Studying programming environments in use

In NWPER ' 98 The Eighth Nordic Workshop on Programming Environment Research, Bergen/Norway, Sunday June 14-Tuesday June 16 1998

Studying programming environments in use: between principles and praxis Olav W. Bertelsen and Susanne Bødker Department of Computer Science University of Aarhus Ny Munkegade, bldg 540, DK-8000 Aarhus C. [email protected], [email protected]

A b s t r a c t . This paper describes a study of an object oriented source code debugger. This study was carried out in order to support redesign of the debugger, and consists of several parts, of which the most significant are a study of the debugger in use, and a redesign workshop. The paper presents and discusses these two parts of the study, both with respect to the findings and with respect to the methods applied. It further uses the findings to discuss how we may understand debugging, in relation or contrast to programming, and how this understanding apply to redesign of the debugger. Three main issues are of importance: the contrast between objectoriented principles and programming praxis, the demarcation of what the programmer needs and wants to be concerned with when debugging, and the problems of separating debugging from programming. The paper points out how it is indeed helpful for design to understand more of the actual praxis of users. In particular, programming cannot be understood in fragmented terms structured according to the artefacts used because principles embedded in programming environments may contradict with the praxis they are intended to support. On the methodical level, the paper demonstrates how studies of debugging and programming supply important input to re-design activities.

1

Introduction

In this paper we report on the findings of the study of an object oriented source code debugger, Valhalla, which is part of the Mjølner BETA programming environment. The study was done to inform the redesign of the debugger. The issues raised by the study concern the methodology of programming environment research; how is it possible to study programming environments in a meaningful way, as well as the more general features of object-oriented programming and debugging. It was a basic premise of the study that the use quality of an artefact is constituted in use, and that artefacts like a debugger can only be understood in terms of the context of programming. In the study we conducted two main activities: evaluation of the debugger in use, and a redesign workshop centred around a paper mock-up of a new

version of the debugger. The two activities were related in the sense that the studies of use produced input to the redesign workshop. The paper is structured as follows: A presentation of the Valhalla debugger, the center-piece of our investigation is followed by a a presentation of out empirical study. The findings are presented and discussed, leading to a discussion of how we may perceive the relationship between programming and debugging in practice, and how this need to influence debugger re-design. A number of problems with Valhalla-in-use are discussed and followed by a discussion of methodology for studying and redesigning debuggers.

2

The Valhalla debugger

Valhalla is a source-level debugger for the BETA language (Madsen et al. 1993) and is a part of the Mjølner BETA System. The study was of version 1.3 of Valhalla, thus, unless otherwise explicitly stated Valhalla means Valhalla version 1.3. The status of Valhalla 1.3 was that it was not yet a part of the “commercial Mjølner package”; it was mostly used by designers at Mjølner and project programmers in a number of projects in the Computer Science Department. The aim of Valhalla is to help the programmer locate errors in BETA programs by tracing their execution at the BETA source level. Thus, Valhalla users often know the code that they inspect and the purpose of the program rather well. And the debugger is launched because of an error in the program. There is at least one more reason for using Valhalla, namely to be able to look at a program execution in order to learn about the program. Thus, Valhalla is in some cases also of interest to persons who hardly know the code, and who are in a non-error situation. Valhalla includes the following functionality: Controlling program execution: The program execution may be controlled e.g. by setting breakpoints and single stepping at the BETA source level. Runtime errors are caught by Valhalla which will display the offending object and code. From there, the program state can be browsed to locate the cause of error. The debugged program executes in a process of its own, being watched by Valhalla, but otherwise unaffected. Static browsing of program text: Static program browsing is supported through group and code viewers making it possible to browse the different relationships present in a BETA program. These include super-pattern relationships, links from name applications to the corresponding declarations, the binding of slots to fragments, and so on. Finally, search for expressions denoting objects and patterns may be performed. Dynamic browsing: Dynamic browsing refers to the possibility of browsing the dynamic state of the BETA program execution. This includes inspection of the call chain and object states whenever the program execution is stopped at a breakpoint or as a result of a runtime error. The user interface of Valhalla consists of a main window containing a menu and a number of different windows (viewers) displaying different aspects of the debugged program.

3

Empirical study

Our background for starting the study of Valhalla was a general concern for how one may study computer applications in use, more than a specific concern for debugging or Valhalla. However, it was very important not just to study any computer application in use, but rather to situate the study so that it would have an impact on redesign of the computer application. With Valhalla, it was possible to get into that position. We are not ourselves Valhalla users. Though we both understand object-oriented programming in general terms none of us have ever done any serious Beta programming. Thus we came to the study of Valhalla as outsiders, with no preconceived opinions about Valhalla. At the same time, we had an excellent chance of being close to the everyday activity of the Beta programmers because we all have offices along the same corridor. Our contribution was mainly a methodological one: We have been involved with several studies of computer applications in use (e.g. Bødker 1996, Bødker & Grønbæk 1996) and in developing methods for active cooperation between users and designers (see e.g. Greenbaum & Kyng 1991). Our particular concern in this case was for studying a rather complex computer application such as Valhalla, the use of which is on the one hand very specific, namely debugging, on the other hand severely entangled with a large and complicated set of other computer applications that support in their various ways the totality of programming. Furthermore, we have a research concern for the integration of design and use of computer applications, and the Mjølner Beta programmers seemed to be a good example of a community where this integration exists. The study was done through the following activities: • Interview with the programmer in charge of maintaining and developing Valhalla, as well as designing the next version. This was done in order to understand the principles behind Valhalla, as well as the overall plans and ideas for the direction in which Valhalla was developing. • Interviews and demonstration with one frequent user of Valhalla to get an initial identification of the kinds of problems that programmers had. • Our own novice attempts to explore Valhalla based on the user manuals, in order to understand Valhalla better as well as to understand how Valhalla is presented to users, as an intended use that may well be very different from the actual use by experienced users (Carroll & Carrithers 1984). • Observations and video taping of a programmer’s use of Valhalla. Since debugging is deeply embedded in programming, and difficult to isolate. We installed a video camera that the programmer would switch on when using the debugger. Programming/debugging is most often done by single individuals rather than groups, and therefore, the programmer was asked to think aloud when debugging (Karat 1988). • The redesign workshop.

3.1

The redesign workshop

Following the study of Valhalla in use, we conducted a workshop aimed at clarifying the future development of Valhalla. Participants in the workshop were the developer responsible for Valhalla (A), and two designers from Mjølner Informatik (B and C). The insights achieved during the “evaluation in use” served as an important background for the workshop which was divided into three phases. Firstly, a brief report of the findings from the evaluation was given, as described below. Secondly, a “Brain storm” centred around why, who, how, what questions of debugging. Thirdly, presentation of the new design by means of a paper mock-up constructed during the morning, and a simulated debugging session with the new version. We choose to structure the brainstorm around the questions: Why? Who? How? and What? in relation to object-oriented source-level debugging. This structure was pragmatically chosen because of successful experience with the use of such questions, and was backed up theoretically by the three levels of human activity corresponding to the questions why? what? and how? (See Bødker 1996, Bærentsen, 89) Based on feedback from users and discussions at Mjølner Informatik, A had made a design specification for a new version of Valhalla. This new design was presented by means of a paper mock-up, produced during the morning before the workshop. The mock-up was based on the specification of the new version and the standard example Beta program from the current Valhalla manual. Although A had a very clear vision of the new design, the process of building the mock-up provoked discussions and decisions about non-trivial details. The presentation and enactment of the new debugger served as the vehicle for a long discussion about the new design, and about how the different parts of the Mjølner Beta environment should be integrated in the future. Several times the discussion became very technical, but frequently the focus turned back to usability issues. No specific conclusions were drawn from these discussions. At the same time, they yielded new insights among the Mjølner designers into the environment they were building and into the activity this environment was aimed at supporting.

4

Evaluation and redesign of Valhalla

In the following, we present the main findings from the two parts of the study: the observational study of Valhalla-in-use, and the redesign workshop. 4.1

Studying the debugger in use

The number of windows. In Valhalla, any new undertaking is shown in a window: any object, stack etc. is shown in its own independent window. The number of windows launched is in itself big. Furthermore, several of the windows show snapshots, thus e.g. a new request for the stack later in the session opens yet another stack window. It is at a later point impossible to see which windows belong together, somehow

representing the “status” of the program execution, or which version of e.g. the stack is the most recent, and which contain outdated information. The programmers typically do not want to keep the snapshots/state information between breaks in the program execution, but each window has to be closed individually. Keeping track of history. Knowing which windows are created when and in which context is, as outlined above, one of the problems of keeping track of history. Furthermore, the programmers expressed a wish to be able to go backwards in the program execution, or to restart from a well-defined state. Staying within one’s own code. Though Valhalla delimits which code may be looked at through the fragment system, there were three major problems with respect to which code one looks at. Firstly, Valhalla stops on signals that do not always come from errors in the code being debugged. Secondly, using “step into” takes the programmer through the execution of all the code including the basic libraries. What happens there is often of little interest to the programmer, and even worse: it is often exactly the step before getting outside ones own code that is interesting. This leaves the programmer in a difficult situation since there is no way of getting back. Thirdly, in various larger settings it is important to be able to debug only one’s own part of a larger application, e.g for the Valhalla programmer to be able to debug Valhalla, and not the rest of the Mjølner BETA system. 4.2

The redesign workshop

In the redesign workshop it turned out that the only participant using the debugger regularly was A. The two other participants were using more traditional techniques like inserting printouts of variables in the source code, which made it possible to trace critical variables, during the entire execution, particularly the initial phase, to detect the errors when they were generated. Their own explanation for not using Valhalla was lack of education, but it also pointed to the fact that Valhalla did not support tracing of variables, but only views on “discrete” states of objects. Valhalla is intended to be an object-oriented debugger in contrast to traditional source level debuggers with only one code window and one trace window and no support for monitoring the relation between different parts of the program. However, the essential thing to monitor in most debugging situations is dynamics. This discrepancy can be understood as an instance of the general tension between principles and praxis. The Mjølner BETA system is built upon a conceptual framework focusing on static aspects of modelling rather than dynamic aspects of execution. Thus, Valhalla was constructed to conform to the principles, rather than to support programming praxis. During the brainstorm it was questioned whether anything was gained from the object-oriented debugging approach, and it was agreed that the next version should support “traditional” source level debugging in addition to the object-oriented approach, e.g. by integrating a report generator or scripting of calculations relative to specified watch points. The advantage of this strategy was, according to B, that

programmers could start using the tracing facilities and then gradually approach the new “religion”. However, in dealing with totally unexpected errors no information can be obtained from traces, because you don't know what to trace. In such situations the objectoriented approach is more likely to help the programmer. Furthermore, Valhalla had been a useful tool for reporting errors in basic libraries etc. Thus, the object-oriented approach is useful when dealing with different layers of a program simultaneously. The principles versus praxis tension also turned up in the discussion of the difficulty of navigating between the excess of windows in Valhalla. It was discussed whether programmers have, or should be allowed to have, a textual understanding of the program they are working on. A claimed that “if you are working on the text then you are working in a procedural manner” (as opposed to an object-oriented manner), but later on B claimed that “the program text is a suitable means of navigation”. It was stressed that “the object-oriented programmer” is not working on the code-text but on the structure of the program, that is hierarchies, patterns, objects. The view on the textual representation as well as the view on specific executions are only means for viewing the structure. Here, the principles of object-orientation prevented the Mjølner designers from acknowledging the central role of textual representations in programming praxis, including their own praxis. However, they agreed that a less fragmented (object-oriented) and more sequential (textual) way of viewing the program would be fruitful. In the new design all of Valhalla resides inside a single window. The main reason for this is that it makes it possible to draw connections between sub-window displaying objects, data or source code. The display of source code in the new Valhalla is done using a subset of the Mjølner hyper-structure editor Sif. 4.3

Summary of findings

The most obvious problem with the studied version of Valhalla was the number of windows and the resulting risk of getting lost. This problem was easy to discover through isolated studies of the use of the debugger. What turned out during the workshop was that Valhalla mainly supports one aspect of debugging (the objectoriented view on runtime instantiations), and that this aspect wasn’t enough debugging support in real programming. The textual aspects turned out to be equally important; both support for tracing and support for viewing the code. The study of the Mjølner BETA debugger elicited the contradiction between the object-oriented principles of working with structure and the actual text oriented programming praxis of both users and designers of the Mjølner BETA system.

5

Debugging and Programming

The perspective on debugging underlying the studied version of Valhalla is that debugging is an isolated process that has to do with finding and correcting errors, and

this process is supported by a separate tool, the debugger. This debugger, Valhalla, is invoked whenever the programmer starts debugging. At the same time, however, it is equally relevant to see bugs, and thus debugging, as a profoundly integral part of programming. This is partly because programming is a human activity like any other, where trial and error is an important part of how we shape things and how we learn (see Engeström 1987). Partly, object-oriented programming in particular is an area where design strategies such as programming-by-example (see Goldberg & Robson, 1983), etc., where programmers deliberately do an experimental modification of existing code, are commonplace. A bug is very difficult to define. At one level we all know bugs, such as undefined variables, syntax errors and calls of procedures with parameters of a wrong type. In practical programming it does not stop there, however. Often the debugger does not stop where the bug is, bugs have ramifications that take the programmer to strange corners, and it requires a great deal of reconstruction (and skill) to find out about where the programmer did wrong. This means that the object of debugging in most practical cases is the whole of the complex of objects that one deals with, or at least a major part of it. A call with parameters of the wrong type, may be where the bug shows up, but it is very likely not where the programmer made his mistake. This call occurs within the context of the entire complex of objects, as well as within the context of the particular program execution. What the programmer may choose to do when debugging is dependent on this whole situation and context, and just as the bug is not planned for, the debugging cannot be pre-planned and fully anticipated. The programmer acts in the particular situation. Nobody make bugs at will. A bug will almost always be a result of something that we do as part of some purposeful action focused at something else: creating an object that would do something specific. However, this should not lead us to think that a bug is a result of something thought through, but wrongly. Human activity is much more experimental than that, and in any activity, including programming, we make use of the fact that our materials speak back to us (Schön, 1983). A bug is often a result of such an experiment, and in a way, the question of debugging could be recast as a question of how the programming platform speaks back to use, when we use it (i.e. program). Because debugging is acting in situations of significant complexity, it is essential for our investigation of the tools for debugging, in this case Valhalla that they are studied in use. However, because of the complexity, it is difficult to study programming environments in use: it is hard to isolate typical episodes, and to isolate and plan for situations of debugger use. This means that what we would get out of testing Valhalla in well-understood examples would remain toy stories that would say little about the real problems of debugging. Exemplifying this from our own work: we got very little out of the interviews/demos that we did with programmers as regards substantial examples of how they act in use: how they handle the excessive number of windows, etc. and the complexity of coping with many windows on the screen hardly showed up in the demos. This leaves us no hope but to study real use, introducing immediately the next sequence of methodological problems: since Valhalla use is incidental and sporadic, we

really needed to study, and make recordings of very long-lasting programming activities in order to hope for catching substantial and surprising debugging situations. We opted for the alternative of asking the programmers to turn on the video camera whenever they launched Valhalla, and hoped they would be able to think aloud while doing the debugging (the latter since there was no guarantee that we were right there at that very moment, which would often be in the middle of the night).

6

General(isable) problems in Valhalla

6.1

Principles of object-orientation versus programming praxis

In most cases a specific design artefact is intended to be used in a specific way, and just as often the actual way the design artefact mediates design is very different from this intention. In most cases, the relation between the principles of intended use and the actual praxis with the artefact is far from trivial (Bertelsen, 1998). In our study of Valhalla, we saw similar problems. The tool was based on strict object-oriented principles, whereas the users to a great extent worked with programs as text. The tension between principles of object-orientation and actual programming praxis was most clearly expressed during the redesign workshop. Valhalla was intended to be an object-oriented debugger in contrast to traditional source level debuggers. It is a part of the Mjølner BETA system, which is built upon a conceptual framework emphasising that programming is modelling (see Madsen et al. 1993, Ch. 18), but also focusing on static aspects rather than the dynamic of execution. However, in debugging it is important to be able to monitor the dynamics of execution — the debugger was conforming to principles rather than supporting praxis. In discussing the navigational problems in the use of Valhalla, the principles versus praxis tension turned up again. The designers discussed whether programmers had, or should be allowed to have, a textual understanding of programs. It was emphasised that “the true object-oriented programmer” is not working on code text but on the structure of the program, i.e. hierarchies, patterns, objects. The view on the textual representation as well as the view on specific executions are only means of viewing the structure. The principles of object-orientation prevented the designers, from acknowledging the central role of textual representations in programming praxis, including their own praxis. Valhalla was primarily intended as a part of the hidden curriculum for the reeducation of programmers to do true object-oriented programming. The designers were unsure if anything was gained from the object-oriented debugging approach. The discussion of their own use of Valhalla, had forced them into explicating their own debugging praxis. To meet the demands of their own praxis their debugger had to include support for more “traditional” debugging, e.g. tracing of variables, this was agreed to be featured in the next version along with object-oriented

features. This was not considered a setback for the vision of object-oriented programming activity, rather, according to one of the designers, programmers could start using the tracing facilities and then gradually approach the new “religion”. In other words, the new instrument, structure debugging, had not been consolidated in the praxis of the programmers. The praxis of old time programmers to handwire print statements was surviving the vision of new ways of doing object-oriented programming and debugging. Thus, the introduction and consequent use of an objectoriented programming language into a programming praxis is no guarantee for the successful transition of the programming activity to be object-oriented. This is realised in the Mjølner community (Knudsen (ed.) 1994) and is the reason for the semi religious talk about structure over text, during the workshop. The examples above point to a general contradiction between prescription and praxis. The general discrepancy in design artefacts between what they prescribe and the praxis they induce can in some situations be understood as a contradiction between the design artefact producing activity and design praxis. This is the situation in the case of the Mjølner programming environment (here Valhalla and Sif). The tool designers think that programmers need to understand their own work as constructing objectoriented structure, whereas the programmers, even when they are intimately familiar with the structure-not-text idea, continue to write program text. In this case, the programmers who produced the instruments also represent the more advanced way of doing debugging through structures. The funny twist in the case of the Mjølner environment is that the same persons who produce the new the instrument are also, in their own programming praxis declining to adopt these principles. 6.2

Demarcation of context

As we have indicated, our studies have showed that the programmer in programmingand-debugging is concerned with a unit that is larger than the object he is writing code for, yet smaller that the whole universe, i.e. programming environment. To make a program that draws a circle in a window I should be able to concentrate on the size of the window, the things to put in it, etc., whereas I ought to be able to trust that the window as such works. There are several aspects to this: The window code has been placed in a library by somebody else, who has supposedly debugged this code. Thus I have no general intent of debugging this code, though at times I may want to inspect it. The Valhalla debugger could not tell the difference between my code, and that of others, in the sense that we have described the difference here. From other studies we have seen that placing code in public is a process that is different from creating ones personal code (see Bødker in preparation) in that to place code in public, it needs to be “wrapped-up”, be made self-contained and to have clear boundaries. The obvious problem of a separation between code I trust and my code in the case of the Mjølner designers is that to them, this distinction hardly exists: they are themselves potentially interested in debugging all parts of the code, an additional problem of the overlap between the design community as designers and as users.

At a very specific level what happens e.g. when debugging the circle in window is that the actual bug often seems to occur in code that was not written by me, whereas it is difficult, as described earlier, to find the point where I step out of my code and into somebody else’s territory. Furthermore, this point may depend on the entire context of my code, and of the context and of the history of the program execution. And it may depend on the complexity of the problem I’m facing, how much of the surrounding code I’m interested in. We have no answers to what exactly would be a good design in support of this way of thinking, but we do believe that further studies of specific examples of debugging, and the elaboration of contexts of concerns and borders of objects, would be extremely useful in future redesign of Valhalla. 6.3

Debugging separated from programming

Looking at debugging as a process that is, or can be, separated from programming turned out to be problematic in two ways: first of all, for practical situations, debugging is deeply embedded in programming, which can be seen from the fact that it was next to impossible to reconstruct debugging sessions of substance in a lab or demo setting. Secondly, a separation of the two processes leans heavily on a perspective where programming is done first, and in full detail, following which the code is debugged. This perspective is in sharp contrast to a more experimental one, where programs are experimentally constructed, using existing code as material (or clay, as Kay (1977) would phrase it), and where human thinking through trial and error is supported. We are certainly aware that it is not this aspect as much as the modelling capabilities of object orientation that has been the driving force of the Mjølner BETA system. Furthermore, the closer integration of the different parts of the Mjølner BETA environment (hyperstructure editor, debugger case tool etc.) in later version, and the Mjølner designers description of Valhalla as also being a good tool for learning about code that others have written, indicates that Mjølner does not base the environment on a naive top down approach. We find it hard to believe that anybody would argue in favour of the full separation of programming and debugging. An integrated, experimental perspective on debugging and programming does not prevent anybody from designing separate tools to support the two. Just as we eat with both knife and fork, we can certainly apply more than one tool for debugging-andprogramming. What we would argue, however is that such a perspective, at the same time as it would be very fruitful for the further development of the programming environment, demands of the designers that they engage profoundly in understanding real use. It is easy to design a debugger that handles the anticipated bugs, way more difficult to design for the unanticipated cases. With the closer integration, in later versions of the Mjølner BETA programming environment, Valhalla can be seen as a viewer emphasising the dynamic aspects of the program, more than merely a tool for finding errors.

7

How to study debugging - Discussion of method

Two interdependent methodical problems arose in relation to studying the debugger in use. The first problem was related to debugging being an integrated part of programming, happening at unforeseen occasions. Bugs are not isolated phenomena only related to the piece of code the programmer is working on when bugs occur. It is next to impossible to generate bug situations that in a realistic way can be used as a basis for laboratory testing; thus, we were destined to hang around at the programmer’s office until bugs occurred. This led to the second problem, related to making sense of the recorded data. The resolution on the video tapes was so low that the only thing we could distinguish on the recorded UNIX screen was the numerous windows popping up. The tapes partly worked for stimulated recall interviews, but we were unable to make micro level analysis. A topic for further research is on methods for collecting data on programming, both at the technical level of being able to see what is going on, and at the level of finding a meaningful basic unit to study (debugging is obviously a too small unit). Further empirical studies of the objectoriented principle of structure and its role in practical programming, is another topic for further research. This will involve analysis on what is the object of programming work (text, structure, application, use praxis). The workshop centred around the paper mock-up of a proposal for the next version of the debugger served several purposes. To some extent, the workshop yielded insights which we had not been able to achieve through the study of Valhalla in use. As opposed to sheer interviewing or discussion, simulated work with real artefacts or mock-ups is an efficient vehicle for eliciting established praxis; modes of working, rationales, etc. This use of mock-ups in analysis rather than in the verification of design ideas is captured in Mogensen’s (1992) term “provotype” (as opposed to prototype). Further, the mock-up was a successful vehicle for the general discussion of the principles on which the design of the debugger was based. The mock-up was drawing the discussion in the direction where it was based more on the reality of programming praxis and less on pure detached principles. Thus, the workshop helped navigating between the enforcement of object-oriented principles and the pragmatic support of programming praxis in the design of the next version of Valhalla. The direct purpose of the study of Valhalla was to give input to the design of the next version of the debugger. This gave clear criteria for the level of detail etc., that was needed in the study, and it also to some degree eliminated the need for detailed reporting of results of the workshop since the users of these results were participating directly in the development of new insights.

8

Conclusions

This paper has reported on a project aiming at the analysis of one version of a debugger, supplying input for the design of the next version of the same tool. With respect to this original aim the project was successful. However, the study furthermore

points to general conclusions about programming environments and general methodical implications for the study of programming environments. With respect to understanding programming and designing programming environments the study showed that programming cannot be understood in fragmented terms structured according to the artefacts used (e.g. understanding debugging as being the same as debugger use); that in real-world programming the notion of context and border between, e.g. my code and basic library code, is important; and that principles embedded in programming environments may contradict with the praxis they are intended to support. On the methodical level, the study points to a need for further work on recording what happens on very high resolution screens, and how to do micro-level analysis based on such data. However, the paper has shown how studies of artefacts in use can provide important input to workshop activities, which in turn can be very useful instruments for provoking discussions of established praxis, as well as for exploring new designs. The key issue remains to provide insight into programming practice to the process and confront these with programming principles.

Acknowledgements We are grateful to the people from Mjølner and from the Devise project who willingly participated in the study.

References Bertelsen, O. W. (1998). Elements to a theory of design artefacts: a contribution to critical systems development research, Ph.D.-Thesis. Aarhus: Aarhus University, DAIMI PB531. Bærentsen, K. (1989). Menneske og maskine [Man and Machine]. In Hedegaard, Hansen & Thyssen (eds.). Et virksomt liv [An active life]. Aarhus: Aarhus University Press. pp. 142-187. Bødker, S. (in prep.) Mature object oriented system development, submitted for publication Carroll, J. M. & Carrithers, C. (1984) Training Wheels in a user interface. In Communications of the ACM, vol. 27, pp. 800-806. Engeström, Y. (1987). Learning by expanding: an activity-theoretical approach to developmental research. Helsinki: Orienta-Konsultit Oy. Goldberg, A. & Robson, D. (1983) Smalltalk-80: the language and its implementation. Reading, MA: Addison-Wesley Publishing Company. Karat, J. (1988). Software Evaluation Methodologies, in Hellander, M. (ed.) Handbook of Human-Computer Interaction. Amsterdam: North Holland. Kay, A. (1977). Microelectronics and the Personal Computer, Scientific American 237, pp. 41-47. Knudsen, J. L. (1994). Teaching object-oriented methodology and languages. In Knudsen, J.L., Löfgren, M., Madsen, O.L. & Magnusson, B. (eds.). Object-Oriented Environments: the Mjølner Approach. New York: Prentice Hall. pp. 578-600.

Madsen, O. L., Møller-Pedersen, B. & Nygaard, K. (1993). Object-Oriented Programming in the BETA Programming Language. Wokingham, England: ACM Press/AddisonWesley Publishing Company. Mogensen, P. (1992). Towards a Provotyping Approach in Systems Development. In Scandinavian Journal of Information Systems, vol. 4. pp. 31-55. Schön D. A. (1983). The Reflective Practitioner - How Professionals Think in Action. New York: Basic Books.