applications, quality management, software ... a multimedia environment buying network services ... and selected multimedia applications issues are used to.
Interop '95
M. Bogen, H.-L. Hausen, R. Worst
Quality of Service in a Multimedia Environment Manfred A. Bogen Hans-Ludwig Hausen Rainer Worst GMD—German National Research Center for Computer Science D-53754 Sankt Augustin comprising production or service provision and also for their organisation.
Keywords
Beside the needs for quality drivers from the market there are requirements defined by interests groups (e.g. unions) or government (e.g. European directive for workstations). This rises the need for well-defined, cost-effective means for the specification, testing, measurement, assessment and (eventually) certification of the quality of products, processes, and services.
Quality of Service (QoS), QoS characteristics, quality management, network services, CSCW applications, quality management, software evaluation, multimedia Abstract Engineers and Managers today are co-operating in a multimedia environment buying network services from a variety of service providers. Quality of Service is an important issue for them. In order to avoid extra costs, the handling of Quality of Service has to be considered as an integral part of quality management. The International Organisation for Standardisation's ISO 9000, on the other hand, provides guidelines which can be used by manufacturers and service providers to establish a quality-conscious approach for development, testing, manufacturing and support operations. In such a case ISO 9000 certification represents a first step in the process of improving quality.
As industrial standards for quality management systems and standards have been designed for the development of IT systems, they are not yet applicable to CSCW and multimedia applications. On the other hand, conducting Quality of Service management as separate activity will cause organisational problems and will finally cause unjustifiable costs. A solution to this is the integration of methods and procedures for quality specification and evaluation form both areas into one coherent framework. The reaming parts of the present paper provide an outline of such a framework. Networking for CSCW and selected multimedia applications issues are used to demonstrate the applicability of the approach proposed.
Quality of Service is well-defined in the different protocol layers, but the standard ISO 9000 does not immediately help in the area of groupware, CSCW or multimedia applications. The problem is that those quality standards are more related to the quality management and not to the product or service quality itself which is the most important issue for the end user, consumer and engineer. The paper identifies major quality criteria in a multimedia environment and proposes methods how to assess them in adopting standardised methods for quality management. I. Introduction For both computer-supported co-operative work and multimedia applications, success will be determined if and when they are accepted by industry and finally by the professional as well as by the nonprofessional users. More and more industrial companies, therefore, are forced to develop a qualityaware approach for their business processes
Figure 1: Local Multimedia Environment
1
Interop '95
M. Bogen, H.-L. Hausen, R. Worst • availability for different operating systems • a wide range of media types • hyper links • sophisticated synchronisation and presentation facilities • adequate local response times (no delays please! 0,2 s for simple operations, 2 s for complex operations, 20 s: the user gives up.)
II. Quality in A Distributed Multimedia Environment In this chapter, we describe our understanding of distributed multimedia environment and related applications, we identify a popular multimedia and CSCW application and we define major quality characteristics for this application.
and others.
II.A The Scenario
As soon as the multimedia environment is distributed (see figure 2), other requirements are related to the network, the protocols used and the communication and co-operation, such as
In alignment with [1] we define a multimedia document as a document containing different media types such as text, sound, image, graphics, animation (text, pictures and graphics in motion), movie, interaction (the user has influence on the action), simulation (executable computer programs) and possibly in the future smell.
• scalability (dynamic groups) • connection types (1-1, 1-N, N-N, N-M, ...) • reliability (unreliable, limited, overall) • performance (delay, throughput, ...) • resource allocation • adequate remote response times • mail-based access to multimedia information • intelligent authoring tools
Figure 1 shows a typical local multimedia environment where different user requirements (requirements on user-related quality characteristics) have to be met [1], such as
• • •
Network
End system
User
Figure 2: Distributed Multimedia Environment
2
Interop '95
M. Bogen, H.-L. Hausen, R. Worst
Communication
Co-operation
Co-ordination
Electronic mail
Common message store
Group decision support systems
Shared windows Multi-user editors Real-time group editors
Common calendars
Co-ordination agents
Electronic bulletin boards
Time marking and keeping
Computer real-time conferences Computer teleconferencing Desk-top conferencing Video conferencing
Shared information sources
Work flow management systems
Electronic meeting rooms Electronic class rooms Intelligent agents Directory Table 1: Classification of Multimedia and CSCW Applications • it allows live and interactive collaboration, • it needs few system resources (storage) as videos are passed through (don't ask for network resources!), • it supports real-time document handling, and • it transports even non-verbal information (did she smile?).
II.B. A Typical Multimedia and CSCW Application CSCW (computer-supported co-operative work) is a multidisciplinary or interdisciplinary field for social and computer scientists. The disciplines are: distributed systems (operating systems, databases), communications, human-computer interaction (interfaces), artificial intelligence, social theory/ sociology.
The user's needs and the communication habits within a company will determine in the future how video technology is used. Four major issues are relevant here:
CSCW looks at how groups work and seeks to discover how technology (especially computers) can help them work. CSCW is concerned with bringing computing face-to-face with groups of people as they work.
• Quality • Connectivity • Real-time scenarios • Costs (last not least).
In the multimedia environment described, different multimedia and CSCW applications are possible, related to the communication and co-operation of the users and related to the co-ordination of processes and actions (see table 1).
Costs have to be calculated - according to accounting procedures - of the most appropriate service providers with respect to cost drivers such as • • • •
In chapter V, we will take a real-time video conferencing system or service (in contrast to storeand-forward video mail) as an example for a multimedia or CSCW application. Multiple participants cooperate and communicate in their distributed multimedia environment over the network (e.g. Internet) using their workstations (desktop video conferencing) or special conference rooms.
connection installation price only, contingent pricing, access price costing, pricing per number or type of users or per size of institution.
We concentrate on the following questions: What means quality in these areas? What is quality assurance and quality assessment here? How should a quality management system look like and how can quality characteristics be measured?
There are several, cost-independent reasons why real-time conferencing became so popular in the last years:
3
Interop '95
M. Bogen, H.-L. Hausen, R. Worst
III. Quality management and Quality Systems
to reinvent the organisation. BPR requires a revolution in the enterprise under consideration [3]. No matter how an organisation attempts to get quality into its products or services, there is always a need to make evident whether the process meets its requirements. Therefore, it would be useful for a provider to refer to commonly accepted standards instead of dealing with the different approaches to quality assessment of the various purchasing organisations and customers. A solution to this problem was recently established in manufacturing, but it becomes evident that by and by service providers are asked for their quality systems' compliance to established standards, too.
In this chapter we briefly describe the most relevant international standards for quality management systems widely applied today in IT industry . It is shown that they are useful for serviceproviders and users, too. Generally, quality becomes more and more important in international IT and telecommunication market. Hence, many enterprises stress their commitment to quality. Particular incentives are quality-related awards, such as the European Quality Award or the Malcolm Baldrige Award in the USA. The winning organisations use these qualifications to present themselves as leading edge companies with regard to quality.
III.A. International Standard ISO 9000 Those commonly accepted regulations in manufacturing are the international standards ISO 9000 to ISO 9004, which often are called "ISO 9000" all together. Three of these standards represent distinct forms of functional or organisational capability [4] suitable for two-party contractual purposes:
Because of the variety of organisations, products and services, there is a broad range for specific means of ensuring quality. A well-known approach is Total Quality Management (TQM), which is a management approach of an organisation, centred on quality, based on the participation of all its members and aiming at long term success through customer satisfaction, and benefits to the members of the organisation and to society [2]. TQM encourages a process of continuous improvement and is therefore evolutionary. Against that, the issues of Business Process Re engineering (BPR) are analysis of business processes and use of advanced information technology in order
•
ISO 9001, Quality systems—Model for quality assurance in design/development, production, installation and servicing,
•
ISO 9002, Quality systems—Model for quality assurance in production and installation,
•
ISO 9003, Quality systems—Model for quality assurance in final inspection and test.
User
QoS Requirements QoS Data
Service Provision
Feedback Figure 3: External Quality Assurance
4
Audit
Interop '95
M. Bogen, H.-L. Hausen, R. Worst and effective operation of a quality system that helps to evaluate, control and improve the quality of service.
ISO 9001 to ISO 9003 are the basis of external quality assurance (see figure 3). A certificate of an authorised body with regard to one of these three standards attests that a quality system is established and maintained within the audited organisation. Without such a certificate it is nearly impossible for a supplier to get orders from manufacturing enterprises.
Management should develop and document a quality policy. Its realisation requires the identification of primary goals. Then, management has to translate the primary goals into a set of quality objectives and activities. These might be:
Additionally, there are some guidelines for selection and use of ISO 9000 series and for their application to several fields, such as ISO 9004, Part 2: Guidelines for services [5]. This guide might be particularly useful for organisations that provide multimedia services. Therefore, we summarise its content in the following sections.
• • • • •
and especially for multimedia and CSCW applications
III.B. Interaction with Customers The interface with customers is the most important aspect of a quality system. The establishment of effective communication between customers and the service provider's personnel is an outstanding task, because it is critical to the customer's perception of the service quality. This perception can be influenced by creating an appropriate image of the service provider in addition to the essential service delivery.
• • •
describing the service, its scope and its availability,
•
stating how much the service will cost,
•
explaining the interrelationships between service, delivery and cost,
•
explaining to customers the effect of any problems, and how they will be resolved, should they arise,
•
ensuring that customers are aware of the contribution they can make to service quality,
•
providing adequate and readily accessible facilities for effective communication,
•
determining the relationship between the service offered and the real need of the customer.
Enhance productivity of teams and groups Support office work as team work Distribute information efficiently and timely in a form that can be utilised by a group or organisation.
To achieve the quality objectives, it is important to define explicitly responsibility and authority of all quality-related personnel. Nevertheless, it is not only these personnel who create quality. To achieve continuous improvement, all the personnel must commit to quality. The behaviour and performance of every individual directly impact on the quality of service.
It is necessary to listen to the customers. It is also necessary to keep them informed. According to [5], effective communication with customers involves: •
Offer a reliable service React quickly Achieve high performance Let customers feel comfortable Understand your customers
Not only the communication with the customers but also the communication within the service organisation is a regular part of daily work. For a provider of multimedia services, e.g. communication via multimedia workstations would be a typical addition to other methods of in-house communication. Therefore, all the personnel should have the knowledge and skill to use such equipment. III.D. Service Quality Loop The main issue of ISO 9004-2 is the concept of a "service quality loop", which is shown in figure 4. Particular attention should be paid to the service processes and feedback measures: • • • •
III.C. Management Responsibility It is within management's responsibility to establish the necessary policy for service quality. Management has to commit itself to the development
5
Marketing process, Design process, Service delivery process, Service performance improvement.
analysis
and
Interop '95
M. Bogen, H.-L. Hausen, R. Worst
Service Provider Service specification
Design process
Service brief
Interface
Service delivery specification Quality control specification
Marketing process
Service delivery process
Interface Supplier Customer
Customer Supplier Service needs
Improvement instruction
Improvement instruction
Service result
Evaluation result
Improvement instruction
Supplier's assessment
Service performance analysis and improvement
Customer's assessment
Assessment report Assessment report
Figure 4: Service Quality Loop according to ISO 9004-2 service provider. While the customer's measurement generally is based on comparison, the service provider has the chance to conduct real measurement, i.e. to attach real numbers to the characteristics and to apply operations to them like in physics. In other words, the service provider can make those service characteristics quantitative that are qualitative for the customer.
III.D.1. Marketing process First of all, it is necessary to collect market information and to determine and promote the demand for a service. This is to be done by marketing using surveys or interviews. We do not see any specific aspects for the domain of multimedia services with regard to marketing. The result of the marketing process is a service brief, which defines the customers' needs and the related service organisation's capabilities.
Examples of service characteristics that might be relevant for multimedia systems are: • • • • •
III.D.2. Design process Based on the service brief, the results of the design process are the service specification, the service delivery specification and the quality control specification.
Transmission delay Image clarity Audio capability Voice recognition Number of cameras
The service delivery specification defines the means and methods used to deliver the service. It describes clearly the service delivery characteristics that have a direct impact on the quality of service, including definition of resource requirements, of number and skill of personnel required and of reliance on sub-contractors for purchased products and
The service specification defines the service to be provided. It describes clearly the service characteristics subject to customer evaluation. The characteristics need to be capable of evaluation by the service provider against defined standards of acceptability. They are measurable by the customer or 6
Interop '95
M. Bogen, H.-L. Hausen, R. Worst
services. Service delivery characteristics may not always be observable by the customer, but it also needs to be capable of evaluation by the service provider against defined standards of acceptability. So the service provider should make also the service delivery characteristics quantitative. Examples of such characteristics that might be relevant for multimedia systems are: • • •
IV. A QoS Evaluation Method In this section we describe a method originally developed for software evaluation and how the quality evaluation should be done by an audit entity (notary, certification body, arbitration committee etc. ) for a multimedia service or application. An evaluation method comprises • the evaluation process • evaluation levels • evaluation techniques • evaluation modules
Bandwidth usage Connection types Standards used
The quality of the services supplied by a multimedia service provider also depends on purchased products and services. This is particular important with regard to software products required by the service provider. A careful selection of subcontractors and software suppliers is recommended.
IV.A. The evaluation process An evaluation process consists of five activities which are performed by the evaluator: • ACT1 analysis of the actual requirements for the evaluation.
Quality control should be considered as an integral part of the service processes. The quality control specification defines the procedures for evaluating and controlling the characteristics of service and service delivery. The design of quality control involves identifying and analysing the key activities in each process to select the relevant characteristics with regard to service quality and on the other hand defining methods for evaluating and controlling these characteristics.
The requirements may consist of the list of characteristics that are considered of importance to the purpose of the product or service. The list may be enriched by reference to evaluation levels for each characteristic. In addition, the requirements may refer to standards and regulations with which the product should comply. • ACT2 specification of the evaluation that is to comply with the requirements identified by the analysis.
III.D.3. Service delivery process The characteristics of service and service delivery are controlled by controlling the process that delivers the service. Fortunately, multimedia services tend to become mechanised in contrast to manual personalised services. Hence, there is a opportunity to apply structured and disciplined quality system principles effectively to them.
The specification shall be made on the basis of the product and process information. It shall consist of two steps. The first step is to identify parts or key attributes to be inspected and evidence from the development process to be investigated. The second step shall be to define those characteristics to be evaluated, for each piece of product and process information, and their corresponding evaluation level
For multimedia services, remedial action is often impossible, since customer assessment of any nonconformity is immediate, but the customer is located far away from the service provider and the service provider generally does not take notice of the customers reaction. Therefore, it is not possible to rely on final inspection to influence service quality at the customer interface.
• ACT3 design of the evaluation which shall choose evaluation techniques and plan the evaluation project to satisfy the specification. This activity should make use, as much as possible, of the library of standard evaluation modules. • ACT4 performing the evaluation which shall consist in inspecting, measuring and testing product parts and elements of process evidence.
III.D4. Service performance analysis and improvement Service quality improvement requires a continuous evaluation of all service processes. The information needed for this process of improvement will be available from supplier assessment, customer assessment and quality audits.
The results of these actions should be produced and recorded according to the definition of the various evaluation modules used; • ACT5 reporting on the result generated from performing the above four activities.
The evaluation method described in the next section adopts established methods and procedures to specify, evaluate and control the quality of multimedia services.
This report shall be delivered to the client of the evaluation.
7
Interop '95
M. Bogen, H.-L. Hausen, R. Worst
Input from the Client client information
client evaluation requirements
Input from the Evaluator
product and process information
evaluation techniques
Analyse Ev. Requirements evaluation requirements
evaluator information
evaluation modules library
categorised evaluation items
Specify the Evaluation Design and Plan the Evaluation
evaluation specification
evaluation plan
Perform the Evaluation evaluation results
Report evaluation report
Figure 5: The Evaluation Process IV.B Evaluation levels
IV.C. Evaluation techniques
The selection of evaluation levels as function of the context of use of the product has been studied [6]. The evaluation levels may be selected independently for each of the relevant quality characteristics. When selecting the levels, several aspects should be considered. For example, important aspects are those related to safety, to economy, to security, to the environment and to the marketing of the product when appropriate.
The feasibility and the cost effectiveness of evaluation depend on the availability of appropriate evaluation techniques. They must be pertinent, i.e. the result that can be derived from their application must be representative of the actual quality of the product or service and they must be cost-effective, i.e. the cost of their application must be related to the confidence obtained in the quality. Evaluation techniques can be considered under the following headings:
For a relevant quality characteristic, the risks involved by the non conformity of the product to requirements relating to this characteristics, as well as benefits from high quality, should be assessed for all the relevant aspects. For some of these aspects, table 2 provides the suggested relationship between risks and levels to be selected. When several aspects need to be considered, the most stringent level should be selected.
•
automatic analysis of documents
The most used technique in software evaluation is the static analysis of source code providing complexity measurements. This can be complemented with similar measurements on other development documents if they are written with some formalised language, or the measurement of readability indices for natural language documents, or measurements of the extend of the application of development rules (e.g. programming rules for source code).
For the issues of economic risks and marketing benefits, the cost of the evaluation should be considered.
8
Interop '95
M. Bogen, H.-L. Hausen, R. Worst
L e v e l
Safety
Economy
A
Many people killed
Financial disaster
Strategic data and (company will not services survive)
Unrecoverable environmental damage
B
Threat to human life
Large economic loss Critical (company data and endangered) services
Recoverable environmental damage
C
Damage to property
Local pollution
Few people disabled
Significant economic Protection loss ( c o m p a n y against affected) error risk
Small damage to property;
Negligible economic loss
No risk
D
Security
No risk
Environment
No risk to people
Table 2: Evaluation Levels and Risks •
metrics has always been a failure and it is now recognised that expertise is needed to interpret the raw numerical results and to combine them with other metrics values. This expertise must take into account factors like the application domain of the system under consideration, or the culture of the organisation that produced it. Another aspect to be considered is that most techniques are not applicable to all cases. The conditions for application of an evaluation technique may be very specific and trying to apply it outside its context would lead to meaningless results. These considerations advocate for the encapsulation of basic metrics definitions with the expertise needed to choose them and interpret the measured values, with the conditions for applications and with other aspects such as the kind of results that can be expected from the use of that technique.
inspections
This is a generic term for a variety of techniques like peer reviews or walk-trough; there are approaches to both formalise and specialise these inspections: for various attributes or characteristics of a product, check-lists have been developed; the score obtained can be considered as a measure. The concept of an evaluation module could be summarised as a document describing which technique can be used to assess a given quality characteristic at a given evaluation level. It also gives some guidance to the evaluator on deciding whether the product passes sufficiently well for the related evaluation level. The fact that many quantitative evaluation techniques are applicable is not sufficient for large scale use in industry. In order to make this approach attractive, several barriers need to be overcome, such as the lack of ready-to-use packages, the difficulty for experts to reach consensus or the lack of a global framework applicable to their specific case.
The goal of an evaluation module is to capitalise the expertise gained by practitioners using some assessment techniques extensively. This expertise is formalised in a document that describes an evaluation technique, a standard way of applying it and the result it can provide when used during the assessment task. An evaluation module can be considered as a black box that, when applied to suitable input, produces a report and a quantitative evaluation result. As such, it can be considered as a higher level metric capturing some feature of the software product or service.
When quantitative aspects of software assessment are evoked, one usually thinks of "metrics". The drawback is that the notion of metrics is often fuzzy and interpreted in a very restrictive way, for example, the number of lines of code or the test effort. The attempt to define absolute thresholds for such basic 9
Interop '95
M. Bogen, H.-L. Hausen, R. Worst • •
V. Example: Evaluation of a Video Conferencing Service
The refinement into sub characteristics is shown in tables 3-6.
To make a comprehensive assessment of a service, the elements to be decision relevant and the relevant set of characteristics have to be selected. For each characteristic a quantification has to be defined. On the other hand, one might consider the software quality standard ISO 9126 and software ergonomics standard ISO 9214.
Based on these tables, the evaluation method described in chapter IV is applicable. Instrumentation of the evaluation is done with Evaluation Modules. At present we have about 100 applicable Evaluation Modules for software evaluation. ISO/IEC JTC1 SC7 will standardise those modules.
A few user- and network-related quality characteristics have already been mentioned in section II). For a video conferencing system the requirements are even more specialised and we propose a subdivision into quality-related characteristics and sub characteristics. As characteristics we see • •
standards usage characteristic add-ons characteristic
Special Evaluation Modules for the measurement of the characteristics not related to software have to be written. By this bottom-up approach a consensus on specific assessment techniques would be obtainable before there is a complete framework on QoS with regard to multimedia services.
network characteristic system characteristic Sub characteristic
Possible
Network used
Internet, (B-) ISDN, Satellite, leased organisation-dependent lines, ATM, SMDS; analogue telephone lines, Novell, Ethernet, ... 56 to 512 Kbps and more < 100 kbps see above N-M < 200 ms (overseas telephone calls) 5 • 50 frames per second (fps), > 10 fps full motion: 30 fps
Bandwidth usage Connection types Transmission delay Video/ frame rate (performance)
Acceptable
Table 3: Network Characteristic Sub characteristic
Possible
Hardware equipment needed (Sender/ Receiver) Software equipment needed (Sender/ Receiver) Media types supported
workstation, video card, camera, modem, cost-dependent audio-video module, Codec integrated user interface, video application cost-dependent software, converters, Codec text, sound, image, graphics, animation, movie, interaction, simulation, smell,... 80 * 96 pixels to 1152 * 870 pixels quarter of a screen 4-bit grayscale 24 bit per pixel, ...
Window size Window colours Image clarity Video resolution Video signal formats Number of cameras
Acceptable
160*120, ..., 1024 * 768 NTSC, PAL, SECAM, ... < 16 Table 4: System Characteristic
10
Interop '95
M. Bogen, H.-L. Hausen, R. Worst Sub characteristic
Possible
encoding codec videoconferencing service documents compression
G.721, G.722, G.725 H.221, H.242, H.261 F.400, H.320 HyTime, IIF, MHEG, ODA, GIF JPEG, MPEG, JBIG, T.80
Table 5: Standards Usage Characteristic Sub characteristic
Possible
Audio capability Voice recognition Background file transfer and e-mail Application sharing Joint editing Joint viewing Notebook Hyperlinks
yes/ no yes/ no yes/ no yes/ no yes/ no yes/ no yes/ no yes/ no
Table 6: Add-ons Characteristic From the end users' point of view, these tables will help them talk to potential service providers and compare them. A bit of homework, however, will still be necessary.
VI. Conclusion Quality evaluation is often discussed as a complex procedure for managing all potential attributes and topics. This type of quality management is not very effective because it does not help identify the decisionrelevant criteria. This paper has demonstrated the feasibility of quality evaluation for multimedia services. As shown in the example, which was simplified for better understanding, one has to identify what has to be evaluated (video conferencing) and the basic components of that service. Then one has to search for the characteristics. The next step is to quantify the required characteristics and to instrument the process that is defined in the Evaluators Guide by Evaluation Modules.
And finally it has been shown that principles of quality management, such as given by the ISO 9000 series, can be applied successfully to QoS assessment. Top level management might like this because it allows the integration of QoS management into the quality management which might be already an integral part of the organisation. VII. References [1] Chris Adie, Network Access to Multimedia Information, Edinburgh University Computing Service, May 1994
The advantage of the approach presented is twofold. First, one gets an applicable method for QoS evaluation. If the characteristics are carefully selected, one can achieve efficient evaluations. Second, we have embedded the QoS evaluation into the evaluation of software systems, thereby reducing the effort of QoS evaluation, because we can reuse product or process evaluation reports in evaluating, for example, reliability.
[2] ISO, „ISO 8402, Quality management and quality assurance; Vocabulary," 1991. [3] M. Hammer, J. Champy, „Reengineering the corporation: a manifesto for business revolution," HarperCollins Publishers, 1993. [4] ISO, „ISO 9001 Quality systems—Model for quality assurance in design/development, production, installation and servicing," 1987.
What should be done? In the context of the proposed evaluation procedure, dedicated QoS Evaluation Modules might be developed and standardised, which could be used to define state of the art of a service. That might lead to real work for standardisation bodies.
[5] ISO, „ISO 9004 Quality management and quality system elements; Part 2: Guidelines for services," 1991. [6] H.-L. Hausen, D. Welzel, „Guides to Software Evaluation (consisting of: The Evaluator's Guide, The Evaluation Module Development Guide),"
11
Interop '95
M. Bogen, H.-L. Hausen, R. Worst
Arbeitspapiere der GMD, No. 746, GMD Sankt Augustin, April 1993.
Author Information
[7] Manfred Bogen, Hans-Ludwig Hausen, Rainer Worst: Handling of QoS Characteristics, in: Computer Networks and ISDN Systems 26 (Suppl. 2 double issue) (1994), pp107-118, Elsevier Science B.V., Holland
Manfred Bogen has been working as a scientist at GMD since 1983, where he has been active in the area of group communication, X.400 development, and X.400 standardisation. In 1987 he became head of the German Networking, Research and Information Centre and responsible for the provision of value-added services. He studied computer science at the University of Bonn and is co-author of two books about X.400 and about distributed group communication. At present, he is the convenor of the TERENA working group on network operation (WGNOP) and a member of the TERENA Technical Committee and the Internet Society.
[8] B. Shneiderman, Response Time and Display Rate in the Human Performance with Computers, Comp. Surveys 16, 1984 [9] Internet Activities Board, „Guidelines for Internet Measurement Activities,„ RFC1262, October 1991. [10] ftp://ftp.gmd.de/gmd/SW-Quality
Hans-Ludwig Hausen received degrees in electrical engineering and in computer science from the Technical University of Berlin. He worked as teacher and consultant at the Technical University of Berlin and later at GMD. Within GMD he has worked on computer-aided software engineering environments and on software quality engineering. During the last five years, he has been working in several European projects and initiatives on software quality and productivity. In ISO he has defined a software quality and certification scheme.
[11] McCall, J.A.; Richards, P.K.; Walters, G.F.:"Factors in Software Quality". Springfield, Va, NTIS, AD-A049 014, November 1977 [12] L. Bannon, K. Schmidt: "CSCW: Four Characters in Search of a Context", in: "Proceedings of the 1st European Conference on CSCW", London, 1989 [13] J. M. Bowers, S. D. Benford (Editors): "Studies in Computer Supported Cooperative Work • Theory, Practice and Design", North-Holland, 1991
Rainer Worst received a degree in computer science from the University of Bonn. He worked three years in a major software house as system analyst and programmer. Then he joined GMD in 1984, where he became head of the software methods and tools group. From 1991 to 1993, he was hired out to the Federal Ministry for Research and Technology, where he was responsible for parts of the programme Quality Management. He is now working at GMD's software engineering department and is focusing on quality systems.
[14] S. Greenberg (Editor): "Computersupported Cooperative Work and Groupware", Academic Press, Harcourt Brace Jovanovich, Publishers, Computers and People Series, London, 1991 [15] I. Greif (Editor): "ComputerSupported Cooperative Work: A Book of Readings", Morgan Kaufmann Publishers, San Mateo, 1988 [16] Chris Adie, A Survey of Distributed Multimedia , Edinburgh University Computing Service, January 1993 [17] Cornell University: gated.cornell.edu: /pub/video [18] D. Tsichritzis, J. Schäfer, POLIKOM • Feeding Innovation into Telegovernment, GMD, Sankt Augustin, 1992 [19] Manfred Bogen, Karl-Heinz Weiß: Group Coordination in a Distributed Environment, in: Group support systems, Thomas Kreifelts, Horst Santo (Eds.), Bericht der GMD Nr. 234, pp67-86, R. Oldenbourg Verlag, München/ Wien 1994, Artikel
12
