Computer-Assisted Evaluation of Undergraduate Courses ... - ece.ufrgs

224

IEEE TRANSACTIONS ON EDUCATION, VOL. 49, NO. 2, MAY 2006

Computer-Assisted Evaluation of Undergraduate Courses in Frequency-Domain Techniques for System Control José Felipe Haffner, Luís Fernando Alves Pereira, and Daniel Ferreira Coutinho, Member, IEEE

Abstract—An innovative computer-assisted teaching technique is proposed as an effective way to help in the assessment of undergraduate courses in classical control theory. Using an iterative Matlab/Simulink based algorithm, the task of identifying an unknown linear system from its frequency response is given to the students. At each step, the student improves his/her approximation to minimize the frequency response of the mismatch and decides whether the estimate is sufficient. The proposed self-evaluation task gives the students a means to analyze Bode diagrams and system identification techniques, nontrivial issues in introductory courses in control systems theory. Index Terms—Control systems education, educational software, frequency-domain techniques, self-evaluation, transfer function identification.

I. INTRODUCTION

T

ODAY, new educational methods and practices constitute a great challenge to educators in almost all areas of teaching. Course structures encourage teaching practices in which students “learn by doing,” with classroom teachers presenting the theoretical basis. The students build their knowledge in further hours of study outside the lecture room. Teaching/learning activities must necessarily involve extensive student participation. Therefore, the successful teaching is directly related to the time spent by teachers in lecture preparation (lecture notes, examples, etc.), and the successful learning is directly dependent on the students’ effort to solve the topics given to them [1]. Expressed in this form, the problem of teaching and learning is oversimplified, giving the false impression that a trivial solution might be expressed as follows: “The professor’s job is to prepare and present material, and to assess the results, and the student’s duty is to learn and to study hard.” Equally trivial is the conclusion that each student has a different pace of learning characterized by various factors. Among them, the time that each student spends on the tasks given to him/her, the ease with which he/she arrives at the solution of problems associated with the material presented, the student’s degree of interest in the material, and the degree of empathy in the relation between the professor and the student makes a problem that appears to be easy quite complex.

Manuscript received July 19, 2004; revised December 7, 2005. The authors are with the Department of Electrical Engineering, Pontifícia Universidade Católica do Rio Grande do Sul, Porto Alegre-RS 90619-900, Brazil (e-mail: [email protected]; [email protected]; [email protected]). Digital Object Identifier 10.1109/TE.2006.872406

This paper describes an approach that attempts to solve the problem, one that has been used successfully by teachers in courses related to control systems within the disciplines of electrical engineering, computer engineering, and control and automation engineering at the Pontifical Catholic University of Rio Grande do Sul (PUCRS) in southern Brazil. The model for “learning by doing” has been presented to students in the form of a dynamic and iterative procedure. As pointed out by Johansson et al. in [2], iterative tools can stimulate the development of the student’s engineering understanding that is very important in courses founded on a strong mathematical background. An important issue to mention is that the courses implementing the proposed self-evaluation technique exist in the majority of electrical and mechanical Bachelor of Science level courses in engineering around the world, where the educational methods are based on expositive teaching with constant evaluation by means of tests, exams, and individual or group homework. The final grade assigned to the student is a function of all forms of assessments, although the formal evaluation is still considered in the course (that is, the overall teaching-evaluation process is handled by the course lecturer). The student is given an alternative way of learning some important issues through an iterative task. The proposed task is carried out individually by the student, and he/she can obtain that final grade that depends on these skills on the topic to be addressed. The method lies well within the capability of most students belonging to the computer-literate generation, where the results of the interactions are given in real time, stimulating and encouraging them to develop still further their abilities in solving the proposed task. One topic in control systems in which this method is applied concerns the frequency response of linear, time-invariant systems. The frequency response is a topic of particular interest, because the basic fundamentals of classical control theory are compulsory material in most engineering courses [3], [4]. A good grounding for students about to embark on work relating to methods of frequency response is fundamental for the understanding of various topics in the field of control systems since the response characteristics of linear, time-invariant systems are used, for example, in assessing robustness, model identification, order reduction, harmonic linearization, and controller design. The difficulties of teaching material related to frequency response are widely recognized since the phenomena are described in a domain unknown to, or little explored by, most students. Such difficulties become the first barrier to students’ understanding of the material and often result in a marked loss of

0018-9359/$20.00 © 2006 IEEE

HAFFNER et al.: EVALUATION OF COURSES IN FREQUENCY-DOMAIN TECHNIQUES

motivation and a consequent lack of interest in topics presented in the classroom, topics that directly affect their course grades. Many successful initiatives have been proposed by several researchers toward the development of iterative tools used in the teaching/learning in the area of automatic control, such as [1], [2], [6], [7]. Computer-assisted interactive teaching is described in this paper as an effective way to help both professors and students in teaching/learning techniques of frequency response. More precisely, this paper presents an interactive proposal for identifying linear, time-invariant systems following the approach described by Nise in [8] and used each teaching semester by an average of 200 students in the area of control systems, which uses the simulation environment Matlab/Simulink as described in the following sections. In spite of several educational aspects related to this topic, this paper is focused on the technical aspects of the proposed technique. II. STUDENT ASSESSMENT IN CONTROL COURSES AT PUCRS The theoretical basis for control systems studies at PUCRS is divided into two courses: analysis and design. In the first course, the students are introduced to the concepts related to the analysis of open- and closed-loop systems using the classical frequency-domain methodology. The second one works with the design problem considering frequency- and time-domain techniques. Each course consists of three parts that are evaluated separately by standard assessment methods. To stress the student’s knowledge on the frequency-domain methodology,1 in both courses, out-of-class student activities are also regularly used as an integral part of performance evaluation. Working with an average contingent of 200 students, preparing and evaluating work for out-of-class study, although fundamental for student development, are often difficult tasks to execute, not only in terms of preparing different tasks but also in terms of evaluating and awarding grades for each piece of work undertaken by each student. In addition, the successfully proposed tasks are directly related to the time spent by students and to the level of interest in each student to accomplish the out-of-class activity. After several discussions between professors and teaching assistants, the proposed idea is to introduce a self-evaluation, out-of-class study that should be enjoyable to the students and at the same time an efficient technique for students’ evaluation. The culture associated with self-study is introduced in the first part of the introductory course in control systems, where students learn to use the simulation environment Matlab through a self-explanatory program developed in Matlab itself. In this program, the student learns interactively the use of vectors, matrices, writing mathematical functions, and displaying these functions graphically. The student also learns to use the programming elements of Matlab in the environment for simulating dynamic systems integrated with Matlab, known as

1The frequency-domain techniques are considered one of the most important topics in undergraduate courses in system control [3], [8].

225

Simulink. From the moment at which students learn to use simulation software adequately for all exercises, including those developed in practical laboratory classes, students make use of Matlab/Simulink, especially in out-of-class exercises. Various types of out-of-class activity exist, but in all such activities certain characteristics must be followed. The first characteristic concerns the specific nature of each task allocated, where each student receives an individual (and unique) work. The individual characteristics are achieved through the automatic generation of different dynamic systems, uniquely for each student. The second characteristic is related to the degree of interactivity that the proposed exercises must show, which is satisfied by giving the student the possibility to evaluate each step in the work’s development. From the exercises, the students should be able to draw conclusions about the validity of the results, whether or not progress is being made in the right direction. The third characteristic is associated with the building of the motivation that the student needs to complete the work, which is achieved by informing the student of the score he/she was given at each step or phase in working through an exercise. III. FREQUENCY-RESPONSE ITERATIVE PROCEDURE The problem given to students is that of identifying the transfer function starting from the system’s Bode diagram. In general, the student’s task is to determine adequately the poles, zeros, and dc-gain of the system to be identified. The method used to achieve this task is described in [8] and consists basically of the step-by-step reconstruction of the system transfer function given by the equation (1) with and , , and , . The final objective of the given exercise is to obtain a transfer function which approximates (1), of the form (2) The task is undertaken by students by means of constructing (2) using terms such as

and

(3)

where , , , and . These parameters must be appropriately inserted in the numerators and denominators of the transfer functions identified by students. The identification procedure consists of obtaining successive differences between the magnitude and phase curves of the real and the identified system. At each interaction, students have access to the Bode diagrams of the errors in magnitude and phase, which will give clues to terms not yet included. After all poles and zeros have been included in the transfer function identified by the student, the work begins to locate which of these poles and zeros are the

226


TABLE I INPUT PARAMETERS TO THE PROGRAM SYSTEMG.P

Fig. 1.

Generating the frequency response

G(s).

nearest possible to the singularities of the systems being identified. In general terms, the problem is to optimize the function2

(4) where the vector contains the coefficients in the numerator and denominator of the approximated transfer function. The task expressed in (4), a formal optimization problem, might alarm the students. Nevertheless, they can solve the problem in a straightforward way using as a basis the visual information of the two curves: the magnitude and phase error plots between the transfer functions (1) and (2). IV. SETTING THE STUDENT BASIS ON FREQUENCY-RESPONSE METHODS Although the main task given to the students is to identify a linear time-invariant (LTI) system by frequency-response methods, the magnitude and phase data are only available to the student after he/she demonstrates a certain level of knowledge on the subject. Therefore, a Matlab based program, called systemg.p, is supplied to the students for generating a different transfer function for each individual student enrolled in the course. The systemg.p program is already compiled, and students have no access to the source code, where the input arguments given to the students are presented in Table I. In the Matlab environment, the student must type the commands (shown in Fig. 1) which correspond to data related to the frequency response in an interval defined by a lower frequency up to rad/s and the upper limit of frequency limit, which is divided into 1000 steps of frequency. Fig. 1 shows how the example data in Table I are inputted by the students (the first student in group 450). The student must complete a set of ten questions about frequency response analysis before the program systemg.p returns to the Matlab environment. The students respond to these questions based on concepts presented with their graphical interpretation, as shown in Figs. 2–6. The first screen presents to the student the dc gain definition in both time- and frequency-domain, as illustrated in Fig. 2. At this 2In fact, the cost function also takes into account the relative distance between the true pole-zero location and their estimates, as shown later in this paper.

point, the student is encouraged to determine the dc gain assisted by the mouse cursor function, using either the Bode diagram or the system step response. Another important concept introduced to the student is the relative degree of a single-input/single-output (SISO) LTI system. Similar to Fig. 2, the students receive the concept definition and its graphical meaning in view of the Bode diagram as illustrated in Fig. 3. In light of Fig. 3, the student should answer the following questions. 1) What is the system phase in high frequencies? 2) What is the system relative degree? 3) What are the number of finite zeros and poles of the system? 4) What are their locations from the magnitude and phase plots? Questions 1) and 2) appear simplistic to answer. The remaining questions are not so simple, since the student needs to recognize with some accuracy the location of the system poles and zeros by means of their influence on the Bode diagram. Completing the student knowledge on frequency-domain methods, the steady-state response of sinusoidal signals is introduced to the student establishing a connection between the frequency- and time-domains. Basically, a new window is presented to the student with the sine input and the respective output signal in which the time axis is normalized from zero to one, as given in Fig. 4. From the information regarding the amplitude and phase of the sinusoidal response, the student should be able to determine the following: 1) the frequency of the input signal; 2) the gain in decibels to be added to the system so that the is satisfied. condition In addition, for a given input signal as shown in Fig. 5, the student has to determine the output response in steady-state using the information given by the Bode diagram in Fig. 2. In summary, the student must answer a total of ten questions about the basics on frequency-response of SISO LTI systems. The evaluation of these questions is based on the criteria and stand for the small and stated in Table II, where large singularities of the system, respectively. Questions 1–3 in Table II are automatically answered whenever the function systemg.p is called.


227

Fig. 2. Definition and graphical interpretation of the dc gain.

Fig. 3.

Relative degree: definition and graphical meaning.

If the student is able to answer all questions correctly, as illustrated in Fig. 6, the data regarding the Bode diagram (magnitude, phase, and frequency) of the unknown system are then available to the student, allowing the estimation of the system transfer function.

Whenever a question is not correctly answered, the student must start the process from the beginning. No hint about the wrong subject is given to the students. This avoidance of error prevents a trial-and-error process. Obviously, at each tentative response the system is the same, but the values of the frequency

228


Fig. 4. Sinusoidal steady-state response: input and output signals.

Fig. 5. Sine input signal with known amplitude and frequency.

and amplitude in questions 9–12 are randomly generated (again avoiding trial and error mechanisms). At this stage, the student is encouraged to look for help with the lecturers and assistants of control systems courses. V. ENSURING THE UNIQUENESS AND FAIRNESS OF EACH STUDENT EXERCISE To each individual student enrolled in the course, a different is generated by means of the program transfer function

systemg.p. An important issue to be considered when designing all transfer functions is to guarantee that the level of difficulty should be as fair as possible. in obtaining the estimation , the relative degree, the number of Therefore, the order of complex singularities, and the width of the bandwidth are similar in all system models, and the level of difficulty in estimating them is in some way equalized, guaranteeing the fairness of each student’s exercise. Table III shows the values assumed when designing the systems to be identified.


Fig. 6.

229

Set of questions prior to the system identification.

TABLE II EVALUATION CRITERIA ON THE STUDENT BASIS

Also, all systems are stable and minimum phase to avoid different levels of difficulty to solve the proposed task. A question that arises is how a relatively great number of systems are generated with similar characteristics, yet at the same time with reasonably different Bode diagrams. A possible solution could be obtained by means of the function rmodel.m of Matlab that generates random systems of the same order. One has, however, no way of controlling the relative degree, the number of complex singularities, and the bandwidth. The solution used by the authors is to construct randomly a large database of systems having an order of four or five and choose the systems with the characteristics stated in Table III. VI. SYSTEM IDENTIFICATION EXERCISE To simplify the task proposed to the students, the interactivity of the exercises is achieved through the use of a second program called test.m, which the students must use to test whether the apis satisfactory. Since the technique used for proximation

TABLE III CRITERIA USED TO DESIGN THE UNKNOWN SYSTEMS

identification, as proposed in [8], consists of including first- and second-order terms as given in (3), the open code of the test.m program and main programming blocks are supplied to the students. Each block of the program refers to the singularities that the students must include, presenting for each block added to the magnitude and phase errors of relative to . More precisely, the first step to start the system identification task is to plot the Bode diagram from the data obtained from the program systemg.p, as illustrated in Fig. 7. From a visual inspecin the test.m tion, the student may write the first block of program, as shown in Fig. 8. In this case, the student included rad/s and . The a second-order term with and values was obtained through the first approximation of Bode diagram observation, using the cursor facilities of Matlab over the magnitude and phase curves, as demonstrated in Fig. 7. considering the dc-gain Thus, the first approximation of is (5) which is still far from , as illustrated by the Bode diagram for magnitude and phase errors between the real transfer funcand the approximation , shown in Fig. 9. tion , the student can include a To improve the approximation second block composed by a well defined pair of real zero-pole behavior between the frequencies 30 (zero) and 200 rad/s (pole), and another one between the frequencies 300 (pole) and 1000 rad/s (zero). As a result, a more accurate approximation of

230

Fig. 7.


Bode diagram of the system to be identified.

Fig. 8. Line commands containing the first term of

G(s).

is obtained, as illustrated in Fig. 10, which is given by the following transfer function: (6)

At this stage, a student’s question might be: Is the estimate identified in (6) a good approximation of the real system transfer function? An answer to this question is given by means of a third program called score.p, whose use is described in the next section of this paper.


Fig. 9.

Fig. 10.

Bode diagram for the error between the functions

G(s) and G(s).

Magnitude and phase error of the second approximation.

231

232

Fig. 11.


Bode diagram of

G(s) (continuous line) and of G(s) (dotted line).

VII. EVALUATION OF PERFORMANCE After the first attempt on estimating the transfer function, the student approximation should be evaluated. Therefore, the professors must consider some criterion to assess how near the lies magnitude and phase plots of the identified function . The Bode in relation to the transfer function of interest diagrams of the two transfer functions are shown in Fig. 11. The inspection of this figure shows that the Bode diagrams are similar and could perhaps be considered a good approximation . Possibly, if the student is satisfied with the result of of his/her work, which visually appears to be a good approximation, he/she would end the search at this point. Perhaps, the result of the professors’ assessment of the work would not be as good as the student’s expectations, again raising the old problem of methods of evaluation. In the end, how can the student know if he/she has done a good work before receiving the professors’ final judgment? The answer to this question is given by the third program issued to students, called score.p. With this program, students are able to find out, in simple terms, how good their work has been. Using the program score.p, the student is told about the result of his/her work, in real time. In addition, whenever the identified system is not accurate, some hints are given to the . For the student to students to improve the approximation use the program score.p, he/she must enter the set of parameters given in Table IV. Fig. 12 shows the screen resulting from the use of the score.p program to evaluate the student’s performance, taking the

TABLE IV PARAMETERS USED AS ARGUMENTS TO THE PROGRAM SCORE.P

given by (6) as the first attempt toward transfer function . Upon activating the score.p program, the stuidentifying dent receives a set of messages, such as the score awarded, the cause of a nonmaximum score, and the position of the identified singularities. The student’s score varies over an interval from 0.0 (minimum score) to 10.0 points (maximum). In the example (see Fig. 12), the student received an intermediate score of 8.9 points. If the student considers that the score is not high enough, he/she is encouraged to continue working on the exercise until a better grade is achieved by a fine-tuning on the worst approximated singularity. Using the Bode diagrams in Figs. 11 and 12, the student must perceive that between frequencies 10 and 10 rad/s, the phase curve of the identified system lies below the phase curve of and that the singularities of located near this frequency interval have to be readjusted. Such observations help the student to understand visually the influence exerted by the positions of poles and zeros on the system’s


Fig. 12.

Screen showing the results of using the score.p program.

Fig. 13.

An improved approximation generates a better score.

233

234


TABLE VII GRADE DISTRIBUTION FROM THE SINGULARITIES LOCALIZATION

TABLE V MEAN GRADES IN THE FIRST COURSE IN CONTROL SYSTEMS

TABLE VI GRADE DISTRIBUTION IN TERMS OF THE MAGNITUDE AND PHASE ERRORS

magnitude and phase curves. The process of fitting poles and is repeated so that the student decides what score zeros of he/she wishes to get for the work. Reallocating the pole from 300 to 330 rad/s, the student obtains the result in Fig. 13 after applying the score.p program to the new approximation. Comparing the result presented by the program score.p, the student has achieved a significant improvement on the transfer function originally identified while reaching the maximum score. The improvement is only possible because at the end of each fitting cycle giving the poles and zeros of (6), the student runs the program score.p evaluating whether the proposed or make it worse. changes improve the approximation of In Table V, the proposed methodology is compared with two standard out-of-class works with similar levels of difficulty. Two student groups over the course of two years performed the three types of exercises, and the interactivity characteristics of Exercise 2 (proposed evaluation in this paper) has led to an overall performance improvement. VIII. GENERATING THE SCORE A crucial point on the proposed approach is the determination of the student’s grade by the program score.p. The grade should evaluate the student successful in realizing adequately the proposed job and, at the same time, avoiding a trial-and-error process to get the maximum grade. Hence, the final grade of each student is generated by the average between two criteria, where the score ranges from 0 (minimum) to 10 (maximum). The first criterion is obtained from a comparison between the original Bode diagram and the identified one, taking into account the argument of the cost function as defined in (4). In this case, the grade is established as stated in Table VI considering is detera linear distribution. In addition, the first criterion mined as follows: (7) where stands for the score of the magnitude error and the phase error, both considering (4). The second criterion measures the relative distance between the singularities from the real and the identified transfer functions. Here, the score is assigned considering a linear distribution from the minimum and maximum scores as attributed in Table VII.

For each zero and pole of the transfer function , the average error is calculated for the equivalent singularities. Then, an intermediate score is attributed to each pair, taking into account a linear distribution of the scores specified in Table VII. The score, with respect to singularities localization error, is then computed by means of the following expression:

(8)

where is the th zero of , the th identified zero, the th pole of , the th identified pole, a linear function that assigns each singularity score as above, and the score which represents the singularities error. The subsequent discovery is that the above criterion can be determined only whenever one can infer what the equivalent singularities and are. between To generate the final score, the following procedure is applied. 1) Determine the score from and considering Table VI and the formula defined in (7). from Table VII and (8). 2) Similarly, compute . 3) Obtain the intermediate score 4) If , then ; else . , then ; else . 5) If In the above algorithm, the objective of the fourth step is to avoid an erroneous equivalence among the real and the identified singularities that may appear when the mismatch between and is large. In addition, the fifth step is added to the score computation to overcome useless trial and error tentatively to refine the score. IX. CONCLUDING REMARKS An innovative technique to the automatic evaluation is a proposal for the preparation of teaching material bringing with it alternative methods and trends for application in engineering courses, using less time spent in the lecture room and more time dedicated to out-of-class study. The idea is to present the student with a kind of interactive procedure in which better performance coincides with time spent in study and the consequent level of proficiency in the work. The objective of automatic evaluation is not only to produce a score at the end of the work but also to provide support for the student. This support is obtained at each stage of the work’s development by generating graphs illustrating concepts and messages showing how the student can overcome the problematic situation. Initial results obtained with student groups in which the methodology has been applied have been remarkable, notably in terms of student satisfaction and interest. Currently, the authors are extending the approach for the design of frequency-domain compensators.


ACKNOWLEDGMENT The authors would like to thank the anonymous reviewers for their valuable comments improving the quality of the paper and the revision on the writing carried out by Dr. M. das Graças Furtado Feldes and Dr. A. Sanfelice Bazanella.

235

José Felipe Haffner received the B.E. and M.S. degrees in electrical engineering from Pontifícia Universidade Católica do Rio Grande do Sul (PUCRS), Porto Alegre, Brazil, in 1986 and 1998, respectively. He is a Lecturer of electrical, control and computer engineering at PUCRS. His current research interests are control of electrical machines and computerassisted education technology applied in control systems area.

REFERENCES [1] G. J. C. Copinga, M. H. C. Veraegen, and M. J. J. M. van de Ven, “Toward a web-based study support environment for teaching automatic control,” IEEE Control Syst. Mag., vol. 20, no. 4, pp. 8–19, Aug. 2000. [2] M. Johansson, M. Gáfvert, and K. J. Aström, “Interactive tools for education in automatic control,” IEEE Control Syst. Mag., vol. 18, no. 3, pp. 33–40, Jun. 1998. [3] P. A. Dorato, “Survey of control systems education in the United States,” IEEE Trans. Educ., vol. 33, no. 3, pp. 306–310, Aug. 1990. [4] B. W. Bequette and B. A. Ogunnaike, “Chemical process control education and practice,” IEEE Control Syst. Mag., vol. 21, no. 2, pp. 10–17, Apr. 2001. [5] D. S. Bernstein, “The frequency domain,” IEEE Control Syst. Mag., vol. 20, no. 2, pp. 8–14, Apr. 2000. [6] R. C. Garcia and B. S. Heck, “An interactive tool for classical control design education,” in Proc. Amer. Control Conf., San Diego, CA, Jun. 1999, pp. 1460–1464. [7] V. Petridis, S. Kazarlis, and V. G. Kaburlasos, “ACES: an interactive software platform for self-instruction and self-evaluation in automatic control systems,” IEEE Trans. Educ., vol. 46, no. 1, pp. 102–110, Feb. 2003. [8] N. S. Nise, Control System Engineering, 2nd ed. Menlo Park, CA: Addison-Wesley, 1996, pp. 598–601.

Luís Fernando Alves Pereira received the B.E. degree in electrical engineering from Pontifícia Universidade Católica do Rio Grande do Sul (PUCRS), Porto Alegre, Brazil, in 1987 and the M.S. and Dr. degrees from Instituto Tecnológico de Aeronáutica, São José dos Campos, Brazil, in 1989 and 1995, respectively. He is currently an Associate Professor of electrical, control and computer engineering at PUCRS, where he works on theoretic and applied nonlinear control.

Daniel Ferreira Coutinho (S’00–M’03) received the B.E. degree in electrical engineering from Universidade Federal do Rio Grande do Sul (UFRGS), Porto Alegre, Brazil, in 1989 and the M.S. and Dr. degrees from Universidade Federal de Santa Catarina (UFSC), Florianópolis, Brazil, in 1993 and 2003, respectively. He was a Visiting Researcher at the School of Electrical and Computer Engineering, University of Newcastle, Australia, in 2001. He is currently an Associate Professor of electrical and control engineering at Pontifícia Universidade Católica do Rio Grande do Sul, Porto Alegre. His research interests are the robust control and filtering of nonlinear systems with applications to dc–dc power converters, power systems, and induction motors.