Guest Editorial on Measurement - IEEE Xplore

Guest Editorial on Measurement ■ Charles Wilker and Dominique Schreurs

T

he guest editors would like to acknowledge the significant help of Kate Remley, MTT-11 vicechair, in preparing this focused issue. Test and measurement is often only an afterthought in many projects. In the RF community, substantial time, energy, and resources are spent in the optimization of the design models, design layouts, and fabrication processes, but all of these depend on good measurements. Good designs rely on having good models, which could not be derived without the help of good measurements. Dependable fabrications rely on having dependable materials and dependable processes, which also could not be developed without the help of good measurements. Quality products rely on meeting specifications, which could not be verified without good measurements. Depending on the needs of the project, the measurements may need to be repeatable, reproducible, precise, accurate, valid, reliable and/or a combination of all of these. One would also like to have confidence Charles Wilker, ARFTG liaison to MTT-S, is with DuPont, Wilmington, Delaware, USA. Dominique Schreurs, chair of MTT-11, is with Katholieke Universiteit Leuven, Leuven, Belgium. Digital Object Identifier 10.110 9/MMM.2007.901160

6

© CREATAS

in the measurement, which implies a detailed knowledge of the measurement errors and uncertainties. The ability to perform good measurements is thus a critical and necessary part of any project. Fortunately, there are resources available within the Microwave Theory and Techniques society (MTT-S) devoted to helping engineers with their test and measurement needs. This issue of IEEE Microwave Magazine is being sponsored by two separate technical organizations, Microwave Measurement

Technical Committee (MTT-11) and Automatic Radio Frequency Techniques Group (ARFTG). MTT-11 is one of the 23 technical committees organized within MTT-S. MTT-11 promotes activities related to microwave measurement technology and science. This includes furthering the investigation, understanding, art, and practice of microwave measurement. The committee works to increase both the visibility and vitality of the measurement community within

August 2007

MTT-S. It stimulates, develops, and sponsors quality technical sessions and workshops within IMS and other venues, provides Web-based technical measurement forums moderated by experts, develops quality educational tools, and develops other ways of disseminating information related to microwave measurements. The technical committee works with the MTT-S Standards Coordinating Committee in developing and establishing IEEE standards and best measurement practices in the microwave and millimeter-wave field. For more information about MTT-11, please visit our Web site at www.mtt.org/ committees/mtt-11. ARFTG is not a part of the IEEE but an independent, international, nonprofit technical organization formed in 1972, which is interested in all aspects of RF and microwave test and measurement. Each of the two ARFTG conferences held every year are technically cosponsored with MTT-S. ARFTG membership includes a broad if eclectic mix of government metrologists, academic researchers, and industrial practitioners, and like MTT-11, ARFTG provides a forum for the development of new RF and microwave test and measurement technologies, methodologies, and instrumentation. There is always ample opportunity at every ARFTG conference for detailed technical discussions with others facing similar test and measurement challenges. The members of ARFTG often find that these interactions are their best source of ideas and information for their current projects. So come and join us at our next conference, short course, or workshop. You’ll find that the atmosphere is informal and quite friendly. For more information about ARFTG, please visit our Web site at www. arftg.org. The main topic of this issue of IEEE Microwave Magazine is the theory and practice of RF and microwave measurement, focusing on uncertainty, new techniques, and methods for verification. This issue is composed of a number of articles that address various aspects of interest to the RF and microwave test and measurement community.

8

• Measurement Uncertainty, Traceability, and the GUM GUM is an acronym for the Guide to the Expression of Uncertainty in Measurement first published in 1993 by the International Organization for Standardization (ISO). It is a standardized method for evaluating and expressing the uncertainty of a measurement. However, bearing in mind that the GUM runs to around 100 pages in length and contains some fairly advanced mathematics and statistics, the article is a more digestible version appropriate for the beginner or casual observer in this field. • Confidence in VNA Measurements The ARFTG Measurement Comparison Program (MCP) is an alternate way to look at your overall VNA-based measurement processes. The ARFTG MCP is designed to give a snapshot of your capability by anonymously comparing S-parameter measurements from your VNA to those of other labs. By using this comparison, you can uncover problems (if there are any) in your measurement process that you may not even know you had. Additionally, it can promote confidence in your capabilities. Many accreditors are now accepting the ARFTG MCP as proof of proficiency testing. • The Sampling Oscilloscope as a Microwave Instrument Many modern high-speed oscilloscopes are well suited for precise measurements of microwave waveforms, modulated signals, and nonlinear phenomena. Like all microwave instruments, characterizing and correcting for the instrument’s imperfections are fundamental to making accurate measurements. With suitable corrections, these high-speed sampling oscilloscopes can be used not only to debug circuits, but also to perform metrology-grade measurements at microwave frequencies up to 100 GHz. Most of the equipment required is readily available in most microwave labs: a vector network analyzer, a microwave

signal generator, and, of course, a sampling oscilloscope! • The History of the Automatic RF Techniques Group ARFTG was founded in 1972 to create a more cohesive voice for the RF and microwave test and measurement community to solicit support from the manufacturers of test instrumentation. The primary focus of the first few meetings was on the automation and calibration of network analyzers. The early years of ARFTG were more like a users group than the professional measurement society ARFTG would become. In fact, the early meetings were not a pleasant place for the representatives of the instrumentation manufacturers. The magic of these early test systems was embedded and hidden from the user in the software, which was then not available in source form. ARFTG was formed to help the manufacturers realize and appreciate the needs of the users. • Key Nonlinear Measurement Events There are two informal meetings where microwave engineers can learn techniques, share experiences, and discuss the latest research in nonlinear and large-signal measurements. The ARFTG Nonlinear Measurements Workshop is an annual event that covers a topic of interest related to nonlinear measurements in a presentation/discussion format. The Nonlinear Vector Network Analyzer (NVNA) Users’ Forum is an informal discussion group that meets three times a year and is devoted to sharing information and issues related to instrumentation utilized in vector large-signal network analysis of microwave circuits and systems that contain nonlinear elements. • EM-Component-Based Design of Planar Circuits The techniques developed by the test and measurement community for deembedding and calibration have been used to enhance the capabilities of electromagnetic (EM) design software. A new design methodology is introduced based on the development of perfectly

August 2007

Measurement Concepts Let us spend a little time discussing the some of the concepts commonly understood (or maybe misunderstood) around making good measurements. Some of the questions that we would like to be able to answer are: Do all of the measurements produce the same result? How confident is one with the calibration and thus the measured value? How does one know what one has measured? Is the result correct or more precisely, valid? What are the properties of a good measurement system or process? One definition of a good measurement might be simply that the measurement satisfies one’s needs. As stated previously, we would like to make a measurement that is repeatable, reproducible, precise, accurate, valid, reliable and/or a combination of all of these with some understanding of the measurement uncertainties. What Is Measurement Repeatability? Measurement repeatability is a determination of the performance of a measurement process run repeatedly under essentially the same conditions. According to the Guidelines for Evaluating and Expressing the Uncertainty of NIST Measurement Results, repeatability conditions include the same measurement procedure, the same observer, the same measuring instrument used under the same conditions, the same location, and repetition over a short period of time. Repeatability should provide an indication of the size of the uncontrolled random errors, e.g., noise that are present in a measurement process. A measurement is said to be repeatable when this variation is smaller than some agreed limit and is the best that can be hoped for in the overall measurement process. Ideally, one would like a measurement process that is at least an order of magnitude better than the desired measurement resolution. A measurement process that is not repeatable may require a large amount of averaging and thus in the end, may not be useful. What Is Measurement Reproducibility? Measurement reproducibility is a determination of the performance of a measurement process run repeatedly under typical measurement conditions. A measurement process usually requires some disassembly of the measurement system to change the device under test (DUT) to another device or circuit or perhaps to insert a calibration artifact. Reproducibility should provide an indication of the total of all of the random errors, e.g., connector repeatability, operator repeatability, fixture repeatability, etc. Again, a measurement is said to be reproducible when this variation is smaller than some agreed limit. Ideally, one would still like a measurement process that is at least an order of magnitude better than the desired measurement resolution. A measurement process that is not reproducible may require a large amount of averaging and thus in the end, may not be useful.

10

What Is Measurement Precision? Measurement precision is a determination of the performance of a measurement process run repeatedly under the entire range of measurement conditions. Measurement errors generally fall into one of three categories: random errors and two types of systematic errors, bias and drift. A measurement process that is repeatable and reproducible has controlled the random errors. Bias errors, on the other hand, cannot be removed by repeating measurements or averaging large numbers of results. However, since bias errors do not change over time, they can be corrected by calibration. Drift errors, however, can shift over time and may include changes in the laboratory temperature or humidity, wear of the test fixtures or connectors, or differences between operators or instruments. Precision is an indication of the performance of an uncalibrated measurement process where the variations due to the drift errors are smaller than some agreed limit. Ideally, one would still like a measurement process that is at least an order of magnitude better than the desired measurement resolution. A measurement process that this repeatable and reproducible but not precise may require frequent calibration but in the end is still probably useful. What Is Measurement Accuracy? Measurement accuracy is a determination of the performance of a calibrated measurement process run repeatedly under the entire range of measurement conditions. The intent of any calibration process is to remove all of the bias errors from the measurement process. Note that additional sources of error can be introduced by the calibration process itself, including random errors associated with the raw data for the calibration artifacts (from connector repeatability, noise effects, etc), random errors associated with the calibration process, and systematic errors resulting from imprecise knowledge about the calibration standards. Note that the results of a measurement process can be accurate but not precise, precise but not accurate, neither, or both. Ideally, a measurement is both accurate and precise, with measurements all close to and tightly clustered around some value. The accuracy and precision of a measurement process is usually established by repeatedly measuring some traceable reference standard. Accurate measurements are essential in many fields, and since all measurements are necessarily approximations, a great deal of effort must be taken to make measurements as accurate as possible. The study of measurement is called metrology. Metrology is defined by the International Bureau of Weights and Measures as “the science of measurement, embracing both experiment and theoretical determinations at any level of uncertainty in any field of Science and Technology.” Metrology is a very broad field and may be divided into three subfields:

August 2007

• Scientific or fundamental metrology concerns the establishment of measurement units, unit systems, development of new measurement methods, realization of measurement standards, and the transfer of traceability from these standards to users in society. • Applied or industrial metrology concerns the application of measurement science to manufacturing and other processes and their use in society, ensuring the suitability of measurement instruments, their calibration, and quality control of measurements. • Legal metrology concerns regulatory requirements of measurements and measuring instruments for the protection of health, public safety, the environment, enabling taxation, protection of consumers, and fair trade.

able, reproducible, precise, accurate, and valid with an acceptable measurement uncertainty needs also to be robust. A robust measurement process always measures a similar value for the same artifact within the measurement uncertainty. Measurement robustness may include frequent calibration to properly compensate for drift errors, careful monitoring of the measurement uncertainty, and frequent cross checks between operators, instruments and/or laboratories, etc. Ultimately, one would like to be able to trust the measurement, this is measurement reliability. What Is a Good Measurement? A good measurement in the end is simply that which satisfies ones needs. A metrologist might require days on a multimillion dollar instrument to make a good measurement in order to achieve the minimum measurement uncertainty and maximum measurement validity possible. An engineering student might require only a few moments with an ohm meter to make a good measurement to satisfy his or her professor. A high throughput manufacturing engineer might require only a few microseconds to make a good measurement with just enough uncertainty to guarantee a specification. The common thread for all of these measurements is the assessment of the measurement uncertainty that is used to monitor the measurement process and to express the confidence of the measurement result.

What Is Measurement Validity? Measurement validity is a determination of the performance of a calibrated measurement process in gauging the actual (or true) value of some quantity. Since any a priori knowledge of this value must be derived from other measurements, it is extremely difficult if not impossible to know with absolute certainty the actual (or true) value of any quantity. (This is as much a statement of philosophy as of engineering.) The best that can be hoped for is that the entire RF community comes to a common understanding that an agreed upon measurement of an agreed upon thing has an agreed upon value. Hidden or unknown bias errors may be complex, and their effects are only observable if and when they are removed. Therefore, one can never be certain if the calibration process has removed all of the bias errors. The RF community as a whole has used a number of methods to reduce the uncertainty of the measurement validity or equivalently, to increase the agreement between common measurement processes. Some of these methNot Repeatable and Not Reproducible Repeatable but Not Reproducible ods include performing a subsequent Therefore Not Precise, Calibration and Therefore Not Precise, Calibration and measurement with more sophisticated Therefore Accuracy Will Be Problematic, Therefore Accuracy Will Be Problematic, equipment, measuring a transfer stanValidity Is Unknown Validity Is Unknown dard, or participating in a measurement comparison program. Note that this is one of the traditional roles of a national laboratory. The results of a measurement process can be valid but not precise, precise but not valid, neither, or both. What Is Measurement Reliability? Measurement reliability is a determination of the performance of a calibrated measurement process in actual use. A measurement process that is repeat-

12

Repeatable, Reproducible and if Stable Over Time, Precise, but Requires Calibration to Become Accurate and Possibly Valid

Repeatable, Reproducible, Precise, Well Calibrated and Therefore Accurate but Is It Valid?

August 2007

calibrated internal ports into the EM analysis software. These ports allow the insertion of separately analyzed components into an overall analysis of a complete circuit. Both the potential and the limitations of this approach are carefully examined. • MTT-11 Web Page: A Multitude of Information Resources on Microwave Measurements There are many valuable resources available on the MTT-11 Web site, www.mtt.org/committees/mtt-11. The most important feature is the online forums Web pages, which are divided into five different areas: General Measurements, OnWafer S- and Noise-Parameter Measurements, Signal Integrity and Multiport Measurements, Nonlinear and Loadpull Measurements, and Coaxial VNA Measurements. These forums provide a way to discuss issues of measurements with top researchers in the area. IEEE members simply sign up and are then given access to past and future posts of questions

August 2007

Good measurements are an integral on measurement-related issues for part of modern science, modern engineersome of the key topics of the day. ing and the modern RF • Benefits of ARFTG industry. Good designs Membership rely upon having good ARFTG is a technical The ability to models that could not be organization interest- perform good derived without the help ed in all aspects of RF measurements of good measurements. and microwave test Dependable fabrications and measurement. is a critical rely upon having dependThe most stimulating and necessary able materials and part of the ARFTG part of any dependable processes that experience is the also could not be developportunity to inter- project. oped without the help of act one-on-one with colleagues, experts, and vendors of good measurements. Quality products the RF and microwave test and mea- rely upon meeting specifications that surement community. Whether could not be verified without good meayour interests include high-through- surements. The ability to perform “good” put production or one-of-a-kind measurements is thus a critical and intemetrology measurements, complex gral part of any RF project. Poor measurements can lead to systems or simple circuit modeling, small signal S-parameter or large- bad science, bad engineering, poor signal non-linear measurements, designs, poor fabrication processes, phase noise or noise figure, from dc poor products as well as wasted to lightwave, you will find within time, energy and especially money! ARFTG a kindred spirit or maybe So, use the resources available within even an expert. For more informa- MTT-11 and ARFTG to get the help tion about ARFTG, please visit the needed to make good measurements and successful projects. Web site at www.arftg.org.

13