Design and Implementation of a Robust and Reconfigurable Data Acquisition and Post-Processing System Matthew Alan Gray
Robert Glenn Robertson
Department ofElectrica1 and Computer Engineering, Mississippi State University, MSU. MS 39762, USA ma@@ ece.msstate.edu
Department of Electrical and Computer Engineering Mississippi State University, MSU, MS 39762. USA
[email protected]
William Buchanan, M.S.
Randolph F. Follett, Ph.D.
Department of Electrical and Computer Engineering Mississippi State University, MSU, MS 39762. USA huchanan@ ecc.msstate .edu
Department of Electrical and Computer Engineering Mississippi State University, MSU. MS 39762,USA
[email protected]
aequisition,lgging. and processing can be integral processes involved in the characterization of complex systems. However, the tasks involved in these processm become rumbenomewhen massive amounts of data are involved. Though data can be logged to disk with relative ease, identification and past-proeesring of the data can be difficult. If identification information is stored along with the measured data, p a n i n g and past-procewing can he greatly simplified. and the meaning of stored data can be reliably represented as more t h a n j u ~ Pt set of time-indexed unit-less VIIYII. Providing a platform that totally or partially automates these processes can free the tester 01 researcher from the hborioui t a s k o f p a n i n g stored data forpostpracessing. AbsImcI-Data
I. INTRODUCTION
The process of acquiring data, storing it to disk, and retrieving it later for post-processing is utilized in numerous applications. ranging from the testing and upcrational verification ofexisting devices to the measurement and characterization of new ones. In many cases, it is feasible to do each individual ask in said process manually, but when the acquisition of data results in relatively large datasets. the handling of data can become cumbersome and susceptible to human error. Identification ofrelevant information also becomes a problem ifthe format in which the data is stored has not been defined in such a way as to prevent any ambiguity between individual data streams in the dataset. The development of a system for automating these and other processes could relieve the operator of such w o m e s as data identification and parsing ofdata for analysis.
0-7803-7697-8/03/$17.00 0 2003 IEEE
242
An important consideration in the design of such a platform is the physical speed limitations ofthe hardware performing the acquisition. This limitation, more often than not, is a function of the data access rates ofthe hardware storage medium (in the case of a PC, this equates to the hard disk). For example, if the characterization o f a system requires the acquisition and processing ofeight channels at 40,000 samples per second (40 kS/s), this wouldresult in 320;OOO values being streamed to disk every second. lfeach value were represented in double-precision floating-point format, this would require a sustained stream-to-disk rate of roughly 2.44 megabytes per second. Though conventional hard disks can provide data burst rates in excess ofthe required stream-to-disk ratc in the aforementioned example, there is no guarantee that the disk can maintain the required rate reliably. Possible hardware considerations for such requirements may include a RAID-0 or RAID-5 multiple disk array configuration to allow for parallel disk accesses. thus increasing disk access performance by a factor of n, where n is the number of nonredundant hard disks in the RAID array configuration. Data archival is another important issue to address, considering that acquisitions that occur over significant intervals of time can generate large amounts of data. In the previous example, an acquisition taken over one hour with the aforementionod parameters a n generate about 8 . 5 8 gigabytes. Though tape backup systems c m handle this capacity many times over, recovering the data can take a considerable amount oftime. On the other hand. more accessible high-capacitymedia types such as DVDRecordable media can onlyhandle around 4.7 gigabytes (a
little over half an hour worth of acquired data) at the present time. Robustness and configurahility of the platform for use in various applications is also important in that it extends the lifetime of the application in terms of usability, but also provides a powerful tool to the researcher of tester that can be used where requirements range from the characterization of high-power high-frequency components to the analysis of signals t o m low-voltage noise-sensitive sensors. Such robustness can be achieved by selecting hardware that can provide configurable gains and sampling rates. and designing software to allow programmatic mauipulation of these parameters by the user through an intuitive user interface. Modularization ofthe processing algorithms (since the processing involved in the analysis of a sensor is considerably different from the processing involved in the characterization ofa multiple-phase power source) can also contribute to the robustness of the system. allowing incremental upgrades to the functionality of the platform in the form of plug-in modules providing specialized processing algorithms. This proposed paper will describc the procurement. assembly, and testing ofthe data acquisition system specified in the previous paragraphs. Demonstration of the capabilities using representative data will be provided.
It must have a strictly defined file format with embedded information for identifying data streams within the file. It must provide the ability to immediately peruse recently acquired data in order to determine correctness and relevance of data prior to processing. It must provide a process and memory efficient past-processing algorithm superior to that of its predecessor. It must be relatively easy to operate. Although other considerations such as reusability in other applications are important. these are the primary goals of the design. A diagram illustrating the overall system architecture is shown in Figure 1
,\7.L%-I.v,
i.i’ba0,
11. SYSTEM OVERVIEW
For the system described in this paper. the challenge was to improve upon an already established process of data acquisition, storage, and post-processing of measurements taken from each phase of a 208V. 400 Hz, three-phase signal. Using t h e existing method. voltage and current measurements were sampled at 40;OOO samples a second t o m each phase and dumped to disk as a contiguous array of single-precision floating point values. Portions ofthe resulting data were parsed and processed in MATLAB using a script that loaded all the values into memory. performed calculations on the values, and then dumped all the resulting values to disk. If the resulting data set was too big, there would not be enough physical memory to hold the values and perform the calculations at the same time. The fact that onlyrelatively small portions ofthe resulting data could be analyzed at a time, and that the size of the data generated could he significant (IO minutes would equate to 549.3 MB), caused this process to be very time consuming. not to mention the h c t that someone who was not intimately famihar with the sequential format of the resulting file would have difficulty deciphering its contents, For the new design to be practical in replacing the old method. it had to meet the following criteria:
Figure 1 -Proposed System Architecture
The design effort focuses on two issues: hardware and software. The hardware selected for the data acquisition system consist of some form of data acquisition hardware, a computer with a suitable hard disk, and any sort of signal conversion hardware required for converting the signal to a measurable range. The soflware developed for this data acquisition systcm will handle control and synchronization of hardware components during dat? acquisition as well as providing a mechanism for the immediate viewing of data before and after post-processing. A mechanism for adding more functionality, such as lfferent post-processing algorithms. will be provided in the form of modules or “plug-ins .”
243
IO. HARDWARE The hardware consists of three components: the data acquisition device, the host computer, and the signal conversion hardware. Selection of the first two involves the consideration of the physical requirements of its intended application. The signal conversion hardware must be designed and implemented around the first two components. For the data acquisition component of the hardware, the National Instruments PCI-6052E multifunction data acquisition card was selected. This card was chosen not only because it would support high acquisition speeds over multiple channels_ but also because it provided a 16-bit resolution and independently configurable gains for each channel (although these configurable gains are not entirely necessary for the particular application for which this system was designed, it allows measwement of a number of different signals over considerably different ranges). An illustration of the PCI-6052E is shown in Figure 2 below.
particular storage medium, a suitable storage medium capable of handling incoming data rates such as those generated by the data acquisition hardware must h c selected. To allow for expandability of the system, a storage medium capable of supporting multiple data acquisition devices of the aforementioned type was chosen. In this particular case. it was assumed that there would be a potential for the use of four PCI-6052E multifunction data acquisition cards: this would generate 2.54MB of data per second. Most conventional hard disks have burst rates high enough to handle this much data, hut few can provide a sustained rate fast enough to guarantee that it will be able to store data to disk fast enough without compromising thc intcgrity of thc data resulting from dropped samples that could not be written before new data arrives. Sustained stream to disk data rates for a conventional hard disk are shown in Figure 3.
I
I
Stream-bDisk Rates
.
.
. . . . . . &sbs(rm$leS)orwupMaR.O. (UF4-W
.
,
Figure 3 - Stream-to-Disk rates of a typical IDE h a r d
disk with variable buffer sizes
Figure 2 - D a t a Acquisitian Hardware This data acquisition device allows the user to acquire up to eight differential channels or up to sixteen singled-ended (referenced or non-referenced) channels simultaneously at a maximum overall sampling rate of 333,000 samples per second. This rate is divided among the number of channels in use, allowing a maximum sampling rate per channel of41,6?5 samples per second if eight diffcrential channels are in use, or 20,812 samples per second if 16 single-ended channels are in use. This creates a maximum possible data stream rate of 650.39kB per second if the stream is represented by 16-bit integers, 1.27MB per second if the stream is converted to single-precision floating point values, and 2.54MB per second if the stream is converted to double-precision floating point values. In order to preserve the integrity of the data stream coming t o m the data acquisition hardware as it is written to a
244
Note that the buffer size represents the size of the buffer in memory that is being written to disk. As can he observed from Figure 2, a conventional hard disk may not he capable of guaranteeing a suitable sustained data rate. Instead. it was concluded that a server-class system hosting a level 3 RAID multiple disk array was needed. with three hard disks; two non-redundant disks for parallel write accesses. This setup doubles the maximum possible sustained stream-to-disk rate, with one redundant drive for storing parity information in the case of hardware kilure. Figure 4 illustrates a RAID-3 configuration, as it is commonly known.
Figure 4
- RAID-3 Configuration
I
Source Input 8 Measurement Poi-
IV. SOFTWARE The software chosen for the system was custom written in the National Instruments LabVIEW development environment for reasons of compatibility with the data rcquisition hardware and its reasonable learning curve. The central and most transparent component of the data zcquisition system is the message passing interface that landles sends, receives, and decodes messages to and t o m strictly defmed data input modules. data output modules. and cats conversion modules. The interhce to each category of nodules was strictly defined, creating a kamework for lreveloping new inodules for supporting new hardware. new 1 ata storage or output methods. or new processing methods. The primary input module for the data acquisition system is a cnstom designed dam acquisition engine that allows an dvanced level of confignrability of the hardware being used n this case. the PCI-6052E).~Thisallows the operator to individual channels on the data acquisition board or different ranges and gains, and allows the operator to rovide detailed descriptions of each channel. This onfiguration allows others who handle the data to be able tv precisely identify each channel of data. rather than simply seeing a stream oftime-indexed; unit-less values. The primary conversion module consists of an algorithm that takes multiple streams of data, representing a 208V, 400 Hz, three-phase signal, and performing power analysis equations on each stream of data. These calculations are performed on small portions of data at a time automatically, thus conserving memory resources by not loading an entire dataset into memory prior to processing. The primary output module. or file-streaming engine, handles the task of outputting the acquired data to a tile. This module handles the task of saving data stream identification information as well 8s ensuring that the acquired data is written to disk in an acceptable format. Control of the d a h acquisition system by the operator is possible by means of a custom user interface that allows the
DIODE
DIODE
GND
(multiplied) within software by the pre-determined divider constant to yield the correctly scaled voltage and current data utilized for system characterization. During system implementation, the voltage divider and voltage clamp circuits were replicated to add additional channels. The present implementation utilizes a voltage divider box housing four voltage divider and voltage clamp circuits resulting in four channels of data measurement. Future voltage divider boxes will contain additional channels for increased numbers of simultaneous measurements utilizing multiple d a h acquisition channels.
--
Figure 5 -Voltage Clamp Circuit
Dnrine the conceotion ofthis svstem. a need was recognized for the abilityto measure high voltage and ciment measurements utilizing industry standard low voltage data acquisition hardware. This resultod in the development of a voltage divider and a voltage clamp circuit. The voltage divider circuit decreases the source voltage by a predetermined divider constant resulting in an output which falls within the safe measurable range (+/- 10 volts) ofthe data acquisition hardware. The voltage clamp circuit results in a low voltage signal (+/- 50 mV). These outputs may now be safely connected to the data acquisition hardware inputs for measnrement. These measurements are correctly scaled L
245
operator to see real-time feedback of an ongoing acquisition, view any point or interval in previously acquired data, and process previously acquired data to create new data.
V. REFERENCES [ I ] National Instruments_ DAQ PCI E Series User Manual, Austin, TX_ Jul. 2002. [2] Martin, S., “PC-Based Data Acquisition in an Industrial Environment,” PCLBased I n s l n ~ m e n r a t i o n , IEE Colloquium on, 31 Jan. 1990. Pages: 2/1 - X3. [3] Payne. J.R._ M e n , B.A., “High Speed PC-Based Data Acquisition Systems“. Industry Applications Conference, 1995. Thinieth IAS Annual Meeting, IAS ’95, Conference Record of the 1995 IEEE, Volume 3. 8-12 Oct. 1995, Pages: 2140-2145_ vol. 3.
Figure 5 - Screenshot of User Interface The primary feedback mechanism of the user interfacc is the signal waveform graph indicator, shown in Figure 5 . When acquiring data, the waveform graph updates periodically, giving the operator a rough visual indicator of the status of the signal being acquired. When viewing acquired or processed data stored on disk. it allows the operator to select an arbitrary time and duration, and then displays the signal values defined by the time and duration as well as specific signal identification information in the form of an ASCII text string set by the operator at the time of acquisition. Development of the custom software package for the data acquisition system has resulted in a robust and configurable platform for measurement and anal+ of data. In addition. its modular design allows the addition of more features and capabilities. V. CONCLUSION Data acquisition systems are necessary and useful tools for research, testing, and verification related activities, but design of such systems involves the consideration of many factors ranging from the physical capabilities of the hardware used to the inherent properties of the signals being measured. This paper has presented the design and implementation of a custom data acquisition system that is flexible enough so that adaptation to individual data-gathering requirements is a relatively simple matter. In addition, the actual use of the data is greatly simplifird by means of the specific signal identification information incorporated into the data stream as it is being recorded.
246
