A Scalable Datalogging System with Serial Interfaces and Integrated ...

1568

JOURNAL OF ATMOSPHERIC AND OCEANIC TECHNOLOGY

VOLUME 25

A Scalable Datalogging System with Serial Interfaces and Integrated GPS Time Stamping MARIO BEHN, VINCENT HOHREITER,

AND

ANDREAS MUSCHINSKI

Department of Electrical and Computer Engineering, University of Massachusetts—Amherst, Amherst, Massachusetts (Manuscript received 1 May 2007, in final form 18 October 2007) ABSTRACT A scalable datalogging system for micrometeorological, fast-response, in situ, and remote sensing applications is presented. The system is based on a standard x86 MINI-ITX computer and the open-source operating system Linux. Real-time access for debugging and remote system control is implemented via a network interface. A 160-GB, 2.5-in. hard disk drive provides extended local storage. The recorded data can alternately be stored at a remote location using the Network File System (NFS) included in Linux. Accurate time stamping of collected data points is implemented using the open-source software Network Time Protocol (NTP) and a global positioning system (GPS) receiver. The operational capability of the system is demonstrated over a period of several weeks with data from seven ultrasonic anemometer–thermometers and a barometer.

1. Introduction Recent progress in computing, sensing, and timing technology has led to reductions of size, cost, and power consumption of datalogging and sensing systems—in many cases without compromising system requirements. Such progress has enabled the development of increasingly autonomous sensor networks for monitoring many aspects of the natural environment (Hart and Martinez 2006). This is good news for the atmospheric science community because increased automation of both datalogging and data transfer technologies allows greater access to remote and environmentally challenging locations of interest by decreasing the need for human service of systems.

surements are collected at remote sites that are far away from each other. Consider, for example, an array of infrasound sensors. Let c be the sound velocity and L be the spacing of two sensors in the direction perpendicular to the propagation direction. Then, to achieve an angle-ofarrival accuracy of ⌬␣, the uncertainty ⌬t of the time stamps must fulfill the criterion ⌬t ⱕ

Corresponding author address: Andreas Muschinski, Department of Electrical and Computer Engineering, University of Massachusetts—Amherst, 151 Holdsworth Way, Amherst, MA 01003. E-mail: [email protected] DOI: 10.1175/2007JTECHA1024.1 © 2008 American Meteorological Society

共1兲

Now, let us assume that these two infrasound detectors are operated independently from each other during an extended observation period T. Then the relative drift qd ⫽ ⌬t/T between the clocks of the two dataloggers must meet the criterion

a. Motivation Accurate monitoring of the temporal evolution of three-dimensional scalar and vector fields is a prerequisite to progress in meteorology, oceanography, and other areas of the geosciences. The necessity for accurate time stamping becomes a major challenge if mea-

L ⌬␣. c

qd ⱕ

L ⌬␣. cT

共2兲

Now, let us assume that the baseline L of the infrasound interferometer is on the order of the sound wavelength ␭ (the transverse correlation length of natural infrasound waves is typically on the order of ␭). Then, qd ⱗ

⌬␣ ␭ ⌬␣ ⫽ , cT fT

共3兲

where f ⫽ c/␭ is the frequency of the infrasound under consideration. As an example, let us require an angle-

SEPTEMBER 2008

BEHN ET AL.

of-arrival accuracy of ⌬␣ ⫽ 1° ⫽ 0.0175 rad, f ⫽ 1 Hz, and T ⫽ 1 month ⫽ 2.6 ⫻ 106 s, which leads to qd ⱗ 6.7 ⫻ 10⫺9 s s⫺1, or less than 6 ms day⫺1. Such a small drift is not achievable with commercially available clocks. (Drifts of PC clocks are typically on the order of 1 s day⫺1.) In this paper, we present a newly developed datalogger with an embedded computer whose clock is synchronized to coordinated universal time by means of a GPS receiver. In this way, we seek to advance the progress of automated data collection and storage to include accurate time stamping that utilizes existing GPS technology. The system was built entirely from commercial off-the-shelf (COTS) hardware and programmed entirely with open-source software, allowing for maximum freedom of system augmentation in development, utilization, service, and repair. Furthermore, use of standard interfaces (e.g., USB 2.0, RS 232) allows for optimal flexibility of instrumentation needs, essentially creating a plug-and-play datalogger that has great value across a large range of experimental platforms and can simultaneously serve multiple functions in the field. Such functionality renders the present system scalable because it is capable of accommodating a variable number and type of sensors without additional hardware or software restructuring. In the recent literature, Francesc and Domingo-Ferrer (2007) illustrate scalability with respect to communication in computer networks. In an economic sense, scalability can be understood to be the ability to provide an increasing number of systems at a constant cost per unit, which the current system provides by virtue of its technologically standard platform. Of course, the current developments have been achieved on the footing of previous advances in related technology.

b. Available technology 1) COMPUTING Most notably, there have been two important advances in computing technology that have enabled the development of increasingly flexible, versatile, and scalable systems. First, as is well known, computer hardware (e.g., processor, memory, and storage) has dramatically increased in performance and (at the same time) decreased in cost over the previous three decades (Berndt and Rappaport 2001)—the manifestation of which is obvious from the near-ubiquitous presence of cellular telephones and PDA devices, some of which have been programmed to communicate with sensor and measurement equipment (van der Molen et al. 2006). Second, the popularity and prevalence of opensource software (and applications based on open

1569

source) have dramatically increased in the last several years (Lerner and Tirole 2002), allowing users access to system-level code and therefore a degree of flexibility unrealized with proprietary systems.

2) SENSING For many sensor systems, the miniaturization of technology has allowed the integration of the A/D conversion close to the sensor itself (Malcovati et al. 1994). This represents significant progress when compared with the traditional method of transmitting the analog information via a potentially long and noisy cable to a datalogger, where it is converted. Hence, in many modern sensors, the sensor signal containing the information about the measured physical quantity is provided in a digital format. One of the most popular choices for this digital format is a standard serial RS 232 interface because it requires a total of only two wires instead of one wire per data bit in a parallel format. In addition, these sensors often allow for the configuration of their properties via a serial interface. However, despite these advancements, many sensors, especially medium- and high-frequency response sensors, do not have sufficient local storage (if any) to avoid data loss. Consequently, the generated data have to be received and stored using some external means. Many commercially available state-of-the-art dataloggers still contain a large number of analog input ports and (possibly) digital I/O ports in their typical configuration, but few (if any) have inputs for sensors with a standard serial interface. Some dataloggers can be upgraded with more serial ports; however, this usually requires the purchase of an expensive additional option. The total internal storage provided by these dataloggers is typically on the order of a few megabytes, and the typical scan rate is 100 Hz per port. In addition, this relatively small storage must be shared by all the ports of the datalogger. However, storage can usually be increased (as an option) to a few gigabytes by using compact flash (CF) cards. While some dataloggers allow the addition of a GPS unit for position retrieval, we are not aware of any commercially available datalogging systems that would take advantage of the highly accurate timing information provided by GPS technology.

3) TIMING The necessity of attaching a sufficiently exact time stamp to a data point requires an accurate means of timekeeping. Fortunately, the development of UTC time and the increasing prevalence of GPS technology have provided a standard by which an accurate time stamp can be obtained. An overview of timekeeping

1570

JOURNAL OF ATMOSPHERIC AND OCEANIC TECHNOLOGY

VOLUME 25

TABLE 1. Time-stamping accuracies for different GPS/NTP implementations (ordered by achievable accuracy). Level

Implementation

Accuracy (s)

Limited by

0

Standard kernel PC clock set manually Standard kernel NTP/GPS Low-latency kernel NTP/GPS Low-latency kernel PPS API NTP/GPS Customized hardware, GPS, oven-controlled quartz

1

Human reaction time

Climatology

10⫺1 10⫺3 10⫺5

SL Reduced SL ISR latency

10⫺7

Hardware delays, GPS clock accuracy

Self-logging thermometer (⬃1 Hz) Sonic anemometer, video recording Sonic anemometer, fine wire anemometer, video recording Cosmic ray counting, high pulse repetition frequency (PRF) radar

1 2 3 4

technology can be found in Jespersen and FitzRandolph (1999). Table 1 summarizes some timekeeping technology with respect to GPS technology, which, in combination with the Network Time Protocol (NTP) and the pulse-per-second (PPS) technique, provides an excellent option for global time stamping with accuracies on the order of microseconds. However, it is important to examine some of the limiting factors of a system using NTP, GPS, and PPS. The accuracy with which NTP synchronizes the local PC clock to the GPS clock depends only on the computer hardware, the accuracy of the reference clock, and software latencies (SLs), which are operating system scheduling and processing delays (Deeths and Brunette 2001). An advanced discussion of NTP is given by Mills (2006). Typical hardware delays are on the order of a few nanoseconds, because they consist of gate delays, bus speed limitations, and other related hardware properties. These delays are usually negligible compared to the accuracy of the reference clock itself (typically about 200 ns) and the operating system software latencies (typically on the order of 10–100 ms). A detailed assessment of the accuracy of several GPS clocks can be found in Mannermaa et al. (1999), who found that 10 of 11 GPS receivers tested provided a time within ⫾200 ns of the official (true) time. Given the factors influencing time-stamping accuracy, it becomes possible to compare different levels of implementation. The most straightforward implementation of timekeeping (indicated in Table 1 by level 0) is to reset the system clock manually at regular intervals. However, because it will eventually drift away, it is essential to repeatedly correct it with respect to a reference clock (e.g., GPS). The limiting factor of such an implementation is the accuracy with which a human can reset the clock (e.g., 1 s). A similar, but automatic, function is implemented in NTP and executed continuously to compensate for any drift (level 1 in Table 1). Because the function is automatic, no (slow) human interaction is needed and the limiting factor becomes the SL of the computer system (e.g., ⱕ0.1 s). Using an operating sys-

Application examples

tem with a low-latency and preemptive scheduling kernel patch (level 2 in Table 1) reduces software latencies to about 1 ms (Heursch and Rzehak 2001). To further overcome the software latencies, handling of the time synchronization can be shifted into the operating system kernel, by using the PPS signal provided by a GPS and an Interrupt Service Routine (ISR) to perform the local clock update. An implementation of PPS is described in Mogul et al. (2000). Using this approach (level 3 in Table 1), an accuracy on the order of a few microseconds can be realized. Additionally, a specialized hardware solution can be used (level 4 in Table 1), such as that described in Brouwer et al. (2002), where a GPS receiver (Motorola Oncore) was used in conjunction with external counter hardware to time stamp cosmic ray data and examine time-correlated events detected by systems separated by many kilometers. Using customized software/hardware developed by Pryke and Lloyd-Evans (1995) along with the PPS signal, an RMS error of 37 ns was reported. A similarly customized system was also used by Berns and Wilkes (2000), in which case the time stamps of two data recordings located 250 km apart were reported with an accuracy of 100 ns.

2. System a. System requirements To provide a low-cost system with maximum functionality and versatility that could fulfill a broad range of applications and conditions encountered during atmospheric and meteorological field studies, the following system features are desirable: 1) implementation of readily available standard hardware to support system components that are easily interchangeable, upgradable, and low cost; 2) all system software based on open-source technology to allow maximum flexibility in adaptation to requirements and source code–level maintenance; 3) use of latest GPS technology to feature accurate

SEPTEMBER 2008

1571

BEHN ET AL.

FIG. 1. Operational schematic of a datalogging system with the input, processing, and output sections. Timing information is received by the GPS and input to the network time protocol process, which uses it to stabilize the PC clock. Up to eight sensors send data via the USB-to-serial-port multiplexer/demultiplexer to the datalogging process. The datalogging process then detects when new data are ready, creates a UTC time stamp from the (stabilized) PC clock, and outputs the data into a dedicated file for each sensor.

4)

5)

6)

7)

time stamping and spatial determination of experimental components; interchangeable and expandable number of serial inputs to allow connection with up to 32 serial sensors; sufficiently low power consumption to maximize system operation time when operated remotely (e.g., battery- or solar-powered applications); remote/online access to system controls and data stream to provide real-time troubleshooting and data analysis capabilities; protection from a range of environmental conditions to increase system reliability and robustness and avoid system breakdown; and

8) adequate data storage to allow for extended datalogging periods over months, even with highfrequency response sensors.

b. Design 1) OPERATIONAL

OVERVIEW

Figure 1 shows an operational schematic of the datalogging system. The input, processing, and output sections are clearly identified. The input section consists of the sensors, the GPS receiver, and the serial-to-USB multiplexer (the USB to 8-serial-port adapter). The processing section of the system consists of the MINIITX PC and the software running on it, which contains

1572

JOURNAL OF ATMOSPHERIC AND OCEANIC TECHNOLOGY

the NTP and the datalogging program. Both are separate operating system processes. The NTP task ensures that the local PC clock, which is used to time stamp the input data, is disciplined and corrected by the GPS reference clock. Operating concurrently, the datalogging process is informed of new data by a signal at any of the serial ports. When a signal is present, the data are fetched immediately from the serial port and a time stamp is added. Finally, the data are output to disk.

2) STANDARDIZATION To realize a highly scalable and versatile datalogging system, the current design is based on a standard x86 processor; a motherboard (VIA Eden MII); and the MINI-ITX form factor, which provides a significantly larger degree of standardization than many commercially available dataloggers (see Fig. 2). This enables us to use a common PC operating system and simplifies the addition of new hardware. Some other advantages provided by using standardized computer systems are lower cost, more reliable driver software, higher degree of integrated hardware, increased choice of vendors, and greater ease in exchanging and upgrading parts.

3) OPEN-SOURCE

SOFTWARE

For the operating system, we haven chosen to use the open-source operating system Linux (Bokhari 1995), which provides more freedom to adjust and troubleshoot as compared to the closed-source (proprietary) operating systems typically used in commercial datalogging systems. There are also many other reasons why an open-source system is favorable, such as independence from a specific vendor, unrestricted distribution and modification of source code, cost, and a larger development community. Because a wired network interface is already included in the MINI-ITX computer, and software tools such as the Secure Shell (Ylonen and Lonvick 2006) are already part of the Linux operating system, real-time access to monitor the recorded data and the datalogging system can easily be implemented. For extended local storage, a 160-GB, 2.5-in. hard disk drive (Seagate Technology) was installed. Alternatively, the recorded data can also be stored directly at a remote storage location using the Network File System (NFS) described by Shepler et al. (2003), which is also included in Linux. Another software package included in Linux is the Simple Network Time Protocol (SNTP), described in Mills (1991). For the datalogging system presented here, we use NTP in combination with a GPS

VOLUME 25

receiver (Motorola Oncore), available with an integrated antenna (Synergy Systems).

4) TIME

STAMPING

For the datalogging system presented here, a timing system of level 3 (see Table 1) was chosen because it offers the best accuracy that can be achieved with a software solution alone and satisfies system requirements. For the PPS implementation, we have followed Giometti (2006). A comparable system with respect to timing performance, which uses an integrated GPS receiver, is given by Hwang et al. (2004). This system, however, was a custom design based on a printed circuit board with an Advanced Reduced Instruction-Set Computing (RISC) Machine (ARM) microcontroller— a feature that requires substantial development time and is not as readily available as a standard product.

5) SERIAL

SENSOR INPUTS

The need for multiple serial sensors, each connected with an asynchronous serial (RS 232) line (with a data transfer rate up to 230 400 bit s⫺1), requires some way of multiplexing these data lines. This is because the MINI ITX computer itself only contains two serial ports. However, because the MINI ITX computer has an integrated high-speed USB 2.0 interface, a lowpower-consumption USB adapter was selected as a multiplexing unit. The low-power-consumption feature allows the adapter to be powered entirely from the USB bus itself instead of using an additional external power supply. It should further be noted that with this technique, 8 serial ports per USB port can be connected, which—with 4 USB ports on the MINI-ITX computer—provides up to 32 serial ports without any additional power supply needed. We note that the Linux OS is already equipped with software to support a large number of serial ports. This support can be enabled as an option in the Linux kernel configuration file (if this option is enabled, the kernel must be recompiled). A detailed discussion regarding this feature can be found in Hankins and Lawyer (2007).

6) POWER

SUPPLY AND CONSUMPTION

The need for a battery-based dc power supply results from the fact that for many operating conditions, no (extended) power infrastructure exists. This implies a need for both a high-capacity battery and the lowest possible power consumption. To fulfill the first need, a 115-Ah deep-cycle battery was selected as a dc power source. Table 2 summarizes the steady-state power consumption of the components of the datalogging system and compares it to a typical commercially available system (Campbell Scientific CR 1000).

SEPTEMBER 2008

BEHN ET AL.

1573

FIG. 2. Photograph of the system. 1: Weather-resistant enclosure, with internal dimensions 12 in. (30.48 cm) ⫻ 14 in. (35.56 cm) ⫻ 5.5 in. (13.97 cm). 2: Datalogging system (MINI ITX PC). 3: 160-GB 2.5-in. notebook hard disk drive. 4: Input voltage regulator. 5: MINI ITX power supply. 6: Power distribution terminal 7: Fuse.

7) ONLINE

ACCESS

For the purpose of online access, which aids in monitoring data and system integrity, a wireless 802.11 b/g 54-Mbit s⫺1 network card and a wired 10/100-Mbit s⫺1 network connection were added to the system. The availability of a wireless connection proves itself especially useful during operation in difficult terrain or oth-

erwise limited-access conditions because online access can be established anywhere within the radio range of the wireless network and is not constrained by a network cable. However, it is also true that the wireless network consumes additional power (about 1 W, as seen in Table 2). To reduce this effect, it is possible (by using the Linux cron-agent) to enable the wireless system for preset monitoring and service intervals only.

1574

JOURNAL OF ATMOSPHERIC AND OCEANIC TECHNOLOGY

VOLUME 25

TABLE 2. Power consumption and cost of datalogger components. Current system

Comparable commercial system (Campbell Scientific)

Part

Power (W)

Cost ($)

Part

Power (W)

Cost ($)

MINI ITX PC WLAN ⫹ antenna (max 56 Mbit s⫺1) 32 USB RS 232 (max) Subtotal Timing GPS 2.5-in. 160-GB hard disk drive Total

10.55 0.95 3.80 15.30 1.28 2.00 18.58

400 200 600 1200 550 120 1870

CR1000 datalogger RF 401 ⫹ antenna (max 9.6 kbit s⫺1) 29 RS 232 (max) Total

0.40 5.20 2.40 8.0

1390 1275 6160 8825

This allows for the benefits of a wireless connection while reducing the impact on the power budget.

8) PROTECTION

FROM ENVIRONMENT

Because the datalogging system will usually be set up outside, it will be susceptible to the environment. This includes multiple kinds of precipitation, dust, or salt carried by wind, high winds, and extreme radiative fluxes. To prevent damage to the electronics as a result of ambient conditions, the datalogging system was mounted inside a weather-resistant enclosure (Campbell Scientific) as shown in Fig. 2. All connectors to the sensors, the GPS, the power supply (110 Vac or 12 Vdc from battery), and the network are watertight (see Fig.

3). Another issue is the potential formation of condensate inside the system enclosure, which could damage the electronics. Condensate can only form if the temperature of the system or the case drops below the dewpoint of the inside air. However, the heat generated by the system components will reduce the chance of this occurring. In addition, if a humidity problem is suspected, a bag of desiccant material (e.g., silica gel) can be placed in the enclosure to reduce the humidity inside the enclosure.

9) REQUIRED

DATA STORAGE

For the present sonic anemometer (R. M. Young Model 81000), 38 bytes per sample are sent via the

FIG. 3. Front view of external interface of datalogging unit. All connections are watertight. 1: Serial port with 12-Vdc power supply line. 2: Wireless network (WLAN) antenna connected to panel mount N-type RF connector. 3: Local area network (LAN) connector. 4: Connector to GPS unit. 5: Battery connector. 6: Ground terminal. During operation, a 110-Vac connector is added below the LAN connector and to the right of the battery and the GPS connectors.

SEPTEMBER 2008

serial interface (in ASCII text mode) when configured to report the three wind vector components, the temperature, and an error code. A 28-byte time stamp is then appended to each record by the datalogging program. The entire 66-byte record (38 ⫹ 28) is then stored in a dedicated file for each sensor. Using the maximum sampling rate for the present sonics (32 Hz), the data rate is given by 66 bytes ⫻ 32 Hz ⫻ 24 h day⫺1 ⫻ 3600 s h⫺1 ⬇ 175 MB day⫺1 for a single sonic anemometer. The amount of required storage can easily reach tens of gigabytes, depending on the duration of operation, the number of sensors, their individual sampling rates, the amount of data transmitted per sample, and the storage format. The 160-GB drive provides enough storage for many applications. For example, simultaneous logging of data from eight sonics of the type we used (66 bytes per data record, 32 records per second) would accumulate to approximately 40 GB per month, such that the 160-GB drive would overflow only after 4 months of continuous operation. While storage can be used more efficiently if data are stored in binary format, a human-readable ASCII file format was chosen to allow for ease of data monitoring. This decision, in combination with the availability of online access, provided a simple yet powerful tool to monitor data and ensure system integrity in real time.

c. Testing To gain a greater understanding of the operational limits and performance of the datalogging system, two bench-top tests were performed. First, a temperature module was added to monitor the device temperature of the CPU, the hard disk drive, the MINI ITX power supply, and the 120 Vac–120 Vdc converter. Second, the NTP-provided peerstats log file was analyzed to assess the timekeeping accuracy.

1) SYSTEM

1575

BEHN ET AL.

OPERATING TEMPERATURES

To determine the likelihood of overheating or undercooling the datalogging system beyond the manufacturer-specified range of 0°–50°C, a temperature sensor module with four temperature probes was connected to the second onboard serial port of the MINI ITX PC. These four probes were located at the power supply, the CPU, the hard disk drive, and the ac–dc converter. With an ambient temperature of 27°C, a steady-state temperature of 42.5°C was measured at the power supply, 45.5°C at the CPU, 39.5°C at the hard disk drive, and 46.8°C at the ac–dc converter. As expected, the temperatures inside the enclosures were considerably higher than the ambient temperature because of selfheating. This is beneficial at the lower end of the am-

FIG. 4. Offset between the local PC clock and the GPS clock under steady-state conditions during a 4-h interval. Note that the offset variations stay well within the bounds of ⫾10 ␮s.

bient operating temperature range but a disadvantage at the upper end.

2) TIME

STAMPING

To test the timekeeping ability of NTP in the laboratory setting, the GPS antenna was mounted on a windowsill, pointed toward the sky, and allowed to synchronize the local PC clock to UTC time using NTP. Using the ability of NTP to log the offset between the reference clock (here, the GPS satellite clock) and the local PC clock in a peerstats file, the offset was extracted and plotted in Fig. 4. For the displayed steadystate condition, the offset between the local PC clock and the GPS clock stayed well within ⫾10 ␮s, in agreement with the requirement for a level 3 system (see Table 1).

3. Field operation Extensive field operation of the current datalogging system was carried out as part of the Meteor Crater Experiment (METCRAX) performed in October 2006 at the Meteor Crater (also known as Barringer Crater, located at 35°01.931⬘N, 111°01.631⬘W) near Winslow, Arizona. The environmental conditions and testing experiences are summarized below.

a. Setup and maintenance Two prototypes of the datalogging system were built and used during METCRAX. A total of seven sonic anemometers and one barometer were connected to them. In the following, we focus on one of the systems, which was mounted (shown in Fig. 5) at the base of a

1576

JOURNAL OF ATMOSPHERIC AND OCEANIC TECHNOLOGY

VOLUME 25

TABLE 3. Time-stamp format.

FIG. 5. Datalogging system set up at the east rim of the Arizona Meteor Crater as part of the METCRAX field experiment. Sonic anemometers are visible on the left side of the tower, and a barometer is located beneath the datalogger enclosure. The GPS antenna can be seen on the top left of the tower. The system was operated continuously from 4 Oct to 27 Oct 2006.

6.4-m meteorological tower at 0.25 m above ground level (AGL) at the crater rim. The height above sea level of the entire unit was 1715 m as measured by a handheld GPS. Four sonic anemometers were set up at heights of 0.5, 1.5, 2.5, and 5.95 m; three of them are visible in Fig. 5. In addition, one barometer (located underneath the datalogger system in Fig. 5) and four internal temperature sensors were connected. All sensors were operated and data were logged in a nearcontinuous fashion from 4 October to 27 October 2006. For increased operating time, the system ran on two deep-cycle batteries, connected in parallel. Interruptions of a few minutes were made at approximately 48-h intervals [at approximately 1200 local time (LT)] to service the system and extract preliminary data.

b. Conditions and observations In total, 23.7 GB of data were collected from seven sonic anemometers and one barometer over the course of the field campaign. Except for the scheduled service maintenance, no system outage was observed. Performance was reliable from both a hardware and software standpoint during the operation time. Wind speeds ranged from 0 to ⬇30 m s⫺1, and ambient temperatures varied between ⫺5° and 35°C. Examples of recorded data are shown in Tables 3 and 4. The recorded internal system temperatures ranged from 10° to 42°C, well within the limits of the computing hardware specifications of 0° to 50°C. To expand this range, two temperature hardening strategies can be

Year

Month

Day

Hour

Minute

Seconds

2006 2006 2006 2006 2006

10 10 10 10 10

07 07 07 07 07

00 00 00 00 00

00 00 00 00 00

00.019295 00.051271 00.083268 00.115269 00.147271

applied. The first involves simply maintaining the temperature within the specified range. This could require a separate cooling system and/or a heating system, depending on the specific environmental conditions. However, if there are active components involved, these will contribute to the system power consumption. In a battery-operated system, this could shorten the operation time significantly. The second approach would be to use parts that are rated for an extended temperature range, which would avoid the compromise on the battery-based system operation time. However, the MINI ITX computer is currently not available with an industrial- or military-grade temperature rating. It should be noted that in conditions on the lower bound of the operational temperature range, self-heating by the working processor will compensate for a significant portion of the cooling, as observed during the field operation. While the system is not rated for applications that require no moving mechanical parts (on account of the spinning notebook hard disk drive), it continued to log data from all sensors even during periods of heavy precipitation, heavy sustained winds, and lightning activity, which induced significant environmental stress. To harden the system for even more adverse conditions, the hard disk drive could be replaced with a CF memory. Compact flash memories have no moving parts and a power consumption of typically only a few tens of milliwatts; however, the cost per unit of storage is currently much greater compared with traditional magnetic hard disk drive devices. As noted above, approximately 5 times as much storage could be obtained by switching from an ASCII data storage format to a binary data storage format. However, this would make TABLE 4. Data format. u (m s⫺1)

␷ (m s⫺1)

w (m s⫺1)

Temperature (°C)

Error flag

4.81 5.19 3.37 2.57 2.26

1.70 2.35 1.93 2.72 3.04

⫺0.65 ⫺0.34 ⫺0.90 ⫺1.13 ⫺0.85

19.67 19.64 19.82 19.60 19.58

0 0 0 0 0

SEPTEMBER 2008

BEHN ET AL.

online monitoring more complex because the data would have to be translated into a human readable format instead of being displayed with a simple ASCII/text editor. Another possible optimization to conserve storage space would be to reduce redundancy within the individual time stamps and data by compressing the stored file (i.e., run-length coding).

4. Discussion While the datalogging system has performed satisfactorily during the field operation and in the laboratory, it is useful to discuss potential system-load limitations in more detail. Data flow proceeds from the RS 232 serial interface into the multiplexer, from the multiplexer into the processing unit via the USB 2.0 bus, and finally from the processing unit to the hard disk drive, where it is permanently stored (see Fig. 1). The component with the lowest bandwidth along this path from data source (the sensor) to data sink (the file on the hard disk drive) will set the datalogging limit for the system.

a. Device limit According to the USB standard (Garney and Lueker 2000), it is possible to connect up to 127 devices to one USB bus. The MINI ITX computer has four USB ports, and each of these ports operates as a separate USB bus. In the current configuration, this means that each bus has to provide enough bandwidth for the eight RS 232 serial ports attached to it. Considering that the attached RS 232 serial-to-USB converters operate at full speed on the USB side, the theoretical bandwidth is 12 Mbit s⫺1. Following the throughput analysis given in Garney (1996), these 12 Mbit s⫺1 are divided into 12 000-bit (⬅1500 bytes) 1-ms frames. The resulting 1500 byte ms⫺1 is an ideal value because communication overhead (e.g., start of frame, end of frame, clock adjustment, hub polls, control transfer) reduces the maximum payload down to typically 1300 byte ms⫺1. However, even this is not a guaranteed payload because USB transmits its clock and data over the same line. By doing this, a separate clock line was saved in the design of USB, but a clock-recovery mechanism known as bit stuffing became necessary. Bit stuffing can consume up to 17% of the remaining payload, thus leaving (in the worst case) only 1084 byte ms⫺1 for the payload. The USB transfer mode from the serial ports is a bandwidth-guaranteed interrupt transfer, which in this case means that the USB hub polls each of the interrupt devices for data at a set interval and allows up to 64 bytes to be transferred in the interval-per-interrupt de-

1577

vice. There are also always 14 bytes of communication information added to these. With the interval set to the lowest value of 1 ms, up to 13 interrupt devices can be operated at the USB bus at full speed.

b. Data generation limit To assess the maximum amount of data generated by one of the RS 232 serial ports, note that a maximal data transfer rate of 230 400 bit s⫺1 by the sensor (including one stop bit, one start bit, and eight data bits) results in 23 byte ms⫺1. This is easily within the range provided by the 64-byte (per serial port) interrupt device, and even allows the number of interrupt devices to be increased to 28 per USB bus. We can then compute the maximum aggregated data rate of the 32 sensors as 32 ⫻ 23 byte ms⫺1 ⫽ 720 KB s⫺1 (where 1 KB ⫽ 1024 bytes). This is far below the memory, processing, and storage bandwidths of 100, 600, and 100 Mbit s⫺1, respectively. Because of the framing of data into 1-ms intervals by the USB bus, there are a maximum of 1000 unambiguous time stamps per second attached to each of the serial ports. So before the data are written to a file, a total of 32 000 time stamps have to be added to the data. Assuming that the time-stamp information is uncompressed and therefore consumes about 30 bytes, the total sustainable storage rate requirement becomes 32 000 ⫻ 30 byte s⫺1 ⫹ 720 KB s⫺1 ⫽ 1657.5 KB s⫺1, which is still only a small fraction of the storage bandwidth of 100 Mbit s⫺1. Nevertheless, this maximum storage rate can be further reduced by exploiting the redundancy present in the time-stamping data.

c. Data-rate limit: Single fast sensor In the case of running one 16-bit sensor at the highest possible sampling rate, the maximal RS 232 line speed sets the limit. Assuming that the data transmitted by the sensor to the USB multiplexer and then to the processing section are only raw (e.g., binary) data, the maximal sampling rate is given by 230 400 bit s⫺1/20 bit s⫺1 Hz⫺1 ⫽ 11 520 Hz.

d. Line-length limit Following the standard of the RS 232 serial interface (Electronic Industries Association 1969), the length between the sensor and the datalogging system is limited for all line rates to a maximum of 15.6 m. However, by converting the signal into a differential serial format (RS 422), the range can be increased to several hundred meters; the specific length depends on the exact line speed.

1578

JOURNAL OF ATMOSPHERIC AND OCEANIC TECHNOLOGY

5. Conclusions A datalogging system is presented based on MINI-ITX PC technology in combination with the open-source operating system Linux, GPS-based positioning and timing, and NTP and PPS technologies for enhanced timestamping accuracy. The system has been field tested and the limitations assessed with respect to the sampling rate, operating temperature, time-stamping accuracy, and data rate. The datalogging system presented provides a costeffective solution for many micrometeorological applications and in that context is unique in both its ability to time stamp data with an accuracy on the order of microseconds and its storage capacity of 160 GB. Flexibility with respect to maintenance and available hardware is maximized by using open-source software and a massproduced computer system. Future work should concentrate on extending the range of test conditions and decreasing the power consumption in order to generally enhance system performance and meet the specific challenges of more demanding applications and adverse environmental conditions. Acknowledgments. This material is based on work supported in part by the U.S. Army Research Laboratory and the U.S. Army Research Office under Grant 49393-EV and by the National Science Foundation under Grant ATM-0444688. The senior author (AM) is the first holder of the Jerome M. Paros Endowed Professorship in Measurement Sciences at the University of Massachusetts, and he thanks Jerry and Linda Paros for their generous support. The authors are grateful to Yonghun Cheon for assistance in the field, and to Daniel Wolfe and Christopher Fairall for providing access to the Boulder Atmospheric Observatory site. Thanks are also due to the two anonymous reviewers for their constructive comments. REFERENCES Berndt, E. R., and N. J. Rappaport, 2001: Price and quality of desktop and mobile personal computers: A quarter-century historical overview. Amer. Econ. Rev., 91, 268–273. Berns, H., and R. J. Wilkes, 2000: GPS time synchronization system for K2K. IEEE Trans. Nucl. Sci., 47, 340–343. Bokhari, S., 1995: The Linux operating system. Computer, 28, 74–79. Brouwer, W., and Coauthors, 2002: The ALTA global positioning satellite based timing system. Nucl. Instrum. Methods Phys. Res., 493A, 79–89. Deeths, D., and G. Brunette, 2001: Using NTP to control and synchronize system clocks—Part 1: Introduction to NTP. Sun Microsystems, Inc., 20 pp. [Available online at http://www .sun.com/blueprints/0701/NTP.pdf.] Electronic Industries Association, 1969: EIA standard RS-232-C: Interface between data terminal equipment and data communications equipment employing serial binary data interchange. Electronic Industries Association, 11 pp.

VOLUME 25

Francesc, S., and J. Domingo-Ferrer, 2007: Scalability and security in biased many-to-one communication. Comp. Networks, 51, 1–13. Garney, J., cited 1996: An analysis of throughput characteristics of Universal Serial Bus. [Available online at http:// www.usb.org/developers/whitepapers/bwpaper2.pdf.] ——, and J. Lueker, cited 2000: Universal Serial Bus specification revision 2.0. [Available online at http://www.usb.org/ developers/docs/.] Giometti, R., cited 2006: Linux implementation (LinuxPPS). [Available online at https://ntp.isc.org/bin/view/Dev/ LinuxImplementationLinuxPPS.] Hankins, G., and D. S. Lawyer, cited 2007: Serial HOWTO. [Available online at http://tldp.org/HOWTO/Serial-HOWTO. html.] Hart, J. K., and K. Martinez, 2006: Environmental sensor networks: A revolution in the earth system science? Earth Sci. Rev., 78, 177–191. Heursch, A., and H. Rzehak, Eds., 2001: Rapid Reaction Linux: Linux with low latency and high time accuracy. Proc. Fifth Annual Linux Showcase and Conf., Oakland, CA, USENIX, 27–38. [Available online at http://www.usenix.org/ publications/library/proceedings/als01/heursch.html.] Hwang, S.-Y., D.-H. Yu, and K.-J. Li, 2004: Embedded system design for network time synchronization. Embedded and Ubiquitous Computing, Lecture Notes in Computer Science Series, Vol. 3207, Springer, 96–106. Jespersen, J., and J. Fitz-Randolph, 1999: From Sundials to Atomic Clocks: Understanding Time and Frequency. Dover, 308 pp. Lerner, J., and J. Tirole, 2002: Some simple economics of open source. J. Ind. Econ., 2, 197–235. Malcovati, P., C. A. Leme, P. O’Leary, F. Maloberti, and H. Baltes, 1994: Smart sensor interface with A/D conversion and programmable calibration. IEEE J. Solid-State Circuits, 29, 963–966. Mannermaa, J., K. Kalliomäki, T. Mansten, and S. Turunen, 1999: Timing performance of various GPS receivers. Proc. Joint Meeting of the European Frequency and Time Forum and the IEEE Int. Frequency Control Symp., Besancon, France, IEEE, 287–290. Mills, D. L., 1991: Internet time synchronization—The network time protocol. IEEE Trans. on Commun., 39, 1482–1493. ——, 2006: Computer Network Time Synchronization: The Network Time Protocol. CRC Press, 286 pp. Mogul, J., D. Mills, J. Brittenson, J. Stone, and U. Windl, 2000: Pulse-per-second API for UNIX-like operating systems, version 1.0. RFC 2783. [Available online at http://tools.ietf.org/ html/rfc2783.] Pryke, C. L., and J. Lloyd-Evans, 1995: A high-performance GPSbased autonomous event time-tagging system with application in a next-generation Extensive Air Shower array. Nucl. Instrum. Methods Phys. Res., 354A, 560–566. Shepler, S., B. Callaghan, D. Robinson, R. Thurlow, C. Beame, M. Eisler, and D. Noveck, 2003: Network File System (NFS), version 4 protocol. RFC 3530. [Available online at http:// www.ietf.org/rfc/rfc3530.txt.] Van der Molen, M., M. J. Zeeman, J. Lebis, and A. J. Dolman, 2006: EClog: A handheld eddy covariance logging system. Comput. Electron. Agric., 51, 110–114. Ylonen, T., and C. Lonvick, 2006: The Secure Shell (SSH) protocol architecture. RFC 4251 (proposed standard). [Available online at http://www.ietf.org/rfc/rfc4251.txt.]