fleet of 250 cruisers will be equipped with the system. I. INTRODUCTION .... list. Application 1. Application 2. Application n. Text-to- speech engine coordinated.
Project54: Introducing advanced technologies in the police cruiser Andrew L. Kun, W. Thomas Miller, William H. Lenharth {andrew.kun, tom.miller, whl}@unh.edu University of New Hampshire, ECE Department, Kingsbury Hall, Durham, NH 03824 Abstract –1The Project54 effort aims to improve the ability of police to manipulate data in mobile units as well as to provide a way to seamlessly integrate all in-car electronic devices. Work is being done the integration of in-car hardware and software, user interface integration, and the integration of the cruiser into a wireless data network. The system is being tested in three NH State Police cruisers. The entire NH State Police fleet of 250 cruisers will be equipped with the system. I. INTRODUCTION Project54 [1] is a cooperative effort between the University of New Hampshire and the New Hampshire Department of Safety. The project has two goals. First, it aims to improve the ability of police to collect and interpret data in, and exchange data between, mobile police units. This improved ability to manipulate data will increase the effectiveness of the officers, and what is extremely important, it will improve their safety. Second, the project aims to provide a seamless way to integrate the controls of all of the equipment within a police cruiser. The overall Project54 system needs to accomplish several goals, graphically represented in Figure 1. The system has to make the cruiser part of a computer-aided dispatch system, connected to central databases through a digital radio. It also has to allow the officer in the cruiser to access central databases, such as state motor vehicle records, criminal records, etc. The cruiser has to be equipped with a GPS receiver, a camera, a fingerprint checking device, and possibly other electronic devices. The officers should have a voice interface available allowing them to issue voice commands and receive voice feedback. Voice commands have to control all of the cruiser’s equipment (e.g. lights, sirens, radars, GPS, etc.). Some of the cruiser’s functionality should be accessible remotely. For example this remote access will provide the officer with the ability to operate the The authors wish to thank the US DOJ for funding Project54 through grant #1999-DD-BX-0082.
cruiser’s lights or access its databases from outside the cruiser. Having remote access to some of the cruiser’s functions is important because officers spend considerable time outside the cruiser. The specialized devices and software that are necessary to accomplish the desired functionality shown in Figure 1 are most often built without an emphasis on compatibility. This means that individual police departments have to create unique solutions that take into account their needs and the equipment they have. Other departments with different needs and base equipment will not benefit much from these experiences. They will probably start from scratch and come up with their own system. This increases system cost and decreases system performance for the majority of departments. The approach taken by Project54 is to tackle this problem by working on: a) hardware and software integration of the electronic devices in the mobile units; b) user interface integration of the devices and software available in the cruiser; c) network integration of the mobile units with a central base as well as with each other, through network communications.
Figure 1 System goals II.
BACKGROUND
Researchers at Texas A&M worked on improving the functionality of police cruisers using advanced
electronics through the ALERT project [2, 3]. The project addressed the need to integrate various electronic devices in police cruisers. However the project did not result in the acceptance of a standard for integration; rather it was a proof-of-concept effort. Wahab et al. worked on creating a dashboard system for intelligent vehicles that would be able to handle data and speech communication [4]. They paid special attention to combating noise in the car that interferes with voice communication. Muthusamy et al. created a prototype system for information retrieval in cars [5]. The system has a voice interface (input and output), and it can be used for voice dialing, Internet operations, and help with navigation.
The Project54 common IDB interface hardware uses Version 2.0B of the Control Area Network (CAN) protocol as the method of data transmission on the IDB. The CAN protocol, developed by Robert Bosch GmbH, uses differential signaling. Differential signaling works well in noisy environments such as an automobile.
Speech recognition in cars is of great interest to researchers as well as industry. An ambitious effort uniting car manufacturers, telecommunications equipment manufacturers, mobile telephone network operators, and universities is the SpeechDat-Car data collection effort. SpeechDat-Car collected in-car speech data from multiple languages [6, 7]. The collected data provides databases that can be used to improve the performance of speech recognition in cars.
Figure 2 The common IDB interface allows IDB messages to be passed to the aftermarket device and responses from the device to be put on the IDB.
III. THE PROJECT54 SYSTEM A. Hardware integration The Project54 system accomplishes hardware integration by using the Intelligent Transportation System Data Bus (IDB). All devices in the cruiser are connected to the IDB. The IDB standard prescribes both the hardware components and the protocol needed to accomplish control tasks and to exchange information between the car’s systems [8]. Most devices meant for police cruisers are not able to connect to the IDB; however most of them have serial and/or parallel inputs and outputs. Therefore we developed an embedded device that can interface to electronic products in a serial and parallel fashion, and can communicate with the IDB. Using this device, messages from the IDB can be translated and sent to a connected device, and responses from that device can be put on the IDB, as shown in Figure 2. We call the device the common IDB interface. As a means to connect the aftermarket devices to the IDB the interface provides two types of connection. The first type is a serial port that uses standard RS232 signaling. The second type of connection is a port that offers five parallel TTL level signals that can be used for a variety of applications including switching.
B. Software integration The software integration effort resulted in a software architecture based on Microsoft’s Component Object Model (COM) [9]. This architecture provides a standard way to implement control software for incar devices. The standard is published on the project web site (www.catlab.unh.edu). The software architecture was designed with two goals in mind. First, we wanted to create an open and modular architecture, which would allow the addition and exchange of components. Such a system would allow the easy addition of software components for the control of new hardware. It would also allow the upgrading of existing software components as more advanced ones become available. The second goal was to create the software architecture such that speech can easily be used as the user interface for all applications. We decided to accomplish this by allowing developers to write applications for our system without having to learn how to use speech recognition (SR) and text-tospeech (TTS) engines directly. Figure 3 shows a detailed view of the Project54 software system. At the center of the system is the application manager. The manager sends control messages to the individual applications. The applications control individual functions of the police cruiser (for example one application controls the light bar while a different application controls the radar unit). The applications talk to three modules in the system. The first one is the message coordinator. This
module receives status messages from the applications. These messages are forwarded to the manager. The second module is the logging coordinator. This module provides a centralized means for all applications to log events and errors. Finally, the speech output coordinator is responsible for an orderly output of speech messages to the user. Multiple applications may want to send speech output. Without the coordinator they would have to find a way to negotiate which one of them can send output first. They would also have to know how to talk to the TTS engine, which is not necessary in this system. Initialization list
Grammar
Keyboard and mouse input Application manager
Command list
status messages
Message coordinator
control messages
Logging coordinator
recognition results
Speech output coordinator
logging messages
Speech recognition engine audio object feedback
applications. Recognition results (words or phrases) are sent to the correct application using the command list. The application receives the recognition result in text format and decides on a proper course of action. C. User interface integration The primary user interface in the Project54 system is the speech interface. The speech user interface is integrated by designing the system so that every application of the software system (Figure 3) has access to speech input from the user and can output spoken messages through the speech output coordinator. Almost all functions of the system can be controlled using speech input and almost all feedback is available in the form of speech feedback. The exceptions are some remote data queries which require complicated data input and which can produce poorly structured responses.
coordinated speech output Text-tospeech engine
Application 1 Application 2
Application n
speech output
Figure 3 Software system architecture The manager receives input from a mouse, a keyboard, and an SR engine. Speech is the preferred means of communication in the Project54 system. However, interacting with speech engines is not trivial. The Project54 software system centralizes this responsibility by designating the manager as the only part of the system that can interact with an SR engine. As a result individual applications do not need to know how to do this. In fact, they do not even need to know that the user uses speech as the input to the system. All they need to know is how to receive control input from the manager. Application developers have to be aware of the existence of a grammars used by the SR engine. The grammars are supplied to the engine in the form of text files. The format of this file is determined by the SR engine. A commonly used industry standard (SAPI 4) exists which describes a standard for grammar file format. The manager uses two system files in its operation. One of them is the initialization list file. This file lists all the dynamic link libraries (DLLs) that have to be loaded and all the executable files that have to be run in order for the system to work. The DLLs and the executable files contain the individual applications. The second file is the command list file. The command list is used to associate recognition results from the SR engine with actions of specific
Figure 4 Graphical user interface A window-based graphical user interface (GUI) is the second input/output modality available in the system. The graphical user interface provides all the functionality of the speech user interface and extends this functionality slightly. Figure 4 shows the GUI used by the Project54 application manager. This GUI appears on the LCD touch screen. On the left there are two columns of virtual buttons. These buttons can be used to move from the application manager GUI to the GUI of any other application. Of course, the user can start up the desired application by issuing a voice command – the GUI is only a backup. While Figure 4 shows the application manager GUI, all GUIs have the same look and feel – they all have columns of virtual buttons on the left, the same background, the same text fields, etc. This is accomplished by providing a GUI service DLL which all applications call when creating their GUIs.
D. Network integration Through Project54 the NH State Police is installing a centralized law enforcement Records Management System (RMS) and a Computer Aided Dispatch (CAD) system, with Automatic Vehicle Location (AVL) and mapping capabilities. In-car software developed by Project54 allows officers to access NH and other databases. The software also provides continuous updates about the position of the police cruiser to the CAD and AVL systems. The final data system architecture has been optimized to provide maximal functionality considering the low-bandwidth and possibly intermittent wireless data channels typically encountered in public-safety operating environments. IV. IMPLEMENTATION Figure 5 shows the Project54 system hardware installed in a New Hampshire State Police (NHSP) cruiser. The center console contains the embedded computer, the radio, the radar, and miscellaneous electronics. The vehicle is equipped with an LCD touch screen. The touch screen can be used as a second input and output modality in case the speech interface is not available. There is also a directional microphone on the dashboard, as well as a push-totalk button on the steering wheel. The directional microphone reduces the influence of sounds that are not coming from the driver of the vehicle – this improves SR engine performance. The push-to-talk button is used to start recognition by the SR engine.
As of November 2001, Project54 has equipped three New Hampshire State Police cruisers with the new system. One of these cruisers is being used for testing purposes at the University of New Hampshire, while the other two are currently on the road in everyday use by police officers. These first cruisers are used to test the system in operation. The entire fleet of the New Hampshire State Police (250 cruisers) will be equipped with the system, starting in the summer of 2002. REFERENCES 1.
http://www.catlab.unh.edu/
2.
http://alert.tamu.edu/
3.
J. A. Ochoa, J. Witz, M. Eshani, and J. Morgan, “Advanced law enforcement vehicle electronics and associated power,” Proceedings of the 18th Digital Avionics Systems Conference, St. Louis, MO, October 24-29, 1999
4.
A. Wahab and T. E. Chong, “Intelligent dashboard with speech enhancement,” Proceedings of the International Conference on Information, Communications and Signal Processing, pp. 993-997, Singapore, September 9-12, 1997
5.
Y. Muthusamy, R. Agarwal, Y. Gong, and V. Viswanathan, “Speech-enabled information retrieval in the automobile environment,” Proceedings of the International Conference on Acoustics, Speech, and Signal Processing, Phoenix, AZ, May 15-19, 1999
6.
M. Sala, F. Sanchez, H. Wengelnik, H. van den Heuvel, A. Moreno, E. Deregibus, G. Richard, and E. le Chevalier, “Speechdat-Car: Speech databases for voice driven teleservices and control of in-car applications,” Proceedings of the EAEC 99, Barcelona, 1999
7.
Peter A. Heeman, David Cole, and Andrew Cronk, “The U.S. SpeechDat-Car data collection,” Proceedings of the 7th European Conference on Speech Communication and Technology (Eurospeech), Aalborg, Denmark, September 3-7, 2001
8.
Michael E. Martin, “The Project54 Common Interface for the Intelligent Transportation Systems Data Bus Version 4,” Technical Report ECE.P54.2001.3, ECE Department, University of New Hampshire, Durham, NH, 2001
9.
Dale Rogerson, Inside COM, Microsoft Press, Redmond, WA, 1997
Figure 5 System hardware V. CONCLUSION Project54 has created a system that integrates the hardware and software of in-car devices, as well as their user interfaces. The system also addresses network integration issues. The system uses a standard speech-based user interface which allows hands-free and eyes-free operation of in-car devices.
