Evaluating Different Vehicle Audio Environments

PIAZZA ET AL.

SOFTWARE BASED SYSTEM FOR AUDIO QUALITY EVALUATION

Audio Engineering Society

Convention Paper 6003 Presented at the 116th Convention 2004 May 8–11 Berlin, Germany

This convention paper has been reproduced from the author’s advance manuscript, without editing, corrections, or consideration by the Review Board. The AES takes no responsibility for the contents. Additional papers may be obtained by sending request and remittance to Audio Engineering Society, 60 East 42nd Street, New York, New York 10165-2520, USA; also see www.aes.org. All rights reserved. Reproduction of this paper, or any portion thereof, is not permitted without direct permission from the Journal of the Audio Engineering Society.

Evaluating different Vehicle Audio Environments through a novel Software-based System Stefano Squartini1 , Francesco Piazza1 , Romolo Toppi2 , Massimo Navarri2 , Walter Lori2 , Ferruccio Bettarelli3 , Emanuele Ciavattini3 , and Ariano Lattanzi3 1

DEIT, Universit´ a Politecnica delle Marche, Ancona, 60131, Italy

2

Faital Spa, San Donato Milanese, Milano, 20097, Italy

3

Leaff Enginnering Srl, Jesi, Ancona, 60035, Italy

Correspondence should be addressed to Stefano Squartini ([email protected]) ABSTRACT An original software-based system, featuring two different tools, is here proposed for vehicle audio quality assessment. The first one performs the acquisition of relevant data for system modelling and for cancelling the undesired effects of the acquisition chain. The second offers a user-friendly interface for real time simulation of different car audio systems and for both objective and subjective evaluation, where the listening procedure is directly experienced at PC workplace. The validity of this approach has been examined through a subjective listening test set (more than 50 participants and three cars involved), developed by means of a dedicated software environment and based on appropriate ITU recommendations. Experimental results have shown that the quality rating delivered by conventional in-car procedure is confirmed when the software-based approach is used. AES 116TH CONVENTION, BERLIN, GERMANY, 2004 MAY 8–11 1

1. INTRODUCTION Scientific literature and industrial research have been showing an increasing interest in automotive audio system design. The design process needs to know the local and global acoustic characteristics of the sound environment (the car cockpit), in order to tune the value of the design system variables (those concerning loudspeaker and amplifiers typically) and set the desired level of general listening pleasantness. In fact, as observed by Shively in [1], the ”boundary conditions” on such variables are strictly determined by the environment under study. This means to consider the spectral, spatial and temporal characteristics of the vehicle listening environment, and define a complete set of solutions regarding the automotive amplifier scheme, loudspeaker design parameters and placement. However the design procedure needs audio quality assessment through objective and subjective measurements, in order to validate choices made by the designer. The set of quantities [1] used for the former kind of surveying could not suffice, as [2] recommends, and a subjective test frame is required, even due to the complex nature of near zero-reverberation-time rooms as vehicle audio systems, caused by the deep interactions occurring between the sound sources and the sound field [1]. This justifies the relevant role that subjective assessment fulfils in audio system design. It is widely accepted [1], [2], [3] that a correct procedure of listening tests must have the properties of repeatability, comparability and statistical significance. The evaluating process consists on testing each kind of loudspeaker (the audio system designing core) under test in each vehicle, through a variable set of listening surveys in variable conditions and with the objective of comparing the audio systems assessed to conclude which loudspeakers satisfy prefixed quality thresholds. It follows that such a kind of procedure demands a relevant amount of industrial resources, in terms of loudspeakers and vehicles (having separate audio systems can facilitate comparability among them), of people involved in listening tests (to give a statistical relevance to the global quality assessment) and of time (needed to perform a reliable number of tests). Therefore it seems interesting under the point of view of industry application to create an alternative method able to simplify the procedure of subjective evaluation without losing its helpfulness for the design process. Some

helpful suggestions can be found in literature [4], [5], where interesting solutions based on auralization [6] have already been proposed, also through the development of dedicated software tools for automatic audio system characterization and evaluation processes. A similar approach is considered and hence implemented in the automated framework discussed in this paper (and partially appeared in [7]) to model the vehicle audio system. The proposed framework is an integrated software-based system, comprising three different tools each of them addressing specific tasks (see Fig. 1). The modelling is performed by means of a combined solution of measurement devices and software tool, namely Automotive Recording Tool (ART). This can be considered the basic support for what processed by the second tool, the SQ Tool (SQ), where audio responses of car audio systems to different audio source signals can be analyzed and simulations of in-car listening experiences carried out. This leads to double effective capabilities, i.e. the chance of performing both objective and subjective evaluation. In particular, the development of the latter one has strictly required the implementation of a dedicated software (SeLen). Such a tool has been thought to facilitate the steps of test setup and to automate data processing resulting from testing. It supported also the preparation and fulfillment of the listening test set considered in this paper. Experimental results have allowed the authors to highlight the relevant capabilities of the automated procedure for comparing different car audio systems. Section II deals with the question of vehicle audio system modelling: MLS theory and the inverse filtering procedure needed for cancelling the undesired effects due to the acquisition frame. Section III, IV describe the basic software tools elaborated, Automotive Recording Tool and SQ Tool respectively. Section V presents the software solution to guide subjective evaluation tests and the processing of related results. The listening tests performed are reported in Section VI: comments on obtained outcomes and some ideas to develop are shown in Section VII. 2. VEHICLE AUDIO SYSTEM MODELLING The best solution for vehicle audio system modelling seems to be the black-box approach that describes the acoustic environment separating the source and the listening sensor (loudspeakers and human ear

PIAZZA ET AL.

SOFTWARE BASED SYSTEM FOR AUDIO QUALITY EVALUATION output vectors, and respectively, according to x[n] = H[n] ∗ s[n]

Fig. 1: Architecture of the implemented software.

typically) by modelling its input/output behavior through a transfer function (under hypotheses of causality, linearity and time independence). This can be seen as a proper series/parallel combination of simpler functions each related to a particular sound transfer path and so to a specified corrupting action on the original signal. M.Ziemba [8] has considered up to 8 different transfer functions; a simplified version of that consists of viewing the sound system as the cascade of three different elements, respectively modelling the loudspeaker, the effect of the cabin and the listener presence, usually addressed as Head Related Transfer Functions (HRTF). That is the choice effectively made, with the additional requirement of representing all involved signals in discrete time domain (choosing an appropriate sampling rate in order to avoid aliasing). Indeed, DSP techniques have been widely used on purpose, describing the overall relationship between the source signal (that drives a single loudspeaker) and the output sensor signal (coming from a microphone reproducing the acoustic signal caught by human ear) through a single discrete time filter (a FIR filter modelling the relative impulse response, IR), with a sampling frequency of 44.1kHz and neglecting the nonlinear behavior of loudspeakers. It follows that, given a car interior, a discrete-time MIMO model with suitable dimensions (n source inputs, m listening outputs) is needed to deliver a complete characterization of the automotive sound system. The parameters of the model are the taps of each FIR filter in the matrix , linking input and

˜ s[z] x ˜[z] = H[z]˜

(1)

in time and Z domains. They must be estimated to get an optimum fit of the phenomena, under adequate hypotheses and by means of an optimization criterium. Hence, they are MxNxL coefficients where the filter length is L. Many techniques [9] can be employed to identify the model parameters, both full band or subband-based. As aforementioned, an interesting one is based on the MLS (Maximum Length Sequences) [7], [10]. This kind of procedure can be followed also to mitigate some undesired effects that could degrade the quality of the final result. The causes of their origin is double: a) the presence of devices in the acquisition chain used for recording aims (although the Signal-to-Noise ratio can be kept very high during the acquisition phase, the effects of such devices, like microphones, mixer, sampler, etc., cannot be ignored); b) the way through which the subjective tests are performed( all the listening phase is carried out by means of commercial headphones receiving the output of the identified model solicited by suitable input signals). These undesired effects have to be modelled and removed. Under usual hypotheses of causality, linearity and time-independence, the modelling task can be performed through the combination of a recording action, where the input signal is directly fed to the headphones preceding the acquisition chain, and the estimation of the coefficients of FIR filters. Similarly to [1], it can be written: ˜ u [z]˜ x ˜[z] = H s[z]

x ˜i [n] =

X

˜ u,ij [n]˜ H sj [n]

(2)

j

The estimated transfer matrix Hu [z] must be inverted X ˜ u,ij [n]˜ ˜ u [z] = (H ˜ u [z])−1 (3) s˜i [n] = G xj [n] G i

and then multiplied to the recorded IR matrix H[z] to take the undesired effects out. This allows to cancel the presence of headphones during the listening phase automatically. The problem of obtaining a stable and robust inverse of a linear discrete-time filter can be quite difficult, especially when the system is non-minimum-phase. However it is always

AES 116TH CONVENTION, BERLIN, GERMANY, 2004 MAY 8–11 3

PIAZZA ET AL.


Fig. 2: Structure of acquisition chain. possible to approximate the true inverse by a FiniteImpulse-Response (FIR) filter [9], [11]. 3. AUTOMOTIVE RECORDING TOOL (ART) This software tool guides the user in all steps of the acquisition phase. The overall process is fully automated, with the only requirement being the user to choose the elements of the acquisition chain, the passenger’s position and the right recording option to load. Every acquisition action results in generating a set of files containing all information needed for simulation purposes handled in the software tool that is going to be described in the next section. Such files are managed in a completely automated way. The implemented software, based on ASIO 2.0 protocol [12], offers a user-friendly interface that does not require any kind of previous knowledge to perform the recording task. Such an operation has the structure of the acquisition chain as its core, depicted in Fig. 2. It is composed by a Bruel Kjaer Head Torso Simulator 4128, with ear simulators and microphones, a Yamaha 01V digital mixer, two Crown Macrotech 1200 power amplifiers, a Layla 24 bit digital multitrack recording system connected to a Pentium IV PC running a Microsoft Windows 2000 operating system and the relative software equipment. Four are the basic steps to let recording phase come to

fulfillment. They appear in the main ART screen, shown in Fig. 3 and are: calibration, car modelling, headphone modelling and CD player (CDP) modelling. 3.1. Calibration Amplification channels must be calibrated in order to make repeatable and homogeneous all subsequent measurements. After choosing the number of channels, the software will guide the user on loading the right test signals. Therefore the output levels of amplifiers must be tuned in order to have the same value for every channel, in dependence of the used measurement tools (e.g., Voltmeter and load impedances). 3.2. Car Modelling First (the setup phase), the number of channels, the positions to acquire and the map of hardware connections to Layla ins/outs must be set. Second (the session phase), the real recording operation can be launched by playing MLS signals and recording data perceived by the dummy head placed inside the car. The resulting data (together with setup information) will be ready to be stored in a complete softwareguided-way for further processing (see next section). It is also possible to select MLS signals, audio signals or both of them (automatically set in a cascade


PIAZZA ET AL.


Fig. 3: ART screenshot. configuration) for recording aims. 3.3. Headphone Modelling This step is formally analogue to the previous one: the only difference consists of considering headphones as source devices, completely bypassing the audio system where the mannequin is placed in. No setup phase occurs in this section, as well as in the following one. 3.4. CDP Modelling Also the CD player effect must be taken into account to get a complete characterization of the acoustic ambient. This aim is accomplished by acquiring the CD player responses when it is solicited by MLSs, similarly to the procedure described in the previous modelling sections. In fact, what performed consists on playing appropriate MLS tracks recorded on a CD through the CD player under test and acquiring the output signals, required for further IR calculation. However, a relevant aspect must be considered.

The digital signal reproduced by CD reader must be D/A converted before feeding the input channels of each loudspeaker. The converter sampling frequency is different from that one relative to the original signal: this fact leads to misinterpreting the crosscorrelation between the digital versions (at identical sampling frequencies) of input and output signals, resulting in erroneous IR estimation. Suitable algorithms have been employed to re-sample the CD output before giving it to the tool for IR calculation. Preliminary tests have shown sensible audible improvements, confirming the efficacy of implemented approach to face the converter effect. 4. SOUND QUALITY (SQ) TOOL Data resulting from the acquisition phase performed by the previous software tool are directly used by the SQ Tool to generate files concerning the ”car audio system”, the ”headphones equalization” and the ”CDP equalization” through the application of


PIAZZA ET AL.


suitable algorithms (Fig. 1). Such files let any common audio file be real-time filtered, hence allowing to evaluate the quality of the vehicle audio systems under test by real-time listening at the PC workplace. SQ Tool comprises the following operative subsections, available in the main SQ screen (Fig. 4). 4.1. Setup • Files relative to car audio systems and headphones drivers are here selected (Set-Up folder) together with the audio track (PCM code WAVE format, 16 bits quantization, 44.1Khz sampling rate) that will be played during the listening phase. Different choices can be made and loaded into memory for further cross objective/subjective testing. • Selected ambients, containing data resulting from recording phase, are modelled (Generate folder) by means of suitable algorithms, according to what pointed out in section II. 4.2. Filtering operations: real time and off line • In the Real Time folder the simulated reproduction of selected audio track in the chosen ambient can be carried out, once all the needed IRs, headphone and CDP drivers have been loaded. Such a reproduction consist of filtering the audio input by the FIR matrix model (section II), thus implementing a real-time PC based car audio simulator. The simulator employs an efficient frequency domain approach to convolution and is able to deal easily in real-time, on a Pentium III processor, with filter matrices whose elements have up to 1024 taps each. • The filtering operation can be easily switched from one ambient to another with no interruptions, hence supporting subjective comparisons among different audio systems (up to three different combinations of ambients and CDPs, to be chosen from their relative available lists ). • It is also possible to change the virtual position of the user (including head orientation and elevation) in the cockpit relative to a particular ambient, with no interruptions and low delay effect.

• Normalization factor can be applied (by means of different procedures) to the ambient undertest in order to decrease the impact of audio level differences. • The Saving folder provides the chance of off-line filtering a complete audio track through a single combination of modelled IR and headphone driver, organizing the resulting data files in a easy way for future elaborations, as those described in the following subsection. 4.3. Objective Analysis • Objective measurements of audio system quality can be performed on the offline elaborated data describing the audio response of each considered ambient. The Analyze folder provides tools to visualize the output signal of the audio system in different position in the cabin and in time/frequency domain, tools to add prefiltering operation to the modelling procedure and tools to evaluate some commonly used parameter values relative to single IRs as well as cross correlations between ears. Moreover, a graphical interface has been developed to help the user during the retrieving process. According to this, it is possible during playing to know which the listening position and head pan/tilt involved are. Then a visual feedback has been added to the audio simulator to make the listening experience more effective, i.e. it is possible to be ”virtually seated” in the car seeing the interior from the proper point of view. Fig. 5 shows some screenshots from the simulator visual feedback. Note the capability of moving around in the car cockpit using the navigator bar, and the effect of ”virtually” moving the head on the right. Although it employs visual processing, this feature seems to greatly increase the listener feeling, especially for subjective evaluation purposes, as underlined in [13]. 5. S.E.L.EN The acronym S.e.L.en stands for Subjective evaluating Listening environment, and covers the need of performing an automatic and repeatable subjective test set on various audio files. Such a tool can


PIAZZA ET AL.


Fig. 4: SQ Tool screenshot.

Fig. 5: Visual cues. Three different positions: Driver 0 and 45 right pan, rear left passenger. be functionally divided into two distinct blocks, organized as follows, in compliance with the ITU-R BS.1116-1 Recommendation [14]:

capabilities of the involved subjects, while the latter helps them to improve such capabilities by suggesting the correct answers.

Prescreening Test Set. Two different modules are provided therein (see Fig. 6): one is for training and the other for selecting candidates to take part to real subjective evaluation tests. The former allows assessing the audio listening

Subjective Listening Test Set. The user is guided through a series of listening sessions during which some kind of evaluating opinions are asked to him about the proposed audio files (see Fig. 7). They can be proper blind tests, as


PIAZZA ET AL.


Fig. 6: The Preescrening test Set-up window in SeLen environment.

Fig. 7: The Subjective test Set-up window in SeLen environment.

defined in the aforementioned recommendation, or quality tests, thought to asses the quality of listened audio tracks with the support of a proposed list of typical subjective parameters. All the cited tests can vary in dependence of the particular type of evaluation to be performed, even maintaining the structure explained above. Anyway, SeLen have been studied to facilitate both the test preparation and the consequent implementation through user listening phase: in fact its flexibility lets any administrator to set all tests of interest without being pressed by strict schemes or guidelines. Fig. 8 and Fig. 9 reports two typical screenshots for preescreening and subjective listening tests respectively. Italian has been chosen as the language for these forms, to facilitate the audience comprehension of test questions, although English can be easily used. The information management is based on a suitable database (ODBC model, OLEDB interface) employed to register all users’ profiles (general data obtained by filling a user registration form and results coming from their prescreening tests) and the subjective test responses to be interpreted. Such a choice does not require particular skills from administrator end. Moreover, together with the ”friend

Fig. 8: Preescrening test screenshot.

ness” of implemented software interface, it lets the user concentrate on listening evaluation, without worrying about the management of his answers. Regarding to the choice of parameters considered in quality tests, it must be observed that they have been selected considering what proposed in literature [2], [15], [16] and needs of loudspeaker compa-


PIAZZA ET AL.


Fig. 9: Subjective Listening test screenshot. nies. 6. SUBJECTIVE EVALUATION OF AUDIO QUALIY: EXPERIMENTAL RESULTS In this section subjective evaluation tests are described and relative results reported. The goal of such an experimental session consisted on validating the effectiveness of the proposed software-based approach for vehicle audio quality assessment, by comparing it with the conventional in-car procedure. This has been done by evaluating three different car audio systems through the two approaches and verifying that the audio quality ratings resulting from the assessment processes are identical. This idea has been applied by considering two different classes of subjects: common people and audio experts (i.e. people working at Faital Spa loudspeaker construction company). In both situations the same conclusion has been drawn, confirming the effectiveness of the implemented idea. Nissan Micra 1.2 Visia, Nissan Primera 1.9D and Mercedes C220 are the cars selected for testing. 6.1. Pre-screening Tests The performed Prescreening Tests have been built up following recommendations in [14], but it does not comprehend a training section. Indeed, they

have been thought just to assess the acoustic capabilities of subjects and to avoid that their possible acoustic impairments could seriously affect test results. The work of S. Olive [17] seems to support the idea that exists a strong correlation between judgements of a restrict number of experts and those of an audience not specifically trained, as highlighted also by our experimental results. This has let us consider common subjects for our tests, preliminarily discarding any kind of training phase or pre-required reliability. A grade score has been assigned for each test in order to get a global reliability score for each subject to be compared with a prefixed threshold. Pre-screening tests can be divided into four relevant categories (excerpt of question list is reported translated in English in table1): 1. Pure tone perception test. Pure tones, divided into four groups (250Hz, 500Hz, 1kHz, 2kHz, 4kHz, 8kHz; 63Hz, 125Hz, 12kHz; 16kHz; 18kHz), are given to both channels alternatively and randomly. Mandatory access range: 250Hz8kHz. Trial timing: 2-3 seconds. 2. Phase distortion test. 1kHz signal is given to both channels simultaneously three times, the last two ones are 180 phase shifted respect with the first one. Trial timing: 5-10 seconds.


PIAZZA ET AL.


3. Spectral distortion test. Reference track: ”Tom’s Diner”, S. Vega. It has to be compared to each of the spectral distorted versions, randomly selected (3dB over three bands: 0-350Hz, 350-4kHz, 4kHz-20kHz). Trial timing: 10 seconds maximum. 4. Saturation distorsion test. Reference track: ”Tom’s Diner”, S. Vega. Occurrence of saturation on the single track must be detected. Trial timing: 20 seconds. Table 2 shows the selecting criteria for user listening capabilities. The final subject quality evaluation is given by (4); the minimum selecting value has been fixed equal to 62 over 100. SQItotal = SQIP ureT ones + SQIP haseDist +

(4)

Prescreening Test Questions

Insert ”1” if you perceive the sound in the LEFT ear, ”2” if you perceive it in the RIGHT one, ”0” if you do not hear any sound. Mark with ”1” which signal is equal to reference ”A”. Insert ”0” otherwise. Mark with ”1” the track containing the smallest low frequency amount respect with the reference ”A”. Insert ”0”s if all tracks are identical. Insert ”1” if the signal sounds distorted. Insert ”0” otherwise.

+SQISpecDist + SQISatDist The number of subjects initially involved in this phase to select a suitable group for the listening tests was 71. A global skill threshold equal to 6 (in the range 0-10) has been fixed on purpose, yielding a listening group of 56 people with more than 8 as averaged global skill. Fig. 10 describes the form collecting the data associated to each subject. 6.2. Subjective Listening (Quality) Tests This kind of tests have been developed taking into account the typical industrial needs of loudspeaker companies and also the principal issues known from related works in scientific audio community ([2]-[19]). Such tests are based on a selected set of quality terms: low-frequency extension, tonal balance, high frequency extension, separability/intellegibility, image/localization/spaciousness, dynamics, punch, transient response, distortion. A set of 25 tracks of different musical genres has been chosen and submitted to subjects for audio quality evaluation. Three selected listening terms for each track: this selection has been made a-priori on the basis of audio characteristics of the tracks. Each subject has been required to give a score (in the half-point graded scale, from 0 to 5) to each term involved, resulting in a total of 3*25 quality scores for each car. A particular procedure has been performed to yield a value for the global quality index (GQI) associated to each car from these scores.

Table 1: Type of questions, corresponding to the four different types of pre-screening tests.

• In-car testing procedure. Three selected cars have been placed in the same room. Each subject had to fill a form relate to the car audio system under test inserting a score for each entry corresponding to one of the listening terms associated to the playing track. The form comprises also a special section to list pros and cons of the listening experience performed. The test session is made of three parts, one for each car, fulfilled in sequence by each subject, contemporarily with other two people of the listening group. One single listening chance has been allowed for each track. Each subject has been correctly placed in the driver’s position in the cabin by adjusting the height level of the seat. It has been let the subjects acquaint with the vehicle audio systems just before the session by listening a sample track, not included in the official form. This type of test set has not required any time limitations to the involved subject; the 20-30mins is average time required per car listening test. • Software-based testing procedure. This is the phase where audio output at headphones (used by subjects for listening at PC workplace) are


PIAZZA ET AL.


Fig. 10: SeLen screenshot of the form collecting the prescreening phase user’s data reproduced by processing the input audio tracks in SQ Tool environment. Each subject has required to fill the same form addressed in the incar testing procedure, with the only difference that all the process is based on SeLen program interface. This type of test set has not required any time limitations to the involved subject; the 20-30mins is average time required per car listening test. The latter procedure has been carried out approximatively two weeks later the former. At the same time, the SeLen upload test function has been used to load all the forms relative to the in-car listening procedure and start the Post-screening process to delete the contribution of those subjects introducing bias and so not relevant for statistical interpretation of results (according to guidelines proposed in [14]. Fig. 11 and Fig. 12 reports the average scores (and relative standard deviation) for all listening quality terms and the GQI in case of Mercedes C220, for in-car and software-based test procedure respec-

tively. Table 3 illustrates the average GQI for all test sessions with common subjects (”com”) and expert listeners (”ex”) involved. These results let us conclude that the quality rating for vehicles considered is identical for the two kinds of test sessions described, both for common and expert people. However, two observations must be done. First of all, the quality scores are slightly lower for the case of automated software-based approach: this is due to the lack of physical and listening sensations (like punch or distortion) in headphone mode tests, instead experienced in listening tests inside the cockpit. Moreover, smaller average scores have been registered in case of expert listeners with respect to the case of common subjects. This can be easily explained observing that expert ears perform their evaluation taking into account higher references derived by their background. It is important to stress that these facts do not change the significance of the achieved results and the validating meaning they present, in fact the relative differences among the different cars are equally reproduced by both ap-


PIAZZA ET AL.


pure tone perception

phase distortion

spectral distortion

saturation distortion

number of questions

2*6

2*3

2*1

2*1

3

6

3

weights

1

1

1.5

2

5

6

8

10

24

16

minimum SQI

12

Table 2: Selecting criteria for user listening capabilities in pre-screening tests. Weights and minimum subjective quality index (SQI) associated to each test are reported.

GQI

car1

car2

car3

ex software

3.1 ± 0.3

2.9 ± 0.4

2.5 ± 0.3

ex in-car

3.6 ± 0.5

3.3 ± 0.2

3.0 ± 0.2

com software

3.8 ± 0.3

3.6 ± 0.4

3.5 ± 0.4

com in-car

3.9 ± 0.4

3.7 ± 0.4

3.5 ± 0.5

Table 3: GQI (mean and std) for automated and in-car testing procedure, for both kind of listening groups involved. Car1=Mercedes C220 Car2=Nissan Primera 1.9D Car3=Nissan Micra 1.2 Visia

uation. Related experiments have shown that the quality rating yielded by conventional in-car evaluating procedure is confirmed when the software based approach is used, both when common subjects (all assessed reliable for listening sessions)and expert listeners are involved. Further developments are needed, as those here listed: • Prescreening test set could be improved by employing more performing scores and decision rules, in compliance with [18], [3], also taking into account the relevance of the listener training phase, as recently discussed [17], [18].

proaches.

• Further quality tests could be used to consider a larger range of audio aspects in experiments, improving the reliability on the evaluating capabilities of the proposed method.

7. CONCLUSIONS A fully automated process for vehicle audio system characterization has been proposed and its reliability for the evaluation of loudspeaker quality illustrated, hence proving its usefulness for industrial purposes. This process is based on two software tools, one (ART) for data acquisition and the other (SQ) for proper vehicle audio system modelling. An independent software environment (SeLen) has been developed for arranging proper listening tests for subjective evaluation purposes. It has been used by the authors to build up two test sets (pre-screening and blind) in order to test the effectiveness of the automated software-based solution for audio quality eval-

• The listening experience at the PC workplace is slightly biased by the use of headphones, that do not allow to reproduce properly some physically-related listening experiences such as bass punch. However headphones represents a very cheap and practical listening device, specially in industrial environments, even allowing an easy portability among different locations, also at geographical distances. It is therefore feasible to substitute headphones with properly designed fixed loudspeaker systems, introducing suitable 3D audio algorithms for improved sound reproduction. This aspect is actually under study.


PIAZZA ET AL.


Fig. 11: Mercedes C220: mean and std results for in-car testing procedure. Data elaborated by statistical processing in SeLen.

Fig. 12: Mercedes C220: mean and std results for automated testing procedure. Data elaborated by statistical processing in SeLen. • Several efforts have been recently made to derive a suitable link between objective and subjective evaluation of audio systems, at least for some of the quality terms [19], [16]. A relevant approach is represented by the PEAQ method, described in [20], that allows to deliver a scalegraded value (analogous to that obtained by a subjective analysis), without the intervention of the human subjects as in usual assessment tests. The authors are investigating the possibility to support such a method within the proposed software-based process of acquisition and characterization of audio systems.

8. REFERENCES [1] R. Shively, “Automotive Audio Design (A Tutorial),” presented at the AES 109th convention, Los Angeles, USA, 200 September 22–25.

[2] AES recommended practice for professional audio - Subjective evaluation of loudspeakers. 1996. [3] F.E. Toole, ”Subjective Measurements of Loudspeakers Sound Quality and Listener Performance”, J. Audio Eng. Soc., vol. 33, no.1/2, pp 2-32, January-February 1985. [4] E. Granier, M. Kleiner, B.I. Dalenback, and P. Svensson. ” Experimental Auralization of Car Audio Installations”, J. Audio Eng. Soc, vol. 44, n. 10, pp. 835-849, October 1996. [5] A. Farina, and E. Ugolotti, ”Subjective comparison of different car audio systems by the auralisation technique”, presented at the AES 103rd Convention, New York, USA, 26-29 September 1997.


PIAZZA ET AL.


[6] M. Kleiner, B.I. Dalenback, and P.Svensson, ” Auralization - An Overview”, J. Audio Eng. Soc, vol. 41, pp. 861-875, November 1993. [7] F. Piazza, S. Bartoloni, F. Bettarelli, A. Lattanzi, R. Toppi, M. Navarri, and M. Pontillo, ”Quality Measurement and Evaluation of Vehicle Audio Systems”, presented at SAE 2002 World Congress, Detroit, Canada, May, 2002. [8] M. Ziemba, ”Test Signals for the Objective and Subjective Evaluation of Automotive Audio Systems”, presented at the AES 110th Convention, Amsterdam, Netherlands, May 12-15, 2001. [9] S. Haykin, Adaptive Filter Theory. Prentice Hall, Englewood Cliffs, New Jersey, USA, 1996. [10] J. Borish, and J.B. Angell, ”An Efficient Algorithm for Measuring the Impulse Response Using Pseudorandom Noise”, J. Audio Eng. Soc., vol. 31, no. 7, July 1983.

[17] S. Olive, ”Differences in Performance and Preference of Trained versus Untrained Listeners in Loudspeaker Tests: A Case Study,” presented at the AES 114th Convention, Amsterdam, Netherlands, March 22-25, 2003. [18] D. Clark, ”Listening Technology for Automotive Sound Systems”, presented at the AES 114th Convention, Amsterdam, Netherlands, February 22-25, 2003. [19] S.H. Shin, and J.G. Ih, ”A Study on the Correlation Between Subjective and Objective Data of Loudspeakers”, Sound Quality Symposium, Dearborn, USA, August 22, 2002. [20] ITU Recommendation ITU-R BS.1387-1, ”Method for the objective measurements of perceived audio quality”, Geneva, Switzerland, 2001.

[11] S.T. Neely, and J.B. Allen. ”Invertibility of a room impulse response”, J. Acoust. Soc. Am., vol. 66(1), July, 1979. [12] Steinberg ASIO 2.0 Audio Streaming Input Output Development Kit. [Online]. Available: ftp://dist.steinberg.de/asiosdk2. [13] P. Odya, A. Czyzewski, and B. Kostek, ”Determination of Influence of Visual Cues on Perception of Spatial Sound”, presented at the AES 110th AES Convention, Amsterdam, Netherlands, May 12-15, 2001. [14] ITU Recommendation ITU-R BS.1116-1, ”Method for the Subjective assessment of small impairments in audio systems including multichannel sound systems”, Geneva, Switzerland, 1997. [15] F.E. Toole, and S.E. Olive, ”Subjective Evaluation”, in J. Borwick, ed., Loudspeaker and Headphone Handbook - Third Edition, Focal Press, London, UK, 2001. [16] D. Clark, ”Perceptual Transfer Function Measurement for Automotive Sound Systems”, presented at the AES 108th Convention, Paris, France, February 19-22, 2000. AES 116TH CONVENTION, BERLIN, GERMANY, 2004 MAY 8–11 14