Non-Uniform Sampling in Statistical Signal ... - Automatic Control

Linköping Studies in Science and Technology. Dissertations. No. 1082

Non-Uniform Sampling in Statistical Signal Processing Frida Eng

Department of Electrical Engineering Linköpings universitet, SE–581 83 Linköping, Sweden Linköping 2007

Thesis Cover — Application Snapshots Human heart

Death valley, as seen from the

Supernova 1987A in the

http://en.wikipedia.org/ Large Magellanic Cloud, as Space Shuttle’s synthetic aperwiki/Image:Humhrt2.jpg seen by the Hubble telescope. ture radar instrument.

Copyright released by owner

Courtesy NASA and STScI

Courtesy NASA/JPL–Caltech

http://hubblesite.org/

http://photojournal.jpl.nasa. gov/catalog/PIA01349

In the public domain

In the public domain Ultra sound Copyright Frida Eng

Glowing sun Oscilloscope From LiU’s picture data base Thanks to Peter Johansson, ES, ISY, LiU Copyright Frida Eng

Tire Copyright Frida Eng

Scania R470 flat nose truck. http://en.wikipedia.org/ wiki/Image:Scania_R470 _topline.JPG

Licensed under Creative Commons Attribution Share Alike 1.0 License

Part of Full Internet map, 22 Nov 2003, with colors separating routers on different continents. http://www.opte.org/maps/

Licensed under a Creative Commons License

Non-Uniform Sampling in Statistical Signal Processing © 2007 Frida Eng [email protected] www.control.isy.liu.se Division of Automatic Control Department of Electrical Engineering Linköpings universitet SE–581 83 Linköping Sweden

ISBN 978-91-85715-49-7

ISSN 0345-7524

Printed by LiU-Tryck, Linköping, Sweden 2007

To Maja, who gives me perspective

Abstract Non-uniform sampling comes natural in many applications, due to for example imperfect sensors, mismatched clocks or event-triggered phenomena. Examples can be found in automotive industry and data communication as well as medicine and astronomy. Yet, the literature on statistical signal processing to a large extent focuses on algorithms and analysis for uniformly, or regularly, sampled data. This work focuses on Fourier analysis, system identification and decimation of non-uniformly sampled data. In non-uniform sampling (NUS), signal amplitude and time stamps are delivered in pairs. Several methods to compute an approximate Fourier transform (AFT) have appeared in literature, and their posterior properties in terms of alias suppression and leakage have been addressed. In this thesis, the sampling times are assumed to be generated by a stochastic process, and the main idea is to use information about the stochastic sampling process to calculate a priori properties of approximate frequency transforms. These results are also used to give insight in frequency domain system identification and help with analysis of down-sampling algorithms. The main result gives the prior distribution of several AFTs expressed in terms of the true Fourier transform and variants of the characteristic function of the sampling time distribution. The result extends leakage and alias suppression with bias and variance terms due to NUS. Based on this, decimation of nonuniformly sampled signals, using continuous-time anti-alias filters, is analyzed. The decimation is based on interpolation in different domains, and interpolation in the convolution integral proves particularly useful. The same idea is also used to investigate how stochastic unmeasurable sampling jitter noise affects the result of system identification. The result is a modification of known approaches to mitigate the bias and variance increase caused by the sampling jitter noise. The bottom line is that, when non-uniform sampling is present, the approximate frequency transform, identified transfer function and anti-alias filter are all biased to what is expected from classical theory on uniform sampling. This work gives tools to analyze and correct for this bias.

v

Preface Is it ever easy to decide on the direction for a research project, that should span over several years, give enough challenges for the student, fully utilize the competence of the research group and give results to fill a PhD thesis? Probably quite a few professors can answer that question. I cannot. This thesis is the result of a project that quite dramatically changed direction somewhere in the middle. Going from a study of a particular application, control of packet data networks, that lead to a theoretical analysis of issues with nonuniform sampling. The mix is still quite nice, with a profound motivation for the deep theoretical work, although some might say that a closed loop is missing, since the theories are not applied to the original application. To those I say, you are very welcome to do this, but my focus is still more on the theoretical side, since the results are intriguing. Whatever your concerns, I hope you enjoy the time spent with this work, otherwise it was not time well spent. One of my wishes is to, if only a tiny bit, express the joy that working with this kind of mathematics can be.

Frida Eng February 2007 Linköping

vii

Acknowledgments I will here mention names that partly explains why this thesis is what it is. Maybe not the whole truth, but I hope you all know why you are mentioned. Otherwise, just go ahead and ask.1 I was born and raised by mom and dad, and I guess that is part of the explanation, although I am not sure how. Anyway, I love you both, and hope you can keep up for a few more lines. Thanks also to Lasse and Christine for taking such good care of my dear parents. Although I hate to say it, Uncle Johan was a major source of inspiration for moving to Linköping, starting at Y and also aiming for a PhD diploma. Now, when I am grown up (!) I can even enjoy your company. My professor Fredrik Gustafsson helped me to start the PhD journey, a long time ago: As has been the case for all our “common projects”, the beginning was a bit shaky, but the final result is pretty good. Thanks for helping me find the way. I don’t know what this group would be without Lennart Ljung leading it. I agree with everything that has already been said in previous acknowledgments, but also: Thanks for knowing about pregnancy and family life and for not kicking me out when I broke your rib. And Ulla Salaneck, I think you have heard it all before, but I honestly would have been lost without your help. The working environment would be quite awful without all the other colleagues, you never know what the coffee breaks or junkmail will bring. Memorable things are (not limited to): KRAV with Ragnar; sound with Johan S; history with Torkel; anything with Anna; Kent; coffee machines, gender issues and local news with almost everyone; and, of course, private girl talk with Henrik O. A little more namedropping is required: My co-supervisor Fredrik Gunnarsson always has time for questions, when he is here . . . Thanks for being supportive, especially for a confused first-year PhD student. Martin Enqvist has been a good friend ever since I moved into “his” town. I enjoy having you around although giving you a hard time about all the crosses in the margin. Gustaf Hendeby sighs as heavily as I do every time I have to knock on his door. I believe we make an excellent LaTeX/BibTeX-team by now. I only hope I can return any favor some day. Except from the previous three, a couple of other people also read and improved parts of the thesis: Ola, Jonas J, Jonas G and David T were involved the last time I wrote a book, and Henrik O have helped this time. Johanna and Linnéa: The fikas (and more) have been great, so thanks for starting our network. I hope I can stay in touch with . . . us. My out-of-the-office friends: I hope you are still with me, after this awfully long period of silence. Finally, Anders, Thanks for trying to read between the lines. I love you!

1 For those who don’t know me: I occasionally use jokes and irony, and the people in question will understand. Don’t worry, I don’t offend my loved ones, if I don’t have to.

ix

Contents

1 Introduction 1.1 Thesis Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Contributions and Relevant Publications . . . . . . . . . . . . . . . .

1 1 2

I

5

Background Theory and Application Overview

2 Applications 2.1 Introduction . . . . . . . . . . . 2.2 Packet Data Networks . . . . . . 2.3 Automotive Applications . . . . 2.3.1 Tire Pressure Monitoring 2.3.2 Non-Round Wheels . . . 2.4 Medical Applications . . . . . . 2.5 Radar . . . . . . . . . . . . . . 2.6 Astronomy . . . . . . . . . . . . 2.7 Hardware . . . . . . . . . . . . . 2.7.1 A/D Converters . . . . . 2.7.2 Oscilloscope . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

7 7 8 9 9 10 11 12 12 13 13 14

3 Non-Uniform Signal Processing in Literature 3.1 Signal Model . . . . . . . . . . . . . . . . . . . . 3.2 Non-Uniform Sampling . . . . . . . . . . . . . . 3.3 Frequency Analysis . . . . . . . . . . . . . . . . 3.3.1 Fast Algorithms for the Fourier Transform 3.4 Reconstruction and Estimation . . . . . . . . . . 3.4.1 Basis Expansions . . . . . . . . . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

15 16 17 19 20 22 22

. . . . . . . . . . .

xi

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

xii

Contents

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

24 25 26 27 27

4 Packet Data Traffic – A Motivating Example 4.1 Background on Packet Data Networks . . . . . . . 4.1.1 Network Preliminaries . . . . . . . . . . . . 4.1.2 Flow Control . . . . . . . . . . . . . . . . . 4.1.3 Active Queue Management . . . . . . . . . 4.1.4 Control Structures and Interactions . . . . . 4.1.5 In-house Work . . . . . . . . . . . . . . . . 4.2 Performance Issues in Queue Management . . . . . 4.2.1 Problems with TCP . . . . . . . . . . . . . 4.2.2 Measurements Used in Current Research . . 4.2.3 Using the Bottleneck Queue . . . . . . . . 4.3 Modeling and Identification of the Queue Dynamics 4.3.1 Frequency Analysis and Filtering . . . . . . 4.3.2 AR Modeling of Nonzero-Mean Signals . . . 4.4 Control of the Queue Length . . . . . . . . . . . . 4.4.1 Simulation Results . . . . . . . . . . . . . . 4.5 Summary . . . . . . . . . . . . . . . . . . . . . . . 4.6 Open Problems . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

31 32 32 35 37 39 41 42 42 44 45 46 47 49 51 53 57 57

5 Summary of the Thesis 5.1 Main Contributions . . . . . . . . . . . 5.1.1 Frequency Domain Analysis . . . 5.1.2 Frequency Domain Identification 5.1.3 Analysis of Down-Sampling . . . 5.2 Future Work . . . . . . . . . . . . . . . 5.2.1 Frequency Domain Analysis . . . 5.2.2 Identification . . . . . . . . . . . 5.2.3 Other Aspects . . . . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

59 59 59 60 62 63 64 64 65

3.5

3.4.2 Iterative Solutions . . . . . . . . . . . 3.4.3 Other Reconstruction Cases . . . . . . Optimal Sampling . . . . . . . . . . . . . . . 3.5.1 Benefits from Non-Uniform Sampling . 3.5.2 Alias-Free Signal Processing . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . .

. . . . . . . .

. . . . .

. . . . . . . .

. . . . . . . .

Bibliography

67

II

77

Included Papers

A Frequency Domain Analysis of Signals with Stochastic 1 Introduction . . . . . . . . . . . . . . . . . . . . . . 2 Problem Definition . . . . . . . . . . . . . . . . . . 2.1 Signal Model . . . . . . . . . . . . . . . . . 2.2 Stochastic Frequency Model . . . . . . . . . 2.3 Non-Uniform Sampling . . . . . . . . . . . . 3 Approximation of the Fourier Transform . . . . . . .

Sampling Times . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

79 81 83 83 84 85 86

xiii

4

5

6

7

3.1 Two Transform Expressions . . . . . . . . 3.2 Related Work . . . . . . . . . . . . . . . 3.3 Summarizing the Expressions . . . . . . . 3.4 The Periodogram . . . . . . . . . . . . . Main Results . . . . . . . . . . . . . . . . . . . . 4.1 Mean and Covariance of Y( f ) . . . . . . . 4.2 Mean and Covariance of W( f ) . . . . . . 4.3 The Periodogram . . . . . . . . . . . . . Asymptotic Analysis for ARS . . . . . . . . . . . 5.1 Asymptotic Analysis of the Mean Value . 5.2 Asymptotic Distribution . . . . . . . . . . 5.3 Distribution of the Transform Magnitude . 5.4 The Periodogram . . . . . . . . . . . . . Examples and Discussion . . . . . . . . . . . . . 6.1 The Stochastic Frequency Window . . . . 6.2 The Periodogram . . . . . . . . . . . . . 6.3 Addition of Amplitude Noise . . . . . . . Conclusions . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . .

. 87 . 87 . 88 . 88 . 88 . 89 . 90 . 93 . 93 . 93 . 96 . 96 . 97 . 98 . 98 . 98 . 102 . 102

A Covariance of W( f ) Using ARS . . . . . . . . . . . . . . . . . . . . . 103 B Covariance of W( f ) Using SJS . . . . . . . . . . . . . . . . . . . . . 104 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 B Identification with Stochastic Sampling Time Jitter 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . 3 Bias Compensation . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1 Bias in Yd ( f ) . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 First Moment of W . . . . . . . . . . . . . . . . . . . . . . 3.3 Bias Compensated Least Squares Estimate . . . . . . . . . . 3.4 Illustrations . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Covariance Compensation . . . . . . . . . . . . . . . . . . . . . . . 4.1 Covariance of Yd ( f ) . . . . . . . . . . . . . . . . . . . . . . 4.2 Covariance of W( f ) . . . . . . . . . . . . . . . . . . . . . . 4.3 Weighted Bias Compensated Least Squares . . . . . . . . . 4.4 Maximum Likelihood . . . . . . . . . . . . . . . . . . . . . 4.5 Implementation Issues . . . . . . . . . . . . . . . . . . . . . 5 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1 Known First Order OE Model with Unknown Jitter pdf . . . 5.2 Unknown Second Order OE Model with Known Jitter pdf . . 5.3 Unknown Second Order OE Model with Unknown Jitter pdf 6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . .

109 111 112 114 115 115 116 117 118 119 119 120 121 121 121 122 125 126 127 128

xiv C Downsampling Non-Uniformly Sampled Data 1 Introduction . . . . . . . . . . . . . . . . . . . 2 Interpolation Algorithms . . . . . . . . . . . . . 2.1 Interpolation in the Time Domain . . . 2.2 Interpolation in the Convolution Integral 2.3 Interpolation in the Frequency Domain . 2.4 Complexity . . . . . . . . . . . . . . . . 3 Numeric Evaluation . . . . . . . . . . . . . . . 4 Application Example . . . . . . . . . . . . . . . 5 Theoretic Analysis of Algorithm C.2 . . . . . . 6 Illustration of Theorem C.1 . . . . . . . . . . . 7 Conclusions . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . .

Contents

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

131 133 136 136 137 138 138 139 142 145 148 150 150

1 Introduction

In statistical signal processing, the sampling times are most often taken to be equally spaced. However, several applications indicate that non-uniform sampling is important. The major work performed on non-uniform sampling is for when the sampling times can be specified, and the signal processing community lacks tools to deal with standard issues like identification and decimation for signals sampled at non-uniform times. With a stochastic view, this thesis aims to fill this gap, and it provides tools to deal with errors induced by non-uniform sampling. Much is gained by studying a priori properties of frequency transforms and estimates, and the tools can be used for several signal processing problems.

1.1

Thesis Outline

For the reader not interested in sequential, but optimized, reading, hopefully this brief description of the different chapters and papers in the thesis can help. To clearly motivate the need for theories on non-uniform sampling, several applications are described in Chapter 2. Both the reason for the sampling being non-uniform, and what kind of problems that need to be solved, are pointed at. Much has been done in the area of non-uniform sampling, and Chapter 3 sets a common notation and also sorts different research efforts in to categories, based on the type of problem that is solved. Here, also references to the coming contributions in the thesis are made. Earlier work by the author, on modeling and control of packet data traffic, clearly showed the need for different solutions, given non-uniform sampling, and in Chapter 4 this work is presented together with pointers to the open problems that this resulted in. The main contributions of the thesis are described in Chapter 5, with chosen figures and equations to describe the results in the following 1

2

1

Introduction

papers. Here, guidelines for future work are also given. Each of the included papers investigate different aspects of non-uniform signal processing. Paper A contains theoretical analysis of a priori properties in the frequency domain, such as mean value and covariance of the frequency transform. Also, asymptotic analysis is possible when the number of measurements increases. The results separate the effect of the signal, the non-uniform sampling, and the finite number of samples in a nice way. A slightly different problem is considered in Paper B, where the sampling times are corrupted by unknown noise, and the underlying model is desired. It is shown that, inclusion of the knowledge of the sampling procedure removes the bias in the identification. Paper C considers resampling from a set of fast non-uniform samples, given by the application, to a slower uniformly sampled set, chosen by the user. This is done together with low-pass filtering, and the results from Paper A are used to perform analysis of the down-sampling procedure.

1.2

Contributions and Relevant Publications

The main contributions of the thesis are stated here, together with the relevant publications where the author of the thesis is the main contributor. Note that Mrs. Eng was Miss Gunnarsson before May 2004. As mentioned before, Section 5.1 further explains the contributions. • Investigation of performance measurements in networks, overview and model-based improvement of a control problem in packet data networks. This has previously been published in: Gunnarsson, Frida; Gunnarsson, Fredrik; and Gustafsson, Fredrik (2001). TCP performance based on queue occupation. In Towards 4G and Future Mobile Internet, Proceedings for Nordic Radio Symposium 2001. Nynäshamn, Sweden. Gunnarsson, Frida; Gunnarsson, Fredrik; and Gustafsson, Fredrik (2002). Issues on performance measurements of TCP. In Radiovetenskap och Kommunikation. Stockholm, Sweden. Gunnarsson, Frida; Gunnarsson, Fredrik; and Gustafsson, Fredrik (2003). Controlling Internet queue dynamics using recursively identified models. In IEEE Conference on Decision and Control. Maui, Hawaii, USA. In the thesis, the control problem for packet data networks is described in Chapter 4. • Stochastic analysis of frequency transforms based on signals with nonuniform sampling times. Derivation of first and second order moments, as well as asymptotic analysis is performed. The publications concerning this contribution are:

1.2

Contributions and Relevant Publications

3

Gunnarsson, Frida; Gustafsson, Fredrik; and Gunnarsson, Fredrik (2004). Frequency analysis using nonuniform sampling with applications to adaptive queue management. In IEEE International Conference on Acoustics, Speech, and Signal Processing. Montreal, Canada. Eng, Frida and Gustafsson, Fredrik (2005a). Frequency transforms based on nonuniform sampling — basic stochastic properties. In Radiovetenskap och Kommunikation. Linköping, Sweden. Eng, Frida; Gustafsson, Fredrik; and Gunnarsson, Fredrik (2007). Frequency domain analysis of signals with stochastic sampling times. In IEEE Transactions on Signal Processing. Submitted. Paper A covers this analysis in the thesis. • Identification in the frequency domain, with unknown sampling time jitter. The bias is removed by including knowledge about the sampling time jitter when performing frequency domain identification. The relevant publications are: Eng, Frida and Gustafsson, Fredrik (2005b). System identification using measurements subject to stochastic time jitter. In IFAC World Congress. Prague, Czech Republic. Eng, Frida and Gustafsson, Fredrik (2006). Bias compensated least squares estimation of continuous time output error models in the case of stochastic sampling time jitter. In IFAC Symposium on System Identification (SYSID). Newcastle, Australia. Eng, Frida and Gustafsson, Fredrik (2007c). Identification with stochastic sampling time jitter. In Automatica. Provisionally accepted as regular paper. In the thesis, the identification approach is presented in Paper B. • Analysis of down-sampling when sampling times are non-uniform, given the results in Paper A. The impact of the sampling times on the low-pass filter is derived and the estimate is proved to be asymptotically unbiased. Publications about this contribution are: Eng, Frida and Gustafsson, Fredrik (2007a). Algorithms for downsampling non-uniformly sampled data. In European Signal Processing Conference (EUSIPCO). Poznan, ´ Poland. Submitted. Eng, Frida and Gustafsson, Fredrik (2007b). Down-sampling nonuniformly sampled data. In EURASIP Journal of Advances in Signal Processing. Submitted. Paper C is devoted to down-sampling in this thesis.

4

1

Introduction

Part I

Background Theory and Application Overview

5

2 Applications

With relevant applications, theoretical work is easy to motivate. This chapter gives an overview of applications where non-uniform sampling occurs, and also states some of the problems that need to be solved in the respective cases. Section 2.1 is an overview of the applications and their problems, related to the work presented in this thesis. Then, Sections 2.2–2.7 describe the applications in more detail. These sections are aimed for readers with a particular interest in a certain area. Much of the presentation here is taken from previous work, see the given citations.

2.1

Introduction

A number of applications use non-uniform sampling inherently or by choice. They include: • Packet data traffic, where calculations are done at the arrival of a data packet. This application is described in Section 2.2, and Chapter 4 addresses some of its problems. • Automotive applications, where rotating toothed wheels are used to obtain measurements of angular velocity. These types of applications are described in Section 2.3. • Medical applications, which often use human activities to decide on measurement, such as peaks in the electrocardiogram, ECG, or lung volume, see more in Section 2.4. • Radar, where frequency estimation is used to detect movements by Doppler effects. The non-uniform sampling is introduced to increase performance and to avoid jamming, see Section 2.5 for more details. 7

8

2

Applications

Table 2.1: Summary of applications with non-uniform sampling Application packet data automotive medical radar astronomy

freqest X X X X X

ident X X X

control X

downsamp X X X

X

• Astronomical time series, which are collected during long time spans with weather conditions and equipment failures causing non-uniform spacing of the data. Section 2.6 describes the problem using a collection of datasets. Different hardware can also induce non-uniform sampling, for example, due to clock imperfections, a few cases are described in Section 2.7. Several signal processing actions can be of interest for systems or signals with non-uniform sampling. For example, • Fourier analysis aims to find the main frequency content of a signal. • Identification aims to find a mathematical model to describe the dynamics of a signal. • Automatic control aims to vary an input to enforce a desired behavior of a signal. • Downsampling is used, when the non-uniform samples are much closer than needed, to obtain the wanted information from a signal. The relation between different applications and actions are given in Table 2.1, indicating potentially interesting areas in non-uniform signal processing for the respective application. In this section, we have seen several examples of where we find non-uniform signal processing, also what kind of actions that can be of interest. The main focus of this thesis is to further analyze the different actions, without focusing on any particular application. The following sections include a more thorough description of the different applications, for the interested reader.

2.2

Packet Data Networks

Internet is one of the largest man-made systems in the world, and it is continuously growing. The algorithms controlling it are both situated at end-nodes (e.g., home PCs) and within the network (e.g., core routers). In routers, some of the algorithms controlling the data flows are based on packet data arrivals, and are at these instants assumed to collect measurements, calculate decision variables, and perform a suitable action. It is clear that data packets will arrive totally at random, and therefore several of the network calculations have to deal with non-uniform

2.3

9

Automotive Applications

Figure 2.1: Tire with 100 % (left) and 70 % (right) inflation. Thanks to Peter Lindskog and Urban Forssell at Nira Dynamics AB. sampling. This is mainly done in a heuristic fashion, but many research efforts aim at improving both models and control and also try to measure at different (slower) time scales. In Chapter 4, these problems are exemplified and discussed further. Also, a thorough background to Internet and packet data traffic is given there.

2.3

Automotive Applications

In the vehicles of today, numerous sensors are included and massive amounts of data are analyzed for different purposes. For example, toothed wheels are inserted in the wheels with sensors registering when teeth pass the sensor, giving a nonuniformly sampled signal, affected by the wheel speed. The measurements were originally only recorded for the ABS system, and are now finding more uses. The sensor values are times tm , that can be used for estimation of the angular velocity, ωˆ m =

2π L(tm − tm−1 )

(2.1)

when there are L identical teeth. This angular velocity estimate can be used for analysis of, for example, tire pressure in cars, and non-round wheels in trucks.

2.3.1

Tire Pressure Monitoring

A Tire Pressure Monitor System, TPMS, was presented in Persson (2002b) based on the wheel speed sensor. This presentation is entirely based on the work in that publication. Using the correct tire pressure is important, both for safety, economical and environmental reasons. In Figure 2.1, two tires with 100 % and 70 % inflation are shown. Lately, there is also laws in U.S. making it mandatory with TPMSs in new cars due to an increased number of fatal accidents with under-pressurized tires. Two types of TPMSs are available, direct and indirect.

10

2

Applications

120

instantaneous angular speed estimate, ω [rad/s]

84.5 95

84

100 83.5 83

90

80

82.5 60

82

85

81.5 40

81 80 80.5

20

80 75

79.5

0 0

500

1000

time, t [s]

600

650 time, t [s]

700

750

705

710

715

720

time, t [s]

Figure 2.2: Estimate of the angular velocity (2.1), given measurements from one wheel speed sensor of a car. Data is gradually zoomed in as indicated by the gray-shaded area. The direct system uses a pressure sensor mounted in the wheel, communicating with the central system of the car. This is expensive and requires new hardware in every car. The indirect system utilizes the available sensors and estimates the tire pressure from them. One way is to perform wheel radius analysis, since a decrease in the tire pressure decreases the effective wheel radius. Another way to perform indirect tire pressure monitoring is by vibrational analysis, since the rubber in the tire is excited from road roughnesses. The resonance frequency is highly dependent on the tire pressure, and can thus be used to detect changes in the pressure. This method is suggested in Persson (2002b). The algorithm corrects for sensor errors, i.e., unideal toothed wheels giving errors in (2.1), and converts the data from the event sampled sequence to a uniformly sampled one by interpolation, and then down-samples the data including lowpass filtering. It is argued that a combination of the two indirect TPMSs increases performance. Persson (2002b) also discusses interesting future estensions, which includes estimation directly from the non-uniform data, more efficient downsampling, and spectral analysis of non-uniformly sampled data. An example of measurements from one of the wheel speed sensors in a car is shown in Figure 2.2. In this particular case, the number of teeth are L = 48, and the average sampling interval is 2.3 ms over the whole interval.

2.3.2

Non-Round Wheels

The Master’s thesis Nilsson (2007) studies detection of non-round wheels in trucks, with the purpose of reducing the oscillations in the cabin. Vibrations in truck cabins can be amplified when tires are non-round. At speeds around 90 km/h, the frequency from the non-round wheels coincide with the resonance

2.4

11

Medical Applications

Figure 2.3: An example of an ECG curve. From http://en.wikipedia.org/ wiki/Image:EKG2.png. Licensed under GNU Free Documentation License. frequency of the cabin, and this is experienced as very annoying. Garages have difficulties in measuring the roundness of wheels, and, since trucks can have about 10 wheels, changing and testing them one by one is a long process. The non-round wheel is modeled as rotating around an offset, δ, from the hub, which gives a sinusoidal varying angular speed, ω, during one revolution of the wheel. Using several revolutions, the variations can be detected as a function of the angle of the wheel, θ, and the offset can be estimated, both by Fourier analysis and AR-modeling of the estimated radius, r(θ) = v/ω(θ), where v is the speed of the truck. In this context, problems with imperfect sensors are also present, and conversion from non-uniform samples to a uniform grid is of interest.

2.4

Medical Applications

There are numerous applications with medical connections, and here we study the case of the electrocardiogram, ECG, which is used to monitor the electrical activity in a person’s heart. An example of an ECG is given in Figure 2.3. There are times when the whole ECG curve is measured, both invasive and non-invasive, see for example, Wallin (2005) and Wikström (2005) for analysis of artifacts in the invasive case. It is also common to record the times of the highest peaks in the ECG, and use them for estimation (van Steenis and Tulen, 1991). Let tm be the time of the peak of the mth heartbeat, the instantaneous heart rate is then rm =

1 , tm − tm−1

(2.2)

which gives a non-uniform sampling of the heart rate, r(t). This is very similar to the automotive case, cf. (2.1). Healthy persons experience periodic fluctuations in the instantaneous heart rate, known as respiratory sinus arrhythmia, RSA. A decrease in the magnitude of the fluctuations normally indicates some kind of heart problem, and one of the main reasons for studying the heart rate variability is to predict the ability to recover after for example a heart attack. In order to detect heart rate variability, the heart rate spectrum can be investigated, in particular its low frequency content. An accurate heart rate spectrum

12

2

Applications

can also be used to study the heart condition after a transplant, e.g., rejection probability (Sands et al., 1989), or investigate patients with a problem known as the Guillain-Barre syndrome (Flachenecker et al., 1997). The heart rate is also used in order to study fetal health during both pregnancy and labour. For example, the spectrum can be used to assess the maturation of the Autonomous Nervous System (Romano et al., 2003).

2.5

Radar

A radar is an electronic system used for detection, location and classification of objects. It uses remote sensing by emitting signals of a certain wavelength and detecting the echo signal. Many radar applications result in non-uniform data on a two-dimensional grid, see for example, Grönwall (2006) and Belmont et al. (2003), where the latter describes problems with switching to a uniform grid in order to perform spectral estimation for vessel movements. One-dimensional radar measurements, e.g., long range radars or pulse radars use, frequency estimation to detect movements by Doppler effects. Usually, the wanted frequency is much larger than the possible Nyquist frequency, and therefore different non-uniform sampling techniques are of interest to increase performance (Alavi and Fadaei, 1994). Also, for military applications, detection and jamming of the radar is better avoided by the use of non-uniform sampling (Pribić, 2004).

2.6

Astronomy

In stellar physics, the luminosity of variable stars are recorded to describe their frequency contents, see Roques and Thiebaut (2003) and van der Ouderaa and Renneboog (1988). Measurements are done over long time spans, usually several years, and the measurement series are corrupted by both weather conditions and telescope failures. This gives long periods with missing data. In Hertz and Feigelson (1997), a total of 13 datasets are described, giving an idea of the problems in astronomical time series. For example, detection of periodicities in both the number of sunspots and observed properties of other stars is an important task for astronomers. A glowing sun with its spots is shown on the cover of the thesis. The data is often non-uniformly sampled as well as sparse and noisy. It is believed that a majority of the stars are in what is known as binary systems, where two stars orbit each other, and this is one of the properties that have to be detected with the mentioned type of data. Measurements of the radial velocity of two different stars are shown in Figure 2.4, and it is obvious that the sampling is quite non-uniform. An extremely important example in astronomy is the detection of neutrinos from the supernova explosion observed in 1987, SN1987A. One picture taken after the supernova explosion, by the Hubble telescope, is on the cover of this thesis. In this type of measurement the arrival times of the neutrinos, or other individual

2.7

13

Hardware

15

radial velocity [km/s]

10

5

0

−5

−10 21−Jul−1968

03−Dec−1969

17−Apr−1971

29−Aug−1972

11−Jan−1974

Figure 2.4: Radial velocities of two different stars. Data is taken from Hertz and Feigelson (1997). Error bounds for the measurements are shown with vertical bars. particles, are recorded, and the result is non-uniformly spaced data, which often is relatively sparse. For example, only 20 neutrinos were recorded from the supernova explosion, SN1987A. Figure 2.5 shows the non-uniform arrival times of the neutrinos recorded by two sensors.

2.7

Hardware

Here are a few examples of where non-uniform sampling may appear due to hardware, to further stress the importance of the area.

2.7.1

A/D Converters

The thesis Elbornsson (2003) thoroughly describes different errors arising in interleaved A/D converters, A/D-Cs. Interleaved A/D-Cs are necessary in order to increase sampling rates. If N A/D-Cs are used to sample a signal, the nth A/D-C samples at time tk,n = (kN + n)T, to get an overall inter-sampling time T. The results from the individual A/D-Cs are multiplexed to get the sampled sequence y(kT). Individual A/D-Cs suffer from time errors, δn , due to the internal clocks, and the sampling clock also suffers from noise, which gives rise to jitter, τk,n . The actual sampling time for A/D-C n is therefore, tk,n = (kN + n)T + δn + τk,n . Here δn is constant for individual A/D-Cs and τk,n is random. The resulting sequence t1,1 , t1,2 , . . . , t1,N , t2,1 , . . . , t2,N , . . . is clearly non-uniform. The thesis states

14

2

Applications

IMB

Kamioka

0

2

4

6

8

10

12

relative neutrino arrival time, [s]

Figure 2.5: Arrival times of the neutrinos after the supernova explosion, SN1987A. A total of 20 arrivals were measured by two units. Data is taken from Hertz and Feigelson (1997). methods to correct for the time errors, δn , and later work by Janik and Bloyet (2004) investigates the jitter errors, τk,n , further.

2.7.2

Oscilloscope

In Verbeyst et al. (2006), a 50 GHz sampling oscilloscope is studied and its time base jitter is identified to have standard deviation 0.965 ps, which corresponds to approximately 80% of the sampling interval. The model is that the actual measurement time is corrupted by zero-mean Gaussian noise, tk = kT + τk , and the second order moment of τk is identified in experiments. This identification is important when other effects, such as amplitude distortion, of the oscilloscope is studied. For example, the paper shows that a bias of more than 10% is introduced on additive noise estimates, if the time base distortion is not compensated for.

3 Non-Uniform Signal Processing in Literature Non-uniform, irregular, uneven, staggered and non-equidistant sampling in time are all names that have been given to the type of sampling this thesis deals with. This chapter presents the research problems and suggested solutions that exist in the area. The preceding Chapter 2 on applications gave an idea about what the different problems are, but here the research efforts are categorized, without any connection to specific applications. We start with two sections to define the notation that is used in this thesis: for the signal model in Section 3.1, and for non-uniform sampling in Section 3.2. The sampling is done at times tm to get sample values ym from a continuous-time signal s(t), see Figure 3.1: ym = s(tm ).

(3.1)

The publications are then sorted in the following topics: Frequency analysis described in Section 3.3 focuses on the following problem. Problem 3.1. Given measurements ym at times tm , how do we best characterize the frequency content in the original signal, s(t)? Reconstruction and estimation discussed in Section 3.4, can be stated as solving the following problem. Problem 3.2. Given measurements ym at times tm , how do we find the best approximation of the original continuous-time signal, s(t)? Optimal sampling investigated in Section 3.5, where focus is on solving the following problem. Problem 3.3. Given a signal s(t), how do we optimally place the sampling instants tm , and what is optimality in this case? 15

16

3

Non-Uniform Signal Processing in Literature

2.5 2 1.5 1

s(t)

0.5 0 −0.5 −1 −1.5 −2 −2.5 time [s]

Figure 3.1: A continuous-time signal, that is non-uniformly sampled.

There is also work on fast implementations, Section 3.3.1, mainly for the Fourier transform. Overviews of non-uniform sampling can be found in, for example, Papoulis (1977), Bilinskis and Mikelsons (1992) and Marvasti (2001), and these collections also cover parts of the ideas presented here.

3.1

Signal Model

We consider a deterministic continuous-time signal, s(t), with Fourier transform S( f ), Z s(t) =

S( f )ei2π f t d f,

(3.2)

which is non-uniformly sampled M times. The sample times, tm , are stochastic variables with probability density functions (pdfs), ( pm (t) =

d dt P(tm P(tm

≤ t) = t)

continuous, discrete,

(3.3)

for m = 1, . . . , M, and the number of samples, M, is deterministic. From the continuous-time signal (or function) s(t) we get a stochastic observation, ym , of the corresponding deterministic sample value: ym = s(tm ). One example of non-uniform sampling of a signal s(t) is given in Figure 3.1.

(3.4)

3.2

17

Non-Uniform Sampling

The sample value, ym , is a function of the stochastic variable tm and its stochastic properties can be investigated accordingly, for example, Z E[ym ] = E[s(tm )] = s(t)pm (t) dt, Var(ym ) = E[s(tm ) ] − E[s(tm )] = 2

!2

Z

Z 2

2

s(t) pm (t) dt −

s(t)pm (t) dt .

Any transform of these stochastic function values or measurements, ym , can be investigated to study its a priori properties, for example, for parameter estimation or frequency analysis. In this work, we consider the case of frequency analysis of a continuous-time signal, s(t), sampled non-uniformly and then transformed to approximate the true Fourier transform, S( f ). Sampling of stochastic processes will not be considered here, but all sensors give corrupt measurements, therefore ym = s(tm ) + em

(3.5)

is considered, where em is stochastic measurement noise, usually independent and identically distributed with zero mean. In some of the analyses, the additive measurement noise is not important, and the effects can be included afterwards. For example, by adding zero-mean measurement noise, we get E[s(tm ) + em ] = E[s(tm )] and Var(s(tm ) + em ) = Var(s(tm )) + Var(em ).

3.2


Non-uniform sampling can occur in different forms and this section lists the most common descriptions. The probability distribution for the sampling times, tm , was given as pm (t) in (3.3). Depending on the type of sampling, pm (t) can be deduced from the pdf, pτ (t) of the sampling noise, τm . In this work we use the sampling noise, τm , to construct the non-uniform sampling instants, tm , based on different sampling models. For additive random sampling, ARS, the sampling times are constructed by adding the sampling noise to the previous sampling time, tm = tm−1 + τm =

m X

τk ,

t0 = 0

(3.6)

k=1

where τm ∈ (0, ∞) and E[τm ] = T. This means that E[tm ] = mT, while the variance increases with m. The pdf is given as a convolution of the sampling noise pdf m times, (m)

pm (t) = pτ ? . . . ? pτ (t) , pτ (t). | {z } m times

(3.7)

18

3


Table 3.1: Summary of different sampling models, additive random sampling (ARS), stochastic jitter sampling (SJS) and missing data (MD). Type ARS SJS MD

Update tm = tm−1 + τm tm = mT + τm tm = tm−1 + τm

E[tm ] mT mT > mT

τm ∈ (0, ∞) (−T/2, T/2) {T, 2T, . . .}

pm (t) (m) pτ (t) pτ (t − mT) —

For example, the exponential distribution, pτ (t) = T−1 e−t/T , gives a Poisson sampling process. The central limit theorem gives that pm (t) will approach a Gaussian distribution when m goes to infinity, since it is the pdf of a sum of m independent identically distributed variables, c.f., (3.6). Additive random sampling is one of the considered sampling types in Paper A, while Paper C only uses this model to describe the sampling procedure. For stochastic jitter sampling, SJS, the sampling noise is added to the expected sampling time, tm = mT + τm ,

(3.8)

with τm ∈ (−T/2, T/2) and E[τm ] = 0. In this case the variance is constant over time and the pdf is given directly by the pdf for τm , pm (t) = pτ (t − mT).

(3.9)

One natural distribution is the rectangular distribution, pτ (t) = 1/T, −T/2 < t < T/2, but it is also possible to imagine a truncated Gaussian distribution or other bounded distributions. The sampling noise can both be known and unknown. Paper B considers the case of unknown jitter noise, while Paper A discusses known jitter noise as one of the sampling procedure models. Another case is the problem with missing data, MD, where the underlying sampling procedure is uniform but sometimes samples are missed. This can, for example, be described with a discrete sampling noise, tm = tm−1 + τm ,

(3.10)

and τm ∈ {T, 2T, . . .}. This can be seen as a special case of ARS, with a discrete pdf for the sampling noise, for example, pτ (nT) = P(τm = nT) = p(1 − p)n−1 gives a First success distribution. The expected value E[tm ] > mT whenever E[τm ] > T, which is the case for every nontrivial pdf. Some notes on the missing data problem are given in Section 5.2. The missing data problem is also discussed in Ljung (1999, Ch. 14) with further references. In Table 3.1, these sampling types are summarized with mean value of the sampling times, support of the sampling noise, the update equation for the sampling times, and the pdf.

3.3

19

Frequency Analysis

3.3

Frequency Analysis

In Chapter 2 we saw that in many applications, such as radar, image processing, astronomy and cardiology, the frequency content in the sampled signal is of interest. Problem 3.1 is considered in the following literature. Wojtkiewicz and Tuszynski ´ (1992) have chosen to start from the z-transform, Y[z] =

∞ X

ym z−m ,

(3.11)

m=0

for sampled signals and construct the Dirichlet transform, Y(x) =

M X

ym e−xtm ,

(3.12)

m=1

with x = σ + iω. This transform is argued to be better suited for analysis of non-uniformly sampled signals, since it preserves information about the time instants. The sampling is considered deterministic and the inverse transform is also derived. Only the case of jitter sampling is discussed in the analysis. Lomb (1976) and Scargle (1982) use ym = a sin(2π f (tm − τ)) + b cos(2π f (tm − τ)) + vm

(3.13)

as a model and from this uses least squares fitting to find a and b. The time shift τ is chosen such that PM m=1 sin(4π f tm ) , (3.14) tan(4π f τ) = PM m=1 cos(4π f tm ) for easier computations, since it ensures that the cross-term M X

cos(2π f (tm − τ)) sin(2π f (tm − τ)) = 0,

for all f.

(3.15)

m=1

This gives the periodogram  2  2   X   1 X   PY ( f ) = ym sin(2π f (tm − τ)) +  ym cos(2π f (tm − τ))  ,  M m m

(3.16)

≈ (a2 + b2 ) Comparing the Dirichlet transform (3.12), with x = i2π f , and the Lomb-Scargle periodogram, we get PY ( f ) = |Y(i2π f )|2 /M. Lomb and Scargle also perform probability calculations and correlation analysis between frequencies when the true signal is sinusoidal and the measurement noise is Gaussian. In Qi et al. (2002) the same signal model is used but extended to a sum over several frequencies, X ym = a0 + vm + ak sin(2π fk tm ) + bk cos(2π fk tm ). (3.17) k

20

3


The coefficients ak and bk are then considered varying and are estimated recursively using the Kalman filter, for an a priori chosen set of frequencies fk . This gives a useful algorithm when the frequency content varies over time, but there is no closed form expression like (3.16). Estimation of the spectrum when ym is given as samples from a stochastic process is given attention in three papers by Masry and co-authors. Masry and Lui (1976); Masry (1978b); and Masry et al. (1978) consider Poisson sampling, i.e., the sampling times are given from tm = tm−1 + τm and τm is taken from an exponential distribution with mean β. First the spectrum is estimated using PY ( f ) =

log M M−k 1 XX ym ym+k cos(2π f (tm+k − tm )), πβM

(3.18)

k=1 m=1

and the estimate is shown to be asymptotically unbiased, when M → ∞, for any value on β, Z∞ Cov(y(t), y(t + τ)) cos(2π f τ) dτ

E[PY ( f )] →

M → ∞,

(3.19)

0

when the sample values ym are taken from a Gaussian process, y(t). In the second paper the estimate is generalized with the inclusion of a window function so that log M M−k 1 XX ym ym+k wM (tm+k − tm ) cos(2π f (tm+k − tm )), πβM k=1 m=1 Z wM (t) = H( f )ei2π f / log M d f,

PY ( f ) =

(3.20a) (3.20b)

and H( f ) is any symmetric real-valued function scaled so that wM (0) = 1. Also here, the bias and variance of PY ( f ) are studied. Finally, the third paper compares estimation of the spectrum from uniform sampling to Poisson sampling, and in particular the case for finite sample sizes, as opposed to the asymptotic analysis performed in the first two papers. As mentioned the analysis is based on Poisson sampling, so the usefulness is limited when sampling times are given, see also Section 3.5. Paper A studies the frequency analysis problem in this thesis, with the major contribution being a priori analysis of existing transforms. The Dirichlet transform seen above will then be thoroughly examined.

3.3.1

Fast Algorithms for the Fourier Transform

This thesis is not concerned with the speed of the implementation, although it is of course always of importance when algorithms are designed for real-life. For the uniform sampling case, the FFT is instrumental because of its good implementation performance. For non-uniform time sampling, new considerations have to be made. We mention two contributions to this area.

3.3

21

Frequency Analysis

Algorithm 3.1 Fast Implementation of (3.21) (Potts et al., 2001, Alg. 12.1) Given: M, α > 1, fk , ym , and φ( f ). 1. Precompute m φ( fk − ) αM Z ck = φ( f )ei2π f (k−1) d f,

m = 1, . . . , αM, k = 1, . . . , M,

2. Form ( aˆk =

yk /ck , 0,

k k

= 1, . . . , M, = M + 1, . . . , αM

3. Use the FFT to get αM

am =

1 X −i2πkm/αM aˆk e , αM

m =1, . . . , αM,

k=1

4. Finally ˆ fk ) = Y(

αM X

am φ( fk −

m=1

m ), αM

k = 1, . . . , M,

is the approximation of the transform.

Potts et al. (2001) show a fast algorithm for computing an approximation of Y( fk ) =

M X

ym e−i2π fk m/M ,

k = 1, . . . , M,

(3.21)

m=1

at frequencies fk below the Nyquist rate. This is done by approximation of the transform as well as zero-padding. An oversampling factor α > 1 is introduced and Y( fk ) is approximated with ˆ fk ) = Y(

αM X m=1

am φ( fk −

m ), αM

(3.22)

where φ( f ) is some function with the same period as Y( f ). One of the suggested algorithms is summarized in Algorithm 3.1, and the paper also shows how to use this when the non-uniform grid is instead in time domain. This is also extended to include non-uniform sampling in both time and frequency. It is shown that Algorithm 3.1 is considerably faster than computing (3.21) directly.

22

3


The characteristics of the computation error are investigated for several choices of the approximation kernel φ( f ). Another approach is taken by Mednieks (1999). In this approach the sampling times are assigned based on a fine grid and on average the Nth point is used, tm = tm−1 + (N + τm )δ,

(3.23)

called additive pseudorandom sampling. The randomness is given by the integer variable τm , E[τm ] = 0, and δ is the granularity of the time grid, which gives E[tm ] = mNδ. This enables the use of the FFT algorithm on the extended sequence ( y˜ k =

ym , 0,

k = tm /δ , otherwise

k = 1, . . . , MN,

(3.24)

to find the Fourier transform in a fast way. This method is improved for this special case of sampling in order to reduce alias effects and increase implementation speed further. The resulting transform is exactly the same as using the Dirichlet transform on the sequence ym when tm is given by (3.23).

3.4

Reconstruction and Estimation

Reconstruction of a signal is particularly useful for certain image applications, and for A/D and D/A conversions. Recovery of the sampled signal at specific time points are important for a wide range of applications, for example, when analyses are performed on a slower time scale than the original sampling. Here, Problem 3.2 is considered. The estimate or reconstruction of s(t) is denoted sˆ(t) in this presentation.

3.4.1

Basis Expansions

More than 40 years ago, Beutler (1966) promised that for signals s(t) with a Fourier transform (even more general in fact), there exist functions hm (t) such that s(t) = lim

M→∞

M X

s(tm )hm (t)

(3.25)

m=1

converges uniformly for a fixed t, when the number of measurements on a fixed interval increases. It remains to find the basis functions hm (t) and hope that the error when using a finite M is small enough. Several contributions follow in this direction.

3.4

23


Yao and Thomas (1967) discuss the existence of Lagrange expansions, where a signal can be represented as sˆ(t) =

M X

ym hm (t),

(3.26a)

m=1

G(t) , (t − tm )G0 (tm ) Y G(t) ∝ (1 − t/tm ).

hm (t) =

(3.26b) (3.26c)

m

The requirements on the sampling sequence for this expansion to exist are given in the often cited non-uniform sampling theorem recapitulated in Theorem 3.1. Theorem 3.1 (Non-Uniform Sampling Theorem (Yao and Thomas, 1967, Thm. 2)) All functions s(t), with frequency content strictly below 1/2T Hz, possess a sampling expansion given by s(t) =

∞ X

s(tm )hm (t)

m=−∞

for each sampling sequence tm fulfilling |tm − mT| < L < ∞, |tm − tk | > δ > 0,

m , k, m, k = 0, ±1, ±2, . . . .

The expansion is given by (3.26). When L = 0 the well-known Shannon expansion with sinc functions can be recovered from (3.26). Russell (2002) uses the same idea and develops a recovery algorithm to calculate hm (t) and, ultimately, sˆ(t). Extensive analysis is performed to enable easy real-time implementation. Benedetto (1992) uses frame theory1 to derive reconstruction formulas for band-limited signals and specific assumptions on the sampling sequence, we refer to the article for details. A function s(t), which is band-limited on [−ω, ω], can be reconstructed with a basis function, hm (t) = h(t − tm ), which is band-limited on a wider band and has a frequency transform equal to 1 on [−ω, ω], as follows X sˆ(t) = am h(t − tm ), (3.27) m

where the parameters am ≈ Kym . The constant K is defined by the sampling times tm , and an exact expression for am is also given in the article. This is a special way of defining the basis function hm (t). Eldar (2003) considers a more abstract sampling procedure using inner product constructions and frame theory. For a special choice of sampling functions, xm (t), 1 We

refer to Benedetto (1992); Eldar (2003) and references therein for an introduction to frames.

24

3


this approach gives an alternative way of computing the coefficients, am , in (3.27). First, assume that the samples are given by an inner product with the original signal, and define X∗ such that ym = hxm , si,

y , X∗ s.

⇐⇒

(3.28)

Then, also define W so that sˆ(t) =

X

am wm (t) , aW.

(3.29)

m

The coefficients am collected in the vector a are then found using a pseudo inverse, a = (X∗ W)† y.

(3.30)

This can be seen as an orthogonal projection of s(t) onR the space spanned by the measurements. Choosing xm (t) = δ(t−tm ) and hxm , si = s(t)xm (t) dt gives a perfect match with our usual understanding of sampling, i.e., ym = s(tm ). The only thing left is to interpret X∗ and W accordingly, which might be non-trivial.

3.4.2

Iterative Solutions

Marvasti et al. (1991) consider stable sampling sets, tm , and band-limited signals, s(t). Stable sampling sets uniquely define the sampled signal (see for example Marvasti, 1987). The reconstruction of s(t) is done recursively, by defining the sample operator, S, X S x(t) = x(tm )δ(t − tm ), (3.31) m

giving a train of impulses. The kth recursion, s(k) (t), is then s(k+1) (t) = λ PS s(t) + (P −λ PS)s(k) (t), = P s(k) (t) + λ S s(t) − s(k) (t)

(3.32) (3.33)

Here P is the ideal band-limiting operator, so that s(t) and s(k) (t) have the same bandwidth for all k. Marvasti (1996) studies this reconstruction further, for the special case of tm = mT + τm with bounded τm . Feichtinger and Gröchenig (1994) give explicit iterative algorithms for reconstruction of signals band-limited to [−ω, ω], and also state an upper bound on the reconstruction error. One example is the Adaptive Weights Method, which requires oversampling. This method is investigated further in Feichtinger et al. (1995), and here a specialized implementation algorithm is given for polynomials of degree R and period 1, s(t) =

R X k=−R

ak ei2πkt .

(3.34)

3.4

25


Algorithm 3.2 presents a condensed version of the algorithm, with ha, bi being the inner product of two vectors. With some effort, it might be possible to extract a fast algorithm for calculation of the Dirichlet transform from this as well. The paper also includes a numerical comparison of several reconstruction algorithms.

Algorithm 3.2 Fast Reconstruction (Feichtinger et al., 1995, Thm. 1) Let R be the polynomial degree (3.34) and let 0 ≤ t1 < . . . < tM < 1 be an arbitrary sequence of sampling points with M ≥ 2R + 1. Set t0 = tM − 1, tM+1 = t1 + 1 and wm = (tm+1 − tm−1 )/2 and compute γk =

M X

wm e−i2πktm ,

k = 0, 1, . . . , 2R,

(3.35a)

m=1

bk =

M X

s(tm )wm e−i2πktm ,

|k| ≤ R.

(3.35b)

m=1

Let b = (b−R , . . . , bR ) and T be the matrix with elements {T}l,k = γl−k , for |l|, |k| ≤ R. Initialize r(0) = q(0) = b and a(0) = 0. From n ≥ 1, iteratively compute a(n) = a(n−1) +

hr(n−1) , q(n−1) i (n−1) q , hTq(n−1) , q(n−1) i

(3.36a)

r(n) = r(n−1) −

hr(n−1) , q(n−1) i Tq(n−1) , hTq(n−1) , q(n−1) i

(3.36b)

q(n) = r(n) −

hr(n−1) , Tq(n−1) i (n−1) q . hTq(n−1) , q(n−1) i

(3.36c)

After N ≤ 2R + 1 steps Ta(N) = b is solved, and sˆ(t) =

R X

(N)

ak ei2πkt

(3.37)

k=−R

gives the reconstructed function.

3.4.3

Other Reconstruction Cases

Masry and Cambanis (1981) focus on estimation of a signal, s(t), that is corrupted by a static nonlinearity, f (x), for example, the sign-function, and show convergence when the sampling frequency increases. The reconstruction is done by adding noise to the samples, ym = f (s(mT) + em ).

(3.38)

26

3


It is assumed that the signal is limited by a known constant |s(t)| ≤ b and that the distribution of em is chosen wisely, for example a rectangular distribution em ∈ Re[−b, b]. Masry and Cambanis describe some cases when a static function g(x) can be used to find s(t) from an estimate of the mean m(t) = E[ f (s(t) + e)].

(3.39)

ˆ It is shown that g(m(t)) → s(t) when T → 0, and the convergence properties are also discussed. In Souders et al. (1990), the main goal is to study effects of timing jitter, and s(t + τ) is studied when pτ (t) is given. It is shown that the mean is a biased estimator of s(t). This is further discussed in Paper B, where also the effects of a finite number of samples are considered. In the publication pair Kybic et al. (2002a,b), generalized sampling in higher domains is discussed. For this problem, the theory boils down to a cubic spline minimizing the second derivative of the reconstructed function. The result is known from previous spline literature, and both publications give extensive references to other work. The resulting reconstruction is given by defining sˆ(t) = a0 + a1 t +

M X

λm |t − tm |3 ,

(3.40)

m=1

and then solving the linear equation system sˆ(tm ) = ym , X λm = 0,

(3.41a) (3.41b)

m

X

λm tm = 0,

(3.41c)

m

for a0 , a1 and λm . A special case of reconstruction is resampling where only certain values of the underlying function, s(t), are sought. Paper C discusses this when the task is down-sampling and low-pass filtering of the sampled sequence. The algorithms in this section are then too complicated or have too hard requirements on the sampling sequence, tm , to be of use. Therefore, a more practical approach is taken.

3.5

Optimal Sampling

Optimal sampling is possible when the sampling points can be chosen before hand. It can be of importance, for example, for anti-jamming in radars, suppression of alias frequencies in the frequency transform, and placement of sensors for spatial sampling. Examples of this have already been given in previous sections, yet some more specialized ones exist. Here, Problem 3.3 is considered. This aspect of non-uniform signal processing is outside the scope of this thesis, and the research is therefore only briefly presented. The most common optimality

3.5

27

Optimal Sampling

criterion is suppression of alias frequencies, also known as alias-free sampling found in Section 3.5.2, but other contributions concerning the benefits of nonuniform sampling exist as well.

3.5.1

Benefits from Non-Uniform Sampling

Bilinskis and Mikelsons (1992) present several aspects on randomization in the sampling procedure, for example, that correlation between inter-sampling times can be beneficial. Work similar to Lemma A.1 given later in Paper A are also presented. Jacod (1993) investigates an estimation problem for a stochastic process and shows an optimal sampling procedure in the sense of maximizing the Fisher information for all the parameters. The interest is in asymptotic properties as the number of sampling instances, M, tends to infinity. Bland and Tarczynski (1997) empirically motivate non-uniform sampling and give some user guidelines on sampling time placements. In Papenfuss et al. (2003), an algorithm for hardware implementation of jittered sampling (tm = mT + τm , where τm is a random process) promises 40 times the bandwidth of the corresponding uniform sampling process, τm = 0.

3.5.2

Alias-Free Signal Processing

Digital alias-free signal processing (DASP) and alias-free sampling (AFS) are investigated in numerous works. The original mentioning of DASP methods can be found in Shapiro and Silverman (1960) and the methods have been enhanced after that. The main idea is the ability to choose placement of sampling points in order to reduce or remove aliasing. Shapiro and Silverman (1960) give a condition for when the sampling is alias free. Given additive random sampling, tm = tm−1 + τm ,

(3.42)

the requirement is that the characteristic function ϕτ ( f ) = E[e−i2π f τ ], for the sampling noise, τ, should be one-to-one on the real axis. Beutler (1970) discusses AFS for a specific class, S, of spectra, and gives a similar, but more general definition than Shapiro and Silverman. The sampling sequence, tm , is alias-free with respect to the class S if no two random processes with different spectra yield the same correlation sequence r[n] = E[ym+n ym ].

(3.43)

This will also correspond to demands on ϕτ ( f ), namely: The sampling sequence is alias free if Z∞

Z ϕτ ( f ) h( f ) d f = 0, for n = 1, 2, . . ., n

−∞

⇒

h( f ) d f = 0.

(3.44)

28

3


Note that the correlation sequence is formed independently of the actual sampling times, tm . Here, several examples of alias-free sampling time distributions are given, for example, it is shown that a uniform sequence where each sample is missing with probability q < 1, is alias-free if the sampled signal s(t) is band-limited to the Nyquist frequency (then the uniform sequence, mT, is also alias-free). It is also noted, that a formula for finding the spectra is still missing, or was at the time, see Section 3.3. The definition from Shapiro and Silverman is explored further in Masry (1978a) together with a new definition of AFS, which is ensured when g∞ (t) =

∞ X

pm (|t|) > 0, almost everywhere,

(3.45)

m=1

where pm (t) is the probability density function for the sampling time tm . The definitions concern the ability to separate the covariance sequence of the processes with different spectra, when sampled using a certain sampling sequence. The sequence tm , is assumed to be stationary, as well as the sampling intervals, tm −tm−1 . Only the sampling intervals were assumed stationary in the previous definitions. It is also shown here that the two definitions by Shapiro and Silverman and Masry do not coincide, and Masry argues that the second one is more practical for reconstruction purposes. Bland and Tarczynski (1997) find the optimum conditions on sampling time placements for maximum alias frequency suppression. Vandewalle et al. (2004) investigate the advantages of aliasing at image reconstruction. Here, two uniformly sampled sets are used, first tm = mT and second tm = mT + t1 where t1 is a small shift. The resulting two sequences are then combined to enhance image quality. Tarczynski and Allay (2004) study two ways of incorporating a window, w(t), in the Dirichlet transform. First, the sampling times are uniformly distributed over [0, tM ], pm (t) =

1 , tM

t ∈ [0, tM ],

(3.46)

and M tM X ym w(tm )e−i2π f tm (3.47) Y( f ) = M m=1 R giving an unbiased estimate of s(t)w(t)e−i2π f t dt. Second, the window is used in the placement of the sampling times,

pm (t) = where the constant A ensures

R

w(t) , AtM

t ∈ [0, tM ],

(3.48)

pm (t) dt = 1. The transform

Y( f ) = A

M tM X ym e−i2π f tm M m=1

(3.49)

3.5

Optimal Sampling

29

R is also an unbiased estimate of s(t)w(t)e−i2π f t dt. The authors also discuss the purpose of DASP without formal definitions of AFS. Note that the probability distribution is the same for all sampling times (pm (t) = pτ (t)) and they are independent of each other, as opposed to the case with ARS described previously.

30

3


4 Packet Data Traffic – A Motivating Example Packet data traffic on the Internet was discussed in Section 2.2 as one application that could benefit from non-uniform signal processing. This chapter will explore this further by a performance study, and a test of improved controllers, for the active queue management, AQM, problem. This presentation is aimed for the reader interested in this particular application or in more bridges between applications and the theoretical work given in later papers. This chapter does not present solutions to the non-uniform sampling problems in the application, but exemplifies and discusses them. The presentation is based on Gunnarsson et al. (2002), Gunnarsson et al. (2003) and Gunnarsson (2003). It also includes a summary of the problems in non-uniform sampling that arise for this application, together with references to the relevant papers in the thesis. By controlling network flows or network queue lengths, using AQM, the performance of a network can be improved. This work focuses on the flow level, and is based on current solutions, but the ideas easily carry over to independent settings. The approach is system theoretic, which is fairly new for network research. Several contributions have emerged that study network problems including queue management from a system theoretic viewpoint, e.g., Altman et al. (2000); Park et al. (2003); Hollot et al. (2001); Low et al. (2002); Kunniyur and Srikant (2004); Jacobsson and Hjalmarsson (2006). The rest of the chapter is organized as follows. Section 4.1 gives relevant background information on the complex Internet system. Section 4.2 starts by defining performance for a network and discusses performance assessments based on network queue lengths, together with an overview of performance measures used by other researchers. Section 4.3 develops models based on filtered measurements of the queue length. The models are recursively identified to adopt to varying network settings. In Section 4.4, various controllers are described and evaluated. 31

32

4

Packet Data Traffic – A Motivating Example

The discussions are accompanied by explicit examples on a dataset from the network simulator, ns-2(NS, 2003, ver. 2.1b8a). Finally, Section 4.6 summarizes the non-uniform signal processing problems and discusses how a solution could be found.

4.1

Background on Packet Data Networks

Usually when studying Internet issues, the focus is on a certain level in the system or on a particular problem. To understand the complexity of controlling the traffic, an understanding of the whole picture is needed. Different levels interact and counteract in their demands on the communication. Design of new algorithms require evaluation of the performance over the full network. Simulations can be done with different open source simulators, such as ns-2 (Fall and Varadhan) and REAL (Keshav, 1997), or using proprietary environments, for example, ULC-1000AN developed by OPNET (OPNET, 2007). In order to make a good design, on the other hand, models of different components and how they affect the performance of the specific task are needed. Due to the complexity of the network, simplifications at different levels are made to facilitate easier analysis. When designing for a certain layer, it is common to model lower layers as a delay, possibly varying, and a queue. In most cases, this gives a sufficient accuracy, but the behavior of the variation can be difficult to model and is strongly affected by the specific underlying protocols. The design parameter for the queue is the maximum length and the service rate which mostly is affected by the underlying topology. Standards and proposals in the network architecture are described in Requests For Comments, RFCs. Anyone can write an RFC and they have different categories such as standards track, informational, draft and experimental. As the name suggests, anyone can have opinions on the content, and originally the aim was to make Internet a product of consensus. More information about available RFCs can be found at http://rfc-editor.org, where also information about how to publish new ones are available.1

4.1.1

Network Preliminaries

To understand the structure of the network application, some background knowledge is necessary. First, a short overview of some of the key events during the first years of computer network history is given. Then, the building blocks of the Internet communication system, the layers and their functionality, are discussed. Internet History To give an overview of the time horizon for the Internet evolution, some of the key events are presented here in chronological order. A continuously developing 1 Note

that there are also a number of April Fool’s Day RFCs that can be entertaining to read

4.1


33

time line can be found in Zakon (2003), where also more references to and copies of the original papers are listed. In 1961, the first paper on packet switching, which was written by Leonard Kleinrock, was published and later on there was also a book on the subject by the same author. In 1962, a series of memos from J.C.R. Licklider were published, envisioning a globally interconnected set of computers, where everyone could access data from any site. In 1965, the first wide-area network connecting two computers was built by Lawrence Roberts. This experiment showed that packet switching was needed and circuit switching was totally inadequate. In 1966, several papers on packet networks were presented at a conference and, in 1968, the design of the first packet switches started at a small firm called BBN. Besides the team at BBN lead by Frank Heart, Robert Kahn was involved in the overall network architectural design, network topology and economics were designed and optimized by Roberts, and the network measurement system was prepared by Kleinrock’s team at UCLA. The first node in the network was installed at UCLA in 1969. By the end of the year four hosts,at the universities in Stanford, Santa Barbara and Utah, were connected together. In 1970, the first host-to-host protocol, Network Control Protocol (NCP), was designed and after the final implementation, the users of the network could start with application implementations. In 1972, the network was demonstrated to the public by Roberts and electronic mail was introduced. In 1973, the design of what was to become the protocol suite TCP/IP (Transmission Control Protocol/Internet Protocol) was started by Robert Kahn and Vinton Cerf. Much more information including references to the mentioned publications can be found also in Leiner et al. (2000, Sec. Origins of the Internet). Layer Structure To manage the large system of interacting computers, the information flow on the Internet is structured into layers. Each layer is responsible for a certain task and does not know anything about how layers above or below carry out their tasks. The actual data traverses the layer stack top down at the sender but the layer structure is transparent and information is seemed to be carried between the corresponding layers at the sender and the receiver using headers attached to the data. Every packet contains a header with information about the packet and data, which is delivered. Every layer adds a new header with information relevant for the corresponding layer at the receiver. The task of each layer is carried out using different protocols. Protocols are rules on how to handle files and packets at different layers. Different layers can use different protocols, depending on the type of transmission. Example 4.1: Data Transfer When the goal is to deliver a file from sender A to receiver B, different things happen at different layers. Both horizontal and vertical communication take place. The structure is described in Figure 4.1 together with names of some of the protocols for each layer.

34

4


Application

Application

FTP, HTTP

FTP, HTTP

Transport TCP, UDP

Transport TCP, UDP

Network IP(v6) Link hardware

Network

Network

Link

Link

Network IP(v6) Link hardware

Figure 4.1: The explicit data transfer (dashed) and the communication (solid) in the layered network structure. Each bar represents a physical position in the network, such as routers and servers.

Application A file is sent from A to B. When the whole file is successfully delivered a notification is sent to the sending application. The application delivers the IP address and port number to be used to the layer below. Transport The file is separated into packets, each with a header stating the size of the packet and the port to which it should be delivered. After each packet is successfully delivered an acknowledgment is received, otherwise it is retransmitted. The IP address is forwarded downwards. Network The packet is provided a header with the source and destination address, the IP addresses of A and B respectively, producing an IP frame. After each frame is sent, the next one is prepared. The address of the next computer in the network is determined and delivered to the next layer. Link The frame is given a header with the local address of the next computer and delivered on the physical link. The physical link can be both a wire and a radio link. In the latter case, a number of internal layers are used to enhance the communication over the radio link. In this case the delivery of the file is always successful since the transport layer makes sure that lost packets are retransmitted. For other application types, for example, real-time transmissions, this might not be a desired behavior and the correction can instead be made in the application, e.g., by using redundant data, coding or retransmissions.

In Figure 4.2, the principal components of a network are shown. The senders are at the ends and the packets traverses links and queues to the receiver. The receipts or acknowledgments, ACKs, go the other way. The setup can be compared to Figure 4.1 where the link and network layer exists in all nodes but the higher layers are only present in the sources and destinations.

4.1

35


flow 1 source

packets

flow 2 destination

queues

links

ACKs

flow 2 source

flow 1 destination data direction

Figure 4.2: A simple network with packets and receipts, queues and links. Each end node contains all the layers and the core nodes contains the two lower layers, see Figure 4.1.

A generic layer provides a service to the upper layer and uses a service from the lower layer. Decisions are made on whether to forward additional data downward based on quality measures. The lower layer is thus fed with a varying data rate and the lower layer needs to try to provide its service fast enough. This four layer model belongs to the TCP/IP protocol suite, which is a collection of protocols for all the different layers. In Stevens (1994, Ch. 1), a more formal description of the layers is given. The different parts are described further in the following sections.

4.1.2

Flow Control

TCP includes mechanisms for control of the send rate in order to interact with all traffic and prevent overloading or congestion of the network. As shown previously in Figure 4.2, the flow control performed by TCP resides in the end nodes, senders and receivers. The sender transmits data in packets and the receiver transmits ACKs in returning packets. Both the sender and the receiver use TCP mechanisms. At the packet arrival times, tk , control variables are calculated. TCP keeps an internal window, the congestion window ck , which is adapted as a measure of the capacity of the network, in bytes or packets. TCP also keeps track of the amount of data that is sent but not yet acknowledged, the outstanding data or flight size, f (t). The received advertised window, ak , which is delivered with the arriving packet, is the capacity of the receiver. These three, ck , f (t) and ak , are used to limit the outgoing send rate, rTCP (t), rTCP (t) = min(ck − f (t), ak ), for t ≥ tk

(4.1)

36

4


The congestion control algorithm is specified in RFC 2581 (1999). It states the updating mechanism for ck based on acknowledged data. A threshold, s, should be used to separate different parts of the update, slow start and congestion avoidance. During slow start, ck < s, c is increased with the size of one packet at the arrival of a new ACK. When ck ≥ s, the congestion avoidance phase is entered and the update is done with approximately one packet for each round trip time period, TRT , RTT. The round trip time is a measure of the time it takes from a packet is sent until it is acknowledged. The flight size, f (t), was defined as the sent but not yet acknowledged data and can therefore be calculated as the sent packets during the last round trip time Zt f (t) =

rTCP (τ) dτ.

(4.2)

t−TRT

Representing c and s in packets, rather than bytes, we get ( ck =

ck−1 + 1 1 ck−1 + ck−1

ck−1 ≤ s . ck−1 > s

(4.3)

There are two ways of noticing a failure of delivery for TCP, timeout and dupacks. Time out Time out occurs when one packet’s individual RTT is higher than a constant times the mean RTT. This indicates a traffic jam and several lost packets and therefore c is reinitialized. How to calculate and measure the RTT was described in Gunnarsson (2003, App. 2.A). Dupacks The ACKs inform the sender about expected deliveries. Therefore, subsequent ACKs with the same number indicate a loss. TCP interprets three duplicate ACKs, dupacks, as a single packet loss. The threshold, s, is also updated when a data loss is detected in one of the ways described above. The updates of c and s after a congestion is detected are given by ( ck =

1 s

time out, 3 dupacks,

(4.4)

s = max( f (t)/2, 2). The TCP algorithm is summarized in Algorithm 4.1. The specification of TCP has evolved over time and the one described here is the most used, called TCP Reno. Later version have more refined the mechanisms that trigger loss recoveries and also the updating of ck during congestion avoidance. A short overview of these updates were given in Gunnarsson (2003, App. 2.A).

4.1

37


Algorithm 4.1 TCP Flow Control At each arrival time, tk , the current congestion window, ck is calculated as        ck =      

1 s ck−1 + 1 1 ck−1 + ck−1

time out, 3 dupacks, ck−1 ≤ s, ck−1 > s.

The slow-start threshold, s, is updated on congestion according to s = max( f (tk )/2, 2). The flight size, f (t), changes continuously as Zt f (t) =

rTCP (τ) dτ. t−TRT

The send rate, rTCP , is given by rTCP (t) = min(ck − f (t), ak ), for t ≥ tk , where ak denotes the received advertised window.

4.1.3

Active Queue Management

The network layer protocols are present in all nodes in the network, for example in the ones shown in Figure 4.2. Active queue management (AQM) is a set of suggested additions to the network layer. The protocols use dropping and marking to inform TCP (explicitly or implicitly) about congestion dangers. Since TCP lowers its send rate, rTCP , when packets are lost, early dropping could be used to gently lower the send rate instead of causing a time out. This is supposed to attenuate oscillations in the queue length, prevent overflow and minimize empty queue time. Random Early Detection (RED) is one AQM suggestion, that uses a simple dropping profile based on the current queue length to decide which packets should be dropped. The longer the queue, the higher the probability of dropping an arriving packet. This technique was introduced in Floyd and Jacobson (1993) and has been widely adopted for further research. More specifically, when a packet arrives at the router at time, tk , a filtered value, yk , of the queue length, qk , is calculated as yk = (1 − λ)m yk−1 + λqk .

(4.5)

Here, m = 1 if qk > 0 and otherwise it is used to account for the time the queue

38

4


1

pM 00

qm

qM

Figure 4.3: The probability of dropping a packet as a function of the filtered queue length. The parameters qm , qM and pM are design variables.

was zero. If r is the service rate of the queue, ( 1, qk > 0, m= (tk − te )r, qk = 0. The time te is used to denote the time when the queue became empty, i.e., the smallest te such that q(t) = 0, te ≤ t ≤ tk . The probability, p, of dropping the new packet is then calculated as   0,    pM (y − q ), pk =  m k qM −qm    1,

yk ≤ qm , qm < yk ≤ qM , yk > qM ,

(4.6)

i.e., according to a linear function between the constants qm and qM . The constant 0 < pM < 1 is suggested to be in the order of 0.1 for good performance. An example of pk as a function of yk is given in Figure 4.3 and Algorithm 4.2 summarizes the actions. Explicit Congestion Notification (ECN) is described in RFC 3168 (2001). It is a technique that enables marking of packets instead of dropping, by using the TCP header. This requires changes in the TCP implementation as opposed to only dropping the packets, and the issue becomes how TCP should respond to these marks. Several proposals for improvements on the dropping/marking decision mechanisms have been made. Most of them are based on the same structure as is proposed for RED and a few of them are described in Gunnarsson (2003). A nice comparison of a few algorithms can be found in Zhu et al. (2002) and another overview is given in Asfand-E-Yar et al. (2004). Basically, (4.6) is replaced in the different proposals, for some of them (4.5) as well. There are also techniques that use RED functionality to achieve service differentiation. RFC 2474 (1998) describes differentiated services for IP traffic, DiffServ, where the RED parameters in (4.6) are tuned differently for each traffic class. DiffServ is also described in Kilkki (1999).

4.1

39


Algorithm 4.2 Random Early Detection (RED) At each arrival time, tk , the queue length qk is measured. A filtered estimate, yk , is calculated as yk = (1 − λ)m yk−1 + λqk , with the scaling factor, m, being ( m=

1 qk > 0, (tk − te )r qk = 0,

where r is the service rate of the queue and te = min{τ : q(t) = 0, τ ≤ t ≤ tk }. τ

The probability of drop, p, is then   0,    pM (y − q ), pk =  m k qM −qm    1,

4.1.4

yk ≤ qm , qm < yk ≤ qM , yk > qM .

Control Structures and Interactions

The standards described previously imply some control structure for each layer. These descriptions are here translated into block diagrams and cross effects between layers will be identified. As shown in Figure 4.2, packets on a network traverse queues and links. The queues add a queuing delay depending on their length and service rate and the links add a transmission delay. Therefore it is common to model the effects of a network on sent packets as a queue and a delay. Using block diagram representation, a pure delay, y(t) = x(t − T),

(4.7)

is written using the Fourier transform e−sT . The queue is fed with a certain arrival rate, ra , and sends packets with a service rate, rs . The difference between these two will be the increase in queue length. The total queue length at time t will be the accumulated difference from the initial time, i.e., the integral ( 1s ) of ra − rs , Zt q(t) =

(ra (τ) − rs (τ)) dτ.

(4.8)

0

The block diagram representations of the delay and of the queue are shown in Figure 4.4.

40

4

x(t)

e

−sT


rs (t) ra (t) + −

y(t) = x(t − T )

1 s

q(t) =

Rt 0

ra (τ ) − rs (τ ) dτ

Figure 4.4: A delay (left) and a queue (right) shown in block diagram representation. For the delay, x is the input and y is the output. For the queue, ra is the arrival rate and rs is the service rate.

rq (t) CT CP

rT CP (t)

e−sT1

− +

1 s

+

e−sT2

q(t)

−

qref Figure 4.5: The block diagram for a TCP connection. d(t) is the available capacity for that flow in the queue and can be considered as a disturbance. T1 and T2 are transport delays for the connection and Tq is the queuing time, i.e., TRT = T1 + T2 + Tq .

Transmission Control Protocol The control strategy in the transport layer is provided by TCP and was described in Section 4.1.1. The controller, CTCP , acts based on notifications of deliveries and drops (q > qmax = qre f ) and it controls the send rate, rTCP , into the network. Figure 4.5 shows the network as delays and a queue, which is how TCP sees it. The queue increases with the delivery rate, rTCP (t − T1 ) and decreases with the ˙ = rTCP (t − T1 ) − rq (t). The total round trip time of the service rate rq (t), i.e., q(t) network, TRT , will depend on both the transmission delays, T1 and T2 , and the queuing time, Tq . The queuing time can be found from solving the equation t+Tq Z rq (τ) dτ = q(t),

(4.9)

t

i.e., the time it would take to empty a queue of length q(t) with the specified send rate, rq . Since TCP Reno only acts based on explicit drops, the block diagram could be extended with a switch. The control signal back to CTCP is 1 if q > qre f and 0 otherwise.

4.1

41


qref q(t) H

CAQM

u(t)

p(t)

Figure 4.6: The block diagram for a AQM queue. The blocks can be added to TCP’s block diagram, Figure 4.5, between the queue, q, and the return delay, T2 .

Active Queue Management The additional functionality provided by the AQM mechanisms adds to the control loops. In Figure 4.6, a block diagram for an AQM scheme is shown. The queue length, q, is filtered through H. A control decision, u, is calculated in CAQM based on qre f and limited to produce the dropping probability, p. The dropping probability was before either 0 or 1, either there was room in the queue or there was not. Incorporating the AQM control with the TCP representation shows that AQM wants to change the reference for TCP, qre f , from being the maximum queue length to something lower. The blocks should be included between the queue, q, and the return delay, T2 , in Figure 4.5. The AQM block diagram shows that only drops are used for the control in TCP. This part is hidden in CTCP in Figure 4.5. Several TCP connections will add to the queue length in the AQM queue and each drop affects one of them. If DiffServ, Section 4.1.3, is deployed, parallel AQM links with different controllers and different reference values will divide the TCP flows among them according to traffic class.

4.1.5

In-house Work

A number of Master’s theses have studied problems with packet data traffic in different contexts. Gunnarsson (2000) studied problems with TCP congestion control and laid the foundation for the work presented here on packet data. TCP in other contexts was studied by Persson (2002a), focusing on ad hoc networks, and Brunberg (2002) in a Digital Radio Broadcast framework. Tronarp (2003) focused on evaluation of Quality of Service, QoS, in ad hoc networks. Björsson (2003), Knutsson (2004), and Zhao and Wu (2005) included 3rd generation radio network capabilities in ns-2 (NS, 2003, ver. 2.1b9a), and also performed simulation studies. Studies of performance in 3G radio nets have been done by Adolfsson (2003), Fredholm and Nilsson (2003), and Eriksson (2004).

42

4

4.2


Performance Issues in Queue Management

Good network performance means different things depending on who is asked. The network provider might have a different measure than the user. To find a good performance measure, the different issues that can be identified will be discussed here. For each user, transmission time and reliability are important. When TCP is used, the reliability is assured and the only consideration is the transmission time. Thus, one issue in network performance is the time to transmit a certain number of bytes, measured on a per flow (or per user) basis. The network provider wants to fully utilize the available capacity and divide it among the users. This gives the requirement that there should always be packets processed by the slowest node. The issue of dividing the resources in an efficient way is not easy to resolve. Efficiency can be measured from both the overall network and from the single user, and optimizing the efficiency of the network might not be fair among the users and might cause users to change network provider. The two extreme cases are “total fairness”, which means that every user gets exactly the same share of the resources, and “best effort”, where no guarantees are given about delay or bandwidth share, and the matter has been discussed in various scheduling contexts (see for example Kelly, 2003). We will not discuss the matter of resource sharing more here. Flow control is the main way to of vary the network load. As was described in the history review of Internet development in Section 4.1.1, the original flow control was performed at the end nodes, by a predecessor to the TCP protocol. After an immense increase in both size and number of available applications, the Internet traffic does not behave as it did initially. In this section, the previously identified problems caused by TCP are described, together with the research efforts that have been made based on these problems. Current research about performance measurements based on the network queues is also discussed.

4.2.1

Problems with TCP

For a long time, the transport protocol TCP has been seen as the main troublemaker, when performance is concerned. A number of problems have been identified and corresponding solutions have been proposed. Here, the research efforts have been grouped according to the problems they address. A more thorough description of each problem can be found in the corresponding references. The problems are also described in correspondence with the previous block diagram representation, see Figure 4.5. 1. TCP provokes packet losses to get an idea about the state of the network. We get implicit feedback only when something goes wrong. • This problem has been addressed with the inclusion of explicit feedback by Mascolo (1999). • This problem arises because the controller, CTCP , in Figure 4.5, only bases its decisions on packet drops.

4.2


43

2. The use of the available bandwidth oscillates. Thus the load of the network varies and the network becomes slower than it has to be. • This problem has been addressed using adaptive queue management, AQM, starting with RED and extensions such as ECN (Floyd and Jacobson, 1993; Floyd, 1994). • Many research efforts aim to improve performance of the initial AQM algorithms, e.g., Kunniyur and Srikant (2004); Deng et al. (2002) • In the block diagram, Figure 4.5, this is an effect of the design of the controller dynamics, i.e., the algorithm for calculating rTCP based on measurements (drops). 3. There has been a shift in the Internet traffic mix. (Measurements can be found in Balakrishnan et al. (1998); Padhye et al. (1998).) (a) Due to short transmissions, TCP is “always” (85%) in slow start. (b) Originally less than 1% of the packets were lost, now it is as much as 5 − 7%, which means a lot of retransmissions, (13% of the transmissions), most of which are due to timeouts. • The newer versions of TCP, such as Vegas, address this problem (Brakmo and Peterson, 1995) • The problem with changing traffic characteristics is also discussed in Jacobsson et al. (2006). • The original design of CTCP was done based on other assumptions about the rest of the system. A simple way is to say that the characteristics of the noise, or available capacity, rs , has changed. 4. No difference is made between different traffic classes, such as FTP files, video streams or e-mail. • This falls under the area of quality of service, QoS, and DiffServ has been developed to solve this problem (Kilkki, 1999). • This is the responsibility of the network, and the problem lies in the construction of the controller CAQM , Figure 4.6. The same parameter settings are used for different flows or types of flows. 5. TCP always assumes that packet losses are due to congestion. • This is a faulty assumption in wireless networks where packet losses can be caused by bad links or complete link failures. Some solutions have been proposed for ad hoc networks, where all nodes are both routers and potential receivers (Chandran et al., 2001; Liu and Singh, 2001). • Some of the proposed changes use this feature to improve performance, especially in adaptive queue management (Floyd and Jacobson, 1993; Kunniyur and Srikant, 2004).

44

4


• This is due to the design of the controller, CTCP , and how it interprets drops. Many proposals solve a problem by treating some of the other problems as features of TCP and include them in the knowledge of the system. In this way, queue management is moved from the end nodes into the queues, i.e., the routers. As mentioned before, this family of techniques is called adaptive queue management, AQM. The idea is that it is easier to include changes in some of the routers and improve performance locally instead of having to include changes in most of the TCP implementations to improve somewhere. Since any TCP implementation can be used over any router it seems logical to place the improvements in the router of interest.

4.2.2

Measurements Used in Current Research

With all the research efforts put into the problem of improving TCP, performance evaluations become more and more important. Some of the measures that are used in different contexts are briefly described below. Sachs and Meyer (2001); Meyer (1999); Peisa and Meyer (2001) use the time to transmit a certain number of bytes or the packet bit rate to evaluate mobile Internet access, to analyze TCP and file transfers in 3rd generation mobile network. Misra et al. (1999) develop a model of the distribution of the queue length and the congestion window and use these to investigate TCP’s performance. Brakmo and Peterson (1995); Mascolo et al. (2001) consider throughput (bytes/s) and Jain’s fairness index (Jain, 1991), as performance measures to analyze the improvements of TCP when using different TCP versions, such as Reno, Vegas and Westwood. Hollot et al. (2001) models RED queue behavior using a probabilistic viewpoint and use the model to calculate RED parameters that achieve stability and also fulfill other control criteria. Park et al. (2003) study the oscillations of the queue length and try to attenuate them. Jacobsson et al. (2004); Altman et al. (2000) use the overall throughput and the number of packet losses as performance measures. After the discussion above and previously in the section, a personal view is presented for network performance measuring. The emphasize is put on the network queue since the earlier discussions showed that throughput, oscillations in the queue and other queue mechanisms are important in the performance discussion.

4.2


4.2.3

45

Using the Bottleneck Queue

We will now take the perspective of a network provider who wants satisfied customers. The focus will be on a specific part of the network, namely the bottleneck queue. Definition 4.1 (The bottleneck queue). The bottleneck queue of a flow is the queue that is the slowest one for that flow and, consequently, a queue where that flow has more than one packet at a time. There can of course be many bottlenecks in one network, but only one per flow and time instant. When investigating the network performance from the bottleneck queue perspective, we can measure The time the queue is unused. This indicates how efficiently the network is used. An idle bottleneck queue indicates a waste of capacity. The fluctuations of the queue length. This illustrates the ability to find a steady state of the network. The more stable the queue length is, the more stable is also the round trip time experienced by each connection. A stable queue will also make it easier for new connections if the total network is experienced as being in steady state, and thus scalability is improved. The fraction of dropped packets. This fraction represents how fast changes of the network are noticed. A larger number of dropped packets indicates a longer response time to overflows. The throughput. The throughput, also, signals if the queue often is empty. Of course, a higher throughput is better. It is also interesting for the owner of this bottleneck queue if the income is based on delivered bytes or packets. The average queue length A long average queue implies large round trip delays and a high probability to drop packets, while a short average queue means high probability for an unused queue. Control objective: The conclusion is that we want to control the tails of the queue length distribution to keep them small. If this is done, drops and empty queues are avoided. We also want to make the distribution narrow around a, not too high and not too low, mean value, which assures small variations in the queue. This, once again, points at the importance of having information about the actual queue, in particular its length.

46

4

4.3


Modeling and Identification of the Queue Dynamics

In order to achieve the control objective, the block diagram description, introduced in Section 4.1.4, will be used. Figure 4.7 is essentially the same as Figure 4.6 with P describing the dynamical relationship between the drop decision and the arrival rate to the queue, rnw . This is the considered system and it is a connection of TCP links, similar to the one seen in Figure 4.5. From the queue perspective this system is unknown: the number of flows, their respective round trip times and the type of TCP controller they obey. It is only known that, after some delay, the queue length decreases when packets are dropped and it decreases more rapidly if a larger number of packets are dropped at the same time. The queue send rate, rs , is given and the AQM controller, CAQM , bases the dropping decision on the filtered queue length, y.

rs qref

CAQM

+

u

p −1

P

rnw

− +

1 s

q

−

y

H

Figure 4.7: Block diagram of the queue system. The AQM part is the same as in Figure 4.6. The filtered queue length is denoted y. The system P describes the network effects on the queue length after a certain drop is done.

Several modeling efforts that focus on the bottleneck queue have been made. For example, Asfand-E-Yar et al. (2004) find a closed form model of the RED mechanisms in order to analyze AQM performance; Jacobsson et al. (2006) develop a model from the acknowledgment count to the bottleneck queue length and hereby study the impact of different traffic types; Deb and Srikant (2006) model a single bottleneck queue with the aim to predict steady state properties of the arrival process. A nice overview of the estimation problems in congestion control is given in Jacobsson and Hjalmarsson (2004). The focus in this work is on a simple model of the queue length dynamics without any knowledge of the rest of the system P. The qualitative knowledge about P will be used in simple controllers in later sections. To exemplify and motivate the modeling choices throughout this section, a typical data set is used. Example 4.2: Dataset Using ns-2 (NS, 2003, ver. 2.1b8a) realistic network data can be produced. We measure the queue length, q, at the bottleneck node in a network. The network

47


20

20

15

15

15

10

5

Queue length, q

20

Queue length, q

Queue length, q

4.3

10

5

0

5

0 0

20

40

60

80

100

10

0 20

30

time [s]

40

50

time [s]

30

32

34

36

38

40

time [s]

Figure 4.8: The motivation data example from a bottleneck queue with the number of active transmissions varying from 0 to 100. Both the full time span and shorter segments are shown. The data is taken from ns-2 (NS, 2003).

has between 0 and 100 sources sending TCP traffic over the bottleneck queue at the same time. The sending periods for the sources are selected randomly. Figure 4.8 shows the measured queue length over time. Inspection of the plots shows that the basic variation, for this network setup, is in the order of 0.6 Hz.

4.3.1

Frequency Analysis and Filtering

The Fourier transform, Xc ( f ), can be used to describe the frequency content of a signal, x(t), Z∞ Xc ( f ) =

x(t)e−i2π f t dt.

(4.10)

−∞

This demands that Xd , is used

R

|x|2 < ∞. If the signal is sampled, x(kT) k = 1, . . . , N, the DFT,

Xd ( f ) = √

1

N X

NT

k=1

x(kT)e−i2π f kT .

(4.11)

Xd ( f ) is what will be used in this work. Example 4.3: Frequency Content Visual inspection of primarily the right part of Figure 4.8 shows that the signal is oscillating rapidly around the basic variation. Frequency analysis of the queue

48

4


1 10

1 10

1 10

0 10

0 10

0 10

−1 10

−1 10

−1 10

−2 10

−2 10

−2 10

−3 10

−3 10

−3 10

0

5

10

15

20

frequency [Hz]

0

0.5

1

1.5

frequency [Hz]

2

10

11

12

13

14

15

frequency [Hz]

Figure 4.9: The frequency content of the dataset described in Example 4.2. |Xs ( f )|2 is shown for the shortest time interval. Focusing on different frequency ranges shows the peak around 0.6. The high frequency variations are spread over several frequencies.

length shows, see Figure 4.9, that a majority of the frequency content is around 0.6 Hz. The faster variations that were visible in the time plot cannot be seen in a single frequency peak.

Filtering of the signal is performed to reduce the fast oscillations and the noise. The control can then be based on the long term variations (0.6 Hz). The noise will be reduced by a simple first order low pass filter, H(s) = which attenuates frequencies higher than

a , s+a a 2π

(4.12)

Hz more than 3 dB.

Example 4.4: Filtering The main frequency content in the shorter time segment is below f = 1 Hz. With a = 6.5 the low pass filter effectively attenuates frequencies above 1.03 Hz more than 3 dB. The result is clearly visible in a comparison of the two signals, see Figure 4.10. The phase delay introduced by the filter is inevitable in the real-time implementation, and the filter choice will be a trade-off between smoothing and delay.

4.3

49


queue length

20

15

10

5 33.5

34

34.5

35

35.5

36

36.5

37

37.5

38

time [s]

Figure 4.10: Comparing the filtered signal (thick line) and the original one (thin line) for a short time.

4.3.2

AR Modeling of Nonzero-Mean Signals

Auto regressive (AR) models are models of a signal based only on the signal itself, see for example Ljung (1999) for a thorough discussion on AR models. The basic model equation contains the signal and time shifts of it. Here, we study variations around a fixed unknown level b, and get y(t) = ϕT (t)θ + v(t) ϕ(t) = −y(t − T) . . . −y(t − nT) T θ = a1 . . . an b

1

T

(4.13)

where ϕ(t) contains the known signal and θ are the parameters to be estimated. This model structure can capture the basic frequencies of a signal. The frequencies are given by the angles of the complex roots, x0 , to the characteristic polynomial xn + a1 xn−1 + . . . + an = 0.

(4.14)

The frequencies are thus f =

arg(x0 ) 1 , π 2T

(4.15)

where T is the sampling interval as before. Recursive identification is performed using recursive least squares (RLS), see for example Ljung (1999). The algorithm is summarized in Algorithm 4.3. The AR model structure is particularly simple to handle in this type of identification. After an estimate of the current model parameters is obtained, future values of the signal, y, can be predicted. For a model with sample time T, one step ahead

50

4


Algorithm 4.3 Recursive Least Squares (RLS) (Ljung, 1999, Sec. 11.2) Calculate a new estimate of θ at time t according to ˆ − T) ε(t) = y(t) − ϕT (t)θ(t P(t − T)ϕ(t)ϕT (t)P(t − T) 1 P(t) = P(t − T) − λ λ + ϕT (t)P(t − T)ϕ(t)

!

K(t) = P(t)ϕ(t) ˆ = θ(t ˆ − T) + K(t)ε(t) θ(t) Here, λ is the forgetting factor chosen as λ < 1. predictions are calculated as ˆ ˆ + T) = ϕT (t + T)θ(t). y(t

(4.16)

Studying (4.13) we see that ϕ(t + T), in the expression above, contains values ˆ are the most recent of y from time t and older. The estimated parameters θ(t) estimate. Predictions further away into the future can also be calculated using (4.16) recursively, but of course with larger uncertainty, both because of noise and of model changes. In the case of an AR model, when future values of y are needed in ϕ the predicted values are used instead. The accuracy of the prediction is thus effected by the accuracy of earlier predictions as well as of the accuracy of the model estimation. Example 4.5: AR Modeling An AR model has been estimated from the data set described in Example 4.2. It turned out that a fifth order AR model gave a good fit. A segment of the filtered ˆ is shown in Figure 4.11, queue length, y, and the estimated model output, y, together with the two estimated frequencies. For this example, RLS with λ = 0.999 was used. The basic frequency is around 0.6 Hz, while the second one varies over time, cf., the frequency plots in Figure 4.9 where no distinct second peak was visible. We compare the one-step ahead predictor (4.16) for the AR model with the ˆ + T) = y(t). To measure the performance of the estimates the trivial predictor y(t relative distance between the measurements and the estimates is calculated, sP ˆ2 |y − y| P 2 . R=1− |y| A value close to 1 indicates a good fit. For the AR model R = 0.83 and for the trivial predictor R = 0.75. From this we see that the knowledge about the data is increased by the identified AR model.

51

Control of the Queue Length

20

1

18

0.9 freqeuncy estimate from AR model, [Hz]

4.4

16

queue length

14 12 10 8 6 4 2

0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1

0

0 25

30

35

40

20

30

time [s]

40

50

60

70

80

time [s]

Figure 4.11: To the left is a comparison of the filtered queue length, y(t) (thin) ˆ (thick), from Example 4.5. To the right is ˆ = ϕT (t)θ(t) and model output, y(t) the corresponding frequency estimates, given by (4.15).

4.4


In Section 4.2.1, a number of problems with TCP were discussed. The control of the queue dynamics that will be discussed here addresses Problems 2 (bandwidth oscillation) and 3b (increase of packet losses). The control technique utilizes the identification procedure discussed earlier and considers Problem 5 (packet losses always lowers the send rate) as system knowledge that can be used in the control design. As discussed previously, the goals in queue management are to attenuate oscillations and to keep the queue from emptying and overflowing. The previous section indicated that the queue dynamics is well described by an AR model. To reduce the oscillations, derivative action is needed and a few simple versions will be described here. The control is primarily based on the ideas in the existing RED algorithm. In standard RED, the packet dropping probability is first calculated and then the packet is dropped or accepted based on this probability. A natural extension is to drop packets only when the queue is increasing, which is a very simple inclusion of derivative action into the controller. Derivative action changes the control decision based on the derivative of the signal, and a large derivative typically yields a lower control signal. The idea with the adaptively estimated AR model is to predict the evolution of the queue length in the near future and to use this prediction as a base for control decisions. Combining prediction and this type of derivative action with the basic RED algorithm produces four alternatives, which are described below. For reference,

52

4

the saturation operator is first defined:   0,    x, sat(x) =     1,


x≤0 0 0 k( y(t u(t) = ˆ˙ + T) ≤ 0 0 y(t p(t) = sat(u(t)) For all cases, the constant r can be seen as the reference value towards which the queue length is controlled. Since only the bottleneck queue is considered, the queue length is increased, by increasing TCP send rates, until the control kicks in, y(t) > r. The difference between the first two approaches can be seen as the difference between using the shifted model or the identified AR model in Example 4.5. To implement the derivative action, an estimate of the derivative is needed. The derivative action can be included in other ways, the more typical setup gives a fifth alternative.

4.4

53


5. Explicit D action An extension is to use derivative action more explicitly ˙ u(t) = kp (y(t) − r) + kd y(t) p(t) = sat(u(t)) in which case it becomes important to have reliable derivative estimates. This alternative can of course also be used with predictions. Here we restrict the investigation to methods 1—4, and use a simple backward difference approximation of the derivative. A discussion about the derivative estimation for this case was done in Gunnarsson (2003, Sec. 3.3).

4.4.1

Simulation Results

Here, the performance of the described control strategies is analyzed using simulations in ns-2 (NS, 2003, ver. 2.1b8a). A special version of the RED queue module has been implemented, which allows testing of both identification and control performance. The setup is a network with varying number of flows and only one queue is studied. For comparative studies, identical realizations of the network are used, i.e., the start and stop times for the flows are fixed in each simulation to get comparable and reproducible conditions for all controllers. Since it is not likely that one of the update times ti matches t−kT, k = 1, 2, . . . , n, simple linear interpolation between the two surrounding measurement points is used whenever an old value of y is needed. Here, more advanced methods for down-sampling of non-uniformly sampled signals could be beneficial. Because of the uncertainty in the derivative calculation, controller number 5 has not been implemented. All the other control versions can easily be implemented using the generic Algorithm 4.4, where a simple backward difference estimate is used for the derivative. During the simulations, Mp = 0.2, Mq is set to the actual total queue length and mq = Mq /2. The question arises what the suitable sampling time, T, is. Using the estimated model, the oscillating frequency, f , can be found and T should then, as a rule of 1 . Spectral analysis may thumb, be chosen as a tenth of the period, i.e., T ≈ 1f 10 also be used to find the main frequency. Here, methods for spectral analysis of non-uniformly sampled signals would be appropriate. Simulations indicate that it is the send rate of the queue that is the main cause of the varying frequency, see Figure 4.12. Here, the scenarios have been adapted 1 1 to work for T = 0.2 s ≈ 10 0.6 , the results are valid no matter what value we choose but the choice is still a design parameter. To improve performance, automatic detection of an accurate time constant should be considered. A few performance measures are summarized in Table 4.1. These numbers are calculated for one single run with exactly the same setup for all four controllers. In Figure 4.13 the resulting queue length is shown for the four controllers. The following observations can be made from Table 4.1 and Figure 4.13:

54

4


Algorithm 4.4 Generic AQM Implementation y1 − y2 , Td ( y −m Mp M1q −mqq , u= 0,

yd =

yd > 0, otherwise,

p = sat(u). The variables y1 , y2 and Td are used to get the different versions from p. 52, 1. P: y1 = y(t), y2 = 0 and Td = y1 − y2 , ˆ + T), y2 = y(t) and Td = y1 − y2 , 2. P pred: y1 = y(t 3. PD: y1 = y(t), y2 = y(t − T) and Td = T, ˆ + T), y2 = y(t) ˆ and Td = T. 4. PD pred: y1 = y(t

service rate

60

50

40

30

queue length

20

10

0 50

55

60

65

70

75

80

85

90

95

time [s]

Figure 4.12: Comparing different send rates (top thick line) with the corresponding queue length (lower thin line). A lower send rate in the queue seems to slow down the oscillations of the queue length.

4.4

55


2. P predictions 30

25

25

queue length, q, [packets]


1. P 30

20 15 10 5 0 50

55

60

65

20 15 10 5 0 50

70

55

time [s]

25

25

20 15 10 5

55

60

time [s]

65

70

65

70

4. PD predictions 30



3. PD 30

0 50

60

time [s]

65

70

20 15 10 5 0 50

55

60

time [s]

Figure 4.13: A segment of the queue length, q, when using the four different controllers. Both differences in the height and the freqeuncy of the variations can be seen.

56

4


Table 4.1: Comparison of the different control schemes, see p. 52 for a description. The number of drops and the total throughput is studied. Type 1. 2. 3. 4.

P P pred PD PD pred

Drops early+hard = tot 55 + 250 = 305 66 + 247 = 313 82 + 207 = 289 62 + 235 = 297

Sent unique packets 3260 3569 3126 3342

• An increase in throughput is accompanied with an increase in the total number of drops. • Model predictions can improve the throughput (7–10%), both in basic RED (compare 1 to 2) and in RED with D action (compare 3 to 4). • The simple derivative action used does not increase the throughput. • In both cases the derivative action reduces the number of hard drops, compare 1 to 3 and 2 to 4, which also was part of the stated control objective. • The derivative action lowers the span of the main variations in the queue length, which explains the reduced number of hard drops. • Using predictions reduces the number of highest peaks in the variations and increases the frequency. Reducing the highest peaks should also indicate a lower number of hard drops, while an increased basic frequency indicates that using the predictions removes the frequencies captured in the model. Ordering the algorithms becomes hard, since the aim is to both reduce the number of hard drops and to increase the throughput. It is apparent that predictions should be used and, actually the fourth algorithm (PD pred) performs second best for both cases. The evaluation is promising for the model based control, and further investigations of prediction and improved derivative action would be interesting. Closed-loop Identification The system performs in closed loop and the output affects later inputs. This can be a problem when system identification is used. In this case, the model identification is based on finding variations and basic frequencies and the control goal is to attenuate the oscillations. This means that a perfect controller will make it hard to estimate the model and a bad model will make the controller worse. These properties will make it impossible to find a perfect controller for this system. However, the delay between a packet loss and a reaction in the queue length is so large that it becomes impossible to control the system perfectly because of this. Therefore, it is worth the effort to use model based control for the network

4.5

Summary

57

system. Closed-loop identification and validation problems for congestion control are discussed in Jacobsson and Hjalmarsson (2006).

4.5

Summary

Network performance can be studied from a queue perspective. The control objectives for the queue is to attenuate oscillations, prevent overflow and avoid an empty queue. Queue management can improve network performance by intelligently dropping packets. The standard algorithm, RED, for AQM can be improved by using prediction. To attenuate queue length oscillations derivative action has been tested. The throughput was not improved using the simple derivative action, but the oscillations were attenuated. Explicit derivative action was not included in the evaluation runs. The experiments indicate that model based estimation of the derivative can give improvements. In the simulation run, only a difference approximation was used to find the derivative. The improved controller was based on accurate modelling of the queue length dynamics. It seems that a biased AR model is well suited for this purpose.

4.6

Open Problems

This chapter was given as a motivating example to the work with non-uniform sampling. Here we summarize the issues that could improve the work presented here, and give references to the text. It is clear that, in this application, the non-uniform sampling is given by events (packet arrivals), and the interesting measurement, the queue length, is completely described from this sampling process. The problems that arise are summarized below. Problem 1 In order to find a good value of the sampling time T, the low frequency content in y(t) is needed, see Example 4.3 and the comments on p. 53. Theory In this thesis, further work in this area is done in Paper A. Application The theories can be applied quite easily to the network data in order to perform off-line frequency estimation, as well as other off-line investigations of the spectrum. Problem 2 Low pass filtering and down-sampling of q would give y only in the interesting time points, see Example 4.4, Section 4.3.2 and the notes on p. 53. Theory The problem of down-sampling from non-uniform sampling times is addressed in Paper C. Application It is an easy task to implement down-sampling for the network queue, given the results. The problem is probably to satisfy the realtime demands. A solution to this problem might be to perform the

58

4


model identification and control actions on a slower time scale than the packet arrival times. Problem 3 Identification directly from non-uniform samples is another issue that gives improvements. This could give continuous-time models without the dependence on the choice of T. Theory Paper B discusses this matter for a specific case, but recursive identification from non-uniform samples is still an open question. Application Off-line identification could be used to get a nominal model, that could be used in real-time implementations, while the recursive identification still would follow the current procedure. Problem 4 This applications have high demands on speed, which makes the implementation performance important. Theory Some thoughts on this are given in Paper C, while it is a major problem in Paper B, and more research needs to be done.

5 Summary of the Thesis

This chapter gives a little foretaste of what has been done, and can be investigated further in the included papers, and also makes some suggestions on what could be done in the future, based on these results.

5.1

Main Contributions

Here, the main results of the included papers are given, in a very compact way, with selected equations and figures from the papers. The notation in Chapter 3 is still valid.

5.1.1

Frequency Domain Analysis

In Paper A, two types of transforms and two sampling models are studied. We exemplify with the Dirichlet transform, Y( f ) =

M X

ym e−i2π f tm ,

(5.1)

m=1

and additive random sampling, ARS, tm = tm−1 + τm ,

m = 1, . . . , M.

(5.2)

The results give analytical expressions for mean value and covariance of the transform Y( f ), with the effects from the original signal, s(t), the non-uniform sample times, tm , and the limited number of samples, M, clearly separated. Here, the characteristic function (CF), ϕτ (t) = E[e−i2π f τ ], 59

(5.3)

60

5

Summary of the Thesis

of the sampling noise, τm , plays a crucial role. For example, in uniform sampling the discrete-time spectrum is given by a convolution of the spectrum of s(t) with the normalized Dirchlet kernel, dM ( f ) =

1 − e−i2π f MT . 1 − e−i2π f T

(5.4)

This convolution gives the aliasing effect due to a finite number of samples. For ARS, this corresponds to convolution with dARS M (f) =

1 − ϕτ ( f )M 1 − E[e−i2π f τ ]M = . 1 − ϕτ ( f ) 1 − E[e−i2π f τ ]

(5.5)

The results for the transform Y( f ) lead us to results for the periodogram PY ( f ) = |Y( f )|2 . The mean value of PY ( f ) is straightforward to find, and given some asymptotic analysis (M → ∞), the variance of PY ( f ) can also be calculated. As an example, we study the signal s(t) = 1.2 sin(2πt) + 0.8 sin(4πt),

(5.6a)

ym = s(tm ),

(5.6b)

tm = tm−1 + τm ,

(5.6c)

using ARS with τm taken from a rectangular distribution between 0.1 and 1.3, pτ (t) =

1 , 1.2

t ∈ [0.1, 1.3].

(5.7)

The periodogram and its first two moments for this signal are shown in Figure 5.1. The mean value clearly shows the two frequency peaks, but the sampling noise contributes to several other peaks for the single realization.

5.1.2

Frequency Domain Identification

For the study of identification of non-uniform sampled systems, we consider the special case of stochastic jitter sampling with unknown jitter noise, i.e., s(t) = (gθ ? u)(t),

(5.8a)

ym = s(mT + τm ) + em ,

(5.8b)

τm ∈ pθ (τ)

(5.8c)

and the values of τm are unknown. The input u(t) is assumed known, or at least its Fourier transform, U( f ), and also the measurement noise, em , has known properties. Both the system gθ (t) and the jitter sampling noise pdf pθ (τ) are parameterized with θ. We use the discrete time Fourier transform (DTFT) of this sequence, Y( f ) =

M X m=1

ym e−i2π f mT ,

(5.9)

5.1

61

Main Contributions

0.5

0.4

0.3

0.2

0.1

0 −1

−0.5

0

0.5

1

1.5

2

2.5

3

3.5

4

Figure 5.1: Replica of Figure A.3b. Mean value (thick line), confidence band (gray shade) and one realization (thin line) of the periodogram, when the signal is a multi-sine, (5.6), and the sampling is ARS with rectangular noise between 0.1 and 1.3. and derive mean value and variance for Y( f ). These moments are once again functions of the characteristic function of the sampling noise, τm , and of the original signal. This enables us to calculate optimal parameters for both the unknown system, gθ (t), and unknown sampling noise pdf, pθ (τ), based on minimization in the frequency domain. For example, a least squares setting is θˆ = arg min θ

X 2 Y( fk ) − µY ( fk ; θ) ,

(5.10)

k

where µY can be varied to give different criteria. LS A least squares criterion is given with µY ( fk ; θ) =

X

s(mT; θ)e−i2π fk mT ,

m

i.e., simply the DTFT of the sequence s(mT). BCLS Bias compensation is achieved when µY is given by the mean value of Y( f ), µY ( fk ; θ) =

X

E[s(mT + τm ); θ]e−i2π fk mT ,

m

which is given explicitly in the paper. AsML The criterion is also extended in the paper to give an asymptotic Maximum Likelihood estimate, using also the covariance of Y( f ).

62

5


Table 5.1: Replica of Table B.2. Results for identification of a second order system with known sampling pdf. Mean value and standard deviation of θˆ = (kˆ 0 zˆ pˆr pî )T for the three estimates LS, BCLS and AsML, as well as the true value, θ0 , are shown. k0 6.25 5.8468 0.1723 6.2536 0.0479 6.2499 0.0480

θ0 E[θˆ LS ] Stdev[θˆ LS ] E[θˆ BCLS ] Stdev[θˆ BCLS ] E[θˆ AsML ] Stdev[θˆ AsML ]

pr −1.5 −1.5876 0.0448 −1.4981 0.0385 −1.5008 0.0343

pj 2 1.9508 0.0045 2.0019 0.0415 2.0000 0.0401

The results for identification of a second order system, gθ (t) = L−1 (Gθ (s)), parameterized as Gθ (s) =

k0 , (s − (pr + ipi ))(s − (pr − ipi ))

θ = ( k0

pr

pi )T ,

(5.11a) (5.11b)

are given in Table 5.1. The sampling is done with T = 2 and τm is taken from a zero-mean rectangular distribution covering the whole sampling interval. The main advantage is that a bias error is removed when estimating based on the mean value of Y( f ) instead of the DTFT of s(mT). Figure 5.2 shows the true system Gθ0 (i2π f ) together with Gθ (i2π f ) using θ = E[θˆ LS ] from the estimation without bias compensation. Using θ = E[θˆ BCLS ] or θ = E[θˆ AsML ] does not give a visible difference from the true system.

5.1.3

Analysis of Down-Sampling

When down-sampling is studied, we consider ARS, tm = tm−1 + τm ,

(5.12)

ym = s(tm ) + em

(5.13)

and the aim is to uniformly resample and filter the signal, s(t), at a lower rate, T E[τm ], given a low-pass filter h(t). Optimally, this would give Z z(nT) = h(nT − t)s(t) dt, (5.14) and we analyze three different ways of realizing this equation when ym is given. It is shown that interpolation of the convolution integral is an appropriate procedure, both in terms of performance and complexity. In the analysis, results from

5.2

63

Future Work

magnitude |G(i2π f)|

0 10

−0.1 10

−0.2 10

−0.3 10 −2 10

−1 10 frequency, f [Hz]

Figure 5.2: (Replica of Figure B.5). A comparison of |Gθ (i2π f )| (thin line), given by (5.11) and θ = E[θˆ LS ] from Table 5.1, with the true |Gθ0 (i2π f 0)| (thick line). When using θˆ BCLS or θˆ AsML the difference from the true system is not visible. Paper A show that this implementation corresponds to computing Z zˆ (nT) =

˜ h(nT − t)s(t) dt,

(5.15)

˜ ˜ f )) is given by the original filter h(t) and the non-uniform where h(t) = F −1 (H( sampling properties, such that ˜ f) = H(

M X

τm h(tm )e−i2π f tm ,

(5.16)

m=1

recognized as the Riemann transform of h(tm ). For all cases, this results in a ˜ f )] , H( f ). However, for smooth filters, h(t), with bias, zˆ (nT) , z(nT), since E[H( compact support ˜ f )] → H( f ), E[H(

M → ∞.

(5.17)

˜ f )] is shown For four different rectangular distributions on τm , the resulting E[H( in Figure 5.3, where H( f ) is a second order Butterworth filter.

5.2

Future Work

There is never enough time before a deadline. Therefore, here are some more or less elaborated ideas that this work could continue with.

64

5


1

0.97

0.94

0.91

0.88

Re(0.1,0.3) Re(0.3,0.5) Re(0.4,0.6) Re(0.2,0.6) H(f)

0.85 −2 10


Figure 5.3: Replica of Figure C.5. The true H( f ) (thick line) compared to ˜ f )]| (5.16), when H( f ) is a second order butterworth filter, for four | E[H( different rectangular sampling noise distributions, and M = 250.

5.2.1

Frequency Domain Analysis

The last part of Paper A partly depends on the likely statement in (A.37), where it is argued that the transforms, for example in (5.1), have asymptotic complex Gaussian distribution. This statement would benefit from a rigorous proof using an existing or new central limit theorem. The setup is as follows: If Y( f ) =

M X

s(tm )e−i2π f tm

(5.18)

m=1

P where tm = m k=1 τk and τm are independent identically distributed random variables, what can then be said about the probability distribution of Y( f ) when M approaches infinity? Discussions with Professor Timo Koski, Mathematical statistics, Linköping University, made us believe that existing theorems need to be refined for this particular case, and probably will result in requirements on the underlying function s(t). It is likely that James Davison: Stochastic Limit Theory. Advanced Texts in Econometrics. Oxford University Press 1994, ISBN 0-19-877402-8, can be of some help here.

5.2.2

Identification

Missing data identification is the case when a uniform sampling sequence is distorted with missing samples. Modeling this as an ARS sequence tm = tm−1 + τm ,

(5.19)

5.2

65

Future Work

with τm ∈ [T, 2T, . . .] being discrete sampling noise, can enable us to use the same setup as in Paper B for frequency domain identification. This would give identification of the continuous-time system, when only the pdf of the missing samples is known and not their exact location. The frequency domain identification requires the whole measurement sequence to be recorded before identification is started. An analysis of possibilities to implement this recursively and also of the error that this would give, would be beneficial. Implementing a recursive update of the Dirichlet transform could be done as Ym ( f ) =

m X

yk e−i2π f tk = Ym−1 ( f ) + ym e−i2π f tm ,

(5.20)

k=1

which is quite straightforward. Using Ym ( f ) for identification in each step then requires calculation of E[Ym ( f )] for different parameter settings and also a recursive update of the parameter estimate. The main problem here is probably computation of this mean value in each time step, since this turned out to be fairly complex in Paper B. In all the different identification cases, it is interesting to see how good the estimate can get at best. Therefore, calculating the Cramér-Rao Lower Bound, for the identified parameters, would be interesting.

5.2.3

Other Aspects

Both for recursive implementation, but also for normal frequency analysis, efficient implementation of algorithms is crucial. Ideas for fast implementation of the sum X gm e−i2π f tm (5.21) m

exist in literature, and could be useful for the results in both Paper A and Paper C. For the identification, the problems probably need to be re-formulated to fit in an optimization framework, and then solved using existing solvers. The main obstacle is the mean value and covariance, since they are constructed using integrals. Approximations using numerical analysis may be needed to speed up this part. In that case, analysis of error propagation to the identified parameters is also needed. Given the large amount of applications where non-uniform sampling occurs, it is of course of great interest to test and modify theories for specific cases. Also, this would most probably give more ideas for future research.

66

5


Bibliography

Adolfsson, Klas (2003). TCP performance in an EGPRS system. Master’s thesis, Department of Electrical Engineering, Linköpings universitet, Sweden. LiTHISY-EX-3163. URL http://www.ep.liu.se Alavi, Hossein and Fadaei, Mehdi (1994). Frequency estimation of low rate nonuniformly sampled signals using neural networks. In IEEE Singapore International Conference on Communication Systems ICCS, vol. 1, (pp. 229–233). Singapore. Altman, Eitan; Ba¸sar, Tamer; and Hovakimian, Naira (2000). Worst-case ratebased flow control with an ARMA model of the available bandwidth. In Annals of Dynamic Games, vol. 6:pp. 3–29. Asfand-E-Yar; Awan, Irfan; and Woodward, Michael E (2004). Stochastic analysis of Internet traffic congestion control using active queue management (AQM) scheme. In Performance Modelling and Evaluation of Heterogenous Networks, HET-NETs. West Yorkshire, England. URL http://www.comp.brad.ac.uk/het-net/HET-NETs04/ Balakrishnan, Hari; Padmanabhan, Venkata N; Seshan, Srinivasan; Stemm, M; and Katz, Randy H (1998). TCP behavior of a busy Internet server: analysis and improvements. In Joint Conference of the IEEE Computer and Communications Societies, INFOCOM, vol. 1, (pp. 256–262). San Fransisco, California, USA. Belmont, Michael R; Horwood, J Mike K; and Thurley, Richard WF (2003). Nonuniform sampling issues arising in shallow angle wave profiling LIDAR. In IEEE/OES Seventh Working Conference on Current Measurement Technology, (pp. 135–139). San Diego, California, USA. 67

68

Bibliography

Benedetto, John J (1992). Irregular sampling and frames. In Chui, Charles K (Ed.), Wavelets: A Tutorial in Theory and Practice. Academic Press Inc. Benedetto, John J and Ferreira, Paulo JSG (Eds.) (2001). Modern Sampling Theory. Birkhäuser Verlag. Beutler, Frederick J (1966). Error-free recovery of signals from irregularly spaced samples. In SIAM Review, vol. 8(3):pp. 328–335. Beutler, Frederick J (1970). Alias-free randomly timed sampling of stochastic processes. In IEEE Transactions on Information Theory, vol. 16(2):pp. 147–152. ISSN 0018-9448. Bilinskis, Ivar and Mikelsons, Arnolds (1992). Randomized Signal Processing. Prentice Hall. Björsson, Anders (2003). Simulation analysis of RLC/MAC for UMTS in Network Simulator version 2. Master’s thesis, Department of Electrical Engineering, Linköpings universitet, Sweden. LiTH-ISY-EX-3399. URL http://www.ep.liu.se Bland, Denise M and Tarczynski, Andrzej (1997). Optimum nonuniform sampling sequence for alias frequency suppression. In IEEE International Symposium on Circuits and Systems. Hong Kong. Brakmo, Lawrence S and Peterson, Larry L (1995). TCP Vegas: end to end congestion avoidance on a global Internet. In IEEE Journal on Selected Areas in Communications, vol. 13:pp. 1465–1480. Brunberg, Henrik (2002). Distributing digital radio over an IP backbone. Master’s thesis, Department of Electrical Engineering, Linköpings universitet, Sweden. LiTH-ISY-EX-3317. URL http://www.ep.liu.se Chandran, Kartik; Raghunathan, Sundarshan; Venkatesan, Subbarayan; and Prakash, Ravi (2001). A feedback-based scheme for improving TCP performance in ad hoc wireless networks. In IEEE Personal Communications, vol. 8(1):pp. 34–39. ISSN 1070-9916. Deb, Supratim and Srikant, R (2006). Rate-based versus queue-based models of congestion control. In IEEE Transactions on Automatic Control, vol. 51(4):pp. 606–619. ISSN 0018-9286. Deng, Xidong; Yi, Sungwon; Kesidis, George; and Das, Chita R (2002). Stabilized virtual buffer (SVB) - an active queue management scheme for Internet qualityof-service. In IEEE Global Telecommunications Conference, Globecom, vol. 2, (pp. 1628–1632). Elbornsson, Jonas (2003). Analysis, Estimation and Compensation of Mismatch Effects in A/D Converters. Ph.D. thesis, Linköpings universitet, Sweden. Thesis No. 811. URL http://www.control.isy.liu.se/

Bibliography

69

Eldar, Yonina C (2003). Sampling with arbitrary sampling and reconstruction spaces and oblique dual frame vectors. In Journal of Fourier Analysis and Applications, vol. 9(1):pp. 77–96. Eng, Frida and Gustafsson, Fredrik (2005a). Frequency transforms based on nonuniform sampling — basic stochastic properties. In Radiovetenskap och Kommunikation. Linköping, Sweden. Eng, Frida and Gustafsson, Fredrik (2005b). System identification using measurements subject to stochastic time jitter. In IFAC World Congress. Prague, Czech Republic. Eng, Frida and Gustafsson, Fredrik (2006). Bias compensated least squares estimation of continuous time output error models in the case of stochastic sampling time jitter. In IFAC Symposium on System Identification (SYSID). Newcastle, Australia. Eng, Frida and Gustafsson, Fredrik (2007a). Algorithms for down-sampling non-uniformly sampled data. In European Signal Processing Conference (EUSIPCO). Poznan, ´ Poland. Submitted. Eng, Frida and Gustafsson, Fredrik (2007b). Down-sampling non-uniformly sampled data. In EURASIP Journal of Advances in Signal Processing. Submitted. Eng, Frida and Gustafsson, Fredrik (2007c). Identification with stochastic sampling time jitter. In Automatica. Provisionally accepted as regular paper. Eng, Frida; Gustafsson, Fredrik; and Gunnarsson, Fredrik (2007). Frequency domain analysis of signals with stochastic sampling times. In IEEE Transactions on Signal Processing. Submitted. Eriksson, Jonas (2004). Providing Quality of Service for Streaming Applications in Evolved 3G Networks. Master’s thesis, Department of Electrical Engineering, Linköpings universitet, Sweden. LiTH-ISY-EX-3493. URL http://www.ep.liu.se Fall, Kevin and Varadhan, Kannan (Eds.) (). The ns manual. URL http://nsnam.isi.edu/nsnam/ Feichtinger, Hans G and Gröchenig, Karlheinz (1994). Theory and practice of irregular sampling. In Benedetto, John J and Frazier, Michael W (Eds.), Wavelets: Mathematics and Applications. CRC Press. Feichtinger, Hans G; Gröchenig, Karlheinz; and Strohmer, Thomas (1995). Efficient numerical methods in non-uniform sampling theory. In Numerische Matematik, vol. 69(4):pp. 423–440. Flachenecker, Peter; Hartung, Hans-Peter; and Reiners, Karlheinz (1997). Power spectrum analysis of heart rate variability in Guillain-Barre syndrome. a longitudinal study. In Brain, vol. 120(10):pp. 1885—1894.

70

Bibliography

Floyd, Sally (1994). TCP and explicit congestion notification. In ACM Computer Communication Review, vol. 24(5):pp. 10–23. Floyd, Sally and Jacobson, Van (1993). Random early detection gateways for congestion avoidance. In IEEE/ACM Transactions on Networking, vol. 1(4):pp. 397–413. Fredholm, Kenth and Nilsson, Kristian (2003). Implementing an application for communication and quality measurements over UMTS network. Master’s thesis, Department of Electrical Engineering, Linköpings universitet, Sweden. LiTH-ISY-EX-3369. URL http://www.ep.liu.se Grönwall, Christina (2006). Ground Object Recognition using Laser Radar Data - Geometric Fitting, Performance Analysis, and Applications. Ph.D. thesis, Linköpings universitet, Sweden. Thesis No. 1055. URL http://www.control.isy.liu.se/ Gunnarsson, Frida (2000). Problems and Improvements of Internet Traffic Congestion Control. Master’s thesis, Department of Electrical Engineering Linköpings universitet, Sweden. LiTH-ISY-EX-3098. Gunnarsson, Frida (2003). On modeling and control of network queue dynamics. Licentiate Thesis No. 1048, Department of Electrical Engineering, Linköpings universitet, Sweden. Gunnarsson, Frida; Gunnarsson, Fredrik; and Gustafsson, Fredrik (2001). TCP performance based on queue occupation. In Towards 4G and Future Mobile Internet, Proceedings for Nordic Radio Symposium 2001. Nynäshamn, Sweden. Gunnarsson, Frida; Gunnarsson, Fredrik; and Gustafsson, Fredrik (2002). Issues on performance measurements of TCP. In Radiovetenskap och Kommunikation. Stockholm, Sweden. Gunnarsson, Frida; Gunnarsson, Fredrik; and Gustafsson, Fredrik (2003). Controlling Internet queue dynamics using recursively identified models. In IEEE Conference on Decision and Control. Maui, Hawaii, USA. Gunnarsson, Frida; Gustafsson, Fredrik; and Gunnarsson, Fredrik (2004). Frequency analysis using nonuniform sampling with applications to adaptive queue management. In IEEE International Conference on Acoustics, Speech, and Signal Processing. Montreal, Canada. Hertz, Paul L. and Feigelson, Eric D. (1997). A sample of astronomical time series. In Subba Rao, T.; Priestley, M. B.; and Lessi, O. (Eds.), Applications of Time Series Analysis in Astronomy and Meteorology. Chapman & Hall. ISBN 0-41263800-2. URL http://xweb.nrl.navy.mil/timeseries/timeseries.html

Bibliography

71

Hollot, C. V.; Misra, Vishal; Towsley, Don; and Gong, Wei-Bo (2001). A control theoretic analysis of RED. In Joint Conference of the IEEE Computer and Communications Societies, INFOCOM. Anchorage, Alaska, USA. Jacobsson, Krister and Hjalmarsson, Håkan (2004). Estimation problems in congestion control. In On the State of the Art in Controlled Bandwidth Sharing. Euro-NGI. URL http://eurongi.enst.fr/archive/127/jra21_deliverable_final.pdf Jacobsson, Krister and Hjalmarsson, Håkan (2006). Closed loop aspects of fluid flow model identification in congestion control. In IFAC Symposium on System Identification (SYSID). Newcastle, Australia. Jacobsson, Krister; Hjalmarsson, Håkan; and Möller, Niels (2006). ACK-clock dynamics in network congestion control – an inner feedback loop with implications on inelastic flow impact. In IEEE Conference on Decision and Control. San Diego, San Fransisco, USA. Jacobsson, Krister; Möller, Niels; Johansson, Karl-Henrik; and Hjalmarsson, Håkan (2004). Some modeling and estimation issues in control of heterogeneous networks. In International Symposium on Mathematical Theory of Networks and Systems, MTNS. Leuven, Belgium. Jacod, Jean (1993). Random sampling in estimation problems for continuous Gaussian processes with independent increments. In Stochastic Processes and their Applications, vol. 44(2):pp. 181–204. ISSN 0304-4149. Jain, Raj (1991). The art of computer systems performance analysis: Techniques for experimental design, measurement, simulation and modeling. In ACM SIGMETRICS Performance Evaluation Review, vol. 19(2):pp. 5–11. Janik, Jean-Marie and Bloyet, Daniel (2004). Timing uncertainties of A/D converters: theoretical study and experiments. In IEEE Transactions on Instrumentation and Measurement, vol. 53(2):pp. 561–565. ISSN 0018-9456. Kelly, Frank (2003). Fairness and stability of end-to-end congestion control. In European Journal of Control, vol. 9:pp. 159–176. Keshav, Srinivasan (1997). An Engineering Approach to Computer Networking. Addison Wesley. Description of the REAL simulator. URL http://www.cs.cornell.edu/skeshav/real/ Kilkki, Kalevi (1999). Differentiated Services for the Internet. Macmillan Technical Publishing. ISBN 1-57870-132-5. Knutsson, Björn (2004). Simulation of radio resource management for UMTS. Master’s thesis, Department of Electrical Engineering, Linköpings universitet, Sweden. LiTH-ISY-EX-3414. URL http://www.ep.liu.se

72

Bibliography

Kunniyur, Srisankar and Srikant, R (2004). Analysis and design of an adaptive virtual queue algorithm for active queue management. In IEEE/ACM Transactions on Networking, vol. 12(2):pp. 286–299. Kybic, Jan; Blu, Thierry; and Unser, Michael (2002a). Generalized sampling: a variational approach. I. Theory. In IEEE Transactions on Signal Processing, vol. 50(8):pp. 1965–1976. ISSN 1053-587X. Kybic, Jan; Blu, Thierry; and Unser, Michael (2002b). Generalized sampling: a variational approach. II. Applications. In IEEE Transactions on Signal Processing, vol. 50(8):pp. 1977–1985. Leiner, Barry M; Cerf, Vinton G; Clark, David D; Kahn, Robert E; Kleinrock, Leonard; Lynch, Daniel C; Postel, Jon; Roberts, Larry G; and Wolff, Stephen (2000). A brief history of the Internet. URL www.isoc.org/internet/history/brief.shtml Liu, Jian and Singh, Suresh (2001). ATCP: TCP for mobile ad hoc networks. In IEEE Journal on Selected Areas in Communications, vol. 19(7):pp. 1300–1315. ISSN 0733-8716. Ljung, Lennart (1999). System Identification – Theory for the User. Prentice Hall, 2 ed. Lomb, N R (1976). Least-squares frequency analysis of unequally spaced data. In Astrophysics and Space Science, vol. 39:pp. 447–462. Low, Steven H; Paganini, Fernando; and Doyle, John C (2002). Internet congestion control. In IEEE Control Systems Magazine, vol. 22(1):pp. 28–43. ISSN 02721708. Marvasti, Farokh (1987). Zero-Crossings and Nonuniform Sampling of Single and Multidimensional Signals and Systems. Nonuniform Publications. Marvasti, Farokh (1996). Nonuniform sampling theorem for bandpass signals at or below the Nyquist density. In IEEE Transactions on Signal Processing, vol. 44(3):pp. 572–576. ISSN 1053-587X. Marvasti, Farokh (Ed.) (2001). Nonuniform Sampling: Theory and Practice. Kluwer Academic Publishers. Marvasti, Faroukh; Analoui, Mostafa; and Gamshadzahi, Mohsen (1991). Recovery of signals from nonuniform samples using iterative methods. In IEEE Transactions on Signal Processing, vol. 39(4):pp. 872–878. Mascolo, Saverio (1999). Congestion control in high-speed communication networks using the Smith principle. In Automatica, vol. 35(12). Mascolo, Saverio; Casetti, Claudio; Gerla, Mario; Sanadidi, M. Y.; and Wang, Ren (2001). TCP westwood: Bandwidth estimation for enhanced transport over wireless links. In International Conference on Mobile Computing and Networking, (pp. 287 – 297). Rome, Italy.

Bibliography

73

Masry, Elias (1978a). Alias-free sampling: An alternative conceptualization and its applications. In IEEE Transactions on Information Theory, vol. 24(3):pp. 317–324. Masry, Elias (1978b). Poisson sampling and spectral estimation of continuous-time processes. In IEEE Transactions on Information Theory, vol. 24(2):pp. 173–183. Masry, Elias and Cambanis, Stamatis (1981). Consistent estimation of continuoustime signals from nonlinear transformations of noisy samples. In IEEE Transactions on Information Theory, vol. 27(1):pp. 84–96. ISSN 0018-9448. Masry, Elias; Klamer, Dale; and Mirabile, Charles (1978). Spectral estimation of continuous-time processes: Performance comparison between periodic and Poisson sampling schemes. In IEEE Transactions on Automatic Control, vol. 23(4):pp. 679–685. ISSN 0018-9286. Masry, Elias and Lui, Ming-Chuan C (1976). Discrete-time spectral estimation of continuous-parameter processes – a new consistent estimate. In IEEE Transactions on Information Theory, vol. 22(3):pp. 298–312. Mednieks, Ints (1999). Methods for spectral analysis of nonuniformly sampled signals. In International Workshop on Sampling Theory and Application. Loen, Norway. Meyer, Michael (1999). TCP performance over GPRS. In IEEE Wireless Communications and Networking Conference. New Orleans, Louisiana, USA. Misra, Archan; Baras, John S; and Ott, Tennis (1999). The window distribution of multiple TCP’s with random loss queues. In IEEE Global Telecommunications Conference, Globecom, vol. 3, (pp. 1714–1726). Rio de Janeiro, Brazil. Nilsson, Martin (2007). Estimation of Radial Runout. Master’s thesis, Department of Electrical Engineering, Linköpings universitet, Sweden. LiTH-ISY-EX– 07/3949–SE. URL http://www.ep.liu.se NS (2003). Ucb/lbnl/vint network simulator, ns-2. URL http://nsnam.isi.edu/nsnam/ OPNET (2007). OPNET Technologies Co., Ltd. URL http://www.opnet.com van der Ouderaa, Edwin and Renneboog, Jean (1988). Some formulas and applications of nonuniform sampling of bandwidth-limited signals. In IEEE Transactions on Instrumentation and Measurement, vol. 37(3):pp. 353–357. ISSN 0018-9456. Padhye, Jitendra; Firoiu, Victor; Towsley, Don; and Kurose, Jim (1998). Modeling TCP throughput: A simple model and its empirical validation. In Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication, ACM SIGCOMM. Vancouver, Canada.

74

Bibliography

Papenfuss, Frank; Artyukh, Yuri; Boole, E; and Timmermann, Dirk (2003). Optimal sampling functions in nonuniform sampling driver designs to overcome the Nyquist limit. In IEEE International Conference on Acoustics, Speech, and Signal Processing. Hong Kong. Papoulis, Athanasios (1977). Signal Analysis. McGraw-Hill. Park, Kyung-Joon; Lim, Hyuk; Ba¸sar, Tamer; and ho Choi, Chong (2003). Antiwindup compensator for active queue management in TCP networks. In Control Engineering Practice, vol. 11(10):pp. 1127–1142. Peisa, Janne and Meyer, Michael (2001). Analytical model for TCP file transfers over UMTS. In IEEE Third Generation Wireless Communications Conference (3G’Wireless). San Francisco, California, USA. Persson, Katarina (2002a). TCP/IP in tactical ad hoc networks. Master’s thesis, Department of Electrical Engineering, Linköpings universitet, Sweden. LiTHISY-EX-3206. URL http://www.ep.liu.se Persson, Niclas (2002b). Event Based Sampling with Application to Spectral Estimation. Licentiate Thesis No. 981, Department of Electrical Engineering, Linköpings universitet, Sweden. URL http://www.control.isy.liu.se/ Potts, Daniel; Steidl, Gabriele; and Tasche, Manfred (2001). Fast Fourier transform for nonequispaced data: A tutorial. In Benedetto and Ferreira (2001), chap. 12. Pribić, Radmila (2004). Radar irregular sampling. In IEEE International Conference on Acoustics, Speech, and Signal Processing. Montreal, Canada. Qi, Yuan; Minka, Thomas; and Picard, Rosalind (2002). Bayesian spectrum estimation of unevenly sampled nonstationary data. In IEEE International Conference on Acoustics, Speech, and Signal Processing. Orlando, Florida, USA. RFC 2474 (1998). Definition of the Differentiated Services Field (DS Field) in the IPv4 and IPv6 Headers. Proposed standard. K. Nichols, S. Blake, F. Baker and D. Black. RFC 2581 (1999). TCP Congestion Control. Proposed standard. M. Allman, V. Paxson and W. Stevens. RFC 3168 (2001). The Addition of Explicit Congestion Notification (ECN) to IP. Proposed standard. K. Ramakrishnan, S. Floyd and D. Black. Romano, Maria; Cesarelli, Mario; Bifulco, Paolo; Sansone, Mario; and Bracale, Marcello (2003). Study of fetal autonomous nervous system’s response by means of FHRV frequency analysis. In International IEEE EMBS Conference on Neural Engineering, (pp. 399–402). Capri Island, Italy.

Bibliography

75

Roques, Sylvie and Thiebaut, Carole (2003). Spectral contents of astronomical unequally spaced time-series: contribution of time-frequency and time-scale analyses. In IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 6, (pp. 473–6). Hong Kong. ISSN 1520-6149. Russell, Andrew I (2002). Regular and Irregular Signal Resampling. Ph.D. thesis, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA. Sachs, Joachim and Meyer, Michael (2001). Mobile Internet - performance issues beyond the radio interface. In Aachen Symposium on Signal Theory (ASST). Aachen, Germany. Sands, Kenneth E F; Appel, Marvin L; Lilly, Leonard S; Schoen, Frederick J; Mudge, Jr, Gilbert H; and Cohen, Richard J (1989). Power spectrum analysis of heart rate variability in human cardiac transplant recipients. In Circulation, vol. 70:pp. 76—82. Scargle, Jeffrey D (1982). Studies in astronomical time series analysis II: Statistical aspects of spectral analysis of unevenly spaced data. In The Astrophysical Journal, vol. 263:pp. 835–853. Shapiro, Harold S and Silverman, Richard A (1960). Alias-free sampling of random noise. In Journal of the Society for Industrial and Applied Mathematics, vol. 8(2):pp. 225–248. Souders, T Michael; Flach, Donald R; Hagwood, Charles; and Yang, Grace L (1990). The effects of timing jitter in sampling systems. In IEEE Transactions on Instrumentation and Measurement, vol. 39(1):pp. 80–85. ISSN 0018-9456. van Steenis, H. G. and Tulen, J. H. M. (1991). A comparison of two methods to calculate the heart rate spectrum based on non-equidistant sampling. In International Conference of the IEEE Engineering in Medicine and Biology Society. Philadelphia, Pennsylvania, USA. Stevens, W Richard (1994). TCP/IP illustrated vol 1: The protocols. Addison Wesley. Tarczynski, Andrzej and Allay, Najib (2004). Spectral analysis of randomly sampled signals: Suppression of aliasing and sampler jitter. In IEEE Transactions on Signal Processing, vol. 52(12):pp. 3324–3334. ISSN 1053-587X. Tronarp, Otto (2003). Quality of Service in Ad Hoc Networks by Priority Queuing. Master’s thesis, Department of Electrical Engineering, Linköpings universitet, Sweden. LiTH-ISY-EX-3343. URL http://www.ep.liu.se Vandewalle, Patrick; Sbaiz, Luciano; Vandewalle, Joos; and Vetterli, Martin (2004). How to take advantage of aliasing in bandlimited signals. In IEEE International Conference on Acoustics, Speech, and Signal Processing. Montreal, Canada.

76

Bibliography

Verbeyst, Frans; Rolain, Yves; Schoukens, Johan; and Pintelon, Rik (2006). System identification approach applied to jitter estimation. In Instrumentation and Measurement Technology Conference, (pp. 1752 – 1757). Sorrento, Italy. ISSN 1091-5281. Wallin, Johan (2005). Catheter-Compensated Blood Pressure Measurement. Master’s thesis, Department of Electrical Engineering, Linköpings universitet, Sweden. LiTH-ISY-EX-3620. URL http://www.ep.liu.se Wikström, Martin (2005). Compensating for Respiratory Artifacts in Blood Pressure Waveforms. Master’s thesis, Department of Electrical Engineering, Linköpings universitet, Sweden. LiTH-ISY-EX-3654. URL http://www.ep.liu.se Wojtkiewicz, Andrzej and Tuszynski, ´ Michal (1992). Application of the Dirichlet transform in analysis of nonuniformly sampled signals. In IEEE International Conference on Acoustics, Speech, and Signal Processing. San Fransisco, California, USA. Yao, Kung and Thomas, John B (1967). On some stability and interpolatory properties of nonuniform sampling expansions. In IEEE Transactions on Circuits and Systems [legacy, pre - 1988], vol. 14(4):pp. 404–408. ISSN 0098-4094. Zakon, Robert Hobbes’ (2003). Hobbes’ Internet timeline v6.0. URL http://www.zakon.org/robert/internet/timeline/ Zhao, Haichuan and Wu, Jian Qiu (2005). Implementation and simulation of HSDPA functionality with ns-2. Master’s thesis, Department of Electrical Engineering, Linköpings universitet, Sweden. LiTH-ISY-EX-3647. URL http://www.ep.liu.se Zhu, Chengyu; Yang, Oliver W W; Aweya, Lames; Ouellette, Michel; and Montuno, Delfin Y (2002). A comparison of active queue management algorithms using the OPNET modeler. In IEEE Communications Magazine, vol. 40(6):pp. 158–167. ISSN 0163-6804.

Part II

Included Papers

77

A Frequency Domain Analysis of Signals with Stochastic Sampling Times Frida Eng, Fredrik Gustafsson and Fredrik Gunnarsson Edited version submitted for review to IEEE Transactions on Signal Processing.

Preliminary version published as Technical Report No LiTH-ISY-R-2772, Dept. of Electrical Engineering, Linköpings universitet. Download from http://www.control.isy.liu.se/publications/

79

1

81

Introduction

Frequency Domain Analysis of Signals with Stochastic Sampling Times Frida Eng, Fredrik Gustafsson and Fredrik Gunnarsson Dept. of Electrical Engineering, Linköpings universitet SE-581 83 Linköping, Sweden Email: [email protected] Abstract In non-uniform sampling (NUS), signal amplitudes and time stamps are delivered in pairs. Several methods to compute an approximate Fourier transform (AFT) have appeared in literature, and their posterior properties in terms of alias suppression and leakage have been addressed. In this paper, the sampling times are assumed to be generated by a stochastic process. The main result gives the prior distribution of several AFTs expressed in terms of the true Fourier transform and variants of the characteristic function of the sampling time distribution. The result extends leakage and alias suppression with bias and variance terms due to NUS. Specific sampling processes as described in literature are analyzed in detail. The results are illustrated on simulated signals, with particular focus to the implications for spectral estimation.

1

Introduction

Non-uniform sampling (NUS) occurs in many applications where uniform sampling is either not possible or practically not achievable: Event based sampling in the time domain In queue processes, the signal will change level for each arrival and departure, which occur naturally on a nonuniform time grid. One example is routers in data networks, where data packets come and go and the queue length is a central control parameter. Here, it would not make sense to sample the queue length uniformly. Another example is analysis of astronomical data, where for instance the signal may be the number of received neutrinos. Missing data is another kind of event process, where the sampling times can be modeled as stochastic. Uniform sampling in other domains Angular speeds are often computed from regular angular displacement. One example is angular speed sensors, e.g., ABS in cars, that give a certain number of pulses for each wheel revolution. The samples are uniform in the angle domain, but non-uniform in the time

82

A Frequency Domain Analysis of Signals with Stochastic Sampling Times

domain. Another example is the heart rate spectrum, which is computed from the peak times of an ECG signal. Again, the sampling is uniform in the heart rate domain, but non-uniform in the time domain. User chosen non-uniform grid In some applications, a non-uniform time grid is desirable, e.g., to avoid detection in military radar applications. Also the time grid can be chosen to avoid aliasing or increase other performance parameters. In all cases, a posteriori analysis can be based on interpolating the signal to a uniform time grid and use classical Fourier transform (FT) based approaches to frequency, spectral or model-based analysis. The interpolation error is in Schoukens et al. (1992) shown to be quite small if the interpolation is done correctly. There are also several methods to compute an approximate FT (AFT) (Beutler, 1970; Benedetto, 1992; Feichtinger and Gröchenig, 1994; Beurling, 1989). Posterior analysis of AFTs in terms of alias suppression and leakage is a fairly well explored area (Lomb, 1976; Scargle, 1982; Wojtkiewicz and Tuszynski, ´ 1992). The reverse problem of designing NUS to maximize alias suppression is also well studied (Papenfuss et al., 2003; Bland and Tarczynski, 1997; Shapiro and Silverman, 1960; Masry, 1978, 1983; Hall and Yin, 2004). This article addresses the problem of a priori analysis of the stochastic properties of certain natural Fourier transform approximations. More specifically, we derive the asymptotic distribution for the approximate Fourier transform (AFT) in terms of the true signal’s Fourier transform, and a stochastic model of the sampling times, before the pairwise measurements of the signal and time are performed. This is done for some important cases of sampling time noise. In this way, one can compute an a priori confidence region for each frequency in the AFT. Once the signal and time stamps are observed, we get one observation of the AFT (from the distribution we have derived), that will lie in this confidence region with a certain probability. Focus is on frequency resolution, alias suppression and stochastic uncertainty in the AFT. Given the results, it is possible to evaluate intricate design questions for the applications mentioned above, and some examples include: How well can the eigen-frequencies of a car tire be determined, given that the sampling times are perturbed by the car speed and road roughness? Is it possible to find the oscillations in a router in the Internet, to enable improvements of its control protocols? Given that neutrinos arrive according to a Poisson process, how many arrivals are needed to find the dominating frequencies with a certain accuracy? How does the outlier probability influence the bias in spectral estimates? In Section 2, the signal model and stochastic frequency model are defined, and different non-uniform sampling schemes are described. A discussion of interpolation techniques for approximating the true transform is done in Section 3. The main part of the work is the analysis of the stochastic properties of two linear frequency transforms, with the main results in Section 4, together with extensions for the periodogram. In Section 5, asymptotic analysis of the mean value and the distribution is done, which further increase the knowledge about the AFT and the periodogram. Finally, Section 6 gives examples of both the AFT and the

2

83

Problem Definition

Table A.1: Notation and relevant references. s(t) S( f ) tk pk (t) τk pτ (t) ϕτ ( f ) Tk yk wk (·) Y(·) W(·) an1 g( · |t) f ( · ; θ)

continuous-time signal true Fourier transform of signal s(t) non-uniform stochastic sampling time probability density function (pdf) for tk sampling noise, for example, tk = kT + τk pdf for the sampling noise, τ characteristic function for τ inter-sampling times, tk − tk−1 , t0 = 0 stochastic sample of s(t) at time tk stochastic frequency weight stochastic frequency transform of yk stochastic frequency window the sequence ak , k = 1, .., n. deterministic value of the stochastic function g(·) conditioned on t the function f (·) has implicit dependence on θ

(A.1) (A.1) (A.3) (A.2) Section 2.3 Section 4 (A.3) (A.4) & (A.19) (A.7) (A.25)

periodogram. The article is concluded in Section 7.

2

Problem Definition

We start by describing the signal model, and clarifying the notation. This leads to a description of the stochastic frequency model. Different stochastic sampling models are listed at the end. The notation adopted in this work is summarized in Table A.1, together with relevant references to descriptions or definitions in the text.

2.1

Signal Model

We consider a deterministic continuous-time signal, s(t), with Fourier transform S( f ), Z∞ s(t) =

S( f )ei2π f t d f,

(A.1)

−∞

which is non-uniformly sampled N times. The sample times, tk , are stochastic variables with probability density functions (pdfs), ( pk (t) =

d dt P(tk P(tk

≤ t) = t)

continuous density, discrete density,

(A.2)

84


for k = 1, . . . , N, and the number of samples, N, is deterministic. In the following, we consider only continuous density functions, pk (t). From the continuous-time signal (or function) s(t) we get a stochastic observation, yk , of the corresponding deterministic sample value: yk = s(tk ).

(A.3)

Thus, yk is a function of the stochastic variable tk and its stochastic properties can be investigated accordingly. For example, its mean is Z∞ E[yk ] = E[s(tk )] =

s(t)pk (t) dt. −∞

Any transform of these stochastic function values or measurements, yk , can be investigated to study its a priori properties, for example, for parameter estimation or frequency analysis. In this work, we consider the case of frequency analysis of a continuous-time signal, s(t), sampled non-uniformly and then transformed to approximate the true Fourier transform, S( f ). Sampling of stochastic functions will not be considered here. However, it will be clear that additive measurement noise easily can be included in the descriptions, since only linear transformations of the measurements are of interest. By adding zero-mean measurement noise, ek , to the signal sample we, for example, get E[s(tk ) + ek ] = E[s(tk )].

2.2

Stochastic Frequency Model

The transforms of interest in this work are linear transforms of the measurements, that can be calculated when the measurement times are given, i.e., Y( f |tN 1)

=

N X

yk wk ( f |tk ),

(A.4)

k=1

where Y( f |tN ) is the deterministic approximation, AFT, of the true transform, S( f ). 1 Here, wk ( f |tN ) are weights to be decided, see the coming discussion in Section 3, 1 and tN denotes the sequence of the sampling times, 1 N tN 1 = (tk )k=1 .

(A.5)

For uniform sampling, tk = kT, the discrete time Fourier transform (DTFT) is commonly used Ydtft ( f ) = T

N X

s(kT)e−i2π f kT ,

(A.6)

k=1

i.e., it corresponds to weighting with wk ( f |kT) = Te−i2π f kT . The literature contains definitions both with and without the scaling factor T. The well-known discrete Fourier transform (DFT) is the DTFT evaluated at discrete frequency points, normally without the scaling.

2

85

Problem Definition

Using Bayes’ rule we can derive the pdf for the random variable Y( f ), denoted pY( f ) (y), by marginalization of the stochastic sampling times, Z∞ pY( f ) (y) =

N pY( f |tN ) (y)p(tN 1 ) dt1 , 1

(A.7)

−∞

where the joint pdf of the sampling times, p(tN ), will depend on the type of 1 sampling process. We also note that, since Y( f |tN ) is deterministic, pY( f |tN ) (y) = 1 1 N δ(y − Y( f |t1 )). This gives, with (A.7), the a priori distribution of Y( f ), Z N N pY( f ) (y) = δ(y − Y( f |tN (A.8) 1 ))p(t1 ) dt1 , which would require a change of variables in order to be solved. The a priori distribution of Y( f ) enables investigation of, for example, the mean value and covariance achieved for different sampling processes, expected leakage and aliasing. A quick investigation shows that this expression is mainly for theoretical discussions, since it is virtually impossible to evaluate, so we will use other means to explicitly derive mean value and covariance for the stochastic AFT.

2.3


Non-uniform sampling can occur in different forms and this section lists the most common sampling models. The probability distribution for the sampling times, tk , was given as pk (t) in (A.2). Depending on the type of sampling, pk (t) can be deduced from the probability density function, pτ (t) of the sampling noise, τk . For additive random sampling (ARS), the sampling times are constructed by adding the sampling noise to the previous sampling time, tk = tk−1 + τk =

k X

τn ,

t0 = 0,

(A.9)

n=1

where τk ∈ (0, ∞) are i.i.d. stochastic noise and E[τk ] = T. This means that E[tk ] = kT, while the variance increases with k. The pdf is given as a convolution of the sampling noise pdf k times, (k)

pk (t) = pτ ? . . . ? pτ (t) , pτ (t). | {z }

(A.10)

k times

For example, the exponential distribution, pτ (t) = 1/Te−t/T , gives a Poisson sampling process. The central limit theorem gives that pk (t) will approach a Gaussian distribution when k goes to infinity, since it is the pdf of a sum of k independent identically distributed variables, c.f., (A.9). For stochastic jitter sampling (SJS), the sampling noise is added to the expected sampling time, tk = kT + τk ,

(A.11)

86


with τk ∈ [−T/2, T/2] being i.i.d. stochastic noise and E[τk ] = 0. In this case the variance is constant over time and the pdf is given directly by the pdf for τ, pk (t) = pτ (t − kT).

(A.12)

One natural distribution is the rectangular distribution, 1 , t ∈ [−T/2, T/2], T or more generally the beta distribution pτ (t) =

(A.13)

1 tα−1 (1 − t)β−1 , t ∈ [0, 1] (A.14a) B(α, β) Γ(α)Γ(β) B(α, β) = , α, β > 0, (A.14b) Γ(α + β) R where Γ(q) = tq−1 e−t dt is the gamma-function1 . The beta distribution is the rectangular distribution when α = β = 1. Another special case is the problem with missing data (MD), where the underlying sampling procedure is uniform but sometimes samples are missed. This can, in the current context, be described with a discrete sampling noise, pτ (t) =

tk = tk−1 + τk ,

(A.15)

and τk ∈ {T, 2T, . . .}. This can be seen as a special case of ARS, with a discrete pdf for the sampling noise, for example, pτ (nT) = P(τk = nT) = p(1−p)n−1 gives a “First success” distribution. The expected value E[tk ] > kT whenever E[τk ] > T, which is the case for every nontrivial pdf. The missing data problem is also discussed in Ljung (1999, Ch. 14) with further references. In Table A.2, these sampling types are summarized with mean value of the sampling times, support of the sampling noise and the update equation for the sampling times and the pdf. Table A.2: Summary of different sampling models, additive random sampling (ARS), stochastic jitter sampling (SJS) and missing data (MD). Type ARS SJS MD

3

Update tk = tk−1 + τk tk = kT + τk tk = tk−1 + τk

E[tk ] kT kT > kT

τk ∈ (0, ∞) (−T/2, T/2) {T, 2T, . . .}

pk (t) (k) pτ (t) pτ (t − kT) —

Approximation of the Fourier Transform

Non-uniform sampling, both deterministic and stochastic, is described in Bilinskis and Mikelsons (1992) and Marvasti (2001). Among other things, the literature describes how the Fourier transform (FT) can be approximated when the sampling times are deterministic. 1 Γ(m)

= (m − 1)! for integer values, m > 0.

3

87

Approximation of the Fourier Transform

3.1

Two Transform Expressions

As stated earlier, we will investigate the use of linear transforms of the form given in (A.4). Our interest is focused on approximating the continuous-time Fourier transform Z∞ S( f ) =

Z∞ −i2π f t

s(t)e −∞

dt =

I(t) dt,

(A.16)

−∞

with such a transformation, Y( f |tN ), of the discrete-time measurements. These 1 approximations are done based on knowledge of the sample times, tk , and actual measurements, yk = s(tk ), and can then be used to design the transform, via (A.7), of the stochastic sample values that we aim for. Using zero order spline interpolation of the integrand I(t), corresponds to Riemann integration and yields Y( f |tN 1)=

N X

Tk yk e−i2π f tk ,

(A.17)

k=1

called the Riemann transform (RieT). In several contexts, the Dirichlet transform (DirT) has been proposed as a suitable frequency transform for non-uniformly sampled signals (Bilinskis and Mikelsons, 1992; Tarczynski and Allay, 2004). It can be deduced, for example, by approximating the signal s(t) with an impulse train. The Dirichlet transform is given as Y( f |tN 1)=

N X

yk e−i2π f tk .

(A.18)

k=1

Except for the scaling factor T, both expressions coincide with the DTFT (A.6) when tk = kT.

3.2

Related Work

There are other numerical integration methods, for example, Runge-Kutta integration and Newton-Cotes formulas, that could be used for transforms similar to RieT. In Bland et al. (1996), several integration methods are also investigated for non-uniform sampling. In Persson (2002) and van Steenis and Tulen (1991), the RieT is used in applications. The DirT can be compared with the Lomb-Scargle periodogram (Lomb, 1976; Scargle, 1982), which is a least squares fitting of the transform at each frequency, and is approximately the |DirT|2 , only scaled with N. In Gunnarsson et al. (2004), a few more expressions are derived, for example, with spline interpolation and sinc basis expansion (for band limited functions). The expressions are much more complicated to calculate and for almost uniform sampling, they basically coincide with the RieT and DirT.

88

3.3


Summarizing the Expressions

In summary, this work focuses on two AFTs given by (A.4) with, ( −i2π f t k, e n = 0, DirT, n −i2π f tk wk ( f |tk ) = (Tk ) e = Tk e−i2π f tk , n = 1, RieT,

(A.19)

where n ∈ {0, 1} indicates the transform type used. In the remainder of this paper, the sampling times, tk , are stochastic variables, and the frequency transform, Y( f ), is a random variable with a certain probability distribution. We will investigate the properties of this distribution depending on the type of sampling chosen from Table A.2 and the type of transform in (A.19).

3.4

The Periodogram

Here, we define the periodogram PY ( f ) of Y( f ) as PY ( f ) = |Y( f )|2 ,

(A.20)

and get another stochastic variable. Other definitions of PY can include scaling with the number of samples N, or the total time interval, tN or N E[Tk ], which does not affect the results. The periodogram for zero-mean Gaussian signals is known to be exponentially distributed. Stoica and Moses (2005) and Ljung (1999) shows that X 2 1 −i2π f kT 2 vk e (A.21) N ∈ AsExp(1/σ ), when vk is (uniformly sampled) zero-mean white Gaussian noise. We will return to this when discussing the distribution of the periodogram in the case of nonuniform sampling. Note that in the literature, the above periodogram is said to be chi-squared distributed with two degrees of freedom, which is the same as the exponential distribution. We choose the exponential notation to give good correspondance with the following investigation.

4

Main Results

The condensed result of this work is given in this section, and it is discussed and exemplified in the following sections. It is obvious that the distribution of τk plays a crucial role for the stochastic properties of the AFT. This will be shown via the influence of the characteristic function, ϕτ (ω) = Eτ [eiωτ ]. It will be convenient to define an extended version of the characteristic function ϕm ( f ) , E[(Tk )m e−i2π f τk ],

m = 0, 1, . . .

(A.22)

where Tk is the inter-sampling time and m = 0 gives the standard definition when f = −ω/2π. The parameter m will show parts of the difference between the two

4

89

Main Results

transforms, RieT and DirT. A strong connection with the uniform sampling case will be shown with the appearance of the normalized Dirichlet kernel (also known as the aliased sinc function) dSJS ( f ) , e−i2π f T N

sin(π f NT) 1 − e−i2π f NT . = e−iπ f (N−1)T −i2π f T sin(π f T) 1−e

(A.23a)

A more general definition will also occur, dARS N (f) ,

1 − ϕ0 ( f )N 1 − E[e−i2π f τk ]N = . 1 − ϕ0 ( f ) 1 − E[e−i2π f τk ]

(A.23b)

The superscripts indicate that these factors will show the difference between the sampling types.

4.1

Mean and Covariance of Y( f )

In the following we study the a priori properties, and hence the conditioning on tN is removed. We consider a signal s(t) with Fourier transform S( f ) sampled 1 non-uniformly at random times tk , with probability densities, pk (t), resulting in random signal samples yk = s(tk ). The sampling times are constructed from random sampling noise, τk , using one of ARS and SJS, see Table A.2. If we use the inverse Fourier transform description, (A.1), we get   ∞ Z∞ N N   X X Z  i2πψt   k  S(ψ)e S(ψ)W( f, ψ) dψ, (A.24a) Y( f ) = yk wk ( f ) = dψ wk ( f ) =   k=1

k=1 −∞

−∞

where W( f, ψ) =

N X

ei2πψtk wk ( f ).

(A.24b)

k=1

The definition of the weights, wk ( f ), in (A.19), implies that N X

W( f, ψ) = W( f − ψ) ,

(Tk )n e−i2π( f −ψ)tk ,

(A.25)

k=1

which gives a nice convolution formulation in (A.24a), and allows us to state the first two moments of Y. Theorem A.1 (Mean Value and Covariance of Y) The mean value, µY , and covariance, RY , of the stochastic frequency transform, Y( f ) (A.24), are N µY ( f ; S, pN 1 ) = S ? µW ( f ; p1 ),

RY ( f, ψ; S, pN 1) !

2 We use ? to denote single convolution,

=S?

f ? g(x) = f (ξ)h(x − ξ, y − η) f ∗ (η)dξdη, respectively.

R

2 ?RW ( f, ψ; pN 1 ),

(A.26a) (A.26b)

f (ξ)g(x−ξ) dξ, and ?? for double, f ??h(x, y) =

90


with µW ( f ; pN 1 ) , E[W( f )]

(A.27a)

RW ( f, ψ; pN 1 ) , Cov(W( f ), W(ψ)).

(A.27b)

and

Proof: By using (A.24) we get the mean value of Y as µY ( f ; S, pN 1)

Z∞ , E[Y( f )] = E[ S(φ)W( f − φ) dφ] −∞

Z∞ =

S(φ) E[W( f − φ)] dφ = S ? µW ( f ; pN 1 ). −∞

In a similar manner the covariance is obtained, RY ( f , ψ; S, pN 1 ) , Cov(Y( f ), Y(ψ)) = E[(Y( f ) − µY ( f ))(Y(ψ) − µY (ψ))∗ ]  ∞  ∞ ∗ Z  Z     = E[ S(φ) W( f − φ) − µW ( f − φ) dφ  S(η) W(ψ − η) − µW (ψ − η) dη ]    −∞ −∞ " ∞ ∗ = S(φ) E[ W( f − φ) − µW ( f − φ) W(ψ − η) − µW (ψ − η) ]S(η)∗ dφ dη −∞

= S ? ?RW ( f, ψ; pN 1 ), and the proof is complete.

The connection to the original sampled signal is completely captured in the convolutions with S( f ). This is why we can focus on the stochastic frequency window, W( f ), and its properties. For a single sinusoid, the transform is given as Y( f ) = (W( f − f0 ) − W( f + f0 ))/2i, and the stochastic frequency window is the sampling effect on a single frequency. This is completely in accordance with the uniform sampling case.

4.2

Mean and Covariance of W( f )

As shown in Theorem A.1, we can focus on the stochastic frequency window, W( f ), to describe the properties of the original transform. The mean value and covariance can be described completely with the following theorem. Theorem A.2 (Mean Value and Covariance of W) The mean value, µW , and covariance, RW , (defined in (A.27)) of the stochastic frequency window, W(·) (A.25), are ST µW ( f ; pN 1 ) = ϕn ( f )dN ( f ),

RW ( f, ψ; pN 1)

=

T ST ΦST n ( f, ψ) DN ( f, ψ)

(A.28a) − µW ( f )µW (ψ) , ∗

(A.28b)

4

91

Main Results

where the explicit expressions are given in Table A.3. The sampling type, ST, is one of {SJS, ARS} and the transform type affects n ∈ {0, 1}, giving four combinations, summarized in Table A.4. Proof: When considering ARS, (A.9), the inter-sampling time is exactly the sampling noise, Tk = τk . The stochastic properties of the frequency window, W(·), will depend on the finite number of samples, N, and the stochastic properties of the sampling noise. The mean value of the stochastic frequency window, for the case of ARS, is µW ( f ; pN 1 ) = E[W( f )] =

N X

E[(Tk )n e−i2π f tk ]

k=1

=

N X

E[(Tk )n

k Y

e−i2π f τn ] =

n=1

k=1

= ϕn ( f )

N−1 X

N X

E[(Tk )n e−i2π f τk ]ϕ0 ( f )k−1

k=1

ϕ0 ( f )k = ϕn ( f )dARS N ( f ),

k=0

which corresponds to (A.28a) and the definitions in Table A.3. For SJS, (A.11), the first moment is very similar, E[W( f )] =

N X

e−i2π f kT E[(Tk )n e−i2π f τk ] = ϕn ( f )

k=1

N X

e−i2π f kT = ϕn ( f )dSJS ( f ), N

k=1

which also corresponds to the given definitions. The calculations for the covariance of W(·) are straightforward and similar to the ones above. They are given in Appendices A and B to keep the presentation clean. Note that for SJS and DirT, the expressions are more compact than they appear, since n = 0 and then the term nκ( f )ϕ0 (−ψ) = 0. For uniform sampling and a uniform (boxcar) time window, the frequency window, W, is deterministic and becomes exactly the normalized Dirichlet kernel, dSJS ( f ), and the corresponding transform, N Z∞ Y( f |kT) =

S(ψ)dSJS ( f − ψ) dψ, N

(A.30)

−∞

is the DTFT, (A.6). Previous publications include analysis of the stochastic frequency window, W( f ). Studies of the expected value are made in Gunnarsson et al. (2004), and signal estimation in the frequency domain for non-uniformly sampled signals is investigated in Eng and Gustafsson (2007). Remark A.1. For decreasing sampling noise variance the different sampling types tend to uniform sampling. The results for the mean value agree with this since,

92


Table A.3: Explicit expressions needed for evaluation of Theorem A.2. The indicator n ∈ {0, 1} specifies the transform.

ϕm ( f ) = E[(Tk )m e−i2π f τk ], m = 0, 1, 2, . . . ( −i2π f NT e−i2π f T 1−e , f , m/T, 1−e−i2π f T dSJS ( f ) = N N, f = m/T.  N  1−ϕ ( f )   1−ϕ00 ( f ) , ϕ0 ( f ) , 1, dARS N (f) =    N, ϕ0 ( f ) = 1.  ϕn ( f )ϕn (−ψ)   ϕn ( f )ϕn (−ψ) + nκ( f )ϕ0 (−ψ)   ϕ2n ( f − ψ) ΦSJS n ( f, ψ) =    ϕn ( f )ϕn (−ψ) + nκ(−ψ)ϕ0 ( f )  ϕn ( f )ϕn (−ψ)

(A.29a) (A.29b) (A.29c)       ,   

n = 0, 1,

κ( f ) = T2 ϕ0 ( f ) − 2ϕ2 ( f ) + Tϕ1 ( f ) + 2 E[τ2k ]ϕ0 ( f ),    ϕn ( f )ϕn ( f − ψ)    , n = 0, 1, ϕ2n ( f − ψ) ΦARS n ( f, ψ) =    ϕn (−ψ)ϕn (ψ − f )   N k−2  P P −i2π( f k−ψl)T  e     k=1 l=1   −i2π f T SJS  e dN ( f − ψ)      , DSJS ( f, ψ) =  dSJS ( f − ψ)  N N  i2πψT  SJS  e dN ( f − ψ)    N N  P P −i2π( f k−ψl)T    e

(A.29d)

(A.29e) (A.29f)

(A.29g)

k=1 l=k+2

 N k−1  P P  ϕ ( f )k−l−1 ϕ0 ( f − ψ)l−1  k=1 l=1 0   dARS ( f − ψ) DARS  N ( f, ψ) =  N  P N k−1 P  ϕ0 (−ψ)k−l−1 ϕ0 (ψ − f )l−1  k=1 l=1

      .    

(A.29h)

Table A.4: Collection of expressions for different setups of (A.28a) and (A.28b) in Theorem A.2. Table A.3 defines the functions.

ARS SJS

µW RW +µµ∗ µW RW +µµ∗

DirT (n = 0)

RieT (n = 1)

ϕ0 ( f )dARS (f) N

ϕ1 ( f )dARS (f) N

ΦARS ( f, ψ)T DARS ( f, ψ) 0 N

ΦARS ( f, ψ)T DARS ( f, ψ) N 1

ϕ0 ( f )dSJS (f) N

ϕ1 ( f )dSJS (f) N

ΦSJS ( f, ψ)T DSJS ( f, ψ) 0 N

ΦSJS ( f, ψ)T DSJS ( f, ψ) N 1

5

93

Asymptotic Analysis for ARS

intuitively, we have ϕn ( f ) → T n ,

when Var(τk ) → 0, E[τk ] = 0,

(A.31a)

SJS dARS N ( f ) → dN ( f ), when Var(τk ) → 0, E[τk ] = T,

(A.31b)

and hence E[DirT] and E[RieT] approach the DTFT.

4.3

The Periodogram

With the previous results we can explicitly get the mean of the periodogram. Corollary A.1 (Mean of the Periodogram) The mean value µP ( f ) of the periodogram PY ( f ) (A.20) is µP ( f ) = E[|Y( f )|2 ] = |µY ( f )|2 + RY ( f, f ), where µY and RY are given by Theorem A.1. Proof: The result is given by the definition of the variance RY ( f, f ) = E[|Y( f ) − µY ( f )|2 ] = E[|Y( f )|2 ] − |µY ( f )|2 = µP ( f ) − |µY ( f )|2 .

5


When the number of measurements increases to infinity more conclusions can be made. Here, we give results for the asymptotic mean value, and discuss asymptotic distributions for the transform, its magnitude, and the periodogram.

5.1

Asymptotic Analysis of the Mean Value

In the previous analysis, a finite number of samples, N, was considered. The following lemma investigates the term dARS ( f ), when N goes to infinity. The periN SJS odic counterpart, dN ( f ), is well known and thoroughly studied, it will converge to an impulse train as N increases. The lemma will provide us with a tool to study the asymptotic mean value of W( f ) and Y( f ) for ARS. Note that, in the lemma, we discuss distributions from functional theory and not probability theory. The assumptions in the lemma will be discussed later. Lemma A.1 (Asymptotic Properties of dARS ) N Consider the case of ARS. Assume that the continuous-time function h(t) with FT H( f ) fulfills the following conditions: 1. h(t) and H( f ) belongs to the Schwartz class, S.3 3h

∈ S ⇔ tk h(l) (t) is bounded, i.e., h(l) (t) = O(|t|−k ), for all k, l ≥ 0.

94


PN

2. The sum gN (t) =

pk (t) obeys Z Z lim gN (t)h(t) dt = k=1

N→∞

1 1 h(t) dt = H(0), µT µT

(A.32)

for this h(t). 3. The initial value is zero, h(0) = 0. Then, it holds that Z lim

N→∞

dARS N ( f )H( f ) d f =

1 H(0). µT

(A.33)

Proof: The proof is conducted using distributions.4 Let ∆r denote the distribution corresponding to the function r. We use Z h∆r , hi = r(t)h(t) dt to denote the action of the distribution corresponding to r on h ∈ S. Now, the aim is to show that h∆dARS , Hi → N

1 hδ, Hi, µT

N→∞

under the given assumptions on H( f ). We note the following, the CF, ϕ0 ( f ), is the FT of the pdf of the sampling noise, pτ (t), which means that ϕ0 ( f )k = FT[pk (t)] = FT[pτ ? . . . ? pτ (t)]. Also, the FT can be moved between the functions in the integral, which, in particular, means that Z Z h∆r , hi = r(t)h(t) dt = FT[r(t)]( f )H( f ) d f = h∆R , Hi. Now, with this clarified, we can continue:  Z X Z  N−1 N−1 X   k h∆dARS , Hi = ϕ0 ( f ) H( f ) d f = FT[pk (t)]( f ) H( f ) d f 1 + N k=0

k=1

Z =

H( f ) d f +

Z X N−1

Z FT[pk (t)]( f )H( f ) d f = h(0) +

gN−1 (t)h(t) dt

k=1

→ h(0) +

1 H(0), µT

N → ∞.

(A.34)

4 The theory for distributions can be revisited in Gausquet and Witkomski (1999) if necessary. For the proof we need to know that hδ, hi = h(0) for h, ∈ S.

5

95


This proves the lemma, since h(0) = 0 was assumed. In particular, h∆dARS , ϕ0 Hi =

Z X N

N

ϕ0 ( f )k H( f ) d f =

Z X N

k=1

Z =

FT[pk (t)]( f )H( f ) d f

k=1

gN (t)h(t) dt →

1 H(0), µT

N → ∞,

(A.35)

which actually shows that the inverse FT of ϕ0 ( f )H( f ) is always 0 at the origin, when H( f ) fulfills the assumptions in the lemma. Let us study the conditions on h(t) and H( f ) given in Lemma A.1 a bit more. The restrictions from the Schwartz class could affect the usability of the lemma. However, all smooth functions with compact support (and their Fourier transforms) are in S, which should suffice for most cases. It is not intuitively clear how hard (A.32) is. Note that, for any ARS case with continuous sampling noise distribution, pk (t) is approximately a Gaussian for higher k, and we can confirm that, for a large enough fixed t, gN (t) =

N X k=1

√

1

−

2πkσT

e

(t−kµT )2 2kσ2 T

→

1 , µT

N → ∞.

(A.36)

The integral in (A.32) can then serve as some kind of mean value approximation, and the edges of gN (t) will not be crucial. Also, condition 3 further restricts the behavior of h(t) for small t, which will make condition 2 easier to fulfill. The convergence of dARS ( f ) is also discussed in Bilinskis and Mikelsons (1992, Ch. 3), N for the special case of g∞ (t) = 1/µT for all t. Discussions on how to achieve this equality is also done. Now we can study the mean value of Y( f ) for ARS, when N → ∞. Lemma A.1 and Theorem A.1 give Z∞ E[Y( f )] =

Z S(ψ)ϕn ( f − ψ)dARS N ( f − ψ) dψ =

S( f + ψ)ϕn (ψ)dARS N (ψ) dψ

−∞

1 → ϕn (0)S( f ) = µT

(

S( f )/µT , S( f ),

DirT, RieT.

when H(ψ) =R S( f + ψ)ϕn (ψ) fulfills the requirements in the lemma. For the RieT, condition 3, S( f + ψ)ϕ1 (ψ) dψ = 0, is probably the hardest to check, while this condition is automatically fulfilled for DirT, by the special case shown in the proof. The constant µT is the average inter-sampling time and thus the DirT can be scaled in advance to get asymptotically unbiased estimates.

96


5.2

Asymptotic Distribution

The stochastic transform Y( f ) given by (A.24) and (A.25), is asymptotically a sample from the stochastic distribution N Y( f ) ∈ AsN(µY ( f ; S, pN 1 ), RY ( f, f ; S, p1 )),

(A.37)

where AsN denotes the asymptotic Gaussian distribution. By this we mean that Y has mean value µY and covariance RY , both depending on N, and Y converges in distribution to a Gaussian random variable as N goes to infinity. It is clear that the real and imaginary parts of Y( f ) are independent, since cos and sin are orthogonal basis functions. The transform is a sum of random variables and both Y( f ) and the normalized ¯ f ) = 1 Y( f ), will converge to a Gaussian distribution as N increases. version, Y( N This can be proved by use of a general central limit theorem, and was also indicated by the numerical investigations in Eng and Gustafsson (2005). In this work, we are satisfied with the fact that the construction of the transform makes the asymptotic Gaussian distribution likely.

5.3

Distribution of the Transform Magnitude

Since the Fourier transform is a complex variable, it will be of importance to study its absolute value. Knowing the distribution of Y( f ) to be Gaussian, the asymptotic mean value of the amplitude could be found using the Rice distribution, i.e., (A.37) implies that |Y( f )| ∈ AsRic, where AsRic denotes the asymptotic Rice distribution. This gives us complete knowledge of the moments of |Y( f )|. For the stochastic frequency window W( f ), it is therefore possible to calculate the mean value E[|W( f )|] and covariance Cov(|W( f )|) numerically, using our expression for µW ( f ) and RW ( f ) derived earlier and formulas from the Rice distribution. The Rice distribution is presented in Rice (1945), and below we recapitulate the main ideas. If z is a complex Gaussian stochastic variable with mean µz , and variance Rz , then |z| is Rice distributed with known mean µ|z| and variance R|z| . Also, the stochastic variable r = |z − µz | is Rayleigh distributed with known mean µr and variance Rr . These values are related as |µz | ≤ µ|z| , max(Rr − 2µr |µz |, 0) ≤ R|z| ≤ Rz .

(A.38)

The moments of the variable |z| can be described, using moments of z, as − E[|z|k ] = Rk/2 z e

|µz |2 Rz

Γ( k+2 2 ) Γ(1)

1 F1 (

|µz |2 k+2 , 1; ), 2 Rz

(A.39)

5

97


where Γ(q) was given in (A.14). The function 1 F1 (α, β; x) is the confluent hypergeometric function defined as ∞ X Γ(α + n)Γ(β)xn F (α, β; x) = . 1 1 Γ(α)Γ(β + n)n! n=0 For even values of k it is possible to get polynomial expressions for E[|z|k ], for example, E[|z|2 ] = |µz |2 + Rz ,

(A.40a)

E[|z| ] = |µz | + 4|µz | Rz + 4

5.4

4

2

2R2z .

(A.40b)

The Periodogram

The asymptotic distribution of Y( f ) will enforce the asymptotic distribution on the periodogram, PY ( f ). We note that known connections between the Gaussian, Rice, Rayleigh and exponential distributions gives the following statements for a complex random variable z,   |z| ∈ Ric    |z − µ| ∈ Ray(R/2) z ∈ N(µ, R) ⇒     |z − µ|2 ∈ Exp( 1 ). R

This means that (A.37), together with the fact that the real and imaginary parts of Y( f ) are independent, implies |Y( f ) − µY ( f )|2 ∈ AsExp(

1 ), RY ( f )

|Y( f ) − µY ( f )|2 = PY ( f ) + |µY ( f )|2 − X( f ) X( f ) = 2 NT is assured. This is not in exact correspondence with the problem formulation, but assures that the results for different τm -distributions are comparable. The samples are corrupted by additive measurement noise, u(tm ) = s(tm ) + e(tm ),

(C.7)

where e(tm ) ∈ N(0, σ2 ), σ2 = 0.1. The filter is a second order LP filter of Butterworth type with cut-off frequency 1 , 2T i.e., h(t) = H(s) =

√ π − π√ t π 2 e T 2 sin( √ t), T T 2 s2 +

√

(π/T)2 2π/Ts + (π/T)2

t > 0,

.

(C.8) (C.9)

This setup is used for 500 different realizations of f j , τm and e(tm ). We will test four different rectangular distributions (C.6): τm ∈ Re(0.1, 0.3),

µT = 0.2,

σT = 0.06

(C.10a)

τm ∈ Re(0.3, 0.5),

µT = 0.4,

σT = 0.06

(C.10b)

τm ∈ Re(0.4, 0.6),

µT = 0.5,

σT = 0.06

(C.10c)

τm ∈ Re(0.2, 0.6),

µT = 0.4,

σT = 0.12

(C.10d)

and the mean values, µT , and standard deviations, σT , are shown for reference. For every run we use the algorithms presented in the previous section and compare their results to the exact, continuous-time, result, Z z(kT) = h(kT − τ)s(τ) dτ. (C.11) We calculate the root mean square error, RMSE, s 1 X |z(kT) − zˆ (kT)|2 . λ, N

(C.12)

k

The algorithms are ordered according to lowest RMSE, (C.12), and Table C.1 presents the result. The number of first, second and third positions for each algorithm during the 500 runs, are also presented. Figure C.1 presents one example of the result, though the algorithms are hard to separate by visual inspection.

A number of conclusions can be drawn from the previous example: • Comparing a given algorithm for different non-uniform sampling time pdfs, Table C.1 shows that pτ (t), in (C.10), has a clear effect on the performance.

3

141

Numeric Evaluation

Table C.1: RMSE values, λ in (C.12), for estimation of zˆ (kT), in Example C.1. The number of runs where respective algorithm finished 1st , 2nd and 3rd , are also shown.

Alg. C.1 Alg. C.2 Alg. C.3 Alg. C.1 Alg. C.2 Alg. C.3 Alg. C.1 Alg. C.2 Alg. C.3 Alg. C.1 Alg. C.2 Alg. C.3

E[λ] Stdev(λ) 1st Setup in (C.10a) 0.281 0.012 98 0.278 0.012 254 0.311 0.061 148 Setup in (C.10b) 0.338 0.017 9 0.325 0.015 175 0.330 0.038 316 Setup in (C.10c) 0.360 0.018 6 0.342 0.015 144 0.341 0.032 350 Setup in (C.10d) 0.337 0.015 59 0.331 0.015 117 0.329 0.031 324

2nd

3rd

258 195 47

144 51 305

134 277 89

357 48 95

82 329 89

412 27 61

133 285 82

308 98 94

3

2

1

0

−1

−2

−3 100

110

120

130

140

150

160

170

180

190

200

time, t

Figure C.1: The result for the four algorithms, given Example C.1, and a certain realization of (C.10c). The dots are u(tm ) and z(kT) are interconnected with the line, while the triangles are Algorithm C.1, the crosses are Algorithm C.2 and the squares are Algorithm C.3.

142

C

Downsampling Non-Uniformly Sampled Data

• Comparing the algorithms for a given sampling time distribution shows that the lowest mean RMSE is no guarantee of best performance at all runs. Algorithm C.2 has the lowest E[λ] for setup (C.10a), but still performs worst in 10% of the cases, and for (C.10d) Algorithm C.3 is number 3 in 20% of the runs, while it has the lowest mean RMSE. • Usually, Algorithm C.3 has the lowest mean RMSE, but its standard deviation is more than twice as large, compared to the other two algorithms. • Algorithms C.1 and C.2 have similar RMSE statistics, though, of the two, Algorithm C.2 performs slightly better in the mean, in all the four tested cases. Remark C.4. It is important to note that the performance depends on the setup. For example, Algorithm C.2 needs the downsampling factor M/N to be significantly larger than 1 for the Riemann approximation to be good. In the examples above it is at least a factor 10. The algorithms are comparable in performance and complexity. In the following we focus on Algorithm C.2, because of its nice analytical properties, its on-line compatibility, and, of course, its slightly better performance results.

4

Application Example

As a motivating example, consider the ubiquitous wheel speed signals in vehicles that are instrumental for all driver assistance systems and other driver information systems. The wheel speed sensor considered here gives L = 48 pulses per revolution, and each pulse interval can be converted to a wheel speed. With a wheel radius of 1/π, one gets 24v pulses per second. For instance, driving at v = 25 m/s (90 km/h) gives an average sampling rate of 600 Hz. This illustrates the need for speed-adaptive downsampling. Example C.2 Data from a wheel speed sensor, like the one discussed above have been collected. An estimate of the angular speed, ω(t), ˆ m) = ω(t

2π , L(tm − tm−1 )

(C.13)

can be computed at every sample time. The average inter-sampling time, tM /M, is 2.3 ms for the whole sampling set. The set is shown in Figure C.2. We also indicate the time interval that the following calculations are performed on, a sampling time T = 0.1 s gives a signal suitable for driver assistance systems.

4

143

Application Example

110

instantaneous angular speed estimate, ω [rad/s]

100 90 80 70 60 50 40 30 20 10 0 0

200

400

600

800

1000

1200

1400

time, t [s]

Figure C.2: The data from a wheel speed sensor of a car. The data in the gray area is chosen for further evaluation. It includes more than 600 000 measurements. For the data set in Example C.2, there is no true reference signal, but in an off-line test like this, we can use computationally expensive methods to compute the best estimate. For this application, we can assume • independent measurement noise, e(tm ), and • bounded second derivative of the underlying noise-free function s(t), i.e., |s(2) (t)| < C, which in the car application means limited acceleration changes. Under these conditions, the work by Fan and Gijbels (1996), helps with choosing the optimal filter h(t), to get the most accurate estimates. When estimating a function value z(kT) from a sequence u(tm ) at times tm , a local weighted linear approximation is investigated. The underlying function is approximated locally with a linear function m(t) = θ1 + (t − kT)θ2

(C.14)

and m(kT) = θ1 is then found from minimization, θˆ = arg min θ

M X

(u(tm ) − m(tm ))2 KB (tm − kT),

(C.15)

m=1

where KB (t) is a kernel with bandwidth B, i.e., KB (t) = 0 for |t| > B. The Epanechnikov kernel KB (t) = (1 − (t/B)2 )+ ,

(C.16)

144

C


94 93 92

angular speed, ω [rad/s]

91 90 89 88 87 86 85 84 83 675

680

685

690

695

700

time, t [s]

Figure C.3: The cloud of data points, u(tm ) black, from Example C.2, and the optimal estimates, zopt (kT) gray. Only part of the shaded interval in Figure C.2 is shown. is the optimal choice for interior points, t1 + B < kT < tM − B, both in minimizing MSE and error variance. Here, subscript + means taking the positive part. This corresponds to a non-causal filter for Algorithm C.2. This gives the optimal estimate zˆ opt (kT) = θˆ 1 , using the non-causal filter given by (C.14)–(C.16) with B = Bopt from Fan and Gijbels (1996), Bopt

15σ2 (kT) = 2 C M f (kT)

!1/5 ,

(C.17)

where σ2 (kT) is the variance of u(kT), C is the upper limit of the second derivative of s(kT), and f (kT) is the density of the time points, tm . In order to find Bopt , the values of σ2 (kT), C and f (kT) were roughly estimated from data in each interval [kT − T/2, kT + T/2], and a mean value of the resulting bandwidth was used for Bopt . The result from the optimal filter, zˆ opt (kT), is shown compared to the original data in Figure C.3, and it follows a smooth line nicely. Now, we test four different estimates zˆ E (kT) : the casual filter given by (C.14), (C.15), the kernel (C.16) for −B < t ≤ 0 and B = 2Bopt ; zˆ BW (kT) : a causal Butterworth filter, h(t), in Algorithm C.2; The Butterworth filter is of order 2 with cut-off frequency 1/2T = 5 Hz, as defined in (C.8), zˆ m (kT) : the mean of u(tm ) for tm ∈ [kT − T/2, kT + T/2]; zˆ n (kT) : a nearest neighbor estimate.

5

145

Theoretic Analysis of Algorithm C.2

79.8 79.6

angular speed, ω [rad/s]

79.4 79.2 79 78.8 78.6 78.4 78.2 78 647

648

649

650

651

652

653

654

time, t [s]

Figure C.4: A comparison of three different estimates for the data in Example C.2: optimal zopt (kT) (thick line), casual "optimal" zE (kT) (gray line), and causal Butterworth zBW (kT) (thin line). Only a part of the shaded interval in Figure C.2 is shown. and compare them to the optimal zopt (kT). Figure C.4 shows the two first estimates, zE (kT) and zBW (kT), compared to the optimal zopt (kT). It is clear that the two causal estimates vary more than the non-causal optimal one. Table C.2 shows the root mean square errors compared to the optimal estimate, zˆ opt (kT), calculated over the interval indicated in Figure C.2. From this, it is clear that the casual “optimal” filter, giving zˆ E (kT), needs tuning of the bandwidth, B, since we have no result for the optimal choice of B in this case. Both the filtered estimates, zE (kT) and zBW (kT), are significantly better than the simple mean, zm (kT). The Butterworth filter performs very well, and is also much less computationally complex than using the Epanechnikov kernel. It is promising that the estimate from Algorithm C.2, zˆ BW (kT), is close to zˆ opt (kT), and it encourages future investigations. Table C.2: RMSE from optimal estimate, ple C.2. casual “optimal”, zˆ E (kT) 0.0793

5

Butterworth, zˆ BW (kT) 0.0542

q

E[|ˆzopt (kT) − zˆ ∗ (kT)|2 ], in Exam-

local mean, zˆ m (kT) 0.0964

nearest neighbor zˆ n (kT) 0.3875


Here we study the a priori stochastic properties of the estimate, zˆ (kT), given by Algorithm C.2. For the analytical calculations, we use that the convolution is

146

C


symmetric, and get zˆ (kT) =

M X

τm h(tm )u(kT − tm ),

m=1

=

M X

Z τm

Z i2πη(tm )

H(η)e

dη

U(ψ)ei2πψ(kT−tm ) dψ,

m=1

" =

H(η)U(ψ)ei2πψkT

τm e−i2π(ψ−η)tm dψ dη,

m=1

" =

M X

H(η)U(ψ)ei2πψkT W(ψ − η; tM 1 ) dψ dη,

(C.18)

with W( f ; tM 1 )=

M X

τm e−i2π f tm .

(C.19)

m=1

Let, −i2π f τ

ϕτ ( f ) = E[e

Z ]=

e−i2π f τ pτ (τ) dτ = F (pτ (t))

denote the characteristic function for the sampling noise τ. Here F is the Fourier transform operator. Then Theorem 2 in Eng et al. (2007) gives E[W( f )] = −

1 dϕτ ( f ) 1 − ϕτ ( f )M , 2πi d f 1 − ϕτ ( f )

(C.20)

and also an expression for the covariance, Cov(W( f )), not repeated here. These known properties of W( f ) make it possible to find E[ˆz(kT)] and Var(ˆz(kT)) for any given characteristic function, ϕτ ( f ), of the sampling noise, τk . The main results in Eng et al. (2007) is used to derive the following theorem. Theorem C.1 (Properties of zˆ ) The estimate given by Algorithm C.2 can be written as zˆ (kT) = h˜ ? u(kT),

(C.21a)

˜ = F −1 (H ? W( f ))(t), h(t)

(C.21b)

˜ is given by where h(t)

with W( f ) as in (C.19).

5

147


Furthermore, if the filter h(t) is smooth with compact support1 , and h(0) = 0, then E[ˆz(kT)] → z(kT),

Z X M

if

m=1

E[ˆz(kT)] = z(kT),

Z X M

if

m=1

Z

1 pm (t)h(t) dt → µT 1 pm (t)h(t) dt = µT

h(t) dt,

M → ∞,

(C.21c)

Z h(t) dt,

∀M,

(C.21d)

with µT = E[τm ], and pm (t) is the pdf for time tm . Proof: First of all, (C.2) gives Z z(kT) =

H(ψ)U(ψ)ei2πψkT dψ,

(C.22a)

and from (C.18) we get Z zˆ (kT) =

Z U(ψ)ei2πψkT

H(η)W(ψ − η) dη dψ | {z }

(C.22b)

˜ H(ψ)

˜ f ) as the filter operation on the continuouswhich implies that we can identify H( time signal u(t), and (C.21a) follows. If h(t) is smooth with compact support, h(t) and H( f ) belongs to the Schwartz class, and Lemma 1 in Eng et al. (2007), together with (C.20), gives Z E[W( f )]H( f ) d f → H(0). (C.22c) We then get Z E[ˆz(kT)] =

Z U(ψ)ei2πψkT

H(η) E[W(ψ − η)] dη dψ

Z →

H(ψ)U(ψ)ei2πψkT dψ = z(kT).

(C.22d)

Using the same technique as in the proof of the mentioned Lemma 1, the third property follows. ˜ f ) is the Remark C.5. From the investigations in Eng et al. (2007) it is clear that H( AFT of the sequence h(tm ), cf., the AFT of u(tm ) in step (2) of Algorithm C.3. Algorithm C.3 can be investigated analogously. Theorem C.2 The estimate given by Algorithm C.3 can be written as zˆ (kT) = h˜ ? u(kT), 1 More

(C.23a)

generally, it is sufficient that h(t) ∈ S, the Schwartz class (Gausquet and Witkomski, 1999).

148

C


˜ is given by the inverse Fourier transform of where h(t) ˜ f) = 1 H( 2NT

 N−1 2N−1 X  X −i2π( f − fn )kT −i2π( f − fn )kT  H( fn )e W( fn − f ) + H(− fn )e W(− fn − f )  n=0

n=N

and W( f ) was given in (C.19). Proof: First observe that real signals u(t) and h(t) gives U( f )0 = U(− f ) and H( f )0 = H(− f )0 , respectively, the rest is completely in analogue with the proof of Theorem C.1, with one of the integrals replaced with the corresponding sum. In Eng et al. (2007), it is argued that the requirements on h(t) in (C.21c) are not impossible. Since ARS is given by a sum of independent random variables, the central limit theorem will make pm (t) converge towardPa Gaussian distribution (with µ = mµT and σ2 = mσ2T ) and for higher t the sum, pm (t), then converges to a Pconstant. The integral then serves as averaging over the initial time span, where pm (t) , 1/µT . The fact that h(0) = 0 is required is usually the most restrictive of the demands. A non-causal filter is desired in theory, but often not possible in practice. The performance of zˆ BW (kT) compared to the optimal non-causal filter in Table C.2 is thus encouraging.

6

Illustration of Theorem C.1

Theorem C.1 shows that the originally designed filter H( f ) is effectively replaced ˜ f ) by using Algorithm C.2. Since H( ˜ f ) only depends on by another linear filter H( ˜ f ) to exclude the effects of the H( f ) and the sampling times tm , we here study H( signal, u(t), on the estimate. First, we illustrate (C.21b), showing the influence of the sampling times, or, the ˜ We use the four different sampling noise distributions distribution of τm , on E[H]. in (C.10), using the original filter h(t) from (C.8) with T = 4 s. Figure C.5 shows ˜ f ), when the sampling noise distribution is changed. We the different filters H( ˜ f )]|, and that it seems conclude that both the mean and the variance affect | E[H( ˜ f ) by multiplying zˆ (kT) with a possible to mitigate the pass-band effect of H( constant depending on the filter h(t) and the sampling time distribution pτ (t). Second, we show that EH˜ → H when M increases, for a smooth filter with compact support, (C.21c). Here, we use h(t) =

π t − df T 2 1 cos( ) , 4d f 2 df T

|t − d f T| < d f T

(C.24)

where d f is the width of the filter. The sampling noise distribution is given by (C.10b). Figure C.6 shows an example of the sampled filter h(tm ). Here, the time shift d f T is used in order to produce a smooth causal filter and fulfill the demand h(0) = 0 in a easy way. This in turn introduces a delay of d f T in the resampling procedure. We choose the scale factor d f = 8 for a better view

6

149

Illustration of Theorem C.1

1

0.97

0.94

0.91

Re(0.1,0.3) Re(0.3,0.5) Re(0.4,0.6) Re(0.2,0.6) H(f)

0.88

0.85 −2 10


˜ f )]| given by (C.8) Figure C.5: The true H( f ) (thick line) compared to | E[H( and Theorem C.1, for the different sampling noise distributions in (C.10), and M = 250.

0.035

0.03

0.025

0.02

0.015

0.01

0.005

0 0T

4T

8T

12T

16T

20T

24T

time, t [s]

Figure C.6: An example of the filter h(tm ) given by (C.24), (C.10b) and T = 4.

150

C


0 10

−1 10

−2 10

−3 10

H(f) M=160 M=150 M=140 M=120 M=100

−4 10 −2 10


˜ f )]| for increasFigure C.7: The true filter, H( f ) (thick line), compared to E[H( ing values of M, when h(t) is given by (C.24). of the convergence (higher d f gives slower convergence). The width of the filter covers approximately 2d f T/µT = 160 non-uniform samples, and more than 140 of them are needed for a close fit also at higher frequencies. The convergence of the magnitude of the filter is clearly shown in Figure C.7.

7

Conclusions

This work investigated three different algorithms for downsampling non-uniformly sampled signals, each using interpolation on different levels. Numerical experiments showed that interpolation of the convolution integral presents a good and stable down-sampling alternative, and makes theoretical analysis possible. The algorithm gives asymptotically unbiased estimates when the original filter is smooth and has h(0) = 0. Also, the analysis showed the connection between the original filter and the actual filter implemented by the algorithm.

Acknowledgment The authors wish to thank NIRA Dynamics AB for providing the wheel speed data, and Ph.D. Jacob Roll for interesting discussions on optimal filtering.

References Aldroubi, Akram and Gröchenig, Karlheinz (2001). Nonuniform sampling and reconstruction in shift-invariant spaces. In SIAM Review, vol. 43(4):pp. 585— 620.

References

151

Beutler, Frederick J (1966). Error-free recovery of signals from irregularly spaced samples. In SIAM Review, vol. 8(3):pp. 328–335. Bourgeois, Marc; Wajer, Frank; van Ormondt, Dirk; and Graveron-Demilly, Danielle (2001). Reconstruction of MRI images from non-uniform sampling and its application to intrascan motion correction in functional MRI. In Benedetto, John J and Ferreira, Paulo JSG (Eds.), Wavelets: Mathematics and Applications, chap. 16. Birkhäuser Verlag. Dey, Sourav R; Russell, Andrew I; and Oppenheim, Alan V (2006). Precompensation for anticipated erasures in LTI interpolation systems. In IEEE Transactions on Signal Processing, vol. 54(1):pp. 325–335. ISSN 1053-587X. Eldar, Yonina C (2003). Sampling with arbitrary sampling and reconstruction spaces and oblique dual frame vectors. In Journal of Fourier Analysis and Applications, vol. 9(1):pp. 77–96. Eng, Frida; Gustafsson, Fredrik; and Gunnarsson, Fredrik (2007). Frequency domain analysis of signals with stochastic sampling times. In IEEE Transactions on Signal Processing. Submitted. Fan, Jianqing and Gijbels, Irène (1996). Local Polynomial Modelling and Its Applications. Chapman & Hall. Feichtinger, Hans G and Gröchenig, Karlheinz (1994). Theory and practice of irregular sampling. In Benedetto, John J and Frazier, Michael W (Eds.), Wavelets: Mathematics and Applications. CRC Press. Ferreira, Paulo JSG (2001). Iterative and noniterative recovery of missing samples for 1-D band-limited signals. In Marvasti (2001), chap. 5. Gausquet, Claude and Witkomski, Patrick (1999). Fourier Analysis and Applications. Springer Verlag. Gunnarsson, Frida; Gustafsson, Fredrik; and Gunnarsson, Fredrik (2004). Frequency analysis using nonuniform sampling with applications to adaptive queue management. In IEEE International Conference on Acoustics, Speech, and Signal Processing. Montreal, Canada. Lacaze, Bernard (2001). Reconstruction of stationary processes sampled at random times. In Marvasti (2001), chap. 8. Marvasti, Farokh (Ed.) (2001). Nonuniform Sampling: Theory and Practice. Kluwer Academic Publishers. Marvasti, Faroukh; Analoui, Mostafa; and Gamshadzahi, Mohsen (1991). Recovery of signals from nonuniform samples using iterative methods. In IEEE Transactions on Signal Processing, vol. 39(4):pp. 872–878. Mitra, Sanjit K (1998). Digital Signal Processing. McGraw-Hill. ISBN 0-07-0429537.

152

C


Papoulis, Athanasios (1977). Signal Analysis. McGraw-Hill. Ramstad, Tor A (1984). Digital methods for conversion between arbitrary sampling frequencies. In IEEE Transactions on Signal Processing, vol. 32(3):pp. 577–591. ISSN 0096-3518. Russell, Andrew I (2000). Efficient rational sampling rate alteration using IIR filters. In IEEE Signal Processing Letters, vol. 7(1):pp. 6–7. ISSN 1070-9908. Russell, Andrew I (2002). Regular and Irregular Signal Resampling. Ph.D. thesis, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA. Russell, Andrew I and Beckmann, Paul E (2002). Efficient arbitrary sampling rate conversion with recursive calculation of coefficients. In IEEE Transactions on Signal Processing, vol. 50(4):pp. 854–865. ISSN 1053-587X. Saramäki, Tapio and Ritoniemi, Tapani (1996). An efficient approach for conversion between arbitrary sampling frequencies. In IEEE International Symposium on Circuits and Systems, vol. 2, (pp. 285–288). Atlanta, Georgia, USA. Unser, Michael (1999). Splines: a perfect fit for signal and image processing. In IEEE Signal Processing Magazine, vol. 16(6):pp. 22–38. ISSN 1053-5888. Yao, Kung and Thomas, John B (1967). On some stability and interpolatory properties of nonuniform sampling expansions. In IEEE Transactions on Circuits and Systems [legacy, pre - 1988], vol. 14(4):pp. 404–408. ISSN 0098-4094.

PhD Dissertations Division of Automatic Control Linköpings universitet

M. Millnert: Identification and control of systems subject to abrupt changes. Thesis No. 82, 1982. ISBN 91-7372-542-0. A. J. M. van Overbeek: On-line structure selection for the identification of multivariable systems. Thesis No. 86, 1982. ISBN 91-7372-586-2. B. Bengtsson: On some control problems for queues. Thesis No. 87, 1982. ISBN 91-7372593-5. S. Ljung: Fast algorithms for integral equations and least squares identification problems. Thesis No. 93, 1983. ISBN 91-7372-641-9. H. Jonson: A Newton method for solving non-linear optimal control problems with general constraints. Thesis No. 104, 1983. ISBN 91-7372-718-0. E. Trulsson: Adaptive control based on explicit criterion minimization. Thesis No. 106, 1983. ISBN 91-7372-728-8. K. Nordström: Uncertainty, robustness and sensitivity reduction in the design of single input control systems. Thesis No. 162, 1987. ISBN 91-7870-170-8. B. Wahlberg: On the identification and approximation of linear systems. Thesis No. 163, 1987. ISBN 91-7870-175-9. S. Gunnarsson: Frequency domain aspects of modeling and control in adaptive systems. Thesis No. 194, 1988. ISBN 91-7870-380-8. A. Isaksson: On system identification in one and two dimensions with signal processing applications. Thesis No. 196, 1988. ISBN 91-7870-383-2. M. Viberg: Subspace fitting concepts in sensor array processing. Thesis No. 217, 1989. ISBN 91-7870-529-0. K. Forsman: Constructive commutative algebra in nonlinear control theory. Thesis No. 261, 1991. ISBN 91-7870-827-3. F. Gustafsson: Estimation of discrete parameters in linear systems. Thesis No. 271, 1992. ISBN 91-7870-876-1. P. Nagy: Tools for knowledge-based signal processing with applications to system identification. Thesis No. 280, 1992. ISBN 91-7870-962-8. T. Svensson: Mathematical tools and software for analysis and design of nonlinear control systems. Thesis No. 285, 1992. ISBN 91-7870-989-X. S. Andersson: On dimension reduction in sensor array signal processing. Thesis No. 290, 1992. ISBN 91-7871-015-4. H. Hjalmarsson: Aspects on incomplete modeling in system identification. Thesis No. 298, 1993. ISBN 91-7871-070-7. I. Klein: Automatic synthesis of sequential control schemes. Thesis No. 305, 1993. ISBN 917871-090-1. J.-E. Strömberg: A mode switching modelling philosophy. Thesis No. 353, 1994. ISBN 917871-430-3. K. Wang Chen: Transformation and symbolic calculations in filtering and control. Thesis No. 361, 1994. ISBN 91-7871-467-2. T. McKelvey: Identification of state-space models from time and frequency data. Thesis No. 380, 1995. ISBN 91-7871-531-8. J. Sjöberg: Non-linear system identification with neural networks. Thesis No. 381, 1995. ISBN 91-7871-534-2. R. Germundsson: Symbolic systems – theory, computation and applications. Thesis No. 389, 1995. ISBN 91-7871-578-4.

P. Pucar: Modeling and segmentation using multiple models. Thesis No. 405, 1995. ISBN 91-7871-627-6. H. Fortell: Algebraic approaches to normal forms and zero dynamics. Thesis No. 407, 1995. ISBN 91-7871-629-2. A. Helmersson: Methods for robust gain scheduling. Thesis No. 406, 1995. ISBN 91-7871628-4. P. Lindskog: Methods, algorithms and tools for system identification based on prior knowledge. Thesis No. 436, 1996. ISBN 91-7871-424-8. J. Gunnarsson: Symbolic methods and tools for discrete event dynamic systems. Thesis No. 477, 1997. ISBN 91-7871-917-8. M. Jirstrand: Constructive methods for inequality constraints in control. Thesis No. 527, 1998. ISBN 91-7219-187-2. U. Forssell: Closed-loop identification: Methods, theory, and applications. Thesis No. 566, 1999. ISBN 91-7219-432-4. A. Stenman: Model on demand: Algorithms, analysis and applications. Thesis No. 571, 1999. ISBN 91-7219-450-2. N. Bergman: Recursive Bayesian estimation: Navigation and tracking applications. Thesis No. 579, 1999. ISBN 91-7219-473-1. K. Edström: Switched bond graphs: Simulation and analysis. Thesis No. 586, 1999. ISBN 91-7219-493-6. M. Larsson: Behavioral and structural model based approaches to discrete diagnosis. Thesis No. 608, 1999. ISBN 91-7219-615-5. F. Gunnarsson: Power control in cellular radio systems: Analysis, design and estimation. Thesis No. 623, 2000. ISBN 91-7219-689-0. V. Einarsson: Model checking methods for mode switching systems. Thesis No. 652, 2000. ISBN 91-7219-836-2. M. Norrlöf: Iterative learning control: Analysis, design, and experiments. Thesis No. 653, 2000. ISBN 91-7219-837-0. F. Tjärnström: Variance expressions and model reduction in system identification. Thesis No. 730, 2002. ISBN 91-7373-253-2. J. Löfberg: Minimax approaches to robust model predictive control. Thesis No. 812, 2003. ISBN 91-7373-622-8. J. Roll: Local and piecewise affine approaches to system identification. Thesis No. 802, 2003. ISBN 91-7373-608-2. J. Elbornsson: Analysis, estimation and compensation of mismatch effects in A/D converters. Thesis No. 811, 2003. ISBN 91-7373-621-X. O. Härkegård: Backstepping and control allocation with applications to flight control. Thesis No. 820, 2003. ISBN 91-7373-647-3. R. Wallin: Optimization algorithms for system analysis and identification. Thesis No. 919, 2004. ISBN 91-85297-19-4. D. Lindgren: Projection methods for classification and identification. Thesis No. 915, 2005. ISBN 91-85297-06-2. R. Karlsson: Particle Filtering for Positioning and Tracking Applications. Thesis No. 924, 2005. ISBN 91-85297-34-8. J. Jansson: Collision Avoidance Theory with Applications to Automotive Collision Mitigation. Thesis No. 950, 2005. ISBN 91-85299-45-6. E. Geijer Lundin: Uplink Load in CDMA Cellular Radio Systems. Thesis No. 977, 2005. ISBN 91-85457-49-3. M. Enqvist: Linear Models of Nonlinear Systems. Thesis No. 985, 2005. ISBN 91-8545764-7. T. B. Schön: Estimation of Nonlinear Dynamic Systems — Theory and Applications. Thesis No. 998, 2006. ISBN 91-85497-03-7.

I. Lind: Regressor and Structure Selection — Uses of ANOVA in System Identification. Thesis No. 1012, 2006. ISBN 91-85523-98-4. J. Gillberg: Frequency Domain Identification of Continuous-Time Systems Reconstruction and Robustness. Thesis No. 1031, 2006. ISBN 91-85523-34-8. M. Gerdin: Identification and Estimation for Models Described by Differential-Algebraic Equations. Thesis No. 1046, 2006. ISBN 91-85643-87-4. C. Grönwall: Ground Object Recognition using Laser Radar Data – Geometric Fitting, Performance Analysis, and Applications. Thesis No. 1055, 2006. ISBN 91-85643-53-X. A. Eidehall: Tracking and threat assessment for automotive collision avoidance. Thesis No. 1066, 2006. ISBN 91-85643-10-6.

Non-Uniform Sampling in Statistical Signal ... - Automatic Control

Non-Uniform Sampling in Statistical Signal ... - Automatic Control

Suggest Documents

Noise amplification of periodic nonuniform sampling - Signal

Noise amplification of periodic nonuniform sampling - Signal ...

Digital Repetitive Control under Nonuniform Sampling ... - UPCommons

Automatic Control 2 - Sampling

Multiband signal processing by using nonuniform sampling and ...

Multiband signal processing by using nonuniform sampling and ...

Automatic Control 2 - Sampling - IMT Lucca

Nonuniform sampling of nonbandlimited signals - IEEE ... - Sweet

Nonuniform Sampling of Periodic Bandlimited Signals - Electrical ...

Nonuniform Sampling: Bandwidth and Aliasing - Probability Theory ...

Granger causality analysis with nonuniform sampling ...

Nonuniform Sampling: Theory and Practice - Google Sites

Nonuniform sampling and reconstruction with Hilbert frames

Periodically Nonuniform Sampling Of Bandpass Signals - California ...

Nonuniform Sampling of Complex-Valued Harmonic ...

Download Nonuniform Sampling: Theory and Practice (Information ...

Parameter Selection in Periodic Nonuniform Sampling of ... - arXiv

Uniform Versus Nonuniform Sampling when Tracking in Clutter

Nonuniform Sampling and Reconstruction in Shift-Invariant Spaces∗

The correction of nonuniform signal intensity profiles in ... - Springer Link

Synchronization of sampling in distributed signal ... - CiteSeerX

Statistical quality control based on ranked set sampling ... - ThaiScience

Estimation of the Optimal Statistical Quality Control Sampling ... - PLOS

Statistical Signal Processing