Introduction to the Issue on Differential Geometry in Signal Processing

IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING, VOL. 7, NO. 4, AUGUST 2013

573

Introduction to the Issue on Differential Geometry in Signal Processing

A

PPLICATIONS, tools and underpinning technological advances are primary drivers of signal processing research. New tools and better technology, especially increased computing power, create opportunities not only for new application areas but also for revisiting existing application areas. From differential geometry come tools that can lead to greater insight and, ultimately, better algorithms, for signal processing applications. Importantly, digital signal processing (DSP) architectures are sufficiently powerful to implement potentially nonlinear solutions stemming from geometric insights. There are various ways in which differential geometry, and more generally, topology and geometry, play a fundamental role in signal processing. Before elaborating on this, it is striking to realize that half the human brain is devoted directly or indirectly to processing visual information: to reason about a problem, it makes sense then to “visualize” it. The algebraic fact that a matrix satisfies becomes significantly more meaningful when coupled with the mental image of projection onto a linear subspace. One purpose of differential geometry is describing dynamical systems evolving on spaces other than Euclidean space. For instance, the orientation of an aeroplane is naturally described by a rotation, precisely, an element of the special orthogonal group , and hence the differential equations governing the motion of an aeroplane evolve not in Euclidean space but on a generalization of Euclidean space known as a manifold. Although more abstract definitions are possible that do not rely on an ambient Euclidean space, in the first instance, a manifold is a “smooth,” or “particularly nice,” subset of Euclidean space. Simple examples are spheres and tori. While linear algebra provides tools for working with linear subsets of Euclidean space, differential geometry provides tools for working with curved subsets of Euclidean space. (The special orthogonal group can be thought of as a curved subset of the space of all three-by-three matrices.) The adjective “differential” implies that concepts from calculus have various extensions to manifolds, such as differentiation, integration, Taylor series and so forth. “Geometry” signifies the possibility of measuring the distance between two points, measuring the angle between two intersecting curves, measuring curvature of a surface about a point, and defining a generalization of straight lines (geodesics), among other things. The most direct way differential geometry enters signal processing is when a signal or a parameter naturally belongs to a manifold other than a linear space. Traditional examples include manoeuvring aircraft, robotic arms comprising various joints, and camera-pose estimation problems. A more subtle example is when an ostensibly vector-valued or matrix-valued parameter is of interest, but inherent ambiguity means only partial inforDigital Object Identifier 10.1109/JSTSP.2013.2265514

mation about that parameter can be gained from observations, in which case it may be advantageous to consider the parameter as belonging to what is known as a quotient space, where the ambiguity has been “quotiented out.” In certain instances, this quotient space is a manifold. A prominent example in signal processing of a quotient manifold is the Grassmann manifold. From an applications perspective, if a particular application naturally involves a manifold, a guiding principle is that the shape (precisely, the topology and geometry) of the manifold may give inspiration for new algorithms. (Even if the algorithms end up based on linear approximations, knowing how best to approximate a curved surface comes down to an understanding of the geometry of the problem at hand.) From the perspective of developing tools, a fundamental question is how can algorithms designed for Euclidean space—optimization algorithms, parameter estimation algorithms, filtering algorithms and so forth—be generalized to work on manifolds. One advantage of working with manifolds rather than Euclidean space is that differential geometry is designed to suppress the dependence on any particular coordinate system. Often the choice of coordinates is an artefact unrelated to the underlying problem and can prove a distraction. Working in greater generality with manifolds can therefore lead to new insights into the Euclidean setting as well. A less obvious way differential geometry (and differential topology) relates to signal processing is along the lines of Smale’s concept of topological complexity. That is to say, and roughly speaking, the geometry of a problem itself is studied, such as the problem of finding the roots of a polynomial. In the area of optimization on manifolds, this was the impetus for proposing to study not the geometry of the underlying manifold but the geometry of the class of cost functions of interest on that manifold, an approach termed Optimization Geometry. A better understanding of the geometry of a problem can be expected to lead to better algorithms for solving that problem. While differential geometry certainly has had a positive impact on signal processing, especially over the past decade, the best is surely yet to come. Spivak’s well-known introductory text to differential geometry is five volumes long, and by necessity is selective in the topics it introduces. Yet the applications of differential geometry in signal processing generally rely only on several chapters worth of knowledge. Two speculative areas come to mind where differential geometry may have a significant impact. A shorter-term prospect is the generalization of signal processing algorithms to compact manifolds. From a mathematical perspective, compact manifolds are particularly nice to work with, and are distinct from Euclidean space. The basic hope is that if highly efficient algorithms can be found for certain problems on compact manifolds — analogous to the Kalman filter for the linear filtering problem, for example — then these problems can serve as approximations

1932-4553/$31.00 © 2013 IEEE

574

IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING, VOL. 7, NO. 4, AUGUST 2013

for more general problems, just as the extended Kalman filter approximates nonlinear filtering problems by linear ones. A longer-term prospect is the use of geometry to study problems and algorithms in signal processing, as mentioned earlier in the context of topological complexity and optimization geometry. Among other things, this may lead to numerical algorithms with guaranteed performance: an a priori bound can be given on the number of operations required in order to achieve a particular accuracy. A. Special Issue The guest editors’ motivation for this special issue is to take a random sample of how differential geometry is currently being used in signal processing. While the sample is undoubtedly biased — even authors that received the call-for-papers may not have wished to submit a paper for other reasons — it is nevertheless interesting to see the breadth of applicability of differential geometry reflected in the papers appearing in this special issue. When browsing this special issue, it may prove useful to the reader to examine what “level of generality” of manifolds is considered in each paper. Particularly nice manifolds to work with are compact Lie groups. A Lie group is both a manifold and a group, and the two structures are compatible (see the Primer by Manton in this special issue for details). The special orthogonal group mentioned earlier is a compact Lie group. Being “nice” means there is an extensive theory available, including a generalization of the Fourier transform known as (non-commutative) harmonic analysis. Next come Lie groups in general. Homogeneous spaces are a generalization of Lie groups (and can be thought of as the quotient space of one Lie group quotiented by another). A special case are symmetric spaces. Then come Riemannian manifolds (and special cases of such) and finally (non-Riemannian) smooth manifolds. In each case, roughly speaking, there is less structure to work with. Usually, but not always, it is easier to start with more structure then look for generalizations, rather than working immediately with arbitrary smooth manifolds. In terms of arranging the papers in this special issue, one may have anticipated a top-level classification into deterministic or stochastic. Under deterministic could have come, for example, optimization on manifolds and the use of Riemannian geometry

for measuring distances and angles, such as for signal classification. Under stochastic, a division into information geometry, statistics on manifolds, and stochastic processes on manifolds suggests itself. Here, information geometry, roughly speaking, refers to treating probability distributions as points on a manifold. Sometimes in signal processing, information geometry is used to “visualize” iterative algorithms and suggest improvements to them. When looking over the papers in this special issue, it is interesting to see that some fall into two or more such categories. It was difficult for us to organize the papers into a small number of categories. Arranging papers in alphabetical order by surname of the lead author seemed no worse a choice than any other, and was duly adopted. With the above as guiding principles for what to look for, the reader is invited to enjoy the selection of papers in this special issue. ACKNOWLEDGMENT We thank Professor V. Krishnamurthy, the former editor-inchief, for inviting and supporting this special issue, and similarly, we thank Professor F. Pereira, the current editor-in-chief, for continuing this support and ensuring the editorial process ran smoothly. Of particular help was the administrative support expertly provided by D. Tomaro, R. Wollman and C. Skey. A special thanks to the reviewers for their timely and detailed reviews; the relentless increase each year in the number of journal paper submissions places greater burdens on reviewers. JONATHAN H. MANTON, Lead Guest Editor University of Melbourne Victoria 3010, Australia DAVID APPLEBAUM, Guest Editor University of Sheffield Sheffield S7 3RH, U.K. SHIRO IKEDA, Guest Editor The Institute of Statistical Mathematics Tokyo 190-8562, Japan NICOLAS LE BIHAN, Guest Editor Centre National de la Recherche Scientifique (CNRS) Grenoble 38402 , France

Jonathan Manton received the B.Sc. degree in mathematics and the B.Eng. degree in electrical engineering in 1995 and the Ph.D. degree in 1998, all from the University of Melbourne. He holds a Distinguished Chair at the University of Melbourne with the title Future Generation Professor. He is also an Adjunct Professor in the Mathematical Sciences Institute at the Australian National University.

INTRODUCTION TO THE ISSUE ON DIFFERENTIAL GEOMETRY IN SIGNAL PROCESSING

575

David Applebaum received an MA in pure and applied mathematics from St Andrews University in 1979 and obtained an MSc (1982) and PhD (1984) in mathematical physics from the University of Nottingham. He is currently Professor of Mathematics at the University of Sheffield. His research interests include stochastic calculus (especially processes with jumps), stochastic differential and partial differential equations and probability theory on Lie groups and manifolds.

Shiro Ikeda received the B. Eng., M. Eng., and Dr. Eng. degrees in information physics from the University of Tokyo, Tokyo, Japan, in 1991, 1993, and 1996, respectively. He has been an Associate Professor at the Institute of Statistical Mathematics, Tokyo since 2003. His research interests are in the areas of statistical signal processing, learning theory, and information geometry.

Nicolas le Bihan was born in 1974 in Morlaix, France. He received the B.Sc. degree in physics from the Université de Bretagne Occidentale (UBO, Brest), France, in 1997, and the M.Sc. and Ph.D. degrees in signal processing, respectively in 1998 and 2001, both from the Institut Polytechnique de Grenoble (Grenoble INP), France. Since 2002, he has been Chargé de Recherche at the Centre National de la Recherche Scientifique (CNRS) and is working with the Department of Images and Signals at the GIPSA-Lab (CNRS UMR 5083) in Grenoble, France. His research interests include statistical signal processing on groups and manifolds and its applications in polarized wave physics, waves in disordered media, and geophysics.