Graph Laplacians and discrete reproducing kernel

GRAPH LAPLACIANS AND DISCRETE REPRODUCING KERNEL HILBERT SPACES FROM RESTRICTIONS AND CAMERON-MARTIN

arXiv:1501.04954v1 [math.FA] 20 Jan 2015

PALLE JORGENSEN AND FENG TIAN

Abstract. We study reproducing kernel functions, and associated reproducing kernel Hilbert spaces (RKHSs) H over infinite, discrete and countable sets V . Numerical analysis builds discrete models (e.g., finite element) for the purpose of finding approximate solutions to PDE-boundary value problems; typically using multiresolution-subdivision schemes, applied to the continuous domains. In this paper, we turn the tables: our object of study is realistic infinite discrete models in their own right; and we then use an analysis of suitable continuous counterpart PDE-boundary value problems, but now serving as a tool for obtaining solutions in the discrete world.

Contents 1. Introduction 2. Preliminaries 3. Discrete RKHSs 3.1. Examples from elliptic operators 3.2. The Cameron-Martin space HCM (Ω) 4. Infinite network of resistors 5. The Discrete RKHSs H (V ) References

1 3 6 7 10 12 17 22

1. Introduction A reproducing kernel Hilbert space (RKHS) is a Hilbert space H of functions on a prescribed set, say V , with the property that point-evaluation for f ∈ H is 2000 Mathematics Subject Classification. Primary 47L60, 46N30, 46N50, 42C15, 65R10, 05C50, 05C75, 31C20; Secondary 46N20, 22E70, 31A15, 58J65, 81S25. Key words and phrases. Unbounded operators, harmonic analysis, Hilbert space, reproducing kernel Hilbert space, discrete analysis, infinite matrices, Dirichlet problem, Gaussian free fields, graph Laplacians, distribution of point-masses, Green’s function (graph Laplacians), orthogonal systems, transforms, optimization. 1

2

continuous with respect to the H -norm. They are called kernel spaces, because, for every x ∈ V , the point-evaluation for functions f ∈ H , f (x) must then be given as a H -inner product of f and a vector kx , in H ; called the kernel. The RKHSs have been studied extensively since the pioneering papers by Aronszajn in the 1940ties, see e.g., [Aro43, Aro48]. They further play an important role in the theory of partial differential operators (PDO); for example as Green’s functions of second order elliptic PDOs; see e.g., [Nel57, HKL+ 14, Trè06]. Other applications include engineering, physics, machine-learning theory (see [KH11, SZ09, CS02]), stochastic processes (e.g., Gaussian free fields), numerical analysis, and more. See, e.g., [AD93, ABDdS93, AD92, AJSV13, AJV14, BTA04]. Also, see [LB04, HQKL10, ZXZ12, LP11, Vul13, SS13, HN14]. But the literature so far has focused on the theory of kernel functions defined on continuous domains, either domains in Euclidean space, or complex domains in one or more variables. For these cases, the Dirac δx distributions do not have finite H -norm. But for RKHSs over discrete point distributions, it is reasonable to expect that the Dirac δx functions will in fact have finite H -norm. Here we consider the discrete case, i.e., RKHSs of functions defined on a prescribed countable infinite discrete set V . We are concerned with a characterization of those RKHSs H which contain the Dirac masses δx for all points x ∈ V . Of the examples and applications where this question plays an important role, we emphasize two: (i) discrete Brownian motion-Hilbert spaces, i.e., discrete versions of the Cameron-Martin Hilbert space [Fer03, HH93]; (ii) energy-Hilbert spaces corresponding to graph-Laplacians. Our setting is a given positive definite function k on V × V , where V is discrete (see above). We study the corresponding RKHS H (= H (k)) in detail. A positive definite kernel k is said to be universal [CMPY08] if, every continuous function, on a compact subset of the input space, can be uniformly approximated by sections of the kernel, i.e., by continuous functions in the RKHS. We show that for the RKHSs from kernels kc in electrical network G of resistors, this universality holds. The metric in this case is the resistance metric on the vertices of G, determined by the assignment of a conductance function c on the edges in G. The problems addressed here are motivated in part by applications to analysis on infinite weighted graphs, to stochastic processes, and to numerical analysis (discrete approximations), and to applications of RKHSs to machine learning. Readers are referred to the following papers, and the references cited there, for details regarding this: [AJS14, AJ12, AJL11, JPT15, JP14, JP11a, DG13, Kre13, ZXZ09, Nas84, NS13].

3

While the natural questions for the case of large (or infinite) networks, “the discrete world,” have counterparts in the more classical context of partial differential operators/equations (PDEs), the analysis on the discrete side is often done without reference to a continuous PDE-counterpart. The purpose of the present paper is to try to remedy this, to the extent it is possible. We begin with the discrete context (Theorem 2.5). And we proceed to show that, in the discrete case, our analysis depends on two tools, (i) positive definite (p.d.) functions, and associated RKHSs, and (ii) (resistance) metrics. Both may be studied as purely discrete objects, but nonetheless, in several of our results (including the corollaries in sections 4 and 5), we give contexts for continuous counterparts to the two discrete tools, (i) and (ii). We make precise how to use the continuous counterparts for computations in explicit discrete models, and in the associated RKHSs. In Theorems 3.11 and 5.4 we give such concrete (countable infinite) discrete models which can be understood as restrictions of analogous PDE-models. While, in traditional numerical analysis, one builds clever discrete models (finite element methods) for the purpose of finding approximate solutions to PDE-boundary value problems. They typically use multiresolution-subdivision schemes, applied to the continuous domain, subdividing into simpler discretized parts, called finite elements. And with variational methods, one then minimize various error-functions. In this paper, we turn the tables: our object of study are the discrete models, and analysis of suitable continuous PDE boundary problems serve as a tool for solutions in the discrete world.

2. Preliminaries Definition 2.1. Let V be a countable and infinite set, and F (V ) the set of all finite subsets of V . A function k : V × V → C is said to be positive definite, if XX (2.1) k (x, y) cx cy ≥ 0 (x,y)∈F ×F

holds for all coefficients {cx }x∈F ⊂ C, and all F ∈ F (V ). Definition 2.2. Fix a set V , countable infinite. (1) For all x ∈ V , set kx := k (·, x) : V → C as a function on V .

(2.2)

4

(2) Let H := H (k) be the Hilbert-completion of the span {kx : x ∈ V }, with respect to the inner product DX E XX X := cx dy k (x, y) (2.3) cx kx , dy ky H

modulo the subspace of functions of zero H -norm. H is then a reproducing kernel Hilbert space (HKRS), with the reproducing property: hkx , ϕiH = ϕ (x) , ∀x ∈ V, ∀ϕ ∈ H .

(2.4)

Note. The summations in (2.3) are all finite. Starting with finitely supported summations in (2.3), the RKHS H is then obtained by Hilbert space completion. We use physicists’ convention, so that the inner product is conjugate linear in the first variable, and linear in the second variable. (3) If F ∈ F (V ), set HF = closed span{kx }x∈F ⊂ H , (closed is automatic if F is finite.) And set PF := the orthogonal projection onto HF .

(2.5)

A reproducing kernel Hilbert space (RKHS) is a Hilbert space of functions on some set S. If S comes with a topology, it is natural to study RHHSs H consisting of continuous functions on S. Except for trivial cases, the Dirac “function”  1 y = x δx (y) = 0 y ∈ S\ {x} is not continuous, and as a result, δx will typically not be in H . We will show that, even if S is a given discrete set (countable infinite), we still often have δx ∈ / H for naturally arising RKHSs H ; see also [JT15]. Below, we shall concentrate on cases when S = V is a set of vertices in a graph G with edge-set E ⊂ V × V \ (diagonal), and on classes of RKHSs H of functions on V , for which δx ∈ H for all x ∈ V . Definition 2.3. The RKHS H (= H (k)) is said to have the discrete mass property (H is called a discrete RKHS ), if δx ∈ H , for all x ∈ V . The following is immediate. Proposition 2.4. Suppose H is a discrete RKHS of functions on a vertex-set V, as described above; then there is a unique operator ∆, with dense domain dom (∆) ⊂ H , such that (∆f ) (x) = hδx , f iH ,

for all x ∈ V , and all f ∈ dom (∆).

(2.6)

5

The operator ∆ from (2.6) will be studied in detail in Section 4 below. In special cases, it is called a graph-Laplacian; and it is a discrete analogue of ∆ = −∇2 . Given V , let F (V ) denote the set of all finite subsets of V . Theorem 2.5. Given V , k : V × V → R positive definite (p.d.). Let H (= H (k)) be the corresponding RKHS. Then the following three conditions (i)-(iii) are equivalent; x1 ∈ V is fixed: (i) δx1 ∈ H ; (ii) ∃Cx1 < ∞ such that for all F ∈ F (V ), the following estimate holds: XX |ξ (x1 )|2 ≤ Cx1 ξ (x)ξ (y) k (x, y) (2.7) F ×F

(iii) For F ∈ F (V ), set KF = (k (x, y))(x,y)∈F ×F

(2.8)

as a #F × #F matrix. Then sup F ∈F (V )


KF−1 δx1 (x1 ) < ∞.

(2.9)

Proof. (i)⇒(ii) For ξ ∈ l2 (F ), set X hξ = ξ (y) ky (·) ∈ HF . y∈F

Then hδx1 , hξ iH = ξ (x1 ) for all ξ. Since δx1 ∈ H , then by Schwarz: XX hδx , hξ i 2 ≤ kδx k2 ξ (x)ξ (y) k (x, y) . 1 1 H H

(2.10)

F ×F

But hδx1 , ky iH = δx1 ,y

 1 y = x 1 = ; hence hδx1 , hξ iH = ξ (x1 ), and so (2.10) 0 y = 6 x1

implies (2.7). (ii)⇒(iii) Recall the matrix KF := (hkx , ky i)(x,y)∈F ×F as a linear operator l2 (F ) → l2 (F ), where X (KF ϕ) (x) = KF (x, y) ϕ (y) , ϕ ∈ l2 (F ) .

(2.11)

y∈F

By (2.7), we have  ker (KF ) ⊂ ϕ ∈ l2 (F ) : ϕ (x1 ) = 0 .

(2.12)

6

Equivalently, ker (KF ) ⊂ {δx1 }⊥ and so δx1 F ∈ ker (KF )⊥ = ran (KF ), and ∃ ζ (F ) ∈ l2 (F ) s.t. X ζ (F ) (y) k (·, y). δx1 F = y∈F | {z }

(2.13)

(2.14)

=:hF

Claim. PF (δx1 ) = hF , where PF = projection onto HF . Proof of the claim. We only need to prove that δx1 − hF ∈ H HF , i.e., hδx1 − hF , kz iH = 0, ∀z ∈ F.

(2.15)

But, by (2.14), LHS(2.15) = δx1 ,z −

X

k (z, y) ζ (F ) (y) = 0.

y∈F

 Monotonicity: If F ⊂ F 0 , F, F 0 ∈ F (V ), then HF ⊂ HF 0 , and PF PF 0 = PF by easy facts for projections. Hence kPF δx1 k2H ≤ kPF 0 δx1 k2H ,

hF := PF (δx1 )

and lim kδx1 − hF kH = 0.

F %V

(iii)⇒(i) Left to the reader; see also [JT15].



3. Discrete RKHSs Given a discrete set V , #V = ℵ0 , let KV : V × V → C (or R) be p.d., and HV = H (KV ) the corresponding RKHS. We study when δx is in HV , for all x ∈ V ; see Definition (2.3). We show below (Theorem 3.11; also see Section 4) that every fundamental solution for a Dirichlet boundary value problem, bounded open domain Ω in Rν , allows for discrete restrictions, vertices sampled in Ω; which have all the desired properties. Remark 3.1. To get the desired conclusions, consider K : Ω × Ω → R continuous p.d., and {fn }n∈N an ONB for H (K) = RKHS = CM (Cameron Martin Hilbert space; see Section 3.2.) We need suitable restricting assumptions on Ω ⊂ Rν : (i) (ii) (iii) (iv)

open bounded connected smooth boundary ∂Ω

7

(Some of the restrictions may be relaxed, but even this setting is interesting enough.) The conditions (i)-(iv) are satisfied by the covariance function k (s, t) = s ∧ t − st on Ω × Ω, Ω = (0, 1) ⊂ R, for Brownian bridge (Examples 3.2 and 5.6). But we also have the covariance function k (s, t) = s ∧ t for the Brownian motion, and in this case, we may take Ω = R+ , or Ω = R; and these are examples with unbounded domain Ω ⊂ Rν . 3.1. Examples from elliptic operators Given a bounded open domain Ω ⊂ Rν , let  ν  X ∂ 2 = −∇2 ∆=− ∂xj

(3.1)

j=1

with corresponding Green’s function K, i.e., the fundamental solution to the Dirich let boundary value problem, so that K : Ω×Ω → C, ∆K (s, ·) = δs , and K (s, ·) ∂Ω ≡ 0. We assume that Ω ⊂ Rν has finite Poincaré constant CP , i.e., ( ˆ ) 2 ˆ ˆ |f |2 dx ≤ CP f dx + |∇f |2 dx , ∀f ∈ HCM (Ω) ; (3.2) Ω

Ω

Ω

see [Ami78, DL54]. Example 3.2 (Brownian bridge). ν = 1, Ω = (0, 1), and k (s, t) = s ∧ t − st;

(3.3)

i.e., the covariance function of the Brownian bridge. Proposition 3.3. It is immediate that ks (t) := k (s, t) satisfies  ks (0) = ks (1) = 0,

and

−

d dt

(3.4)

2 ks = δ (s − t) .

Proof. Direct verification, see [JT15]. Sketch: Fix s, 0 < s < 1, then  t (1 − s) 0 < t ≤ s ks (t) = s (1 − t) s < t < 1 and (see Fig 5.2)  1 − s 0 < t ≤ s d ks (t) = −s dt s0

0

ˆ

and K=

0

∞

1 P2 (dλ) = ∆−1 0 λ

(3.17)

(in general an unbounded operator.) To see that K is p.d. (see (3.14)): Step 1. If ϕ ∈ Cc∞ (Ω); then (enough to consider the real valued case) ˆ ˆ K (x, y) ϕ (x) ϕ (y) dxdy Ω Ω ˆ ∞ 1 = hϕ, Kϕi2 = kP2 (dλ) ϕk22 ≥ 0. λ 0 Step 2. Approximate (δx ) with Cc∞ (Ω).



Corollary 3.6. Let K be as in (3.11), then K satisfies that ∆K = δ (x − y) on Ω × Ω, and K (x, ·) ∂Ω ≡ 0. Proof. By well-known facts from elliptic operators, the conclusion is equivalent to (3.13). 

10

Lemma 3.7. Let k : Ω × Ω → C be a p.d. kernel, and let H = H (k) be the  corresponding RKHS. Then for every subset V ⊂ Ω, H V = f V ; f ∈ H is a  RKHS HV ; and if ϕ ∈ HV , then kϕkHV = inf kf kH ; f V = ϕ .  Proof. See [Aro43, Aro48]. Consider H NV (closed), where NV = f ∈ H ; f V = 0 . Moreover the inf is attained; f ∈ H .  Corollary 3.8. Let k, V , and H = H (k) be as above. For f ∈ H , write f = f0 + f1

(3.18)

w.r.t. the orthogonal splitting (two closed subspaces): H = NV ⊕ NV⊥ , f0 ∈ NV , f1 ∈ NV⊥ ,

(3.19)

then if ϕ ∈ HV , we have kϕkHV = kf1 kH ,

(3.20)

where ϕ = f V . Proof. With the splitting (3.18), we get f V = f1 V ; so ϕ = f = f1 , and V

V

(3.21)

kf k2H = kf0 k2H + kf1 k2H ≥ kf1 k2H so by (3.21), kϕkHV = kf1 kH .



The purpose of the next section is to study these restrictions (discrete) in detail, from cases where H is one of the classical continuous RKHSs. 3.2. The Cameron-Martin space HCM (Ω) The Cameron–Martin Hilbert space is a RKHS (abbreviated C-M below) which gives the context for the Cameron–Martin formula which describes how abstract Wiener measure changes under translation by elements of the Cameron–Martin RKHS. Context: Abstract Wiener measure is quasi-invariant (under translation), not invariant; and the C-M RKHS serves as a tool in a formula for computing of the corresponding Radon-Nikodym derivatives, the C-H formula; see e.g., [HH93]. The technical details involved vary, depending on the dimension, and on suitable boundary conditions, see below. Let Ω ⊂ Rν , satisfying conditions (i)-(iv); i.e., Ω is bounded, open, and connected in Rν with smooth boundary ∂Ω. Let K : Ω × Ω → R continuous, p.d., given as the Green’s function of ∆0 for the Dirichlet boundary condition, see (3.9). Thus, ∆0 is positive selfadjoint, and ∆K = δ (x − y) on Ω × Ω

(3.22)

11

K (x, ·) ∂Ω ≡ 0

(3.23)

see Corollary 3.6. Let HCM (Ω) be the corresponding Cameron-Martin RKHS. For ν = 1, Ω = (0, 1), take n HCM (0, 1) = f f 0 ∈ L2 (0, 1) , f (0) = f (1) = 0, ˆ 1 o 0 2 2 f dx < ∞ kf kCM :=

(3.24)

0

For ν > 1, let   ˆ 2 2 2 |∇f | dx < ∞ , HCM (Ω) = f ∇f ∈ L (Ω) , f ∂Ω ≡ 0, kf kCM := Ω   ∂ ∂ ∂ where ∇ = , ,··· , . ∂x1 ∂x2 ∂xν

(3.25)

Remark 3.9. In the case of Ω = (0, 1), ν = 1, and for K (s, t) = s ∧ t − st, we have HCM (0, 1) as in (3.24). The following decomposition holds: K (s, t) =

∞ X sin (nπs) sin (nπt)

(nπ)2

n=1

,

(s, t) ∈ Ω × Ω.

Proof. Use Fourier series; or the fact that   sin (nπt) ∞ nπ n=1 is an ONB in HCM (0, 1).



In general, ν > 1, there exists ONB {fn }n∈N in HCM (Ω) (see (3.25)), such that ∞ X

K (x, y) =

fn (x) fn (y) on Ω × Ω.

n=1



Proof. A result from the theory of RKHS.

Lemma 3.10 (reproducing property). Let K be the kernel of ∆0 for the Dirichelet boundary condition; and let HCM (Ω) be the Cameron-Martin space in (3.25). Then hKx , f iCM = f (x) , Proof. Note that LHS(3.26)

∀f ∈ HCM , ∀x ∈ Ω.

ˆ =

(∇Kx ) (∇f ) dy ˆ
− ∇2 Kx f (y) dy Ω

=

Ω

(by (3.23),(3.25))

(3.26)

12

ˆ ∆Kx f (y) dy

= (by (3.22))

Ω

ˆ

δ (x − y) f (y) dy = f (x) ;

= (by (3.22))

Ω

where dy = dy1 dy2 · · · dyν denotes the Lebesgue measure in Ω ⊂ Rν .



We show below that: Theorem 3.11. Then (1) Discrete case: Fix V ⊂ Ω, #V = ℵ0 , where V = {xj }∞ j=1 , xj ∈ Ω. Let H (V ) = RKHS of KV := K V ×V ; then δxj ∈ H (V ). (2) Continuous case; contrast: Kx ∈ HCM (V ), but δx ∈ / HCM (Ω), x ∈ Ω. The proof will be given in the next section. To see that δx ∈ / HCM (Ω), we use (3.24) when ν = 1, and (3.25) when ν > 1. In general, by elliptic regularity, HCM (Ω) is a RKHS of continuous functions; and δx is not a function, so not in HCM (Ω). But the RKHS of KV := K V ×V is a discrete RKHS, and δx ∈ H (V ); proof below. 4. Infinite network of resistors Here we introduce a family of positive definite kernels k : V × V → R, defined on infinite sets V of vertices for a given graph G = (V, E) with edges E ⊂ V × V \(diagonal). There is a large literature dealing with analysis on infinite graphs; see e.g., [JP10, JP11b, JP13]; see also [OS05, BCF+ 07, CJ11]. Our main purpose here is to point out that every assignment of resistors on the edges E in G yields a p.d. kernel k, and an associated RKHS H = H (k) such that δx ∈ H , for all x ∈ V . (4.1) Definition 4.1. Let G = (V, E) be as above. Assume 1. (x, y) ∈ E ⇐⇒ (y, x) ∈ E; 2. ∃c : E → R+ (a conductance function = 1 / resistance) such that (i) c(xy) = c(yx) , ∀ (xy) ∈ E;  (ii) for all x ∈ V , # y ∈ V | c(xy) > 0 < ∞; and (iii) ∃o ∈ V s.t. for ∀x ∈ V \ {o}, ∃ edges (xi , xi+1 )0n−1 ∈ E s.t. xo = 0, and xn = x; called connectedness.

13

Figure 4.1. Examples of configuration of resistors in a network.

1

1

Figure 4.2. Example of configuration of resistors in a “large” network. Given G = (V, E), and a fixed conductance function c : E → R+ as specified above, we now define a corresponding Laplace operator ∆ = ∆(c) acting on functions on V , i.e., on F unc (V ) by X (∆f ) (x) = cxy (f (x) − f (y)) . (4.2) y∼x 1 See Fig 4.1-4.2 for examples of networks of resistors: cxy = res(x,y) , if (x, y) ∈ E. Let H be the Hilbert space defined as follows: A function f on V is in H iff f (o) = 0, and 1X X cxy |f (x) − f (y)|2 < ∞. (4.3) kf k2H := 2 (x,y)∈E ⊂V ×V

Lemma 4.2 ([JP10]). For all x ∈ V \ {o}, ∃vx ∈ H s.t. f (x) − f (o) = hvx , f iH ,

∀f ∈ H

1

(4.4)

14

where hh, f iH =

 1X X  cxy h (x) − h (y) (f (x) − f (y)) , 2

∀h, f ∈ H .

(4.5)

(x,y)∈E

(The system {vx } is called a system of dipoles.) Proof. Let x ∈ V \ {o}, and use (4.2) together with the Schwarz-inequality to show that X 1 X |f (x) − f (o)|2 ≤ cxi xi+1 |f (xi ) − f (xi+1 )|2 . cxi xi+1 i

i

An application of Riesz’ lemma then yields the desired conclusion. (c) Note that vx = vx in (4.4) depends on the choice of base point o ∈ V , and on conductance function c; see (i)-(ii) and (4.3). 

The resistance metric R(c) (x, y) = res (x, y) is as follows: o n 2 2 (c) R (x, y) = inf K |f (x) − f (y)| ≤ K kf kH , ∀f ∈ H . Now set k (c) (x, y) = hvx , vy iH ,

∀ (xy) ∈ (V \ {o}) × (V \ {o}) .

(4.6)

It follows from a theorem that k (c) is a Green’s function for the Laplacian ∆(c) in the sense that ∆(c) k (c) (x, ·) = δx (4.7) where the dot in (4.7) is the dummy-variable in the action. (Note that the solution to (4.7) is not unique.) Finally, we note that ∆vx = δx − δo ,

∀x ∈ V \ {o} .

(4.8)

And (4.8) in turn follows from (4.4), (4.2) and a straightforward computation. Corollary 4.3. Let G = (V, E) and conductance c : E → R+ be as specified above. Let ∆ = ∆(c) be the corresponding Laplace operator. Let H = H (k c ) be the RKHS. Then hδx , f iH = (∆f ) (x) (4.9) and δx = c (x) vx −

X y∼x

holds for all x ∈ V .

cxy vy

(4.10)

15

Proof. Since the system {vx } of dipoles (see (4.4)) span a dense subspace in H , it is enough to verify (4.9) when f = vy for y ∈ V \ {o}. But in this case, (4.9) follows from (4.7) and a direct calculation. (For details, see [JT15].)  Corollary 4.4. Let G = (V, E), and conductance c : E → R+ be as before; let (c) ∆(c) be the Laplace operator, and HE the energy-Hilbert space in Definition 4.1 (see (4.3)). Let k (c) (x, y) = hvx , vy iHE be the kernel from (4.6), i.e., the Green’s

function of ∆(c) . Then the two Hilbert spaces HE , and H k (c) = RKHS k (c) , (c)

(c)

are naturally isometrically isomorphic via vx 7−→ kx where kx = k (c) (x, ·) for all x∈V. Proof. Let F ∈ F (V ), and let ξ be a function on F ; then

X

2 XX

ξ (x)ξ (y) k (c) (x, y) ξ (x) kx(c) =

x∈F H (k(c) ) F ×F XX = ξ (x)ξ (y) hvx , vy iHE (4.6)

=

F ×F

X

x∈F

2

ξ (x) vx

HE

.

The remaining steps in the proof of the Corollary now follow from the standard completion from dense subspaces in the respective two Hilbert spaces HE and
H k (c) .  In the following we show how the kernels k (c) : V ×V → R from (4.6) in Lemma 4.2 are related to metrics on V ; so called resistance metrics (see, e.g., [JP10, AJSV13].) Corollary 4.5. Let G = (V, E), and conductance c : E → R+ be as above; and let k (c) (x, y) := hvx , vy iHE be the corresponding Green’s function for the graph Laplacian ∆(c) .
Then there is a metric R = R(c) = the resistance metric , such that R(c) (o, x) + R(c) (o, y) − R(c) (x, y) 2 holds on V × V . Here the base-point o ∈ V is chosen and fixed s.t. k (c) (x, y) =

hvx , f iHE = f (x) − f (o) ,

∀f ∈ HE , ∀x ∈ V.

(4.11)

(4.12)

Proof. See [JP10]. Set R(c) (x, y) = kvx − vy k2HE .

(4.13)

We proved in [JP10] that R(c) (x, y) in (4.13) indeed defines a metric on V ; the so called resistance metric. It represents the voltage-drop from x to y when 1 Amp is fed into (G, c) at the point x, and then extracted at y.

16

The verification of (4.11) is now an easy computation, as follows:

=

R(c) (o, x) + R(c) (o, y) − R(c) (x, y) 2 2 2 kvx kHE + kvy kHE − kvx − vy k2HE 2

= hvx , vy iHE = k (c) (x, y)

(by (4.6)). 

Corollary 4.6. The functions R(c) (·, ·) which arise as in (4.11) and (4.13) are conditionally negative definite, i.e., for all finite subsets F ⊂ V and functions ξ on P F , such that x∈F ξx = 0, we have: XX ξ x ξy R(c) (x, y) ≤ 0. (4.14) F

F

(Note that (4.14) follows from (4.13).) Moreover, if R(c) arises as a restriction of a metric on Rν , then there is a quadratic form Q on Rν (possibly Q = 0), and a positive measure µ on Rν such that ˆ 1 − cos (x · ξ) (c) R (o, x) = Q (x) + dµ (ξ) , (4.15) |ξ|2 Rν ˆ

where

Rν

dµ (ξ) < ∞. 1 + |ξ|2

(4.16)

Proof. For details on this last point, see for example [AJV14] and [BTA04].



Proposition 4.7. In the two cases: (i) B (t), Brownian motion on 0 < t < ∞; and (ii) the Brownian bridge Bbri (t), 0 < t < 1, (see Fig 4.3) the corresponding resistance metric R is as follows: (i) If V = {xi }∞ i=1 ⊂ (0, ∞), x1 < x2 < · · · , then (V )

RB (xi , xj ) = |xi − xj | .

(4.17)

(ii) If W = {xi }∞ i=1 ⊂ (0, 1), 0 < x1 < x2 < · · · < 1, then (W )

Rbridge (xi , xj ) = |xi − xj | · (1 − |xi − xj |) . (W )

(4.18)

In the completion w.r.t. the resistance metric Rbridge , the two endpoints x = 0 and x = 1 are identified; see also Fig 4.3.

17 0.10

0.05

0.2

0.4

0.6

0.8

1.0

-0.05

Figure 4.3. Brownian bridge Bbri (t), a simulation of three sample paths of the Brownian bridge. The Brownian bridge Bbri (t) is realized on a probability space Ω (' C ([0, 1])) such that Bbri (0) = Bbri (1) = 0, and E (Bbri (s) Bbri (t)) = s ∧ t − st = kB (s, t) ,

(4.19)

E (B (s) B (t)) = s ∧ t;

(4.20)

´ where E (· · · ) = Ω · · · dP , and P denotes Wiener measure on Ω. If {B (t)}t∈∈[0,1] denotes the usual Brownian motion, B (0) = 0, with covariance

then we may take for Bbri (t) as follows:   t Bbri (t) = (1 − t) B , 1−t

∀t ∈ (0, 1) .

(4.21)

5. The Discrete RKHSs H (V ) Let V , K be as above. To get that δxi ∈ H (V ), we may specify a graph G = (V, E) with vertices V and edges E ⊂ V × V \ {diagonal}, and assume that, for all xi ∈ V ,  # xj xi ∼ xj < ∞, ∀xi ∈ V (5.1) (finite neighborhood set); see Fig 5.1. Fix a base-point o ∈ V . Set HE := all functions f on V s.t. f (o) = 0, with 1 XX 1 kf k2E = |f (xi ) − f (xj )|2 ; 2 res (i, j) i

j

where res (i, j) denotes resistance between xi and xj , and cij :=

1 res (i, j)

is the conductance. Then HE (G) is a RKHS for the graph G = (V, E), i.e., the energy Hilbert space.

18

x

Figure 5.1. ∀i, # {j s.t. (xi xj ) ∈ E} < ∞, every vertex xi has a finite set of neighbors. Note that e x := K (xi , ·) ∈ HE (G) K i V

and D E ex , f K i

HE

= f (xi ) ,

∀xi ∈ V, ∀f ∈ HE (G) .

e x , defines a Lemma 5.1. The mapping PG : CM (Ω) → HE (G), Kxi 7−→ K i projection of the Cameron-Martin space (see (3.25)) onto HE (G). e x = Kx , is an isometry. In fact, Proof. Note that JV : HE (G) → CM (Ω), JV K j j we have that

X

2

X

2

ex ξj K = ξ K ;

j xj j j j HE (G) CM (Ω) ´  where kf k2CM = Ω |∇f |2 dx, for all f ∈ CM (Ω). Now, PG = JV JV∗ . Corollary 5.2. CM (Ω) HE (G) = {f ∈ CM (Ω) ; f (xj ) = 0, ∀xj ∈ V } (See also Lemma 3.7.) Proof. The same as in the proof for the case of Brownian bridge: D E ex , f JK = f (xj ) , ∀f ∈ CM (Ω) . j CM (Ω)



The desired result follows from this. Definition 5.3. Set res (xi , xj ) = R(c) (xi , xj )

2 = Kxi − Kxj H

(5.2) E

= K (xi , xi ) + K (xj , xj ) − 2K (xi , xj )

19

(We proved in [JP10] that res (xi , xj ) in (5.2) indeed defines a metric on V ; the so called resistance metric.) Let X 1 (f (xi ) − f (xj )) (∆V f ) (xi ) := res (x , x ) i j x ∼x j

i

be the graph-Laplacian; where xj ∼ xj iff (xi xj ) ∈ E. Theorem 5.4. Let V ⊂ Ω and ∆0 , K, H (V ) be as above; assume (5.1), i.e., finite neighbor in G. Then δxi ∈ H (V ), and (∆V f ) (xi ) = hδxi , f iH (V ) ,

∀f ∈ H (V ) .

Proof. Follows from earlier analysis. One shows that ! X X 1 1 δxi = Kxi − Kx ∈ H (V ) . res (xi , xl ) res (xi , xj ) j x ∼x x ∼x l

j

i

i

(It is a finite sum based on the assumption (5.1).)



Remark 5.5. Let Ω ⊂ Rν as above. If ν > 1, then the kernel K (s, t), (s, t) ∈ Ω × Ω, has a singularity at x = y, by contrast to ν = 1 (see below.) But we can still construct discrete graph Laplacians. Fix Ω, and let K be the kernel s.t. K (x, ·) ∂Ω ≡ 0, ∀x ∈ Ω, ∆K = δ (s, y) = δ (x − y) ,

(x, y) ∈ Ω × Ω

where ∆ = −∇2 as a s.a. operator on L2 (Ω), with  dom (∆) = f ∈ L2 (Ω) ∆f ∈ L2 (Ω) , f ∂Ω = 0 . So K satisfies the Dirichlet boundary condition, but K has a singularity at x = y, i.e., at the diagonal of Ω × Ω if ν > 1. Fix V = {xi }∞ ⊂ Ω, discrete. Fix E ⊂ V × V \ (diagonal) s.t. ∀xi ∈ V ,  1 # j xj ∼ xi < ∞, finite neighbor sets. Set X 1 (∆V f ) (xi ) = (f (xi ) − f (xj )) res (xi , xj ) x j xj ∼xj

and get the corresponding energy-Hilbert space     XX 1 |f (xi ) − f (xj )|2 < ∞ . HE (V ) = f on V   res (xi , xj ) i

j

20 ks

s(1 - s)

s

1

s

1

t

d (ks ) dt

1-s

-s

t

Figure 5.2. ν = 1, ks (t) = k (s, t).

Example 5.6. For ν = 1, let k = s ∧ t − st (5.3)

ks (t) := k (s, t) . Then, ks0 (t) = (1 − s) χ[0,s] (t) − sχ[s,1] (t) −ks00 (t) = δs (t) = δs,t P  2 For ν > 1, set ∆ = −∇2 = ν1 ∂x∂ j , where   ∂ ∂ ∇ = grad = ,··· , . ∂x1 ∂xν

(5.4)

(5.5)

Set Cν =

 

1 2π  ν−2 |S ν−1 |

ν=2

ν>2  Ω = x ∈ Rν |x| < 1 n Xν S ν−1 = x ∈ Rν |x|2 =

j=1

(5.6) o x2j = 1

21

x*

D=Ω

x 0

∂Ω = S1

 D = Ω = x ∈ Rν |x| < 1  ∂Ω = S 1 = x ∈ Rν |x| = 1 Figure 5.3. The case of ν = 2. x 7→ x∗ , reflection around S 1 .

and, ∀ (x, y) ∈ Ω × Ω,

K (x, y) =

 n 1  − C    ν |x−y|ν−2

1 (|x||x∗ −y|)ν−2

   



1 2π

log



|x||x∗ −y| |x−y|

o

ν>2 (5.7) ν=2

where in (5.7), we set x∗ :=

 

x |x|2

∞

x ∈ Ω\ {0} x = 0.

For ν = 2, see Fig 5.3. Special case of  K = G − Poisson G (x, ·)



∂Ω

where

 − 1 log |x − y| ν = 2 2π G (x, y) = (Newton potential) 1 Cν ν > 2. ν−2 |x−y|

(5.8)

(5.9)

Theorem 5.7. For Ω ⊂ Rν , bounded open, smooth boundary ∂Ω, let m∂Ω be the surface measure (see [Trè06]), then ∆0 with boundary condition f ∂Ω = 0 with  dom∆0 = f ∈ L2 (Ω) ∆f ∈ L2 (Ω) , f ∂Ω = 0 is s.a., and with K as in (5.10), ∆K = δ (s − y) , with K (x, ·) ∂Ω = 0, ∀x ∈ Ω.

(x, y) ∈ Ω × Ω

22

Proof. For x, y ∈ Ω, let Py be the Poisson kernel, and set i h K (x, y) = G (x, y) − Py G (x, ·) ∂Ω

(5.10)

view G (x, ·) as a function on the boundary ∂Ω. We have i ˆ h Py (b) G (x, b) dm∂Ω (b) , Py G (x, ·) = ∂Ω

∂Ω

where dm∂Ω (·) denotes the standard measure on the boundary ∂Ω of Ω; see [Trè06]. Then i h Ω 3 y −→ Py G (x, ·) ∂Ω i h is harmonic in Ω, and limy→b Py G (x, ·) = G (x, b). So, from (5.10), ∂Ω

K (x, b) = 0,

∀x ∈ Ω, b ∈ ∂Ω,

and so K in (5.10) is the Dirichlet kernel.



Acknowledgement. The co-authors thank the following colleagues for helpful and enlightening discussions: Professors Daniel Alpay, Sergii Bezuglyi, Ilwoo Cho, Ka Sing Lau, Paul Muhly, Myung-Sin Song, Wayne Polyzou, Gestur Olafsson, Keri Kornelson, and members in the Math Physics seminar at the University of Iowa. References [ABDdS93] Daniel Alpay, Vladimir Bolotnikov, Aad Dijksma, and Henk de Snoo, On some operator colligations and associated reproducing kernel Hilbert spaces, Operator extensions, interpolation of functions and related topics (Timişoara, 1992), Oper. Theory Adv. Appl., vol. 61, Birkhäuser, Basel, 1993, pp. 1–27. MR 1246577 (94i:47018) [AD92] Daniel Alpay and Harry Dym, On reproducing kernel spaces, the Schur algorithm, and interpolation in a general class of domains, Operator theory and complex analysis (Sapporo, 1991), Oper. Theory Adv. Appl., vol. 59, Birkhäuser, Basel, 1992, pp. 30–77. MR 1246809 (94j:46034) , On a new class of structured reproducing kernel spaces, J. Funct. Anal. 111 [AD93] (1993), no. 1, 1–28. MR 1200633 (94g:46035) [AJ12] Daniel Alpay and Palle E. T. Jorgensen, Stochastic processes induced by singular operators, Numer. Funct. Anal. Optim. 33 (2012), no. 7-9, 708–735. MR 2966130 [AJL11] Daniel Alpay, Palle Jorgensen, and David Levanony, A class of Gaussian processes with fractional spectral measures, J. Funct. Anal. 261 (2011), no. 2, 507–541. MR 2793121 (2012e:60101) [AJS14] Daniel Alpay, Palle Jorgensen, and Guy Salomon, On free stochastic processes and their derivatives, Stochastic Process. Appl. 124 (2014), no. 10, 3392–3411. MR 3231624 [AJSV13] Daniel Alpay, Palle Jorgensen, Ron Seager, and Dan Volok, On discrete analytic functions: products, rational functions and reproducing kernels, J. Appl. Math. Comput. 41 (2013), no. 1-2, 393–426. MR 3017129

23

[AJV14]

Daniel Alpay, Palle Jorgensen, and Dan Volok, Relative reproducing kernel Hilbert spaces, Proc. Amer. Math. Soc. 142 (2014), no. 11, 3889–3895. MR 3251728 [Ami78] Charles J. Amick, Some remarks on Rellich’s theorem and the Poincaré inequality, J. London Math. Soc. (2) 18 (1978), no. 1, 81–93. MR 502660 (80a:46016) [Aro43] N. Aronszajn, La théorie des noyaux reproduisants et ses applications. I, Proc. Cambridge Philos. Soc. 39 (1943), 133–153. MR 0008639 (5,38e) , Reproducing and pseudo-reproducing kernels and their application to the par[Aro48] tial differential equations of physics, Studies in partial differential equations. Technical report 5, preliminary note, Harvard University, Graduate School of Engineering., 1948. MR 0031663 (11,187b) [BCF+ 07] Brighid Boyle, Kristin Cekala, David Ferrone, Neil Rifkin, and Alexander Teplyaev, Electrical resistance of N -gasket fractal networks, Pacific J. Math. 233 (2007), no. 1, 15–40. MR 2366367 (2010d:28006) [BTA04] Alain Berlinet and Christine Thomas-Agnan, Reproducing kernel Hilbert spaces in probability and statistics, Kluwer Academic Publishers, Boston, MA, 2004, With a preface by Persi Diaconis. MR 2239907 (2007b:62006) [CJ11] Ilwoo Cho and Palle E. T. Jorgensen, Free probability induced by electric resistance networks on energy Hilbert spaces, Opuscula Math. 31 (2011), no. 4, 549–598. MR 2823480 (2012h:05341) [CMPY08] Andrea Caponnetto, Charles A. Micchelli, Massimiliano Pontil, and Yiming Ying, Universal multi-task kernels, J. Mach. Learn. Res. 9 (2008), 1615–1646. MR 2426053 (2010b:68130) [CS02] Felipe Cucker and Steve Smale, On the mathematical foundations of learning, Bull. Amer. Math. Soc. (N.S.) 39 (2002), no. 1, 1–49 (electronic). MR 1864085 (2003a:68118) [DG13] Marta D’Elia and Max Gunzburger, The fractional Laplacian operator on bounded domains as a special case of the nonlocal diffusion operator, Comput. Math. Appl. 66 (2013), no. 7, 1245–1260. MR 3096457 [DL54] J. Deny and J. L. Lions, Les espaces du type de Beppo Levi, Ann. Inst. Fourier, Grenoble 5 (1953–54), 305–370 (1955). MR 0074787 (17,646a) [Fer03] Xavier Fernique, Extension du théorème de Cameron-Martin aux translations aléatoires, Ann. Probab. 31 (2003), no. 3, 1296–1304. MR 1988473 (2004f:60087) [HH93] Takeyuki Hida and Masuyuki Hitsuda, Gaussian processes, Translations of Mathematical Monographs, vol. 120, American Mathematical Society, Providence, RI, 1993, Translated from the 1976 Japanese original by the authors. MR 1216518 (95j:60057) + [HKL 14] S. Haeseler, M. Keller, D. Lenz, J. Masamune, and M. Schmidt, Global properties of Dirichlet forms in terms of Green’s formula, ArXiv e-prints (2014). [HN14] Haakan Hedenmalm and Pekka J. Nieminen, The Gaussian free field and Hadamard’s variational formula, Probab. Theory Related Fields 159 (2014), no. 1-2, 61–73. MR 3201917 [HQKL10] Minh Ha Quang, Sung Ha Kang, and Triet M. Le, Image and video colorization using vector-valued reproducing kernel Hilbert spaces, J. Math. Imaging Vision 37 (2010), no. 1, 49–65. MR 2607639 (2011k:94032)

24

[JP10]

[JP11a] [JP11b]

[JP13] [JP14] [JPT15] [JT15] [KH11]

[Kre13]

[LB04]

[LP11] [Nas84]

[Nel57] [NS13] [OS05] [SS13] [SZ09] [Trè06]

Palle E. T. Jorgensen and Erin Peter James Pearse, A Hilbert space approach to effective resistance metric, Complex Anal. Oper. Theory 4 (2010), no. 4, 975–1013. MR 2735315 (2011j:05338) Palle E. T. Jorgensen and Erin P. J. Pearse, Gel0 fand triples and boundaries of infinite networks, New York J. Math. 17 (2011), 745–781. MR 2862151 (2012k:05233) , Resistance boundaries of infinite networks, Random walks, boundaries and spectra, Progr. Probab., vol. 64, Birkhäuser/Springer Basel AG, Basel, 2011, pp. 111– 142. MR 3051696 , A discrete Gauss-Green identity for unbounded Laplace operators, and the transience of random walks, Israel J. Math. 196 (2013), no. 1, 113–160. MR 3096586 , Spectral comparisons between networks with different conductance functions, J. Operator Theory 72 (2014), no. 1, 71–86. MR 3246982 Palle Jorgensen, Steen Pedersen, and Feng Tian, Spectral theory of multiple intervals, Trans. Amer. Math. Soc. 367 (2015), no. 3, 1671–1735. MR 3286496 P. Jorgensen and F. Tian, Discrete reproducing kernel Hilbert spaces: Sampling and distribution of Dirac-masses, ArXiv e-prints (2015). Sanjeev Kulkarni and Gilbert Harman, An elementary introduction to statistical learning theory, Wiley Series in Probability and Statistics, John Wiley & Sons, Inc., Hoboken, NJ, 2011. MR 2908346 Christian Kreuzer, Reliable and efficient a posteriori error estimates for finite element approximations of the parabolic p-Laplacian, Calcolo 50 (2013), no. 2, 79–110. MR 3049934 Yi Lin and Lawrence D. Brown, Statistical properties of the method of regularization with periodic Gaussian reproducing kernel, Ann. Statist. 32 (2004), no. 4, 1723–1743. MR 2089140 (2006a:62053) Sneh Lata and Vern Paulsen, The Feichtinger conjecture and reproducing kernel Hilbert spaces, Indiana Univ. Math. J. 60 (2011), no. 4, 1303–1317. MR 2975345 Z. Nashed, Operator parts and generalized inverses of linear manifolds with applications, Trends in theory and practice of nonlinear differential equations (Arlington, Tex., 1982), Lecture Notes in Pure and Appl. Math., vol. 90, Dekker, New York, 1984, pp. 395–412. MR 741527 (85f:47002) Edward Nelson, Kernel functions and eigenfunction expansions, Duke Math. J. 25 (1957), 15–27. MR 0091442 (19,969f) M. Zuhair Nashed and Qiyu Sun, Function spaces for sampling expansions, Multiscale signal analysis and modeling, Springer, New York, 2013, pp. 81–104. MR 3024465 Kasso A. Okoudjou and Robert S. Strichartz, Weak uncertainty principles on fractals, J. Fourier Anal. Appl. 11 (2005), no. 3, 315–331. MR 2167172 (2006f:28011) Oded Schramm and Scott Sheffield, A contour line of the continuum Gaussian free field, Probab. Theory Related Fields 157 (2013), no. 1-2, 47–80. MR 3101840 Steve Smale and Ding-Xuan Zhou, Online learning with Markov sampling, Anal. Appl. (Singap.) 7 (2009), no. 1, 87–113. MR 2488871 (2010i:60021) François Trèves, Basic linear partial differential equations, Dover Publications, Inc., Mineola, NY, 2006, Reprint of the 1975 original. MR 2301309 (2007k:35004)

25

[Vul13]

[ZXZ09]

[ZXZ12]

Mirjana Vuletić, The Gaussian free field and strict plane partitions, 25th International Conference on Formal Power Series and Algebraic Combinatorics (FPSAC 2013), Discrete Math. Theor. Comput. Sci. Proc., AS, Assoc. Discrete Math. Theor. Comput. Sci., Nancy, 2013, pp. 1041–1052. MR 3091062 Haizhang Zhang, Yuesheng Xu, and Jun Zhang, Reproducing kernel Banach spaces for machine learning, J. Mach. Learn. Res. 10 (2009), 2741–2775. MR 2579912 (2011c:62219) Haizhang Zhang, Yuesheng Xu, and Qinghui Zhang, Refinement of operator-valued reproducing kernels, J. Mach. Learn. Res. 13 (2012), 91–136. MR 2913695

(Palle E.T. Jorgensen) Department of Mathematics, The University of Iowa, Iowa City, IA 52242-1419, U.S.A. E-mail address: [email protected] URL: http://www.math.uiowa.edu/~jorgen/ (Feng Tian) Department of Mathematics, Wright State University, Dayton, OH 45435, U.S.A. E-mail address: [email protected] URL: http://www.wright.edu/~feng.tian/