Calculation of intrinsic and parameter-effects curvatures for nonlinear ...

Communications in Statistics - Simulation and Computation

ISSN: 0361-0918 (Print) 1532-4141 (Online) Journal homepage: http://www.tandfonline.com/loi/lssp20

Calculation of intrinsic and parameter-effects curvatures for nonlinear regression models Douglas M. Bates , David C. Hamilton & Donald G. Watts To cite this article: Douglas M. Bates , David C. Hamilton & Donald G. Watts (1983) Calculation of intrinsic and parameter-effects curvatures for nonlinear regression models, Communications in Statistics - Simulation and Computation, 12:4, 469-477, DOI: 10.1080/03610918308812333 To link to this article: http://dx.doi.org/10.1080/03610918308812333

Published online: 27 Jun 2007.

Submit your article to this journal

Article views: 40

View related articles

Citing articles: 9 View citing articles

Full Terms & Conditions of access and use can be found at http://www.tandfonline.com/action/journalInformation?journalCode=lssp20 Download by: [Dalhousie University]

Date: 11 September 2015, At: 04:20

COMMUN. STATIST.-SIMULA.

COMPUTA.,

1 2 ( 4 ) , 469-477

(1983)

CALCULATION OF INTRINSIC AND PARAMETER-EFFECTS CURVATURES FOR NONLINEAR REGRESSION MODELS

D o u g l a s M. B a t e s

David C. H a m i l t o n

Department of S t a t i s t i c s U n i v e r s i t y of Wisc., Madison

D e p t . o f Math., S t a t . & Comp. S c i . Dalhousie University, Halifax

Donald G. W a t t s D e p a r t m e n t of M a t h e m a t i c s a n d S t a t i s t i c s , Queen's U n i v e r s i t y , K i n g s t o n

Downloaded by [Dalhousie University] at 04:20 11 September 2015

Key Words and Phrases: orthogonal vectors; rotation o f coordinates; basis for normal acceleration space.

PURPOSE AND DESCRIPTION Purpose: T h i s a l g o r i t h m u s e s f i r s t a n d s e c o n d d e r i v a t i v e s of t h e model f u n c t i o n w i t h r e s p e c t t o t h e model p a r a m e t e r s a t t h e l e a s t s q u a r e s estimates

to

calculate

curvature arrays design

-

data

is

Watts,

&

p a r a m e t e r - e f f ects

set c o m b i n a t i o n .

parameter-effects vector

the

associated with

curvatures

are

a particular

calculated

and

B employed

-

the in

residual (Hamilton,

B a t e s , 1 9 8 2 ) t o compensate f o r t h e e f f e c t s of i n t r i n s i c 'Ihe ( s c a l e d )

m a t r i x R f r o m a QR d e c o w o s i t i o n

t h e f i r s t d e r i v a t i v e m a t r i x and t h e i n v e r s e they

may b e s t o r e d and

transformation 1981).

intrinsic

Root mean s q u a r e i n t r i n s i c and

used t o produce t h e m t r i x

nonlinearity.

that

and

n o n l i n e a r model

of

parameters

of R a r e

used t o evaluate

as

A s u g g e s t i o n of ( H a m i l t o n ,

described i n 1980)

is

of

returned so

t h e e f f i c t s of (Bates &

a

Watts,

used t o reduce t h e

d i m e n s i o n of t h e i n t r i n s i c c u r v a t u r e a r r a y and t o s a v e s t o r a g e a n d t ime

.

469 Copyright @ 1983 by Marcel Dekker, Inc.

0361-0918/83/1204-0469$3.50/0

BATES, HAMILTON, AND WATTS

470 Theory and Method: The

general

theory

and

geometric

curvature a r r a y s is given i n (Bates & (Hamilton,

l98O),

interpretation

Watts,

1980).

of

the

A s noted i n

a r e a t ms t p ( p + 1 ) / 2 i n d e p e n d e n t second

there

d e r i v a t i v e v e c t o r s i n a model w i t h p p a r a m e t e r s ,

s o t h e dimension

of t h e normal a c c e l e r a t i o n s p a c e of t h e s o l u t i o n

l o c u s i s a t most

By d e t e r m i n i n g a b a s i s f o r t h e components of t h e s e c o n d

p(p+1)/2. derivative

vectors

orthogonal

to

the

first

derivatives

and

it i s possible t o express t h e

rotating coordinates appropriately,

i n t r i n s i c c u r v a t u r e a r r a y v e r y compactly. The method of Watts,

(Bates 6 matrix

calculating t h e curvature arrays

described i n

1980) i n v o l v e d f o r m i n g a QR d e c o m p o s i t i o n of t h e

of f i r s t d e r i v a t i v e s a n d ,

.

d e r i v a t i v e v e c t o r s by Q'

later,

pre-multiplying

second

These two s t e p s and t h e f u r t h e r s t e p

of d e t e r m i n i n g a b a s i s f o r t h e normal a c c e l e r a t i o n s p a c e a r e


simply

accomplished

decomposition derivative assume

in

this

and t h e non-redundant

the f i r s t derivative

require

algorithm

of a n augmented m a t r i x ,

that

t h e augmented

D

second

,

by

forming

i n t e r c h a n g e s of t h e s e c o n d

al.,

&,

derivative vectors.

matrix

is

non-singular

matrix

be

non-singular.

b u t do

not

To g e t a

we a l l o w

column

r o u t i n e from LINPACK (Dongarra e t

b e g i n by

w h e r e s2

is of

d i v i d i n g t h e r e s i d u a l and d e r i v a t i v e v e c t o r s by t h e r e s i d u a l mean s q u a r e . t h e matrix

D

,

We

the f i r s t p

contain the f i r s t derivative vectors

xi,

t h e n form a QR

columns of

i = 1, 2,

..., p,

r e m a i n i n g p ( p + 1 ) / 2 columns, t h e s e c o n d d e r i v a t i v e v e c t o r s 1, 2 ,

W e

d e r i v a t i v e s columns u s i n g t h e p i v o t i n g

t h e QR d e c o m p o s i t i o n

decomposition

is,

QR

1979). We

s

a

composed of t h e f i r s t

u n i q u e QR d e c o m p o s i t i o n w i t h R u p p e r t r a p e z o i d a l

scheme i n

more

..., p , in

j = i, i+I,

the

order

decomposition,

a * . ,

which and t h e

4'

p , i n symmetric s t o r a g e mode;

V 1 1 , V 1 2 ~ ~ 2 , V 1 3 * X 2" 3 ~' "V -PP

.

i =

that This

D = QRE

produces t h e n

by

n

orthogonal

matrix

Q

(n i s

t h e number o f

'

471

CURVATURES FOR NONLINEAR REGRESSION MODELS E,

observations),

a p(p+3)/2 s q u a r e permutation

t h e u p p e r l e f t p by p s u b m a t r i x i s t h e i d e n t i t y , trapezoidal

matrix

with

t h e u p p e r l e f t p by

m a t r i x i n which and R ,

a n upper

p submatrix

upper

triangular.

R

Partitioning

as

which i s u s e d i n c o m b i n a t i o n w i t h R t o produce t h e 11' 12 p a r a m e t e r e f f e c t s c u r v a t u r e a r r a y , and w i t h R t o produce t h e

provides R

2

i n t r i n s i c curvature array.

R I I i s t h e same a s R = L~

i n ( B a t e s I.

W a t t s , 1980).


To c a l c u l a t e t h e c u r v a t u r e a r r a y s , t h r e e dimensional a r r a y

and p o s t - m u l t i p l i e d

effected

by

L

.

These

two

Finally

the

by

multiplications are

of

the last

p(p+1)/2

a m a t r i x composed of p r o d u c t s of e l e m e n t s

t h e r e b y c r e a t i n g t h e symmetric array.

the

s t o r a g e mode m s t be p r e - m u l t i p l i e d

by

a single post-multiplication

columns of R by

p f a c e of

formed by expanding t h e second d e r i v a t i v e

columns from t h e symmetric

L

each p by

storage

B

matrix

and

v e r s i o n of the

nus

of L

,

t h e curvature curvatures

are

v e c t o r ( t h a t is,

the

calculated. Only t h e normal p a r t component

which i s

f i r s t derivatives) because

of

the residual

orthogonal

is

the residual

used i n vector is

t o t h e columns of

t h e matrix

t h e c a l c u l a t i o n of B

.

his

not perfectly orthogonal

of is

to the

f i r s t d e r i v a t i v e s i n p r a c t i c e which c a n c a u s e d i s c r e p e n c i e s i n B

.

Constants: The only machine level

for

orthogonal

determining

dependent

constant is

t h e d i m e n s i o n of

EPS,

t h e tolerance

the acceleration

t o t h e s p a n of t h e f i r s t d e r i v a t i v e s .

space

We recommend a

472


v a l u e of

t h e s q u a r e r o o t of t h e r e l a t i v e

is

which

t h e l a r g e s t number s u c h

t h a t l+e=l i n

floating

e,

point

'Ihis is a r a t h e r l o o s e c r i t e r i o n f o r determining t h e

arithmetic.

r a n k of a r e c t a n g u l a r m a t r i x a n d i t d i m e n s i o n of t h e o r t h o g o n a l However,

machine p r e c i s i o n ,

t h e c o n t r i b u t i o n of

i s p o s s i b l e t h a t t h e computed

a c c e l e r a t i o n space w i l l be too small. any

additional

faces

of

N

t h e A..

a r r a y t o t h e i n t r i n s i c curvature w i l l be n e g l i g i b l e . RELATED ALGORITHMS This package

a l g o r i t h m u s e s a number of subprograms f r o m t h e L i n p a c k (Dongarra

et

al.,

1979).

A b r i e f d e s c r i p t i o n of

the

p u r p o s e of each t h e s e subprograms i s g i v e n belcw.

Subprogram

Purpose


SQRDC

u s e s H o u s e h o l d e r t r a n f ormat i o n s t o compute t h e QR d e c o m p o s i t i o n of a m a t r i x w i t h op t i o n a l column p i v o t i n g

STRDI

computes t h e d e t e r m i n a n t a n d / o r i n v e r s e o f a real triangular w t r i x

SQRSL

A p p l i e s t h e o u t p u t of SQRDC

t o compute

coordinate transformations,

projections,

and l e a s t s q u a r e s s o l u t i o n s SCOPY

Copies a v e c t o r t o a n o t h e r v e c t o r

SDOT

R e t u r n s t h e d o t p r o d u c t of two v e c t o r s

SSCAL

M u l t i p l i e s a v e c t o r by a c o n s t a n t TEST RESULTS

Most of

t h e computation i n

Linpack r o u t i n e s Timing r e s u l t s o n al.,

for

which

this

extensive

algorithm is testing

has

done by been

the done.

some of t h e s e r o u t i n e s i n g i v e n i n ( D o n g a r r a e t

1979). ACKNOWLEDGEMENTS This

research has

Foundation under

been

s u p p o r t e d by

research grant

t h e National

llMCS-8102732 a n d by

S c i e n c e s a n d E n g i n e e r i n g R e s e a r c h C o u n c i l of Canada.

Science

the Natural

CURVATURES FOR NONLINEAR REGRESSION MODELS BIBLIOGRAPHY Bates,D.M. & Watts,D.G.,(1980). Relative Curvature Measures of Nonlinearity. J. R. Statist. Soc. B, 3,1-25. Parameter Transformations for Bates,D.M. & Watts,D.G.,(1981). Improved Approximate Confidence Regions in Nonlinear Least Squares, Ann. Statist., 2, 1152-1176. Dongarra,J.J., Moler,C.B., Bunch,J.R., & Stewart,G.W.,(1979). LINPACK User's Guide., Philadelphia: S.I.A.M. Hamilton,D.C. (1980). Experimental Design for Nonlinear Regression Models., Ph.D. Thesis, Queen's University at Kingston, Canada.


Hamilton, D.C. Watts, D.G. & Bates, D.M.,(1982). Accounting - for Intrinsic Nonlinearity in Nonlinear Regression Parameter Inference Regions., Ann. Statist., l0, 386-393.

APPENDIX ALGORITHM SUBROUTINE CRVTRS (NOBS,NPAR,NTOTAL,EPS,LDDER,LDRINV,LDB, DERIVS,RESID,RHORES,WORK,JPVT,RINV,B,NNORM, * RPARAM,RINTRN,INFO ) INTEGER NOBS,NPAR,NTOTAL,LDDER,LDRINV,LDB, JPVT (NPAR),NNORM,INFO REAL EPS,DERIVS(LDDER,NTOTAL),RESID(NOBS),RHORES, * WDRK (NTOTAL,NTOTAL ) ,RINV(LDRINV,NPAR ) ,B (LDB ,NPAR ) i~ RPARAM,RINTRN

*

CRVTRS USES A QR DECOMPOSITION OF THE FIRST AND SECOND DERIVATIVES OF A NONLINEAR REGRESSION MODEL TO COMPUTE THE INTRINSIC AND PARAMETER EFFECTS CURVATURE ARRAYS AND THE ASSOCIATED R.M.S. CURVATURES ON ENTRY NOBS NPAR NTOTAL EPS LDDER

INTEGER. NUMBER OF OBSERVATIONS. (ROWS OF DERIVS ) INTEGER. NUMBER OF PARAMETERS. INTEGER. NUMBER OF FIRST AND SECOND DERIVATIVES. (COLUMNS OF DERIVS). EQUAL TO NPAR*(NPAR + 3)/2 REAL. TOLERANCE LEVEL FOR COMPARISONS TO DETERMINE THE NUMBER OF NORMAL ACCELERATION DIMENSIONS. INTEGER.


LDRINV LDB DERIVS

RESID RHORES WORK JPVT

LEADING DIMENSION O F THE MATRIX DERIVS. M I S T BE AT LEAST AS LARGE AS MAXO(NTOTAL,NOBS ). INTEGER. LEADING DIMENSION O F THE MATRIX RINV. INTEGER. LEADING DIMENSION OF THE MATRIX B. REAL(LDDER,NTOTAL) F I R S T AND SECOND PARTIAL DERIVATIVES FORM THE COLUMNS (SECOND DERIVATIVES I N SYMMETRIC STORAGE MODE). REAL (NOBS ) RESIDUAL VECTOR AT THE E S T I M A T D PARAMETERS. REAL. VARIANCE ESTIMATE, S-SQUARED. REAL(NTOTAL,NTOTAL) WRK I S A W)RK ARRAY O F AT LEAST NTOTAL**2 E L m N T S . INTEGER (NPAR) J P V T I S AN INTEGER WRK ARRAY USED TO STORE THE P I V O T S FROM THE QR DECOMPOSITION.

ON RETURN


DERIVS

RESID RHORES RINV B

NNORM RPARAM RINTRN INFO

REAL (LDDER,NTOTAL ) THE (SCALED) R-TILDE COMPRISES THE F I R S T NPAR COLUMNS. A * . ( I N SYMMETRIC STORAGE EIODE) COMPRISES THE NEXT NPAR*(NPAR+l)/Z COLUMNS. REAL (NOBS ) Q-TRANSPOSE*(RESIDUAL VECTOR) REAL. THE STANDARD RADIUS S*SQRT(NPAR ). REAL (LDRINV,NPAR). THE INVERSE OF THE MATRIX R-TILDE. REAL(LDB ,NPAR ). THE MATRIX O F INNER PRODUCT OF THE RESIDUAL VECTOR AND THE ORTHOGONAL ACCELERATION VECTORS. INTEGER. THE NUMBER O F NORMAL ACCELERATION DIMENSIONS. REAL. THE R.M.S. PARAMETER EFFECTS CURVATURE. REAL. THE R.M.S. I N T R I N S I C CURVATURE. INTEGER. INFO CONTAINS ZERO I F THE CALCULATIONS COULD BE PERFORMED. OTHER VALUES ARE 1 NPAR.LE.0 2 NOBS .LT. NPAR 3 NTOTAL DOES NOT AGREE WITH W A R 4 RHORES .LT. 0.0 5 LDDER .LT. NTOTAL 6 F I R S T DERIVATIVES ARE SINGULAR

.

SUBPROGRAMS AND FUNCTIONS

-

CURVATURES FOR NONLINEAR REGRESSION MODELS

475

LINPACK SQRDC, SQRSL, STRDI BLAS (DIRECTLY) SCOPY, SDOT, SSCAL BLAS (INDIRECTLY) SAXPY, SCOPY, SDOT, SSCAL, SSWAP, SNRM2 FORTRAN FLOAT, MAXO, MINO, SQRT


INTERNAL VARIABLES INTEGER NPPl,NPAIRS,NPRPl,NROWS,I,J,K,II,JJ,KK REAL ZERO,ONE,TWO,FOUR,ACCUM,RHOINV,DIAG,TEMP,RMSFAC,SDOT DATA ZERO,ONE,TWO,FOUR / 0 .OEO, 1.OE0,2. OEO,4.OEO / CHECK DIMENSIONS AND POSITIVITY OF RHORES INFO=l IF (NPAR-LE.0) RETURN INFO=2 IF (NOBS.LT .NPAR ) RETURN INF0=3 IF (NTOTAL.NE. ( (NPAR* (NPAR+~) )/2 ) ) RETURN INFOz4 IF (RHORESeLE. ZERO ) RETURN INF0=5 IF (LDDER-LT-NTOTAL) RETURN SCALE DERIVATIVES AND RESID RHORES=SQRT(FLOAT(NPAR )*RHORES ) RHOINV=ONE/RHORES CALL SSCAL(NOBS,RHOINV,RESID,l) DO 10 I-1,NTOTAL CALL SSCAL(NOBS,RHOINV,DERIVS(l,I),l) JPVT(1 )=0 10 CONTINUE FORM QR DECOMPOSITION DO 20 I=l,NPAR JPVT (I )=I 20 CONTINUE CALL SQRDC (DERIVS,LDDER,NOBS,NTOTAL,WORK,JPVT,WORK(l, 2), 1) MJLTIPLY RESID BY Q-TRANSPOSE CALL SQRSL (DERIVS,LDDER,NOBS,NTOTAL,WORK,RESID, * WORK(1,2),RESID,WORK(1,2),WORK(1,2),WORK(1,2),01000,I) ZERO TRAILING ENTRIES OF COLUMNS NROWS=MAXO(NTOTAL,NOBS) DO 30 I=l,NTOTAL J=MINO(I+l,NOBS+1) K=NROWS+l-J C K L SCOPY(K,ZERO,0,DERIVS (J,I), 1 ) IF (I.LE.NPAR) CALL SCOPY(NPAR,DERIVS(~,I),~,RINV(~,I),~) 30 CONTINUE CALCULATE RINV CALL STRDI(RINV,LDRINV,NPAR,WORK,O11,1) INFO=6 IF (I.NE.0) RETURN INFOSO

476

C

C C


C C

C


CALCULATE NNORM NNORM=O NPPl=NPAR+l TEMP=DERIVS(NPPl,NPPl) IF (TEMP-EQ.ZERO) GO TO 50 DO 40 I=NPPl,NTOTAL IF (EPS .LE.ABS (DERIVS (1,I )/TEMP ) ) NNORMzI-NPAR 40 CONTINUE 50 CONTINUE REVERSE THE EFFECT OF THE PIVOTS ON THE COLUMNS OF DERIVS DO 60 I=NPPl,NTOTAL CALL SCOPY(NTOTAL,DERIVS(l,I),l,WORK(1,1),1) 60 CONTINUE DO 70 I=NPPl,NTOTAL J=JPVT(I) CALL SCOPY(NTOTAL,WORK(l,I ), l,DERIVS(l, J), 1 ) 70 CONTINUE FORM MATRIX TO PERFORM PRE- AND POST-MULTIPLICATIONS BY RINV NPAIRS=NTOTAL-NPAR K=O DO 110 I=l,NPAR DO 100 J=l,I K=K+ 1 KK=O DO 9 0 II=l,NPAR DO 80 JJ-1,I1 KK=KK+ 1 WORK(KK,K)=RINV(II, I )*RINV(JJ, J) * +RINV(JJ,I)*RINV(II,J) IF (1I.EQ.J.T) WORK(KK,K)=WORK(KK,K)/TWO 80 GONTINUE 90 CONTINUE 100 CONTINUE 110 CONTINUE NPRPl=NPAIRS+l DO 130 I=l,NTOTAL DO 120 J=l,NPAIRS WORK(J,NPRPl )=SDOT(NPAIRS,DERIVS (1,NPPl ),LDDER, * WORK(l,.J),l) 120 CONTINUE CALL SCOPY(NPAIRS,WORK(l,NPRPl ), 1,DERIVS(1,NPPl ),LDDER) 130 CONTINUE CALCULATE THE B MATRIX K=NPAR DO 150 I=l,NPAR DO 140 J=l,I K=K+ 1 B(1, J)=SDOT(NPAIRS,DERIVS (NPPl,K),l,RESID(NPPl), 1) B(J,I)=B(I,J)

CURVATURES FOR NONLINEAR REGRESSION MODELS


140 CONTINUE 150 CONTINUE C CALCULATE R.M.S. CURVATURES RPARAM=ZERO RINTRN=ZERO DO 170 I=l,NTOTAL ACCUM=FOUR*SDOT(NPAIRS,DERIVS (1,NPPl) ,LDDER,DERIVS (I,NPPl), * LDDER ) DIAG=ZERO DO 160 J=1 ,NPAR K=NPAR+(J* (J+l) )/2 DIAG=DIAG+DERIVS(I,K) ACCUMxACCUM-TWO*DERIVS (I,K)**2 160 CONTINUE IF (I .LE SNPAR) RPARAM=RPARAM+ACCUM$DIAG**Z IF (I.GT-NPAR) RINTRN=RINTRN+ACCUM+DIAG**Z 170 CONTINUE RMSFAC=ONE/FLOAT(NPAR*(NPAR+Z)) RPARAM=SQRT(RPARAM*RMSFAC) RINTRN=SQRT (RINTRN*RMSFAC ) RETURN END

R e c e i v e d b y E d i t o r i a l Board member December, 1982.

Recomended b y W i l l i a m J . Kennedy, Iowa S t a t e U n i v e r s i t y , Ames , I A