Jun 27, 2007 - curvatures for nonlinear regression models. Douglas M. Bates , David C. Hamilton & Donald G. Watts. To cite this article: Douglas M. BatesĀ ...
Communications in Statistics - Simulation and Computation
ISSN: 0361-0918 (Print) 1532-4141 (Online) Journal homepage: http://www.tandfonline.com/loi/lssp20
Calculation of intrinsic and parameter-effects curvatures for nonlinear regression models Douglas M. Bates , David C. Hamilton & Donald G. Watts To cite this article: Douglas M. Bates , David C. Hamilton & Donald G. Watts (1983) Calculation of intrinsic and parameter-effects curvatures for nonlinear regression models, Communications in Statistics - Simulation and Computation, 12:4, 469-477, DOI: 10.1080/03610918308812333 To link to this article: http://dx.doi.org/10.1080/03610918308812333
Published online: 27 Jun 2007.
Submit your article to this journal
Article views: 40
View related articles
Citing articles: 9 View citing articles
Full Terms & Conditions of access and use can be found at http://www.tandfonline.com/action/journalInformation?journalCode=lssp20 Download by: [Dalhousie University]
Date: 11 September 2015, At: 04:20
COMMUN. STATIST.-SIMULA.
COMPUTA.,
1 2 ( 4 ) , 469-477
(1983)
CALCULATION OF INTRINSIC AND PARAMETER-EFFECTS CURVATURES FOR NONLINEAR REGRESSION MODELS
D o u g l a s M. B a t e s
David C. H a m i l t o n
Department of S t a t i s t i c s U n i v e r s i t y of Wisc., Madison
D e p t . o f Math., S t a t . & Comp. S c i . Dalhousie University, Halifax
Donald G. W a t t s D e p a r t m e n t of M a t h e m a t i c s a n d S t a t i s t i c s , Queen's U n i v e r s i t y , K i n g s t o n
Downloaded by [Dalhousie University] at 04:20 11 September 2015
Key Words and Phrases: orthogonal vectors; rotation o f coordinates; basis for normal acceleration space.
PURPOSE AND DESCRIPTION Purpose: T h i s a l g o r i t h m u s e s f i r s t a n d s e c o n d d e r i v a t i v e s of t h e model f u n c t i o n w i t h r e s p e c t t o t h e model p a r a m e t e r s a t t h e l e a s t s q u a r e s estimates
to
calculate
curvature arrays design
-
data
is
Watts,
&
p a r a m e t e r - e f f ects
set c o m b i n a t i o n .
parameter-effects vector
the
associated with
curvatures
are
a particular
calculated
and
B employed
-
the in
residual (Hamilton,
B a t e s , 1 9 8 2 ) t o compensate f o r t h e e f f e c t s of i n t r i n s i c 'Ihe ( s c a l e d )
m a t r i x R f r o m a QR d e c o w o s i t i o n
t h e f i r s t d e r i v a t i v e m a t r i x and t h e i n v e r s e they
may b e s t o r e d and
transformation 1981).
intrinsic
Root mean s q u a r e i n t r i n s i c and
used t o produce t h e m t r i x
nonlinearity.
that
and
n o n l i n e a r model
of
parameters
of R a r e
used t o evaluate
as
A s u g g e s t i o n of ( H a m i l t o n ,
described i n 1980)
is
of
returned so
t h e e f f i c t s of (Bates &
a
Watts,
used t o reduce t h e
d i m e n s i o n of t h e i n t r i n s i c c u r v a t u r e a r r a y and t o s a v e s t o r a g e a n d t ime
.
469 Copyright @ 1983 by Marcel Dekker, Inc.
0361-0918/83/1204-0469$3.50/0
BATES, HAMILTON, AND WATTS
470 Theory and Method: The
general
theory
and
geometric
curvature a r r a y s is given i n (Bates & (Hamilton,
l98O),
interpretation
Watts,
1980).
of
the
A s noted i n
a r e a t ms t p ( p + 1 ) / 2 i n d e p e n d e n t second
there
d e r i v a t i v e v e c t o r s i n a model w i t h p p a r a m e t e r s ,
s o t h e dimension
of t h e normal a c c e l e r a t i o n s p a c e of t h e s o l u t i o n
l o c u s i s a t most
By d e t e r m i n i n g a b a s i s f o r t h e components of t h e s e c o n d
p(p+1)/2. derivative
vectors
orthogonal
to
the
first
derivatives
and
it i s possible t o express t h e
rotating coordinates appropriately,
i n t r i n s i c c u r v a t u r e a r r a y v e r y compactly. The method of Watts,
(Bates 6 matrix
calculating t h e curvature arrays
described i n
1980) i n v o l v e d f o r m i n g a QR d e c o m p o s i t i o n of t h e
of f i r s t d e r i v a t i v e s a n d ,
.
d e r i v a t i v e v e c t o r s by Q'
later,
pre-multiplying
second
These two s t e p s and t h e f u r t h e r s t e p
of d e t e r m i n i n g a b a s i s f o r t h e normal a c c e l e r a t i o n s p a c e a r e
Downloaded by [Dalhousie University] at 04:20 11 September 2015
simply
accomplished
decomposition derivative assume
in
this
and t h e non-redundant
the f i r s t derivative
require
algorithm
of a n augmented m a t r i x ,
that
t h e augmented
D
second
,
by
forming
i n t e r c h a n g e s of t h e s e c o n d
al.,
&,
derivative vectors.
matrix
is
non-singular
matrix
be
non-singular.
b u t do
not
To g e t a
we a l l o w
column
r o u t i n e from LINPACK (Dongarra e t
b e g i n by
w h e r e s2
is of
d i v i d i n g t h e r e s i d u a l and d e r i v a t i v e v e c t o r s by t h e r e s i d u a l mean s q u a r e . t h e matrix
D
,
We
the f i r s t p
contain the f i r s t derivative vectors
xi,
t h e n form a QR
columns of
i = 1, 2,
..., p,
r e m a i n i n g p ( p + 1 ) / 2 columns, t h e s e c o n d d e r i v a t i v e v e c t o r s 1, 2 ,
W e
d e r i v a t i v e s columns u s i n g t h e p i v o t i n g
t h e QR d e c o m p o s i t i o n
decomposition
is,
QR
1979). We
s
a
composed of t h e f i r s t
u n i q u e QR d e c o m p o s i t i o n w i t h R u p p e r t r a p e z o i d a l
scheme i n
more
..., p , in
j = i, i+I,
the
order
decomposition,
a * . ,
which and t h e
4'
p , i n symmetric s t o r a g e mode;
V 1 1 , V 1 2 ~ ~ 2 , V 1 3 * X 2" 3 ~' "V -PP
.
i =
that This
D = QRE
produces t h e n
by
n
orthogonal
matrix
Q
(n i s
t h e number o f
'
471
CURVATURES FOR NONLINEAR REGRESSION MODELS E,
observations),
a p(p+3)/2 s q u a r e permutation
t h e u p p e r l e f t p by p s u b m a t r i x i s t h e i d e n t i t y , trapezoidal
matrix
with
t h e u p p e r l e f t p by
m a t r i x i n which and R ,
a n upper
p submatrix
upper
triangular.
R
Partitioning
as
which i s u s e d i n c o m b i n a t i o n w i t h R t o produce t h e 11' 12 p a r a m e t e r e f f e c t s c u r v a t u r e a r r a y , and w i t h R t o produce t h e
provides R
2
i n t r i n s i c curvature array.
R I I i s t h e same a s R = L~
i n ( B a t e s I.
W a t t s , 1980).
Downloaded by [Dalhousie University] at 04:20 11 September 2015
To c a l c u l a t e t h e c u r v a t u r e a r r a y s , t h r e e dimensional a r r a y
and p o s t - m u l t i p l i e d
effected
by
L
.
These
two
Finally
the
by
multiplications are
of
the last
p(p+1)/2
a m a t r i x composed of p r o d u c t s of e l e m e n t s
t h e r e b y c r e a t i n g t h e symmetric array.
the
s t o r a g e mode m s t be p r e - m u l t i p l i e d
by
a single post-multiplication
columns of R by
p f a c e of
formed by expanding t h e second d e r i v a t i v e
columns from t h e symmetric
L
each p by
storage
B
matrix
and
v e r s i o n of the
nus
of L
,
t h e curvature curvatures
are
v e c t o r ( t h a t is,
the
calculated. Only t h e normal p a r t component
which i s
f i r s t derivatives) because
of
the residual
orthogonal
is
the residual
used i n vector is
t o t h e columns of
t h e matrix
t h e c a l c u l a t i o n of B
.
his
not perfectly orthogonal
of is
to the
f i r s t d e r i v a t i v e s i n p r a c t i c e which c a n c a u s e d i s c r e p e n c i e s i n B
.
Constants: The only machine level
for
orthogonal
determining
dependent
constant is
t h e d i m e n s i o n of
EPS,
t h e tolerance
the acceleration
t o t h e s p a n of t h e f i r s t d e r i v a t i v e s .
space
We recommend a
472
BATES, HAMILTON, AND WATTS
v a l u e of
t h e s q u a r e r o o t of t h e r e l a t i v e
is
which
t h e l a r g e s t number s u c h
t h a t l+e=l i n
floating
e,
point
'Ihis is a r a t h e r l o o s e c r i t e r i o n f o r determining t h e
arithmetic.
r a n k of a r e c t a n g u l a r m a t r i x a n d i t d i m e n s i o n of t h e o r t h o g o n a l However,
machine p r e c i s i o n ,
t h e c o n t r i b u t i o n of
i s p o s s i b l e t h a t t h e computed
a c c e l e r a t i o n space w i l l be too small. any
additional
faces
of
N
t h e A..
a r r a y t o t h e i n t r i n s i c curvature w i l l be n e g l i g i b l e . RELATED ALGORITHMS This package
a l g o r i t h m u s e s a number of subprograms f r o m t h e L i n p a c k (Dongarra
et
al.,
1979).
A b r i e f d e s c r i p t i o n of
the
p u r p o s e of each t h e s e subprograms i s g i v e n belcw.
Subprogram
Purpose
Downloaded by [Dalhousie University] at 04:20 11 September 2015
SQRDC
u s e s H o u s e h o l d e r t r a n f ormat i o n s t o compute t h e QR d e c o m p o s i t i o n of a m a t r i x w i t h op t i o n a l column p i v o t i n g
STRDI
computes t h e d e t e r m i n a n t a n d / o r i n v e r s e o f a real triangular w t r i x
SQRSL
A p p l i e s t h e o u t p u t of SQRDC
t o compute
coordinate transformations,
projections,
and l e a s t s q u a r e s s o l u t i o n s SCOPY
Copies a v e c t o r t o a n o t h e r v e c t o r
SDOT
R e t u r n s t h e d o t p r o d u c t of two v e c t o r s
SSCAL
M u l t i p l i e s a v e c t o r by a c o n s t a n t TEST RESULTS
Most of
t h e computation i n
Linpack r o u t i n e s Timing r e s u l t s o n al.,
for
which
this
extensive
algorithm is testing
has
done by been
the done.
some of t h e s e r o u t i n e s i n g i v e n i n ( D o n g a r r a e t
1979). ACKNOWLEDGEMENTS This
research has
Foundation under
been
s u p p o r t e d by
research grant
t h e National
llMCS-8102732 a n d by
S c i e n c e s a n d E n g i n e e r i n g R e s e a r c h C o u n c i l of Canada.
Science
the Natural
CURVATURES FOR NONLINEAR REGRESSION MODELS BIBLIOGRAPHY Bates,D.M. & Watts,D.G.,(1980). Relative Curvature Measures of Nonlinearity. J. R. Statist. Soc. B, 3,1-25. Parameter Transformations for Bates,D.M. & Watts,D.G.,(1981). Improved Approximate Confidence Regions in Nonlinear Least Squares, Ann. Statist., 2, 1152-1176. Dongarra,J.J., Moler,C.B., Bunch,J.R., & Stewart,G.W.,(1979). LINPACK User's Guide., Philadelphia: S.I.A.M. Hamilton,D.C. (1980). Experimental Design for Nonlinear Regression Models., Ph.D. Thesis, Queen's University at Kingston, Canada.
Downloaded by [Dalhousie University] at 04:20 11 September 2015
Hamilton, D.C. Watts, D.G. & Bates, D.M.,(1982). Accounting - for Intrinsic Nonlinearity in Nonlinear Regression Parameter Inference Regions., Ann. Statist., l0, 386-393.
APPENDIX ALGORITHM SUBROUTINE CRVTRS (NOBS,NPAR,NTOTAL,EPS,LDDER,LDRINV,LDB, DERIVS,RESID,RHORES,WORK,JPVT,RINV,B,NNORM, * RPARAM,RINTRN,INFO ) INTEGER NOBS,NPAR,NTOTAL,LDDER,LDRINV,LDB, JPVT (NPAR),NNORM,INFO REAL EPS,DERIVS(LDDER,NTOTAL),RESID(NOBS),RHORES, * WDRK (NTOTAL,NTOTAL ) ,RINV(LDRINV,NPAR ) ,B (LDB ,NPAR ) i~ RPARAM,RINTRN
*
CRVTRS USES A QR DECOMPOSITION OF THE FIRST AND SECOND DERIVATIVES OF A NONLINEAR REGRESSION MODEL TO COMPUTE THE INTRINSIC AND PARAMETER EFFECTS CURVATURE ARRAYS AND THE ASSOCIATED R.M.S. CURVATURES ON ENTRY NOBS NPAR NTOTAL EPS LDDER
INTEGER. NUMBER OF OBSERVATIONS. (ROWS OF DERIVS ) INTEGER. NUMBER OF PARAMETERS. INTEGER. NUMBER OF FIRST AND SECOND DERIVATIVES. (COLUMNS OF DERIVS). EQUAL TO NPAR*(NPAR + 3)/2 REAL. TOLERANCE LEVEL FOR COMPARISONS TO DETERMINE THE NUMBER OF NORMAL ACCELERATION DIMENSIONS. INTEGER.
BATES, HAMILTON, AND WATTS
LDRINV LDB DERIVS
RESID RHORES WORK JPVT
LEADING DIMENSION O F THE MATRIX DERIVS. M I S T BE AT LEAST AS LARGE AS MAXO(NTOTAL,NOBS ). INTEGER. LEADING DIMENSION O F THE MATRIX RINV. INTEGER. LEADING DIMENSION OF THE MATRIX B. REAL(LDDER,NTOTAL) F I R S T AND SECOND PARTIAL DERIVATIVES FORM THE COLUMNS (SECOND DERIVATIVES I N SYMMETRIC STORAGE MODE). REAL (NOBS ) RESIDUAL VECTOR AT THE E S T I M A T D PARAMETERS. REAL. VARIANCE ESTIMATE, S-SQUARED. REAL(NTOTAL,NTOTAL) WRK I S A W)RK ARRAY O F AT LEAST NTOTAL**2 E L m N T S . INTEGER (NPAR) J P V T I S AN INTEGER WRK ARRAY USED TO STORE THE P I V O T S FROM THE QR DECOMPOSITION.
ON RETURN
Downloaded by [Dalhousie University] at 04:20 11 September 2015
DERIVS
RESID RHORES RINV B
NNORM RPARAM RINTRN INFO
REAL (LDDER,NTOTAL ) THE (SCALED) R-TILDE COMPRISES THE F I R S T NPAR COLUMNS. A * . ( I N SYMMETRIC STORAGE EIODE) COMPRISES THE NEXT NPAR*(NPAR+l)/Z COLUMNS. REAL (NOBS ) Q-TRANSPOSE*(RESIDUAL VECTOR) REAL. THE STANDARD RADIUS S*SQRT(NPAR ). REAL (LDRINV,NPAR). THE INVERSE OF THE MATRIX R-TILDE. REAL(LDB ,NPAR ). THE MATRIX O F INNER PRODUCT OF THE RESIDUAL VECTOR AND THE ORTHOGONAL ACCELERATION VECTORS. INTEGER. THE NUMBER O F NORMAL ACCELERATION DIMENSIONS. REAL. THE R.M.S. PARAMETER EFFECTS CURVATURE. REAL. THE R.M.S. I N T R I N S I C CURVATURE. INTEGER. INFO CONTAINS ZERO I F THE CALCULATIONS COULD BE PERFORMED. OTHER VALUES ARE 1 NPAR.LE.0 2 NOBS .LT. NPAR 3 NTOTAL DOES NOT AGREE WITH W A R 4 RHORES .LT. 0.0 5 LDDER .LT. NTOTAL 6 F I R S T DERIVATIVES ARE SINGULAR
.
SUBPROGRAMS AND FUNCTIONS
-
CURVATURES FOR NONLINEAR REGRESSION MODELS
475
LINPACK SQRDC, SQRSL, STRDI BLAS (DIRECTLY) SCOPY, SDOT, SSCAL BLAS (INDIRECTLY) SAXPY, SCOPY, SDOT, SSCAL, SSWAP, SNRM2 FORTRAN FLOAT, MAXO, MINO, SQRT
Downloaded by [Dalhousie University] at 04:20 11 September 2015
INTERNAL VARIABLES INTEGER NPPl,NPAIRS,NPRPl,NROWS,I,J,K,II,JJ,KK REAL ZERO,ONE,TWO,FOUR,ACCUM,RHOINV,DIAG,TEMP,RMSFAC,SDOT DATA ZERO,ONE,TWO,FOUR / 0 .OEO, 1.OE0,2. OEO,4.OEO / CHECK DIMENSIONS AND POSITIVITY OF RHORES INFO=l IF (NPAR-LE.0) RETURN INFO=2 IF (NOBS.LT .NPAR ) RETURN INF0=3 IF (NTOTAL.NE. ( (NPAR* (NPAR+~) )/2 ) ) RETURN INFOz4 IF (RHORESeLE. ZERO ) RETURN INF0=5 IF (LDDER-LT-NTOTAL) RETURN SCALE DERIVATIVES AND RESID RHORES=SQRT(FLOAT(NPAR )*RHORES ) RHOINV=ONE/RHORES CALL SSCAL(NOBS,RHOINV,RESID,l) DO 10 I-1,NTOTAL CALL SSCAL(NOBS,RHOINV,DERIVS(l,I),l) JPVT(1 )=0 10 CONTINUE FORM QR DECOMPOSITION DO 20 I=l,NPAR JPVT (I )=I 20 CONTINUE CALL SQRDC (DERIVS,LDDER,NOBS,NTOTAL,WORK,JPVT,WORK(l, 2), 1) MJLTIPLY RESID BY Q-TRANSPOSE CALL SQRSL (DERIVS,LDDER,NOBS,NTOTAL,WORK,RESID, * WORK(1,2),RESID,WORK(1,2),WORK(1,2),WORK(1,2),01000,I) ZERO TRAILING ENTRIES OF COLUMNS NROWS=MAXO(NTOTAL,NOBS) DO 30 I=l,NTOTAL J=MINO(I+l,NOBS+1) K=NROWS+l-J C K L SCOPY(K,ZERO,0,DERIVS (J,I), 1 ) IF (I.LE.NPAR) CALL SCOPY(NPAR,DERIVS(~,I),~,RINV(~,I),~) 30 CONTINUE CALCULATE RINV CALL STRDI(RINV,LDRINV,NPAR,WORK,O11,1) INFO=6 IF (I.NE.0) RETURN INFOSO
476
C
C C
Downloaded by [Dalhousie University] at 04:20 11 September 2015
C C
C
BATES, HAMILTON, AND WATTS
CALCULATE NNORM NNORM=O NPPl=NPAR+l TEMP=DERIVS(NPPl,NPPl) IF (TEMP-EQ.ZERO) GO TO 50 DO 40 I=NPPl,NTOTAL IF (EPS .LE.ABS (DERIVS (1,I )/TEMP ) ) NNORMzI-NPAR 40 CONTINUE 50 CONTINUE REVERSE THE EFFECT OF THE PIVOTS ON THE COLUMNS OF DERIVS DO 60 I=NPPl,NTOTAL CALL SCOPY(NTOTAL,DERIVS(l,I),l,WORK(1,1),1) 60 CONTINUE DO 70 I=NPPl,NTOTAL J=JPVT(I) CALL SCOPY(NTOTAL,WORK(l,I ), l,DERIVS(l, J), 1 ) 70 CONTINUE FORM MATRIX TO PERFORM PRE- AND POST-MULTIPLICATIONS BY RINV NPAIRS=NTOTAL-NPAR K=O DO 110 I=l,NPAR DO 100 J=l,I K=K+ 1 KK=O DO 9 0 II=l,NPAR DO 80 JJ-1,I1 KK=KK+ 1 WORK(KK,K)=RINV(II, I )*RINV(JJ, J) * +RINV(JJ,I)*RINV(II,J) IF (1I.EQ.J.T) WORK(KK,K)=WORK(KK,K)/TWO 80 GONTINUE 90 CONTINUE 100 CONTINUE 110 CONTINUE NPRPl=NPAIRS+l DO 130 I=l,NTOTAL DO 120 J=l,NPAIRS WORK(J,NPRPl )=SDOT(NPAIRS,DERIVS (1,NPPl ),LDDER, * WORK(l,.J),l) 120 CONTINUE CALL SCOPY(NPAIRS,WORK(l,NPRPl ), 1,DERIVS(1,NPPl ),LDDER) 130 CONTINUE CALCULATE THE B MATRIX K=NPAR DO 150 I=l,NPAR DO 140 J=l,I K=K+ 1 B(1, J)=SDOT(NPAIRS,DERIVS (NPPl,K),l,RESID(NPPl), 1) B(J,I)=B(I,J)
CURVATURES FOR NONLINEAR REGRESSION MODELS
Downloaded by [Dalhousie University] at 04:20 11 September 2015
140 CONTINUE 150 CONTINUE C CALCULATE R.M.S. CURVATURES RPARAM=ZERO RINTRN=ZERO DO 170 I=l,NTOTAL ACCUM=FOUR*SDOT(NPAIRS,DERIVS (1,NPPl) ,LDDER,DERIVS (I,NPPl), * LDDER ) DIAG=ZERO DO 160 J=1 ,NPAR K=NPAR+(J* (J+l) )/2 DIAG=DIAG+DERIVS(I,K) ACCUMxACCUM-TWO*DERIVS (I,K)**2 160 CONTINUE IF (I .LE SNPAR) RPARAM=RPARAM+ACCUM$DIAG**Z IF (I.GT-NPAR) RINTRN=RINTRN+ACCUM+DIAG**Z 170 CONTINUE RMSFAC=ONE/FLOAT(NPAR*(NPAR+Z)) RPARAM=SQRT(RPARAM*RMSFAC) RINTRN=SQRT (RINTRN*RMSFAC ) RETURN END
R e c e i v e d b y E d i t o r i a l Board member December, 1982.
Recomended b y W i l l i a m J . Kennedy, Iowa S t a t e U n i v e r s i t y , Ames , I A