STRATEGIC PLANNING AND FINANCIAL ...

Journal of Management Studies 28:4 July 1991 0022-2380 $3.50

STRATEGIC PLANNING AND FINANCIAL PERFORMANCE: A META-ANALYTIC REVIEW* BRIAN K . BOYD

Blue Cross and Blue Shield of Michigan

ABSTRACT After two decades of research, the effect of strategic planning on a firm's performance is still unclear. While some studies have found significant benefits from planning, others have found no relationship, or even small negative effects. Interpretation of these findings is confounded by the fact that many of these studies base their findings on a small number of firms. This article uses metaanalysis to aggregate the results of 29 samples on a total of 2496 organizations. Cumulation of previous studies found modest correlations between planning and nine performance measures. Extensive measurement problems suggest that these findings underestimate the true relationship between planning and performance. INTRODUCTION Open systems models, including contingency theory and resource dependence, argue that a firm's survival is dependent on its ability to adapt .successfully to a changing environment. Strategic planning is one tool to manage environmental turbulence, and began to diffuse in the United States during the 1960s (Ringbakk, 1972). While once the exclusive domain of manufacturing firms, strategic planning is now practised by service and retail organizations (Burt, 1978), financial institutions (Gup and Whitehead, 1983) and even non-profit organizations (Odom and Boxx, 1988; Wortman, 1979). Formal strategic planning is an explicit and ongoing organizational process (Armstrong, 1982; Steiner, 1979) with several components, including establishment of goals and generation and evaluation of strategies. Steiner suggests that an effective strategic planning system will link long-range strategic goals with both mid-range and operational plans. Planners collect data, forecast, model and construct alternative future scenarios. Ostensibly, these activities should allow organizations to outperform other firms which did not engage in planning. As Hofer noted, early planners took this assumption on faith: Does strategic planning pay? For a substantial time, those involved in the strategic planning area have had to accept as a tenet of faith the belief that strategic planning was indeed worthwhile. This belief was justified with the theoretical arguments of Ansoff and others, but there was no research evidence to provide support for these beliefs (1976, p. 262). Address for reprints: Brian Boyd, Blue Cros.s and Blue Shield of Michigan, 600 Lafayette East. Detroit, MI 48226. USA

354

BRIAN K. BOYD

The first empirical test of this relationship was conducted by Thune and House (1970), who studied 36 firms in six industry groups. Within each industry group they compared groups of formal and informal planners on five economic measures. Planners had 44 per cent higher earnings per share than non-planners, and similar results were found for earnings on common equity (planners 38 per cent better) and earnings on total capital (planners 32 per cent better). They also evaluated performance before and after firms initiated formal planning programmes, and found similar performance benefits. The outcome of this study confirmed many firms' hopes about the usefulness of strategic planning. Following Thune and House were numerous papers conducting similar analyses. Two decades later there are dozens of empirical tests of the planningperformance relationship. Unfortunately, this larger body of research is less clear, and less encouraging, than Thune and House's original fmdings. While some studies report strong benefits of planning (Karger and Malik, 1975; Rhyne, 1986), many report no quantifiable benefit (Grinyerand Norburn, 1975; Kudla, 1980), and others (Fulmer and Rue, 1974; Whitehead and Gup, 1985) have even found that planners perform worse on some measures than their non-planning counterparts. Several papers have reviewed this body of empirical work in an effort to integrate these findings. Armstrong compared the numbers of studies reporting positive, null, or negative benefits to formal planning, and concluded that these studies supported the usefulness of formal planning, but that 'serious research problems were found in these studies, so few conclusions could be drawn about how to plan and when to plan' (1982, p. 209). Pearce et al. critiqued 18 studies and concluded that empirical support for the effect of formal planning'has been inconsistent and contradictory' (1987, p. 671). Pearce et al. identified measurement of formal planning as a critical issue in this avenue of research. As Hunter et al. explain, these secondary analyses are as important as the original studies: At one time in the history of psychology and the social sciences, the pressing need was for more empirical studies examining the problem in question. In many areas of research, the need today is not additional empirical data but some means of making sense of the vast amounts of data we have accumulated (1982, pp. 26-7). A logical extension of these narrative reviews is to aggregate statistically the previous research. Meta-analysis (Glass ^/a/., 1978; Hunters? a/., 1982) reviews a body of empirical work, and estimates a weighted 'average' correlation between two variables. Meta-analysis uses summary data usually available in published papers, and does not require access to the original data. Narrative and statistical aggregations often yield different results, even when reviewing relatively few studies (Cooper and Rosenthal, 1980), and Hunter et al. (1982) argued that metaanalysis is less biased than a narrative or verbal integration. There are several approaches to meta-analysis (see Bangert-Drowns, 1986 for a summary and critique), and this article uses the technique advocated by Hunter et al. (1982). The purpose of the present study is two-fold. The first goal is to provide a statistical aggregation of previous research on formal planning and financial performance. Meta-analysis enables cumulation of these studies while accounting

STRATEGIC PLANNING AND FINANCIAL PERFORMANCE

355

for differences in sample size or strength of effect. Previous reviews have highlighted the prevalence of measurement error in these studies. Thus, a second goal is to demonstrate the effect of measurement error on these results. METHOD Definition of Strategic Planning The studies analysed here measure strategic planning in a variety of ways. Eight papers dichotomize their sample into planners versus non-planners. Other studies classify their samples into three, four, or even five groups, based on sophistication, structure, commitment, and even quality of planning programmes. Initially, this might suggest that a meta-analysis of these studies is inappropriate, as each measures a somewhat different variable. However, as Pearce et al. (1987) observed, these measures of planning really fall into only two categories: formalization of planning processes, and perceived importance of planning. Additionally, a goal of meta-analysis is to generalize over a body of research which may incorporate different types of models and data (Farley and Lehmann, 1986). Selection of Studies The usefulness of a meta-analysis is a direct function of the thoroughness of the literature review. While there are many narrative reviews on this topic {e.g. Armstrong, 1982; Greenley, 1986; Hofer, 1976; King, 1983; Shrader ^/a/., 1984), most are not exhaustive reviews ofthe literature. A comprehensive review should include both manual and automated searches (Cooper, 1982, 1984). Journals in management, planning, and strategy were scanned, as well as relevant journal/ citation indices. This included scanning the tables of contents of Strategic Management Joumal, Long Range Planning, Academy of Management Joumal/Review/Executive, Joumal of Management Studies, Managerial Planning, Joumal of Business Strategy, Joumal of Business Research, Planning Review, American Journal of Small Business, And Management Science for at least the last 10 years. The preceding five years ofthe Academy ofManagement Proceedings were also included. This was supplemented with a computerized search ofthe business periodicals component ofthe Dialog database. The literature reviews of these articles were examined, and used as a source of additional articles. The literature search excluded foreign language journals, and included two studies conducted outside the United States. One empirical study cited previously (Harju, 1981) could not be obtained. Neither Klein (1979), nor Whitehead and Gup (1985) were cited in any paper reviewed here, nor in Dialog. A total of 49 journal articles and book chapters was identified which addressed the relationship between strategic planning and firm performance. Of these, only a relatively small subset are appropriate for meta-analysis. As shown in table I, 13 papers were critical reviews or essays. Five studies were excluded because of no quantifiable effect size. An effect size is a measure ofthe strength of the effect of one variable upon another, and can be estimated using group means, sample size, and variances, or by using sample size and /-values. For example, given the (-value of a comparison between mean scores for planners and non-planners on some performance measure, the effect size V can be estimated as: rt^{t'^N2)

BRIAN K. BOYD

Some studies may report whether or not lvalues are significant, but not the actual lvalue. Alternately, means for two groups may be reported but not their standard deviations. Correlational studies cannot be included if they report nonsignificant correlations merely as 'NS'. Table I. Studies excluded from the meta-analysis by rationale Article

Journal

Essays and Reviews

Armstrons (1982) Brahm and Brahm (1987) Camillus (1975) Child (1972) Gerstner (1972) Gluck etal. (1980) Greenley (1986) Hofer (1976) Karger (197.'}) King (1983) Lorange (1979) Pearce et al. (1987) Shrader^/ al. (1984)

Strategic A'ianagemerit Journal

unpublished conference paper

i

Long Range Planning Sociotogy Business Horizons Harvard Business Review Long Range Planning Journal of Business and Economics Long Range Planning Strategic Management Journal Strategic Planning: A New View of Business Policy and Planning Academy of Management Review Journal of Management

No quantifiable effect size

Gershcfski (1970) Herold (1972) Perkins and Sugden (1971) Prasad (1984) Robinson and Pearce (1983)

Management Science Academy of Management Journal Formal Planning Systems The Bankers Magazine Strategic Management Journal

Data reported in previous study

Klein (1981) Kudla (1981) Malik and Basu (1986) Malik and Karger (1975) Rue and Fulmer (1973) Shapiro and Kallman (1978)

Journal of Bank Research Review af Business and Economic Re.fearch Business Horizons Management Review Academy of Management Proceedings Planning Review

Unpublished papers

Guynes (1969) Lindsay el al. (1981) Najjar (1966) Sheehan (1975)

dissertation conference paper dissertation dissertation

Authors were contacted requesting additional data for studies where effect sizes could not be estimated. Robinson and Pearce (1983), Prasad (1984), and Herold (1972) indicated that data from their studies were unavailable. Additional information was provided by Ernest Kallman (Kallman and Shapiro, 1978) and Robert Ackdsberg (Ackelsberg and Arlow, 1985). Additional data required for Thune and House (1970) was found in Thune's (1967) master's thesis. Supplemental information used to compute effect sizes is noted in the appendix.

STRATEGIC PLANNING AND FINANCIAL PERFORMANCE

357

Many authors reported their fmdings more than once. Klein's {198\) foumal of Bank Research paper is taken from his book, first published in 1977. Kudla reported the same data for papers in 1980 and 1981, and Kallman and Shapiro published their findings in two journals in 1978. A total of six papers provided duplicate information. An additional factor is the decision whether or not to include unpublished studies in the meta-analysis. Rosenthal suggested that published research may be a biased sample of studies actually conducted on a topic. Known as the 'file drawer problem', it is ' . . . that journals are filled with the 5 % of the studies that show Type I errors, while the file drawers back at the lab are filled with the 95% of the studies that show nonsignificant results' (1979, p. 638). Also, as Cooper (1984) indicates, studies are less likely to be published if they: (1) report null results, or (2) report results contrary to 'conventional wisdom'. Excluding unpublished studies, then, would create an upward bias in the estimation of effect sizes. Hunter et al. (1982) argued that the disparity between published and unpublished papers is a function of methodological problems. Smith and Glass (1977) evaluated 375 studies and found that studies published in books had the largest effect sizes, followed by journal articles, then disserations, and unpublished papers having the smallest effect sizes. Hunter et al. suggested that this disparity reflects greater measurement error and other design problems associated with unpublished papers.''' From this dialogue, three possibilities are suggested for non-publication of a manuscript: editorial bias against null effects; results counter to 'conventional wisdom'; or methodological problems. Nearly one third ofthe published studies on planning and performance reported either null effects, or that planners actually do worse than non-planners (Fulmer and Rue, 1974; Grinyer and Norburn, 1975; Kallman and Shapiro, 1978; Klein, 1979; Kudla, 1980; Leontiades and Tezel, 1980; Whitehead and Gup, 1985), suggesting that methodological limitations are the most likely candidate for non-publication. Four unpublished papers were excluded from this analysis, but are noted in table I for reference. After removing all of these studies, 21 papers were left which were suitable for meta-analysis. While this seems a small number for this technique, two factors significantly improve the sample size: first, many studies conducted analyses on more than one industry group. Fulmer and Rue (1974), for example, studied four samples: service industries, durable goods, and non-durable goods, with separate analysis for large and small durable goods producers. Whitehead and Gup (1985) examined four samples of commercial banks based on firm size. Second most studies tested between three and five measures of performance. Karger and Malik (1975) tested 13 measures, and Ansoff «•/ al. (1970) tested 21. The 21 studies analysed here include 29 separate samples, and a total of 2496 organizations. Overall, this yielded 105 evaluations of strategy's effect on performance. Brief descriptions of each study are given in table II. Analysis

Meta-analysis generates an effect size for each statistical test. This effect size is an estimate ofthe strength ofthe relationship between two variables. While Glass et al. (1978) encourage the use of W as a measure of effect size, Hunter et al. (1982) argue that V is superior. V is a correlation coefficient, so it ranges

358

BRIAN K. BOYD

0

•c u a. 3 >

«

c

u

.2P ?J

a-s

V

MC

n

3 XI

«g

u ES o

CO

V

B o

z

to >^

V.

_

-D (_

^ ^- 9 £ « °

OwSSii

O I-u —.

•o 3 o, i; c

-o =

C r- u

J

c •"o, o u

TJ 6

S c.S tn

O

;::^

CO CO

ta

I

B

B

u

0n)"S3JD



S H£ S ^

°

>.TJ

359

STRATEGIC PLANNING AND FINANCIAL PERFORMANGE

be

5J bn.t: i

C G

^

'Q.'Q.

sl

O C ni «

P

z

be C C u C K N Q.

r-H

•

..—

•-»

u C

a. 3

G

G

0

o

n (0

Z

2

"a. " p

Cor npai per lance aga inst indu ave rage Ind ustr

2

n 1

V

S

C P C

ers and ers

u

O

u

gni fican

V

—

G S nn

.5 'c

plan

a c2

tje

O

ate gory gni fican

V

ia

-o

in

E

-7-1

U

I o

< o

Z

2

• «

tio

G

, 0

u O u

0 2


U O

— P -3 t.

.1

" .,, 2 -3

u. W

^ H "0

c .ii

E E o

2 -q a Fn g cr « f u u bo

M >

el

1 a

C Q-

-S O

c

p T3

E

c o

rO bn-'-' be G " c

bo u

W —•

C •-

It

plani

5

TT Q

a C G u Ji C O rt 3 ^ ^ •2" ' "

p ffl

U

o Z TJ

3

u c

•c

O

1

aiii

ffl « u

360

JOIJ.

sure

BRIAN K. BOYD

n V

B •

*

o

Plan

1V2

0

fc

CLH

a.a ra ^

2

^^ »

3

H S u al b ,aj

5 ,i: ^

C

o



rt c r . " - c ^ ^ c JS

u

I n

d

1 c

CO

CO CT^ 00 CT>

V

U

ra u

>s

a.

oT P S sJ

U

t-

c C

H H ^ r^ fl cn

u

U

U

STRATEGIC PLANNING AND FINANCIAL PERFORMANCE

•c

u

3

C 3

•EC

tn

?i 5 o »J

V

"S "a- 1- 5

u

eu

c cffi

aa

n

n:

^

C

E S

o 0 a5

S

-o a

•Ml

3 f"ra

P

™

cr c

^

c

C _C

3

X

B ii >

3

S1

u b.

CJ

u

B B

fft

2

•o

4-

rog

ra

e

c a

U

3

« c «; c c "a c "a c A c A JH 5 iS o a. c CL. c . u

3

.2

In

be p

a. u«i u m

fl

u

u Ml

c 5 2 « CM _

• —

a.

o

u

c

c 2 0 c g c

a c >

o

s o 5 i;

Iros

a. a.

b..

O

^

2i

5c

inc

?< ? 5 a. 3

-J

t4

361

362

. BRIAN K. BOYD

from + / - 1. Also, r, unlike d, can be corrected when the two groups {e.g. planners and non-planners) have unequal sample size - a common occurrence in these studies. When comparing the mean between two groups, r is equivalent to the point biserial correlation (McNemar, 1969), which compares group means and variances and estimates the correlation that would result if the dichotomous variable (planning) were actually measured on a continuous scale. The type of analysis used in each study is noted in table II, and the appendix notes any other transformations used to calculate size effect sizes. Once individual effect sizes have been calculated, they can be cumulated statistically across studies, weighted by sample size. A small study with a large effect size, for example, may be offset by a very large study reporting no effect. The next step involves calculating the effect size variance, which can be partialled into sampling error and population variance. A x^ test assesses whether the effect size varies significantly across studies. Hunter et al. indicate, however, that this statistic should be interpreted with caution: This statistic can be used for a formal test of no variation, though it has very high statistical power and will therefore reject the null hypothesis given a trivial amount of variation across studies. Thus if the chi square is not significant, this is strong evidence that there is no true variation across studies; but if it is significant, the variation may still be negligible in magnitude (1982, p. 47).

RESULTS As a bseline measure, the cumulated effect size across all 105 measures was calculated. As shown in table III, the overall effect of planning on performance is very weak (r = 0.1507). The comparison of error and population variance yields a X = 385.31, which is highly significant. This statistic suggests that there are sytematic differences in effect size between performance measures. Much of the variance between these estimates is imposed artificially by cumulating multiple estimates from individuEil studies. The 'worst' estimate was an effect size of - 0.45 reported by Karger and Malik on capital spending. At the other extreme, Ackelsberg and Arlow reported almost perfect correlations for changes in sales and earnings. One quarter of the estimates were effect sizes greater than 0.40 (a full listing of effect sizes and variables is available from the author). The large ^ is readily explained since different performance measures are being cumulated: the overall measure includes effect size estimates for return on assets, profitability, and other measures. If there is a relationship between planning and performance, the effect may vary from indicator to indicator. Consequencely, effect sizes were cumulated separately for nine performance measures: earnings growth, deposit growth, earnings per share growth, sales growth, priceearnings ratio, profitability, return on assets, return on equity, and return on investment. These were the most common performance measures used in the studies, and 72 of the 105 effect size estimates were on these nine variables. Most of the studies used controlled for industry effects.

STRATEGIC PLANNING AND FINANCIAL PERFORMANCE

363

Table III. Meta-analysis summary statistics Category

N

r

d^r

ALL MEASURES

9066

0.1507

0.0425

Earnings growth

1013

0.2075

Ackelsberg and Arlow Ansoff et a!. Bracker et al.' Bracker and Pearson" Burt' Fulmer and Rue" Fulmer and Rue* Fulmer and Rue* Fulmcr and rue' Karger and Malik* Wood and LaForge*

98 62 217 188

151 67 53 115 13 35

0.9995 0.1000 0.1120 0.1708 0.5050 -0.1212 -0.0234 0.0198 0.2559 0.6353 0.6763

Deposit growth

514

0.0643

Sapp and Seiler* Whitehead and Gup* Whitehad and Gup* Whitehead and Gup* Whitehead and Gup*

302 42 56 60 54

0.1000 0.0051 0.0288 0.0191 -0.0021

Earnings per share growth

172

0.2817

62 13 61 36

0.1117 0.4428 0.1201 0.7900

1243

0.2458

Ansoff et al. Karger and Malik* Leontiades and Tezel Thune and House* Sales growth Ackelsberg and Arlow Ansoff et al. Bracker et al. Bracker and Pearson* Fulmer and Rue* Fulmer and Rue* Fuimer and Rue* Fulmer and Rue* Karger and Malik* Klein* Klein* Klein* Leontiades and Tezel Pearce el al. Thune and House*

14

124 62 217 188 151 67 53

115 13 9 29 21 61 97 36

a^e

o^p

X^

0.0111

0.0314

385.31"

0.0965

0.0099

0.0866

97.75**

0.0019

0.0096

0.0000

0.98

0.0757

0.0197

0.056

13.02

0.0838

0.0107

0.0731

104.16**

0.9991 0.2399 0.2520 0.2226 0.0010 -0.1583 0.0352 0.0526 0.5567 0.1854 0.0200 0.0964 0.1414 0.4200 0.3300

continued

364

BRIAN K. BOYD

Table III continued Category

N

Price-earnings ratio

123

0.2385

13 61 49

0.4063 0.2000 0.2420

Profitability

840

0.0722

Fulmer and Rue* Fulmer and Rue* Fulmer and Rue* Fulmer and Rue* Kallman and Shapiro* Klein* Ktein* Klein* Pearce et at.

151 67 53 115 298 9 29 21

97

-0.0868 -0.0820 0.1882 0.1803 0.0570 0.0173 0.0319 0.1449 0.3000

Return on assets

404

0.0960

21 13 61 97

0.2152 0.7298 0.2195 0.1800 -0.0123

Karger and Malik* Leontiades and Tezel Welch

Grinyer and Norburn* Karger and Malik* Leontiades and Tezel Pearte et at. Whitehead and Gup* Whitehead and Gup* Whitehead and Gup* Whitehead and Gup*

42 56

0.0217

-0.0179

0.46

0.0151

0.0106

0.0045

12.68

0.0259

0.0194

0.0065

10.46

0.0061

0.0087

-0.0026

5.54

0.0428

0.0113

0.0315

33.47'*

-0.0164

60 54

-0.0356 -0.0460

Return on equity

909

0.0806

Kaliman and Shapiro' Leontiades and Tezel Rhyne* Sapp and Seiler* Whitehead and Gup* Whitehead and Gup* Whitehead and Gup* Whitehead and Gup*

298 61 36 302 42 56 60 54

0.1100 0.1361 0.3622 0.0790 -0.0317 -0.0261 -0.0204 -0.0122

Return on investment

782

0.1050

Burt* Fulnier and Rue* Fulmer and Rue* Fulmer and Rue* Fulmer and Rue* Kallman and Shapiro* Karger and Malik* Rhyne' Wood and LaForge*

14 151 67 53 115 298 13 36 35

0.6403 -0.0959 -0.1627 0.1053 0.1876 0.0830 0.7239 0.4683 0.5808

' controlled for industry effects " significant at a - 0.05

0.0037

STRATEGIC PLANNING AND FINANCIAL PERFORMANCE

365

The effect size for earnings growth is 0.2075, and is based on a sample size of 1013. Wood and LaForge (1979), Karger and Malik (1975), Burt (1978) and Bracker and Pearson (1986) all found strong, positive effect sizes, while Fulmer and Rue (1974) reported three samples of small or negative effects. These differences lead to a significant x^ = 97.75. Without the unusually large Ackelsberg and Arlow (1985) estimate, the effect size is much lower at r= 0.1227, but the x^ of 29.74 is still significant. Deposit growth produced consistently low effect size estimates. Using banking data from Whitehead and Gup (1985) and Sapp and Seiler (1981) (A'= 514), r= 0.642. The x^ = O-97 indicates that variation across these studies is due entirely to sampling error. Growth in earnings per share is based on four studies and a sample size of 172. The effect size r= 0.2817, and the difference in variance is significant (x'= 13.02). Sales growth is the most common performance measure, and is cumulated from 15 samples and 1243 firms, and produced r= 0.2458. Again, removal of the Ackelsberg and Arlow (1985) estimate lowers the estimate of Fto 0.1623. For this smaller effect size, difference in variance across studies is still significant (X' = 26.07). The price-earnings ratio had a strong effect size in this study, with r = 0.2385. The correlation variance is 0.0037, and is due almost entirely to sampling error. This finding was based on a sample size of 123 firms cumulated from three studies. For profitability, individual studies found small negative, and moderate positive effect sizes. The cumulated eflect size is r= 0.722, based on a sample size of 840. The variance across studies is non-significant. Return on assets was reported in eight studies, with a sample size of 404. The effect size is 0.0960, with an effect size variance of 0.0259. Variance across studies is not significant. Return on equity also yielded a very small effect size: T= 0.0806, with a sample size of 909. The x^ for this estimate is not significant. Finally, return on investment had a moderate effect size. Using a sample size of 782, r= 0.1050. Two studies found negative correlations, while others found strong positive effect sizes. This yielded a significant x^ = 33.47. Rhyne's (1986) estimate of return on investment differed from standard calculations of ROI.' ' Consistency of Effects In addition to differences in magnitude of effect sizes, performance measures also reported substantial variation in consistency of effect size estimates. Growth measures, for example, revealed a very wide range of estimates across studies, ranging from extremely strong positive effects to very weak negative effects. Profitability measures generally yielded smaller, but more consistent effect size estimates. Cumulated effect sizes for profitability and various return measures ranged from 0.07 to 0.11, with only moderate or no significant variation across studies. Finally, two items of particular interest to shareholders (growth in earnings per share and the P/E ratio) produced consistently high correlations with no significant variation across studies.

366

' •1, •

BRIAN K. BOYD DISCUSSION

A major limitation of previous research is the sample size ot individual studies. Welch (1984) studied 49 firms, Grinyer and Norburn (1975) 21 firms, and Burt (1978) only 14 firms. Individually, the statistical power of these studies is not great. The meta-analysis allows estimates based on a pool of over 2000 organizations, representing a substantial improvement in statistical power. Measurement Error

Initial results from the meta-analysis would indicate that the effect of strategic planning on firm performance is very weak, as most ofthe aggregate effect sizes range between 0.05 and 0.15. However, these findings assume perfect measurement of both planning and performance. Presence of measurement error will consistendy lower the estimate of a correlation coefficient or effect size. Consequendy, if a cumulated effect size r=O.U is observed, and it is known that some measurement error occurred, it can be inferred that the population effect size is greater than 0.11. As noted in previous reviews, the potential for measurement error abounds in these studies. One source of potential error concerns the quality of planning programmes. Most studies simply classify firms as planners or non-planners, or organize them into ordinal categories. In practice, this usually only measures the degree of planning formalization, and not the quality of plans being made or firm's commitment to these plans. Additionally, one must consider the appropriateness ofa plan to a given situation (Lorange, 1979). Complicating this issue is the nature of data collection. Most studies {e.g. Karger and Malik, 1975; Pearce et al., 1987; Welch, 1984) mailed surveys to CEOs. However, as Grinyer and Norburn (1975) noted: 'because planning processes are complex, and a spontaneous reaction to questions was thought to be important, the limitations of mailed questionnaires made them particularly inappropriate'. Grinyer and Norburn interviewed roughly five executives in each firm to get a more detailed assessment of planning in each organization. A second major criticism is that some studies use cross-sectional data. Strategic decision-making today may not show up in the balance sheet for several years. Despite this, several papers correlate planning and performance data from the same year. Use of longitudinal data would allow assessment of a dynamic relationship between planning and performance. Measurement error is clearly a major limitation when interpreting these findings. If the reliabilities of planning and performance indicators were known, the observed effect size could be corrected by the following formula:

Where r^ and r^^ are the reliabilities of planning and performance, ^"observed is the observed effect size, and r^^^ is the true population effect size. For example, assume that performance is measured without error, and that measurement of planning has relatively low reliability - 0.40, perhaps. The r^^ for ROI would change from 0.11 to 0.18; earnings per share growth would improve from 0.28

STRATEGIC PLANNING AND FINANCIAL PERFORMANCE

367

to 0.46. In meta-analysis, it is standard to correct for measurement error (or attenuation) before cumulating the effect sizes. However, most papers included in this study either use single items or do not report reliabilities when using index measures. Since the reliabilities for planning and performance cannot actually be assessed here, the effect sizes for the meta analysis should be interpreted as the lower bounds of the true effect size. The effect of measurement error can be readily illustrated in a structural model. Structural equation modelling (Joreskog and Sorbom, 1986) combines factor analysis and regression, and a major advantage of this approach is that it is possible to control for the effects of measurement error in a regression. One can readily estimate a structural model of the effect of planning on EPS growth, using the effect size reported in table III. This model is shown in figure la. The structural coefficient for the effect of formal planning on EPS growth is 0.280, and the corresponding r^ for this model is 0.078. One can re-estimate this relationship while controlling for measurement error. The 6/^ and 0, terms represent measurement error, and can also be described as (1 - reliability) for a variable (Hayduk, 1987). In this example, we assume the reliability of the planning measure is 0.60 - measurement error for planning (Of,) is consequently (1 - 0.60) or 0.40. We also assume a reliability of 0.95 in measuring EPS growth, and 0, is set at 0.05. This model is shown in figure Ib. When measurement error is considered, the relationship between these variables is dramatically different. The structural parameter is estimated as 0.467, and the r'^ for this model is nearly double at 0.138, demonstrating that the effect size estimates in table III underestimate the true relationship between planning and performance. la, Standard regression model Planning

* EPS Growth 0.280

Ib. Fixed parameters for tneasuremeni error e^

•' Planning 0.40

* EPS Growth 0.467

9^

>

0.05

Figure 1. Effect of measurement error Moderating

Variables

One criticism of studies linking planning and performance is the lack of controls for moderating effects. Beard and Dess (1981), for example, concluded that industry type is a major determinant ofa firm's profitability. Sampling firms in a cross-section of industries might then cloud the effect of planning on performance. A second moderating variable is firm size. Robinson and Pearce, after completing a study of strategic planning in small firms, concluded that 'firm size is a critical contingency variable in strategic management research and theory development' (1983, p. 204). However, the studies included in this meta-analysis do attempt to control for these potential mediators. Of the 21 studies analysed here, 18 control for industry effects. Some, like Grinyer and Norburn (1975), compare performance of sample

368

BRIAN K. BOYD

firms to industry averages. Most, though, conduct separate analyses for each industry group sampled. Highlighting this problem, Rhyne (1986) offers effect sizes with and without industry controls. ROI, for example, yields an effect size of 0.575 without industry controls, and a smaller effect size of 0.406 when controlling for industry differences. Only four studies confronted the moderating effect of firm size. Whitehead and Gup (1985) conducted separate analyses based on firm size, while Sapp and Seiler (1981) included size as a predictor in their analysis of variance. Unfortunately, the way in which most of these were designed does not allow moderators to be cumulated in the meta-analysis. Suggestions for Future Research

What suggestions can be made to improve future research in this area? The first suggestion is the use of a structural model to incorporate estimates of mca.surement error for both firedictor and dependent variables. This approach helps assure that propositions "can be developed and tested unencumbered by the contamination of measurement error that may seriously bias parameter estimates and threaten the validity of conclusions drawn from their values' (Herting, 1985, pp. 263-4). To control for measurement error, it is necessary to measure both planning and performance with multiple indicators. Multiple indicators are desirable when a construct can be measured in several ways, as well as in the face of measurement error (James ^^ a/., 1982). A prerequisite for such analysis is a more rigorous measurement of strategic planning processes. Most studies treat planning as a nominal or ordinal value at best: planners versus non-planners, or a series of ordered categories {e.g. budgeting, operational planning, long-range planning, strategic planning). An alternative to the use of planning categories is the use of indicators of strategic planning activity within firms. Armstrong (1982), for example, described five components of formal strategic planning: specification of objectives; generation of strategies; evaluation of strategies; monitoring results; and seeking commitment to these plans from organizational members. A set of survey items could be developed to measure organizational activity on each of these components. Armstrong's framework has several advantages over categorical measures of planning: first, it subsumes measurement used in previous studies: i.e. generation of plans and their evaluation would reflect formaJization of planning; commitment would reflect perceived importance of planning. A second benefit is that this model assumes several dimensions to the strategic planning process. A common critique of studies in this area is that simply measuring the formality of a planning process is an oversimplification of the f)lanning-performanee relationship. Thus, using Armstrong's framework, it would be possible to determine the effect of specific aspects of the planning process (or interactions between these elements) on performance measures. A second component of this model is a series of indicators for performance. Several studies (Bagozzi and Phillips, 1982; Keats, 1983) have argued that organizational performance should be measured with multiple rather than single indicators. The effect of planning processes would then be related to dimensions of performance, rather than single indicators. Woo and Willard (1983) identified

STRATEGIC PLANNING AND FINANCIAL PERFORMANCE

369

four dimensions of organizational performance: profitability, relative market position, change in profitability, and growth. A fmal suggestion for future research concerns moderating effects. As Pearce et al. (1987) observed, 'controls' for industry effects in previous studies usually only involve separate analyses by industry group. While this method offers a control of performance relative to competitors, this does not measure moderating effects of industry differences. Strategic planning may be much more useful in a turbulent environment than a placid one (Armstrong, 1982). Consequently, the correlation between planning and performance may be stronger in a turbulent environment, and weaker in a placid environment. Industry-specific studies reported positive findings for electronics (Bracker et al., 1988), dry cleaning (Bracker and Pearson. 1986), and retailing (Burt, 1978); null efTects were reported for motor carriers (Kallman and Shapiro, 1978), Studies of banking institutions were split evenly between reporting null effects (Klein, 1979; Whitehead and Gup. 1985) and positive effects (Sapp and Seiler, 1981; Wood and LaForge, 1979). Studies of non-profit organizations reported strong benefits of strategic planning (Odom and Boxx, 1988; Van de Ven, 1980). Objective measures of organizational environment {e.g. Dess and Beard, 1984) could be incorporated into the model to test for an interaction between strategic planning and environmental turbulence on firm performance. One final issue is the number of studies excluded from this analysis. Of the 26 empirical papers which test the impact of strategic planning on firm performance, one-fifth were unusable in the meta-analysis because of inadequate reporting. Early and recent papers are equally at fault for poor reporting. While recognizing space limitations of academic journals, the necessary statistics could be added very easily: it requires no more space to include the value of a nonsignificant correlation than a 'NS' in a table. Similarly, references to nonsignificant /-values could be expanded to include the actual statistic. At least one author noted that these statistics were included in the original manuscript, and deleted by the journal at time of publication.

CONCLUSION Early adopters of strategic planning took comfort in the findings of Thune and House, Ansoff f/ a/., and other initial studies regarding the financial rewards of strategic planning. Unfortunately, later analyses were not as reassuring. Firms which are questioning the need for strategic planning should remember two points from this body of research: first, existing research is subject to a great deal of measurement error, thus seriously underestimating the benefits of planning. Second, while the average effect size is small, many firms do report significant, quantifiable benefits from participating in the strategic planning process. Identification of methodological limitations in these empirical studies suggests several avenues for future research. More rigorous measurement for formal planning, controls for industry effects, and separate analyses for different dimensions of organizational performance can improve substantially our level of understanding in this area.

570

BRIAN K. BOYD APPENDIX DERIVATION OF EFFECT SIZES

All mean based studies used the point biserial correlation as an estimate of the effect size. The point biserial is described by Hunter et al. (1982, p. 98) and McNemar (1969, p. 214). Correlational studies used the zero order correlation between planning and performance. All other transformations used here are described either in Hunter et al. or McNemar. Ackelsherg and Arlow (1985): effect size estimates based on table A3, comparison for all firms. Standard deviations were not published, and provided subsequently by Robert Ackelsberg as: change in sales change in earnings

planners

non-planners

0.543 0.724

0.296 0.492

Ansoff etal. (1970): reported several averaging measures for performance. Effect sizes based on their methods 2 and 3. Bracker, Keats, and Pearson (1988): effect size based on univariate i^-tests, using the transformation: Bracker and Pearson (1986): effect size based on univariate F-tests reported in table 2. Burt (1978): reported separate analyses for growth in profits and ROI averaged over both two and three years. Effect size estimates based on an average of the two. Karger and Malik (1975): effect size data can be computed only on the subset for machinery industry, A^= 13. Kallman and Shapiro (1978): effect size estimates based on F statistics reported by Kallman (1976, p. 76) from table 12 of his dissertation. Two formulae each were used to measure earnings/revenue and earnings/capital. These were averaged to estimate respective effect sizes. Effect sizes are: revenue income/revenue income/capital income/equity

0.041 0.057 0.083 0.110

Klein (1979): reported separate analyses for small, medium and large banking institutions. Effect sizes are based on the /"-statistic. Leontiades and Tezel (1980): reported a chi-square analysis. McNemar (p. 225) suggests use of the fourfold point correlation, or the transformation: Odom and Boxx (1988): effects sizes based on /^-statistics. Pearce, Robbins, and Robinson (1987): reported Pearson correlations between planning formality and performance measures. Rhyne (1986): reported means for four levels of planning: budgeting, annual planning, long range planning and strategic planning. Means for budgeting and

STRATEGIC PLANNING AND FINANCIAL PERFORMANCE

371

strategic planning were used as anchors of the planning-non-planning continuum. Rhyne computes absolute mean differences, and also differences which standardize for industry effects. The latter offer more conservative effect sizes, and are the values used in this meta-analysis. The effect sizes for absolute and standardized measures are (estimates in parentheses are corrected for attenuation using a reported Cronbach's alpha = 0.75): absolute measures:

roi

0.574834

(0.66376)

roe

0.403695

(0.466146)

roi

0.405600 0.313632

(0.468346) (0.362161) Sapp and- Seiler (1981): reported an ANOVA between planning and four performance measures, while controlling for size, location, scope, and holding company. The main effect for planning is reported as an F-statistic. i Thune and House (1970): reported ANOVA for tests of hypotheses with and without controls for industry differences. Data on the former is shown graphically in figure 3 of their article. Additional data to compute effect size estimates was taken from table 4-3-B of Thune's (1967) master's thesis. Estimates are: relative measures:

roe

sales earn/share stock price earn/equity earn/capital

0.33 0.79 0.31 0.56 0.53

Wood and LaForge (1979): effect size based on their averaging method A, for comprehensive versus non-planners. NOTES • Chuck McSween and Nica Mrowzinski assisted with the literature review and data collection. Ernest Kallman and Robert Ackelsberg provided additional data from their respective studies. I am also indebted to three anonymousyA/5 reviewers who provided helpful suggestions and comments. 11 ] The 3 unpublished dissertations listed in table I all reported null effects. [2] Credit for this observation lies with an anonymousyA/^ reviewer. REFERENCES and ARLOW, P. (1985). 'Small businesses do plan and it does pay off. Long Range Planning, 18, 5, 61-7.

ACKELSBERG, R .

ANSOFF, H . I., AVNER, J., BRANDENBURG, R . G . , PORTNER, F . E . and RADOSEVICH, R .

(1970). 'Does planning pay? The effects of planning on success of acquisitions in American firms'. Long Range Planning, 3, 2, 2-7. ARMSTRONG, J. S. (1982). 'The value of formal planning for strategic decisions: review of empirical research'. Strategic Management Journal, 3, 197-211. BAGOZZI, R . P . and PHILLIPS, L . W . (1982). 'Representing and testing organizational theories: a holistic construal'. Administrative Science Qitarterlj, 17, 459-89.

372

. BRIAN K. BOYD

(1986). 'Review of developments in meta-analytic method'. Psychological Bulletin, 99, 388-99. BEARD, D . and DESS, G . (1981). 'Corporate level strategy, business level strategy, and firm performance'. Academy of Management Journal, 24, 663-88. BRACKER,J. S. and PEARSON, J . N . (1986). 'Planning and fmancial performance of small, mature firms'. Strategic Management Journal, 7, 503-22. BRACKER, J . S., KEATS, B . W . and PEARSON, J . N . (1988). 'Planning and fmancial performance among small firms in a growth industry'. Strategic Management Journal 9, 591-603. BRAHM, R . and BRAHM, C . B . (1987). 'Formal planning and organizational performance; assessing emerging empirical research trends'. Presented at the 1987 National Meetings of the Academy of Management. BuRT, D. N. (1978). 'Planning and performance in Australian retailing'. Long Range Planning, 11, 3, 62-6. CAMILLUS,J. C . (1975). 'Evaluating the benefits of formal planning*. Long Range Planning, 8, 3, 33-40. CHILD, J. (1972). 'Organizational structure, environment and performance: the role of strategic choice'. Sociology, 6, 1-22. COOPER, H . M . (1982). 'Scientific guidelines for conducting integrative research reviews'. Review of Educational Research, 52, 291-302. COOPER, H . M . (1984). The Integrative Research Review. Beverly Hills, Cal.: Sage. COOPER, H . M . and ROSENTHAL, R . (1980). 'Statistical versus traditional procedures for summarizing research findings'. Psychological Bulletin, 87, 442-9. DESS, G . G . and BEARD, D . (1984). 'Dimensions of organizational task environments'. Administration Science Quarterly, 29, 52-73. FARLEY, J. U. and LEHMANN, D . R . (1986). Meta-analysis in Marketing: Generalization of Response Models. Lexington, Mass.; Heath. FuLMER, R. M. and RUE, L. W . (1974). 'The practice and profitability of long-range planning'. Managerial Planning, 22, 6, 1-7. GERSHEFSKI, G . W . (1970). 'Free for all; corporate models - the state of the art'. Management Science, 16, B-303-12. GERSTNER, L . (1972). 'Can strategic planning pay off?' Business Horizons, 15, 5-16. GLASS, G . V., MCGAW, B . and SMITH, M . L. (1978). Meta-analysis in Social Research. Beverly Hills, Ca!.; Sage. GLUCK, F . W . , KAUFMAN, S. P. and WALLECK, A. S. (1980). 'Strategic management for competitive advantage'. Harvard Business Review, 58, 4, 154-61. GREENLEY, G. (1986). 'Does strategic planning improve company performance?' Long Range Planning, 19, 2, 101-9. GRINVER, P . H . and NORBURN, D . (1975). 'Planning for existing markets: perceptions of executives and financial performance'. Journal of the Royal Statistical Society, (A), 138, 70-97. G U P , B . E . and WHITEHEAD, D . (1983). 'Shifting the game plan: strategic planning in financial institutions'. Economic Review Federal Reserve Bank of Atlanta, 68 (December), 22-33. GuYNES, C. S. (1969). 'An analysis of planning in large Texas manufacturing firms'. Unpublished doctoral dissertation, Texas Tech University. HARJU, P. (1981). Attitude of Strategic Managers toward Formalized Corporate Planning. Turku, Finland; School of Economics. HAYDUK, L . A . (1987). Structural Equation Modeling with LISREL: Essentials and Advances. Baltimore: Johns Hopkins University Press. HEROLD, D . M . (1972). 'Long-range planning and organizational performance: a crossvaluation study'. Academy of Management Journal, 15, 91-102. BANGERT-DROWNS, R . L .

STRATEGIC PLANNING AND FINANCIAL PERFORMANCE

373

J. R. (1985).'Multiple indicator models using LISREL'. In Blalock, H. M. Jr. (Ed.), Causal Models in tke Social Sciences New York: Aldine, 263-319. HOFER, C . (1976). 'Research on strategic planning: a survey of past studies and suggestions for future eiforts'. Journal of Economics and Business, 28, 261-86. HUNTER, J. E., SCHMIDT, F . L . and JACKSON, G . B . (1982). Meta-analysis: Cumulating Research Findings Across Studies. Beverly Hills, Cal.: Sage. JAMES, L . R . , MULIAK, S. A. and BRETT, J. M. (1982). Causal Analysis. Beverly Hills: Sage. JoRESKOG, K. G. and SORBOM, D . (1986). LISREL VI: Analysis of Linear Structural Relationships by the Method of Maximum Likelihood. Mooresville, Ind.: Scientific Software. KALLMAN, E . A. (1976). 'An empirical study oflong-range planning in the motor freight industry'. Unpublished doctoral dissertation, City University of New York. KALLMAN, E. A. and SHAPIRO, H . J. (1978). 'The motor freight industry - a case against planning'. Long Range Planning, 11, 81-6. KARGER, D . W , (1973). 'Integrated formal long range planning and how to do it'. Long Range Planning, 6, 4, 31-4. KARGER, D . W . and MALIK, Z . A. (1975). 'Long-range planning and organizational performance'. Long Range Planning, 8, 6, 60-4. KEATS, B . N . (1983). 'On the measurement of strategic performance'. Paper presented at the Strategic Management Society annual conference, Paris. KING, W , R . (1983). 'Evaluating strategic planning systems'. Strategic Management Journal, 4, 263-77. KLEIN, H . E . (1979). Growth, Profit, and Long Range Planning in Banks. Ann Arbor, Mich.: University of Michigan Press. KLEIN, H . E . (1981). 'The impact of planning on growth and profit'. Journal of Bank Research, 12, 2, 105-9. KUDLA, R . J. (1980). 'The effects of strategic planning on common stock returns'. Academy of Management Journal, 23, 1, 5-20. KUDLA, R . J. (1981). 'Strategic planning and risk'. Review of Business and Economic Research, 17, 1-14. LEONTIADES, M . and TEZEL, A. (1980). 'Planning perceptions and planning results'. Strategic Management Journal, 1, 65-76. LINDSAY, W . M . , BOULTON, W . R . , FRANKLIN, S. and RUE, L . W . (1981). 'Strategic management effectiveness: a longitudinal study'. Paper presented at the Academy of Management conference. LORANGE, P. (1979). 'Formal planning systems: their role in strategy implementation'. In Schendel, D. E. and Hofer, C. W. (Eds.), Strategic Management: A New View of Business Policy and Planning. Boston: Little, Brown, 226-41. MAL(K, Z . A . and BASU, S. (1986). 'Formal integrated long-range planning: its impact on fmancial risk decisions'. Business Horizons, 29, 2, 80-2. MALIK, Z . A. and KARGER, D . W . (1975). 'Does long-range planning improve company performance?' Management Review, 64, 27-31. MCNEMAR, Q . (1969). Psychological Statistics. New York: Wiley. NAJJAR, M . A. (1966). 'Planning in small manufacturing companies: an empirical study'. Unpublished doctoral dissertation, Ohio State University. ODOM, R. Y . and Boxx, W. R. (1988). 'Environment, planning processes, and organizational performance of churches'. Strategic Management Journal, 9, 197-205. PEARCE, J. A., FREEMAN, E. B . and ROBINSON, R . B . (1987). 'The tenuous link between formal strategic planning and performance', Academy of Management Review, 12, 658-75. PEARCE, J. A., ROBBTNS, D . K . and ROBINSON, R . B. (1987). 'The impact of grand strategy and planning formality on financial performance'. Strategic Management Journal, 8, 125-34. HERTING,

374

. .

BRIAN K. BOYD

:

A. and SUGDEN, B . (1971). 'Purposes and effectiveness of formal planning systems'. In Vancil, R. F. (Ed.), Formal Planning Systems, 1971. Cambridge, Mass.: Harvard Business School, 234-50. PRASAD, S . B . (1984). 'The paradox of planning in banks'. The Bankers Magazine, 16, 3, 77-81. RHYNE, L . C . (1986). 'The relationship of strategic planning to financial performance'. PERKINS,

Strategic Management Journal, 7, 423-36.

(1987). 'Contrasting planning systems in high, medium, and low performance companies', yourfia/ of Management Studies, 24, 363-85. RiNGBAKK, K. A. (1972). 'The corporate planning life cycle - an international point of view'. Long Range Planning, 5, 3, 10-20. ROBINSON, R . B . and PEARCE, J. A. (1983). 'The impact of formalized strategic planning on financial performance in small organizations'. Strategic Management Journal, 4, 197-207. ROSENTHAL, R . (1979). 'The "file drawer problem" and tolerance for null results'. RHYNE, L . C .

Psychological Bulletin, 86, 638-41. RUE, L . W .

and

FULMER, R . M .

(1973). 'Is long-range planning profitable?' Proceedings

of the Academy of Management, Boston, Mass.; 66-73.

and SEILER, R . E . (1981). 'The relationship between long-range planning and financial performance of US commercial banks'. Managerial Planning, 30, 2, 32-6. SHAPIRO, H . and KALLMAN, E . (1978). 'Long-range planning is not for everyone'. Planning Review, 6, 5, 27-9. SHEEHAN, G . (1975). 'Long range strategic planning and its relationship to firm growth and variability: an explorative, empirical investigation'. Unpublished doctoral dissertation. University of Western Ontario, Canada. SHRADER, C . B . , TAYLOR, L . and DALTON, D . R . (1984). 'Strategic planning and SAPP, R . W .

organizational performance: ?icv\\\c2\a^'pva.\&3X. Journal of Management, 10, 2, 149-71. SMITH, M . L . and GLASS, G . V. (1977). 'Meta-analysis of psychotherapy outcome studies'. American Psychologist, 3 2 , 752-60. STEINER, G . A . (1979). Strategic Planning: What Every Manager Must Know. New York:

Free Press. (1967). 'An investigation into the effect of formal long range planning in selected industries'. Unpublished master's thesis, City College, City University of New York. THUNE, S. S. and HOUSE, R . J. (1970). 'Where long-range planning pays off. Business Horizons, 13, 4, 81-7. VAN DE VEN, A. H. (1980). 'Problem solving, planning, and innovation, Part 2. Test of the program planning model'. Human Relations, 33, 711-40. WELCH, J . B. (1984). 'Strategic planning could improve your share price'. Long Range Planning, 17, 2, 144-7. WHITEHEAD, D . D . and G U P , B . E . (1985). 'Bank and thrift probability: does strategic THUNE, S. S.

planning really pay?' Economic Review Federal Reserve Bank of Atlanta, 70, October,

14-25. Woo, C. Y. and WILLARD, G . (1983). 'Performance representation in business policy research: discussion and recommendation'. Paper presented at the Academy of Management annual conference, Dallas, Texas. WOOD, D . R . and LAFORGE, R . L . (1979). 'The impact of comprehensive planning on financial performance'. Academy of Management Journal, 22, 516-26. WoRTMAN, M. S., JR. (1979). 'Strategic management: not-for-profit organizations'. In Schendel, D. E. and Hofer, C . W. (Eds.), Strategic Management: A New View of Business Policy and Planning, 353-73. Boston: Little, Brown.