1 Testing Strategy Formulation and Implementation ...

Testing Strategy Formulation and Implementation Using Strategically Linked Performance Measures Dennis Campbell, Srikant Datar, Susan L. Kulp, and V.G. Narayanan* Harvard Business School Current Draft: December 2006 ABSTRACT: This study investigates whether strategically linked performance measures reveal information about the quality of a firm’s business strategy. The strategy literature describes business strategies using the concepts of formulation, implementation, and fit. The management accounting literature links these strategy concepts with the selection and use of performance measures. Building on these two streams we examine whether and how the performance measurement system can be used to distinguish between formulation, implementation, and fit problems. We analyze balanced scorecard data from a field-site which formulated, implemented, and subsequently abandoned an innovative operating strategy. Managers learned the strategy was ineffective over a two year period. We find that the company’s strategically linked performance measures systematically reveal more timely information about problems with the strategy. Furthermore, the performance measures distinguish between problems with strategy formulation, implementation, and fit. The results are consistent with a well implemented, but poorly formulated, strategy at the research site. Additionally, the results imply a poor fit between the strategy and the firm’s internal resources. These results provide evidence that strategically linked, balanced scorecard measures can be used (1) to evaluate the strategy promptly and (2) to distinguish between strategy formulation, implementation, and fit problems. I.

INTRODUCTION

Management control theories argue that performance measurement systems consisting of financial and non-financial metrics linked to the firm’s unique strategy should facilitate learning through testing, validating, and revising the hypothesized relationships that describe the strategy (e.g., Eccles 1991; Kaplan and Norton 1996, 2000; Ittner and Larcker 2005; Julian and Scifres 2002; Shreyogg and Steinmann 1987).

For example, Kaplan and Norton (1996) contend that balanced scorecards give

decision makers the ability to detect whether the company’s strategy is working or failing. We examine this idea by empirically investigating how specific information about the quality of a firm’s business strategy is revealed in strategically linked performance measures of a balanced scorecard (BSC). We conduct an ex-post audit of strategy outcome, strategy implementation, employee capability, and financial performance measures of Store24, a New England convenience store chain. In FY 1998 Store24 initiated a new store-level strategy to differentiate itself by improving customer experiences. *

The authors thank Store24 for use of its data. We thank Chris Ittner, Robert Kaplan, Ken Koga, Michael Maher, Joan Luft, Tatiana Sandino, Philip Stocken, Dan Weiss, two anonymous referees, and seminar participants at the AAA Annual Meeting in Orlando, Boston University, the EIASM conference, Harvard University, Management Accounting Section Mid-year Meeting in San Diego, Michigan State University, Ohio State University, University of Arizona, University of Michigan, and University of Southern California for their helpful comments and suggestions.

1

There was, however, significant variation in how much and how well individual stores executed against Store24's implementation plan, in how customers valued this strategy, and in financial performance across stores. Based on customer feedback during the next two years, Store24 reverted back to a traditional strategy that emphasized speed of service and operational efficiency.

Store24 monitored store

performance via a set of performance measures formulated in a BSC. This site provides an ideal setting to offer empirical evidence on the extent to which strategically linked performance measures reveal specific information about a firm’s business strategy. In particular, we are able to benchmark the information revealed in analyses of the relationships among the firm’s performance measures against field-based evidence on the actual problems discovered by management over subsequent time periods. The strategy literature identifies formulation as the ends (objectives and goals) and implementation as the means (action plans and allocation of resources) of the strategy (Snow and Hambrick 1980). The management accounting literature on strategic control systems links these concepts of strategy formulation and implementation with the selection of performance measures. In particular, the BSC framework advocates choosing performance metrics related to key financial and customer objectives, the firm's internal processes for achieving these objectives, and organizational capabilities necessary to execute its internal processes. Moreover, performance measures should be explicitly linked in hypothesized "cause-and-effect" relationships that depict the firm's strategy (Kaplan and Norton 1996; 2004). Improvements in measures of organizational capabilities are expected to drive improvements in the execution of internal processes which in turn lead to customer and financial outcomes. Thus, the BSC framework explicitly recognizes interrelationships between strategy-specific measures of financial and customer outcomes and input-oriented "performance drivers" related to the firm's internal processes and organizational capabilities. Managers formulate specific strategies based on ex-ante expectations about how the strategy will translate into organizational objectives (e.g., increased profitability). Moreover, managers translate these

2

action plans into internal processes that will implement the formulated strategy.1 We conceptualize the quality of strategy formulation as the marginal effect of increases in strategy specific customer outcome measures on the firm’s financial objectives. We view the quality of strategy implementation as the marginal effect of increases in input-oriented internal process measures on the firm’s strategy specific customer outcome measures.

A performance measure related to internal processes may not be a leading

indicator of financial performance if (1) the action plan chosen to implement the strategy, as represented by input-oriented internal process measures, does not improve specific strategic customer outcomes (poor strategy implementation) or (2) the formulated strategy improves customer outcomes, but does not deliver expected financial outcomes (poor strategy formulation). We use the framework described in the preceding paragraphs to illustrate how strategically linked performance measures in Store 24’s BSC can be used to systematically reveal information about problems with the firm’s strategy.

Store24’s performance measurement system contained information to

differentiate between poor strategy formulation and poor strategy implementation.

As part of the

customer perspective, Store24 management measured the extent to which individual stores provided an entertaining experience (i.e., a strategy-specific customer outcome measure). Store24 management also developed a store-level action plan to implement this strategy, mapped the action plan into operating standards, and measured store-level conformance with these standards as part of its BSC internal process perspective (i.e. a strategy-specific input measure). Thus, all stores worked on executing against these operating standards to implement the new strategy. There was, however, significant variation in how well the strategy was implemented in different stores and in how customers experienced the implementation. Measures of unique internal processes (hereafter, 'input measures') are positively related to strategyspecific customer outcome metrics (hereafter, 'outcome measures') while outcome measures are negatively related to financial performance. The results are consistent with a well implemented, but poorly formulated, strategy at our research site.

1

For a framework articulating the interrelationships among the choice of strategic objectives, action plans, and performance measures, see Ittner and Larcker (2001).

3

Theory from the strategy literature suggests that problems with a particular business strategy may arise due to lack of "fit" with internal resources such as employee capabilities (Amit and Schoemaker 1993; Dierickx and Cool 1989). In this paper, strategic fit with internal resources is conceptualized in three ways. First, the marginal effect of increases in measures of a firm’s internal capabilities on strategyspecific input measures captures the extent to which the firm's internal capabilities drive its ability to execute its internal processes. Second, the marginal effect of measures of internal capabilities on the quality of strategy implementation captures complementarities between the firm's internal capabilities and the processes it uses to satisfy customers. Third, the marginal effect of measures of internal capabilities on the quality of strategy formulation captures complementarities between the firm’s internal capabilities and its chosen strategy. Our results indicate that cross-sectional differences in store capabilities account for differences in the success of Store24’s strategy.

Low employee skill levels do not directly affect strategy

implementation. But in stores with low employee skills, even when outcome measures are high, financial performance is poor. Conversely, in stores with high employee skills, when outcome measures are high, financial performance is strong. These results are consistent with a "poor fit" hypothesis in which regardless of how thoroughly Store24 implements its strategy, for the strategy to succeed, store level employee capabilities need to be high. Our study makes three contributions to the accounting literature on performance measurement. First, we describe and illustrate a method to use performance measurement systems to analyze and evaluate strategy implementation and formulation.

Several studies in management accounting

demonstrate relationships among financial performance metrics and non-financial measures such as product quality and customer satisfaction (e.g., Banker, et. al. 2001; Ittner and Larcker 1998b; Nagar and Rajan 2001).

However, these studies do not explicitly analyze measures of a firm’s strategy and

capabilities and, consequently, the extent to which such measures provide information useful for timely detection of problems with the strategy. In our study, contrary to prior studies, improvements in outcome

4

measures are negatively (or not) related to a variety of financial measures because the strategy, though well-implemented, is poorly formulated. Second, despite the academic evidence that non-financial performance measures typically lead financial performance, Ittner and Larcker (1998b) document that many executives do not tie together firm-specific non-financial metrics with lagging accounting measures.2

Our paper shows that the

relationships between non-financial performance measures and financial performance depend on characteristics of the strategy captured by those measures. A lack of a relationship between firm-specific non-financial metrics and accounting returns may be informative about (1) the firm’s strategy formulation, (2) its strategy implementation, and (3) the effect of a firm's internal capabilities on strategy implementation or the fit of the formulated strategy with the firm’s internal capabilities. We provide some of the first field-based empirical evidence on the potential for a set of strategically linked financial and non-financial performance measures to distinguish among these three alternatives. Third, we extend prior research on the relationships between non-financial performance measures and financial performance by examining the potential moderating effect of employee capabilities. Prior research suggests that business models are typically depicted by linear relationships between financial and non-financial performance metrics (Rucci et al. 1998, Kaplan and Norton 1996; 2000). Except for Ittner and Larcker (1998b), prior empirical work typically ignores potential nonlinearities in relationships among performance measures. Moreover, these studies do not examine interactions among non-financial performance measures as a source of nonlinearity that may moderate these relationships (Ittner and Larcker 1998a). The results in this paper are subject to the caveat that the field-based nature of our research limits the generalizability of our findings. However, the unique nature of a firm’s strategy dictates that the performance measures and links between these measures, articulated in the firm’s business model, are likely to be firm-specific. Future research should provide additional evidence from other settings of the extent to which business model-based performance measurement systems capture information useful for 2

Consistent with this, Store24 management did not perform statistical analyses linking the performance measures together, although the metrics were consistently collected across stores and across time.

5

monitoring strategic progress. Our main contribution is to describe a method that can be generally applied to other settings and industries to isolate the effects of strategy formulation, strategy implementation, and strategic fit. The remainder of the paper proceeds as follows. In section II we discuss prior literature, and in Section III we develop hypotheses. Sections IV and V present our research site and empirical research design. Results are presented in section VI. We conclude the paper in section VII. II.

THE LINK BETWEEN THE STRATEGY LITERATURE AND STRATEGIC CONTROL SYSTEMS FRAMEWORKS

In this section, we discuss links among the notions of strategy formulation, implementation, and fit with internal resources found in the strategy literature and emerging frameworks of strategic performance measurement found in the management accounting literature. Strategy Formulation vs. Strategy Implementation There does not appear to be clear consensus on the definitions of strategy formulation and strategy implementation within the strategy literature. However, several conceptual papers distinguish these concepts based on the choice of strategic objectives and the choice of action plans to achieve those objectives, respectively. Notably, Andrews (1971) put forth a general definition of strategy as: … a pattern of major objectives, purposes, or goals and essential policies and plans for achieving those goals, stated in such a way as to define what business the company is in or is to be in and the kind of company it is or is to be. Andrews' definition explicitly identifies two separate processes, formulation and implementation, and the interrelation between these two concepts (Sloan 2005). Similarly, Chandler (1962) refers to strategy as "… the determination of the basic long-term goals and objectives of the enterprise and the adoption of courses of action and allocation of resources necessary for carrying out those goals." As with Andrews, this definition of strategy distinguishes between formulation and implementation by encompassing both elements of ends (goals and objectives) and means (courses of action and allocation of resources). Subsequent strategy researchers continue the dichotomy between choosing strategic objectives (strategy formulation) and detailing action plans to achieve those objectives (strategy implementation) (Snow and Hambrick 1980).

6

Strategic Fit with Internal Resources and Capabilities More recent research introduces a strategy’s "fit" with the firm's internal resources and capabilities. The resource-based view of the firm (RBV) posits that an organization’s unique, valuable, and difficult to replicate resources and capabilities form the basis for sustainable competitive advantage (Amit and Schoemaker 1993; Dierickx and Cool 1989).3 Others (e.g., Itami and Roehl 1987; Dierickx and Cool 1989; Nanda 1996, Hitt et al. 2001) classify resources such as brand name, customer loyalty, technical know-how, firm-specific human capital, and employee skills as strategic. The RBV concept of interconnectedness of asset stocks (Dierickx and Cool 1989) posits complementarities among accumulations of various “invisible assets” or resources such as human capital. This literature focuses on the role of strategic resources and capabilities in successful strategy formulation and implementation. Strategic Control Systems and the Balanced Scorecard Recent management accounting research incorporates these strategy frameworks by articulating linkages between performance measure choice, strategy formulation, and strategy implementation. The value-based management framework (Ittner and Larcker 2001) emphasizes interrelationships among the choices of strategic objectives, action plans, and performance measures. Proponents of strategic or “business model” based performance measurement systems advocate formulating performance measurement systems around a diverse set of financial and non-financial performance metrics linked to the firm’s unique strategy (e.g., Eccles 1991; Kaplan and Norton 1996). The literature on management control systems has long argued that one role of control and performance measurement systems is the facilitation of strategic feedback and learning (Ittner and Larcker 2005), This literature echoes the basic means-ends concepts found in the strategy literature by emphasizing strategic feedback and learning as a process of systematically using data generated by the firm's control systems to evaluate strategic plans, activities, and ultimately, results (Schreyogg and Steinmann 1987; Julian and Scifres 2002). Similarly, in settings where there is uncertainty over the firm's "profit drivers", Dye (2004) demonstrates that performance measurement systems consisting of 3

The RBV explains cross-sectional differences in strategy choices and outcomes. Related insights apply to Store24, a decentralized company in which stores are heterogeneous with respect to demographics and employee capabilities.

7

"intermediate" measures of firm processes facilitate experimentation in that they allow managers to determine which of several underlying processes are most strongly linked to future profitability. Kaplan and Norton's (1996; 2004) BSC framework is perhaps most explicit in advocating that performance measures be chosen based on hypothesized relationships between measures of financial objectives and unique measures of nonfinancial "performance drivers".

This framework can be

conceptualized as consisting of strategic outcome metrics in the financial and customer perspectives and strategic input metrics in the internal process and learning and growth perspectives. These measures should be explicitly linked in a series of hypothesized "cause-and-effect" relationships that represent the firm's strategy (Kaplan and Norton 1996; 2004). With its emphasis on intangible assets (e.g., employee capabilities) as the basis for successful strategy implementation, the BSC framework directly parallels the notion of "fit" from the strategy literature. III.

A FRAMEWORK FOR STRATEGIC HYPOTHESIS TESTING

In this section, we construct a set of general hypotheses guided by the literature in strategy and strategic performance measurement. We describe how to distinguish among problems with strategy formulation, strategy implementation, and strategic fit. Managers formulate strategies based on ex-ante expectations about how the strategy will translate into organizational objectives (e.g., customer satisfaction or profitability). Moreover, managers develop action plans to implement the strategy and detail the internal processes needed to achieve the stated objectives. Consider the relationship between a performance outcome measure, PO , such as profit and strategy input measures, S I , related to the firm's internal processes for achieving its strategic objectives:

PO = f ( S I , ε P ) where ε P represents factors that affect performance other than S I . effectiveness of internal processes by examining

Managers can evaluate the

∂PO . Problems with the strategy as formulated and ∂S I

8

implemented are revealed unambiguously if

∂PO ≤ 0 .4 This suggests the following straightforward ∂S I

hypothesis, stated in null form, as a starting point for evaluating the performance of a given strategy. H10: Ceteris Paribus strategy inputs are positively related to financial performance. Figure 1 summarizes this and subsequent hypotheses. H10 will be rejected if the input metrics show no (or a negative) relationship with financial performance. This may be caused by two reasons: (1) the action plan and internal processes chosen to implement the strategy do not result in the achievement of strategy-specific objectives or (2) the formulated strategy does not deliver expected returns, that is, achieving the chosen strategic objectives does not result in superior financial performance. Distinguishing between problems in the strategy formulation (e.g. choice of strategy-specific objectives) and problems with strategy implementation (e.g. choice of action plans to achieve strategy-specific objectives) would be possible if an intermediate strategy-specific customer outcome metric, SO , were available.5 In this case, we have

PO = g ( SO , ε SO ) SO = h( S I , ε SI ) where ε SO and ε SI represent factors that affect performance other than SO and S I , respectively. Problems with the strategy as formulated and implemented would be revealed unambiguously if

∂PO ∂PO ∂SO ∂P ∂S ∂P ∂S = × ≤ 0 .6 This occurs if either: (1) O ≤ 0 and O > 0 or (2) O > 0 and O ≤ 0 . ∂S I ∂SO ∂S I ∂SO ∂S I ∂SO ∂S I Case 1 is consistent with good implementation but poor formulation of strategy. Input measures (unique internal processes chosen to implement a strategy) are positively related to customer outcomes, but these customer outcomes are not positively related to the firm's overall financial performance objectives. The

4 5

We assume throughout that higher values of Si indicate better performance. The performance outcome PO is distinct from the strategy-specific customer outcome SO . PO represents a high-

level performance objective such as profit. SO represents a strategy-specific objective such as customer experience or satisfaction with unique product or service attributes. Note that firms often derive a measure of expected performance. In such cases, the strategy’s performance would be evaluated relative to this target, rather than relative to zero. 6

9

second case is consistent with poor implementation but good formulation of strategy. Input measures are not positively related to customer outcomes, but the customer outcomes are positively related to the firm's overall financial performance objectives. In a BSC framework with strategically linked input- and output-oriented metrics, this suggests the following hypotheses for evaluating whether observed problems with a given strategy’s performance is due to poor implementation or poor formulation: H20: Ceteris Paribus strategy inputs are positively related to strategy-specific customer outcomes. H30: Ceteris Paribus strategy-specific customer outcomes are positively related to financial performance.7 H20 or H30 could be rejected if the given strategy requires the presence of complementary intangible assets to succeed. The RBV literature suggests that returns to formulating and implementing a strategy may depend on the level of complementary strategic resources. Much of this literature argues that specialized complementary resources provide the basis for sustainable competitive advantage (Teece 1986; Tripsas, 1997).

Empirical research in this area demonstrates that strategic resources, such as

human capital, interact with strategy inputs and strategy outcomes to affect performance (Hitt et. al. 2001). That is, the marginal effects of customer outcomes on financial performance (quality of strategy formulation) and input measures on customer outcomes (quality of strategy implementation) are determined by whether the level of a complementary strategic resource is below the level necessary for positive returns to the formulated strategy. We refer to these strategic resources as internal capabilities and focus on manager and employee skills. Thus, we have the following hypotheses for evaluating whether observed problems in strategy formulation and strategy implementation are due to poor fit with internal capabilities: H40: Ceteris Paribus the marginal impact of increases in strategy inputs on strategyspecific customer outcomes is positively related to the level of internal capabilities.

7

Our framework could also be used to identify cases where unique internal processes chosen to implement strategy are negatively related to strategy-specific outcomes and strategy-specific outcomes are negatively related to overall performance objectives. In this case, unique internal processes chosen to implement strategy are positively related to overall performance objectives, but the detailed analysis would highlight problems of poor implementation and poor formulation that make such a strategy unsustainable.

10

H50: Ceteris Paribus the marginal impact of increases in strategy-specific customer outcomes on financial performance is positively related to the level of internal capabilities. Thus far, the hypotheses have focused on identifying problems with strategy formulation, implementation, and fit from the perspective of senior management. It is at this level where strategic objectives are chosen and strategy inputs (unique internal processes) for implementing those objectives are selected. However, problems of strategic "fit" may also arise if a company's internal capabilities do not allow it to achieve the desired strategic inputs needed to successfully execute the implementation plan, suggesting the following straightforward hypothesis. H60: Ceteris Paribus internal capabilities are positively related to strategy inputs. IV.

RESEARCH SITE

Store24 is a privately held convenience store retailer in New England, the 4th largest in the region. Its stores, located through Massachusetts, New Hampshire, Rhode Island, and Connecticut, are grouped into nine geographic divisions, each with its own division manager. Stores are homogenous in many aspects of their operations including compensation, technology, management structure, and product pricing, but they vary in size, geographic location, market demographics, and product mix. The company’s primary product categories include cigarettes, beverages, snacks, prepared foods, and lottery tickets. Revenues totaled approximately $180 million in fiscal year 1998 (May 1, 1998 to April 30, 1999). Store24 employed 800 people including 740 store managers and crew and 60 corporate level employees. The skills and experience of these employees vary widely overall and across stores. Store24 operates in a mature environment with competition from convenience stores, gasoline retailers, and drug stores. Traditionally, convenience store retailing focused on short-term productivity (e.g., inventory and cash control). As the convenience store industry matured and competition intensified, marketing, customer service, and brand name emerged as differentiating factors. Before FY 1998 and after FY 1999, Store24 did not differentiate itself; rather it focused on excelling at traditional service quality metrics such as physical environment (cleanliness and store layout) and quality of the customer experience (fast, friendly service) (Fitzsimmons and Fitzsimmons, 2001).

11

During FYs 1998 and 1999 (that is from May 1, 1998 to April 30, 2000), Store24 formulated a strategy aimed at increasing same-store sales and margins because growing via new sites was difficult. “Location is a primary driver of store performance. However, we are stymied on the growth front due to a lack of acceptable new sites. This has led to a focus on optimizing our existing sites through an increasing emphasis on store-level marketing and operations,” explained Store24’s CFO. To achieve its goals, Store24 changed its strategy to creating entertaining in-store atmospheres that would differentiate its stores from those of competitors. The Differentiation Strategy Store24 implemented this new, innovative store-level strategy during the first quarter of FY 1998 (i.e., beginning May 1, 1998). It aimed to differentiate its stores while maintaining performance on traditional productivity measures. Successful retailers, such as Disney stores, offer “fun and interactive” shopping experiences. Store24’s CEO believed that adopting a similar strategy would improve financial performance. Store24 provided a fun in-store atmosphere by emphasizing specific themes. Store-level strategy execution centered on a large display case (i.e., “endcap”) featuring themeoriented promotional items and store decorations that fostered employee interaction with customers. For example, during the old movie theme stores featured life-size cutouts of movie stars, endcaps contained high-margin videos of old movies, and old movies became a conversation piece. The themes sought to attract urban adults between the ages of 14 and 29 years, a growing market segment and Store24’s target market. A senior manager explained, “The [Differentiation] strategy was really playing off of the urban, young adult market. Marketers know that this demographic gets bored easily and needs to be stimulated. We wanted this group to always see new and different things in the store.” In contrast to the basic service quality component, store-managers were accorded autonomy in implementing the differentiation strategy. That is, although all stores were required to implement the new strategy, how they implemented or how much they implemented varied across stores. Corporate defined a theme and provided the endcaps, but store employees possessed considerable flexibility in strategy execution. Thus, manager and crew skills were at least as important as theme choice to the strategy’s

12

success. Store24’s controller explained, “Our best managers really took the strategy to heart. The strategy served as an outlet for manager and crew creativity. However, other managers put minimal effort into this strategy and even stocked traditional items such as chips on the endcaps saying they needed the product space.” The differentiation strategy, as originally conceived, centered on the physical environment. But the interaction between store employees and customers was crucial to the strategy’s success. Senior management intended the themes and promotions to serve as points of interaction that would help Store24 establish relationships with customers and cross-sell high margin products. Explained a senior executive, “The endcaps and displays under the [differentiation strategy] had the dual intention of building a rapport with customers and bumping up the average sales per customer. We felt that store management and crew could use the displays as “ice-breakers” in talking with customers. In addition, the margins on the promotional items featured under the [differentiation] strategy were typically two to four times the margins of our traditional products.

When customers were browsing or “window shopping” we

encouraged store crew to direct the customer’s attention to these promotional items.” Store24 looked to its differentiation strategy to attract new customers and increase store sales, specifically, sales of highermargin, strategy-specific products, and thereby boost store profits. Performance Measurement System Store24 used a balanced scorecard-based performance measurement system.

The company

collected information on a variety of performance measures at various levels of the organization and at various frequencies. Management collected store-level financial performance metrics quarterly. It monitored store-level customer measures less frequently. Between the 1st and 4th quarters of FY 1999, an external research firm solicited feedback from customers at 65 stores about Store24, its product selection, and other factors that would persuade them to shop at Store24 more often. Customers ranked unique attributes related to the differentiation strategy that they found appealing; among these was “fun place to shop,” “entertaining,” and “unexpected.” Additionally, the research firm conducted semiannual telephone surveys of self-identified convenience store customers in Store24’s major markets to

13

assess the likelihood of customers shopping at Store24, name recognition of Store24, and, for Store24 customers, the quality of merchandise, price, and store cleanliness. Store24 translated the components of its strategy into a set of store-level operating standards and measured store-level conformance to these standards via walk-through audits conducted twice per quarter. During these announced visits management evaluated store performance on various dimensions including in-store image, in-stock position, and store appearance. The walk-through audit score quantified the store-level implementation of Store24’s operating strategy. For FYs 1998 and 1999, the standards reflected both the differentiation strategy as well as traditional service quality metrics.

A store’s

differentiation score referred to a separate measure of conformance to only standards related to the differentiation strategy such as actions in terms of themes and products that would make Store24 a fun and entertaining place to shop. Store24 also measured conformance to store-level operating standards through monthly surprise visits or “mystery shops.” The mystery shop review, which consisted of twenty high-level questions, helped to ensure the validity of the walk-through audit scores. Scores on the announced and unannounced visits are significantly and positively correlated. Senior and division management considered employee skills critical to consistent implementation of the store-level operating strategy. Accordingly, Store24 measured manager and crew skills through biannual evaluations of performance in guest interactions, merchandising, machinery maintenance, store condition, adherence to policy, loss prevention, and problem solving.

Store manager and crew

compensation was tied to, for example, store-level profit and strategy implementation measures. To encourage implementation of the differentiation strategy specifically, employee rewards were based on both the differentiation score and total walk-through audit score. As a result of these measures and incentives, all stores implemented the new strategy. But, implementation of the differentiation strategy was not straightforward. Beyond the physical environment and stocking of new products, it required store staff to establish relationships with customers and sell high-margin products. Implementation of the strategy varied significantly among stores. Even when stores implemented the strategy well, there was variation in how customers experienced the new strategy.

14

There was also significant variation across stores in profitability. These variations allow us to draw conclusions about both strategy formulation and implementation. We analyze Store24’s BSC data, including the walk-through audit scores, financial performance measures, and employee metrics, to learn more about the hypothesized causal links around the strategy.8 Strategy Change Store24 incorporated the differentiation component during FYs 1998 and 1999. During this time, management monitored the scorecard. Store-level execution of operating standards (strategy-inputs) declined and then gradually increased over this period (Figure 2), and the strategy-specific customer outcome measure followed the same pattern. In each quarter of FY 1999 Store24 posted a higher profit than in the corresponding quarter of FY 1998 (Figure2). Store24 management, however, could not attribute the strong financial performance to the new strategy as growth in profits closely tracked industry averages. In FY 2000, based on negative customer feedback, Store24 concluded that the differentiation strategy had failed and refocused its strategy on traditional service quality activities.9 See Figure 3 for a timeline of events related to Store24’s strategy change. Based only on trends in the balanced scorecard metrics, it was difficult for management to definitively disentangle problems with strategy formulation from those with strategy implementation. That is, it wasn’t easy to pinpoint why the strategy failed. Store24 senior management also identified potential problems related to the “fit” of the differentiation strategy with the existing level of employee capabilities at its stores. Senior management believed successful store-level implementation of this strategy required performance in complementary, difficult to measure activities. To leverage the environment into financial performance, skilled employees needed to establish customer relationships. Senior management believed that high skill levels enhanced and low skill levels limited, the relationship between implementation of the differentiation strategy and store performance. Explained Store24’s CFO, “Managers and crew that were already skilled in our core [efficiency based] strategy and other basic store operations such as cash, labor, and inventory control, 8

We omit the mystery shop scores due to their correlation with walk-through audit scores and data availability. We cannot disaggregate mystery shop scores into basic service quality and differentiation strategy implementation measures. 9 Store24 received negative feedback from in-store comment cards, telephone surveys and focus groups.

15

were able to devote considerably more time to implementing the [differentiation] strategy and to tailor this strategy based on knowledge of their customers.

These skills made it easier to build the

[differentiation] strategy on top of the basics.” The success of local strategy implementation relied on manager and crew interactions with customers and local market knowledge. Absent these complementary activities, differentiation implementation might not translate into improved store performance, and might, in fact, adversely affect performance, particularly on the productivity dimension. Using the information learned by management over time about problems with the strategy as a benchmark, we seek to examine the insights derived from systematic analysis of the scorecard measures. V.

EMPIRICAL RESEARCH DESIGN

Our sample consists of financial, non-financial and customer performance measures for 65 stores during fiscal years 1998 and 1999 (i.e., during implementation of the differentiation strategy). To obtain scores on store-level differentiation, we disaggregate the walk-through audit scores into their constituent components. We have data for store-level implementation of the differentiation strategy for the fourth quarter of FY 1998 and the second and third quarters of FY 1999. We supplement Store24’s balanced scorecard data with information on store competition and demographics gathered during the same time period. To gain familiarity with the business environment we interviewed Store24 senior management and reviewed company documents about the measurement system and strategic learning process. Finally, we interviewed five store managers about store-level execution of the differentiation strategy. Empirical Variables Financial Performance To improve its financial performance, Store24 can: i) increase customers; ii) increase spending per customer; or iii) increase the efficiency and effectiveness of store personnel (decrease costs). Operating profit (Profit) summarizes these categories at the store level; it is defined as revenues (Sales) from general merchandise, lottery tickets, money orders, and phone cards less cost-of-goods sold, utilities expense, and labor expense. This measure reflects the financial components that Store24 believes store-

16

level management can influence and is the primary measure used by management to evaluate overall store financial performance. We measure Profit as annual operating profit during FY 1999. This is the period we are able to match with available strategy input measures, strategy outcome measures, and measures of employee capabilities. FY 1999 is the second year of Store24's differentiation strategy, allowing enough time for any start-up problems in implementation to be worked out. In all analyses, we scale Profit by square feet of store selling space. Non-financial Performance Measures Measure of Strategy Inputs. Store-level measures of strategy inputs capture store-level activities that management believes drive strategy success. Senior and mid-level management measure performance by conducting walk-through audits twice per quarter. Management awards points based on compliance with 78 operating standards related to in-store image, in-stock position, merchandising and marketing management, and facilities appearance. A percentage score is calculated by dividing total awarded points by total potential points.10 We disaggregate stores’ total operational audit scores into scores that reflect the store’s compliance with operating standards (strategy input measures) for the differentiation strategy.11 Input_Diff reflects a store’s percentage score on operating standards related to differentiation; it reflects how well each store executed this strategy or the quality of the “inputs”.

We use the strategy input

measure taken at the beginning of FY 1999 in all our empirical analyses (Input_Diff). Measure of Basic Service Quality. During the walk-through audit, Store24 management also measures basic service quality items such as in-store image, fast service, and in-stock position. BSQ is the average percentage score on operating standards related to basic service quality taken over the same period as our measure of strategy inputs.

10

Mystery shop scores are positively and significantly correlated with walk-through audit scores and cannot be disaggregated. Adding mystery shop scores to the analyses does not change the results. 11 Due to extra credit points for strong implementation of Differentiation, a store’s score on Input_Diff can reach 135%. Employees were compensated based on a separate measure of this strategy normalized by total available points. Thus, they were induced to invest in this implementation.

17

Measure of Strategy-Specific Customer Outcomes. A third-party research firm conducted in-store customer interviews at a subset of stores throughout FY 1999.12 Customers rated the attributes they “liked most about this particular Store24,” including whether Store24 was “entertaining,” “a fun place to shop,” and “unexpected.” We collect the metrics specific to the differentiation strategy; these metrics comprise a reliable set as evidenced by a Cronbach’s coefficient alpha of 0.9596. Each attribute is measured as the proportion of surveyed customers who stated that they liked this characteristic about a particular store; Outcome_Diff is the average of these measures.

Outcome_Diff reflects whether

customers observe and value the new strategy; it represents a strategy-specific customer outcome measure resulting from implementation of the differentiation strategy (strategy input measures). Employee Capabilities. Store24 relies on its employees to execute strategy at the point of customer contact. Thus, we take measures of manager and crew skills as our primary measures of the firm’s strategic resources. Store24 evaluates its managers during the 2nd and 4th quarters of each fiscal year. Managers are rated, on a five-point scale, on many dimensions including ability to retain, train, and interact with crew; customer service; merchandising; time and labor management; maintaining store safety; and technology use. A store manager’s skill rating (MgrSkill) is the average score across all dimensions. Crew skills are rated on a five-point scale along similar dimensions; all non-management employee scores are averaged to devise a store’s crew skill rating (CrewSkill).

In all subsequent

empirical tests, we use the skill metrics taken in the beginning of FY 1999.13 Were Store24’s senior management simply to infer skill ratings from actual store performance, a store’s manager and crew skills ratings would reflect store performance rather than exogenous skill levels. As shown in Table 2, neither manager nor crew skills exhibit significant correlations with Profit.. Thus, on average, senior executives do not provide higher skill ratings to employees in better performing stores. Data on individual employee skill ratings for a sample of 20 stores reveals variation in skill ratings across

12

Data was collected for approximately 15-20 stores per quarter. Our results are invariant to the use of average skills throughout FY 1999 rather than taking the skill metrics at the beginning of FY 1999.

13

18

individual employees within a particular store, reflecting senior management’s desire to identify individual skills rather than infer skill-level from store performance. Control variables. Store24 collects demographic information for the half-mile radius around each store. Many of these demographics relate to population and foot traffic in the trading area of a given store and are highly correlated. Because many of these variables are correlated we use factor analysis to identify the underlying constructs and find one population factor with an eigenvalue greater than one. Population represents daily activity around the store location. It comprises primarily the student population (pre-high school, high school, and college), pedestrian count rating, and population density. Income is an estimate of the median level of annual disposable income available to a family for grocery and convenience store purchases in the surrounding area.which Store24 obtains from a third-party research firm. Because we expect high income and/or large population areas to offer more sales potential, these variables should relate positively to financial performance. Finally, having more competing stores in the area is expected to be associated with lower financial performance. To control for this effect, we include Competition which reflects the number of competing stores within a half-mile radius of each store. We also control for unobservable location characteristics by including rent per square foot (Rent). Store24 pays a premium to rent facilities in locations with, for example, high visibility. Cross-sectional differences in Rent should capture store location differences which we do not directly control for in our analyses. Finally, we include a measure of store size (SQFT), measured as square feet of retail selling space, and a variable that indicates whether a store is open 24 hours per day (24Hours). Methodology We test the baseline hypothesis, H1, by estimating the following equation: PROFITi t = α 0 + α1 Input _ Diffi + α 2 MgrSkilli + α 3CrewSkilli +α 4 BSQi + α 5Competitioni + α 6 Populationi + α 7 Incomei + α 8 24 Hoursi +α 9 SquareFeeti +α10 Renti +ε i

(1)

Where PROFITi denotes operating profit for store i during FY 1999. We estimate this equation using OLS on a cross-sectional sample of 65 stores.

To reduce collinearity due to the inclusion of the

19

interaction terms and to maintain interpretability of the coefficients, we mean center the interaction variables prior to estimation (Aiken and West 1991). If the strategy-input measure leads to improved financial performance, we expect α1 to be positive and significant. Finding no (a negative) relationship implies that improved strategy implementation is not (negatively) associated with improved performance, signaling problems with strategy formulation, strategy implementation or strategy fit. Consistent with the framework outlined in section III, we test for problems in strategy implementation (H2), strategy formulation (H3), and strategy fit (H4, H5, and H6) by using OLS to estimate the following equations. PROFITi = γ 0 + γ 1Outcome _ Diffi + γ 2Outcome _ Diffi × MgrSkilli + γ 3Outcome _ Diffi × CrewSkilli + γ 4 MgrSkilli + γ 5CrewSkilli + γ 6 BSQi + γ 7Competitioni + γ 8 Populationi + γ 9 Incomei + γ 10 24 Hoursi + γ 11SquareFeeti + γ 12 Renti + ηi

Outcome _ Diff

t i

(2)

= β 0 + β1 Input _ Diffi + β 2 Input _ Diffi × MgrSkilli + β 3 Input _ Diff i × CrewSkilli + β 4 MgrSkilli + β 5CrewSkilli + ε i

(3)

Input _ Diff i = α 0 + α 1 MgrSkilli + α 2 CrewSkilli + μ i

(4)

Equation (2) is analogous to equation (1) where the outcome measure replaces Store24’s internal input measure14. Equation (3) tests the relationship between the outcome measure and Store24’s input measure.15 A positive correlation, β1, indicates relatively good implementation of the differentiation strategy because the outcome measure correlates with the input metrics.

β1>0, γ 1 ≤0 would provide

evidence in favor of H20 and against H30 implying a good implementation of a bad strategy. Conformance to operating standards (strategy inputs) leads to the desired strategy-specific customer

14

In untabulated tests, we estimate equation 2 separately for stores where Outcome_Diff was measured during the first 6-months and second 6-months of FY 1999 respectively. In these tests, for stores measured in the first (second) 6-months, we measure manager and crew skills as the average of skills as measured during the end of the fourth quarter of FY 1998 (second quarter of FY 1999) and the second quarter of FY 1999 (fourth quarter of FY 1999). The results from estimation of equation 2 on each of these sub-samples are substantively similar to those reported in Table 5 on the full sample of stores. These results mitigate the potential that the findings in our paper are due to any mismatch in performance measurement periods within Store24. 15 Note that we do not include demographic and other store location characteristics as controls in Equation 3. There is no a priori reason to believe that strategy-specific outcomes should be driven by these factors. However, we have estimated Equation 3 using the same controls as in Equations 1 and 2 and results are substantively similar.

20

outcome (customers view stores as “entertaining"), but the strategy-specific customer outcome does not translate into improved store financial performance. β1≤0, γ 1 >0 would provide evidence against H20 and in favor of H30; it is consistent with bad implementation of a good strategy. Strategy-specific customer outcomes (more entertaining stores) are associated with higher financial performance; however the strategy input measures do not lead to higher levels of strategy-specific customer outcomes. To test the complementary impact of Store24’s strategic capabilities on the relationships between input,

outcome

and

financial

performance

measures,

we

rely

on

the

interaction

terms,

λ2Outcome _ Diffi × MgrSkilli and λ3Outcome _ Diffi × CrewSkilli , for strategy formulation tests and β 2 Input _ Diffi × MgrSkilli and β 3 Input _ Diffi × CrewSkilli

for strategy implementation tests.

Significant coefficients on these variables indicate that the level of internal capabilities impacts the relationships among input measures, outcome measures and financial performance (H4 and H5). Finally, we use equation (4) to investigate the final part of the "strategic fit" hypothesis (H6) by examining the relationship between performance on the input metric (Input_Diff) and the level of internal capabilities (MgrSkill, CrewSkill). We include MgrSkill, and CrewSkill in equations (2) and (3) to account for any main effects of employee capabilities on store financial performance.16 Although scaling by store size (Square Feet) alleviates concerns with heteroskedasticity, we calculate p-values based on both OLS standard errors and Mackinnon and White’s (1985) heteroskedasticity consistent “HC3” standard errors with no substantive differences in results.17 RESULTS Descriptive Statistics

Table 1 provides descriptive statistics and Table 2 presents the correlation matrix for the sample of 65 stores. Note that the stores exhibit wide cross-sectional variability in both Store24’s input measure

16

Managers with high skills may, for example, more effectively manage labor and inventory costs which would have a direct effect on store-level financial performance. 17 White’s test for heteroskedasticity is not as reliable in small samples (Mackinnon and White 1985, Long and Ervin 2000). Long and Ervin (1997) suggest using the HC3 estimator for standard errors when heteroskedasticity is suspected. Although we have no a priori reason to suspect heteroskedasticity, we check p-values based on HC3 estimators for robustness (untabulated).

21

(Input_Diff) and outcome measure (Outcome_Diff). Outcome_Diff is negatively related to Profit.

The univariate correlations suggest that

Additionally, the outcome measure is significantly

positively related to Store24’s input measure (Input_Diff). Together, this provides preliminary evidence that the differentiation strategy was well implemented, as Store24’s view of good implementation corresponds to the customer outcome, but possibly poorly formulated due to the negative relation of the customer outcome with financial performance. Since stores vary on other factors that might affect financial performance (e.g., location and skills) we refrain from making conclusions based on these univariate tests. Competition, Population, Income, Sqft and Rent all exhibit significant correlations with Profit. Thus, these seem to be powerful controls for unobserved location characteristics that might affect store performance. Tests of H1 (Identifying Problems with Strategy Formulation and/or Implementation)

Table 3 reports the results of estimating the relationship between Profit and Store24’s assessment of stores’ internal conformance with strategic operating standards.

On average, the input metric,

Input_Diff, is not associated with Profit. This suggests that store-level effort to implement the new strategy was not translating into store-level profits. Manager skills significantly and positively relate to profit as does population in the surrounding area; competition is negatively related to profit. Rent per square foot is positively related to profit suggesting that higher rents are proxying for characteristics associated with better store locations. These results highlight that the hypothesized link between internal implementation of the action plans related to the new strategy and financial performance does not exist. However, it is unclear whether the strategy is poorly formulated or poorly implemented. Tests of H2 and H3 (Distinguishing between Problems of Formulation vs. Implementation)

Table 4 contains results from estimation of equation (3). On average, Store24’s input metric (Input_Diff) positively relates to the outcome measure (p