IIE Transactions (2001) 33, 837±846
A ¯exible simulation tool for manufacturing-cell design, II: response surface analysis and case study MARIA DE LOS A. IRIZARRY1 , JAMES R. WILSON2; and JAIME TREVINO3 1
Department of Industrial Engineering, University of Puerto Rico, Mayaguez, PR 00681, USA Department of Industrial Engineering, North Carolina State University, Raleigh, NC 27695, USA E-mail:
[email protected] 3 Electric Systems Technology Institute, ABB Power T & D Company, Inc., Raleigh, NC 27606, USA 2
Received April 1998 and accepted January 2000
We present a two-phase approach to design and analysis of manufacturing cells based on simulated experimentation and response surface methodology using a general manufacturing-cell simulation model. The ®rst phase involves factor-screening simulation experiments to identify design and operational factors that have a signi®cant eect on cell performance as measured by a comprehensive annual cost function. In the second phase of experimentation, we construct simulation (response surface) metamodels to describe the relationship between the signi®cant cell design and operational factors (the controllable input parameters) and the resulting simulation-based estimate of expected annual cell cost (the output response). We use canonical and ridge analyses of the estimated response surface to estimate the levels of the quantitative input factors that minimize the cell's expected annual cost. We apply this methodology to an assembly cell for printed circuit boards. Compared to the current cell operating policy, the simulation metamodel-based estimate of the optimum operating policy is predicted to yield average annual savings of approximately $425 000, which is a 20% reduction in annual cost. In a companion paper, we detail the structure and operation of the manufacturing-cell simulation model.
1. Introduction In this paper and a companion paper (Irizarry et al., 2000), we develop and demonstrate a simulation-based methodology for design and analysis of manufacturing cells. In Irizarry et al. (2000), we discuss a general manufacturing-cell simulation model for evaluating the eects of world class manufacturing practices on expected cell performance; moreover, we formulate a comprehensive annual cost function to facilitate comparison of alternative cell con®gurations. In this paper we explain how the simulation model is used in conjunction with appropriate techniques for response surface modeling and analysis to determine an optimal policy for cell design and operation. The rest of this paper is organized as follows. Section 2 contains a brief review of the literature on the use of simulation metamodels for design and analysis of manufacturing cells. We present our proposed cell design methodology in Section 3 along with a case study involving the assembly of printed circuit boards. Finally in Section 4 we summarize our main ®ndings, and we make recommendations for future work.
Corresponding author
0740-817X
Ó 2001 ``IIE''
2. Simulation metamodels for design and analysis of manufacturing cells Simulation metamodels (that is, regression or response surface models of simulation-generated outputs) provide an eective mechanism for simplifying the interpretation of results from a well-planned experiment by explaining a selected system response as a function of relevant input factors. Much of the recent work in this area concerns manufacturing applications (Yu and Popplewell, 1994). Jothishankar and Wang (1993) used metamodels to estimate the eects of eight predictor variables on the mean interdeparture time of ®nal assembled products from a JIT kanban system. Hira and Pandey (1983) developed regression models to evaluate the eciency of a manual ¯ow line having ®nite-capacity intermediate storage buffers. Lin and Chiu (1993) used metamodels to study the behavior of a ®xed ¯ow automatic robotics cell; in particular they sought to characterize: (i) long-run average cell behavior under steady-state operating conditions; and (ii) short-run (transient) cell performance under the in¯uence of machine breakdowns and job changes. Three recent studies focused on the use of simulation metamodels to optimize the performance of a manufacturing system. Lin and Cochran (1987) studied the
838 behavior of a complex ¯ow line for the fabrication of printed circuit boards; and they developed a seven-step procedure to estimate and minimize expected product ¯ow time. Madu and Chanin (1992) formulated a simulation metamodel of a maintenance ¯oat system to determine the number of standby units and repair technicians needed to minimize the expected cost of system operation subject to a lower-bound constraint on average equipment utilization. Shang and Tadikamalla (1993) used response surface methodology to optimize the expected throughput of an automated printed circuit board manufacturing line. Hurley (1994) developed a simulation-based methodology for estimating and optimizing the eects of design and operation factors on the expected performance of manufacturing cells. This methodology was applied to a speci®c manufacturing-cell devoted to the assembly of service parts for electric-power generation equipment. Hurley considered operational issues such as: lot sizing; number of cell operators; setup times; machine processing times; and material handling distances. Other factors included in the study were: maintenance programs; quality control strategies, and scheduling policies. A key feature of this work was the formulation of an annual cost function to measure overall cell performance. For each combination of the qualitative input factors, a simulation metamodel was constructed to describe the relationship between the quantitative input factors and expected cell performance; then a grid-search technique was used to determine the optimum levels of the quantitative input factors. The optimum levels of the qualitative input factors were determined using the Ryan±Einot±Gabriel± Welsch multiple-comparisons procedure (Welsch, 1977). We build upon the work of Hurley (1994) to develop the two-phase approach to cell design and analysis that is discussed in the rest of this paper.
3. Cell design via simulation-based response surface modeling and analysis: a case study In Irizarry et al. (2000), we detail a general manufacturing-cell simulation model for evaluating the eect of world class manufacturing practices on cell performance expressed in terms of an annual cost function with 10 major cost components: inventory carrying cost; lateness cost; setup cost; material handling cost; storage equipment cost; production labor cost; maintenance cost; quality cost; layout cost; and ¯oor space cost. Figure 1 depicts our overall approach to manufacturing-cell design and analysis based on the use of this simulation model together with eective techniques for response surface estimation and optimization. This methodology consists of four major steps: (i) selection of cell design and operation issues; (ii) development of a comprehensive cell performance measure; (iii) identi®cation of critical design
Irizarry et al. and operation factors for the cell; and (iv) optimization of cell performance as a function of the cell's critical design and operation factors. Step (ii) is discussed in the companion paper (Irizarry et al., 2000). In Sections 3.1±3.3 below, we describe steps (i), (iii), and (iv); and we illustrate the application of this methodology to the assembly cell for printed circuit boards that was introduced in Irizarry et al. (2000). Complete details on our methodology for cell design and analysis are given in Irizarry (1996). 3.1. Selection of cell design and operation factors In each individual application, the selection of relevant cell design and operation issues is situation-speci®c. All the issues elaborated in this section are relevant to the design of a new manufacturing-cell. If the objective of the analysis is to recon®gure an existing cell, then we need only consider issues that are relevant to restructuring that cell. The methodology proposed in this paper was applied to a cell for the assembly of printed circuit boards. A diagram of the current layout of the cell is given in Irizarry et al. (2000) and is not reproduced here to conserve space. The objective of this case study was to analyze product quality and the cell's customer service level. We sought to evaluate the impact of implementing one or more of the following world class manufacturing practices: reduced setup times (that is, quick changeovers); smaller unit load sizes; smaller lots; minor stoppages at the pick-and-place machines; quality at the source; or autonomous and preventive maintenance. The cell material handling system, stang requirements, inventory buer sizes, and part scheduling practices remained constant throughout the study. The cell operational issues addressed in this study ± that is, setup policy (SU), unit load size (UL), lot size (LT), machine minor stoppages (ST), quality policy (QL), and maintenance policy (MA) ± are the input factors for the factor-screening experimentation. 3.2. Identi®cation of critical design and operation factors The identi®cation of input factors with a signi®cant eect on cell performance is vital to the design and operation of a cellular manufacturing system (Buzacott, 1985). Since it is likely that only a few of the selected cell design and operation factors will have a signi®cant impact on cell performance, we must perform a factor-screening experiment as the ®rst step of our procedure; then in further experimentation, we consider only the signi®cant input factors. Table 1 summarizes the input factors that are relevant to the assembly cell for printed circuit boards along with the factor levels selected for the factorscreening experiment. In the case study, the factor-screening experiment was a 26 1 fractional factorial design in two blocks of 16
Flexible simulation tool for manufacturing-cell design II
839
Fig. 1. Methodology for design and analysis of manufacturing cells.
design points each. This is a resolution V design ± that is, no main eect or two-factor interaction is confounded with any other main eect or two-factor interaction (Montgomery, 1991). Within each block of 16 design points, we used the method of common random numbers to induce a block eect and thereby improve the eciency with which all main eects and two-factor inter-
actions are estimated (Schruben and Margolin, 1978). The objective of the screening experiment was to identify the important main eects and two-factor interactions. We evaluated cell performance at each design point using the generic cell simulator that we introduced in Irizarry et al. (2000). We set the total run length at 243 000 minutes, including a warm-up period of 9000 minutes
840
Irizarry et al.
Table 1. Description of factor levels for the factor-screening experiment Factor Level 1 (coded value )1)
Level 2 (coded value +1)
SU
quick changeovers (75% reduction) small (»10% of lot size) small lots (individual customer orders) bigger rolls of components quality at the source autonomous and preventive maintenance
UL LT ST QL MA
long (unreduced) setup times large (»50% of lot size) large lots (groups of customer orders) small rolls of components traditional inspections breakdown maintenance
after which all accumulated performance statistics were cleared. We performed a statistical analysis of the results of the factor-screening experiment for the case study using SAS's general linear model procedure GLM (Anon, 1990). In particular, we examined the estimated residuals for evidence of nonnormality. We judged the sample skewness and excess kurtosis of the estimated residuals to exhibit no signi®cant departures from normality. Stemand-leaf plots and normal probability plots of the estimated residuals provided further validation of the normality assumption. See Irizarry (1996) for a complete discussion of the techniques used to validate the assumptions underlying the factor-screening analysis described in the rest of this section. In the regression analysis for the factor-screening experiment of the case study, the signi®cance probabilities (P -values) for the corresponding estimated regression coecients were used to identify signi®cant main eects and two-factor interactions. For a signi®cance level of 5%, the signi®cant main eects were setup time (SU), unit load size (UL), lot size (LT), minor stoppages (ST), and maintenance (MA). The signi®cant two-factor interactions were: lot size with unit load size (LT UL); lot size with setup time (LT SU); lot size with minor stoppage (LT ST); and setup time with minor stoppages (SU ST). In the factor-screening analysis for the case study, the estimated regression coecient for the maintenance policy (MA) was highly signi®cant; however, this factor was not found to interact with any other factor. Taken over all design points (runs) with breakdown maintenance (that is, the current maintenance policy with coded value MA 1), the average cell operating cost is $4780 000. By contrast, the average annual cell operating cost taken over all design points (runs) with autonomous and preventive maintenance (MA 1) is $5440 000. Hence, the investments and interruptions caused by the high level of maintenance were found to have a negative impact on cell performance. We concluded that maintenance should be set at its low level (MA 1) in all subsequent experimentation.
3.3. Optimization of cell performance 3.3.1. Analysis of ®tted metamodels After the factor-screening analysis is complete, a second experiment is required to construct an adequate approximation to the target response surface and to estimate the optimal settings for the signi®cant input factors as well as the expected optimal response. A response surface experimental design is used to perform this task. In the case study, the corresponding simulation experiment included two qualitative factors (lot size, LT, and machine stoppages, ST) each at two levels; and for every combination of these qualitative input factors, we performed a 32 full factorial simulation experiment to estimate the expected annual cell operating cost as a function of the selected quantitative factors (setup time, SU, and unit load size, UL). As depicted in Fig. 2, the overall simulation macroexperiment consisted of four separate 32 factorial experiments corresponding to the nodes labeled 1 through 4 in the displayed ``experiment tree''. Moreover, as represented by branches a, b, and c in Fig. 2, three independent simulation runs were performed at each design point of each 32 factorial experiment, yielding a total of 27 runs per experiment and a total of 108 independent simulation runs in the entire macroexperiment. Table 2 summarizes the input-factor levels used in all these scenarios. As in the factor-screening experiment, a coded factor level of 1 corresponds to the current cell operating policy. We analyzed the simulation results for each of the experiments corresponding to nodes 1±4 of Fig. 2 using the SAS response surface regression procedure RSREG with a 5% signi®cance level (Anon, 1990). We estimated each response surface using a metamodel that is a secondorder (quadratic) function of the components of the design point X
SU; UL. Canonical and ridge analyses of the estimated metamodel yielded the optimal design point X^ , that is, the stationary point of the ®tted response surface; (Box and Draper, 1987, 332±381). Table 3 summarizes the
Fig. 2. Layout of the simulation macroexperiment.
Flexible simulation tool for manufacturing-cell design II
841
Table 2. Description of factor levels for the response surface macroexperiment Factor Level 1 ()1) SU UL LT ST
Level 2 (0)
Level 3 (+1)
long (unreduced) setup times quick changeovers (37.5% reduction) large (»50% of lot size) medium (»30% of lot size) large lots (groups of customer orders) small rolls of components
quick changeovers (75% reduction) small (»10% of lot size) small lots (individual customer orders) bigger rolls of components
Table 3. Results from the statistical analysis of the simulation macroexperiment Experiment 1 2 3 4
Signi®cant regressors
Estimated optimum response Y^ (X^ ) ($)
UL, SU, UL2, SU2 SU, SU2 UL, SU, UL2, SU2 SU, SU2
1954 )1425 1937 )161
905 522 638 378
Residual skewness and excess kurtosis )0.0071, 0.6259, 0.0677, )0.6946,
)1.0607 1.7765 )0.5288 5.3033
Standard error of Y^ (X^ ) ($) 5 82 4 2290
136 588 322 593
results of this analysis. Figures 3 and 4 contain plots of the estimated response surfaces for experiments 1 and 3, respectively. (We have suppressed the plots of the initial estimates of the response surfaces for experiments 2 and 4 because of obvious problems with these response surfaces that we address in Section 3.3.2.) We analyzed the residuals using the SAS procedure UNIVARIATE, which includes moments, quantiles, stem-and-leaf plots, box plots, and normal probability plots (Anon, 1990). We found that experiments 1 and 3 exhibited similar behavior in the shape of their estimated response surfaces, with both metamodels having the same signi®cant regressors as well as similar values for the predicted mean response at the corresponding stationary point. The P -values in the lack-of-®t tests for these metamodels are 0.948 and 0.964, respectively. Recall that each P -value represents the smallest level of signi®cance at which the observed value of the goodness-of-®t test statistic would cause us to reject the null hypothesis that the corresponding metamodel has the correct functional form. Since the observed P -values are much larger than the signi®cance levels commonly used for hypothesis testing, we found the metamodels for experiments 1 and 3 to exhibit no signi®cant lack-of-®t.
We found the sample skewness and excess kurtosis of the estimated residuals for experiments 1 and 3 to be consistent with the basic assumption of normally distributed simulation responses. Since the standard error of the estimated mean response Y^
X^ at the stationary point X^ is relatively small for experiments 1 and 3, we concluded that the corresponding response surfaces were estimated with sucient accuracy for the purposes of the case study. The original metamodels developed for experiments 2 and 4 resulted in negative predictions for the expected annual cost at the stationary point and in other regions of the input-factor space. Moreover, in experiments 2 and 4 we found that the estimated residuals exhibited signi®cant departures from normality ± especially in terms of the sample skewness and excess kurtosis of those residuals. Finally the standard error of Y^
X^ is much larger for experiments 2 and 4 than it is for experiments 1 and 3; and we found that introducing additional design points and then ®tting a higher-order metamodel did not improve the behavior of the estimated residuals as measured by their sample variance, skewness, and excess kurtosis. We concluded that in order to obtain usable response surface models in experiments 2 and 4, we must do the
Fig. 3. Estimated response surface for experiment 1.
Fig. 4. Estimated response surface for experiment 3.
842
Irizarry et al.
Table 4. Design points (SU, UL) added to estimate third-order response surfaces for experiments 2 and 4 Experiment 2
Experiment 4
(0.25, 0) (0.5, 0) (0.75, 0) (0.5, )0.5718)
(0.25, 0) (0.5, 0) (0.75, 0) (0.2, 0.5625) (0.2, )0.5718)
following: (i) identify and apply an appropriate variance stabilizing transformation of the original simulationgenerated responses (Box and Draper, 1987); and (ii) augment the set of regressors (independent variables) in the corresponding metamodels to include relevant higherorder terms. These conclusions are consistent with our previous experience in building simulation metamodels for certain types of textile production systems as discussed, for example, in Powell (1992). 3.3.2. Follow-up analysis for re®tted metamodels We examined several transformations of the response in our search for metamodels with a better ®t to the results observed in experiments 2 and 4 (Box and Draper, 1987, 280±293). Moreover, we added design points to these experiments to allow estimation of higher-order metamodels. Based on examination of the ®tted response surfaces for experiments 2 and 4 as described in Table 3, we augmented these experiments with three replications of each of the design points speci®ed in Table 4. Statistical analyses of these more complex metamodels were performed using SAS's general linear model procedure GLM (Anon, 1990). Table 5 contains a summary of the results of this follow-up analysis for experiments 2 and 4; and Figs. 5 and 6 respectively display the corresponding response surfaces. In the two-dimensional region of interest for experiment 2, we obtained the best results using the logarithmic transformation Z
X f Y
X lnY
X
1
of the simulation-based estimate Y
X of the cell's expected annual cost at each design point X
SU; UL.
Fig. 5. Estimated response surface for the logarithm transformation.
As shown in Table 5 and Fig. 5, we estimated the response surface for experiment 2 by ®tting a metamodel to the transformed response Z
X that is a third-order polynomial in the components of X . In the response surface ®tting scheme based on the transformation (1) for experiment 2, we estimated the variance of the prediction at each design point X using the delta method so that we took (Stuart and Ord, 1994, Section 10.6(a)) 2 ^ d f Y^
X VarY^
X : VarZ
X
2 dy For each design point X in response surface experiment 2 or 4, let W
X denote the row vector whose ®rst element is one and whose other elements are the corresponding regressors de®ned in Table 5; thus, for example, at each design point X in experiment 2, we have W
X W
SU; UL 1; SU; UL; SU2 ; UL2 ; SU3 ; UL3 : If D denotes the overall design matrix for a response surface experiment so that each row of D has the form W
X for the associated design point X in the experiment, then we have the standard regression result ^ r ^2Z W
X
DT D 1 WT
X ; VarZ
X ^2Z r
is the residual mean square for the regression where involving the transformed responses de®ned by (1). Combining (2) and (3) and inserting our ®nal estimate X^
Table 5. Results for selected transformations of the response in experiments 2 and 4
Signi®cant regressors Normalizing transformation Z(X)=f [Y(X)] ^ X^ at Predicted annual cost Y^
X^ =f 1 Z
^ the stationary point X ($) Standard error of Y^
X^ ($) Residual skewness and excess kurtosis
3
Experiment 2
Experiment 4
SU, UL, SU2, UL2, SU3, UL3 Z(X)=ln[Y(X)] 1647 899
SU, UL2, SU2, SU3 Z(X)=10 000/Y(X) 1730 652
26 649 0.2690, 1.4656
12 454 )0.0404, 0.5661
Flexible simulation tool for manufacturing-cell design II
843
Comparing the results presented in Table 3 for experiment 4, we found the predictions in Table 5 based on (7) and (8) to be much better behaved. Moreover for the transformed response (6), we found that the corresponding estimated residuals displayed no marked departures from normality. 3.3.3. Con®dence intervals for ®nal ®tted metamodels In summary, the ®nal ®tted simulation metamodels for experiments 1 through 4 are, respectively, Y^1 1986 904 Fig. 6. Estimated response surface for the inverse transformation.
of the stationary point into the result, we obtain our ®nal estimate of the optimal untransformed response for experiment 2, Y^
X^ f
1
^ X^ g expfZ
^ X^ g; fZ
5:75 10
3
8:87 10
23 320 UL 49 268 SU2 ; 0 1 14:367 2 0:021 4 UL 0:046 04 SU Y^2 exp@ 2:89 10 4 UL2 1:362 5 SU2 A; 0:024 5 UL3 0:914 6 SU3 Y^3 1967 606
31 675 UL
66 290 SU
22 845 UL2 51 536 UL3 ; and
4
10 000 SU 3:20 10 6 UL2
1 q d ^Z W
X^
DT D 1 WT
X^ SEY^
X^ f Y^
X^ r dy q ^ X^ ^ rZ W
X^
DT D 1 WT
X^ :
5 expZ
Comparing the results presented in Table 3 for experiment 2, we found the predictions in Table 5 based on (4) and (5) to be much better behaved. Moreover for the transformed response (1), we found that the corresponding estimated residuals displayed no marked departures from normality. The transformation judged to provide the best estimate of the response surface for experiment 4 was Z
X f Y
X 10 000=Y
X :
6
We estimated the response surface for experiment 4 by ®tting a metamodel to the transformed response Z
X in (6) that is a third-order polynomial in the components of X . Figure 6 depicts the ®tted response surface. Proceeding along the lines of (4) and (5) for the transformation (6), we obtained for experiment 4 ^
66 788 SU
2
4
and the associated standard error Y^4
34 686 UL
^X Y^
X 10 000=Z
7
^ X^ 2 g^ rZ SEY^
X^ f10 000=Z
q W
X^
DT D 1 WT
X^ :
8
^
and
2:55 10
3
SU2 1:71 10
3
SU3
;
where: Y^i is the estimated mean response at design point X
SU; UL within experiment i for i 1, 2, 3, 4. In each of the four experiments, an approximate 100
1 a% con®dence interval for the mean response at the stationary point is Y^
X^ t1
^ ^ ;
a=2;m SEY
X
9
where: m is the degrees of freedom for the corresponding residual mean square; t1 a=2;m is the 1 a=2 quantile of Student's t-distribution with m degrees of freedom (we took a 0:05); and 8 q > ^Y W
X^
DT D 1 WT
X^ > >r > > > > < for experiments 1 and 3; 1 q SEY^
X^ > > d ^ ^ > ^Z W
X^
DT D 1 WT
X^ f Y
X r > > dy > > : for experiments 2 and 4.
10 In the part of display (10) describing experiments 1 and 3 ^2Z ) is ^2Y (respectively, r (respectively, experiments 2 and 4), r the residual mean square for the regression performed on the untransformed (respectively, transformed) simulation responses. Table 6 summarizes the ®nal results for the ®tted metamodels in all four experiments. The con®dence intervals given in Table 6 clearly distinguish the scenarios described by experiments 1 and 3
844
Irizarry et al.
Table 6. Con®dence intervals for the predicted optimal mean response Experiment 1 2 3 4
Y^ (X^ ) ($) 1954 1647 1937 1730
905 899 638 652
95% con®dence interval on optimal mean response ($) [1944 [1587 [1928 [1705
from the scenarios described by experiments 2 and 4. Moreover since the overlap of the con®dence intervals for experiments 2 and 4 is so small and the corresponding 95% con®dence interval on the dierence between the expected optimal costs for experiments 2 and 4 does not contain zero, we concluded that experiment 2 will yield the overall optimal setting for the input factors and thus the minimum-cost cell con®guration. The coded values for the estimated optimal unit load size and setup time in experiment 2 are 0.5 and 0.2, respectively, as shown in Table 6. The optimal coded value for setup time corresponds to a 45% reduction in setup time. The optimal coded value for unit load size corresponds to seven images per panel for products 1 through 7 and 3.5 images per panel for products 8 through 13. Since the number of images per panel for products 1 through 7 must be an even number, it is not practical to force a unit load size of seven for these products. In this case we evaluated unit load sizes of six and eight for products 1 through 7, and we evaluated unit load sizes of three and four for products 8 through 13. The optimal input-factor settings for experiment 2 included the use of small rolls of components at the pick-and-place machine (ST 1) and small lot sizes (LT 1). Factors that were ®xed on the basis of the factor-screening experiment included traditional inspections (QL 1) and breakdown maintenance (MA 1). We evaluated the metamodel for experiment 2 at the modi®ed factor settings (that is, unit load sizes of six and eight for products 1 through 7 and unit load sizes of three and four for products 8 through 13) to understand the eects of these small changes in unit load sizes on the cell's total annual cost. We found the average observed dierence ($286) to be negligible when compared to the annual cost ®gures displayed in Table 6. This insensitivity of the estimated mean response to changes in unit load size in the vicinity of the stationary point led us to conclude that there was a range of near-optimal settings for the unit load size. Therefore, the cell designer has ¯exibility in setting the operating conditions to achieve optimal cell performance. We validated the ®nal metamodel for experiment 2 by assessing the statistical and practical signi®cance of the dierence between the metamodel-predicted mean response and the simulated-generated mean response at the same stationary point. The results from 20 independent
222, 355, 648, 221,
1965 1708 1946 1756
588] 443] 628] 083]
^ UL
^ SU
0.6378 0.5 0.6175 0.7
0.6268 0.2 0.6091 0.2
replications of the simulation model at the stationary point yielded an average annual cost of $1740 000 with a sample standard deviation of $14 000. We used these results to build an approximate 100
1 a% con®dence interval for the dierence between the metamodel- and simulation-based estimates of optimal cell performance that is given by n o1=2 Y^
X^ Y
X^ t1 a=2;g SE2 Y^
X^ SY2
X^ ;
11 where: SY2
X^ is the estimated variance of the simulation mean response Y
X^ based on 20 independent replications of the simulation at the design point X^ ; and g, our approximation to the ``eective'' degrees of freedom for the complex variance estimator SE2 Y^
X^ SY2
X^ in (11), is taken to be $ , !% n o2 4 ^ ^ SY4
X^ SE Y
X ; g SE2 Y^
X^ SY2
X^ m n 1
12 with m denoting the degrees of freedom for the residual mean square in the regression analysis used to estimate the associated response surface, and n denoting the number of independent simulation runs performed at X^ . (The basis for (11) and (12) can be found in Equations (15.4.15)±(15.4.17) of Hald (1952)). Table 7 summarizes the result of evaluating (11) and (12). Table 7 shows that the 95% con®dence interval (11) for the dierence between the metamodel-predicted mean response and the mean response from the simulation runs does not contain zero. Therefore, we concluded that there is a statistically signi®cant dierence between the metaTable 7. 95% con®dence interval for dierence between metamodel- and simulation-based predictions of optimal cell performance Simulation-based estimates
Metamodel-based estimates
^
Y
X =$1736 060 p SY
X^ =$14 091= 20=$3151 n=20
Y^
X^ =$1647 899 SEY^
X^ $26 649 m=31
a=0.05, g=31, t1 ^
A 95% CI for EY^
X
^
a=2; g =2.040
Y
X is [)$142 903, )$33 418]
Flexible simulation tool for manufacturing-cell design II model- and simulation-based estimates of the optimal expected cell cost. However, of greater importance is the practical signi®cance of the dierence between these two estimates as measured by the percentage deviation of the metamodel-based estimate from the simulation-based estimate, Percentage deviation 100fY^
X^
Y
X^ g=Y
X^
5%:
In our experience, a 5% deviation of a metamodel-based performance estimate from the corresponding simulationbased estimate is not practically signi®cant; and thus we concluded that the metamodel could be used eectively to evaluate cell performance under other scenarios within the region of interest in the input-factor space. The optimum settings of the input factors identi®ed via the response surface analysis are as follows: a 45% reduction in machine setup times; a unit load size of six for products 1 through 7 and a unit load size of four for products 8 through 13; a small lot size; small rolls at the automatic insertion machines; traditional inspections; and breakdown machine maintenance. We estimated the savings that would result from implementing the optimal cell con®guration using an analysis similar to that formulated in displays (11) and (12) by comparing the following simulation-generated statistics: (i) the average annual cost for the current cell con®guration based on eight runs as reported in the companion paper (Irizarry et al., 2000); and (ii) the average annual cost for the estimated optimum cell con®guration based on 20 runs as reported in Table 7. This approach yielded a 95% con®dence interval for the expected cost savings of $425 000 $13 000, which represents a 20% reduction in the cell's annual cost. See Irizarry (1996) for complete details on this analysis of the expected cost savings that would result from optimizing the cell con®guration.
4. Conclusions and recommendations Simulation metamodels provide an eective way of performing ``what-if'' analyses in the vicinity of the optimum. Canonical and ridge analyses of the estimated response surface help to identify subregions of the inputfactor space in which cell performance is nearly optimal. This results in greater ¯exibility in the selection of cell operating conditions. As illustrated in the case study, we believe that validation of the ®tted simulation metamodel should be based on consideration of both the statistical and practical signi®cance of the dierence between the simulation- and metamodel-based estimates of the expected cell performance at selected design points. The methodology proposed in this paper provides users with a systematic approach for design and analysis of
845
manufacturing cells. It facilitates the design process and the allocation of resources by focusing attention on those cell design and operation issues with a signi®cant impact on cell performance as identi®ed by a factor-screening experiment. The methodology incorporates response surface analysis to identify the setting(s) of cell design and operational factors that optimize cell performance. The availability of simulation tools with capabilities similar to those included in the generic cell simulator should motivate more companies to combine simulation and optimization techniques for better results. The case study presented in this paper revealed a number of areas in which further methodological developments are needed. To increase the eciency of the initial factor-screening experiment beyond what can be achieved by straightforward application of the variance reduction technique of common random numbers, we may seek to apply the Schruben±Margolin correlationinduction strategy for controlling the random-number streams that drive the manufacturing-cell simulation model (Schruben and Margolin, 1978). However, it is not clear that the Schruben±Margolin strategy may be routinely applied to manufacturing-cell simulations; and diagnostic techniques such as those formulated in Tew and Wilson (1992) may be required to identify situations in which the Schruben±Margolin strategy may be validly applied. In some situations in which we apply a variance stabilizing transformation of the form Z
X f Y
X to obtain improved metamodel estimates, we may be able to construct a con®dence interval for the mean response at the stationary point that is more reliable than (9). If the transformed response Z
X is more nearly normal than the original response Y
X , then a more nearly exact 100
1 a% con®dence interval for the mean response at the stationary point has the form ^ X^ t1 a=2;m SEZ
^ X^ g; f 1 fZ
13 where: m is the degrees of freedom for the corresponding ^ X^ by ^2Z ; and we calculate SEZ
residual mean square r taking X X^ in (3). An important direction for future research is to identify situations in which the alternative con®dence interval estimator (13) substantially outperforms conventional con®dence intervals of the form (9) and thus is strongly preferred in practice. Perhaps the most urgent need is for follow-up work on new experimental designs for the response surface phase of our procedure. In our experience conventional ®rstand second-order metamodels often fail to approximate adequately the response surface associated with a largescale simulation model; and in such a situation, we require a well-developed methodology for augmenting the original response surface experiment with new design points so that more ¯exible metamodels may be eciently estimated, validated, and ®nally applied to improve the design and operation of the system under study.
846 Acknowledgements This work was partly supported by NSF Grant DDM9215432. We thank Barbara J. Hurley for many enlightening discussions on the development of the simulation metamodels for the case study. We also thank David Goldsman, the Simulation Department Editor, and two anonymous referees for several suggestions that improved the exposition.
References Anon (1990) SAS/STAT User's Guide, Version 6, 4th edn, Vol. 2, SAS Institute, Cary, NC. Box, G.E.P. and Draper, N.R. (1987) Empirical Model-Building and Response Surfaces, John Wiley & Sons, New York. Buzacott, J.A. (1985) Modeling manufacturing systems. Robotics and Computer-Integrated Manufacturing, 2(1), 25±32. Hald, A. (1952) Statistical Theory with Engineering Applications, John Wiley & Sons, New York. Hira, D.S. and Pandey, P.C. (1983) A computer simulation study of manual ¯ow lines. Journal of Manufacturing Systems, 2(2), 117± 125. Hurley, B.J. (1994) Simulation-based optimization of design, operation, and control factors for a manufacturing cell. Ph.D. dissertation, Department of Industrial Engineering, North Carolina State University, Raleigh, NC. Irizarry, M. (1996) Simulation-based methodology for the design of world-class manufacturing cells. Ph.D. dissertation, Department of Industrial Engineering, North Carolina State University, Raleigh, NC. Irizarry, M., Wilson, J.R. and Trevino, J. (2001) A ¯exible simulation tool for manufacturing cell design, I: model structure, operation, and case study. IIE Transactions, 33(10), 827±836. Jothishankar, M.C. and Wang, H.P. (1993) Metamodeling a just-intime kanban system. International Journal of Operations and Production Management, 13(8), 18±36. Lin, L. and Chiu, F. (1993) Manufacturing cell operating characteristics. European Journal of Operational Research, 69, 424±437. Lin, L. and Cochran, J.K. (1987) Optimization of a complex ¯ow line for printed circuit board fabrication by computer simulation. Journal of Manufacturing Systems, 6(1), 47±57. Madu, C.N. and Chanin, M.N. (1992) A regression metamodel of a maintenance ¯oat problem with Erlang-2 failure distribution. International Journal of Production Research, 30(4), 871±885. Montgomery, D.C. (1991) Design and Analysis of Experiments, 3rd edn, John Wiley & Sons, New York. Powell, K.A. (1992) Interactive decision support modeling for the textile spinning industry. Master's thesis, Department of Industrial Engineering, North Carolina State University, Raleigh, NC. Schruben, L.W. and Margolin, B.H. (1978) Pseudorandom number assignment in statistically designed simulation and distribution
Irizarry et al. sampling experiments. Journal of the American Statistical Association, 73(363), 504±525. Shang, J.S. and Tadikamalla, P.R. (1993) Output maximization of a CIM system: simulation and statistical approach. International Journal of Production Research, 31(1), 19±41. Stuart, A. and Ord, J.K. (1994) Kendall's Advanced Theory of Statistics, Vol. 1: Distribution Theory, 6th edn, Edward Arnold, London. Tew, J.D. and Wilson, J.R. (1992) Validation of simulation analysis methods for the Schruben±Margolin correlation-induction strategy. Operations Research, 40(1), 87±103. Welsch, R.E. (1977) Stepwise multiple comparison procedures. Journal of the American Statistical Association, 72(359), 566±575. Yu, B. and Popplewell, K. (1994) Metamodels in manufacturing: a review. International Journal of Production Research, 32(4), 787±796.
Biographies Maria de los A. Irizarry is an Assistant Professor in the Department of Industrial Engineering at the University of Puerto Rico, Mayaguez Campus. She teaches and performs research in the areas of simulation and ergonomic work design. She is a member of IIE, APICS, and HFES. She is a registered professional engineer. She received a Ph.D. degree from North Carolina State University and a Master's degree from Texas A&M University, both in Industrial Engineering. James R. Wilson is Professor and Head of the Department of Industrial Engineering at North Carolina State University. His current research interest are focused on the design and analysis of large-scale simulation experiments. He also has an active interest in applications of operations research techniques to all areas of industrial engineering. From 1988 to 1992, he served as a Departmental Editor of Management Science. Since 1997 he has served as an Area Editor of ACM Transactions on Modeling and Computer Simulation. He has also held various oces in TIMS (now INFORMS) College on Simulation; and he currently serves as the corepresentative of that organization to the Board of Directors of the Winter Simulation Conference. He is a member of ASA, ACM, IIE, and INFORMS. Jaime Trevino is Vice President and Director of the Electric Systems Technology Institute (ETI) at ABB Power T & D Company. ETI is the ABB research and development organization for the Americas. He was previously Manager of Production Technologies, Engineering and Manufacturing Productivity, and Information Technology at ABB. He is also an Adjunct Assistant Professor at North Carolina State University, where for more than 10 years he has taught courses in facilities design, material handling systems, just-in-time production systems, and manufacturing strategy. He has also worked at Georgia Tech and Monterrey Tech as a faculty member and researcher. He is the author of many publications in the ®elds of material handling, just-in-time production systems, and facilities design. Contributed by the Simulation Department
