ing neural networks in the combined continuous/discrete problem but the ..... minimum and maximum design variable and state values contained in the 160 IOP ...
Optimization of Mixed Discrete/Continuous Design Variable Systems Using Neural Networks R.S. Sellar S.M. Batilly J.E. Renaud z Department of Aerospace and Mechanical Engineering University of Notre Dame Notre Dame, Indiana inadequate or at least untested when faced with the task of automating or optimizing the design of mixed discrete/continuous systems. A second and sometimes equally important issue is the vast amount of information required for the design of complex, multidisciplinary engineering systems. The computational capabilities provided by current computers can allow the designer the opportunity to produce vast amounts of data and consider many potential design variations. Providing a framework for these design studies and a method of storage of the information generated so that it can be useful in practical design decision making is critical to the success of automated design efforts. These issues result in questions as to how one can generate optimal or at least improved designs given real constraints on time and computational resources. This is particularly important in combinatorial design problems involving discrete design variables which often require an extremely large number of objective function evaluations. For these problems the reduction of the number or cost of these objective function evaluations is a high priority. The purpose of this study was to address issues pertaining to design space representation and evaluation for structural design problems which contain both discrete and continuous design variables. The method by which the design space de nition (response surface mapping) for the mixed discrete/continuous design variable problem was addressed in this study was through the use of arti cial neural networks. These networks have been shown to provide a useful tool for storage and manipulation of design data obtained through conventional analysis techniques for systems of either continuous[1, 2, 3] or discrete[1, 4, 5, 6] design variables, but problems containing both continuous and discrete variables have only recently been considered in some detail [7, 8]. This paper employs feedforward, back propagation neural networks to provide an approximation to the mixed discrete/continuous design space and to replace conventional numerical analysis methods at the system level in the optimization process. The problem considered in this paper is an extension of a preliminary study initially presented in Reference [9]. This earlier work indicated the potential for using neural networks in the combined continuous/discrete problem but the design space considered did not provide a particularly demanding application and the subspace design problem was rather limited. The methods considered in the present study were evaluated by application to a relatively straightforward structural design problem. This problem was selected since it contained the basic characteristics of both continuous and discrete design variables. The design space
ABSTRACT Emerging multidisciplinary design techniques must be capable of dealing with a wide variety of disciplinespeci c design methodologies. Of particular concern is the development of methods which can e ectively integrate continuous and discrete design variables. This paper documents a study in which material selection, structural arrangement and component sizing are considered for a single, simple structural system. A hierarchical design problem is formulated in which component sizing based upon nite element analysis is performed at the subsystem level. Arti cial neural networks are used to provide response surface mapping of the subsystem for use at the system level. The system level problem which includes the discrete material selection design variables and continuous structural arrangement design variables is formulated as a discrete problem and optimum designs were identi ed using simulated annealing algorithms for two di erent merit functions. This study identi es requirements for the e ective use of neural network mapping of the subspace and how this representation of the subspace in uences the system level optimization. INTRODUCTION Optimization methods have a long history and these methods have played important roles in engineering analysis and design. The foundation for many of the current optimization methods used in engineering design are analytic or numerical techniques which are well-suited for speci c classes of problems. As the engineering community attempts to expand the in uence of optimization methods into the realm of \multidisciplinary design optimization" it is faced with the problem of adapting optimization methods to problems which are more complex and represent a variety of problem classes. The current study was concerned with just one case in which both continuous and discrete design variables were required for a single design problem. There are a variety of optimization techniques that can be used to nd the solution to a design problem when the system in question is composed solely of either continuous (spar cap cross sectional area, wing sweep angle,: : : ) or discrete (material choice from a nite set, number of spars,: : : ) design variables. These techniques, however, are in many cases Graduate Research Assistant, Member AIAA y Professor, Associate Fellow AIAA z Clark Equipment Assistant Professor, Member
AIAA c 1994 by Stephen M. Batill. Published by the Copyright American Institute of Aeronautics and Astronautics, Inc. with permission.
1
was rather complex in character but the problem itself was simple enough to allow for graphical presentation of various results. The sections which follow brie y describe the speci c design problem, the way in which neural networks were used to map the design space for this problem, and the methodology used to obtain optimal solutions through the use of neural network representations of the design space. The paper concludes with a detailed discussion of results for the application of these methods to the sample structural design problem.
4
4000 pounds
#3
3 #4 #2 #5
THE DESIGN PROBLEM The problem presented in this paper is the conceptual design of a structure for which the system design vector is comprised of both discrete and continuous design variables. The \design problem" is an example of a two-step hierarchic design. The rst step, referred to as the subspace optimization, contained continuous design variables and the second step, or system level optimization, contained both continuous and discrete variables. The design of a structure typically involves the selection of the basic geometric arrangement (con guration), the selection of the materials to be used in the structure (the material system), and nally the sizing of the components which make up the structure. The sequence in which these decisions are made can signi cantly in uence the design. There have been many methods developed which allow for the sizing of structural components for a given con guration and material properties. These are often based upon nite element representations of the structure. In this paper the selection of the component sizes for a given structure and material combination is referred to as a \subspace" optimization and is the rst step in this design process. Including the selection of the material system and con guration in the design optimization may allow for even better designs but these types of design variables are more dicult to include in many of the current structural optimization schemes. This is often the result of the fact that the nite element method used to model and analyze the structure is not as easily adaptable to variations in con guration and material properties as it is to variations in component sizing. An additional complexity is introduced to the problem due to the discrete nature of some of the con guration and material property design variables and the diculty associated with using either optimality criteria or gradient-based optimization algorithms in the design process. Combinatorial optimization algorithms, usually associated with discrete design variables require large numbers of analyses and are often quite costly when using nite element structural analysis methods. In the current study, nite element analysis techniques were used to model and analyze the structure at the subspace level. For a given con guration and material composition, a gradient-based algorithm was used to determine the least weight design subject to yield, local buckling and minimum gage constraints. Then the design space represented by the con guration and materials design variables was evaluated to determine the con guration and materials which yielded the best of the least weight designs. Two measures of merit were used at the system level to evaluate the suitability of the designs. Weight, the same merit function used at the subsystem level, was used as well as a hypothetical \cost" function which was based upon an analytic combination of weight and performance as measured by structural deformation. Details on this cost function are provided below.
1
2
#1 δy
4000 pounds δx
Figure 1: Schematic of Five-bar Truss Design Load Condition In order to provide graphical representations of the design space for this sample problem, the candidate structure was selected so that the number of discrete design variables was kept small. The ubiquitous ve bar truss structure provided a realistically complex design space and at the same time allowed the subspace optimization to be accomplished quickly and without investing large amounts of time in the development of an extensive set of analysis and optimization software. Described below is the simple structural example examined in this study, the method of analysis and the subspace design algorithm. The Five Bar Truss The problem under consideration was the design of a statically indeterminate ve bar truss in which the system level design variables were the material choice for each of the rod elements [aluminum-(1) or titanium-(2) thus yielding 25 or 32 discrete material system combinations] and the x-coordinate of Node 2 (a continuous design variable). The baseline geometry of the truss is shown in Figure 1. In this paper a ve-integer designation was used to identify the material combination, e.g. a 12211 indicator implies element #1 is aluminum, element #2 is titanium, and so on. Node #2 was allowed to vary over a rather wide range from 3 inches to 13 inches about the baseline value of 10 inches. At the subsystem level the design variables were the cross-sectional areas of each of the truss elements. The truss was fully constrained in both x and y directions at Nodes 1 and 4 and a single loading condition was applied at the unrestrained nodes as indicated in Figure 1. By posing the mixed continuous/discrete problem in such a manner the region of the design space associated with each of the 32 material combinations can be displayed as a continuous functional relationship between weight and the x-location of Node 2. Similarly the entire discrete design space for a single Node 2 x-location can be graphically represented with a single bar chart. Subspace Design In the nite element algorithm used for the subspace analysis the truss elements were modeled as rod 2
j
j
j
j
1.00 Material Combination 11112 Material Combination 22212 Material Combination 21222 Weight (pounds)
0.90
0.80
0.70
0.60 0.0
2.0
4.0 6.0 8.0 10.0 12.0 Node 2 Location (inches)
14.0
Figure 2: Non-parametric Representation of Weight Versus Con guration Subspace Horizontal Displacement (inches)
0.05
0.04
0.03
0.02 Material Combination 11112 Material Combination 22212 Material Combination 21222
0.01
0.00 0.0
2.0
4.0 6.0 8.0 10.0 12.0 Node 2 Location (inches)
14.0
Figure 3: Non-parametric Representation of Horizontal Displacement Versus Con guration Subspace -0.010 Vertical Displacement of Node 2 (inches)
elements capable of supporting only axial loads. An IMSL routine developed to solve constrained, non-linear minimization problems using successive quadratic programming (NCONF) [10] was used to determine the least weight design for a speci ed con guration and material combination. These designs were used in two ways in this study. A set of these designs were used as the training data for the subsequent neural network representations of the subspace. Description of the neural networks and their training is included below. For this particular problem the subspace design algorithm could also be used to provide a detailed, non-parametric representation of the subspace for eventual comparison with and evaluation of the neural network representations. As with most gradient-based optimization algorithms, the \optimal" design can be sensitive to the initial design. This is particularly troublesome when one wishes to automate the process and develop a large number of designs. To avoid this problem, each subspace, least-weight design optimization was started from four di erent initial designs and the least weight design of the four solutions was selected. Figure 2 presents the nonparametric representation of the weight design space for three arbitrarily selected material combinations. This was developed by determining the least weight design for each material combination and for Node 2 x-locations over the entire range with a increment in the Node 2 x-location of 0.2 inches (51 designs per material combination). This illustrates the general character of the space with a minimum located near the middle and a local maximum located near the left side boundary of the space. Figures 3 and 4 present the Node 2 horizontal and vertical displacements for optimum weight designs for three material combinations over the range of con gurations described earlier. These serve to illustrate the complexity of the design problem for even this rather simple structure. Since there were three di erent constraints (yield, buckling and gage) on the truss elements, there were di erent constraints active at various points over the range of Node 2 x-locations for each material combination. Thus the load paths through the structure could change rather signi cantly with only a small variation in the Node 2 x-location (for a particular material combination) and though the space de ned by the minimum weight designs was rather smooth, the space de ned by the de ections was far more complex. System Level Cost Function and Design Space At the system level each design was evaluated based upon two di erent measures of merit. If weight was used at the system level for system optimization then the design space would be represented as in Figure 2. As an alternative to using weight for the system level design a \cost" function was proposed that included both weight and system performance measured in terms of structural deformation. A simple expression of the form, b d f Cost = a(W eight) + c x2 + e y2 + g was used to de ne the measure of merit for the system level optimization. The coecients were selected in order to develop a merit function of sucient complexity to challenge the parametric representation of the design space, i.e. the neural network. Figure 5 illustrates the non-parametric cost design space for the same three material combinations shown in Figure 2. This design space has severe gradients, slope discontinuities and local minima. Figure 6 shows the non-parametric design space for each of the 32 material combinations for the x = 8.0
-0.020
-0.030
-0.040
-0.050 0.0
Material Combination 11112 Material Combination 22212 Material Combination 21222 2.0
4.0 6.0 8.0 10.0 12.0 14.0 Node 2 Location (inches)
Figure 4: Non-parametric Representation of Vertical Displacement Versus Con guration Subspace 3
Some of the issues involved in using neural networks for design space mapping are how to con gure the neural network, how much training data is required to \suciently" train the neural network, and how the training data is selected. These issues are not independent and are brie y discussed in the following sections.
1.70 1.65
Cost
1.60
Neural Network Con guration One of the diculties that arises in the process of using neural networks in any application is determining an appropriate network con guration. This depends upon the complexity of the function being represented, the amount and distribution of the training data and the time allotted for training. In the current study since there were 6 design variables at the system level, there were 6 input neurons. Based upon earlier studies it was decided to select networks with a single hidden layer. The number of neurons in the hidden layer was varied from 10 to 40 in this study. Since the subspace design provided three output variables, weight, horizontal displacement and vertical displacement of Node 2, it was possible to train a single network with all three outputs or three networks with a single output. Both options were considered in this study and will be discussed. Since there were only two material choices for each rod element, the material property inputs to the neural network were either 0.1 or 0.9 corresponding to material 1 or 2 for each nite element. This is one of a number of approaches which could be used to represent discrete design variables. The appropriate choice for design variable representation in neural networks using discrete design variables is still an issue requiring further study.
1.55 1.50 Material Combination 1 1 1 1 2 Material Combination 2 2 2 1 2 Material Combination 2 1 2 2 2
1.45 1.40 0.0
2.0
4.0 6.0 8.0 10.0 12.0 Node 2 Location (inches)
14.0
Figure 5: Non-parametric Representation of System Cost versus Con guration 1.70
Cost
1.65
Neural Network Training Once the network con guration has been selected{ however it is accomplished{it is necessary to train the network to approximate the design space. But how much design space information in the form of training data is required given that the impetus for application of neural networks is to reduce the cost associated with running complex analysis methods? Also, how is training data distributed in the design space such that the neural network is able to adequately represent the design space? Neural network representations of the subspace for this design problem were obtained for a number of neural networks and combinations of training data. The backpropagation algorithm in NETS was utilized to train the networks to a given tolerance or other termination criteria which will be discussed later. Training set sizes ranging from 20 to 160 IOP sets were used. An IOP (Input/Output Pair) consists of an input vector (structural con guration and material combination) and the corresponding output vector, the least weight and/or Node 2 horizontal or vertical displacements. Initially a training data set of 160 designs was randomly selected. From this set, subsets of 80, 40 and 20 IOPs were randomly chosen, each successive training set being a subset of the previous. An alternative training set of 160 IOPs which were uniformly distributed throughout the design space was also developed. For this study each of the 3 network con gurations (10, 20 and 40 hidden neurons) were trained as either single networks or sets of three networks for each of the output quantities resulting in 12 di erent networks. These were then trained with each of the training data sets (20,40,80 and 160 random distribution and 160 uniform distribution) resulting in 60 di erent neural networks. Space does not permit the detailed discussion of each network but this
1.60
1.55
Material Combination
Figure 6: Non-parametric Representation of System Cost Versus Material Composition inches node location. Obviously the \shape" depends upon the sequencing of the material combinations and it illustrates the discrete nature of the material design space. DESIGN SPACE MAPPING In order to reduce the computational expense required to optimize design spaces at the system level, neural networks were employed as approximations to the design spaces represented by the con guration and material design variables. Because of the success of feedforward, back propagation neural networks with a sigmoid activation function in representing structural design spaces which contain only continuous design variables [1] and design spaces which contain only discrete design variables [4], these networks were utilized for the approximation of design spaces containing both discrete and continuous design variables. The NETS [11] computer program was used to create the neural networks used in this study. 4
A second exhaustive search technique involved a detailed assessment of the neural network representations of the design space. This was conducted using a modi ed Newton's method independently for each of the 32 discrete material combinations. This represented an exhaustive search of the material space (discrete design variables) and a gradient-based search to identify the optimum value of the continuous design variable. The material combination was xed and the Node 2 x-location which provided the least weight or cost, based upon the neural network, was determined. The gradients required for the Newton's method were determined by nite differences using the neural networks. Then the minimum weight design was simply selected from the set of 32 individual designs determined for each of the material combinations. As with all numerical search procedures, the results were sensitive to the starting location for the search. Three di erent starting locations were used and the results shown later in the paper indicate the starting location which provided the optimum design based upon the neural network representation of the design space. These approaches to identifying the best designs using either measure of merit for both the actual design space and for the neural network representation of the combined continuous/discrete problem were only possible due to the small size of overall design space for this sample problem{one of the reasons that this problem was selected.
does illustrate the potential for additional complexity in an already dicult design problem. The training data was scaled between 0.1 and 0.9 for network training after the desired number of training points were generated. The training data was scaled for each network based upon the range de ned by the minimumand maximumdesign variable and state values contained in the 160 IOP set. The question of training data scaling is still an open issue and does in uence the ability of the network to extrapolate from the training set. Finally, this paper uses a shorthand notation to identify a given neural network. For example a 6-203(80) designation describes a network with 6 input neurons, 20 neurons in the only hidden layer, and 3 output neurons which was trained using 80 IOPs. A u or r is used to identify whether the 160 IOPs are uniformly or randomly distributed. SYSTEM LEVEL DESIGN OPTIMIZATION There were two basic concerns in this study. The rst was the use of arti cial neural networks to represent the design space for a mixed continuous/discrete design problem. The second was the development of methods to exploit the neural networks representation of the space in order to determine the optimum design at the system level. At the system level two di erent measures of merit were considered. Optimum system designs were either based upon weight or the cost function described earlier. When weight was used as the measure of merit, it was the same merit function as was used in the subspace optimization. One would expect that the true optimum design would be determined at the system level by using the same merit function at the system and subsystem levels. The cost function was introduced at the system level in order to provide a more challenging situation for the system level optimization routines' combined continuous/discrete optimization procedure. It is recognized that applying a di erent merit function at the system level from that at the subsystem level may result in less than optimal system level designs and this issue will be addressed in future e orts. The attractiveness of using neural networks to approximate the design space is that the cost (and time) of objective function evaluations is signi cantly reduced, thus the eciency of the optimization routine, while still important, is not an overriding concern. In this study, two concepts for optimization of systems containing both discrete and continuous design variables were explored. Comparison of these methods was conducted in order to assess the e ectiveness of the optimization concepts in determining the optimal solution from the neural network representations of the design subspace. Exhaustive Searches In order to obtain benchmark designs to which other designs were compared, two types of exhaustive searches were performed. The rst was a search of the 1632 discrete designs (32 material combinations and 51 Node 2 x-locations) which constituted the complete, non-parametric design space. Though developing this extensive description of the design space in a realistic problem would be prohibitively expensive, it was straightforward in this case and provided the \exact" solution for comparison, though this was limited by the resolution of the continuous design variable x2. These \exact" solutions are referred to as the \Design Space Optimum" in later discussions.
Simulated Annealing (SA) Generally a simulated annealing algorithm is applied to systems containing discrete variables and is effective for determining the region in the design space in which the global optimum solution exists [4]. In this study simulated annealing was explored as an optimization alternative since the inputs to the neural network were predominantly discrete (5 discrete variables and only one continuous variable). The issue was then how to handle the continuous design variable in this type of algorithm. The principle upon which simulated annealing is based is that candidate designs are generated and accepted as the current design if the \energy" of the new design is less than or marginally greater than the previous design. Figure 7 presents a owchart outlining the logic used in the simulated annealing algorithm. In this application the energy of a design is simply the objective function (the weight or cost of the structure). The probability of accepting a candidate design is based on the energy (weight/cost) of the previous design as compared to the energy (weight/cost) of the current design. Although SA algorithms are most often used in optimization problems containing discrete design variables, there is no requirement that the design vector be comprised only of discrete design variables. The use of simulated annealing in discrete optimization problems is in uenced by the candidate design selection process and convergence properties. In traditional SA algorithms candidate designs are randomly determined and are entirely independent of the current \best" design. In this study potential designs were determined by perturbing the current design. This was achieved by altering a random subset of the total number of design variables (from 1 - 6 in this case). In this way an entirely new design vector was obtained approximately 17% of the time; therefore, the remaining 83% of the time design vectors which were generated contained at least some of the information from the current design. 5
range of the continuous design variable, simulated annealing was performed using this coarsely discretized design space. Based upon the best design identi ed using this coarse resolution of the design space a new allowable range for the continuous design variable was selected and the design space was again discretized. The resolution of the discretization in this reduced range about the preliminary solution was 50% ner, and simulated annealing was performed again. This procedure was continued until no change in the discrete design variables was detected between successive discretizations. The owchart for this algorithm is presented in Figure 8.
Initial Design
Modification of the Design Vector
Computation of System Cost
Reduce Design Acceptance Criteria
design modifications
Initial Design
Modification of the Design Vector
Accept or Reject New Design
Computation of System Cost
NO
Design YES Convergence?
design modifications
Reduce Design Acceptance Criteria Accept or Reject New Design
STOP
Figure 7: Simulated Annealing (SA) Method Flow Chart
Rediscretize continuous design variable YES
This procedure retained the ability to escape from local extrema in the design space. This was made possible by the way in which the continuous design variable was perturbed. The continuous design variable (Node 2 x-location in this case) was altered by perturbing it about the previous value with a perturbation size from 0.1% - 20% of the total range. The amount by which the variable was altered was discrete in the sense that it is xed by the resolution of the computing system or by the choice of minimum and maximum percent change. However, by changing the continuous variable by a random percentage of itself, it was possible for the variable to take on any value in the design space. This allowed the algorithm to approach the optimal solution without requiring the use of traditional continuous design space optimization strategies.
NO
Design YES Convergence?
Change in Truss Element Materials? NO
STOP
Newton’s Method with Fixed Materials
Figure 8: Successive Simulated Annealing (SSA) Method Flow Chart Once this procedure converged, it was assumed that the algorithm had provided the material system which would allow for the best design. As one nal step in the process a simple con guration optimization was performed to nalize the Node 2 x-location. An optimization again using Newton's method for a xed set of material properties was performed with the starting value of the Node 2 x-location obtained from the most recent SA analysis.
Successive Simulated Annealing (SSA) The Successive Simulated Annealing (SSA) algorithm is a heuristic based on traditional simulated annealing techniques. The primary di erence between Successive Simulated Annealing and the simulated annealing(SA) approach described above is the way in which the continuous design variable was manipulated. In the SSA procedure the continuous design variable was discretized in a rather \coarse" manner over the entire 6
analysis. The results are for the 11211 material combination over the complete range of Node 2 x-locations and represent only the 6-20-3 network results. The randomly distributed training data which were included in each IOP set are indicated on the gures. For the 20 IOP case there was only a single design present in the training data set and this occurred near x2 = 10.5 inches. The neural network representation for the 6-20-3(20) network was obviously marginal at best. The general trend in the design space may have been captured but none of the more speci c space attributes were represented. As the training set was increased to 40 IOP (recall that the smaller training sets were simple subsets of the larger sets) there were four designs in the IOP set for this material combination. Since 10% of the entire IOP set was present for this single material combination one might expect an improved design space representation as indicated in Figure 9b. Though the general character of the neural network representation improved for the 620-3(40) network, details particularly near the minimum weight for this material combination were still lacking. Though the overall training set for the 6-20-3(80) network was twice that of the previous network, it did not introduce any new designs into this particular material combination as shown in Figure 9a. Even though no new training data has been added, the network representation of the weight space has improved in comparison with the 40 IOP case for this material combination. The nal network presented is the 6-20-3(160r) in Figure 9b. One additional training point was added for this material combination. This allowed the network to begin to identify the convex behavior near the left boundary of the design space. This last network was trained to a somewhat higher error tolerance and thus the error at the training points is somewhat higher, even though the overall representation of the design space has improved. Recall that weight is only one of the neural network output parameters. Figures 10a and 10b illustrate the network representation of the vertical displacement of Node 2. Results for the same material combination and network discussed above are presented. The vertical displacement space is obviously more complex and re ects the changing constraint surfaces encountered during the least-weight, subspace design. The 6-20-3(20) network obviously does not e ectively represent the actual space and only the 6-20-3(160r) network is able to even approximate the sharp variations which occur. One issue that should be noted is that the random selection of the training data has resulted in two training data points which are quite close to each other. The proximity of these two points does not necessarily improve the overall network representation and only adds to training time and cost. Another issue considered as part of this study was the comparison of neural networks trained with a single output to those trained with multiple outputs. Figure 11 shows the weight space characterization for two di erent networks, the 6-40-3(40) and the 6-40-1(40). Both were trained using the same IOP set but one had only weight as the network output, the other had all three output variables. For this material combination it appears as if the 6-20-3(40) network provides a better representation. This type of information was developed for a large number of networks and unfortunately no de nite conclusions could be drawn from the results. This is an issue which will require further attention. The graphical results presented have shown comparisons between the nonparametric representation of the design space and the neural networks for variations
RESULTS AND DISCUSSION Neural Network Representation of the Design Space Since one of the primary goals of this study was to approximate design spaces using neural networks, it was useful to have the ability to graphically represent the actual design space for comparison with the neural network representation. In the following gures the curves representing the neural networks were developed by simply propagating a range of input values through the trained neural networks. The discussion which follows highlights some of the observations associated with the neural network representations of the design spaces. Figures 9a and 9b illustrate the in uence of training data density on the ability of the neural network to adequately represent the design space. 0.90
Weight (pounds)
0.80
0.70
0.60
0.50 0.0
6-20-3 (20) 6-20-3 (80) 20 IOPs 80 IOPs Non-Parametric 2.0
4.0 6.0 8.0 10.0 12.0 Node 2 X-Location (inches)
14.0
(a) 20 and 80 IOP sets 0.90
6-20-3 (40) 6-20-3 (160r) 40 IOPs 160 IOPs Non-Parametric
Weight (pounds)
0.80
0.70
0.60
0.50 0.0
2.0
4.0 6.0 8.0 10.0 12.0 Node 2 X-Location (inches)
14.0
(b) 40 and 160 IOP sets Figure 9: In uence of Training Set Size on 6-20-3 Network Weight Estimate These gures compare structural weight as predicted by the neural network with the extensive non-parametric representation developed directly from the nite element 7
1.0 6-40-3 (40) 6-40-1 (40) Training Data (40 IOPs) Non-Parametric (FEM)
Weight (pounds)
Vertical Displacement of Node 2 (inches)
0.9
0.7
-0.020
0.6
-0.030
0.5 0.0
6-20-3 (20) 6-20-3 (80) 20 IOPs 80 IOPs Non-Parametric
-0.050
2.0
4.0 6.0 8.0 10.0 12.0 14.0 Node 2 X-Location (inches)
-0.020
-0.030
6-20-3 (40) 6-20-3 (160r) 40 IOPs 160 IOPs Non-Parametric
-0.050
-0.060 0.0
2.0
4.0 6.0 8.0 10.0 12.0 Node 2 X-Location (inches)
14.0
in the continuous design variable. Figure 12 presents the comparison for the 6-40-3(160r) network for a Node 2 x-location of x2 = 9.0 inches. This Node 2 x-location was selected since it was near the minimum weight design. The discrete nature of the design space makes the comparison somewhat more dicult but it does indicate the capabilities associated with the neural network representation. For some material combinations the neural network maps the design space quite well, while for others it is less accurate. The nal method used to evaluate the network representations of the design space was based upon a quantitative comparison between the networks and the complete non-parametric representation of the design space. The maximum and RMS (root-mean-square) errors for all 1632 designs in the non-parametric design space were determined for each network and each training data set considered in this study. Tables 1 and 2 present maximum and RMS errors for the weight and Tables 3 and 4 provide similar data for the vertical displacement of Node 2.
(a) 20 and 80 IOP sets
-0.040
2.0
Figure 11: Comparison of Single and Multiple Output Neural Networks
-0.040
-0.060 0.0
Vertical Displacement of Node 2 (inches)
0.8
6,n,3 IOPs n Random 10 20 40 20 35.13 30.09 30.38 40 20.17 31.57 19.47 80 17.95 10.30 12.06 160 5.22 7.56 6.92 Uniform 160 3.64 6.35 7.52
4.0 6.0 8.0 10.0 12.0 14.0 Node 2 X-Location (inches)
(b) 40 and 160r IOP sets Figure 10: In uence of Training Set Size on 6-20-3 Network Vertical Displacement Estimate
10 29.61 27.30 11.80 11.32
6,n,1 n 20 34.20 18.01 13.75 6.93
40 28.21 27.06 14.77 11.41
4.74
4.70
5.21
Table 1: Network Maximum% Error - Weight Parameter The values listed in the tables have been normalized by the absolute value of the maximum value of the parameter in the non-parametric data set. The most obvious trend present in the data is the reduction in the errors with increased size of the training data set, as would be expected. It should be noted that all of the networks were not necessarily trained to the same error tolerance (maximum di erence between the scaled network and training data). Most were trained with the same number of training cycles (150,000) or to the minimum tolerance of approximately 0.1%. In some cases training was 8
0.80 Non-Parametric 6-40-3 (160r)
Weight
0.75
0.70
0.65
0.60
0.55
Material Combination
Figure 12: Neural Network Representation of Discrete Design Space Using a 6-40-3(160r) Network IOPs Random 20 40 80 160 Uniform 160
10 6.54 2.64 1.30 0.72
6,n,3 n 20 6.20 4.71 1.59 0.87
40 5.50 3.23 1.64 0.77
0.59 0.68 0.98
10 9.14 6.22 2.76 1.35
6,n,1 n 20 9.24 4.80 2.94 1.27
1.11 1.22
6,n,3 IOPs n Random 10 20 40 20 15.03 14.03 13.10 40 10.60 11.15 12.29 80 8.38 9.84 10.28 160 4.88 6.81 8.35 Uniform 160 4.33 5.82 4.07
40 10.78 6.60 2.89 1.92 1.22
Table 2: Network RMS % Error - Weight Parameter IOPs Random 20 40 80 160 Uniform 160
10 76.20 70.74 55.79 50.29
6,n,3 n 20 69.76 61.46 72.82 56.40
40 67.97 73.93 68.07 67.81
48.58 41.12 45.41
10 69.71 62.77 62.34 48.62
6,n,1 n 20 71.40 61.10 67.33 69.49
10 21.61 19.22 15.77 11.53
6,n,1 n 20 25.79 18.61 16.95 14.77
40 24.11 22.25 16.31 14.24
7.18
7.63
7.17
Table 4: Network RMS % Error - Vertical Displacement Parameter
40 69.82 81.18 68.19 61.27
yields the same ratio between maximum (and RMS) error for these trained networks. Even though the network trained with 160r IOPs had a training error which was approximately 100 times greater than that for the network trained with 40 IOPs, the network representation of the design space by the 6-20-3(160r) network yielded lower maximumand RMS errors than the 6-20-3(40) representation. Once the neural network training was completed it was also possible to compare the network-predicted cost function (an analytic combination of the network outputs) with the non-parametric cost function as determined from the FEM analysis. A sample of this comparison for the 6-20-3 neural network with 80 and 160 (uniformly distributed) IOP sets and for the 11211 material combination is shown in Figure 13. Since the cost space is strongly in uenced by the displacement parameters, it too is rather complex. Although the neural network represents the general character of the cost function for the 80 and 160u IOP sets, it does not provide a particularly
42.33 42.01 43.76
Table 3: Network Maximum % Error - Vertical Displacement Parameter terminated by the number of training cycles and therefore e ective training errors were di erent. For certain networks [e.g. 6-20-3(160r)] the maximum error during training was 2 orders of magnitude higher (see Table 5) than that of the same network trained with fewer IOPs [e.g. 6-20-3(40)]. Although these errors may not be compared directly, normalization by the magnitude of the largest value of the output parameters in the training data sets for the 160r and 40 IOP sets, respectively, 9
IOPs Random 20 40 80 160 Uniform 160
10 0.001 0.021 0.059 0.132
6,n,3 n 20 0.001 0.007 0.025 0.135
40 0.001 0.002 0.010 0.046
0.083 0.059 0.036
10 0.001 0.005 0.030 0.051
6,n,w n 20 0.001 0.003 0.013 0.047
40 0.001 0.002 0.016 0.047
0.036 0.050 0.049
10 0.001 0.009 0.041 0.047
6,n,dx n 20 0.001 0.015 0.021 0.031
40 0.001 0.002 0.011 0.025
0.050 0.050 0.050
10 0.001 0.002 0.029 0.130
6,n,dy n 20 0.001 0.001 0.015 0.040
40 0.001 0.001 0.004 0.030
0.050 0.050 0.050
Table 5: Network Maximum Training Error (Scaled Training Data) Table 6 presents the results for the 6-40-3(80) network. All three optimization methods converged to the same material combination (11111) and to the same Node 2 x-location (consistent with the convergence criteria).
accurate representation of the space.
Optimization Material Node 2 Actual Method (I.C.) Combination Location Weight Weight EX (8.0) 11111 7.906 0.578 0.623 SA 11111 7.986 0.578 0.622 SSA 11111 7.849 0.578 0.624 Design Space 11211 Optimum (g b y y b) 8.800 0.582
Cost
1.65
1.60 6-20-3 (80) 80 IOPs 6-20-3 (160u) 160u IOPs Non-Parametric
1.55
1.50 0.00
2.00
Table 6: Optimization Results, Weight Merit Function, 6-40-3(80) Network This was not the same design as indicated by the nonparametric representation of the design space which occurred for material combination 11211. A comparison of the non-parametric representation of the two design spaces and their associated neural network approximations is provided in Figure 14.
4.00 6.00 8.00 10.00 12.00 14.00 Node 2 Location (inches)
Figure 13: Comparison of Cost Merit Function with 6-20-3(80) and 6-20-3(160u) Network Representations
0.90
System Level Optimization System level optimizations using both measures of merit, weight and cost, were performed using each of the neural network representations of the subspace and each of the optimization algorithms discussed above. Each of the optimization algorithms used both the continuous design variable, Node 2 x-location, and the discrete design variables, material combination. As one would expect, the quality of the optimization results are directly related to the quality of the neural network representation of the design space. Only limited results are included here to highlight certain characteristics of this combined continuous/discrete design problem. The results in this section are presented in two forms. Table 6 shows the optimum weight design as determined by each of the optimization strategies: 1) exhaustive material space/gradient based continuous variable search [EX], 2) simulated annealing [SA] or 3) successive simulated annealing [SSA]. The EX solution also indicates which of the three initial conditions for the gradient search on the x2 parameter resulted in the least weight design. The entry labeled \Design Space Optimum" represents the least weight design as identi ed by the 1632 individual designs which formed the nonparametric representation of the design space. The Design Space Optimum also indicates the constraint which was active for each of the structural elements: n-no constraint, g-minimum gage, b-local buckling or y-yield stress.
MC 1 1 1 1 1 IOPs for 1 1 1 1 1 N-P, 1 1 1 1 1 MC 1 1 2 1 1 IOPs for 1 1 2 1 1 N-P, 1 1 2 1 1
Weight (pounds)
0.80
0.70
0.60
0.50 0.0
2.0
4.0 6.0 8.0 10.0 12.0 Node 2 X-location (inches)
14.0
Figure 14: Weight Design Space Represenation for Optimization Results, 6-40-3(80) Network This gure clearly indicates the reason for the discrepancy. The material combination 11111 contained only a single training point and its neural network representation is clearly less accurate than that for the 11211 material combination. The 11211 material combination contained four training points and though none occurred in the immediate vicinity of the least weight design, the 6-40-3(80) network represents this material combination more accurately than material combination 11111. Each 10
of the optimization procedures identi ed the global optimum as indicated by the neural network but the inadequate neural network representation led to an incorrect global optimum. It is anticipated that if additional training data was added to the IOP set in the vicinity of the network-identi ed optimum then the network representation would improve in this region, resulting in selection of a di erent optimal solution. Table 7 and Figure 15 provide similar results for the 6-40-3(160r) network. Again each of the optimization algorithms yielded the same nal design.
network representation for the 11122 combination is obviously slightly lighter than that for the 11112 combination. Though each network is fairly e ective in representing the qualitative nature of the design space, they each lack sucient detail in the vicinity of the design space minimum. If one realizes that the potential of neural network representation may lie in identifying the region in which improved designs reside and thus direct the collection of additional training data in those regions, then the results presented above are promising. Optimization Material Node 2 Actual Method (I.C.) Combination Location Cost Cost EX (4.5) 11122 4.917 1.530 1.506 SA 11122 4.770 1.530 1.499 SSA 11122 4.861 1.530 1.504 Design Space 11112 Optimum (g n y n b) 4.600 1.493
Optimization Material Node 2 Actual Method (I.C.) Combination Location Weight Weight EX (8.0) 11221 8.203 0.589 0.585 SA 11221 7.996 0.590 0.587 SSA 11221 8.146 0.589 0.585 Design Space 11211 Optimum (g b y y b) 8.800 0.582
Table 8: Optimization Results, Cost Merit Function, 6-40-3(160u) Network
Table 7: Optimization Results, Weight Merit Function, 6-40-3(160r) Network 0.80
1.60
0.70 Cost
Weight (pounds)
0.75
0.65 0.60 0.55 0.50 0.45 0.0
MC 1 1 2 2 1 IOPs, 1 1 2 2 1 Non-Parametric, 1 1 2 2 1 MC 1 1 2 1 1 IOPs, 1 1 2 1 1 Non-Parametric, 1 1 2 1 1 2.0
4.0 6.0 8.0 10.0 12.0 Node 2 X-location (inches)
1.55 MC 1 1 1 2 2 IOPs, 1 1 1 2 2 N-P, 1 1 1 2 2 MC 1 1 1 1 2 IOPs, 1 1 1 1 2 N-P, 1 1 1 1 2
1.50
1.45 0.0
2.0
4.0 6.0 8.0 10.0 12.0 Node 2 X-location (inches)
14.0
Figure 16: Cost Design Space Represenation for Optimization Results, 6-40-3(160u) Network
14.0
Figure 15: Weight Design Space Represenation for Optimization Results, 6-40-3(160r) Network
CONCLUSIONS The primary goal of this study was a preliminary investigation of the design of systems which contain both discrete and continuous design variables. The existence of the discrete design variables suggests the use of optimization algorithms which require large numbers of analysis function or subspace evaluations. In order to allow for these repetitive calculations, arti cial neural networks were used to provide response surface mapping for this mixed continuous/discrete system. It was determined that neural networks can be used to represent these mixed design spaces when the networks are developed using adequate amounts of \properly distributed" training data. Quantifying the term \adequate" appears to be an issue of current concern and worthy of further investigation. One obvious conclusion is that the amount of training data necessary to e ectively represent the design space increases with the complexity of the design space. It appears that an iterative procedure to identify and distribute training data within the design space is required.
The slight di erences between the three algorithms is again attributable to the methods used to update the design (SA) and the tolerance on the convergence criteria (EX and SSA). As shown in Figure 15, the 11221 material combination and the 11211 material combination yield almost identical designs. For this particular case the 11221 material combination had six training data points, one of which was quite near the least weight for that combination. The 11211 material combination did not have a training point in the vicinity of the global optimum and thus the neural network representation for that material combination was not as accurate in that region. The last example presented is for the design based upon the cost merit function using the 6-40-3(160u) network. All three optimization algorithms again identi ed the same material combination, 11122, as illustrated in Table 8. This material combination along with that identi ed from the non-parametric data are shown in Figure 16. The similarity between the designs is illustrated over a majority of the design space. The neural 11
[6] L. Berke and P. Hajela. Application of Arti cial Neural Nets in Structural Mechanics. presented at the Lecture Series on Shape and Layout Optimization of Structural Systems at the International Center for Mechanical Sciences, Udine, Italy, July 16-20 1990. NASA TM 102420. [7] J.M. Ford and C.L. Bloebaum. A Decomposition Method for Concurrent Design of Mixed Discrete/Continuous Systems. In Advances in Design Automation, Vol. 2. ASME, 1993. [8] C.L. Bloebaum and H.-W. Chi. A Concurrent Decomposition Approach for Mixed Discrete /Continuous Variables. AIAA/ASME/ASCE/AHS/ASC 35th Structures, Structural Dynamics and Materials Conference, AIAA 94-1363, April 1994. [9] J.E. Renaud, R.S. Sellar, S.M. Batill, and P. Kar. Design Driven Coordination Procedure for Concurrent Subspace Optimization in MDO. AIAA/ASME/ASCE/AHS/ASC 35th Structures, Structural Dynamics and Materials Conference, AIAA 94-1482, April 1994. [10] IMSL User's Manual. Version 1.1, January 1989. [11] Paul T. Ba es. NETS 2.0 User's Guide. Technical Report JSC-23366, NASA Lyndon B. Johnson Space Center, September 1989.
In the design process it would be ideal to represent the design space using as few training data points as possible since they are typically generated through costly analysis procedures. For the simple structural design problem presented here, which contained both discrete and continuous design variables, it was demonstrated that the character of the design space could be wellrepresented with relatively little training data. Even though certain details were not recognized in the neural network approximation to the design spaces, general trends were preserved and this did allow for the identi cation of \near-optimum" designs. It has also been shown that design methods presented in this paper can be used for mixed discrete/continuous design variable problems. These two methods were based upon existing discrete design variable optimization strategies. Though they proved adequate for the current problem they could result in excessively large problems for systems of practical importance. It was also noted that they were limited by the ability to accurately predict system characteristics through the use of arti cial neural networks. The current work does illustrate the complexity associated with design problems which di er in fundamental character and which are considered together in an \multidisciplinary" design environment. Though the current work illustrates two approaches, neural networks for response surface mapping and simulated annealing for discrete design optimization, and potential bene ts, it also indicates limitationsassociated with each and only through application to a wide variety of problems can the potential of both approaches be exploited. ACKNOWLEDGMENTS This work was supported in part by the National Aeronautics and Space Administration Langley Research Center Grant NAG-1-1561, Dr. Jaroslaw Sobieski, Project Monitor. The authors want to thank Mr. Brian Capozzi for his assistance with the computer simulations and graphical presentation of the results. REFERENCES [1] S. Batill and R. Swift. Preliminary Structural Design-De ning the Design Space. Final Report WL-TR-93-3004, February 1991. [2] R. Swift and S. Batill. Application of Neural Nets to Preliminary Structural Design. AIAA/ASME /AHS/ASC 32nd Structures, Structural Dynamics and Materials Conference, AIAA 91-1038, 1991. [3] D. Rehak, C. Thewalt, and L. Doo. Neural Network Approaches in Structural Mechanics Computations. In Computer Utilization in Structural Engineering, Proceedings of the ASCE Structures Congress, 1989. [4] R. Swift and S. Batill. Simulated Annealing Utilizing Neural Networks for Discrete Design Variable Optimization Problems in Structural Design. AIAA/ASME/AHS/ASC 33rd Structures, Structural Dynamics and Materials Conference, AIAA 92-2311, February 1992. [5] P. Hajela and L. Berke. Neurobiological Models in Structural Analysis and Design. AIAA/ASME /AHS/ASC 31st Structures, Structural Dynamics and Materials Conference, AIAA 90-1133, 1990. 12
