In the Classroom

Chem. Educator 2003, 8, 1–8

(web)1

Neural Computing in Product Formulation R. C. Rowe and Elizabeth A. Colbourn* PROFITS Group, School of Pharmacy, University of Bradford, Bradford, W Yorks BD7 1DP, UK and Intelligensys Ltd., Billingham, Teesside TS23 4EA, UK, [email protected] Received January 2, 2003. Accepted March 4, 2003.

Abstract: Artificial intelligence techniques increasingly are being used to improve product formulations by developing models that relate alterations in ingredients and processing conditions to changes in observed properties. From relatively few applications in the early to mid-1990s, the use of neural computing in its broadest sense is gaining acceptance worldwide in a number of industry sectors. The new generation of formulators can expect to use these techniques routinely, making it timely for educators to be aware of this emerging new field. This paper outlines the key concepts underlying neural networks, fuzzy logic, genetic algorithms, and neurofuzzy systems, and reviews how these technologies have been used, singly and in combination, to model and optimize formulations in areas like pigments and dyes, adhesives, paints and coatings, and oils and lubricants.

Introduction Formulation impacts upon nearly every aspect of contemporary life, and chemists entering industry can be expected to understand its importance. Fuels and lubricants for cars, paints and coatings used to protect and enhance surfaces, personal care products like shampoos and creams, and household cleaning products all involve mixing several ingredients using a variety of possible processing conditions. Even pharmaceutical drugs need to be formulated before they can be administered to a patient. Formulation often involves complex interactions between ingredients and between ingredients and processing conditions. Understanding these interactions may at times seem more like an art than a science, and traditionally formulation expertise has been acquired only after a long period of apprenticeship in a particular field. Therefore, experienced formulators represent a substantial investment, and in turn they have to invest a significant proportion of their time in training their successors, rather than in applying their expertise to new problems and new products. New product development, however, remains vital for industry to remain competitive, and long timescales are no longer acceptable in the development of new formulations and in training novice formulators. Some of the drivers are commercial, because companies that get products into the marketplace more quickly generally benefit from higher profits. Some of them may be legislative, for example where environmental protection demands that ecologically unfriendly products be replaced immediately and customers expect continually improved products, and formulation innovation is required to meet their needs. Techniques drawn from the general field of artificial intelligence, supported by more traditional techniques like visualization and statistics, can help to speed product development, especially now that fast, inexpensive PCs are readily available. Some of the technologies that have proved most useful include artificial neural networks, genetic algorithms, fuzzy logic, and combinations thereof. For these techniques to be accessible to formulators, who are generally chemists rather than mathematicians, they need to be accessed

via a user-friendly interface and to include appropriate default parameters and heuristics (rules of thumb). Integrated computer programs are now becoming commercially available, and are gaining increasing acceptance in industry. It is important that the next generation of formulators be aware that such methods are used in industry. In this paper we present an overview of the most important artificial intelligence techniques, necessary to appreciate the chemical literature in which they are applied. An in-depth understanding of the technologies cannot be given here, because it would take several books to cover the field in full detail. To show the relevance to chemical product formulation, we include a review of applications carried out in academe and industry for a range of formulation domains. These show that neural computing is a versatile technique that is being applied across many industry sectors, impacting worldwide, including, notably, within developing countries. Technologies Neural Networks. Sometimes called artificial neural networks or ANNs, these were first investigated in the 1940s as a way to mimic mammalian intelligence and learning. The basic unit both of the mammalian nervous system and of the ANN is the neuron. In the case of mammalian systems, each neuron collects input stimuli and (provided the signal is sufficiently strong) triggers an output to the next neuron in the assembly. Artificial neurons behave in a similar fashion—they are processing units that collect mathematical inputs, and produce an output signal that is passed to the next processing unit or node. Amongst the simplest neurons are the perceptrons, which weight the inputs, sum them, then transform the sum with a smoothing function to produce an output function. A single artificial neuron on its own is of little value. Like mammalian systems, their power lies in connecting them to other units in a neural network, and, like mammalian systems, such connected systems are able to “learn.” Figure 1 shows the type of connectivity that is most commonly found, the input is assigned to an input node, and the input nodes are then

© 2003 The Chemical Educator, S1430-4171(03)0xxxx-x, Published on Web xx/xx/2003, 10.1333/s00897030687a, xxxxxxaa.pdf

2

Chem. Educator, Vol. 8, No. X, 2003

Rowe and Colbourn

Input 1

Input 2 Property 1 Input 3 Property 2 Input 4

Input 5 Input layer

Hidden layer (3 nodes)

Output layer

Figure 1. Neural network with 5 inputs, 2 outputs, and 3 nodes in a single hidden layer.

connected to the nodes (processing units) in the so-called hidden layers. These, in turn, are connected to the output nodes, with one output node assigned to each measured output. For formulation, the input nodes correspond to ingredients, ingredient amounts, and process conditions. Output nodes correspond to measured properties that are specific to each formulation domain. For example, for a paint formulation, outputs might cover aspects like gloss, coverage, toughness, weather resistance, and so on. The neural network, therefore, connects the inputs with the outputs, so that the user can make a change to an input and, by a mechanism known in neural network jargon as feed forward, see how a property is altered in response to this change. The power of neural networks arises because they can “learn” a model from existing data. That is, neural networks are capable of recognizing patterns, including cause-and-effect relationships, from data points that are shown to them, using supervised learning. In supervised learning, data are presented to the neural network as records, each of which corresponds to a formulation experiment; each record contains both inputs and outputs as defined above. The network is initialized by putting small random weights on each of the nodes. Using these small random weights, an output is calculated, and the calculated output is compared with the observed one. The weights are adjusted, typically by relating the weight change to the difference between the calculated and observed output values. The method used most frequently is called backpropagation [1] and the changes in the weights between one iteration and the next are calculated using the expression ∆wik(n) = –ε*δEp/δwik+α*∆wik(n – 1)

(1)

Here, ∆wik(n) is the change in weight for the node connecting node i and node k for the nth iteration of the process. Ep is the difference between the calculated and observed outputs for the pth pattern. (A pattern, in the case of formulation, corresponds to a single experimental recipe.) Equation 1 refers to standard incremental backpropagation, where the weights are updated after each pattern. An alternative, standard batch backpropagation, updates the weights only after all the patterns have been presented to the network. Because standard batch backpropagation involves summing a large number of energy differences, it can be very

dependent on the number of data records in the training set, so some scaling is usually employed. There are two adjustable parameters in the backpropagation expression. Both must be non-negative numbers. The learning rate, ε, allows control of the average size of the weight changes and α is the momentum. The use of the momentum function can speed up training in very flat regions of the error surface, and it suppresses weight oscillations in step “valleys.” One problem with standard backpropagation (whether incremental or batch) arises because ideally these parameters should be changed as the network is trained. Even for relatively wellbehaved problems, the choice of appropriate values can be problematic. Various algorithms and heuristics, summarized in the literature [2] have been developed to automate the selection and updating of the learning rate and momentum, including two known as RPROP and Quickprop. These make training the network more foolproof for the nonexpert. Rules of thumb indicate that 85% of all problems can be learned by a neural network with a single hidden layer, and this is the network architecture most commonly used in formulation. Choosing the number of nodes in the hidden layer also requires expertise, but again there are rules of thumb that can be used to simplify use of neural networks for nonexperts. These relate the number of hidden nodes to the number of inputs and the number of data records that are available. Because neural networks learn from the data presented to them, they are applicable to a wide range of problems and they can encapsulate relationships that are highly nonlinear, even discontinuous. They are rapid in execution, allowing “what if” scenarios to be developed quickly by the user. Their major disadvantage is that they are by nature black boxes; the relationships that the network finds cannot be expressed easily in mathematical form. The primary risk in developing a model is that of over-training, a situation in which the neural network starts to reproduce the noise specific to a particular sample in the training data, which may cause it to lose its ability to predict accurately. One way of testing the network for predictivity is to reserve some of the data, that is, to exclude it from the training data set in a process called network validation. The network can be used to predict the outputs for these reserved data records, and the calculated outputs can be compared with the observed values. If they are satisfactorily close, the network is sufficiently predictive. It is important to note that the network does not learn from the test data; these records are used solely to test the network's predictive capabilities. Because neural networks are entirely data driven, the implication is that data must be available to train the network, and the issue of “how much data” is an important one. Generally, popular belief is that large quantities of data are needed to train neural networks. In practice, a good model can be developed from 3 to 4 times as many experiments as there are inputs, provided that the data are of good quality. Most formulation data, involving carefully performed experiments, fit this quality criterion. This is fortunate for the formulators, who wish to minimize the number of experiments they need to do. Genetic Algorithms. Frequently, the aim of the product formulator is to develop a product that is optimized to meet specific market requirements. The challenges are two-fold. First, formulators often seek conflicting properties, where enhancing one property can result in serious degradation of


Neural Computing in Product Formulation

Neural Network


3

P

Generate new formulations

Mutation

Obj = Crossover

Genetic Algorithm

∑ (di ( pi ) * weighti ) i =1

P

∑ weighti

(2)

i =1

Predict Properties

No Fit Solution?

Select fittest solutions (best performers)

Yes Stop: Optimum Formulation

Figure 2. Genetic algorithms combined with neural networks for formulation modeling and optimization.

another. Secondly, there can be many possible input variables, which interact subtly, making it hard to understand what is happening. In mathematical terms, n inputs generate an (n + 1)-dimensional hyperspace in which a single property is a function of all n inputs. This can mean that the formulator is tempted to oversimplify the problem, for example varying only two or three of the possible inputs in an attempt to get a system that is relatively easy to visualize in three dimensions. In principle the neural network model could be used, in a series of “what if” experiments to try to find the optimum solution. In practice, this is too time-consuming and relies too much on luck to find the best solution. Consequently, an optimization technique that is capable of dealing with complex problems is required. Genetic algorithms, initially introduced by Holland in the 1970s [3], provide a search strategy that, when combined with a model to generate possible candidate solutions, has proved exceptionally effective in optimizing multidimensional problems. The technique aims to mimic what happens in reproduction, allowing both inheritance of parent properties and mutation. Genetic algorithms operate on a population of solutions, ultimately selecting the “fittest” for the defined requirements. There are three essentials if genetic algorithms are to be used for optimization. The first is the ability to define a criterion of fitness. The second is the ability express the solution as a chromosome, that is, in a form such as a character string, generally as an array of bits or characters, that encodes the values for the different parameters being optimized and that can be stored on a computer. The third requirement is for genetic “operators” that mimic sexual reproduction (sometimes called crossover or recombination) and mutation. In crossover, as the name suggests, genetic material is exchanged between the solutions by swapping bits from one solution with bits from another. Typically, the user can control the degree to which crossover is used. Various selection criteria can be used to pick the individual solutions for mating, but generally only the fittest solutions are chosen and the worst solutions are excluded. For formulators, the chief issue for genetic algorithm optimization is defining a suitable criterion of fitness. Generally, this is an objective function, Obj, so called because it describes the objective of the optimization. In formulation, it frequently takes the form of a weighted sum of the desirability of each property summed over all properties, as shown in eq 2.

Here, the expression is normalized by dividing by the sum of all the weights. In formulation, the weights (weighti) correspond to the relative importance of each property, so they allow the formulator to emphasize the properties of greatest significance to a particular user. This in turn means that formulations can be customized to some extent for each client. The use of weights also allows the formulator to deal with conflicting properties (e.g., hard tablets that disintegrate quickly) and to examine the trade-offs involved in sacrificing one property in order to improve another. The objective function in eq 2 also requires the desirability, di, for each property. The desirability is a function describing whether the property should be as high as possible, as low as possible, or in some range in the middle. As described below, this can be achieved using fuzzy logic. In using the genetic algorithm, then, an initial population of solutions is generated, and the fitness of each is determined using the objective function. The most-fit solutions are used as parents in the next generation of solutions. There is competition between families of solutions and between parents and children within each family. Ultimately, the most-fit solutions—those that most closely match the defined requirements—are obtained. Because genetic algorithms explore the whole of the ndimensional hyperspace using a stochastic (random) approach, they are generally effective at finding a global optimum on a complex hypersurface, especially if the amount of crossover allowed is fairly high. More directed search methods (like steepest descent, or conjugate gradients) can result in the system becoming locked into a local optimum, without finding the global one. Because genetic algorithms require a computation of the fitness for each possible solution, though, they can be compute-intensive. Again, this is a technology that has benefited significantly from the advent of powerful inexpensive computers. The combination of neural networks with genetic algorithms has proved exceptionally fruitful for formulation optimization, because it allows the same neural network models to be used both for “what if” predictions and for optimization, as illustrated in Figure 2. This combination is necessary for realistic optimization. In theory it is possible to use the properties as inputs to a neural network and the ingredients as outputs (an effect-and-cause model, rather than cause and effect); however, in practice (as illustrated in Figure 3) for nonlinear problems like formulation, this approach can give models that are not consistent with the cause-and-effect models that have already been generated. Fuzzy Logic. Conventional logic, which can be traced back to the Greeks, depends on the premise that a hypothesis can be classified either as true or false. This maps onto conventional set theory, so that a hypothesis either lies in the true set, or else lies wholly outside it. That is, the membership in the true set is either 0 (for false hypotheses) or 1 (for true ones). The problem with conventional or “crisp” logic, though, is that it does not map intuitively onto everyday life. An oft-cited example is of “comfortable” room temperature. If 20 ºC is a comfortable


4


Ingredients & Process Conditions

Model 1

Measured Properties

Rowe and Colbourn

Measured Properties

Ingredients & Process Conditions Model 2

Figure 3. Final ingredients and process conditions are not the same as the initial ones, if you try to use two neural network models.

Figure 4. Fuzzy sets FAST and SLOW used to describe cure time of a coating.

Figure 5. Fuzzy desirabilities for tablet hardness and disintegration time. Tablets with hardness > 10 kg are 100% desirable; tablets with disintegration time < 300s are 100% desirable.

temperature, does this mean that 19 ºC, or 21ºC, is not comfortable? Crisp logic would insist that this is so, or else it would require the user to set up very complicated linguistic rules describing the concept comfortable. The same issues arise when describing any variable in linguistic terms. For example, for coating, a cure time of 250 seconds might be judged as fast, but this might not mean that a time of 260 seconds is too slow. Fuzzy logic allows intuitive statements and descriptors to be used. Fuzzy logic is based on the theory of fuzzy sets, introduced in the 1960s by Lotfi Zadeh [4]. For fuzzy sets, membership functions can take any continuous value between 0 or 1. In effect, in addition to the black and white of conventional logic, fuzzy logic allows for shades of grey. In the context of the cure time example, two sets, “fast” and “slow” can be defined, as illustrated in Figure 4. Here, cure times in a range around 300 seconds have memberships in both fast and slow sets. Fuzzy logic forms a useful basis for defining desirability functions describing the values that different properties should take. For example, in the case of the hardness of a

pharmaceutical tablet, higher hardness values are preferred. Figure 5a shows the desirability function for the case where tablet hardness should take a value greater than 10. For pharmaceutical tablets, we may also be looking for a short disintegration time so that the drug gets into the system as quickly as possible. Figure 5b shows the case where disintegration time should take values less than 300 seconds. This figure shows that disintegration times below 300 seconds are 100% desirable, and that desirability falls off in a linear fashion as the disintegration time is increased. A simple implementation of crisp logic would force us to say that a tablet with disintegration time of 299 seconds was totally undesirable; fuzzy logic allows us to assign a desirability of just less than 100% to such a formulation; however, because the desirability is used in the fitness assessment (shown in eq 2), then the higher the disintegration time, the more the formulation giving such a value is penalized in the optimization. Fuzzy logic is also widely used in process control, because it allows rules to be expressed in a simple linguistic format “IF (A) THEN (B),” with an associated confidence level that is related to the membership functions. This ability to express objectives in simple terms also allows fuzzy logic to be combined with other technologies, like neural networks, as discussed below, to give powerful data-mining techniques that can be used in formulation. Neurofuzzy Logic. Current work in artificial intelligence brings together the above-mentioned techniques to produce hybrid systems. One combination that is emerging to be useful in formulation is neurofuzzy logic [5, 6], which as the name suggests combines the learning capabilities of neural networks with the intuitive linguistic rules of fuzzy logic. This means that the black-box neural network attains a degree of transparency, leading to so-called “grey-box” models, which are easier to interpret and understand than neural network models. Neurofuzzy logic is gaining increased acceptance as a powerful data-mining technique, because it produces useful actionable rules from input data, in the form IF (precedent 1) AND (precedent 2) AND…THEN (consequent). Some neurofuzzy implementations are capable of developing parsimonious models that ascertain which input variables are important for specific properties [7], with no requirement for prior knowledge from the user. Thus, knowledge is discovered automatically by data, although the reasons why may remain obscure. For formulators, neurofuzzy logic can provide a useful tool in deciding the direction to follow for experimentation and in determining which variables affect particular properties. As yet, there is just one commercial implementation of neurofuzzy logic specifically for product formulators [8]; application of neurofuzzy logic in formulation is still very new. Applications to Formulation

Prior to 1995, there were few reported applications of neural computing in product formulation. Since that time, the number has increased rapidly with applications being reported in adhesives, dyes, paints, pharmaceuticals, and many more fields. The following examples, discussed by specific application areas, show some of the possibilities offered by the use of these new technologies.


Neural Computing in Product Formulation Adhesives. Neural networks were first used to model and optimize a new adhesive for personnel at the Lord Corporation (Chemical Products Division) in the early 1990s [9, 10]. The network was trained using data from 38 formulations, and the model was used to optimize the components of the promoter to provide a better level of performance. Neural networks have now been extended to the design of aqueous pressure-sensitive adhesives for paper labels at Dow Europe in Switzerland [11] and olefin thermosets at Honeywell Electronic Materials [12]. In both cases, neural networks proved effective in designing improved products. Agrochemicals/Agriculture. There have been two reports of applications in agrochemicals: the design of water dispersible granules at Zeneca Agrochemicals [13] and aqueous suspension concentrates at AgrEvo [14]. Both used neural networks to model the properties of their respective formulations and predict strategies for improvement. In the latter study, the optimized formulation was prepared and monitored to confirm the predictions. In the agricultural field in Japan [15], genetic algorithms have been applied successfully to the optimization of a feed mixture for livestock. Dyes and Pigments. Predicting the color of a dye or pigments mixture is extremely difficult, due to the intrinsic nonlinearities of the problem; however, initial experiments with dye mixtures for fibers, sponsored by Courtaulds in the early 1990s, proved that it was possible to use neural networks with a high degree of success [16, 17]. Since then, the technology has been applied successfully to pigments in Germany [18] and fluorescent dyes in both China [19] and Brazil [20]. Fats and Spreads. The final characteristics of such products as margarines and spreads depend on the physical and chemical properties of the oils and fats used in their formulation, and it is not surprising that neural networks have been used in this field. In an extensive evaluation in Brazil, Block et al. [21, 22] have shown that it is possible to train networks using the solid fat contents of a variety of formulated fats with three different raw materials and to be able to predict successfully the properties of new formulations considered acceptable by a panel of experts. Lubricating Oils. Predicting lubricant performance in engine tests is both difficult and costly, and the ability to interchange base oils while maintaining product quality is an interest of significant importance to the lubricant marketing industry. In pioneering work, personnel from Texaco [23] have shown that it is possible, using neural networks, to correlate the relative distribution of the molecular types (such as aromatics, naphthalenes, paraffins, and certain sulfurcontaining species) to lubricant performance in engine tests. In all cases, the predictions from neural networks were comparable to those using statistics and obviate the need for engine tests. Paints and Coatings. Paints and coatings are essentially complex mixtures of mutually interacting components, formulated for a specific application condition. Applications of neural computing in the field generally fall into two distinct groups: those concerned with the prediction of color and those concerned with the prediction of an end-use property. Work on the former was originally started in 1995 at the Kansai Paint Company in Osaka, Japan by Mizutani et al. in 1995 [24] and has resulted in a computer system that combines the three principal constituents of neural computing—neural networks,


5

genetic algorithms, and fuzzy logic—providing precise color recipe outputs [25]. End-use properties modeled and predicted using neural networks have included acid resistance, studied at the Glidden Company [9, 10], and rheological and mechanical properties, studied at the COLOR Company in Slovenia [26– 29]. In all cases neural networks modeled the systems successfully and efficiently, giving results comparable to statistical techniques. Paper and Board. Many interdependent variables affect the paper-coatings process, from coating formulation to machine settings; hence, optimization is very difficult. The formulation consists of clays (e.g. kaolin), calcium carbonate, titanium dioxide, latex binders, and other binders, all mixed to provide optimum brightness and opacity. In 1995, Novak et al. from Ljubljana, Slovenia demonstrated [30] that neural networks could be used to model paper coatings with differences between the predictions of experimental results well within TAPPI standards. This concept has recently been extended and combined with genetic algorithms to optimize coated-paper brightness [31]. Pharmaceuticals. Over the past few years, neural networks have gained more acceptance in modeling pharmaceutical formulations than in any comparable field. Applications now exist for immediate and controlled release tablets, skin creams, hydrogel ointments, emulsions, and film coatings. For controlled release tablets, the first studies were carried out in the early 1990s by Hussain et al. at the University of Cincinnati. They modeled the in vitro release characteristics of a number of drugs prepared from two hydrophilic polymers [32] and found that neural networks with a single hidden layer generally performed well in predicting drug-release profiles. Later studies at the University of Florida [33] and the University of Tennessee, Memphis [34], using similar formulations, confirmed these findings, as have recent studies at Hoshi University in Japan [35, 36]. Neural networks have also been used in Slovenia [37] and in China [38] to model the release characteristics of drugs from polymer matrices. More recently, work in this area has been extended to model osmotic pumps in China [39], and Malaysian researchers report work on pellet formulations [40]. For all these applications, a simple neural network with a single hidden layer was sufficient to give a good model for predicting drug release. Immediate release tablets have also been investigated, with two papers in the mid-1990s reporting the earliest studies. In the first, Turkoglu et al., working at the Universities of Marmara and Cincinnati, modeled tablet formulations [41] in order to maximize tablet strength and select the best lubricant. In the other, Kesavan and Peck, from Procter and Gamble Pharmaceuticals and Purdue University, modeled a tablet formulation of caffeine [42] in order to relate both formulation and processing variables with granule and tablet properties. Both of these investigations showed that neural networks performed better than conventional statistical methods. Subsequently, the data of Kesavan and Peck were reanalyzed using a combination of neural networks and genetic algorithms [43]. This study showed that the optimum formulation depended both on the relative importance placed on the output properties and on the constraints applied both to the levels of the ingredients used in the formulation and to the processing variables. Many optimum formulations could be produced, depending on the trade-offs that could be accepted for different aspects of product performance.


6


In a series of papers, Bourquin et al. from Novartis and the University of Basel in Switzerland have highlighted the pros and cons of neural networks for formulating immediate-release tablets [44–48]. In another study using both neural networks and genetic algorithms, Dang [49], working in Vietnam, has illustrated the usefulness of combining these technologies in formulating antacid tablets. Rocksloh et al. [50] in Germany have used neural networks to successfully optimize the formulation of tablets containing plant extracts. So-called topical formulations are applied directly to the skin. By their very nature they have many components, and neural networks have been applied to deal successfully with this complexity. The first work was performed on hydrogel formulations containing anti-inflammatory drugs at Hoshi University, Japan, in 1997 [51], followed up by further studies in 1999 [52] and 2001 [53]. Lipophilic semisolid emulsion systems have been studied in Slovenia [54, 55] and transdermal delivery formulations of melatonin in Florida [56]. In all cases, the superiority of neural networks over conventional statistics has been reported. Neural networks have also bee used to study the effect of ingredients on the phase behavior of emulsions in New Zealand [57], microemulsions in the UK [58], and colloidal-drug delivery systems in Malaysia [59]. In pharmaceutical formulation, film coatings are applied to tablets and granules either to protect the active drug against light or moisture or to control its release. The aim is generally to produce an opaque coating with good optical properties that does not crack, because cracking could lead to dose dumping. Neural networks have been used to model the release of drugs from coated pellets by Khan et al. at Texas Tech University at Amarillo [60]. Release profiles were optimized from the model and the formulations prepared. In every case the observed release was very close to that predicted from the model. In a pioneering study by Rowe and Woolgar [61], neurofuzzy computing has been applied to film coatings containing titanium dioxide as an opacifier. Formulation variables included polymer molecular weight, pigment type, pigment size and variation, pigment concentration, and film thickness. Output properties were crack velocity and opacity. The neurofuzzy technique discovered that crack velocity depended primarily on pigment size, size distribution, and concentration. Opacity, on the other hand, depended on pigment size, pigment concentration, and film thickness. In line with expectations, the technique discovered for itself that opacity was high when the films were thick or had high concentrations of pigment, increasing our confidence that sensible and useful rules will be developed from input data. Refractories and Flame Retardant Polymers. Work in this area has been undertaken primarily in China and Japan. Sugawara et al. [62] from the Krosaki Harima Corporation in Japan have recently applied neural networks to optimization of refractory formulations for the hot repair of BOF. Results from the evaluation of a trial product showed that the properties predicted by the model agreed very well with those found experimentally. For flame-retardant polymer composites, neural networks have been incorporated into an expert system to design new products [63, 64]. Rubber Compounding. Despite the fact that neural networks and genetic algorithms were first proposed as a method of developing and optimizing recipes for rubber compounding in 1996 [65], it was not until 1999 that applications first appeared in the literature. Borosy [66],

Rowe and Colbourn working in Hungary, has evaluated the concept using four different data sets, each containing several rubber mixtures with their respective properties. He concluded that the concept was suitable for mapping the quantitative nonlinear relations commonly found in the field of rubber compounding. Neural networks have been used recently in Argentina [67] to successfully predict the rheological properties of rubber compounds. Implications and Ramifications

The above examples show that neural networks have been used worldwide (including in developing countries). As discussed above, neural computing can be used for a wide range of application domains in the general area of product formulation and gives models at least as good as those obtained using statistical modeling. Its power lies especially in their ability to capture nonlinear relationships in the data without requiring prior knowledge from the user. From the applications cited earlier in this paper, the benefits that were seen included • effective use of incomplete data sets • rapid analysis of data • ability to accommodate more data and retrain the network (refine the model) • effective exploration of the total design space, irrespective of complexity • ability to accommodate constraints and preferences In a recent survey of 75 UK companies [68] covering the use of 93 neural computing applications in all business sectors, the major benefits identified were improved quality, improved response times, and increased productivity. 84% of users were satisfied or very satisfied with their systems, with only 3% expressing dissatisfaction. Specifically for the domain of product formulation, business benefits have been given as: • enhancement of product quality and performance at low cost • shorter time to market • development of new products • improved customer response • improved confidence Of the users mentioned in the previously cited study, 39% found problems related to software and lack of development skills. This situation is changing. In order that neural systems be used effectively by chemists with a modest mathematical background, it is important that software packages incorporate “rules of thumb” and default parameters that ensure the technologies are robust across a range of applications, so the user does not need artificial intelligence expertise. New commercial systems that meet this criterion are emerging; these integrate neural technologies with visualization and data manipulation capabilities within an easy-to-use interface to provide a valuable decision support system for product formulators. Even when commercial packages are used, however, they can be applied to advantage only if other specific conditions are met. The problem must be numeric in nature, and reasonable quantities of data must be available to develop an adequate model. The greatest benefits are achieved for multidimensional problems, because for these it is difficult to


Neural Computing in Product Formulation


7

express any analytical model and difficult to abstract the rules by any other mechanism than neural computing. The next generation of formulators in industry will find themselves using these techniques routinely and to an increasing extent. It is therefore vital that they understand the key concepts, the key requirements especially for their data, and the limitations and advantages of using neural computing in product formulation.

32. Hussain, A. S.; Shivanand, P.; Johnson, R. D. Drug. Dev. Ind. Pharm. 1994, 20, 1739–1752.

References and Notes

33. Ebuke, N. K.; McCall, T.; Chen, Y.; Meyer, M. C. Pharm. Dev. Technol. 1997, 2, 225–232.

1.

34. Chen, Y.; McCall, T. W.; Baichwal, A. R., Meyer, M. C. J. Controlled Release 1999, 59, 33–41.

Rumelhart, D. E.; Hinton, G. E.; Williams, R. J. In Parallel Distributed Processing: Explorations in the Microstructures of Cognition; MIT Press: Cambridge, MA 1986, Vol. 1, 318–362.

28. Leskovsek, N.; Tusar, L.; Tusar, M.; Zupan, J. Rheology 1995, 95, 140–145. 29. Tusar, L.; Tusar, M.; Leskovsek, N. Surface Coatings International 1995, 10, 427–434. 30. Novak, G.; Gornik, M.; Malesic, I. Das Papier 1995, 49, 220–223. 31. Kumar, A.; Hand, V. C. Eng. Chem. Research 2000, 39, 4956–4962.

35. Takahara, J.; Takayama, K.; Nagai, T. J Controlled Release 1997, 49, 11–20.

2.

Schiffman, W.; Joost, M; Werner, R. Proc. of the European Symposium on Artificial Neural Networks, ESANN '93; 1993, 97– 104.

3.

Holland, J. H. Adaptation in Natural and Artificial Systems; The University of Michigan Press: Ann Arbor, MI, 1975.

37.

4.

Zadeh, L. Information and Control 1965, 8, 338–353.

5.

Jang, J. S. R.; Sun, C. T.; Mizutani, E. Neuro-fuzzy and Soft Computing; Prentice Hall: Englewood Cliffs, NJ, 1997.

38. Sheng, H.; Wang, P.; Tu, J-S.; Yuan, L.; Pin, Q-N. Chinese J. Pharmaceuticals 1998, 29, 352–354.

36. Takayama, K.; Morva, A.; Fujikawa, M.; Hattori, Y., Obata, Y.; Nagai, T. J. Controlled Release 2000, 68, 175–186. Zupancic Bozic, D.; Vrecer, F.; Kozjek, F. Eur. J. Pharm. Sci. 1997, 5, 163–169.

39. Wu, T.; Pan, W.; Chen, J.; Zhang, R. Drug Dev. Ind. Pharm. 2000, 26, 211–215.

6.

Kasabov, N. K. Neurocomputing 2001, 41, 25–45.

7.

Bossley, K. M. PhD Thesis, University of Southampton, UK, 1997.

40. Peh, K. K.; Lim, C. P.; Quek, S. S.; Khoh, K. H. Pharm. Res. 2000, 17, 1384–1388.

8.

Roskilly, S. J.; Colbourn, E. A. FormRules; Intelligensys Ltd: Billingham, UK, 2002.

41. Turkoglu, M.; Ozarslan, R.; Sakr, A. Eur. J. Pharm. Biopharm. 1995, 41, 315–322.

9.

Gill, T.; Shutt, J. Scientific Computing and Automation; 1992, 5, 19– 26.

42. Kesavan, J. G.; Peck, G. E. Pharm. Dev. Technol. 1996, 1, 391–404.

10.

Verduin, William H. TAPPI Journal 1994, 7, 100–104.

11.

Calvert, D. J. Proc. 20th Annual Meeting Adhesives Society; 1997 369–371.

12. Nguyen, M. N., Chien, I. Y.; Knoll, P. M. Proc 2000 Electronic Components and Technology Conference IEEE; 2000, 878–881. 13.

Bell, G.; Skelton, S. E. In Proc Symposium Formulation–from Design to Delivery; Royal Society of Chemistry, London, 1998.

14. Webb, M. A. In Proc Symposium Formulation—from Design to Delivery; Royal Society of Chemistry: London, 1998. 15. Furuya, T; Satake, T; Minami, Y. Computers and Electronics in Agriculture 1997, 18, 129–135. 16. Bishop, J. M.; Bushnell, M. J., Westland, S. Col. Res. Applic. 1991, 16, 3–8. 17. Bishop, J. M.; Bushnell, M. J., Usher, A; Westland, S. Proc. Int. Conf. Artificial Neural Networks and Genetic Algorithms; 1993, 719–725. 18. Spehl, J.; Wölker, M.; Kettler, W. H.; Pelzl, J. Proc. Int. Conf. Neural Networks 1994, 5, 3336–3341. 19. Xin, J. H. J China Textile University (Eng. Ed.) 1999, 16, 46–48. 20. Bezerra, C. de M., Hawkyard, C. J., J. Soc. Dyer. Col. 2000, 116, 163–169. 21. Block, J. M.; Barrera-Arellano, D.; Figueiredo, M. F.; Gomide, F. A. C. J. Am. Oil Chem. Soc. 1997, 74, 1537–1541. 22. Block, J. M.; Barrera-Arellano, D.; Figueiredo, M. F.; Gomide, F. A. C.; Sauer, L. J. Am. Oil Chem. Soc. 1999, 76, 1357–1361. 23. Firmstone, G. P.; Smith, M. P., Stipanovic, A. J. Soc. Automot. Eng. Special Publication 1116 1995, 201–208. 24. Mizutani, E.; Takagi, H.; Auslander, D. M., Proc. IEEE Conf. on Evolutionary Computing 1995, 2, 533–538. 25. Mizutani, E.; Takagi, H.; Auslander, D. M.; Yang, J-S. R. IEEE Transactions on Systems Man and Cybernetics 2000, 30, 537–550. 26. Leskovsek, N.; Tusar, L.; Tusar, M.; Zupan, J. Kovine Zlitive Technol. 1994, 28, 321–325. 27. Leskovsek, N.; Tusar, L.; Tusar, M.; Zupan, J. Kovine Zlitive Technol. 1995, 29, 258–259.

43. Colbourn, E. A.; Rowe, R. C. Pharm. Tech. Eur. 1996, 8(9), 46–55. 44. Bourquin, J.; Schmidli, H.; van Hoogevest, P.; Leuenberger, H. Pharm Dev. Technol. 1997, 2, 95–109. 45. Bourquin, J.; Schmidli, H.; van Hoogevest, P.; Leuenberger, H. Pharm Dev. Technol. 1997, 2, 111–121. 46. Bourquin, J.; Schmidli, H.; van Hoogevest, P.; Leuenberger, H. Eur. J. Pharm. Sci., 1998, 6, 287–300. 47. Bourquin, J.; Schmidli, H.; van Hoogevest, P.; Leuenberger, H. Eur. J. Pharm. Sci. 1998, 7, 5-16. 48. Bourquin, J.; Schmidli, H.; van Hoogevest, P.; Leuenberger, H. Eur. J. Pharm. Sci. 1998, 7, 17–28. 49. Do, M. Q.; Dang, V. G.; Le, Q. N. Tap Chi Duoc Hoc 2000, 6, 16– 19. 50. Rocksloh, K.; Rapp, F-R.; Abu Abed, S.; Muller, W.; Reher, M.; Gauglitz, G.; Schmidt, P. C. Drug Dev. Ind. Pharm. 1999, 25, 1015– 1025. 51. Takahara, J.; Takayama, K.; Isowa, K.; Nagai, T. Int. J. Pharm. 1997, 158, 203–210. 52. Takayama, K.; Takahara, J.; Fujikawa, M.; Ichikawa, H.; Nagai, T. J. Controlled Release 1999, 62, 161–170. 53.

Wu, P-C.; Obata, Y., Fujikawa, M.; Li, C. J.; Higashiyama, K.; Takayama, K. J. Pharm. Sci. 2001, 90, 1004–1014.

54. Gasperlin, M.; Tusar, L.; Tusar, M.; Kristl, J.; Smid-Korbar, J. Int. J. Pharm. 1998, 168, 243–254. 55. Gasperlin, M.; Tusar, L.; Tusar, M.; Smid-Korbar, J.; Zupar, J., Kristl, J. Int. J. Pharm. 2000, 196, 37–50. 56. Kandimalla, K. K., Kanikkannon, N., Singh, M. J. Controlled Release 1999, 61, 71–82. 57. Alany, R. G.; Agatonovic-Kustrin, S.; Rades, T.; Tucker, I. G. J Pharm. Biomed. Anal. 1999, 19, 443–452. 58.

Richardson, C. J.; Mbanefo, A.; Aboofazeli, R.; Lawrence, M. J.; Barlow, D. J. J Coll. Interface Sci. 1997, 187, 296–303.

59. Agatonovic-Kustrin, S.; Alany, R. G. Pharm. Res. 2001, 18, 1049– 1055. 60.

Vaithiyalingam, S.; Khan, M. A. Int. J. Pharm. 2002, 234, 179–193.


8


Rowe and Colbourn

61. Rowe, R. C.; Woolgar, C. G Pharm. Sci. Technol. Today 1999, 2, 495–497.

66. Borosy, A. P. Chemometrics and Intelligent Laboratory Systems 1999, 47, 227–238.

62.

67. Schwartz, G. A. Rubber Chem. and Technol. 2001, 74, 116–123.

Sugawara, M.; Ikebe, T.; Iwashita, K.; Yamato, T. Taikabutsu 2001, 53, 2–9.

63. Xia, J.; Xiao, C.; Wang, J.; Huang, C. J Fire Sciences 2001, 19, 3– 17.

68. Rees, C. Neural Computing—Learning Solutions—User Survey. Department of Trade and Industry, London, UK, 1996.

64. Xia, J.; Wang, J.; Huang, C. J Fire Sciences 2001, 19, 309–328. 65. White, L. European Rubber Journal 1996, 178, 20–23.
