Direct marketing is another application better served by Neural Network techniques than traditional statistical ... division, marketing campaigns can be focused on each segment independently . Cluster .... and their amazing potential. Neural ...
Artificial Neural Networks and their Applications in Business
by The Pegasus Group: Mario Rosso Phillip Randolph
Submitted in partial fulfillment of the requirements for the graduate course MBA 253 Seminar in Information Techno logy California State University Fresno Fall 2001
Appendix A
A -0
Introduction and History World War II and its aftermath produced an explosion of scientific research. During the War, millions of dollars were spent on developing new technologies in an effort to speed the process of attaining victory. After the war, however, scientist began to seek ways to build on these newly developed technologies for peace-time purposes. One such technology was the computer. While the earliest electronic computers were little more than calculators, their development was seen as an overwhelming accomplishment in the field of science and technology. Scientists from around the world rallied around this new technology, and ideas and theories regarding how it could be used abounded. Of particular interest to many scientists was the use of computers as a replacement for the human thinking process. Thus, the field of Artificial Intelligence was created. During the early 1950’s research involving the function of the human brain was converging with research into computing. While biological scientists were developing theories on how the brain works, mathematicians were researching the application of these theories to the new field of computing. In 1956, researchers gathered at the Dartmouth Summer Research Project on Artificial Intelligence, held at Dartmouth College. The first of its type, this gathering produced two competing schools of thought for research into the field of Artificial Intelligence. These were “Expert-Systems” and “Neural Computing.” Neural Computing advocates believed that the processes by which the brain works could be reproduced through algorithmic modeling to produce semi-reasoning computer programs. Expert-systems advocates, however, believed that the process by which artificial intelligence was achieved was not of concern. The real concern was the results produced by the process.
Although the two competing schools of thought continued, Expert-Systems gained widespread acceptance as the preferred method of developing artificial intelligence. As a result, ES was able to solicit high levels of funding for research while Neural Computing took a back seat. Recently, however, neural computing has seen a wide-spread resurgence due to the limitations of ES. Expert systems typically require a complete understanding of the field in which it is applied. An expert-systems approach to artificial intelligence requires an intimate understanding of interactions by a human ‘expert’ who could successfully predict outcomes. With ES, an “expert would help to program a computer to mimic that understanding so that it could ultimately make its own predictions.” Essentially, the expert codifies his human knowledge into a set of rules which will guide the computer in determining the proper outcomes. The limitations, however, are that these systems can only be applied where human experts exist. In areas where there are no real experts, such as in accurately predicting the growth or decline of the economy, a different approach is required. Many computer scientists believe that neural computing in the form of Neural Networks may fill this gap in artificial intelligence. Neural Networks Professor Kevin Gurney of the University of Sheffield, defines a Neural Network as “an interconnected assembly of simple processing elements, units or nodes, whose functionality is loosely based on the animal neuron. The processing ability of the network is stored in the interunit connection strengths, or weights, obtained by a process of adaptation to, or learning from, a set of training patterns.” (http://www.shef.ac.uk/psychology/gurney/notes/l1/l1.html) Neural Networks, in contrast to Expert systems, do not require an explicit set of rules. Instead, the logic behind Neural Networks stems from the accepted biological model of how the
brain works. The network makes up its own rules that match the data it receives to the result it's told is correct. Neural Networks, like the brain, consist of numerous artificial “neurons” or processor units. Each unit receives inputs, both excitatory and inhibitory, from a number of other units and, if the strength of the signal exceeds a given threshold, sends signals to other units. The action, so to speak, is in the connections among these units. Herb Brody of Technology Review states, “Each neuron takes many input values, multiplies each by a "weighting" factor, adds together the products of these multiplications, and mathematically operates on this sum to produce another set of values. These numbers may translate directly into an answer--say, the identity of a spoken word or a written character.” The term “Neural Network” is inherently a biological one. While the term Neural Network can be used to define either a network of interconnected, organic nerve cells in biological organisms, the subject of this work is to identify attributes and applications of computer-based Neural Network systems. A more correct term for software-based Neural Networks would be Artificial Neural Networks or ANN. For the purpose of simplicity, from this point forward, the term Neural Network is used throughout this work to describe computer-based ANNs. Initially, a Neural Network must undergo a period of training to develop its “own rules.” During the training phase, the network is fed a stream or batch of input data, and it determines the strength or weight of the relationships between each variable and a given output. The training process determines input weights that will allow a Neural Network to function properly based on the training data. Provided that you train a Neural Network with appropriate and complete data, the network will determine appropriate outputs from inputs that are slightly
different from inputs that are entered during training. In other words, as the patterns of relationships are determined between the inputs and the given outputs, the network can predict different outcomes based on variations among various input variables. The real “intelligence” behind Neural Networks is its ability to automatically make corrections in the weights and relationships it gives to each variable entered. This is through a method called “backpropagation.” As the network determines the patterns, relationships, interaction effects, and weights of each variable as they are related to a given output, a set of “rules” is developed. However, if more data is entered into the system in the form of inputs and outputs, the network will test its set of rules. If the rules do not produced the desired outputs, the network goes back to determine what rules need to be changed. In essence, the backpropagation process adjusts the weights of the various relationships until they meet the given output. The weight of each relationship or variable is increased or decreased in small increments until the weights that produce the proper output are determined. As the proper output is attained, the network develops a new set of “rules” to guide itself in making new outputs. NEURAL NETWORK APPLICATIONS Data Mining Dramatic reductions in the cost of data storage have facilitated a substantial increase in the amount of data currently being collected and stored by organizations. In order for managers to gain value from this stored data and to improve the decision-making process, hidden patterns must be discovered. Neural networks are well-suited to the task of recognizing patterns in enormous amounts of data such as is commonly found in data warehouses. Some of the attributes of Neural Networks that make possible effective data mining activities include the ability to explore all possible interrelationships among key elements, model complicated
problems and multi-dimensional data and execute pattern extraction with greater speed than conventional tools. Marketing Forecasting retail sales is necessary to make decisions relating to efficient capacity planning. Issues such as ordering of raw materials, manpower requirements and inventory levels depend on accurate and timely retail sales forecasts. Traditionally, Multiple Regression Analysis (MRA) was performed to forecast sales. Utilizing Neural Network methods instead of MRA results in improvements in accuracy and user friendliness. Independent variables present at the input layer of a retail sales forecasting Neural Network might include market demand for product, disposable income of consumers, size of population, price of product, price of substitute products and price of complementary products. Dependent variables at the output layer will include sales and the amount of time to reach a break-even point. Within the hidden layers of the Neural Network, weights for the network can be initialized to arbitrary values. During the training phase, historical data is cycled through the network a number of times. As differences are identified between computed outcomes and correct historical data, the weights within the hidden layers will change and adapt. Finally, actual data can be fed into the network and reliable forecasts can be computed. Direct marketing is another application better served by Neural Network techniques than traditional statistical methods. One of the objectives of direct marketing is to minimize the amount of mailings to people who are not as likely to purchase the goods and services being marketed. The conventional statistical technique utilized to achieve this objective is Discriminant Analysis; however, Neural Network has proven to be a more efficient method for
identifying prospective customers. Independent variables such as age, sex, income, occupation, education, social class and geographic location serve as the inputs for the network. The outputs will include groups of prospective and non-prospective customers. Target marketing seeks to segment a large market into one or more smaller groups. After division, marketing campaigns can be focused on each segment independently . Cluster Analysis is traditionally used as the conventional tool for market segmentation. In applying Neural Network technology to this problem, independent variables such as demographics, socioeconomic levels, geographic location, purchasing behavior, consumption and attitudes toward product are presented at the input layer. The outputs of the network will include the desired number of market segments. Manufacturing The pattern recognition capabilities of Neural Networks make them well-suited for production environments. Applications in quality assurance, process control, process optimization and failure prediction are becoming increasingly common and accepted. Lucent Technologies, Inc. combines Neural Network with genetic algorithms to improve process control in its semiconductor fabrication labs. Variations in film thickness increase production costs and reduce semiconductor yields. The complex, 450-step production process involved in the fabrication of Lucent’s semiconductors, combined with the small tolerances of the devices, make it nearly impossible to identify which minute adjustments would have the most impact on improving process control. To solve this problem, engineers trained a Neural Network with plasma etching time data taken from a database of previous production lots. The resulting system reduced film thickness variation from 730Å to 220Å.
Another example of Neural Network applications in process control comes from the petroleum industry. Quality attributes for gasoline, such as octane, can be inferred through the use of Neural Networks. Traditionally, such measures of quality had to be obtained during processing using physical analyzers such as gas chromatographs. This physical analysis method caused delays in production and required the use of expensive, maintenance- intensive equipment. Neural networks utilizing simple inputs such as temperature, flow rates and pressure have reduced, and in some cases eliminated, the need for online physical analysis methods. Automobile manufacturers are utilizing Neural Network technology to improve the setting of warranty policies. By loading a Neural Network with sensor data obtained in new car stress testing and then comparing this data with the patterns of previous model tests, durability information can be deduced The resulting mean time between failure (MTBF) calculations are important in the establishment of warranty policies. Additional Neural Network applications in manufacturing include the Nestor Development System (NDS) software package. Designed by Nestor, Inc. to run on personal computers and Sun Microsystems workstations, it has the ability to recognize visual patterns for part identification. The system can recognize parts even after slight modifications have been made in their shape or in changing lighting environments. The NDS package also has applications in electronics manufacturing due its ability to recognize patterns in electronic signals. Financial Forecasting and prediction are important elements of investment decision making. While most investment decisions are made by individuals with expertise in the field of finance, Neural Networks are beginning to challenge the performance of human fund managers. Of the many
factors which make Neural Networks well suited to forecasting applications, the ability to analyze enormous amounts of historical performance data and to identify relationships and patterns in that data are of paramount importance. Data associated with financial markets is readily available and can easily be codified for use in computerized Neural Network systems. In addition, human factors such as ego and caution that may influence a decision maker’s judgment are eliminated when using Neural Networks. Financial forecasting Neural Networks can be separated into two categories. The first group is concerned with prediction of future events from the analysis of known facts. Examples of this type of Neural Network include programs that are designed to predict bankruptcies or potential loan defaults. A second category of forecasting Neural Network is concerned with Time Series Forecasting. Time Series Forecasting examines past events and data and attempts to predict future trends. An example of this type of Neural Network application includes systems that are used to predict stock and bond market trends and prices. As is the case with all Neural Networks, an investment forecasting Neural Network must be trained to recognize positive and negative relationships to produce useful outputs. Training is accomplished by utilizing either a supervised or an unsupervised learning technique. Supervised learning involves comparing the outputs of the Neural Network to actual historical outputs and adjusting the weights of the hidden layer to minimize the difference between actual and calculated results. Unsupervised learning does not involve the comparison of calculated and actual outputs. Instead, unsupervised learning methods rely on the network itself to find positive relationships within the large amounts of data that are fed into it. To illustrate the successfulness of Neural Networks in financial applications, four examples are provided herein. The first example involves an experiment conducted in 1990
involving stock market prediction. Using a Neural Network to analyze data from the Tokyo Exchange, researchers were able to predict the future direction pattern of the market with 94 percent accuracy. A second example involves bond rating. A sample of bond issues from 40 companies were used to test a Neural Network’s ability to rate bonds. Thirty of the bond issues were used as input data to train the network while the remaining ten were used to compare against the results generated by the network. The Neural Network was able to predict the bond ratings with an accuracy rate of 88 percent, compared with only 65 percent for regression analysis techniques. Third in the list of examples involves an Neural Network used by LBS Capital Management Inc. in Clearwater, Florida. The system was used to pick stocks for purchasing between the years 1991 and 1993. It performed consistently better than the Standard & Poor’s 400 Index each year. A final example involves the commodities trading market. A Neural Network trained with three years of trading data realized a 52 percent profit after operating for six months on an original investment of $3,000. Figure 1 below illustrates a sample of results from a number of Neural Network investment experiments.
Accuracy in Predicting Investments 90% 80% 70% 60% 50%
S&P 500
Euro NIKKEI Crude Oil Futures
40% 30% 20% 10% 0% Figure 1Neural Network Investment Accuracy
Yen
Risk Assessment Applications in risk analysis stand out as some of the most successful and wide-spread Neural Network business applications developed to date. Many large financial institutions ha ve incorporated Neural Network based fraud detection systems into their organizations. One such system, Visa U.S.A.’s Cardholder Risk Identification Service, produces an alert when irregular purchasing behavior is detected. Credit ratings can also be ge nerated using systems such as Neural Technologies’ Decider. Standard & Poors (S&P) is currently using Decider as the engine powering its Neural Network based CreditModel system. CreditModel is an Internet delivered service that allows financial institutions doing business with S&P to rate the credit risk of potential customers. CreditModel’s inputs include the customer’s revenues, debt load, and prior S&P credit ratings. Using CreditModel’s Neural Network to determine a bank’s credit rating has brought the cost of S&P’s service down from $40,000, using human analysis, to $400. Similarly, the process of approving or denying mortgages and loans can be delegated to an Neural Network. Sears Mortgage Corporation is using Neural Networks to streamline and augment its mortgage analysis. Up to 70 percent of Sears’ “clear-cut” mortgage applications are processed by its Neural Network, while human underwriters scrutinize the remaining “tough” applications. Among the thirty pieces of data input into the Sears’ neural net are applicant’s zip code, age of the home, appraised value of the home, monthly income, proposed housing expense and credit rating. The network is initially trained by running iterations on 2,500 application acceptances and 2,500 rejections. Outputs simply establish whether the application is to be accepted or rejected.
Other Applications The number of fields benefiting from Neural Network technology is vast, and identifying them all is beyond the scope of this report. Many of these applications are difficult to classify but deserve to be noted. Among these are systems that utilize the pattern recognition strengths of Neural Networks in applications such as the reading of Japanese Kanji characters and English hand writing on checks, identifying battle-field targets and classifying radar signals, and locating features on seismic maps that aid petrochemical industry geologists in the exploration of new natural resources. Two of the most surprising and interesting applications of Neural Networks technology discovered while researching this report include race horse paternity determination and network failure prediction systems. Researchers at the University of California School of Veterinary Medicine at Davis are combining Neural Network powered pattern recognition systems with video cameras to verify race horse pedigree. Proof of parentage, verified through a series of 142 reaction tests on a sample of blood, is a requirement for thoroughbred breeders to race a horse. Laboratories at Davis were becoming overwhelmed by as many as 72,000 reaction tests in a single day. To solve this problem, scientists at Davis installed a Neural Network system produced by San Diego-based, Science Applications International Corporation. The successful system has inspired research into similar biological testing applications for humans. Neural network based failure prediction and alert systems, such as Computer Associates’ Neugents product, seek to alert engineers of impending failures before an event actually occurs. Neugents software is currently being used to identify and learn historical patterns in data network
infrastructures to facilitate preventative corrective action before a system fails or its performance degrades. Summary and Conclusions The application of Artificial Neural Networks in business and industry is an emerging field with enormous potential. While still largely unknown or misunderstood, this rapidly evolving field holds promise for a vast number of tasks that were previously accomplished through human cognitive activity. Throughout the course of this study, Neural Networks were defined and a number applications were highlighted. While not an exhaustive list, many of these applications, such as those in fraud detection, process control, market prediction and pattern recognition, should give the reader an introductory understanding of Artificial Neural Networks and their amazing potential. Neural Networks are at last being touted as a worthy alternative to their Artificial Intelligence counterparts, Expert Systems. In the future, humans will delegate more and more of their cognitive and decision making activities to Neural Networks and Expert Systems. This fact should make for some exciting possibilities and concerns that deserve further study.
Bibliography Ady, Snit K. “Neural Network: Classification Method Helps Seismic Interval Interpretation.” The Oil and Gas Journal 96, no. 37, 14 Sept 1998, 47. Brambilla, A. and F. Trivella. “Estimate Product Quality with ANNs.” Hydrocarbon Processing 75, no. 9 (Sept 1996): 61-66. Brody, Herb. “The Neural Computer.” Technology Review 93, no. 6 (August-Sept 1990): 42-49. Cheng, Wei, Bruce W. McClain and Christopher Kelly. “Artificial Neural Networks Make Their Mark as a Powerful Tool for Investors.” Review of Business 18, no. 4 (Summer 1997): 4-10. Davidson, Clive. “A Mind of their Own.” Computer Weekly, 14 Jan 1993, 26-24. Enticknap, Nicholas. “Knowledge is the Key.” Computer Weekly, 30 Nov 1995, 48-49. “Fab Control gets Neural Guidance.” Electronic Engineering Times, 16 December 1996. FU, LIMIN. “Knowledge Discovery Based on Neural Networks.” Communications of the ACM 42, no. 11 (Nov 1999): 47. Getz, Arlene. “Out in The Cold: Travel nightmare: far from home with a frozen creditcard account.” Newsweek, 30 Oct. 2000, 102. Gianturco, Michael. “The Eye is a Computer.” Forbes 146, no. 1. 12 Nov 1990, 32-35. Hoffman, Thomas. “Neural Nets Spot Credit Risks.” Computerworld, 16 July 1999, 38. Knight, Kevin. “Connectionist Ideas and Algorithms.” Communications of the ACM 33, no. 11 (Nov 1990): 58-75. “Planes That Think for Themselves.” Poptronics 1, (July 2000): 15. Perkin, Julian. “Survey – FT-IT: Software Agents ‘Negotiate’ Production Schedules: Artificial Intelligence in Manufa cturing.” Financial Times (London), 1 November 2000, 7. Quitttner, Jeremy. “Credit Union Unit Gets New Tools to Fight Fraud.” American Banker 161, no. 64, 3 April 1996, 15. Ray, Gary. “Neural Networks.” PC Week, 5 Sept 1988, 86, 5. Roberts, Leslie. “Are Neural Nets Like the Human Brain?” Science 243, no 4890 (Jan 27, 1989): 481-482.
Romberg, Denise. “CA’s ‘Neugents’ Forcast Failures.” Computing Canada 25, 1 Jan 1999, 17. Smith, J. Clarke. “A Neural Network – Could it Work for You?” Financial Executive 6, no. 3 (May-June 1990): 26-31. Venugopal,V. and W. Baets. “Neural Networks and their Applications in Marketing Management.” Journal of Systems Management 45, no. 9 (Sept 1994): 16-22. Wright, Maury. “Neural Networks Tackle Real-World Problems.” EDN, 8 Nov 1990, 7987.
APPENDIX A Commercial Neural Network Software Applications (source: http://www.emsl.pnl.gov:2080/proj/neuron/neural/products/) Marketing • •
• •
• •
•
•
•
Adaptive Decision Systems o Application: evaluates direct marketing decisions BehavHeuristics Inc. o Application: forecasts demand of airline flights o Product: Airline Marketing Assistant Britvic o Application: predicting sales of soft drinks (e.g., Tango Orange Man) Churchill Systems o Application: optimize marketing Strategy o Product: The Target Market System o Customers: Veratex Corp. Microsoft o Application: direct mail marketing NeuralWare o Application: determine which customers should receive catalogs o Customer: Spiegal Inc. HNC Software Inc. o Application: predictive modeling system for direct marketers o Product: DataBase Mining® Marksman HNC Software Inc. o Application: Web advertising placement server o Product: SelectCast major US supermarket chain o Application: finding the connection between the sales of diapers and beer
Real Estate • •
Day & Zimmerman, Inc. o Application: real estate appraisal HNC Software o Application: property valuation system o Product: Automated Real Estate Appraisal System (AREAS)
Market Trading • • •
Carl & Associates o Application: stock forecasting Citibank o Application: trends analysis in international markets Daiwa Securities Co., Ltd. and NEC Corporation o Application:stock price forecasting
•
• • • • • •
•
• •
Econostat Ltd. o Application: bond portfolio management o Product: Global Bond Gerber Baby Foods o Application: manage cattle futures trading G.R. Pugh & Company o Application: corporate bond rating John Deere & Company o Application: pension fund management LBS Capital Management Inc. o Application: mutual fund management Mellon Equity Associates o Application: portfolio management Neural Applications Corporation o Applications: stock forecasting o Product: NetProfit NeuroDimension, Inc. o Application: Technical analysis and prediction of stocks and other financial data o Product: TradingSolutions O'Sullivan Brothers Investments, Ltd. o Application: monetary market trading Walkrich Investment Advisors o Application: common stock valuation
Fraud Detection • •
• •
•
•
Dunn and Bradstreet o Application: check approval HNC Software o Application: credit card fraud detection o Product: Falcon o Customers: First Bank USA, Household Credit Services, and National Colonial Bank USA MasterCard o Application: identify deviations in spending habits Nestor Inc. and Fraud Detection Systems o Application: credit card fraud detection o Customers: Bank of America, Canadian Imperial Bank of Commerce, and Europay International S.A. Belgium NeuroMetric Vision System Inc. o Application: signature verification from checks o Product: Check Signature Verification System o Customers: US banking industry and Federal Reserve System Visa o Application: identify deviations in spending habits
Credit Rating • •
•
Adaptive Decision Systems o Application: credit scoring Chase Financial Technologies o Application: forecasting credit worthiness o Product: Creditview HNC Software o Application: mortgage underwriting and risk- management o Product: Colleague o Product: Aquarius
Process Controllers • • •
• •
•
•
Fujitsu o Application: controller for continuous casting Honeywell o Application: process control systems Neural Applications Corporation o Application: controller for steel making o Product: Intelligent Arc Furnace Nippon Steel o Application: controller for continuous casting Pavilion Technologies o Application: process control, waste reduction o Product: Process Insights® o Customers: Eastman Kodak Siemens o Application: process control of rolling mills o Customers: Krupp Hoesch Stahl, HYLSA, Thyssen, Voest Alpine, Hanbo, Gallatin Steel, Texaco's Puget Sound Refinery o Application: refinery process control system
Quality Control Systems • • • • •
Anheuser-Busch o Application: beer testing Applied Intelligent Systems o Application: vision recognition systems CTS Electronics o Application: loudspeaker defect classifications Dunlop o Application: tire testing Intel o Application: computer chip manufacturing quality control
• • • •
Monsanto o Application: predicting quality of plastics Netrologic, Inc. o Application: welding quality control NLK - Celpap o Application: predicting quality of paper Volvo o Application: diesel knock testing, paint inspection
