734
IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING,
VOL. 15,
NO. 3,
MAY/JUNE 2003
Knowledge-Based Search in Competitive Domains Steven Walczak Abstract—Artificial Intelligence programs operating in competitive domains typically use brute-force search if the domain can be modeled using a search tree or alternately use nonsearch heuristics as in production rule-based expert systems. While brute-force techniques have recently proven to be a viable method for modeling domains with smaller search spaces, such as checkers and chess, the same techniques cannot succeed in more complex domains, such as shogi or go. This research uses a cognitive-based modeling strategy to develop a heuristic search technique based on cognitive thought processes with minimal domain specific knowledge. The cognitive-based search technique provides a significant reduction in search space complexity and, furthermore, enables the search paradigms to be extended to domains that are not typically thought of as search domains such as aerial combat or corporate takeovers. Keywords—Search, knowledge-based, chunking, opponent modeling, heuristic pruning, chess.
æ 1
INTRODUCTION
D
EEP Blue’s recent victory over Gary Kasparov, the human world chess champion, has validated the use of brute-force search techniques augmented with specialized hardware as a viable means for emulating human performance on complex search tasks [8]. Deep Blue’s success was enabled by special circuitry running with an IBM RS6000 parallel computer with 256 processors and brute force search techniques that analyze 200 million positions per second and achieve a maximum depth of 14 ply (a ply is a single player’s move in the game of chess) [8], [29]. However, even at a search depth of 14 ply, computer chess programs face the horizon effect. That is, their move choices are based on limited (incomplete) tactical information. As an example, Bratko and Michie [9] have demonstrated that the simple KRKN (King and Rook versus King and kNight) endgame which is solvable by most average strength chess players has a solution that may be 52 ply deep in a search tree, thus preventing the chess program from discovering a guaranteed win using brute-force search due to the horizon effect (i.e., not having complete look-ahead information). Traditionally, the artificial intelligence community has been divided over which methods will improve tree search and, consequently, game performance. One group believes that the depth of the search is the most important criterion affecting the playing ability of game programs. The second group focuses on increasing performance through the use of domain knowledge. In support of the first approach, Adelson-Velsky et al. [1] state that increasing the depth of search of a game tree increases the quality and reliability of game play. Berliner and Ebeling [6] specifically state that a deep tree search will outperform a shallower search that uses more knowledge.
. The author is with the College of Business and Administration, University of Colorado at Denver, Campus Box 165, PO Box 173364, Denver, CO 80217-3364. E-mail:
[email protected]. Manuscript received 2 June 2000; revised 16 Feb. 2001; accepted 8 June 2001. For information on obtaining reprints of this article, please send e-mail to:
[email protected], and reference IEEECS Log Number 112227. 1041-4347/03/$17.00 ß 2003 IEEE
In competitive domains, techniques for improving search depth include forward pruning, iterative deepening, and other heuristic methods for limiting the focus of a search. A detailed description of game-tree search methods is given in [5]. Chess domain knowledge is encapsulated in the position evaluation functions. Additional domain knowledge may be used in the form of opening and ending books where rote lines of play for specific positions are recorded. Unfortunately, chess programs do not acquire any knowledge from these opening and closing books, just current accepted lines of play. Due to the lack of any strategic knowledge in chess playing computer systems that rely on the brute-force search strategy, chess grandmasters and masters typically try to play unusual moves to place the computer into unfamiliar territory, i.e., take it out of it’s opening book [26]. Kasparov, following his victory in the 1996 Deep Blue match, indicated that this was his strategy and grandmaster V. Anand [3] stated that Kasparov was playing like “Mr. Anti Deep Blue” during the 1997 rematch. The second group of artificial intelligence practitioners claims that a better way to improve program performance in competitive domains like chess is to increase the amount of domain knowledge that is available to the program. A more knowledge-based approach to movement selection would decrease the effectiveness of the unusual play strategy typically employed by competent human competitors of chess programs. Additionally, now that Deep Blue has shown that brute force search can produce effective grandmaster level play for the domain of chess, which has a search space of approximately 10123 nodes, other more complex competitive games are becoming a focus for artificial intelligence research. Among these games are shogi (Japanese chess), with a search space of approximately 10226 , with it’s 9 9 board and 20 pieces, and the ability to reintroduce pieces into the game via a “drop move,” and go with a search space of approximately 10360 [27]. Published by the IEEE Computer Society
WALCZAK: KNOWLEDGE-BASED SEARCH IN COMPETITIVE DOMAINS
The quality of knowledge has a direct effect on the playing ability of the game program [7], [25]. Northwestern University’s CHESS4.6 was able to defeat other chess programs of the time that searched several ply deeper into the game tree based on the strength (knowledge) of its evaluation function [15], [44]. Other than incorporating more and more knowledge into an evaluation function, analogical reasoning or pattern-based reasoning may be used to accurately prune tree-based searches or produce move selection directly. Pattern-based solutions to games or portions of games have proven to be highly effective. Pattern-based game programs exist in checkers [33], Othello [14], chess [6], and go [43] and are frequently used in real-time interactive gaming programs. The chess and go programs [6], [43] use patterns to identify common board positions that have some well-known solutions in order to limit the tree search for a specific solution. Perhaps the most well-known example of using previous game knowledge and patterns for playing a chess-like game is Samuel’s [33] checkers playing program, which has recently been imitated by Epstein et al. [16] in their Hoyle program that plays zerosum two person games like tic-tac-toe or shisima. The extent of pattern-based problem solving (and play) in chess is typically limited to the end-game portion of the game. In chess, Quinlan’s [31] ID3 program has been used to model solutions to King-Rook end games, with other types of pattern-based end game play addressed by various other researchers [10], [44]. Wilkins [44], [45] also discusses how to use patterns in active middle game play, where the evaluation function is only concerned with piece count. Pattern-based techniques have been used in tsume shogi [20], which is similar to solving chess end-game problems for the domain of shogi. The difficulty with using pattern-based approaches in previous research is that the patterns are limited to specific portions (and sometimes styles of play) of the game. Additionally, storage and usage of patterns typically requires large amounts of space that slows down the program’s playing speed. Walczak [35] introduced the idea of using pattern-based, tree-search reductions based on cognitive processing versus domain-specific knowledge with the Inductive Adversary Modeler (IAM) program. The IAM program, covered in more detail in Section 2, acquires visual cognitive patterns of geometrically related pieces through observation of multiple games of an opponent. Since the cognitive patterns are acquired through direct observation, they are applicable to the entire duration of a chess game, thus overcoming traditional problems associated with pattern based search techniques. These chunks are then used to determine the moves an opponent will select by following the cognitive principle of reduction, thus reducing the complexity of the game tree in the same way a skilled chess player achieves cognitive economy through maneuvering a chess game into a familiar position. Other recent research [22], [23], called opponent model search, has focused on incorporating differing models of opponent skills and goals. The problems with opponent model search are that it assumes perfect knowledge of an opponent’s evaluation criteria (and, consequently, does not
735
account for cognitive limitations opponents face when evaluating the current domain) and it is simply a modification of traditional alpha-beta search routines using two evaluation functions instead of one. Hence, the IAM-based search reduction from cognitive patterns is significantly different from opponent model search as described by Iida et al. [22], [23]. In this article, the IAM methodology for acquiring cognitive chunks is briefly outlined in Section 2. Applying the cognitive patterns, through emulation of game player’s behavioral patterns, to reduce game tree search and an analysis of the advantage gained is presented in Section 3. Finally, extending the IAM search reduction method to other domains is discussed in Section 4.
2
THE IAM PATTERN ACQUISITION APPLICATION METHODOLOGY
AND
IAM emulates expert chess players as well as experts in almost any domain, by observing the performance of an opponent prior to engaging in play [21]. The idea of modeling expert performance through learning by observation is similar to the rote memorization of games performed by Samuel’s checkers playing program in the early to mid1960’s [33]. Waterman used observation of opponents to select betting strategies from a set of heuristic rules [40]. In fact, modeling expert performance by copying what experts have already done is the philosophy behind the use of opening books in chess programs. The key question is what needs to be learned from observations of expert opponents? The IAM program acquires the cognitive/perceptual patterns that experts use to organize and make sense of their complex domains. Acquisition of cognitive patterns (at least for the domain of chess) is based on the empirical evidence and theory provided by Chase and Simon [12], who claim that chess experts acquire domain relevant chunks. The chunks used in chess represent configurations of pieces on the chessboard. Use of the chunks during game situations provides a cognitive economy to the chess expert by reducing the complexity of the current situation and, thus, provides a better evaluation of the current game situation. Therefore, the pattern learning performed by the IAM program enables the IAM program to partially simulate the way in which human experts evaluate board positions. The chunking theory of learning [24], which states that performance in a domain improves via the acquisition of knowledge about patterns and chunks, provides additional theoretical support for the use of cognitive chunks or patterns for simulating the search mechanisms of human experts. A vast amount of additional research also supports the concept that expertise is a function of skilled pattern recognition [13] and recall of domain relevant patterns [11], [12], [18]. Ericsson and Smith [17] go even further and claim that a monotonic relation exists between recall of chunks and expert performance. Further evidence to support IAM’s perceptual chunk acquisition methodology is provided by Simon and Gilmartin [34] who track the eye movements of chess experts over a board and identify an expert’s focus of attention around collections of pieces on the board that
736
IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING,
correspond to the chunk definition used by IAM. Charness [11] has performed a longitudinal study of a developing chess expert and claims that the chess expert’s increase in performance is a result of acquiring additional perceptual chunk knowledge and that expert chess players do not search any deeper into a game tree than intermediate level players. Thus, chess expertise is a function of knowledge of patterns and other domain dependent knowledge that enable more accurate evaluation of domain situations and has little to do with search speed. Chase and Simon [12] indicate that human chess experts may acquire from 10,000 to 100,000 chunks during their progress towards expertise. These chunks are collections of pieces that are related primarily through proximity and similarity, meaning that piece chunks consist of like colored pieces that are next to each other on the board and chains of these types of related pieces. Chase and Simon’s definition of chess cognitive chunks is similar to the Gestalt psychology principles of organization that all humans use to interpret sensory stimulus from the world and it implies that specific patterns on the chessboard will be more meaningful than others [4]. The IAM program learns cognitive chunks for individual opponents by examining the records of previous games played by the opponent. The purpose of this article is not to describe the IAM pattern learning algorithm (described in detail in [35], [36], [37]), but rather to investigate and evaluate the contributions to search strategies of pattern-based learning. The patternbased learning utilized in IAM is quasi-domain-independent method, since perceptual knowledge of visual clustering by humans is the main acquisition strategy. The perceptual patterns are then used to automatically prune search trees based on the chunks of pieces displayed by chess-like game players during the course of a game. A brief overview of the IAM methodology is presented to familiarize the reader with the origin of the chunk patterns used in the tree pruning process. After each move made by the studied opponent, IAM searches the current board configuration of pieces to identify collections of pieces that are contiguous and of the same color as the opponent being modeled. A contiguous piece is any piece in one of the eight adjacent squares to the square being examined. The current limitation of IAM is that it only acquires contiguous same-colored pieces. This is a result of previous research by Chase and Simon [12] (satisfying most parts of their definition of a cognitive chunk) and also of the principles of organization from Gestalt psychology [4]. Once a piece is identified as having another similarly colored contiguous piece, the operation is performed recursively on all of the contiguous pieces to form larger patterns. All identified chunks are placed into a temporary database of probable cognitive chunks. If the same chunk or an identical transposition of the chunk through either a vertical or horizontal axis occurs at least once in each of a minimum of two games, then the chunk is removed from the temporary knowledge base and placed into a permanent knowledge base of cognitive chunks for the specific opponent being studied. In addition to the configuration of the pieces that make up the cognitive pattern, knowledge
VOL. 15,
NO. 3,
MAY/JUNE 2003
concerning the time of the game (first to last game moves during which the pattern was visible), the color of the pieces making up the pattern, the eventual win/loss result of the game, and chunk frequency (number of games observed versus total games analyzed) are stored in the knowledge bases. The chunks learned from a grandmaster opponent by the IAM program are comparable in both size and complexity to the chunk measurements reported by Chase and Simon [12] for players of at least international master caliber [38]. The similarity of the chunks acquired by IAM to the Chase and Simon measurements lends credence to the validity and probable use of the chunks by the modeled grandmaster player. A castled king chunk is a common piece of chess knowledge for most players. Fig. 1 illustrates a specific castled king chunk acquired for the grandmaster M. Botvinnik by the IAM program, with the chunk shown in a bolded box. The displayed chunks occurred in three separate games and were repeated by Botvinnik in many of his subsequent games. Thus, the cognitive perceptual chunks are learned by the chess player (or other chess-like game players) in order to acquire cognitive economy during the processing of complex game positions. The cognitive patterns/chunks learned by IAM are then used in a current game situation through analogical reasoning to predict the move selection of the modeled chess opponent. Prior to every move made by an opponent in the search of the game tree, the game board is scanned to identify partial patterns visible at that point. A partial pattern is a pattern that is similar to one in the permanent knowledge base, but is missing a piece (or two in the case of a king and rook combination). Next, a completion algorithm determines if a single move by the opponent can complete the pattern and thereby afford the opponent greater cognitive economy. It is possible that no partial patterns currently exist on the game board, in which case no move prediction is made. All patterns that may be completed are then weighted using cognitive psychology principles [4]. One such principle is that the larger the pattern, the greater the cognitive economy. Another important principle is that the more frequently a pattern has been associated with an opponent’s moves in the past, the more likely it is to be used by the opponent in future games [36]. The most significant (largest weight) pattern that may be completed is then used to predict the move choice of the opponent. The prediction algorithm transforms the cognitive (perceptual patterns) of pieces formed by an opponent into a behavioral pattern that implies recreation of the pattern to gain cognitive economy. Previous results [36] indicate that, with a minimum of four previous games (recall that at least two games must be analyzed to produce any repeated patterns), the IAM method can achieve a sustainable prediction level of 20 percent of an opponent’s moves throughout a chess game. As new patterns are learned by IAM, an incremental improvement in the move choice prediction level is achieved. This incremental improvement typically increases the move prediction by IAM up to an average of 25 percent of all opponents’ moves. After a sufficient number of patterns are learned, where sufficient is dependent on the opponent and implies a
WALCZAK: KNOWLEDGE-BASED SEARCH IN COMPETITIVE DOMAINS
737
Fig. 1. Pattern learned by IAM for Botvinnik from the Hague-Moscow Tournament.
general coverage of the most frequently used cognitive chunks, the IAM prediction algorithm achieves a 20 to 32 percent prediction accuracy, again averaging approximately 25 percent. As an example, the IAM algorithm has succeeded in predicting 28 percent of Kasparov’s move choices, while it only predicts 27 percent of Karpov’s and just over 20 percent of Botvinnik’s move choices. The effect of predicting one fifth to one quarter or more of an opponent’s moves can significantly impact the play of a game. While certain nonquiescent positions (an imminent capture or threat to the king) may result in forced moves, the majority of moves in a chess game (as well as go and shogi) are at the discretion of the player. Using an analogy to demonstrate the power of predicting moves, imagine the outcome of a poker game if you could accurately tell when an opponent is bluffing or not bluffing 25 percent of the time, or if a US football coach knew the exact play selection of an opposing team for every fourth play in the game. Similarly, the ability to know ahead of time exactly how a chess (or go or shogi) player will respond to a specific board situation can significantly improve the performance of a chess program in developing plans and counter-plans that operationalize these board positions. Prediction of move choice based on completion of cognitive patterns/chunks assumes that skilled players will automatically attempt to reduce the complexity of the domain situation with respect to recognized chunks (with respect to their learning, though perhaps not their opponent’s). As an example, we return to the chunk that was assembled by Botvinnik during three of the first four games
from the Hague-Moscow Tournament that is shown in Fig. 1. Through an 88 game study of Botvinnik, the chunk shown in Fig. 1 (or minor variations of the chunk) was observed in 76 percent of the games analyzed, indicating that Botvinnik consciously or subconsciously chose to maneuver his chess games into positions that contained his familiar chunk. The effect of using cognitive patterns to predict opponent moves is analyzed in Section 3. The patterns and knowledge used by the IAM program to this point is relatively domain independent including factors such as proximal similar color pieces and the time and frequency these patterns are observed. Only the move completion algorithm requires knowledge of legal moves in the domain to determine if a pattern may be completed. The domain independence of the IAM methodology implies that the same cognitive pattern learning and partial pattern recognition algorithms may be used robustly in other domains. Walczak and Dankel [37] report some preliminary results of applying IAM to the game of go with the identification of tactical go patterns, and Walczak and Grimbergen [39] have applied IAM to the domain of shogi (with it’s 10226 search space) with similar results to earlier chess domain implementations. A method for modeling military domains using the IAM methodology and the consequent learning of cognitive patterns appropriate to these military domains has also been demonstrated [36]. Although a domain independent search enhancement technique is a goal, ignoring domain dependent knowledge
738
IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING,
may produce a strategic knowledge horizon effect, similar to a search horizon effect that produces suboptimal move selection by brute force search techniques. The addition of just a minimum amount of domain dependent knowledge, in this case not leaving a major piece “en prise” (susceptible to capture), reduces incorrect move predictions by IAM due to the strategic horizon effect by 5 percent [36]. The IAM knowledge-based search strategy is not meant to replace evaluation algorithms, since the IAM search method does not incorporate domain specific knowledge, but rather to works in conjunction with established evaluation functions.
3
ANALYSIS OF IAM PATTERN-BASED TREE SEARCH REDUCTIONS
The IAM methodology requires acquisition of patterns from a collection of games played by a single opponent. A minimum of two games played by a single adversary is required by IAM to fulfill the definition of a cognitive pattern occurring in multiple games. There is no maximum for the number of games that may be analyzed by IAM to produce a knowledge base of cognitive patterns displayed by an opponent. Evaluation of the IAM methodology is performed by evaluating the board positions, using the pattern completion algorithm, from subsequent games that are played by the same opponent. As an example, the chess games of M. Botvinnik from 1948 through 1963, during his eight World Chess Championship tournaments, are used to model the cognitive patterns of Botvinnik. Starting with the 1948 tournament, the complete set of games from all tournaments up to a specific point in time are used to acquire cognitive chunks. Then, the current chunk knowledge base of IAM for Botvinnik is used to predict moves made in all of the games from the next tournament. Contrarily, the evaluation of IAM in predicting the move selection of G. Kasparov is performed by using only his games from the 1990 World Chess Championship. After the first two games were played and analyzed by IAM to acquire patterns, the IAM knowledge base of Kasparov patterns is used to predict moves in the next game in the tournament. This procedure is repeated, adding one game at a time, through the first 13 games in the tournament, producing a prediction set for 11 different games that occur sequentially in time.
3.1 Search Size and Speed A concern of game program researchers with the use of knowledge directed search is the amount of overhead required for the knowledge base to direct the search. This concern arises in part from the large size of the knowledge bases. As an example, opening books in the late 1980’s and early 1990’s contained from 5,000 to 60,000 bytes of information [30] and are even larger today. Specialized hardware [6], [29] has been used in the domain of chess to speed up database accesses as well as to generate new positions for the game tree search. Several researchers have commented on the need to minimize opening book size [2], [42]. The large size of chess program’s opening books is due to following the game theory concept of accounting for an adversary’s capabilities [46] (possible moves). For chess and other adversarial
VOL. 15,
NO. 3,
MAY/JUNE 2003
domains, the optimum tactics and strategy is achieved by “playing the man, not the board [41].” The same concerns for size and speed of access must be addressed by any knowledge-based search method. Since the IAM method learns the cognitive patterns used by specific adversaries to produce behavioral patterns, “playing the man and not the board,” the amount of relevant data that needs to be loaded is much smaller than the space required for standard opening books. Only those patterns associated with the current opponent need to be loaded. Many of IAM’s patterns for a specific opponent may have also been acquired for multiple other opponents. However, if a specific pattern cannot be associated with a specific opponent (through observation of previous games), it is pointless to attempt to utilize that pattern for predicting the opponent’s moves. The patterns produced by the IAM cognitive pattern learning method are 35 bytes in size. This breaks down into: 16 bytes for the actual pattern; 12 bytes to record the start, stop, and average game turn or “window of play” during which the pattern has been observed, 4 bytes to record the frequency of the pattern, 1 byte to register the shape of the pattern, and 1 byte each to record a win/loss and a black/ white usage flags. No special encoding is used to represent the data values and, so, the use of bitmaps and other encoding techniques can further reduce the size of each learned pattern. As an example, the version of IAM used for this research runs on a Sun 3 workstation on a Unix-based operating system that uses 4 bytes to represent integers instead of the 2 bytes commonly used on personal computers. Converting the IAM program to a personal computer platform would immediately reduce the size of the stored patterns by 6 bytes, affording 17 percent savings in the size of each stored pattern. Additionally, the last 3 bytes of information may be converted to a binary/bit representation, thus reducing the knowledge representation space requirements by two bytes per pattern. The quantity of patterns learned for an opponent tends to increase as a function of the number of historical games analyzed. For a 12 game study of Kasparov (taken from the 1990 world chess championship match against Karpov), the IAM cognitive pattern learning method acquired 33 patterns that would require only 1,155 bytes of memory to store the entire pattern knowledge base. Likewise, for an 80 game study of M. Botvinnik, 138 patterns were learned, requiring only 4,830 bytes of memory. The small size of the pattern knowledge base enables it to be stored in memory and thereby increases efficiency since only memory transfer rates are needed instead of disk access to read in a much larger opening book database. These small knowledge bases are still quite effective in predicting opponent move selection, with prediction rates for Kasparov averaging approximately 27 percent of all moves in a game and prediction rates for Botvinnik averaging almost 20 percent of all moves in a game. The better performance of IAM for predicting Kasparov may be due to the cognitive psychology “recency effect” [36], which states that any person is more likely to reuse information that has occurred more recently in time. Recall that the Kasparov patterns were acquired during a single tournament and used
WALCZAK: KNOWLEDGE-BASED SEARCH IN COMPETITIVE DOMAINS
739
to predict moves in games that occurred later in the same tournament. The Botvinnik patterns were acquired from games that spanned his entire career as world chess champion, 1948-1961 (with the 1963 tournament only being used for evaluation) and, thus, represent the extended learning of cognitive patterns. A future research topic will be to analyze the use of a smaller one or two year window for determining which patterns learned by IAM for an opponent to use in predicting move choice. Preliminary results indicate that those chunks which are closer in time to the game that is to be predicted will have a higher accuracy in predicting move choices of the opponent. This future research would also evaluate the “forgetting” or deletion of patterns from IAM’s knowledge base for a specific opponent if the pattern is not replicated within the moving one or two year window of experience. Alternatively, the current IAM system implements a form of memory decay by decreasing the frequency weight (the ratio of times the chunk has been observed to the total number of games analyzed) of chunks as new games are studied. Over time, a chunk that was used frequently during an earlier stage of learning will see a reduction in use by the chess player. As the number of games observed by IAM for the chess player continues to rise, the frequency of observation weight will converge to zero. As the chunk’s frequency weight diminishes, the chunk will be less likely, but still available, to suggest an opponent’s move choices.
3.2 Search Efficiency The IAM application of patterns to board position to determine an opponent’s move choice currently works in millisecond time. Further decreases in the size of the chunks as well as other possible optimizations of the software code will decrease the time constraints even further and enable IAM to realize microsecond retrieval times. How does this impact current brute force search engines? Deep Blue was reported to search 200 million positions per second [29] during the 1997 match against Kasparov. If the potential microsecond time is achieved, then using the IAM prediction algorithm (recall that the algorithm may only be run at every other level in the game tree as it can only predict the opponent’s move choice) reduces the quantity of searchable positions by a factor of 100 to 2 million positions per second. Is this a price that game search engines should pay? The projection ability of certain types of knowledge in the chess domain is 9 ply for tactical knowledge (looking ahead 1 or 2 ply in the game tree), 25 ply for positional knowledge (understanding of positional properties of the game such as defensive pawn-chains and castled kings), and 45 ply for strategic knowledge (understanding the goals and motivations of your opponent) [15]. IAM enables tactical and some positional knowledge to be used during the search process instead of waiting for the domain dependent evaluation algorithms at the end of the search. The average speed of a search algorithm is given by the equation: Oðstandard searchÞ ¼ k1 N m ¼ OðN m Þ
ð1Þ
[35], where k1 is the evaluation algorithm time, m is the depth of the search (14 ply for Deep Blue), and N is the average number of positions evaluated at each ply. A fair estimate of the average number of position evaluated at each ply is approximately 30. As each new ply is constructed via the position generation software or hardware, if that ply represents an opponent’s move, the IAM prediction mechanism is activated. If a prediction is made, then all of the remaining branches other than the predicted position are automatically pruned. The cost of analyzing the board positions is present regardless of subsequent pruning triggered by a prediction. The average speed of the cognitive knowledge augmented search routine is given by the equation: Oðprediction searchÞ ¼
p p k2 k1 N mð1ÿ2Þ ¼ O N mð1ÿ2Þ 2 ð2Þ
(adapted from [35]), where k2 is the speed of the prediction algorithm and p is the percentage of move predictions made by the IAM prediction algorithm. Both k2 and p are halved since the prediction algorithm only runs at every other ply. Whenever a prediction is made, then the resulting cost is k1 k2 2 , since only a single branch is opened for that level. Since p is greater than or equal to zero and is normally greater than zero, the new prediction-based search is faster than a standard brute force search. The IAM program has been used to model the cognitive patterns of four grandmasters; Kasparov, Karpov, Spassky, and Botvinnik [36], [37]; as well as a new chess master, Seltzer [38]. The number of correct predictions per game for the grandmaster and master chess players ranges from almost 20 percent to just over 46 percent (the 46 percent prediction rate was achieved using IAM for predicting Kasparov moves during one game of his 1990 World Chess Championship match, with average performance at just over 27 percent for Kasparov), with an average prediction rate across all analyzed chess players of just over 25 percent. Plugging the 25 percent prediction rate into (2), the resulting exponent is 0.875. This translates into an improvement over the original search algorithm that previously searched to a depth of 8 ply that now takes the time of a 7 ply search and should be able to achieve a corresponding 9 ply search. Similarly, a search depth of 14 ply is extended to 16 and should a 24 ply search ever be realized, this would be extended to a 27 ply search. To illustrate the potential increase in search efficiency, the IAM prediction algorithm was able to predict seven out of eight moves for Kasparov in the 13th game of the 1990 World Chess Championship. A brute force search to a depth of 16 ply would open approximately 1:96 1023 game tree positions, while the IAM pruned 18 ply search opens only 1:3 1013 game tree positions. Even though this is a 2 ply deeper search than performed by Deep Blue, it can be seen that the cost of 1:98 108 positions per second can well be afforded as it would take an unenhanced search algorithm similar to that of Deep Blue 1 1016 seconds (just over 31 million years) to search all the additional positions required to equal the pruned search space of the IAMenhanced search algorithm.
740
IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING,
VOL. 15,
NO. 3,
MAY/JUNE 2003
Fig. 2. Board position analyzed by IAM, Sargon 4, and GnuChess.
One concern when using a predictive measure to prune the search tree is with incorrect predictions. In the example given above, where seven out of eight positions generated a predicted move, one of the eight predictions was incorrect. In this case, the pruning still occurred and the error was not identified until the opponent makes a move that negates the prediction. Several methods have been discussed [36], [37] for improving the correctness of predictions including the use of the weighted frequency of observation value as a belief value and only accepting predictions with a minimum weight greater than 0.5. Through the use of observation frequency, the percentage of correct predictions will be reduced slightly (by about 10 percent since some correct predictions have smaller frequency), but nearly 100 percent of the incorrect predictions are eliminated. Thus, instead of positively knowing one quarter (25 percent, which equals the average performance of the IAM move prediction method) of all the moves that an opponent will make during a game, the IAM prediction method will only identify 22.5 percent of the moves that will be made by the adversary. Although this produces, on average, fewer predictions, the result is that the predictions are now highly accurate and methods for overcoming incorrect predictions become less significant. Another possibility is to enable a reduced search of multiple paths. Even when the IAM prediction algorithm does make a mistake, it frequently still identifies the correct piece to be moved [36]. Using the correct piece strategy means that for each ply of the opponent, not only is the predicted move expanded for the piece, but also all other moves that can be made by that piece are expanded in the game tree. Although this causes a greater tree expansion, it still significantly prunes the game tree. Of the 30 average moves available for any game turn, on average, less than 10 of these possible moves are made by any piece, which still enables an average search tree reduction of 33.3 percent. The investigation of alternate search pruning strategies based on the cognitive patterns of an opponent is a future research topic.
3.3 Better Search Paths For adversarial domains, the use of domain knowledge can be dangerous by leading us into a false sense of security. Samuel [32] warns that programs using a mini-max algorithm must take into consideration the intent of the adversary. Humans use goal directed tree search with the direction and method of the search changing as new information becomes available [19]. This last statement means that the preceding events in a game and tournament affect the future strategic decisions of the adversary, which is the basic idea behind the IAM cognitive pattern learning and prediction method. An interesting example of how the use of traditional domain knowledge in evaluation algorithms may produce incorrect game tree analysis is given by examining some personal computer chess programs. Fig. 2 shows the board position prior to the 21st move to be made by Botvinnik, during one of his games against Keres in the HagueMoscow Tournament on May 4th and 5th, 1948, and the chunk used in predicting the move. The chunk shown in Fig. 2 is acquired by IAM for Botvinnik through analyzing previous games in the Hague-Moscow Tournament. Two commercial chess programs, selected for their availability and their ability to display their active search trees, were utilized to analyze this position and select the next move to be made. The SARGON 4 program, searching to a depth of 6 ply, selected the Rc1 move and the GNUCHESS program, searching to a depth of 8 ply, selected the Ne5 move for the position shown in Fig. 2. IAM predicted that Botvinnik would move the knight located at d3 to either the b4 or e5 square. Both move choices were predicted with equal likelihood since each move will create the same chunk, thus resulting in a two node expansion at this level of the game tree. Each of the commercial programs, through analysis of their revealed search trees, never considered the Nb4 move in their respective searches. Botvinnik actually made the Nb4 move choice that was one of the two IAM predictions. This example illustrates that the IAM cognitive pattern method is capable of alerting existing chess programs to move choices that are either not considered at all, as above, or not considered to be optimal selections.
WALCZAK: KNOWLEDGE-BASED SEARCH IN COMPETITIVE DOMAINS
741
Fig. 3. Castle chunks used in multiple games by Kasparov, Seltzer, and other grandmasters.
Another example comes from the early days of Deep Blue (Deep Thought at that time), when the chess program first played Kasparov [28]. Apparently, a bug in the evaluation algorithm caused Deep Blue to avoid castling. This general bug may have also caused the program to decrease the likelihood of Kasparov castling as a result of decreasing the value of a castled king in its evaluation algorithm and thereby produced an incorrect evaluation of the game situation. Since castling is a common pattern of pieces, it is normally learned by the IAM pattern learning method and in fact was learned for all five chess players previously studied. Additionally, the castled pattern involves a minimum of two major pieces (the king and a rook as well as others such as a bishop in a fianchetto position) and usually three pawns, which gives it a strong prediction weight. An example of a learned castle pattern for Botvinnik, acquired from three games played in the Hague-Moscow Tournament during 1948 is shown in Fig. 1. From Fig. 1, this specific castle pattern contains six pieces with three of them being major (nonpawn) pieces and, hence, offers a cognitive economy to Botvinnik when employed. Castle patterns are frequently used and known by most experienced chess players. Castle patterns similar to the one shown for Botvinnik in Fig. 1, but captured for Kasparov, Seltzer, and the other chess grandmasters (Karpov and Spassky) are shown in Fig. 3. Use of these IAM cognitive patterns for Kasparov would have alerted Deep Thought to give a much stronger evaluation to the castle move that was eventually made by Kasparov. Other common chess patterns acquired by IAM include various pawn chains, bishop with pawn patterns, and rook with rook or queen patterns.
4
EXTENDING IAM
TO
OTHER DOMAINS
Using the cognitive pattern-based pruning in the domain of chess has been shown to provide increases in search depth and to be able to identify stronger search paths in the domain of chess. As stated earlier in this article, the relative domain independence of the pattern learning and search tree pruning method can lend itself to application in other competitive domains. Preliminary work has been accomplished in learning go patterns [36]. The IAM method applied to the game of go has acquired the common defensive patterns of eyes and other patterns including false eyes, tiger’s mouth, and ladders [37]. Current research is investigating the application of the IAM method to the domain of shogi [39]. Unlike chess, the search tree for shogi expands markedly towards the end-game portion of play which presents new
levels of search difficulty for traditional search based approaches to chess-like games. Thus far, two shogi masters (Habu and Tanigawa) have been modeled using the IAM method [39]. The size and quantity of patterns obtained for these two shogi masters is similar to the size and quantity of patterns obtained for chess grandmasters on a similar number of games, indicating that the cognitive processing abilities of shogi masters are equivalent to chess masters. Following modifications to the IAM-based prediction algorithm to account for the different types of pieces and corresponding piece movements, the shogi master pattern knowledge bases produced an average prediction rate of 25 percent for both of the shogi masters. This indicates that a search tree reduction similar to that achieved in the domain of chess may also be achieved for shogi. Application of the IAM cognitive pattern methodology to other nongame domains requires some additional work. The content of a relevant cognitive chunk must be defined in each new domain. As an example, in the domain of corporate finance, patterns of monetary and stock movements of competitors may be used to identify the likelihood of a corporate takeover or merger. The corporate takeover version of IAM would then monitor financial transactions (including stock purchases) of competitors to identify learned patterns associated with hostile takeovers or mergers. The knowledge patterns learned by the IAM method could potentially enable corporate entities to be better defended or prepared for such adversarial actions. Modifications to the IAM pattern representation algorithm for military domains would involve changing the domain representation to better account for two-dimensional terrain or three-dimensional movement (e.g., air, space, or submarine combat). The square board used in chess may be translated into a hexagonal terrain map representation that would accurately model distance perspectives between units [36] with a corresponding translation in the perceptual pattern acquisition methodology to account for the different shaped representation of the domain. Patterns of troops or equipment can then be learned and used to predict ensuing military tactics. Although the definition of a pattern may change and the rules for applying the pattern may change between domains, the domain independent method of learning patterns based on the previous performance of a potential opponent is widely applicable. Domains that require little redefinition of the pattern learning method include games, military land combat (as explained above), and certain types of law enforcement. Other possible domains that may require large modifications to the definition of acceptable patterns include political negotiation, hostage negotiation
742
IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING,
VOL. 15,
NO. 3,
MAY/JUNE 2003
Fig. 4. Relative increase in prediction coverage as acquired patterns increase.
(pattern of negotiation by the hostage takers), economic and financial domains, and athletics [35] (patterns that precede specific actions by the opposing player/team).
5
SUMMARY
In the previous sections, the idea of using patterns that partially simulate the cognitive processing of domain experts in competitive domains to automatically prune tree searches has been examined. The technique is currently limited to domains that may be considered as competitive since the opponent being modeled via the cognitive patterns must have a high motivation for reuse of the patterns [37]. However, as mentioned previously, many varied domains may easily be perceived as competitive in nature. As the number of cognitive patterns grow (total predictions 7 total moves made), with possible respect to a recentcy effect, the prediction percentage of opponent move choices increases as shown in Fig. 4 (Fig. 4 is a composite of prediction values from the Kasparov and Botvinnik knowledge bases and is consistent with IAM’s other chess pattern knowledge bases). Knowledge bases for four chess grandmasters, one chess master, and two shogi masters (equivalent to a chess grandmaster) have achieved an average 25 percent prediction accuracy (correct predictions 7 total moves made) for all moves in a game. The 25 percent prediction rate yields a 12.5 percent increase in the number of ply that can be searched using the automatic pruning method described. Hence, a chess or shogi program that currently searches to 14 ply may now search to 16 ply. This increase in the search depth of traditional search engines further pushes
back the horizon effect and significantly increases the quality of play for these programs. The patterns in the knowledge base are small enough and focused on a single opponent, thus limiting the quantity, such that they may easily be stored in RAM memory and thereby perform at optimal speeds when compared to traditional opening and end-game books. Another concern for using a knowledge-based automatic pruning approach is that opponents may not always do what is cognitively economical, but may choose to increase the relative (with respect to their pattern knowledge) complexity of the current domain situation. A marginal increase in the expansion of the search tree, but still greatly reducing the overall size of a brute force search, is achieved by examining all moves of the predicted piece to be moved. Additionally, the use of cognitive patterns can reveal search paths that may not be considered by a traditional evaluation algorithm, but would be followed by the opponent. These new search paths produce a more realistic evaluation of the domain situation by “playing the man and not the board,” which in turn may produce better quality play by the game program. With the success of Deep Blue in defeating a world champion chess player, artificial intelligence research will begin to explore new research venues that have significantly larger search trees and different tree dynamics, such as shogi with a search tree of 10226 that expands towards the end-game [27]. Success of artificial intelligence programs in these new domains may well depend on acquiring more knowledge to solve the problem instead of just applying faster technologies.
WALCZAK: KNOWLEDGE-BASED SEARCH IN COMPETITIVE DOMAINS
REFERENCES [1] [2] [3] [4] [5] [6] [7]
[8] [9] [10] [11] [12] [13] [14] [15] [16]
[17] [18]
[19] [20] [21] [22]
[23]
[24] [25] [26] [27]
G.M. Adelson-Velsky, V.L. Arlazarov, and M.V. Donsky, Algorithms for Games. New York: Springer-Verlag, 1988. I. Altho¨fer, “Data Compression Using an Intelligent Generator: The Storage of Chess Games as an Example,” Artificial Intelligence, vol. 52, no. 1, pp. 109-113, 1991. V. Anand, “More Questions Than Answers,” http:// www.research.ibm.com/deepblue/home/may11/story_3.html, 1997. J.R. Anderson, Cognitive Psychology and Its Implications. New York: Freeman, second ed., 1985. A. Barr and E.A. Feigenbaum, The Handbook of Artificial Intelligence. vol. 1, Reading, Mass.: Addison-Wesley, 1981. H. Berliner and C. Ebeling, “Pattern Knowledge and Search: The SUPREM Architecture,” Technical Report CMU-CS-88-109, Carnegie-Mellon Univ., Pittsburgh, Penn., 1988. H. Berliner and G. Goetsch, “A Quantitative Study of Search Methods and the Effect of Constraint Satisfaction,” Technical Report CMU-CS-84-147, Carnegie-Mellon Univ., Pittsburgh, Penn., 1984. D.A. Blum, “Kasparov Versus Deep Blue: The Showdown Between Human and Computer,” PC Artificial Intelligence, vol. 15, no. 1, pp. 49-51, 2001. I. Bratko and D. Michie, “An Advice Program for a Complex Chess Programming Task,” The Computer J., vol. 23, no. 4, pp. 353359, 1980. M.S. Campbell, “Chunking as an Abstraction Mechanism,” PhD thesis, Carnegie-Mellon Univ., Pittsburgh, Penn., 1988. N. Charness, “Expertise in Chess: The Balance Between Knowledge and Search,” Toward A General Theory of Expertise, K.A. Ericsson and J. Smith, eds., pp. 39-63, 1991. W.G. Chase and H.A. Simon, “Perception in Chess,” Cognitive Psychology, vol. 4, no. 1, pp. 55-81, 1973. N.J. Cooke, “Modeling Human Expertise in Expert Systems,” The Psychology of Expertise, Cognitive Research and Empirical Artificial Intelligence, R. Hoffman, ed., pp. 29-60, 1992. K.A. DeJong and A.C. Schultz, “Using Experience-Based Learning in Game Playing,” Proc. Fifth Int’l Conf. Machine Learning, pp. 284290, 1988. A. Elithorn and R. Banerji, Artificial and Human Intelligence. New York: Elsevier, 1984. S.L. Epstein, J. Gelfand, and J. Lesniak, “Pattern-Based Learning and Spatially Oriented Concept Formation in a Multi-Agent, Decision-Making Expert,” Computational Intelligence, vol. 12, no. 1, pp. 199-221, 1996. K.A. Ericsson and J. Smith, “Prospects and Limits of the Empirical Study of Expertise: An Introduction,” Toward A General Theory of Expertise, K.A. Ericsson and J. Smith, eds., pp. 1-38, 1991. K.A. Ericsson and J. Staszewski, “Skilled Memory and Expertise: Mechanisms of Exceptional Performance,” Complex Information Processing The Impact of Herbert A. Simon, D. Klahr and K. Kotovsky, eds., pp. 235-267, 1989. P.W. Frey, Chess Skill in Man and Machine. New York: SpringerVerlag, 1983. R. Grimbergen, “Using Pattern Recognition and Selective Deepening to Solve Tsume Shogi,” Proc. Game Programming Workshop in Japan ’96, pp. 150-159, 1996. A. Horowitz, The World Chess Championship A History. New York: Macmillan, 1973. H. Iida, J.W.H.M. Uiterwijk, H.J. Van den Herik, and I.S. Herschberg, “Potential Applications of Opponent-Model Search, Part 2: Risks and Strategies,” Int’l Computer Chess Assoc. J., vol. 17, no. 1, pp. 10-14, 1994. H. Iida, J.W.H.M. Uiterwijk, H.J. Van den Herik, and I.S. Herschberg, “Thoughts on the Application of Opponent-Model Search,” Advances in Computer Chess, D.F. Beal, ed., vol. 7, pp. 6178, 1995. J. Laird, P. Rosenbloom, and A. Newell, Universal Subgoaling and Chunking. Norwell, Mass: Kluwer Academic, 1986. K. Lee and S. Mahajan, “A Pattern Classification Approach to Evaluation Function Learning,” Artificial Intelligence, vol. 36, no. 1, pp. 1-25, 1988. D.N. Levy, “Chess Master Versus Computer,” Advances in Computer Chess, D.F. Beal, ed., vol. 4, pp. 181-194, 1986. H. Matsubara, H. Iida, and R. Grimbergen, “Natural Developments in Game Research: From Chess to Shogi to Go,” Int’l Computer Chess Assoc. J., vol. 19, no. 2, pp. 103-112, 1996.
743
[28] M. Newborn, Kasparov Versus Deep Blue Computer Chess Comes of Age. New York: Springer, 1997. [29] M. Newborn, “History at the Chess Table,” http:// www.research.ibm.com/deepblue/learn/html/e.8.5.html, 1997. [30] M. Newborn and D. Kopec, “The Twentieth Annual ACM North American Computer Chess Championship,” Comm. ACM, vol. 33, pp. 92-104, 1990. [31] J.R. Quinlan, “Learning Efficient Classification Procedures and Their Application to Chess End Games,” Machine Learning An Artificial Intelligence Approach, R.S. Michalski, ed., pp. 463-482, 1983. [32] A.L. Samuel, “Some Studies in Machine Learning Using the Game of Checkers,” IBM J. Research and Development, vol. 3, no. 3, pp. 210229, 1959. [33] A.L. Samuel, “Some Studies in Machine Learning Using the Game of Checkers,” IBM J. Research and Development, vol. 11, no. 6, pp. 601-617, 1967. [34] H.A. Simon and K. Gilmartin, “A Simulation of Memory for Chess Positions,” Cognitive Psychology, vol. 5, pp. 29-46, 1973. [35] S. Walczak, “Using Inductive Inference of Past Performance to Build Strategic Cognitive Adversary Models,” PhD thesis, Univ. of Florida, Gainesville, Fla., 1990. [36] S. Walczak, “Pattern-Based Tactical Planning,” Int’l J. Pattern Recognition and Artificial Intelligence, vol. 6, no. 5, pp. 955-988, 1992. [37] S. Walczak and D.D. Dankel, “Acquiring Tactical and Strategic Knowledge With a Generalized Method for Chunking of Game Pieces,” Int’l J. Intelligent Systems, vol. 8, no. 2, pp. 249-270, 1993. [38] S. Walczak and P. Fishwick, “A Quantitative Analysis of Pattern Production and Its Relationship to Expert Performance,” J. Experimental and Theoretical Artificial Intelligence, vol. 9, no. 1, pp. 83-101, 1997. [39] S. Walczak and R. Grimbergen, “Pattern Analysis and Analogy in Shogi: Predicting Shogi Moves from Prior Experience,” Knowledge and Information Systems, vol. 2, no. 2, pp. 185-200, 2000. [40] D.A. Waterman, “Generalization Learning Techniques for Automating the Learning of Heuristics,” Artificial Intelligence, vol. 1, nos. 1-2, pp. 121-170, 1970. [41] S. Webb, Chess for Tigers. New York: Pergamon Press, 1986. [42] J.F. White, “The Amateur’s Book-Opening Routine,” Int’l Computer Chess Assoc. J., vol. 13, no. 1, pp. 22-26, 1990. [43] B. Wilcox, “Reflections on Building Two Go Programs,” SIGART Newsletter, vol. 94, pp. 29-43, 1985. [44] D.E. Wilkins, “Using Patterns and Plans in Chess,” Artificial Intelligence, vol. 14, no. 2, pp. 165-203, 1980. [45] D.E. Wilkins, “Using Knowledge to Control Tree Searching,” Artificial Intelligence, vol. 18, no. 1, pp. 1-51, 1982. [46] F.C. Zagare, Game Theory Concepts and Applications. Newbury Park, Calif.: Sage Publications, 1984. Steven Walczak received the PhD degree in computer science (artificial intelligence) from the University of Florida and the MS degree in computer science and BS degree in mathematics from the Johns Hopkins University and the Pennsylvania State University, respectively. He is an associate professor of information systems in the College of Business at the University of Colorado at Denver. Dr. Walczak’s research interests are in the area of applied artificial intelligence systems (including expert systems, neural networks, and intelligent agents) and knowledge management. His research has been previously published in Transactions on Systems, Man, and Cybernetics, Journal of Management Information Systems, Decision Support Systems, and many other journals. He is an active member of the Florida Artificial Intelligence Research Symposium, the Decision Sciences Institute, and has previously been a member of the IEEE Computer Society and the American Association for Artificial Intelligence.
. For more information on this or any other computing topic, please visit our Digital Library at http://computer.org/publications/dlib.
