Automatic Page Layout Using Genetic Algorithms for Electronic Albuming Joe Geigel1 and Alexander Loui2 Research and Development, Eastman Kodak Company, Rochester, NY 14650-1816 ABSTRACT In this paper, we describe a flexible system for automatic page layout that makes use of genetic algorithms for albuming applications. The system is divided into two modules, a page creator module which is responsible for distributing images amongst various album pages, and an image placement module which positions images on individual pages. Final page layouts are specified in a textual form using XML for printing or viewing over the Internet. The system makes use of genetic algorithms, a class of search and optimization algorithms that are based on the concepts of biological evolution, for generating solutions with fitness based on graphic design preferences supplied by the user. The genetic page layout algorithm has been incorporated into a web-based prototype system for interactive page layout over the Internet. The prototype system is built using client-server architecture and is implemented in Java. The system described in this paper has demonstrated the feasibility of using genetic algorithms for automated page layout in albuming and web-based imaging applications. We believe that the system adequately proves the validity of the concept, providing creative layouts in a reasonable number of iterations. By optimizing the layout parameters of the fitness function, we hope to further improve the quality of the final layout in terms of user preference and computation speed. Keywords: Albuming, genetic algorithms, page layout, image positioning, graphic design
1. INTRODUCTION With the proliferation of digital cameras, low-cost scanners, and Internet imaging applications, the volume of digitized photographs has steadily increased in recent years. Efficient organization of large volume of digital photographs is a challenging task for the consumers as well as professional photographers. In an electronic albuming system, the layout of the final album page is one of the most important functions among other functions such as event clustering, duplicate detection, and image quality screening.1,2 Page layout is the final step of an albuming system that will arrange and layout pictures within an album in a manner pleasing to the consumer. In this paper, we describe a flexible system for generation of album page layouts. The system makes use of genetic algorithms and is designed for distributed systems allowing for printing or viewing over the Internet. The paper is structured as follows: In Section 2, an overview of the page layout problem is given. A brief tutorial on genetic algorithms is presented in Section 3. In Section 4, the application of genetic algorithms to our problem is described, followed by an overview of the prototype system architecture in Section 5, and some results in Section 6. We conclude and discuss future work in Section 7.
2. THE PAGE LAYOUT PROBLEM The role of a page layout system is fairly simple. Given a set of images to be placed in an album, a page layout algorithm must distribute the images amongst a set of pages and then layout the images on each individual page. The work performed by page layout system can be categorized into two separate, yet equally important tasks: (1) Page Creation – Given a set of images, the system must distribute these images amongst a set of album pages. Simply put, each image must be assigned a page upon which the image will appear. 1 2
Work performed while author was at Kodak. He can currently be reached via e-mail at
[email protected] E-Mail:
[email protected]
(2) Image Placement – Once the images have been assigned to pages, each individual page must be laid out by positioning the images assigned to it. Thus, for each image, placement, rotation, and scaling of the image on the page must be determined. There are several software applications, and graphic design tools, that can assist the user in manual creation of photo albums. In general, these applications provide the user with a straightforward means accomplishing the two tasks mentioned above. The drawback with such applications is that they require a large amount of user interaction. In addition, a common approach used by such software is to employ templates. Using this approach, predefined layouts are made available with empty areas that are filled in with user images. A user or system merely needs to specify which images should be placed in which empty area to complete an album page. Templates provide a straightforward and working solution to the layout problem. However, this solution is somewhat limiting as variation in page layout design can only be achieved by the addition of new templates. The real challenge in automatic generation of page layouts is allowing an advanced level of creativity in the layouts produced. With the introduction of scrapbooking as a social activity (as what is promoted by Creative Memories,3) there has been a recent resurgence of interest in capturing memories and telling stories using scrapbooks. The albums produced by people in these communities can range from extremely basic, where images are laid out using a fixed grid or template, to extremely complex, where images are seemingly scattered randomly on a page. With regards to album layout, our goal is to produce albums that more closely resemble scrapbooks as opposed to a simple collection of pictures. Achieving this goal in an automated fashion is a difficult problem, as the means by which creative scrapbookers generate page layouts for their albums is usually not easily expressed. Creation of a scrapbook is primarily a subjective and artistic task. Few, if any, concrete rules are applied in the scrapbook generation process, and those that are tend to be individual, based on personal preferences. The subjective nature of creative page layout poses a real challenge to any page layout system. In particular, template-based layout approaches are somewhat limiting because the range of possibilities for a page layout are bounded by the collection of available templates.
3. GENETIC ALGORITHMS Genetic algorithms4,5 are a class of adaptive methods that can be used to solve search and optimization problems involving large search spaces. The search is performed using the idea of simulated evolution (survival of the fittest). These algorithms maintain and manipulate “generations” of potential solutions or “populations”. With each generation, the best solutions (as determined by a problem specific fitness function) are genetically manipulated to form the solution set for the following generation. Like in real evolution, solutions can be combined (via mating/crossover) or undergo random mutation. In addition, inferior solutions can, by chance, survive from generation to generation. The search process is performed in an iterative manner as illustrated in Fig. 1. First, an initial population of solutions is generated. This initialization is typically done randomly although there is nothing prohibiting in using some sort of heuristic for this process. Once a population is established, individual solutions are judged based on the fitness function and suitable individuals are chosen for mating and mutation. Next, mating and mutation are performed on the chosen solution, resulting in a number of new individuals. These new individuals are generated using the genetic operators of mutation and crossover. A new generation of solutions is then created from these newly generated offspring. The process is iterated until a defined stopping criterion is met. When applying a genetic algorithm to a given problem, three major tasks must be performed: (1) Coding - Genetic algorithms maintain populations of problem solutions. During implementation, these solutions are represented by some data structure (known as a genome). In the coding task, a data structure is chosen to represent the genome for the problem space and a mapping from the data structure fields to the problem domain is established. Common genome data structures used in genetic algorithms includes lists, arrays, and trees.
Initialize population
Select individuals for crossover (based on fitness function
Crossover
Mutation
Insert new offspring into population
Are stopping criteria satisfied?
Finish
Figure 1 - Flowchart for genetic algorithm process
(2)
Definition of Genetic Operators – New solutions are created via crossover and mutation of individuals from previous generations. Given a particular genome structure, the means for performing these operations must be defined. During crossover, one or more children solutions are derived from two or more parents. With mutation, new individuals are generated by mutation of a single solution. There exists standard crossover and mutation operators available for genomes encoded using commonly used data structures like lists, arrays, and trees. Example genetic operations are illustrated in Fig. 2.
(3)
Fitness – The most challenging and application specific task in applying genetic algorithms to a problem domain is in the definition of a fitness function. The fitness function is responsible for judging individual solutions and returning a score based on its evaluation. In essence, the fitness function defines the difference between a good solution to a problem and a bad one. Much care must be taken in defining the fitness function, as the genetic algorithm will converge on solutions deemed “fit” by this function.
4. APPLYING GENETIC ALGORITHMS TO PAGE LAYOUT 4.1. Motivation The idea behind using genetic algorithms for page layout is motivated by a system created by Karl Sims that uses genetic algorithms for creating 2D textures.6 The motivation behind his work was mostly artistic whereby the artist directly determined the fitness of each solution by visual inspection. His system allowed for random exploration of the texture space with solutions converging based on the likes and dislikes of the artist. Page layout has also been found to be more of an artistic task rather than a mechanical one. Genetic algorithms seem appropriate for such artistic tasks since, unlike other more brute force algorithms, the genetic algorithm does not attempt to mimic or model any particular process by which solutions are created. Instead, solutions are generated randomly and evaluated after the fact. This is quite analogous to way creative scrapbookers approach the layout problem. When asked, scrapbookers could not explain the process by which they generated their page layouts; however, they certainly know a good layout when they see it. The layout problem has a multidimensional problem space. Consider the image distribution task mentioned in Section 2. If we represent the placement of an image using 4 values (corresponding to x position, y position, rotation, and scaling), the solution space can ranging from a dimensionality of 4 (in the case of a single image to be placed) to as high as 40 (in the case of 10 images to be placed). Genetic algorithms have been proven successful for problems with similarly large dimensional solution spaces. In particular, a genetic algorithm approach has been successfully applied to the similar problem of standard and macro cell placement in VLSI design.7 Although the goals of image layout and VLSI cell layouts are different, the basic problem, i.e., the placement of shapes on a 2D canvas, is the same. Thus, there is some precedent in using genetic algorithms for this type of problem.
Crossover
Crossover
Mutation
Before Before After
Mutation
Before
Before
After
After
After
Figure 2a - Genetic operators for trees. For crossover a random node is selected for each parent and the subtrees from these chosen nodes are swapped. For mutation, nodes from the same tree are randomly chosen, then swapped.
Figure 2b - Genetic operators for arrays. For crossover a position within the array is randomly selected and the array values after the chosen position are swapped between the 2 parents. For mutation, 2 random array elements are chosen and swapped.
4.2 Genetic Page Layout Engine In Figure 3, an overview of a page layout engine that makes use of genetic algorithms is presented. The engine takes as input a collection of images to be placed in an album. The system assumes that the images are categorized by event, duplicates and duds have been removed from the collection,2 and that each image has been assigned an emphasis value indicating the visual appeal of the image.8 The system produces a series of album pages. Separate modules in the engine perform each of the layout tasks mentioned above. The Page Creator module is responsible for assigning each image to an album page. The page groupings created by this module are passed to the Image Placement module, which positions the images on each individual page. Final page layouts are specified in a textual, XML-based description,9 which are interpreted by a layout compiler that creates images corresponding to each album page. Both the Page Creator and Image Placement module make use genetic algorithms and make use of their own separate genetic engines. Details on the application of genetic algorithms by each of the individual modules are discussed in Sections 4.4 and 4.5.
4.3. Fitness Parameters As with any application that utilizes genetic algorithms, the most important development task involves the definition of the fitness function. In defining the fitness for page layout, it would be ideal to take an approach similar to Sims6 whereby the fitness is determined directly by the user’s visual evaluation. However, in designing the system, one of the goals was to minimize the user input once the images to be placed in the album have been identified. Because of the subjective nature of album evaluation, some indication of the kind of layouts to be produced is required. Page layout is one of the major tasks of those skilled in the art of graphic design. In defining a means for a user to specify album layout preference, we rely on the principles of graphic design. Upon surveying a number of graphic design texts10,11 and engaging in discussions with some graphic designers, we extract a number commonly mentioned principles used in evaluation of layout design:
Album pages
User Images (sorted by event) Page layout engine
Layout Final Compiler Pages (in XML)
User Parameters
Page
Image
Creator
Placement
Genetic Engine
Page Grouping
Page Evaluation
Genetic Engine Layout Evaluation
Figure 3 - Overview of genetic page layout engine architecture
Balance – An equal distribution of weight on the page. This principle refers to the symmetry (or asymmetry) of the page with respect to color, size, shape, and texture. Spacing – Describes the basic layout of images on a page. Spacing parameters can be used to describe the feel of a layout in terms of white space, or randomness. Chronology – Describes whether the placement of images on the page matches with the temporal order in which the pictures were taken. Emphasis – What stands out most gets noticed first. Emphasized elements will be the focal point of a page. Although generally achieved using element size, emphasis can also be maintained by use of color, shape, framing, and texture. Unity – Elements that belong together look like they belong together. Unity is achieved by grouping, repetition, and the use of grids (a subdivision of space into rows, columns, and margins). Each engine module will judge solutions based on a number of different criteria from the categories above. The user’s preference for each criterion will be expected as input to each of the modules. We call these values preference parameters. These preference parameter values can be obtained directly by querying the user. Once established, these preferences can be stored as part of a user profile and extracted whenever a particular user requests an album be made. In addition to preference parameters, the two modules also provide for a set of importance parameters. These importance parameters define how important it is that the system, for a given criterion, obtains a solution where the layout produced evaluates to the exact value of the preference parameter for that criterion. Another way of looking at it is that the importance parameters indicate how much variation the system is allowed with respect to a given criterion. For example, one of the evaluation criteria for the Image Placement module is white space. A preference of 1.0 indicates that a layout with a larger amount of white space is desired, whereas a preference of 0.0 is indicates that a layout with very little white space is preferred. Setting the preference for white space to be 0 with an importance of 1, the system will do everything it can to assure that the layout has as little white space as possible. Note that this is not the same as saying that the importance of white
space is 0. In the above example, white space is very important, it's just that the user wants very little of it. An importance of 0 indicates to the system that it should not even pay attention to the white space preference and, as a result produce a solution with as much or little white space as the system deems appropriate. Finally, each module has a number of system parameters. These parameters are not directly used during the layout evaluation process. Instead, these parameters provide flexibility for the modules ensuring that they can be used by different albuming systems on a variety of output devices. Examples of system parameters include page dimensions, resolution (dpi), minimum and maximum number of pages per album, etc. 4.4. Page Creation The role of the Page Creator module is to place each image onto an album page. Alternately, one can think of this module as being responsible for assigning to each image a page number, with this number corresponding to the page on which the image will be placed. The genome for the page creator module makes use of a tree-based structure that represents the visual grouping of images as presented on a set of album pages. This is illustrated in Fig. 4.
Entire collection (album)
Individual pages
Subgroups within pages
images
Figure 4 - Genome for page creation module
The root of the tree represents the entire photo album. Interior nodes represent a structure of hierarchical visual groupings of images, which, in turn, are represented by the leaf nodes.3 The first layer of the tree indicates the partitioning of the album into pages. Nodes below the first layer represent visual groupings within pages. The notion of this hierarchical visual grouping is illustrated in Fig. 5. This figure shows the encoding of an album with two pages and a possible layout solution that maintains the visual grouping relationships. Standard crossover and mutation operators for tree structures, as illustrated in Fig. 2a, are used by the Page Creator module. Note that the crossover operator may result in a solution that contains images appearing in multiple places within the tree or in solutions where particular images are omitted. These types of offspring are unacceptable and will be assigned a score of 0.0 during evaluation. Because of this, the probability of performing a crossover has been lowered thus favoring mutation over crossover when constructing the individuals of future generations. The determination of fitness used by the page creator module is a combination of a number of factors. First, a solution is evaluated and scored based on a number of different criteria. For each criterion, the score achieved by the solution is compared to the preference of the user as defined by the preference parameters supplied to the module. This comparison results in a score indicating the suitability of the solution given the preferences of the user. Finally, the final fitness is 3
This encoding is inspired by the space-filling algorithm presented in.12 A tree genome is also employed in7 while applying GAs to VLSI macro cell layout. The original intention was that the image distribution module would use visual subgroups to aid in laying out the images on individual pages. Subsequently, it was found that the image placement module could operate effectively without consideration of visual subgroups. Thus, it may be advantageous to replace this tree based encoding with a simpler data structure based on arrays.
obtained by scaling these suitability values on a criterion by criteria basis using the importance parameters, also supplied by the user.
Page 1
album
Page 1
Page 2
Page 2
Encoding
Possible Layouts
Figure 5 - Encoding the genome for page creation module
Criteria used by the page creator module include: Balance The balance score is a measure of whether there are an equal number of images on each page. A solution with an equal number of images on all of its pages will obtain a perfect balance score of 1.0. Emphasis In considering emphasis, the evaluation method attempts to judge how equally distributed the emphasis is amongst the pages in the album. It is undesirable to have too much emphasis on a single page, because this will limit the capability of the image placement module to adequately emphasize the images that have high emphasis values. At the same time, a page with too little emphasis on a page will force the image placement module to emphasize images that do not properly deserve emphasis. Chronology In evaluating chronology, each solution is judged based on how closely the order of the images as presented on the pages match that of the chronology of the images. Unity Evaluation of unity is performed by considering the images belonging to events and subevents2 and their proximity to each other when constructing the pages for the album. Two types of scores are computed when evaluating unity: •
Page Unity – Each page is evaluated to determine the percentage of images that belong to the same event that appear on the page. If all the images on a page belong to the same event, the page unity score is 1.0. If several events share the same page, a penalty based on the fraction of events of which images appear on the page is applied.
•
Event Unity – Each event is evaluated to determine the percentage of images in the same event that fall on the same or subsequent pages. For each page on which images of a particular event appears, the fraction of images from that page that belong to that event is determined. If an event totally dominates a page, a score of 1.0 is added to the total Event Unity score. Otherwise, the fraction of images on that page not belonging to the event in question is applied as a penalty and subtracted from the total Event Unity Score. It is acceptable for an event to scan multiple pages. For those events that do, a bonus is applied when the pages on which the event images appear are subsequent. A penalty, based on the distance between the pages on which the images appear, is applied if this is not the case.
4.5. Image Placement The role of the Image Placement module is to position images on individual album pages. For each image to be laid out on a page, its placement can be given by the 4-tuple (x, y, s, θ) where: x is the x coordinate of the center of the image, y is the y coordinate of the center of the image, s is a scaling factor4, and θ is the amount of rotation about the center vertical axis of the image. A floating-point array is used as the genome for the image placement module. As illustrated in Fig. 6, each element of the array corresponds to the individual member of the 4-tuple defined above. The complete genome is comprised of the 4-tuples of all images to be placed. Thus, for a page with N images, the length of the genome is 4 * N.
Image 1
Image n
. . . Rotation (θ) Scaling (s) Position (y) Position (x)
Figure 6 - Genome encoding for image placement solution
All elements of the array are floating point values ranging from 0 to 1. This is to assure that all genes (i.e., array elements) are considered equal when performing genetic operations. Gene values are mapped to actual placement coordinates, scaling, and rotation values before evaluation of a particular layout solution. The Image Placement module, as illustrated in Fig. 2b, uses standard crossover and mutation operators for arrays. An additional mutation operator, which chooses a random gene within the array and deviate its value by a random amount, is also used alternately by the module. Care is taken to assure that gene values remain between 0 and 1 after the mutation. Similar to the page creator module, the fitness for the image placement module is a combination of raw evaluations in a number of given criteria considered with respect to the preference and importance parameters supplied by the user. Seven different criteria are used in evaluating a page layout. These criteria can be categorized as follows. Spatial Criteria • White space - The white space score is a measure of the amount of white space on the page and is determined by calculating the percentage of the page area not filled by any images. • Overlap - In evaluating overlap, the maximum overlap between any two images is considered. The overlap score is defined as the maximum percentage of any image area covered by another image over all of the images placed on the page. • Rotation - The rotation score is an indication of the total amount of image rotation on an album page. It is derived by averaging the absolute values of the image rotations over all of the images.
4
Note that the aspect ratios of the images are preserved. Thus, equal scaling is performed in both the horizontal and vertical directions.
Balance Criteria • Spatial Balance - Spatial balance is evaluated by comparing the image areas in the four quadrants of the album page. The page is first split horizontally into equal halves. The amount of page area containing images for both halves are determined and the ratio of the half with the smaller image area over the half with the larger image area are calculated. For a spatially balanced layout, this ratio should be close to 1.0. The same procedure is performed in the vertical direction. The final spatial balance score is the average of the two ratios. • Rotational Balance - The score for rotational balance is calculated in a manner similar to that of rotation. Like with the rotation evaluation, the rotation values over all the images are averaged. However, unlike the evaluation for rotation, the actual rotation values and not the absolute values of these rotations are considered when calculating the average. The rational behind this evaluation is that for a rotationally balanced layout, the summation of all the rotations should amount to 0. • Border Symmetry - The border symmetry score evaluates how closely the edges of the image on the page form a natural boundary. For each edge of the album page (top, bottom, left, right), a border distance is determined by considering the image endpoint closest to the page edge and calculating the distance from the edge to the endpoint. The mean distance amongst the four edges is determined and the average distance from the mean amongst the four edges is calculated. Emphasis - The emphasis score measures the proportionality of the size of the images with respect to the emphasis values assigned to the images. The rationale behind this evaluation stems from the notion that images with large emphasis values should take up more space on the page. The comparison made during evaluation is relative to the sizes of all of the images. For each image, the size relative to the largest image in the group is calculated and expressed as a percentage. This percentage is then subtracted from the emphasis value assigned to the image. Then mean difference amongst all the images on the page is calculated and this average is subtracted from 1.0, assuring that an emphasis score of 1.0 indicates a strong positive correlation between image size and emphasis values.
5. SYSTEM PROTOTYPE The genetic layout algorithm has been incorporated into a web-based prototype system for interactive page layout over the Internet. The demo is built using a client-server architecture illustrated in Fig. 7. It is assumed that the images to be placed in an album have been previously downloaded to the server. Preference and importance parameter values are supplied by the user at the client via a Java-enabled Web browser. Once received by the server, the page layout is created on the server side using a page layout engine implementing the algorithm presented above. Communication components for both the client and server are written in Java 1.2.2. The layout engine is written in C++ (using13 for genetic programming). On the client side, an applet is used to communicate requests from the user. A message center, resident on the server side, receives these requests and commands the layout engine appropriately. Once the server has completed a page layout, a JPEG file containing the results of the algorithm is created using the XML description of the layout, and placed in a common file area on the server. This file is retrieved by the client using a standard HTTP request. Once a satisfactory layout has been created, this same, device independent XML layout specification can be used to drive a printer thus creating a hard copy of the layout.
Datastore
HTTP Server
Java Applet
Client (Web Browser)
Internet
Layout Engine
Layout Compiler XML Layout Desc.
Layout Server
Figure 7 – System prototype architecture
JPEG
Layout Compiler
To printer
A simple messaging protocol was developed for the prototype consisting of 5 different requests to which the server can respond: (1)
START – This message initiates the initial connection between a client and the server. If the server is not idle (i.e., is currently serving another client), the server will send back a busy message. Otherwise, it will record a job ID (which is generated by the client) and IP address of the source of the incoming request and return an acknowledgement.
(2)
PARAMS – This message contains the parameter set to be used for the layout. This message also causes the server message center to have the layout engine initiate the layout creation process.
(3)
PROGRESS_REQUEST – This message is a request for the current progress of the page layout process. The response of this message is a number from 0–100, indicating the percentage of the layout creation process completed. 100 will be sent back if the process is complete.
(4)
SCORE_REQUEST – This is a request for the fitness score of a completed layout. The score is calculated by the layout generator app returned to the server message. This request should only come after a layout is complete.
(5)
FINISH – This message indicates that the client is done with the server. When received, the server will sever the connection with the client and mark it as idle.
6. RESULTS 6.1. Page Creation Module To test the page creation module, the algorithm was run using a test set derived from actual user images. The set represents a typical consumer role of film consisting of 40 images taken from a variety of different events. In these tests, it took approximately 10,000 generations to reach convergence, producing solutions with scores ranging from 0.75–1.0. The computation time required generating a solution averaged between 5–10 min..5 When the preferences for each criterion were considered individually, the module produced expected results. It was found that a high unity and chronology preferences result in solutions that tend to lump images together on the same page, whereas high emphasis and balance preferences tend to favor solutions where images are more spread out amongst pages. An interesting compromise is reached when combining the effects of two criteria. In Table 1, the results of a page creation run with equal preference of emphasis and unity is presented. The page set for this example consisted of pictures from six different events (labeled A, B, C, D, E, F). Images in the table are presented as bracketed couplets labeled with the event to which it belongs and a value between 0.0–1.0 indicating the amount of emphasis the image should receive. Note that images belonging to the same event tend to remain together. At the same time, the introduction of emphasis eliminates the tendency to combine a large number of images onto one page. Page 1: [A 0.5] Page 2: [A 0.4] [A 0.33] Page 3: [B 0.6] [B 0.46] Page 4: [B 0.6] Page 5: [B 0.54] [B 0.56] Page 6: [A 0.9] Page 7: [C 0.8]
Page 8: [C 0.34] [C 0.58] [C 0.28] Page 9: [C 0.43] [C 0.55] [C 0.58] Page 10: [B 0.49][B 0.49][B 0.45] [B 0.42] Page 11: [D 0.49] [D 0.36] [D 0.28] Page 12: [D 0.5] [D 0.54] Page 13: [D 0.27] [D 0.5] [D 0.5] Page 14: [C 0.67] [C 0.34] [C 0.28]
Page 15: [C 0.75] [C 0.4] Page 16: [E 0.55] [E 0.52] Page 17: [E 0.62] Page 18: [E 0.76] Page 19: [E 0.7] Page 20: [E 0.9] Page 21: [F 0.4]
Table 1 – Page creator example (equal emphasis and unity preferences)
5
Timing values obtained running un-optimized code on a 450 MHz Pentium II PC running Microsoft Windows NT.
6.2. Image Distribution Module The image distribution module was tested using image sets with number of images ranging from 1–10 per set. Each set represents a group of images that are to be distributed on a single page. In these tests, the algorithm was run for 300 generations. The creation of the final solutions took, on the average, 1–3 min to compute. The convergence rate of the algorithm was quite good, with final solutions obtaining an average fitness of 0.8–1.0. In Fig. 8, a sampling of layouts produced by the image distribution module, for a page containing 4 images is given. The effects of the choice of preference for white space (Fig. 8a), overlap (Fig. 8b), and spatial balance (Fig. 8c) are illustrated. The variety of page layouts that can be produced by the module is best illustrated when modifying several preference parameters at once. In Fig. 8d, the preferences for white space, rotation, and overlap are all modified between the two layouts. The result is two layouts with two entirely different feels to them, the first being very traditional and rigid, the second having more of a whimsical quality to it.
Preference = 0.0
Preference = 1.0
(a) Effects of white space preference
Preference = 0.0
Preference = 1.0
(c) Effects of spatial balance preference
Preference = 0.0
Preference = 1.0
(b) Effects of overlap preference
Rotation = 0.0 Overlap = 0.0 White Space = 0.8
Rotation = 1.0 Overlap = 1.0 White Space = 0.1
(d) Varying preferences of several criteria
Figure 8 – Image distribution module examples
7. CONCLUSIONS AND FUTURE WORK In this paper, a flexible system for automated album page layout is described. The layout process is divided into two subtasks, page creation whereby images are distributed amongst a set of pages, and image placement whereby images are positioned on individual pages. The system makes use of genetic algorithms for generating solutions for both of these subtasks with fitness based on graphic design preferences supplied by the user. The use of genetic algorithms has resulted in a variety of creative layouts, many difficult to achieve using standard albuming templates.
The system described in this paper has demonstrated the feasibility of using genetic algorithms for automated page layout in albuming and web-based imaging applications. We believe that the system adequately proves the validity of the concept, providing creative layouts in a reasonable number of iterations. By optimizing the layout parameters of the fitness function, we hope to further improve the quality of the final layout in terms of user preference and computation speed. Having shown the advantage of this approach, there are many future enhancements from which the system can benefit. The system certainly generates a number of interesting and creative solutions to the layout problem. However, none of these solutions have been judged by users nor has the relationship between the setting of preference parameters and the layouts produced been investigated. An obvious next step in integration of these techniques into an actual albuming system would be to conduct a formal human factors study whereby the validity of the preference and importance parameters would be tested. The study might investigate not only whether the layouts produced given a set of layout and importance parameters meet users expectations, but also test whether specifying a layout using these parameters is intuitive or not. In addition, user feedback can be used to develop new criteria for evaluation and thus further refine the fitness functions. Even though the Sims approach6 in defining genetic fitness does not meet with the goals of the original system, having the user visually evaluate layout solutions is still a very desirable paradigm to pursue. Querying the user will not only provide immediate feedback, but this feedback can also be used as input to a learning algorithm thus providing a means of learning user preferences with regards to layout. During our implementation of the system, defaults defined by the genetic algorithm library employed13 were used in controlling the basic genetic algorithm. The library however, does allow for the specification of genetic parameters such as number of individuals per generation, number of generations to run, and probabilities of crossover and mutation. A thorough analysis of these parameters with respect to our layout application should be performed in order to ascertain the most appropriate values to use for our particular problem and encoding schemes. It is expected that choosing the right values for these parameters will greatly improve the efficiency of the system. Finally, with the exception of the use of the image appeal (emphasis) value, the system produces layouts by considering only the geometry of the images and does not account for the actual content of the image. The addition of further fitness evaluation criteria that considers what’s in the picture is an area for future investigation.
REFERENCES 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13.
A. Loui and M.D. Wood, “A software system for automatic albuming of consumer pictures”, Proceedings of ACM Multimedia ’99, Orlando, FL November 1999. A. Loui and A. Savakis, “Automatic image event segmentation and quality screening for albuming applications,” Proceedings IEEE International Conference on Multimedia and Expo, New York, NY, July 30-Aug. 2, 2000. Creative Memories, http://www.creative-memories.com/ J. Holland, Adaptation in Natural and Artificial Systems. The University of Michigan Press, Ann Arbor, 1975. D. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning, Addison-Wesley, 1989. K. Sims, “Artificial evolution for computer graphics”, Proceedings of SIGGRAPH ‘ 91, pp 319—328. P. Mazumder and E. M. Rudnick, Genetic Algorithms for VLSI Design, Layout & Test Automation, Prentice-Hall, 1999. S. Etz, A. Savakis, and A. Loui, “Evaluation of image appeal in consumer photography,” Proceedings SPIE Electronic Imaging 2000, San Jose, CA, January 2000. World Wide Web Consortium, Extensible Markup Language (XML) 1.0, http://www.w3.org/xml, February 1998. A. Arntson, Graphic Design Basics, 3rd Ed., Harcourt Brace College Publishers, Fort Worth, 1998. L. Siebert and L. Ballard, Making a Good Layout, Northern Light Books, Cincinnati, OH, 1992. B. Shneiderman, “Tree visualization with tree-maps: 2-d space filling approach”, ACM Transactions on Graphics, Vo1 11, No. 1, January 1992, pp 92-99. M. Wall, “GALib: A C++ library of genetic algorithm components”, Massachusetts Institute of Technology, Mechanical Engineering Department, http://lancet.mit.edu/ga/, April 1996.
