Learning and Intelligent Optimization for ...

Learning and Intelligent Optimization for Computational Materials Design Innovation 1

2

Amir Mosavi , and Timon Rabczuk 1

2

Institute of Automation, Obuda University, Budapest, Hungary Institute of Structural Mechanics, Bauhaus University Weimar, Weimar, Germany

Abstract. Material design is crucial for the long-lasting success of any technological sector and industry. Computational materials design innovation is an emerging area of materials science. It is a rapidly evolving field of challenges and opportunities aiming at development and application of multiscale methods to predict and simulate innovative materials with high accuracy. Yet it requires an adaptive solver to model a wide range of materials design problems. This research contributes to the development of a computational toolbox for the virtual design and simulation of innovative materials. A toolbox for simulation-based

optimization

of

advanced

materials

is

introduced to model, simulate, and predict the fundamental properties and behavior of multi-scale materials. The proposed computational toolbox is a simple yet powerful concept presenting an integration of advanced machine learning and intelligent optimization techniques. Keywords: machine learning · optimization · materials design

1

Introduction

Infrastructure, logistics, energy, safety, and further technologies of high public interest require advanced customized materials. In fact materials design is crucial for the long-lasting success of any technological sector, and yet every technology is founded upon a particular materials design set. This is why the pressure on development of new high-performance materials for use as high-tech structural and functional components becoming greater than ever. Although the demand for materials is endlessly growing, experimental materials design is attached to high costs and time-consuming procedures of synthesis. This is why simulation technologies have become completely essential for materials design innovation [19]. Naturally the research community highly supports the advancement of simulation technologies as it represents a massive platform for further development of scientific methods and techniques in all branches of material design innovations. Yet computational materials design innovation is a new paradigm in which the usual route of materials selection and prototyping is enhanced by concurrent materials design simulations and computational applications. Designing new materials is a multi-dimensional problem where multiple criteria of design need to be satisfied. Consequently materials design innovation would require advanced optimization and decision-support tools. In addition the performance and behavior of a new material must be predicted in different environments and conditions. This is why prediction and optimization are the essential requirements of a reliable simulation system for material design innovation [6]. In fact predictive analytics and optimization algorithms are the essential computation tools to tailor the atomic-scale structures and chemical compositions and microstrutures of materials for desired mechanical and transport properties such as high-strength, hightoughness, high thermal and ionic conductivity, high irradiation and corrosion

resistance [7]. Via manipulating the atomic-scale dislocation, phase transformation, diffusion, and soft vibrational modes the material behavior on plasticity, fracture, thermal, and mass transport at the macroscopic level can be predicted and optimized accurately [19]. The framework of a predictive simulation-based optimization of advanced materials, which yet to be realized, represents a central challenge within material simulation technology [2]. Consequently material design innovation is facing the ever-growing need to provide a computational toolbox that allows the development of tailormade molecules and materials through the optimization of material behavior predicted in large-scale simulations [8]. The goal of such toolbox within the context of computational materials science is to provide insights over the properties of materials and phenomena associated with their design, synthesis, processing, characterization, and utilization [20].

2

Computational Materials Design Innovation

Computational materials design innovation aims at development and application of multiscale methods to simulate advanced materials with high accuracy [25]. A key to meet the ever-ongoing demand on increasing performance, quality, specialization, and price reduction of materials is the availability of simulation tools which are accurate enough to predict and optimize novel materials on a low computation cost. A major challenge one faces when developing such tools is the hierarchical nature inherent to all materials. Accordingly to understand a material property on a given length and time scale it is crucial to optimize and predict the mechanisms on shorter length and time scales all the way down to the most fundamental mechanisms describing the chemical bond. Consequently the materials systems are to be studied under consideration of their underlying nano-structures as well as their exposure to extreme environmental conditions. Such design process is highly

complex and requires international collaboration and data sharing as well as interdisciplinary contributions [1]. This is why the pioneer research institutes have evolved from isolated numerical approaches to an integrative systems science via developing open-source advanced software toolboxes to help the scientific community collaborate, share information, model, simulate and predict the fundamental properties and behavior of nanoscale and mesoscale materials [6]. As the result of such collaborations scientists have become closer to the idea of combining advanced thermodynamic and electronicstructure methods. 2.1

Interdisciplinary Research

Thermodynamic and electronic structure calculations of materials [21], systematic storage of the information in database repositories [6], materials characterization and selection, and gaining new physical insights [9] account for big data technologies. In addition making decision for the optimal materials design needs optimization tools as well as an efficient decision support system for post-processing [22]. This is considered as a design optimization process of the microstructure of materials with respect to desired properties and functionalities. Such process requires a smart agent which learns from dataset and makes optimal decisions. The solution of this inverse problem with the support of the virtual test laboratories and knowledge-based design would be the foundation of tailor-made molecules and materials toolbox. With such an integrated toolbox at hand the virtual testing concept and application is realized. This challenging task can only be accomplished through a variety of scale bridging methods which requires machine learning and optimization combined [4]. In fact developing a toolbox for materials design innovations requires a great deal of understanding about big database technologies, prediction, and optimization algorithms. Predicting the overall

microstructure behavior of existing materials, as well as the ability to test the behavior of new materials at the atomic, microscopic and mesoscopic scale is desired which leads to virtually modify the materials properties [7]. Virtual applications and virtual test laboratories further allows efficient experiments with entirely new materials and molecules [21]. Basic machine learning technologies such as artificial neural networks [23], and genetic algorithms [10], Bayesian probabilities and machine learning [8], data mining of spectral decompositions [6], refinement and optimization by cluster expansion [21], structure map analysis and neural networks [4], and support vector machines [24], have been recently used for this purpose. 2.2

Research Gap

Computational materials design innovation to perfect needs to dramatically improve and put crucial components in place. To be precise, data science, data mining tools, efficient codes, big data technologies, advanced machine learning techniques, intelligent and interactive optimization, open and distributed networks of repositories, fast and effective descriptors, and strategies to transfer knowledge to practical implementations are the research gaps to be addressed [6]. In fact the current toolboxes rely only on a single algorithm and address limited scales of the design problems. In addition there is a lack of a reliable visualization tool to better involve engineers in the design loop [12]. Considering further research gap; the absence of robust design systems, lack of the post processing tool for multicriteria decisionmaking, lack of big data tools for an effective consideration of huge materials database are a number to mention. To conclude, the process of computational materials design innovation requires a set of up-to-date solvers to cover a wide ranges/scales of problems. Furthermore an advanced database would require cloud computing, big data technologies, learning from data

capabilities and a powerful set of intelligent optimization tools which are currently missing [24]. The other problem with current open-source software toolboxes is that the modeling solution is not flexible as it cannot be configured according to the problem at hand. Instead an adaptive solver that can be configured and set to rule a wide range of design problems is desired. Nevertheless, as the software toolboxes require concrete specification on the mathematical model before starting the solution process, this has been a reason why computation tools for materials design have not been effective. In fact a solver rather requires to update the solution model. Consequently the vision of this work is to develop a fully automated toolbox, where the solver determines the optimal choices via computational power available in cloud environments [5]. Ultimately the vision is to construct a knowledge-based virtual test laboratory to optimize the hybrid materials microstructure systems at either atomic, microscopic and mesoscopic scales. Whether building atomistic, continuum mechanics or multiscale models, the open-source toolbox can provide a platform to rearrange the appropriate solver according to the problem at hand. Such platform contributes in advancement of innovative materials database and leads to design innovative materials with the optimal functionality for a wider range of applications.

3

Learning and Intelligent Optimization as a Solver

The increasing availability of huge amounts of data in computational materials design from sources as diverse as database of atomictic, quantum mechanical, discrete and continuum modelling, multiscale experiments, chemical compounds, structures, and digital libraries, represents a great challenge for computing systems. Big data is produced and it is continuously expanding. Such a complex body of information asks for the most recent advances in machine learning to scale to the big data and complexity to direct

prediction for an optimal design [5]. In fact materials design can be seen as a high potential research area and a continuous source of challenging problems for machine learning. Accordingly the vision would be an automated system so that only data and desired outputs need to be provided where machine learning is used for predicting the multiscale 3D materials structures. Furthermore machine learning is used to combine heterogeneous and noisy sources of information from evolutionary, similarity and experimental data in order to contribute in discovering relational structures. In the proposed toolbox every individual design task, according to the problem at hand, can be modeled on the basis of the solvers within the toolbox. In the other words concentration on the current design model is essential as the design objectives in different scenarios differs. To obtain a design model the methodology does not ask to specify a model, but it experiments with the current system. The appropriate model is created in the toolbox and further is used to identify a better solution in a learning cycle. The methodology is based on transferring data to knowledge to optimal decisions through machine learning and intelligent optimization [5]: a workflow that is referred to as prescriptive analytics [3]. In addition an efficient big data technology [13] is used to build models and extract knowledge. Consequently a large database containing the thermodynamic and electronic properties of existing and hypothetical materials is interrogated in the search of materials with the desired properties. Knowledge exploits to automate the discovery of improving solutions i.e. connecting insight to decisions and actions [18]. As the result a massively parallelized multiscale materials modeling tools that expand atomisticsimulation-based predictive capability is established which leads to rational design of a variety of innovative materials and applications. A variety of solvers integrated within the toolbox include several algorithms for data mining, machine learning, and predictive analytics which are tuned by cross-validation. These solvers provide the ability of learning

from data, and are empowered by Reactive Search optimization [3]; the intelligent optimization tool that is integrated into the toolbox. The proposed machine learning and intelligent optimization toolbox fosters research and development in intelligent optimization and Reactive Search [5]. Reactive Search advocates the integration of sub-symbolic machine learning techniques into local search heuristics for solving complex optimization problems. The word reactive hints at a ready response to events during the search through an internal online feedback loop for the self-tuning of critical parameters [3,12]. In fact Reactive Search is the effective building block for solving complex discrete and continuous optimization problems which can cure local minima traps. Further, Cooperating Reactive Search [5] coordinates a collection of interacting solvers through an organized subdivision of the configuration space which is adapted in an online manner to the characteristics of the problem. And the development of such framework is guided by a number of design principles i.e., general-purpose optimization, global optimization, multi-scale search, adaptation, and tunable precision. Recently such implementation has been used for a number of problems e.g. material selection [17], geometrical [16], construction [20], welded bean design [14], and Robotics [18].

4

Conclusions

Computational materials design innovation is an emerging area of materials science aiming at development of multiscale methods to predict and simulate innovative materials. Yet it requires an adaptive solver to rule a wide range of materials design problems. Learning and intelligent optimization proposes a suitable platform for developing a computational toolbox for the virtual design and predictive simulation-based optimization of advanced materials to model, simulate, and predict the fundamental properties and behavior of

multiscale materials. The proposed computational toolbox is a simple yet powerful concept presenting an integration of advanced machine learning and intelligent

optimization

techniques.

With

a

strong

interdisciplinary

background this research connects computer science and engineering, and further strengthens the research direction of digital engineering.

References 1. Artrith, N.H., Alexander, U.: An implementation of artificial neuralnetwork potentials for atomistic materials simulations. Computational Materials Science 114: 135-150 (2016) 2. Bayer, F.A., et al.: Robust economic Model Predictive Control using stochastic information. Automatica 74: 151-161 (2016) 3. Battiti, R., Brunato, M., Mascia. F.: Reactive search and intelligent optimization. Springer Science & Business Media 45 (2008) 4. Bhadeshia, H., Dimitriu, R.C., Forsik, S., Pak, J.H., Ryu, J.H.: Performance of neural networks in materials science. Mater. Sci. Technol.25, 504–510 (2009) 5. Brunato, M., Battiti, R.: Learning and intelligent optimization: one ring to rule them all. Proceedings of the VLDB Endowment 6, no. 11: 1176-1177 (2013) 6. Curtarolo, S., et al.: The high-throughput highway to computational materials design. Nature materials 12.3: 191-201 (2013) 7. Ceder, G.: Opportunities and challenges for first-principles materials design andapplications to Li battery materials. Mater. Res. Soc. Bull. 35, 693–701 (2010) 8. Fischer, C.C., et al.: Predicting crystalstructure by merging data mining with quantum mechanics. Nature Mater 5, 641–646 (2006)

9. Jain, A., et al.: A high-throughput infrastructure for density functional theory calculations. Comp. Mater. Sci. 50, 2295–2310 (2011) 10. Johannesson, G.H. et al. Combined electronic structure and evolutionary search approach to materials design. Phys. Rev. Lett. 88, 255506 (2002) 11. Lencer, D., et al.: A map for phase-change materials. Nature Materials 7, 972-977 (2008) 12. Mosavi, A.: Decision-making software architecture; the visualization and data mining assisted approach. International Journal of Information and Computer Science (2013) 13. Manyika, J. et al.: Big data. Competition and Productivity, McKinsey Institute. (2011) 14. Mosavi, A., Vaezipour, A.: Reactive Search Optimization; Application to Multiobjective Optimization Problems. Applied Mathematics 3, no. 10 1572: (2012) 15. Mosavi, A.: A multicriteria decision making environment for engineering design and production decision-making. Int. Journal of Computer Applications 69, no. 1 (2013) 16. Mosavi, A.: Decision-Making in Complicated Geometrical Problems. International Journal of computer Applications 87, no. 19 (2014) 17. Mosavi, et al.: Reconsidering the multiple criteria decision making problems of construction workers with the aid of grapheur. ANSYS and EnginSoft Conference (2011) 18. Mosavi, A., Varkonyi-Koczy, A.R.: Integration of Machine Learning and Optimization for Robot Learning. Intelligent Systems and Computing, 349-355 (2017) 19. McDowell, D.L., Backman, D.: Simulation-Assisted Design and Accelerated

Insertion

of

Materials,

Computational

Microstructure-Property Relationships. (2010)

Methods

for

20. Mosavi, et al.: Multiple criteria decision making integrated with mechanical modeling of draping for material selection of textile composites. Composite Materials (2012) 21. Setyawan,

W.,

Curtarolo,

S.:

High-throughput

electronic

band

structurecalculations: challenges and tools. Comp. Mater. Sci. 49, 299– 312 (2010) 22. Saito, T.: Computational materials design. 34. Springer Science & Business Media, (2013) 23. Stucke, D. P., Crespi, V.H.: Predictions of new crystalline states for assemblies of nanoparticles. Nano Lett. 3, 1183–1186 (2003) 24. Sumpter,

B.G.,

Noid,

D.W.:

On

the

design,

analysis,

and

characterizationof materials using computational neural networks. Annu. Rev. Mater. Sci.26, 223–277 (1996) 25. Xiong, L., et al.: Prediction of phonon properties of 1D polyatomic systems using concurrent atomistic-continuum simulation, Applied Mechanics, 84: 1665 (2014)