Optimization in Python for EM ... Evolutionary optimization is based on Evolutionary Algorithms .... Fully Bayesian estimation applied to anisotropic Gaussian.
Machine Learning for Multiobjective Evolutionary Optimization in Python for EM Problems • Anatoliy Boryssenko, AE Partnership, United States • Naftali Hercovici, Raytheon, United States
Talk Outline Objectives: how to create a computational framework in Python using existing Machine Learning and other free, open source libraries. • Computational EM (CEM) and Evolutionary Algorithms • Surrogate, Model-Based Optimization • Python programming language and libraries • Gaussian process regression with scikit-learn • Single objective examples • Dual-objective Pareto examples
Looping CEM in EA CEM: • Computational electromagnetics involves using computationally efficient approximations to Maxwell's equations to calculate EM devices and wave propagation in different media. • Modern CEM tools allow simulating many EM/antenna problems to assist in design EA: • Evolutionary optimization is based on Evolutionary Algorithms (EA) inspired by biological evolution, such as reproduction, mutation, recombination, and selection. • Modern EA are robust tools for global multi-objective tuning
Direct EA-CEM Optimization "Brute-force" optimization with full-wave CEM engines looped inside EA (GA, PSO, DE, ...) search engines is not straightforward:
• Optimization can normally succeed with results to believe over the basis of high fidelity simulations. • High-fidelity (fine mesh) CEM models are CPU time and RAM hungry. • EAs requires too many calls of CEM engines for high fidelity models and become inefficient, ineffective and unaffordable.
Dual-band array unit cell • CEM-EA: HFSS-DE in Matlab • Design parameters: 6+ • Design objective: S11 bandwidth mask not yet reached after 375+ HFSS calls with 25-35 minutes per call
Fitting Cosenant^2 EL Pattern Mask • CEM-EA: HFSS-DE in Matlab • Design parameters: 6+ • Objectives: fitting pattern mask at three frequencies was not yet reached after 185+ HFSS calls with 30-45 minutes per call.
Smart Alternatives? • "Brute-force" approaches with many "expensive" CEM calls could work by using expensive computer hardware like clusters. • The above option is often unaffordable for many companies, engineers, researchers and students. • Cost reduction would be rather better choice for everybody. • Alternative to direct optimization is surrogate modelling that offer fast model predictions which might facilitate such costly analyses. • Nevertheless many efforts, still many challenges bottlenecks hide behind the implementation and applications of these techniques.
Current Status Industry: Many companies seem started proposing new "smart" optimization add-on module for their CEM engines. Proprietary nature of commercial codes exclude deep understanding how those algorithm works and how they are really efficient. That is also substantial extra cost to already $$ expensive tools. Academia: Many ongoing developments including the CEM community.
My Wish List Could Be Assumed that the background theory of surrogate optimization is understood then several practical questions emerge:
• How cold I start experimenting with this? • Do I need to write all codes from scratch? • Could Do-It-Yourself (DIY) be possible here? • How to minimize coding and use as max as possible what already exist?
Surrogate Approach Flowchart
ML Major Algorithms Many different ML algorithms could be mentioned with more under development and to come. • Linear & Logistic Regressions • Gaussian Process Regression (GPR) • Support Vector Machines (SVM) • K-Nearest Neighbors • Random Forests • K-Means Clustering • Principal Components Analysis (PCA)
ML Could be the Answer Machine learning is undoubtedly on the rise driven by Big Data, Data Mining and other similar terms that slowly climbe into ‘buzzword’ territory. This is in large part due to misuse and simple misunderstanding of the topics that come with the term. The chart below illustrate the Google Trends analysis of interest in the term over the last few years.
Python is Better • Has a Healthy, Active and Supportive Community • Is Source-Opened, Accessible, Free, Efficient • is Interactive, Interpreted, Modular, Dynamic • Is Object-oriented, Portable, High level • Has Some Great Corporate Sponsors • Includes Amazing Libraries - Compare Matlab File Exchange depository to Python Package Index (PyPI)
Python Use Growth From Stackoverflow Blog: The Incredible Growth of Python
Python is Chosen by Many What are the Most Disliked Programming Languages?
Python scikit-learn library http://scikit-learn.org scikit-learn examples
Surrogate Approach Flowchart
Gaussian Process Short Math Kriging or Gaussian process regression is a method of interpolation for which the interpolated values are modeled by a Gaussian process governed by prior covariances.
GPR to Build Surrogate Fully Bayesian estimation applied to anisotropic Gaussian Processes (GP).
Image from Google
GPR for Infill - Exploitation min of the surrogate function
Image adapted from Google
GPR for Infill - Exploration max uncertainty of the surrogate function
Other mixed strategies: Probability of Improvement (PI), EI, LCB, ... Images adapted from Google
1D-1D sinc Test Function
sinc 1D/1D low noise video
GPR Approach to Pareto Front
Image from Google
2D/2D Fonseca and Fleming
Resistive Loading Dipole • f1 for S11 and f2 for gain bandwidths, respectivelly • under constraints: f1 ≤ − 10dB and f1 ≥ 1dB
Resistive Loading Dipole • f1 for S11 and f2 for gain bandwidths, respectivelly • under constraints: f1 ≤ − 6dB and f1 ≥ 7dB
3-Element Yagi-Uda Array • f1 for S11 and f2 for gain bandwidths, respectivelly • under constraints: f1 ≤ − 10dB and f1 ≥ 1dB
3-Element Yagi-Uda Array
Software Demo
This is the Last Slide
Bonus Slides
Major Acronyms Acronyms Meaning AI Artificial Intelligence API Application Program Interface DE Differential Evolution EA Evolutionary Algorithm GPR Gaussian Process Regression ML Machine Learning MOEA Multiobjective Evolutionary Algorithm NSGA-II Non-dominated Sorting Genetic Algorithm II PSO Particle Swarm Optimization
Major Vocabulary Terms X Y 1D-1D ND-MD Y=F(X) feval seval
Meaning Variable of design (parameter) space Variable of response space 1-D design and 1-D response dimensions N-D design and M-D response dimensions Function to relate X and Y True (original) function evaluation Surrogate (model-based) evaluation
