Why Should We Develop Simulation Models in Pairs? Bernard P. Zeigler Professor Emeritus, University of Arizona Chief Scientist, RTSync Corp.
[email protected] 50th Anniversary Titans Talk Winter Simulation Conference
December 2017
Outline • The conventional approach to model construction for simulation: focus on a single model follow a more or less structured development cycle.
• Motivation Why put in twice the time and effort to develop two models rather than one? Short Answer: extra effort at the beginning may payoff in the end Longer Answer: Will show how the pair-of-models approach leads to be better results…
• Principles of Valid Simplification Theory behind the approach Multiresolution Model Families
• Constructive Agenda Software tools to implement the theory
• Details and Examples • Take Aways
Motivation • Why build target and simpler models – it’s hard enough to build one? • Because: The Simpler model can give more easily attained insight be more general be used to test the target (more complex model) proxy for the target explain the target’s behavior
• Usually under restricted conditions: what are they?
Principles of Valid Simplification • Conditions required for valid aggregation (lumping) Uniform structure within blocks Indifference of block interaction to identity of block members
• Mappings (“morphisms”) express lumping under strict conditions Systems formalize simulation models System morphisms express mappings and their preservation properties
• Approximate morphisms allow relaxation of conditions • Software tools computationally check morphisms
Constructive Agenda • Develop principles for constructing base-lumped model pairs • Develop software tools for testing morphisms especially approximate morphisms
• Tools should be broadly applicable • Goal: Reduce additional effort for pair-of-models approach
Pair-of-Models…Multi-resolution Model Family Embed in Multi-resolution hierarchy Base Model
Pair-of-Models Base Model
Lumped Model
Lumped Model
Base Model
Lumped Model
Insert as component
Examples • Brain simulation scale modeling Pair-of-models approach to impossibility of full scale simulation
• Combat modeling attrition multiresolution Illustrates concepts and tools for pairs-of-models development Avoid ontological mismatch
• Multiprocessor Interconnect effect on speedup DEVS Markov models & tools used to derive Amdahl’s law with interprocessor communication
• Surrogate for design of AI Classifier Captures some essential elements with small but sufficient configurations to make the search non-trivial Simple enough that correct outcomes are known and algorithm performance can be readily evaluated
Taxonomy for Base-Lumped Model Methodology Using Pair-of-Models Which Model Base Model
Behavior Generation
Lumped Model
Analytic
Applied Math
Checking Direction L-of-B
Simulation
Logic-based
LM is analyzed to Check BM LM is simulated to replace BM to address question/purpose BM is simulated to Check LM
B-of-L
Address Question/Purpose Lumped Model
Base Model
Examples: Roles in Taxonomy Example
Instance of LM is simulated to replace BM to address question/purpose
Comment LM (further simplified) is analyzed to check LM
Combat modeling attrition multiresolution
BM is simulated to check LM
LM replaces BM components with same I/O interface (messages)
Multiprocessor Interconnect effect on speedup
LM is analyzed to check BM
LM has simplified I/O interface (message rate) so cannot replace BM
Brain simulation scale modeling
Surrogate for design of AI LM is simulated to replace BM to Classifier address question/purpose
Replacement is temporary to enable design of Classifier
Examples: Roles in Taxonomy Example Brain simulation scale modeling
Instance of LM is simulated to replace BM to address question/purpose
Comment LM (further simplified) is analyzed to check LM
Combat modeling attrition multiresolution
BM is simulated to check LM
LM replaces BM components with same I/O interface (messages)
Multiprocessor Interconnect effect on speedup
LM is analyzed to check BM
LM has simplified I/O interface (message rate) so cannot replace BM
Surrogate for design of AI LM is simulated to replace BM to Classifier address question/purpose
Replacement is temporary to enable design of Classifier
Examples: Roles in Taxonomy Example Brain simulation scale modeling
Instance of LM is simulated to replace BM to address question/purpose
Comment LM (further simplified) is analyzed to check LM
Combat modeling attrition multiresolution
BM is simulated to check LM
LM replaces BM components with same I/O interface (messages)
Multiprocessor Interconnect effect on speedup
LM is analyzed to check BM
LM has simplified I/O interface (message rate) so cannot replace BM
Surrogate for design of AI LM is simulated to replace BM to Classifier address question/purpose
Replacement is temporary to enable design of Classifier
Examples: Roles in Taxonomy Example Brain simulation scale modeling
Instance of LM is simulated to replace BM to address question/purpose
Comment LM (further simplified) is analyzed to check LM
Combat modeling attrition multiresolution
BM is simulated to check LM
LM replaces BM components with same I/O interface (messages)
Multiprocessor Interconnect effect on speedup
LM is analyzed to check BM
LM has simplified I/O interface (message rate) so cannot replace BM
Surrogate for design of AI Classifier
LM is simulated to replace BM to address question/purpose
Replacement is temporary to enable design of Classifier
Conditions required for valid aggregation (lumping) • Uniformity and Indifference in 1 block – Brain simulation • Uniformity and Indifference in 2 blocks – Combat Modeling Each force groups Red and Blue, studied for attrition Non-uniformity handled by approximate morphism
• Uniformity and indifference in N+1 blocks – Multiprocessor N vs 1 queue Non-uniformity handled by special solution
Brain simulation Consider model of neural net too large to simulate e.g. billions of neurons SpiNNaker https://www.theregister.co.uk/2017/1 0/19/steve_furber_arm_brain_intervi ew/
Question: what steady state firing rate would it settle into? Neural net simulation
Real neural cortex Construct
X
?
steady state firing level
Want a model that can be simulated and whose rate would predict the full scale counterpart. Analogy: Scale model e.g. run scaled airplane in wind tunnel and scale results back to full scale. Question: how to scale to achieve this?
What is the smallest feasible scaled simulation and how to construct it? Real neural cortex
steady state firing level Infer Construct Scaled Neural net simulation
! Feasibly Simulate
Scaled steady state firing level
Feasibility of scaling Real neural cortex
100 billion neurons
• Turns out that with same model of neuron, the avg number of neighbors must be kept the same while drastically reducing size of net. • So smallest scale model is fully connected net of size equal to avg number of neigbors+1, • i.e., scale reduction is net size, 1011, to neighbor size, 104 = 107
10,000 neurons with 100 million connections *The Human Brain in Numbers: A Linearly Scaled-up Primate Brain, Suzana Herculano-Houze https://faculty.washington.edu/chudler/facts.html#neuron
• Note: Net size of 104 is feasible but needs 108 connections
Viewed as Pair-of-Models Real System
Base model
Experimental Frame
Lumped model
• As in aeronautics, knowing how to scale can provide smart way forward in brain simulation • But knowing how to scale is knowing how to construct a base-lumped model pairs
In this case Generalizes Mean Field Approximation
Why Should We Develop Simulation Models in Pairs? End of Part 1
Why Should We Develop Simulation Models in Pairs? Part 2 Bernard P. Zeigler Professor Emeritus, University of Arizona Chief Scientist, RTSync Corp.
[email protected] 50th Anniversary Titans Talk Winter Simulation Conference
December 2017
So far… • Motivation: Why build target and simpler models – it’s hard enough to build one? o Because: The Simpler model can o o o o o
give more easily attained insight be more general be used to test the target (more complex model) proxy for the target explain the target’s behavior
• Conditions required for Valid Simplification: o Uniform structure within blocks o Indifference of block interaction to identity of block members
• Brain simulation scale modeling Pair-of-models approach to impossibility of full scale simulation
Combat modeling attrition multiresolution
Encounters between higher level combat units with win/loss probabilities, durations and remaining force determined from lower level unit model
Base Model With Single Weapon Units Lumping Morphism 1
Lumped Model With Brigade Units Lumping Morphism 2
Hofmann, Marko A. "Criteria for decomposing systems into components in modeling and simulation: Lessons learned with military simulations." Simulation 80, no. 7-8 (2004): 357-365
Lumped Model of Lumped Brigade With Model Single of Lumped Brigade With Model Single of Weapon Units Brigade With Single Weapon Units Weapon Units
Combat modelingUniformity violations: Vulnerability, firing power attrition multiresolution Uniformity and Indifference Conditions:
Emitted fire uniformity: Every unit distributes the same total fire volley to the opposing group.
Received fire uniformity: Every Indifference violations: unit receives the same fire volley Front line vs rear positions from the other group. Avoid ontological mismatch: current highresolution simulations (such as Janus that employs a time-step approach for simulating target acquisition) do not explicitly keep track of the times at which LOS is established or lost for specific target-observer (unit) pairs needed for valid Lanchester-type aggregated-replay model using reliable statistical estimation techniques for determining model parameters
Combat modeling attrition multiresolution Uniformity and Indifference Conditions:
Emitted fire uniformity: Every unit distributes the same total fire volley to the opposing group.
Received fire uniformity: Every unit receives the same fire volley from the other group. Avoid ontological mismatch: current highresolution simulations (such as Janus that employs a time-step approach for simulating target acquisition) do not explicitly keep track of the times at which LOS is established or lost for specific target-observer (unit) pairs needed for valid Lanchester-type aggregated-replay model using reliable statistical estimation techniques for determining model parameters
System Morphism
Shot Red Lumped Group
Shot
Blue Lumped Group
Experimental Frame
Approximate Morphisms – Lumpability Zone
Multiprocessor Interconnect effect on speedup: Amdahl’s Law Speedup sequential time / parallel time N * CompTime CompTime CommDelay ( N , CommNet ) So the speedup relative to the maximum possible is : Speedup RelativeSpeedup N CompTime CompTime CommDelay ( N , CommNet )
Departures from strict lumpability requirements and their effects • uniformity of structure, i.e., all processors have the same properties, here the same processing time •
indifference of the CommNet service to the processors’ requests, i.e.., there is no priority for processors in the underlying queueing discipline
Requirement 1) is violated when processors have varying processing times – the departure is measured by variance and shown to reduce speedup Requirement 2) is violated where processors have prioritized access to the network: Special trick here: • We can extend the lumped model to compute the communication delays experienced by the processors in iterative fashion. Since a processor can only employ the communication medium if all processors with higher priority are busy with their computation the interaction of components can be handled in a serial manner • General case requires iteration to fixed point solution
Initial concept exploration and successful demonstrations done with Surrogate for Linear Discriminant Analysis Base Model
Base Data
Limped Data
Experimental Frame Maximize Signal-to-Noise Ratio
Projective mapping w , that Optimally Separates classes
Lumped Model
The lumped model is required to be equivalent to the base model in generating Data that will provide the same kind of projective mapping when fed to the Process that derives the mapping by maximizing the signal to noise ratio • Captures some essential elements with small but sufficient configurations to make the search nontrivial • Simple enough that correct outcomes are known and algorithm performance can be readily evaluated
• Simplicity of the model and its intuitive behavior help to illustrate the target technology
Multiresolution Model Families: Example Global Warming Questions
Model components
Approximate Parameter Morphisms
Computational Checking of Homomorphism Listing 1. Computational test of homomorphism 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17.
Create a Simulator, bsim and attach it to the BaseModelAndFrame, bmf Create a Simulator, lsim and attach it to the LumpedModelAndFrame, lmf Access the Transducer, btrans component of bmf Access the Transducer, ltrans component of lmf Tell bsim to start (which starts bmf in its initial state) Tell lsim to start (which starts lmf in its initial state, assumed to be corresponding to that of bmf) Until all iterations done { Tell bsim to do one state transition of bm, Tell lsim to do one state transition of lm, Query btrans for the current state of bm, bmState Query ltrans for the current state of lm, lmState bmState’ = Correspondence(bmState) Compare: If bmState’ ~= lmState, break and report not homomorphic } Tell bsim to stop Tell lsim to stop Report homomorphism confirmed
Workflow for Approximate Morphism Evaluation Example Operations and Workflow Orchestrators Simulation control n-arry Operations e.g., Operation(base, lumped,ef)
Workflow Implications
makeSimulation(sim, base, lumped, ef) StartSimulation(sim) RunSimulation(sim,iteration) getResults(sim)
StartSimulation(base,lumped) => StartSimulation(base) and StartSimulation(lumped) etc.
Evaluation Compare(base, lumped,ef) ComputeLumpabilityZone (base, lumped,ef) ComputeBackgroundLevel (base,ef)
CompareWith(base, lumped) => StartSimulation(base,lum ped)
Take-Aways: Pairs-of-Models Approach • Why put in twice the time and effort to develop two models rather than one? • Working with base-lumped pairs may Be better than: constructing a target model and later developing simpler one under stress Offer value added:
give more easily attained insight be more general be used to test the target (more complex model) proxy for the target
explain the target’s behavior
• Benefits of Multiresolution Family of Models
Can be Organized By Approximate Morphisms Able perform mutual cross-calibration (Paul Davis) Avoid harmonization issues of the underlying ontologies Better reconcile and correlate model behaviors
• Principles of Valid Simplification Conditions: Uniformity and Coupling Indifference Approximate Morphisms relax conditions – open up possible pairings
• Constructive Agenda Software tools to implement the theory
For more in-depth exposition: • Videos: Bernard Zeigler: JDF Part 1 Exact and Approximate Morphisms Bernard Zeigler: JDF Part 2 Simulation Morphisms and Model Property Preservation
• Books B.P. Zeigler, A. Muzy, and E. Kaufman, Theory of Modeling and Simulation: Discrete Event & Iterative System Computational Foundations, 3rd Ed. 2018 Bernard Zeigler, Simulation-based Evaluation of Morphisms for Model Library Organization In Model Engineering for Simulation, Ed Zhang, 2018
