Rapid Adaptive Realistic Behavior Modeling is Viable for Use in Training Margery J. Doyle and Antoinette M. Portrey L-3 Communications Link Simulation and Training 711/HPW Air Force Research Lab Warfighter Readiness Research Wright-Patterson Air Force Base, OH, 45433
DISTRIBUTION A: Approved for public release: distribution is unlimited. 88 ABW-2014-1036 13 March 2014; 88 ABW-2014-0173 22 January 2014
Not-So-Grand-Challenge (NSGC): Overview • NSGC: White Space • NSGC Phase I: Problem Statement • NSGC Phase I: Goals • NSGC Phase I: Approach • NSGC Phase I: Accomplishments • NSGC Phase I: Conclusions • Phase II & III Way Forward
Federation of Models and Tools Needed to Support Large Complex White Space
Copyright © 2013 Charles River Analytics Inc. – All Rights Reserved
NSGC Phase I: Problem Statement • Rapidly evolving strategic environments = new threat profiles • Requiring the warfighter to adapt operational strategies and training – Rapidly modifiable capabilities-based training
» Persistent intelligent agent/architectures/models » Display adaptive behaviors to evolving threat environments » Must be quickly and easily reconfigurable – Example: Use enemy aircraft maneuvers data to generate CGFs within a week
• Requires assessing the viability of using rapidly developed GOTs/COTs models in Live Virtual Constructive (LVC) capabilities-based training research environments 7/18/2014
4
NSGC Phase I: Goals • Develop data consumable by, and meaningful for, models • Integrate rapid modeling capability into DMO/LVC testbed using Distributed Interactive Simulation (DIS) protocols • Evaluate model capability to adapt and exhibit realistic behavior
• AFRL Warfighter Readiness Research Division • Main problem: achieving integration and interoperability at AFRL • Define data parameters, thresholds required to define and develop agent models • Develop a testbed and model-to-DIS interface (m2DIS) • Allow models to talk & listen to the DIS party line
• Industry and Gov’t Partners
• Specify and develop believable agent models • Integrate with the existing infrastructure at AFRL • Demonstration model approach and behaviors
7/18/2014
5
NSGC Phase I: Approach Enterprise Architecture • Flexible and Modular
• Instance-based Interaction • Model to DIS (m2DIS) • Rapid Technical Integration • Semantic Interoperability
• Grounded in Domain • SME Knowledge Elicitation • Taxonomy • Ontology
• Model Based Systems Engineering • Multi-modal Validation • Training context • Direct Observations
• Warfighter • Systems & Software Eng.
Architecture, Integration, and Interoperability x Phase NSGC Phase I Architecture NSGC Phase II Architecture
Syntactic: integrated systems lack a shared meaning of exchanged data Semantic Interoperability: Two integrated systems share a common understanding of what the shared data means 7/18/2014
Pragmatic: integrated systems share a common context and purpose (i.e., LVC training research context) Zeigler & Hammonds (2007), Mittal et.al (2008) 7
Tractable Traceable Problem Space • Provided two well defined F-16 air-to-air scenarios complete with the expected behaviors and commands (m2DIS)needed to act in the world • Information such as time, space, and positional information (TSPI) were communicated in DIS and could be collected/used by the models Visual Identification 6: Allow Red Force models to interact with non-reactive Blue Force players while red air performs a stern (baseline) intercept.
7/18/2014
Sweep-2: Allow Red force to initially present a range problem for Blue force. Whereby Blue force would typically execute a Grind tactic.
8
High Level Sample of a Model
Copyright © 2013 Charles River Analytics Inc. – All Rights Reserved
NSGC Phase I: Validation and Verification (V&V) • Validation has two main components – Structural validation • An internal examination of the assumptions, architecture, and algorithms in the context of intended use
– Output validation • How well results compare with the “perceived real world.” – In this case, a model’s ability to represent realistic adversarial behavior
• Verification – Transformational accuracy of model development process • Transformation of functional need requirements – Functional need to concept model (descriptive functional properties) – Into actionable model (executable functional properties) in context Adapted a model, integration, interoperability V&V process from Petty (2010) and Wang, Tolk, and Wang (2009). 7/18/2014
10
Phase I: Stakeholder Centric Results 4
Pilot SME
3.5
Engineer
All Perspectives
3 2.5 2 1.5 1 0.5 0 Alion
Team 1
7/18/2014
Aptima
Team 2
CHI
Team 3
CRA
Team 4
PALM
Team 5
SoarTech
Team 6
Stottler Henke
Team 7
All Teams
All Teams
11
Phase I: SME Team x Function Results 3.5
3
2.5
2
1.5
1
0.5
0 Alion Team 1
7/18/2014
Aptima2 Team
CHI 3 Team
CRA 4 Team
PALM Team 5
SoarTech Team 6
Stottler TeamHenke 7
Combat Realism
Maneuver Execution
Control of Intercept Geometry
Simulated SA/ Information Sharing
Target Sorting
Maintains Formations/ Reforms
12
Phase I: Eng. Team x Function 4.5 4 3.5 3 2.5 2 1.5 1 0.5 0 Combat Realism
Maneuver Execution Team 1
7/18/2014
Control of Intercept Geometry
Team 2
Team 3
Simulated SA/ Information Sharing
Team 4
Team 5
Team 6
Target Sorting
Maintains Formations/ Reforms
Team 7
13
Approach Fit to MBSE Paradigm Criteria System Behavior 1 VID-6 Scenario 1a Individual behavior 1b Team behavior 2 Sweep-2 Scenario 2a Individual behavior 2b Team behavior System Structure 3 Architecture /Tech Modularity MBSE Process 5 Requirements
Team
Team
Team
Team
Team
Team
Team
Partial Failed
Partial Failed
Full Partial
Partial Partial
Full Full
Full Partial
Full Full
Partial Partial
-
Full Partial
Partial Partial
Full Partial
Full Full
Full Full
Withheld
Withheld
Withheld
Withheld
Withheld
Withheld
Withheld
-
-
Yes
Yes
Yes
Yes
Yes
-
Yes
Partial
Yes
Yes
Yes
Yes
6
Modeling tool
Withheld
Withheld
Withheld
Withheld
Withheld
Withheld
Withheld
7
Model design Complexity (Feature set) Instrumentation
Simple
Simple
Moderately complex
Simple
???????
Complete
Complete
Yes
Yes
Yes
Yes
Yes
Yes
Yes
9
Testing & Evaluation methodologies
Partial
Yes
Yes
Yes
Yes
Yes
Yes
10
Integration with AFRL NICE (Model execution) Model Behavior Validation
Success
Success
Success
Success
Success
Success
Success
1.57
1.33
1.0
1.68
2.35
2.91
2.52
Available
Available
Available
Available
Available
Available
Available
8
11
12
Verification Method
NSGC Phase I: Accomplishments • Technical syntactic integration achieved • Achieved use of less-scripted, more realistic models
– Established a model, integration, interoperability V&V process adapted from Petty (2010) and Wang, Tolk, and Wang (2009). – Established tractable and repeatable model behavior method through use of two common scenarios across architecture types
• Separated physics model from behavior model: supporting semantic interoperability through use of m2DIS
– Physics modelers handle development of the environment, platform, and flight model – Behavior/Cognitive modelers handle decision, perception, choice of tactic, goal oriented behavior – Acting in their environment through use m2DIS commands that drive the platform to act in the world
• • • •
Supporting architecture-agnostic model interoperation Set stage for achieving sematic interoperability Increasing realism, speed, and accuracy when modeling new behaviors Creating a rapidly modifiable threat system
NSGC Phase I: Accomplishments Cont. • Created excellent partnership with industry
– Leverages disparate modeling methodologies – Architectures integrated with DIS and threat system – Creating rapid modeling capability
• NSGC Enterprise Architecture supports rapid integration of new CONOPS/TTPs models for use in LVC/ DMO training, research, and ops • Various methods of model development and use are now being considered • Modularity allowed for evaluations of external validity while preserving scientific boundaries for validation of theoretical underpinnings inherent in an architecture • Conceptualized Environment Abstraction for enhanced model situation awareness – Modular and hierarchical agents can now be used by multiple platforms
• Developed a Fighter Combat Situation Recognition Taxonomy 7/18/2014
16
NSGC Phase II and III Goals • Phase II: Achieve Semantic Interoperability – Environment Abstraction
• Add semantic constructs to environmental data • EA data used by different collaborator models ( i.e., shared ontology) – Facilitating dynamic situation awareness – True semantic interoperability – Lays foundation for use of a federation of models and scalability
• Achieve agent tactical awareness, understanding, “what if” prediction capability and learning – For use of new tactics, an adaptive behavior
• Develop for usability, reusability, and generalizability
– Foster easy rendering capability to make models composable by common users without need to use propriety software/systems
• Strengthen operational realism • Reduce to practice the rapid modeling capability
– Allows for incorporation of changes to doctrine, i.e., new Tactics, Techniques and Procedures (TTPs) derived from lessons learned
7/18/2014
17
Questions?
7/18/2014
18
CONTRACT No.: FA8650-05-D-6502 Task Order 013: Operational Development and Validation of CompetencyBased Methods and Tools for Enhancing Human Performance in Air and Space Warfighting Systems Point of Contact: Dr. Winston "Wink" Bennett 711 HPW/RHAS 2620 Q Street (Bldg 20852) Wright Patterson AFB, OH 45433-7955 Phone (Comm): 937.938.2550 Phone (DSN): 798-2550 Fax: 937.904.8797 Email:
[email protected] 7/18/2014
19
Backup Slides
7/18/2014
20
The Future in Detail • EA Supports Realistic Behavior • Purposeful action requires •
Real world perception, knowledge of what is relevant, effective decision making and the execution of context goal relevant behavior
• Agent must know • • • • •
What action/s a situation affords Know when to act Know why it should take action (~goal driven) Know what to act upon Know how to act within the limits of what affordances exist
• Inclusive of proactive actions, reactions, and passive observance
• That is, to have structural-functional procedural understanding of environment, situation, and one’s own capabilities and limitations to determine what action can or should be taken in the world • Situation Awareness and recognition of affordances in context • Situation Understanding (e.g., why, what, how)
7/18/2014
21
