Cumulative cost of over $60 billion. Hurricane Katrina (2005) slammed NO. At risk of damage from coastal storms. ⢠6.5 million U.S. homes and. ⢠$1.5 trillion of ...
A Learning Science Protocol for Evaluation of a Sustainability-based Learning Environment Mahdi M.N. Abadi1, Babak Bahaddin1, Reza Feyzi Behnagh1, Michael A. Deegan2, David F. Andersen1, Luis F. Luna-Reyes1, Rod MacDonald3, Eliot Rich1 1. University at Albany, 2. US Army Corps of Engineers, and 3. Initiative for System Dynamics in Public Sector
Major Project Components
Problem Domain Hurricane Sandy (2012) hit the NY/NJ Coast Cumulative cost of over $60 billion Hurricane Katrina (2005) slammed NO
Steps Refinement of the current SBLE Refinement of facilitation plans to: Promote evidence-based approach to decision and policy making Assessing actual insights participants develop during the experience
Major Evaluation Streams
Simulation Core Logic +
development density R
+
+
desired protection
level of protection gap
tourism pressure on land development
+
+
Protection from Mitigation Projects
B
-
development to carrying capacity ratio -
Natural Barriers +
perceived protection from natural resources and mitigation projects
+
Evaluation of Learning in USACE Workshop
Sustainability Curriculum as System-Based Lessons for
Evaluate Classroom-Based SBLE
Existing Products First Draft of Coastal Protection Sustainability Lessons
Final Draft of Coastal Protection Sustainability Lessons
Evaluate NJ Workshop First USACE SBLEBased Coastal Protection Planning Workshop
Aim 2: Can a SBLE Effectively Teach System Sustainability Lessons for Coastal Protection?
Second USACE SBLEBased Coastal Protection Planning Workshop
Final Draft of Coastal Protection Sustainability Lessons
Evaluate Second Workshop Third USACE SBLEBased Coastal Protection Planning Workshop
Aim 1: What are General Systems-Based Sustainability Lessons? What are System Sustainabililty Lessons for Coastal Protection?
Evaluate Final Workshop Aim 3: Is the System Sustainability Curriculum for Coastal Protection Relevant in Actual Practice?
Stocks as System Building Blocks
Figure 2. Major Products, Research Questions, and Evaluation Activities
System Dynamics Lessons
Use and Evaluate Coastal Protection SBLE in Controlled Classroom Setting
Several key system stocks are fundamental Accumulations of human activity Stocks of natural resources Mitigating adverse impacts of human activity (investment) Coastal risk is driven by: Long-term construction near the coast Degraded natural protective barriers Protective barriers
Feedback Drives Behavior
Endogenous Point of View
Feedback between stocks & flows is the basis of behavior
Aspects of behavior are the result of policies and endogenous actions
Feedback among the three classes of stocks over time determine long term system behavior and often lead to unintended and undesirable outcomes
Well-intended environmental policies may not be sustainable because they may trigger “better before worse” patterns of system behavior
See Figure 1
Attempts to protect the coast solely by investing in the built environment may be a self-defeating policy
Evaluations of SBLE with Controls for Classroom-Based Activities Classroom Based
-
protecting the community
Evaluation of Classroom Learning
System-Based Lessons for Sustainable Coastal Protection
risk perception B
+
-
If you build it, they will come
Updated CoastalProtectSim Model and SBLE Materials
Existing CoastalProtectSim Model and Materials
Coastal ProtectSim Cockpit
Land Development
R
+
development pressure for tourism
USACE Experience with Coastal Protection Policy & Management and Shared Vision Planning
Decision makers have difficulty reaching consensus about how to apply climate information in a complex policy context Regulations and incentives needed to preserve community viability Emerge as a negotiated result that may take many years to complete Coastal managers may inadvertently increase long term risks by encouraging construction and revitalization
Simulation-Based Learning Environment in USACE
Coastal Protection
Challenges Models require data integration Topological Physical Population
Workshops
(1) System Sustainability Curriculum, (2) SBLE, and (3) USACE Coastal Protection Workshop
Provide stakeholders the mechanism to understand the effects of their actions and inactions on their sustainable future: Fundamental lessons in management The effects of complexity Reasons for unanticipated outcomes
Hurricane Katrina (2005) slammed New Orleans Total cost to the US Economy: $110 billion Health risks: Hepatitis A, cholera, TB, typhoid/yellow fever, malaria, WNV Death toll: 1,836 lives Loss of jobs: 400,000
Refined Coastal Protection Simulation Model [1] Simulation-Based Learning Environment in Classroom [2]
Sustainability Principles and Lessons
Goals
Lessons for the Coastal Protection
At risk of damage from coastal storms 6.5 million U.S. homes and $1.5 trillion of assets
US Army Corps of Engineers
Major Component Contextualization Pre Test
Natural Resources
Treatment
Figure 1. Major Stocks and Feedback Loops in the CoastalProtectSim Post Test [1] Deegan, M.; Stave, K.; MacDonald, R.; Andersen, D.; Ku, M.; Rich, E. Simulation-Based Learning Environments to Teach Complexity: The Missing Link in Teaching Sustainable Public Management. Systems 2014, 2, 217-236 [2] Ku, M. (n.d.). Teaching and Learning about Complexity in Public Policy Decision Making: Using a Simulation-Based Learning Environment in the MPA Classroom. Journal of Public Affairs Education (forthcoming).
Evaluation
SBLE
Field Based Control
Pointe Claire Case Materials Background on Coastal Protection Structured survey Written essay on selected topics Pre-simulation interaction with policy issues Strategic Structured interactions with task/group analysis simulator exercises Debriefing and written or “think aloud” products Structured survey materials Model-based experimentation with same/different context Reasoning-based written final assignment Multiple measures possible, e.g. --measure improvement from runs Measure the difference between the results of the final written assignments and results from the control group
SBLE Problem Defining and other exercises drawn from GMB experiences appropriate to selected group Pre workshop interviews Other instruments (e.g. social network analysis) Pre-simulation interaction with policy issues with written products Structured interactions with simulator Debriefing and written or “think aloud” products Qualitative debriefing based on best practice from group model building scripts Follow up with relevant instruments Use qualitative approach to assess realism and relevance of overall exercise
