Joshua New, Ph.D. Oak Ridge National Laboratory
[email protected] 865-241-8783
Seminar 55 Simulation Calibration Autotune Calibration
Learning Objectives • Describe how ASHRAE Guideline 14 defines calibration criteria for energy simulation • Describe how high performance computing (HPC) resources can be used to efficiently distribute simulation runs across multiple servers • Describe how machine learning algorithms can be used to support the development of efficient calibration techniques • Describe the disadvantages of each of the three calibration techniques presented • Describe the advantages of each of the three calibration techniques presented • Describe realistic scenarios for model calibration that can be utilized by practitioners today ASHRAE is a Registered Provider with The American Institute of Architects Continuing Education Systems. Credit earned on completion of this program will be reported to ASHRAE Records for AIA members. Certificates of Completion for non-AIA members are available on request. This program is registered with the AIA/ASHRAE for continuing professional education. As such, it does not include content that may be deemed or construed to be an approval or endorsement by the AIA of any material of construction or any method or manner of handling, using, distributing, or dealing in any material or product. Questions related to specific materials, methods, and services will be addressed at the conclusion of this presentation.
Acknowledgements • • • • • • • • •
Amir Roth – DOE’s BTO Aaron Garrett – JSU Jibonananda Sanyal – ORNL Richard Edwards – UT Mahabir Bhandari – ORNL Som Shrestha – ORNL Buzz Karpay – Karpay Associates XSEDE OLCF
Outline/Agenda • Motivation • What is Autotune? – Calibration as search
• How does it work? – Methods for speeding up the search
• How good is it? – Calibration process and accuracy
• How can I use it? – Deployment as web service
Motivation 3,000+ building survey, 23-97% monthly error ASHRAE G14 Requires
Using Monthly utility data
CV(RMSE)
15%
NMBE
5%
Using Hourly utility data
CV(RMSE)
30%
NMBE
10%
5
The Autotune Idea Automatic calibration of software to data E+ Input Model
. . .
6
Calibration as Search Problem/Opportunity: ~3000 parameters per input file
2 minutes per simulation = 83 hours
7
Supercomputers for Buildings • • • •
EnergyPlus is a desktop app Writes files during a simulation Use RAMdisk Balance simulation memory vs. result storage • Validate simulation output • Bulk write data to disk • Design of Experiments for Uncertainty Quantification • In-Situ data analysis • Scalable Architecture for Big Data Mining • 270TB of simulation data
CPU Wall-clock Data EnergyPlus Cores Time (mm:ss) Size Simulations 16 18:14 5 GB 64 32
18:19
11 GB
128
64
18:34
22 GB
256
128
18:22
44 GB
512
256
20:30
88 GB
1,024
512
20:43
176 GB
2,048
1,024
21:03
351 GB
4,096
2,048
21:11
703 GB
8,192
4,096
20:00
1.4 TB
16,384
8,192
26:14
2.8 TB
32,768
16,384
26:11
5.6 TB
65,536
32,768
31:29
11.5 TB
131,072
65,536
44:52
23 TB
262,144
131,072
68:08
45 TB
524,288 8
Suite of Machine Learning • Linear Regression • Non-Linear Regression
• Feedforward Neural Network
• Self-Organizing Map with Local Models
• Regression Tree (using Information Gain)
• Support Vector Machine Regression
• Time Modeling with Local Models
• K-Means with Local Models
• Recurrent Neural Networks
• Gaussian Mixture Model with Local Models
• Ensemble Learning (combinations of multiple algorithms)
• Genetic Algorithms
Integrated mixture of Commercial, Research, and Open Source software 9
MLSuite Architecture Data Preparation
PBS
XML
Supercomputer #1
MLSuite Linux #1
30x LS-SVMs validation folds 1-10 input orders 1-3
…
Linux #218
Supercomputer #2
MLSuite Example • EnergyPlus – 2-10 mins for an annual simulation !- ALL OBJECTS IN CLASS Version, 7.0; !- Version !- SIMULATIONCONTROL === SimulationControl, No, !-Do Zone Sizing Calc No, !-Do System Sizing Calc …
• ~E+ - 4 seconds AI agent as surrogate model, 90x speedup,
