Lesson 12 - Exercise documentation
New vocabulary for this lesson:
- Situation Manual (SITMAN)
- Exercise Plan (EXPLAN)
- Controller and Evaluator Handbook (C/E Handbook)
- Master Scenario Events List (MSEL)
- Procedural Flow
- Exercise Evaluation Guides (EEGs)
- Exercise Evaluation Plan (EVALPLAN)
No matter what kind of exercise you run, you’ll need documents to help guide you and your exercise participants. Let’s briefly discuss some typical exercise documentation needs:
Situation Manual (SITMAN)
What is a Situation Manual (SITMAN)?It’s a handbook provided to all participants in discussion-based exercises, particularly TTXs. The SitMan provides background information on the exercise scope, schedule, and objectives. It also presents the scenario narrative that will drive participant discussions during the exercise.
This is a guide to the exercise that is handed out to participants. While it doesn’t need to be more than a few pages, it gives background information on the exercise scope, schedule, and objectives. It also presents the scenario narrative that participants will use during the exercise. The SITMAN should mirror any multimedia briefing, supporting the scenario narrative and allowing participants to read along while watching events unfold.
Exercise Plan (EXPLAN)
What is an Exercise Plan (EXPLAN)? This is a general information document that helps operations-based exercises run smoothly. It is published and distributed prior to the start of exercise and provides a synopsis of the exercise.
An Exercise Plan (EXPLAN) is typically used for operations-based exercises. It provides a summary of the exercise and is distributed before the exercise starts. It discusses the exercise objectives and scope, and assigns tasks and responsibilities. The EXPLAN doesn’t contain detailed scenario information and is generally intended for exercise participants and observers.
What’s included in an Exercise Plan?
- Purpose of the exercise
- General objectives
- Overall exercise strategy
- Exercise Planning Group structure
- Agencies involved - tasks and responsibilities
- Safety and security (in general terms)
- Exercise type and basic information
- Basic administrative and/or logistical support
Information in the Situation Manual and the Exercise Plan is sometimes combined into a Player Handbook.
A Player Handbook contains a list of instructions for players, as well as information about player responsibilities and functions to be performed during the exercise. It helps the players in understanding the ground rules, the overall objectives and scope of the exercise, limits of play, simulation plans, and the debriefing process.
The player handbook should include a section on Administrative Notes and include information such as:
- Location of restrooms.
- Lunch time.
- Parking locations.
If you don’t include it, you can be certain participants will be asking for this information!
What’s In a Player Handbook?
- A Schedule of player exercise briefings
- Provisions to review emergency management plans, policies, procedures
- Scenario overview
- A list of exercise objectives
- Procedures for preparation of exercise generated messages, logs and reports
- Emergency Operating Centre (EOC)_procedures
- Expected player actions
- Administrative requirements
- Recommended pre-exercise training events
Exercise Control Plan
What is an Exercise Control Plan? This plan provides exercise controllers and simulators with guidance concerning procedures and responsibilities for exercise control, simulation, and support. It explains the exercise concept as it relates to controllers and simulators, establishes the basis for control and simulation of the exercise, and establishes and defines the communications, logistics, and administration structure needed to support control and simulation during the exercise.
This document contains more detailed information about the exercise scenario and describes the controller staff roles and responsibilities. As the controller staff notes contain information on the scenario and exercise administration, it’s best to only distribute the Exercise Control Plan to those individuals specifically designated as controllers and evaluators.
What’s In an Exercise Control Plan?
- General objectives
- Concept of play (exercise scope, scenario narrative, location of players)
- Specific functional objectives
- Procedures, responsibilities, assignments and support
- Exercise Planning Group structure
- Exercise timelines (including pre- and post exercise activities)
- Emergency call-off procedures, safety and security
- Artificialities, assumptions and simulations
- Master scenario events list (MSEL) for the exercise
- Communications capabilities, structure and procedures
- Checklists or any other job aids needed (including maps, reference etc)
Controller and Evaluator (C/E) Handbook
What is a Controller and Evaluator Handbook (C/E Handbook)?It’s an exercise overview and instructional manual for controllers and evaluators. A supplement to the Exercise Plan (ExPlan), it contains more detailed information about the scenario, and describes controllers’ and evaluators’ roles and responsibilities. Because the C/E Handbook contains information on the scenario and exercise administration, it should be distributed only to those individuals specifically designated as controllers or evaluators.
This document supplements the EXPLAN. It contains more detailed information about the exercise scenario, and describes the roles and responsibilities of exercise controllers’ and evaluators. Because the C/E Handbook contains information on the scenario and exercise administration, it goes only to controllers and evaluators.
Master Scenario Events List (MSEL)
What is a Master Scenario Events List (MSEL)? It’s a chronological timeline of expected actions and scripted events to be injected into exercise play by controllers to generate or prompt player activity. It ensures necessary events happen so that all objectives are met, and provides guidance for controllers and/or simulators in keeping the exercise on schedule.
The Master Scenario Events List (MSEL) links simulation to action, makes the exercise experience relevant for players, and uses an incident or activity that is intended to prompt players to action. Each Master Scenario Events List (MSEL) record contains a designated scenario time, an eventsynopsis, the name of the controller responsible for delivering a particular inject and any special delivery instructions, the task and objective to be demonstrated, the expected action, the intended player, and a note-taking section.
It’s important that messages are entered in their proper sequence so the exercise will maintain “flow” and controllers can monitor the tracking of the messages.
Let’s take a look at a sample Master Scenario Events List (MSEL), and see what information can go in it.
Sample Master Scenario Events List
Plane radios tower: losing engine power and altitude
1. Tower notifies dispatch centre.
2. Dispatcher alerts police, fire, medical services to proceed to airport.
Pilot reports major
vibrations/noise. Requests runway designation.
1. Tower designates runway; notifies dispatcher of runway and potential for mass casualty incident.
2. Dispatcher relays runway info to police, fire, medical.
3. Dispatcher notifies hospitals.
4. Crash/Fire Rescue initiates ICS; notifies Dispatcher of Crash Position and staging locations.
5. Dispatcher relays Crash Position and staging locations to police, fire, medical.
Hospital calls dispatcher requesting more information
1. Dispatcher gets potential number of casualties and relays info to hospital.
2. Hospital notifies other medical facilities.
Media calls dispatcher requesting
Exercise Evaluation Guides (EEG)
What are Exercise Evaluation Guides (EEG)?These documents support the exercise evaluation process by providing evaluators with consistent standards for observation, analysis, and After Action Report (AAR) development.
Exercise Evaluation Guides (EEG’s) are developed to help with exercise evaluation, incorporating the critical tasks to be completed in an exercise. These are developed by the controllers and evaluators to evaluate the exercise and to produce a final exercise report.
The main point that an exercise report needs to focus on is whether the main aim of the exercise was met. Developers of Exercise Evaluation Guides (EEGs) are either experienced exercise evaluators, and/or practitioners who are subject matter experts but may have little or no exercise evaluation experience.
Exercise Evaluation Guides (EEGs) provide evaluators with information on what they should expect to see during an exercise, space to record observations, and questions to address after the exercise as a first step in the analysis process.
Exercise Evaluation Plan (EVALPLAN)
What is an Exercise Evaluation Plan (EVALPLAN)? It’s typically used for operations-based exercises of a large scope and scale. This document provides specific guidance to exercise evaluators. It’s designed to help exercise evaluators understand their roles and responsibilities in exercise data collection and evaluation in order to conduct an effective analysis of the exercise and produce a comprehensive After Action Report (AAR)/Improvement Plan (IP).
The evaluation process for exercises includes a formal exercise evaluation, integrated analysis, and an After Action Report/Corrective Action Plan (AAR/CAP). This process begins during exercise planning and ends when improvements have been implemented and validated in future exercises.
8 Steps in the Evaluation Process
Step 1: Plan and organize the evaluation
Step 2: Observe the exercise and collect data
Step 3: Analyze data
Step 4: Conduct an exercise debrief
Step 5: Develop the draft AAR
Step 6: Identify improvements and corrective actions that need to be implemented
Step 7: Finalize and issue the AAR
Step 8: Track implementation
What’s in an Evaluation Plan?
- Purpose of the handbook
- General objectives
- Concept of play (scope, scenario narrative, location of players)
- Specific functional objectives
- Timelines (including pre- and post-exercise activities)
- Emergency call off procedures
- Artificialities, assumptions, and simulations
- Evaluation management & structure
- Evaluation team training
- Evaluation team responsibilities and procedures
- Evaluation reporting and documentation
- Administrative and logistical support
- Communications procedures and support
Final Exercise Report
A major multi-agency exercise can be both costly and time consuming to arrange and undertake. A final exercise report should be compiled as soon as is practical after the debrief in order to provide feedback to participating organizations on their performance during the exercise. The report should contain the goals, objectives and planned outcomes of the exercise, along with an outline of the scenario and the planning process. The report should also contain an evaluation section in which positive and negative observations are recorded and recommendations made. Needed improvements that have been agreed to are noted to help in converting lessons learned from the exercise into concrete, measurable steps that result in improved capabilities. This report is used as a basis for the revision of plans and procedures, and in determining future training needs.