Adaptive Capacity Labs

Process and Scope of an Assessment

The assessment project uses an iterative approach to explore the relationship between incidents, incident responses, post-incident analysis and synthesis, and how the resulting learning is captured and retained across the workplace. The process is case-based and uses incidents as cases. Iteration brings the critical elements to the fore quickly and generates focus.

We typically anticipate four cycles of activity, approximately two weeks per cycle.

Cycle 1

Initial exploration of existing practices and state of learning from incidents​​

Cycle 2

Deep-dive on 1-2 incidents, tracing their influence on anticipation/prep for future events

Cycle 3

Focused exploration based on Cycle 2 findings

Cycle 4

Focused exploration based on Cycle 2/3 findings, and synthesize results into artifacts and/or training.

Cycle 1: Exploration and mapping

The first iteration is used to trace and map the formal, nominal, and typical after-incident analyses and sharing processes across a group of incidents, focusing on recent events.

During this iteration, we review:

    • How incidents are identified, managed, and logged
    • How the individuals and groups responsible for management analyze, makes sense of, and classify incidents
    • How this information flows into formal and informal incident handling systems
    • How further investigation and development of countermeasures takes place
    • How countermeasures are prioritized
    • How these activities vary depending on severity, complexity, and urgency

The result of this work is a map of incident handling and an incident collection that characterizes the recent company experience.

Cycle 2: Deep dive

Cycle 2 will identify opportunities for a deep look at one or two events, and explore how preparation for upcoming events and future designs are formed. Planning for this activity will occur at the end of Cycle 1.

The results of Cycle 2 will be used to plan the following cycles.

Cycles 3 and 4:

Detailed planning for Cycle 3 and 4 occurs at the end of Cycle 2.

Some typical candidate areas for attention are:

    • What are the current sources of insight and hypothesis generation during and after incidents?
    • How are countermeasures developed and chosen?
    • Where is the “memory” of important past events?
    • What themes are present in (and absent from!) post-mortem reviews?
    • How does anticipating future events shape the work to prepare for them?
    • What can an organization do to build capacity around post-incident debriefings (“post-mortems”)?
    • What makes an incident a good case for post-incident review?
    • What tools are needed to make post-event reviews efficient?
    • What influence (positive or negative) do post-incident review have concerning bringing new employees up to speed with current risks, esoteric knowledge of systems behavior, etc.?
    • How does authority and/or responsibility migrate in the midst of responding to an incident?

 

Incidents can have a significant influence on long-term business decisions, create uneven perceptions of risk across teams, and create barriers to the velocity and robustness of change that a company needs to stay competitive. Tracing this often invisible influence is part of our assessment.

Even though the focus of the project is an assessment, this project itself tends to create momentum for change within an organization:

    1. Focusing attention on learning from incidents sends a powerful message to the team about the importance of post-incident review.
    2. Engagement between ACL and engineering staff in dev and ops will generate interest in related topics, new routes of expression, and energy that can be used to move forward in this area.

Ongoing opportunities during the project

During the cycles, there might be opportunities to capture data from new incidents. Our continued presence at the company makes it possible to take advantage of these opportunities to observe the response to incidents, the after-incident reactions, the production of specific countermeasures, and the dissemination of lessons learned from these ‘live’ events.

Checkpoints

Naturally, we stay in contact throughout the project with stakeholders. There are usually short progress/midcourse correction meetings near the end of each cycle. These sessions keep the leadership team up to date on the project.

Contact us