How We Work

The methods and techniques we use come from the study of cognitive work in high-consequence, high-tempo domains. Primarily, these are qualitative research methods and include:

cognitive work analysis (CWA)
cognitive task analysis (CTA)
process tracing

These methods and techniques were developed to characterize complex work domains with deep technical structures and high-risk boundaries. The methods draw on carefully-developed techniques for the study of real work and tested in aviation, nuclear power, medicine, and military systems.

In many cases, these methods are used to reconstruct what people do and say during an incident and also during the after-incident reviews. The approach also uses other data collections, (previous post-incident reports, logged and known follow-up tasks, “runbooks,” code and/or architecture documentation, chat or audio/video transcripts, etc. as probes during our interviews.

These data are used to triangulate accounts and perspectives. The results are used to draw conclusions about knowledge, perspectives, confidence, assessments, etc.

For example, here is a diagram outlining how we use this approach in doing our Assessment projects:

For more information about these methods and approaches and examples of them being used, consider these resources and studies:

Albolino S, Cook R, O’Connor M (2007). Sensemaking, safety, and cooperative work in the intensive care unit. Cogn Tech Work 9: 131-137.

Allspaw J. (2015) Trade-Offs Under Pressure: Heuristics And Observations Of Teams Resolving Internet Service Outages. Masters Thesis. Lund, Sweden: Lund University.

Cook RI, Woods DD (1994). Operating at the sharp end: The complexity of human error. In Bogner MS, ed., Human error in medicine. Hillsdale, NJ: Lawrence Erlbaum. pp. 255-310.

Crandall, B., Klein, G., & Hoffman, R. R. (2006). Working minds: A practitioner’s guide to cognitive task analysis. Cambridge, MA, US: MIT Press.

Flanagan JC (1954). The Critical Incident Technique. Psychological Bulletin. 51:327 -358.

Johannesen L. (2008) Maintaining Common Ground: An Analysis of Cooperative Communication in the Operating Room. In Nemeth CP, ed. Improving Healthcare Team Communication: Building on Lessons from Aviation and Aerospace. Burlington VT: Ashgate Press, 179-203.

Johannesen L, Cook RI, Woods DD. (1994) Grounding Explanations in Evolving, Diagnostic Situations. Houston, TX: NASA-Johnson Space Center, 1994. [NASA-CR-198027]

Klein, G. A., Calderwood, R., & Macgregor, D. (1989). Critical decision method for eliciting knowledge. IEEE Transactions on systems, man, and cybernetics, 19(3), 462-472.

Klein G, Feltovich PJ, Bradshaw J., Woods DD. (2005) Common Ground and Coordination in Joint Activity. In Rouse WR, Boff KB, eds. Organizational simulation. New York: Wiley, 2005, 139-184.

Klein, G.A., & Hoffman, R.R., (1992) Seeing the Invisible: Perceptual-cognitive aspects of expertise. In M. Rabinowitz (Ed.), Cognitive Science Foundations Of Instruction (pp. 203-226). Mahwah, NJ: Erlbaum

Nemeth CP, Cook RI, Woods DD. (2004) The Messy Details: Insights From the Study of Technical Work in Healthcare. IEEE Transactions on Systems, Man, and Cybernetics – Part A: Systems and Humans. 34: 689-692.

Nemeth C, Kowalsky J, Brandwijk M, Kahanna M, Klock AP, Cook RI (2008). Between Shifts: Healthcare Communication in the PICU. In Nemeth C, ed. Improving Healthcare Team Communication: Building on Lessons from Aviation and Aerospace. Burlington VT: Ashgate, 135-153.

Rasmussen J, Lind M. (1981) Coping with complexity (Report RISØ-M-2293). Roskilde, Denmark: Risø National Laboratory.

Sarter N (2006). The Critical Incident Technique: A Method for Identifying System Strengths and Weaknesses Based on Observational Data (Chapter 594). In Karwowski W, ed., International Encyclopedia of Ergonomics and Human Factors, (Second Ed.). CRC Press.

Shattuck LG, Woods DD (1994). The Critical Incident Technique: 40 Years Later. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 38: 1080-1084.

Watts J, Woods DD, Patterson ES. (1996) Functionally distributed coordination during anomaly response in space shuttle mission control. Third Annual Symposium on Human Interaction with Complex Systems, 1996. (HICS ’96 Proceedings). Dayton, OH: IEEE, 1996, 68-75.

Watts-Perotti, J. and Woods, D. (2009). Cooperative Advocacy: An Approach for Integrating Diverse Perspectives in Anomaly Response. Computer Supported Cooperative Work (CSCW), 18(2-3), pp.175-198.

Wears, R. L., & Webb, L. K. (2011). Fundamental On Situational Surprise: A Case Study With Implications For Resilience. In Hollnagel, E., Rigaud, É., & Besnard, D. (Eds.), Proceedings of the fourth Resilience Engineering Symposium: June 8-10, 2011, Sophia Antipolis, France. Presses des Mines.

Woods DD. (1994) Cognitive demands and activities in dynamic fault management: abductive reasoning and disturbance management. In Stanton NA, ed. Human factors in alarm design. Bristol, PA: Taylor & Francis, 63 – 92.

Woods DD. (1988) Coping with complexity: the psychology of human behaviour in complex systems. In Goodstein LP, Andersen HB, Olsen SE, eds. Tasks, errors, and mental models. Bristol, PA: Taylor & Francis, 1988, 128-148.

Woods DD. (1993) Process Tracing Methods for The Study of Cognition Outside of the Experimental Laboratory. In Klein GA, Orasanu J, Calderwood R, Zsambok CS, eds. Decision making in action: Models and methods. Westport, CT: Ablex Publishing, 228-251, 1993.

Woods DD, Dekker SWA, Cook RI, Johannesen L, Sarter N. (2010) Behind Human Error, 2nd Edition. Farnham, UK: Ashgate Press..

Woods, D. & Hollnagel, E. (2006). Anomaly Response, in Joint Cognitive Systems : Patterns in Cognitive Systems Engineering. (pp. 69-95) Boca Raton: CRC/Taylor & Francis.