Question: Measurement of Situational Awareness

[nmap]

About 7 weeks ago, I posted a thread about how to develop situational awareness; there were many superb responses, and I appreciate every one of them. LINK to S.A. Thread

I’ve entered into a college class on reflective leadership, and a primary project for the semester permits me to attempt to improve my S.A. (Yes, I know I’m rather old for this sort of thing, but it’s worthwhile to keep the mind active.).

The challenge I face now is how to demonstrate – or, if you prefer, measure – improvements in situational awareness. Some ideas I’ve had include getting a partner to show me a photograph for one minute – I would be tasked with writing down significant details. Alternatively, a video might be used in similar manner. Finally, there is the option of the classic police academy exercise of a staged argument, followed by writing up the events.

Let me hasten to add that I am not engaged in academic dishonesty. I’ve saved all the comments from the thread, and have showed the results to the professor. I’ve even quoted NDD (with attribution, of course!).

This also has a practical use. If we suppose that S.A. is a skill that can be developed, there ought to be a way to measure it. On the other hand, if it’s something that cannot be developed, there should be some way to measure the difference. Might I add that if it is subject of improvement, the ability to measure implies the ability to improve the training to get better results.

So…any ideas?

[The Reaper]

Kim's Game.

TR

[Razor]

If you don't mind going a bit more elaborate, you can enlist a small group of people to carry on a discussion that doesn't actively include the 'test subject' (the one demonstrating his level of SA). Give the discussion group a topic, and instruct them to talk to each other about the topic indirectly, without specifically naming or describing the topic. The testee then listens to the conversation and watches the body language of the discussion group, and is asked after a short period of time to name the topic of the conversation. It forces one to use verbal hints, non-verbal cues and incorporate context to gather information. You can go one step further and ask the test subject to attempt to smoothly inject himself into the conversation as an outsider of the group.

[nmap]

Thank you, TR and Razor, for the good ideas!

I located a piece in the library - notice the similarity between the approaches you've both mentioned, and the excerpt:

SAGAT is an objective measure of SA. SAGAT
employs periodic, randomly-timed freezes in a simulation
scenario during which all of the operator's displays are
temporarily blanked. At the time of the freeze a series of
queries are provided to the operator to assess his or her
knowledge of what was happening at the time of the freeze.
For instance, queries may ask a military pilot where other
aircraft are located, which aircraft pose a threat, ownship
airspeed or altitude and whether a given aircraft is hostile. The
queries typcially cover SA elements at all three levels of SA
(perception, comprehension and projection).

[snip...]

Operator's responses to these queries are scored based on what was
actually happening in the simulation at the time of each freeze
(within operationally determined tolerance zones).

This is all from Endsley, M. R., Stephen, J. S., Thomas, D. H., & Darryl, G. C. (1998). A comparative analysis of SAGAT and SART for evaluations of situation awareness. Proceedings of the Human Factors and Ergonomics Society ... Annual Meeting, 1, 82. (no link available)

Anyway, your ideas are a lot more practical. Thank you again!