Measuring Self-Awareness

Over at Slate is an interesting interview with Dylan Evans, who has coined the term "risk intelligence", which really turns out to be a kind of self-awareness regarding what one does and does not know.

[Risk intelligence] is the ability to estimate probabilities accurately, it's about having the right amount of certainty to make educated guesses. That's the simple definition. But this apparently simple skill turns out to be quite complex. It ends up being a rather deep thing about how to work on the basis of limited information and cope with an uncertain world, about knowing yourself and your limitations.

In my limited exposure to the concept, it seems to be mis-named, and I would guess it might be because the author does not regard absolute certainty as possible for philosophical reasons.

Intrigued that someone seemed to be attempting to measure a quality I have often thought others to be lacking in, I took the test. Here is the writeup of my results.

The RQ score ranges from 0 (low RQ) to 100 (high RQ). Your score is 79.57. Such a score is high. Risk intelligence can be measured by calculating something called a "calibration curve". The red line displayed to your right is your calibration curve. A perfect calibration curve would lie exactly on the blue diagonal line, so the area between the curve and the diagonal would be zero. Nobody is perfectly calibrated, but people with high risk intelligence come very close to this ideal.

By now, you may have realized there is an easy way to game this test. If you always select the 50% category unless you are pretty certain that a statement is true or false - and if the test contains equal numbers of true and false statements - you will score very highly, perhaps very near 100.

What did I find myself doing during the test? I considered why I agreed or disagreed with a given statement -- or thought I had no basis for doing so, for the questions I realized I knew nothing about. In doing so, I usually ended up with some rough estimate of how confident I was in my (dis)agreement, as well as what else I would need to know to be more sure one way or the other. This was really an attempt to use a percentage as a metaphor for what I thought my level of knowledge about a topic was, rather than an actual estimate of a probability.

Based on past experience, I think it is possible for people with this kind of self-awareness to appear to be less confident than they actually are. This can happen when one communicates his level of uncertainty poorly (or too conservatively) or when one is dealing with someone who lacks this form of self-awareness (and hence equates certainty with confidence or views admitted uncertainty with suspicion). I can even recall doing the former while in the latter situation quite a few times when I was younger. (This ultimately culminated in me being ignored after correcting someone I absolutely knew to be wrong! Fortunately, the stakes were low.)

-- CAV