Practice

A confirmation prompt tests attention, not intention

Twenty-five million warnings were measured in the field. The best-performing ones stopped most people; the worst were bypassed seven times in ten. What none of them could do was tell a correct decision from a confident mistake.

Akhawe and Felt instrumented Chrome and Firefox and recorded 25,405,944 warning impressions. Clickthrough on malware and phishing warnings ran between nine and twenty-three per cent; on Firefox's SSL warning, thirty-three per cent; on Chrome's, 70.2 per cent. The spread shows that warning design changes behaviour substantially. It also marks the ceiling: every one of those warnings was asking a question only an inattentive person can fail.

Two failures lead to an unwanted action, and a prompt can detect one of them. In the first the person intended something else and the action departed from the intention — a misdirected click, a wrong row, a slip in Reason's terminology, where the plan was sound and the execution was not. Against this a prompt performs respectably: the interruption breaks the automatic sequence, the action is named back, and a person who reads the name recognises the mismatch. In the second the person intended exactly this action, on a belief that happens to be wrong. Reason calls that a mistake, and the check the prompt imposes has already been performed internally, to the person's satisfaction, before the prompt appeared. The prompt invites a review they have no reason to conduct.

So the prompt samples attention. It asks whether somebody is present enough to notice a discrepancy, and answers usefully where a discrepancy exists to be noticed. Where the discrepancy is between the world and the person's model of it, the prompt has access to neither.

The field data is usually cited for the high figure and read as showing that warnings do not work. The authors drew closer to the opposite conclusion, and it is the more useful one: the variation across warnings is large, so design has substantial purchase on the outcome. The variation also shows what the design is working on. Firefox required three clicks to bypass and allowed stored exceptions; Chrome required one. Clickthrough tracked effort and frequency of exposure rather than anything about the correctness of the underlying decision. Almuhimedi and colleagues later found clickthrough on Chrome's malware warning varying with the person's browsing history, which is a statement about exposure, not judgement.

Exposure also degrades the mechanism in a way no prompt can design around. A warning shown on every occurrence of a condition, most of them benign, trains the response that clears it, until the response is part of the action sequence rather than an interruption to it. That gives prompts a property unlike most interface decisions: each additional one reduces the effectiveness of the others. A product with one confirmation at its single irreversible step has an instrument. A product with eleven has a ritual.

Checks that survive a confident mistake all require information the mistaken belief cannot supply. Asking somebody to type the name of the item being deleted requires them to look at what is actually selected; the belief that the right row is selected does not generate the characters, the row does. Delay with reversal gives the confidently wrong person what a prompt never can — the consequence, visible, while correction is still cheap. Separation removes the slip entirely, and with it the need for the prompt that guarded against it.

Sources

Akhawe, D., and Felt, A. P. (2013). Alice in Warningland: A large-scale field study of browser security warning effectiveness. Proceedings of the 22nd USENIX Security Symposium, 257–272. Impression count, clickthrough ranges and the SSL figures are as reported there, as is the observation about click counts and stored exceptions.

Almuhimedi, H., Felt, A. P., Reeder, R. W., and Consolvo, S. (2014). Your reputation precedes you: History, reputation, and the Chrome malware warning. Proceedings of the Tenth Symposium on Usable Privacy and Security.

Reason, J. (1990). Human Error. Cambridge University Press. Source of the slip/mistake distinction.

All sources are paraphrased; nothing above is quoted. The account of habituation reflects the general warning-science literature rather than a specific quantitative result, and is stated without a figure for that reason.

← All writing