Practice

Training is a control of last resort

The safety world has a hierarchy of controls. Covering it in onboarding sits at the bottom of that hierarchy, and it is the first thing most design teams reach for.

Risk-control standards rank warnings, manuals and training last, and agree on why: such measures work only where the person retained the instruction, has it to hand at the moment of decision, and has not lost it in between. Decay after a year of nonuse is measured at d = −1.4. Most of what product teams record as mitigation belongs to that tier under another name.

The hierarchy of controls is not a controversial document. Elimination, substitution, engineering controls, administrative controls, protective equipment: five tiers settled by industries required to defend their reasoning in public, with training and procedure placed fourth. NIOSH gives the reason plainly. The upper tiers control exposure without significant human interaction; the lower two require continuing effort from the people working under them. A control that must be remembered before it operates borrows the memory of whoever is expected to remember it, and fails on the day that memory is under load.

The FDA's human factors guidance is precise about the mechanism, which is what makes it useful well outside medicine. Information-based controls rank last because they depend on the person recalling the instruction, on the labelling being available during use, and on knowledge not having decayed in the interval. Three conditions, all of which must hold at once. The third has been measured: Arthur and colleagues, reviewing 189 data points on trained skills that then went unused, found decay reaching d = −1.4 beyond a year, with cognitive and accuracy-based tasks going fastest. That is a fair description of most work done with software, and it concerns nonuse rather than fallibility. It says nothing about a task performed daily and everything about the annual task, the once-per-account task and the recovery task attempted after something has already gone wrong — which are the tasks onboarding is most often asked to carry.

Teams rarely propose training under that name. They propose a tooltip, an empty state, a help article, a line in the release notes, a confirmation dialogue. Each is information for use, and each occupies the tier the training session would have occupied. The dialogue impersonates an engineering control most convincingly, because it sits in the flow of work and presents as a barrier; what it does is ask somebody who has already resolved to proceed whether they wish to proceed. Against a slip it works tolerably. Against a confident mistake it does little, because the check it imposes is one that person has already satisfied themselves on.

The objection is one of proportion, and it misreads what the hierarchy ranks. Severity governs how much ought to be spent on a remedy; it has no bearing on which remedies work. A measure requiring the person to remember, unprompted, at the right moment behaves the same way whether the consequence is a reactor trip or a cancelled subscription. Descending the hierarchy deliberately is a legitimate engineering decision with a name: a residual risk, accepted knowingly, with the reason recorded. What will not survive the same scrutiny is the tooltip written against a hazard and the matter marked closed.

Sources

NIOSH, Hierarchy of Controls. Centers for Disease Control and Prevention. The ordering and its rationale are paraphrased from this source.

U.S. Food and Drug Administration (2016), Applying Human Factors and Usability Engineering to Medical Devices, Section 7. The three failure conditions are paraphrased from this guidance.

Arthur, W., Bennett, W., Stanush, P. L., and McNelly, T. L. (1998). Factors that influence skill decay and retention: A quantitative review and analysis. Human Performance, 11(1), 57–101.

ISO 12100:2010, Safety of machinery. Three-step method: inherently safe design, safeguarding and protective measures, information for use.

All sources are paraphrased; nothing above is quoted. Effect sizes are as reported in the source meta-analysis.

← All writing