Practice
Training is a control of last resort
The safety world has a hierarchy of controls. Covering it in onboarding sits at the bottom of that hierarchy, and it is the first thing most design teams reach for.
Risk-control standards rank warnings, manuals and training last, and agree on why: such measures work only where the person retained the instruction, has it to hand at the moment of decision, and has not lost it in between. Decay after a year of nonuse is measured at d = −1.4. Most of what product teams record as mitigation belongs to that tier under another name.
The hierarchy of controls is not a controversial document. Elimination, substitution, engineering controls, administrative controls, protective equipment: five tiers, settled decades ago by industries required to defend their reasoning in public, with training and procedure placed fourth. Nothing in the ordering is a matter of taste, and nothing in it was arrived at cheaply.
NIOSH publishes the version most often reproduced, and gives the reason for the sequence plainly. The upper three tiers "control exposures without significant human interaction". The lower two, training among them, "require significant and ongoing effort by workers and their supervisors". A control that must be remembered before it operates borrows the memory of whoever is expected to remember it, and fails on the day that memory is under load.
Machinery safety arrives at the same place with fewer tiers. ISO 12100 sets out three steps — inherently safe design, then safeguarding and protective measures, then information for use, where warnings, manuals and training all sit. Medical devices follow an equivalent sequence in ANSI/AAMI/ISO 14971, and the FDA's human factors guidance, summarising that standard, declines to be diplomatic about it: "Design modifications to the device and its user interface are generally the most effective means for eliminating or reducing use-related hazards."
The tier counts differ; the bottom of each is the same place. Where a design puts an error-prone step in front of a person, the remedy that counts is a change to the step. Instruction is what remains once the ways of changing it have run out.
The mechanism
The FDA guidance is unusually precise about how information-based controls fail, which is what makes it useful well outside medicine. They rank last, it says, because "they rely on the user to remember or refer back to the information, labeling might be unavailable during use, and knowledge gained through training can decay over time". Three conditions, and all three must hold at once. The instruction has to have been retained. It has to be present at the point of decision. And the interval between the two has to have been short enough for the retention to survive it.
The third has been measured. Arthur, Bennett, Stanush and McNelly's 1998 meta-analysis in Human Performance drew 189 independent data points from 53 articles on trained skills that then went unused. Immediately after training, the effect size for decay is d = −0.01. After more than 365 days of nonuse it is d = −1.4. Cognitive tasks decayed faster than physical ones and accuracy-based tasks faster than speed-based ones; between them, a fair description of most work done with software.
The scope of that finding is easily overstated, so it is worth being exact. It concerns nonuse, not fallibility. It says nothing about a task performed several times a day, and everything about the annual task, the once-per-account task, and the recovery task attempted when something has already gone wrong. Those are the tasks onboarding is most often asked to carry, and they are the tasks for which instruction has been measured to work least well.
Substitutes
Teams rarely propose training under that name. They propose a tooltip, an empty state, a help article, a recorded walkthrough, a line in the release notes, a briefing for the support desk, a confirmation dialogue. Each is information for use, and each occupies the tier the training session would have occupied. None is worthless — the standards that rank information lowest also require it to be there, and ISO 12100 treats information for use as a mandatory step rather than a discretionary one. But a hazard against which a tooltip has been written has been documented, not mitigated.
The confirmation dialogue deserves separating from the rest, because of all of them it impersonates an engineering control most convincingly. It sits in the flow of work, it interrupts progress, it presents as a barrier. What it does is ask somebody who has already resolved to proceed whether they wish to proceed. Against the user who has slipped it works tolerably. Against the user who is confidently mistaken it does very little, because the check it imposes is one that person has already carried out to their own satisfaction — the failure human factors calls premature closure. Requiring the value to be entered again independently is an engineering control. Putting the destructive action somewhere other than beside the routine one is an engineering control. A dialogue asking whether the user is sure is a warning notice with a button beneath it.
Proportion
The objection raised at this point is one of proportion. Nobody is harmed when an appointment is cancelled in error, and the apparatus of nuclear safety sits oddly on a settings page. The stakes are lower. They are also not what the hierarchy ranks.
What it ranks is the reliability of each class of control. The mechanism it describes — a measure requiring the person to remember, at the right moment, unprompted — behaves the same way whether the consequence is a reactor trip or a subscription cancelled by mistake. Severity governs how much ought to be spent on a remedy. It has no bearing on which remedies work.
Two circumstances fall outside the argument. Irreducible domain knowledge is one: a radiographer must understand radiography, and no interface will supply it. Sustained daily use is the other, where repetition maintains the skill and the interval on which decay depends never opens — expert tools, used constantly, by fluent operators. Instruction is worth a great deal in both. In neither is it the thing standing between a foreseeable error and its consequence.
That leaves the infrequent task, the first-run task, and the recovery task attempted under pressure by somebody who last considered the product at the previous incident. For those, training is the control chosen because it was the cheapest to write down. Reason's distinction between the person approach and the system approach is the same observation at a different scale: retraining and exhortation have the appearance of action, and leave the conditions that produced the error exactly as they were.
Control at the level of design
A design-level control alters what the product does rather than what it says. The step responsible for the error is removed. Measured state is shown in place of commanded state, so that a disagreement between the two is visible rather than remembered. The destructive action is moved away from the routine one. The dangerous option is made unavailable for as long as the condition that makes it dangerous holds, in the way a permit system locks out a control rather than reminding an operator not to touch it. A value is entered independently rather than confirmed. Each of these sits a tier above any warning that might have been written instead, and each can be checked afterwards by somebody who was not in the room when the decision was taken.
The audit is straightforward enough. For the last several risks a team has closed, the tier each remedy actually occupied can be recorded as an outside reviewer would classify it rather than as it was intended. Where nothing rises above information for use, that is the finding, and it is worth having in writing.
Descending the hierarchy on purpose remains a legitimate engineering decision, and it has a name. A residual risk, accepted knowingly with the reason recorded, can be defended years later by whoever inherits it. That is the standard the regulated industries hold themselves to, and it is not an onerous one.
What will not survive the same scrutiny is the tooltip written against a hazard and the matter marked closed. It leaves the error where it was, transfers responsibility for catching it to the person least equipped to carry it, and records the transfer as a mitigation.
Sources
NIOSH, Hierarchy of Controls. Centers for Disease Control and Prevention. https://www.cdc.gov/niosh/hierarchy-of-controls/about/index.html
ISO 12100:2010, Safety of machinery — General principles for design — Risk assessment and risk reduction. Three-step method: inherently safe design, safeguarding and protective measures, information for use.
U.S. Food and Drug Administration (2016), Applying Human Factors and Usability Engineering to Medical Devices: Guidance for Industry and Food and Drug Administration Staff, Section 7. Both quoted sentences are from this guidance. The ANSI/AAMI/ISO 14971 ordering is as summarised there, not quoted from the standard itself.
Arthur, W., Bennett, W., Stanush, P. L., and McNelly, T. L. (1998). Factors that influence skill decay and retention: A quantitative review and analysis. Human Performance, 11(1), 57–101.
Reason, J. (2000). Human error: models and management. BMJ, 320(7237), 768–770.
Standards and guidance are quoted directly where quotation marks appear; American spellings within quotations are as published. Reason is referenced rather than quoted. Effect sizes are as reported in the source meta-analysis.