Design Verification

Why 95/95? How to Set a Reliability Target You Can Defend

Reliability and confidence targets for design verification: from harm severity to a standing severity-to-target mapping.

NPItoProductGuy

NPItoProductGuy

— 5 min read
Share

Part 2 of the Defensible V&V series. Previously: Part 1, 59 or 8? How to Derive a V&V Sample Size You Can Defend (https://npitoproductguy.com/59-or-8-vv-sample-size-you-can-defend/).
Reliability and confidence targets for design verification: from harm severity to a standing severity-to-target mapping.
Part 1 of this series showed that a sample size must be derived, not chosen. But every derivation starts from a target, and "95/95 is what we always use" fails the same audit as "59 is what we always test." This article covers the first link in that chain: where the reliability and confidence targets come from, and how to document them so your next review goes differently.

The question behind the number

Every verification plan states a target near the top of its first page: demonstrate 95% reliability at 95% confidence. It is so familiar that most teams stop seeing it. Then a reviewer asks why 95/95, and the room discovers the number has no source. Nobody chose it; it accumulated. A target without a source has the same flaw as a sample size without a derivation: it can be repeated, but not defended.
The target is not a statistical choice. It is a risk decision expressed in statistical terms, and it must trace back to the risk management file.

What the two numbers mean

The two numbers answer different questions. Reliability R is a claim about the product: the minimum proportion of units that conform to the requirement, under the conditions the test represents. Confidence C is a claim about the demonstration: how strongly the sample evidence supports that R was achieved. Formally, C is the confidence level of a lower bound on R.
In one sentence: we want 95% confidence that at least 95% of units conform.
Two clarifications prevent most errors here:

  • R covers only the conditions you tested. If the requirement must hold at end of life, test units that represent end of life: aged, cycled or conditioned. A 95/95 result on fresh units says nothing about year five.
  • R and C are not interchangeable. Raising C makes you more certain about the same claim; raising R makes the claim itself stronger. Reviewers probe this distinction directly.

The chain from harm to target

The target is derived from risk, and every link is written down:

  1. Severity of harm. Which harm does this requirement protect against, and how severe is it? Take severity from the risk analysis, not from an RPN score. If the requirement controls several hazards, the most severe one sets the target.
  2. Role in risk control. Is the requirement a risk control measure, and is it the only barrier between the hazard and the harm? A sole control justifies a higher target than one backed by independent layers.
  3. Risk acceptability. The risk management policy defines which residual risks are acceptable. Severity, together with the control's role, determines how reliable the requirement must be.
  4. Confidence. How much statistical assurance does the consequence justify? Higher severity raises R first. It raises C as well when a wrong demonstration would itself be unacceptable.
  5. Written down once. The mapping lives in a procedure as a standing table from severity class to R/C target, approved before any project starts. It is not renegotiated in each protocol.
    If the target is set in the protocol, it is an opinion. If it is set in the procedure, it is policy.

An example mapping (fictional)

Every cell in the standing table has a price in units before a single test is planned. The last column shows that price for the binomial success-run route with zero failures allowed:

Severity (per risk analysis) Example harm R / C target Units, zero failures
Negligible Transient discomfort, no treatment needed 80% / 90% 11
Moderate Minor, reversible injury 90% / 90% 22
Serious Injury requiring medical intervention 95% / 95% 59
Critical Life-threatening or irreversible harm 99% / 95% 299
This table is a fictional example, not a recommendation. Each organization builds its own mapping from its own risk acceptability policy, and quality management approves it. Note the pattern: severity raises reliability first, and confidence rises only where a wrong demonstration would itself be unacceptable. Some organizations go further, requiring 99% confidence for critical requirements or adding a column that separates a sole risk control from a layered one.
Requirements with no link to any harm, such as purely cosmetic ones, do not belong in this table. Their targets come from customer or business requirements and are documented the same way.

The target sets the sample size

Move one severity class down, from serious to moderate, and the test shrinks from 59 units to 22. Move from negligible to critical and it grows from 11 to 299, a factor of 27. On the variables route, the target sets the k factor instead, and with it the margin your process needs.
That is why the mapping is a business decision owned by quality management, not something each project improvises. A team that quietly applies 80/90 to a serious harm buys a cheap test and an indefensible file. A team that applies 99/95 to a negligible harm spends budget nobody asked for. The standing table prevents both.

Worked example (fictional)

Return to the fictional component from Part 1. It has two verified requirements. The first is its 45 N output-force limit, which protects against a serious harm recorded as hazard HZ-07 in the risk file. The procedure maps serious severity to 95/95, so the protocol inherits that target, and the protocol needs either 59 units with zero failures on the success-run route or, as Part 1 showed, 8 measured units on the variables route. The second requirement protects against a negligible harm, transient discomfort, so its target is 80/90 and 11 units suffice.
Same component, two targets, both correct. Each test's rationale cites the same chain: the hazard ID, the severity class from the risk analysis, the version of the mapping table, and the resulting sample size. Those citations, not the numbers, are what survive the audit.

Common failure modes

  • The target appears in the protocol with no source. The reviewer asks why 95/95, and the room goes quiet.
  • Two projects, same severity, different targets. Inconsistency across files is a finding in itself, even when each target looks reasonable on its own.
  • Severity taken from an RPN score instead of the risk analysis. An RPN combines severity with occurrence and detection; the mapping needs severity alone.
  • A requirement that controls several hazards is mapped to the wrong one. The most severe hazard sets the target.
  • Fresh units tested for an end-of-life claim. The demonstrated R covers only the conditions tested.
  • The mapping exists, but a project overrides it after seeing the cost. That is changing the plan after seeing the result, and the warning from Part 1 applies to targets too.

Where this sits in the regulations

ISO 14971:2019 requires criteria for risk acceptability in the risk management plan, and ISO/TR 24971:2020 gives guidance on setting them. The severity-to-target mapping is where that policy meets verification planning. ISO 13485:2016, clauses 7.3.6 and 7.3.7, requires verification and validation plans to include statistical techniques with a rationale for sample size, and that rationale starts from a justified target. Since February 2, 2026, the FDA's Quality Management System Regulation (QMSR) has incorporated ISO 13485:2016 by reference, so the same expectation applies to devices sold in the US.
No regulation will hand you a target. What the regulations require is a target that is defined, justified and applied consistently. The standing mapping table meets that requirement on a single page a reviewer can sign.

Tools

Read more