6.6.4 Fault tolerance measures

What this heading covers

The evidence has to be a failure, not a design. A redundant architecture is a claim until a component is stopped and the product keeps working. The scenarios should be the ones the product is designed for, and they should include the failure of a dependency as well as an internal component. The measure should record the effect on users, because a product that continues in a degraded mode may still fail the task.

Record position: family 19 of 40 in clause 6, under 6.6 Reliability, which carries 4 families. It is one of the 40 families a programme has to map its measures to, and its elements are listed below. The heading is 6.6.4 in the public contents; the text behind it is not public.

How a measure for this family is built

Count the failure scenarios in which the stated service continued within the stated degradation as the numerator over the scenarios tested. Name the component, the failure mode and the acceptable degradation. For a stricter measure, record the time to detect and the time to switch over.

What to put in place before publication

1. Write the acceptable degradation before injecting the failure.
2. Include a dependency failure, not only an internal one.

Evidence to keep

Two questions to ask

• Which component failure was injected, and what degradation was acceptable?
• Was a dependency failure included, or only an internal one?

Where teams usually fall short

• A failover described in the architecture document and never exercised.

What the number cannot tell you

It cannot tell you how often the failure happens. Tolerance is a property under a scenario, not a probability.

What happens next

Recoverability is the family for what happens after the failure wins.