6.5.3 Learnability measures

What this heading covers

Learning can be measured by time to competence, by the proportion of users who complete a task on the first attempt, or by the help needed to get there. Each is legitimate and they answer different questions. The population and the starting knowledge matter: an expert user learns a new product faster, and a study of experts says little about new users. The measure should name the task, the user group and the definition of learned that was used.

Record position: family 10 of 40 in clause 6, under 6.5 Interaction capability, which carries 8 families. It is one of the 40 families a programme has to map its measures to, and its elements are listed below. The heading is 6.5.3 in the public contents; the text behind it is not public.

How a measure for this family is built

Count users who reached the defined competence on the first attempt as the numerator over the users in the study. If effort is the quantity, measure the time or the number of help requests to reach competence and report the median with the spread. Keep the task simple and representative, and record prior experience.

What to put in place before publication

1. Define competence before the study, not after seeing the results.
2. Record prior experience for each participant.

Evidence to keep

Two questions to ask

• How was learned defined before the study?
• Were any participants already trained on the product?

Where teams usually fall short

• A learnability score from a training session the participants had already attended.

What the number cannot tell you

It cannot tell you how much help they needed. Two users can both reach competence, one alone and one with a trainer, and the measure treats them the same.

What happens next

Operability is what a user does once the product is learned. The two measures often diverge.