Skip to content
KPI Systems and Balanced Scorecard | Part 6 of 8

Where the Numbers Come From: Collection and Reliability

From a system, by hand or by sample. Each method has its own error sources, its own effort and its own check, and every figure needs a recount from raw data.

A figure is only as good as the way it is produced. Three collection methods cover almost everything in a plant, and each one fails in its own way.

Three methods

From a system. Read from ERP, MES or machine data through a saved report, not retyped. Cheap per reading and easiest to scale. It fails through a wrong filter, through master data changed without notice, and at the shift boundary, where report and shift book disagree about which day a value belongs to.

By hand. Counted or read at the line and entered once a day or week. It fails through missed entries at shift change, through rounding, and through entries made from memory at the end of the week.

By sample. An audit or a survey with a defined sample size and method. It fails through a sample that is too small, through always sampling the same shift, and through self-selection in surveys.

In the example plant, of 24 figures, 14 come from a system, 7 are collected by hand and 3 by sample. Before the cut, 89 of 146 figures were collected by hand.

What collection costs

Effort is invisible until somebody adds it up. Minutes per reading times readings per period is enough.

Group Hours per month
From a system, 14 figures 6.5
By hand, 7 figures 10.2
By sample, 3 figures 3.9
Total, 24 figures 20.6

Before the first cut the same plant spent about 55 hours per month counting and typing. The reduction came from fewer figures and from moving six of them out of manual collection.

Choosing the rhythm

A figure measured more often than it can be acted on produces noise and meetings. A figure measured less often than the decision cycle arrives too late.

Daily suits figures a team can react to within the shift: plan attainment, defects by type, the OEE of the bottleneck. Weekly suits decisions made within the week: on-time delivery, schedule attainment, scrap cost. Monthly suits figures that need a closed period or a survey: conversion cost per unit, absenteeism, the satisfaction score.

The example plant measures 4 figures daily, 11 weekly and 9 monthly.

The reliability check

Twice a year, five figures are recomputed from the raw data, with their owners, independently of the report.

  1. Pick five, two from systems, two by hand, one by sample.
  2. Recompute the last closed period from the raw data.
  3. Compare. A difference above the tolerance of the figure is a finding, not a rounding.
  4. Write the cause on the definition sheet under known error sources and fix the collection.
  5. Repeat with five other figures in six months, so that every figure is checked within three years.

The first check in the example plant found two of five values wrong. On-time delivery had been measured on the ship date instead of the agreed arrival date, which flattered the figure by 2.6 points. Scrap cost had excluded the rework hours booked to a separate cost center. Both were corrected, dated as a new version of the definition sheet, and communicated before the next review.

Two findings out of five is normal for a first check. A check that finds nothing usually means the recount was not independent.

Write the error sources down

Every definition sheet carries a short list of the ways its value can go wrong. For first pass yield: units adjusted at the station and passed on without an entry. For the OEE of a line: planned time wrongly set, micro stops not counted. For a survey score: a sample that is too small and always the same contacts.

That list is what lets the next person reproduce the number, and the first place to look when a figure jumps without a reason.

The next part shows the trusted numbers in a way that can be read, and turns the reading into a decision.

Newsletter

One Lean idea.
Every two weeks.