Skip to content
Measurement Systems and Gauge Management | Part 5 of 6

Attribute Checks: Do Your Inspectors Agree?

How to study a visual check or a go and no-go gauge with thirty parts, three inspectors and two rounds, and what agreement, misses and kappa tell you.

A visual check has no display and no certificate, so it is rarely treated as a measurement. It is one. It has two possible answers, and it varies between people and between Monday and Friday.

The check

At final inspection of the shaft line, every shaft is checked for scratches on the seal seat. A scratch across the seat causes a leak at the customer. The drawing says “no scratches”, the inspectors have years of experience, and complaints still arrive.

The study

  1. Select 30 parts: about a third clearly good, a third clearly bad and a third at the boundary. The boundary parts carry the information.
  2. The people who own the specification set a reference decision for each part. Here: 18 good, 12 bad.
  3. Three inspectors judge every part twice, in random order, under the conditions of daily work, without seeing earlier decisions.

That gives 180 decisions. Four counts per inspector are enough:

  • the same decision in both rounds,
  • both decisions equal to the reference,
  • bad parts called good,
  • good parts called bad.

The result

Inspector Same in both rounds Both equal to reference Bad called good Good called bad
A 29 of 30 28 of 30 1 of 24 2 of 36
B 27 of 30 25 of 30 3 of 24 4 of 36
C 25 of 30 22 of 30 6 of 24 5 of 36
All three 21 of 30 10 of 72 11 of 108

Each inspector made 24 decisions on bad parts and 36 on good parts. Only 21 of 30 parts were judged right by everyone in both rounds.

How to read it

Misses first. A bad part called good reaches the customer. The target in a study is zero. Ten misses in 72 decisions mean that one bad part in seven passes the check.

Agreement with the reference. Common practice expects 90 percent or more per inspector. Inspector A reaches 93 percent, B 83 percent and C 73 percent.

Agreement with yourself. Inspector C gave a different answer in the second round for 5 of 30 parts.

Kappa. Agreement of 80 percent sounds good until you remember that two coins agree half the time. Kappa compares the agreement with the reference to the agreement expected by chance. One is perfect, zero is chance. Values above about 0.75 are read as good, below about 0.40 as poor. The three inspectors reach 0.90, 0.76 and 0.62.

The parts. The rates say how large the problem is. The parts say what it is. Parts 7 and 12 were judged wrong by all three inspectors. The inspectors were not careless. The boundary was not defined.

The fix

The team laid the wrongly judged parts on a table and agreed with the customer where a mark becomes a scratch. Then they changed the conditions of the check:

  • four limit samples at the workstation that show the boundary,
  • a lamp with 1,000 lux instead of the hall light,
  • a magnifier and a fixed viewing sequence,
  • one hour of training with the 30 parts of the study.

The second study

All three inspectors Before After
All six decisions equal to the reference 21 of 30 28 of 30
Lowest agreement of one inspector 22 of 30 28 of 30
Bad parts called good 10 of 72 0 of 72
Good parts called bad 11 of 108 4 of 108

No bad part passed. The four remaining false alarms came from two good parts close to the boundary.

What applies to gauges too

A go and no-go plug gauge is studied in the same way, with parts measured beforehand so that the reference is known.

Limit samples age and disappear, so they belong in the gauge list with a number and a due date.

The last part of this series turns to the day every plant fears: a gauge is found out of tolerance.

Newsletter

One Lean idea.
Every two weeks.