Skip to content
Measurement Systems and Gauge Management | Part 3 of 6

Choosing the Gauge: Resolution, Permissible Error and Tolerance

Two shares show in one minute whether a gauge fits a tolerance, and the inspection plan at the workstation fixes the choice.

A digital caliper shows two decimals. It looks precise enough for a tolerance of ±0.05 mm. It is not, and the reason is printed in its data sheet.

Two shares

The first share compares what the gauge can show with what it has to judge:

resolution share = smallest step the gauge shows / tolerance width

A common rule of thumb asks for 10 percent at most and 5 percent or less where possible. The tolerance width is the upper limit minus the lower limit.

The second share compares what the gauge may get wrong with the tolerance:

error share = 2 × permissible error / tolerance width

The permissible error is the largest deviation a gauge of this type may show and still pass calibration. It is a plus and minus value, so the band is twice as wide. As a rule of thumb, this band should take 20 percent of the tolerance at most and 10 percent or less where possible. Both limits are practice, not a standard.

Eight characteristics

The shaft line has eight measured characteristics on a shaft and a bushing. The check took one hour.

Characteristic Gauge Verdict Resolution Error
Bearing seat 32.000 ±0.010 Micrometer Acceptable 5.0 % 20 %
Outer diameter 30.000 ±0.012 Micrometer Acceptable 4.2 % 17 %
Overall length 148.0 ±0.2 Caliper Acceptable 2.5 % 15 %
Shoulder width 12.00 ±0.05 Caliper Not suitable 10.0 % 60 %
Runout 0.030 maximum Lever indicator Acceptable 6.7 % 20 %
Bore 20.000 +0.021 Bore gauge Acceptable 4.8 % 19 %
Length 40.00 ±0.02 Caliper Not suitable 25.0 % 150 %
Flange thickness 10.00 ±0.05 Micrometer Good 1.0 % 4 %

One good, five acceptable, two not suitable. The caliper on the shoulder width passes the resolution rule exactly and fails the error rule by a factor of three: its permissible error of ±0.03 mm takes 60 percent of a tolerance of 0.10 mm. On the bushing length it takes more than the whole tolerance.

The plant replaced both. A micrometer with disc anvils now measures the shoulder width with an error share of 8 percent, and a micrometer measures the bushing length with 10 percent.

What the shares do not see

The two shares describe the gauge on paper. Four questions belong to the choice as well:

  • Can the gauge reach the measuring point? A micrometer next to a larger diameter measures somewhere else.
  • Is the measuring force defined? Thin walls give under the anvil. A ratchet or a fixed force helps.
  • Is the part clean and at room temperature? A steel shaft of 32 mm grows by about 0.004 mm when it is 10 degrees warmer.
  • Variable or attribute? A plug gauge is fast and says good or bad. It gives no trend.

A quick look at the spread

Before a full study, twenty readings show what the gauge does in a hand. One operator measured a setting master of 32.0000 mm twenty times with the micrometer of grinding cell G2. The average was 32.0007 mm, the standard deviation 0.0008 mm. Six standard deviations are 0.0048 mm, or 24 percent of the tolerance of the bearing seat.

One person and one part already use a quarter of the tolerance. The micrometer had passed both shares on paper. The paper check is necessary, but it is not proof.

Fix the choice at the workstation

The result belongs on the inspection plan at the machine: characteristic, gauge by type and place, measuring point with a picture, frequency, record and reaction. If the plan only says “measure diameter”, the operator takes the gauge that is closest. The inspection plan is the operator view of the control plan, which the series on FMEA and Control Plan covers.

One more field belongs on every check sheet: the gauge number. It looks like bureaucracy until a gauge fails.

The next part takes the micrometer of the bearing seat through a full study with ten parts and three people.

Newsletter

One Lean idea.
Every two weeks.