Open Gauge Documentation
CalibrationWorked examples

Example 1 — Temperature RTD, linear fit, full uncertainty budget

A two-calibration example exercising linear regression, all four uncertainty contributions at once, and all three decision rules.

Exercises: linear regression, all four Type A/B budget contributions at once (fit residuals, reference standard, resolution, sensor nominal accuracy), the reference-standard-uncertainty auto-fetch feature, and the three decision rules diverging on identical data.

Every number below was produced by running Open Gauge's real calculation engine against the input data shown, then independently re-derived by hand from the underlying formulas to confirm they match. If you enter the same setup and data through the UI, you should see the same numbers (up to the display rounding described at the end).

This example has two calibrations: first you calibrate a reference thermometer (so it has an expanded uncertainty on file), then you calibrate the working sensor against that reference — this is what makes Open Gauge auto-populate the "reference standard" budget row.

1.0 Setup — two assets

Asset A — reference standard (Assets → New Asset):

FieldValue
NameReference PT100 Standard
Manufacturer / Modelanything, e.g. Fluke / 5628
Asset typesensor

One channel:

FieldValue
Channel IDCH1
Physical quantityTemperature
Range min / max0 / 100
Unit°C
Calibration rolechecked (reference standard)

Leave accuracy/resolution/uncertainty blank on this asset — only its calibration record's expanded uncertainty matters here (computed in 1.1).

Asset B — working sensor:

FieldValue
NameProcess Line Thermometer
Manufacturer / Modelanything, e.g. Generic / PT100-A
Asset typesensor

One channel:

FieldValue
Channel IDCH1
Physical quantityTemperature
Range min / max0 / 100
Unit°C
Accuracy value0.5
Accuracy unit°C (a real unit means "absolute" — see The "% FS" convention)
Resolution0.1
Resolution unit°C
Measurement uncertainty0.3
Uncertainty unit°C
Calibration roleunchecked (not a reference standard)

Leave "Output signal unit" blank on both channels so the measured-value unit defaults to °C — this avoids unit-conversion complexity for this example.

1.1 Calibrate the reference standard first

On Asset A, start a calibration: type internal (or external — doesn't matter here, no reference asset needed), distribution Normal, confidence 95%, no accuracy spec needed (skip channel accuracy — it isn't set on this channel).

Data points (reference = a higher-tier standard's value; measured = this reference thermometer's reading):

#Reference (°C)Measured (°C)
10.0000.010
250.00050.020
3100.00099.985

The math

Fitting reference=ameasured+b\text{reference} = a \cdot \text{measured} + b (degree 1) to these 3 points gives a=1.00025a = 1.00025, b=0.01750b = -0.01750.

Residuals (reference − calculated):

PointCalculatedResidual
1−0.007498+0.007498
250.015004−0.015004
399.992495+0.007505

n=3n = 3 points, k=2k = 2 parameters → residual degrees of freedom =1= 1.

  • Type A standard uncertainty = sample std dev of residuals (ddof=1\text{ddof}=1) = 0.012994 °C.
  • No Type B contributions configured on this channel, so the budget has exactly one row:
SourceDistributionudof
fit_residuals (Type A)normal0.0129941
  • Combined uncertainty: uc=0.0129942=0.012994 °Cu_c = \sqrt{0.012994^2} = \mathbf{0.012994\ °C}.
  • Coverage factor: for a normal distribution at 95% confidence, k=Φ1(0.975)1.96k = \Phi^{-1}(0.975) \approx \mathbf{1.96}.
  • Expanded uncertainty: U=kuc=1.96×0.012994=0.0254670.025 °CU = k \cdot u_c = 1.96 \times 0.012994 = 0.025467 \approx \mathbf{0.025\ °C} (2 sig figs).

Expected results in the wizard (Step 3)

FieldValue
0.99999993
RMSE0.010609
Max error0.015004
Combined uncertainty0.012994 (shown rounded: 0.013)
Expanded uncertainty (±)0.025467 (shown rounded: 0.025)

Save this calibration. Write down the Expanded uncertainty (0.025 °C) — Asset B's calibration fetches this automatically in the next step.

1.2 Calibrate the working sensor against the reference standard

On Asset B, start a calibration: type internal, reference asset Reference PT100 Standard (Asset A) — as soon as you pick it, the wizard should show "Ref. standard U: 0.025 °C (last calibration of Reference PT100 Standard)." If it shows a manual-entry field instead, the fetch didn't find a calibration on Asset A — go back and confirm 1.1 was saved.

Distribution Normal, confidence 95%, decision rule Guard band (tolerance − U), and check "Incl. sensor nominal accuracy" (folds in the channel's 0.3 °C manufacturer spec).

Data points:

#Reference (°C)Measured (°C)
10.000.05
220.0020.08
340.0039.95
460.0060.12
580.0079.90
6100.00100.07

The math

Fit: a=1.00038a = 1.00038, b=0.04739b = -0.04739.

PointCalculatedResidual
10.002631−0.002631
220.040260−0.040260
339.917828+0.082172
460.095510−0.095510
579.883044+0.116956
6100.060727−0.060727

n=6n = 6, k=2k = 2 → residual dof =4= 4. Max error = 0.116956 (point 5) → %FS error = 0.116956/100×100%=0.117%0.116956/100 \times 100\% = \mathbf{0.117\%}.

Uncertainty budget (four rows this time):

SourceHow it's derivedu
fit_residuals (Type A)std dev of the 6 residuals, dof = 40.083509
reference_standard (Type B)0.025467 (Asset A's U) ÷ k=20.012734
resolution (Type B)0.1/120.1/\sqrt{12}0.028868
sensor_nominal_accuracy (Type B)0.3 ÷ k=20.150000
  • Combined: uc=0.0835092+0.0127342+0.0288682+0.1500002=0.174554 °Cu_c = \sqrt{0.083509^2 + 0.012734^2 + 0.028868^2 + 0.150000^2} = \mathbf{0.174554\ °C}.

  • Effective degrees of freedom (Welch-Satterthwaite — only fit_residuals has finite dof; the three Type B rows are exactly-known and drop out of the sum):

    νeff=uc4uA4/dofA=0.17455440.0835094/476.4\nu_{\text{eff}} = \frac{u_c^4}{u_A^4 / \text{dof}_A} = \frac{0.174554^4}{0.083509^4/4} \approx 76.4
  • Expanded (normal distribution — νeff\nu_{\text{eff}} isn't used for "normal," only for "t"/"chi_squared"): k=Φ1(0.975)1.96k = \Phi^{-1}(0.975) \approx 1.96, U=1.96×0.174554=0.3421200.34 °CU = 1.96 \times 0.174554 = 0.342120 \approx \mathbf{0.34\ °C} (2 sig figs).

Decision rule (guard band, spec = ±0.5 °C absolute):

guard=U=0.342120,max error+guard=0.116956+0.342120=0.4590760.5    CONFORMS\text{guard} = U = 0.342120, \quad \text{max error} + \text{guard} = 0.116956 + 0.342120 = 0.459076 \le 0.5 \implies \textbf{CONFORMS}

Expected results

FieldValue
Combined uncertaintyshown rounded: 0.17
Expanded uncertainty (±)shown rounded: 0.34
ν_eff (in the Expanded tooltip)≈ 76.4
StatementCONFORMS to ±0.5 (absolute), decision rule = Guard-banded acceptance

1.3 Bonus check — watch the three decision rules disagree

Repeat 1.2 with the channel's Accuracy value changed to 0.4 (edit the channel, save, then re-run the same 6 points through Step 3 of a new calibration) and try all three decision rules with "Incl. sensor nominal accuracy" still checked. Everything above is unchanged (same fit, same budget, same U=0.342120U = 0.342120) — only the pass/fail flips:

Decision ruleCheckResult
Simple acceptance0.1169560.40.116956 \le 0.4CONFORMS
Guard band (− U)0.116956+0.342120=0.4590760.40.116956 + 0.342120 = 0.459076 \le 0.4? NoDOES NOT CONFORM
Shared risk (+ U)0.1169560.342120=0.2251640.40.116956 - 0.342120 = -0.225164 \le 0.4CONFORMS

This is the cleanest way to confirm the decision-rule feature is wired correctly: identical data and identical spec, three different verdicts depending only on which rule is selected.

On this page