TB 9-6625-133-40
Table 1. Calibration Description - Continued.
Test instrument
Performance specifications1
parameters
Accuracy: normal gate mode:
Range: 10 Hz to 1 MHz
(10 ppm + 2 digits)
Accuracy
Range
(ppm of reading + mΩ)
Temperature
0 Ω-199.990 Ω
9.5 + 0.18
Readout 4
200 Ω-1999.900 Ω
9.5 + 0.6
All specifications are manufacturer's absolute specifications 365 day, TCal 5C, 99% confidence maximum resolution.
1
0.1 ppm of range uncertainty added to DCV, DCI and resistance performance check limits for 7 digit read rate/resolution.
2 >300 V, < 10 kHz add: 0.0004 x (Reading 300)2 ppm.
>300 V, >30 kHz add: 0.0024 x (Reading 300)2 ppm.
3 Same specifications for TRUΩ.
4 Verified during resistance checks. Does not include accuracy of probe/sensor.
EQUIPMENT REQUIREMENTS
calibration procedure. This equipment is issued with Secondary Reference Calibration
Standards Set, NSN 4931-00-621-7878. Alternate items may be used by the calibrating
activity. The items selected must be verified to perform satisfactorily prior to use and must
bear evidence of current calibration. The equipment must meet or exceed the minimum use
certify this instrument after combining the uncertainties of the calibration standards with
the TI Relative to Standards specification using the root sum square summation method
recommended by the manufacturer.
5. Accessories Required. The accessories required for this calibration are common
usage accessories, issued as indicated in paragraph 4 above, and are not listed in the
calibration procedure. The following peculiar accessories are required: Fluke high-integrity
low-thermal emf 4-wire shorting PCB and low thermal cables with 6 mm gold plated copper
spade terminals, part of the Fluke Measurement Lead Kit model, 8508A-LEAD.
Table 2. Minimum Specifications of Equipment Required.
Manufacturer and model
Common name
Minimum use specifications
(part number)
AC MEASUREMENT
Range:
100 mV ac
Fluke Model 5790A
STD
Accuracy: 20 Hz 89 ppm (100 ppm)
(13534003)
55 Hz 84 ppm
(MEASUREMENT
1 kHz 77 ppm
STANDARD)
3 kHz 138 ppm
10 kHz 138 ppm
30 kHz 227 ppm
60 kHz 468 ppm
100 kHz 468 ppm
