TB 9-6625-2023-35
Table 1. Calibration Description - Continued.
Test instrument
parameters
Performance specifications
Range: 0 to 10,000 kΩ in 6 ranges2
Ohms
Accuracy (high resolution off): 0.1 kΩ range:
(0.005% of reading + 6 digits)
1 kΩ range:
(0.005% of reading + 1 digit)
10 kΩ range:
(0.007% of reading + 2 digits)
100 kΩ range:
(0.004% of reading + 3 digits)
1000 kΩ range: (0.014% of reading + 5 digits)
10,000 kΩ range: (0. 100% of reading + 5 digits)
Accuracy (high resolution on): 1 kΩ range:
(0.0040% of reading + 6 digits)
10 kΩ range:
(0.0065% of reading + 6 digits)
100 kΩ range:
(0.0040% of reading + 7 digits)
1000 kΩrange:
(0.0140% of reading + 6 digits)
10,000 kΩ range: (0.1000% of reading + 6 digits)
1Frequencies
> 100 kHz are specified for 1V and 10V ranges only.
specified are for the 4-wire k Ω function. Add 0.0004 k Ω to all readings for the 2-wire k Ω function.
2Accuracies
EQUIPMENT REQUIREMENTS
4. Equipment
Required. Table 2 identifies the
specific equipment to be
used in this
calibration procedure. This equipment is issued with Secondary Transfer Calibration
Standards Set, AN/GSM-286. Alternate items may be used by the calibrating activity.
The items selected must be verified to perform satisfactorily prior to use and must bear
evidence of current calibration. The equipment must meet or exceed the minimum use
5. Accessories Required. The
accessories required for this
calibration are
common
usage accessories, issued as indicated in paragraph 4 above, and are not listed in this
calibration procedure.
Table 2. Minimum Specifications of
Equipment Required
Manufacturer and model
Common name
Minimum use specifications
(part number)
CALIBRATOR
Range: -10 V to 1000 V dc
John Fluke, Model 5700A/CT
Accuracy: 100 mV 0.00375%
(p/o MIS-35947); w/power
0.002375%
1V
amplifier, John Fluke Model
0.002075%
10 V
5725A (5725A)
0.002575%
100 V
1000 V 0.002575%
Range: 100 Ω to 10 MΩ
Accuracy: 100 Ω 27.5 ppm
1 kΩ
15 ppm
10 kΩ 17.75 ppm
100 kΩ 16 ppm
1 MΩ 36.5 ppm
10 MΩ 0.02515 %