TB 9-6625-2023-24
Table 1. Calibration Description - Continued
Test instrument
parameters
Performance specifications
Range: 0 to 10,000 k: in 6 ranges2
Ohms
Accuracy (high resolution off):
0.1 k: range:
(0.005% of reading + 6 digits)
1 k: range:
(0.005% of reading + 1 digit)
10 k: range:
(0.007% of reading + 2 digits)
100 k: range:
(0.004% of reading + 3 digits)
1000 k: range:
(0.014% of reading + 5 digits)
10,000 k: range:
(0. 100% of reading + 5 digits)
Accuracy (high resolution on):
1 k: range:
(0.0040% of reading + 6 digits)
10 k: range:
(0.0065% of reading + 6 digits)
100 k: range:
(0.0040% of reading + 7 digits)
1000 k:range:
(0.0140% of reading + 6 digits)
10,000 k: range:
(0.1000% of reading + 6 digits)
1Frequencies
> 100 kHz are specified for 1 V and 10 V ranges only.
2Accuracies specified are for the 4 wire k: function. Add 0.0004 k: to all readings for the 2 wire k: function.
EQUIPMENT REQUIREMENTS
4. Equipment Required. Table 2 identifies the specific equipment to be used in this
calibration procedure. This equipment is issued with AN/GSM-286; AN/GSM-287; or
AN/GSM-705. Alternate items may be used by the calibrating activity. The items selected
must be verified to perform satisfactorily prior to use and must bear evidence of current
calibration. The equipment must meet or exceed the minimum use specifications listed in
table 2. The accuracies listed in table 2 provide a four-to-one ratio between the standard
and TI. Where the four-to-one ration cannot be met, the four-to-one accuracy of the
equipment selected is shown in parenthesis.
5. Accessories Required. The accessories required for this calibration are common
usage accessories, issued as indicated in paragraph 4 above, and are not listed in this
Table 2. Minimum Specifications of Equipment Required
Manufacturer and model
Common name
Minimum use specifications
(part number)
-10 V to 1000 V dc
CALIBRATOR
Range:
Fluke, Model 5720A (5720A)
r0.00375%
(p/o MIS-35947); w amplifier,
Accuracy: 100 mV
r0.002375%
Fluke 5725A/AR (5725A/AR)
1V
r0.002075%
10 V
r0.002575%
100 V
r0.002575%
1000 V
100 : to 10 M:
Range:
Accuracy: 100 :
r27.5 ppm
1 k:
r15 ppm
10 k:
r17.75 ppm
100 k:
r16 ppm
1 M:
r36.5 ppm
10 M:
r62.8 ppm