Questions tagged [accuracy]
The accuracy tag has no usage guidance.
141
questions
1
vote
0
answers
44
views
"50 Ohm" vs "50 Ohm terminated" in ACV accuracy specs
In the accuracy specifications for an AC voltage measurement with a Keysight 3458A, there is a distinction between source impedance of "50 Ohm" and "50 Ohm terminated" for the &...
- 256
0
votes
1
answer
203
views
What is the best and simplest way to generate multiple accurate reference voltages?
I need several reference DC voltages to use them to calibrate hundreds of circuit boards manufactured in our facility.
The voltages to be calibrated are
5V
10V
100V
200V
500V
The accuracy required ...
0
votes
2
answers
92
views
Multimeter Range and Resolution
I'm confused as to how my multimeter takes readings, especially since its not an auto-range multimeter. I have to switch to the desired range to carry out measurements. Unfortunately, the resolution ...
- 339
1
vote
3
answers
215
views
Easiest way to get highly-accurate frequency reference?
I just want to check the calibration of my frequency counter, so anything constant under 250 MHz would work(?)
Mains are synced to atomic clocks, to maintain wall clock accuracy, but only ...
- 28.9k
1
vote
2
answers
81
views
Understanding accuracy specification for an electronic load box
I am using a BK Precision 8600 electronic load box. I am using it in constant resistance mode (CR mode). I have two resistances I am using, 15Ω and 5Ω, and I am interested in determining the max/min ...
- 21
1
vote
3
answers
212
views
Shunt's thermal EMF (Seebeck effect) vs Kelvin connection?
While reading this Vishay document about shunts, I saw that temperature mismatch between the shunt's terminals can introduce a "parasitic" voltage drop which will in turn affects the current ...
- 896
0
votes
2
answers
133
views
Highly accurate DIY friendly (cheap) power load
To avoid XY question, I start with my motivation and then my question.
I have an Emporia Vue2 (in-panel) power sensor but it is known not to be very accurate. For me there is a discrepancy between the ...
- 1,326
2
votes
1
answer
73
views
Which INL is better for this ADC?
I am using the following ADC.
I want to know given the figures below, which INL is better?
How do I interpret the x-axis (% of FS)? What is my full-scale (FS) value? My Vin ranges from 10mV to 50mV ...
- 2,391
1
vote
1
answer
144
views
How to design an accurate (1 μV error) meter
I want to design an accurate LV meter.
The range is from 2.5mV to 50mV both AC, with a 5uV accuracy. This comes from a shunt resistor that is connected to AC main (230V AC). Hence it will go below 0V. ...
- 2,391
2
votes
1
answer
125
views
Long Term Drift of IC
I wanted to calculate Long Term Drift of output voltage of an IC and in datasheet the value is given as 60 ppm/√kHr. I dont know what exactly this means as there is square root of kHr. Can anyone pls ...
- 347
3
votes
3
answers
546
views
Is there a specific definition for the tolerance value of a resistor?
Is there any authoritative definition for what the tolerance value of a resistor actually specifies, or is it just a de-facto common understanding, or are the details left up to individual ...
- 33
2
votes
3
answers
296
views
Improvement in current sensor accuracy
I want to improve the accuracy of a current sensor. 49% of the error I have at the moment is from the tolerance of the shunt resistor and 26 percent of that it is from the op amp offset voltage.
How ...
- 347
0
votes
1
answer
241
views
DC offset to an ADC reduces dynamic range which reduces accuracy. How to quantify accuracy due to loss of dynamic range?
It’s understood that the dynamic range of the ADC should match the max amplitude of the signal to achieve maximum accuracy. If it doesn’t, you lose digital codes which reduces accuracy.
How can one ...
- 113
1
vote
3
answers
323
views
The most precise resistor with respect to temperature ppm
What is the most precise resistor available (measured in terms of ppm drift if
measurements are repeated over time)?
I just want a passive box with two terminals. The temperature will be kept between ...
- 554
12
votes
3
answers
2k
views
How much does temperature affect the 555 IC?
I read that one of the disadvantages of the 555 timer is that it varies in timing accuracy depending on temperature.
How much can accuracy vary, and why does this happen? Is temperature the only thing ...
- 2,113