Digital Sensor Calculator

ADC counts, measured value and resolution

Counts to value
Value to counts
---
Measured value
Full-scale count
---
Number of codes
---
Value per count
---
Quantisation error, ±½ LSB
---
Percent of full scale
---
Calculation method and assumptions +

An analogue to digital converter reports its input as an integer count. An n bit converter produces 2n distinct codes, numbered 0 to 2n − 1. Mapping those codes onto a measured range gives the value the count represents. PV is the process variable, the standard instrumentation term for the measured quantity, whatever it happens to be.

Codes = 2n   Full-scale count = 2n − 1
PV = PVmin + Count / (2n − 1) × (PVmax − PVmin)
Count = round[ (PV − PVmin) / (PVmax − PVmin) × (2n − 1) ]

Why the divisor is 2n − 1 and not 2n

Both divisors appear in datasheets, and they answer different questions. This tool uses the endpoint mapping, where count 0 sits exactly at the bottom of the range and the highest count sits exactly at the top, so the step between adjacent counts is the span divided by 2n − 1. That is the convention for scaling a reported count back into engineering units, and it is what the value per count above reports.

The other convention describes the converter's own quantisation step, the width of one code bin, which is the full-scale input span divided by 2n. It is the right figure when reasoning about the converter as a quantiser rather than about the number it reported. At 12 bit the two differ by one part in four thousand and nothing depends on which you pick. At 8 bit the difference is 0.4 %, and at low bit counts it is worth being explicit about which one a datasheet means.

Resolution is not accuracy

The quantisation error above is the floor set by the arithmetic alone. Real converters add noise, so the effective number of bits is lower than the nameplate figure, sometimes by several bits on a 24 or 32 bit device, where the last bits are usually below the noise floor of the converter itself. Offset, gain and linearity errors sit on top of that, and the sensor in front of the converter usually contributes more error than the converter does. Treat the value per count as the smallest change the number can express, rather than as the smallest change the instrument can measure.

What this assumes

The mapping is linear, unipolar and unsigned, with count 0 at the bottom of the measured range. A bipolar or signed converter, an offset binary or two's complement output, or a sensor that uses only part of the converter's input span needs its own mapping. Where a transmitter drives a 4-20 mA loop into an ADC that covers 0 to 25 mA, the counts corresponding to 4 mA and 20 mA are what belong in the two range fields, not the extremes of the converter.

Application notes

TALK TO OUR TDLAS EXPERTS

Interested in learning more?

Talk to an engineer or contact us to learn more about our products and technology, and explore how our next-generation TDLAS gas analyzers can solve your gas analysis applications.