There is no universal multiplier that converts a GNSS accuracy figure between CEP, RMS, 2DRMS and “95%.” A valid comparison first fixes the dimensions, reference position and statistic. Conversions to a probability radius then require an error-distribution model. For unbiased, circular Gaussian horizontal errors, useful exact relationships exist. Bias, unequal axis errors or a different meaning of “RMS” can invalidate those shortcuts.
This matters when comparing receiver specifications or interpreting an evaluation report. A smaller number is not automatically a smaller error if the two numbers describe different quantities.
Establish what the error is measured against
For horizontal error, compare each reported position with a suitable reference in the same coordinate frame and at the corresponding epoch. Let E and N be its east and north errors, in metres. Its horizontal radial error is:
r = √(E² + N²)
A vertical error is one component. A three-dimensional radial error also includes the up component. Neither can be substituted for r without additional information. ESA-hosted Navipedia's positioning-error reference defines measured horizontal, vertical and three-dimensional errors separately.
Also identify the centre. Distances from a surveyed truth reference assess errors against that reference. Distances from the average of the receiver's own positions assess scatter around its mean and remove that run's mean offset. The two reports need not give the same result.
RMS is not a probability level
For n retained horizontal error samples, define horizontal RMS error here as:
HRMS = √[(1/n) Σ(Eᵢ² + Nᵢ²)]
This is a root mean square of distances from the reference. It weights larger errors more strongly than a median does, but it does not specify what fraction of samples falls inside that radius. A 95th-percentile horizontal radius, R95, answers a different question: the radius containing 95% of the horizontal-error distribution, or the corresponding percentile of a stated test sample.
CEP is commonly used for the 50th-percentile horizontal radius. In this article, 2DRMS means twice the horizontal RMS quantity defined above. A bare label “RMS” remains insufficient: it may concern one coordinate, the horizontal distance, a three-dimensional distance, measurement residuals, or a model-derived estimate rather than measured errors. Ask for the actual definition, not an extra decimal place.
NovAtel's GPS Position Accuracy Measures, APN-029 gives a vendor example of conversion tables with explicit assumptions about axis standard deviations and vertical-to-horizontal geometry. Those assumptions belong to that table; they are not properties of every GNSS receiver or every test.
A conversion that works under a stated horizontal model
Assume that E and N are independent Gaussian errors, both have zero mean, and both have the same standard deviation σ. The reference is the true position, so the error cloud is circular and centred on truth. The radial error then follows a Rayleigh distribution with:
Pr(r ≤ a) = 1 − exp[−a²/(2σ²)]
For a desired probability p, solve this expression for its radius:
aₚ = σ √[−2 ln(1 − p)]
Here a and σ are in metres, p is a probability between zero and one, and ln is the natural logarithm. NIST's Rayleigh distribution documentation gives this cumulative-distribution relationship. The following values are calculated from that model, not measured receiver performance.
| Horizontal statistic | Meaning used here | Value under the stated circular Gaussian model |
|---|---|---|
| CEP | 50% horizontal-error radius | 1.1774σ |
| HRMS | Root mean square horizontal error | √2σ ≈ 1.4142σ; about 63.21% probability inside this radius |
| R95 | 95% horizontal-error radius | 2.4477σ ≈ 2.0789 × CEP ≈ 1.7308 × HRMS |
| 2DRMS | Twice HRMS | 2√2σ ≈ 2.8284σ; about 98.17% probability inside this radius, not exactly 95% |
Consequently, “double the RMS to get 95%” is not an exact horizontal conversion under this model. The familiar approximately two-standard-deviation interval concerns a one-dimensional normal quantity. NIST's coverage-factor guidance states the normal-distribution and reliable-standard-uncertainty conditions for that interval. A horizontal radius is a different random variable.
Hypothetical specification comparison: Offer A quotes 1.00 m CEP and Offer B quotes 1.80 m horizontal R95. If both really follow the same unbiased circular Gaussian model and have comparable operating conditions, A converts to about 2.079 m R95. On that conditional basis, B has the smaller R95 despite its larger printed number. If the model or test conditions are unknown, this conversion is an illustration, not evidence that B is the better product.
Bias can leave a tight cloud in the wrong place
Consider four invented horizontal error vectors, in metres relative to a known reference: (1,0), (3,0), (2,1) and (2,−1). Their mean error is (2,0).
The squared distances from truth are 1, 9, 5 and 5, so horizontal RMS error is √5 ≈ 2.236 m. Subtract the mean first and all four squared distances become 1. The RMS scatter about the mean is then 1.000 m. The same four positions have not become more accurate; the calculation changed its reference centre.
With the same 1/n convention for both quantities, the general identity is:
HRMS² = centred horizontal mean-square scatter + mean(E)² + mean(N)²
This is why standard deviation about the mean and RMS error against truth coincide only when the mean error is zero. Do not mix the 1/n identity with an n−1 sample-variance estimator without adjusting the formula. Nor should four teaching points be used to infer a population's 95th-percentile accuracy.
A known offset may have an explainable cause, but silently subtracting it changes the reported result. If a calibration or coordinate transformation is part of the intended system, describe it and validate its result against an appropriate reference. Report an uncorrected bias separately rather than hiding it inside a claim of low random scatter.
Unequal axes and outliers break the simple multiplier
If east and north have different variances, or their errors are correlated, a covariance ellipse can describe the horizontal uncertainty more appropriately than a circular model. A joint 95% ellipse is not the same object as two separate one-axis 95% intervals or a circle labelled “95%.” A single radial RMS number does not specify the ellipse shape or a unique percentile radius.
Bias shifts the error distribution away from truth. Heavy tails, occasional wrong solutions and a mixture of operating modes can also change the relationship between RMS and percentiles. These are reasons to examine the actual errors or a justified model, rather than treating 2.0789 as a universal CEP-to-R95 factor.
The DOP and position-accuracy guide explains how assumed measurement uncertainty propagates through geometry. The separate measurement-residual guide explains why residual RMS can be small while coordinates are wrong. Neither residual RMS nor DOP supplies the missing horizontal-error distribution for this conversion.
Keep measured accuracy separate from live uncertainty
A receiver's reported standard deviations describe its estimated uncertainty under its model. They are not a direct measurement of the unknown true error at that epoch. For example, NovAtel's BESTPOS documentation defines separate latitude, longitude and height standard-deviation fields alongside solution status and type. Those are NovAtel interface definitions, not ResiNav outputs.
For an evaluation report, retain the reference uncertainty, coordinate frame, antenna/reference point, time alignment, sample interval, test duration and operating conditions. State whether statistics cover all outputs or only selected solution states, and report unavailable intervals separately. Filtering to successful epochs can answer a useful conditional question, but it is not the accuracy-and-availability result for the whole run. Correlated samples also do not provide the same amount of independent evidence as equally many independent trials.
When a measured 95th percentile is supplied, identify the percentile convention and retained population. Ninety-five percent of one finite run is not a guarantee for every future environment, and a 95% error radius is not a 95% confidence interval for the estimated mean.
Use the integrity and monitoring guide for the separate decision about uncertainty outputs, protection levels and operational response. This article does not assign a CEP, covariance output or accuracy guarantee to a ResiNav model. Its practical result is a comparable claim: which error, relative to which reference, summarized by which statistic, under which conditions.
NEXT STEP
Turn available platform information into an engineering review.
Use the Technology centre to frame the discussion, review relevant application scenarios, then send the available platform and receiver details for confirmation.