ENGINEERING REFERENCE

GNSS Measurement Residuals: Why a Good Fit Is Not Position Truth

A GNSS measurement residual describes the difference between an observation and a model prediction. A small post-fit residual means that the estimated solution fits the observations closely. It does not, by itself, show that the estimated position is close to the true position. Some measurement errors can be absorbed into estimated position, clock or other states, leaving little disagreement after the fit.

This distinction matters when comparing receiver logs, investigating an interference event or evaluating a mitigation method. Residuals answer a consistency question. Position error answers a truth-reference question. Both are useful, but they are not interchangeable results.

First establish which residual you are reading

A pre-fit residual is the observation minus the prediction made before the current parameter adjustment. It can contain the effect of an inaccurate starting position or clock as well as measurement noise and remaining modelling errors. ESA's GNSS measurements modelling explains how these observed-minus-modelled quantities enter the navigation equations.

A post-fit residual uses the prediction after estimating the states from the measurements. It therefore reflects the discrepancy left after the estimator has had an opportunity to fit the data. A large pre-fit value followed by a small post-fit value can simply mean that the state estimate changed. It is not, on its own, evidence that a bad observation was repaired.

In a sequential filter, an innovation normally compares a new observation with a prediction based on the prior state. ESA's Kalman-filter formulation separates prediction from the update using measurements. A vendor's field called “residual” might instead describe a post-update quantity, a normalized statistic or a different processing stage. Read the field's definition before comparing it with a least-squares post-fit residual. Code, carrier-phase and Doppler residuals also require their own units and models.

The fit divides an error into two different effects

Consider one linearized adjustment written as y = G δx + e. Here y is the pre-fit observed-minus-computed vector, G is the design matrix, δx is the state correction and e represents remaining observation errors. The predicted observation correction is G δx̂, so the post-fit residual is r = y − G δx̂.

For ordinary least squares with a full-column-rank G, the estimator selects δx̂ to minimize the sum of squared residuals. ESA's parameter-adjustment derivation gives the corresponding state and residual equations. One consequence is particularly important: an error pattern that can be represented as a change of the estimated states can be fitted rather than left in r.

In this linearized model, write an observation-error pattern as e = G a + u, where u is orthogonal to the columns of G. The fit absorbs G a as the state error a. The post-fit residual contains u. This is a statement about the model's geometry, not a claim that real errors always have a convenient known decomposition.

That is why residual magnitude and position-error magnitude need not move together. For the related but different question of how measurement uncertainty propagates through satellite geometry, see GNSS DOP vs position accuracy.

A small worked example with a wrong answer and zero residual

The following is an invented two-state teaching model, not a full three-dimensional GNSS solution or a receiver test. Only one position coordinate x and a clock term b are unknown. Both are expressed in metres, with clock time already converted to an equivalent range. All other coordinates and corrections are assumed known.

Four observation corrections have the form yᵢ = aᵢ x + b + eᵢ, with coefficients aᵢ equal to −1, −0.5, +0.5 and +1. Assume the true corrections are x = 0 and b = 0, and use equal weights. The least-squares estimates are:

x̂ = (−y₁ − 0.5 y₂ + 0.5 y₃ + y₄) / 2.5

b̂ = (y₁ + y₂ + y₃ + y₄) / 4

Now compare three deliberately constructed observation-error patterns. “Residual RMS” below means √[(r₁² + r₂² + r₃² + r₄²) / 4]. It is not a degrees-of-freedom-adjusted uncertainty estimate or a confidence bound.

Observation-error pattern, in metres Post-fit residual RMS Error absorbed into estimated states
−5, −1, +7, +11 0 m x̂ = +8 m; b̂ = +3 m
+1, −1, −1, +1 1 m x̂ = 0 m; b̂ = 0 m
−4, −2, +6, +12 1 m x̂ = +8 m; b̂ = +3 m

In the first case, every observation lies exactly on the model with x = 8 m and b = 3 m. The fit is perfect, but the estimated position coordinate is wrong by 8 m. The second case has a nonzero residual without a position error in this model. Adding those patterns produces the third case: the same 1 m residual RMS coexists with an 8 m position error.

The example does not predict the probability of these patterns in a real receiver. It demonstrates why a residual alone cannot uniquely determine position error.

Weights, exclusions and extra states change the comparison

A weighted fit minimizes a weighted discrepancy rather than treating every observation equally. ESA's weighted least-squares description defines the weight matrix as part of the estimator. Changing it can change the estimated states and the distribution of residuals even when the original observations are unchanged.

Similarly, a smaller residual after excluding observations is not automatically a better position result. The estimator is now fitting a different set, potentially with different geometry and redundancy. As an implementation-specific example, NovAtel's Waypoint measurement settings describe an outlier-detection option that rejects individual satellite measurements and recomputes the position and residuals. That is a Waypoint processing behaviour, not a description of ResiNav products.

In a full-rank, exactly determined linear system, there is no redundant observation direction left for an ordinary least-squares residual test: the fitted residual can be zero. Real navigation software may also estimate biases, use constraints, combine epochs or apply robust methods. Consequently, “number of satellites minus four” is not a universal statement of every processor's usable redundancy.

Normalization also needs care. Dividing a residual by an observation's input standard deviation is not automatically the same as using the correct post-fit residual covariance. The fit itself couples the residuals. A threshold must belong to the statistic and assumptions for which it was designed.

Consistency monitoring and truth-reference testing are complementary

Residual-based checks can identify inconsistencies that would be hidden in a position-only plot. Their usefulness does not make them a direct accuracy certificate. ESA's RAIM fundamentals distinguishes consistency testing from protection-level calculations and explains the role of geometry and statistical assumptions. Passing a consistency test does not prove the absence of every possible fault.

For an evaluation, keep three results distinguishable:

  • Measurement fit: the defined pre-fit, post-fit or normalized statistic, together with its observation set and processing configuration.
  • Solution error: position or velocity compared with a suitable independent reference, with aligned time tags, coordinate frames and reference-point definitions.
  • Integrity or uncertainty statement: the documented bound or estimate, its assumptions and its coverage, rather than the residual RMS relabelled as a bound.

This separation also helps interpret interference-mitigation results. A lower residual might reflect cleaner observations, changed weighting, exclusions or errors absorbed into the state. Establish which happened before treating it as proof of improved position accuracy. The broader distinction between signal availability, measurement changes and final solution results is covered in interference mitigation and measurement bias.

What a useful residual comparison actually establishes

Compare like-for-like statistics from the same processing stage, observable type and units. Keep track of the states estimated, observations retained, weighting and any filter or exclusion changes. Where the question is actual positioning accuracy, add independent reference evidence instead of substituting a low residual for it.

For the separate uncertainty question, see how correlation and model assumptions can make GNSS uncertainty too optimistic, and compare the reported intervals with suitably referenced errors.

A residual is valuable because it shows what the chosen model did not fit. Understanding what the model could fit is the other half of interpreting it.

NEXT STEP

Turn available platform information into an engineering review.

Use the Technology centre to frame the discussion, review relevant application scenarios, then send the available platform and receiver details for confirmation.

Request an engineering review Explore Technology