State Estimation, Smoothing, and Hidden-State Inference

How hidden-state models organize filtering and smoothing, why Kalman and forward-backward recursions matter, and how past-only and past-plus-future inference differ.
Modified

April 26, 2026

Keywords

state estimation, filtering, smoothing, Kalman filter, hidden Markov model

1 Role

This is the fifth page of the Signal Processing and Estimation module.

Its job is to move from static denoising and Wiener-style estimation into sequential inference for hidden states.

The previous page said:

  • noisy observations can be handled by MMSE and linear-MMSE estimation
  • Wiener filtering is the stationary spectral version of that story

This page adds:

  • hidden states evolve over time
  • observations reveal those states only indirectly
  • filtering and smoothing are different inference tasks

2 First-Pass Promise

Read this page after Noise Models, Wiener Filtering, and MMSE Estimation.

If you stop here, you should still understand:

  • what a hidden-state model is
  • how prediction, filtering, and smoothing differ
  • why Kalman filtering is the load-bearing linear-Gaussian recursion
  • why forward-backward style inference is the discrete-state analogue

3 Why It Matters

Many real systems are not observed directly.

What we see is only a noisy projection of an evolving hidden state:

  • object position seen through noisy sensors
  • latent regime of a communication channel
  • speaker or phoneme state behind audio features
  • hidden biological state behind observed sequences
  • tracked system state behind measurements arriving over time

So estimation is not only:

  • denoise one measurement

It is often:

  • infer a trajectory of latent states from a sequence of observations

This is where signal processing meets control, tracking, sequential statistics, HMMs, and modern probabilistic ML.

4 Prerequisite Recall

  • MMSE and LMMSE gave a static least-squares view of noisy estimation
  • Wiener filtering gave a stationary-process version of linear estimation
  • probability and statistics provide conditional distributions and posterior thinking
  • state-space viewpoints from control explain why a system can have internal memory not directly observed

5 Intuition

5.1 Observation Is Not State

A sensor reading is often not the thing we care about.

It is a noisy or partial function of the underlying state.

So we need to infer the hidden state from what we observe.

5.2 Prediction, Filtering, And Smoothing Ask Different Questions

These three questions sound similar but are not the same:

  • prediction: what will the state be next, given the past?
  • filtering: what is the current state, given observations up to now?
  • smoothing: what was the state at an earlier time, after seeing later observations too?

Smoothing is usually stronger because it gets to use future evidence.

5.3 Recursive Estimation Is A Memory-Saving Principle

A good sequential estimator should not reread the full observation history every time a new measurement arrives.

It should update a compact summary:

  • the current posterior distribution
  • or, in simpler settings, the current mean and covariance

That compact summary is the belief state or filtered state estimate.

5.4 Kalman Filtering Is Hidden-State Inference For Linear Gaussian Models

When the dynamics are linear and the noise is Gaussian, recursive estimation becomes especially clean.

The estimator cycles through:

  • prediction from the model
  • correction from the new measurement

That is the Kalman filter.

5.5 Forward-Backward Is The Discrete-State Analogue

For hidden Markov models with discrete latent states, the same structural story appears in a different algebra:

  • a forward pass accumulates information from the past
  • a backward pass brings in information from the future

Their combination gives smoothed posteriors over hidden states.

6 Formal Core

Definition 1 (Definition: Hidden-State Model) A hidden-state model describes an evolving latent state x_t together with observations y_t generated from that state.

At first pass, the model has two pieces:

  • state evolution
  • observation generation

In discrete-state language, this is a hidden Markov model.

In continuous linear-Gaussian language, this is a state-space model.

Definition 2 (Definition: Prediction, Filtering, and Smoothing) For a hidden-state model:

  • prediction estimates x_{t+1} from observations up to time t
  • filtering estimates x_t from observations up to time t
  • smoothing estimates x_t from observations up to a later horizon T > t

At first pass, the corresponding distributions are:

\[ p(x_{t+1}\mid y_{1:t}), \qquad p(x_t\mid y_{1:t}), \qquad p(x_t\mid y_{1:T}). \]

Theorem 1 (Theorem Idea: Recursive Filtering) Filtering updates the current state belief by combining:

  • propagation through the dynamics model
  • conditioning on the newest observation

So recursive state estimation is a predict-update cycle.

Theorem 2 (Theorem Idea: Forward-Backward Smoothing) In hidden-state models, smoothing combines a forward pass carrying past information with a backward pass carrying future information.

For discrete HMMs, this is the forward-backward algorithm.

For linear-Gaussian models, this becomes Kalman smoothing.

Definition 3 (Definition: Linear-Gaussian State-Space Model) A linear-Gaussian hidden-state model has the form

\[ x_{t+1}=Ax_t+w_t,\qquad y_t=Cx_t+v_t, \]

where w_t and v_t are Gaussian noise terms.

Theorem 3 (Theorem Idea: Kalman Filtering) For linear-Gaussian models, the filtered posterior remains Gaussian, so recursive state estimation can be tracked through means and covariances.

At first pass, this is the reason the Kalman filter is computationally practical:

  • the entire posterior does not need to be stored symbolically
  • mean and covariance are enough

7 Worked Example

Take the scalar random-walk model

\[ x_{t+1}=x_t+w_t, \qquad y_t=x_t+v_t, \]

where:

  • x_t is hidden position
  • y_t is noisy measurement
  • w_t is process noise
  • v_t is measurement noise

Suppose we observe:

  • y_1
  • then later y_2

At time 1, the filtered estimate of x_1 uses only y_1.

After y_2 arrives, the smoothed estimate of x_1 can use both y_1 and y_2.

So smoothing can revise earlier beliefs:

  • if y_1 was noisy but y_2 strongly supports a nearby trajectory, the estimate of x_1 becomes sharper

This is the central qualitative point:

  • filtering is online
  • smoothing is retrospective

The Kalman filter gives the forward recursion.

The Kalman smoother adds a backward refinement step.

8 Computation Lens

When you meet a sequential inference problem, ask:

  1. what is hidden, and what is observed?
  2. is the latent state discrete, continuous, or mixed?
  3. do we need online filtering or offline smoothing?
  4. is the model linear-Gaussian, so Kalman methods apply?
  5. if not, is the right mental model closer to HMM forward-backward, particle filtering, or approximate inference?

These questions usually determine whether the right tool is simple recursion, Kalman filtering, HMM inference, or a more approximate method.

9 Application Lens

9.1 Tracking And Navigation

State estimation is the backbone of tracking position, velocity, orientation, and other latent physical states from sensor streams.

9.2 Communication And Speech

Hidden-state models organize decoding and sequential latent-structure inference when the observed signal is only an emitted trace of a hidden process.

9.3 Modern Probabilistic ML

Latent-variable sequence models, probabilistic state-space models, and many sequence-inference pipelines are direct descendants of this filtering-and-smoothing viewpoint.

10 Stop Here For First Pass

If you stop here, retain these five ideas:

  • hidden-state models separate latent dynamics from noisy observations
  • prediction, filtering, and smoothing are different inference tasks
  • filtering uses the past, smoothing uses both past and future
  • Kalman filtering is the linear-Gaussian recursive estimator
  • forward-backward style inference is the discrete hidden-state analogue

11 Go Deeper

The strongest next page is:

The strongest adjacent live pages are:

12 Optional Deeper Reading After First Pass

13 Sources and Further Reading

Back to top