Signals, Convolution, and Linear Time-Invariant Systems

How signals become mathematical objects, why linear time-invariant systems are determined by impulse response, and how convolution expresses their full input-output behavior.
Modified

April 26, 2026

Keywords

signal, system, convolution, LTI, impulse response

1 Role

This is the first page of the Signal Processing and Estimation module.

Its job is to build the load-bearing first-pass language:

  • what a signal is
  • what a system is
  • why linearity and time invariance matter
  • why convolution is the universal representation of LTI behavior

Everything later in the module leans on this page.

2 First-Pass Promise

Read this page before anything else in Signal Processing and Estimation.

If you stop here, you should still understand:

  • the difference between a signal and a system
  • the meaning of linearity and time invariance
  • what an impulse response is
  • why convolution is the right formula for an LTI system

3 Why It Matters

A huge number of engineering and ML objects can be treated as signals:

  • audio
  • sensor traces
  • images
  • communication waveforms
  • discretized trajectories

And a huge number of useful operations can be treated as systems:

  • smoothing
  • filtering
  • delay
  • echo
  • channel distortion

The reason signal processing becomes mathematically powerful is that many useful systems are approximately:

  • linear
  • time invariant

Once that happens, the whole system is captured by one object: its impulse response.

That is what makes convolution the first real organizing equation of the subject.

4 Prerequisite Recall

  • a vector space point of view helps because signals can be added and scaled
  • discrete-time signals are often sequences x[n]
  • continuous-time signals are often functions x(t)
  • integrals and sums are both accumulation operators, depending on whether time is continuous or discrete

5 Intuition

5.1 A Signal Is A Structured Quantity That Varies Over Time Or Index

At first pass, a signal is simply a number attached to each time or index.

Examples:

  • sound pressure over time
  • pixel intensity over location
  • sensor voltage over time

The important point is not the physical unit.

It is that the object has ordered structure.

5.2 A System Maps Signals To Signals

A system takes an input signal and returns an output signal.

Examples:

  • a delay system shifts the input
  • a blur system averages neighboring values
  • a channel distorts and scales a waveform

The system is the rule, not the signal itself.

5.3 Linearity Means Superposition Works

If a system is linear, then:

  • response to a sum is the sum of responses
  • response to a scaled signal scales the output

This lets us decompose a complicated input into simpler pieces and then recombine the outputs.

5.4 Time Invariance Means Shifts Do Not Change The Rule

If a system is time invariant, then delaying the input just delays the output by the same amount.

So the system behaves the same way at every time location.

That is what makes one impulse response reusable everywhere.

5.5 Convolution Is Shift-And-Add

At first pass, convolution is best read operationally:

  • break the input into shifted impulses
  • each shifted impulse produces a shifted impulse response
  • weight those shifted responses by the input values
  • add everything up

That is the core LTI picture.

6 Formal Core

Definition 1 (Definition: Signal) A signal is a function or sequence that assigns a value to each time or index.

Typical examples are:

  • discrete-time: x[n]
  • continuous-time: x(t)

Definition 2 (Definition: System) A system is a mapping that takes an input signal x and produces an output signal y.

We often write this schematically as

\[ y = T x \]

for some transformation T.

Definition 3 (Definition: Linear Time-Invariant System) A system is:

  • linear if it preserves addition and scaling
  • time invariant if shifting the input shifts the output by the same amount

Definition 4 (Definition: Impulse Response) The impulse response h of an LTI system is the output produced by the unit impulse input.

At first pass, h is the system’s response to the most concentrated possible input.

Theorem 1 (Theorem Idea: LTI Systems Are Determined By Convolution) If a system is linear and time invariant, then its output is the convolution of the input with the impulse response.

Discrete time:

\[ y[n] = \sum_k x[k]\, h[n-k]. \]

Continuous time:

\[ y(t) = \int x(\tau)\, h(t-\tau)\, d\tau. \]

This is the main structural theorem of the page.

Theorem 2 (Theorem Idea: Convolution Is Superposition Of Shifted Responses) Convolution expresses the output as a weighted sum or integral of shifted copies of the impulse response.

So the output is built from:

  • input coefficients or amplitudes
  • shifts of one reusable template h

7 Worked Example

Consider the discrete-time moving-average system

\[ y[n] = \frac{1}{2}x[n] + \frac{1}{2}x[n-1]. \]

Its impulse response is

\[ h[0]=\frac12,\qquad h[1]=\frac12, \]

and h[n]=0 otherwise.

If the input is

\[ x[0]=2,\qquad x[1]=4,\qquad x[2]=0, \]

then convolution says:

\[ y[n] = \sum_k x[k] h[n-k]. \]

Compute the first few outputs:

  • y[0] = 2 \cdot \frac12 = 1
  • y[1] = 4 \cdot \frac12 + 2 \cdot \frac12 = 3
  • y[2] = 0 \cdot \frac12 + 4 \cdot \frac12 = 2

So the system smooths sudden changes by averaging neighboring samples.

That is the first-pass meaning of convolution in action.

8 Computation Lens

When you meet an LTI system, ask:

  1. what is the impulse response?
  2. is time discrete or continuous?
  3. can the output be read as a local averaging, delay, or weighted accumulation rule?
  4. is the system finite-memory like a short filter, or long-memory like a decaying response?

Those questions usually make the formula readable before you do any algebra.

9 Application Lens

9.1 Filtering And Smoothing

Many denoisers and smoothers are convolution systems in disguise.

9.2 Communication Channels

Echo, multipath, blur, and bandwidth effects are often modeled by an impulse response acting through convolution.

9.3 ML And Representation Pipelines

Convolutional architectures, spectral methods, and many sequence-processing tools reuse the same shift-and-aggregate viewpoint.

10 Stop Here For First Pass

If you stop here, retain these five ideas:

  • a signal is an ordered function or sequence
  • a system maps one signal to another
  • linearity gives superposition
  • time invariance makes one response template reusable everywhere
  • convolution is the universal input-output formula for LTI systems

11 Go Deeper

The strongest next page is:

The strongest adjacent live pages are:

12 Optional Deeper Reading After First Pass

13 Sources and Further Reading

Back to top