Constrained Optimization

How local extrema change under constraints, why Lagrange multipliers arise from tangency of level sets, and how constraint geometry bridges multivariable calculus to full optimization.
Modified

April 26, 2026

Keywords

constrained optimization, Lagrange multipliers, constraint geometry, extrema, multivariable calculus

1 Role

This page is the constraint-geometry page of multivariable calculus.

Its job is to explain why optimization under a constraint is different from unconstrained optimization, and why Lagrange multipliers are really a statement about tangent geometry rather than a formal trick.

2 First-Pass Promise

Read this page after Jacobians and Hessians.

If you stop here, you should still understand:

  • why constrained extrema need different logic from unconstrained extrema
  • why \(\nabla f\) and \(\nabla g\) become parallel at a constrained extremum
  • how to solve a basic one-constraint Lagrange multiplier problem
  • how this page leads naturally into the deeper optimization module

3 Why It Matters

In unconstrained optimization, a local optimum often appears where the gradient vanishes.

But with a constraint, you are not allowed to move in every direction. You can only move along the constraint surface.

That changes the logic:

  • the full gradient of the objective may not be zero
  • what matters is whether any allowed direction improves the objective
  • geometry of tangency becomes central

This is the first place where multivariable calculus really starts to look like optimization.

It matters directly because:

  • engineering design problems often optimize under fixed resource or geometry constraints
  • ML objectives sometimes include normalization or explicit equality constraints
  • later KKT conditions are built on exactly this constrained first-order picture

4 Prerequisite Recall

  • gradients are normal to level sets
  • linearization gives the first-order local model near a point
  • Hessian language explains curvature, but first-order geometry already determines the basic Lagrange condition

5 Intuition

Suppose you want to optimize \(f(x,y)\) subject to a constraint

\[ g(x,y)=c. \]

The constraint is a curve. Along that curve, the function \(f\) may increase or decrease.

At a constrained optimum, the level curve of \(f\) just touches the constraint curve instead of cutting across it. That means the two curves are tangent there.

Since gradients are normal to level curves, the normals must be parallel:

\[ \nabla f = \lambda \nabla g. \]

That is the whole geometric content of the Lagrange multiplier condition.

So the multiplier method is not a mysterious algebra trick. It is a way of encoding tangency of the objective level set and the constraint set.

6 Formal Core

Definition 1 (Constrained Extremum) A constrained extremum of \(f\) subject to \(g(x,y)=c\) is a point on the constraint set where \(f\) is locally maximal or minimal among nearby points that also satisfy the constraint.

This is different from an unconstrained extremum because movement is restricted to the constraint set.

Proposition 1 (Lagrange Multiplier Condition) If \(f\) and \(g\) are differentiable, \(f\) has a local constrained extremum at a point on the smooth constraint \(g(x,y)=c\), and \(\nabla g \ne 0\) there, then

\[ \nabla f = \lambda \nabla g \]

for some scalar \(\lambda\).

This says the two gradients are parallel.

Proposition 2 (Meaning Of The Multiplier) At a first-pass level, \(\lambda\) measures how the objective is aligned with the constraint normal direction.

Later in optimization, multipliers gain richer interpretations as sensitivities and dual variables, but here the core idea is geometric proportionality.

7 Worked Example

Find the maximum and minimum of

\[ f(x,y)=x^2+y^2 \]

subject to

\[ x+y=1. \]

First write the constraint as

\[ g(x,y)=x+y. \]

Then

\[ \nabla f=(2x,2y), \qquad \nabla g=(1,1). \]

The Lagrange condition is

\[ (2x,2y)=\lambda(1,1). \]

So

\[ 2x=\lambda,\qquad 2y=\lambda, \]

which implies

\[ x=y. \]

Combine this with the constraint \(x+y=1\):

\[ 2x=1 \quad \Rightarrow \quad x=y=\frac12. \]

So the only critical candidate is

\[ \left(\frac12,\frac12\right). \]

Its value is

\[ f\left(\frac12,\frac12\right)=\frac14+\frac14=\frac12. \]

This is the constrained minimum.

Why is there no constrained maximum?

Because on the line \(x+y=1\), the points can go arbitrarily far from the origin in opposite directions, making \(x^2+y^2\) arbitrarily large. So the objective has no constrained maximum on this unbounded constraint set.

This example is useful because it shows both pieces of the method:

  • solve the gradient-proportionality system
  • still reason globally about the actual constraint set

8 Computation Lens

A practical first-pass workflow for one smooth equality constraint is:

  1. write the objective \(f\) and constraint \(g=c\)
  2. compute \(\nabla f\) and \(\nabla g\)
  3. solve \[ \nabla f = \lambda \nabla g \] together with the constraint equation
  4. evaluate the objective at the candidate points
  5. check whether the geometry of the constraint allows a max, a min, both, or neither

This keeps the method grounded in meaning rather than only algebraic elimination.

9 Application Lens

This page is the clean bridge into the optimization module:

  • here, the story is geometric tangency and constrained local extrema
  • later, the optimization module extends this into Lagrangians, KKT conditions, inequalities, duality, and algorithms

So if the optimization module feels abstract, this page should be the place where the first constrained idea becomes intuitive.

10 Stop Here For First Pass

If you can now explain:

  • why constrained extrema are different from unconstrained ones
  • why \(\nabla f\) and \(\nabla g\) become parallel
  • how to solve a simple one-constraint Lagrange multiplier problem
  • why candidate points still need interpretation in the geometry of the constraint set

then this page has done its main job.

11 Go Deeper

The strongest next steps after this page are:

  1. Vector Fields and Divergence / Curl, for the field-based branch of multivariable calculus
  2. Constrained Optimization, KKT, and Lagrangians, to see the full optimization-side generalization
  3. Optimization, to see constrained geometry become certificates, multipliers, and algorithms

12 Optional Deeper Reading

13 Optional After First Pass

If you want more practice before moving on:

  • solve one optimization problem on a circle or line constraint
  • draw the objective level curves and the constraint curve to see the tangency
  • ask whether the constraint set is bounded enough to guarantee both max and min

14 Common Mistakes

  • setting \(\nabla f=0\) as if the problem were unconstrained
  • forgetting to include the constraint equation after writing \(\nabla f=\lambda\nabla g\)
  • solving only the algebra and not checking the geometry of the constraint set
  • assuming every candidate automatically gives both a max and a min
  • treating the multiplier as magic instead of as a tangency condition

15 Sources and Further Reading

Back to top