Free Energy Minimisation

Free energy minimisation, as described in Karl Friston’s Free Energy Principle (FEP), means that ==any System that persists over time – that maintains its boundary – can be described as minimising a quantity called variational free energy.==

Formal structure

Variational free energy is an ==upper bound on surprisal== (negative log-evidence), which measures how unlikely a given sensory state is under the system’s Model. Formally:

Free energy ≥ Surprisal

or equivalently:

F = E_q[ln q(x) − ln p(y, x)]

where q(x) is the system’s approximate posterior (its “belief” about hidden causes), p(y, x) is the generative model (joint probability of sensory data and causes), and y is the sensory data at the Markov Blanket.

This decomposes into two terms:

Complexity: the KL divergence between the system’s beliefs and its prior expectations – how much the system has had to update its model.
Inaccuracy: the expected negative log-likelihood – how poorly the model predicts the actual sensory input.

Minimising free energy therefore means finding the simplest accurate model of what is causing sensory input.

Because free energy bounds surprisal from above, minimising it also minimises surprisal on average, which is the information-theoretic Entropy of the system’s sensory states. A system that minimises free energy over time is one that keeps itself in a relatively small, characteristic set of states – its Attractor.

This is why To exist means to minimise surprise: to persist is to resist the second law of thermodynamics, i.e. to keep one’s sensory entropy low.

Two strategies for minimisation

There are exactly two ways to reduce the gap between model and world:

Perception: update the internal model to better fit the incoming data. This is “bringing the model closer to the world”.
Action: change the world (via active states on the Markov Blanket) so that it conforms to the model’s expectations. This is “bringing the world to the model”.

These are not two separate systems but two faces of the same imperative. Together, they constitute what Friston calls active inference.

Intuitive understanding

Despite the name, variational free energy is an information-theoretic, not a physical-energetic quantity. It measures model–world fit, not energy available to do work.

An intuitive way to understand it: free energy measures the cost of ongoing Sensemaking.

Every living system – from a bacterium to a person – is continuously making sense of what is happening to it. Not necessarily consciously: a bacterium “makes sense” of a sugar gradient by swimming up it, using an Implicit Model embodied in its chemotaxis machinery. There is always a model of “how things should be going” that the system brings to its encounter with the world.

When the world behaves roughly as the model predicts, sensemaking is cheap – free energy is low. When something unexpected happens, there is a gap between prediction and encounter. Sensemaking becomes expensive – free energy spikes. The two strategies (perception and action) are the two ways of closing this gap.

Crucially, what counts as “making sense” is not given by the world but by the system’s own model – by the kind of thing it is. A fish’s model expects water; a human’s expects a body temperature around 37°C. The model does not passively predict the world; it defines the envelope of viable states for that system. The landscape the system navigates is constituted by its model, not objectively given.

Variational free energy as a Lyapunov function

The variational free energy functional serves as a Lyapunov function for the system’s dynamics: the system’s flow through its State Space follows a gradient descent on this function.See Attractor for details on Lyapunov functions and gradient flows. This connects free energy minimisation directly to the Attractor structure described by the system’s dynamics.

Unlike thermodynamic free energy, where minimisation leads to equilibrium (maximum entropy, no structure), minimising variational free energy keeps the system far from thermodynamic equilibrium – in the structured, low-entropy states characteristic of being alive. The system actively maintains itself near its expected states. This is Self-organisation.

Scale-free character

Free energy minimisation is a Scale-free Abstraction: it applies at every level where a Markov Blanket can be identified – from cells to organisms to social systems. What changes across scales is the content of the generative model, not the principle itself.

References