Estimating agent-based models by matching random features

1An agent-based model is a probability distribution you can only sample from

An agent-based model (ABM) is a computer program that tracks many individual agents: people, animals, firms, cars. Each agent has a state and follows simple rules that involve chance and depend on nearby agents. Nobody writes down a formula for the behavior of the whole population. The large-scale pattern comes from running the rules.

The running example on this page is a small epidemic. There are 1,024 people arranged on a 32 by 32 grid whose edges wrap around. Each person is susceptible, infected, or recovered. Every day:

• A susceptible person meets their four grid neighbors and one person picked at random from the whole grid. Each infected contact passes on the disease with probability β.
• An infected person recovers with probability γ.
• A recovered person loses immunity with probability 0.03, and any susceptible person catches the disease from outside the grid with probability 0.0005. These two numbers are treated as known. They keep the epidemic from dying out.

The unknown parameter vector is θ = (β, γ), so the number of parameters is p = 2. We never see the grid. We see two numbers per day: the percentage of positives among 400 randomly tested people (the test has sensitivity 0.95 and specificity 0.99), and the count of new cases. The observed data are a time series X₁, …, X_n where each X_t has d = 2 entries.

Two facts from this figure drive everything that follows.

For a fixed θ, the output is random. The model therefore defines a probability distribution P_θ over whole time series. A run of the program is one draw from P_θ.

Changing θ changes the distribution, not any particular value. Estimation means finding the θ whose distribution most plausibly produced the series we observed. The true value is written θ₀.

2Why maximum likelihood is out of reach

The standard recipe for estimation is maximum likelihood. Write the probability of the observed data as a function of the parameter, L(θ) = p(x^obs; θ), and choose the θ that makes it largest. Bayesian inference needs the same function. Both require a formula for p(x; θ), or at least a way to compute it.

In an ABM the observed numbers depend on the hidden history h of every agent on every day. The probability of the data is a sum over all hidden histories that could have produced it:

The situation is lopsided. Drawing a sample from P_θ is cheap. Evaluating the probability that P_θ assigns to a given series is infeasible. Methods built for this situation are called likelihood-free or simulation-based inference.

3The simulation-based idea: match summaries, and the trouble with choosing them

If we cannot compute probabilities, we can still compare. Pick a summary: any function T that turns a data set into a few numbers. Compute T on the real data. Then search for the θ whose simulated data give about the same T. The method of simulated moments, indirect inference, and approximate Bayesian computation are all versions of this loop.

The whole method stands or falls on the choice of T. Good summaries must be (i) easy to compute, (ii) sensitive to the parameters, and (iii) identifying: two different parameter values must not produce the same expected summary. The third property is the hard one. The figure below tries the most natural summary for our epidemic, the average percentage of positive tests.

In practice, scientists either hand-craft summaries using deep knowledge of the model and then check properties (i) to (iii) by trial and error, or they train neural networks to learn summaries from many simulated data sets, which costs a lot of computation. The proposal examined here is a third route: choose the summaries at random, from a family of functions rich enough that a handful of them will identify the parameters with probability one.

4A random feature is a random question about a distribution

Start with a fact from probability theory. A distribution is fully described by the expected values it gives to a rich enough family of test functions. One such family is the sines and cosines. The characteristic function of a random variable X is

It plays the same role as the moment generating function, with the advantage that it always exists. The uniqueness theorem says that two distributions with the same ψ at every frequency ω are the same distribution. So "what is E[cos(ωX)]?" is a question about the distribution, and the collection of all such questions has a unique answer sheet for each distribution.

A random Fourier feature is one of these questions, drawn at random:

Random Fourier features come from machine learning (Rahimi and Recht, 2007), where they are used to speed up kernel methods. Here they do a different job: they act as probes that tell probability distributions apart. In statistics any function of the data is called a statistic. Machine learning calls it a feature, and the papers follow that usage. The frequency ω and phase α are drawn once, before looking at the data, and then held fixed. The expected value of the feature under parameter θ is written Φ(θ) = E_θ[φ(X)]. A trigonometric identity shows that it reads off the characteristic function at the random frequency: Φ(θ) = cos(α) Re ψ_θ(ω) − sin(α) Im ψ_θ(ω).

The simplest test case has one parameter: independent draws from a normal distribution with unknown mean μ and variance 1. There Φ(μ) = e^−ω²/2 cos(ωμ + α), which the figure plots on the right.

Three things to take from this figure. First, the sample average of the feature lands close to its expected value: that is the law of large numbers at work. Second, the feature responds smoothly to the parameter, so it carries information about μ. Third, one feature is not enough. The curve on the right goes up and down, so several values of μ share the same feature value. A single question has an ambiguous answer.

5Why 2p + 1 random features are enough

Use k features at once and stack their expected values into a vector Φ(θ) = (Φ₁(θ), …, Φ_k(θ)). As θ moves through the p-dimensional parameter space, Φ(θ) traces a p-dimensional shape inside k-dimensional space. The parameters are identified by the features when this map is one-to-one: the shape never passes through the same point twice. Then knowing Φ tells you θ.

For the normal mean, p = 1 and the shape is a curve. The figure lets you compare k = 2 with k = 2p + 1 = 3.

The string picture and the counting argument

Drop a long piece of string on a table and it will cross itself. Wave the same string around in the air and freeze it: the chance that two separate points of the string occupy the same point in space is zero. A curve needs three dimensions to avoid itself. A p-dimensional surface needs 2p + 1.

Here is why that number appears. A failure of identification is a pair of different parameter values θ and θ′ with Φ(θ) = Φ(θ′). The pair (θ, θ′) has 2p free coordinates. Each feature contributes one equation, Φ_i(θ) = Φ_i(θ′). With 2p + 1 independent equations and only 2p unknowns, the system is overdetermined, and for randomly chosen equations it has no solutions apart from the trivial ones with θ = θ′.

What the theorem says

The idea has a long history in geometry and in nonlinear dynamics. Whitney (1936) showed that a typical smooth map of a d-dimensional manifold into 2d + 1 dimensions is one-to-one. Takens (1981) used this to show that the state of a d-dimensional dynamical system can be rebuilt from 2d + 1 delayed readings of a single measurement. Sauer, Yorke and Casdagli (1991) replaced "typical" with a probabilistic "almost every". Those results concern generic smooth maps, which one cannot sample from. Wieck-Sosa and Shalizi (2026) prove the version needed for statistics, using a recent result called the finite witness theorem (Amir et al., 2023). In plain terms:

Nothing in the statement requires understanding the model, inspecting the data, or knowing which aspects of the output are informative. That is the practical appeal for ABMs, where such understanding is hard to come by.

One caution. The theorem is about identification: the expected feature values determine θ. It does not say how precisely θ can be estimated from a finite, noisy data set. Precision depends on how strongly the features react to θ relative to their sampling noise. Steps 6 to 9 deal with data.

6From expectations to data: average the features over time

The theorem speaks of expected values. We have one observed time series, not an expectation. With independent data, the sample mean estimates the expectation. With time series the observations depend on each other, but the same trick still works under two conditions: the process is stationary (its statistical behavior does not drift over time) and its memory fades (what happened long ago has little influence on today). Then the ergodic theorem, the law of large numbers for dependent data, says that averages over time converge to expectations.

Features that see the dynamics

A feature that looks at one day at a time only learns about the distribution of single days. To learn how today depends on yesterday, let each feature read a short window of m + 1 consecutive days. With d numbers per day the window holds (m + 1) × d numbers, and the feature assigns a random frequency to each one:

Each Ω_i,j is a vector of d independent N(0, 1) draws and the dot is the dot product. For a model where tomorrow depends only on today (a Markov model) the papers suggest m = 1. Our epidemic observed through noisy tests is not exactly Markov in the observed series, but m = 1 works well. The summary that gets matched is the time average of each feature:

When the process is not stationary

Many ABMs are run from a special starting state and never settle: an outbreak that rises and burns out, a market after a shock, a city that grows. Then the expected feature value changes over time, and a single average over the whole series blurs the early and late behavior together. The second estimator in Wieck-Sosa and Shalizi (2026) replaces the global average by rolling-window averages: the mean feature value over the last w days, computed at every day, for both the observed and the simulated series. The parameter estimate minimizes the average squared gap between the two rolling curves. The supporting theory uses a regime in which more data means denser sampling of the same episode, called infill asymptotics.

7The full recipe

Here is the time-average estimator from start to finish, for an ABM with p parameters whose output is a d-dimensional time series of length n.

1. Set up. Choose the parameter box Θ. Put each output series in units where its fluctuations are of order 1, using constants fixed in advance.
2. Draw the questions. Draw k = 2p + 1 random Fourier features: frequencies Ω from N(0, 1), phases α from Uniform(−π, π). Do this once and store them.
3. Question the data. Compute the k time averages on the observed series. Call the result F^obs.
4. Question the simulator. For a candidate θ, run the ABM s times, compute the same k time averages on each run, and average over runs. Call the result F̄^sim(θ).
5. Measure the mismatch. Q(θ) = ‖F^obs − F̄^sim(θ)‖, the Euclidean distance between two points in k-dimensional space.
6. Search. Hand Q to a global optimizer. The estimate θ̂ is the minimizer.

In Python the core fits in a few lines. Here run_abm stands for your simulator and must return an array with n rows and d columns.

import numpy as np
from scipy.optimize import differential_evolution

p, d, m, s = 2, 2, 1, 10
k = 2 * p + 1
rng = np.random.default_rng(1)
Omega = rng.normal(size=(k, (m + 1) * d))     # random frequencies, drawn once
alpha = rng.uniform(-np.pi, np.pi, size=k)    # random phases, drawn once

def features(X):                              # X has shape (n, d), already rescaled
    n = len(X)
    W = np.hstack([X[j:n - m + j] for j in range(m + 1)])   # all windows of m+1 days
    return np.cos(W @ Omega.T + alpha).mean(axis=0)         # k time averages

F_obs = features(X_obs)

def Q(theta):
    runs = [features(run_abm(theta, n=len(X_obs), seed=r)) for r in range(s)]
    return np.linalg.norm(F_obs - np.mean(runs, axis=0))

fit = differential_evolution(Q, bounds=[(0.03, 0.25), (0.05, 0.35)], seed=2)
theta_hat = fit.x

Practical choices

8Live estimation of the grid epidemic

This figure runs the whole recipe in your browser. The true parameters generate 300 days of observed data. Five random features (2p + 1 with p = 2) are drawn, each reading a two-day window of both series. A small differential evolution search, 12 candidates improved over 20 generations with s = 3 runs per candidate, looks for the minimizer of Q. The budget is kept small so that it finishes in a few seconds. The papers use s = 10 and a full optimizer run.

Things worth trying, in order:

• Estimate, then look at the bottom-right panel. The simulated series never lines up with the observed one day by day, and it should not. The method matches distributions, through the feature averages in the panel above, not trajectories.
• Map the mismatch. Compare with the valley in step 3. With five random features, chosen with no knowledge of epidemiology, the smallest mismatch sits in a compact region near the truth instead of running along the whole valley. The region is usually narrower in γ than in β, which means this data set pins down recovery better than transmission. Pale patches far from the truth can appear because cosines are periodic: a distant parameter value can match some of the features by coincidence, though not all five at once.
• Draw new features and estimate again, several times. The estimates scatter around the cross. Some feature draws are more informative than others, which is part of the method's variance. Now and then an estimate lands far from the truth. With only 300 days of data the observed feature averages are noisy, and for an unlucky draw a distant parameter value can match them better than the truth does. In one such case checked while building this page, the mismatch at the truth was 0.17 and the mismatch at the wrong estimate was 0.07, so the optimizer did its job and the noise in the data was responsible. In 30 independent repetitions of this demo, each with fresh data and fresh features, 2 estimates landed far from the truth. Longer series make this rarer: the experiments in the papers show estimates tightening as n grows from 100 to 1,000.
• Move the truth and check that the estimates follow it.

9Uncertainty, model checking, and limits

Standard errors by parametric bootstrap

A point estimate needs a measure of uncertainty. Because we can simulate, the parametric bootstrap is the natural tool: generate a fresh data set from the fitted model at θ̂, re-estimate the parameters from it, and repeat. The spread of the re-estimates approximates the sampling variability of θ̂. The Bootstrap button in step 8 does this six times (a real analysis would use hundreds) and reports the standard deviations. Both papers recommend this approach. Wieck-Sosa and Shalizi also state that a separate manuscript establishes asymptotic normality of the estimators, which would justify confidence sets of the familiar "estimate ± 2 standard errors" type.

Checking the fit with fresh features

Shalizi (2021) points out a convenient check. Draw additional random features that were not used for fitting. Compute them on the observed data and on many simulations at θ̂. If the observed values fall far in the tails of the simulated ones, the model at its best-fitting parameters still fails to reproduce some aspect of the data. Since the fresh features played no part in the fit, this check does not reuse the data in a circular way.

What the evidence covers and what it does not

The numerical experiments in the three sources use time series models: normal and t location families, the logistic map with and without observation noise, a Hénon map, the Lorenz-63 equations, a moving-average process with intractable noise, a state-space model, an SIR differential-equation model observed through binomial testing, and a structural time series with a change-point. Estimates concentrate around the truth and tighten as n grows from 100 to 1,000. None of these experiments is a spatial agent-based simulation. Shalizi (2021) motivates the method with ABMs, citing a spatial-network epidemic model with three free parameters and 630-dimensional data, and the theory covers any simulator whose output can be written as a function of the parameters and a stream of random inputs with fading influence. An ABM that reports aggregate time series fits that description. The grid epidemic on this page is a demonstration, not a validation study.

Conditions to think about for your own ABM

• The model must be identifiable from what you observe. Random features cannot recover a distinction that the observed output does not carry. While building this page, a version that observed only the positive-test series separated β and γ poorly: estimates slid along the valley from step 3. Adding the new-case series fixed it. That was a choice about what to observe, not about which summary to compute.
• Smooth dependence on parameters. The theorem assumes the characteristic function is analytic in θ. ABMs with sharp thresholds or phase transitions inside the parameter box deserve extra care, and integer-valued or categorical parameters fall outside the theory.
• Fading memory. Consistency needs the influence of past random inputs to decay. Models that lock in permanently, for example an epidemic that goes extinct and never returns, or a market that tips to one standard forever, break this. The small import probability in our model exists for this reason.
• Identification is not efficiency. 2p + 1 features guarantee a unique answer in the limit, not the smallest variance. Both sources mention refinements: weighting the distance by the inverse covariance matrix of the features, or treating the feature vector as Gaussian and maximizing the resulting synthetic likelihood (Wood, 2010).
• Output that is not a time series. For ABMs whose output is a spatial field or a network, the authors expect the principle to carry over with a different family of random features suited to that structure. They list this as future work.
• Computation. Every evaluation of Q costs s runs of the ABM, and a global search needs hundreds to thousands of evaluations.

A checklist

• Write down the p free parameters and a box that contains plausible values.
• Decide which output series are observed, and whether the process is stationary (time averages) or a one-off episode (rolling windows).
• Make the simulator accept a seed, and fix the seeds used inside the objective.
• Rescale each series to order 1 with constants chosen in advance.
• Draw 2p + 1 features once, with a window of m + 1 time points, and save them with your results.
• Before touching real data, simulate data at known parameters and confirm that you recover them.
• Report bootstrap standard errors and a fresh-feature check of the fit.