Project

AutomataGPT

Forecasting and ruleset inference for 2D cellular automata, from the AutomataGPT paper (Advanced Science, under revision).

Read the paper →
AutomataGPT overview diagram from the paper
Figure: AutomataGPT architecture and evaluation tasks combining autoregressive rollouts with a ruleset inference head (adapted from the paper).

AutomataGPT pairs a causal transformer forecaster with a ruleset inference head to learn cellular automata dynamics directly from rollouts. Trained across diverse 2D rules and neighborhoods, the model delivers long-horizon stability while recovering hidden transition rules from short sequences.

Paper takeaways

  • Autoregressive attention captures multi-neighborhood interactions and maintains sharp structure over long rollouts.
  • A ruleset head infers transition probabilities directly from observed states, supporting in-context rule discovery without labeled rules.
  • Pretraining across many CA families boosts zero-shot transfer to unseen rules and lattices compared to convolutional NCA baselines.

Results that matter

  • Stable predictions for gliders, oscillators, and complex pattern cascades across diverse 2D automata.
  • Ruleset inference recovers high-fidelity transition tables from short partial trajectories, enabling rapid simulator cloning.
  • Ablations show attention depth and shared tokenization materially improve both forecasting and rule recovery.