Skip to content

Data Model

The types that make up a loaded experiment, and the columnar store that holds live entity data while it runs. Everything here is the post-load form from internal/engine/model/types.go (plus regime.go, planner.go, distributions.go). The Raw* parse-time mirrors are covered in Config YAML and Process Flow → loading.

The types fall into six conceptual groups. The first five describe the loaded Config object graph; the sixth is the runtime store.

Solid edges are containment (a parent owns a slice of children); the dashed edge is a reference by index. Boxes are coloured by conceptual group.

The loaded Config object graph Config contains taxonomy definitions, catalogs, state machines, a resolved regime and a derived execution plan. State machines contain fields, families, reference slots, states and messages_accepted groups. States contain events, probability distributions and messages_accepted groups. Events contain conditions and actions. Actions emit messages. Config TaxonomyDefinition Catalog StateMachine Regime derived ExecutionPlan derived Field FieldFamily ReferenceSlot State MessagesAccepted machine-level Event ProbabilityDistribution [turns][1+events] MessagesAccepted state-level Condition Action Message SimulationConfig Schema Lifecycle Execution Messaging & Scheduling
Each containment edge is one-to-many (a Config has many StateMachines, a State has many Events, …). Cross-references by index — Action→State (transition), Action→StateMachine (new_entity), ReferenceSlot→target StateMachine — are described per group below rather than drawn, to keep the graph legible.

Config is the root object the loader produces. It holds the machine catalog, the shared taxonomy dimension catalog, named lookup Catalogs, and two derived members — the resolved Regime and the ExecutionPlan (both computed at load, neither declared directly).

TypeKey fieldsNotes
ConfigName, Taxonomy, Catalogs, PolicyTables + PolicyTableIndex, StateMachines + StateMachineIndex, MessageNames, MessageCascadeDepth, Regime, Plan, TimingOrdered []*StateMachine with a name→index map. PolicyTables follow the same pattern as Catalogs. GetStateMachine(name) is the named lookup.
PolicyTableName, Policies map[string][]ProbabilityDistributionA named set of probability matrices. Policy keys are taxonomy values; the matrix layout matches the owning state’s events.
TaxonomyDefinitionName, Dependency, Distribution, ConditionalDistributions, Weights, Lookups, Options, Values + ValueIndexOne of five sampling strategies. Dependency names another entry, forming a DAG resolved into Config.TaxonomyOrder.
CategoricalDistributionmap[string]float64Implements UnmarshalYAML to coerce mixed int/float and Normalize().
CatalogName, Entries, AttrNames + AttrIndex, SKUIndexA named table of attribute rows (e.g. products). Backs catalog fields and field families.
  • Five taxonomy strategies: Distribution (flat probabilities), ConditionalDistributions (keyed by a dependency value), Lookups (deterministic 1:1), Weights (integer weights keyed by a dependency), or Options (uniform random). See Config YAML → taxonomy for examples of each.
  • Value interning: every categorical value is interned into Values []string + ValueIndex. Taxonomy fields are stored on entities as dense ValueIdx integers (init -1), compared with integer ops, and rendered back to strings only at export.
  • Catalog-backed fields either sample a CatalogIdx (func: random) or derive a value from an attribute (func: from_catalog).

A StateMachine defines the shape of an entity: its fields, its typed references to other entities, and its states.

TypeKey fieldsNotes
StateMachineName, LoadFile, ReferenceSlots + ReferenceIndex, InitState, Fields, Families, States + StateIndex, MessagesAccepted, FieldMapping, FieldIndex, AgeIdx/TurnIdx, IntCount/FloatCount/StringCountCarries everything the store needs to allocate and address an entity’s columns.
FieldName, ValueType, Func, TaxonomyName, CatalogName, InitInt/InitFloat/InitString, IsSystem, IsFamilyThe flat unit. ValueType is one of String/Int/Float.
ReferenceSlotName, EntityIdxEntityIdx is the index into Config.StateMachines of the referenced type (-1 until linkReferences resolves it).
FieldFamilyName, CatalogName, BaseIdx, Size, InitExpands to one field per catalog entry: stock[0], stock[1], …
FieldSlotValueType, SliceIndexOne entry of FieldMapping: maps a flat FieldIdx to its backing column.
  • References, not parent/child: a machine declares typed reference slots (e.g. Verification references merchant). They define connectivity and feed the planner’s hazard graph but do not constrain spawn order.
  • Field flattening: the YAML categories (taxonomy, catalog, families, counters, dimensions, measures) flatten into one []Field. Four system fields are prepended (StateMachine, ID, TurnNumber, Age), then one <ref>_id system string field per reference slot. counters→int, dimensions→string, measures→float.
  • No reflection: buildFieldMapping precomputes FieldMapping so the columnar store resolves a FieldIdx to (ValueType, SliceIndex) directly.

The Markov layer: each State carries the events that may fire from it and a per-turn probability matrix.

TypeKey fieldsNotes
StateName, Description, Decision, EventSelectorKind, Events, Probabilities, MessagesAccepted, PolicyDecision is materialized at load ("stochastic" or "first_wins"); EventSelectorKind is the resolved SelectorKind for event dispatch. State-level MessagesAccepted groups take precedence over machine-level ones. Policy (nil when unused) links to a policy table for per-entity probability override.
ProbabilityDistribution[]float64One row of the matrix. Normalize() scales it to sum 1.0 at load.

What happens when an event fires: conditions gate it, then actions produce effects.

TypeKey fieldsNotes
EventName, Conditions, ActionsAll conditions must pass (AND) for the actions to run.
ConditionField (FieldIdx), Operator, ValueType, IntValue/FloatValue/StringValue, Bracket, PathOperators: is, atLeast, atMost (strings support is only). Path is a compiled *PathExpr for indirect field access (self.ref.field); nil for direct fields.
ActionType, StateIdx, Field, Operator, Value, EntityIdx, ReferenceIdx, MessageIdx, Payload, Delay, References, Bracket, ApplyAll, Path, ValuePathOne struct covers every action type; unused index fields are -1. Path/ValuePath are compiled *PathExpr for indirect field/reference access.
BracketExprFamilyBaseIdx, FamilySize, Source, SourceIdxResolves family[payload.key | self.field | sender.field] to a concrete member index at runtime.
PathExprHops []PathHop, TerminalFieldIdx, TerminalMachineIdx, TerminalIsRefA compiled self.ref.ref.field expression: a chain of PathHop dereferences terminating in a field or reference slot on the target machine. Compiled at load by parsePathExpr.
PathHopRefSlotIdx, TargetMachineIdxA single reference-slot dereference. At runtime, GetReference resolves the entity handle at each hop.

Runtime action types (after loader normalization):

  • transition — set a new state on self; resets Age to 0.
  • updateset/add/subtract on a self field; optionally apply: all over a family or a bracket-resolved member.
  • new_entity — spawn another machine, optionally binding the child’s reference slots via a References map.
  • set_reference — resolve and write a reference slot.
  • emit_message — enqueue a Message to a reference slot or back to the sender, with an optional delay.

Indexed references: actions store integer indices — StateIdx, EntityIdx, ReferenceIdx, MessageIdx — resolved in linkReferences. update_within_state is accepted as an alias for update; update_reference is desugared into an emit_message plus an auto-generated handler (see Scheduler → messaging).

TypeKey fieldsNotes
MessageNameIdx, Sender, PayloadImmutable; constructed at runtime by emit_message, never parsed from YAML. Payload values are positional.
MessagesAcceptedMessage, MessageIdx, Decision, SelectorKind, Handlers, ProbabilitiesGroups handlers for a single message type. Decision is always materialized ("stochastic" or "first_wins"). SelectorKind controls dispatch: FirstValidWins or StaylessProbabilistic. Lives on a State (authoritative) or the StateMachine (fallback). Probabilities is an age-indexed matrix with no stay column — each column maps directly to a handler.
HandlerName, Conditions, Actions, MessageIdxA resolved message handler within a MessagesAccepted group. Mirrors Event: named, with conditions gating execution and actions firing when selected. MessageIdx is copied from the parent group.
RegimeClock, Work, Update, OrderFour declared scheduling axes. Concurrency is not an axis — the engine derives it.
ExecutionPlanSeedTypes, Partitions, IndependentDerived by finalizeSchedule: seed types (no inbound spawn edge), and union-find partitions.

Several named int types prevent accidental transposition between distinct index spaces. The compiler rejects passing one where another is expected:

TypeIndexes into
FieldIdxa machine’s Fields / FieldMapping
SlotIdxa per-type backing slice (ints/floats/strings)
ValueIdxTaxonomyDefinition.Values
ReferenceIdxa machine’s ReferenceSlots
MessageIdxConfig.MessageNames
CatalogIdxCatalog.Entries
PolicyTableIdxConfig.PolicyTables
SelectorKindenum: FirstValidWins, ProbabilisticWithStay, StaylessProbabilistic

At experiment time the live entities of each machine type are stored together, column-wise, in an EntityStore (store.go / store_archetype.go). The loaded Config types above are the read-only schema; the store holds the mutable rows. One Archetype holds every entity of one machine type as a struct-of-arrays, and code addresses an individual entity by an EntityHandle{machineIdx, row} — its position in that storage.

The archetype store as struct-of-arrays One archetype for the Merchant machine. Columns hold States, integer fields, float fields, string fields, reference handles and NextWake. Rows zero to two are active; rows three and four are pending spawns. An EntityHandle points at machine zero, row two. Archetype["Merchant"] — struct-of-arrays States IntColumns FloatCols StringColumns RefColumns NextWake Age visits mrr email verif. merchant row 0 Acquired 3 7 100.0 a@x verified {2,0} 0 row 1 Visitor 1 2 20.0 unver. {-1,-1} 0 row 2 Visitor 0 1 45.0 unver. {-1,-1} 0 row 3 Visitor 0 0 10.0 unver. {-1,-1} 0 row 4 Visitor 0 0 59.0 unver. {-1,-1} 0 active pending ActiveCount = 3 TotalCount = 5 EntityHandle{machineIdx:0, row:2} an entity is a (machineIdx, row) coordinate — never scanned for
Rows [0, ActiveCount) are live; [ActiveCount, TotalCount) are pending spawns made visible by ActivatePending() at the next turn. References are stored as target EntityHandles; the <ref>_id strings are derived from them only at export.
TypeRole
EntityStore (interface)Create/activate entities, Access(handle), ForEach, get/set references and state, export fields. Abstracts the storage layout.
ArchetypeEntityStoreThe only implementation: archetypes []*Archetype + a name→index map.
ArchetypeOne per machine type. Parallel IntColumns/FloatColumns/StringColumns, a States column, RefColumns (per slot), a NextWake column, ActiveCount/TotalCount, and an optional snapshot.
ArchetypeViewA cursor over one row: GetInt/SetInt/AddInt (and float/string), GetReference, GetState, GetNextWake, GetMachine.
EntityHandleThe {machineIdx, row} coordinate. O(1) access via FieldMapping; never scanned.