2. File Specification: Config Schema (config.json)

2.1. Metadata

These top-level keys describe the study and are used for dataset indexing and citation.

The stimuli_count field provides a quick summary of dataset size. Participant demographics (participants_info) and the total number of ratings (judgment_count) are now specified in the Human Data files; see the Human Data Schema.

2.2. Experiment Flow (experimentFlow)

The experimentFlow array defines the structure and order of the trials presented to the participant. Each entry is one experimental condition, and the interface presents a single condition (chosen uniformly at random) to each participant. The list should enumerate every condition included in the experiment. If conditions differ so substantially that they require separate instructions, assets, or config logic, consider splitting them into separate experiment folders instead. Make this decision on a case-by-case basis.

Each condition holds one or more sequences. A sequence is a fully-specified ordering of the condition's blocks; the interface presents a single sequence (chosen uniformly at random) to each participant. Randomization is expressed by enumerating sequences — rather than flagging a block to be shuffled, you write out each concrete ordering you want as its own sequence. This keeps every ordering shown to participants explicit and reproducible.

Note: the former randomization flags (block_randomization, stimuli_randomization) have been removed. All order randomization is now expressed directly by listing multiple sequences — each a concrete, explicit ordering.

2.2.1. Example experimentFlow

"experimentFlow": [
  {
    "condition": "condition_1",
    "sequences": [
      {
        "seq_id": "seq_1",
        "blocks": [
          [ "instruction_01", "test_trial_01" ],
          [ "trial_1_1", "trial_1_2", "trial_1_3" ]
        ]
      },
      {
        "seq_id": "seq_2",
        "blocks": [
          [ "instruction_01", "test_trial_01" ],
          [ "trial_1_3", "trial_1_2", "trial_1_1" ]
        ]
      }
    ]
  },
  {
    "condition": "condition_2",
    "sequences": [
      {
        "seq_id": "seq_1",
        "blocks": [
          [ "instruction_02", "test_trial_02" ],
          [ "trial_2_1", "trial_2_2", "trial_2_3" ]
        ]
      }
    ]
  }
]

In this example:

• There are two experimental conditions. The interface randomly presents one of the two conditions to each participant. You do not need to specify the condition label if there is only one condition.
• condition_1 has two sequences (seq_1, seq_2). Each participant assigned to this condition sees one of them, chosen uniformly at random, and the seq_id they saw is recorded with their data. Both sequences share the same first block (an instruction module "instruction_01" and a test trial "test_trial_01"); they differ only in the order of the three trials in the second block — [trial_1_1, trial_1_2, trial_1_3] versus [trial_1_3, trial_1_2, trial_1_1]. This is how order randomization is expressed: enumerate the orderings you want.
• condition_2 has a single sequence (one fixed order).
• Within a sequence, blocks is a list of blocks; each block is a list of component ids presented in that order.
• All IDs (for instructions, test trials, quizzes, and regular trials) must match the id fields defined in their respective files (instruction.jsonl or trial.jsonl).

2.3. Trial Layout Configuration (trial_layout_config)

The optional trial_layout_config object controls the visual layout of trial panels (stimuli and queries) rendered during the experiment. By default, the interface uses a two-column layout with the left panel displaying stimuli and the right panel displaying queries. Use this configuration to adjust the column arrangement and the default width of each panel.

2.3.1. Example

Two-column layout

"trial_layout_config": {
  "layout": "2-columns",
  "default_width": ["40%", "50%"]
}

The trial interface will render two side-by-side columns where the stimuli panel takes up 40% of the width and the queries panel takes up 50%, leaving a small gap between them.

Single-column layout

"trial_layout_config": {
  "layout": "1-column",
  "default_width": ["80%"]
}

The trial interface will stack stimuli above queries in a single column that occupies 80% of the viewport width, centered on the page.

2.4. Full config.json Example

Here is what a complete config.json file looks like, combining the metadata and the experiment flow. The id strings in the blocks array can refer to:

• Trial objects from your Trial Schema (trial.jsonl) file (e.g., "trial_1_1")
• Instruction, test trial, or comprehension quiz modules from your Instruction Schema (instruction.jsonl) file (e.g., "instruction_01", "test_trial_01", "comprehension_quiz_01")

All IDs must match the id fields defined in their respective source files.

{
  "experimentName": "Epistemic Language Understanding",
  "description": "Participants observe short animations and answer questions about the agent's goals and beliefs.",
  "paperDOI": "https://doi.org/10.1111/example.doi.12345",
  "taskType": [
    "Social Cognition",
    "Theory of Mind"
  ],
  "responseType": [
    "multi-choice",
    "single-slider"
  ],
  "contributors": [
    "Jane Doe",
    "John Smith"
  ],
  "stimuli_count": 48,
  "trial_layout_config": {
    "layout": "2-columns",
    "default_width": ["40%", "50%"]
  },
  "experimentFlow": [
    {
      "sequences": [
        {
          "seq_id": "seq_1",
          "blocks": [
            [ "instruction_01", "test_trial_01", "trial_1_1", "trial_1_2", "trial_1_3" ],
            [ "trial_2_1", "trial_2_2", "trial_2_3", "trial_2_4", "comprehension_quiz_01" ]
          ]
        }
      ]
    }
  ]
}