2. File Specification: Config Schema (config.json)

The config.json file provides high-level metadata about the experiment and defines the overall structure (or "flow") of the trials presented to the participant.

2.1. Metadata

These top-level keys describe the study and are used for dataset indexing and citation.

Key Type Description
experimentName String The full title of the experiment or study.
description String A brief paragraph describing the task.
paperDOI String The DOI link to the original publication (e.g., "https://doi.org/...").
taskType Array of Strings Keywords describing the cognitive task (e.g., "Social Cognition", "Decision Making").
responseType Array of Strings All response types used, e.g., ["multi-choice", "slider", "free-text"].
contributors Array of Strings List of researcher names who prepared the data.
stimuli_count Integer Total number of unique stimuli/trials defined for this experiment (i.e., the number of entries in trial.jsonl that are referenced by config.json).

The stimuli_count field provides a quick summary of dataset size. Participant demographics (participants_info) and the total number of ratings (judgment_count) are now specified in the Human Data files; see the Human Data Schema.

2.2. Experiment Flow (experimentFlow)

To control the order and grouping of multiple trials, an experimentFlow array is used. Each entry in this array is a dictionary describing a full flow definition (e.g., for a specific experimental condition). The list should enumerate every condition included in the experiment. If conditions differ so substantially that they require separate instructions, assets, or config logic, consider splitting them into separate experiment folders instead. Make this decision on a case-by-case basis.

Key Type Description
blocks Array of Arrays of Strings A nested array. Each inner array is a "block" of trials. Each string is a stimuli_id from trial.jsonl.
block_randomization Boolean If true, the order of the "blocks" (the inner arrays) is randomized for each participant.
stimuli_randomization Boolean If true, the order of the trial stimuli_ids within each block is randomized.
experimental_condition String (optional) An optional label identifying the experimental condition associated with this flow definition.

2.2.1. Example experimentFlow

"experimentFlow": [
  {
    "experimental_condition": "condition_1",
    "blocks": [
      [ "trial_1_1", "trial_1_2", "trial_1_3" ],
      [ "trial_2_1", "trial_2_2" ]
    ],
    "block_randomization": true,
    "stimuli_randomization": false
  },
  {
    "experimental_condition": "condition_2",
    "blocks": [
      [ "trial_3_1", "trial_3_2" ]
    ],
    "block_randomization": true,
    "stimuli_randomization": false
  }
]

In this example:

  • There are two experimental conditions. The interface will randomly present one of the two conditions to the participant. You do not need to specify the experimental condition field if there is only one condition.
  • Within the first experimental condition, there are two blocks of trials. Because block_randomization is true, the experiment will randomly present either Block 1 (["trial_1_1", ...]) or Block 2 (["trial_2_1", ...]) first.

2.3. Full config.json Example

Here is what a complete config.json file looks like, combining the metadata and the experiment flow. The stimuli_id strings (e.g., "trial_1_1") in the blocks array refer to the trial objects defined in your Trial Schema (trial.jsonl) file.

{
  "experimentName": "Epistemic Language Understanding",
  "description": "Participants observe short animations and answer questions about the agent's goals and beliefs.",
  "paperDOI": "https://doi.org/10.1111/example.doi.12345",
  "taskType": [
    "Social Cognition", 
    "Theory of Mind"
  ],
  "responseType": [
    "multi-choice",
    "single-slider"
  ],
  "contributors": [
    "Jane Doe",
    "John Smith"
  ],
  "stimuli_count": 48,
  "experimentFlow": [
    {
      "blocks": [
        [ "trial_1_1", "trial_1_2", "trial_1_3" ],
        [ "trial_2_1", "trial_2_2", "trial_2_3", "trial_2_4" ]
      ],
      "block_randomization": true,
      "stimuli_randomization": false
    }
  ]
}