Appendix - Trial Schema

This appendix contains additional examples and detailed specifications for the Trial Schema.

A.1. Example with Multiple Media URLs and Dimensions

When multiple media URLs are provided, they are rendered side by side. You can specify dimensions for each using the dimension array:

{
  "id": "trial_comparison_01",
  "stimuli": [
    {
      "input_type": "img",
      "title": "Compare the Two Scenarios",
      "fontsize": 20,
      "media_url": [
        "assets/scenario_a.png",
        "assets/scenario_b.png"
      ],
      "dimension": [
        { "width": 400, "height": 300 },
        { "width": 400, "height": 300 }
      ]
    }
  ],
  "queries": [
    {
      "prompt": "Which scenario shows the agent's goal more clearly?",
      "type": "multi-choice",
      "tag": "scenario_comparison",
      "option": ["Scenario A", "Scenario B", "Equally clear"],
      "required": true
    }
  ]
}

In this example, two images are displayed side by side, each with dimensions of 400x300 pixels. If dimension is not specified but width and height are provided, all media URLs will use those same dimensions.

A.2. Slider Config (slider_config)

This object is required for single-slider and multi-slider types.

Key Type Description
min Number The minimum value of the slider (e.g., 0).
max Number The maximum value of the slider (e.g., 100).
default_value Number The value where the slider handle starts.
labels Array of Objects An array of label objects to display under the slider. See table below.
show_label_values Boolean If true, displays the numeric value (from value) alongside the text (from label).
num_clicks Number (optional) Default is 1. If set to a value greater than 1, the slider becomes a multi-click slider, allowing the user to click the slider N times. Each click records a separate value. Useful for collecting multiple ratings or repeated measurements on the same slider.

Each object in the labels array has the following structure:

Key Type Description
value Number The numeric position on the slider where this label should appear.
label String The text to display at that position (e.g., "Neutral", "Strongly Agree").

A.3. Optional Stimulus Fields

The following fields are optional for each stimulus object in the stimuli array:

Field Type Description
title String A heading displayed before the stimulus.
fontsize Number Font size for the title (in pixels).
width Number Width of the media in pixels (applies to all media URLs if dimension is not specified).
height Number Height of the media in pixels (applies to all media URLs if dimension is not specified).
dimension Array of Objects When multiple media_urls are provided, specify dimensions for each. Each object has width and height (Numbers). The array length must match the media_url array length.

A.4. Symbolic Representation (symbolic_representation/)

The symbolic_representation/ directory is optional. When an experiment provides it, it holds structured world-state for models — grid worlds, physics scenes, event sequences — typically one JSON file per trial or condition.

When the directory is present, the following contract applies:

held_out/exp1/
  trial.jsonl
  symbolic_representation/
    trial_comparison_01.json     # linked to trial_comparison_01 by name
    trial_comparison_02.json

A full content schema for the files inside symbolic_representation/ is out of scope for now. The corpus contains several incompatible top-level shapes across experiments, and specifying a single structure requires dedicated design work. Validation therefore covers only that files parse as JSON and can be linked to trials.