Survey Data Import Workflow

This document describes the current PRISM workflow for importing survey definitions from Excel and converting wide survey exports (.xlsx or LimeSurvey .lsa) into a PRISM/BIDS-style dataset.

Key points:

  • Survey and biometrics templates are imported from separate Excel files.

  • Survey templates can be multilingual (DE/EN) in the library.

  • Dataset sidecars must be single-language to validate against the stable schema; use --lang at conversion time.

1) Start from the provided Excel templates

Do not combine survey + biometrics in one Excel.

  • Survey template: demo/import_survey_template/survey_import_template.xlsx

  • Biometrics template: demo/import_survey_template/biometrics_import_template.xlsx

Both templates include placeholder participant-facing instruction text (DE/EN) that you should replace with the exact wording used in your study.

2) Survey Excel format (minimal vs advanced)

The survey importer is header-based (column names). The template contains all supported columns.

Minimal columns (start here)

Per item/row:

  • VariableName (item ID; becomes the TSV column name)

  • Group (instrument ID; becomes the survey template file name survey-<group>.json)

  • One of:

    • Question_de / Question_en (recommended), or

    • Question (fallback, single-language)

  • One of:

    • Scale_de / Scale_en (recommended), or

    • Scale (fallback)

Advanced optional columns

Item-level validation/semantics:

  • Units, DataType, AllowedValues

  • MinValue, MaxValue, WarnMinValue, WarnMaxValue

  • TermURL, Relevance

  • AliasOf, Session, Run

Instrument-level metadata (repeat in any row; first non-empty per Group wins):

  • OriginalName_de, OriginalName_en, ShortName

  • Version_de, Version_en, StudyDescription_de, StudyDescription_en

  • Citation, Construct, Keywords

  • Instructions_de, Instructions_en (participant-facing instructions)

  • Respondent, AdministrationMethod, SoftwarePlatform, SoftwareVersion

  • Languages, DefaultLanguage, TranslationMethod

Multilingual (i18n) JSON output

The imported survey library JSON uses the repo’s i18n convention:

  • Study.OriginalName, Study.Version, Study.Description, Study.Instructions are language maps like { "de": "…", "en": "…" }.

  • Each item’s Description is { "de": "…", "en": "…" }.

  • Each item’s Levels is a dict of dicts: { "0": {"de": "Nie", "en": "Never"}, ... }.

3) Import the survey Excel into a library

Recommended:

python prism_tools.py survey import-excel --excel metadata.xlsx --output survey_library

Or (equivalent low-level script):

python scripts/excel_to_library.py --excel metadata.xlsx --output survey_library

This produces survey_library/survey-*.json survey templates.

4) Convert wide survey exports into a dataset

Convert a wide .xlsx export (one row per participant, one column per item) into a PRISM dataset:

python prism_tools.py survey convert \
  --input responses.xlsx \
  --library survey_library \
  --output my_dataset \
  --lang de

Notes:

  • --lang selects the language for i18n templates and writes schema-valid single-language sidecars.

  • Survey IDs / task names are normalized to alphanumeric for BIDS-safe filenames. For example, demo_survey becomes demosurvey in filenames; the mapping report prints the normalized value.

5) Validate

python prism.py my_dataset

Biometrics (separate Excel)

Biometrics templates are imported separately:

python prism_tools.py biometrics import-excel --excel biometrics.xlsx --output biometrics_library