What lives where
| Location | Contents |
|---|---|
| GitHub | Scripts, configuration, taxonomies, templates, prompts, documentation and small examples. |
| Google Drive or institutional storage | Raw corpora, processed data, Pass 1 outputs, informed Pass 2 outputs, blind-validation outputs, adjudication files, logs and review workbooks. |
| Colab or local Python | Execution environment for the reproducible pipeline. |
Pass 2 inputs
The production Pass 2 scripts require both Pass 1 annotation files:
- Sense Pass 2 reads Sense Pass 1 and Function Pass 1.
- Function Pass 2 reads Function Pass 1 and Sense Pass 1.
This enables informed critical review. The --blind option hides both Pass 1 outputs for a separate validation sample.
Keep blind and informed outputs separate
outputs/en/sense_pass2_informed.csv
outputs/en/function_pass2_informed.csv
outputs/en/sense_pass2_blind.csv
outputs/en/function_pass2_blind.csvDo not append blind and informed rows to the same CSV, because the review condition is part of the provenance.
Required identifiers
All files must preserve the stable row_id. This is what allows sampled sentences, sense annotations, function annotations and human review decisions to be joined safely.
Secrets and large data
API keys and large corpus files must not be committed to GitHub. Keep secrets in environment variables or protected storage and preserve corpus licensing conditions.