Pipelines and writers

Pipeline keys

run_pipeline.sh maps short keys to .piper files:

Key	File	Use
`s_core_relations_smoke_drug`	`pipelines/combined/s_core_relations_smoke.piper`	Default STAR run. Sections, dictionary lookup, WSD, assertion, drug NER, relations, smoking status, and reporting.
`s_core_relations_fast`	`pipelines/combined/s_core_relations_fast.piper`	Faster combined run. No WSD, smoking, or drug NER.
`core_sectioned_smoke`	`pipelines/combined/core_sectioned_smoke.piper`	Older combined shape.
`core`	`pipelines/core/core_wsd.piper`	Core concept run.
`sectioned`	`pipelines/sectioned/sectioned_core_wsd.piper`	Sectioned concept run. Not the normal STAR output path.
`smoke`	`pipelines/smoke/sectioned_smoke_status.piper`	Smoking status focused run.
`drug`	`pipelines/drug/drug_ner_wsd.piper`	Drug-focused run with RxNorm CSV when CSV writers are enabled.

The normal STAR command does not pass --pipeline because the default is already s_core_relations_smoke_drug.

pipelines/combined/s_core_relations_smoke.piper starts with:

threads 3

The wrapper replaces that line with the autoscaled or user-provided thread count before cTAKES runs. The default pipeline loads or adds these stages:

Stage	File line behavior
Tokenizer/POS/chunker	`TsFullTokenizerPipeline`, `ContextDependentTokenizerAnnotator`, `POSTagger`, `TsChunkerSubPipe`.
Section fallback	`tools.segment.FallbackDocumentSegmentAnnotator`.
Dictionary lookup	`TsDictionarySubPipe`.
Local WSD	`tools.wsd.BestConceptDisambiguatorAnnotator` with resources under `resources/tools/wsd/`.
Assertion	`TsAttributeCleartkSubPipe` plus `tools.assertion.DefaultSubjectAnnotator`.
Drug NER	`tools.drug.deterministic.DeterministicDrugMentionAnnotator`.
Relations	`deterministic_fast_relation_subpipe`.
Modifier cleanup	`ModifierSpanNormalizer` and `DeterministicAnnotationCanonicalizer`.
Smoking status	deterministic wrappers around smoking step 1 and step 2.
Progress	`tools.debug.ProgressLogger`.
Writers	Present in the source `.piper`, then changed by `run_pipeline.sh` based on `--writers`.

The source .piper files list XMI/CSV/HTML writers, but the wrapper edits the effective pipeline before the run.

Profile	What the wrapper keeps
`sqlite`	Removes XMI, HTML, RxNorm CSV, concept CSV, and CUI count BSV. Adds `AnnotationSqliteWriter`. This is the default.
`standard`	Keeps XMI, concept CSV, RxNorm CSV, and CUI count BSV. Removes HTML.
`all`	Keeps XMI, CSV/BSV, and HTML.
`csv`	Keeps CSV/BSV writers and removes XMI/HTML.
`lean`	Removes XMI, HTML, and RxNorm CSV.
`none`	Removes all reporting writers.

The exact file cTAKES ran is always copied to:

outputs/<batch>/run_pipeline.effective.piper