Autoscale and disk safety

run_pipeline.sh checks CPU, RAM, Linux CommitLimit, and free disk before choosing threads and heap. Useful knobs:
export CTAKES_AUTOSCALE_MEM_PERCENT=70
export CTAKES_AUTOSCALE_COMMIT_PERCENT=75
export CTAKES_AUTOSCALE_DISK_RESERVE_MB=20480
export CTAKES_AUTOSCALE_MAX_THREADS=8
run_input.sh also sizes batch concurrency so multiple folders do not overrun memory or scratch disk.

Offline kit

Build an offline kit on a machine with internet access:
bash scripts/get_bundle.sh
bash scripts/get_java.sh
bash scripts/build_offline_kit.sh
That creates:
Ctakes_USD_offline_<tag>.tgz
Ctakes_USD_offline_<tag>.tgz.sha256
The offline kit includes:
repo files
CtakesBun-bundle/
build/tools
dictionary manifest/inventory files when present
release_inventory.json
.ctakes_deps/java/ when available
.ctakes_env
On the locked-down machine:
tar -xzf Ctakes_USD_offline_<tag>.tgz
cd Ctakes_USD_offline_<tag>
bash scripts/flight_check.sh

Validation

Quick environment check:
bash scripts/flight_check.sh
Optional 100-note validation:
bash scripts/validate_mimic.sh
validate_mimic.sh runs the sample notes under samples/mimic/ and compares against the checked-in manifest for the selected pipeline when the default 100-note pack is used.

Common failures

Java missing:
bash scripts/get_java.sh
export CTAKES_JAVA_HOME="$PWD/.ctakes_deps/java/current"
bash scripts/flight_check.sh
Bundle missing:
bash scripts/get_bundle.sh
bash scripts/flight_check.sh
RxNorm CSV folder missing: The normal run writes RxNorm rows into annotations.sqlite. If the site needs CSV files, rerun with:
bash scripts/run_pipeline.sh -i input/batch1 -o outputs/batch1_standard --writers standard
Memory error:
export CTAKES_AUTOSCALE_MEM_PERCENT=60
bash scripts/run_pipeline.sh -i input/batch1 -o outputs/batch1
Need the exact command without running notes:
bash scripts/run_pipeline.sh -i input/batch1 -o outputs/batch1 --dry-run
Need the exact paths the run will write:
bash scripts/run_pipeline.sh -i input/batch1 -o outputs/batch1 --print-write-plan

What not to change during a site run

Do not edit these for a normal STAR batch:
pipelines/combined/s_core_relations_smoke.piper
resources/dictionary_configs/KidneyStone_SDOH_offline.xml
tools/
CtakesBun-bundle/
If those change, rerun flight_check.sh, then run a small validation batch before processing real site notes.