Skip to content
All projects
Research programmeArchitect

Global Outcome Tracking Hub — 7-Project TB Consolidation

BCHPR · INSPIRE TB research portfolio · 2023 – present

Mission-critical ETL hub aggregating TB patient-outcome data from 7 independent research projects into a unified REDCap database with longitudinal patient-journey tracking — 116,000+ records across 390+ treatment sites.

Highlights

  • Hub-and-spoke ETL aggregating Health Camps (HC-#####) · INSPIRE TB · Start4All (SC[1-5]-####) · Rapid TB (PP[1-6]-####) · Wave 11 Facility / HC · FujiLAM II (FC[1-9]-####).
  • Multi-project ID pattern recognition and normalisation across 7 distinct regex patterns.
  • Patient journey state machine: screening → TB detection → treatment → 6-month outcomes.
  • Automated data-drift detection via DataFrameComparator catches upstream schema changes before they corrupt the hub.
  • Chunked parallel REDCap upload, fault-tolerant and resumable after failure.
  • SQLite state tracking enables idempotent re-runs — no double counting even after mid-run crash.
  • ~8 min processing cycle on 20,000+ records per run.
  • Cut time-to-insight for treatment outcome reporting from weeks to real-time.