Data engineering

Deep dives into data systems I built end to end

Each project breaks down motivation, architecture, implementation strategy, outcomes, and tradeoffs so teams can evaluate both execution quality and thinking model.

Architecture-first deliveryEvent-driven + lakehouse systemsReliability and governance by design

Healthcare cloud data platform

The team needed one trusted platform to combine wearable streams and operational batch feeds without splitting analytics across separate systems.

Key outcomes

  • Platform scope: One shared platform for streaming + batch analytics delivery.
  • Delivery alignment: Cross-functional standards coordinated across a 4-engineer team.
KafkaADLS Gen2Delta LakeSparkDruidSupersetAzure
View deep dive

Clinical + wearable medallion pipeline

Healthcare analytics needed trustworthy progression from raw events to governed metrics while satisfying compliance requirements for sensitive data.

Key outcomes

  • Data trust model: Medallion architecture established as the standard for sensitive analytics datasets.
  • Compliance posture: Governance controls aligned to HIPAA/GDPR-oriented operating requirements.
KafkaDelta LakeSparkAzure MonitorLog AnalyticsRBAC
View deep dive

Legacy-to-cloud data modernization

Legacy services and fragmented pipelines slowed analytics delivery and made reliability difficult to scale across teams.

Key outcomes

  • Modernization progress: Legacy analytics workloads transitioned to cloud-native platform patterns.
  • Operational resilience: Airflow-based backfill and SLA workflows formalized production operations.
PythonKafkaAirflowSparkSnowflakedbtFastAPIAzure
View deep dive