Something went wrong. Try again.
A training to familiarize TIDE analysts with parquet datasets and python
Something went wrong. Try again.
3.4 kB · 77 lines
Python
at main
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778# pattern: Imperative Shell"""Exercise 5: Joins — MERGE with IN= flags, done sanely.
Before you start: read lessons/05_joins.md and run demos/demo05.py.
How to run (platform notes: README, "How to run the scripts"): python exercises/ex05_joins.py
Each TODO below is a task statement. Stuck? Lesson 05's "Your turn" sectionhas three hint tiers per TODO (nudge -> pseudocode -> partial code) — openthem one at a time. A NotImplementedError means "not started yet"; anAssertionError means "attempted, wrong result" (lesson 00). Success prints: All checks passed.
Part (b) is a miniature Sentinel enrollment-coverage QC check: whichdispensings have no matching drug-covered enrollment span? The data-modelground rules (dispensing grain, span multiplication, the date window) are inlesson 05 — read them first; the code is much easier once you know them."""
from pathlib import Path
import polars as pl
DATA_DIR = Path(__file__).parent.parent / "data"
def main() -> None: # ------------------------------------------------------------------ # # TODO 1: Read all three Parquet files into df_demographic, # df_enrollment, df_dispensing. # ------------------------------------------------------------------ # raise NotImplementedError("TODO 1: read demographic, enrollment, dispensing")
# ------------------------------------------------------------------ # # TODO 2(a): Inner join dispensing with demographic on PatID and store # the joined row count in inner_demo_count. Every dispensing has a # matching patient, so the count should equal df_dispensing.height. # ------------------------------------------------------------------ # raise NotImplementedError("TODO 2(a): inner join dispensing with demographic")
# ------------------------------------------------------------------ # # TODO 2(b): Count the dispensings covered by a drug-covered enrollment # span (DrugCov "Y", RxDate within Enr_Start..Enr_End). Store the count # of unique covered dispensings — grain PatID + RxDate + Rx — in # n_covered. # ------------------------------------------------------------------ # raise NotImplementedError("TODO 2(b): find drug-covered dispensings")
n_total = df_dispensing.height
# ------------------------------------------------------------------ # # TODO 2(c): Anti-join to count the dispensings NOT covered by any # drug-covered span; store the count in n_uncovered. # ------------------------------------------------------------------ # raise NotImplementedError("TODO 2(c): anti-join to find uncovered dispensings")
# --- Assertions ---------------------------------------------------- # assert inner_demo_count == df_dispensing.height, ( "Every dispensing should match a demographic row" ) assert n_covered < n_total, ( f"Covered dispensings ({n_covered}) should be less than total " f"({n_total}) — the ~5% uncovered dispensings should be missing" ) assert n_uncovered > 0, "Expected some uncovered dispensings — check your anti-join" assert n_uncovered == n_total - n_covered, "Uncovered = total - covered"
print(f" Total dispensings: {n_total:>8,}") print(f" Covered dispensings: {n_covered:>8,}") print(f" Uncovered dispensings: {n_uncovered:>8,}")
print("All checks passed.")
if __name__ == "__main__": main()