Something went wrong. Try again.
A training to familiarize TIDE analysts with parquet datasets and python
Something went wrong. Try again.
3.4 kB · 80 lines
Python
at main
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081# pattern: Imperative Shell"""Exercise 3: Filter and Select — WHERE, KEEP/DROP, and the Polars expression model.
Before you start: read lessons/03_filter_select.md and run demos/demo03.py.
How to run (platform notes: README, "How to run the scripts"): python exercises/ex03_filter_select.py
This is a fill-in-the-blank exercise. Replace each ____ with the rightfunction, method, attribute, or value. Until every blank is filled, runningthe file stops with an AttributeError or NameError that names '____' — thatmeans "you missed a blank" (lesson 00 shows how to read the traceback).When your answers are right you'll see: All checks passed.
SAS framing: - df.filter() ≈ WHERE (in DATA step or PROC) - df.select() ≈ KEEP - df.drop() ≈ DROP - df.sort() ≈ PROC SORT"""
from pathlib import Path
import polars as pl
DATA_PATH = Path(__file__).parent.parent / "data" / "dispensing.parquet"
def main() -> None: # ------------------------------------------------------------------ # # TODO 1: Read data/dispensing.parquet into a DataFrame. # Blank: the polars function that reads a Parquet file (same as ex02). # ------------------------------------------------------------------ # df = pl.____(DATA_PATH)
# ------------------------------------------------------------------ # # TODO 2: Filter for dispensings in the year 2021. # Blanks: the row-picking method, and the year to compare against. # ------------------------------------------------------------------ # df_2021 = df.____(pl.col("RxDate").dt.year() == ____)
# ------------------------------------------------------------------ # # TODO 3: From the 2021 dispensings, keep rows with RxAmt > 30 AND # RxSup >= 30. The & combines the two conditions; note the parentheses. # Blanks: the two threshold values. # ------------------------------------------------------------------ # df_filtered = df_2021.filter((pl.col("RxAmt") > ____) & (pl.col("RxSup") >= ____))
# ------------------------------------------------------------------ # # TODO 4: Keep only the PatID, RxDate, Rx, RxAmt columns (drops RxSup). # Blank: the column-picking method (KEEP, not WHERE). # ------------------------------------------------------------------ # filtered = df_filtered.____("PatID", "RxDate", "Rx", "RxAmt")
# ------------------------------------------------------------------ # # TODO 5: Sort by RxDate, newest first. # Blank: the keyword argument's value. # ------------------------------------------------------------------ # filtered = filtered.sort("RxDate", descending=____)
# ------------------------------------------------------------------ # # TODO 6: Count the filtered rows. # Blank: the row-count attribute (no parentheses). # ------------------------------------------------------------------ # n_2021 = filtered.____
print(f"Filtered rows: {n_2021}")
# --- Assertions ---------------------------------------------------- # assert n_2021 == 15_698, f"Expected 15698 filtered rows, got {n_2021}" assert set(filtered.columns) == {"PatID", "RxDate", "Rx", "RxAmt"} assert filtered.height == 15_698 assert filtered["RxDate"].dt.year().max() == 2021
print("All checks passed.")
if __name__ == "__main__": main()