Something went wrong. Try again.
A training to familiarize TIDE analysts with parquet datasets and python
Something went wrong. Try again.
3.3 kB · 80 lines
Python
at main
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081# pattern: Imperative Shell"""Exercise 2: Read and Inspect — PROC CONTENTS + PROC PRINT (OBS=) + PROC MEANS.
Before you start: read lessons/02_read_and_inspect.md and run demos/demo02.py.
How to run (platform notes: README, "How to run the scripts"): python exercises/ex02_read_and_inspect.py
This is a fill-in-the-blank exercise. Replace each ____ with the rightmethod, attribute, or value. Until every blank is filled, running the filestops with an AttributeError or NameError that names '____' — that means"you missed a blank" (lesson 00 shows how to read the traceback). When youranswers are right you'll see: All checks passed.
SAS framing: - df.schema ≈ PROC CONTENTS (variable names, types) - df.head(10) ≈ PROC PRINT (OBS=10) - df.describe() ≈ PROC MEANS (min, max, mean, std, etc.) - df.null_count() ≈ counts of missing values (like PROC FREQ with MISSING)"""
from pathlib import Path
import polars as pl
DATA_PATH = Path(__file__).parent.parent / "data" / "dispensing.parquet"
def main() -> None: # ------------------------------------------------------------------ # # TODO 1: Read data/dispensing.parquet into a DataFrame. # Blank: the polars function that reads a Parquet file. # ------------------------------------------------------------------ # df = pl.____(DATA_PATH)
# ------------------------------------------------------------------ # # TODO 2: Print the shape — a (rows, columns) tuple. # Blank: the attribute (no parentheses) holding that tuple. # ------------------------------------------------------------------ # print(f"Shape: {df.____}")
# ------------------------------------------------------------------ # # TODO 3: Print the schema (column names and dtypes). # Blank: the attribute (no parentheses) mapping names to dtypes. # ------------------------------------------------------------------ # print(f"Schema: {df.____}")
# ------------------------------------------------------------------ # # TODO 4: Print the first 10 rows. # Blank: how many rows to show. # ------------------------------------------------------------------ # print("Head (10):") print(df.head(____))
# ------------------------------------------------------------------ # # TODO 5: Print summary statistics. # Blank: the method (with parentheses) that summarizes every column. # ------------------------------------------------------------------ # print("Describe:") print(df.____())
# ------------------------------------------------------------------ # # TODO 6: Print the null count for each column. # Blank: the method that counts missing values per column. # ------------------------------------------------------------------ # print("Null counts:") print(df.____())
# --- Assertions ---------------------------------------------------- # assert df.shape[0] == 199_613, f"Expected 199613 rows, got {df.shape[0]}" assert "RxDate" in df.columns assert df.schema["RxDate"] == pl.Date, f"Expected Date, got {df.schema['RxDate']}" assert "PatID" in df.columns and df.schema["PatID"] == pl.Int64
print("All checks passed.")
if __name__ == "__main__": main()