Deduplicate the daily export

Python · Python · beginner · greenfield

Deduplicates the export by order_id, keeping the latest row per id using pandas' built-in dedupe with a recency preference. Returns the cleaned frame.

Requirements

• Given a DataFrame of orders, keep one row per order_id.
• When an order_id repeats, keep the row with the most recent `updated_at`.
• Return the deduplicated DataFrame.

Files touched

• etl/dedupe.py

--- etl/dedupe.py
+import pandas as pd
+
+
+def dedupe_orders(df: pd.DataFrame) -> pd.DataFrame:
+    return df.drop_duplicates(subset="order_id", keep="latest")

Review this PR

Python practice