Outdated data
Old history piles up
The system keeps learning from every past week, even months-old ones, as if they mattered as much as last week. It's what happens when nobody cleans up a data pipeline.
Shelf-Life Guard
Bad data quietly makes delivery promises wrong. We built a safeguard that fixes most of it.
When a courier picks up a package, the system tells the customer when it will arrive. That estimate comes from an AI model that relearns every week from recent deliveries. If the delivery records feeding it go bad, the promises go bad too, and nothing on screen warns anyone.
Backed by trained model updates across experiments on real deliveries from Chongqing, all scored against the real delivery times.
Each update retrains the models from scratch in about seconds, so they're easy to keep current as new data comes in.
of lost accuracy restored
of bad data caught
Same package, two answers
calculated live on this page
Loading the prediction model…
Without the Guard
–
With the Guard
–
Part 1 of 3
What goes wrong with the data
Old data wasn't the real problem. Unreliable sources were.
Nobody hacked anything. We recreated three ordinary ways delivery data goes bad and measured what each one did to the promises.
Outdated data
Stats stop updating
The courier and neighborhood averages the model reads stop refreshing at the end of June, and the model keeps using them. It's what happens when a nightly job fails and nobody notices.
Sloppy scans
Sloppy scanning
couriers stop scanning at the door. They mark a whole stack delivered at the end of the shift, and their device uploads some records twice. To the model, their deliveries look hours slower than they really were.
Old data wasn't the problem
- More history never hurt. Learning from every past week was as accurate as learning from the last .
- Stats that stopped updating cost nothing: min per package.
- What does go stale is the model. One that's never retrained gets min worse per package.
- Bad sources mattered far more: sloppy scans added min of error per package, outdated data only .
How it works: exactly what we changed
Every change was applied to an in-memory copy of the LaDe Chongqing delivery data and written to a modification log. The untouched data is the reference. Scoring always uses the real delivery times.
- Outdated data (“stale”): training uses all past weeks at equal weight instead of the last , and the courier and area history features are frozen with data through : predictions used stale stats.
- Sloppy scans (“low quality”): of couriers (). Bulk scans move the delivery time to the courier's last scan that day, for of their packages in week , rising to . From week , of their records are uploaded twice: wrong times and duplicates in all.
- Both at once (“combined”): all of the above together. We also tested this.
Part 2 of 3
What we built: the Guard
Three checks, placed where bad data gets in.
For each package, the prediction system gives the most likely delivery time and the promise: a time we expect to beat 4 times out of 5. It relearns from the last weeks of deliveries every week. The Guard wraps around that loop.
1 Check at the door
Every new record is screened for telltale signs: many packages marked delivered in the same minute, the same package uploaded twice, impossible times or distances.
Suspicious records aren't deleted. They count for only as much, so nothing is lost if the check is wrong.
2 Weigh by trust
Recent records count more than old ones. Couriers whose records keep disagreeing with others working the same area at the same hour count less.
Old stats are kept, because they were still useful. A person gets an alert instead.
3 Final check before promising
The promise is tuned to be on time about of the time.
If a package is unusual, like a brand-new area or a prediction the model is very unsure about, a dispatcher decides instead of the system.
How it works: the technical version
The prediction system is two LightGBM gradient-boosted regressors: a median model and an 80th-percentile (P80) model, predicting delivery duration in minutes. Both are refit from scratch every week on the previous weeks.
- L1 ingest checks: + packages from one courier scanned in the same minute, a repeated package id, a non-positive duration, a GPS fix older than min, or a courier more than km from the stop. Flagged rows get training weight × and are excluded from history features. The checks never read the injection labels.
- L2 trust-weighted training: weight = recency × reputation × flag. Recency halves every weeks. Reputation comes from the courier's flag rate and robust disagreement with peers in the same area × hour, shrunk toward neutral with a -record prior and applied at λ = . Per-courier weight is capped at × the median.
- L3 serve-time check: conformal P80 calibrated per area type on the most recent week. Defer when the area is unseen in the window, GPS is stale, or the P80 − median interval is more than × the area type's median width.
Dispatcher
Gives customers the promise. Handles the packages the Guard flags.
Data team
Reviews suspicious records and couriers every week. Can restore a courier's trust.
Model owner
Approves each weekly update before it goes live, and can roll it back.
Operations manager
Decides where the Guard runs and when to stop it.
Part 3 of 3
Does it work?
Yes. It undoes most of the damage and costs almost nothing when the data is clean.
We locked the Guard's settings first, then tested it once on four weeks it had never seen, including China's Golden Week holiday and a large surge in volume.
– minless error per package when couriers scan sloppily ( → min)
–of the extra error from sloppy scanning removed
–removed when sloppy scans and outdated data hit together
–––of packages handed to a person to check
–extra error when nothing is wrong: essentially zero
Before and after
Average error per package, in minutes: how far the predicted delivery time was from the real one. Same packages, with and without the Guard, in the final test.
Week by week
Extra error compared with clean data, in minutes per package. The yellow area is the error the Guard removes. Shaded weeks are the final test.
Removing the right records is what matters
How it works: proving the cause, link by link
Sloppy scans, design weeks. Solid: without the Guard. Dashed: with it.
Data: bad records, share of what the model learns from What it learns: KS distance, training labels vs. true durations Model: promise shift on a fixed set of test packages, min Decision: extra error vs clean data, min
Controls and dose-response: development weeks, seed 0. Main results: mean over 3 seeds.
How good is the check at the door?
The check at the door is the one place the system makes a yes/no call: is this incoming record suspicious? Here's how often it got that right in the final test, across every run with sloppy scans, compared with the records we know we broke.
How it works: definitions
- Positive class
- A training record the injector changed (a bulk-scanned time or a duplicate re-upload). The label is used only to score the check; the Guard never reads it.
- Prediction
- Flagged by any L1 check.
- Pooled
- Counts summed over the low-quality and combined conditions.
- What it drives
- Training weight × , exclusion from history features, and the courier's flag rate. No package is rejected on a flag.
- Counting unit
- Note
- The delivery-time models themselves are regressors, scored by error in minutes, not as classifiers.
The trade-offs
What it costs to run the Guard:
Security and privacy
- No data leakage. The models only see what's known when a package is picked up: no future deliveries, no test-week outcomes. The labels we used to plant bad data are never model or Guard inputs, and an automated test scrambles them to prove nothing changes.
- No personal data published. The site shows only totals and trained models, never raw records, courier IDs or locations.
- Your data stays with you. The Try it page runs entirely in your browser; an uploaded file is never sent anywhere.