A tax audit asked a small towing company for three years of records, every motor-club call with its location and payment, so the sales tax could be worked out exactly. The records existed. One of the owners showed me where, in a national billing portal. Listing the calls took five to fifteen seconds, and opening one took another five to fifteen. Then came thirty to sixty seconds of reading and copying before the page refreshed and it started over with the next call. There were about 28,000 calls, and the deadline was a couple of months away.
The portal exports one call at a time, and only after you've opened its detail view, so there was no bulk export to ask for. Even with one, the numbers would have needed fixing. The largest motor club billed at a flat tax rate, and the audit wanted the rate for wherever each call actually happened.
So the records came out the way a person would have collected them, one call at a time. A Python script went month by month, opened every record and saved it to disk. It paused half a second between records and kept a progress file, so an interruption only lost the time since the last checkpoint. The first hour turned up a rate limit and some records with mismatched fields. I adjusted the script and let it run, and the pull was done before sunrise.
Three years of records, two ways
The raw records said what happened, but the audit wanted where and at what rate. First the pipeline fetched every call on the portal club's list by ID into a sheet. Then it merged and normalized that with the other two clubs' exports. Only then did the tax work start, on one master sheet. The addresses were reduced to a unique set and each got a zip code. The zip codes were reduced the same way and each got a rate. That came to a few hundred lookups instead of tens of thousands.
The pipeline
Started Friday night, done by Monday. The whole date range came out enriched in one CSV in the company's Google Drive, which anyone with Excel can open, and everything was submitted on time. The script was a throwaway tool for a one-off job. The records are plain files now, and whoever the company hires for the next audit can read them.