The situation

A published government dataset needed to reach a business system.

The data lives on a public website, released as a ZIP archive containing a CSV.

Doing it by hand: visit the site, download the archive, extract it, find the right file inside, open it, check it looks right, then get it into the target system. Every time the data updates.

Not difficult. Just repetitive, and easy to skip on a busy day — which means working from stale data without realising.

The design decision that matters

Two tools, with a clear split.

UiPath handles getting the file. Download, extract, find, validate, upload.

A workflow tool handles the contents. Parsing, mapping, checking for existing records, creating them.

Why split it at all?

Because the two jobs have different failure modes and different maintenance profiles.

Getting a file from a public website depends on that website. It will change eventually, and when it does, that step needs attention.

Processing the contents depends on your own systems and your own rules. It changes when your business changes.

Keeping them separate means a website redesign affects one small step, not the whole pipeline.

Step 1 — Download

The archive is fetched by HTTP request, not by driving a browser.

This is worth pointing out, because RPA is assumed to mean screen automation. Here it does not.

Where a direct URL exists, requesting it is better in every way — faster, more reliable, easier to debug, and unaffected by page layout changes.

Navigating a site with a browser is a last resort, for when a login is required or the link is generated per session.

The archive is saved to a known location.

Step 2 — Extract and find

The archive is extracted to a working folder.

Then the automation looks for the file it needs — searching the folder and its subfolders.

And here is a detail worth borrowing: if several matching files are found, take the newest by modification time.

Why that rule: archives sometimes contain more than one file, or a previous extraction has left older files behind. “Newest” is a simple, deterministic rule that does the right thing in both cases — rather than “the first one found”, which depends on file system ordering and can silently pick the wrong file.

Step 3 — Validate

Two checks before anything is sent.

Was a file found at all? If not, something went wrong upstream — an empty archive, a failed download, a changed structure.

Does the file actually exist where the automation expects? Extraction can fail in ways that leave the path but not the file.

Failing here is much better than failing later. A missing file caught before upload is a clear, actionable error. A missing file discovered downstream is a confusing problem in another system.

Step 4 — Upload the whole file

The entire CSV is sent in one request to a webhook the workflow tool exposes.

Not row by row. The whole file, once.

Three reasons this is the right choice:

Fewer executions. A file with hundreds of rows sent individually means hundreds of calls. Sent whole, it is one.

Less duplicate risk. A partial send that gets retried can produce duplicates. One file either arrived or it did not.

Clearer separation. The automation’s job is delivering the file. Understanding what is inside it belongs to the next system.

The upload is a standard multipart form post with the file attached — the same mechanism a browser uses to upload a file, done directly.

Step 5 — Check the response

The automation captures the HTTP status and the response.

Why this matters: an upload that returns a success status but a response saying nothing was processed is a failure wearing a success badge. Only reading the response body reveals it.

A successful upload is logged.

Step 6 — Error handling

Every stage is wrapped in try-catch — download, extraction, validation, upload.

When something fails:

The error message and stack trace are logged. It runs unattended, so that log is all anyone will have the next morning.

The error is re-thrown, so the job is marked as failed.

That last point is the important one. An automation that catches an error and exits successfully is the worst possible outcome — the orchestrator records a success, nobody is alerted, and the data silently did not arrive. Somebody discovers it weeks later when a report is short.

A failed job is visible. A quietly-succeeded failure is not.

What the pattern generalises to

This shape comes up whenever you need data from a source you do not control.

The general form:

One tool retrieves the file — using the least fragile method available, and doing nothing else.

It validates that it got something real.

It hands the whole thing to a tool built for processing.

That tool parses, checks against what already exists, and creates records.

The mistake is using one tool for all of it — usually the RPA tool, because it technically can. It works, and then every change anywhere breaks the whole chain.

What we would tell anyone building this

Use an HTTP request if a direct URL exists. Do not drive a browser out of habit.

Pick files deterministically. “Newest by modification time” is a real rule. “The first one found” is not.

Validate before handing off. Cheap to check, expensive to discover later.

Send whole files. Fewer executions, less duplicate risk, cleaner failures.

Re-throw errors. A job that silently succeeds after failing is worse than one that fails loudly.

The short version

Two tools, one job each. One gets the file. The other understands it.

The retrieval step uses an HTTP request rather than screen automation, which makes the fragile part far less fragile than RPA’s reputation suggests.

Validate before handing off, send the whole file at once, and re-throw errors so a failure is visible rather than silent.

That last one is what separates a pipeline you can trust from one that quietly stops working.