Importing files

#Importing files

Importing data moves rows. Files are the other half, and they move differently: rows go through the import wizard, but the file bytes themselves are copied storage to storage by a tool built for it.

This page covers which tool to use, and — the part that is easy to get wrong — how to keep imported records pointing at the right file afterwards.

of gigabytes through the API would be slower (every byte crossing the network twice), would hit request limits, and would mean handing over storage credentials that we would rather never hold.

---

#1. Pick the route for your source

Where your files are now	Use	Why
S3, R2, or any S3-compatible bucket	Cloudflare Super Slurper	Built for exactly this. Runs on Cloudflare's side — no local bandwidth, no long-running job on your machine.
S3-compatible, but Super Slurper won't take it	rclone	Handles unusual endpoints, resumes, filters, and partial copies.
Files on the old server's local disk	rclone from that machine	The files are only reachable there. Copy from the disk straight to R2.
A handful of files	Upload them normally	Below a few hundred, the wizard's own upload is less setup than a tool.

#The bucket-to-bucket copy

Both tools do the same job: copy every object into a prefix in your EmuView R2 bucket. Keep the copy flat — same filenames, one prefix — which is what makes step 2 possible.

collections** on the source page, open directus_files, and pick where the objects should land. The command comes back filled in with the real bucket and prefix, and the same screen can check afterwards that the objects arrived. Everything below is what it is doing, for anyone who would rather do it by hand.

You are choosing between three destinations:

Destination	When
A new container	The usual answer. Named after the source, so a re-sync later lands in the same place rather than duplicating every object.
A folder in a container you have	You already have somewhere files go — a media library — and want these inside it.
An existing container, as it is	Consolidating into one place. The cost is that "remove everything this import brought" stops being a single prefix.

# rclone: the source bucket as it stands, into a prefix of your own
rclone copy s3-source:my-old-bucket \
  r2:sveltesync-files/imported/cms-example-com/ \
  --transfers 16 --checksum --progress

copy, not sync — sync deletes anything at the destination the source does not have, which matters the moment you point it at a container that already holds something. --checksum rather than --size-only, so a re-run replaces an object that a previous copy truncated.

With Super Slurper, set the destination prefix to the same imported/cms-example-com/ and let it run.

not arrived yet is indistinguishable from one pointing at a file that was lost — and you will not find out until somebody clicks it.

---

#2. Keep the filenames the source used

This is the whole trick, and it costs nothing if you do it up front.

Most systems — Directus included — store a file under an opaque name on disk (a3f1c0de-….jpg) while showing you a friendly one. Your records reference the opaque name. So:

Copy the objects under the names they already have. Do not let the tool rename or restructure them. A flat copy preserves the link between a record and its file for free, because the value already in the record still matches the object now sitting in R2.

Rewriting the names during the copy means every record has to be rewritten to match, in lockstep, from a mapping table — which is a bespoke script, and one that can only run after the import has finished. It is a lot of work to buy a tidier bucket listing.

---

#3. Bring the file records across

Your old system has a table describing the files — Directus calls it directus_files. You have two options, and they answer different needs.

#Option A — import it as an ordinary collection

Treat the files table as data, like any other. Run it through the import wizard: create a collection (legacy_files, say), map the columns you care about — filename, title, alt text, caption, uploaded date — and import.

the list until you turn on Show system collections at the top of the source page. Three of them appear when you do — files, users and folders. Directus's own internal tables (sessions, activity, revisions) are never listed, because there is nothing here for them to become.

• Good for: keeping editorial metadata, and any records that reference files by the old system's id. Relations resolve exactly as they do for any other imported collection.
• Trade-off: these are rows describing files, not files as EmuView knows them. They will not appear in file containers, will not count towards storage reporting, and downloads are not served through the files API.
• Keeping the link: map the source's storage filename into a text column. That value plus your prefix is the object's location in R2.

This is the lighter option, and it is usually the right one when the files are mostly attachments hanging off records you are also importing.

#Option B — import them as EmuView files

Land them as real platform files instead, so they appear in file containers, count towards storage, and are served by the files API with the same access rules as anything uploaded here.

• Good for: a media library you will keep using and adding to.
• Trade-off: more setup. Each file needs a record whose stored key points at the object you copied in step 1, which is why the flat copy matters.
• How: POST /api/v1/migrations/:id/files/import, once you have chosen a container and verified the copy. It reads one page of the source's file list and writes a file record for each object, without moving a byte — the keys it records are composed exactly the way the copy command and the verification step compose them.

#Running the import

It works a page at a time, and the response tells you where you are:

{ "page": 1, "created": 200, "repaired": 0, "unchanged": 0, "rejected": [], "nextPage": 2 }

Keep calling it with the page from nextPage until nextPage is null.

It is safe to re-run. A second pass over the same page repairs what is already there rather than duplicating it, so if a request times out or the window closes, ask for the same page again. unchanged counting up instead of created is what a completed page looks like the second time — not an error.

A row the source lists with no stored object is skipped and reported as skippedWithoutObject, because a file record pointing at nothing is worse than a missing one: everything downstream treats it as a real file.

the import needs a principal with a person behind it. A plain API key has none, and the import refuses it up front with upload_needs_person rather than failing part way through the manifest. Run it from the dashboard, or with a delegated key — one bound to a user, which presents that person's identity.

Files land against your project's storage quota as they are imported. If the quota is reached the import stops at that point in the manifest and names what it refused; raise the quota and run the same page again, and nothing that already landed is written twice.

single byte — the objects are already in R2 under stable names. That is the other reason step 2 is worth getting right.

---

#4. Check before you rely on it

Whichever option you chose, confirm a sample rather than assuming:

1. Pick five records that reference files, ideally including the oldest and the largest.
2. Confirm the object exists in R2 at the prefix you copied into.
3. Open one through the app, not just the bucket listing — that exercises the path a visitor takes.

A missing object at this stage means the copy did not finish, or the prefix does not match what the records hold. Both are cheap to fix now and expensive to discover in six months.

---

#Passwords, users, and the rest

The file bytes and the file table are the parts that need a tool. Everything else about a migration — users, permissions, flows — is covered in Importing data.

One thing worth repeating here because it surprises people: an API token cannot read password hashes out of Directus, so imported users arrive without passwords and need a reset link. The import wizard says so when you connect a source, and it is not something a file copy changes.