Active Storage on several nodes, each with its own disk

Active Storage's Disk service assumes one machine. Run the same Rails app on three nodes, each with its own disk, and a file uploaded through node A returns a 404 when a request for it lands on node B.

The usual fixes are to move to S3 or another object store, mount a shared NFS volume, or pin requests to nodes. Sometimes none of those fit. The files must stay on hardware you control, the network share is the thing you are trying to retire, or the load balancer isn't yours to configure.

We hit that on a client app and wrote activestorage-multi_node_disk. Version 0.1.0 is now on rubygems.org.

The idea

A file stays on the disk of the node that received it. Any node can read, check or delete it: the service looks on its own disk first, then asks the other nodes and keeps the first copy that matches the blob.

Nothing is recorded about where a file lives. There is no new column, no lookup table and no backfill, so the blobs you already have keep working the day you switch.

Setup

Add the gem:

gem "activestorage-multi_node_disk"

Change one line in config/storage.yml and list the nodes:

local:
  service: MultiNodeDisk
  root: <%= Rails.root.join("storage") %>
  nodes: [app1.internal, app2.internal, app3.internal]

Give every node the same token in credentials:

bin/rails credentials:edit   # multi_node_disk: { token: <the same random value on every node> }

And turn on token checks in an initializer (bin/rails g active_storage:multi_node_disk:install writes it for you):

ActiveStorage::MultiNodeDisk.configure do |config|
  config.auth = :token
end

That's the whole setup. has_one_attached, blob.download, blob.open, purge and purge_later work as they do now.

How it works

MultiNodeDiskService is a subclass of Active Storage's DiskService, and Active Storage loads it from service: MultiNodeDisk the same way it loads Disk or S3. It overrides only what has to change:

• Upload writes to this node's disk, exactly as Disk does.
• Read (download, download_chunk, open) checks this disk first. On a miss, it asks every other node at once, at most 8 at a time, under one deadline. The first copy whose size and checksum match the blob wins, and a copy that doesn't match is skipped and reported.
• exist? checks this disk, then whether any node has the file.
• Delete removes the local copy, then asks every other node to delete theirs. A 404 counts as done.

Everything else stays local, as with Disk: url (use Active Storage's proxy mode, which reads through download), compose, and delete_prefixed for variants, which any node can regenerate.

Each node answers the others at one endpoint:

GET | HEAD | DELETE  /rails/active_storage/multi_node_disk/blobs/:key

It lives in an isolated engine under Active Storage's routes prefix and is mounted ahead of your routes, so a catch-all can't hide it. The endpoint serves only from its own disk. A request that arrives from another node never fans out again, so there are no loops and no request storms.

What it costs

A local hit costs one File.exist?, plus one indexed blob lookup for a whole-file download. A miss on N nodes sends N − 1 requests. Only the node that has the file sends any bytes; the others answer 404 after a File.exist?. For the three to five nodes a typical stage runs, that is cheap enough that we dropped our first plan to record each file's node: saving a few 404s didn't justify an extra write on every upload and a backfill.

Errors Rails already understands

The service raises subclasses of Active Storage's own errors, so your existing rescues keep working:

Error	Superclass	When
NotHere	ActiveStorage::FileNotFoundError	every node said "not here"
Unavailable	ActiveStorage::Error	a node that may have it didn't answer (retry)
Mismatch	ActiveStorage::IntegrityError	the only copies found don't match the blob

Deletes: inline or in a job

The local copy is always deleted at once. For the other nodes, pick a mode:

• :sync (default) deletes inline and raises Unavailable naming any node that didn't confirm.
• :job hands the remote deletes to ActiveStorage::MultiNodeDisk::DeleteJob, which retries until every node confirms and is safe to run twice.

config.delete_mode = :job
config.delete_job_queue = :maintenance

Who is calling

The gem checks who is calling its endpoint. It never decides who may see a file; that belongs to your app, on your app's own pages.

auth	The endpoint needs
:none (default)	nothing
:token	Authorization: Bearer <token>, else 401
:ip	the caller in allowed_ips, else 403
[:ip, :token]	both

:none is the default so a first install works on a closed network. The README warns about it and the gem logs a warning at boot in production. Use :token anywhere else.

Under :ip, the allowlist defaults to the nodes' own addresses, resolved from DNS and refreshed every 60 seconds. The caller is the socket's peer, and X-Forwarded-For counts only when that peer is one of your trusted_proxies, because a forwarded header is something any caller can write.

An audit trail

When files are payroll exports or member lists, someone eventually asks who pulled which file, and from where. Every answered request and every remote fetch or delete becomes one versioned JSON document:

{"v":1,"at":"2026-10-17T12:00:00.000Z","action":"read","result":"served","status":200,
 "key":"abc","filename":"members.csv","byte_size":22,"node":"app2","from_node":"app1",
 "remote_ip":"10.0.0.1","auth":["token"],"hop":true,"duration_ms":3.1,
 "requester":{"id":42,"label":"hr@example.com"},"requester_verified":true}

It goes to a JSON Lines file by default (tmp/multi_node_disk/audit.log), or to a table, a callable, or nowhere. It never contains a token, file contents or a signed URL. The same document is also sent as an audit.multi_node_disk notification.

The requester is up to you:

config.audit_requester = ->(request) { {id: Current.user&.id, label: Current.user&.email} }

and for jobs:

ActiveStorage::MultiNodeDisk.with_requester(id: user.id, label: user.email) do
  blob.open { |file| import(file) }
end

The requester travels with the request to the node that serves the file, so the audit line on node B says which user on node A asked. Node B marks it verified only when token or IP auth passed.

If you want the audit table, bin/rails g active_storage:multi_node_disk:install --audit writes the migration. It has one JSON payload column (jsonb on PostgreSQL, json on MySQL 8, a CLOB on Oracle), an index on created_at only, and names that fit Oracle's 30-character limit. Nothing runs until you migrate.

Testing your app

The gem ships a test helper that runs fake nodes in process, each answered by the gem's real endpoint:

RSpec.configure { |config| config.include ActiveStorage::MultiNodeDisk::TestHelper }

it "reads a file another node holds" do
  node_b = multi_node_disk_node("app2")
  multi_node_disk_move(blob, to: node_b)

  multi_node_disk_cluster("app2" => node_b, "app3" => :missing) do |requests|
    expect(blob.download).to eq(bytes)
  end
end

A node can also be :wrong, :timeout or :down, so you can spec what your app does when a node is out.

Rules we kept

• Extend Active Storage, don't wrap it. One service class and one changed line in storage.yml, with no adapter layer in between.
• No runtime dependencies beyond Rails. Net::HTTP, system DNS, Rails credentials, Socket.gethostname. If you want HTTPX, IPv4 pinning or your own token scheme, there is a hook for it in the initializer.
• No database unless you ask for one. Only the opt-in audit table touches it.
• Nothing at /. No root route, no catch-all, no changes to default_url_options.

The README's Advanced section lists every hook, including the full config of the app the gem was written for.

Try it

gem "activestorage-multi_node_disk"

It needs Ruby 3.3+ and Rails 8.1. The source, issues and changelog are on GitHub. It's MIT-licensed, and bug reports and pull requests are welcome.