Magento Open Source Amazon S3 remote storage

The remote storage module moves a defined set of file categories from the web server's local disk to an Amazon S3 bucket. Everything outside those categories stays on local disk. Before you plan the bucket layout, you need a clear list of what moves and what does not. The two groups Adobe describes are media files and generated files. Media files are the images and documents that enter the store through the admin, such as product and category images. Generated files are the derived copies Magento creates from those sources, such as resized image variants. Adobe defines the exact categories for each release line, so confirm that list in Adobe's documentation for your version before you assign prefixes or set lifecycle rules in the bucket.

The part that stays on local disk matters as much as the part that moves. Application code, compiled output, configuration and any directory the module does not list remain on each node's filesystem. In a multi-node web tier, that means every node still needs its own copy of the code and must agree on the same configuration. Build a mapping table with one row for each directory your store writes to. For each row, record the owning component, whether Adobe lists it as a moved category for your release, and who approved the decision. Include directories written by custom modules and third-party extensions, because those writes are easy to miss in a standard installation review.

After the switch, the storefront and admin still request media by URL, so the visible behavior of the catalog should not change for shoppers. What changes is where the bytes come from and how writes get there. Confirm with Adobe how the media base URL is set for your module version, and whether that URL points directly at the bucket or at a content delivery network placed in front of it. Adobe also notes PHP file function caveats. Any custom code that reads or writes media with local filesystem functions, or that assumes a file exists at a local path, must be reviewed and tested against the remote storage behavior before go-live. A file written to local disk on one node will not be visible to the other nodes, so treat any local write path as a defect to fix.

Plan the review as a test exercise, not a configuration step. Run admin uploads, product image regeneration, category image changes, a cache flush, and any scheduled jobs that create files, and confirm each result on every web node and in the browser. Include custom code paths in the test list, along with the extensions that generate invoices, exports or catalog feeds. Ask your implementation partner or your own developers to produce a written list of every custom path that touches media or generated files, with a pass or fail result for each. Keep that list with the rollback plan, because the decision about which categories move will shape your backup scope, your bucket permissions and your restore steps later.

Remote storage matters on a multi-node web tier because every web node must read and write the same media location. Once two or more nodes serve one store, an image uploaded through the admin on one node has to resolve on every other node, and any file generated by one node, such as a resized product image or a cached asset, has to be visible to the rest. With local media directories, each node holds its own copy, and the store shows broken images or stale files depending on which node answers a request. Moving media and generated files to Amazon S3 with the remote storage module gives all nodes one shared target, but only if every node is configured identically.

The main failure mode is partial configuration. A node that was built before the remote storage settings were applied, or a node restored from an older image, may still write to local disk while its neighbors write to the bucket. Cache and deployment steps make this worse. A static content deploy or a configuration cache refresh run on one node does not automatically correct another node's environment settings. Before you roll out changes, decide the order in which nodes receive the new code and environment values, and keep one node out of rotation until it has passed the checks below. Confirm the sequence in your own deployment tooling, because the exact commands depend on how your nodes are built and whether they share an env.php source.

To confirm that all nodes share one configuration, compare the effective settings on each node rather than trusting the change log. Check the remote storage driver and bucket values through the CLI on every node, and compare the resulting configuration output across nodes, including the storage driver and the path prefix. Confirm that the database media storage setting has been disabled before the cutover, since a node that still reads media from the database will behave differently from one that reads from S3. Record the comparison as part of the release evidence so that the next person can see what was verified and when.

After a node is added, replaced or scaled, repeat the comparison before it receives traffic, then run a short functional check. Upload a test image in the admin, load a storefront page that uses it from the new node, and confirm that the file resolves through the expected URL. Check that generated images are written to the bucket rather than to the local media directory, and that the new node does not hold a divergent copy. Autoscaled or rebuilt nodes are the most common source of drift, so treat the node image or provisioning template as part of the configuration baseline. Ask your partner or internal team to confirm how the bucket credentials reach new instances, whether through instance roles or stored secrets, because a missing credential on one node can appear as intermittent media failures rather than a clear error.

Plan for the cost and operational side as well. Running several nodes against one bucket adds request volume, and the IAM policy must allow every node to read, write and delete only the paths it needs. Decide who owns the bucket policy, who rotates credentials, and who approves a change to the storage settings, since a single misconfigured value affects every node at once. Ask any implementation partner to state in the written scope which node rollout steps, verification checks and rollback actions are included, and which remain with your platform team.

The first design decision is which cache sits in front of the media files. Adobe documents Varnish as the full-page cache for self-managed installations, and it documents Fastly for cloud-hosted Adobe Commerce. A CDN in front of Amazon S3 is a separate layer from both, so you need to decide how it fits your own edge setup before you change where media is stored. Confirm with your edge team whether your CDN can use the S3 bucket or a web-tier origin, which hostname shoppers will request, and how Varnish will treat responses for product images. Check the Varnish version you plan to run against the system requirements for your Magento Open Source release line.

The CDN and the media base URL settings must agree. If shoppers request images from the bucket, a CDN hostname and the store domain at different times, you can get mixed content warnings, broken images in cached pages, or old images after a catalog update. Plan a test in staging that replaces one product image, confirms the new file is served through the CDN, and records each purge or invalidation step in the runbook. Ask your CDN provider how it handles cache expiry for objects that keep the same file name, since regenerated image cache files often do.

Adobe also notes caveats for PHP file functions in general terms. Some functions behave differently or are restricted in a managed application environment, and the code that calls them assumes a local filesystem. With remote storage, media reads and writes can pass through a storage adapter instead of the local disk. Code that calls file functions directly on paths under the media directory may therefore find no file, write to the wrong place, or delete only a local copy. Review the exact function list and any module-specific limits in the official documentation before anyone writes custom code that touches media, because the list can change between releases.

Start with an inventory of custom modules, theme code, import scripts, cron jobs and third-party extensions that read, write, delete or build paths under the media directory. Classify each touchpoint and ask the team that wrote it, or the extension vendor, whether it uses the framework's storage abstraction. The number of touchpoints is one of the main drivers of effort and of any quote for this work, so ask for each item to be estimated separately. Test every item in staging with the same storage configuration as production, including the CDN path, before the change goes live.