Ceph RGW S3 API compatibility: what the gateway covers and why it matters before migration
An application team or platform owner moving S3 workloads onto Ceph RGW needs to know which calls will work, which will behave differently and which will fail before any data moves. The Ceph Object Gateway (RGW) offers S3-compatible and Swift-compatible APIs, and its documentation covers AWS STS, IAM and SNS APIs. Compatibility, however, is decided feature by feature, not by protocol name, so a workload that uses only basic object operations may fit well while one that depends on a specific bucket policy condition, lifecycle transition or event filter may not. Treat this page as a fit check that tells you what to verify, not as a guarantee that any given application will run unchanged.
The cost model is also different from a commercial service. Ceph's source is licensed under LGPL-2.1 or LGPL-3, and the project does not offer a paid edition, so the license fee is zero and the spend is the implementation work: the cluster itself, gateway deployment, identity and key management, testing of your SDKs and the application changes that compatibility gaps force. Ask any partner for a quote that separates the compatibility assessment, the gateway build and the ongoing operation, and that states which of your applications are in scope for testing.
Start by inventorying the S3 calls each application makes, including the SDK and version, the authentication method, any presigned URL use, multipart uploads, versioning, object tagging, lifecycle and notification configuration. Then compare that list against the current Ceph release notes and the RGW documentation for the release you plan to run, and confirm each gap with a test against a lab gateway rather than relying on a feature list. Record the results per application, because a missing feature in one SDK path may not matter to another team.
Which S3 feature areas need the closest look
Core bucket and object calls are the first thing most applications touch, but the gaps usually sit in the less common operations around them. Start by listing every S3 call your application makes, then check each one against the RGW S3 documentation for the release you run. Treat the list as your test plan, not as a yes or no answer.
- Bucket listing and pagination: Confirm the list-objects variants your SDK uses, including continuation tokens and delimiters, and test with buckets holding many keys.
- Object tagging and metadata: Check tag limits, user metadata handling and whether tags survive copy operations as your code expects.
- Copy and conditional requests: Verify server-side copy, byte-range reads and conditional headers such as
If-Match. - Bucket configuration calls: Confirm CORS, website and logging settings individually, since a client that calls an unsupported one may fail at setup rather than at first use.
A partial implementation usually shows up as a 501 or a malformed response on a secondary call, after the main upload path already works. Confirm each operation in your release notes and document the result.
Versioning and object lock change how deletes, overwrites and retention behave, so client code that assumes simple overwrites can silently keep old data or fail on retention checks. Verify both features against the documentation for your release, including how the retention and legal hold headers are accepted and enforced.
- Version listing and delete markers: Confirm that listing versions, restoring prior versions and deleting specific version IDs behave as your backup or sync tool expects.
- Object lock modes: Check which retention modes and legal hold settings are documented, and whether they can be set at bucket creation only.
- Bucket-level defaults: Confirm whether default retention applies to new objects and how it interacts with existing ones.
- Lock and replication interplay: Verify behavior when locked objects are replicated to another zone.
A partial implementation often means the write succeeds but the retention header is ignored or stored differently, which is a compliance problem rather than an error. Test deletion and overwrite paths explicitly, and record the results for your audit file.
Lifecycle rules run in the background and can delete or move data long after an application is deployed, so mismatches are easy to miss. Check the lifecycle documentation for your release, then compare each rule type with what your application or retention policy requires.
- Expiration rules: Confirm expiration by age, by prefix and by tag, and whether expiration applies to noncurrent versions.
- Storage class transitions: Check which transitions are supported and whether they are one-way.
- Remote tier behavior: Verify what happens to metadata, restore requests and access times when objects move to a remote S3 tier.
- Rule evaluation timing: Confirm how often rules run and how to observe their progress.
A partial implementation often means a rule is accepted at upload time but evaluated differently from what the policy text states. Clients that read an object after transition may receive an error code they do not handle. Set a test bucket with short-lived test objects and confirm the outcome before you rely on the rule.
Bucket policies, ACLs, presigned URLs and server-side encryption decide who can read data and how it is protected at rest. Each has its own documented limits, and they interact, so verify them together rather than one at a time.
- Bucket policies and ACLs: Confirm which policy condition keys and principals are evaluated, and whether ACL grants are honored alongside policies.
- Presigned URLs: Check signature version support, expiry limits and whether URLs work through your load balancer or proxy.
- Multipart uploads: Verify part size limits, the abort-incomplete-upload behavior and whether checksums are computed as the SDK expects.
- Server-side encryption: Confirm which of SSE-C, SSE-KMS and SSE-S3 are documented, the key backend options and whether HTTPS is required for encrypted requests.
A partial implementation often means a policy condition is ignored, which fails open for access control. Presigned URL failures usually appear only after deployment behind a proxy. Test each access path with a user that has the minimum permissions and record the result.
Bucket notifications, identity APIs and temporary credentials connect RGW to the rest of your platform. These areas depend on the release and on the endpoint types you configure, so confirm each one against the Object Gateway documentation for your version and against your identity design.
- Notification targets: Confirm which event types and endpoint types (such as Kafka, AMQP or HTTP) are documented, and how failed deliveries are retried or dropped.
- IAM APIs: Check which IAM actions exist for users, groups, roles and policies, and whether your tooling calls any that are missing.
- STS operations: Verify which assume-role and token calls are available, their session duration limits and how role trust policies are evaluated.
- Identity federation: Confirm which external authentication methods your design needs, such as LDAP or OpenID-based sign-in, and whether they map to RGW identities as planned.
A partial implementation often means a notification is accepted but not delivered, or an IAM call returns an error your tooling does not expect. Confirm these areas with a test account and log the responses before you move production clients.
Compatibility checks and the test method each one fits
This table pairs each compatibility check area with what to verify, how to test it, and the situation where that test earns its time. Read it as a sequence: inventory first, then SDK and authentication behavior, then IAM, replication and multisite limits. The main takeaway is that a feature list alone does not prove fit, because your applications depend on specific request patterns, policies and failure modes.
| Check area | What to verify | How to test it | When it fits |
|---|---|---|---|
| Feature inventory | Which S3 operations, bucket settings, object features and storage classes your applications call, compared against the RGW release you plan to run | Extract the API calls from application code, gateway logs and SDK traces, then run each against a staging RGW with the same release and configuration | Start here for every migration, before any SDK or policy testing |
| SDK behavior | Retry handling, multipart upload thresholds, checksum headers, virtual-host versus path-style addressing and error codes your code parses | Run the exact SDK versions in production against staging buckets, including failure injection such as dropped connections and throttled responses | When applications use SDK features beyond basic get, put and list calls |
| IAM and STS policy evaluation | Account, user, group and role policies, condition keys, and whether assumed-role sessions behave as your access model expects | Write a test matrix of allow and deny cases per principal and resource, run each through the SDK, and confirm the denials as well as the grants | When teams need separate IAM spaces or temporary credentials for workloads |
| Replication and multisite limits | Which replication features your zones need, how object and metadata sync behaves during outages, and which settings must match across zones | Build a two-zone lab, write and delete objects in both zones, interrupt the link, and measure convergence and conflicts against your recovery targets | When data must be copied to a second site or used for disaster recovery |
| Keystone authentication | Whether your identity model maps cleanly to Keystone projects and roles, and how token expiry and revocation reach RGW | Authenticate test users from each role, confirm the mapped access, then revoke a token and confirm the request fails | When an OpenStack environment already issues identities for storage access |
| LDAP authentication | Which directory attributes map to RGW users, how group membership is resolved, and what happens when the directory is unreachable | Test a bind with valid, expired and disabled accounts, and simulate a directory outage to observe the error returned to clients | When existing directory accounts must reach object storage without duplicate credentials |
| Open Policy Agent integration | Which decisions are delegated to OPA, the policy input RGW sends, and the latency and fail-open or fail-closed behavior | Load a test policy bundle, send allow and deny requests, and stop the OPA service to confirm the configured failure behavior | When authorization rules must be centralized and audited outside RGW |
| Multi-factor authentication | Whether your applications can supply the one-time code on each request, or whether they need a separate path for automation | Attempt signed requests with and without the second factor using the SDKs in use, and record which automated jobs break | When human or privileged access to buckets must require a second factor |
Running an S3 compatibility test before cutover
This sequence turns a list of applications into a recorded pass or fail result for each S3 operation they depend on, so the cutover decision rests on evidence and accepted gaps are written down before migration starts.
Build the S3 call inventory
Your engineers and DeployOpen's reviewers scan application code, SDK configuration, and existing gateway or proxy logs to list every S3 operation each application calls, including multipart uploads, presigned URLs, versioning, tagging, lifecycle, notifications, and bucket policies. Group the results by application and by SDK language and version. The step is done when each application has a named owner and a complete list of operations, and anything found only in logs is flagged for confirmation with its owner.
Stand up a test RGW zone
Deploy a non-production Ceph Object Gateway zone on the Ceph release you plan to run, using cephadm on hardware or virtual machines that resemble production. Confirm the release choice against the Ceph releases page, since support windows differ between active releases. Load a seed dataset that includes bucket layouts and object sizes close to production. The check is a working endpoint with a test user, a healthy cluster status, and a documented build record that names the exact release and configuration.
Run each SDK version against the zone
Execute the inventory's operations with the exact SDK versions in production, using scripted test cases rather than manual clicks. Cover small objects, objects above the multipart threshold, large multipart uploads, range reads, and failed-request retries, along with the virtual-host and path-style addressing your clients use. Keep the test harness in version control so it can be rerun after every change. The check is a result file for each SDK version showing the operations executed and their outcomes.
Record pass or fail per operation
Log every failing call with the operation name, request parameters, HTTP status, error code, and the SDK and release versions involved. Separate hard failures from behavior differences, such as a response header that differs or a listing order that changes, because they carry different risk. Confirm each failure against the Ceph Object Gateway documentation for the release under test. The step closes when every operation in the inventory has a recorded pass, fail, or not-tested status, with no blank rows left in the matrix.
Review policy and STS behavior with identity owners
Walk through the bucket policies, IAM users, roles, and STS calls that applications rely on with the identity team. Verify that the intended authorization outcomes match what RGW enforces, including denials, temporary credential expiry, and role assumption from each identity source you plan to use. Record any policy construct that RGW does not evaluate the way your existing platform does. The check is a signed-off list of identity scenarios with expected and observed results, reviewed by the people who own the policies.
Fix blocking gaps and retest
For each hard failure, decide with the application owner whether to change the application, change the SDK version, adjust the design, or accept the gap. Make the agreed changes in the test environment and rerun the affected test cases, plus a regression run of the full suite once changes are in. The step is done when no blocking failure remains open without a recorded decision.
Document accepted gaps and cutover criteria
Write a compatibility report that lists supported operations, accepted gaps with their owners and workarounds, and the test evidence for each. Define the cutover criteria in writing, such as which failures block migration and who signs off on accepted differences. Ask DeployOpen or your own team to confirm that the report matches the final test run. The output is a signed compatibility report that becomes the reference for the migration runbook and for any later audit questions about data access.
Security checks that belong in the compatibility test
Feature lists do not show whether an application stays safe after cutover. The checks below test the behaviors that security reviews and application owners depend on: who can reach what, how clients verify the endpoint, which identities are accepted, what encryption they assume, and which software versions are running. Run them against your own RGW endpoint with the SDKs and identity setup your applications use.
Policy enforcement matches expectations
Bucket policies, IAM policies and ACLs can grant overlapping access, and an application that works under one model may fail or over-reach under another. Build a test matrix with an allowed and a denied request for each role your applications use, including cross-account reads, prefix-scoped writes and deletes. Confirm that a denied request returns the same error class your code expects. Verify the results against the policy evaluation documented for your release, and record any case where the outcome differs from what your access review assumed.
HTTPS and certificate validation
SDKs differ in how they validate certificates, trust custom certificate authorities, and handle hostnames for virtual-host style buckets. Test each SDK against the exact hostname and certificate chain your RGW endpoint presents, including any load balancer in front of it. Confirm that clients reject an untrusted or expired certificate instead of falling back to plain HTTP or skipping verification. Check that the CA bundle is deployed to every client image, since a missing bundle often gets fixed by disabling verification.
Authentication backend behavior
RGW can authenticate users through Keystone, LDAP and multi-factor authentication, and each backend changes how tokens, sessions and failures reach the client. Test a valid login, an expired credential, a revoked user and a wrong MFA code for each backend your applications rely on. Confirm which errors the SDK surfaces and whether retries cause lockouts or repeated prompts. Ask your identity team to confirm that disabling a user in the directory stops S3 access within the window your policy requires.
Encryption behavior clients rely on
Applications often assume that server-side encryption is on, that a specific algorithm is used, or that encrypted objects can be read after a key rotation. Test uploads that request encryption headers, reads of objects written before and after any key change, and requests that must be rejected when the required headers are missing. Confirm with the vendor documentation which encryption modes your release supports and whether a client can detect the mode used, so audit evidence matches what the application sends.
Current RGW and SDK versions
Compatibility results only hold for the versions you tested. Record the RGW release and the SDK versions in the test report, and plan retests when either side moves. Ceph's supported release list shows which series are maintained with backports and security fixes, so confirm your cluster runs one of them before you sign off. Schedule a retest after each minor update, and keep the SDK pinned in your build so an unreviewed client release cannot change behavior in production.
Questions teams ask before testing S3 compatibility on Ceph RGW
How long does an S3 compatibility fit check take on Ceph RGW?
No fixed duration applies to every application, because the time is set by how much your applications use. The main drivers are the number of distinct S3 calls each application makes, the number of SDK languages and versions in use, and how complex your IAM model is. An application that uses a handful of object operations against a single bucket is far quicker to check than one that relies on versioning, lifecycle rules, bucket notifications, presigned URLs and per-team policies. Ask any partner to show how they count calls and SDKs before they commit to a schedule, and confirm the final timeline against the written scope.
What does a Ceph RGW compatibility check cost?
Ceph is licensed under LGPL-2.1 or LGPL-3, so there is no license fee to pay for the software itself. The cost of a fit check is mainly the labor to inventory applications, build test cases, run them against a test environment and document the results. If you hire a partner, the quote should break out that labor by application, SDK and IAM area, and state which items are one-time and which need repeating after each Ceph upgrade. Your own infrastructure, storage and staff time are separate costs that the quote should name. Ask what a retest after an upgrade would include.
What does our team need to provide for a compatibility test?
A useful test needs three inputs from your side. First, an application inventory listing each system that talks to object storage, the SDK and version it uses, the S3 operations it performs and any IAM policies or presigned URL patterns it depends on. Second, a test environment with a Ceph RGW endpoint that matches the production setup as closely as possible, including TLS, the endpoint hostname format and the authentication method. Third, a named owner for each application who can confirm expected behavior, approve test data and sign off on results. Without these, results describe the test setup rather than your applications.
Can our internal team run a Ceph RGW S3 compatibility test alone?
Yes, if your team has engineers who can read the Ceph Object Gateway documentation, write repeatable SDK tests and run a Ceph test cluster. The work is manageable when the application count is small and the feature set is limited to basic object operations. It becomes harder when multiple SDK languages, multipart uploads, versioning, lifecycle, notifications or IAM policies are in scope, because each area needs its own test cases and careful reading of which features the gateway supports. Many teams split the work: internal staff own the inventory and application owners, while an outside partner builds the test suite. Decide this based on the gaps in your own team's skills.
Does a passing S3 compatibility test certify Ceph RGW for our use?
No. A compatibility test shows how your specific applications behaved against your specific Ceph version and configuration on the day you ran it. It is evidence for your own decision, not a certification, and it does not replace the vendor's documentation or an independent assessment. DeployOpen does not issue audit reports, attestations or certifications. If your organization needs a formal assessment for a regulatory or contractual requirement, ask the assessor which evidence they will accept, and confirm with them whether your test records meet it. Keep the test scripts, versions and results so the work can be repeated after upgrades.
Which parts of the S3 API should we test first on Ceph RGW?
Start with the operations your applications depend on most, then test the features that are most likely to be partial or missing in your environment. Engineers commonly check basic bucket and object operations, multipart uploads, presigned URLs, object tagging, versioning, lifecycle rules, bucket policies and any IAM or STS calls. Confirm each item against the current Ceph Object Gateway documentation rather than assuming support from the S3 name alone. Record the exact Ceph release you test against, because feature coverage changes between releases.
How do we check which Ceph release our S3 tests should target?
Test against the Ceph release you plan to run in production, not the one you have in a lab. The Ceph releases page lists the actively maintained releases and their estimated end-of-life dates, so check that your chosen release will still receive backports and security fixes for the period you need. Confirm the release, patch level and RGW configuration with your platform team before writing test cases, and repeat the test when you move to a newer release.
Limits to confirm for your Ceph release and environment
Ceph RGW feature support depends on the release you run, how you deploy it and how your applications call it. The cards below list the areas where compatibility claims most often go wrong, so you can confirm each one against the exact release and configuration you plan to operate.
Feature status by release
The RGW feature set changes between Ceph releases, and some S3 operations are only available or behave differently on specific versions. Start from the Object Gateway documentation for your release, then compare the feature list with the release notes for the version you will deploy. Record the exact version in your compatibility matrix, because a feature that works on one release may be absent or partial on another. Ask your vendor or release channel to confirm any operation your applications depend on.
Multisite replication constraints
Multisite replication covers configuration, metadata and object data, but its limits differ by feature, and some settings must be consistent across zones before replication works as expected. Confirm which bucket features replicate in your release, how versioning and lifecycle actions behave across zones, and how sync is monitored. Test a failed sync and a conflicting write in a lab before you rely on the design for recovery objectives.
IAM and STS policy coverage
RGW supports AWS STS, IAM and SNS APIs, but policy language coverage is not the same as full AWS IAM. Write down the policy patterns your applications and teams use, such as conditions on prefixes, cross-account access, role assumption and session tags, then test each one against your release. Confirm how the IAM accounts model maps to your tenancy design before you migrate policies from another provider.
Notifications and SNS behavior
Bucket notifications depend on topics, delivery endpoints and how RGW handles endpoint failures. Confirm which event types and filters your release supports, how persistent delivery retries, and what happens when an endpoint stays unavailable for longer than the retry window. Check whether your pipelines assume exactly-once delivery, since notification delivery should be treated as at-least-once unless your release documentation says otherwise.
Large bucket performance
Compatibility is not only about API calls succeeding. Buckets with very large object counts depend on index sharding, and listing, versioning and lifecycle scans can slow down as the index grows. Benchmark your own object sizes, key naming and list patterns on the planned hardware, and confirm the resharding behavior of your release. Do not assume a feature that works on a small test bucket will perform the same at production scale.
Version-dependent and partial features
Some S3 features are partial, bundled with other components or only available in specific releases, including server-side encryption modes, object lock style retention and storage class transitions. Build a list of every S3 operation your applications call, mark each as supported, partial or missing on your release, and treat any partial result as a design constraint. Re-check the list at each upgrade, because the status can change between major releases.
How to test the SDKs your applications use
The SDK your application uses, and how it is configured, affects S3 compatibility as much as the server does, so test the client stack before you judge the gateway. The same RGW endpoint can look compatible with one SDK version and broken with another. The settings that most often change results are checksum behavior, the multipart upload threshold, path-style versus virtual-host addressing, retry and timeout values, and how presigned URLs are generated and expired. Newer SDK releases have changed default checksum handling, so a version that worked last year may send requests that a gateway rejects or stores differently. Record the exact SDK version, language runtime and every non-default setting for each application, because a compatibility result is only valid for that combination.
Virtual-host addressing deserves its own check. It requires DNS names and wildcard certificate coverage for bucket subdomains, so an application that works in path-style mode in a test zone may fail in production once it switches styles. Multipart thresholds matter in a similar way: an SDK that splits large uploads at a different size exercises different code paths in the gateway, including part sizes, aborted uploads and completion. Presigned URLs should be generated by the application's own SDK and then fetched from a client outside the cluster, with the clock and proxy path matching production. Confirm with your Ceph version's Object Gateway documentation which checksum algorithms and addressing modes it accepts before you assume parity.
Run each production SDK version, with its production configuration, against a test zone that mirrors your RGW endpoints, certificates and load balancer. Script the calls your applications actually make, such as put, get, multipart upload, list with pagination, copy, delete, tagging, versioning and presigned access, and record which calls succeed unchanged, which need a code or configuration change, and which fail. Keep that record with the SDK version so that an upgrade of either the application or the Ceph release triggers a rerun. Treat any call that needs a workaround as a migration task with an owner, not as a footnote. Ask your implementation partner or internal team to include retry and timeout tests under simulated endpoint failure, since those behaviors decide whether a transient gateway error becomes a user-facing outage.
How DeployOpen scopes an S3 compatibility assessment
The sequence turns a list of applications into a written, tested view of which S3 features your workloads can rely on in Ceph RGW, and what must change before migration. It ends with a documented result and a migration plan your team can act on.
Inventory applications and S3 feature use
DeployOpen works with your application owners to list every workload that calls S3 today, the SDKs and versions each one uses, and the operations it depends on, such as multipart upload, versioning, object lock, lifecycle, notifications, presigned URLs and IAM calls. The output is a written inventory that maps each workload to the features it needs, so the assessment tests what your business actually runs rather than the full S3 surface.
Agree scope and the RGW test environment in writing
Before any testing starts, DeployOpen and your team agree the scope document: which applications and SDK versions are in, which Ceph release and RGW configuration will be tested, whether the test cluster matches production settings such as TLS, authentication backend and load balancer, and what counts as a pass. Your team confirms the target release against the Ceph releases page, since support windows and backports differ by release. The output is a signed-off scope, acceptance criteria and the list of features explicitly out of scope.
Build the RGW test environment
DeployOpen stands up or reuses a non-production RGW endpoint in your infrastructure, with users, accounts and buckets that mirror the production model, plus a test copy of the TLS and endpoint configuration. Clusters are deployed with cephadm or your existing method, and the environment is documented with versions, configuration keys and access paths. The step is complete when a smoke test from each client network reaches the endpoint and the environment record matches what production will run.
Run SDK and application test suites
Each in-scope application runs its own test suite or a DeployOpen test pack against RGW, covering object CRUD, large and multipart uploads, listing and pagination, metadata, ACLs, versioning, lifecycle and any notification or IAM calls it makes. Failures are recorded with the request, response code and the SDK version, so the team can separate RGW gaps from client configuration errors. Where a feature is partial or missing, DeployOpen tests the workaround your application would need and records it as a finding.
Test IAM, STS and access control paths
Where workloads use IAM policies, roles, temporary credentials or cross-account access, DeployOpen tests those paths against the agreed account and user model, including denied requests that should fail. The team checks policy evaluation and credential expiry behavior against what the applications expect. The output is a table of tested access scenarios with the expected and observed result for each.
Document results and classify each gap
DeployOpen writes a compatibility report that lists every tested operation as supported, partial or missing for each application, with the evidence attached: request traces, logs and test output. Each gap is classified by impact and by whether it can be solved with application changes, configuration, a different feature or a decision by your team. The report is accepted when your application owners confirm that the findings match their own observations.
Hand off the migration plan and retest
DeployOpen turns the accepted findings into a migration plan that names the application changes, configuration decisions and open risks, with an owner for each item. Where changes are made, the affected tests are rerun and the results are added to the report. The sequence closes when your team has a plan, a retest record and a list of checks to repeat after any Ceph upgrade or RGW configuration change.
