Debug a failing S3 upload with a local request feed and object browser

When an S3 upload fails, it’s tempting to keep changing the SDK configuration until something works. That usually leaves you with a successful request and no idea which change fixed it.

A better approach is to follow the evidence through the whole path:

1. What endpoint did the client target?
2. Did a request reach the local server, and what response did it return?
3. If the request succeeded, what bucket, key, and content state were actually stored?

That separates connection and configuration problems from request-level failures and application bugs. This article uses Objectively’s request feed and object browser to make those checks visible. Objectively is my own macOS local S3 development tool, so treat the recommendation as a disclosed product suggestion and compare it with a debugger, proxy, emulator, or test framework when one of those is a better fit.

Start with a small, safe fixture

Use a disposable bucket name and a synthetic file. Don’t debug with production credentials, private documents, or a bucket that another test is using.

The local environment provides copy-ready connection details. Use the endpoint and region shown by the environment you are running, keep the test credentials separate from real AWS credentials, and confirm the current product documentation for any addressing requirements.

set -eu

export AWS_ACCESS_KEY_ID=objectively_local_access_key
export AWS_SECRET_ACCESS_KEY=objectively_local_secret_key
export AWS_ENDPOINT_URL=http://localhost:9000
export AWS_REGION=us-east-1

export LOCAL_RUN_ID="$(uuidgen | tr -d '-' | tr '[:upper:]' '[:lower:]')"
export LOCAL_BUCKET="debug-s3-upload-$LOCAL_RUN_ID"
export LOCAL_KEY=fixtures/upload.txt
export LOCAL_FILE="$(mktemp "${TMPDIR:-/tmp}/objectively-upload-fixture.XXXXXX")"
export LOCAL_BUCKET_MARKER="${LOCAL_FILE}.bucket-created"

printf 'local S3 debugging fixture\n' > "$LOCAL_FILE"

The placeholders above are deliberately not credentials. Copy the connection values shown by the local environment, keep them in the shell or an ignored file, and check that the endpoint is local before running the client. AWS documents the supported environment variables and endpoint precedence.

Create the bucket and make one known-good control request with the AWS CLI’s s3 mb and s3 cp commands:

set -eu

require_loopback_endpoint() {
  local endpoint="${AWS_ENDPOINT_URL:-}"
  local port
  case "$endpoint" in
    http://localhost:*) port="${endpoint#http://localhost:}" ;;
    http://127.0.0.1:*) port="${endpoint#http://127.0.0.1:}" ;;
    http://\[::1\]:*) port="${endpoint#http://\[::1\]:}" ;;
    *)
      printf 'Refusing AWS operation: AWS_ENDPOINT_URL is not a loopback HTTP endpoint.\n' >&2
      exit 1
      ;;
  esac
  case "$port" in
    ''|*[!0-9]*)
      printf 'Refusing AWS operation: AWS_ENDPOINT_URL must have a numeric port.\n' >&2
      exit 1
      ;;
  esac
}

require_loopback_endpoint
if ! aws --endpoint-url "$AWS_ENDPOINT_URL" s3 mb "s3://$LOCAL_BUCKET"; then
  printf 'Bucket creation failed; refusing to continue with upload.\n' >&2
  exit 1
fi

printf '%s\n%s\n%s\n' \
  "$AWS_ENDPOINT_URL" "$LOCAL_BUCKET" "$LOCAL_FILE" > "$LOCAL_BUCKET_MARKER"

require_loopback_endpoint
if ! aws --endpoint-url "$AWS_ENDPOINT_URL" s3 cp \
  "$LOCAL_FILE" "s3://$LOCAL_BUCKET/$LOCAL_KEY"; then
  printf 'Upload failed; leaving the owned fixture for inspection or cleanup.\n' >&2
  exit 1
fi

Use this control request to establish a known request shape and an object you can find in the browser. It says nothing about full S3 compatibility. If your application fails whilst this control request works, the local server is probably reachable and the next useful comparison is the application’s endpoint, signing configuration, addressing style, bucket, and key.

Before treating the control as successful, check the command output and confirm the object in the Objectively UI.

Read the evidence in order

The fastest diagnosis is usually the least glamorous one: write down the values that the failing client is supposed to use. Keep these beside the client error:

AWS documents endpoint configuration through explicit client settings, environment variables, shared configuration, and CLI options. During debugging, an explicit endpoint on the failing command or client is easier to reason about than an inherited profile. AWS’s service-specific endpoint configuration documentation describes the precedence rules.

If the local endpoint’s documented addressing requirement is path-style, a virtual-hosted request such as bucket.localhost differs from localhost/bucket; enable the SDK’s path-style option and inspect the request that arrives. The exact URL, host header, and signing details depend on the SDK and endpoint.

If no request appears

An empty request feed is useful evidence, but it doesn’t identify one cause. Work through the boundary between the client and the server:

Check that the local server is running

Confirm that the environment is running and that the configured port is the port in the client. If the default port is occupied, the documented troubleshooting path is:

lsof -i :9000

Only stop a process you own and know is disposable. Otherwise, reconfigure that process or change the Objectively environment’s port and use the new endpoint in the client. A client aimed at an old port will never produce a request in the feed for the new environment.

Check the final endpoint, not just the source code

Environment variables, shared AWS configuration, and SDK defaults can override what you think the application is using. Log or inspect the final host and port without printing the secret. If the failing process runs from an IDE, test runner, shell script, or separate launch service, check the environment available to that process rather than the one in your terminal.

Check addressing style

If the SDK moves the bucket into the hostname, the request may fail before it reaches a loopback server that expects path-style addressing. The request you want to compare is structurally like:

http://localhost:9000/debug-s3-upload/fixtures/upload.txt

The exact URL, host header, and signing details depend on the SDK. Don’t paste a secret or full signed URL into an issue while inspecting them.

Remember what the feed can and cannot mean

Some server-side errors may appear in the feed, but connection failures, wrong ports, client-side signing errors, and other failures can occur outside what it records. “Nothing appeared” means the feed has no recorded request, not that the server definitely received nothing.

If a request appears with an error

Now the question changes from “where did the request go?” to “what did the server receive, and what did it reject?”

Use the current request-feed entry to line up a client error with the request’s rough shape, but do not treat it as a packet capture or full trace. Record only the fields the current UI exposes.

Match one client attempt with the available request evidence. Then compare the values your application intended to send. A useful diagnostic record can look like this:

14:32:10.418  upload attempt  ->  client error 403

That line is illustrative; use the actual client and request evidence from your own diagnostic run. Do not assume that every field you want is available in the current feed.

The next check depends on the evidence:

The table is a set of branches, not a promise that one status code uniquely identifies one root cause. S3-compatible servers and SDKs can render errors differently. Preserve the client error and the exact feed entry together.

If the request succeeded but the application is wrong

A successful HTTP response only tells you that the local server accepted that operation. It doesn’t prove that your application used the bucket, key, content, or response it expected.

Open the bucket in the object browser and check the state directly, using only the controls exposed by the current product:

• Is the expected bucket present?
• Is the object under the expected prefix and key?
• Can you inspect the stored object and compare it with the fixture you intended to upload?
• Did the application accidentally upload a different filename or an empty fixture?

The object browser is a way to browse and manage local files. Treat any deletion exposed by the current UI as final: use disposable data and preserve evidence before removing anything.

If the object is correct in the browser but the application displays stale or incorrect state, investigate the application’s read path, parsing, cache, and UI update separately.

The request log cannot show that final state. The feed tells you what the endpoint handled; the browser lets you compare that with the resulting local object state.

Make one correction and repeat the same request

Once you have a hypothesis, change one thing. Don’t simultaneously change the endpoint, credentials, region, addressing mode, bucket, and key. A successful retry after six changes is not a diagnosis.

For example, if the feed is empty and the client was using a stale port, update only the endpoint and repeat the same upload. If the feed shows the wrong key, keep the endpoint and credentials fixed and correct only the key construction. If the feed shows an authentication error, verify the access key, secret, and region belong to the same environment before changing the operation.

Repeat the control request and the application request with a fresh timestamp. Capture:

1. The client configuration with secrets redacted.
2. The client error or response.
3. The matching request-feed evidence.
4. The object-browser state after the request.

The before-and-after evidence should make the change explainable. If it doesn’t, reduce the fixture again and test the client with a simple bucket or object operation before returning to the application’s larger upload flow.

Clean up after capturing the evidence

Inspect the request feed and object browser, save the redacted screenshots or notes you need, and only then remove the disposable fixture. The bucket name includes a UUID so it should not collide with another local run; still, keep it dedicated to this diagnostic. Re-check that the endpoint is loopback immediately before cleanup. Use the AWS CLI s3 rm and s3 rb commands only with that verified local endpoint.

set -eu

: "${AWS_ENDPOINT_URL:?Run the setup block first}"
: "${LOCAL_BUCKET:?Run the setup block first}"
: "${LOCAL_KEY:?Run the setup block first}"
: "${LOCAL_FILE:?Run the setup block first}"
LOCAL_BUCKET_MARKER="${LOCAL_FILE}.bucket-created"

# Run this only after capturing the request and object evidence.
require_loopback_endpoint() {
  local endpoint="${AWS_ENDPOINT_URL:-}"
  local port
  case "$endpoint" in
    http://localhost:*) port="${endpoint#http://localhost:}" ;;
    http://127.0.0.1:*) port="${endpoint#http://127.0.0.1:}" ;;
    http://\[::1\]:*) port="${endpoint#http://\[::1\]:}" ;;
    *)
      printf 'Refusing cleanup: AWS_ENDPOINT_URL is not a loopback HTTP endpoint.\n' >&2
      exit 1
      ;;
  esac
  case "$port" in
    ''|*[!0-9]*)
      printf 'Refusing cleanup: AWS_ENDPOINT_URL must have a numeric port.\n' >&2
      exit 1
      ;;
  esac
}

require_loopback_endpoint

if [ ! -f "$LOCAL_BUCKET_MARKER" ]; then
  printf 'Refusing cleanup: this walkthrough did not record ownership of the bucket.\n' >&2
  exit 1
fi

MARKER_ENDPOINT="$(sed -n '1p' "$LOCAL_BUCKET_MARKER")"
MARKER_BUCKET="$(sed -n '2p' "$LOCAL_BUCKET_MARKER")"
MARKER_FILE="$(sed -n '3p' "$LOCAL_BUCKET_MARKER")"
if [ "$MARKER_ENDPOINT" != "$AWS_ENDPOINT_URL" ] \
  || [ "$MARKER_BUCKET" != "$LOCAL_BUCKET" ] \
  || [ "$MARKER_FILE" != "$LOCAL_FILE" ]; then
  printf 'Refusing cleanup: the ownership marker does not match the current fixture.\n' >&2
  exit 1
fi

require_loopback_endpoint
aws --endpoint-url "$AWS_ENDPOINT_URL" s3 rm \
  "s3://$LOCAL_BUCKET/$LOCAL_KEY"
require_loopback_endpoint
aws --endpoint-url "$AWS_ENDPOINT_URL" s3 rb "s3://$LOCAL_BUCKET"
rm -f -- "$LOCAL_FILE" "$LOCAL_BUCKET_MARKER"

The cleanup is deliberately scoped to the named bucket, its one fixture object, and the temporary file. If the diagnostic created additional objects, inspect the bucket first and remove only the disposable test data.

What this local loop does not prove

This workflow is good at making a local development path visible. It has clear limits:

• A request-feed entry is not a full trace. Use only the metadata the current product exposes; do not assume it includes headers, bodies, authentication details, or response payloads.
• Feed history and retention can change with the product. Check the current documentation or UI before relying on persistence across an app restart.
• A local success is not proof of AWS compatibility. Test the operations your application relies on against the supported local subset and run the appropriate real-AWS integration test from your cloud test boundary.
• Testing against a second AWS-shaped local endpoint can broaden emulator coverage, but it is not evidence of Amazon S3 production behaviour, IAM, durability, networking, quotas, or billing.
• A local HTTP endpoint is not a production security model. Keep credentials and fixtures local, and don’t infer TLS, IAM policy behaviour, ACLs, or versioning from a successful upload.
• A request that never reaches the handler cannot appear in a server-side feed. Use the client’s own logs or a deliberately selected network diagnostic when you need evidence before the endpoint.

That last distinction matters. The goal isn’t to make the local server look like AWS in every detail; it’s to shorten the path from an opaque upload error to a falsifiable next check.

Keep the evidence safe and useful

When you turn a diagnosis into an issue or a support report, use a synthetic fixture and redact:

• access keys, secrets, session tokens, and signed query parameters;
• private bucket names, local usernames, absolute paths, and customer data;
• request bodies or previews that contain anything other than the fixture;
• screenshots that leave a credential visible in an environment panel or terminal.

Include the app version, macOS version, SDK and client version, endpoint host/port, addressing style, operation, exact client error, and the matching feed evidence. If no supported diagnosis fits, that is enough information for a minimal reproducible issue without uploading the original data.

If you’re developing on a Mac and want this request-to-object loop in a native local S3 tool, Objectively is one option to evaluate. Use the product’s current connection details, test the operations your application needs, and keep the final integration check against AWS where AWS behaviour matters.