[Self-Hosted] media-proxy intermittently fails S3 reads with "transport chunk exceeded its byte bound"

(#872) Bug Needs triage media self-hosting

Observed behaviour

On my self-hosted Fluxer instance, media-proxy intermittently fails while reading larger MP4 files from the bundled SeaweedFS S3 backend. Failed requests return HTTP 502 with: object storage operation failed: object storage response transport chunk exceeded its byte bound The exact same object and transformation request can sometimes succeed with HTTP 200 and sometimes fail with HTTP 502. For one 122,994,218-byte MP4, repeated identical requests produced: 8 successful / 12 failed 5 successful / 15 failed 3 successful / 17 failed A subsequent diagnostic run failed 5/5 When a failure occurs, SeaweedFS reports broken pipe after the media-proxy connection closes. The amount transferred before the connection closes varies substantially, from only a few MB to over 100 MB. As a control test, I downloaded the exact same 122,994,218-byte S3 object directly from SeaweedFS using the official AWS CLI container on the same Docker network. All 20/20 downloads succeeded with the correct file size. The problem also occurs with a freshly uploaded copy of the same MP4, so it does not appear to be limited to migrated/old SeaweedFS objects. Expected behaviour: media-proxy should reliably read and process an object that the configured SeaweedFS S3 backend can deliver successfully.

Reproduction steps

Start a self-hosted Fluxer instance using the standard Docker Compose configuration with the bundled SeaweedFS S3 backend. Upload a larger MP4 file. The file used for this test is 122,994,218 bytes (approximately 118 MB). Allow Fluxer to process the attachment. Request a transformed preview of the MP4, for example: /media/attachments/<guild-or-channel-id>/<attachment-id>/video.mp4?format=webp&width=64 Repeat the exact same request several times sequentially. Observe that some requests return HTTP 200 while others return HTTP 502. On failed requests, media-proxy logs: object storage operation failed: object storage response transport chunk exceeded its byte bound At the corresponding time, SeaweedFS reports a broken pipe while streaming the object to media-proxy. The problem does not require concurrent requests and persists after restarting media-proxy. Direct S3 downloads of the same object using AWS CLI succeed reliably.

Build information

Stable Web 2026.926.115042, Windows NT 10.0 (x64), Chrome 152.0.0.0, Locale en-US

Platform

Self-hosting

Evidence

Relevant media-proxy error: reason="storage_error" object storage operation failed: object storage response transport chunk exceeded its byte bound During five consecutive failures, SeaweedFS terminated streaming after approximately: 110 MB 9.5 MB 92.7 MB 3.75 MB 2.5 MB SeaweedFS reported: GetObjectHandler: failed to stream ... write: broken pipe Independent control test: Same SeaweedFS server Same Docker network Same S3 credentials Same bucket Same 122,994,218-byte object Official AWS CLI client 20 consecutive complete downloads Result: 20/20 successful Every downloaded object was exactly 122,994,218 bytes Additional troubleshooting already performed: Restarted media-proxy: issue persists Confirmed media-proxy is not OOM killed Peak observed media-proxy memory was approximately 104 MiB of its 512 MiB limit Tested large raw object reads successfully Tested large HTTP range reads successfully Temporarily increased SeaweedFS S3 idle timeout to 3600: no improvement; change reverted Reproduced while bypassing the external Nginx reverse proxy Reproduced with sequential requests, so concurrency is not required Reproduced with a newly uploaded copy of the file

5 comments

Sign in with Fluxer to comment and vote.
Comment by @FrigidSouls
RexSystem 1 vote originally by @FrigidSouls on GitHub OP
Additional A/B testing rules out SeaweedFS 4.47 as the source of the regression. I still have my previous self-hosted Fluxer installation and its original SeaweedFS data volume. That environment uses:
  • SeaweedFS 4.34
  • The exact same fluxer-media-proxy:v1 image/digest as the new installation:sha256:740e236db7ad9efd9cc1213ba769d6d8e9e24a079975a1cb431539d58fceba5c
I started only the old SeaweedFS 4.34 container and tested the same 122,994,218-byte MP4. Direct AWS CLI S3 downloads against SeaweedFS 4.34: SUCCESS=20 FAILURE=0 I then started the existing media-proxy container and accessed media-proxy directly over the Docker network, bypassing Nginx and Caddy. 20 identical sequential requests for: /attachments/<id>/<id>/video.mp4?format=webp&width=64 produced: SUCCESS=15 FAILURE=5 All five failures reported the same error: object storage operation failed: object storage response transport chunk exceeded its byte bound Therefore the issue reproduces with both SeaweedFS 4.34 and 4.47 while direct S3 GETs against both versions succeed 20/20. Current comparison:
  • SeaweedFS 4.34 + AWS CLI: 20/20
  • SeaweedFS 4.47 + AWS CLI: 20/20
  • media-proxy + SeaweedFS 4.34: 15/20
  • media-proxy + SeaweedFS 4.47: intermittently fails, with multiple tests ranging from 8/20 to 3/20 successful
  • media-proxy image digest is identical in both environments
This appears to further isolate the problem to the current media-proxy S3 response-reading path rather than a specific SeaweedFS version.
Comment by @FrigidSouls
RexSystem 1 vote originally by @FrigidSouls on GitHub OP
Additional control test: I reproduced the issue on a clean Debian 12 VM. Test environment:
  • Debian 12 (bookworm), kernel 6.1
  • Docker Engine 29.8.1
  • SeaweedFS 4.34
  • Same restored SeaweedFS data
  • Same ghcr.io/fluxerapp/fluxer-media-proxy:v1 image/digest as the original deployment
  • Exact media-proxy runtime environment copied from the original container, with only the SeaweedFS endpoint changed for the test Docker network
  • Requests made directly to media-proxy over the Docker network, bypassing Nginx/Caddy
Before testing media-proxy, I performed 20 complete downloads of the same 122,994,218-byte MP4 directly from SeaweedFS S3 using AWS CLI. All 20 succeeded. I then made 20 identical requests directly to media-proxy: /attachments/.../video.mp4?format=webp&width=64 Results:
  • HTTP 200: 10/20
  • HTTP 502: 10/20
Every failed request reported the same error: object storage operation failed: object storage response transport chunk exceeded its byte bound Successful and failed requests were interspersed, so the same object and identical request can either succeed or fail. This reproduces the problem on Debian 12 as well as Debian 13, and with SeaweedFS 4.34 as well as 4.47. Combined with reliable direct S3 downloads, this appears to rule out the Debian version, reverse proxy, migrated object corruption, and a SeaweedFS-4.47-specific regression as primary causes. The remaining evidence points toward the media-proxy S3 response-reading/client path or an interoperability issue between that client path and SeaweedFS's S3 responses.
Comment by @TheyMadeMeChangeMyUsername
RexSystem 1 vote originally by @TheyMadeMeChangeMyUsername on GitHub
the stock .ENV file absolutely REFUSED to spin up any 'storage containers' or whatever when media was being sent within a chat and just threw an error i ended up switching to mini and seems to work great for me now!
seaweedfs:
    image: chrislusf/seaweedfs:4.47
    command: ["mini", "-dir=/data"]
    volumes:
      - seaweedfs-data:/data
    networks:
      - fluxer
mini uses single-node provisioning - the stock .ENV file was looking for other mounts - which didnt exist (debian 13 LXC in proxmox)