Edit history

[Self-Hosted] media-proxy intermittently fails S3 reads with "transport chunk exceeded its byte bound" has not been edited, so there are no earlier versions.

Current version | Original by Rex
Show

[Self-Hosted] media-proxy intermittently fails S3 reads with "transport chunk exceeded its byte bound"

Observed behaviour

On my self-hosted Fluxer instance, media-proxy intermittently fails while reading larger MP4 files from the bundled SeaweedFS S3 backend. Failed requests return HTTP 502 with: object storage operation failed: object storage response transport chunk exceeded its byte bound The exact same object and transformation request can sometimes succeed with HTTP 200 and sometimes fail with HTTP 502. For one 122,994,218-byte MP4, repeated identical requests produced: 8 successful / 12 failed 5 successful / 15 failed 3 successful / 17 failed A subsequent diagnostic run failed 5/5 When a failure occurs, SeaweedFS reports broken pipe after the media-proxy connection closes. The amount transferred before the connection closes varies substantially, from only a few MB to over 100 MB. As a control test, I downloaded the exact same 122,994,218-byte S3 object directly from SeaweedFS using the official AWS CLI container on the same Docker network. All 20/20 downloads succeeded with the correct file size. The problem also occurs with a freshly uploaded copy of the same MP4, so it does not appear to be limited to migrated/old SeaweedFS objects. Expected behaviour: media-proxy should reliably read and process an object that the configured SeaweedFS S3 backend can deliver successfully.

Reproduction steps

Start a self-hosted Fluxer instance using the standard Docker Compose configuration with the bundled SeaweedFS S3 backend. Upload a larger MP4 file. The file used for this test is 122,994,218 bytes (approximately 118 MB). Allow Fluxer to process the attachment. Request a transformed preview of the MP4, for example: /media/attachments/<guild-or-channel-id>/<attachment-id>/video.mp4?format=webp&width=64 Repeat the exact same request several times sequentially. Observe that some requests return HTTP 200 while others return HTTP 502. On failed requests, media-proxy logs: object storage operation failed: object storage response transport chunk exceeded its byte bound At the corresponding time, SeaweedFS reports a broken pipe while streaming the object to media-proxy. The problem does not require concurrent requests and persists after restarting media-proxy. Direct S3 downloads of the same object using AWS CLI succeed reliably.

Build information

Stable Web 2026.926.115042, Windows NT 10.0 (x64), Chrome 152.0.0.0, Locale en-US

Platform

Self-hosting

Evidence

Relevant media-proxy error: reason="storage_error" object storage operation failed: object storage response transport chunk exceeded its byte bound During five consecutive failures, SeaweedFS terminated streaming after approximately: 110 MB 9.5 MB 92.7 MB 3.75 MB 2.5 MB SeaweedFS reported: GetObjectHandler: failed to stream ... write: broken pipe Independent control test: Same SeaweedFS server Same Docker network Same S3 credentials Same bucket Same 122,994,218-byte object Official AWS CLI client 20 consecutive complete downloads Result: 20/20 successful Every downloaded object was exactly 122,994,218 bytes Additional troubleshooting already performed: Restarted media-proxy: issue persists Confirmed media-proxy is not OOM killed Peak observed media-proxy memory was approximately 104 MiB of its 512 MiB limit Tested large raw object reads successfully Tested large HTTP range reads successfully Temporarily increased SeaweedFS S3 idle timeout to 3600: no improvement; change reverted Reproduced while bypassing the external Nginx reverse proxy Reproduced with sequential requests, so concurrency is not required Reproduced with a newly uploaded copy of the file