Storage deduplication for file uploads

(#3888) Feature Open: Under consideration api infra performance self-hosting

Thread

Comment by sodiboo
Does Fluxer even need to do anything here? I feel like there are performance tradeoffs to this, and the S3 backing storage for media uploads is perfectly capable of implementing this on behalf of a Fluxer instance. as far as i can tell, SeaweedFS, which ships by default in the self-hosted compose recipes, does not in fact deduplicate files, but for example Garage does. It seems natural that an instance operator can choose an S3 implementation according to their needs.
when I send that silly cat gif from my saved media ten times in a row
it also feels natural to me that this should not cause a reupload, but just link to the same instance of the file each time. but that doesn't need global deduplication; this just needs the client not to reupload stuff unnecessarily.