Similar to Support Resumable / Chunked Uploads for Large Files via JMAP Endpoint
Fundamentally that wouldn’t work because stalwart always buffers requests and responses into RAM. This also means that someone with Permission::UnlimitedUploads can OOM Stalwart, which is neat.
I propose that a newtype be created that’s an enum of either Vec<u8> or some kind of AsyncRead which will be used in BlobStore.put_blob and BlobStore.get_blob. http_proto::request::fetch_body would have to be changed to have this newtype too. Said newtype could also have a method to coerce into a Vec<u8> for “normal” sized requests.
The back-ends for BlobStore::Fs, BlobStore::S3, and BlobStore::Azure all seem to work pretty well with ranges too, one could create a readable stream that only fetches within the specified range.
Though there’s some caveats with uploading new blobs
- This won’t be useful in deployments that re-use the data store as the blob store. (How would this be handled? Error?)
- The stream will have to be reset-able since…
BlobHash’s need to be computed before the blobs are committed- It’s a requirement for azure
Since the streams need to be reset-able when creating blobs, maybe requests could be buffered to temporary files instead of RAM if the in-memory size limit is exceeded. From my exploration so far, streams that would be created by fetching blobs don’t seem to need to be reset-able.
Edit: I might work on a fork on this soon.
