Loading training content into Odoo, a 480 MB video took down the worker handling it. A larger batch failed partway and then refused to retry, reporting that the attachment collided with an existing file.
Root cause
Creating an attachment from raw content holds the entire file in memory, and more than once while it is hashed and written. On a hosted worker with a memory limit, a large file is enough to get the process killed. The collision on retry was the half-written file from the first attempt.
A streaming approach
- Receive the upload in chunks and write each chunk to a temporary file, updating a checksum as you go.
- Move the finished file into the filestore at the path its checksum dictates.
- Create the attachment record without content, then set its stored filename, size and checksum directly.
Peak memory is now about one chunk, whatever the file size.
Related limits worth knowing
- The stored file size is a 32-bit integer in the database. A generated file over 2 GB fails with “integer out of range” unless you handle that case.
- On Odoo.sh, staging builds are full copies of production and count toward the storage quota. A few large generated files can exhaust it. Purge generated files after download.
- A shell session without the right data directory reads and writes a different filestore from the live site.