This should be fixed via #3153. Feel free to retest and close the issue if you can confirm it's fixed!
Meilisearch indexes with half of the CPU cores it can see by default, and every indexing thread gets its own buffers on top of MAX_INDEXING_MEMORY. The container has no CPU limit, so on a many-core machine like yours it sees every core, and the indexer goes well past 768mb no matter what the indexing memory is set to. Queued indexing work survives restarts, which is why it kept getting OOM-killed every 13 to 15 minutes.
The defaults are now:
FLUXER_MEILISEARCH_MAX_INDEXING_THREADS=2 (new)
FLUXER_MEILISEARCH_MEMORY_LIMIT=1536mb
FLUXER_MEILISEARCH_MAX_INDEXING_MEMORY=256mb
With those, a 300k message backlog indexed fine at about 1 GB peak in testing, while the old defaults were killed within seconds. The 8 GB and 4 GB host profiles in the docs were updated to match. The change is in the Compose files, so sh install.sh --update picks it up. If you've set your own limits, you can keep them and add the thread setting.
Thread
Comment by Hampus
MAX_INDEXING_MEMORY. The container has no CPU limit, so on a many-core machine like yours it sees every core, and the indexer goes well past 768mb no matter what the indexing memory is set to. Queued indexing work survives restarts, which is why it kept getting OOM-killed every 13 to 15 minutes. The defaults are now:
With those, a 300k message backlog indexed fine at about 1 GB peak in testing, while the old defaults were killed within seconds. The 8 GB and 4 GB host profiles in the docs were updated to match. The change is in the Compose files, soFLUXER_MEILISEARCH_MAX_INDEXING_THREADS=2(new)FLUXER_MEILISEARCH_MEMORY_LIMIT=1536mbFLUXER_MEILISEARCH_MAX_INDEXING_MEMORY=256mbsh install.sh --updatepicks it up. If you've set your own limits, you can keep them and add the thread setting.