Affects Version/s: 11.10.4
Fix Version/s: None
Component/s: Search - Solr
I've recently come across a particular case when a 5 MB *.vsdx file (Microsoft Visio diagram) lead the Solr Index thread into what looks like a memory leak that very quickly fills up the heap, spikes CPU usage (mainly caused by GC) and eventually results in java.lang.OutOfMemoryError: GC overhead limit exceeded. (was able to reproduce locally with a fresh instance)
The worst part is that once the Solr indexing fails with the OOM exception, the heap remains full. Even worse, after a restart, since the attachment was not properly indexed, Solr starts over again, leading to another OOM, practically locking access to the wiki. (this part, after a restart, I did not reproduce locally, Solr saying 1 documents added and finishing the indexing job at startup).
Here is a thread dump from my initial remote XWiki instance where I noticed the issue, just before the heap managed to get full again (i.e. after a restart):
And the top heap consumers around the same time, retrieved with glowroot's "Heap histogram" view:
I tried locally with a sample vsdx file and everything was indexed fine and I was able to even find text from inside the diagram. However, once I have attached the same problematic vsdx file, I immediately got an OOM locally as well, as mentioned. Some particular content must be tripping apache Tika.
Not sure if the solution is to upgrade Tika or to add some sort of exclude option (file name/path/reference) to the Solr indexer, in order to allow users to work around problematic content.
Note: I am unable to attach the problematic file publicly due to privacy reasons.