Uploaded image for project: 'XWiki Platform'
  1. XWiki Platform
  2. XWIKI-22820

Solr field cache uses a lot of RAM

    XMLWordPrintable

Details

    • Unknown
    • N/A
    • N/A

    Description

      Steps to reproduce:

      1. Have a huge number of documents indexed (like a million documents) with a modest amount of RAM (like 6 GB) and internal Solr.
      2. Perform some searches with a lot of results (like *, searching for all documents)

      Expected result:

      The wiki is still working and most memory is still available to XWiki.

      Actual result:

      The wiki crashes because of an OOM error and most memory is consumed by Solr's field cache. This can be analyzed with the following Groovy snippet:

      {{groovy wiki="false"}}
      import org.apache.solr.common.params.ModifiableSolrParams;
      
      def solr = services.component.getInstance(org.xwiki.search.solr.internal.api.SolrInstance)
      ModifiableSolrParams params = new ModifiableSolrParams();
      params.set("group", "core");
      params.set("prefix", "CACHE.core.fieldCache");
      params.set("qt", "/admin/metrics");
      
      solr.query(params)
      {{/groovy}}
      

      In an instance, it reported 5.3 GiB total size. More information about the problems of this cache can be found at https://support.hcl-software.com/community?id=community_blog&sys_id=ddedbfb11be58914a2f48661cd4bcbd8

      In the concrete example, the fields used the following amounts of RAM:

      id: 5.02 GB
      space_facet: 446.97 MB
      space_exact: 266.94 MB
      class: 95.76 MB
      attauthor: 95.76 MB
      mimetype: 95.70 MB
      title_sort: 81.11 MB
      name_exact: 44.72 MB
      creator: 27.56 MB
      author: 27.56 MB
      locale: 15.09 MB
      doclocale: 15.01 MB
      type: 8.96 MB
      wiki: 5.84 MB
      

      Attachments

        Activity

          People

            MichaelHamann Michael Hamann
            MichaelHamann Michael Hamann
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: