XWiki Platform
  1. XWiki Platform
  2. XWIKI-2874

Allow XWiki to scale with large number of objects

    Details

    • Type: Improvement Improvement
    • Status: Open Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 1.7 M3
    • Fix Version/s: None
    • Component/s: Model
    • Labels:
    • keywords:
      patch
    • Pull Request Status:
      Awaiting Contributor feedback
    • Similar issues:
      XWIKI-1415Use SVN revision number as XWiki's build number when displaying the version
      XWIKI-750EditObjects should allow editing a specified object
      XWIKI-7339Watchlist documents property does not scale
      XWIKI-2938XWiki Objects' number field values are not preserved in previous versions of a page in case of object deletion
      XWIKI-13009REST API breaks if an object deletion causes objects to not be consecutively numbered
      XWIKI-12279Allow getting number of active installs for a given extension
      XWIKI-6658The configured default image quality for scaled JPEGs is ignored
      XWIKI-2339Cannot import large attachments
      XWIKI-6530Scaled images are not exported to PDF
      XWIKI-334Impossible to create xwiki:properties with name equal to xwiki:object jcr properties

      Description

      It seems pages get very slow to load when documents have lots of objects attached to them. For example a page with 1000 comments will load very slowly.

      We need to find a strategy or practice (generic or not) that allows XWiki pages to load fast even when they have lots of objects.

      1. testNewDocumentLoader.groovy
        4 kB
        CalebJamesDeLisle
      2. XWIKI-2874-newLoadXWikiDoc-0.01.patch
        16 kB
        CalebJamesDeLisle
      3. XWIKI-2874-newLoadXWikiDoc-0.02.patch
        24 kB
        CalebJamesDeLisle

        Issue Links

          Activity

          Hide
          CalebJamesDeLisle added a comment -

          I have developed a test script to find problems with the new document loader. Just because the loader works on my installation of XE doesn't mean it will work with every database in existence. This test script depends on fixing the bug XWIKI-4377. loader0.03 (which is only a slight change from 0.02) does not pass this test so I will remove it.

          Show
          CalebJamesDeLisle added a comment - I have developed a test script to find problems with the new document loader. Just because the loader works on my installation of XE doesn't mean it will work with every database in existence. This test script depends on fixing the bug XWIKI-4377 . loader0.03 (which is only a slight change from 0.02) does not pass this test so I will remove it.
          Hide
          CalebJamesDeLisle added a comment -

          There is an email describing the change here:
          http://xwiki.markmail.org/message/bndsnhwh3saeoc5i

          Is it worthwhile to rework the loader to load multiple documents simultaneously? We could have a request batcher which holds requests and executes them periodically (say every 50ms.) It might improve performance on a large database.
          Is this worthwhile?

          Show
          CalebJamesDeLisle added a comment - There is an email describing the change here: http://xwiki.markmail.org/message/bndsnhwh3saeoc5i Is it worthwhile to rework the loader to load multiple documents simultaneously? We could have a request batcher which holds requests and executes them periodically (say every 50ms.) It might improve performance on a large database. Is this worthwhile?
          Hide
          Sergiu Dumitriu added a comment -

          XWIKI-8948 should allow to simplify the patch a bit.

          Show
          Sergiu Dumitriu added a comment - XWIKI-8948 should allow to simplify the patch a bit.
          Hide
          Vincent Massol added a comment -

          FTR we have the issue on extensions.xwiki.org where for example the IRC Bot Application has 12 dependencies and there have been over 70 releases of it. That results in 840 xobjects. Thus opening it using the object editor takes a very long amount of time (or times out). Not only we need to allow having unlimited number of xobjects in documents, we also need to paginate the object editor. Note that viewing the doc doesn't cause a problem and only editing it in object editor does, so there be something in it that takes time (maybe accessing the class metadata for each xobject?).

          Show
          Vincent Massol added a comment - FTR we have the issue on extensions.xwiki.org where for example the IRC Bot Application has 12 dependencies and there have been over 70 releases of it. That results in 840 xobjects. Thus opening it using the object editor takes a very long amount of time (or times out). Not only we need to allow having unlimited number of xobjects in documents, we also need to paginate the object editor. Note that viewing the doc doesn't cause a problem and only editing it in object editor does, so there be something in it that takes time (maybe accessing the class metadata for each xobject?).
          Hide
          Thomas Mortagne added a comment -

          What takes this long is probably just the generation of such a big html. Making the object editor more asynchronous might help this too (like loading the objects when you open the section associated to the class).

          Show
          Thomas Mortagne added a comment - What takes this long is probably just the generation of such a big html. Making the object editor more asynchronous might help this too (like loading the objects when you open the section associated to the class).

            People

            • Assignee:
              Unassigned
              Reporter:
              Vincent Massol
            • Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Date of First Response: