Uploaded image for project: 'XWiki Platform'
  1. XWiki Platform
  2. XWIKI-17536

Improve XWiki Export / Filter stream error handling to be able to export regardeless of some page failing and stability of the wiki instance

    XMLWordPrintable

Details

    • Improvement
    • Resolution: Unresolved
    • Major
    • None
    • 7.4
    • Filter
    • None
    • Unknown

    Description

      Because of issues like XWIKI-17535 or XWIKI-17534 or potentially other import errors, the filter stream export is lacking reliability, leading to needing to use an alternative method of exporting a wiki (https://snippets.xwiki.org/xwiki/bin/view/Extension/Large%20Wiki%20Export%202/).

      Looking at the type of error that happened an analysis has shown that it would be necessary to:

      1/ Being able to export to individual files one disk and not stop when a page fails

      -> this is required because just one XAR can make a huge file which could fail and could also break in the middle of the export (java OOM or other issues)

      2/ Being able to export using a restart on failure

      -> if the export fails (which could be avoided) or the wiki crashes or restart during a long export (this cannot easily be avoided) we would need a method to restart on failure

      -> this could be implemented by saving to a directly and if the file exists not export. Ideally the export process would should be minimal on that specific page (no serialization or reading of the xwiki page) so that the process starts again at the next non exported page

      3/ Having the possibility to add some pages to ignore in the export

      -> this is required in case a specific page is bringing down the wiki at export time

      4/ Having a proper error reporting of failed pages

      -> it is necessary to know which pages failed to export so that they can be handled manually (without versions or as text). Currently Filter stream does give error reporting but does not go to the end. Standard XWiki XAR import fails without any error message of which page failed and where (this is also true from import which won't give feedback if there is a timeout)

      An additional problem is that current improvement to the reliability of filter streams or XAR export cannot be used on older versions of XWiki. This will mean that it will still be necessary to use scripts like https://snippets.xwiki.org/xwiki/bin/view/Extension/Large%20Wiki%20Export%202/ when other systems fail

      Attachments

        Activity

          People

            Unassigned Unassigned
            ludovic Ludovic Dubost
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: