Uploaded image for project: 'Batch Import Application'
  1. Batch Import Application
  2. BATCHIMP-46

Improve memory usage when adding attachments to imported pages

    XMLWordPrintable

Details

    • Improvement
    • Resolution: Fixed
    • Major
    • 2.4
    • 2.2, 2.3
    • API
    • Unknown

    Description

      The objective of this ticket is to improve the batch import API to use java InputStream API when adding attachments to imported pages, the improvement is useful to avoid import failure caused by memory issues when attaching big files which are currently loaded entirely on memory.

      Steps to reproduce

      • On an XWiki instance (version 14.4.6) install the Batch Import application version 2.3
      • Create a CSV file containing the data of the page to be imported, see screenshot
      • Create a zip file (files.zip) that contains a big file, (file size > 1GB), named attachment.zip
      • Create a test page and edit it using object editor
      • Attach an object of class to " BatchImport.BatchImportClass" to the test page
      • Set the object properties as follows :
        • fileupload = true
        • datafilename = The path to the zip file that contains the attachments (path to files.zip)
      • Save the changes
      • Open the test page in view mode and follow the batch import steps until the last one (import), here is the mapping to be used :

      Expected result : The page has been correctly imported and the file attached to the page

      Current result : The import failed with a "java.lang.OutOfMemoryError: Java heap space" error, see screenshot :

      Attachments

        1. csv-files.png
          csv-files.png
          28 kB
        2. imported-page.png
          imported-page.png
          673 kB
        3. import-error.png
          import-error.png
          1010 kB
        4. import-mapping.png
          import-mapping.png
          1.01 MB

        Activity

          People

            mouhb Mohamed Boussaa
            mouhb Mohamed Boussaa
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: