Uploaded image for project: 'XWiki Platform'
  1. XWiki Platform
  2. XWIKI-15009

Open files leak in HTML export

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Major
    • 9.11.5, 10.3
    • 9.11.1
    • Old Core
    • None
    • debian 9.3
    • Unknown
    • N/A
    • N/A

    Description

      Running Jetty 9.4.6.v20170531 on a debian 9.3

      We recently moved from XWiki 6.4.4 to XWiki 9.9 then 9.11.1.

      6.4.4 to 9.9 migration hapened on Jan/15-16

      I think the issue started raising up starting from XWiki 9.11.1, which happened on Jan/31. We can observe that on the attached graph (sorry for the Y-axis ...)

      The number of opened file limit for the java process was historically 32768 (for all versions), and it has never reach it until two days ago where I raised to 65535.

      You can see on the graph that from a fresh start, it's doesn't always raise at the same rate and can reach the limit more or less rapidly.

      Now, I have inspected the number of opened files, here is a report with about 1 hours interval :

      time is 9:38
      
      number of *single* files (on filesystem) opened by the process : 906
      (sudo lsof -p 32287 | egrep '(REG|DIR)' | awk '{print $9'} | sort | uniq -c |sort -n | wc -l)
      
      number of handlers (all types) for the process : 13923
      (sudo lsof  -p 32287 | wc -l)
      
      partial handlers listing, by unique file
      (sudo lsof -p 32287 | egrep '(REG|DIR)' | awk '{print $9'} | sort | uniq -c |sort -n)
      
      ( ... < 23 )
           23 /appdata/webapps/root/resources/icons/xwiki/favicon144.png
           23 /appdata/webapps/root/resources/icons/xwiki/favicon16.png
           23 /appdata/webapps/root/resources/icons/xwiki/favicon.ico
           23 /appdata/webapps/root/resources/icons/xwiki/favicon.svg
           35 /appdata/webapps/root/resources/icons/datamodel/calendar.png
           36 /appdata/webapps/root/resources/uicomponents/widgets/datepicker/dateTimePicker.js
           39 /appdata/webapps/root/resources/icons/silk/arrow_refresh.png
           39 /appdata/webapps/root/resources/icons/silk/lock_edit.png
           39 /appdata/webapps/root/resources/icons/silk/page_white_copy.png
           39 /appdata/webapps/root/resources/icons/silk/page_white_edit.png
           40 /appdata/webapps/root/resources/icons/silk/bullet_arrow_up.png
           41 /appdata/webapps/root/resources/icons/silk/textfield_rename.png
           44 /appdata/webapps/root/resources/icons/silk/bullet_arrow_down.png
           63 /appdata/webapps/root/resources/icons/silk/user.png
           76 /appdata/webapps/root/resources/icons/xwiki/ajax-loader-large.gif
          183 /appdata/webapps/root/resources/icons/silk/link.png
          185 /appdata/webapps/root/resources/icons/silk/cancel.png
          185 /appdata/webapps/root/resources/icons/silk/delete.png
          185 /appdata/webapps/root/resources/icons/silk/eye.png
          185 /appdata/webapps/root/resources/icons/silk/textfield_add.png
          193 /appdata/webapps/root/resources/icons/silk/tag_yellow.png
          227 /appdata/webapps/root/resources/icons/xwiki/pagination-controls.png
          368 /appdata/webapps/root/resources/icons/silk/add.png
          368 /appdata/webapps/root/resources/icons/silk/comment.png
          368 /appdata/webapps/root/resources/icons/silk/comments_add.png
          368 /appdata/webapps/root/resources/icons/silk/folder.png
          368 /appdata/webapps/root/resources/icons/silk/note_add.png
          368 /appdata/webapps/root/resources/icons/silk/note.png
          368 /appdata/webapps/root/resources/icons/xwiki/accordion.png
          368 /appdata/webapps/root/resources/icons/xwiki/parent_edit.gif
          368 /appdata/webapps/root/resources/uicomponents/require/deferred.js
          369 /appdata/webapps/root/resources/icons/silk/attach.png
          369 /appdata/webapps/root/resources/icons/silk/bullet_add.png
          369 /appdata/webapps/root/resources/icons/silk/bullet_toggle_minus.png
          369 /appdata/webapps/root/resources/icons/silk/bullet_toggle_plus.png
          369 /appdata/webapps/root/resources/icons/silk/chart_organisation.png
          369 /appdata/webapps/root/resources/icons/silk/cross.png
          369 /appdata/webapps/root/resources/icons/silk/help.png
          369 /appdata/webapps/root/resources/icons/silk/page_white_text.png
          369 /appdata/webapps/root/resources/icons/silk/pencil.png
          369 /appdata/webapps/root/resources/icons/silk/tick.png
          369 /appdata/webapps/root/resources/icons/xwiki/external-link.png
          369 /appdata/webapps/root/resources/icons/xwiki/spinner.gif
          369 /appdata/webapps/root/resources/icons/xwiki/spinner-white.gif
          369 /appdata/webapps/root/resources/js/prototype/prototype.js
          369 /appdata/webapps/root/resources/js/xwiki/compatibility.js
          369 /appdata/webapps/root/resources/js/xwiki/eventsBridge.js
          369 /appdata/webapps/root/resources/js/xwiki/markerScript.js
          369 /appdata/webapps/root/resources/js/xwiki/meta.js
      
      time is 10:37
      number of *single* files (on filesystem) opened by the process : 914
      (sudo lsof -p 32287 | egrep '(REG|DIR)' | awk '{print $9'} | sort | uniq -c |sort -n | wc -l)
      
      number of handlers (all types) for the process : 13874
      (sudo lsof  -p 32287 | wc -l)
      
      partial handlers listing, by unique file
      (sudo lsof -p 32287 | egrep '(REG|DIR)' | awk '{print $9'} | sort | uniq -c |sort -n)
      
      ... (< 24)
           24 /appdata/webapps/root/resources/icons/xwiki/favicon144.png
           24 /appdata/webapps/root/resources/icons/xwiki/favicon16.png
           24 /appdata/webapps/root/resources/icons/xwiki/favicon.ico
           24 /appdata/webapps/root/resources/icons/xwiki/favicon.svg
           36 /appdata/webapps/root/resources/icons/datamodel/calendar.png
           37 /appdata/webapps/root/resources/uicomponents/widgets/datepicker/dateTimePicker.js
           39 /appdata/webapps/root/resources/icons/silk/arrow_refresh.png
           39 /appdata/webapps/root/resources/icons/silk/lock_edit.png
           39 /appdata/webapps/root/resources/icons/silk/page_white_copy.png
           39 /appdata/webapps/root/resources/icons/silk/page_white_edit.png
           40 /appdata/webapps/root/resources/icons/silk/bullet_arrow_up.png
           41 /appdata/webapps/root/resources/icons/silk/textfield_rename.png
           44 /appdata/webapps/root/resources/icons/silk/bullet_arrow_down.png
           66 /appdata/webapps/root/resources/icons/silk/user.png
           77 /appdata/webapps/root/resources/icons/xwiki/ajax-loader-large.gif
          186 /appdata/webapps/root/resources/icons/silk/link.png
          189 /appdata/webapps/root/resources/icons/silk/cancel.png
          189 /appdata/webapps/root/resources/icons/silk/delete.png
          189 /appdata/webapps/root/resources/icons/silk/eye.png
          189 /appdata/webapps/root/resources/icons/silk/textfield_add.png
          196 /appdata/webapps/root/resources/icons/silk/tag_yellow.png
          230 /appdata/webapps/root/resources/icons/xwiki/pagination-controls.png
          373 /appdata/webapps/root/resources/icons/silk/add.png
          373 /appdata/webapps/root/resources/icons/silk/comment.png
          373 /appdata/webapps/root/resources/icons/silk/comments_add.png
          373 /appdata/webapps/root/resources/icons/silk/note_add.png
          373 /appdata/webapps/root/resources/icons/silk/note.png
          373 /appdata/webapps/root/resources/icons/xwiki/accordion.png
          373 /appdata/webapps/root/resources/icons/xwiki/parent_edit.gif
          373 /appdata/webapps/root/resources/uicomponents/require/deferred.js
          374 /appdata/webapps/root/resources/icons/silk/attach.png
          374 /appdata/webapps/root/resources/icons/silk/bullet_add.png
          374 /appdata/webapps/root/resources/icons/silk/bullet_toggle_minus.png
          374 /appdata/webapps/root/resources/icons/silk/bullet_toggle_plus.png
          374 /appdata/webapps/root/resources/icons/silk/chart_organisation.png
          374 /appdata/webapps/root/resources/icons/silk/cross.png
          374 /appdata/webapps/root/resources/icons/silk/folder.png
          374 /appdata/webapps/root/resources/icons/silk/help.png
          374 /appdata/webapps/root/resources/icons/silk/page_white_text.png
          374 /appdata/webapps/root/resources/icons/silk/pencil.png
          374 /appdata/webapps/root/resources/icons/silk/tick.png
          374 /appdata/webapps/root/resources/icons/xwiki/external-link.png
          374 /appdata/webapps/root/resources/icons/xwiki/spinner.gif
          374 /appdata/webapps/root/resources/icons/xwiki/spinner-white.gif
          374 /appdata/webapps/root/resources/js/prototype/prototype.js
          374 /appdata/webapps/root/resources/js/xwiki/compatibility.js
          374 /appdata/webapps/root/resources/js/xwiki/eventsBridge.js
          374 /appdata/webapps/root/resources/js/xwiki/markerScript.js
          374 /appdata/webapps/root/resources/js/xwiki/meta.js
      

      It doesn't sound good to me that a single file gets that much handler opened on it, and that those numbers are keeping increasing.

      Regards,

      Attachments

        Issue Links

          Activity

            People

              tmortagne Thomas Mortagne
              mh Martin
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: