XWiki Platform

Too many open files

Details

  • Type: Bug Bug
  • Status: Closed Closed
  • Priority: Major Major
  • Resolution: Duplicate
  • Affects Version/s: 2.6, 2.7
  • Fix Version/s: None
  • Component/s: None
  • Labels:
    None
  • Environment:
    Linux 2.6.31-22-generic #69-Ubuntu SMP Wed Nov 24 08:51:08 UTC 2010 i686 GNU/Linux
    tomcat6 (from ubuntu distribution)
  • keywords:
    attachments, open-files
  • Difficulty:
    Unknown
  • Documentation:
    N/A
  • Documentation in Release Notes:
    N/A
  • Similar issues:
    XWIKI-251Too many title types in the WYSIWYG editor
    XWIKI-5084Too many JAR dependencies for generating an id in IdGenerator
    XWIKI-68History page too big when the document has many versions
    XWIKI-883access right cannot be updated if too many users or groups selected
    XWIKI-2122Too many empty lines when using IE in the Rights Management UI
    XWIKI-9124Distribution Wizard upgrades too many documents even when there are no changes in them
    XWIKI-5585Attachment action icons too far from attachment names
    XWIKI-8705Attachment on-disk file cache leaks open file descriptor
    XWIKI-8681XWikiAttachmentContent on-disk file cache leak open file descriptors
    XWIKI-4037Export of image/files fails if page name too long

Description

I tried to upgrade from version 2.4.30451 to version 2.7 (linux ubuntu, postgresql). In 2.4 I was able to export a full dump of the wiki (around 550MB) after the update (including the 2.7 xar) the export fails.

I think for one if an attachment is not available, or missing then the whole export fails and it is extremely hard to figure out which wiki page actually triggered the error.

Second even if all those errors are resolved, the export fails with the error "Too many open files" (see the snippet below). I then checked the number of open files. On 2.7 I am seeing a large number of open files even after a restart. Of course there are all the system and library files, but there is also a large number of temporary files open (and those remain open for at least one hour of inactivity). At the bottom, there is an example of some of these files. I guess the lucene indexing after the restart generates these (there were 181 open files with similar names). If I try to export the wiki, the number of open files goes into the thousands and most of them are of the kind quoted below.

I then tried version 2.6 with the same symptoms. I reverted to 2.4 and everything works fine again.

Version 2.7 seems a lot faster on export and also seems to require a lot less resources, and I would be very happy about that.

I hope this information is helpful to you. Please feel free to ask for more specific information.

Thanks so much,
--Elaina.

— Logfile snippet 1 —
Dec 25 15:19:23 herald jsvc.exec[10046]: 2010-12-25 15:19:23,861 https://xxxxxLinux herald 2.6.31-22-generic #69-Ubuntu SMP Wed Nov 24 08:51:08 UTC 2010 i686 GNU/Linux/xwiki/bin/export/XWiki/Export?editor=globaladmin&section=Export WARN web.XWikiAction - Uncaught exception: Error number 11015 in 11: Exception while exporting#012Wrapped Exception: Failed to get InputStream #012com.xpn.xwiki.XWikiException: Error number 11015 in 11: Exception while exporting#012Wrapped Exception: Failed to get InputStream#012#011at com.xpn.xwiki.web.ExportAction.render(ExportAction.java:67)#012#011at com.xpn.xwiki.web.XWikiAction.execute(XWikiAction.java:216)#012#011at com.xpn.xwiki.web.XWikiAction.execute(XWikiAction.java:117)#012#011at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:431)#012#011at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:236)#012#011at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1196)#012#011at org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:432)#012#011at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)#012#011at javax.servlet.http.HttpServlet.service(Ht
Dec 25 15:19:23 herald jsvc.exec[10046]: tpServlet.java:717)#012#011at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)#012#011at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)#012#011at com.xpn.xwiki.web.ActionFilter.doFilter(ActionFilter.java:129)#012#011at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)#012#011at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)#012#011at com.xpn.xwiki.wysiwyg.server.filter.ConversionFilter.doFilter(ConversionFilter.java:152)#012#011at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)#012#011at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)#012#011at com.xpn.xwiki.plugin.webdav.XWikiDavFilter.doFilter(XWikiDavFilter.java:68)#012#011at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)#012#011at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationF
Dec 25 15:19:23 herald jsvc.exec[10046]: ilterChain.java:206)#012#011at org.xwiki.container.servlet.filters.internal.SavedRequestRestorerFilter.doFilter(SavedRequestRestorerFilter.java:218)#012#011at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)#012#011at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)#012#011at org.xwiki.container.servlet.filters.internal.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:112)#012#011at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)#012#011at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)#012#011at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)#012#011at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)#012#011at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)#012#011at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)#012#011at org.apache.catalina.co
Dec 25 15:19:23 herald jsvc.exec[10046]: re.StandardEngineValve.invoke(StandardEngineValve.java:109)#012#011at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)#012#011at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856)#012#011at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)#012#011at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:454)#012#011at java.lang.Thread.run(Thread.java:636)#012#012#012Wrapped Exception:#012#012java.io.FileNotFoundException: /tmp/tomcat6-temp/upload_31503983_12d1e97e78d__8000_00004753.tmp (Too many open files)#012#011at java.io.FileInputStream.open(Native Method)#012#011at java.io.FileInputStream.<init>(FileInputStream.java:137)#012#011at org.apache.commons.fileupload.disk.DiskFileItem.getInputStream(DiskFileItem.java:230)#012#011at com.xpn.xwiki.doc.XWikiAttachmentContent.getContentInputStream(XWikiAttachmentContent.java:211)#012#011at com.xpn.xwiki.doc.XWikiAttachment.toXML(XWikiAttachment.java:409)#012#011at com.xpn.xwiki.doc.XWikiDocument.toXML(XWikiDocument.java:3900)#012#011at com.x
Dec 25 15:19:23 herald jsvc.exec[10046]: pn.xwiki.doc.XWikiDocument.toXML(XWikiDocument.java:3978)#012#011at com.xpn.xwiki.doc.XWikiDocument.addToZip(XWikiDocument.java:3665)#012#011at com.xpn.xwiki.plugin.packaging.Package.addToZip(Package.java:1038)#012#011at com.xpn.xwiki.plugin.packaging.Package.export(Package.java:341)#012#011at com.xpn.xwiki.plugin.packaging.PackageAPI.export(PackageAPI.java:226)#012#011at com.xpn.xwiki.plugin.packaging.PackageAPI.backupWiki(PackageAPI.java:298)#012#011at com.xpn.xwiki.web.ExportAction.exportXAR(ExportAction.java:262)#012#011at com.xpn.xwiki.web.ExportAction.render(ExportAction.java:60)#012#011at com.xpn.xwiki.web.XWikiAction.execute(XWikiAction.java:216)#012#011at com.xpn.xwiki.web.XWikiAction.execute(XWikiAction.java:117)#012#011at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:431)#012#011at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:236)#012#011at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1196)#012#011at org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:432)#012#011at javax.servlet.h
Dec 25 15:19:23 herald jsvc.exec[10046]: ttp.HttpServlet.service(HttpServlet.java:637)#012#011at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)#012#011at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)#012#011at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)#012#011at com.xpn.xwiki.web.ActionFilter.doFilter(ActionFilter.java:129)#012#011at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)#012#011at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)#012#011at com.xpn.xwiki.wysiwyg.server.filter.ConversionFilter.doFilter(ConversionFilter.java:152)#012#011at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)#012#011at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)#012#011at com.xpn.xwiki.plugin.webdav.XWikiDavFilter.doFilter(XWikiDavFilter.java:68)#012#011at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilt
Dec 25 15:19:23 herald jsvc.exec[10046]: erChain.java:235)#012#011at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)#012#011at org.xwiki.container.servlet.filters.internal.SavedRequestRestorerFilter.doFilter(SavedRequestRestorerFilter.java:218)#012#011at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)#012#011at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)#012#011at org.xwiki.container.servlet.filters.internal.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:112)#012#011at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)#012#011at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)#012#011at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)#012#011at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)#012#011at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)#012#011at org.apache.ca
Dec 25 15:19:23 herald jsvc.exec[10046]: talina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)#012#011at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)#012#011at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)#012#011at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856)#012#011at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)#012#011at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:454)#012#011at java.lang.Thread.run(Thread.java:636)

------end logfile ----

---- open files ----
jsvc 28191 tomcat6 357w REG 8,1 14631 655616 /tmp/tomcat6-temp/upload_474e3796_12d2399566c__8000_00002483.tmp
jsvc 28191 tomcat6 358w REG 8,1 32314 655622 /tmp/tomcat6-temp/upload_474e3796_12d2399566c__8000_00002487.tmp
jsvc 28191 tomcat6 359w REG 8,1 65685 655629 /tmp/tomcat6-temp/upload_474e3796_12d2399566c__8000_00002499.tmp
jsvc 28191 tomcat6 360w REG 8,1 1865969 655621 /tmp/tomcat6-temp/upload_474e3796_12d2399566c__8000_00002485.tmp
jsvc 28191 tomcat6 361w REG 8,1 216645 655623 /tmp/tomcat6-temp/upload_474e3796_12d2399566c__8000_00002489.tmp
jsvc 28191 tomcat6 362w REG 8,1 210714 655624 /tmp/tomcat6-temp/upload_474e3796_12d2399566c__8000_00002491.tmp
jsvc 28191 tomcat6 363w REG 8,1 174376 655625 /tmp/tomcat6-temp/upload_474e3796_12d2399566c__8000_00002493.tmp
jsvc 28191 tomcat6 364w REG 8,1 265307 655626 /tmp/tomcat6-temp/upload_474e3796_12d2399566c__8000_00002495.tmp
jsvc 28191 tomcat6 365w REG 8,1 3231201 655628 /tmp/tomcat6-temp/upload_474e3796_12d2399566c__8000_00002497.tmp
jsvc 28191 tomcat6 366w REG 8,1 65685 655631 /tmp/tomcat6-temp/upload_474e3796_12d2399566c__8000_00002501.tmp
jsvc 28191 tomcat6 367w REG 8,1 100421 655632 /tmp/tomcat6-temp/upload_474e3796_12d2399566c__8000_00002503.tmp
jsvc 28191 tomcat6 368w REG 8,1 97256 655633 /tmp/tomcat6-temp/upload_474e3796_12d2399566c__8000_00002505.tmp
jsvc 28191 tomcat6 369w REG 8,1 1804006 655634 /tmp/tomcat6-temp/upload_474e3796_12d2399566c__8000_00002507.tmp
jsvc 28191 tomcat6 372w REG 8,1 2003474 655635 /tmp/tomcat6-temp/upload_474e3796_12d2399566c__8000_00002509.tmp
jsvc 28191 tomcat6 373w REG 8,1 6041408 655637 /tmp/tomcat6-temp/upload_474e3796_12d2399566c__8000_00002511.tmp
jsvc 28191 tomcat6 374w REG 8,1 2021116 655639 /tmp/tomcat6-temp/upload_474e3796_12d2399566c__8000_00002513.tmp
jsvc 28191 tomcat6 375r REG 8,1 700446 655640 /tmp/tomcat6-temp/upload_474e3796_12d2399566c__8000_00002515.tmp
---- end open files ----

  1. Text File
    TooManyOpenFilesXWiki.txt
    28/Mar/11 16:47
    10 kB
    Roman Arkadijovych Muntyanu

Issue Links

duplicates

Bug - A problem which impairs or prevents the functions of the product. XWIKI-8705 Attachment on-disk file cache leaks open file descriptor

  • Blocker - Blocks development and/or testing work, production could not run.
  • Closed - The issue is considered finished, the resolution is correct and has been pushed to production. Issues which are not closed can be reopened.

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Andreas Jonsson added a comment -

There is a new release of apache commons file upload - version 1.2.2 - that among othere things contains a fix for "Temporary files have not been deleted, if an error occurred in FileUploadBase.parseRequest();".

https://issues.apache.org/jira/browse/FILEUPLOAD-160

Currently version 1.2.1 is used in XWiki. An upgrade might fix this?

Show
Andreas Jonsson added a comment - There is a new release of apache commons file upload - version 1.2.2 - that among othere things contains a fix for "Temporary files have not been deleted, if an error occurred in FileUploadBase.parseRequest();". https://issues.apache.org/jira/browse/FILEUPLOAD-160 Currently version 1.2.1 is used in XWiki. An upgrade might fix this?
Hide
Permalink
Sergiu Dumitriu added a comment -

Looking at their issues, it seems that FILEUPLOAD-160 was actually fixed for 1.3 (unreleased at the moment), and there aren't any actual issues fixed between 1.2.1 and 1.2.2 (see https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310476&version=12315238 ).

Show
Sergiu Dumitriu added a comment - Looking at their issues, it seems that FILEUPLOAD-160 was actually fixed for 1.3 (unreleased at the moment), and there aren't any actual issues fixed between 1.2.1 and 1.2.2 (see https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310476&version=12315238 ).
Hide
Permalink
Roman Arkadijovych Muntyanu added a comment -

Experience same errors on export on

  • Linux 2.6.18-164.9.1.el5 #1 SMP Tue Dec 15 21:04:57 EST 2009 i686 i686 i386 GNU/Linux
  • XWiki Enterprise 2.6.33065
  • Tomcat 6.0.29
Show
Roman Arkadijovych Muntyanu added a comment - Experience same errors on export on Linux 2.6.18-164.9.1.el5 #1 SMP Tue Dec 15 21:04:57 EST 2009 i686 i686 i386 GNU/Linux XWiki Enterprise 2.6.33065 Tomcat 6.0.29
Hide
Permalink
Sergiu Dumitriu added a comment -

Probably fixed in r35751.

Show
Sergiu Dumitriu added a comment - Probably fixed in r35751.
Hide
Permalink
Roman Arkadijovych Muntyanu added a comment - - edited

I have made an upgrade to XWiki Enterprise 3.0-rc-1.35909 and experience "too many open files" issue during XWiki export.

Environment 

Linux 2.6.18-164.9.1.el5 #1 SMP Tue Dec 15 21:04:57 EST 2009 i686 i686 i386 GNU/Linux
XWiki Enterprise 3.0-rc-1.35909
Tomcat 6.0.29
The software is installed on ESX virtual machine

The size of our XWiki xar-file is ~200MB (if to measure by latest successful export).
I was able to download from 195MB in the beginning of my tests to 24MB in the end (in different attempts) of 200MB before the export has interrupted.

Checking the number of open files in different ways exposed such info

# ls -l /proc/<tomcatProcessId>/fd/ >tmp.txt ## results in tmp.txt containing 650-900 rows (that is open file records)
                                             ## either during export or after export fails
                                             ## and ~400 open tmp files in ./apache-tomcat-6.0.29/temp/ folder
# lsof | wc -l      ## minimum 3461 maximum 4342 open files depending on the test
3461
# cat /proc/sys/fs/file-nr  ## minimum 1644 , maximum 2304 open files depending on the test
1664    0       255889

I've also encountered following exception repeated in catalina.log 150K times (making it 100MB in size)

Mar 28, 2011 5:30:52 PM org.apache.tomcat.util.net.JIoEndpoint$Acceptor run
SEVERE: Socket accept failed
java.net.SocketException: Too many open files
        at java.net.PlainSocketImpl.socketAccept(Native Method)
        at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)
        at java.net.ServerSocket.implAccept(ServerSocket.java:453)
        at java.net.ServerSocket.accept(ServerSocket.java:421)
        at org.apache.tomcat.util.net.DefaultServerSocketFactory.acceptSocket(DefaultServerSocketFactory.java:61)
        at org.apache.tomcat.util.net.JIoEndpoint$Acceptor.run(JIoEndpoint.java:352)
        at java.lang.Thread.run(Thread.java:619)

And the attached file TooManyOpenFilesXWiki.txt shows more of the stack trace.

Hundreds of thousands of java.net.SocketException made me first think this is a different bug. But then I tried the test several times more and were not able to reproduce the socket exception again. Number of temporary files opened by tomcat process varies from test to test (13 to 450). I have compared with diff tool which files are changed between runs and it looks like many temp files (~350) persist.

Show
Roman Arkadijovych Muntyanu added a comment - - edited I have made an upgrade to XWiki Enterprise 3.0-rc-1.35909 and experience "too many open files" issue during XWiki export. Environment Linux 2.6.18-164.9.1.el5 #1 SMP Tue Dec 15 21:04:57 EST 2009 i686 i686 i386 GNU/Linux XWiki Enterprise 3.0-rc-1.35909 Tomcat 6.0.29 The software is installed on ESX virtual machine The size of our XWiki xar-file is ~200MB (if to measure by latest successful export). I was able to download from 195MB in the beginning of my tests to 24MB in the end (in different attempts) of 200MB before the export has interrupted. Checking the number of open files in different ways exposed such info # ls -l /proc/<tomcatProcessId>/fd/ >tmp.txt ## results in tmp.txt containing 650-900 rows (that is open file records) ## either during export or after export fails ## and ~400 open tmp files in ./apache-tomcat-6.0.29/temp/ folder # lsof | wc -l ## minimum 3461 maximum 4342 open files depending on the test 3461 # cat /proc/sys/fs/file-nr ## minimum 1644 , maximum 2304 open files depending on the test 1664 0 255889 I've also encountered following exception repeated in catalina.log 150K times (making it 100MB in size) Mar 28, 2011 5:30:52 PM org.apache.tomcat.util.net.JIoEndpoint$Acceptor run SEVERE: Socket accept failed java.net.SocketException: Too many open files at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390) at java.net.ServerSocket.implAccept(ServerSocket.java:453) at java.net.ServerSocket.accept(ServerSocket.java:421) at org.apache.tomcat.util.net.DefaultServerSocketFactory.acceptSocket(DefaultServerSocketFactory.java:61) at org.apache.tomcat.util.net.JIoEndpoint$Acceptor.run(JIoEndpoint.java:352) at java.lang.Thread.run(Thread.java:619) And the attached file TooManyOpenFilesXWiki.txt shows more of the stack trace. Hundreds of thousands of java.net.SocketException made me first think this is a different bug. But then I tried the test several times more and were not able to reproduce the socket exception again. Number of temporary files opened by tomcat process varies from test to test (13 to 450). I have compared with diff tool which files are changed between runs and it looks like many temp files (~350) persist.
Hide
Permalink
Roman Arkadijovych Muntyanu added a comment -

In addition today I have encountered "too many open" files issue with [Lucene index updating thread].
Configuration is the same

Show
Roman Arkadijovych Muntyanu added a comment - In addition today I have encountered "too many open" files issue with [Lucene index updating thread] . Configuration is the same
Hide
Permalink
Sergiu Dumitriu added a comment -

The problem comes from the attachment storage, which keeps FDs open. Normally, they should be cleared out when attachments are removed from the cache. Can you try playing with the attachment storage cache size?

Show
Sergiu Dumitriu added a comment - The problem comes from the attachment storage, which keeps FDs open. Normally, they should be cleared out when attachments are removed from the cache. Can you try playing with the attachment storage cache size?
Hide
Permalink
Roman Arkadijovych Muntyanu added a comment -

I did not find any property similar to "attachment.storage.cache.size" in neither xwiki.cfg nor xwiki.properties.
What's the configuration property for attachment storage cache size?

Just in case I'd mention that all our attachments are stored in the database.

Show
Roman Arkadijovych Muntyanu added a comment - I did not find any property similar to " attachment.storage.cache.size " in neither xwiki.cfg nor xwiki.properties . What's the configuration property for attachment storage cache size? Just in case I'd mention that all our attachments are stored in the database.
Hide
Permalink
Roman Arkadijovych Muntyanu added a comment -

Working with our IT guys we were able to make the exception disappear (in our environment - see above).
First we have increased the maximum allowed number of open files from 250K to 400K, in this mode we were able to export once out of 5 attempts. Which wasn't that nice.

Then we've set following values in /etc/security/limits.conf

*                -       nofile          8192
*                soft    nofile          65535
*                hard    nofile          65535

after which I was able to successfully do 4 exports one after another.

Hope that will help.

Show
Roman Arkadijovych Muntyanu added a comment - Working with our IT guys we were able to make the exception disappear (in our environment - see above). First we have increased the maximum allowed number of open files from 250K to 400K, in this mode we were able to export once out of 5 attempts. Which wasn't that nice. Then we've set following values in /etc/security/limits.conf * - nofile 8192 * soft nofile 65535 * hard nofile 65535 after which I was able to successfully do 4 exports one after another. Hope that will help.
Hide
Permalink
Vincent Massol added a comment -

Sergiu, you reopened this issue but it's been already released and the fix for is set to 3.0RC1... Could you either change the fix for if what has been committed is useless or create a new issue instead?

Show
Vincent Massol added a comment - Sergiu, you reopened this issue but it's been already released and the fix for is set to 3.0RC1... Could you either change the fix for if what has been committed is useless or create a new issue instead?
Hide
Permalink
Denis Gervalle added a comment -

Finally solve in XWiki 4.4.1

Show
Denis Gervalle added a comment - Finally solve in XWiki 4.4.1
Hide
Permalink
Roman Arkadijovych Muntyanu added a comment -

So what was causing the issue?

Show
Roman Arkadijovych Muntyanu added a comment - So what was causing the issue?
Hide
Permalink
Denis Gervalle added a comment - - edited

Mainly, some FD were open and never closed for never loaded and duplicated attachments contents.
See XWIKI-8681 and XWIKI-8705 comments and commits.

Show
Denis Gervalle added a comment - - edited Mainly, some FD were open and never closed for never loaded and duplicated attachments contents. See XWIKI-8681 and XWIKI-8705 comments and commits.

People

  • Assignee:
    Denis Gervalle
    Reporter:
    Elaina Mountain
Vote (1)
Watch (4)

Dates

  • Created:
    Updated:
    Resolved:
    Date of First Response: