Too many open files

I tried to upgrade from version 2.4.30451 to version 2.7 (linux ubuntu, postgresql). In 2.4 I was able to export a full dump of the wiki (around 550MB) after the update (including the 2.7 xar) the export fails.

I think for one if an attachment is not available, or missing then the whole export fails and it is extremely hard to figure out which wiki page actually triggered the error.

Second even if all those errors are resolved, the export fails with the error "Too many open files" (see the snippet below). I then checked the number of open files. On 2.7 I am seeing a large number of open files even after a restart. Of course there are all the system and library files, but there is also a large number of temporary files open (and those remain open for at least one hour of inactivity). At the bottom, there is an example of some of these files. I guess the lucene indexing after the restart generates these (there were 181 open files with similar names). If I try to export the wiki, the number of open files goes into the thousands and most of them are of the kind quoted below.

I then tried version 2.6 with the same symptoms. I reverted to 2.4 and everything works fine again.

Version 2.7 seems a lot faster on export and also seems to require a lot less resources, and I would be very happy about that.

I hope this information is helpful to you. Please feel free to ask for more specific information.

Thanks so much,
--Elaina.

— Logfile snippet 1 —
Dec 25 15:19:23 herald jsvc.exec[10046]: 2010-12-25 15:19:23,861 https://xxxxxLinux herald 2.6.31-22-generic #69-Ubuntu SMP Wed Nov 24 08:51:08 UTC 2010 i686 GNU/Linux/xwiki/bin/export/XWiki/Export?editor=globaladmin&section=Export WARN web.XWikiAction - Uncaught exception: Error number 11015 in 11: Exception while exporting#012Wrapped Exception: Failed to get InputStream #012com.xpn.xwiki.XWikiException: Error number 11015 in 11: Exception while exporting#012Wrapped Exception: Failed to get InputStream#012#011at com.xpn.xwiki.web.ExportAction.render(ExportAction.java:67)#012#011at com.xpn.xwiki.web.XWikiAction.execute(XWikiAction.java:216)#012#011at com.xpn.xwiki.web.XWikiAction.execute(XWikiAction.java:117)#012#011at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:431)#012#011at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:236)#012#011at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1196)#012#011at org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:432)#012#011at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)#012#011at javax.servlet.http.HttpServlet.service(Ht
Dec 25 15:19:23 herald jsvc.exec[10046]: tpServlet.java:717)#012#011at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)#012#011at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)#012#011at com.xpn.xwiki.web.ActionFilter.doFilter(ActionFilter.java:129)#012#011at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)#012#011at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)#012#011at com.xpn.xwiki.wysiwyg.server.filter.ConversionFilter.doFilter(ConversionFilter.java:152)#012#011at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)#012#011at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)#012#011at com.xpn.xwiki.plugin.webdav.XWikiDavFilter.doFilter(XWikiDavFilter.java:68)#012#011at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)#012#011at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationF
Dec 25 15:19:23 herald jsvc.exec[10046]: ilterChain.java:206)#012#011at org.xwiki.container.servlet.filters.internal.SavedRequestRestorerFilter.doFilter(SavedRequestRestorerFilter.java:218)#012#011at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)#012#011at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)#012#011at org.xwiki.container.servlet.filters.internal.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:112)#012#011at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)#012#011at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)#012#011at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)#012#011at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)#012#011at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)#012#011at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)#012#011at org.apache.catalina.co
Dec 25 15:19:23 herald jsvc.exec[10046]: re.StandardEngineValve.invoke(StandardEngineValve.java:109)#012#011at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)#012#011at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856)#012#011at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)#012#011at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:454)#012#011at java.lang.Thread.run(Thread.java:636)#012#012#012Wrapped Exception:#012#012java.io.FileNotFoundException: /tmp/tomcat6-temp/upload_31503983_12d1e97e78d__8000_00004753.tmp (Too many open files)#012#011at java.io.FileInputStream.open(Native Method)#012#011at java.io.FileInputStream.<init>(FileInputStream.java:137)#012#011at org.apache.commons.fileupload.disk.DiskFileItem.getInputStream(DiskFileItem.java:230)#012#011at com.xpn.xwiki.doc.XWikiAttachmentContent.getContentInputStream(XWikiAttachmentContent.java:211)#012#011at com.xpn.xwiki.doc.XWikiAttachment.toXML(XWikiAttachment.java:409)#012#011at com.xpn.xwiki.doc.XWikiDocument.toXML(XWikiDocument.java:3900)#012#011at com.x
Dec 25 15:19:23 herald jsvc.exec[10046]: pn.xwiki.doc.XWikiDocument.toXML(XWikiDocument.java:3978)#012#011at com.xpn.xwiki.doc.XWikiDocument.addToZip(XWikiDocument.java:3665)#012#011at com.xpn.xwiki.plugin.packaging.Package.addToZip(Package.java:1038)#012#011at com.xpn.xwiki.plugin.packaging.Package.export(Package.java:341)#012#011at com.xpn.xwiki.plugin.packaging.PackageAPI.export(PackageAPI.java:226)#012#011at com.xpn.xwiki.plugin.packaging.PackageAPI.backupWiki(PackageAPI.java:298)#012#011at com.xpn.xwiki.web.ExportAction.exportXAR(ExportAction.java:262)#012#011at com.xpn.xwiki.web.ExportAction.render(ExportAction.java:60)#012#011at com.xpn.xwiki.web.XWikiAction.execute(XWikiAction.java:216)#012#011at com.xpn.xwiki.web.XWikiAction.execute(XWikiAction.java:117)#012#011at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:431)#012#011at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:236)#012#011at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1196)#012#011at org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:432)#012#011at javax.servlet.h
Dec 25 15:19:23 herald jsvc.exec[10046]: ttp.HttpServlet.service(HttpServlet.java:637)#012#011at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)#012#011at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)#012#011at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)#012#011at com.xpn.xwiki.web.ActionFilter.doFilter(ActionFilter.java:129)#012#011at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)#012#011at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)#012#011at com.xpn.xwiki.wysiwyg.server.filter.ConversionFilter.doFilter(ConversionFilter.java:152)#012#011at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)#012#011at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)#012#011at com.xpn.xwiki.plugin.webdav.XWikiDavFilter.doFilter(XWikiDavFilter.java:68)#012#011at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilt
Dec 25 15:19:23 herald jsvc.exec[10046]: erChain.java:235)#012#011at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)#012#011at org.xwiki.container.servlet.filters.internal.SavedRequestRestorerFilter.doFilter(SavedRequestRestorerFilter.java:218)#012#011at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)#012#011at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)#012#011at org.xwiki.container.servlet.filters.internal.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:112)#012#011at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)#012#011at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)#012#011at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)#012#011at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)#012#011at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)#012#011at org.apache.ca
Dec 25 15:19:23 herald jsvc.exec[10046]: talina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)#012#011at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)#012#011at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)#012#011at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856)#012#011at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)#012#011at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:454)#012#011at java.lang.Thread.run(Thread.java:636)

------end logfile ----

---- open files ----
jsvc 28191 tomcat6 357w REG 8,1 14631 655616 /tmp/tomcat6-temp/upload_474e3796_12d2399566c__8000_00002483.tmp
jsvc 28191 tomcat6 358w REG 8,1 32314 655622 /tmp/tomcat6-temp/upload_474e3796_12d2399566c__8000_00002487.tmp
jsvc 28191 tomcat6 359w REG 8,1 65685 655629 /tmp/tomcat6-temp/upload_474e3796_12d2399566c__8000_00002499.tmp
jsvc 28191 tomcat6 360w REG 8,1 1865969 655621 /tmp/tomcat6-temp/upload_474e3796_12d2399566c__8000_00002485.tmp
jsvc 28191 tomcat6 361w REG 8,1 216645 655623 /tmp/tomcat6-temp/upload_474e3796_12d2399566c__8000_00002489.tmp
jsvc 28191 tomcat6 362w REG 8,1 210714 655624 /tmp/tomcat6-temp/upload_474e3796_12d2399566c__8000_00002491.tmp
jsvc 28191 tomcat6 363w REG 8,1 174376 655625 /tmp/tomcat6-temp/upload_474e3796_12d2399566c__8000_00002493.tmp
jsvc 28191 tomcat6 364w REG 8,1 265307 655626 /tmp/tomcat6-temp/upload_474e3796_12d2399566c__8000_00002495.tmp
jsvc 28191 tomcat6 365w REG 8,1 3231201 655628 /tmp/tomcat6-temp/upload_474e3796_12d2399566c__8000_00002497.tmp
jsvc 28191 tomcat6 366w REG 8,1 65685 655631 /tmp/tomcat6-temp/upload_474e3796_12d2399566c__8000_00002501.tmp
jsvc 28191 tomcat6 367w REG 8,1 100421 655632 /tmp/tomcat6-temp/upload_474e3796_12d2399566c__8000_00002503.tmp
jsvc 28191 tomcat6 368w REG 8,1 97256 655633 /tmp/tomcat6-temp/upload_474e3796_12d2399566c__8000_00002505.tmp
jsvc 28191 tomcat6 369w REG 8,1 1804006 655634 /tmp/tomcat6-temp/upload_474e3796_12d2399566c__8000_00002507.tmp
jsvc 28191 tomcat6 372w REG 8,1 2003474 655635 /tmp/tomcat6-temp/upload_474e3796_12d2399566c__8000_00002509.tmp
jsvc 28191 tomcat6 373w REG 8,1 6041408 655637 /tmp/tomcat6-temp/upload_474e3796_12d2399566c__8000_00002511.tmp
jsvc 28191 tomcat6 374w REG 8,1 2021116 655639 /tmp/tomcat6-temp/upload_474e3796_12d2399566c__8000_00002513.tmp
jsvc 28191 tomcat6 375r REG 8,1 700446 655640 /tmp/tomcat6-temp/upload_474e3796_12d2399566c__8000_00002515.tmp
---- end open files ----