Details
-
Bug
-
Resolution: Unresolved
-
Major
-
None
-
12.5
-
None
-
Unknown
-
Description
Symptom
When XWiki stops while it uses a remote Solr server (solr.type=remote), the log can contain an error like the following, and the events that were still waiting in the queue of the asynchronous event store at that moment are not saved: the activity stream and the notifications miss the last actions done before the restart.
ERROR o.x.e.s.s.i.SolrEventStore - Failed to commit ... Caused by: java.lang.IllegalStateException: Connection pool shut down
Cause
- The Solr event store writes asynchronously. When it is disposed, it processes the queued tasks and commits, using the clients of the remote Solr cores, which RemoteSolr closes when it is disposed.
- RemoteSolr has the default dispose priority (the embedded Solr is disposed last, with priority 10000), and nothing orders it after the event store: the event store depends on DefaultSolr, which looks up RemoteSolr at runtime. So the Solr clients can be closed before the event store has flushed its queue.
- In addition, once the store is disposed its processing thread stops after the batch it is working on: the tasks queued beyond that batch are dropped and their futures never complete. New events are still accepted after the store was disposed, and can even block the caller when the queue is full.
Fix
- RemoteSolr is disposed with priority 10000, like the embedded Solr: after the components with the default priority, the event store included, and after the remote observation channels (priority 9000 proposed in XWIKI-25248).
- When the event store is disposed it stops accepting new events (the returned future fails with an EventStreamException), processes every task queued before (within the existing 10 seconds limit), logs a warning when it could not, and fails the tasks it could not process so that nobody waits for them forever.
Attachments
Issue Links
- relates to
-
XWIKI-25248 Stop the remote observation channels last when XWiki stops
-
- Open
-