CollaborationIT#restrictScriptMacroExecution is flickering

The functional test org.xwiki.blocknote.test.ui.AllIT$NestedCollaborationIT#restrictScriptMacroExecution fails intermittently on ci.xwiki.org.

Affected tests

• org.xwiki.blocknote.test.ui.AllIT$NestedCollaborationIT#restrictScriptMacroExecution

Branches: master, stable-18.7.x, stable-18.4.x.

Frequency

Develocity, 28 days to 2026-09-20: 5 failures in 364 executions (1.4%), in 4 of 21 configurations. First failure 2026-08-27, most recent 2026-09-20.

Every failure is on Chrome; for F1 that concentration is significant (32% of runs, p=0.01). The test passes on Firefox in 202 executions across 6 configurations.

F1 - the peer's edit never arrives after rejoining (4 of 5 failures)

org.opentest4j.AssertionFailedError: Unexpected content: one two three four
Failed to execute the [velocity] macro. Cause: [The execution of the [velocity] script macro is not allowed. Check the rights of its last author or the parameters if it's rendered from another script.]. Click on this message for details. ==> expected: <true> but was: <false>
	at org.xwiki.blocknote.test.po.BlockNoteRichTextArea.waitUntilTextContains(BlockNoteRichTextArea.java:273)
	at org.xwiki.blocknote.test.ui.CollaborationIT.restrictScriptMacroExecution(CollaborationIT.java:652)

F2 - same wait, different stale content (1 of 5 failures, 2026-09-04)

org.opentest4j.AssertionFailedError: Unexpected content: one two three four
Failed to render macro: Nested macros are not supported yet ==> expected: <true> but was: <false>
	at org.xwiki.blocknote.test.po.BlockNoteRichTextArea.waitUntilTextContains(BlockNoteRichTextArea.java:228)
	at org.xwiki.blocknote.test.ui.CollaborationIT.restrictScriptMacroExecution(CollaborationIT.java:659)

Analysis

Line 652 is firstTextArea.waitUntilTextContains("five"), immediately after joinSession(firstEditPage, setup). The content the assertion dumps is one two three four plus the unexecuted-macro placeholder, which is the document as it stood before the second tab typed five and clicked Save & Continue. So the first tab rejoins the collaboration session and never receives the peer's pending change; the wait then times out on the pre-rejoin state.

Note that the The execution of the [velocity] script macro is not allowed text in the failure message is part of that stale content and is the expected state at that point of the test. It is not what failed - reading it as a rights problem sends you down the wrong path.

The fix should stay inside the page object or the session-rejoin helper - not an assertion, an expected value or a timeout change.

Verifying a fix

At the 6.4% failure rate across the affected configurations, about 46 consecutive clean runs are needed before the failure can be called gone; the 1 clean run since the last failure is worth p=0.94, i.e. no evidence either way. xwiki/scripts/xwiki-it-repeat.mjs on chrome / mysql latest / tomcat 11-jdk25 is the configuration to reproduce on.

References

• Build scans of the most recent failures: https://community.develocity.cloud/s/nk6emdybvtfkw and https://community.develocity.cloud/s/fsxpl2rmagux4
• CI build: https://ci.xwiki.org/job/XWiki%20Environment%20Tests/job/xwiki-platform/job/master/1379
• XWIKI-23308 tracks the same test method in the predecessor editor (RealtimeWYSIWYGEditorIT#restrictScriptMacroExecution, package org.xwiki.realtime.wysiwyg) - a different class, so not a duplicate, but possibly the same underlying cause.

Filed automatically by the daily CI check.