Uploaded image for project: 'XWiki Platform'
  1. XWiki Platform
  2. XWIKI-24997

XClass is cached as non-existing or outdated when it is saved while a document using it is being loaded (MySQL/MariaDB)

    XMLWordPrintable

Details

    • Bug
    • Resolution: Unresolved
    • Major
    • None
    • 16.10.0
    • Old Core
    • None
    • Unknown

    Description

      What happens

      When XWiki loads a document, it resolves the XClass of each of the document's objects inside the database transaction opened for that load. On MySQL and MariaDB the default transaction isolation level is REPEATABLE READ, under which every read in a transaction is answered from the snapshot taken at the transaction's first read. XWiki never configures an isolation level, so it inherits that default.

      As a result, if another thread saves an XClass after the loading transaction's snapshot was taken but before the loading thread gets around to reading that XClass, the loading thread does not see the save:

      • if the XClass was just created, the loading thread sees it as non-existing;
      • if the XClass was modified, the loading thread sees the previous version, without the new properties.

      XWiki.getXClass() is getDocument(classReference).getXClass(), so what gets stored in the document cache is the class document itself – with no expiry. From that moment on, every reader of that class on that node gets the empty or outdated definition (the class editor and XAR export included) even though the class is perfectly correct in the database, until the cache entry is invalidated by a new save of the class document, or the wiki is restarted.

      Object values are not affected: object properties are loaded from their own rows, independently of the class definition. The damage is limited to everything that needs the definition rather than the data.

      The window is the duration of a document load, so it is small (a few milliseconds) for a small document but grows with the number of distinct XClasses the document's objects use. It is hit most often when classes are written while the wiki is serving requests: extension installation or upgrade, subwiki creation with extensions, the Distribution Wizard, or an administrator editing a class on a live wiki.

      This is not XWIKI-24673 (a different, already-fixed cache race); it reproduces on versions that contain that fix.

      Symptoms

      The failure is silent – no error, no log entry. What breaks is everything that needs the class definition:

      • $doc.display('field') returns an empty string, so sheets and administration sections render labels with no value and forms with no input fields.
      • Anything iterating the class properties produces nothing: edit mode shows no fields, Live Data and class sheet columns configured from the class are missing, $xwiki.getDocument('Space.MyClass').getxWikiClass().propertyList is empty.
      • Creating new objects of the class fails, or creates an object without properties. For a class seen as non-existing, $xwiki.exists('Space.MyClass') returns false.
      • The class editor itself shows the class without the property that was just added, so the save looks lost.
      • XAR export and backups are affected: the export reads the document through the same cache, so an export taken while a node is poisoned silently contains the outdated class definition – and restoring it later makes the loss permanent.
      • The Solr index misses the class document (when the issue occurs while the XClass is created) or contains the old version of the class document, XObject indexing can also be affected due to missing field metadata.
      • Everything is correct again after restarting the wiki (or after re-saving the class document) – this is the strongest indicator of this bug.
      • In a cluster, only the node that cached the stale definition is affected, so the same class can look correct on one node and broken on another.
      • Happens on MySQL and MariaDB and not on PostgreSQL, Oracle or HSQLDB with their default settings – those default to READ COMMITTED.

      What keeps working, so no data is lost in the database: reading values by name ($obj.getValue('field'), getStringValue, ...), and everything built on that – rights objects still apply, notification filters still filter.

      To confirm that the class is intact in the database while the wiki reports otherwise (the class definition is stored in the document row, not in a separate table):

      SELECT XWD_CLASS_XML FROM xwikidoc WHERE XWD_FULLNAME = 'XWiki.XWikiUsers';
      

      Steps to reproduce

      Preconditions: a wiki running on MySQL or MariaDB with the default configuration. Verify with:

      SELECT @@transaction_isolation;
      

      It reports REPEATABLE-READ.

      Variant A – stale class (most realistic)

      1. Log in as administrator, open the class editor of XWiki.XWikiUsers, add a TextArea property myField and save.
      2. While that save is being committed, another request has to load a document holding an XWiki.XWikiUsers object. On an active wiki this needs no setup at all: every request of a logged-in user loads that user's own profile document (for the avatar and the display name), so any concurrent activity by any logged-in user can trigger it.
      3. Reopen the class editor of XWiki.XWikiUsers, or add myField to a section in the User Profile administration section.
      4. Expected: myField is part of the class and can be added to a profile section. Actual: the property is gone from the class editor and cannot be configured or displayed – for every user, in every session, until the wiki is restarted or XWiki.XWikiUsers is saved again. The SQL query above shows that the property is still in the database.

      Variant B – missing class (this is what fails on CI)

      1. Install an extension that ships XClasses on a wiki that is serving requests (or create a subwiki with extensions installed into it).
      2. Load a page that uses one of the newly installed classes.
      3. Expected: the page shows the class's fields. Actual: the page renders as if the class did not exist – an administration section with labels but no fields, $xwiki.exists returning false for the class – while SELECT XWD_CLASS_XML FROM xwikidoc WHERE XWD_FULLNAME = '<class>' shows the definition is there.

      Reliable reproduction

      Both variants depend on hitting a window of a few milliseconds, so they may need several attempts. To make it reliable, widen the window so the class is read late in a long load:

      1. Create N (~400) classes Sandbox.Class1 ... Sandbox.ClassN, each with one String property, plus Sandbox.VictimClass with one String property.
      2. Create Sandbox.Probe with one object of each of those classes, and give the Sandbox.VictimClass object the highest object number – objects are loaded ordered by number, so this makes the victim class the last one resolved.
      3. Delete Sandbox.VictimClass and restart the wiki (to empty the document cache).
      4. Load Sandbox.Probe (this takes ~200 ms with 400 classes) and, ~150 ms into that load, re-create Sandbox.VictimClass with its property from another thread.
      5. Afterwards, Sandbox.VictimClass is reported as non-existing for the rest of the wiki's lifetime, although its row is in the database.

      In an automated test, I was able to reproduce this scenario. On my setup it poisons the cache in 12 of 20 rounds on MySQL at the default isolation level, and passes when the server is started with --transaction-isolation=READ-COMMITTED. Unfortunately, on CI, this testing idea didn't work out, the test passed. The most likely explanation is that the Solr indexer and other async background tasks re-load the XClass into the cache before the test accesses it. As a more reliable reproduction, the test that is part of the fix thus creates its own database transaction performs a read query to trigger the repeated read isolation, then writes the XClass in another thread, waits for the background tasks, clears the cache and then loads the document.

      Workaround

      Save the affected class document again (this invalidates its cache entry), or restart the wiki. To avoid it entirely, add to WEB-INF/hibernate.cfg.xml:

      <property name="hibernate.connection.isolation">2</property>
      

      Fix

      Default to hibernate.connection.isolation to 2 (READ COMMITTED) when the configuration doesn't set one, which is the level the other supported databases already use by default. Administrators who explicitly configured an isolation level keep theirs.

      Upgrade note: InnoDB refuses to run at READ COMMITTED when binary logging is enabled with binlog_format=STATEMENT (ERROR 1665 on every write). MySQL 5.7+ defaults to ROW and MariaDB to MIXED, both of which are fine; installations that explicitly set statement-based logging must change it or configure a different isolation level.

      Note: This issue was investigated and mostly written by Claude Code with Opus 5.

      Attachments

        Issue Links

          Activity

            People

              MichaelHamann Michael Hamann
              MichaelHamann Michael Hamann
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated: