Uploaded image for project: 'XWiki Rendering'
  1. XWiki Rendering
  2. XRENDERING-816

Free-standing references that the parser cannot read back are still rendered free-standing in XWiki Syntax 2.0/2.1

    XMLWordPrintable

Details

    • Unknown

    Description

      The XWiki Syntax 2.0/2.1 renderer prints a free-standing reference (a LinkBlock or
      ImageBlock with freestanding = true) exactly as the resource reference serializer
      returns it, without any escaping and without checking that the parser can read it
      back. Whenever the serialized form is not a valid free-standing reference token, the
      rendered content is silently corrupted: the reference is truncated, changes its type,
      or the link disappears altogether.

      The free-standing tokens are XWIKI_URI, IMAGE and ATTACH in
      xwiki-rendering-wikimodel/src/main/javacc/XWiki20Scanner.jj (the token section of
      XWiki21Scanner.jj is identical). XWIKI_URI is RFC 3986 restricted to ASCII and with
      "," explicitly removed from URI_SUB_DELIMS, so a large part of what a real URL can
      contain terminates the token.

      Rendering each of the following references free-standing to xwiki/2.1 and parsing the
      result again gives (measured on master):

        reference (untyped url)          rendered   parsed back
        http://example.com/a,b           as-is      http://example.com/a
        http://exämple.com/a             as-is      http://ex
        http://example.com/a b           as-is      http://example.com/a
        http://example.com/a}b           as-is      http://example.com/a
        (same for < " | \ ^ ` [ ] and \{ )
        mailto:a,b@example.com           as-is      no link at all
        mailto:a@example.com             as-is      typed mailto reference (type changed)
      
        reference (typed)                rendered            parsed back
        doc:Space.Page                   doc:Space.Page      no link at all
        path:path/to/file                path:path/to/file   no link at all
        attach:foo.png                   attach:foo.png      untyped url reference
      
        free-standing images are broken for every reference, the renderer emits
        image: plus an already type-prefixed reference:
        attach:foo.png                   image:attach:foo.png   untyped url attach:foo.png
      

      References produced by the XWiki syntax parser itself are always untyped url or
      mailto references and do round-trip, so the problem is not reachable by parsing and
      re-rendering plain wiki syntax. It is reachable wherever free-standing references come
      from somewhere else: the xhtml/1.0 and annotated XHTML parsers (WYSIWYG editor save,
      office/HTML import) mark a link free-standing when the label equals the reference,
      other syntaxes do the same for autolinks, and macros can build such blocks directly.
      A URL containing a comma or a non-ASCII character is entirely ordinary there.

      Proposed fix: before printing a reference free-standing, verify that the serialized
      form is read back as the very same reference, and fall back to the full [[...]] syntax
      otherwise. Two cheap checks are enough, no real parse is needed:

      1. Lexical: the whole serialized string must be one free-standing reference token.
        XWIKI_URI / IMAGE / ATTACH are plain regular definitions that transliterate into
        a java.util.regex.Pattern of about 20 lines, shared by both syntaxes. The existing
        conditions 3 and 4 of XWikiSyntaxResourceRenderer#forceFullSyntax already guarantee
        white space or the end of the block on both sides, so "the whole string is one
        token" is exactly the right condition.
      2. Semantic: parsing that string with the matching ResourceReferenceParser
        (xwiki/2.0/link, xwiki/2.1/image, ...) must give back an equal ResourceReference.
        This catches the type losses (doc:, attach:, path:, typed images) and needs only
        the parser component to be injected next to the serializer that the renderer
        already gets.

      Refinement worth having: only fall back when the fallback actually helps. An untyped
      url reference "mailto:a@b.com" comes back as a typed mailto reference both free-standing
      and as [[mailto:a@b.com]], so forcing the full syntax there would only make the output
      noisier without improving fidelity.

      Note that the full syntax does not preserve the free-standing flag itself - [[url]]
      parses back as a non-free-standing link - so the XDOM round-trip stays lossy in that
      one respect, but the reference is preserved.

      The risk of the pattern drifting away from the grammar is covered by making the
      round-trip the test: XWikiSyntaxFreeStandingReferenceTest already renders and re-parses,
      so a parameterized matrix (every ASCII character in scheme/host/path/query/fragment
      position, non-ASCII, every resource type, links and images) validates the feature
      against the real parser rather than against the pattern.

      Related: XRENDERING-815 made the renderer fall back to the full syntax when the
      reference would put a "{{" into the output. That is a macro-safety guarantee only and
      deliberately does not address the round-trip problem described here.

      The free-standing image output image:attach:foo.png looks like a bug of its own — the image reference serializer emits a type prefix that the image token then feeds back as part of an untyped URL.

      Note: this issue description was produced by Claude Code with Opus 5, I have not verified every aspect of it but it seems very plausible and Claude Code did run verification tests to establish the findings in the table above. To be done: test if this can easily be reproduced in the WYSIWYG editor and provide actual reproduction steps if possible.

      Attachments

        Activity

          People

            Unassigned Unassigned
            MichaelHamann Michael Hamann
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: