Details
-
Bug
-
Resolution: Unresolved
-
Major
-
None
-
16.10.0
-
None
-
Unknown
-
Description
The XWiki Syntax 2.0/2.1 renderer prints a free-standing reference (a LinkBlock or
ImageBlock with freestanding = true) exactly as the resource reference serializer
returns it, without any escaping and without checking that the parser can read it
back. Whenever the serialized form is not a valid free-standing reference token, the
rendered content is silently corrupted: the reference is truncated, changes its type,
or the link disappears altogether.
The free-standing tokens are XWIKI_URI, IMAGE and ATTACH in
xwiki-rendering-wikimodel/src/main/javacc/XWiki20Scanner.jj (the token section of
XWiki21Scanner.jj is identical). XWIKI_URI is RFC 3986 restricted to ASCII and with
"," explicitly removed from URI_SUB_DELIMS, so a large part of what a real URL can
contain terminates the token.
Rendering each of the following references free-standing to xwiki/2.1 and parsing the
result again gives (measured on master):
reference (untyped url) rendered parsed back
http://example.com/a,b as-is http://example.com/a
http://exämple.com/a as-is http://ex
http://example.com/a b as-is http://example.com/a
http://example.com/a}b as-is http://example.com/a
(same for < " | \ ^ ` [ ] and \{ )
mailto:a,b@example.com as-is no link at all
mailto:a@example.com as-is typed mailto reference (type changed)
reference (typed) rendered parsed back
doc:Space.Page doc:Space.Page no link at all
path:path/to/file path:path/to/file no link at all
attach:foo.png attach:foo.png untyped url reference
free-standing images are broken for every reference, the renderer emits
image: plus an already type-prefixed reference:
attach:foo.png image:attach:foo.png untyped url attach:foo.png
References produced by the XWiki syntax parser itself are always untyped url or
mailto references and do round-trip, so the problem is not reachable by parsing and
re-rendering plain wiki syntax. It is reachable wherever free-standing references come
from somewhere else: the xhtml/1.0 and annotated XHTML parsers (WYSIWYG editor save,
office/HTML import) mark a link free-standing when the label equals the reference,
other syntaxes do the same for autolinks, and macros can build such blocks directly.
A URL containing a comma or a non-ASCII character is entirely ordinary there.
Proposed fix: before printing a reference free-standing, verify that the serialized
form is read back as the very same reference, and fall back to the full [[...]] syntax
otherwise. Two cheap checks are enough, no real parse is needed:
- Lexical: the whole serialized string must be one free-standing reference token.
XWIKI_URI / IMAGE / ATTACH are plain regular definitions that transliterate into
a java.util.regex.Pattern of about 20 lines, shared by both syntaxes. The existing
conditions 3 and 4 of XWikiSyntaxResourceRenderer#forceFullSyntax already guarantee
white space or the end of the block on both sides, so "the whole string is one
token" is exactly the right condition. - Semantic: parsing that string with the matching ResourceReferenceParser
(xwiki/2.0/link, xwiki/2.1/image, ...) must give back an equal ResourceReference.
This catches the type losses (doc:, attach:, path:, typed images) and needs only
the parser component to be injected next to the serializer that the renderer
already gets.
Refinement worth having: only fall back when the fallback actually helps. An untyped
url reference "mailto:a@b.com" comes back as a typed mailto reference both free-standing
and as [[mailto:a@b.com]], so forcing the full syntax there would only make the output
noisier without improving fidelity.
Note that the full syntax does not preserve the free-standing flag itself - [[url]]
parses back as a non-free-standing link - so the XDOM round-trip stays lossy in that
one respect, but the reference is preserved.
The risk of the pattern drifting away from the grammar is covered by making the
round-trip the test: XWikiSyntaxFreeStandingReferenceTest already renders and re-parses,
so a parameterized matrix (every ASCII character in scheme/host/path/query/fragment
position, non-ASCII, every resource type, links and images) validates the feature
against the real parser rather than against the pattern.
Related: XRENDERING-815 made the renderer fall back to the full syntax when the
reference would put a "{{" into the output. That is a macro-safety guarantee only and
deliberately does not address the round-trip problem described here.
The free-standing image output image:attach:foo.png looks like a bug of its own — the image reference serializer emits a type prefix that the image token then feeds back as part of an untyped URL.
Note: this issue description was produced by Claude Code with Opus 5, I have not verified every aspect of it but it seems very plausible and Claude Code did run verification tests to establish the findings in the table above. To be done: test if this can easily be reproduced in the WYSIWYG editor and provide actual reproduction steps if possible.