Details
-
New Feature
-
Resolution: Unresolved
-
Major
-
None
-
18.7.0
-
Unknown
-
Description
Context
This implements the proposal accepted on
Change the code macro implementation
(+1 from mleduc, tmortagne, mflorea and Wardenburg), whose conclusion was:
1) We agreed to move to a javascript-based syntax highlighting in XS
2) We agreed to keep the code macro and introduce an Admin UI to configure which implementation
to use and by default to use the javascript-based one. And once done and working well, to move the
pygments-based one to contrib.
The reasons recorded there:
- Performance: the current highlighter is written in Python and executed through Jython. It is very
slow on first use, while Jython loads, and still slow afterwards, especially on large content. - Missing features: line numbering, a button to copy the content (XWIKI-21364), and others, which
are far easier to provide client side. - The historical reason for keeping the highlighting server side – the PDF exporter not supporting
JavaScript – no longer applies, since the current PDF exporter does support it.
A client-side implementation already exists in contrib and works:
Prism Macro, along with
org.xwiki.contrib.prismjs:prismjs-code which substitutes it for the code macro
(https://github.com/xwiki-contrib/prismjs/tree/master/prismjs-code). What is left is doing it
properly in XWiki Standard.
Current state in the code
Most of the pluggability is already there:
- CodeMacro resolves a HighlightParser component by language hint, and otherwise falls back
to componentManager.getInstance(HighlightParser.class, "default") – the fallback hint is
hardcoded. - PygmentsParser is declared as {{@Component(roles =
{HighlightParser.class}
)}} with no
@Named, so it is that default hint. - HighlightParser#highlight(String syntaxId, Reader source) receives only the language and the
content: not the macro parameters, and not whether the call is inline. - xwiki-platform-rendering-macro-code declares jython-slim and pygments as direct
dependencies, so the generic macro and the Pygments implementation ship in the same module. - The source parameter needs no work: the generic part of the macro already resolves it and hands
the resulting content to the HighlightParser.
Proposal
- Replace the hardcoded "default" fallback hint by a configured one, and add the Admin UI to
choose among the HighlightParser implementations present in the instance. Per-language
HighlightParser components keep taking precedence, as today. - Extract the Pygments implementation into its own module, under the hint pygments, so that
xwiki-platform-rendering-macro-code stops depending on Jython and Pygments. - Add a client-side HighlightParser implementation to XS, based on Prism.js, and make it the
configured default. - Once that is stable, move the Pygments module to contrib.
Also to be decided while doing 1: whether HighlightParser#highlight needs the macro parameters
and an inline flag, which a client-side implementation may need and the current signature cannot
carry.
Backward compatibility
The risk identified on the thread is language coverage, not architecture:
- Pygments supports far more languages than the JavaScript frameworks. The two that matter to XWiki
are Velocity – supported by Pygments only because the project asked for it – and Groovy. Prism.js
supports both, which is a concrete argument for Prism over highlight.js, which does not. - Language identifiers do not map one to one. Pygments treats bash, sh, ksh, zsh and
shell as synonyms; Prism does not, and zsh would need mapping onto shell. The agreed
approach is to handle such cases individually inside the Prism implementation rather than build a
generic mapping layer.
Existing content using the code macro must keep being highlighted after the switch.
Consequences for other issues
- XWIKI-21364 (copy button) and XWIKI-22832 (line numbers) become much cheaper client side.
- XWIKI-8456 (cache the highlighting result for performance) largely dissolves.
- XWIKI-17107 (upgrade Pygments) and XWIKI-18369 (Pygments lexer patch) become contrib concerns once
the Pygments implementation moves out.