Uploaded image for project: 'XWiki Platform'
  1. XWiki Platform
  2. XWIKI-24577

text/plain REST request bodies are decoded with the platform default charset instead of UTF-8, corrupting non-ASCII content

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Major
    • 18.7.0-rc-1
    • 17.10.0
    • REST
    • None
    • Unit
    • Unknown
    • N/A
    • N/A

    Description

      Problem

      When a client sends content to XWiki through the REST API using a text/plain request body, and that content contains non-ASCII characters (accented letters such as "café", CJK text such as "日本語", the Euro sign "€", emoji, etc.), the characters can be silently corrupted (mojibake) on the server.

      This affects any REST endpoint that accepts a text/plain body, including:

      • setting a page's content or title,
      • setting an object property value,
      • posting a comment,
      • setting tags.

      User-visible symptom

      A user who, for example, uses a script or a third-party application to create or update a page via the REST API with the body café could end up with the page content stored as café (or other garbled text). The corruption is silent: no error is returned, the wrong bytes are simply stored and later displayed.

      Root cause

      The REST request-body readers for text/plain decoded the incoming byte stream using the JVM platform-default charset instead of UTF-8. Concretely, AbstractTextPlainReader.getEntityAsString called the deprecated IOUtils.toString(InputStream) overload, which uses the default charset.

      On servers whose default charset is UTF-8 the bug is invisible, which is why it went unnoticed; but on servers whose default charset is not UTF-8 (e.g. some Windows setups, or JVMs/containers started without -Dfile.encoding=UTF-8 / LANG configured), a UTF-8 request body is misinterpreted and the content is corrupted.

      Fix

      Decode text/plain REST request bodies explicitly as UTF-8, so behaviour is correct and deterministic regardless of the server's platform default charset. Unit tests were added for the whole REST request-body reader package (which previously had none), including a non-ASCII round-trip regression test.

      Attachments

        Activity

          People

            vmassol Vincent Massol
            vmassol Vincent Massol
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: