Details
-
Improvement
-
Resolution: Unresolved
-
Major
-
None
-
0.9
-
None
-
XWiki 18.6.0, ai-llm 0.9.
-
Unknown
-
Description
What happens
XWikiDocumentDocument.getFormattedAttachments() runs every attachment of every indexed document through Tika and appends the extracted text to the content that gets chunked and embedded:
String content = this.tika.parseToString(attachment.getContentInputStream(context)); if (StringUtils.isNotBlank(content)) { return Stream.of(formatHeading(2, attachment.getFilename()) + content); }
There is no MIME allow-list, no size gate, and no configuration option anywhere to turn attachment indexing off. Tika.parseToString applies its default 100k-character write limit, which bounds how much of any single attachment is taken but does not filter which attachments are taken at all.
Why it matters
An administrator who points a collection at a space gets that space's attachments indexed whether or not that is wanted. There is no way to exclude binary formats, scanned documents, exports or archives whose extracted text is noise, and no way to see how much of the index attachments account for. Extracted attachment text also competes for the same result slots as page content, and inherits the chunking behaviour of the page it hangs off.
For large wikis this is also an indexing-cost issue: every attachment is parsed on every re-index.
Suggested fix
Add per-collection options: enable or disable attachment indexing, a MIME type allow-list, and a maximum attachment size. Defaulting to current behaviour keeps existing installs unchanged.
Filed separately from the raw-wiki-syntax issue in the same method, as requested during triage of that report.
Attachments
Issue Links
- is related to
-
LLMAI-176 Raw wiki syntax is fed to the embedder instead of rendered plain text
-
- Open
-