Details
-
Task
-
Resolution: Unresolved
-
Major
-
None
-
0.9
-
None
-
XWiki 18.6.0, ai-llm 0.9.
-
Unknown
-
Description
What happens
SolrConnector.getSolrDocument writes the chunk text into two fields:
solrDocument.addField(AiLLMSolrCoreInitializer.FIELD_CONTENT, chunk.getContent()); solrDocument.addField(AiLLMSolrCoreInitializer.FIELD_CONTENT_INDEX, chunk.getContent());
Both fields are declared as text_general: content via addTextGeneralField(FIELD_CONTENT, false, false) in createSchema, and content_index via setTextGeneralField(FIELD_CONTENT_INDEX, false, false) in the migration to schema version 121000006. The migration's comment reads:
// Add another version of the text field, but indexed as regular text.
which suggests the intent was that content was not analysed as text. On a fresh install it is, so the two fields carry identical analysis of identical text.
Why it matters
No correctness impact – purely index size and indexing cost. The chunk text, which is the bulk of the index, is analysed and stored twice. It is also a small maintenance trap: a future reader of createSchema could reasonably conclude the two fields differ in analysis when they do not.
Suggested fix
Either drop content_index and search content directly, or make the two fields actually differ as the comment implies – e.g. content stored and not indexed, content_index indexed and not stored. Either way a schema migration and re-index would be needed, so this may be worth folding into the next change that already requires one.
Noticed while investigating retrieval relevance in 0.9; filing separately as it is cosmetic relative to the other reports.