Details
-
Improvement
-
Resolution: Unresolved
-
Major
-
None
-
0.9
-
None
-
XWiki 18.6.0, ai-llm 0.9.
-
Unknown
-
Description
What happens
RAGChatRequestFilter.extractURLsAndformat walks the entire searchResults list and emits every unique document URL:
for (Context result : searchResults) { String sourceURL = result.url(); if (!addedUrls.contains(sourceURL)) { sourcesBuilder.append(String.format(SOURCE_STRING, sourceURL)); addedUrls.add(sourceURL); } }
chatWidget.js then splits that string on newlines and lists every entry. So "Sources" is the raw retrieval set with duplicate URLs collapsed – it has no relationship to which chunks the model actually drew on.
Why it matters
With the default budget the user can see up to 10 sources for a question answered entirely from one page. It reads as though the answer synthesised ten documents. That devalues the citation list – users cannot tell a precise retrieval from a noisy one, and cannot tell which link to follow to verify the answer.
Observed
A correct answer drawn from a single page was presented with 8 sources, 7 of them unrelated: 5 semantic + 5 keyword = 10 chunks, 1 removed as a duplicate content match, leaving 9 chunks over 8 distinct URLs.
Suggested fix
The prompt already numbers the chunks – DEFAULT_CHUNK_TEMPLATE emits a document element with an index attribute. So the cleanest option is to ask the model to cite those indices and render only the cited ones. Cheaper alternatives: show the similarity score next to each source, or list only sources above a relevance threshold, or visually separate "used" from "also retrieved".
Attachments
Issue Links
- is related to
-
LLMAI-174 No relevance floor for either retrieval half, and none is configurable
-
- Open
-