Uploaded image for project: 'LLM AI Integration'
  1. LLM AI Integration
  2. LLMAI-170

Chat fails on models that reject an explicit temperature

    XMLWordPrintable

Details

    • Bug
    • Resolution: Unresolved
    • Major
    • None
    • 0.9
    • None
    • Unknown

    Description

      Problem

      Any OpenAI reasoning model (GPT-5 family, o1/o3/o4) rejects an explicit
      temperature other than the default. The chat UI always sends one, so every
      request against such a model fails with:

      Unsupported value: 'temperature' does not support 0 with this model.
      Only the default (1) value is supported.
      

      Non-reasoning models such as gpt-4o-mini are unaffected, so the extension
      works until an administrator configures a current OpenAI model — at which point
      that model is unusable with no way to fix it from the configuration UI.

      Root cause

      The temperature is decided entirely by the client and defaults to 0:

      • AI/Code/Javascript.xml initialises {{promptData = { prompt: '', temperature: 0 }

        }} and, when the Advanced Settings panel was never opened,
        sends that 0 verbatim via getRequestParams().

      • application-ai-llm-chat-webjar/webjar/chatWidget.js likewise defaults
        userSettings.temperature to 0.

      Server-side the value is passed straight through: ChatCompletionsResource
      -> ChatCompletionRequest -> OpenAIChatModel.setModel() (which only
      overrides model and stream) -> RequestHelper.

      Why it cannot be worked around in configuration

      AI.Models.Code.ModelsClass — the class behind the model configuration pages
      — has no temperature-related field. Its properties are: allowGuests,
      contextSize, dimensions, embeddingIndexPrefix, embeddingQueryPrefix, groups,
      maximumParallelism, model, serverName, type. ModelConfiguration.java
      matches. Per LLMAI-16 temperature was deliberately moved into the prompt
      database only, which predates models that reject the parameter outright.

      The only workarounds available to an administrator today are global, not
      per-model:

      1. Set the Advanced Settings slider to 1 (client-side, not persistent).
      2. Create a PromptDB entry with Temperature 1 and tick "Default" — this then
        applies to every model, including those that would benefit from 0.

      Suggested fix

      Add a capability flag to the model configuration (e.g. supportsTemperature,
      default true) and have OpenAIChatModel.setModel() null the parameter out
      when it is false. RequestHelper already serialises with
      JsonInclude.Include.NON_NULL, so a null temperature is omitted from the
      outgoing JSON and the provider applies its own default — the plumbing for this
      already works end to end, only the flag is missing.

      The same restriction applies to top_p on these models, so a small set of
      per-model capability flags would be more durable than special-casing
      temperature alone. A hardcoded model-name list would go stale quickly and is
      best avoided, since the extension also targets non-OpenAI providers behind
      OpenAI-compatible endpoints.

      Steps to reproduce

      1. Configure an OpenAI server and a model entry pointing at a GPT-5 / o-series
        model.
      2. Open the AI chat page, select that model, send any message without touching
        Advanced Settings.
      3. The request fails with the error above. The same prompt against gpt-4o-mini
        succeeds.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              sebels Sebastian Elsner
              Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

                Created:
                Updated: