Skip to content

Fix OpenAI chat tool_call id handling in streaming #243

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ericdallo merged 1 commit into from
Dec 21, 2025
Merged

Fix OpenAI chat tool_call id handling in streaming #243

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# Changelog

## Unreleased
- Fix openai-chat tool call + support for Mistral API #233

## 0.87.1

Expand Down
24 changes: 17 additions & 7 deletions src/eca/features/chat.clj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1021,21 +1021,31 @@
(run-pre-request-hooks! (assoc chat-ctx :message original-text))]
(cond
stop? (do (finish-chat-prompt! :idle chat-ctx) nil)
:else (let [last-user-idx (or (llm-util/find-last-user-msg-idx user-messages)
(dec (count user-messages)))
rewritten (if (and modify-allowed?
last-user-idx
final-prompt)
:else (let [last-user-idx (llm-util/find-last-user-msg-idx user-messages)
;; preRequest additionalContext should ideally attach to the last user message,
;; but some prompt sources may not contain a user role (e.g. prompt templates).
context-idx (or last-user-idx
(some-> user-messages seq count dec))
rewritten (if (and modify-allowed? last-user-idx final-prompt)
(assoc-in user-messages [last-user-idx :content 0 :text] final-prompt)
user-messages)
with-contexts (if (seq additional-contexts)
with-contexts (cond
(and (seq additional-contexts) context-idx)
(reduce (fn [msgs {:keys [hook-name content]}]
(update-in msgs [last-user-idx :content]
(update-in msgs [context-idx :content]
#(conj (vec %)
{:type :text
:text (wrap-additional-context hook-name content)})))
rewritten
additional-contexts)

(seq additional-contexts)
(do (logger/warn logger-tag "Dropping preRequest additionalContext because no message index was found"
{:source-type source-type
:num-messages (count user-messages)})
rewritten)

:else
rewritten)]
with-contexts)))
user-messages)]
Expand Down
1 change: 0 additions & 1 deletion src/eca/features/prompt.clj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,6 @@
"")))
""
refined-contexts)
;; TODO - should be refined contexts?
(when startup-ctx
(str "\n<additionalContext from=\"chatStart\">\n" startup-ctx "\n</additionalContext>\n\n"))
"</contexts>"))
Expand Down
64 changes: 37 additions & 27 deletions src/eca/llm_providers/openai_chat.clj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -396,9 +396,8 @@

;; Atom to accumulate tool call data from streaming chunks.
;; OpenAI streams tool call arguments across multiple chunks, so we need to
;; accumulate the partial JSON strings before parsing them. Keys are either
;; index numbers for simple cases, or "index-id" composite keys for parallel
;; tool calls that share the same index but have different IDs.
;; accumulate partial JSON strings before parsing them. Keys are tool call
;; indices (fallback: IDs) to keep chunks grouped for the active response.
tool-calls* (atom {})

;; Reasoning state machine:
Expand Down Expand Up @@ -431,6 +430,12 @@
:content ""
:buffer "")
(on-reason {:status :started :id new-reason-id})))
find-existing-tool-key (fn [tool-calls index id]
(some (fn [[k v]] (when (or (some-> id (= (:id v)))
(and (nil? (:id v))
(some-> index (= (:index v)))))
k))
tool-calls))
on-tools-called-wrapper (fn on-tools-called-wrapper [tools-to-call on-tools-called handle-response]
(when-let [{:keys [new-messages]} (on-tools-called tools-to-call)]
(let [pruned-messages (prune-history new-messages)
Expand All @@ -449,9 +454,9 @@
:api-key api-key
:url-relative-path url-relative-path
:on-error wrapped-on-error
:on-stream (when stream? (fn [event data] (handle-response event data tool-calls* new-rid)))}))))
:on-stream (when stream? (fn [event data] (handle-response event data tool-calls*)))}))))

handle-response (fn handle-response [event data tool-calls* rid]
handle-response (fn handle-response [event data tool-calls*]
(if (= event "stream-end")
(do
;; Flush any leftover buffered content and finish reasoning if needed
Expand Down Expand Up @@ -496,27 +501,32 @@
{name :name args :arguments} function
;; Extract Google Gemini thought signature if present
thought-signature (get-in extra_content [:google :thought_signature])
;; Use RID as key to avoid collisions between API requests
tool-key (str rid "-" index)
;; Create globally unique tool call ID for client
unique-id (when id (str rid "-" id))]
(when (and name unique-id)
(on-prepare-tool-call {:id unique-id
:full-name name
:arguments-text ""}))
(swap! tool-calls* update tool-key
(fn [existing]
(cond-> (or existing {:index index})
unique-id (assoc :id unique-id)
name (assoc :full-name name)
args (update :arguments-text (fnil str "") args)
;; Store thought signature for Google Gemini
thought-signature (assoc :external-id thought-signature))))
(when-let [updated-tool-call (get @tool-calls* tool-key)]
(when (and (:id updated-tool-call)
(:full-name updated-tool-call)
args)
(on-prepare-tool-call (assoc updated-tool-call :arguments-text args)))))))
existing-key (find-existing-tool-key @tool-calls* index id)
existing (when existing-key (get @tool-calls* existing-key))
tool-key (or existing-key index id)]
(if (nil? tool-key)
(logger/warn logger-tag "Received tool_call delta without index/id; ignoring"
{:tool-call tool-call})
(do
(swap! tool-calls* update tool-key
(fn [existing]
(cond-> (or existing {:index index})
(some? index) (assoc :index index)
(and id (nil? (:id existing))) (assoc :id id)
(and name (nil? (:full-name existing))) (assoc :full-name name)
args (update :arguments-text (fnil str "") args)
;; Store thought signature for Google Gemini
thought-signature (assoc :external-id thought-signature))))
(when-let [updated-tool-call (get @tool-calls* tool-key)]
;; Streaming tool_calls may split metadata (id/name) and arguments across deltas.
;; Emit prepare once we can correlate the call (id + full-name), on first id or args deltas,
;; so :tool-prepare always precedes :tool-run in the tool-call state machine.
(when (and (:id updated-tool-call)
(:full-name updated-tool-call)
(or (nil? (:id existing)) args))
(on-prepare-tool-call
(assoc updated-tool-call
:arguments-text (or args ""))))))))))
;; Process finish reason if present (but not tool_calls which is handled above)
(when finish-reason
;; Flush any leftover buffered content before finishing
Expand All @@ -541,4 +551,4 @@
:on-tools-called-wrapper on-tools-called-wrapper
:on-error wrapped-on-error
:on-stream (when stream?
(fn [event data] (handle-response event data tool-calls* rid)))})))
(fn [event data] (handle-response event data tool-calls*)))})))
Loading