Route reasoning models to reasoning-aware provider clients so reasoning_content round-trips #101

New issue

Closed

opened 2026-08-04 10:00:15 +00:00 by CoreRasurae · 0 comments

CoreRasurae commented

2026-08-04 10:00:15 +00:00

Member

Metadata

Commit Message: feat(llm): route reasoning models to reasoning-aware provider clients
Branch: feature/m2-reasoning-model-provider-support

Background and context

The Actor Configuration Standard (§1.2) deliberately excludes "the internal
mechanics of LLM providers" from its scope, and ADR-2028 (Extended Provider
Routing) routes every non-native provider — including openai_compatible and
every named additional provider — through a bare
langchain_openai.ChatOpenAI(base_url=..., api_key=...) client
(cleveractors.agents.llm_client._build_from_credentials, at commit 882336c).

langchain-openai (currently 1.4.1) explicitly targets the official OpenAI
API specification only. Its own module docstring states that non-standard
response fields added by third-party providers — specifically
reasoning_content — are not extracted or preserved, and recommends a
provider-specific subclass (e.g. openai_compatible_reasoning) when pointing base_url at a
reasoning-capable provider.

As a result, when an llm agent is pointed at a reasoning / "thinking"
model through an OpenAI-compatible endpoint, the model's reasoning_content
is silently dropped on both the response leg (_convert_dict_to_message) and
the request leg (_convert_message_to_dict, which only ever emits
role/content/tool_calls). The multi-turn tool-call loop
(cleveractors.agents.llm.LLMAgent._execute_tool_loop) then replays the
returned AIMessage — now stripped of its reasoning — back to the provider on
the follow-up call after a tool result. Providers that require the reasoning
block to be echoed back reject the request.

This issue tracks Option A of the diagnosis: route reasoning models to a
reasoning-aware provider client so reasoning_content round-trips correctly.

Current behavior

Running an actor whose llm agent uses provider: openai_compatible with a
reasoning model (e.g. deepseek-v4-flash-free) behind a reasoning-capable
proxy fails on the first post-tool-call round with:

Error code: 400 - {'error': {'type': 'invalid_request_error',
'code': 'invalid_request_error', 'message': 'Error from provider (Console):
Upstream request failed: [invalid_request_error] The `reasoning_content` in the
thinking mode must be passed back to the API.'}}

Confirmed diagnostics:

reasoning_content appears in no logged LLM request or response — only in
the error — proving it is dropped end-to-end.
The failure is a non-transient 400, so per ADR-2032 D-8 it is not
retried; it aborts the actor-graph execution and recurs on every attempt.

Expected behavior

An llm agent configured with a reasoning / thinking model behind an
OpenAI-compatible (or provider-specific) endpoint completes a multi-turn
tool-call loop without the reasoning_content ... must be passed back
400 error.
When a provider returns reasoning_content, it is preserved on the resulting
AIMessage and re-sent on the assistant turn that precedes the subsequent
tool result.
Behavior for native providers (openai, anthropic, google) and for
non-reasoning models is unchanged.

Acceptance criteria

An llm agent using a reasoning model through the reasoning-aware
routing path completes at least one tool-call → tool-result → follow-up
round with no invalid_request_error about reasoning_content.
The assistant message replayed by
LLMAgent._execute_tool_loop after a tool call includes the
reasoning_content returned by the model (verified via a fake reasoning
model in a Behave scenario).
Native-provider and non-reasoning-model paths produce byte-identical
request payloads to those produced before this change (no regression).
An ADR extending ADR-2028 is merged that documents the reasoning-aware
routing decision, and docs/specification.md is updated accordingly
before implementation lands.
nox (all default sessions) is green and nox -s coverage_report
reports ≥ 97%.

Supporting information

Root-cause code references (logical location, commit 882336c):
- cleveractors.agents.llm_client._build_from_credentials — routes every
  non-native provider to ChatOpenAI(base_url=...).
- cleveractors.agents.llm.LLMAgent._execute_tool_loop — appends the
  returned AIMessage (reasoning already stripped) before re-invoking.
langchain-openai 1.4.1 BaseChatOpenAI docstring — documents that
reasoning_content is not extracted; recommends ChatDeepSeek /
provider-specific subclasses.
Related ADRs: ADR-2028 (Extended Provider Routing — the decision this
work extends), ADR-2032 (retry scope — explains why the 400 is not
retried).
Scoping note for triage: because this is architectural, the ADR must be
written and approved before implementation (ADR process). If the maintainer
prefers strict one-commit atomicity, this issue may be promoted to an Epic
with the ADR split into its own child issue. It also still needs to be linked
to an appropriate LLM/provider parent Epic during triage (mandatory-parent
rule).

Subtasks

Write ADR (docs/adr/ADR-NNNN-reasoning-aware-provider-routing.md)
extending ADR-2028; submit for review; on approval update
docs/specification.md.
Add the reasoning-aware provider client dependency
(langchain-deepseek / ChatDeepSeek) to pyproject.toml.
Extend provider routing in cleveractors.agents.llm_client to select the
reasoning-aware client for reasoning models / configured providers,
preserving the reasoning_content round-trip while leaving native and
non-reasoning paths unchanged.
Verify LLMAgent._execute_tool_loop re-sends reasoning_content
(ensure additional_kwargs carrying reasoning survive the replay).
Tests (Behave): scenarios proving reasoning_content is preserved across
a tool-call turn using a fake reasoning model in features/mocks/.
Tests (Robot): integration test exercising a reasoning-capable endpoint
through the new routing path.
Verify coverage ≥ 97% via nox -s coverage_report.
Run nox (all default sessions) and fix any errors.

Definition of Done

This issue is complete when:

All subtasks above are completed and checked off.
A Git commit is created where the first line matches the Commit Message in
Metadata exactly.
The commit is pushed to the branch matching the Branch in Metadata exactly.
The commit is submitted as a PR to master, reviewed, and merged.

## Metadata - **Commit Message:** `feat(llm): route reasoning models to reasoning-aware provider clients` - **Branch:** `feature/m2-reasoning-model-provider-support` ## Background and context The Actor Configuration Standard (§1.2) deliberately excludes "the internal mechanics of LLM providers" from its scope, and **ADR-2028 (Extended Provider Routing)** routes every non-native provider — including `openai_compatible` and every named additional provider — through a bare `langchain_openai.ChatOpenAI(base_url=..., api_key=...)` client (`cleveractors.agents.llm_client._build_from_credentials`, at commit `882336c`). `langchain-openai` (currently 1.4.1) explicitly targets the *official* OpenAI API specification only. Its own module docstring states that non-standard response fields added by third-party providers — specifically `reasoning_content` — are **not extracted or preserved**, and recommends a provider-specific subclass (e.g. `openai_compatible_reasoning`) when pointing `base_url` at a reasoning-capable provider. As a result, when an `llm` agent is pointed at a **reasoning / "thinking" model** through an OpenAI-compatible endpoint, the model's `reasoning_content` is silently dropped on both the response leg (`_convert_dict_to_message`) and the request leg (`_convert_message_to_dict`, which only ever emits `role`/`content`/`tool_calls`). The multi-turn tool-call loop (`cleveractors.agents.llm.LLMAgent._execute_tool_loop`) then replays the returned `AIMessage` — now stripped of its reasoning — back to the provider on the follow-up call after a tool result. Providers that require the reasoning block to be echoed back reject the request. This issue tracks **Option A** of the diagnosis: route reasoning models to a reasoning-aware provider client so `reasoning_content` round-trips correctly. ## Current behavior Running an actor whose `llm` agent uses `provider: openai_compatible` with a reasoning model (e.g. `deepseek-v4-flash-free`) behind a reasoning-capable proxy fails on the first post-tool-call round with: ``` Error code: 400 - {'error': {'type': 'invalid_request_error', 'code': 'invalid_request_error', 'message': 'Error from provider (Console): Upstream request failed: [invalid_request_error] The `reasoning_content` in the thinking mode must be passed back to the API.'}} ``` Confirmed diagnostics: - `reasoning_content` appears in **no** logged LLM request or response — only in the error — proving it is dropped end-to-end. - The failure is a non-transient `400`, so per **ADR-2032 D-8** it is not retried; it aborts the actor-graph execution and recurs on every attempt. ## Expected behavior - An `llm` agent configured with a reasoning / thinking model behind an OpenAI-compatible (or provider-specific) endpoint completes a multi-turn tool-call loop **without** the `reasoning_content ... must be passed back` 400 error. - When a provider returns `reasoning_content`, it is preserved on the resulting `AIMessage` and re-sent on the assistant turn that precedes the subsequent tool result. - Behavior for native providers (`openai`, `anthropic`, `google`) and for non-reasoning models is unchanged. ## Acceptance criteria - [ ] An `llm` agent using a reasoning model through the reasoning-aware routing path completes at least one tool-call → tool-result → follow-up round with no `invalid_request_error` about `reasoning_content`. - [ ] The assistant message replayed by `LLMAgent._execute_tool_loop` after a tool call includes the `reasoning_content` returned by the model (verified via a fake reasoning model in a Behave scenario). - [ ] Native-provider and non-reasoning-model paths produce byte-identical request payloads to those produced before this change (no regression). - [ ] An ADR extending ADR-2028 is merged that documents the reasoning-aware routing decision, and `docs/specification.md` is updated accordingly before implementation lands. - [ ] `nox` (all default sessions) is green and `nox -s coverage_report` reports ≥ 97%. ## Supporting information - Root-cause code references (logical location, commit `882336c`): - `cleveractors.agents.llm_client._build_from_credentials` — routes every non-native provider to `ChatOpenAI(base_url=...)`. - `cleveractors.agents.llm.LLMAgent._execute_tool_loop` — appends the returned `AIMessage` (reasoning already stripped) before re-invoking. - `langchain-openai` 1.4.1 `BaseChatOpenAI` docstring — documents that `reasoning_content` is not extracted; recommends `ChatDeepSeek` / provider-specific subclasses. - Related ADRs: **ADR-2028** (Extended Provider Routing — the decision this work extends), **ADR-2032** (retry scope — explains why the 400 is not retried). - **Scoping note for triage:** because this is architectural, the ADR must be written and approved *before* implementation (ADR process). If the maintainer prefers strict one-commit atomicity, this issue may be promoted to an Epic with the ADR split into its own child issue. It also still needs to be linked to an appropriate LLM/provider parent Epic during triage (mandatory-parent rule). ## Subtasks - [ ] Write ADR (`docs/adr/ADR-NNNN-reasoning-aware-provider-routing.md`) extending ADR-2028; submit for review; on approval update `docs/specification.md`. - [ ] Add the reasoning-aware provider client dependency (`langchain-deepseek` / `ChatDeepSeek`) to `pyproject.toml`. - [ ] Extend provider routing in `cleveractors.agents.llm_client` to select the reasoning-aware client for reasoning models / configured providers, preserving the `reasoning_content` round-trip while leaving native and non-reasoning paths unchanged. - [ ] Verify `LLMAgent._execute_tool_loop` re-sends `reasoning_content` (ensure `additional_kwargs` carrying reasoning survive the replay). - [ ] Tests (Behave): scenarios proving `reasoning_content` is preserved across a tool-call turn using a fake reasoning model in `features/mocks/`. - [ ] Tests (Robot): integration test exercising a reasoning-capable endpoint through the new routing path. - [ ] Verify coverage ≥ 97% via `nox -s coverage_report`. - [ ] Run `nox` (all default sessions) and fix any errors. ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the first line matches the Commit Message in Metadata exactly. - The commit is pushed to the branch matching the Branch in Metadata exactly. - The commit is submitted as a PR to `master`, reviewed, and merged.

CoreRasurae added the

labels

2026-08-04 10:00:15 +00:00

CoreRasurae self-assigned this

2026-08-04 14:05:21 +00:00

CoreRasurae referenced this issue from a commit