Skip to content

feat(models): switch default get() model from Gemini 2.5 to Gemini 3.5 Flash - #316

Merged
philipph-askui merged 1 commit into
mainfrom
feat/gemini-3-5-flash
Sep 21, 2026
Merged

philipph-askui merged 1 commit into
mainfrom
feat/gemini-3-5-flash

Conversation

@philipph-askui

Copy link
Copy Markdown
Contributor

Summary

Gemini 2.5 (Flash, Flash-Lite, Pro) is being deprecated from the AskUI API. This PR swaps every 2.5 reference over to gemini-3.5-flash.

  • Default AskUiGeminiGetModel, AskUIImageQAProvider and GoogleImageQAProvider to gemini-3.5-flash. The first two are what ComputerAgent().get() uses over the AskUI API.
  • Add ModelName.GEMINI__3_5__FLASH and ASKUI__GEMINI__3_5__FLASH. The four 2.5 constants stay with a deprecation docstring so existing imports keep working.
  • Update docstring examples, docs/04_using_models.md, docs/05_bring_your_own_model_provider.md and examples/model_providers.py. gemini-2.5-pro is dropped from the docs since the proxy does not serve a 3.5 Pro (returns 404).
  • E2E get tests use the 3.5 constant; the six gemini_pro parametrizations are removed. VLM routing unit tests use 3.5 Flash as their Gemini id.
  • Flash-Lite had no references in the repo.

Verification

  • pdm run qa:fix clean (mypy, ruff format, ruff lint).
  • tests/unit/model_providers: 81 passed.
  • tests/e2e/agent/test_get.py against the live AskUI proxy: 26 of 28 selected tests pass, including PDF, XLSX, DOCX and response-schema cases on 3.5 Flash.
  • The two failures (test_get_with_pdf_too_large_with_default_model, test_get_with_recursive_response_schema[askui]) are pre-existing and reproduce identically on main. They do not depend on the model id.

Known issue: latency (out of scope for this PR)

Schema-constrained get() calls on gemini-3.5-flash are bimodal upstream at Google: roughly half complete in ~3s, the rest stall for 20–50s. This reproduces through both the Vertex and OpenAI-compatible AskUI proxies, is not caused by retries or thinking, and is independent of schema shape. Plain JSON mode (no response_schema) is a consistent 2–4s. Measured latencies on a trivial prompt, 4 runs each:

Model Mode Median Max
gemini-2.5-flash response_schema 0.9s 1.0s
gemini-3.5-flash response_schema 21.2s 48.7s
gemini-3.5-flash no schema 3.3s 4.2s

E2E get tests now take 14–32s each against a 60s per-test timeout, so they may flake in CI until this is addressed (e.g. by switching the get model to plain JSON mode with client-side Pydantic validation).

🤖 Generated with Claude Code

…5 Flash

Gemini 2.5 (Flash, Flash-Lite, Pro) is being deprecated from the AskUI API.

- Default `AskUiGeminiGetModel`, `AskUIImageQAProvider` and
  `GoogleImageQAProvider` to `gemini-3.5-flash`
- Add `ModelName.GEMINI__3_5__FLASH` / `ASKUI__GEMINI__3_5__FLASH`; keep the
  2.5 constants with a deprecation note for backwards compatibility
- Update docstrings, docs and examples; drop `gemini-2.5-pro` since no 3.5
  Pro is served by the proxy
- Update e2e/unit tests to use 3.5 Flash and remove the gemini_pro params

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@philipph-askui
philipph-askui marked this pull request as ready for review September 21, 2026 09:50
@philipph-askui
philipph-askui merged commit 129e261 into main Sep 21, 2026
1 check passed
@philipph-askui
philipph-askui deleted the feat/gemini-3-5-flash branch September 21, 2026 10:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants