Repository navigation
fix(VertexAI): use :embedContent for Gemini embedding models - #14284
Veer Jain (veerjain-1) wants to merge 5 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
This PR fixes Vertex AI embedding generation for gemini-embedding-* models by switching VertexAIEmbeddingClient from the legacy :predict endpoint to :batchEmbedContents (with the correct request/response shapes) while keeping :predict for older embedding models.
Changes:
- Route
gemini-embedding-*models to:batchEmbedContentsand deserialize the correspondingembeddings[].valuesresponse format. - Add new request/response DTOs for the Vertex AI
batchEmbedContentspayload. - Add unit tests and test data validating endpoint selection, payload shape, and response parsing for gemini embedding models.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| dotnet/src/Connectors/Connectors.Google/Core/VertexAI/VertexAIEmbeddingClient.cs | Selects :batchEmbedContents vs :predict based on model id and processes the corresponding response format. |
| dotnet/src/Connectors/Connectors.Google/Core/VertexAI/VertexAIEmbedContentRequest.cs | Introduces the requests[] payload shape required by batchEmbedContents. |
| dotnet/src/Connectors/Connectors.Google/Core/VertexAI/VertexAIEmbedContentResponse.cs | Introduces the embeddings[] response model for batchEmbedContents. |
| dotnet/src/Connectors/Connectors.Google.UnitTests/TestData/vertex_embed_content_response.json | Adds fixture JSON for the batchEmbedContents response format. |
| dotnet/src/Connectors/Connectors.Google.UnitTests/Core/VertexAI/VertexAIClientEmbeddingsGenerationTests.cs | Adds tests ensuring correct endpoint selection, payload structure, and response parsing for gemini embedding models. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
There was a problem hiding this comment.
MAF Automated Review — Iteration 1
Result: Findings reported
Scope: full PR (1 commit(s)): 95c1cd0e7abf
Model: claude-opus-4.8
Overview
The PR cleanly routes gemini-embedding-* models to the :batchEmbedContents endpoint with new internal request/response DTOs while preserving the legacy :predict path, gated by a single construction-time decision that keeps the endpoint suffix and payload shape in lockstep. Routing, dimension threading, and strict ([JsonRequired]) response deserialization are well guarded and covered by new unit tests. The one residual risk is that each requests[] item in the new Vertex batch body omits the per-request model field that the sibling GoogleAI batchEmbedContents implementation deliberately sends, which can cause the exact runtime rejection this PR aims to eliminate. The stubbed test handler cannot catch that live-API contract gap.
Reviewed the supplied pull-request change set across correctness, security/reliability, architecture, and failure behavior.
1 verified finding remained after source verification (1 high) across 1 file. Details are attached to the affected lines below.
Affected areas: dotnet/src/Connectors/Connectors.Google/Core/VertexAI/VertexAIEmbedContentRequest.cs
|
|
||
| public static VertexAIEmbedContentRequest FromData(IEnumerable<string> data, int? dimensions = null) => new() | ||
| { | ||
| Requests = data.Select(text => new EmbedContentRequestItem |
There was a problem hiding this comment.
Each item in the :batchEmbedContents request body is built with only content (and optional outputDimensionality) and never sets a per-request model. The sibling GoogleAI implementation for this same endpoint sets it explicitly (GoogleAIEmbeddingRequest.cs:39, Model = $"models/{modelId}"), because the batchEmbedContents schema treats each requests[] entry as a full embed request whose model is required. If Vertex enforces the same requirement, every gemini-embedding-* call will fail at runtime with an HTTP 400 — the same class of error this PR is meant to fix — and the stubbed test handler cannot detect it since it never validates the body against the live API.
Populate a model field on each EmbedContentRequestItem with the fully-qualified Vertex resource name (projects/{projectId}/locations/{location}/publishers/google/models/{modelId}), or confirm against the Vertex batchEmbedContents REST reference that omission is accepted and document that decision. Please verify against a live endpoint before merge.
There was a problem hiding this comment.
The missing model field is present in the current implementation, and the refreshed branch now has an explicit regression assertion for projects/fake-project-id/locations/us-central1/publishers/google/models/{modelId} on each tested batch request item. All 444 Google connector unit tests pass on current main. No live Vertex AI call was made, so live endpoint validation remains unclaimed.
There was a problem hiding this comment.
Correction after checking the original issue and Google's Vertex REST reference and SDK: this was using the Gemini Developer API batch contract on Vertex. Commit 9935a9f now calls Vertex :embedContent, with a single content request and singular embedding response, one call per input in order. The model is supplied in the URL; a per-item model field does not apply to this contract. gemini-embedding-001 stays on :predict. All 452 connector tests pass, and the two new contract tests fail against the previous implementation. No authenticated live Vertex call was possible because no project/credentials are configured. References: https://docs.cloud.google.com/gemini-enterprise-agent-platform/reference/rest/v1/projects.locations.publishers.models/embedContent and https://github.com/googleapis/js-genai/blob/main/src/models.ts .
|
@microsoft-github-policy-service agree |
|
Hi team! Just a gentle ping on this PR. Let me know if there's anything else you need from my side or if any changes are required. Thanks! |
|
Regarding the automated review feedback about the missing |
|
Hi Giles Odigwe (@giles17), Cozmopolit — friendly ping! This PR has been open for about 2 weeks now. I've addressed the automated review feedback (the missing |
|
Hey team, just checking in on this! Let me know if I need to assign a specific reviewer or make any tweaks. |
|
Hi Giles Odigwe (@giles17), Cozmopolit — another friendly ping on this! It's been a little over a week since my last bump. Happy to make any changes or address any feedback whenever you have a moment to take a look. |
|
Hi Giles Odigwe (@giles17), Cozmopolit — checking in again! Let me know if there's anything I can do to help move this forward. |
|
Hi there! Just a gentle ping to check in on this. Let me know if there's anything else needed from my side to move this forward! |
|
Hi team 👋 Friendly follow-up on this PR (been open ~9 weeks). Happy to rebase on latest main, add tests, or adjust the implementation if there are concerns. Would love to get any feedback to move this forward! |
|
Hi Giles Odigwe (@giles17), Cozmopolit, I refreshed this branch with current main and added an explicit regression assertion for the fully qualified The previous five workflow failures were caused by approval expiration before jobs ran. Could a maintainer approve the fresh CI workflows and review the updated branch ( |
Resolves #14265.
gemini-embedding-2fails when routed through Vertex AI's:predictoperation. Route newer Gemini embedding models to Vertex:embedContent, using its singlecontentrequest and singularembedding.valuesresponse. Keepgemini-embedding-001and legacy text embedding models on:predict.Vertex
embedContentaccepts one content per call. The existing batch-shaped client interface therefore sends one request per input, returns embeddings in input order, forwards dimensions and cancellation, and stops on the first failed request. The model resource is in the URL, not a per-item request field.This corrects the earlier implementation, which mistakenly used the Gemini Developer API's
:batchEmbedContentscontract for Vertex AI.Primary references:
Validation:
git diff --checkpasses.Live authenticated Vertex inference remains unverified: no configured project or credentials are available in the validation environment. No paid API call was made. Multi-input calls make one Vertex request per text; this is synchronous client-side batching, not an asynchronous batch prediction job. Maintainer review and upstream CI remain required.