GenAI Releases

Mistral Large 4 · Oct 29, 2026 · Models

A 1T-parameter multimodal model with a 1M context window can now be self-hosted with vLLM or SGLang, as well as used through the Mistral API.

Bring it up when

Client wants coding, agent or security workloads on its own infrastructure, or needs a European-trained model with no dependency on a hosted API.

Replaces

Could retire a hosted closed model for these workloads, or Mistral Medium 3.5, which Mistral now lists as deprecated in favor of this model.

Watch out for

Businesses running Model-as-a-Service need a separate commercial license from Mistral. Internal use that gives third parties no access is exempt.

Also available from

Meta: Muse Spark Open

Where it runs

Vendor API: YesBedrock: NoAzure: Not confirmedVertex AI: Not confirmed

Claim quality

Vendor-reported: Mistral says it scores 48% on the Artificial Analysis Coding Agent Index, using 47% fewer tokens per task than GLM-5.3.

Sources: 1 2 3 4

Amazon Bedrock · Oct 26, 2026 · Models

Adds a mid-priced GPT-6.1 option under AWS controls, with up to 1M input tokens and image input, and usage inside Bedrock logging and billing.

Bring it up when

Client runs on AWS and has kept OpenAI models out, or runs them separately, because of procurement, IAM or audit requirements.

Replaces

A separate OpenAI account and contract for AWS-hosted workloads, or a proxy in front of the OpenAI API.

Watch out for

On bedrock-runtime it needs a cross-Region inference profile, not in-Region; commercial in-Region prices carry a 10% fee over OpenAI rates.

Also available from

Anthropic: Claude Sonnet 5.5

Where it runs

Vendor API: YesBedrock: YesAzure: YesVertex AI: No

Claim quality

Vendor-reported: AWS reports OpenAI's Agents' Last Exam score 15 points above GPT-5.6 Terra, and tasks over 3x faster than GPT-6 Astra.

Sources: 1 2

OpenAI API · Oct 26, 2026 · Models

Adds a mid-priced tier at $1 input and $5 output per 1M tokens, so image-heavy and screen-driven tasks may not need the Sol or Astra tiers.

Bring it up when

The client automates screen-based or visual work on a higher tier and wants to test whether a cheaper model holds quality on its own images.

Replaces

Could replace GPT-5.6 Terra, which OpenAI's guide names as the migration source, after comparison on representative images.

Watch out for

Tool calling requires the Responses API. Prompts over 272K input tokens cost 2x input and cache rates and 1.5x output for the whole request.

Also available from

Anthropic: Claude Sonnet 5.5

Where it runs

Vendor API: YesBedrock: YesAzure: YesVertex AI: No

Claim quality

No performance claim

Sources: 1 2

Gemini Enterprise Agent Platform · Oct 19, 2026 · Platforms

Gemini 3.5 Flash, 3.5 Flash-Lite and 3.1 Flash-Lite can be tuned and served in production; reinforcement learning fine-tuning remains Preview.

Bring it up when

Client runs a high-volume, well-defined task on a larger model and has labeled examples, so tuning a cheaper model could be compared with prompting alone.

Replaces

Long few-shot prompts, or a larger model used only to hold accuracy on a narrow task.

Watch out for

Fine-tuning is excluded from any SLA. Gemini 3 tuning and serving run on us and eu multi-regions. Provisioned Throughput burndown can carry a premium.

Also available from

AWS: Bedrock model customization
Microsoft: Foundry supervised fine-tuning

Where it runs

Vendor API: NoBedrock: NoAzure: NoVertex AI: Yes

Claim quality

No performance claim

Sources: 1 2 3 4

Muse Spark Open · Oct 15, 2026 · Models

The Muse Spark line now has a self-hostable version with text, image, video and audio input. The model card asks for at least 700GB of VRAM for NVFP4.

Bring it up when

Client wants agentic coding or long-horizon work on its own infrastructure, from GPU servers to two Mac Studios, and can accept a custom license.

Replaces

Could retire a hosted model for these workloads. Meta now lists Muse Spark Open and Muse Glimmer as its open-weight lines, not Llama.

Watch out for

Custom license: products must display "Built with Muse Spark Open"; MaaS firms over $20M revenue need a Meta agreement; policy bars operating critical infrastructure.

Also available from

Mistral: Mistral Large 4

Where it runs

Vendor API: Not confirmedBedrock: NoAzure: YesVertex AI: Not confirmed

Claim quality

Vendor-reported: Meta calls it the most capable US open-weight model to date.

Sources: 1 2 3 4

Nemotron · Oct 15, 2026 · Models

Super VL handles text, images, video and documents in one 120B model, while Bolt 74B and E4B target local NVIDIA systems. Weights, datasets and recipes are published.

Bring it up when

Client wants one open model family across server and edge, or wants to post-train a model on its own data and keep it on premises.

Replaces

Could retire separate vision-language and text models routed behind one agent, or Nemotron 3 Nano for local tasks.

Watch out for

License is OpenMDW-1.1. The Bolt 74B 24GB build fits its memory target through native 2-bit quantization. Throughput claims come from NVIDIA.

Also available from

Meta: Muse Glimmer
Mistral: Mistral Small 4

Where it runs

Vendor API: YesBedrock: NoAzure: YesVertex AI: Not confirmed

Claim quality

Vendor-reported: NVIDIA says Super VL is 5x faster than the next open model in its class.

Sources: 1 2 3 4

Gemini API · Oct 13, 2026 · Models

Adds a higher tier with a 1M-token shared input and output window and long decodes, but explicit caching code must be removed and thinking uses the output limit.

Bring it up when

Client has long-running coding, finance, legal or science agents and wants to test a higher tier against its current model on its own tasks.

Replaces

Splitting long outputs across several calls, and designs built on explicit context caching.

Watch out for

Explicit caching is unsupported. The $2/$10 per 1M token rate ends Nov 30, 2026, then $4/$20. Not on the Free tier.

Also available from

OpenAI: GPT-6 Astra
Anthropic: Claude Fable 5.1

Where it runs

Vendor API: YesBedrock: NoAzure: NoVertex AI: Yes

Claim quality

Vendor-reported: Vendor says it can reason longer on hard problems for better results; the page gives no benchmark figures

Sources: 1 2 3

Claude API · Oct 7, 2026 · Models

Base price is $0.10 / $0.50 per MTok for prompts up to 100K tokens, against $1 / $5 for Haiku 4.5, but thinking is on by default and code written for Haiku 4.5 can return 400 errors.

Bring it up when

The client runs high-volume, latency-sensitive tasks or subagents on a small model, and is comparing per-task cost after recounting tokens on its own traffic.

Replaces

Claude Haiku 4.5, and possibly a mid-tier model used for short tasks. Thinking blocks and tokenizer changes need retesting first.

Watch out for

Prompts over 100,000 tokens pay $0.50 / $2.50 per MTok. Same text counts about 30 percent more tokens than Haiku 4.5. No server-side fallback or fallback credit on refusals.

Also available from

OpenAI: GPT-6 Luna
Google: Gemini 3.5 Flash-Lite

Where it runs

Vendor API: YesBedrock: YesAzure: YesVertex AI: Yes

Claim quality

No performance claim

Sources: 1 2 3 4

Amazon Nova · Oct 5, 2026 · Models

A newer Nova speech-to-speech option with tool calling at Nova 2 Sonic pricing, which can replace separate speech-to-text, LLM and text-to-speech steps.

Bring it up when

Client is planning or running a voice agent for customer service and is weighing a single speech-to-speech model against a multi-model pipeline.

Replaces

A pipeline of separate speech-to-text, LLM and text-to-speech models, and Nova 2 Sonic or the retired original Nova Sonic.

Watch out for

Four Regions at launch and seven languages. The post lists a 256K context window, while the Nova 2 Sonic launch post listed one million tokens.

Also available from

OpenAI: GPT-Live 1
Google: Gemini 3.8 Live

Where it runs

Vendor API: NoBedrock: YesAzure: NoVertex AI: No

Claim quality

Vendor-reported: AWS says improved reasoning, instruction following and tool-calling accuracy and lower latency; no figures given.

Sources: 1

OpenAI API · Sep 29, 2026 · Models

Prices at $2 input and $10 output per 1M tokens up to 272K input, and lets the model delegate to subagents in beta within one Responses request.

Bring it up when

The client is choosing a default for coding or knowledge work, or is planning a move off GPT-5.x before shutdown dates, and wants to test below the top tier.

Replaces

Could replace GPT-6 Sol, GPT-5.6 Sol and a hand-built subagent orchestrator for simple delegation.

Watch out for

Tool calling requires the Responses API, and the `none` reasoning effort is not supported, so Chat Completions tool users must migrate first.

Also available from

Anthropic: Claude Opus 5.5
Google: Gemini 3.8 Flash
Mistral: Mistral Large 4

Where it runs

Vendor API: YesBedrock: YesAzure: YesVertex AI: No

Claim quality

No performance claim

Sources: 1 2 3 4

Claude API · Sep 28, 2026 · Models

List price stays at $2 / $10 per MTok as on Sonnet 5, and cache reads are now 0.05x input. Code written for Sonnet 5 can break in five ways.

Bring it up when

The client runs a mid-tier model in production, is cache-heavy or relies on tool_choice, or still runs Sonnet 4.5, which retires on November 30, 2026.

Replaces

Claude Sonnet 5 and Sonnet 4.5, and designs that force a tool call with tool_choice any or tool.

Watch out for

Forced tool use returns a 400 error. Safeguards can decline requests, including a category for developing competing AI models, and refusals in that category can be billed.

Also available from

OpenAI: GPT-6.1 Terra
Google: Gemini 3.8 Flash

Where it runs

Vendor API: YesBedrock: YesAzure: YesVertex AI: Yes

Claim quality

No performance claim

Sources: 1 2 3 4

Claude API · Sep 22, 2026 · Models

Costs less than Opus 5's $5 / $25 per MTok with a 1M token context, but always-on thinking removes a control that some cost and latency designs rely on.

Bring it up when

The client runs long agent tasks on an earlier Opus model and is weighing the lower list price against migration work and always-on thinking.

Replaces

Claude Opus 5 and Opus 4.x models, and designs using forced tool calls or thinking disabled.

Watch out for

Thinking can't be disabled and forced tool use returns a 400 error. Its biology and cybersecurity classifiers can decline requests. Priority Tier is not supported.

Also available from

OpenAI: GPT-6.1 Sol
Google: Gemini 3.8 Flash
Mistral: Mistral Large 4

Where it runs

Vendor API: YesBedrock: YesAzure: YesVertex AI: Yes

Claim quality

No performance claim

Sources: 1 2 3 4

OpenAI API · Sep 3, 2026 · Models

Top of the OpenAI lineup at $10 input and $50 output per 1M tokens, with async tool calling and mid-turn steering for long-running Responses API work.

Bring it up when

The client has hard multi-step work such as research, coding or document creation that lower tiers fail on, and can measure whether the higher price pays back.

Replaces

Could replace GPT-5.5 Pro or GPT-5.6 Sol in Pro mode for the hardest tasks.

Watch out for

No custom temperature, top_p or logprobs and no `none` reasoning effort. Tool calling requires the Responses API, so existing requests may need changes.

Also available from

Anthropic: Claude Fable 5.1
Google: Gemini 4 Argon

Where it runs

Vendor API: YesBedrock: YesAzure: YesVertex AI: No

Claim quality

No performance claim

Sources: 1 2 3

Gemini API · Sep 2, 2026 · Models

This is the routing target for gemini-2.5-pro from December 1, 2026, so 2.5 users face its breaking API changes and its price on a fixed timeline.

Bring it up when

Client has production traffic on Gemini 2.5 or an earlier Flash and needs to re-test prompts, tools and budgets against a newer default.

Replaces

Gemini 2.5 Pro, and earlier 3.x Flash versions that now route to newer models.

Watch out for

Intro $0.75/$3.75 per 1M tokens runs through Dec 31, 2026, then $1.50/$7.50. thinking_budget, sampling parameters and prefilled turns must be removed.

Also available from

Anthropic: Claude Opus 5.5
OpenAI: GPT-6.1 Sol

Where it runs

Vendor API: YesBedrock: NoAzure: NoVertex AI: Yes

Claim quality

Vendor-reported: Vendor says it substantially reduces failed loops and errors in agent workflows; the page gives no figures

Sources: 1 2 3 4

Amazon Bedrock · Sep 1, 2026 · Models

Adds long-running frontier work on Bedrock, but use requires opting in to a data retention mode that a zero data retention policy may not allow.

Bring it up when

Client wants a model for hours-long coding or analysis work, but its compliance position assumes prompts and outputs are not stored or reviewed.

Replaces

Nothing direct: new capability.

Watch out for

You must set aws_review. All traffic is retained up to 30 days and flagged traffic may get human review by AWS; eligible program customers get ZDR to Dec 31, 2026.

Also available from

OpenAI: GPT-6 Astra
Google: Gemini 4 Argon

Where it runs

Vendor API: YesBedrock: YesAzure: YesVertex AI: Yes

Claim quality

Vendor-reported: AWS says a step up over Claude Fable 5 on hard reasoning tasks, with fewer confident wrong answers; no figures given.

Sources: 1 2

Claude API · Sep 1, 2026 · Models

Long-running agent model at $10 / $50 per MTok with cache reads at $0.25. Its predecessor was suspended for all users from June 12 to July 1 under US export controls.

Bring it up when

The client wants a higher-priced model for long agent work, and has data retention terms, security or coding workloads that classifiers may flag, or low tolerance for losing access.

Replaces

Claude Fable 5, and possibly Opus models on tasks where eval results justify the price difference.

Watch out for

Requires 30-day data retention and is not available under ZDR unless Anthropic expressly authorizes it. Forced tool use returns a 400 error. Classifiers can decline requests.

Also available from

OpenAI: GPT-6 Astra
Google: Gemini 4 Argon

Where it runs

Vendor API: YesBedrock: YesAzure: YesVertex AI: Yes

Claim quality

No performance claim

Sources: 1 2 3 4

Gemini API · Jul 21, 2026 · Models

This is the routing target for gemini-2.5-flash from December 1, 2026 and a listed low-cost option for subagents and high-volume automation.

Bring it up when

Client has large-volume simple workloads or subagents on Gemini 2.5 Flash or Flash-Lite and wants to size a move before routing or retirement dates apply.

Replaces

Gemini 2.5 Flash, and using a larger model for every step of an agent.

Watch out for

Frequency and presence penalty values now return an API error, sampling values are ignored, and requests ending on a model turn fail.

Also available from

Anthropic: Claude Haiku 5.5
OpenAI: GPT-6 Luna

Where it runs

Vendor API: YesBedrock: NoAzure: NoVertex AI: Yes

Claim quality

No performance claim

Sources: 1 2 3 4

Azure AI Foundry · Jun 29, 2026 · Platforms

Azure-committed clients can run Claude with a single Azure bill and Entra ID access. Prompts stay in Azure only on Azure-hosted deployments, and hosting choice changes features.

Bring it up when

The client is committed to Azure and wants Claude alongside GPT models, with a compliance team that needs to know where inference runs and who processes data.

Replaces

Could replace a separate Anthropic contract or a gateway used only to reach Claude from Azure workloads.

Watch out for

Hosted on Azure lacks code execution, Agent Skills, programmatic tool calling and the Files API. Microsoft states no firm dates. Anthropic remains data processor.

Also available from

None confirmed

Where it runs

Vendor API: YesBedrock: YesAzure: YesVertex AI: Yes

Claim quality

No performance claim

Sources: 1 2 3 4

OpenAI on Amazon Bedrock · Jun 1, 2026 · Platforms

An AWS-committed client can buy and govern OpenAI models through AWS. Features differ between the bedrock-runtime and bedrock-mantle endpoints.

Bring it up when

The client's security or procurement rules favor AWS billing and access control, and it wants OpenAI models without a separate vendor relationship.

Replaces

Could replace a direct OpenAI contract or a cross-cloud gateway used only to reach OpenAI models.

Watch out for

Feature coverage differs by endpoint and Region: hosted web search needs Mantle, and the image generation tool is listed as not available on one endpoint.

Also available from

None confirmed

Where it runs

Vendor API: YesBedrock: YesAzure: YesVertex AI: No

Claim quality

No performance claim

Sources: 1 2

Mistral Small 4 · Mar 16, 2026 · Models

One Apache 2.0 model with image input and adjustable reasoning effort covers chat, coding and reasoning, replacing separate Mistral models for each.

Bring it up when

Client needs a mid-size multimodal model with no custom license conditions, and routes across separate reasoning, coding and chat models today.

Replaces

Could retire routing between Magistral, Devstral and Mistral Small, which Mistral says this model consolidates.

Watch out for

Apache 2.0, so no license limit. Self-hosting needs real GPU capacity: the minimum is 4x HGX H100, 2x HGX H200 or 1x DGX B200.

Also available from

NVIDIA: Nemotron 3.5 Super VL

Where it runs

Vendor API: YesBedrock: NoAzure: Not confirmedVertex AI: Not confirmed

Claim quality

Vendor-reported: Mistral says 40% lower end-to-end completion time and 3x more requests per second than Mistral Small 3.

Sources: 1 2

Foundry Models · Nov 18, 2025 · Platforms

One deployment picks a model per prompt by quality or cost mode, with failover and custom subsets. October 2026 raised the pool to 35 models.

Bring it up when

The client on Azure serves a mix of simple and hard prompts on one model, or is weighing a custom routing layer, and can evaluate quality on its own traffic.

Replaces

Could retire custom routing logic and per-use-case model selection, plus manual failover between model deployments.

Watch out for

The effective context window is capped by the smallest underlying model. Session affinity and routing metadata are preview, and Anthropic models need separate deployment.

Also available from

AWS: Bedrock Intelligent Prompt Routing

Where it runs

Vendor API: NoBedrock: NoAzure: YesVertex AI: No

Claim quality

No performance claim

Sources: 1 2 3

Qdrant · Oct 22, 2026 · Vector databases

Keyword ranking can run from a payload index built on text already stored, then combine with dense search through prefetch, without generating and uploading sparse BM25 vectors.

Bring it up when

The client runs hybrid search on Qdrant, or is choosing a store for it, and generates BM25 sparse vectors or runs a separate search engine for the keyword side.

Replaces

Client-side BM25 sparse vector generation, or a separate keyword search engine next to the vector store.

Watch out for

Statistics are per shard, so one document can score differently by shard, and deleted or updated points can skew scores until the optimizer rebuilds segments.

Also available from

Pinecone: Full-text search
Weaviate: Hybrid search with BM25F
Milvus: Full text search (BM25)

Where it runs

Not tied to one model platform

Claim quality

No performance claim

Sources: 1 2 3

Weaviate · Oct 7, 2026 · Vector databases

Late-interaction models such as ColBERT and ColPali can run on a disk-based index instead of holding token vectors in HNSW memory.

Bring it up when

The client is testing late-interaction retrieval or has a large, frequently updated corpus, and index memory cost is what blocks the design.

Replaces

Holding multi-vector embeddings in an in-memory HNSW index, or dropping late interaction because of its memory cost.

Watch out for

Needs MUVERA, fixed at creation, self-provided vectors and 1-bit RQ; other quantizers are rejected. HFresh supports cosine and l2-squared only.

Also available from

Qdrant: On-disk multi-vector search

Where it runs

Not tied to one model platform

Claim quality

Vendor-reported: Weaviate says HFresh needs far less memory than HNSW.

Sources: 1 2 3

Pinecone Database · Sep 2, 2026 · Vector databases

One index can combine BM25, dense and sparse fields, so keyword and vector retrieval can share a store. New REST indexes on API 2026-07 must be schema-based.

Bring it up when

The client runs hybrid retrieval on Pinecone with a separate keyword engine, or needs exact-term matching for product codes, names or legal text in a RAG system.

Replaces

A separate keyword engine such as a BM25 service, or sparse-only keyword retrieval, alongside the vector index.

Watch out for

Schemas are fixed after creation, indexes can't be created in CMEK projects, synonyms aren't supported and capacity can't go from dedicated back to on-demand.

Also available from

Qdrant: BM25 on full-text index
Weaviate: Hybrid search with BM25F
Milvus: Full text search (BM25)

Where it runs

Not tied to one model platform

Claim quality

No performance claim

Sources: 1 2

Milvus · Jul 29, 2026 · Vector databases

Parquet or Iceberg data on object storage can be indexed and searched without copying it into Milvus or running an ETL sync.

Bring it up when

The client already stores embeddings and metadata as lake tables, and the copy into a vector database doubles storage or weakens governance over the data.

Replaces

An ETL pipeline that copies lake tables into a vector database and keeps the copy in sync.

Watch out for

Needs Storage V3, off by default; once data is written in it, downgrades to versions that can't read it are unsupported. Refresh is manual and Iceberg isn't supported on Azure Blob.

Also available from

None confirmed

Where it runs

Not tied to one model platform

Claim quality

No performance claim

Sources: 1 2 3

LlamaIndex · Jun 24, 2026 · Frameworks

RetrieverQueryEngine and CitationQueryEngine can pass image, audio and video blocks to the LLM with multimodal=True, so mixed-media RAG no longer needs a separate engine path.

Bring it up when

Client's RAG corpus holds images, audio, video or scanned pages, and answers or citations need to draw on those directly rather than on extracted text.

Replaces

SimpleMultiModalQueryEngine, which the tracking issue lists as superseded and to be deprecated, and custom synthesis code.

Watch out for

Coverage is partial: router and sub-question engines still summarize as text only, and SQL join and FLARE engines were judged not to fit multimodal.

Also available from

None confirmed

Where it runs

Not tied to one model platform

Claim quality

No performance claim

Sources: 1 2 3

Amazon Bedrock Knowledge Bases · Jun 17, 2026 · Platforms

AWS runs vector storage, ingestion and retrieval, including hybrid search and agentic multi-hop retrieval, so a RAG team has no vector database to operate.

Bring it up when

Client has a RAG system with a self-run vector store and ingestion jobs, or is starting one on AWS and wants connectors for enterprise content.

Replaces

A self-managed vector database and ingestion pipeline, and the bring-your-own-store Knowledge Bases setup.

Watch out for

Listed in nine Regions (US East Virginia, Oregon, Ireland, London, Frankfurt, Tokyo, Singapore, Sydney, GovCloud West), with GovCloud feature differences.

Also available from

Microsoft: Foundry IQ
Google: RAG Engine Serverless mode
OpenAI: File search tool
Pinecone: Pinecone Nexus

Where it runs

Vendor API: NoBedrock: YesAzure: NoVertex AI: No

Claim quality

No performance claim

Sources: 1 2

Foundry IQ · Jun 2, 2026 · Platforms

Multiple agents can share one knowledge base across internal and external sources, using agentic retrieval instead of per-agent RAG pipelines.

Bring it up when

The client on Azure repeats retrieval plumbing per agent or per source, and wants a shared knowledge layer with security handled centrally.

Replaces

Could retire hand-built query planning and multi-source retrieval loops in front of Azure AI Search.

Watch out for

Availability is split: some features are GA and others are preview, depending on the Search REST API version, and portals expose preview-only access.

Also available from

AWS: Bedrock Managed Knowledge Base
Google: RAG Engine Serverless mode
Pinecone: Pinecone Nexus

Where it runs

Vendor API: NoBedrock: NoAzure: YesVertex AI: No

Claim quality

No performance claim

Sources: 1 2

Gemini API · May 5, 2026 · Platforms

A hosted retrieval tool can now index images alongside text and return page numbers and media IDs for citations, without running a vector database.

Bring it up when

Client has a modest document set, wants grounded answers with citations, and doesn't want to operate retrieval infrastructure.

Replaces

A self-run vector database and ingestion pipeline for smaller, simpler corpora.

Watch out for

Can't be combined with Google Search or URL Context grounding, isn't in the Live API, and image search takes PNG or JPEG only, with no audio or video.

Also available from

AWS: Bedrock Managed Knowledge Base

Where it runs

Vendor API: YesBedrock: NoAzure: NoVertex AI: Not confirmed

Claim quality

No performance claim

Sources: 1 2

Gemini API · Apr 22, 2026 · Models

One multimodal embedding space can replace separate per-media retrieval pipelines, but moving from gemini-embedding-001 means a full re-index.

Bring it up when

Client is planning a re-index or adding media to retrieval and needs to weigh a unified embedding model against the migration work.

Replaces

Separate embedding models and indexes per media type.

Watch out for

Spaces are incompatible with gemini-embedding-001, so all data must be re-embedded. Input caps at 8,192 tokens and excess is silently truncated.

Also available from

AWS: Nova Multimodal Embeddings

Where it runs

Vendor API: YesBedrock: NoAzure: NoVertex AI: Yes

Claim quality

No performance claim

Sources: 1 2 3

Gemini Enterprise Agent Platform · Apr 17, 2026 · Platforms

A managed retrieval backend can drop database provisioning, filter on metadata and query several corpora, which shifts the build versus buy line for RAG.

Bring it up when

Client wants managed RAG with metadata filters or retrieval over multiple corpora and is weighing Serverless mode against Spanner mode or its own stack.

Replaces

Provisioned Spanner instances for RAG Engine, and client-side merging of results from several corpora.

Watch out for

Serverless mode doesn't support CMEK, and corpora stay tied to the deployment mode active when they were created.

Also available from

AWS: Bedrock Managed Knowledge Base
Microsoft: Foundry IQ
OpenAI: File search tool
Pinecone: Pinecone Nexus

Where it runs

Vendor API: NoBedrock: NoAzure: NoVertex AI: Yes

Claim quality

No performance claim

Sources: 1 2 3 4

Amazon Bedrock AgentCore · Nov 5, 2026 · Platforms

Agents can get a finished cited answer over MCP from one call instead of running search, retrieval and synthesis steps, with domain and date filters.

Bring it up when

Client agents need current web facts with citations and the team owns a search API, fetch step and synthesis prompt it would rather not run.

Replaces

A custom search, retrieve, synthesize and cite pipeline, or a third-party answer API behind an agent tool.

Watch out for

Only US East (N. Virginia) today, billed per tool call, and it uses geographic cross-Region inference, so regional residency needs a check.

Also available from

OpenAI: Web search tool
Anthropic: Claude web search tool
Google: Grounding with Google Search
Microsoft: Foundry web search tool

Where it runs

Vendor API: NoBedrock: YesAzure: NoVertex AI: No

Claim quality

No performance claim

Sources: 1

Gemini API · Oct 30, 2026 · Platforms

Browser, mobile and desktop control agents move to a GA tool, and admins on Agent Platform can mandate organization-wide confirmation policies.

Bring it up when

Client relies on scripted UI automation or manual steps in legacy apps and is considering a model-driven agent to operate them.

Replaces

Brittle RPA scripts and selector-based browser automation for some workflows.

Watch out for

Client must host the execution environment, and Google advises a sandboxed VM or container. Prompt injection detection is off by default.

Also available from

Anthropic: Claude computer use toolset
OpenAI: Computer use tool
Microsoft: Foundry computer use tool
AWS: Amazon Nova Act

Where it runs

Vendor API: YesBedrock: NoAzure: NoVertex AI: Yes

Claim quality

No performance claim

Sources: 1 2 3

Foundry Agent Service · Oct 28, 2026 · Platforms

Prompt agents can read files, run code and use skills in a Foundry-run loop and sandbox, with the model chosen separately from the orchestration.

Bring it up when

The client builds agents on Azure, wants to stop owning loop and sandbox code, and can run a preview on new agents rather than migrate existing ones.

Replaces

Could retire a custom agent loop and sandbox for tasks that fit the supported tools.

Watch out for

Preview, and switching after creation is not supported. Sandbox compute and memory billing begins December 1, 2026 at $0.0994 per vCPU-hour and $0.0118 per GiB-hour.

Also available from

OpenAI: Agents API
Anthropic: Claude Managed Agents
Google: Gemini API managed agents
AWS: AgentCore harness

Where it runs

Vendor API: NoBedrock: NoAzure: YesVertex AI: No

Claim quality

No performance claim

Sources: 1 2

Amazon Bedrock · Oct 23, 2026 · Platforms

OpenAI models on Bedrock can ground answers on the web inside a Responses API call, now limited to up to 100 allowed or blocked domains.

Bring it up when

Client wants live web grounding on Bedrock, and security asks whether prompts leave AWS or which sites can feed answers.

Replaces

A third-party search vendor with its own keys, billing and review, plus the tool-call loop around it.

Watch out for

external_web_access defaults to true, so set it to false to stay in the AWS boundary. OpenAI models on bedrock-mantle only; not on bedrock-runtime.

Also available from

OpenAI: Web search tool
Anthropic: Claude web search tool
Microsoft: Foundry web search tool

Where it runs

Vendor API: NoBedrock: YesAzure: NoVertex AI: No

Claim quality

No performance claim

Sources: 1 2

Voxtral 3 · Oct 20, 2026 · Models

Transcription, speech synthesis, reasoning and tool calls run in one streaming pipeline, through Mistral's API or self-deployed, so fewer separate voice components are needed.

Bring it up when

Client runs contact-center or service-desk voice agents and needs a say over audio location and retention, or wants fewer vendors in the voice pipeline.

Replaces

Could retire a self-assembled pipeline of separate speech-to-text, LLM and text-to-speech services.

Watch out for

All four models are in Public Preview. Mistral lists them under a Premier license, not as open weights, so self-deployment terms need checking.

Also available from

OpenAI: GPT-Live 1
AWS: Amazon Nova 2.5 Sonic
Google: Gemini Live API

Where it runs

Vendor API: YesBedrock: NoAzure: Not confirmedVertex AI: Not confirmed

Claim quality

Vendor-reported: Mistral says Voxtral Small Transcribe 3 outperforms Whisper, Scribe v2 and Gemini 3.5 Transcribe.

Sources: 1 2 3

LangChain · Oct 14, 2026 · Frameworks

Each gated tool call gets its own interrupt and response schema, so reviewers answer calls separately. LangChain plans to make it the default in the next major release.

Bring it up when

Agents call tools with side effects, and reviewers need to approve, edit or reject each call on its own, with a form built from a schema and answers checked before saving.

Replaces

One batched interrupt per model turn, plus custom client code that validates approval answers.

Watch out for

If one of several answers sent together is invalid, the other tools can run while still shown as pending, and answering them again runs them twice.

Also available from

OpenAI: Agents SDK tool approvals
Microsoft: Agent Framework tool approval

Where it runs

Not tied to one model platform

Claim quality

No performance claim

Sources: 1 2 3

Amazon Bedrock · Sep 29, 2026 · Platforms

A hosted agent loop for OpenAI models with durable sessions, per-agent IAM roles and human approval, inside AWS. No extra charge in preview beyond resources used.

Bring it up when

Client has standardized on OpenAI agent patterns, must run on AWS, and is choosing between a hosted loop and AgentCore or its own framework.

Replaces

A self-run agent loop with custom session state and tool execution around OpenAI models on AWS.

Watch out for

Preview in 3 US Regions, pricing may change at GA. No subagents, no cross-Region profiles, text session input only and no built-in long-term memory.

Also available from

Anthropic: Claude Managed Agents
Google: Gemini API managed agents
Microsoft: Microsoft Copilot harness

Where it runs

Vendor API: NoBedrock: YesAzure: NoVertex AI: No

Claim quality

No performance claim

Sources: 1 2

Gemini API · Sep 17, 2026 · Platforms

Google-hosted sandboxed agents remove the need to run your own harness, but this release changed tool names and parameters for anyone parsing function calls.

Bring it up when

Client is scoping its own agent loop, sandbox and state store and could use a hosted preview for prototypes, on either the Gemini API or Agent Platform.

Replaces

Self-built agent loops and sandbox infrastructure, for prototypes and non-production work.

Watch out for

Preview. Agent Platform terms bar confidential data and production use. Gemini API environments are deleted after 7 days idle with open egress by default.

Also available from

Anthropic: Claude Managed Agents
OpenAI: Agents API
AWS: AgentCore harness
Microsoft: Microsoft Copilot harness

Where it runs

Vendor API: YesBedrock: NoAzure: NoVertex AI: Not confirmed

Claim quality

No performance claim

Sources: 1 2 3 4 5 6

LlamaAgents · Sep 16, 2026 · Frameworks

Workflows persist every step transition through DBOS on SQLite or Postgres and resume after a crash. Server-hosted workflows must now list other stored classes in allowed_types.

Bring it up when

Client runs multi-step LlamaIndex workflows that must survive crashes and restarts, and wants to self-host persistence on SQLite or Postgres rather than write snapshot code.

Replaces

Manual context snapshots for resuming workflows.

Watch out for

One workflow's work cannot be spread across replicas, and changing workflow code while runs are in progress can cause non-determinism on replay.

Also available from

LangChain: LangGraph durable execution
Microsoft: Agent Framework checkpoints

Where it runs

Not tied to one model platform

Claim quality

No performance claim

Sources: 1 2

Agents API · Sep 10, 2026 · Platforms

OpenAI runs session orchestration, compaction and recovery, and the sandbox can be OpenAI-hosted or self-hosted, which removes the need to run an agent loop.

Bring it up when

The client spends engineering time on orchestration and recovery for long-running agents and can accept a beta API tied to one vendor's harness.

Replaces

Could retire custom agent loops, plus remaining designs built on the Assistants API, which shut down on 2026-08-26.

Watch out for

Beta. US data residency only and no Zero Data Retention, even with a self-hosted sandbox, which can rule it out for regulated or EU workloads.

Also available from

Anthropic: Claude Managed Agents
Google: Gemini API managed agents
AWS: AgentCore harness
Microsoft: Microsoft Copilot harness

Where it runs

Vendor API: YesBedrock: YesAzure: Not confirmedVertex AI: Not confirmed

Claim quality

No performance claim

Sources: 1 2 3

GPT-Live · Sep 10, 2026 · Models

Voice conversation can continue while a backend model or agent does reasoning and tools, using OpenAI delegation or the client's own backend.

Bring it up when

The client is designing a voice assistant or phone agent and needs conversation to continue during slow backend work, including with its own backend.

Replaces

Could replace speech-to-speech designs on the Realtime API that pause during tool work.

Watch out for

Sessions cost $0.05 per minute with backend model and tool usage billed on top, and rate limits are measured in concurrent sessions.

Also available from

Mistral: Voxtral 3
AWS: Amazon Nova 2.5 Sonic
Google: Gemini Live API

Where it runs

Vendor API: YesBedrock: NoAzure: YesVertex AI: Not confirmed

Claim quality

No performance claim

Sources: 1 2 3

LangChain · Sep 3, 2026 · Frameworks

One MCPAdapter handles transports, OAuth 2.1 and elicitation through LangGraph interrupts. MultiServerMCPClient code must migrate, and some adapter features changed or were removed.

Bring it up when

Client's LangChain agents use MCP tools through langchain-mcp-adapters, or need per-user auth and human answers when an MCP server asks for input mid-call.

Replaces

The standalone langchain-mcp-adapters package and its MultiServerMCPClient.

Watch out for

In beta and the API may change. It focuses on tools and does not yet wrap MCP prompts or resources, so clients relying on those should stay put or plan a workaround.

Also available from

Microsoft: Agent Framework MCP tools
OpenAI: Agents SDK MCP support

Where it runs

Not tied to one model platform

Claim quality

No performance claim

Sources: 1 2 3

Claude API · Aug 19, 2026 · Platforms

Agents can drive a browser you host through its page structure, not only screenshots. The application still hosts the executor and carries prompt injection risk.

Bring it up when

The client wants an agent to work in web or desktop systems without APIs, and must decide what those sessions can reach and whether a human stays in the loop.

Replaces

Hand-built browser automation scripts around the older beta computer use tool.

Watch out for

Vendor states Claude sometimes follows instructions in page content even when they conflict with yours. Classifiers add a check, not a guarantee. Your application hosts the browser.

Also available from

OpenAI: Computer use tool
Google: Gemini Computer Use tool
Microsoft: Foundry computer use tool
AWS: Amazon Nova Act

Where it runs

Vendor API: YesBedrock: YesAzure: YesVertex AI: Yes

Claim quality

No performance claim

Sources: 1 2

Pinecone Nexus · Aug 6, 2026 · Vector databases

Agents send one KnowQL query and get a cited answer compiled from curated sources, which moves chunking and retrieval-loop work upstream into a managed curation step.

Bring it up when

The client runs agentic RAG where repeated retrieval drives token cost and latency, and would accept a managed compile step and a proprietary query language to reduce it.

Replaces

A hand-built chunking pipeline and an agent retrieve, evaluate and re-retrieve loop.

Watch out for

Nexus BYOC needs an Enterprise plan. Its API and console use a public endpoint because private-only access isn't supported, even if the Database data plane runs privately.

Also available from

Microsoft: Foundry IQ
AWS: Bedrock Managed Knowledge Base

Where it runs

Not tied to one model platform

Claim quality

Vendor-reported: Pinecone says Nexus spends far fewer tokens than agentic RAG and is faster than a retrieval loop. It gives no figures.

Sources: 1 2 3

MCP specification · Jul 28, 2026 · Protocols

Sessions and the initialize handshake are gone, so remote servers scale like ordinary HTTP services, but servers and clients must migrate and carry state explicitly.

Bring it up when

The client is building or operating remote MCP servers behind load balancers, or relies on session IDs, server-initiated requests or SSE resumption.

Replaces

Sticky sessions and Mcp-Session-Id state, the initialize handshake, and held-open streams for server-initiated requests.

Watch out for

Breaking: sessions, ping and SSE resumability are removed, with some migration cost for session-ID users. Tier 1 SDKs are TypeScript, Python, Go and C#; Rust was beta at release.

Also available from

None confirmed

Where it runs

Not tied to one model platform

Claim quality

No performance claim

Sources: 1 2 3

Gemini Enterprise Agent Platform · Jul 15, 2026 · Platforms

Structured profiles and streamed event ingestion can replace a custom memory store, and session and memory compute metering is now in effect.

Bring it up when

Client maintains its own memory extraction and vector store for agents and is deciding between that and a managed memory service.

Replaces

Custom memory extraction pipelines and bespoke per-user profile stores.

Watch out for

CMEK can't be used with the global endpoint, and gemini-embedding-2 needs a global, us or eu endpoint rather than a single region such as us-central1.

Also available from

AWS: AgentCore Memory
Microsoft: Foundry Agent Service Memory
Anthropic: Managed Agents memory stores

Where it runs

Vendor API: NoBedrock: NoAzure: NoVertex AI: Yes

Claim quality

No performance claim

Sources: 1 2 3 4 5

Foundry Agent Service · Jul 9, 2026 · Platforms

Code built on Agent Framework, LangGraph or custom code runs in per-session VM-isolated sandboxes with scale to zero and state restored on resume.

Bring it up when

The client already has agent code in a framework it wants to keep, and is weighing the work of running containers, session state and isolation itself.

Replaces

Could retire self-run container hosting for agents, along with custom session persistence.

Watch out for

Sessions are deleted after 30 days of inactivity and idle timeout tops out at 4 hours. Billing follows CPU and memory across all active sessions.

Also available from

AWS: AgentCore Runtime
Google: Agent Runtime

Where it runs

Vendor API: NoBedrock: NoAzure: YesVertex AI: No

Claim quality

No performance claim

Sources: 1 2 3

Amazon Bedrock AgentCore · Jun 17, 2026 · Platforms

Teams can declare model, tools and instructions as config and let AgentCore run the agent loop, with export to Strands-based code when they outgrow it.

Bring it up when

Client is building agents on AWS, or must leave Bedrock Agents Classic, and is deciding between a declared agent and owning a framework.

Replaces

Custom orchestration loops and session, memory and tool plumbing; also the migration path AWS names for Bedrock Agents Classic.

Watch out for

Export targets Strands-based code today. AWS says multi-agent routing needs custom framework code, so it may not fit routing-mode multi-agent designs.

Also available from

OpenAI: Agents API
Anthropic: Claude Managed Agents
Google: Gemini API managed agents
Microsoft: Microsoft Copilot harness

Where it runs

Vendor API: NoBedrock: YesAzure: NoVertex AI: No

Claim quality

No performance claim

Sources: 1 2

OpenAI API · May 19, 2026 · Protocols

A customer-hosted client opens an outbound path, so private or on-prem MCP servers can serve OpenAI products without public exposure.

Bring it up when

The client's security team blocks public MCP endpoints, but agents on ChatGPT, Codex or the Responses API need internal systems.

Replaces

Could retire public reverse proxies, inbound firewall exceptions or VPN arrangements built only to expose MCP servers.

Watch out for

Defaults cap traffic at 600 MCP requests per minute per tunnel and application and 10 MiB per serialized message, which can constrain busy or file-heavy tools.

Also available from

Anthropic: MCP tunnels

Where it runs

Vendor API: YesBedrock: NoAzure: Not confirmedVertex AI: Not confirmed

Claim quality

No performance claim

Sources: 1 2

LangGraph · May 12, 2026 · Frameworks

Timeouts, failure recovery and resumable drains are now graph features, so agent runs can fail over or stop cleanly during deploys without custom wrapper code.

Bring it up when

Client runs long agents in production and needs hung nodes capped, compensation logic after retries run out, or cooperative stops with a resumable checkpoint during deploys.

Replaces

Custom timeout wrappers, hand-written Saga or compensation routing, and bespoke drain logic around graph runs.

Watch out for

Timeouts apply only to async nodes, and sync nodes with a timeout fail at compile time. Retries are at-least-once, so external side effects must be idempotent.

Also available from

LlamaIndex: Workflows step retries and handlers

Where it runs

Not tied to one model platform

Claim quality

No performance claim

Sources: 1 2 3

Deep Agents · May 12, 2026 · Frameworks

An agent can write JavaScript that computes and chains allowlisted tools inside a scoped runtime, which changes how approvals and isolation need to be designed.

Bring it up when

Client wants agents to compute or chain several tools in code, and has approval rules or untrusted-code concerns that a security reviewer will ask about.

Replaces

Separate model turns for each tool call when code can chain them. It does not replace a sandbox backend for shell or OS work.

Watch out for

Beta. Tool calls made from code skip the normal path, so interrupt_on approvals are not enforced per call. LangChain calls it a scoped runtime, not a full production sandbox.

Also available from

Anthropic: Programmatic tool calling
OpenAI: Programmatic Tool Calling

Where it runs

Not tied to one model platform

Claim quality

No performance claim

Sources: 1 2 3

Claude Managed Agents · Apr 8, 2026 · Platforms

Teams can rent the agent loop, sandbox and session store instead of building them, in exchange for server-side state and a beta product. Billing adds session runtime to tokens.

Bring it up when

The client is about to build or maintain an agent harness, can accept server-side state and Claude-only models, and has no ZDR or HIPAA BAA requirement for this workload.

Replaces

A self-built agent loop with its own sandbox, session persistence and tool execution.

Watch out for

Beta on every model. Not eligible for ZDR or HIPAA BAA coverage. Not listed for Bedrock, Google-managed or Foundry. Adds $0.08 per session-hour of running time to tokens.

Also available from

OpenAI: Agents API
Google: Gemini API managed agents
AWS: AgentCore harness
Microsoft: Microsoft Copilot harness

Where it runs

Vendor API: YesBedrock: NoAzure: NoVertex AI: No

Claim quality

No performance claim

Sources: 1 2 3

Microsoft Agent Framework · Apr 3, 2026 · Frameworks

Gives one open-source SDK with stable APIs and connectors for Foundry, Azure OpenAI, OpenAI, Anthropic, Bedrock, Gemini and Ollama, so it is not tied to one model vendor.

Bring it up when

The client has code on Semantic Kernel or AutoGen, or is choosing a .NET or Python agent framework, and long-term support matters more than newest features.

Replaces

Could replace Semantic Kernel or AutoGen for agent orchestration, or a home-grown abstraction over several model providers.

Watch out for

Not all of it is stable: 1.0 also ships preview features whose APIs may change before reaching stable, so check each feature a design depends on.

Also available from

LangChain: LangGraph
Google: Agent Development Kit
OpenAI: Agents SDK

Where it runs

Not tied to one model platform

Claim quality

No performance claim

Sources: 1 2

Gemini API · Dec 11, 2025 · Platforms

Managed agents, Deep Research and Omni video run through this stateful API, so a new build has to decide its API surface up front.

Bring it up when

Client is starting a Gemini project or planning a refresh and needs to choose between server-side conversation state and its existing generateContent code.

Replaces

Client-managed conversation history and generateContent, which Google calls legacy but still fully supports.

Watch out for

Batch API, explicit caching and Python automatic function calling aren't available yet. Stored interactions last 55 days on paid tier. Gemini 3 use on Agent Platform is Preview.

Also available from

OpenAI: Responses and Conversations state
Anthropic: Claude message threads

Where it runs

Vendor API: YesBedrock: NoAzure: NoVertex AI: Yes

Claim quality

No performance claim

Sources: 1 2 3 4

Arize Phoenix · Oct 21, 2026 · Evals and observability

TypeScript classification evaluators can use a model that picks a label and returns probabilities, changing what a high-volume judge costs and what it records.

Bring it up when

An eval program runs chat-model judges on classification questions, and cost or latency limits how much production traffic gets scored.

Replaces

Chat-model judges used for classification-style evaluators.

Watch out for

Decision models return label probabilities but no explanation, and this release covers the TypeScript evaluators only, so reviewers lose judge reasoning.

Also available from

Langfuse: Decision model evaluators

Where it runs

Not tied to one model platform

Claim quality

Vendor-reported: Arize says a decision model is much faster and cheaper than a chat model judge; no figure given.

Sources: 1

Claude API · Oct 14, 2026 · Evals and observability

Teams can send Claude traces to an existing OpenTelemetry backend from the API or the SDK. Choosing full content also moves prompts and responses into that backend.

Bring it up when

The client already runs an OpenTelemetry backend and wants Claude requests and agent sessions in it, including building evals from real traces.

Replaces

Hand-written logging wrappers around the SDK, or a tracing proxy in front of the API.

Watch out for

Beta. Metadata only by default. Full content lands under your platform's own retention, and HIPAA, FERPA or K-12 restricted organizations get metadata only from destinations.

Also available from

AWS: AgentCore Observability
Microsoft: Foundry client-side tracing
Google: Agent Platform observability
OpenAI: Agents API OTLP trace export

Where it runs

Vendor API: YesBedrock: YesAzure: YesVertex AI: Yes

Claim quality

No performance claim

Sources: 1

Foundry Observability · Sep 24, 2026 · Evals and observability

Recurring trace patterns become findings with evidence, likely causes and proposed next steps, which may shorten agent review once traces accumulate.

Bring it up when

The client already has agent traces on Foundry but relies on manual trace reading to decide what to change, and wants a triage aid before running evaluations.

Replaces

Could reduce manual trace review. It does not replace evaluations, since Microsoft advises testing proposed changes against the same input.

Watch out for

Preview. Tracing and Insights can add monitoring or judge-model costs, and scans are billable, so budget before enabling across an agent fleet.

Also available from

AWS: AgentCore insights
LangChain: LangSmith Engine
Braintrust: Patterns

Where it runs

Vendor API: NoBedrock: NoAzure: YesVertex AI: No

Claim quality

No performance claim

Sources: 1 2

Braintrust · Sep 3, 2026 · Evals and observability

Scheduled agent runs search traces for failure modes and cost trends, saving each finding with its evidence, which shifts discovery away from reading traces by hand.

Bring it up when

The client traces agents in production and its scorers only catch known failures, so new failure modes or cost shifts surface late or through user complaints.

Replaces

Manual trace review and one-off analysis scripts used to discover what to score next.

Watch out for

Public preview, and support is limited to two model families (GPT-6 Astra and GPT-5.6). Each automation's investigation adds to cost.

Also available from

LangChain: LangSmith Engine
Microsoft: Foundry Observability Insights
AWS: AgentCore insights

Where it runs

Not tied to one model platform

Claim quality

No performance claim

Sources: 1 2 3

Arize Phoenix · Jun 26, 2026 · Evals and observability

Evals can live in an existing test suite and fail CI on aggregate quality such as mean correctness or latency, which removes the need for a separate eval runner.

Bring it up when

Engineers already use pytest, Vitest or Jest and want LLM quality regressions to block a merge without adopting a separate eval workflow.

Replaces

Standalone eval scripts run by hand, or custom CI steps that parse eval output.

Watch out for

The TypeScript Vitest and Jest integration is labelled beta; the Python pytest plugin is not. Results record to Phoenix, so a Phoenix instance is part of the design.

Also available from

LangChain: LangSmith pytest integration
Langfuse: Experiments in CI/CD
Braintrust: Evals in CI (eval-action, bt eval)

Where it runs

Not tied to one model platform

Claim quality

No performance claim

Sources: 1

Amazon Bedrock AgentCore · Jun 17, 2026 · Evals and observability

Recommendations, batch evaluation and A/B tests are GA, including for agents outside AWS. Turning traces into ranked failure patterns is still preview.

Bring it up when

Client has live agents with no error signal for wrong behavior, and prompt changes ship on judgment rather than on measured comparison.

Replaces

Manual trace review and ad hoc prompt tuning, or a custom pipeline for agent evaluation and A/B testing.

Watch out for

Failure, intent and trajectory insights are still preview, in 13 Regions, while recommendations, batch evaluation and A/B tests are GA in 14.

Also available from

Microsoft: Foundry Observability Insights
LangChain: LangSmith Engine
Braintrust: Patterns

Where it runs

Vendor API: NoBedrock: YesAzure: NoVertex AI: No

Claim quality

No performance claim

Sources: 1

LangSmith · May 13, 2026 · Evals and observability

Issue discovery, root-cause analysis and draft pull requests can run from production traces, with people reviewing the output instead of triaging traces by hand.

Bring it up when

The team traces agents in LangSmith, failures pile up faster than engineers can diagnose them, and it would accept agent-written pull requests under human review.

Replaces

Manual trace triage, hand-written root-cause notes, and adding regression examples to datasets by hand.

Watch out for

Priced in LSUs with a default cap of 750 LSU per month. Self-hosted needs chart 0.16.0+ and a license entitlement; fix validation and Red Teaming are Cloud-only betas.

Also available from

Braintrust: Patterns
Microsoft: Foundry Observability Insights
AWS: AgentCore insights

Where it runs

Not tied to one model platform

Claim quality

No performance claim

Sources: 1 2 3

Gemini Enterprise Agent Platform · Oct 26, 2026 · Platforms

Plain-language rules can check agent tool calls at runtime across Gemini, Claude, OpenAI-format, Mistral and self-hosted vLLM traffic behind Agent Gateway.

Bring it up when

Client has agents that issue refunds, change records or send data, and needs intent and business-rule checks on each tool call across several model providers.

Replaces

Hard-coded guard logic in agent code and some human approval steps for tool calls.

Watch out for

Preview. Verdicts come from an LLM and may be wrong. Streamed model responses aren't supported, so a streamed tool call might not be blocked.

Also available from

AWS: Policy in AgentCore

Where it runs

Vendor API: NoBedrock: NoAzure: NoVertex AI: Yes

Claim quality

No performance claim

Sources: 1 2 3

Amazon Bedrock Guardrails · Oct 20, 2026 · Platforms

Grounding checks can now cover conversational answers and responses that reason over the source, not only strict summarization or Q&A, at the same price.

Bring it up when

Client uses or rejected contextual grounding because strict checks and limited language support did not fit a conversational assistant.

Replaces

Custom hallucination scoring, or Classic tier grounding on conversational apps.

Watch out for

Requires cross-Region inference with a guardrail profile, which matters for residency. Scores differ from Classic, so retune thresholds; subset of Regions.

Also available from

Microsoft: Groundedness detection
Google: Check Grounding API

Where it runs

Vendor API: NoBedrock: YesAzure: NoVertex AI: No

Claim quality

Vendor-reported: AWS says Standard is more robust than Classic, with more consistent scores for differently worded responses.

Sources: 1

Claude Enterprise · Oct 20, 2026 · Platforms

Enterprise admins can block prompts and tool calls before inference through their own security server. Failure handling sets whether an outage blocks or allows traffic.

Bring it up when

A Claude Enterprise client needs its own security tooling to stop prompts before inference, rather than auditing them afterward with the Compliance API.

Replaces

After-the-fact review through Compliance API polling, or network proxies placed in front of Claude apps.

Watch out for

Claude Enterprise only: API organizations are out of scope, and it is not on Bedrock or Google Cloud. Verdicts are allow or deny, with no redaction. Image-only content is not inspected.

Also available from

Microsoft: Copilot Studio threat detection

Where it runs

Vendor API: YesBedrock: NoAzure: Not confirmedVertex AI: No

Claim quality

No performance claim

Sources: 1

Shieldstral · Aug 4, 2026 · Models

Moderation policies can be written in plain language at inference time, with no retraining, and the classifier runs on a client's own GPU.

Bring it up when

Client self-hosts LLMs, must moderate text and images under its own rules, and does not want to send content to a hosted moderation API.

Replaces

Could retire a hosted moderation API or a fixed-taxonomy guard model that must be retrained for new policies.

Watch out for

Apache 2.0, so no license limit. It answers one yes/no question per request, so a client must set thresholds and fan out queries for several policies.

Also available from

OpenAI: gpt-oss-safeguard
Meta: Llama Guard 4
NVIDIA: Nemotron 3.5 Content Safety

Where it runs

Vendor API: Not confirmedBedrock: NoAzure: Not confirmedVertex AI: Not confirmed

Claim quality

Vendor-reported: Mistral says it matches or outperforms open guard models up to 7x its size.

Sources: 1 2

Amazon Bedrock Guardrails · Jun 16, 2026 · Platforms

Agents can run chosen safety checks at each step with no guardrail resource, getting numeric scores so the application sets thresholds and actions.

Bring it up when

Client has multi-step agents, and one guardrail applied to all prompts and tool results is too blunt or hard to version across workflows.

Replaces

One-size guardrail resources applied per call, or separate detectors for prompt attacks and PII.

Watch out for

Detect-only: enforcement is application code, so a missed check goes unblocked. Seven Regions at launch and a default quota of 1,500 requests per minute.

Also available from

Microsoft: Azure AI Content Safety
Google: Model Armor

Where it runs

Vendor API: NoBedrock: YesAzure: NoVertex AI: No

Claim quality

No performance claim

Sources: 1 2

Azure AI Foundry · May 31, 2026 · Platforms

Microsoft provisions the network boundary for agent outbound traffic and managed private endpoints, so app teams need less virtual network design work.

Bring it up when

The client's regulated agent projects need controlled egress, and teams lack the capacity to design and run custom virtual networks for each project.

Replaces

Could replace a custom virtual network set up per project for agent isolation.

Watch out for

Isolation can't be disabled later and custom networks need redeployment to convert. FQDN outbound rules create a managed Azure Firewall with its own charges.

Also available from

None confirmed

Where it runs

Vendor API: NoBedrock: NoAzure: YesVertex AI: No

Claim quality

No performance claim

Sources: 1 2

Claude API · Nov 2, 2026 · Platforms

Agent loops can send smaller requests instead of the whole history, but the client still needs a full-replay path for when a thread is gone.

Bring it up when

Long agent loops resend a growing history each turn, and upload time or time to first token is a visible problem on the Claude API, Platform on AWS, Google (Anthropic managed) or Foundry.

Replaces

Client-side resending of the full conversation on every turn.

Watch out for

Beta and not ZDR eligible. Best-effort: expires 5 minutes after the last request, then returns 404 and you resend the full conversation. No on-demand compaction.

Also available from

OpenAI: Conversation state (Responses API)
Google: Interactions API stored turns

Where it runs

Vendor API: YesBedrock: NoAzure: YesVertex AI: No

Claim quality

Vendor-reported: Vendor says time to first token stays lower and steadier as the conversation grows; no figure given.

Sources: 1

Amazon Bedrock AgentCore · Oct 22, 2026 · Inference and serving

One governed inference endpoint can reach several providers and fail over on throttling or outages, so clients drop their own retry and failover code.

Bring it up when

Client runs several model providers, sees capacity errors in production, and maintains retry or routing code in each application.

Replaces

Per-client retry and failover code, or a self-hosted LLM gateway used mainly for routing and failover.

Watch out for

The chain allows 1 to 2 alternates, there is no fallback once streaming has started, and every attempt bills as a gateway invocation.

Also available from

Microsoft: Foundry model router

Where it runs

Vendor API: NoBedrock: YesAzure: NoVertex AI: No

Claim quality

No performance claim

Sources: 1 2

Decisions API · Oct 22, 2026 · Platforms

Routing, classification and rubric checks can return typed answers instead of parsed prose, with up to 400 questions per request.

Bring it up when

The client uses a general model plus structured output for high-volume predicates, fixed choices or rubric scoring and pays for latency it does not need.

Replaces

Could retire classification prompts with JSON parsing on the Responses or Chat Completions API.

Watch out for

Only gpt-6-luna is supported, so accuracy on the client's questions rests on one model and cannot be switched to a higher tier.

Also available from

None confirmed

Where it runs

Vendor API: YesBedrock: NoAzure: YesVertex AI: Not confirmed

Claim quality

Vendor-reported: OpenAI says 10x faster than the Responses API.

Sources: 1 2

Braintrust · Oct 20, 2026 · Evals and observability

Long evals can run on provider batch APIs and resume later by polling or webhook, recording results in one experiment, which changes how large eval suites are costed.

Bring it up when

Evals are large enough that synchronous model calls are slow or costly, and the client can accept results arriving later from a provider batch API.

Replaces

Custom scripts that submit batch jobs and stitch the results back into experiment records.

Watch out for

Public preview. Submitting each case alone creates one batch per case, and some providers cap batch counts; Python offers grouped batches via WorkflowBatchTask.

Also available from

None confirmed

Where it runs

Not tied to one model platform

Claim quality

No performance claim

Sources: 1 2 3

OpenAI API · Oct 20, 2026 · Inference and serving

Shared instructions and context can stay cached across gaps of hours, which changes the cost math for bursty or low-frequency workloads.

Bring it up when

The client has long stable instructions or tool definitions, but traffic gaps exceed the default 30-minute cache lifetime, so it keeps paying for writes.

Replaces

Could retire keep-warm requests that exist only to hold a prefix in cache.

Watch out for

12-hour writes cost 2.5x the input price against 1.25x for the 30-minute default, so short-gap traffic can cost more on 12-hour caching.

Also available from

None confirmed

Where it runs

Vendor API: YesBedrock: NoAzure: YesVertex AI: Not confirmed

Claim quality

No performance claim

Sources: 1 2

vLLM · Oct 18, 2026 · Inference and serving

Long-context serving can hold more in the same GPU memory, and on SM100 GLM-5.x now gets an FP8 KV cache by default, so latency baselines can move after an upgrade.

Bring it up when

Client self-hosts with vLLM, long contexts limit concurrency, and buying more GPUs is the only plan on the table.

Replaces

Could reduce the need for extra GPUs bought only for KV cache capacity.

Watch out for

The trade is latency: halved KV memory on GLM-5.x came with a 7-11% TTFT cost in the PR benchmark. Results are model and GPU specific.

Also available from

NVIDIA: TensorRT-LLM FP8 KV cache

Where it runs

Not tied to one model platform

Claim quality

Vendor-reported: vLLM reports 2.3-5.5% more throughput for GLM-5.x FP8 KV cache on SM100 in a PR benchmark.

Sources: 1 2

OpenAI API · Oct 8, 2026 · Inference and serving

A paid tier reduces time between output tokens, which matters for interactive products where users feel streaming speed.

Bring it up when

The client's product is limited by token streaming speed rather than time to first token, and the extra cost per request can be justified by the use case.

Replaces

Could replace Fast mode or a switch to a smaller model chosen only for speed.

Watch out for

GPT-6 Astra Ultrafast has US residency and global processing only, and Ultrafast has its own rate limits apart from Standard and Fast modes.

Also available from

Anthropic: Claude fast mode

Where it runs

Vendor API: YesBedrock: YesAzure: Not confirmedVertex AI: Not confirmed

Claim quality

Vendor-reported: OpenAI said up to 14x faster than Standard when announcing it for GPT-5.6 Sol.

Sources: 1 2

vLLM · Oct 5, 2026 · Inference and serving

Engine restarts no longer have to reload weights from disk, which lowers the cost of rolling config changes and upgrades on large models.

Bring it up when

Client self-hosts large models with vLLM and slow restarts make upgrades, config changes or recovery expensive.

Replaces

Could retire warm standby replicas kept mainly to avoid reload time.

Watch out for

Snapshot restore through CRIU is experimental and limited to a TP1 engine. The weight-cache daemon holds GPU memory across restarts.

Also available from

None confirmed

Where it runs

Not tied to one model platform

Claim quality

No performance claim

Sources: 1 2 3

vLLM · Sep 22, 2026 · Inference and serving

KV pages can spill from GPU to cheaper tiers instead of forcing recomputation, which changes the sizing math for long-context, high-reuse workloads.

Bring it up when

Client runs long-context or agent workloads with repeated prefixes on vLLM, and cache misses drive GPU spend and latency.

Replaces

Could retire external prompt caching layers or GPU capacity bought only to hold KV cache.

Watch out for

HiSparse is for sparse-MLA decode, and vLLM rejects it at startup with cudagraph_mode=FULL. The throughput figure applies to GPU-resident contexts only.

Also available from

NVIDIA: Dynamo with SGLang HiCache

Where it runs

Not tied to one model platform

Claim quality

Vendor-reported: vLLM reports up to 58% more throughput on GLM-5.3-NVFP4 on B200 with long, GPU-resident contexts at high concurrency.

Sources: 1 2 3 4

Azure AI Foundry · Sep 1, 2026 · Platforms

Data residency now has a clearer price: EU Data Zone is 9% above Global, non-US Regional 7-16%, and the new APAC Data Zone 20%. Global pricing is unchanged.

Bring it up when

The client runs on EU Data Zone or non-US Regional deployments, or is building a business case that assumes today's prices carry over to new models.

Replaces

Nothing direct: new pricing. It may reopen the choice between Global and geography-bound deployment where residency is a preference, not a requirement.

Watch out for

Existing EU Data Zone and non-US Regional PTU customers pay more now. Standard customers pay the premium only when moving to models launched on or after September 1.

Also available from

AWS: Bedrock geographic cross-Region
OpenAI: Data residency regional processing

Where it runs

Vendor API: NoBedrock: NoAzure: YesVertex AI: No

Claim quality

No performance claim

Sources: 1 2

Deep Agents · Jul 29, 2026 · Frameworks

Default agents send fewer input tokens per turn, but planning todos are now opt-in, backend factories are gone and tool output formats changed, so upgrades need a code check.

Bring it up when

Client runs or is considering the Deep Agents harness and token spend per turn matters, especially if it relies on default planning todos or parses raw tool output.

Replaces

Trimming the default prompt and tool descriptions by hand to reduce per-turn input tokens.

Watch out for

Breaking: planning todos are off unless TodoListMiddleware is restored, backend shims are removed, and code that parses raw ls, glob or read_file output needs updating.

Also available from

None confirmed

Where it runs

Not tied to one model platform

Claim quality

Vendor-reported: LangChain says a default-agent turn's input tokens drop 65% (5,395 to 1,895), with no quality regression on its own evaluation suite.

Sources: 1 2

LangGraph · May 12, 2026 · Frameworks

Long threads no longer re-serialize a full message list at each step. Deep Agents 0.6 turns this on for message history and agent files, which makes rollback a design decision.

Bring it up when

Client has long-running LangGraph or Deep Agents threads where checkpoint storage or read time is growing, and can accept a beta channel format that blocks simple rollback.

Replaces

Full-state checkpoints on growing channels, and custom trimming of message history to limit checkpoint size.

Watch out for

In beta. Once a thread uses it, downgrading leaves checkpoints unreadable by earlier versions, and the bulk reducer must be associative or state can differ on rebuild.

Also available from

None confirmed

Where it runs

Not tied to one model platform

Claim quality

Vendor-reported: LangChain says DeltaChannel cuts checkpoint overhead for long-running threads; no figure stated.

Sources: 1 2 3 4

Gemini API · Apr 1, 2026 · Inference and serving

Synchronous calls can now be priced by urgency, so some background traffic may not need the Batch API and some critical traffic may not need reserved capacity.

Bring it up when

Client has both user-facing and background workloads on the Gemini API and wants different cost and latency treatment without running two integrations.

Replaces

Batch API pipelines for some background work, and some reserved capacity for traffic spikes.

Watch out for

Preview. Flex is 50% off but can be preempted, with no fallback to Standard, so retries are the client's job. Priority costs 75-100% more than Standard.

Also available from

AWS: Bedrock service tiers
OpenAI: Flex processing
Microsoft: Foundry priority processing

Where it runs

Vendor API: YesBedrock: NoAzure: NoVertex AI: Yes

Claim quality

No performance claim

Sources: 1 2 3

Amazon Bedrock · Nov 26, 2025 · Inference and serving

Inference can now be bought by need: reserved tokens-per-minute, premium Priority, discounted Flex or Standard, on models that support each tier, instead of one on-demand mode.

Bring it up when

Client mixes customer-facing and offline traffic on one on-demand path, or is weighing Provisioned Throughput for predictable capacity.

Replaces

Sending all work to on-demand, or capacity planning around quota increases alone.

Watch out for

Reserved needs an account team and has minimums of 100,000 input and 10,000 output TPM. Billing runs until the reservation is deleted; tier support varies by model.

Also available from

OpenAI: Flex processing
Google: Gemini API Flex inference
Microsoft: Foundry priority processing

Where it runs

Vendor API: NoBedrock: YesAzure: NoVertex AI: No

Claim quality

Vendor-reported: AWS says Reserved targets 99.5% uptime and Priority gives up to 25% better output tokens per second on most models.

Sources: 1 2

pg_pinecone · Oct 27, 2026 · Vector databases

Keeps tables and pgvector queries in Postgres, swaps the search engine under them, and can move one table's search to a Pinecone index and back with a SQL call.

Bring it up when

The client keeps vectors in self-managed Postgres, wants to keep its SQL and row store, and is weighing a move to a separate vector database as the table grows.

Replaces

Moving rows out of Postgres into a separate vector database, plus the sync pipeline that keeps the two stores aligned.

Watch out for

Needs self-managed PostgreSQL 17 or 18 with superuser access. The remote engine and full-text search also need a server setting and a Pinecone index.

Also available from

Microsoft: pg_diskann on Azure PostgreSQL
Google: AlloyDB ScaNN index

Where it runs

Not tied to one model platform

Claim quality

No performance claim

Sources: 1 2

LangSmith · Oct 26, 2026 · Evals and observability

A new AWS data plane now lands on SmithDB unless ClickHouse is requested, and an existing one can't yet turn ClickHouse off, so the storage choice is made at creation.

Bring it up when

The client is creating a LangSmith data plane on AWS and must decide whether to operate SmithDB or ClickHouse for traces, including the SDK changes SmithDB queries require.

Replaces

ClickHouse as the default trace store for new AWS data planes, plus the legacy run query SDK methods.

Watch out for

Existing data planes keep ClickHouse and can't yet turn it off. New SDK methods are required to query traces on SmithDB.

Also available from

None confirmed

Where it runs

Not tied to one model platform

Claim quality

Vendor-reported: LangChain says SmithDB makes core LangSmith experiences faster; no figure was checked.

Sources: 1 2 3

NVIDIA NIM · Oct 15, 2026 · Inference and serving

NIM wraps vLLM and SGLang in validated containers with patch cadence, which turns running raw open-source engines into a build-or-buy decision for self-hosters.

Bring it up when

Client must meet security patch expectations for self-hosted models and is weighing a supported container against running vLLM or SGLang directly.

Replaces

Could retire an in-house vLLM or SGLang image pipeline and its tuning work.

Watch out for

Production Branch and all NVIDIA enterprise support need an NVIDIA AI Enterprise subscription. Some newer models, such as Kimi K3.1 Pro, run in experimental containers.

Also available from

AWS: AWS Deep Learning Containers for vLLM

Where it runs

Not tied to one model platform

Claim quality

Vendor-reported: NVIDIA says throughput optimizations for 28 NIMs reach up to 2.1x relative to the upstream open-source engine.

Sources: 1 2 3 4

Pinecone BYOC · Oct 15, 2026 · Vector databases

Running the data plane in the client's cloud account no longer means sizing dedicated read nodes per index; many indexes can share a pool.

Bring it up when

The client has data residency or network isolation rules that rule out a shared vector service, and its index count made dedicated capacity per index hard to justify.

Replaces

Self-operating an open-source vector database in the client's account, or one dedicated read nodes index per workload.

Watch out for

Enterprise plan only. The pool starts at two nodes, billed even with no on-demand indexes, on top of a platform fee and cloud costs. Integrated embedding isn't supported.

Also available from

Qdrant: Qdrant Hybrid Cloud
Weaviate: Weaviate Bring Your Own Cloud

Where it runs

Not tied to one model platform

Claim quality

No performance claim

Sources: 1 2 3

Langfuse · Aug 17, 2026 · Evals and observability

A single denormalized observations table replaces read-time joins, and monitors, full-text search and v2 APIs come with it. Self-hosters must upgrade infrastructure first.

Bring it up when

The client runs Langfuse v3 itself and is deciding when to upgrade, or is weighing self-hosted Langfuse against a managed option given the infrastructure it requires.

Replaces

Langfuse v3 deployments and their read-time joins between traces and observations.

Watch out for

v4 requires ClickHouse 25.12 or later and PostgreSQL 15 or later. v3 gets security patches through January 2027, with no forced self-hosted cutover.

Also available from

None confirmed

Where it runs

Not tied to one model platform

Claim quality

Vendor-reported: Langfuse says table loads drop from seconds to milliseconds and large-project dashboards load at least 10x faster.

Sources: 1 2 3

Muse Glimmer · Aug 12, 2026 · Models

A text and image agent model can run on one consumer GPU or a Mac with no API key, and it is Apache 2.0, which is a different license from Muse Spark Open.

Bring it up when

Client wants always-on agents on laptops or single GPUs for privacy or per-token cost reasons, and needs a license with no custom usage conditions.

Replaces

Could retire a hosted API for simple agent loops that do not need a larger model.

Watch out for

Apache 2.0, so no custom license terms. No audio input or output, and quantizing to fit a 24-32 GB device may reduce performance.

Also available from

Google: Gemma 4
NVIDIA: Nemotron 3.5 Bolt
OpenAI: gpt-oss

Where it runs

Vendor API: NoBedrock: NoAzure: YesVertex AI: Not confirmed

Claim quality

Vendor-reported: Meta says it leads on the agentic work it was built for, with methodology in its own report.

Sources: 1 2 3 4

Foundry Local · Apr 9, 2026 · Inference and serving

Apps can run models on the user's device with an OpenAI-compatible API, so requests need no cloud dependency and data can stay on the device.

Bring it up when

The client ships desktop or edge software that must work offline or keep data local, and small models could handle part of the workload.

Replaces

Could replace bundling and tuning a custom local inference runtime in an application.

Watch out for

The catalog is deliberately limited to models small enough to distribute. Later bring-your-own-model support covers only compatible ONNX Runtime GenAI models.

Also available from

Google: LiteRT-LM

Where it runs

Not tied to one model platform

Claim quality

No performance claim

Sources: 1 2 3 4

Langfuse · Nov 4, 2026 · Evals and observability

Swapping the base URL and API key routes OpenAI and Anthropic calls through Langfuse, with per-user keys, a model allowlist, and a trace for every call.

Bring it up when

Teams share provider keys across apps or coding agents, and the client can't attribute model cost to a person, key or team, or can't instrument an app it doesn't own.

Replaces

Shared provider keys and per-app instrumentation added only to capture cost and usage.

Watch out for

Public beta: supports OpenAI and Anthropic only, doesn't translate between API formats, and behavior, routes and headers can still change.

Also available from

LangChain: LangSmith LLM Gateway
Braintrust: Braintrust Gateway
Microsoft: AI gateway in API Management
AWS: AgentCore Gateway model routing

Where it runs

Not tied to one model platform

Claim quality

No performance claim

Sources: 1 2

Gemini API · Oct 26, 2026 · Platforms

Prompts and responses from new paid projects are kept unless opted out, so data handling reviews done on older projects may no longer describe new ones.

Bring it up when

Client has data retention or privacy commitments and creates new Gemini API projects on the paid tier, including through automation.

Replaces

Nothing direct: new capability.

Watch out for

Default applies to new projects only; older ones keep their setting. Logs expire after 55 days by default and hold full prompts and responses.

Also available from

None confirmed

Where it runs

Vendor API: YesBedrock: NoAzure: NoVertex AI: No

Claim quality

No performance claim

Sources: 1 2

Claude Models on Google Cloud (Anthropic managed) · Oct 22, 2026 · Platforms

Google Cloud clients now choose between two Claude offerings, and the choice decides who processes data and which features are available.

Bring it up when

The client is committed to Google Cloud billing and IAM and needs Claude API features, and is deciding between the Google-managed and Anthropic-managed offerings.

Replaces

A separate Anthropic contract and API keys alongside Google Cloud, or the Google-managed offering where its feature set falls short.

Watch out for

Anthropic, not Google, is the data processor, and some services run outside Google Cloud. It creates a new organization, and existing ZDR does not carry over.

Also available from

None confirmed

Where it runs

Vendor API: YesBedrock: NoAzure: NoVertex AI: No

Claim quality

No performance claim

Sources: 1

Claude API · Oct 20, 2026 · Platforms

Platforms and resellers can limit a safety enforcement action to one end customer. Without a profile, one customer can trigger action against the whole organization.

Bring it up when

The client runs a platform, gateway or resold API, and one end customer's traffic could trigger a restriction or ban across its whole Anthropic organization.

Replaces

Separate Anthropic organizations per customer as a blast-radius control, or accepting organization-wide enforcement risk.

Watch out for

Google-managed and Amazon Bedrock have no profile endpoints, so profiles must be created in a Claude API or Platform on AWS org. Legacy Bedrock is not supported.

Also available from

OpenAI: Safety identifiers

Where it runs

Vendor API: YesBedrock: YesAzure: YesVertex AI: Yes

Claim quality

No performance claim

Sources: 1

vLLM · Sep 22, 2026 · Inference and serving

Self-hosted text output can carry a keyed watermark at the engine level, with per-request opt-out.

Bring it up when

Client has a policy or provenance need to mark AI-generated text and serves open-weight models itself.

Replaces

Nothing direct: new capability.

Watch out for

The detection endpoint shipped as an example, and per-request opt-out exists, so detection and governance around it remain the client's to build.

Also available from

Google: SynthID Text

Where it runs

Not tied to one model platform

Claim quality

No performance claim

Sources: 1

Amazon Bedrock AgentCore · Aug 31, 2026 · Platforms

A private catalog of agents, MCP servers, tools and skills with approval workflow, search, org-wide sharing and auto-detection of AgentCore resources.

Bring it up when

Client has many teams shipping agents and MCP servers, and the platform or security group cannot say what exists or who approved it.

Replaces

Spreadsheets, wikis or a home-built catalog of agents and MCP servers.

Watch out for

Available in nine AWS Regions, so a data residency requirement outside them rules it out. Check the list for the client.

Also available from

Google: Agent Registry
Microsoft: Agent 365 registry

Where it runs

Vendor API: NoBedrock: YesAzure: NoVertex AI: No

Claim quality

No performance claim

Sources: 1 2

Amazon Bedrock AgentCore · Aug 6, 2026 · Platforms

Policy can now judge a tool call against earlier actions in the session, such as requiring approval first, and gateways can cap traffic by user or group.

Bring it up when

Client is moving agents into sensitive workflows such as payments and needs enforced step order, approvals and per-user limits outside agent code.

Replaces

Sequence checks, approval gates and per-user throttles written into agent or tool code.

Watch out for

Rate limits fail open by default if the rate limit service is unavailable, so AWS says not to rely on them alone as a security boundary.

Also available from

Google: Semantic governance policies

Where it runs

Vendor API: NoBedrock: YesAzure: NoVertex AI: No

Claim quality

No performance claim

Sources: 1 2

Gemini Enterprise Agent Platform · Jun 18, 2026 · Platforms

Gives one enforcement point for agent, tool and MCP traffic with a default-deny access model, plus identity and content filters, for Agent Runtime and Gemini Enterprise.

Bring it up when

Client security team needs a catalog of agents and MCP servers plus central allow and deny rules, and is comparing that with building its own gateway.

Replaces

Per-agent network rules, ad hoc MCP allowlists and home-built agent proxies.

Watch out for

VPC Service Controls covers only gateways created after Sep 8, 2026 using a connectivity template. Registry lacks Access Transparency logs and Access Approval.

Also available from

AWS: AgentCore Gateway and Agent Registry
Microsoft: AI gateway in API Management

Where it runs

Vendor API: NoBedrock: NoAzure: NoVertex AI: Yes

Claim quality

No performance claim

Sources: 1 2 3 4 5

MCP extensions · Jun 18, 2026 · Protocols

Admins can provision MCP server access through the corporate identity provider, so users get connected servers at login without per-app OAuth consent.

Bring it up when

A security or IT team wants MCP access to follow identity provider groups and revocation, and repeated per-server OAuth consent is blocking rollout.

Replaces

Per-user, per-server OAuth consent flows and manually distributed MCP server credentials.

Watch out for

Opt-in and needs client, IdP and server support. Vendors track different ID-JAG draft revisions and some list it as early access, so confirm the exact product combination.

Also available from

None confirmed

Where it runs

Not tied to one model platform

Claim quality

No performance claim

Sources: 1 2

Claude Platform on AWS · May 11, 2026 · Platforms

AWS clients now choose between Bedrock, where AWS operates inference, and this offering, where Anthropic does. The API surface and data processor differ between them.

Bring it up when

The client buys through AWS and wants Managed Agents, Skills or same-day features, and is deciding whether AWS must be the operator and data processor.

Replaces

A direct Anthropic contract beside AWS, or Bedrock where it was chosen only for billing and IAM.

Watch out for

Anthropic, not AWS, processes inference data. Vendor sends FedRAMP High, IL4, IL5 and HIPAA-ready clients to Bedrock. No fast mode. Older workspaces might run outside AWS.

Also available from

None confirmed

Where it runs

Vendor API: YesBedrock: NoAzure: NoVertex AI: No

Claim quality

No performance claim

Sources: 1