Skip to content

Commit c147463

Browse files
committed
feat: add tool choice with .choice()
Backport of the tool-choice feature to the v1 line (AI SDK v6). Add a `.choice()` method that resolves the AI SDK `toolChoice` from the same `{ messages, steps, context }` values as activation. `inferTools()` now returns `toolChoice` alongside `tools` and `activeTools`, spreadable into generateText/streamText and prepareStep. Last-call wins; a resolver returning `undefined` omits `toolChoice` (auto). The resolved value is passed through as-is, no activeTools validation.
1 parent fb332db commit c147463

5 files changed

Lines changed: 466 additions & 22 deletions

File tree

README.md

Lines changed: 71 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -12,16 +12,17 @@
1212

1313
</div>
1414

15-
This library provides a type-safe API to manage [`activeTools`](https://ai-sdk.dev/docs/reference/ai-sdk-core/generate-text#active-tools) for [`generateText()`](https://ai-sdk.dev/docs/reference/ai-sdk-core/generate-text) and [`streamText()`](https://ai-sdk.dev/docs/reference/ai-sdk-core/stream-text) in the AI SDK.
15+
This library provides a type-safe API to manage [`activeTools`](https://ai-sdk.dev/docs/reference/ai-sdk-core/generate-text#active-tools) and [`toolChoice`](https://ai-sdk.dev/docs/ai-sdk-core/tools-and-tool-calling#tool-choice) for [`generateText()`](https://ai-sdk.dev/docs/reference/ai-sdk-core/generate-text) and [`streamText()`](https://ai-sdk.dev/docs/reference/ai-sdk-core/stream-text) in the AI SDK.
1616

1717
### Why?
1818

19-
The AI SDK provides an `activeTools` parameter to control which tools the model can use at any given time. However, managing tool activation becomes complex when you need to:
19+
The AI SDK provides an `activeTools` parameter to control which tools the model can use, and a `toolChoice` parameter to force which tool is called. However, managing this becomes complex when you need to:
2020

2121
- **Statically activate/deactivate tools**: Some tools should be inactive by default and only available after being explicitly activated
2222
- **Dynamically infer tool activation**: Some tools should be activated based on runtime context like the conversation history
23+
- **Force tool choice**: Sometimes the model should be required to call a specific tool, or any tool, based on context
2324

24-
This library wraps standard AI SDK `tool()` definitions with chainable activation methods and resolves `tools` and `activeTools` for any AI SDK function.
25+
This library wraps standard AI SDK `tool()` definitions with chainable activation and choice methods and resolves `tools`, `activeTools`, and `toolChoice` for any AI SDK function.
2526

2627
### Installation
2728

@@ -220,6 +221,50 @@ const toolSet = createToolSet({ tools })
220221
.activateWhen('cancel_order', ({ messages }) => hasUnfulfilledOrders(messages));
221222
```
222223

224+
### Tool Choice
225+
226+
The AI SDK can steer tool calls with [`toolChoice`](https://ai-sdk.dev/docs/ai-sdk-core/tools-and-tool-calling#tool-choice), which controls whether and which tool the model must call. `.inferTools()` resolves a `toolChoice` value alongside `tools` and `activeTools`, so a single tool set drives activation and choice together.
227+
228+
Use `.choice()` with a constant for static choices or use a resolver to decide dynamically:
229+
230+
```typescript
231+
const toolSet = createToolSet({ tools })
232+
.deactivate(['cancel_order'])
233+
// Force the model to call one of the active tools this turn
234+
.choice('required')
235+
// Or resolve dynamically, e.g. force unauthenticated users into search
236+
.choice(({ context }) => (context?.isAuthenticated ? 'auto' : { type: 'tool', toolName: 'search' }));
237+
238+
const { tools, activeTools, toolChoice } = toolSet.inferTools();
239+
240+
const result = await generateText({ model, tools, activeTools, toolChoice, prompt: 'Show me my orders' });
241+
```
242+
243+
A `.choice()` entry is either a constant [`ToolChoice`](https://ai-sdk.dev/docs/ai-sdk-core/tools-and-tool-calling#tool-choice) (`'auto'`, `'required'`, `'none'`, or `{ type: 'tool', toolName }` with `toolName` typed to your tools) or a **resolver**. The resolver runs inside `.inferTools()` with your toolset values (`{ messages, steps, context }`) and returns a `ToolChoice`, or `undefined` to leave the choice unconstrained (the AI SDK then defaults to `'auto'`). Since it sets a single value, it follows the same **last-call wins** rule as activation: the last entry decides, and a resolver returning `undefined` does not fall back to an earlier one, so express any fallback inside the resolver itself.
244+
245+
> [!NOTE]
246+
> `toolChoice` is resolved independently of `activeTools` and passed through as-is. Because the AI SDK filters tools by `activeTools` first, forcing `{ type: 'tool', toolName }` for a tool you have deactivated leaves the model unable to see it and surfaces as a provider error, so keep a forced tool active.
247+
248+
Resolving from `steps` makes it easy to guide a multi-step run. Call `.inferTools()` inside `prepareStep()` to recompute `toolChoice` (and `activeTools`) after each step, for example to force a tool on the first step and then hand control back to the model:
249+
250+
```typescript
251+
import { generateText, stepCountIs } from 'ai';
252+
253+
const toolSet = createToolSet({ tools }).choice(({ steps }) =>
254+
steps?.length === 0 ? { type: 'tool', toolName: 'search' } : 'auto',
255+
);
256+
257+
const { tools } = toolSet;
258+
259+
const result = await generateText({
260+
model,
261+
tools,
262+
stopWhen: stepCountIs(10),
263+
prompt: 'Find my orders and cancel the unfulfilled one',
264+
prepareStep: ({ steps }) => toolSet.inferTools({ steps }),
265+
});
266+
```
267+
223268
### Immutable vs Mutable
224269

225270
By default, `createToolSet()` returns an **immutable** tool set, that means every method returns a new instance and the original is never modified. This is ideal when the tool set is created once in the global scope and shared across requests:
@@ -420,32 +465,47 @@ Conditionally deactivate tools. The predicate receives `{ messages, steps, conte
420465
toolSet.deactivateWhen('search', ({ messages }) => messages && messages.length > 10);
421466
```
422467

468+
#### `.choice(entry)`
469+
470+
Set the toolset's `toolChoice` to a constant `ToolChoice` (`'auto'`, `'none'`, `'required'`, or `{ type: 'tool', toolName }`) or a resolver. A resolver receives `{ messages, steps, context }` and returns a `ToolChoice`, or `undefined` to leave it unconstrained (omitted). Last-call wins. The resolved `toolChoice` is passed through as-is; keep a forced tool active so the AI SDK does not filter it out.
471+
472+
```ts
473+
// Constant
474+
toolSet.choice('required');
475+
toolSet.choice({ type: 'tool', toolName: 'search' });
476+
477+
// Resolver — force a tool on the first step, then let the model decide
478+
toolSet.choice(({ steps }) => (steps?.length === 0 ? { type: 'tool', toolName: 'search' } : 'auto'));
479+
```
480+
423481
#### `.inferTools(input?)`
424482

425-
Evaluate all predicates and return `{ tools, activeTools }`, directly spreadable into `generateText()` or `streamText()`. The input is optional; all fields are optional. Predicates receive `undefined` for fields not provided.
483+
Evaluate all predicates and the tool-choice entry and return `{ tools, activeTools, toolChoice }`, directly spreadable into `generateText()` or `streamText()`. The input is optional; all fields are optional. Predicates and resolvers receive `undefined` for fields not provided.
426484

427485
- `input` (optional):
428486
- `messages` (optional), the current conversation messages
429487
- `steps` (optional), the completed steps of the current run (the `StepResult` array from `prepareStep`)
430-
- `context` (optional), arbitrary values passed to predicates
488+
- `context` (optional), arbitrary values passed to predicates and the tool-choice resolver
489+
490+
The result also includes `toolChoice` when set via `.choice()` (otherwise `undefined`).
431491

432492
```ts
433493
// Static-only (no predicates)
434-
const { tools, activeTools } = toolSet.inferTools();
494+
const { tools, activeTools, toolChoice } = toolSet.inferTools();
435495

436496
// With messages
437-
const { tools, activeTools } = toolSet.inferTools({ messages });
497+
const { tools, activeTools, toolChoice } = toolSet.inferTools({ messages });
438498

439499
// With context
440-
const { tools, activeTools } = toolSet.inferTools({ context: { isAdmin: true } });
500+
const { tools, activeTools, toolChoice } = toolSet.inferTools({ context: { isAdmin: true } });
441501

442502
// With steps (e.g. inside prepareStep)
443-
const { tools, activeTools } = toolSet.inferTools({ steps });
503+
const { tools, activeTools, toolChoice } = toolSet.inferTools({ steps });
444504

445505
// With both
446-
const { tools, activeTools } = toolSet.inferTools({ messages, context });
506+
const { tools, activeTools, toolChoice } = toolSet.inferTools({ messages, context });
447507

448-
const result = await generateText({ model, tools, activeTools, messages });
508+
const result = await generateText({ model, tools, activeTools, toolChoice, messages });
449509
```
450510

451511
#### `.clone(options?)`

examples/tool-choice.ts

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
/**
2+
* Tool choice — force which tool the model must call, resolved from the same
3+
* `{ messages, steps, context }` values that drive activation.
4+
*
5+
* `.choice()` sets the AI SDK's `toolChoice`. It accepts a constant
6+
* (`'auto'` | `'none'` | `'required'` | `{ type: 'tool', toolName }`) or a resolver
7+
* that runs inside `inferTools()` and returns one of those (or `undefined` to leave
8+
* it unconstrained). `inferTools()` returns `toolChoice` alongside `activeTools`, and
9+
* both spread straight into `generateText` and into `prepareStep`'s return.
10+
*
11+
* Here the resolver forces `search` on the first step (so the run always starts by
12+
* gathering data), then hands control back to the model with `'auto'`. The toolChoice
13+
* is recomputed per step from the run's `steps`.
14+
*
15+
* Uses ai-test-kit's MockLanguageModel so it runs without any API keys: the mock
16+
* honors the `toolChoice` it receives — it calls `search` while forced, then answers.
17+
*
18+
* Run: pnpm tsx examples/tool-choice.ts
19+
*/
20+
import { generateText, stepCountIs, tool } from 'ai';
21+
import { Language, MockLanguageModel } from 'ai-test-kit/language';
22+
import { z } from 'zod';
23+
import { createToolSet } from '../src/tool-set.js';
24+
25+
const search = tool({
26+
description: 'Search the web for information',
27+
inputSchema: z.object({ query: z.string() }),
28+
execute: async ({ query }) => ({ results: [`result for "${query}"`] }),
29+
});
30+
31+
const answer = tool({
32+
description: 'Give the final answer to the user',
33+
inputSchema: z.object({ text: z.string() }),
34+
execute: async ({ text }) => ({ answered: text }),
35+
});
36+
37+
const tools = { search, answer };
38+
39+
/**
40+
* Force `search` on the first step (no steps completed yet), then let the model
41+
* decide for every step after. Returning `'auto'` is the explicit fallback —
42+
* with last-call wins, a resolver returning `undefined` would simply omit
43+
* `toolChoice`, which also defaults to `'auto'`.
44+
*/
45+
const toolSet = createToolSet({ tools }).choice(({ steps }) =>
46+
steps?.length ? 'auto' : { type: 'tool', toolName: 'search' },
47+
);
48+
49+
/** Mock model (ai-test-kit): obey the forced toolChoice, otherwise produce final text. */
50+
let searchCallId = 0;
51+
const model = MockLanguageModel.from({
52+
doGenerate: async ({ toolChoice }) => {
53+
const forcedTool = toolChoice?.type === 'tool' ? toolChoice.toolName : undefined;
54+
if (forcedTool === 'search') {
55+
return Language.result([
56+
Language.toolCall({
57+
toolCallId: `call-${++searchCallId}`,
58+
toolName: 'search',
59+
input: { query: 'ai sdk tool choice' },
60+
}),
61+
]);
62+
}
63+
return Language.result('Here is my answer based on what I found.');
64+
},
65+
});
66+
67+
const result = await generateText({
68+
model,
69+
tools,
70+
stopWhen: stepCountIs(10),
71+
prompt: 'Research the topic and then answer.',
72+
prepareStep: ({ stepNumber, steps }) => {
73+
/** Recompute activeTools + toolChoice for this step from the run so far. */
74+
const { activeTools, toolChoice } = toolSet.inferTools({ steps });
75+
console.log(
76+
`step ${stepNumber} → toolChoice: ${JSON.stringify(toolChoice)} → activeTools: [${activeTools.join(', ')}]`,
77+
);
78+
return { activeTools, toolChoice };
79+
},
80+
});
81+
82+
console.log('\nTool-call trace:');
83+
for (const [i, step] of result.steps.entries()) {
84+
for (const call of step.toolCalls) {
85+
console.log(` step ${i} → called ${call.toolName}(${JSON.stringify(call.input)})`);
86+
}
87+
}
88+
89+
console.log('\nFinal text:', result.text);

src/__tests__/tool-set.test-d.ts

Lines changed: 71 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
import type { InferUITool, ModelMessage, StepResult, UIMessage } from 'ai';
1+
import type { InferUITool, ModelMessage, PrepareStepResult, StepResult, UIMessage } from 'ai';
22
import { tool } from 'ai';
33
import { describe, expectTypeOf, test } from 'vitest';
44
import { z } from 'zod';
@@ -407,6 +407,62 @@ describe('immutable toolset', () => {
407407
});
408408
});
409409

410+
describe('choice', () => {
411+
test('should accept the string toolChoice values', () => {
412+
const toolSet = createToolSet({ tools: TOOLS });
413+
toolSet.choice('auto');
414+
toolSet.choice('none');
415+
toolSet.choice('required');
416+
});
417+
418+
test('should accept a tool toolChoice with a known tool name', () => {
419+
const toolSet = createToolSet({ tools: TOOLS });
420+
toolSet.choice({ type: 'tool', toolName: 'plain' });
421+
toolSet.choice({ type: 'tool', toolName: 'cancel' });
422+
});
423+
424+
test('should reject a tool toolChoice with an unknown tool name', () => {
425+
const toolSet = createToolSet({ tools: TOOLS });
426+
// @ts-expect-error — 'unknown' is not in TOOLS
427+
toolSet.choice({ type: 'tool', toolName: 'unknown' });
428+
});
429+
430+
test('should type the resolver input as the inferTools input', () => {
431+
type MyCtx = { isAdmin: boolean };
432+
const toolSet = createToolSet<typeof TOOLS, UIMessage, MyCtx>({ tools: TOOLS });
433+
toolSet.choice((input) => {
434+
expectTypeOf(input.context).toEqualTypeOf<MyCtx | undefined>();
435+
expectTypeOf(input.messages).toEqualTypeOf<Array<UIMessage> | undefined>();
436+
return 'required';
437+
});
438+
});
439+
440+
test('should allow the resolver to return undefined', () => {
441+
const toolSet = createToolSet({ tools: TOOLS });
442+
toolSet.choice(({ steps }) => (steps?.length ? 'auto' : undefined));
443+
});
444+
445+
test('should type toolChoice on the inferTools result', () => {
446+
const toolSet = createToolSet({ tools: TOOLS });
447+
const result = toolSet.inferTools();
448+
expectTypeOf(result.toolChoice).toEqualTypeOf<
449+
| 'auto'
450+
| 'none'
451+
| 'required'
452+
| { type: 'tool'; toolName: 'plain' | 'calc' | 'cancel' | 'edit' | 'archive' }
453+
| undefined
454+
>();
455+
});
456+
457+
test('should spread the inferTools result into a prepareStep return', () => {
458+
const toolSet = createToolSet({ tools: TOOLS }).choice('required');
459+
// The resolved `activeTools` + `toolChoice` are accepted by PrepareStepResult.
460+
const prepareStep = ({ steps }: { steps: Array<StepResult<typeof TOOLS>> }): PrepareStepResult<typeof TOOLS> =>
461+
toolSet.inferTools({ steps });
462+
expectTypeOf(prepareStep).toBeFunction();
463+
});
464+
});
465+
410466
describe('clone', () => {
411467
test('should return immutable toolset by default', () => {
412468
const toolSet = createToolSet({ tools: TOOLS });
@@ -557,6 +613,20 @@ describe('mutable toolset', () => {
557613
});
558614
});
559615

616+
describe('choice', () => {
617+
test('should return this', () => {
618+
const toolSet = createToolSet({ tools: TOOLS, mutable: true });
619+
const result = toolSet.choice('required');
620+
expectTypeOf(result).toEqualTypeOf(toolSet);
621+
});
622+
623+
test('should reject a tool toolChoice with an unknown tool name', () => {
624+
const toolSet = createToolSet({ tools: TOOLS, mutable: true });
625+
// @ts-expect-error — 'unknown' is not in TOOLS
626+
toolSet.choice({ type: 'tool', toolName: 'unknown' });
627+
});
628+
});
629+
560630
describe('clone', () => {
561631
test('should return immutable toolset by default', () => {
562632
const toolSet = createToolSet({ tools: TOOLS, mutable: true });

0 commit comments

Comments
 (0)