Agent Toolkit Plugin

The Agent Toolkit plugin gives a language model conversational control over a TomTom map and its services. You bring the LLM and optionally a chat UI — the plugin handles the tool loop, agent state, and map coordination.

import { MapAgentChat } from './chat';
import { AnalyticsControlPanel } from './ui';
import { useMapAgent } from './useMapAgent';

export function App() {
    const { transport, analyticsState } = useMapAgent();

    return (
        <div className="absolute inset-0 flex flex-row-reverse gap-2 bg-(--sdk-surface-0) p-2 max-sm:flex-col max-sm:gap-0 max-sm:p-0">
            {/* `id="sdk-map"` is required — MapLibre attaches to the DOM node by ID. */}
            <div
                id="sdk-map"
                className="relative flex-1 overflow-hidden rounded-[20px] bg-(--sdk-surface-1) max-sm:min-h-0 max-sm:basis-1/2 max-sm:rounded-none"
            >
                {analyticsState && (
                    <AnalyticsControlPanel analytics={analyticsState.analytics} module={analyticsState.module} />
                )}
            </div>
            {transport ? (
                <MapAgentChat transport={transport} />
            ) : (
                <div className="flex w-[380px] flex-col bg-(--sdk-surface-0) max-sm:w-full">
                    <div className="p-4 text-(--sdk-text-medium)">Initializing assistant...</div>
                </div>
            )}
        </div>
    );
}

What it is

@tomtom-org/maps-sdk-plugin-agent-toolkit is a headless plugin built on Vercel AI SDK . It wires a TomTomMap instance and TomTom services to a curated set of LLM-callable tools, and manages the multi-step tool loop via ToolLoopAgent.

You bring:

• An LLM model instance — any Vercel AI SDK provider (OpenAI, Anthropic, Azure, Google, etc.)
• A chat UI, or programmatic calls to the agent

The plugin provides:

• 60+ built-in tools covering search, routing, traffic, isochrones, map control, MapLibre access, and code-generated analysis
• An intent classifier that selects the right tool subset per message
• Plugin state that persists places, routes, waypoints, reachable ranges, traffic, and derived geometries across tool calls
• An experimental JavaScript-as-input pattern for tools whose work doesn’t fit a fixed schema (see Code generation )
• A composable API to add, remove, or replace any tool

How it works

Every user message goes through two phases before a response is sent:

1. Intent classification

Before the model sees the full tool list, a lightweight pre-pass selects only the tools relevant to this turn. Fewer tools in the prompt means fewer wrong choices and lower token usage.

The classifier is built from each tool’s classificationPrompt — a one-liner that describes when to activate that tool. When you add a custom tool, it participates in classification automatically as long as it has a classificationPrompt.

The default classifier reuses your main model. For cost-sensitive deployments, swap it for a smaller model:

1
import { createMapAgent, createDefaultClassifier } from '@tomtom-org/maps-sdk-plugin-agent-toolkit';
2
import { openai } from '@ai-sdk/openai';
3

4
const agent = createMapAgent(map, {
5
    model: openai('gpt-4o'),
6
    classifier: createDefaultClassifier({ model: openai('gpt-4o-mini') }),
7
});

Or disable classification entirely (all tools always visible):

1
const agent = createMapAgent(map, {
2
    model: openai('gpt-4o'),
3
    classifier: false,
4
});

2. Tool loop

The model calls tools — geocode, search, route, show on map — in sequence or in parallel until it can form a complete answer. Tools never return raw GeoJSON to the model; they return compact summaries (counts, names, labels, status). Full geospatial data is kept in plugin state for follow-up use.

State persists across turns, so the user can say “add a stop after the first one” or “what were those restaurants again?” without repeating context. The agent retrieves prior results from its internal stores via dedicated recall tools.

The tool registry

The plugin ships a DEFAULT_TOOLS registry — a flat record of named ToolEntry objects. Each entry defines what the tool does and how the agent should use it:

1
{
2
    description: string;          // Full tool description for the model
3
    inputSchema: ZodType;         // Validated input parameters
4
    outputSchema?: ZodType;       // Structured output (improves reliability)
5
    execute: (input, state) => Promise<any>;
6

7
    // Classifier metadata
8
    classificationPrompt?: string; // One-liner: when to activate this tool
9
    tags?: string[];               // Category labels (e.g. 'route', 'traffic')
10
    examples?: string[];           // Usage examples shown in the help tool
11
    examplePrompts?: string[];     // Natural-language prompts shown in the help tool
12
    relatedTools?: string[];       // Tools often used together
13
    dependsOn?: string[];          // Tools that must run first
14
}

Tools are grouped by what they do, not which API they wrap:

Some tools accept JavaScript instead of a fixed parameter list — the process*, analyse*, and executeMaplibreCode families. They are covered in their own section: Code generation .

The help tool

The built-in help tool is available to the model at runtime. When a user asks “what can you do?” or “show me routing examples”, the model calls help to surface tool descriptions, usage examples, and example prompts filtered by tag. Users discover capabilities through natural conversation — no documentation browsing required.

Tools that read state vs. fetch data

A key design distinction: some tools fetch fresh data from TomTom services; others read from plugin state. The recall tools (recallPlaces, recallRoutes, recallRanges) exist because tool results are not retained in conversation history. If a user asks “what were those places?” an hour into a session, the model calls a recall tool — it does not guess or hallucinate prior results.

Plugin state

The plugin maintains structured state across the full conversation, organized by feature area:

State is shared across all tools. A place resolved by locatePlace is immediately available to addWaypointsToRoute without the model having to pass it along — tools read directly from shared stores. This prevents the hallucination-prone “read then pass” pattern.

Three of the slices (places, routing, ranges) expose an entry mode: multiple (default) lets several entries render side-by-side; single keeps only the most recent one and auto-drops older entries when flipped. The model surfaces this through the setEntryMode tool when the user asks to “only show one at a time” or “let me overlay multiple”.

You can inspect live state at any time from your application:

1
agent.state.routing.currentRoutes;         // most recent route results
2
agent.state.places.latestPlace;            // most recent place results
3
agent.state.places.entries;                // full history of place entries
4
agent.state.trafficIncidents.entries;      // fetched incident entries
5
agent.state.customGeometries.entries;      // derived polygon entries
6
agent.state.baseMap.mapLibreMap;           // raw MapLibre Map instance

Code generation

A handful of tools take JavaScript as a parameter instead of a fixed argument list. The model writes the code; the plugin compiles it via new AsyncFunction(...paramNames, code) and invokes it with state and a few injected helpers.

This is what lets the agent answer open-ended questions — “bar chart of POI categories”, “union the boundaries of these cities”, “focus the camera on the worst traffic on the route” — without the toolkit needing a dedicated parameter for every variation. The tool surface stays flat: one tool per intent, arbitrary computation inside.

Where it shows up

Execution model

Each tool that accepts code builds an AsyncFunction whose parameters are a fixed set of injected identifiers, then calls it with state. There is no isolation — the function runs synchronously in the host realm. In a browser app that means the code has access to window, document, fetch, dynamic import(), and every global your bundle ships with. The only thing the plugin actively does to the supplied string is strip lines like const turf = require('@turf/turf') that LLMs frequently prepend out of habit (they would otherwise collide with the injected parameter and throw at parse time).

The body works like any other async function body: top-level await is fine, locals and helper consts are fine, and the body must end with return …;. Each tool injects a different set of identifiers:

Two helpers are shared by every process* / analyse* tool:

• turf — @turf/turf : area, bbox, centroid, union, intersect, buffer, distance, booleanPointInPolygon, pointsWithinPolygon, clustersDbscan, convex, etc.
• h3 — h3-js : hex-grid math (latLngToCell, polygonToCells, gridDisk, cellToBoundary).

The remaining identifiers (places, routes, geometries, incidents, placesByEntry, routeUtils, previous, now, log, map) are tool-specific — see the table above for which tool exposes what, and the per-tool API reference for parameter shapes.

The plugin does not pass anything else in. To bring in new data, the agent calls a separate tool first (discoverPlaces, setRoute, findReachableAreas, …) and passes the resulting entry id into the next code call.

Threat model

Because the code is not isolated, the security boundary is the language model and the prompt path you let users drive, not the runtime. Treat model-emitted code the same way you would treat a <script> tag controlled by your LLM provider:

• A jailbroken or compromised model can call fetch(...), read cookies, write to localStorage, exfiltrate user data, or mutate the page DOM via executeMaplibreCode (which holds a live map reference).
• Prompt injection from a tool result (e.g. a place description containing adversarial text) can steer the model into generating malicious code that the model then asks to execute via these tools.

If your deployment context does not tolerate this (e.g. you’re embedding the agent in a page with privileged session cookies), remove the code-generation tools at agent creation:

1
const agent = createMapAgent(map, {
2
    model: openai('gpt-4o'),
3
    tools: {
4
        processPlaces: false,
5
        processGeometries: false,
6
        processRoutes: false,
7
        analysePlaces: false,
8
        analyseRoutes: false,
9
        analyseGeometries: false,
10
        analyseIncidents: false,
11
        executeMaplibreCode: false,
12
    },
13
});

Output contract

analyse* tools accept an outputFormat:

• "json" (default) — any JSON-serializable value. Rendered as text in the chat.
• "chart" — a Chart.js ChartConfiguration ({ type, data, options? }). The chat UI passes the config straight to new Chart(ctx, analysis).

process* tools return a structured envelope. For example, processPlaces accepts { places?, placeConnections?, geometries?, fitOnMap? } — the side effects (new entry written, connections drawn, polygons rendered, camera moved) are derived from which keys the code set.

Return values must be pure JSON — no functions, classes, circular references, or DOM/Map references. The result crosses a JSON boundary before the model sees it, so undefined, NaN, and sparse arrays are normalized via JSON.parse(JSON.stringify(...)) to keep subsequent turns valid.

Example

A user asks “Bar chart of how many places of each category we have.” The model picks analysePlaces and emits:

1
analysePlaces({
2
    name: 'category-bar',
3
    outputFormat: 'chart',
4
    code: `
5
        const counts = {};
6
        for (const p of places.features)
7
            for (const c of (p.properties.poi?.categories ?? []))
8
                counts[c] = (counts[c] ?? 0) + 1;
9
        const entries = Object.entries(counts).sort((a, b) => b[1] - a[1]);
10
        return {
11
            type: 'bar',
12
            data: {
13
                labels: entries.map((e) => e[0]),
14
                datasets: [{ label: 'Places', data: entries.map((e) => e[1]) }],
15
            },
16
        };
17
    `,
18
});

places is a GeoJSON FeatureCollection built from the most recent places entry — or the merged features of every entry listed in placesEntryIDs. The plugin runs the body, validates that the return value is a Chart.js config, stores it on each source entry as _analysis['category-bar'], and returns it to the model. The chat UI receives the same config and renders the chart inline.

Installation

1
npm install @tomtom-org/maps-sdk-plugin-agent-toolkit

Add a Vercel AI SDK provider for your chosen model:

1
npm install ai @ai-sdk/openai   # or @ai-sdk/anthropic, @ai-sdk/azure, etc.

Quick start

1
import { TomTomConfig } from '@tomtom-org/maps-sdk/core';
2
import { TomTomMap } from '@tomtom-org/maps-sdk/map';
3
import { createMapAgent } from '@tomtom-org/maps-sdk-plugin-agent-toolkit';
4
import { openai } from '@ai-sdk/openai';
5
import { DirectChatTransport } from 'ai';
6
import { useChat } from 'ai/react';
7

8
TomTomConfig.instance.put({ apiKey: 'YOUR_API_KEY' });
9

10
const map = new TomTomMap({ mapLibre: { container: 'map' } });
11

12
const agent = createMapAgent(map, { model: openai('gpt-4o') });
13

14
// Wire to any Vercel AI SDK chat interface
15
const { messages, sendMessage } = useChat({
16
    transport: new DirectChatTransport({ agent }),
17
});

Once wired, users can interact naturally:

• “Route from Berlin to Munich avoiding tolls”
• “Show coffee shops near the map center”
• “What traffic incidents are on this route?”
• “Switch to dark mode”
• “How far is the second stop?”

Customizing tools

Tools are resolved by merging the DEFAULT_TOOLS registry with your tools option. You can add, replace, or remove individual entries — the rest remain untouched.

Remove a default tool

1
const agent = createMapAgent(map, {
2
    model: openai('gpt-4o'),
3
    tools: { setLanguage: false },
4
});

Replace a default tool

Pass a ToolEntry with the same key to swap the built-in implementation with yours:

1
const agent = createMapAgent(map, {
2
    model: openai('gpt-4o'),
3
    tools: {
4
        getCurrentLocation: {
5
            description: 'Returns the user position from the company fleet system.',
6
            inputSchema: z.object({}),
7
            execute: async (_input, state) => {
8
                const position = await fleetApi.getDriverPosition();
9
                state.baseMap.userPosition = position;
10
                return { position };
11
            },
12
            classificationPrompt: 'Get the driver\'s current GPS position from the fleet system.',
13
            tags: ['location'],
14
        },
15
    },
16
});

Add a custom tool (BYOD blending)

Bring your own data sources alongside the built-in TomTom tools. Custom tools receive the same state object, so they can hand results to the built-in display flow:

1
import { z } from 'zod';
2
import type { Place } from '@tomtom-org/maps-sdk/core';
3
import type { ToolEntry } from '@tomtom-org/maps-sdk-plugin-agent-toolkit';
4

5
const getFleetVehicle: ToolEntry = {
6
    description: 'Get the current map position of a fleet vehicle by ID and add it to the places history.',
7
    classificationPrompt: 'Locate or display a fleet vehicle on the map by its ID.',
8
    inputSchema: z.object({ vehicleId: z.string().describe('The fleet vehicle identifier') }),
9
    execute: async ({ vehicleId }, state) => {
10
        const position = await fleetApi.getPosition(vehicleId);
11
        const place: Place = {
12
            type: 'Feature',
13
            geometry: { type: 'Point', coordinates: position },
14
            properties: { name: `Vehicle ${vehicleId}` },
15
        };
16
        // Append to the places history; the model can then show it via updatePlacesDisplay,
17
        // route to it via setRoute, or recall it later through recallPlaces.
18
        const entryId = state.places.addPlaceResult(place, `Vehicle ${vehicleId}`);
19
        return { vehicleId, entryId, position };
20
    },
21
    tags: ['location'],
22
    examplePrompts: ['Where is vehicle TT-001?', 'Show fleet vehicle on the map'],
23
};
24

25
const agent = createMapAgent(map, {
26
    model: openai('gpt-4o'),
27
    tools: { getFleetVehicle },
28
});

The custom tool participates in intent classification automatically. If the user says “Where is vehicle TT-001?”, the classifier activates getFleetVehicle alongside any other tools needed to answer the question.

Start from a blank slate

Set includeDefaultTools: false to begin with no built-in tools. Useful when building a narrowly scoped agent that should not stray into general map control. Re-add specific defaults from the DEFAULT_TOOLS registry:

1
import { createMapAgent, DEFAULT_TOOLS } from '@tomtom-org/maps-sdk-plugin-agent-toolkit';
2

3
const agent = createMapAgent(map, {
4
    model: openai('gpt-4o'),
5
    includeDefaultTools: false,
6
    tools: {
7
        getFleetVehicle,
8
        locatePlace: DEFAULT_TOOLS.locatePlace,    // selectively re-add a built-in
9
        flyTo: DEFAULT_TOOLS.flyTo,
10
    },
11
});

Extending the system prompt

Append to the built-in prompt

1
const agent = createMapAgent(map, {
2
    model: openai('gpt-4o'),
3
    systemPromptSuffix: 'Always respond in Spanish. Never show more than 5 places at once.',
4
});

Full replacement

Import BASE_SYSTEM_PROMPT as a baseline to extend rather than starting from scratch. The base prompt contains coordinate order rules, tool-usage guidance, and response formatting instructions that are important for reliability:

1
import { createMapAgent, BASE_SYSTEM_PROMPT } from '@tomtom-org/maps-sdk-plugin-agent-toolkit';
2

3
const agent = createMapAgent(map, {
4
    model: openai('gpt-4o'),
5
    systemPrompt: BASE_SYSTEM_PROMPT + `
6

7
ADDITIONAL INSTRUCTIONS:
8
- This is a logistics application. Prioritize route efficiency over scenery.
9
- Always show estimated arrival times in responses.
10
- When a vehicle ID is mentioned, call getFleetVehicle before anything else.
11
`,
12
});

Per-tool classification prompts

If your custom tool is being activated too broadly or not activated when it should be, tune its classificationPrompt. This is the one-liner the classifier uses to decide whether a user message needs this tool:

1
// Too broad — activates for any location question
2
classificationPrompt: 'Get fleet vehicle data.'
3

4
// Precise — activates only for explicit vehicle ID references
5
classificationPrompt: 'Locate a fleet vehicle by its ID (e.g. "TT-001"); not for general location queries.'

Custom state slices

If your custom tools need to share data across calls, extend ToolState with your own slice. Implement reset() to participate in agent.destroy() cleanup:

1
import type { ToolState, StateSlice } from '@tomtom-org/maps-sdk-plugin-agent-toolkit';
2

3
class FleetState implements StateSlice {
4
    vehicles: Map<string, VehiclePosition> = new Map();
5
    reset() { this.vehicles.clear(); }
6
}
7

8
interface MyState extends ToolState {
9
    fleet: FleetState;
10
}
11

12
const agent = createMapAgent<MyState>(map, {
13
    model: openai('gpt-4o'),
14
    state: { fleet: new FleetState() },
15
    tools: { getFleetVehicle: myFleetTool },
16
});
17

18
// Access live state from your application
19
agent.state.fleet.vehicles;
20
agent.state.routing.currentRoutes;

Observing classification

Use onClassify to inspect which tools were selected for each turn — useful for debugging classification behaviour or building a dev panel:

1
const agent = createMapAgent(map, {
2
    model: openai('gpt-4o'),
3
    onClassify: (result) => {
4
        if (result) {
5
            console.log('Selected tools:', result.activeToolNames);
6
            console.log('Classification took:', result.timeMs, 'ms');
7
        }
8
    },
9
});

Cleanup

1
agent.destroy();  // resets all built-in and custom state slices that implement reset()

API Reference

Agent Toolkit Plugin API Reference

Related guides

• How the SDK works — the Map and Services bundles the agent builds on
• Places module — module used internally for displaying search results
• Routing module — module used internally for rendering routes
• Traffic — traffic tools used by the agent
• AI in the SDK — overview of AI capabilities in and around the SDK