Batch Routing processing speed and quotas

Purpose

This page explains how QPS (queries per second) limits affect the processing speed of the Batch Routing service, what processing times to expect, and how to get QPS limits that fit your traffic.

All processing times on this page are estimates. They are not guaranteed and are not a service level commitment. The actual processing time depends on your other traffic, the type of your routing requests, the current load on Batch Routing and on the underlying Routing API suite endpoints, and temporary service conditions. A batch can take longer than the estimate.

QPS limits

Batch Routing uses two separate QPS limits.

QPS limit	What counts toward the limit	What the limit controls
Batch Routing QPS	Requests to the Batch Routing endpoints. Each submission and each download is one request, whatever the number of batch items.	How many Batch Routing requests you can send per second. Requests over the limit get HTTP `429`.
QPS of the underlying Routing API suite endpoint (for example, Routing API QPS for Calculate Route batch items)	Batch items. Batch Routing executes every batch item as a request to the underlying Routing API suite endpoint, so every batch item counts toward the QPS of that endpoint.	How fast the batch items are processed.

The processing speed depends mainly on the QPS of the underlying Routing API suite endpoint. A higher Batch Routing QPS lets you send more submissions and downloads, but it does not process batch items faster.

Your batches share the QPS of the underlying Routing API suite endpoint with each other and with your direct requests to that endpoint. For example, if you have a Routing API QPS of 10 and you send direct Calculate Route requests at 6 QPS, your batches get the remaining 4 QPS. It also works the other way: your batches can use the whole QPS, and then your direct requests can get HTTP 429.

Default values and how to change them

Unless you have agreed other values with TomTom, the default QPS values apply. For example, the default Routing API QPS is 5. With this value, Batch Routing processes at most 5 Calculate Route batch items per second, which is 18,000 per hour. For all default values, see QPS limits.

The default values are meant for getting started and for small volumes. If your batches take longer than you need, request a higher QPS of the underlying Routing API suite endpoint. With a Routing API QPS of 50, Calculate Route batch items are processed up to about ten times faster than with the default QPS.

Contracted (enterprise) customers can request a QPS change in MyTomTom, as described in QPS management.
On Freemium and Pay As You Grow accounts, the default values apply. To discuss a higher QPS, Contact Sales.

To find the QPS you need, see Estimate the QPS you need.

How batches are processed

Batch Routing puts your batches in a queue and processes them in about the order in which they were submitted. A new batch waits until the batch items of the batches submitted before it have been sent. The processing of consecutive batches can overlap: the next batch can start while the responses for the previous batch are still arriving.

This applies to synchronous and asynchronous batches together. A synchronous batch also waits for your asynchronous batches submitted before it.

Batch Routing processes batch items at most at the QPS of the underlying Routing API suite endpoint. It tries to use your full QPS, but the actual rate can be lower. Batch Routing limits how many batches and batch items it processes at the same time. Long routes and non-default routing options take longer to calculate, so complex routing requests can lower the throughput.

Batch Routing retries batch items that were rejected because of the QPS limit or that failed with a temporary server error. An HTTP 200 response does not mean that every batch item succeeded, so check the status of each batch item in the response.

Estimating processing time

You can estimate the processing time of a batch as:

1
processing time = (batch items in older batches + batch items in the batch) / QPS of the underlying Routing API suite endpoint

If the older batches are already partly processed, the batch can be completed earlier. It can also be completed later, because the actual rate can be lower than your QPS, for example when the routing requests are complex or the service is under high load.

The examples below assume the default Routing API QPS of 5 and no direct Calculate Route requests. In each example, there are no other batches in the queue.

Example	Estimated processing time
1 batch of 100 routes	100 / 5 = 20 seconds, plus a small overhead.
5 batches of 20 routes, submitted at the same time	The batches get the QPS one after another. The first batch is completed after 20 / 5 = 4 seconds, the last one after 100 / 5 = 20 seconds.
1 batch of 10 routes, submitted while one of your batches of 700 routes is being processed	The small batch waits until all batch items of the large batch have been sent. If the large batch has just started, this takes 710 / 5 = 142 seconds.
1 batch of 100 routes, with direct Calculate Route requests at 3 QPS	The batch gets 2 QPS, so 100 / 2 = 50 seconds.
100,000 routes in any number of batches	100,000 / 5 = 20,000 seconds (about 5.6 hours). With a Routing API QPS of 50, it takes about 33 minutes. The batch size does not change this.

The batch size changes the number of submissions and downloads, not the number of batch items processed per second.

Synchronous and asynchronous batches

	Synchronous batch	Asynchronous batch submission
Use for	Lightweight routing requests with quickly-expiring results, for example because of changing traffic conditions	Large volumes of relatively complex routing requests
Maximum number of batch items	100	700
Timeout	HTTP `408` if processing takes longer than 60 seconds	No timeout. The batch stays in the queue until it is processed.
Results	In the response	From the Asynchronous batch download endpoint, for 14 days

Both modes use the QPS of the underlying Routing API suite endpoint in the same way. A synchronous batch times out if it is not completed in 60 seconds. Because it waits for your older batches, this can happen if you submit many synchronous batches at once, if the routing requests are complex, or if you also have asynchronous batches in the queue.

Requests to the Asynchronous batch download endpoint are blocking long-poll requests. HTTP 202 means that the batch is still waiting in the queue or being processed. It is not an error. Retry the request by following the Location header.

Recommendations

Use separate applications for different types of traffic

In one application, your direct requests and all your batches use the same QPS of the underlying Routing API suite endpoint (see QPS limits). They compete for it in both directions. Your batches can use the whole QPS, so your direct requests can get HTTP 429. Your direct requests leave less QPS for your batches, so your batches take longer. In the same way, your asynchronous batches can use the QPS that your synchronous batches need.

To avoid this, use a separate application, each with its own API key, for:

direct requests to the underlying Routing API suite endpoints, which you send without Batch Routing,
synchronous batches, for live traffic that needs results immediately,
asynchronous batches, for large volumes of relatively complex routing requests.

Estimate the QPS for each application separately, and request it for the API key of that application. If you have only one type of traffic, one application is enough.

Estimate the QPS you need

Estimate the QPS of the underlying Routing API suite endpoint from the number of batch items and the processing time you can accept:

1
QPS = (batch items / acceptable processing time in seconds) x 1.2

The factor 1.2 covers the processing overhead of Batch Routing. For example, to process 2 batches of 500 batch items each in 50 seconds, you need (1,000 / 50) x 1.2 = 24 QPS. If several batches must be completed in the same time, add up their batch items. For complex routing requests, add more headroom.

The formula counts batch items only. If you send direct requests from the same application as your batches, add their peak rate to the result. For example, with direct Calculate Route requests at 6 QPS, you need 24 + 6 = 30 QPS. Mixing direct requests and batches in one application is not recommended. See Use separate applications for different types of traffic.

For the Batch Routing QPS, count the submissions and downloads that you send per second.

Use the result as a starting point and request it as described in Default values and how to change them. Then measure the processing time of your batches and adjust the QPS if needed. For help with estimating the QPS, Contact Sales or your account manager.

Keep the queue short

Keep only a few batches in the queue at a time, in total for all the systems that submit them. A second batch in the queue keeps your QPS in use while you download the results of the first one. More batches in the queue do not make processing faster. They only make each batch wait longer.

To get results faster, request a higher QPS of the underlying Routing API suite endpoint. Changing the batch size or sending more batches at the same time does not process more batch items per second.

To calculate travel times and distances between many origins and destinations, consider the Matrix Routing v2 API instead of many Calculate Route batch items.