Processing speed and quotas
Purpose
This page explains the quotas of the Matrix Routing v2 API, how they affect the processing speed, what processing times to expect, and how to get quotas that fit your traffic.
Quotas
The Matrix Routing v2 API uses two separate quotas.
| Quota | What counts toward the quota | What the quota controls |
|---|---|---|
| QPS (queries per second) | Requests to the Matrix Routing v2 endpoints. Each submission, status request and download is one request, whatever the matrix size. | How many requests you can send per second. Requests over the limit get HTTP 429. |
| Resource utilization limit | Route calculations of your matrix jobs that run at the same time. | How many routes are calculated at the same time, and so how fast matrix jobs with many routes are processed. |
Default values and how to change them
The default value of both quotas is 10. The default values are meant for getting started and for small volumes. If your matrix jobs take longer than you need, request a higher resource utilization limit.
- Contracted (enterprise) customers can request a QPS change in MyTomTom, as described in QPS management.
- On Freemium and Pay As You Grow accounts, the default QPS applies. To discuss a higher QPS, Contact Sales.
- To request a higher resource utilization limit, Contact Sales or your account manager.
For all default QPS values, see QPS limits. To find the values you need, see Estimate the quotas you need.
Resource utilization limit
The resource utilization limit sets how many route calculations of your matrix jobs can run at the same time. The higher the limit, the more routes are calculated at the same time, so matrix jobs with many routes are completed sooner. A higher limit does not make a single route calculation faster.
Large matrices have more routes to calculate. Long distances and non-default options make each route calculation take longer, so the route calculations use the limit for longer. With the same limit, such matrix jobs take longer. A higher limit helps large matrices, because more of their routes are calculated at the same time. It does not shorten a single long route calculation.
The processing speed depends mainly on the resource utilization limit. A higher QPS lets you send more submissions, status requests and downloads, but it does not process matrix jobs faster.
All your matrix jobs, synchronous and asynchronous, share the resource utilization limit. With the Multi-Customer feature, each sub-customer has its own limits, so sub-customers do not slow each other down.
How matrix jobs are processed
The Matrix Routing v2 API puts your matrix jobs in a queue and processes them in about the order in which they were submitted. A new matrix job waits until the matrix jobs submitted before it are completed. This applies to synchronous and asynchronous matrix jobs together: a synchronous matrix job also waits for your asynchronous matrix jobs submitted before it.
The processing time depends on the matrix size and on how complex the matrix is to calculate. Matrices with departAt or arriveAt set to any, traffic set to historical, routeType set to fastest, travelMode set to car or truck, and no other parameters are usually the fastest to calculate. For the options with a big impact on the processing time, see the FAQ.
Estimating processing time
No single formula gives the processing time of a matrix job, because it depends too much on the content of the matrix. The examples below show the basic rules. They assume that each matrix job needs the whole resource utilization limit, and that there are no other matrix jobs in the queue, unless the example says otherwise.
| Example | Estimated processing time |
|---|---|
| 1 asynchronous matrix job, with no other matrix jobs in the queue | The matrix job gets the whole resource utilization limit. The processing time depends on the matrix size, the distances between origins and destinations, and the routing options. This is the time that one matrix job needs alone, which the other examples are compared to. |
| 2 asynchronous matrix jobs with the same matrix size and options, submitted at the same time | They get the resource utilization limit one after another. The first one is completed after about the time that one matrix job needs alone, the second one after about twice that time. |
| 1 small asynchronous matrix job, submitted while one of your large asynchronous matrix jobs is being processed | The small matrix job waits until the large one is completed, so it takes about as long as the rest of the large matrix job. |
| The same matrix job with a higher resource utilization limit | Large matrix jobs are processed faster, because more of their routes are calculated at the same time. With twice the limit, a large asynchronous matrix job can take as little as about half the time. Small matrix jobs gain less, and a single long route calculation does not get faster. |
The same matrix job with non-default options, for example avoid or travelMode set to pedestrian | Each route takes longer to calculate, so the matrix job can take many times longer than with the default options. |
| 1 synchronous matrix job, submitted while one of your large asynchronous matrix jobs is being processed | The synchronous matrix job waits until the asynchronous one is completed, so it is likely to time out. |
Small matrix jobs often do not need the whole resource utilization limit. Several matrix jobs can then be processed at the same time and be completed earlier than the examples show. This is not guaranteed, so measure the processing time of matrices that are typical for your traffic. Measured processing times are estimates too. They change with your other traffic and with the load on the service.
Synchronous and asynchronous matrix jobs
| Synchronous Matrix | Asynchronous Matrix Submission | |
|---|---|---|
| Use for | Fast, small matrices of lightweight route calculations | Larger matrix jobs and heavy route calculations |
| Maximum matrix size | 2500 (see API Limitations) | 100M (see API Limitations) |
| Timeout | Cells that are not calculated before the sync request timeout passes get the CELL_TIMEOUT error. | No timeout. The matrix job stays in the queue until it is processed. |
| Results | In the response | From the Asynchronous Matrix Download endpoint, for 24 hours |
A synchronous matrix job fails if the resource utilization limit is not enough to process any part of it before the sync request timeout passes:
- HTTP
429: processing could not start before the timeout, because your older matrix jobs were using the resource utilization limit. HTTP429is also returned when you exceed your QPS. - HTTP
408: processing started, but no part of the matrix was ready before the timeout.
If only part of the matrix is ready before the timeout, the response is HTTP 200 and the remaining cells get the CELL_TIMEOUT error, so check each cell for errors.
If this happens often, send fewer synchronous matrix jobs at the same time, use smaller or lighter matrices, send heavy matrices to the Asynchronous Matrix Submission endpoint, or request a higher resource utilization limit.
For an asynchronous matrix job, a state other than Completed or Failed means that the matrix job is still waiting in the queue or being processed. It is not an error.
Recommendations
Use separate applications for different types of traffic
In one application, your synchronous and asynchronous matrix jobs share the resource utilization limit (see Resource utilization limit). As a result, your asynchronous matrix jobs can use the resource utilization limit that your synchronous matrix jobs need.
To avoid this, use a separate application, each with its own API key, for:
- synchronous matrix jobs, for live traffic that needs results immediately,
- asynchronous matrix jobs, for larger matrices and heavy route calculations.
Estimate the quotas for each application separately, and request them for the API key of that application. If you have only one type of traffic, one application is enough.
If you offer the Matrix Routing v2 API to your own customers, use the Multi-Customer feature so that your sub-customers do not slow each other down.
Estimate the quotas you need
For the QPS, count the submissions, status requests and downloads that you send per second. Status requests also count, so check the status at an interval that fits your QPS.
The resource utilization limit you need depends on the size and options of your matrices, the number of matrix jobs at the same time, and how fast you need the results. Measure the processing time of your typical matrices with your current limit. Then Contact Sales or your account manager with the measured processing times, the routing options you use, the expected volume, and the processing time you need.
Keep the queue short
Keep synchronous matrices small and light. Send large or heavy matrices to the Asynchronous Matrix Submission endpoint.
Keep only a few asynchronous matrix jobs in the queue at a time. More matrix jobs in the queue do not make processing faster. They only make each matrix job wait longer.
To get results faster, request a higher resource utilization limit. Splitting a matrix into more matrix jobs, or sending more matrix jobs at the same time, does not process them faster.