Pagination in Axiom API

You can use pagination for the following endpoints:

• Run Query
• Run Query (Legacy)

Prerequisites#

• Create an Axiom account.
• Create a dataset in Axiom where you send your data.
• Create an API token in Axiom with permissions to ingest data to the dataset you have created.

Pagination mechanisms#

You can use one of the following pagination mechanisms:

• Pagination based on timestamp (stable)
• Pagination based on cursor (public preview)

Axiom recommends timestamp-based pagination. Cursor-based pagination is in public preview and may return unexpected query results.

Timestamp-based pagination#

The parameters and mechanisms differ between the current and legacy endpoints.

Run Query#

To use timestamp-based pagination with the Run Query endpoint:

• Include the limit operator in the APL query of your API request. The argument of this operator determines the number of events to display per page.
• Use sort by _time asc or sort by _time desc in the APL query. This returns the results in ascending or descending chronological order. For more information, see sort operator.
• Specify startTime and endTime in the body of your API request.

Run Query (Legacy)#

To use timestamp-based pagination with the legacy Run Query endpoint:

• Add the limit parameter to the body of your API request. The value of this parameter determines the number of events to display per page.
• Add the order parameter to the body of your API request. In the value of this parameter, order the results by time in either ascending or descending chronological order. For example, [{ "field": "_time", "desc": true }]. For more information, see order operator.
• Specify startTime and endTime in the body of your API request.

Page through the result set#

Use the timestamps as boundaries to page through the result set.

Queries with descending order#

To go to the next page of the result set for queries with descending order (_time desc):

1. Determine the timestamp of last item on the current page. This is the least recent event.
2. Optional: Subtract 1 nanosecond from the timestamp.
3. In your next request, change the value endTime parameter in the body of your API request to the timestamp of the last item (optionally, minus 1 nanosecond).

Repeat this process until the result set is empty.

Queries with ascending order#

To go to the next page of the result set for queries with ascending order (_time asc):

1. Determine the timestamp of last item on the current page. This is the most recent event.
2. Optional: Add 1 nanosecond to the timestamp.
3. In your next request, change the value startTime parameter in the body of your API request to the timestamp of the last item (optionally, plus 1 nanosecond).

Repeat this process until the result set is empty.

Deduplication mechanism#

In the procedures above, the steps about incrementing the timestamp are optional. If you increment the timestamp, there is a risk of duplication. If you don’t increment the timestamp, there is a risk of overlap. Duplicated data is possible for many reasons, such as backfill or natural duplication from external data sources. For these reasons, regardless of the method you choose (increment or not increment the timestamp, sort by descending or ascending order), Axiom recommends you implement some form of deduplication mechanism in your pagination script.

Limits#

Both the Run Query and the Run Query (Legacy) endpoints allow request-based limit configuration. This means that the limit they use is the lowest of the following: the query limit, the request limit, and Axiom’s server-side internal limit. Without a query or request limit, Axiom currently defaults to the limit of 1,000 events per page. For the pagination of datasets that are greater than 1,000 events, Axioms recommends specifying the same limit in the request and the APL query to avoid the default value and contradictory limits.

Examples#

Example request Run Query#

curl -X 'POST' 'https://api.axiom.co/v1/datasets/_apl?format=tabular' \
-H 'Authorization: Bearer API_TOKEN' \
-H 'Content-Type: application/json' \
-d '{
    "apl": "DATASET_NAME | sort by _time desc | limit 100",
    "startTime": "2024-11-30T00:00:00.000Z",
    "endTime": "2024-11-30T23:59:59.999Z"
  }'

Example request Run Query (Legacy)#

curl -X 'POST' 'https://api.axiom.co/v1/datasets/DATASET_NAME/query' \
-H 'Authorization: Bearer API_TOKEN' \
-H 'Content-Type: application/json' \
-d '{
    "startTime": "2024-11-30T00:00:00.000Z",
    "endTime": "2024-11-30T23:59:59.999Z",
    "limit": 100,
    "order": [{ "field": "_time", "desc": true }]
  }'

Example request to page through the result set#

Example request to go to next page for Run Query:

curl -X 'POST' 'https://api.axiom.co/v1/datasets/_apl?format=tabular' \
-H 'Authorization: Bearer API_TOKEN' \
-H 'Content-Type: application/json' \
-d '{
    "apl": "DATASET_NAME | sort by _time desc | limit 100",
    "startTime": "2024-11-30T00:00:00.000Z",
    "endTime": "2024-11-30T22:59:59.999Z"
  }'

Example request to go to next page for Run Query (Legacy):

curl -X 'POST' 'https://api.axiom.co/v1/datasets/DATASET_NAME/query' \
-H 'Authorization: Bearer API_TOKEN' \
-H 'Content-Type: application/json' \
-d '{
    "startTime": "2024-11-30T00:00:00.000Z",
    "endTime": "2024-11-30T22:59:59.999Z",
    "limit": 100,
    "order": [{ "field": "_time", "desc": true }]
  }'

Cursor-based pagination#

Cursor-based pagination is in public preview and may return unexpected query results. Axiom recommends timestamp-based pagination.

The parameters and mechanisms differ between the current and legacy endpoints.

Run Query#

To use cursor-based pagination with the Run Query endpoint:

• Include the limit operator in the APL query of your API request. The argument of this operator determines the number of events to display per page.
• Use sort by _time asc or sort by _time desc in the APL query. This returns the results in ascending or descending chronological order. For more information, see sort operator.
• Specify startTime and endTime in the body of your API request.

Run Query (Legacy)#

To use cursor-based pagination with the legacy Run Query endpoint:

• Add the limit parameter to the body of your API request. The value of this parameter determines the number of events to display per page.
• Add the order parameter to the body of your API request. In the value of this parameter, order the results by time in either ascending or descending chronological order. For example, [{ "field": "_time", "desc": true }]. For more information, see order operator.
• Specify startTime and endTime in the body of your API request.

Response format#

• status (object) — Contains metadata about the response including pagination information.

• status.minCursor (string) — Cursor for the first item in the current page.

• status.maxCursor (string) — Cursor for the last item in the current page.

• status.rowsMatched (integer) — Total number of rows matching the query.

• matches (array) — Contains the list of returned objects.

Page through the result set#

To page through the result set, add the cursor parameter to the body of your API request.

• cursor (string, query) — Optional. A cursor for use in pagination. Use the cursor string returned in previous responses to fetch the next or previous page of results.

The minCursor and maxCursor fields in the response are boundaries that help you page through the result set.

For queries with descending order (_time desc), use minCursor from the response as the cursor in your next request to go to the next page. You reach the end when your provided cursor matches the minCursor in the response.

For queries with ascending order (_time asc), use maxCursor from the response as the cursor in your next request to go to the next page. You reach the end when your provided cursor matches the maxCursor in the response.

If the query returns fewer results than the specified limit, paging can stop.

Examples#

Example request Run Query#

curl -X 'POST' 'https://api.axiom.co/v1/datasets/_apl?format=tabular' \
-H 'Authorization: Bearer API_TOKEN' \
-H 'Content-Type: application/json' \
-d '{
    "apl": "DATASET_NAME | sort by _time desc | limit 100",
    "startTime": "2024-01-01T00:00:00.000Z",
    "endTime": "2024-01-31T23:59:59.999Z"
  }'

Example request Run Query (Legacy)#

curl -X 'POST' 'https://api.axiom.co/v1/datasets/DATASET_NAME/query' \
-H 'Authorization: Bearer API_TOKEN' \
-H 'Content-Type: application/json' \
-d '{
    "startTime": "2024-01-01T00:00:00.000Z",
    "endTime": "2024-01-31T23:59:59.999Z",
    "limit": 100,
    "order": [{ "field": "_time", "desc": true }]
  }'

Example response#

{
  "status": {
    "rowsMatched": 2500,
    "minCursor": "0d3wo7v7e1oii-075a8c41710018b9-0000ecc5",
    "maxCursor": "0d3wo7v7e1oii-075a8c41710018b9-0000faa3"
  },
  "matches": [
    // ... events ...
  ]
}

Example request to page through the result set#

To page through the result set, use the appropriate cursor value in your next request. For more information, see Page through the result set.

Example request to go to next page for Run Query:

curl -X 'POST' 'https://api.axiom.co/v1/datasets/_apl?format=tabular' \
-H 'Authorization: Bearer API_TOKEN' \
-H 'Content-Type: application/json' \
-d '{
    "apl": "DATASET_NAME | sort by _time desc | limit 100",
    "startTime": "2024-01-01T00:00:00.000Z",
    "endTime": "2024-01-31T23:59:59.999Z",
    "cursor": "0d3wo7v7e1oii-075a8c41710018b9-0000ecc5"
  }'

Example request to go to next page for Run Query (Legacy):

curl -X 'POST' 'https://api.axiom.co/v1/datasets/DATASET_NAME/query' \
-H 'Authorization: Bearer API_TOKEN' \
-H 'Content-Type: application/json' \
-d '{
    "startTime": "2024-01-01T00:00:00.000Z",
    "endTime": "2024-01-31T23:59:59.999Z",
    "limit": 100,
    "order": [{ "field": "_time", "desc": true }],
    "cursor": "0d3wo7v7e1oii-075a8c41710018b9-0000ecc5"
  }'