Axiom
Overview

topk

Use topk to find the most common or relevant entries in datasets, especially in log analysis, telemetry data, and monitoring systems. This aggregation helps you focus on the most important data points, filtering out the noise.

The topk aggregation in APL is a statistical aggregation that returns estimated results. The estimation comes with the benefit of speed at the expense of accuracy. This means that topk is fast and light on resources even on a large or high-cardinality dataset, but it doesn’t provide precise results.

For completely accurate results, use the top operator.

Usage#

Syntax#

topk(Field, k)

Parameters#

  • Field: The field or expression to rank the results by.
  • k: The number of top results to return.

Returns#

A subset of the original dataset with the top k values based on the specified field.

Use case examples#

When analyzing HTTP logs, you can use the topk function to find the top 5 most frequent HTTP status codes.

Query

['sample-http-logs']
| summarize topk(status, 5)

Run in Playground

Output

status count_
200 1500
404 400
500 200
301 150
302 100

This query groups the logs by HTTP status and returns the 5 most frequent statuses.

In OpenTelemetry traces, you can use topk to find the top five status codes by service.

Query

['otel-demo-traces']
| summarize topk(['attributes.http.status_code'], 5) by ['service.name']

Run in Playground

Output

service.name attributes.http.status_code _count
frontendproxy 200 34,862,088
203 3,095,223
404 154,417
500 153,823
504 3,497

This query shows the top five status codes by service.

You can use topk in security log analysis to find the top 5 cities generating the most HTTP requests.

Query

['sample-http-logs']
| summarize topk(['geo.city'], 5)

Run in Playground

Output

geo.city count_
New York 500
London 400
Paris 350
Tokyo 300
Berlin 250

This query returns the top 5 cities based on the number of HTTP requests.

  • top: Returns the top values based on a field without requiring a specific number of results (k), making it useful when you’re unsure how many top values to retrieve.
  • topkif: Returns the top k results without filtering. Use topk when you don’t need to restrict your analysis to a subset.
  • sort: Orders the dataset based on one or more fields, which is useful if you need a complete ordered list rather than the top k values.
  • extend: Adds calculated fields to your dataset, which can be useful in combination with topk to create custom rankings.
  • count: Aggregates the dataset by counting occurrences, often used in conjunction with topk to find the most common values.

Other query languages#

Splunk SPL users

Splunk SPL doesn’t have the equivalent of the topk function. You can achieve similar results with SPL’s top command which is equivalent to APL’s top operator. The topk function in APL behaves similarly by returning the top k values of a specified field, but its syntax is unique to APL.

The main difference between top (supported by both SPL and APL) and topk (supported only by APL) is that topk is estimated. This means that APL’s topk is faster, less resource intenstive, but less accurate than SPL’s top.

Splunk example

| top limit=5 status by method

APL equivalent

['sample-http-logs']
| summarize topk(status, 5) by method
ANSI SQL users

In ANSI SQL, identifying the top k rows often involves using the ORDER BY and LIMIT clauses. While the logic remains similar, APL’s topk simplifies this process by directly returning the top k values of a field in an aggregation.

The main difference between SQL’s solution and APL’s topk is that topk is estimated. This means that APL’s topk is faster, less resource intenstive, but less accurate than SQL’s combination of ORDER BY and LIMIT clauses.

SQL example

SELECT status, COUNT(*)
FROM sample_http_logs
GROUP BY status
ORDER BY COUNT(*) DESC
LIMIT 5;

APL equivalent

['sample-http-logs']
| summarize topk(status, 5)

Updated

Was this page helpful?