Most-crawled pages
The paths on your site that AI crawlers fetch most, ordered by total hits. In other words: which of your content the answer engines are actually reading.
Read alongside Website pages, which gives the citation rate per page. A page crawled heavily but never cited is a content problem; a page never crawled is a crawlability problem. The two need different fixes.
GET /v1/brands/{brand_id}/crawler/top-pathsAuthorization
| Header | Type | Required |
|---|---|---|
Authorization | Bearer asky_sk_… | required |
Path parameters
| Parameter | Type | Required | Description |
|---|---|---|---|
brand_id | uuid | required | From List brands |
Query parameters
| Parameter | Type | Required | Description |
|---|---|---|---|
limit | integer | optional | Rows per page, 1 to 100. Defaults to 20. |
cursor | string | optional | Opaque cursor from the previous response. Treat it as a token, never construct one. |
start_date | YYYY-MM-DD | optional | Inclusive lower bound, YYYY-MM-DD in UTC. Narrows the population before metrics are computed. |
end_date | YYYY-MM-DD | optional | Inclusive upper bound, YYYY-MM-DD in UTC. Narrows the population before metrics are computed. |
scope | string | optional | Which requests count. pages (default) keeps the visits to the brand’s own pages plus robots.txt, llms.txt and sitemaps, as Crawler Logs shows them; all adds every other request, old links and scanners included. |
Request
cURL
curl --request GET \
--url 'https://api.askylabs.com/v1/brands/242546de-4cee-4a7d-952e-f6aa60c63ef8/crawler/top-paths?limit=50' \
--header 'Authorization: Bearer <api-key>'Response
200
{
"data": [
{
"path": "/blog/ai-visibility-guide",
"total": 214,
"by_bot": {
"ChatGPT": 121,
"Perplexity": 34,
"Claude": 12,
"Google": 42,
"Microsoft": 2,
"Other": 3
},
"by_crawler": {
"GPTBot": 98,
"ChatGPT-User": 23,
"PerplexityBot": 34
},
"last_visit": "2026-08-13T06:41:19.220Z"
}
],
"pagination": {
"limit": 20,
"has_more": true,
"next_cursor": "eyJ2IjoxLCJvIjoyMCwibCI6MjB9"
},
"meta": {
"period": {
"start_date": "2026-07-14",
"end_date": "2026-08-13"
},
"scope": "pages",
"timezone": "UTC",
"total_paths": 318,
"returned": 50
},
"request_id": "req_1d8f43ba09e747c2b310"
}Response fields
| Field | Type | Description |
|---|---|---|
meta.period | object | required |
meta.timezone | string | required. Always UTC |
meta.scope | string | required. pages or all: which requests the figures count |
data | object[] | required |
data[].path | string | required. Path only, without host |
data[].total | integer | required |
data[].by_bot | object | required. Counts per operator bucket |
data[].by_crawler | object | required. Counts per exact bot name |
data[].last_visit | timestamptz | null | Most recent fetch of this path |
meta.total_paths | integer | required. Distinct paths crawled in the window, before limit |
meta.returned | integer | required |
pagination | object | required. See pagination |
request_id | string | required. Also returned as the x-request-id header. Quote it when contacting support |
Errors
| Status | Code | When |
|---|---|---|
| 400 | invalid_request | A parameter is unknown or malformed. The message names it |
| 401 | invalid_token | The key is missing, malformed, or does not exist |
| 401 | token_expired | The key passed its expiry date |
| 401 | token_revoked | The key was revoked |
| 404 | not_found | The brand does not exist, or this key is not allowed to reach it. The two are deliberately indistinguishable |
| 429 | rate_limited | Over 300 requests per minute. See the Retry-After header |
Notes
Your pages by default. The figures count the visits to pages Asky lists for the brand, plus
robots.txt, llms.txt and sitemaps, as Crawler Logs does. Requests for anything else (old links,
scanners probing for .env files under a crawler’s name) are left out unless you pass scope=all. For a brand
with no pages listed, both scopes count every request.
Bot traffic only. Ordinary human visits are excluded before anything is counted, and they are the large majority of raw log rows. A figure here is therefore never comparable to a pageview count from your analytics tool.
Compare total_paths against returned to know whether you are seeing the whole picture or the
top slice of a longer tail. If has_more is true, page forward with next_cursor to see the
rest — total_paths reports how many exist regardless of which page you are on.