List crawler visits
Individual crawler hits, newest first. One row per fetch: when, which bot, which path.
This is the only endpoint on the surface that uses cursor pagination, because it is the only feed where rows arrive continuously while you are walking it. An offset would skip or repeat rows as new visits land mid-walk.
GET /v1/brands/{brand_id}/crawler/logsAuthorization
| Header | Type | Required |
|---|---|---|
Authorization | Bearer asky_sk_… | required |
Path parameters
| Parameter | Type | Required | Description |
|---|---|---|---|
brand_id | uuid | required | From List brands |
Query parameters
| Parameter | Type | Required | Description |
|---|---|---|---|
limit | integer | optional | Rows per page, 1 to 100. Defaults to 20. |
cursor | string | optional | Opaque cursor from the previous response. Treat it as a token, never construct one. |
start_date | YYYY-MM-DD | optional | Inclusive lower bound, YYYY-MM-DD in UTC. Narrows the population before metrics are computed. |
end_date | YYYY-MM-DD | optional | Inclusive upper bound, YYYY-MM-DD in UTC. Narrows the population before metrics are computed. |
bot_category | string, repeatable | optional | Restrict to these crawler categories. |
crawler | string, repeatable | optional | Restrict to these named crawlers. |
scope | string | optional | Which requests count. pages (default) keeps the visits to the brand’s own pages plus robots.txt, llms.txt and sitemaps, as Crawler Logs shows them; all adds every other request, old links and scanners included. |
Request
cURL
curl --request GET \
--url 'https://api.askylabs.com/v1/brands/242546de-4cee-4a7d-952e-f6aa60c63ef8/crawler/logs?limit=50' \
--header 'Authorization: Bearer <api-key>'Response
200
{
"data": [
{
"id": "0c9a4f21-77b3-4e5d-92aa-6b1e0d38f4c7",
"ts": "2026-08-13T06:41:19.220Z",
"bot_name": "GPTBot",
"bot_category": "ChatGPT",
"path": "/blog/ai-visibility-guide",
"domain": "getasky.com"
}
],
"pagination": {
"limit": 20,
"has_more": true,
"next_cursor": "eyJ2IjoxLCJvIjoyMCwibCI6MjB9"
},
"meta": {
"period": {
"start_date": "2026-08-06",
"end_date": "2026-08-13"
},
"scope": "pages",
"timezone": "UTC"
},
"request_id": "req_5e91c0f2734b48ad9a17"
}Response fields
| Field | Type | Description |
|---|---|---|
meta.period | object | required |
meta.timezone | string | required. Always UTC |
meta.scope | string | required. pages or all: which requests the figures count |
data | object[] | required |
data[].id | uuid | required |
data[].ts | timestamptz | required. When the fetch happened |
data[].bot_name | string | null | Exact crawler, for example GPTBot |
data[].bot_category | string | required. The operator bucket the bot rolls up to |
data[].path | string | null | |
data[].domain | string | null | |
pagination | object | required. See pagination |
request_id | string | required. Also returned as the x-request-id header. Quote it when contacting support |
Errors
| Status | Code | When |
|---|---|---|
| 400 | invalid_request | A parameter is unknown or malformed. The message names it |
| 401 | invalid_token | The key is missing, malformed, or does not exist |
| 401 | token_expired | The key passed its expiry date |
| 401 | token_revoked | The key was revoked |
| 404 | not_found | The brand does not exist, or this key is not allowed to reach it. The two are deliberately indistinguishable |
| 429 | rate_limited | Over 300 requests per minute. See the Retry-After header |
Notes
Your pages by default. The figures count the visits to pages Asky lists for the brand, plus
robots.txt, llms.txt and sitemaps, as Crawler Logs does. Requests for anything else (old links,
scanners probing for .env files under a crawler’s name) are left out unless you pass scope=all. For a brand
with no pages listed, both scopes count every request.
Stop when next_cursor is null, not when a page looks short. The cursor reflects whether a
next page genuinely exists, so a full-looking final page still returns null.
Human traffic is excluded. These are bot visits only, and humans are the large majority of raw log rows, so a count here is never comparable to analytics pageviews.
IP addresses, user agents, referrers and query strings are recorded but not returned. They exist
in the underlying log and are withheld from this endpoint because they are request-level personal
data: an IP plus a timestamp plus a path is identifying, and this surface is built so that a leaked
key cannot become a privacy incident. bot_name and bot_category carry the analytical content
of the user agent without the rest of it.
The cursor encodes a (timestamp, id) tuple rather than a timestamp alone. A timestamp is not a
total order, and paging on it dropped every row that shared a boundary second.