Skip to Content
APIEndpointsList crawler visits

List crawler visits

Individual crawler hits, newest first. One row per fetch: when, which bot, which path.

This is the only endpoint on the surface that uses cursor pagination, because it is the only feed where rows arrive continuously while you are walking it. An offset would skip or repeat rows as new visits land mid-walk.

GET /v1/brands/{brand_id}/crawler/logs

Authorization

HeaderTypeRequired
AuthorizationBearer asky_sk_…required

Path parameters

ParameterTypeRequiredDescription
brand_iduuidrequiredFrom List brands

Query parameters

ParameterTypeRequiredDescription
limitintegeroptionalRows per page, 1 to 100. Defaults to 20.
cursorstringoptionalOpaque cursor from the previous response. Treat it as a token, never construct one.
start_dateYYYY-MM-DDoptionalInclusive lower bound, YYYY-MM-DD in UTC. Narrows the population before metrics are computed.
end_dateYYYY-MM-DDoptionalInclusive upper bound, YYYY-MM-DD in UTC. Narrows the population before metrics are computed.
bot_categorystring, repeatableoptionalRestrict to these crawler categories.
crawlerstring, repeatableoptionalRestrict to these named crawlers.
scopestringoptionalWhich requests count. pages (default) keeps the visits to the brand’s own pages plus robots.txt, llms.txt and sitemaps, as Crawler Logs shows them; all adds every other request, old links and scanners included.

Request

curl --request GET \ --url 'https://api.askylabs.com/v1/brands/242546de-4cee-4a7d-952e-f6aa60c63ef8/crawler/logs?limit=50' \ --header 'Authorization: Bearer <api-key>'

Response

{ "data": [ { "id": "0c9a4f21-77b3-4e5d-92aa-6b1e0d38f4c7", "ts": "2026-08-13T06:41:19.220Z", "bot_name": "GPTBot", "bot_category": "ChatGPT", "path": "/blog/ai-visibility-guide", "domain": "getasky.com" } ], "pagination": { "limit": 20, "has_more": true, "next_cursor": "eyJ2IjoxLCJvIjoyMCwibCI6MjB9" }, "meta": { "period": { "start_date": "2026-08-06", "end_date": "2026-08-13" }, "scope": "pages", "timezone": "UTC" }, "request_id": "req_5e91c0f2734b48ad9a17" }

Response fields

FieldTypeDescription
meta.periodobjectrequired
meta.timezonestringrequired. Always UTC
meta.scopestringrequired. pages or all: which requests the figures count
dataobject[]required
data[].iduuidrequired
data[].tstimestamptzrequired. When the fetch happened
data[].bot_namestring | nullExact crawler, for example GPTBot
data[].bot_categorystringrequired. The operator bucket the bot rolls up to
data[].pathstring | null
data[].domainstring | null
paginationobjectrequired. See pagination
request_idstringrequired. Also returned as the x-request-id header. Quote it when contacting support

Errors

StatusCodeWhen
400invalid_requestA parameter is unknown or malformed. The message names it
401invalid_tokenThe key is missing, malformed, or does not exist
401token_expiredThe key passed its expiry date
401token_revokedThe key was revoked
404not_foundThe brand does not exist, or this key is not allowed to reach it. The two are deliberately indistinguishable
429rate_limitedOver 300 requests per minute. See the Retry-After header

Notes

Your pages by default. The figures count the visits to pages Asky lists for the brand, plus robots.txt, llms.txt and sitemaps, as Crawler Logs does. Requests for anything else (old links, scanners probing for .env files under a crawler’s name) are left out unless you pass scope=all. For a brand with no pages listed, both scopes count every request.

Stop when next_cursor is null, not when a page looks short. The cursor reflects whether a next page genuinely exists, so a full-looking final page still returns null.

Human traffic is excluded. These are bot visits only, and humans are the large majority of raw log rows, so a count here is never comparable to analytics pageviews.

IP addresses, user agents, referrers and query strings are recorded but not returned. They exist in the underlying log and are withheld from this endpoint because they are request-level personal data: an IP plus a timestamp plus a path is identifying, and this surface is built so that a leaked key cannot become a privacy incident. bot_name and bot_category carry the analytical content of the user agent without the rest of it.

The cursor encodes a (timestamp, id) tuple rather than a timestamp alone. A timestamp is not a total order, and paging on it dropped every row that shared a boundary second.

Last updated on