Skip to Content
APIEndpointsWebsite indexing summary

Website indexing summary

A one-call health check on indexing coverage: how many pages are known, how many have actually been fetched, where they sit in the embedding pipeline, how many technical issues are open, and which sitemaps the pages are read from.

Fetch this before drawing conclusions from any per-page citation metric. Coverage is the denominator under all of them.

GET /v1/brands/{brand_id}/website/summary

Authorization

HeaderTypeRequired
AuthorizationBearer asky_sk_…required

Path parameters

ParameterTypeRequiredDescription
brand_iduuidrequiredFrom List brands

Query parameters

None.

Any parameter you pass returns 400 invalid_request naming the offending key, rather than being ignored silently.

Request

curl --request GET \ --url 'https://api.askylabs.com/v1/brands/242546de-4cee-4a7d-952e-f6aa60c63ef8/website/summary' \ --header 'Authorization: Bearer <api-key>'

Response

{ "data": { "total_pages": 318, "fetched_pages": 302, "embedding_status_counts": { "embedded": 288, "pending": 12, "discovered": 14, "failed": 3, "skipped": 1 }, "open_issue_count": 7, "sitemaps": [ { "url": "https://example.com/sitemap_index.xml", "added_by": "user", "url_count": 3, "read_by_scan": true }, { "url": "https://help.example.com/hc/sitemap.xml", "added_by": "user", "url_count": null, "read_by_scan": false }, { "url": "https://example.com/post-sitemap.xml", "added_by": "asky", "url_count": 284, "read_by_scan": true } ], "documentation_sources": [ { "url": "https://docs.example.com", "sitemap_url": "https://docs.example.com/sitemap.xml", "status": "active", "page_count": 32 } ] }, "request_id": "req_bf7210ce49a3486d95f8" }

Response fields

FieldTypeDescription
data.total_pagesintegerrequired. HTML pages known to us
data.fetched_pagesintegerrequired. Pages Asky has read (those with a title). Asky reads a page when a feature needs it, so this is usually below total_pages
data.embedding_status_countsobjectrequired. Page count per embedding state
data.open_issue_countintegerrequired. Unresolved technical issues found on the site
data.sitemapsarrayrequired. The sitemaps pages are read from, at most 25: the ones added on Brand Profile first, then the ones found on the site
data.sitemaps[].urlstringrequired. The sitemap URL
data.sitemaps[].added_bystringrequired. user when added on Brand Profile, asky when found on the site
data.sitemaps[].url_countinteger | nullrequired. URLs the sitemap listed when a scan last read it, null when no scan has read it yet
data.sitemaps[].read_by_scanbooleanrequired. Whether a scan has read it yet. false for one added on Brand Profile since the last scan: no pages from it yet is expected, not a fault
data.documentation_sourcesarrayrequired. Sites read as product documentation for writing, at most 25. Pages read only this way are not counted or listed here; a docs site added as a sitemap is
data.documentation_sources[].urlstringrequired. The documentation site’s address
data.documentation_sources[].sitemap_urlstring | nullrequired. The sitemap it is read from, null when its links are followed instead
data.documentation_sources[].statusstringrequired. active, crawling, paused or failed
data.documentation_sources[].page_countintegerrequired. Pages read from it so far
request_idstringrequired. Also returned as the x-request-id header. Quote it when contacting support

Errors

StatusCodeWhen
400invalid_requestA parameter is unknown or malformed. The message names it
401invalid_tokenThe key is missing, malformed, or does not exist
401token_expiredThe key passed its expiry date
401token_revokedThe key was revoked
404not_foundThe brand does not exist, or this key is not allowed to reach it. The two are deliberately indistinguishable
429rate_limitedOver 300 requests per minute. See the Retry-After header

Notes

Asky lists every page in your sitemaps and reads a page when a feature needs it, so fetched_pages below total_pages is normal. To see whether the site is blocking our crawler, look for the AskyBot is blocked issue among the open technical issues.

A subdomain of your main site with no pages at all, a shop or a help center for example, is usually missing from sitemaps: pages are indexed only through a sitemap, and only for your main site and its subdomains. Add one under Websites on the Brand Profile; the subdomain’s address works too, and its sitemap is found for you.

Last updated on