Update Bot traffic settings

PATCH https://datafa.st/api/v1/admin/websites/{websiteId}/bot-traffic/settings

Update one or more Bot traffic settings for a website. Prefer incremental company and crawler fields so an agent does not overwrite exclusions it did not intend to change.

MCP equivalent: datafast_bot_traffic_settings_update. MCP requires confirm: true before verification, authentication, or ingestion changes because rejected future requests cannot be recovered.

Try on Playground

Request

Path parameters

websiteId

string

required

Website ObjectId used by account tokens to choose which website to query or manage. Website ObjectId. Get it from the dashboard or the List websites API.

Body parameters

verifiedOnly

boolean

true accepts only IP-verified crawlers. false also accepts user-agent-only matches.

requireAuthToken

boolean

true rejects requests without the website Bot traffic token. Create a token first.

disableCompanies

string[]

Disable every known agent for these exact company names.

enableCompanies

string[]

Enable every known agent for these exact company names.

disableCrawlerIds

string[]

Disable exact agent IDs returned by Get settings.

enableCrawlerIds

string[]

Enable exact agent IDs returned by Get settings.

excludedCrawlerIds

string[]

Replace the complete exclusion list. Do not combine with incremental enable or disable fields.

hiddenCrawlers

string[]

Replace crawler IDs hidden from charts only. Does not reduce ingestion or usage.

hideDashboardCard

boolean

Hide or show the Bot traffic dashboard card.

groupBy

string

crawlers or pages.

pageFilterIds

string[]

Replace dashboard page-filter IDs.

Changing hiddenCrawlers, hideDashboardCard, groupBy, or pageFilterIds affects display only. Changing verification, authentication, companies, or agents affects future ingestion. Requests rejected by those settings are not stored and cannot be recovered.

Response

Returns a JSON object with status: "success" and endpoint-specific fields in data (and pagination when the endpoint is paginated).

Response fields

data[].websiteId

string

Website ObjectId used by account tokens to choose which website to query or manage.

data[].domain

string

Tracked domain.

data[].settings

object

Complete normalized settings after the update.

data[].crawlerCatalog

array

Complete company and agent catalog after the update.

Authentication

Requires an owner dft_ account token with settings:write. Website API keys (df_) cannot call this endpoint.

The token must be allowed to access the website in the path, and that website must belong to the token owner.

The account must have active DataFast API access. Read authentication and scopes for token creation and permissions.

Errors

400 — Invalid website ID or JSON, unknown fields, invalid field types, conflicting exclusion updates, or enabling token enforcement before a token exists.

404 — Website not found or the caller is not its owner.

413 — Request body exceeds 8 KB.

See API errors for the standard error envelope, auth failures, validation errors, permission errors, and rate limits.

✍️ Something missing? Suggest features.

🤖 AI agent or LLM? Read this page as markdown

curl -X PATCH "https://datafa.st/api/v1/admin/websites/{websiteId}/bot-traffic/settings" \ -H "Authorization: Bearer dft_xxx" \ -H "Content-Type: application/json" \ -d '{"disableCompanies":["Meta"],"enableCrawlerIds":["OpenAI::ChatGPT-User::answer_fetch"],"verifiedOnly":true}'

{ "status": "success", "data": [{ "websiteId": "665f0b3c4d2e1a0012345678", "domain": "example.com", "settings": { "hiddenCrawlers": [], "excludedCrawlerIds": [ "Meta::meta-externalagent::training" ], "verifiedOnly": true, "hideDashboardCard": false, "groupBy": "crawlers", "pageFilterIds": [], "requireAuthToken": false }, "crawlerCatalog": [] }] }