Source: https://datafa.st/docs/api/website/bot-traffic/settings/retrieve
Markdown source: https://datafa.st/docs/api/website/bot-traffic/settings/retrieve.md
Description: Return a website's Bot traffic display, verification, authentication, and ingestion settings, plus the complete company and agent catalog. Read this first before changing individual crawlers.

# Get Bot traffic settings

`GET https://datafa.st/api/v1/admin/websites/{websiteId}/bot-traffic/settings`

Return a website's Bot traffic display, verification, authentication, and ingestion settings, plus the complete company and agent catalog. Read this first before changing individual crawlers.

MCP equivalent: `datafast_bot_traffic_settings_get`. `hiddenCrawlers` changes the chart only; `excludedCrawlerIds` stops future matching requests before storage.

> **Related:** [Update Bot traffic settings](/docs/api/website/bot-traffic/settings/update) · [Bot traffic tracking](/docs/bot-traffic-tracking)

## Request

#### Path parameters

| Parameter | Type | Required | Description |
| --- | --- | --- | --- |
| `websiteId` | string | Yes | Website ObjectId used by account tokens to choose which website to query or manage. Website ObjectId. Get it from the dashboard or the List websites API. |

## Response

Returns a JSON object with `status: "success"` and endpoint-specific fields in `data` (and `pagination` when the endpoint is paginated).

#### Response fields

| Field | Type | Description |
| --- | --- | --- |
| `data[].websiteId` | string | Website ObjectId used by account tokens to choose which website to query or manage. |
| `data[].domain` | string | Tracked domain. |
| `data[].settings.hiddenCrawlers` | string[] | Crawler IDs hidden from dashboard charts only. |
| `data[].settings.excludedCrawlerIds` | string[] | Crawler IDs excluded from future ingestion. |
| `data[].settings.verifiedOnly` | boolean | Whether only IP-verified crawler matches are accepted. |
| `data[].settings.requireAuthToken` | boolean | Whether incoming requests must include the website token. |
| `data[].settings.hideDashboardCard` | boolean | Whether the Bot traffic card is hidden. |
| `data[].settings.groupBy` | string | `crawlers` or `pages`. |
| `data[].settings.pageFilterIds` | string[] | Selected page identifiers used by the dashboard. |
| `data[].crawlerCatalog[]` | array | Companies with enabled, disabled, or partial status and their agents. |
| `data[].crawlerCatalog[].agents[]` | array | Exact agent ID, user agent, category, description, and enabled state. |

## Authentication

Requires an owner `dft_` account token with `settings:read`. Website API keys (`df_`) cannot call this endpoint.

The token must be allowed to access the website in the path, and that website must belong to the token owner.

The account must have active DataFast API access. Read [authentication and scopes](/docs/api/authentication) for token creation and permissions.

### Errors

**400** — Invalid website ID.

**404** — Website not found or the caller is not its owner.

See [API errors](/docs/api#errors) for the standard error envelope, auth failures, validation errors, permission errors, and rate limits.

## Code examples

### Example request

```bash
curl -X GET "https://datafa.st/api/v1/admin/websites/{websiteId}/bot-traffic/settings" \
  -H "Authorization: Bearer dft_xxx"
```

### Success response

```json
{
  "status": "success",
  "data": [{
    "websiteId": "665f0b3c4d2e1a0012345678",
    "domain": "example.com",
    "settings": {
      "hiddenCrawlers": [],
      "excludedCrawlerIds": ["Common Crawl::CCBot::training"],
      "verifiedOnly": true,
      "hideDashboardCard": false,
      "groupBy": "crawlers",
      "pageFilterIds": [],
      "requireAuthToken": false
    },
    "crawlerCatalog": [{
      "company": "OpenAI",
      "status": "enabled",
      "enabledAgentCount": 3,
      "totalAgentCount": 3,
      "agents": [{
        "id": "OpenAI::ChatGPT-User::answer_fetch",
        "userAgent": "ChatGPT-User",
        "category": "answer_fetch",
        "description": "Fetches pages to answer a ChatGPT user.",
        "enabled": true
      }]
    }]
  }]
}
```