POST
/
crawl
curl --request POST \
  --url https://api.tavily.com/crawl \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '{
  "url": "https://docs.tavily.com",
  "max_depth": 1,
  "max_breadth": 20,
  "limit": 50,
  "query": "Python SDK",
  "select_paths": null,
  "select_domains": null,
  "allow_external": false,
  "categories": null,
  "extract_depth": "basic"
}'
{
  "base_url": "wikipedia.org/wiki/computer",
  "results": [
    {
      "url": "https://en.wikipedia.org/wiki/Computer",
      "raw_content": "<string>"
    }
  ],
  "response_time": 1.23
}

Authorizations

​
Authorization
string
header
required

Bearer authentication header in the form Bearer <token>, where <token> is your Tavily API key (e.g., Bearer tvly-YOUR_API_KEY).

Body

application/json
Parameters for the Tavily Crawl request.
​
url
string
required

The root URL to begin the crawl.

Example:

"https://docs.tavily.com"

​
max_depth
integer
default:1

Max depth of the crawl. Defines how far from the base URL the crawler can explore.

Required range: x >= 1
​
max_breadth
integer
default:20

Max number of links to follow per level of the tree (i.e., per page).

Required range: x >= 1
​
limit
integer
default:50

Total number of links the crawler will process before stopping.

Required range: x >= 1
​
query
string

Natural language instructions for the crawler.

Example:

"Python SDK"

​
select_paths
string[]

Regex patterns to select only URLs with specific path patterns (e.g., /docs/.*, /api/v1.*).

​
select_domains
string[]

Regex patterns to select crawling to specific domains or subdomains (e.g., ^docs\.example\.com$).

​
allow_external
boolean
default:false

Whether to allow following links that go to external domains.

​
categories
enum<string>[]

Filter URLs using predefined categories like 'Documentation', 'Blog', 'API', etc.

Available options:
Careers,
Blog,
Documentation,
About,
Pricing,
Community,
Developers,
Contact,
Media
​
extract_depth
enum<string>
default:basic

Advanced extraction retrieves more data, including tables and embedded content, with higher success but may increase latency.

Available options:
basic,
advanced

Response

200
application/json
Crawl results returned successfully
​
base_url
string

The base URL that was crawled.

Example:

"wikipedia.org/wiki/computer"

​
results
object[]

A list of extracted content from the crawled URLs.

​
response_time
number

Time in seconds it took to complete the request.

Example:

1.23