---
name: tavilyai
description: Use when building AI agents that need live web search, content extraction, site crawling, site mapping, or cited research synthesis. Reach for Tavily when an agent must answer questions with current information, pull content from specific URLs, traverse websites, or generate multi-source reports with citations.
metadata:
    mintlify-proj: tavilyai
    version: "1.0"
---

# Tavily Skill

## Product summary

Tavily is a web search engine optimized for AI agents and LLMs. It provides five core capabilities: **Search** (find sources), **Extract** (pull content from URLs), **Crawl** (traverse sites), **Map** (discover site structure), and **Research** (generate cited synthesis). Unlike generic search APIs, Tavily returns clean, ranked, scored results ready for LLM consumption—no HTML parsing, filtering, or token waste. Access via REST API, Python/JavaScript SDKs, or MCP server. Free tier: 1,000 credits/month (no card required). Base URL: `https://api.tavily.com`. Authentication: Bearer token (`Authorization: Bearer tvly-YOUR_API_KEY`) or keyless header (`X-Tavily-Access-Mode: keyless` for Search/Extract only). Primary docs: https://docs.tavily.com

## When to use

Reach for Tavily when:
- An agent needs to **search the web** for current information (sources unknown)
- You have URLs and need to **extract clean content** without JavaScript rendering or HTML cleanup
- You must **crawl multiple pages** across a site to gather information
- You need to **understand a site's structure** before crawling it
- You need a **finished, multi-source answer with citations** (not raw sources)
- You're building a **RAG pipeline** and need grounded, ranked sources
- You're doing **competitive research, lead enrichment, news monitoring, or meeting prep**
- You need to **track usage by project** or **attribute requests to sessions/users**

Do not use Tavily for: static local files, private/authenticated content, real-time stock prices (use dedicated APIs), or when you only need a single generic web lookup.

## Quick reference

### Endpoints and costs

| Endpoint | Use when | Cost | Rate limit |
|----------|----------|------|-----------|
| `/search` | Sources unknown, need current web context | 1 credit (basic), 2 credits (advanced) | 100 RPM (dev), 1,000 RPM (prod) |
| `/extract` | Have URLs, need clean content | 1 credit per 5 URLs (basic), 2 per 5 (advanced) | 100 RPM (dev), 1,000 RPM (prod) |
| `/crawl` | Traverse multiple pages on a site | Map cost + extraction cost | 100 RPM (both) |
| `/map` | Discover site structure before crawl | 1 credit per 10 pages | 100 RPM (both) |
| `/research` | Need cited synthesis, report, or decision | 4–110 credits (mini), 15–250 (pro) | 20 RPM (both) |

### SDK instantiation

**Python:**
```python
from tavily import TavilyClient
client = TavilyClient(api_key="tvly-YOUR_API_KEY")
response = client.search("query", search_depth="advanced", max_results=5)
```

**JavaScript:**
```javascript
const { tavily } = require("@tavily/core");
const tvly = tavily({ apiKey: "tvly-YOUR_API_KEY" });
const response = await tvly.search("query", { searchDepth: "advanced", maxResults: 5 });
```

**cURL:**
```bash
curl -X POST https://api.tavily.com/search \
  -H "Authorization: Bearer tvly-YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"query": "your query", "search_depth": "advanced", "max_results": 5}'
```

### Key parameters by endpoint

**Search:**
- `search_depth`: `basic` (1 credit, medium latency), `advanced` (2 credits, higher latency), `fast` (1 credit, low latency), `ultra-fast` (1 credit, lowest latency)
- `max_results`: 1–20 (default 5)
- `chunks_per_source`: 1–3 (only with advanced/basic/fast; reranked snippets)
- `include_domains` / `exclude_domains`: Filter by domain
- `include_domains_mode`: `restrict` (hard filter) or `prefer` (soft boost)
- `time_range`: `day`, `week`, `month`, `year`
- `start_date` / `end_date`: YYYY-MM-DD format
- `topic`: `general` or `news`
- `language` / `filter_by_language`: ISO 639-1 code or English name
- `include_raw_content`: Full page content (slower, more tokens)
- `exact_match`: Require verbatim phrase match (narrows results)

**Extract:**
- `urls`: Array of 1–20 URLs
- `query`: Rerank chunks by relevance to query
- `chunks_per_source`: 1–5 (only with query)
- `extract_depth`: `basic` or `advanced`
- `format`: `markdown` (default) or `text`

**Crawl:**
- `url`: Starting URL
- `instructions`: Natural-language guidance for crawling
- `max_depth`: How deep to traverse (1–5)
- `max_breadth`: How many links per page (1–50)
- `extract_depth`: `basic` or `advanced`

**Research:**
- `query`: Research question
- `model`: `mini` (4–110 credits) or `pro` (15–250 credits)
- `include_domains` / `exclude_domains`: Soft preference or hard blocklist
- `output_length`: `short`, `standard`, or `long`

### Session and project tracking

Pass these headers to link related calls and track usage:
- `X-Session-Id`: Opaque session identifier (groups related requests)
- `X-Human-Id`: Opaque end-user identifier (when one key serves many users)
- `X-Project-ID`: Project identifier (organize usage by project)

**Python SDK:**
```python
client.search("query", session_id="session-123", human_id="user-456")
```

**JavaScript SDK:**
```javascript
tvly.search("query", { sessionId: "session-123", humanId: "user-456" })
```

## Decision guidance

### Search vs. Research

| Use Search when | Use Research when |
|---|---|
| You need raw source URLs and content to process yourself | You need a finished, multi-source answer with citations |
| You want to filter/rank results yourself | You want Tavily to synthesize and cite sources |
| You need fast, focused lookups | You need a report, comparison, or decision-ready answer |
| You're building a RAG pipeline | You're answering complex questions that need synthesis |

### Search depth tradeoff

| Depth | Latency | Relevance | Use when |
|-------|---------|-----------|----------|
| `ultra-fast` | Lowest | Lower | Latency is critical; accept lower relevance |
| `fast` | Low | Good | Need quick, targeted snippets |
| `basic` | Medium | High | General-purpose lookups; good balance |
| `advanced` | Higher | Highest | Niche topics, recent pages, multi-faceted queries |

### Extract approach

| Approach | When to use |
|----------|------------|
| Search with `include_raw_content=true` | Quick prototyping; single API call; search results likely relevant |
| Direct Extract API | You have specific URLs; you want to filter/curate before extraction |
| Search → filter by score → Extract | Production pipeline; you want control and quality filtering |

### Domain filtering

| Mode | When to use |
|------|------------|
| `include_domains` with `restrict` | You only trust specific sources (hard filter) |
| `include_domains` with `prefer` | You prefer trusted sources but allow others if needed |
| `exclude_domains` | You want to block irrelevant domains but search broadly |

## Workflow

### Typical agent search workflow

1. **Understand the task.** What does the agent need to find? Is the source known or unknown?
2. **Choose the endpoint.** Unknown sources → Search. Have URLs → Extract. Need site structure → Map. Traverse site → Crawl. Need synthesis → Research.
3. **Set search parameters.** For Search: use `search_depth="advanced"` for quality, `chunks_per_source=3` for evidence, `max_results=5` for focused answers. Add domain filters if source trust matters.
4. **Make the request.** Pass `session_id` and `human_id` if part of a multi-step workflow.
5. **Handle the response.** Check `results` array. Filter by `score` if needed (>0.7 is high confidence). Extract URLs for follow-up Extract calls.
6. **Extract if needed.** If Search found sources, call Extract with those URLs, a focused `query`, and `chunks_per_source=3` to pull relevant content.
7. **Deduplicate and consolidate.** If running multiple searches, dedupe URLs and combine unique content chunks.
8. **Pass to LLM.** Feed ranked, scored, deduplicated results as context. Tavily has already filtered and ranked for you.

### Multi-step research workflow

1. **Search** to find relevant sources (`search_depth="advanced"`, `max_results=10`)
2. **Filter** results by score (>0.5) and domain trust
3. **Extract** from top URLs with a focused query and `chunks_per_source=3`
4. **Deduplicate** content across URLs
5. **Validate** extracted content quality
6. **Pass to LLM** for synthesis or further analysis

### Batch search with concurrency

1. **Prepare queries.** Break complex queries into focused sub-queries.
2. **Set concurrency cap.** Use semaphore: `concurrency ≈ (RPM / 60) × avg_latency_s`. At 100 RPM and 3s latency, start with ~5 concurrent requests.
3. **Implement retry logic.** Tag each result `ok`/`error`. Retry failed queries with exponential backoff.
4. **Deduplicate results.** Merge unique URLs and combine their content chunks.
5. **Track credits.** Pass `include_usage=true` and sum usage across responses.
6. **Monitor rate limits.** Watch for 429 responses; back off if needed.

## Common gotchas

- **Query too long.** Keep queries under 1500 characters. Break complex queries into focused sub-queries.
- **Forgetting chunks_per_source.** Without it, Search returns full page summaries (slower, more tokens). Add `chunks_per_source=3` with advanced/basic/fast depth for reranked snippets.
- **Using include_raw_content for everything.** It's slower and uses more tokens. Use Search for discovery, then Extract for full content.
- **Not filtering by score.** Search results are ranked; filter by `score > 0.5` or `> 0.7` to remove low-confidence matches.
- **Hardcoding API keys.** Always use environment variables or secrets managers. Never commit keys to version control.
- **Ignoring rate limits.** Development keys are 100 RPM; production are 1,000 RPM. Crawl and Research have separate, lower limits (100 and 20 RPM). Implement backoff for 429 responses.
- **Not using session_id for multi-step workflows.** Pass the same `session_id` across Search → Extract → follow-up calls so Tavily can attribute the workflow.
- **Crawling without Map first.** Use Map to understand site structure before crawling; it saves credits and prevents wasted traversal.
- **Forgetting exact_match is narrow.** `exact_match=true` requires verbatim phrase matches; it may return empty results. Use only for due diligence or legal/compliance.
- **Not handling failed extractions.** Extract returns `results` (successes) and `failed_results` (per-URL failures). Always check both.
- **Assuming keyless is free forever.** Keyless access is rate-limited. When you hit the cap, sign up for a free API key (1,000 credits/month) to continue.
- **Mixing keyless and API key.** If you send both `X-Tavily-Access-Mode: keyless` and `Authorization: Bearer`, the API key takes precedence.
- **Not respecting robots.txt.** When crawling, respect site policies and implement appropriate delays.

## Verification checklist

Before submitting work with Tavily:

- [ ] **API key is valid.** Test with a simple search request via the Playground or SDK.
- [ ] **Endpoint choice is correct.** Unknown sources → Search. Have URLs → Extract. Need synthesis → Research.
- [ ] **Parameters are optimized.** Search uses `search_depth="advanced"` and `chunks_per_source=3` for quality. Extract has a focused `query`.
- [ ] **Session tracking is in place.** Multi-step workflows pass consistent `session_id` and `human_id`.
- [ ] **Error handling is implemented.** Catch 429 (rate limit), 400 (bad request), 401 (auth), and handle gracefully.
- [ ] **Results are deduplicated.** If running multiple searches, merge unique URLs and combine content chunks.
- [ ] **Scores are checked.** Filter results by `score > 0.5` or `> 0.7` to remove low-confidence matches.
- [ ] **Rate limits are respected.** Concurrency is capped; backoff is implemented for 429 responses.
- [ ] **Credits are tracked.** Pass `include_usage=true` and monitor total spend.
- [ ] **Responses are validated.** Check `results` and `failed_results` arrays; handle failures per URL.
- [ ] **No hardcoded keys.** API keys are in environment variables or secrets managers.

## Resources

- **Full documentation index:** https://docs.tavily.com/llms.txt
- **Full documentation text:** https://docs.tavily.com/llms-full.txt
- **Agent setup guide:** https://docs.tavily.com/agents.md
- **API Reference:** https://docs.tavily.com/documentation/api-reference/introduction.md
- **Best Practices for Search:** https://docs.tavily.com/documentation/best-practices/best-practices-search.md
- **Best Practices for Extract:** https://docs.tavily.com/documentation/best-practices/best-practices-extract.md

---

> For additional documentation and navigation, see: https://docs.tavily.com/llms.txt