每日备份 2026-06-10 12:57:13
This commit is contained in:
1 parent
fda2ccb8c2
commit
3d2e638c6a
244 files changed
+58890
-3
No files matched your search
@@ -0,0 +1,42 @@
|
||||
---
|
||||
name: firecrawl
|
||||
description: Web search and scraping via Firecrawl API. Use when you need to search the web, scrape websites (including JS-heavy pages), crawl entire sites, or extract structured data from web pages. Requires FIRECRAWL_API_KEY environment variable.
|
||||
---
|
||||
|
||||
# Firecrawl
|
||||
|
||||
Web search and scraping via Firecrawl API.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Set `FIRECRAWL_API_KEY` in your environment or `.env` file:
|
||||
```bash
|
||||
export FIRECRAWL_API_KEY=fc-xxxxxxxxxx
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Search the web
|
||||
```bash
|
||||
firecrawl_search "your search query" --limit 10
|
||||
```
|
||||
|
||||
### Scrape a single page
|
||||
```bash
|
||||
firecrawl_scrape "https://example.com"
|
||||
```
|
||||
|
||||
### Crawl an entire site
|
||||
```bash
|
||||
firecrawl_crawl "https://example.com" --max-pages 50
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
See [references/api.md](references/api.md) for detailed API documentation and advanced options.
|
||||
|
||||
## Scripts
|
||||
|
||||
- `scripts/search.py` - Search the web with Firecrawl
|
||||
- `scripts/scrape.py` - Scrape a single URL
|
||||
- `scripts/crawl.py` - Crawl an entire website
|
||||
@@ -0,0 +1,11 @@
|
||||
{
|
||||
"owner": "ashwingupy",
|
||||
"slug": "firecrawl-search",
|
||||
"displayName": "Firecrawl Search",
|
||||
"latest": {
|
||||
"version": "1.0.0",
|
||||
"publishedAt": 1769544521483,
|
||||
"commit": "https://github.com/clawdbot/skills/commit/7c6e584a7b59012f3c4566b53f748a946d5a9a24"
|
||||
},
|
||||
"history": []
|
||||
}
|
||||
@@ -0,0 +1,85 @@
|
||||
# Firecrawl API Reference
|
||||
|
||||
## Environment
|
||||
|
||||
Set your API key:
|
||||
```bash
|
||||
export FIRECRAWL_API_KEY=fc-xxxxxxxxxx
|
||||
```
|
||||
|
||||
## API Endpoints
|
||||
|
||||
### Search
|
||||
```bash
|
||||
POST https://api.firecrawl.dev/v1/search
|
||||
```
|
||||
|
||||
Request body:
|
||||
```json
|
||||
{
|
||||
"query": "search terms",
|
||||
"limit": 10,
|
||||
"lang": "en",
|
||||
"country": "us"
|
||||
}
|
||||
```
|
||||
|
||||
### Scrape
|
||||
```bash
|
||||
POST https://api.firecrawl.dev/v1/scrape
|
||||
```
|
||||
|
||||
Request body:
|
||||
```json
|
||||
{
|
||||
"url": "https://example.com",
|
||||
"formats": ["markdown", "html", "screenshot"],
|
||||
"onlyMainContent": true,
|
||||
"includeTags": ["h1", "p", "article"],
|
||||
"excludeTags": ["nav", "footer", "aside"]
|
||||
}
|
||||
```
|
||||
|
||||
### Crawl
|
||||
```bash
|
||||
POST https://api.firecrawl.dev/v1/crawl
|
||||
```
|
||||
|
||||
Request body:
|
||||
```json
|
||||
{
|
||||
"url": "https://example.com",
|
||||
"limit": 50,
|
||||
"excludePaths": ["/blog", "/admin"],
|
||||
"scrapeOptions": {
|
||||
"formats": ["markdown"],
|
||||
"onlyMainContent": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Check status:
|
||||
```bash
|
||||
GET https://api.firecrawl.dev/v1/crawl/{job_id}
|
||||
```
|
||||
|
||||
## Response Format
|
||||
|
||||
All responses follow this structure:
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"data": { ... },
|
||||
"status": "completed"
|
||||
}
|
||||
```
|
||||
|
||||
## Rate Limits
|
||||
|
||||
- Search: Check your Firecrawl dashboard
|
||||
- Scrape: Check your Firecrawl dashboard
|
||||
- Crawl: Check your Firecrawl dashboard
|
||||
|
||||
## Pricing
|
||||
|
||||
See https://firecrawl.dev/pricing for current rates.
|
||||
Reference in new issue
Block a user