# Proxy Setup Guide

## Why Proxies Matter

**Datacenter IPs = INSTANT BLOCK** on protected sites like Airbnb, Yelp, and anything with Cloudflare/Datadome.

Anti-bot systems check IP reputation scores. AWS, DigitalOcean, Vultr, Linode IPs have high "fraud scores" and are blocked immediately.

## Proxy Types (Best to Worst for Anti-Bot)

| Type | Detection Risk | Speed | Cost | Best For |
|------|---------------|-------|------|----------|
| **Mobile 4G/5G** | Very Low | Medium | $$$ | Hardest targets |
| **Residential Rotating** | Low | Medium | $$ | General scraping |
| **Residential Static** | Low | Fast | $$ | Session-based work |
| **ISP Proxies** | Medium | Fast | $$ | Light protection |
| **Datacenter** | BLOCKED | Fast | $ | Non-protected sites only |

## Recommended Providers

### Residential/Mobile Proxies
- **Bright Data** (formerly Luminati) - Enterprise grade
- **Oxylabs** - Good residential pool
- **Smartproxy** - Budget friendly
- **IPRoyal** - Cheap residential
- **PacketStream** - P2P residential

### Proxy Format

```
# Without auth
http://host:port
socks5://host:port

# With auth
http://username:password@host:port
socks5://username:password@host:port

# Rotating (some providers)
http://username:password@gate.provider.com:port
```

## Configuration Examples

### Camoufox

```python
from camoufox.async_api import AsyncCamoufox

async with AsyncCamoufox(
    proxy={
        "server": "http://proxy.example.com:8080",
        "username": "user",
        "password": "pass"
    }
) as browser:
    page = await browser.new_page()
```

### curl_cffi

```python
from curl_cffi import requests

response = requests.get(
    "https://example.com",
    proxies={
        "http": "http://user:pass@proxy.example.com:8080",
        "https": "http://user:pass@proxy.example.com:8080"
    },
    impersonate="chrome120"
)
```

### Environment Variable

```bash
export HTTP_PROXY="http://user:pass@proxy.example.com:8080"
export HTTPS_PROXY="http://user:pass@proxy.example.com:8080"
```

## Session Stickiness

**Don't rotate every request!** This looks like a bot.

Real users:
- Stay on same IP for 10-30 minutes
- Make multiple requests from same session
- Have consistent fingerprints

```python
# Good: Sticky session
session_proxy = "http://user-session-abc123:pass@gate.provider.com:8080"

# Bad: Rotate every request
rotating_proxy = "http://user:pass@rotate.provider.com:8080"  # Only for initial requests
```

## Geo-Targeting

Match proxy location to expected user location:

```python
# Scraping US Airbnb listings? Use US proxy
us_proxy = "http://user-country-us:pass@gate.provider.com:8080"

# Scraping UK Yelp? Use UK proxy  
uk_proxy = "http://user-country-gb:pass@gate.provider.com:8080"
```

## Testing Your Proxy

```bash
# Check IP and detection
distrobox-enter pybox -- python -c "
from curl_cffi import requests
r = requests.get('https://httpbin.org/ip', impersonate='chrome120', 
                 proxies={'https': 'http://user:pass@proxy:port'})
print(r.json())
"

# Check for bot detection
distrobox-enter pybox -- python scripts/nodriver-fetch.py \
    "https://bot.sannysoft.com" \
    --proxy "http://user:pass@proxy:port" \
    --screenshot bot-test.png
```

## Cost Optimization

1. **Use datacenter for non-protected sites** - Save residential bandwidth
2. **Cache aggressively** - Don't re-fetch unchanged pages
3. **Use API endpoints** - Faster than full browser rendering
4. **Rotate only when blocked** - Start with sticky sessions
5. **Off-peak scraping** - Some providers charge less at night

## Troubleshooting

| Issue | Cause | Fix |
|-------|-------|-----|
| 403 immediately | Datacenter IP | Use residential |
| 403 after few requests | IP burned | Rotate to new IP |
| Captcha appears | Suspicious behavior | Slow down, add randomness |
| Connection timeout | Bad proxy | Try different proxy/provider |
| SSL errors | Proxy doesn't support HTTPS | Use HTTPS proxy or tunnel |