每日备份 2026-06-10 12:57:13

2026-06-10 12:57:14 +08:00
1 parent fda2ccb8c2
commit 3d2e638c6a
@@ -0,0 +1,239 @@
+# Browser Fingerprint Checks
+
+What anti-bot systems check and how to evade.
+
+## Detection Layers
+
+```
+┌─────────────────────────────────────────────┐
+│           Layer 1: Network/IP               │
+│  • IP reputation score                      │
+│  • Datacenter vs residential                │
+│  • Geo-location consistency                 │
+├─────────────────────────────────────────────┤
+│           Layer 2: TLS Fingerprint          │
+│  • JA3/JA4 hash                            │
+│  • Cipher suite order                       │
+│  • TLS extensions                           │
+├─────────────────────────────────────────────┤
+│           Layer 3: HTTP Headers             │
+│  • User-Agent consistency                   │
+│  • Header order                             │
+│  • Accept-Language                          │
+├─────────────────────────────────────────────┤
+│           Layer 4: Browser Fingerprint      │
+│  • navigator.webdriver                      │
+│  • Canvas/WebGL hash                        │
+│  • Audio fingerprint                        │
+│  • Font enumeration                         │
+├─────────────────────────────────────────────┤
+│           Layer 5: Behavioral               │
+│  • Mouse movements                          │
+│  • Scroll patterns                          │
+│  • Timing analysis                          │
+│  • Click patterns                           │
+└─────────────────────────────────────────────┘
+```
+
+## Specific Checks & Countermeasures
+
+### 1. navigator.webdriver
+
+**What:** `navigator.webdriver` returns `true` for automated browsers.
+
+**Check:**
+```javascript
+navigator.webdriver  // true = bot
+```
+
+**Countermeasure:**
+- Nodriver: Automatically patches
+- Camoufox: Patched at C++ level
+- Playwright-stealth: `delete navigator.webdriver` (weak, can be detected)
+
+### 2. Chrome DevTools Protocol (CDP)
+
+**What:** Presence of CDP side-channels reveals automation.
+
+**Check:**
+```javascript
+// Sites check for CDP artifacts
+window.cdc_adoQpoasnfa76pfcZLmcfl_Array
+window.cdc_adoQpoasnfa76pfcZLmcfl_Promise
+```
+
+**Countermeasure:**
+- Nodriver: Uses CDP but patches artifacts
+- Camoufox: No CDP (native Firefox)
+- Patchright: Patches CDP artifacts
+
+### 3. Canvas Fingerprint
+
+**What:** Each GPU/driver combo renders slightly differently.
+
+**Check:**
+```javascript
+canvas.toDataURL()  // Unique hash per system
+```
+
+**Countermeasure:**
+- Camoufox: Injects deterministic noise
+- Some detection tools look for perfectly identical canvas = bot
+
+### 4. WebGL Fingerprint
+
+**What:** GPU vendor/renderer strings.
+
+**Check:**
+```javascript
+gl.getParameter(gl.RENDERER)  // "ANGLE (Intel HD Graphics)"
+gl.getParameter(gl.VENDOR)    // "Google Inc."
+```
+
+**Countermeasure:**
+- Camoufox: Spoofs to common values
+- Avoid VMs with "llvmpipe" or "SwiftShader" (instant bot flag)
+
+### 5. Audio Fingerprint
+
+**What:** AudioContext processing variations.
+
+**Check:**
+```javascript
+new AudioContext().createOscillator()  // Unique waveform
+```
+
+**Countermeasure:**
+- Camoufox: Adds noise to audio processing
+
+### 6. Font Enumeration
+
+**What:** Bots often have minimal fonts installed.
+
+**Check:**
+```javascript
+// Test if specific fonts render
+document.fonts.check('12px "Segoe UI"')
+```
+
+**Countermeasure:**
+- Install standard fonts in your environment
+- Camoufox: Spoofs font list
+
+### 7. Screen/Window Size
+
+**What:** Unusual dimensions = bot.
+
+**Check:**
+```javascript
+screen.width, screen.height
+window.innerWidth, window.innerHeight
+```
+
+**Countermeasure:**
+- Use standard resolutions: 1920x1080, 1366x768
+- Avoid: 800x600, odd dimensions
+
+### 8. Timezone Consistency
+
+**What:** Timezone must match IP geo-location.
+
+**Check:**
+```javascript
+Intl.DateTimeFormat().resolvedOptions().timeZone
+new Date().getTimezoneOffset()
+```
+
+**Countermeasure:**
+- Set timezone to match proxy location:
+  ```bash
+  TZ="America/New_York" python script.py
+  ```
+
+### 9. Language Consistency
+
+**What:** Browser language should match geo.
+
+**Check:**
+```javascript
+navigator.language       // "en-US"
+navigator.languages      // ["en-US", "en"]
+```
+
+**Countermeasure:**
+- Configure browser locale to match proxy:
+  ```python
+  # Camoufox
+  async with AsyncCamoufox(locale="en-US") as browser:
+  ```
+
+### 10. Hardware Concurrency
+
+**What:** VMs often report 1-2 CPUs.
+
+**Check:**
+```javascript
+navigator.hardwareConcurrency  // 1 or 2 = suspicious
+```
+
+**Countermeasure:**
+- Camoufox: Spoofs to 4-8
+- Or use real hardware, not minimal VM
+
+### 11. Headless Detection
+
+**What:** Multiple signals reveal headless mode.
+
+**Checks:**
+```javascript
+navigator.plugins.length === 0
+navigator.mimeTypes.length === 0
+window.chrome === undefined
+```
+
+**Countermeasure:**
+- Use "new headless" mode: `headless="new"`
+- Or run headed in Xvfb
+- Camoufox headless is well-patched
+
+### 12. Mouse/Keyboard Events
+
+**What:** Bots often have no mouse history.
+
+**Check:**
+```javascript
+// Sites track mouse movement patterns
+document.addEventListener('mousemove', analyze)
+```
+
+**Countermeasure:**
+- Inject random mouse movements
+- Nodriver/Camoufox: Built-in human-like movement
+- Don't teleport cursor - use bezier curves
+
+## Test Sites
+
+Check your stealth setup:
+
+| Site | What It Tests |
+|------|--------------|
+| https://bot.sannysoft.com | Comprehensive bot detection |
+| https://browserleaks.com | All fingerprints |
+| https://nowsecure.nl | Cloudflare detection |
+| https://pixelscan.net | Bot detection score |
+| https://abrahamjuliot.github.io/creepjs | Advanced fingerprinting |
+
+## Tool Comparison
+
+| Check | Camoufox | Plain Playwright |
+|-------|----------|------------------|
+| webdriver | ✅ | ❌ |
+| CDP artifacts | ✅ (no CDP) | ❌ |
+| Canvas | ✅ | ❌ |
+| WebGL | ✅ | ❌ |
+| Audio | ✅ | ❌ |
+| Fonts | ✅ | ❌ |
+| Headless | ✅ | ❌ |
+| TLS | ✅ | ❌ |
+
+Legend: ✅ = patched, ❌ = detected
@@ -0,0 +1,145 @@
+# Proxy Setup Guide
+
+## Why Proxies Matter
+
+**Datacenter IPs = INSTANT BLOCK** on protected sites like Airbnb, Yelp, and anything with Cloudflare/Datadome.
+
+Anti-bot systems check IP reputation scores. AWS, DigitalOcean, Vultr, Linode IPs have high "fraud scores" and are blocked immediately.
+
+## Proxy Types (Best to Worst for Anti-Bot)
+
+| Type | Detection Risk | Speed | Cost | Best For |
+|------|---------------|-------|------|----------|
+| **Mobile 4G/5G** | Very Low | Medium | $$$ | Hardest targets |
+| **Residential Rotating** | Low | Medium | $$ | General scraping |
+| **Residential Static** | Low | Fast | $$ | Session-based work |
+| **ISP Proxies** | Medium | Fast | $$ | Light protection |
+| **Datacenter** | BLOCKED | Fast | $ | Non-protected sites only |
+
+## Recommended Providers
+
+### Residential/Mobile Proxies
+- **Bright Data** (formerly Luminati) - Enterprise grade
+- **Oxylabs** - Good residential pool
+- **Smartproxy** - Budget friendly
+- **IPRoyal** - Cheap residential
+- **PacketStream** - P2P residential
+
+### Proxy Format
+
+```
+# Without auth
+http://host:port
+socks5://host:port
+
+# With auth
+http://username:password@host:port
+socks5://username:password@host:port
+
+# Rotating (some providers)
+http://username:password@gate.provider.com:port
+```
+
+## Configuration Examples
+
+### Camoufox
+
+```python
+from camoufox.async_api import AsyncCamoufox
+
+async with AsyncCamoufox(
+    proxy={
+        "server": "http://proxy.example.com:8080",
+        "username": "user",
+        "password": "pass"
+    }
+) as browser:
+    page = await browser.new_page()
+```
+
+### curl_cffi
+
+```python
+from curl_cffi import requests
+
+response = requests.get(
+    "https://example.com",
+    proxies={
+        "http": "http://user:pass@proxy.example.com:8080",
+        "https": "http://user:pass@proxy.example.com:8080"
+    },
+    impersonate="chrome120"
+)
+```
+
+### Environment Variable
+
+```bash
+export HTTP_PROXY="http://user:pass@proxy.example.com:8080"
+export HTTPS_PROXY="http://user:pass@proxy.example.com:8080"
+```
+
+## Session Stickiness
+
+**Don't rotate every request!** This looks like a bot.
+
+Real users:
+- Stay on same IP for 10-30 minutes
+- Make multiple requests from same session
+- Have consistent fingerprints
+
+```python
+# Good: Sticky session
+session_proxy = "http://user-session-abc123:pass@gate.provider.com:8080"
+
+# Bad: Rotate every request
+rotating_proxy = "http://user:pass@rotate.provider.com:8080"  # Only for initial requests
+```
+
+## Geo-Targeting
+
+Match proxy location to expected user location:
+
+```python
+# Scraping US Airbnb listings? Use US proxy
+us_proxy = "http://user-country-us:pass@gate.provider.com:8080"
+
+# Scraping UK Yelp? Use UK proxy  
+uk_proxy = "http://user-country-gb:pass@gate.provider.com:8080"
+```
+
+## Testing Your Proxy
+
+```bash
+# Check IP and detection
+distrobox-enter pybox -- python -c "
+from curl_cffi import requests
+r = requests.get('https://httpbin.org/ip', impersonate='chrome120', 
+                 proxies={'https': 'http://user:pass@proxy:port'})
+print(r.json())
+"
+
+# Check for bot detection
+distrobox-enter pybox -- python scripts/nodriver-fetch.py \
+    "https://bot.sannysoft.com" \
+    --proxy "http://user:pass@proxy:port" \
+    --screenshot bot-test.png
+```
+
+## Cost Optimization
+
+1. **Use datacenter for non-protected sites** - Save residential bandwidth
+2. **Cache aggressively** - Don't re-fetch unchanged pages
+3. **Use API endpoints** - Faster than full browser rendering
+4. **Rotate only when blocked** - Start with sticky sessions
+5. **Off-peak scraping** - Some providers charge less at night
+
+## Troubleshooting
+
+| Issue | Cause | Fix |
+|-------|-------|-----|
+| 403 immediately | Datacenter IP | Use residential |
+| 403 after few requests | IP burned | Rotate to new IP |
+| Captcha appears | Suspicious behavior | Slow down, add randomness |
+| Connection timeout | Bad proxy | Try different proxy/provider |
+| SSL errors | Proxy doesn't support HTTPS | Use HTTPS proxy or tunnel |