Tag: anti bot

4 articles

mcpreddithacker newsclaude codecursoragentanti-botbatch conversionweb2md

Let Your Agent Read the Sites That Block Crawlers: Reddit, HN, and More

Firecrawl, Jina, and Parallel all fetch from datacenter IPs — which is exactly what Reddit and X block. Route the read through your own browser instead and your agent gets in. Measured across Reddit, Hacker News, and X, with the failures listed too.

2026-09-047 min read

redditclaudeai workflowraganti-botcontext

Send a Reddit Thread to Claude as Context (Without Reddit's Anti-Bot Blocking You)

Reddit blocks Firecrawl, Jina, even its own API for anonymous reads. The workaround: read the thread from inside your logged-in browser. Full workflow.

2026-05-104 min read

firecrawl alternativerag pipelineweb scrapingai workflowmcp serverclaude codecursoranti-botobsidian

Firecrawl Costs Too Much for Hobby RAG — Here's a $9 Alternative That Uses Your Real Browser

Firecrawl Extract is great, but $188/mo is wrong for solo RAG. Flip the architecture: extract inside Chrome with your session. Web2MD does it for $9.

2026-05-077 min read

xiaohongshurednote小红书web to markdownai workflowragchinese social mediaanti-bot

How to Convert Xiaohongshu (RED / 小红书) Posts to Markdown — and Feed Them to Claude or ChatGPT

Xiaohongshu blocks every server-side scraper — anti-bot signing rotates monthly. The reliable way to get RED posts into Markdown: read from your open browser.

2026-05-077 min read