Claudeweb scrapingMarkdownAI toolsChrome extension

Can Claude Scan Websites? What It Can Read, What It Misses, and a Workaround

Zephyr Whimsy2026-11-167 min read

People ask some version of "can Claude scan websites?" all the time. The short answer is yes, partly. Claude can read some web pages on its own. On others it fails without telling you, or it gives you a summary when you wanted the full text.

I spent a couple of weeks giving Claude URLs from my normal work: docs pages, product pages, a few paywalled articles, our internal wiki, and some dashboards I'm logged into. This post covers what worked, what didn't, and the workflow I use now when I need Claude to see a whole page.

The short answer

Depending on your plan and settings, Claude can search the web and fetch a public URL that you paste into the chat. When it works, Claude pulls the page, reads the text, and answers questions about it. For a public blog post or a documentation page, that is often enough.

There are limits, though. Claude fetches the page from Anthropic's servers, not from your browser. That one fact explains almost every failure I hit.

What I tested

I sorted my test URLs into four groups:

  1. Public, static pages (blog posts, docs, Wikipedia). Claude handled these well. It read the content and answered accurately.
  2. JavaScript-heavy pages (single-page apps, some pricing pages, pages that load content after scrolling). The results were mixed. Sometimes Claude got the full content. Sometimes it got an empty shell or only the navigation text.
  3. Paywalled articles. Claude got the teaser paragraph or the paywall message, never the article. That is the correct behavior, because Claude doesn't have my subscription.
  4. Logged-in pages (our Notion workspace, a Jira ticket, a Google Search Console report, a private GitHub repo page). All of these failed. Claude saw a login screen or an error.

The fourth group was the one that mattered most to me. Most of the pages I actually want help with sit behind a login.

Why Claude cannot see what you see

When you open a page in Chrome, your browser sends your cookies and session, runs the page's JavaScript, and renders the final content. When Claude fetches a URL, it makes a fresh request with none of that. As far as the website knows, Claude is an anonymous visitor.

So:

  • Anything behind a login shows Claude a login wall.
  • Anything behind a paywall shows Claude the paywall.
  • Some sites block automated requests outright.
  • Content that only appears after a click, a scroll, or a script run may never load.

There is a second, quieter problem. Even when a fetch succeeds, you don't control what Claude actually keeps. On long pages I sometimes got answers that clearly came from only the first part of the article. Claude didn't say the page had been cut short. The answer just missed things.

Server-side tools have the same limit

Before building my own workflow, I tried the usual tools for turning a URL into text an LLM can read. They are good tools:

  • Jina Reader is fast and free to try. You put r.jina.ai/ in front of a URL and get Markdown back. It works very well on public pages.
  • Firecrawl is the better choice when you need to crawl a whole site, handle many pages, or build a pipeline with an API.
  • MarkDownload is a solid open-source browser extension for clipping pages to Markdown files, and it's popular with Obsidian users.

Jina and Firecrawl run on a server, so they hit the same wall Claude does: no cookies, no session, no access to your logged-in pages. MarkDownload runs in the browser, but it's built for saving notes rather than preparing content for an AI. It has no token count and no way to send the result straight to a chat.

The workaround: convert the page in your browser, then paste

What works reliably for me is to turn the page into Markdown inside my own browser, where I'm already logged in, and then hand that Markdown to Claude.

That is what Web2MD does. It's a Chrome extension. Click it on any page and it converts the rendered content to clean Markdown. Because it runs in your browser session, it sees exactly what you see, including dashboards, internal wikis, tickets, and articles you pay for.

Here's what I got from a logged-in Jira ticket that Claude's own fetch couldn't open:

# PAY-2184: Refund webhook fires twice on partial captures

**Status:** In Progress
**Assignee:** Dana K.
**Priority:** High

## Description

When a merchant issues a partial capture followed by a refund,
the `refund.succeeded` webhook is delivered twice within ~200ms.

## Steps to reproduce

1. Create a payment intent for $100
2. Capture $60
3. Refund $20
4. Observe two webhook deliveries with the same event ID

## Comments

**Sam R.:** Looks like a race in the retry queue. Idempotency key
is set after enqueue, not before.

I pasted that into Claude with "Find the likely root cause and suggest a fix." It had the full ticket, comments included, with none of the sidebar clutter.

Why clean Markdown beats raw copy-paste

You can just select all and paste. I did that for months. The trouble is that raw copy-paste brings along menus, cookie banners, "related articles," footer links, and broken tables. All of that costs tokens and makes the answer worse.

Here's part of a pricing table from a JavaScript-rendered page. A plain copy-paste flattened it into one unreadable line. The Markdown version keeps the structure:

## Plans

| Plan       | Price     | Seats | API calls / month |
|------------|-----------|-------|-------------------|
| Starter    | $0        | 1     | 1,000             |
| Team       | $29/mo    | 5     | 50,000            |
| Business   | $99/mo    | 20    | 500,000           |

> All plans include SSO on annual billing.

Claude reads tables like this correctly. It doesn't do nearly as well with a flattened mess of numbers.

Web2MD also shows a token count before you send anything. That helped me more than I expected. Twice I found a page was about 40k tokens because of an enormous comment thread, so I trimmed it before pasting rather than guessing why Claude's answer felt thin. There's also a one-click send to Claude, ChatGPT, or Cursor, so you don't have to switch tabs and paste by hand.

Privacy

The conversion happens locally in your browser. Page content isn't sent to a third-party scraping server before it reaches the AI tool you chose. For internal tickets and client dashboards, that mattered to me. Remember that whatever you paste into Claude still goes to Anthropic, so the usual judgment about sensitive data still applies.

Honest limits

  • Chrome only (and Chromium-based browsers). There's no Firefox or Safari version right now.
  • Free tier is 3 conversions per day. That covers occasional use. If you convert pages all day, Pro is $9/month. See pricing.
  • It converts one page at a time, as you see it. If you need to crawl 5,000 URLs on a schedule, Firecrawl is the right tool.
  • It doesn't get around anything. If you can't see the content in your browser, Web2MD can't either.

So, can Claude scan websites?

For public, mostly static pages: yes. Paste the URL and Claude will usually do fine.

For anything behind a login or a paywall, or anything heavily rendered with JavaScript, Claude's fetch will often fail or come back incomplete. In those cases, convert the page in your own browser and give Claude the Markdown. You get the full content and a known token count, and you decide exactly what Claude sees.

If you want to try it, install Web2MD and convert the next page Claude can't open. The free tier needs no API key and no signup to start. For more workflows like this, browse the Web2MD blog.

Related Articles

Most Read

last 30 days
  1. #1Can Claude Read Reddit? No — Here's the One-Click Fix (2026)
  2. #2HTML vs Markdown for LLMs: I Wasted 67% of My Tokens for a Year
  3. #3Reducing Token Waste in ChatGPT and Claude: 7 Techniques That Cut Costs 72%
  4. #4Obsidian Web Clipper Official Plugin 2026: Complete Guide + When You Need More
  5. #5Reddit JSON API vs Scraping: The Honest 2026 Comparison for Developers

Latest Articles