> ## Documentation Index
> Fetch the complete documentation index at: https://web2md.org/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# CLI

> Convert URLs to Markdown from the command line — pipe to LLMs, batch process, or save to Obsidian

## Overview

The Web2MD CLI lets you convert any URL to clean Markdown directly from your terminal. Pipe output to LLMs, batch-process URL lists, or ingest content into your Obsidian vault — all without opening a browser.

```bash theme={null}
npx web2md-cli https://example.com/article
```

<Note>
  The CLI requires **Node.js 18+**. Run `node -v` to check your version.
</Note>

## Installation

No installation needed — just run with `npx`:

```bash theme={null}
npx web2md-cli <url> [options]
```

<Warning>
  The official npm package is `web2md-cli`. The package named `web2md` belongs
  to an unrelated project, so do not run `npx web2md` or
  `npm install -g web2md`.
</Warning>

Or install globally for faster startup:

```bash theme={null}
npm install -g web2md-cli
web2md <url> [options]
```

## Modes

Web2MD CLI operates in three modes depending on your configuration:

<CardGroup cols={3}>
  <Card title="Local" icon="laptop">
    **Default mode.** No API key required. Fetches pages and converts locally. Works for most public websites.
  </Card>

  <Card title="Server" icon="server">
    **With API key.** Set `WEB2MD_API_KEY` to unlock Reddit, Fandom/Wikia, and other restricted sites that require server-side handling.
  </Card>

  <Card title="Bridge" icon="bridge">
    **With `--bridge` flag.** Uses your Chrome extension to fetch JS-rendered or login-protected pages that static fetching cannot reach.
  </Card>
</CardGroup>

## Flags

| Flag | Description |
| - | - |
| `--no-images` | Strip image references from output |
| `--no-links` | Strip hyperlinks from output |
| `--meta` | Add YAML frontmatter (title, source, wordCount, tokenCount, readingTime, date) |
| `--json` | Output as JSON `{ markdown, metadata }` |
| `--ndjson` | Output one JSON result per line, suitable for streaming multiple URLs |
| `-o, --output <file>` | Write output to a file |
| `--output-dir <dir>` | Write each URL to a separate `.md` file |
| `--batch <file>` | Read URLs from a file (one per line, `#` = comment) |
| `--vault <dir>` | Obsidian vault mode: saves to `<dir>/raw/` and updates `<dir>/INDEX.md` |
| `--concurrency <n>` | Max parallel fetches (default: 3, max: 20) |
| `--bridge` | Use Chrome extension for JS-rendered or login-protected sites |
| `--force` | Overwrite output files that already exist |
| `--resume` | Skip output files that already exist |
| `--allow-partial` | Return exit code 0 when only some batch items fail |
| `--retries <n>` | Retry network, HTTP 429, and HTTP 5xx failures (default: 2, max: 10) |
| `--timeout <ms>` | Per-request timeout in milliseconds (default: 30000) |
| `-q, --quiet` | Suppress progress messages |

## Commands

Besides converting URLs, the CLI has two subcommands for authentication and Library sync:

| Command | Description |
| - | - |
| `web2md login` | Authenticate once and save your API key to `~/.web2md/config.json` |
| `web2md sync [dir]` | Sync your Web2MD Library to a local directory as Markdown files |

### login

```bash theme={null}
npx web2md-cli login
```

Starts a browser-based device authorization flow: the CLI opens `https://web2md.org` and shows a code to approve. Once approved, your API key is saved to `~/.web2md/config.json` — no need to export `WEB2MD_API_KEY` in every shell. The [MCP server](/docs/advanced/mcp-server#authentication) reads from the same file, so logging in once here also authenticates MCP tool calls.

Prefer to paste a key directly instead of the browser flow?

```bash theme={null}
npx web2md-cli login --key
```

This prompts for a key (`w2m_...`) on stdin and saves it the same way. Get a key from the [dashboard](https://web2md.org/dashboard/api-keys).

### sync

```bash theme={null}
npx web2md-cli sync [dir] [--tag <t>] [--full]
```

Syncs your Library — every page you've saved via the extension, dashboard, or API — to a local directory as Markdown files with frontmatter, organized by tag. Requires being logged in (or `WEB2MD_API_KEY` set).

| Flag | Description |
| - | - |
| `dir` | Target directory (default: `./web2md-vault`) |
| `--tag <t>` | Only sync items with this tag |
| `--full` | Ignore the incremental cursor and re-scan full history |

Sync is incremental by default: it tracks a cursor in `<dir>/.web2md-sync.json` and only pulls new items on subsequent runs. Point Obsidian (or any Markdown-reading tool) at the target directory to browse your Library as notes.

```bash theme={null}
npx web2md-cli login
npx web2md-cli sync ~/Documents/Web2MD-Library
```

## Environment variables

| Variable | Description |
| - | - |
| `WEB2MD_API_KEY` | API key (`w2m_xxx`) for Reddit, restricted sites, and `sync` — takes priority over `~/.web2md/config.json` |
| `WEB2MD_API_URL` | Override the API base URL |
| `WEB2MD_EXTENSION_ID` | Override the Chrome extension ID for `--bridge` mode |

<Tip>
  Add these to your shell profile (`~/.zshrc` or `~/.bashrc`) so they persist across sessions:

  ```bash theme={null}
  export WEB2MD_API_KEY="w2m_your_key_here"
  ```
</Tip>

## Usage examples

### Basic conversion

```bash theme={null}
npx web2md-cli https://example.com/article
```

Prints Markdown to stdout.

### Pipe to an LLM

```bash theme={null}
npx web2md-cli https://react.dev/learn/thinking-in-react | llm "Summarize this page"
```

```bash theme={null}
npx web2md-cli https://docs.python.org/3/tutorial/classes.html | claude "Explain the key concepts"
```

### Save to file

```bash theme={null}
npx web2md-cli https://example.com/article -o article.md
```

```bash theme={null}
npx web2md-cli https://example.com/article --meta -o article.md
```

The `--meta` flag prepends YAML frontmatter with title, source URL, word count, token count, reading time, and date.

### Batch from file

Create a file `urls.txt`:

```text theme={null}
# Research papers
https://arxiv.org/abs/2301.00001
https://arxiv.org/abs/2301.00002

# Blog posts
https://example.com/blog/post-1
https://example.com/blog/post-2
```

Then run:

```bash theme={null}
npx web2md-cli --batch urls.txt --output-dir ./research --concurrency 5
```

Each URL is saved as a separate `.md` file in the `./research` directory.
Filenames contain a short URL hash, so pages with identical titles do not
overwrite one another. Re-run an interrupted batch with `--resume`.

By default, any failed URL makes the command return a non-zero exit code. Use
`--allow-partial` only when partial results should count as success.

### Obsidian vault ingestion

```bash theme={null}
npx web2md-cli --batch urls.txt --vault ~/Documents/MyVault
```

This saves each page to `~/Documents/MyVault/raw/` and updates `~/Documents/MyVault/INDEX.md` with links to all converted pages.

### Reddit with API key

```bash theme={null}
export WEB2MD_API_KEY="w2m_your_key_here"
npx web2md-cli https://www.reddit.com/r/LocalLLaMA/comments/example
```

<Warning>
  Reddit requires a valid API key. Without one, Reddit URLs will fail due to Reddit's bot restrictions.
</Warning>

### Bridge mode

Use the Chrome extension to handle JS-rendered or login-protected pages:

```bash theme={null}
npx web2md-cli --bridge https://app.example.com/dashboard
```

<Note>
  Bridge mode requires the Web2MD Chrome extension to be installed and Chrome to be running. The CLI communicates with the extension via Chrome's native messaging protocol.
</Note>

### JSON output

```bash theme={null}
npx web2md-cli --json https://example.com/article
```

Returns structured output:

```json theme={null}
{
  "markdown": "# Article Title\n\nContent here...",
  "metadata": {
    "title": "Article Title",
    "source": "https://example.com/article",
    "wordCount": 1250,
    "tokenCount": 1680,
    "readingTime": 5,
    "date": "2026-04-11T10:30:00.000Z"
  }
}
```

Useful for programmatic consumption or piping to `jq`:

```bash theme={null}
npx web2md-cli --json https://example.com/article | jq '.metadata.tokenCount'
```

With multiple URLs, `--json` returns one valid JSON array containing a result
or error record for each URL:

```bash theme={null}
npx web2md-cli https://example.com/a https://example.com/b --json
```

For streaming pipelines, use newline-delimited JSON:

```bash theme={null}
npx web2md-cli --batch urls.txt --ndjson
```

## Optimized sites

Web2MD includes built-in adapters for these sites, producing cleaner output than generic conversion:

<AccordionGroup>
  <Accordion title="Sites with optimized support">
    * **Wikipedia** — clean article extraction, infobox handling
    * **arXiv** — paper abstracts and metadata
    * **Hacker News** — threads with comments
    * **GitHub** — Issues and Pull Requests
    * **Stack Overflow** — questions and answers
    * **dev.to** — blog posts
    * **Medium** — articles (bypasses paywall preview)
    * **Substack** — newsletter posts
    * **OpenAI Docs** — documentation pages
    * **Mintlify-based docs** — documentation sites built on Mintlify
    * **Reddit** — posts and comments (requires API key)
  </Accordion>
</AccordionGroup>

## Common workflows

<AccordionGroup>
  <Accordion title="Feed documentation to an AI agent">
    ```bash theme={null}
    npx web2md-cli --batch docs-urls.txt --output-dir ./context --quiet
    ```

    Point your AI agent's context directory at `./context` for grounded answers.
  </Accordion>

  <Accordion title="Build a research corpus">
    ```bash theme={null}
    npx web2md-cli --batch papers.txt --vault ~/Obsidian/Research --meta --concurrency 10
    ```

    Creates an indexed, searchable research vault in Obsidian.
  </Accordion>

  <Accordion title="Strip formatting for LLM input">
    ```bash theme={null}
    npx web2md-cli --no-images --no-links https://example.com/article | llm "Analyze this"
    ```

    Removes images and links to reduce token usage when piping to LLMs.
  </Accordion>
</AccordionGroup>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.