Global (EN)
Setup GuideSaved Tools
Back to All MCP Servers
C

Crawler API

Verified Integration
Author: @jawa7•Category: ServerApplication
GitHub Repository
JSON-RPC 2.0
Protocol Standard
Sub-second
Execution Latency
Active
Operational Status
Tags:#Node.js#Git#Database#GitHub#Claude#Search#PostgreSQL#API#AI

Client Configuration

— Connect Crawler API to Claude Desktop or Cursor in seconds
{
  "mcpServers": {
    "crawler-api": {
      "command": "npx",
      "args": [
        "-y",
        "@modelcontextprotocol/server-crawler-api"
      ],
      "env": {}
    }
  }
}
Paste into ~/Library/Application Support/Claude/claude_desktop_config.json (macOS) or %APPDATA%\Claude\claude_desktop_config.json (Windows).
Architecture & Capabilities

System Overview

Provides a modular web crawler and a REST API for managing and storing webpage data, integrated with an MCP server.

Indexed Date

9/16/2026

License

Open Source

Protocol Layer

stdio / SSE RPC

Frequently Asked Questions

Architecture and operational details for Crawler API

Yes, a key feature is its MCP (Model Context Protocol) server, which enables integration with external LLMs and agents via HTTP. This allows for complex workflows like checking for existing data, crawling new content, and summarizing results for AI processing.

Related MCP Servers

Browse all servers →
C
@carboneio
Carbone

Empower AI assistants to generate and convert documents, manage templates, and automate document workflows efficiently.

ClaudeSearchDatabase+65 FAQs
Learn more
C
@sebastienrousseau
CloudCDN

Provides a multi-tenant, AI-native Content Delivery Network deployable on Cloudflare, featuring sub-100ms TTFB, AI agent controllability, and comprehensive accessibility.

ClaudeSearchDatabase+65 FAQs
Learn more
C

Deploy a Model Context Protocol server on Cloudflare Workers without requiring authentication.

ClaudeSearchDatabase+55 FAQs
Learn more