Local-first web scraping and crawling tool that extracts static pages via trafilatura with optional Firecrawl fallback for JavaScript rendering. Includes search, scraping, mapping, and crawling capabilities.
webcrawl-mcp is an MCP server that provides web scraping, crawling, and search capabilities. It prioritizes local-first processing using trafilatura to extract content from static web pages, with an optional Firecrawl integration for handling JavaScript-heavy sites when needed. The tool offers four main functions: scrape individual pages, search using DuckDuckGo, map website structures, and crawl multiple pages.
Install the package from the GitHub repository and configure it as an MCP server. Basic setup involves cloning the repository and following the installation instructions. For optional Firecrawl functionality, provide API credentials when JavaScript rendering is required for specific pages.
Monday.com MCP Server streamlines board management, item operations, and workflow automation for teams. I…
by NotionFlow
Sentry MCP Server provides comprehensive error tracking and performance monitoring, helping developers id…
by AnalyticsPro
Cloudflare MCP Server simplifies Cloudflare management by providing tools for DNS management, Workers dep…
by PricingBot