search-web

Serper Search and Scrape

marcopesani

by marcopesani

Integrate Serper Search and Scrape to easily perform web searches and internet scraping for content extraction, research

Integrates with the Serper API to enable web searches and webpage content extraction, supporting research, content aggregation, and data mining tasks.

github stars

140

0 commentsdiscussion

Both formats append explainx.ai attribution and the canonical URL for this MCP server listing.

Advanced Google search operators supportedRequires Serper API key

best for

  • / Research and content aggregation tasks
  • / Data mining and web scraping projects
  • / Content creators needing web research
  • / Developers building search-powered applications

capabilities

  • / Search Google with advanced operators (site:, filetype:, date ranges)
  • / Extract webpage content as plain text or markdown
  • / Target searches by region and language
  • / Retrieve metadata and JSON-LD data from pages
  • / Access knowledge graph and related search results

what it does

Performs Google web searches and extracts content from webpages using the Serper API. Supports advanced search operators and returns structured content including metadata.

about

Serper Search and Scrape is a community-built MCP server published by marcopesani that provides AI assistants with tools and capabilities via the Model Context Protocol. Integrate Serper Search and Scrape to easily perform web searches and internet scraping for content extraction, research It is categorized under search web.

how to install

You can install Serper Search and Scrape in your AI client of choice. Use the install panel on this page to get one-click setup for Cursor, Claude Desktop, VS Code, and other MCP-compatible clients. This server runs locally on your machine via the stdio transport.

license

MIT

Serper Search and Scrape is released under the MIT license. This is a permissive open-source license, meaning you can freely use, modify, and distribute the software.

readme

Serper Search and Scrape MCP Server

smithery badge

A TypeScript-based MCP server that provides web search and webpage scraping capabilities using the Serper API. This server integrates with Claude Desktop to enable powerful web search and content extraction features.

<a href="https://glama.ai/mcp/servers/5zk327i0pj"> <img width="380" height="200" src="https://glama.ai/mcp/servers/5zk327i0pj/badge" alt="serper-search-scrape-mcp-server MCP server" /> </a>

Features

Tools

  • google_search - Perform web searches via Serper API

    • Rich search results including organic results, knowledge graph, "people also ask", and related searches
    • Supports region and language targeting
    • Optional parameters for location, pagination, time filters, and autocorrection
    • Supports advanced search operators:
      • site: Limit results to specific domain
      • filetype: Limit to specific file types (e.g., 'pdf', 'doc')
      • inurl: Search for pages with word in URL
      • intitle: Search for pages with word in title
      • related: Find similar websites
      • cache: View Google's cached version of a specific URL
      • before: Date before in YYYY-MM-DD format
      • after: Date after in YYYY-MM-DD format
      • exact: Exact phrase match
      • exclude: Terms to exclude from search results
      • or: Alternative terms (OR operator)
  • scrape - Extract content from web pages

    • Get plain text and optional markdown content
    • Includes JSON-LD and head metadata
    • Preserves document structure

Requirements

  • Node.js >= 18
  • Serper API key (set as SERPER_API_KEY environment variable)

Development

Install dependencies:

npm install

Build the server:

npm run build

For development with auto-rebuild:

npm run watch

Run tests:

npm test                  # Run all tests
npm run test:watch       # Run tests in watch mode
npm run test:coverage    # Run tests with coverage
npm run test:integration # Run integration tests

Environment Variables

Create a .env file in the root directory:

SERPER_API_KEY=your_api_key_here

Debugging

Since MCP servers communicate over stdio, debugging can be challenging. We recommend using the MCP Inspector, which is available as a package script:

npm run inspector

The Inspector will provide a URL to access debugging tools in your browser.

Installation

Installing via Smithery

To install Serper Search and Scrape for Claude Desktop automatically via Smithery:

npx -y @smithery/cli install @marcopesani/mcp-server-serper --client claude

Claude Desktop

Add the server config at:

  • MacOS: ~/Library/Application Support/Claude/claude_desktop_config.json
  • Windows: %APPDATA%/Claude/claude_desktop_config.json
{
  "mcpServers": {
    "serper-search": {
      "command": "npx",
      "args": ["-y", "serper-search-scrape-mcp-server"],
      "env": {
        "SERPER_API_KEY": "your_api_key_here"
      }
    }
  }
}

Cline

  1. Open the Cline extension settings
  2. Open "MCP Servers" tab
  3. Click on "Configure MCP Servers"
  4. Add the server config:
{
  "mcpServers": {
    "github.com/marcopesani/mcp-server-serper": {
      "command": "npx",
      "args": ["-y", "serper-search-scrape-mcp-server"],
      "env": {
        "SERPER_API_KEY": "your_api_key_here"
      },
      "disabled": false,
      "autoApprove": ["google_search", "scrape"]
    }
  }
}

Additional Cline configuration options:

  • disabled: Set to false to enable the server
  • autoApprove: List of tools that don't require explicit approval for each use

Cursor

  1. Open the Cursor settings
  2. Open "Features" settings
  3. In the "MCP Servers" section, click on "Add new MCP Server"
  4. Choose a name, and select "command" as "Type"
  5. In the "Command" field, enter the following:
env SERPER_API_KEY=your_api_key_here npx -y serper-search-scrape-mcp-server

Docker

You can also run the server using Docker. First, build the image:

docker build -t mcp-server-serper .

Then run the container with your Serper API key:

docker run -e SERPER_API_KEY=your_api_key_here mcp-server-serper

Alternatively, if you have your environment variables in a .env file:

docker run --env-file .env mcp-server-serper

For development, you might want to mount your source code as a volume:

docker run -v $(pwd):/app --env-file .env mcp-server-serper

Note: Make sure to replace your_api_key_here with your actual Serper API key.

FAQ

What is the Serper Search and Scrape MCP server?
Serper Search and Scrape is a Model Context Protocol (MCP) server profile on explainx.ai. MCP lets AI hosts (e.g. Claude Desktop, Cursor) call tools and resources through a standard interface; this page summarizes categories, install hints, and community ratings.
How do MCP servers relate to agent skills?
Skills are reusable instruction packages (often SKILL.md); MCP servers expose live capabilities. Teams frequently combine both—skills for workflows, MCP for APIs and data. See explainx.ai/skills and explainx.ai/mcp-servers for parallel directories.
How are reviews shown for Serper Search and Scrape?
This profile displays 37 aggregated ratings (sample rows for discoverability plus signed-in user reviews). Average score is about 4.7 out of 5—verify behavior in your own environment before production use.

Use Cases

Web Research & Information Gathering

Fetch and extract information from websites automatically

Example

Research competitor pricing, scrape product reviews, monitor news mentions

Automate 5-10 hours/week of manual web research

Content Monitoring & Alerts

Track website changes, new content, price updates

Example

Monitor competitor blog for new posts, track stock availability, watch for pricing changes

Stay informed without manual checking, never miss important updates

Data Extraction & Aggregation

Extract structured data from multiple websites

Example

Compile product listings from 10 e-commerce sites, aggregate job postings, collect real estate data

Build datasets 100x faster than manual copying

API-less Integration

Interact with services that don't offer APIs

Example

Check form submissions, validate website functionality, test user flows

Automate interactions with any website, even without API

Implementation Guide

Prerequisites

  • Claude Desktop or Cursor with MCP support
  • Understanding of web scraping ethics and robots.txt
  • Rate limiting awareness to avoid overwhelming target sites
  • Knowledge of legal restrictions on data collection

Time Estimate

20-40 minutes including configuration and testing

Installation Steps

  1. 1.Install web automation MCP server via npm or pip
  2. 2.Configure allowed domains and rate limits in MCP config
  3. 3.Test with simple fetch: 'Get content from example.com'
  4. 4.Progress to extraction: 'Extract all product prices from this page'
  5. 5.Set up monitoring: 'Check this URL daily for changes'
  6. 6.Parse structured data: 'Create CSV from this table'
  7. 7.Respect robots.txt and rate limits always

Troubleshooting

  • 403 Forbidden: Website blocks bots—respect their wishes, use official API instead
  • Rate limit errors: Slow down requests, add delays between fetches
  • Stale data: Target site changed HTML structure—update selectors
  • Timeout errors: Site is slow or blocking—increase timeout, try different user agent
  • JavaScript-rendered content: Use headless browser MCP servers for dynamic sites

Best Practices

✓ Do

  • +Check robots.txt and respect crawl rules
  • +Rate limit requests: 1-2 requests/second maximum
  • +Use official APIs when available instead of scraping
  • +Identify your bot with descriptive user agent
  • +Cache results to minimize repeated requests
  • +Handle errors gracefully with retries and fallbacks
  • +Validate extracted data for accuracy

✗ Don't

  • Don't scrape sites that explicitly forbid it (robots.txt, ToS)
  • Don't overwhelm servers with rapid requests—use rate limiting
  • Don't scrape personal data without consent and legal basis
  • Don't ignore copyright on extracted content
  • Don't assume HTML structure is stable—handle changes
  • Don't use scraped data for commercial purposes without permission

💡 Pro Tips

  • Use CSS selectors or XPath for robust data extraction
  • Set up monitoring alerts for extraction failures (structure changed)
  • Implement exponential backoff for retries on failures
  • Store raw HTML for reprocessing if extraction logic changes
  • Combine with data analysis tools for insights from extracted data
  • Consider using official APIs or RSS feeds as more stable alternatives

Technical Details

Architecture

MCP server handles HTTP requests, HTML parsing, JavaScript rendering (if headless browser), and returns structured data to Claude.

Protocols

  • HTTP/HTTPS
  • WebSocket (for real-time sites)
  • Puppeteer/Playwright (for JavaScript sites)

Compatibility

  • Static HTML sites
  • JavaScript-rendered SPAs (with headless browser)
  • REST APIs
  • GraphQL endpoints

When to Use This

✓ Use When

Use for research automation, content monitoring, data aggregation from multiple sources, and when official APIs don't exist. Best for read-only information gathering.

✗ Avoid When

Avoid for sites with APIs (use API instead), sites that explicitly forbid scraping, when data is copyrighted, or for login-required content without proper authorization.

Integration

  • Scheduled monitoring with change detection
  • Multi-source data aggregation pipelines
  • Fallback to web scraping when API rate limits hit
  • Headless browser for JavaScript-heavy sites

Discussion

Product Hunt–style comments (not star reviews)
  • No comments yet — start the thread.

List & Promote Your MCP Server

Share your MCP server with the developer community

GET_STARTED →
MCP server reviews

Ratings

4.737 reviews
  • Chaitanya Patil· Dec 24, 2024

    Serper Search and Scrape is among the better-indexed MCP projects we tried; the explainx.ai summary tracks the official description.

  • Isabella Jackson· Dec 8, 2024

    Serper Search and Scrape is a well-scoped MCP server in the explainx.ai directory — install snippets and categories matched our Claude Code setup.

  • Lucas Shah· Nov 27, 2024

    I recommend Serper Search and Scrape for teams standardizing on MCP; the explainx.ai page compares cleanly with sibling servers.

  • Piyush G· Nov 15, 2024

    We evaluated Serper Search and Scrape against two servers with overlapping tools; this profile had the clearer scope statement.

  • Isabella Okafor· Oct 18, 2024

    Serper Search and Scrape reduced integration guesswork — categories and install configs on the listing matched the upstream repo.

  • Shikha Mishra· Oct 6, 2024

    Serper Search and Scrape has been reliable for tool-calling workflows; the MCP profile page is a good permalink for internal docs.

  • Yash Thakker· Sep 13, 2024

    Serper Search and Scrape is a well-scoped MCP server in the explainx.ai directory — install snippets and categories matched our Claude Code setup.

  • Anika Martin· Sep 13, 2024

    Serper Search and Scrape reduced integration guesswork — categories and install configs on the listing matched the upstream repo.

  • Yusuf Jain· Sep 1, 2024

    Serper Search and Scrape has been reliable for tool-calling workflows; the MCP profile page is a good permalink for internal docs.

  • Diego Gill· Aug 20, 2024

    We evaluated Serper Search and Scrape against two servers with overlapping tools; this profile had the clearer scope statement.

showing 1-10 of 37

1 / 4