Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.lobstr.io/llms.txt

Use this file to discover all available pages before exploring further.

​
Installation

pip install lobstrio
Requires Python 3.10+.

lobstr go

Combines create, configure, add tasks, run, and download into a single command.

50+ Crawlers

Search and explore every crawler by slug before running.

CSV & JSON Export

Download results with a live progress bar while runs complete in the background.

Webhooks & Cloud

Configure delivery to email, Google Sheets, S3, SFTP, or webhook from the terminal.

​
Quick Start

# Scrape Google Maps in one command
lobstr go google-maps-leads-scraper \
  "https://google.com/maps/search/restaurants+paris" \
  -o leads.csv

# Keyword-based search
lobstr go google-search-scraper "pizza delivery" --key keyword

# With parameters and concurrency
lobstr go google-maps-leads-scraper url1 url2 \
  --param max_results=200 --concurrency 3

# Don't download, just kick off
lobstr go twitter-profile-scraper @elonmusk --no-download

​
Command Reference

​
Crawlers

CommandDescription
crawlers lsList all available crawlers
crawlers search <query>Search crawlers by keyword
crawlers show <slug>Get crawler details by slug
crawlers params <slug>Get supported parameters for a crawler
crawlers attrs <slug>Get output attributes for a crawler

​
Squids

CommandDescription
squid create <crawler> --name "My Scraper"Create a new squid for a crawler
squid lsList all squids in your account
squid show <id>Get squid details by ID
squid update <id> --param key=valueUpdate squid settings or parameters
squid empty <id>Remove all tasks from a squid
squid rm <id>Permanently delete a squid

​
Tasks

CommandDescription
task add <squid> <url1> <url2> ...Add one or more task URLs to a squid
task upload <squid> <file.csv>Bulk upload tasks from a CSV file
task ls <squid>List tasks for a squid
task rm <id>Delete a task by ID

​
Runs

CommandDescription
run start <squid> --waitStart a new run and optionally wait for completion
run ls <squid>List all runs for a squid
run show <id>Get run details by ID
run stats <id>Get statistics for a run
run abort <id>Abort an in-progress run
run download <id> -o results.csvDownload run results to a file

​
Results

CommandDescription
results get <squid> --format csv -o data.csvGet results for a squid and export to file

​
Delivery

CommandDescription
delivery email <squid> --email user@example.comConfigure email delivery for a squid
delivery webhook <squid> --url https://...Configure webhook delivery for a squid
delivery s3 <squid> --bucket my-bucketConfigure Amazon S3 delivery for a squid