CLI - lobstr.io

Installation

pip install lobstrio

Requires Python 3.10+.

Environment variables

Variable	Description
`LOBSTR_TOKEN`	API token, overriding the one saved by `lobstr config set-token`.
`LOBSTR_API_BASE`	Point the CLI at a different API base URL (e.g. a staging or self-hosted instance). Defaults to production (`https://api.lobstr.io/v1/`) when unset.

export LOBSTR_TOKEN=your_api_key

lobstr go

Combines create, configure, add tasks, run, and download into a single command.

55+ Crawlers

Search and explore every crawler by slug before running.

CSV & JSON Export

Download results with a live progress bar while runs complete in the background.

Webhooks & Cloud

Configure delivery to email, Google Sheets, S3, SFTP, or webhook from the terminal.

Quick Start

# Scrape Google Maps in one command
lobstr go google-maps-leads-scraper \
  "https://google.com/maps/search/restaurants+paris" \
  -o leads.csv

# Keyword-based search
lobstr go google-search-scraper "pizza delivery" --key keyword

# With parameters and concurrency
lobstr go google-maps-leads-scraper url1 url2 \
  --param max_results=200 --concurrency 3

# Don't download, just kick off
lobstr go twitter-profile-scraper @elonmusk --no-download

lobstr crawlers search "google maps"
lobstr squid create google-maps-leads-scraper --name "Paris Restaurants"
lobstr task add SQUID_ID "https://google.com/maps/search/restaurants+paris"
lobstr run start SQUID_ID --wait
lobstr results get SQUID_ID --format csv -o results.csv

# Check who you are
lobstr whoami

Command Reference

Crawlers

Command	Description
`crawlers ls`	List all available crawlers
`crawlers search <query>`	Search crawlers by keyword
`crawlers show <slug>`	Get crawler details by slug
`crawlers params <slug>`	Get supported parameters for a crawler
`crawlers attrs <slug>`	Get output attributes for a crawler

Squids

Command	Description
`squid create <crawler> --name "My Scraper"`	Create a new squid for a crawler
`squid ls`	List all squids in your account
`squid show <id>`	Get squid details by ID
`squid update <id> --param key=value`	Update squid settings or parameters
`squid empty <id>`	Remove all tasks from a squid
`squid rm <id>`	Permanently delete a squid

Tasks

Command	Description
`task add <squid> <url1> <url2> ...`	Add one or more task URLs to a squid
`task upload <squid> <file.csv>`	Bulk upload tasks from a CSV file
`task ls <squid>`	List tasks for a squid
`task rm <id>`	Delete a task by ID

Runs

Command	Description
`run start <squid>`	Start a new run and return immediately
`run start <squid> --wait`	Start a run and poll until it completes
`run start <squid> --wait --timeout <s> --poll <s>`	Wait with a time limit and custom poll interval — on timeout, exits non-zero and the run keeps going
`run watch <id> [--timeout <s>] [--poll <s>]`	Attach to a running run and follow its progress
`run ls <squid>`	List all runs for a squid
`run show <id>`	Get run details by ID
`run stats <id>`	Get statistics for a run
`run abort <id>`	Abort an in-progress run
`run download <id> -o results.csv`	Download run results to a file

--wait, --timeout and --download all follow the run to completion. --timeout <seconds> bounds the wait — if the run hasn’t finished by then, the command exits non-zero and prints how to re-attach (lobstr run watch <id>); the run is not aborted and keeps running server-side. --poll <seconds> sets how often status is checked (default 3).

Results

Command	Description
`results get <squid> --format csv -o data.csv`	Get results for a squid and export to file

Delivery

Command	Description
`delivery email <squid> --email user@example.com`	Configure email delivery for a squid
`delivery webhook <squid> --url https://...`	Configure webhook delivery for a squid
`delivery s3 <squid> --bucket my-bucket`	Configure Amazon S3 delivery for a squid
`delivery googlesheet <squid> --url https://...`	Configure Google Sheet delivery for a squid
`delivery sftp <squid> --host ... --username ... --password ... --directory ...`	Configure SFTP delivery for a squid
`delivery test-email --email user@example.com`	Test an email delivery config before activating it
`delivery test-webhook --url https://...`	Test a webhook delivery config before activating it
`delivery test-s3 --bucket my-bucket`	Test an S3 delivery config before activating it
`delivery test-googlesheet --url https://...`	Test a Google Sheet delivery config before activating it
`delivery test-sftp --host ... --username ... --password ... --directory ...`	Test an SFTP delivery config before activating it

​Installation

​Environment variables

lobstr go

55+ Crawlers

CSV & JSON Export

Webhooks & Cloud

​Quick Start

​Command Reference

​Crawlers

​Squids

​Tasks

​Runs

​Results

​Delivery

Installation

Environment variables

Quick Start

Command Reference

Crawlers

Squids

Tasks

Runs

Results

Delivery