CLI - lobstr.io

Installation

pip install lobstrio

Requires Python 3.10+.

Environment variables

Variable	Description
`LOBSTR_TOKEN`	API token, overriding the one saved by `lobstr config set-token`.
`LOBSTR_API_BASE`	Point the CLI at a different API base URL (e.g. a staging or self-hosted instance). Defaults to production (`https://api.lobstr.io/v1/`) when unset.

export LOBSTR_TOKEN=your_api_key

lobstr go

Combines create, configure, add tasks, run, and download into a single command.

55+ Crawlers

Search and explore every crawler by slug before running.

CSV & JSON Export

Download results with a live progress bar while runs complete in the background.

Webhooks & Cloud

Configure delivery to email, Google Sheets, S3, SFTP, or webhook from the terminal.

Quick Start

# Scrape Google Maps in one command
lobstr go google-maps-leads-scraper \
  "https://google.com/maps/search/restaurants+paris" \
  -o leads.csv

# Keyword-based search
lobstr go google-search-scraper "pizza delivery" --key keyword

# With parameters and concurrency
lobstr go google-maps-leads-scraper url1 url2 \
  --param max_results=200 --concurrency 3

# With add-ons (extra credits)
lobstr go youtube-channel-scraper "https://www.youtube.com/@YouTube" \
  --function get_videos

# Don't download, just kick off
lobstr go twitter-profile-scraper @elonmusk --no-download

# Account-backed crawler (e.g. Sales Navigator): auto-picks the one
# healthy, unlocked matching account if there's exactly one
lobstr go sales-navigator-leads-scraper "https://linkedin.com/..."

# ...or name it explicitly (needed when there's more than one candidate)
lobstr go sales-navigator-leads-scraper "https://linkedin.com/..." --account jane@example.com

lobstr crawlers search "google maps"
lobstr squid create google-maps-leads-scraper --name "Paris Restaurants"
lobstr task add SQUID_ID "https://google.com/maps/search/restaurants+paris"
lobstr run start SQUID_ID --wait
lobstr results get SQUID_ID --format csv -o results.csv

# Check who you are
lobstr whoami

Account-backed crawlers (LinkedIn, Sales Navigator, …) need a connected account before they can run. go links one automatically when exactly one healthy, unlocked account matches the crawler’s account type; when it can’t, it names the matching candidates, or the account type to connect, so you can pass --account yourself. It never auto-picks onto a squid that already has an account, so a reused squid (go --name) keeps what it had. A wrong-type account, an unknown account, or --account on a crawler that doesn’t need one each fail with a clear message before anything is created; if linking fails after the squid is created, or the command is interrupted before the run starts, the squid is removed rather than left behind.

Command Reference

Crawlers

Command	Description
`crawlers ls`	List all available crawlers
`crawlers search <query>`	Search crawlers by keyword
`crawlers show <slug>`	Get crawler details by slug
`crawlers params <slug>`	Get supported parameters for a crawler
`crawlers attrs <slug>`	Get output attributes for a crawler

crawlers ls, show, and search show which account type a crawler needs (for example sales-nav-sync) in the “Needs Account” column, in place of a plain yes or no — that’s the value to look for when picking a matching account for --account below. A crawler that needs no account still shows “no”. --json output is unchanged.

Squids

Command	Description
`squid create <crawler> --name "My Scraper"`	Create a new squid for a crawler
`squid ls`	List all squids in your account
`squid show <id>`	Get squid details by ID
`squid update <id> --param key=value`	Update squid settings or parameters
`squid update <id> --function <name>`	Turn an add-on on (`<name>=false` turns it off). Repeatable; names from `crawlers params <crawler>`. Also on `go`.
`squid update <id> --account <username or hash>`	Link a connected account to the squid, in addition to any it already has
`squid update <id> --replace-accounts --account <username or hash>`	Set the squid’s account list outright, detaching any account not passed
`squid empty <id>`	Remove all tasks from a squid
`squid rm <id>`	Permanently delete a squid

Linking an account. --account (repeatable) takes either an account’s username or a hash — a hash prefix is enough. It’s additive: the account is added to whatever the squid already has. --replace-accounts sets the list outright instead, printing which accounts it’s about to detach before doing it; passed with no --account at all, it detaches every account. squid create has no --account — the API only accepts an account list on squid update.

Tasks

Command	Description
`task add <squid> <url1> <url2> ...`	Add one or more task URLs to a squid
`task upload <squid> <file.csv>`	Bulk upload tasks from a CSV file
`task ls <squid>`	List tasks for a squid
`task rm <id>`	Delete a task by ID

Runs

Command	Description
`run start <squid>`	Start a new run and return immediately
`run start <squid> --wait`	Start a run and poll until it completes
`run start <squid> --wait --timeout <s> --poll <s>`	Wait with a time limit and custom poll interval — on timeout, exits non-zero and the run keeps going
`run watch <id> [--timeout <s>] [--poll <s>]`	Attach to a running run and follow its progress
`run ls <squid>`	List all runs for a squid
`run show <id>`	Get run details by ID
`run stats <id>`	Get statistics for a run
`run abort <id>`	Abort an in-progress run
`run download <id> -o results.csv`	Download run results to a file

--wait, --timeout and --download all follow the run to completion. --timeout <seconds> bounds the wait — if the run hasn’t finished by then, the command exits non-zero and prints how to re-attach (lobstr run watch <id>); the run is not aborted and keeps running server-side. --poll <seconds> sets how often status is checked (default 3).

Results

Command	Description
`results get <squid> --format csv -o data.csv`	Get results for a squid and export to file

Delivery

Command	Description
`delivery email <squid> --email user@example.com`	Configure email delivery for a squid
`delivery webhook <squid> --url https://...`	Configure webhook delivery for a squid
`delivery s3 <squid> --bucket my-bucket`	Configure Amazon S3 delivery for a squid
`delivery googlesheet <squid> --url https://...`	Configure Google Sheet delivery for a squid
`delivery sftp <squid> --host ... --username ... --password ... --directory ...`	Configure SFTP delivery for a squid
`delivery test-email --email user@example.com`	Test an email delivery config before activating it
`delivery test-webhook --url https://...`	Test a webhook delivery config before activating it
`delivery test-s3 --bucket my-bucket`	Test an S3 delivery config before activating it
`delivery test-googlesheet --url https://...`	Test a Google Sheet delivery config before activating it
`delivery test-sftp --host ... --username ... --password ... --directory ...`	Test an SFTP delivery config before activating it

​Installation

​Environment variables

lobstr go

55+ Crawlers

CSV & JSON Export

Webhooks & Cloud

​Quick Start

​Command Reference

​Crawlers

​Squids

​Tasks

​Runs

​Results

​Delivery

Installation

Environment variables

Quick Start

Command Reference

Crawlers

Squids

Tasks

Runs

Results

Delivery