Skip to main content
GET
/
v1
/
squids
List Squids
curl --request GET \
  --url https://api.lobstr.io/v1/squids \
  --header 'Authorization: <authorization>'
{
  "total_results": 123,
  "limit": 123,
  "page": 123,
  "total_pages": 123,
  "result_from": 123,
  "result_to": 123,
  "data": [
    {}
  ],
  "next": {},
  "previous": {},
  "data[].id": "<string>",
  "data[].name": "<string>",
  "data[].created_at": "<string>",
  "data[].is_active": true,
  "data[].is_ready": true,
  "data[].crawler": "<string>",
  "data[].crawler_name": "<string>",
  "data[].icon": "<string>",
  "data[].is_public": true,
  "data[].concurrency": 123,
  "data[].params": {},
  "data[].to_complete": true,
  "data[].export_unique_results": true,
  "data[].no_line_breaks": true,
  "data[].last_run_status": "<string>",
  "data[].last_run_at": "<string>",
  "data[].total_runs": 123,
  "data[].next_launch_at": {},
  "data[].google_sheet_fields": {},
  "data[].ftp_fields": {},
  "data[].s3_fields": {},
  "data[].webhook_fields": {},
  "data[].schedule": {},
  "data[].schedule_time": "<string>",
  "data[].download_url": {}
}

Documentation Index

Fetch the complete documentation index at: https://docs.lobstr.io/llms.txt

Use this file to discover all available pages before exploring further.

This endpoint returns a paginated list of all squids belonging to the authenticated user. You can search by name and control pagination with query parameters.

​
Headers

​
Authorization
string
required
Your API authentication token. Value: Token YOUR_API_KEY

​
Query Parameters

​
name
string
Filter squids by name (case-insensitive partial match). Example: Google
​
limit
integer
Number of results per page. Example: 10
​
page
integer
Page number for pagination. Example: 2

​
Response Structure

​
total_results
integer
Total number of squids matching the query. Example: 15
​
limit
integer
Number of results per page. Example: 50
​
page
integer
Current page number. Example: 1
​
total_pages
integer
Total number of pages available. Example: 1
​
result_from
integer
Starting index of results in current page. Example: 1
​
result_to
integer
Ending index of results in current page. Example: 15
​
data
array
Array of squid objects with comprehensive details
​
next
string | null
Full URL to the next page (null if on last page). Example: null
​
previous
string | null
Full URL to the previous page (null if on first page). Example: null

​
Squid Object Fields

​
data[].id
string
Unique squid identifier. Example: "c106a44a98044ef18acc59986ae10967"
​
data[].name
string
Display name of the squid. Example: "Google Maps (1)"
​
data[].created_at
string
Timestamp when the squid was created. Example: "2024-01-15T10:30:00Z"
​
data[].is_active
boolean
Whether the squid is active and can run. Example: true
​
data[].is_ready
boolean
Whether the squid has tasks and is ready to run. Example: true
​
data[].crawler
string
Unique identifier of the associated crawler. Example: "4734d096159ef05210e0e1677e8be823"
​
data[].crawler_name
string
Display name of the crawler. Example: "Google Maps Search Export"
​
data[].icon
string
URL to the crawler’s icon image. Example: "https://cdn.lobstr.io/icons/google-maps.png"
​
data[].is_public
boolean
Whether the crawler is publicly available. Example: true
​
data[].concurrency
integer
Number of concurrent tasks the squid runs. Example: 2
​
data[].params
object
Crawler-specific parameter configuration object. Example: {"language": "en"}
​
data[].to_complete
boolean
Whether to stop after completing all tasks. Example: false
​
data[].export_unique_results
boolean
Whether to export only unique results. Example: true
​
data[].no_line_breaks
boolean
Whether to remove line breaks from exported data. Example: false
​
data[].last_run_status
string
Status of the most recent run (completed, running, error, etc.). Example: "DONE"
​
data[].last_run_at
string
Timestamp of the most recent run. Example: "2024-01-20T15:45:00Z"
​
data[].total_runs
integer
Total number of times the squid has been executed. Example: 5
​
data[].next_launch_at
string | null
Scheduled time for the next automated run (null if not scheduled). Example: "2024-01-21T10:00:00Z"
​
data[].google_sheet_fields
object | null
Google Sheets integration configuration (null if not configured). Example: null
​
data[].ftp_fields
object | null
FTP integration configuration (null if not configured). Example: null
​
data[].s3_fields
object | null
AWS S3 integration configuration (null if not configured). Example: null
​
data[].webhook_fields
object | null
Webhook integration configuration (null if not configured). Example: null
​
data[].schedule
string | null
Cron expression for scheduled runs (null if not scheduled). Example: "0 10 * * *"
​
data[].schedule_time
string
Human-readable schedule description. Example: "Daily at 10:00 AM"
​
data[].download_url
string | null
Temporary signed URL to download results (null if no results available). Example: "https://storage.lobstr.io/downloads/xyz123..."
Use the ‘name’ query parameter to quickly find squids. It performs a case-insensitive partial match on squid names.
The ‘next’ and ‘previous’ fields contain full URLs for easy pagination. Use them directly to navigate through pages.
Check ‘last_run_status’ to see if a squid has completed, is running, or encountered errors. Useful for monitoring multiple squids.
Squids with ‘is_ready: false’ haven’t been configured with tasks yet. Add tasks before starting a run.
The ‘download_url’ field provides a temporary signed URL to download results. URLs expire after a certain time period.

​
Code Examples

curl -X GET "https://api.lobstr.io/v1/squids?name=Google&limit=10&page=1" \
  -H "Authorization: Token YOUR_API_KEY"

​
Response

200
{
  "total_results": 34,
  "limit": 10,
  "page": 1,
  "total_pages": 4,
  "result_from": 1,
  "result_to": 10,
  "data": [
    {
      "id": "e86b29c032024b66aff529e1d43c2bd7",
      "created_at": "2025-02-03T14:24:23Z",
      "name": "My Google Maps Scraper",
      "is_ready": true,
      "is_active": true,
      "concurrency": 2,
      "params": {
        "country": "United States",
        "ratings": "4.0+",
        "language": "English (United States)",
        "max_results": 150
      },
      "crawler": "4734d096159ef05210e0e1677e8be823",
      "crawler_name": "Google Maps Search Export",
      "last_run_status": "DONE",
      "total_runs": 5,
      "last_run_at": "2025-03-05T09:27:17.012423",
      "next_launch_at": null,
      "is_public": true
    }
  ],
  "next": "https://api.lobstr.io/v1/squids?limit=10&page=2",
  "previous": null
}