Skip to main content
POST
/
v1
/
tasks
Add Tasks
curl --request POST \
  --url https://api.lobstr.io/v1/tasks \
  --header 'Authorization: <authorization>' \
  --header 'Content-Type: <content-type>' \
  --data '
{
  "squid": "<string>",
  "tasks": [
    {}
  ]
}
'
{
  "duplicated_count": 123,
  "tasks": [
    {}
  ],
  "tasks[].id": "<string>",
  "tasks[].object": "<string>",
  "tasks[].created_at": "<string>",
  "tasks[].is_active": true,
  "tasks[].params": {},
  "tasks[].module": "<string>"
}

Documentation Index

Fetch the complete documentation index at: https://docs.lobstr.io/llms.txt

Use this file to discover all available pages before exploring further.

This endpoint creates new tasks within a specified squid. Tasks define the specific scraping jobs to be executed, such as URLs to scrape or search parameters. Note: The accepted task input keys (e.g., url, city, activity) depend on the crawler used in your squid. Use the Get Crawler Parameters endpoint to discover valid keys for your crawler.

​
Headers

​
Authorization
string
required
Your API authentication token. Value: Token YOUR_API_KEY
​
Content-Type
string
required
Must be application/json. Value: application/json

​
Request Body

​
squid
string
required
The hash ID of the squid to add tasks to. Example: "a1b2c3d4e5f6g7h8i9j0"
​
tasks
array
required
Array of task objects. Each task contains parameters specific to the crawler. Example: [{"url": "https://example.com"}]

​
Response Field Explanations

​
duplicated_count
integer
Number of tasks that were duplicates and not added. Example: 0
​
tasks
array
Array of created task objects.
​
tasks[].id
string
Unique task identifier (hash). Example: "c5e29d2aba8b77cdc56391e7405302de"
​
tasks[].object
string
Always "task".
​
tasks[].created_at
string
Task creation timestamp (ISO 8601). Example: "2025-02-10T14:19:32.916901"
​
tasks[].is_active
boolean
Whether the task is active and ready to run. Example: true
​
tasks[].params
object
Task parameters specific to the crawler (URL, search terms, etc.). Example: {"url": "https://..."}
​
tasks[].module
string
Identifier of the crawler module this task belongs to.
Use the Get Crawler Parameters endpoint to discover which task keys are valid for your specific crawler.
Duplicate tasks (same parameters) are automatically detected and not added. Check duplicated_count in the response.
Task parameters vary by crawler. Using invalid keys will result in task creation failure.
For bulk task creation, consider using the Upload Tasks endpoint with a CSV file instead.

​
Code Examples

curl -X POST "https://api.lobstr.io/v1/tasks" \
  -H "Authorization: Token YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "squid": "a1b2c3d4e5f6g7h8i9j0",
    "tasks": [
      {"url": "https://www.linkedin.com/in/johndoe"},
      {"url": "https://www.linkedin.com/in/janedoe"}
    ]
  }'

​
Response

200
{
  "duplicated_count": 0,
  "tasks": [
    {
      "id": "6b9d3afdf4e76df374915a50d7a495c4",
      "object": "task",
      "created_at": "2025-02-10T14:19:32.916901",
      "is_active": true,
      "params": {"url": "https://www.linkedin.com/in/johndoe"},
      "module": "4734d096159ef05210e0e1677e8be823"
    },
    {
      "id": "c5e29d2aba8b77cdc56391e7405302de",
      "object": "task",
      "created_at": "2025-02-10T14:19:32.916901",
      "is_active": true,
      "params": {"url": "https://www.linkedin.com/in/janedoe"},
      "module": "4734d096159ef05210e0e1677e8be823"
    }
  ]
}
400
{
  "error": {
    "message": "Invalid task parameters. Each task must contain required fields for the crawler.",
    "type": "validation_error",
    "param": "tasks"
  }
}