Documentation Index
Fetch the complete documentation index at: https://docs.lobstr.io/llms.txt
Use this file to discover all available pages before exploring further.
Configure Amazon S3 delivery to export run results directly to an AWS S3 bucket. Results will be automatically uploaded to your specified bucket and path when runs complete.
Authentication Options
You can authenticate with S3 in two ways:
- AWS Credentials: Provide
aws_access_key and aws_secret_key in the request
- Bucket Policy: If you don’t provide credentials, ensure your bucket policy allows public WRITE permissions
Your API authentication token. Value: Token YOUR_API_KEY
Must be application/json. Value: application/json
Query Parameters
The unique identifier (hash) of the squid for which to configure S3 delivery. Example: c106a44a98044ef18acc59986ae10967
Request Body
Name of the S3 bucket where data will be stored. Example: "my-bucket"
Set to true to enable S3 delivery, false to disable. Example: true
Folder path inside the bucket where results will be saved. Example: "exports"
(Optional) AWS Access Key for authentication. If not provided, bucket must have public WRITE permissions. Example: "AKIAIOSFODNN7EXAMPLE"
(Optional) AWS Secret Key for authentication. If not provided, bucket must have public WRITE permissions. Example: "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
Response Field Explanations
S3 bucket name. Example: "my-bucket"
Whether S3 delivery is active. Example: true
Target folder path in bucket. Example: "exports"
AWS Access Key (returned if provided). Example: "AKIAIOSFODNN7EXAMPLE"
AWS Secret Key (returned if provided). Example: "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
If you don’t provide AWS credentials, your bucket policy MUST allow public WRITE permissions for Lobstr to upload files.
For better security, provide AWS credentials instead of making your bucket publicly writable. Use IAM users with limited permissions.
The target_path creates a folder structure in your bucket. Results will be uploaded to s3://bucket/target_path/filename.csv
Use the Test S3 endpoint to verify your bucket permissions and credentials before activating automatic delivery.
AWS credentials are stored and returned in API responses. Ensure you’re using HTTPS and handle responses securely.
Code Examples
curl -X POST "https://api.lobstr.io/v1/delivery?squid=YOUR_SQUID_HASH" \
-H "Authorization: Token YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"s3_fields": {
"bucket": "my-bucket",
"is_active": true,
"target_path": "exports",
"aws_access_key": "your_access_key",
"aws_secret_key": "your_secret_key"
}
}'
Response
{
"s3_fields": {
"bucket": "my-bucket",
"is_active": true,
"target_path": "exports",
"aws_access_key": "your_access_key",
"aws_secret_key": "your_secret_key"
}
}