The /api/v1/jobs/messages/batches endpoint allows you to create multiple jobs in a single request. This endpoint is ideal for processing multiple tasks asynchronously and is best suited for scenarios where you want to submit several jobs at once and check their status later or receive callbacks upon completion.

POST /api/v1/jobs/messages/batches

HeaderTypeRequiredDescription
Content-TypestringYesMust be application/json
x-api-keystringYesYour API key for authentication

The request body contains an array of job creation objects:

FieldTypeRequiredDescription
modelstringYesAI model to use. Options: gpt-auto, gpt-5, gpt-5-mini. See models for additional details and options.
message_cache_ttlnumberNoCache TTL in seconds (180-3600). Default: 300 seconds
response_formatobjectYesOutput format specification. See details.
messagesarrayYesArray of messages for the conversation
callbackobjectNoOptional callback configuration for webhook notifications. See details.

The callback parameter allows you to receive webhook notifications when jobs complete:

json
{
    "callback_url": "https://your-domain.com/webhook",
    "callback_headers": {
        "Authorization": "Bearer your-webhook-token",
        "X-Custom-Header": "custom-value"
    },
    "callback_payload": {
        "job_reference": "batch_123",
        "user_id": "user_456"
    }
}

  • At least one message with role: "user" is required per job
  • Maximum token size per request is based on user's plan. See details in the pricing page
  • Valid roles: developer, user, assistant

Different user tiers have different batch size limits:

TierMaximum jobs per request
user_free2 jobs
user_go10 jobs
user_pro20 jobs
user_max30 jobs

json
[
    {
        "model": "gpt-auto",
        "response_format": {
            "format": "text"
        },
        "messages": [
            {
                "role": "user",
                "content": "Explain photosynthesis in simple terms"
            }
        ],
        "callback": {
            "callback_url": "https://your-domain.com/webhook",
            "callback_payload": {
                "job_type": "explanation",
                "topic": "photosynthesis"
            }
        }
    },
    {
        "model": "gpt-auto",
        "response_format": {
            "format": "json",
            "schema": {
                "type": "object",
                "properties": {
                    "summary": {
                        "type": "string",
                        "description": "Brief summary of the concept"
                    },
                    "key_points": {
                        "type": "string",
                        "description": "Main key points"
                    }
                },
                "required": ["summary", "key_points"],
                "additionalProperties": false
            }
        },
        "messages": [
            {
                "role": "user",
                "content": "Summarize quantum computing in JSON format"
            }
        ]
    }
]

Status CodeDescription
202Jobs enqueued successfully
403Forbidden - batch size limit exceeded
401Unauthorized - invalid API key
400Bad request - invalid parameters
500Internal server error

json
{
    "success": true,
    "message": "Job enqueued successfully",
    "data": {
        "job_ids": ["550e8400-e29b-41d4-a716-446655440000", "550e8400-e29b-41d4-a716-446655440001"]
    }
}

json
{
    "success": false,
    "message": "Max jobs per batch request exceeded for free user"
}

When you provide a callback configuration, the system will send POST requests to your specified URL when jobs complete.

json
{
    "job_id": "550e8400-e29b-41d4-a716-446655440000",
    "status": "completed",
    "created_at": 1704067200,
    "result": {
        "response_message": "The AI response content here..."
    },
    "error": null,
    "callback_payload": {
        "job_reference": "batch_123",
        "user_id": "user_456"
    }
}

The callback will include:

  • Any custom headers you specified in callback_headers
  • Standard HTTP headers
  • Content-Type: application/json

StatusDescription
completedJob finished successfully
failedJob failed with an error
pendingJob is still processing
assignedJob has been assigned to a worker

After submitting a batch, you can check individual job status using the GET endpoint:

code
GET /api/v1/jobs/{job_id}

typescript
const API_CONFIG = {
    BASE_URL: "https://api.tokenthon.com",
    ENDPOINT: "/api/v1/jobs/messages/batches",
    HEADERS: {
        "Content-Type": "application/json",
        "x-api-key": "your-api-key-here",
    },
};
 
async function createBatchJobs() {
    const batchRequests = [
        {
            model: "gpt-auto",
            response_format: { format: "text" },
            messages: [
                {
                    role: "user",
                    content: "What is the capital of France?",
                },
            ],
            callback: {
                callback_url: "https://your-domain.com/webhook",
                callback_payload: { question_id: "q1" },
            },
        },
        {
            model: "gpt-auto",
            response_format: { format: "text" },
            messages: [
                {
                    role: "user",
                    content: "What is the capital of Germany?",
                },
            ],
            callback: {
                callback_url: "https://your-domain.com/webhook",
                callback_payload: { question_id: "q2" },
            },
        },
    ];
 
    try {
        const response = await fetch(`${API_CONFIG.BASE_URL}${API_CONFIG.ENDPOINT}`, {
            method: "POST",
            headers: API_CONFIG.HEADERS,
            body: JSON.stringify(batchRequests),
        });
 
        const result = await response.json();
 
        if (!response.ok) {
            throw new Error(`Error: ${result.message}`);
        }
 
        console.log("Batch jobs created:", result.data.job_ids);
        return result.data.job_ids;
    } catch (error) {
        console.error("Error creating batch jobs:", error);
        throw error;
    }
}
 
// Usage
createBatchJobs()
    .then((jobIds) => {
        console.log("Job IDs:", jobIds);
        // Store job IDs for later status checking
    })
    .catch((error) => console.error(error));

bash
curl -X POST "https://api.tokenthon.com/api/v1/jobs/messages/batches" \
  -H "Content-Type: application/json" \
  -H "x-api-key: your-api-key-here" \
  -d '[
    {
      "model": "gpt-auto",
      "response_format": {
        "format": "text"
      },
      "messages": [
        {
          "role": "user",
          "content": "Explain machine learning"
        }
      ]
    },
    {
      "model": "gpt-auto",
      "response_format": {
        "format": "text"
      },
      "messages": [
        {
          "role": "user",
          "content": "Explain deep learning"
        }
      ]
    }
  ]'

  • Use the maximum batch size allowed for your tier to optimize throughput
  • Consider job complexity when determining optimal batch sizes

  • Ensure your webhook endpoint can handle concurrent requests
  • Implement idempotency to handle duplicate callback deliveries
  • Include proper error handling and retry logic

  • Monitor batch responses for partially successful operations
  • Implement retry logic for failed jobs using the returned job IDs
  • Log job IDs for debugging and monitoring purposes

  • Implement exponential backoff when polling job status
  • Use callbacks for real-time notifications instead of polling when possible
  • Cache job results to avoid redundant API calls

If some jobs in a batch fail validation:

  • The entire batch request will be rejected
  • No jobs will be created
  • Fix the validation errors and resubmit the entire batch

If jobs fail during processing:

  • Use the job IDs to check individual job status
  • Resubmit failed jobs with corrected parameters
  • Monitor error messages for common issues

If webhook callbacks fail:

  • The system will attempt delivery (retry policy may vary)
  • Implement endpoint monitoring and alerting
  • Consider implementing job status polling as a fallback