Delivery Behavior
Understand BlameTrail's webhook retry schedule, timeout handling, auto-disable behavior, and how to troubleshoot delivery issues.
BlameTrail makes a best effort to deliver every webhook event reliably. When a delivery fails, BlameTrail retries with exponential backoff. Endpoints that fail repeatedly are automatically disabled to prevent wasted resources.
Retry schedule
Failed deliveries are retried up to 5 times with exponential backoff:
| Attempt | Approximate delay | Cumulative time |
|---|---|---|
| 1 (initial) | Immediate | 0 |
| 2 | ~1 minute | ~1 minute |
| 3 | ~5 minutes | ~6 minutes |
| 4 | ~30 minutes | ~36 minutes |
| 5 | ~2 hours | ~2.5 hours |
After 5 failed attempts, the delivery is marked as permanently failed for that event.
Response code handling
BlameTrail interprets HTTP response codes as follows:
| Response | Interpretation | Action |
|---|---|---|
2xx | Success | Delivery complete, no retry |
4xx | Permanent failure | No retry (the request is malformed or rejected) |
5xx | Transient failure | Retry with backoff |
| Timeout (>10s) | Transient failure | Retry with backoff |
| Connection refused | Transient failure | Retry with backoff |
| DNS resolution failure | Transient failure | Retry with backoff |
Important: 4xx responses are not retried
If your endpoint returns a 4xx status code, BlameTrail treats the failure as permanent and does not retry. This means:
- Do not return
400or401for transient errors on your end. - If signature verification fails, return
401-- BlameTrail will not retry, which is correct behavior for an authentication failure. - If your endpoint is temporarily unable to process the event, return
503(Service Unavailable) to trigger a retry.
Timeout
BlameTrail waits 10 seconds for a response. If your endpoint does not respond within this window, the request is considered failed and will be retried.
If your processing takes longer than 10 seconds, acknowledge the request immediately and process the event asynchronously:
app.post("/webhook", (req, res) => {
// Verify signature first
if (!verifySignature(req)) {
return res.status(401).json({ error: "Invalid signature" });
}
// Respond immediately
res.status(200).json({ received: true });
// Process asynchronously
processEventAsync(req.body).catch(console.error);
});Auto-disable
Endpoints that fail 10 consecutive deliveries are automatically disabled. When an endpoint is disabled:
- No further deliveries are attempted.
- Events that would have been delivered are not queued or buffered.
- The endpoint appears as Disabled in the integrations UI.
Re-enabling a disabled endpoint
- Fix the underlying issue (server down, DNS misconfigured, expired certificate, etc.).
- Go to Organization > Integrations > Webhooks.
- Find the disabled endpoint and click Enable.
- Click Send Test to verify it is working.
After re-enabling, the consecutive failure counter resets. New events will be delivered normally. Events that occurred while the endpoint was disabled are not replayed.
Delivery ID
Each delivery includes a unique X-BlameTrail-Delivery header. Use this ID to:
- Deduplicate events on your end (in case of rare double-delivery).
- Reference specific deliveries when troubleshooting with BlameTrail support.
- Correlate webhook logs between your system and BlameTrail.
Troubleshooting
Timeouts
Symptom: Deliveries fail with timeout errors even though your endpoint is running.
Cause: Your endpoint takes longer than 10 seconds to respond.
Solution: Return a 200 response immediately and process the event asynchronously. Do not perform heavy computation, external API calls, or database writes in the request handler before responding.
Signature mismatches
Symptom: All deliveries fail with 401 from your endpoint.
Cause: The signature computed by your verification code does not match the X-BlameTrail-Signature header.
Common fixes:
- Ensure you are using the raw request body (the exact bytes received), not a re-serialized JSON string.
- Verify the signing secret matches the one shown in BlameTrail (copy it again to be sure).
- Check that you are constructing the signed content as
${timestamp}.${rawBody}with a period separator. - Confirm you are using HMAC-SHA256, not SHA256 alone or a different HMAC variant.
Auto-disabled endpoint
Symptom: No deliveries are received after a period of failures.
Cause: The endpoint was auto-disabled after 10 consecutive failures.
Solution:
- Check the endpoint status in Organization > Integrations > Webhooks.
- Fix the root cause (typically a server outage or configuration error).
- Re-enable the endpoint.
- Send a test ping to verify recovery.
Missing events
Symptom: Some events are received but others are not.
Cause: The missing events may have returned 4xx and were not retried, or the endpoint was temporarily disabled.
Solution: Check your server logs for 4xx responses to BlameTrail requests. Ensure your endpoint returns 2xx for successful processing and 5xx for transient failures.