Reliability, retries, and async state

Retry decision matrix

Backoff policy

Use a small initial delay, exponential growth, random jitter, a maximum delay, and a total retry budget. Do not coordinate every worker to retry on the same second. Retry-After is a minimum wait, not permission to retry indefinitely.

const delay = Math.max(retryAfterMs, Math.min(30_000, 500 * 2 ** attempt));
const jitteredDelay = delay + Math.round(Math.random() * Math.min(1_000, delay * 0.2));

JavaScript async-analysis example

const baseUrl = "https://api.prod.ca-central-1.leni.co";
const headers = {
  "X-Api-Key": process.env.LENI_API_KEY,
  "Content-Type": "application/json",
};

const started = await fetch(
  `${baseUrl}/users/run-models`,
  {
    method: "POST",
    headers,
    body: JSON.stringify({
      question: "Summarize current occupancy and its main drivers.",
      user: { modelType: "leniq-pro" },
      wait: false,
    }),
  },
);
if (!started.ok) throw new Error(`Leni returned ${started.status}`);
const run = await started.json();

if (started.status === 202 || run.responseDeadlineReached) {
  const deadline = Date.now() + 15 * 60_000; // Set your own business deadline.
  const sleepWithinDeadline = async requestedMs => {
    const remainingMs = Math.max(0, deadline - Date.now());
    if (remainingMs === 0) return false;
    await new Promise(resolve => setTimeout(resolve, Math.min(requestedMs, remainingMs)));
    return Date.now() < deadline;
  };
  let completed = false;
  let transientAttempt = 0;
  let nextPollDelayMs = (run.pollAfterSeconds ?? 2) * 1000;
  while (Date.now() < deadline) {
    if (!(await sleepWithinDeadline(nextPollDelayMs))) break;
    nextPollDelayMs = (run.pollAfterSeconds ?? 2) * 1000;
    const remainingMs = deadline - Date.now();
    if (remainingMs <= 0) break;
    let message;
    try {
      const polled = await fetch(
        `${baseUrl}/users/me/chat-messages/${run.messageId}`,
        {
          headers: { "X-Api-Key": process.env.LENI_API_KEY },
          signal: AbortSignal.timeout(remainingMs),
        },
      );
      if ([429, 502, 503].includes(polled.status)) {
        const retryAfterSeconds = Number(polled.headers.get("retry-after") ?? 0);
        const defaultBackoffMs = Math.min(30_000, 500 * 2 ** transientAttempt++);
        const retryAfterMs = Number.isFinite(retryAfterSeconds) && retryAfterSeconds > 0 ? retryAfterSeconds * 1000 : 0;
        const retryDelayMs = Math.max(retryAfterMs, defaultBackoffMs);
        const jitterMs = Math.round(Math.random() * Math.min(1_000, retryDelayMs * 0.2));
        nextPollDelayMs = retryDelayMs + jitterMs;
        continue;
      }
      if (!polled.ok) throw new Error(`Poll returned ${polled.status}`);
      transientAttempt = 0;
      ({ message } = await polled.json());
    } catch (error) {
      if (error.name === "TimeoutError" || error.name === "AbortError") break;
      throw error;
    }
    if (message.status === "failed") throw new Error("Leni analysis failed");
    if (message.status === "completed") {
      completed = true;
      break;
    }
  }
  if (!completed) throw new Error("Client polling deadline reached; the Leni run may still be active");

  const result = await fetch(
    `${baseUrl}/users/me/chat-sessions/message-responses`,
    {
      method: "POST",
      headers,
      body: JSON.stringify({ sessionId: run.sessionId, messageId: run.messageId }),
    },
  );
  if (!result.ok) throw new Error(`Result returned ${result.status}`);
  console.log(await result.json());
} else {
  console.log(run);
}

Observability checklist

• Log endpoint, HTTP status, attempt number, duration, and opaque run/message/session IDs.
• Never log project keys, OAuth tokens, complete request bodies, memory contents, or temporary file URLs.
• Track accepted, completed, failed, timed-out, and abandoned work separately.
• Alert on sustained 429, 5xx, stuck processing, or rising terminal-failure rates—not a single transient error.
• Retain the exact terminal payload needed for support while honoring your data-retention requirements.