Back
RespanTRY IT

Respan

Respan is a platform for teams running AI agents and LLM apps in production. It combines an open-source AI gateway, which routes requests to more than 1,000 models through one API, with tracing, monitoring, evaluations and red-teaming, so you can see what your agents did and test changes before they ship.

It's built for engineers and product teams at AI startups and larger companies. Respan says it has processed more than 80 trillion tokens, and customers include Retell, Mem0, Lovable and AlphaSense.

One gateway for every model

You point the SDK you already use at Respan's endpoint and switch models by changing a single name. Fallbacks move a request to the next model on your list the moment one errors or hits a rate limit, so a provider outage doesn't take your app down. Response caching serves repeat requests instantly, and budgets and rate limits can be set per key or per customer, with warnings before spend runs away. The gateway itself is open source.

Tracing what agents actually do

Every LLM call, tool run, retrieval and agent turn becomes a span in a single trace, with its input, output, latency, token count and cost. When an agent gives a wrong answer, you can follow the run step by step and see where it went off course, instead of trying to reproduce the bug.

A Respan trace of a support agent workflow, showing nested spans for embedding, intent classification, planning and drafting a reply on a timeline, beside the latency, cost and input of one generation step

A support agent's run broken into spans. Image from Respan.

A dashboard tracks requests, errors, cost and latency across all traffic, sliced by model or user. Alerts reach Slack or email when a metric crosses a threshold you set.

Evals and red-teaming

Respan's evaluators grade outputs with an LLM judge, a code check, a human reviewer or a mix of them, and turn each result into one score. You can build test sets from real production requests or a CSV. Run two prompts or models against the same set and compare the scores before you ship the winner. The same evaluators can run on a sample of live traffic, so a regression shows up as it happens rather than in user complaints. Red-teaming tools probe your agents for unsafe or broken behavior.

Respan holds ISO 27001 and SOC 2 certification and is HIPAA compliant, with a BAA for healthcare customers on its Enterprise plan. It also operates under GDPR.

Pricing

Respan is freemium. The free plan gives a team the full platform with limits on logs, scores, datasets, evaluators and prompts, and no card is needed. The Team plan, billed yearly, removes most of those limits and adds a private Slack channel and a SOC 2 report. Enterprise adds volume discounts, custom SLAs, a dedicated support engineer and a HIPAA BAA. Model usage through the gateway is billed separately, often below the providers' list prices.

Is Respan free?

Yes. The free plan includes the full platform with usage limits, which is enough to trace and evaluate a small app.

Is Respan's AI gateway open source?

Yes. Respan's AI gateway, which routes requests to more than 1,000 models, is open source.

Alternatives to Respan

  • OpenRouter: Connects developers to hundreds of AI models through one API with shared billing.
  • PromptLayer: Prompt management platform with a versioned prompt registry, evaluations and logging.