Set up Helicone with an AI agent

name: Helicone
description: Use when building LLM applications that need unified API access to 100+ models, observability across requests, cost tracking, prompt management, session debugging, or automatic provider failover. Agents should reach for this skill when users ask about monitoring AI applications, routing between LLM providers, managing prompts without code changes, debugging multi-step agent workflows, or optimizing LLM costs.
metadata:
    mintlify-proj: helicone
    version: "1.0"

Helicone Skill

Product Summary

Helicone is an LLM observability platform and AI Gateway that provides a unified OpenAI-compatible API for accessing 100+ LLM providers (OpenAI, Anthropic, Google, Groq, etc.) with automatic logging, cost tracking, and intelligent routing. The core integration is simple: point your OpenAI SDK to https://ai-gateway.helicone.ai and authenticate with a Helicone API key. Key files and endpoints: AI Gateway at https://ai-gateway.helicone.ai/v1, REST API at https://api.helicone.ai/v1, configuration via Helicone headers (e.g., Helicone-Session-Id, Helicone-Property-*), and prompt management through the dashboard. See Helicone docs for comprehensive reference.

When to Use

Reach for Helicone when:

• Routing & reliability: User needs to access multiple LLM providers, avoid provider outages, or implement automatic failover
• Observability: Tracking requests, costs, latency, errors, or user metrics across LLM calls
• Debugging workflows: Tracing multi-step AI agent flows, sessions, or complex interactions
• Prompt management: Deploying prompt changes without code redeployment, versioning prompts, or managing environments
• Cost optimization: Understanding unit economics, tracking spend by user/feature, or identifying cost spikes
• Session tracking: Grouping related LLM calls, vector DB queries, and tool calls into coherent workflows
• Custom properties: Tagging requests with metadata (environment, feature, user, etc.) for filtering and analysis

Quick Reference

Gateway Integration

Task	Command/Header
Base URL	https://ai-gateway.helicone.ai/v1 (or / for non-v1 endpoints)
Authentication	Authorization: Bearer $HELICONE_API_KEY or Helicone-Auth: Bearer $HELICONE_API_KEY
Switch models	Change model field; gateway auto-routes to cheapest provider
Lock provider	model: "gpt-4o-mini/openai" (only OpenAI)
Fallback chain	model: "gpt-4o-mini/azure,gpt-4o-mini/openai,gpt-4o-mini" (try in order)
Exclude provider	model: "!openai,gpt-4o-mini" (skip OpenAI, use others)

Essential Headers

Header	Purpose	Example
Helicone-Session-Id	Group related requests	"550e8400-e29b-41d4-a716-446655440000"
Helicone-Session-Path	Hierarchy within session	"/task/research/web_search"
Helicone-Session-Name	Session type label	"Customer Support"
Helicone-User-Id	Track per-user metrics	"user-123" or "alice@example.com"
Helicone-Property-[Name]	Custom metadata	Helicone-Property-Environment: production
Helicone-Request-Id	Explicit request ID	UUID format
Helicone-Cache-Enabled	Enable response caching	"true" or "false"
Helicone-RateLimit-Policy	Rate limiting	"1000;w=3600" (1000 requests/hour)

Prompt Management

Task	Syntax
Reference prompt	prompt_id: "abc123" in chat completion request
Specify environment	environment: "production" (or staging, development)
Pass variables	inputs: { "customer_name": "John", "product": "AI Gateway" }
Variable syntax	{{hc:variable_name:type}} in prompt template
Prompt partial	{{hcp:prompt_id:message_index:environment}} to reuse messages

Common Configuration Patterns

// Basic gateway setup
const client = new OpenAI({
  baseURL: "https://ai-gateway.helicone.ai/v1",
  apiKey: process.env.HELICONE_API_KEY,
});

// With session tracking
const response = await client.chat.completions.create(
  { model: "gpt-4o-mini", messages: [...] },
  {
    headers: {
      "Helicone-Session-Id": sessionId,
      "Helicone-Session-Path": "/task/step1",
      "Helicone-Session-Name": "Workflow Name",
    },
  }
);

// With custom properties
const response = await client.chat.completions.create(
  { model: "gpt-4o-mini", messages: [...] },
  {
    headers: {
      "Helicone-Property-Environment": "production",
      "Helicone-Property-Feature": "customer-support",
      "Helicone-User-Id": "user-456",
    },
  }
);

Decision Guidance

When to Use AI Gateway vs. Bring Your Own Keys (BYOK)

Scenario	Use AI Gateway	Use BYOK
New project, no provider accounts	✅ Add credits, no setup	❌
Existing provider accounts, want observability	❌	✅ Configure keys in settings
Need automatic failover across providers	✅ Automatic	⚠️ Manual fallback logic
Compliance: data must stay in region	❌	✅ Use regional deployment
Want to use provider credits	❌	✅ Direct billing
0% markup pricing	✅	❌ (provider bills you directly)

When to Use Sessions vs. Custom Properties

Use Case	Sessions	Custom Properties
Track multi-step workflows	✅ Groups related calls	❌ Tags individual requests
Debug agent flow	✅ Shows hierarchy	❌ Flat metadata
Tag environment/feature	❌	✅ Simple metadata
Reconstruct conversation	✅ Complete trace	❌
Filter by dimension	❌	✅ Dashboard filters

When to Use Prompt Management vs. Hardcoded Prompts

Factor	Prompt Management	Hardcoded
Deploy prompt changes	✅ No code redeploy	❌ Requires deployment
Version control	✅ Built-in	❌ Manual
A/B test prompts	✅ Easy environment switching	❌ Complex
Simple static prompt	❌ Overhead	✅ Fine
Dynamic variables	✅ Native support	❌ String substitution

Workflow

1. Set Up Gateway Integration

• Create Helicone account and generate API key
• Update OpenAI SDK: set baseURL to https://ai-gateway.helicone.ai/v1 and apiKey to Helicone key
• Make a test request; verify it appears in dashboard within seconds
• Requests automatically log with costs, latency, model, and provider

2. Add Observability (Custom Properties & User Tracking)

• Identify dimensions to track: environment, feature, user, conversation, etc.
• Add headers to requests: Helicone-Property-[Name] for metadata, Helicone-User-Id for users
• Use dashboard to filter by properties and analyze costs/performance by dimension
• Query via REST API using request/query-clickhouse endpoint with request_response_rmt.properties filter

3. Implement Session Tracking (for Multi-Step Workflows)

• Generate unique Helicone-Session-Id per workflow (UUID recommended)
• Design session paths using /parent/child hierarchy to represent workflow structure
• Add three headers to each request: Session-Id, Session-Path, Session-Name
• View complete workflow tree in dashboard; trace where failures occur

4. Configure Provider Routing (Optional)

• Default: gateway auto-routes to cheapest provider with automatic failover
• To lock provider: use model: "gpt-4o-mini/openai"
• To set fallback chain: use model: "gpt-4o-mini/azure,gpt-4o-mini/openai,gpt-4o-mini"
• To exclude provider: use model: "!openai,gpt-4o-mini"
• Test failover by simulating provider outage; verify requests succeed

5. Deploy Prompts (Optional)

• Create prompt in dashboard playground with variables using {{hc:name:type}} syntax
• Test with different inputs; save with version and environment
• Reference in code: prompt_id: "abc123", environment: "production", inputs: {...}
• Update prompt in dashboard; changes take effect immediately without code redeploy

6. Monitor & Optimize

• Check dashboard for cost spikes, latency patterns, error rates
• Set up alerts for cost thresholds or error rates
• Use custom properties to identify expensive features/users
• Enable caching for repetitive requests: Helicone-Cache-Enabled: true

Common Gotchas

• Session ID reuse: Using same session ID across different workflows mixes unrelated requests. Generate unique IDs per workflow.
• Session path design: Paths should group by function, not time. All "security checks" use same path even if they happen at different times.
• Custom property filtering: When querying via API, properties MUST be wrapped in request_response_rmt object or query returns empty results.
• Headers vs. request body: Helicone features use headers, not request body fields. Helicone-Session-Id is a header, not a body parameter.
• Model override: If model is not in request body or URL, use Helicone-Model-Override header for cost calculation.
• Prompt variables: Variables use {{hc:name:type}} syntax in dashboard, but are replaced at runtime. Type validation only works for number and boolean; strings accepted for any type.
• Cache key conflicts: Use Helicone-Cache-Seed to separate cache states (e.g., per-user caching).
• Rate limit format: Helicone-RateLimit-Policy uses format [quota];w=[window_ms];u=[unit];s=[segment]. Example: "10;w=1000;u=cents;s=user" = 10 cents per 1000ms per user.
• Fallback chain order: In model: "a,b,c", gateway tries A first, then B, then C. Only first two are used for token limit fallback.
• Provider key priority: BYOK keys always tried first; credits are fallback. If BYOK fails, gateway tries credits automatically.

Verification Checklist

Before submitting work with Helicone:

• API key is set and has write permissions
• Base URL is https://ai-gateway.helicone.ai/v1 (or / for non-v1 endpoints)
• Test request appears in dashboard within 10 seconds
• Session headers (if used) are unique per workflow and follow /parent/child path syntax
• Custom properties are added via Helicone-Property-[Name] headers, not request body
• User tracking uses Helicone-User-Id header for per-user cost analysis
• Prompt references use prompt_id and inputs in request, not hardcoded text
• Provider routing is tested: verify fallback works if primary provider fails
• Cost tracking is enabled: check dashboard for accurate cost attribution
• Caching is configured if using repetitive requests: Helicone-Cache-Enabled: true
• Rate limiting is set if needed: Helicone-RateLimit-Policy header present
• Query filters (if using REST API) wrap properties in request_response_rmt object

Resources

• Comprehensive navigation: See llms.txt for page-by-page documentation index
• Quick start: Getting Started Guide — set up gateway in 2 minutes
• AI Gateway: Gateway Overview — unified API, routing, fallbacks
• Prompt Management: Prompt Overview — versioning, variables, environments
• Sessions & Debugging: Sessions — trace multi-step workflows
• Headers Reference: Header Directory — all supported headers
• Custom Properties: Custom Properties — tagging and filtering
• Provider Routing: Provider Routing — failover and model selection
• Cost Tracking: Cost Tracking Guide — unit economics and optimization

---

For additional documentation and navigation, see: https://docs.helicone.ai/llms.txt