portkey
You are an expert in Portkey, the AI gateway that sits between your app and LLM providers. You help developers add caching, fallbacks, load balancing, request retries, guardrails, semantic caching, bu
SKILL.mdskills/portkey/SKILL.md
GitHubPortkey — AI Gateway for Production LLM Apps
Overview
Portkey routes LLM calls through one endpoint (https://api.portkey.ai/v1) that speaks the OpenAI API and reaches 250+ models across 40+ providers. Behaviour is controlled by a config object: routing strategy (single, fallback, loadbalance, conditional), retry, cache, request_timeout and guardrails. The dashboard logs latency, cost, tokens and errors per request. The gateway itself is open source and can run locally with npx @portkey-ai/gateway (listens on http://localhost:8787/v1, console at /public/). Portkey's docs now brand the hosted product as part of Palo Alto Networks' Prisma AIRS AI Gateway; the API and SDKs are unchanged.
Instructions
Install and call a model
npm install portkey-ai # or: pip install portkey-ai
Add provider credentials once in the Model Catalog (dashboard), which gives each provider a slug such as openai-prod. Then address models as @slug/model:
import { Portkey } from "portkey-ai";
const portkey = new Portkey({ apiKey: process.env.PORTKEY_API_KEY });
const response = await portkey.chat.completions.create({
model: "@openai-prod/gpt-4o",
messages: [{ role: "user", content: "Explain microservices in two sentences." }],
});
console.log(response.choices[0].message.content);
Python is the same shape: Portkey(api_key=os.environ["PORTKEY_API_KEY"]) and portkey.chat.completions.create(model="@openai-prod/gpt-4o", ...). The official OpenAI SDK also works with baseURL: "https://api.portkey.ai/v1" and your Portkey key as apiKey. Virtual keys still work via virtual_key in configs, but Model Catalog slugs are the recommended way.
Routing config
Create a config in the dashboard (it gets an ID like pc-...) or pass the object inline. Per-request, pass it as a second argument or header.
const fallbackConfig = {
strategy: { mode: "fallback" },
targets: [
{ provider: "@openai-prod", override_params: { model: "gpt-4o" },
retry: { attempts: 2, on_status_codes: [429, 500, 503] } },
{ provider: "@anthropic-prod", override_params: { model: "claude-sonnet-4-5" } },
],
request_timeout: 30000,
cache: { mode: "simple", max_age: 3600 },
};
const portkey = new Portkey({ apiKey: process.env.PORTKEY_API_KEY, config: fallbackConfig });
// or: config: "pc-fallback-1a2b3c" (saved config ID)
fallbacktries targets in order;loadbalancesplits traffic by each target'sweight;conditionalroutes on request metadata;singleuses one target.- Target fields:
provider(orvirtual_key),api_key,override_params(always replaces),default_params(only if absent),drop_params,weight. override_params.modelaccepts any model ID your provider currently offers; check the provider's list rather than copying this one.
Caching
cache: { mode: "simple", max_age: 3600 } serves identical prompts from cache. max_age is seconds, minimum 60, maximum 90 days, default 7 days. mode: "semantic" matches similar prompts but is limited to select Enterprise plans. Bypass per request with the header x-portkey-cache-force-refresh: true; partition by user with x-portkey-cache-namespace.
Guardrails
Create guardrail checks in the dashboard (PII, regex, JSON validity, word lists and others) and attach their IDs in the config:
{
"input_guardrails": ["pg-no-pii-4f2a1c"],
"output_guardrails": ["pg-json-valid-9d3b7e"]
}
before_request_hooks and after_request_hooks with { "id": "..." } still work identically. With deny enabled, a failed check returns HTTP 446; with deny off, the request continues and returns 246 with the verdicts.
Examples
Example 1: Survive an OpenAI outage
Request: "If OpenAI returns 429 or 5xx, fall back to Claude automatically."
Use fallbackConfig above. Result: a failed first target is retried twice, then the call goes to @anthropic-prod; both attempts appear in the Portkey logs and the caller still gets an OpenAI-shaped response.
Example 2: Split traffic and cut repeat cost
Request: "Send 80% of traffic to gpt-4o-mini and 20% to gpt-4o, and cache repeats for an hour."
{
"strategy": { "mode": "loadbalance" },
"targets": [
{ "provider": "@openai-prod", "override_params": { "model": "gpt-4o-mini" }, "weight": 0.8 },
{ "provider": "@openai-prod", "override_params": { "model": "gpt-4o" }, "weight": 0.2 }
],
"cache": { "mode": "simple", "max_age": 3600 }
}
Result: roughly 8 in 10 requests use the smaller model; identical prompts within the hour are answered from cache and flagged as cache hits in the logs.
Guidelines
weightonly matters forloadbalance; do not combine it withfallback.- Keep
PORTKEY_API_KEYserver-side. Provider keys belong in the Model Catalog, not in application code. - Semantic caching, budget limits and some guardrails depend on plan; confirm in the dashboard before promising them.
- Budget and rate limits are set in the dashboard per key or workspace, not in the config object.
- A gateway adds a network hop; for latency-critical paths run the open-source gateway close to the app.