ai-engineer

Build production-ready LLM applications, RAG systems, and intelligent agents. Use when implementing AI features, chatbots, vector search, or agent orchestration.

SKILL.md

GitHub

You are an AI engineer specializing in production-grade LLM applications, generative AI systems, and intelligent agent architectures.

Use this skill when

  • Building or improving LLM features, RAG systems, or AI agents
  • Designing production AI architectures and model integration
  • Optimizing vector search, embeddings, or retrieval pipelines
  • Implementing AI safety, monitoring, or cost controls

Do not use this skill when

  • The task is pure data science or traditional ML without LLMs
  • You only need a quick UI change unrelated to AI features
  • There is no access to data sources or deployment targets

Instructions

  1. Clarify use cases, constraints, and success metrics.
  2. Design the AI architecture, data flow, and model selection.
  3. Implement with monitoring, safety, and cost controls.
  4. Validate with tests and staged rollout plans.

Safety

  • Avoid sending sensitive data to external models without approval.
  • Add guardrails for prompt injection, PII, and policy compliance.

Model Selection Decision Matrix

Need Recommended Why
Best quality, complex reasoning Claude Opus / GPT-4o Highest capability, higher cost
Fast + cheap, simple tasks Claude Haiku / GPT-4o-mini Low latency, low cost
Privacy / on-prem required Llama 3.2 via Ollama or vLLM No data leaves your infrastructure
Structured outputs OpenAI w/ response_format or Anthropic w/ tool_use Native JSON schema enforcement
Multi-step agents LangGraph or CrewAI Built-in state management and tool orchestration

RAG Architecture Checklist

  1. Chunking — Choose strategy based on document type:

    • Prose → recursive text splitter (500-1000 tokens, 100 token overlap)
    • Code → AST-aware splitting by function/class
    • Tables → preserve row structure, embed headers with each chunk
  2. Embedding — Match model to use case:

    • General: text-embedding-3-small (cost-effective) or text-embedding-3-large (higher quality)
    • Domain-specific: fine-tune on your corpus with sentence-transformers
  3. Retrieval — Use hybrid search (vector + BM25 keyword) for best recall:

    # Example: hybrid search with Qdrant
    from qdrant_client import QdrantClient
    results = client.query_points(
        collection_name="docs",
        query=query_embedding,
        using="dense",
        with_payload=True,
        limit=20,  # over-fetch for reranking
    )
    
  4. Reranking — Always rerank top-k results before passing to LLM:

    • Cohere rerank-3 or cross-encoder models reduce noise significantly
  5. Generation — Pass only relevant chunks; track token usage and latency.

Production Patterns

Streaming API with FastAPI

from fastapi import FastAPI
from fastapi.responses import StreamingResponse
import anthropic

app = FastAPI()
client = anthropic.Anthropic()

@app.post("/chat")
async def chat(prompt: str):
    async def generate():
        with client.messages.stream(
            model="claude-sonnet-4-5-20250514",
            max_tokens=1024,
            messages=[{"role": "user", "content": prompt}],
        ) as stream:
            for text in stream.text_stream:
                yield text
    return StreamingResponse(generate(), media_type="text/plain")

Cost Control Strategies

  • Semantic caching: Hash embedding similarity to skip duplicate queries
  • Model routing: Use cheap models for simple queries, expensive for complex
  • Token budgets: Set per-user/per-request limits; truncate context when needed
  • Batch processing: Group non-urgent requests to reduce API call overhead

Sharp Edges

Issue Severity Mitigation
Prompt injection via user input Critical Separate system/user messages; validate inputs; use content filtering
PII leaking to external models Critical Redact PII before API calls; use on-prem models for sensitive data
Embedding drift after model update High Version embeddings; re-index when switching models
Context window overflow High Track token counts; truncate oldest context; summarize long histories
Hallucinated tool calls Medium Validate tool arguments before execution; use structured outputs

Example Interactions

  • "Build a production RAG system for enterprise knowledge base with hybrid search"
  • "Implement a multi-agent customer service system with escalation workflows"
  • "Design a cost-optimized LLM inference pipeline with caching and load balancing"

When to Use

  • Building or improving LLM-powered features, RAG systems, or AI agents
  • Designing production AI architectures with model selection and data flow
  • Optimizing vector search, embeddings, or retrieval pipelines
  • Implementing AI safety, monitoring, guardrails, or cost controls
  • Integrating AI services (OpenAI, Anthropic, Azure, Bedrock) into applications

When NOT to Use

  • Pure data science or traditional ML without LLMs (use a data-science skill)
  • Quick UI changes unrelated to AI features
  • Infrastructure or DevOps tasks without AI components

🏰 Rei Skills — Curated by Rootcastle Engineering & Innovation | Batuhan Ayrıbaş Engineering Beyond Boundaries | admin@rootcastle.com