Framework Support

Khaos supports major frameworks through a single @khaosagent entrypoint. You keep your framework code; Khaos handles instrumentation, telemetry, faults, and security testing.

Supported Frameworks

FrameworkSyncAsyncStreaming
OpenAI SDKYesYesYes
Anthropic SDKYesYesYes
Google GeminiYesYesYes
Mistral SDKYesYesYes
Cohere SDKYes-Yes
LangChainYesYesYes
LangGraphYesYesYes
CrewAIYesYes-
AutoGenYesYes-
LlamaIndexYesYesYes
InstructorYesYes-

How It Works

Khaos uses runtime shimming to intercept LLM API calls from your decorated handler:

  1. Decorator entrypoint - Your handler receives user messages and returns a response
  2. Shim injection - Lightweight wrappers capture API calls during a Khaos run
  3. Telemetry collection - Prompts, completions, tokens, latency, costs
  4. Security testing - Adversarial inputs injected through same channel
Terminal
# Just run your agent - Khaos handles the rest
khaos discover
khaos run <agent-name> --sync
Zero overhead in production
Shim overhead is less than 0.0001% and only active during Khaos runs. Your production code runs unchanged.

OpenAI

Full support for the OpenAI Python SDK including chat completions, function calling, and streaming.

Python
# agent.py
from khaos import khaosagent
from openai import OpenAI

client = OpenAI()

@khaosagent(name="openai-agent", version="1.0.0", framework="openai")
def handle(message):
    prompt = (message.get("payload") or {}).get("text", "")
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": prompt}],
    )
    return {"text": response.choices[0].message.content}
Terminal
khaos discover
khaos run openai-agent --sync

Anthropic

Full support for the Anthropic Python SDK including Claude models and streaming.

Python
# agent.py
from khaos import khaosagent
from anthropic import Anthropic

client = Anthropic()

@khaosagent(name="anthropic-agent", version="1.0.0", framework="anthropic")
def handle(message):
    prompt = (message.get("payload") or {}).get("text", "")
    message = client.messages.create(
        model="claude-sonnet-4-20250514",
        max_tokens=1024,
        messages=[{"role": "user", "content": prompt}],
    )
    return {"text": message.content[0].text}

Google Gemini

Full support for the Google Generative AI (Gemini) SDK including content generation and streaming.

Python
# agent.py
from khaos import khaosagent
import google.generativeai as genai

genai.configure(api_key="...")
model = genai.GenerativeModel("gemini-1.5-pro")

@khaosagent(name="gemini-agent", version="1.0.0", framework="google")
def handle(message):
    prompt = (message.get("payload") or {}).get("text", "")
    response = model.generate_content(prompt)
    return {"text": response.text}

Mistral

Full support for the Mistral AI SDK including chat completions and streaming.

Python
# agent.py
from khaos import khaosagent
from mistralai import Mistral

client = Mistral(api_key="...")

@khaosagent(name="mistral-agent", version="1.0.0", framework="mistral")
def handle(message):
    prompt = (message.get("payload") or {}).get("text", "")
    response = client.chat.complete(
        model="mistral-large-latest",
        messages=[{"role": "user", "content": prompt}],
    )
    return {"text": response.choices[0].message.content}

Cohere

Support for the Cohere SDK including chat and streaming responses.

Python
# agent.py
from khaos import khaosagent
import cohere

client = cohere.Client("...")

@khaosagent(name="cohere-agent", version="1.0.0", framework="cohere")
def handle(message):
    prompt = (message.get("payload") or {}).get("text", "")
    response = client.chat(
        model="command-r-plus",
        message=prompt,
    )
    return {"text": response.text}

LangChain

Support for LangChain chains, agents, and tools. Khaos captures all LLM calls made through LangChain's model interfaces.

Python
# agent.py
from khaos import khaosagent
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-4o")

@khaosagent(name="langchain-agent", version="1.0.0", framework="langchain")
def handle(message):
    prompt = (message.get("payload") or {}).get("text", "")
    response = llm.invoke(prompt)
    return {"text": response.content}

CrewAI

Support for CrewAI multi-agent orchestration. Khaos captures all LLM calls across crew members.

Python
# agent.py
from khaos import khaosagent
from crewai import Agent, Task, Crew

researcher = Agent(
    role="Researcher",
    goal="Research the topic",
    backstory="You are an expert researcher.",
)

task = Task(
    description="Research AI safety",
    expected_output="A summary of AI safety research",
    agent=researcher,
)

crew = Crew(agents=[researcher], tasks=[task])

@khaosagent(name="crewai-agent", version="1.0.0", framework="crewai")
def handle(message):
    result = crew.kickoff()
    return {"text": str(result)}

AutoGen

Support for Microsoft AutoGen conversation patterns.

Python
# agent.py
from khaos import khaosagent
import autogen

config_list = [{"model": "gpt-4o", "api_key": "..."}]

assistant = autogen.AssistantAgent(
    name="assistant",
    llm_config={"config_list": config_list},
)

user_proxy = autogen.UserProxyAgent(
    name="user_proxy",
    human_input_mode="NEVER",
)

@khaosagent(name="autogen-agent", version="1.0.0", framework="autogen")
def handle(message):
    prompt = (message.get("payload") or {}).get("text", "")
    user_proxy.initiate_chat(assistant, message=prompt)
    # Get the last response from the conversation
    last_message = assistant.last_message()
    return {"text": last_message["content"] if last_message else ""}

LlamaIndex

Support for LlamaIndex RAG pipelines and query engines.

Python
# agent.py
from khaos import khaosagent
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
from llama_index.llms.openai import OpenAI

documents = SimpleDirectoryReader("data").load_data()
index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine()

@khaosagent(name="llamaindex-agent", version="1.0.0", framework="llamaindex")
def handle(message):
    prompt = (message.get("payload") or {}).get("text", "")
    response = query_engine.query(prompt)
    return {"text": str(response)}

Instructor

Support for Instructor structured output extraction.

Python
# agent.py
from khaos import khaosagent
import instructor
from openai import OpenAI
from pydantic import BaseModel

class User(BaseModel):
    name: str
    age: int

client = instructor.from_openai(OpenAI())

@khaosagent(name="instructor-agent", version="1.0.0", framework="instructor")
def handle(message):
    prompt = (message.get("payload") or {}).get("text", "")
    user = client.chat.completions.create(
        model="gpt-4o",
        response_model=User,
        messages=[{"role": "user", "content": prompt}],
    )
    return {"text": f"Name: {user.name}, Age: {user.age}"}

Automatic Output Extraction

Khaos automatically extracts output text from framework-native objects. You don't need to manually convert responses—just return what's natural:

Python
# LangChain - return the AIMessage directly
@khaosagent(name="langchain-agent", version="1.0.0")
def handle(message):
    prompt = (message.get("payload") or {}).get("text", "")
    response = llm.invoke(prompt)
    return response  # AIMessage - Khaos extracts .content automatically

# CrewAI - return the crew output
@khaosagent(name="crewai-agent", version="1.0.0")
def handle(message):
    result = crew.kickoff()
    return result  # CrewOutput - Khaos extracts .raw automatically

# Or use any common key name
return {"content": "..."}   # OpenAI-style
return {"result": "..."}    # Tool-style
return {"answer": "..."}    # Q&A-style
Supported Keys
Khaos checks these output keys in order: content, text, message, value, result, response, output, answer.

Telemetry Captured

For all frameworks, Khaos captures:

  • Request data - Model, messages, parameters
  • Response data - Completions, tool calls, finish reasons
  • Token usage - Prompt tokens, completion tokens, total
  • Latency - Time to first token, total duration
  • Cost - USD cost based on token pricing
  • Tool calls - Function calls and their results

Running Framework Agents

After adding the @khaosagent decorator, discover and run your agent:

Terminal
# Discover agents in your project
khaos discover

# Run evaluation
khaos run <agent-name> --eval quickstart

# Run with cloud sync
khaos run <agent-name> --eval quickstart --sync

See @khaosagent Decorator for all decorator parameters and advanced patterns.