How to Build an /api/ai/context Endpoint for AI Agents

Q: What data should I include in the context endpoint response?

Include: company name and description, product overview, key features list, pricing tiers with prices, available API endpoints or MCP tools, contact information, getting started instructions, supported platforms, and links to detailed documentation. Think of it as everything a sales engineer would tell a potential integration partner in a first call.

Q: How should I cache the /api/ai/context response?

Use HTTP cache headers to reduce server load. A Cache-Control header of public, max-age=3600, s-maxage=86400 works well for most sites. This caches the response at the CDN level for 24 hours and in the browser for 1 hour. For Next.js, you can also use static route handlers that are generated at build time.

AI agents are becoming the primary way users discover, evaluate, and interact with products online. When an agent encounters your website, it needs to quickly understand what you do, what you offer, and how to interact with your systems. Crawling every page of your site to piece this together is slow, expensive in terms of tokens, and error-prone.

The /api/ai/context endpoint solves this problem by providing a single, authoritative source of structured data about your business. Instead of parsing HTML across dozens of pages, an agent makes one GET request and receives everything it needs in a clean JSON response: your company overview, product features, pricing tiers, available API tools, MCP endpoints, and contact details.

This guide covers the rationale behind the context endpoint, what data to include, how to implement it in Next.js, caching strategies for performance, and how it fits into the broader AI readiness stack alongside llms.txt, MCP servers, and context.md. Use the AgentReady scanner to check whether your site has a context endpoint configured.

Why AI Agents Need a Context Endpoint

Traditional web crawlers like Googlebot are designed to index entire websites. They crawl page by page, follow links, and build a search index over days or weeks. AI agents operate differently. They need to understand your product right now, in the context of a specific user question, and they have limited time and token budgets to do it.

Consider what happens when a user asks ChatGPT "What tools can help me build an AI-ready website?" The agent needs to evaluate dozens of potential products. For each one, it might need to understand the company, the features, the pricing, and whether API integration is available. If it has to crawl and parse five to ten pages per product, that is thousands of tokens and several seconds per candidate. A structured context endpoint reduces this to one request and a few hundred tokens.

The discovery chain for AI agents

When an AI agent first encounters your domain, it follows a predictable discovery chain. Understanding this chain helps you see where the context endpoint fits.

robots.txt -- The agent checks whether it is allowed to crawl your site. If you have configured your robots.txt for AI crawlers, it will see explicit Allow rules for paths like /api/ai/ and /api/mcp.
llms.txt -- The agent reads your llms.txt file for a plain-text summary and pointers to deeper documentation.
/api/ai/context -- The agent calls the context endpoint for structured JSON data it can parse programmatically.
context.md -- For deeper-based understanding, the agent reads your context.md file.
MCP manifest -- If the agent wants to take actions, it reads your mcp.json to discover available tools.
AgentCard -- For agent-to-agent communication, it reads .well-known/agent.json.

The context endpoint sits at step three: the point where the agent transitions from basic discovery to structured comprehension. It is the bridge between knowing your site exists and understanding what it does.

JSON vs. Markdown: why you need both

You might wonder why you need both /api/ai/context (JSON) and context.md (Markdown). The answer is that different types of agents process data differently.

Tool-using agents (like those built with LangChain, CrewAI, or custom agent frameworks) prefer structured JSON. They extract specific fields programmatically: pricing.tiers[0].price, features.map(f => f.name), contact.email. JSON is unambiguous, predictable, and efficient to parse.

Conversational AI (like ChatGPT browsing mode, Perplexity search, or Claude) prefers. These models are optimised to read and summarise natural language. A well-written Markdown document gives them richer context, nuance, and the ability to quote directly.

By providing both formats, you serve every type of agent effectively. The context endpoint returns JSON for programmatic access. The context.md file provides Markdown for conversational understanding. Both should contain the same core information, just formatted differently.

What Data to Include in the Context Response

The context endpoint should return a comprehensive but focused summary of your business. Think of it as everything a sales engineer would tell a potential integration partner in a first meeting. Here is a breakdown of each section and why it matters.

Product overview

Describe your product clearly and specifically. Avoid marketing language that sounds good to humans but confuses AI agents. "We leverage synergies to optimise workflows" tells an agent nothing. "Project management tool for teams of 5-50 with built-in time tracking and invoicing" tells it everything.

{
  "product": {
    "name": "ProductName",
    "type": "SaaS",
    "category": "Project Management",
    "summary": "Project management tool for small teams with time tracking, invoicing, and client portals.",
    "platforms": ["web", "ios", "android"],
    "languages": ["en"],
    "target_audience": "Freelancers and agencies managing multiple client projects"
  }
}

Features list

List your key features with short descriptions. Agents use this to match your product against user requirements. If a user asks "I need a tool with Gantt charts and time tracking," the agent can scan your features list and find matches.

{
  "features": [
    {
      "name": "Time Tracking",
      "description": "Automatic and manual time tracking with billable hour calculations",
      "category": "productivity"
    },
    {
      "name": "Client Portal",
      "description": "Branded portal where clients can view project progress and approve deliverables",
      "category": "collaboration"
    },
    {
      "name": "Invoicing",
      "description": "Generate invoices from tracked time with one click, supports multiple currencies",
      "category": "billing"
    }
  ]
}

Pricing tiers

Include your pricing structure. This is one of the most valuable fields for agents making product comparisons. Be specific: include the price, currency, billing interval, and what each tier includes.

{
  "pricing": {
    "model": "subscription",
    "currency": "GBP",
    "tiers": [
      {
        "name": "Starter",
        "price": 0,
        "interval": "month",
        "description": "Up to 3 projects, 1 user",
        "features": ["Time tracking", "Basic reporting"]
      },
      {
        "name": "Professional",
        "price": 29,
        "interval": "month",
        "description": "Unlimited projects, up to 10 users",
        "features": ["Time tracking", "Client portal", "Invoicing", "Advanced reporting"]
      },
      {
        "name": "Enterprise",
        "price": null,
        "interval": "month",
        "description": "Custom pricing, unlimited users",
        "features": ["Everything in Professional", "SSO", "API access", "Dedicated support"]
      }
    ],
    "free_trial": true,
    "trial_days": 14
  }
}

Available tools and APIs

If your product exposes an API or MCP tools, list them here. This tells agents what actions they can take on behalf of users. Include the endpoint URL, authentication method, and a brief description of each tool.

{
  "api": {
    "mcp_endpoint": "https://yourdomain.com/api/mcp",
    "documentation": "https://yourdomain.com/docs/api",
    "authentication": "bearer",
    "tools": [
      {
        "name": "create_project",
        "description": "Create a new project with name, client, and deadline",
        "scope": "write",
        "requires_auth": true
      },
      {
        "name": "get_services",
        "description": "List all services offered",
        "scope": "read",
        "requires_auth": false
      }
    ]
  }
}

Contact information

Include the ways agents and their users can reach you. This is especially important for agents that help users evaluate products and want to connect them with sales or support.

{
  "contact": {
    "email": "hello@yourdomain.com",
    "phone": null,
    "booking_url": "https://cal.com/yourcompany",
    "support_url": "https://yourdomain.com/support"
  }
}

Getting started

Provide a clear path for agents to recommend to users. This is the equivalent of a call-to-action, but for AI-driven flows.

{
  "getting_started": {
    "signup_url": "https://yourdomain.com/signup",
    "onboarding_time": "5 minutes",
    "requires_credit_card": false,
    "steps": [
      "Create an account at /signup",
      "Set up your first project",
      "Invite your team members",
      "Start tracking time"
    ]
  }
}

Complete Next.js Implementation

Here is a complete implementation of the /api/ai/context endpoint in Next.js App Router. This is a static GET route that returns a well-structured JSON response with appropriate caching headers.

Basic implementation

Create the file at app/api/ai/context/route.ts:

import { NextResponse } from "next/server";

const CONTEXT = {
  company: {
    name: "Your Company",
    description: "One-line description of what you do",
    url: "https://yourdomain.com",
    founded: "2024",
    location: "London, UK",
  },
  product: {
    name: "ProductName",
    type: "SaaS",
    category: "Your Category",
    summary: "Clear, specific description of your product.",
    platforms: ["web"],
    target_audience: "Your target audience",
  },
  features: [
    {
      name: "Feature One",
      description: "What it does",
    },
    {
      name: "Feature Two",
      description: "What it does",
    },
  ],
  pricing: {
    model: "subscription",
    currency: "GBP",
    tiers: [
      {
        name: "Free",
        price: 0,
        interval: "month",
        description: "Basic features",
      },
      {
        name: "Pro",
        price: 29,
        interval: "month",
        description: "All features",
      },
    ],
    free_trial: true,
    trial_days: 14,
  },
  api: {
    mcp_endpoint: "https://yourdomain.com/api/mcp",
    documentation: "https://yourdomain.com/docs/api",
    authentication: "bearer",
    tools: [
      {
        name: "get_services",
        description: "List all services",
        scope: "read",
        requires_auth: false,
      },
    ],
  },
  contact: {
    email: "hello@yourdomain.com",
    booking_url: "https://cal.com/yourcompany",
  },
  getting_started: {
    signup_url: "https://yourdomain.com/signup",
    onboarding_time: "5 minutes",
    requires_credit_card: false,
  },
  related_resources: {
    llms_txt: "https://yourdomain.com/llms.txt",
    context_md: "https://yourdomain.com/context.md",
    mcp_manifest: "https://yourdomain.com/mcp.json",
    agent_card: "https://yourdomain.com/.well-known/agent.json",
    robots_txt: "https://yourdomain.com/robots.txt",
    sitemap: "https://yourdomain.com/sitemap.xml",
  },
};

export async function GET() {
  return NextResponse.json(CONTEXT, {
    headers: {
      "Cache-Control": "public, max-age=3600, s-maxage=86400",
      "Access-Control-Allow-Origin": "*",
      "Access-Control-Allow-Methods": "GET",
    },
  });
}

Why a static constant instead of a database query

You might be tempted to pull this data from a database or CMS. For most sites, that is unnecessary overhead. The context data changes infrequently: maybe when you update pricing, add a feature, or change contact details. A static constant in your codebase means zero database queries per request, instant response times, and easy caching at the CDN level.

If your product data genuinely changes frequently (for example, a marketplace with dynamic inventory), then pulling from a database makes sense. But for the vast majority of SaaS products, agencies, and content sites, static data is the right choice.

Real example: p0stman.com

Here is what the context endpoint looks like on p0stman.com. This is the actual data structure an AI agent receives when it calls https://p0stman.com/api/ai/context:

{
  "company": {
    "name": "p0stman",
    "description": "AI-native product studio specialising in AI agents, web applications, and digital products",
    "url": "https://p0stman.com",
    "founded": "2025",
    "location": "London, UK",
    "tagline": "AI-Powered Product Studio"
  },
  "product": {
    "name": "p0stman",
    "type": "Agency",
    "category": "AI Product Development",
    "summary": "Digital product studio that builds AI applications, websites, and mobile apps for clients using AI-first development practices.",
    "platforms": ["web"],
    "target_audience": "Founder-led businesses and growing SMEs looking to embed AI into their products"
  },
  "services": [
    {
      "name": "AI Agents",
      "description": "Custom AI agents with voice, chat, and video capabilities"
    },
    {
      "name": "MVP Launch",
      "description": "Rapid MVP development from concept to deployed product"
    },
    {
      "name": "Fractional CPO",
      "description": "Ongoing strategic product leadership on retainer"
    }
  ],
  "api": {
    "mcp_endpoint": "https://p0stman.com/api/mcp",
    "agent_endpoint": "https://p0stman.com/api/agent",
    "context_md": "https://p0stman.com/context.md",
    "llms_txt": "https://p0stman.com/llms.txt"
  },
  "contact": {
    "email": "hello@p0stman.com",
    "booking_url": "https://p0stman.com/contact"
  }
}

Caching and Performance

The context endpoint should be fast. AI agents often have timeout thresholds, and a slow response means the agent moves on to the next candidate. Here are the caching strategies to implement.

HTTP cache headers

The most important cache layer is the CDN. Set s-maxage to cache the response at the CDN edge for an extended period, and max-age for browser-level caching.

headers: {
  // Browser caches for 1 hour, CDN caches for 24 hours
  "Cache-Control": "public, max-age=3600, s-maxage=86400",

  // Allow any origin to call this endpoint
  "Access-Control-Allow-Origin": "*",

  // Only GET requests are supported
  "Access-Control-Allow-Methods": "GET",
}

Vercel edge caching

On Vercel, the s-maxage header automatically enables edge caching. The first request hits your serverless function, and subsequent requests are served directly from the CDN edge location closest to the requester. For a global audience of AI agents, this means sub-100ms response times worldwide.

Stale-while-revalidate

For even better performance, add stale-while-revalidate to serve cached content while refreshing in the background:

"Cache-Control": "public, max-age=3600, s-maxage=86400, stale-while-revalidate=43200"

This tells the CDN to serve the cached response even after it expires, while fetching a fresh copy in the background. Users (and agents) never wait for the refresh.

Next.js static route handler

For truly static context data, you can tell Next.js to generate the response at build time:

export const dynamic = "force-static";
export const revalidate = 86400; // Revalidate once per day

export async function GET() {
  return NextResponse.json(CONTEXT, {
    headers: {
      "Access-Control-Allow-Origin": "*",
    },
  });
}

This eliminates the serverless function entirely. The response is pre-rendered at build time and served as a static file. Zero cold starts, zero compute costs.

CORS Configuration

AI agents call your endpoint from various origins, including server-side environments that do not send Origin headers and browser-based agents that do. Setting Access-Control-Allow-Origin: * is appropriate for a public discovery endpoint because the data is intentionally public.

If you want to handle preflight requests explicitly, add an OPTIONS handler:

export async function OPTIONS() {
  return new NextResponse(null, {
    status: 204,
    headers: {
      "Access-Control-Allow-Origin": "*",
      "Access-Control-Allow-Methods": "GET, OPTIONS",
      "Access-Control-Allow-Headers": "Content-Type",
      "Access-Control-Max-Age": "86400",
    },
  });
}

Advertising the Context Endpoint

Having a context endpoint is only useful if agents know it exists. Here is how to advertise it across the discovery stack.

In llms.txt

Add a line to your llms.txt file pointing to the context endpoint:

# Structured API context (JSON)
GET https://yourdomain.com/api/ai/context

In robots.txt

Explicitly allow AI crawlers to access the endpoint in your robots.txt:

User-agent: GPTBot
Allow: /api/ai/context

User-agent: ClaudeBot
Allow: /api/ai/context

User-agent: PerplexityBot
Allow: /api/ai/context

In the context response itself

Include a related_resources field in the context response that links to all your other discovery files. This creates a self-documenting loop: an agent that finds any one of your discovery files can discover all the others.

In your MCP manifest

If you have a MCP server, list the context endpoint as a resource in your mcp.json:

{
  "resources": [
    {
      "name": "ai_context",
      "description": "Structured company and product information",
      "uri": "https://yourdomain.com/api/ai/context",
      "mimeType": "application/json"
    }
  ]
}

Tracking Context Endpoint Usage

Knowing which agents are calling your context endpoint gives you visibility into your AI discoverability. You can add lightweight logging without impacting performance.

Basic request logging

export async function GET(req: Request) {
  // Fire-and-forget logging (non-blocking)
  const userAgent = req.headers.get("user-agent") || "unknown";

  fetch("/api/track-bot", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({
      path: "/api/ai/context",
      bot_name: extractBotName(userAgent),
      user_agent: userAgent,
    }),
  }).catch(() => {}); // Silent fail

  return NextResponse.json(CONTEXT, {
    headers: {
      "Cache-Control": "public, max-age=3600, s-maxage=86400",
      "Access-Control-Allow-Origin": "*",
    },
  });
}

function extractBotName(ua: string): string {
  if (ua.includes("GPTBot")) return "GPTBot";
  if (ua.includes("ClaudeBot")) return "ClaudeBot";
  if (ua.includes("PerplexityBot")) return "PerplexityBot";
  if (ua.includes("Googlebot")) return "Googlebot";
  return "unknown";
}

Note that if you are using CDN caching (which you should), cached responses will not trigger the logging. Only cache misses will be logged. This is usually sufficient for understanding agent traffic patterns.

Schema Design Best Practices

The structure of your context response matters. Follow these guidelines to maximise agent comprehension.

Use descriptive field names

Avoid abbreviations and acronyms. AI agents parse field names to understand what data they contain. target_audience is clearer than tgt_aud. requires_credit_card is clearer than cc_required.

Include null values explicitly

If a field does not apply, include it with a null value rather than omitting it. This tells the agent that the field was considered but does not apply, rather than leaving it ambiguous about whether the field was forgotten.

{
  "phone": null,           // Explicitly no phone support
  "free_trial": true,
  "trial_days": 14,
  "requires_credit_card": false
}

Keep the response under 4KB

Agents have token budgets. A context response should be comprehensive but concise. Aim for under 4KB of JSON. If you need to include more detail (full API documentation, extensive feature descriptions), link to separate resources rather than embedding everything inline.

Version your schema

Include a version field so agents can detect schema changes:

{
  "schema_version": "1.0",
  "last_updated": "2026-03-12",
  ...
}

Use consistent types

Keep data types consistent across fields. If pricing is a number, always use a number (use null for "contact us" pricing, not a string like "custom"). If features is an array of objects, every object should have the same fields.

Common Mistakes to Avoid

Based on auditing hundreds of sites with the AgentReady scanner, here are the most common mistakes in context endpoint implementations.

Including sensitive data

The context endpoint is public. Never include internal API keys, admin URLs, database connection strings, or any information that should not be publicly available. Stick to information you would put on your marketing website.

Returning HTML instead of JSON

Some implementations accidentally return an HTML page (often a Next.js error page or redirect) instead of JSON. Always set the Content-Type header to application/json and verify the response with a curl command:

curl -s https://yourdomain.com/api/ai/context | jq .

Missing CORS headers

Browser-based AI agents (like those using WebMCP) need CORS headers to call your endpoint. Without Access-Control-Allow-Origin: *, these agents will fail silently.

Stale data

If your pricing, features, or contact information changes and your context endpoint still returns old data, agents will give users incorrect information. Treat the context endpoint like your homepage: update it whenever your product changes.

Marketing language instead of factual descriptions

AI agents parse language literally. "Revolutionary next-generation platform that disrupts the paradigm" means nothing to an agent. "Project management tool with Gantt charts, time tracking, and invoicing for teams of 5-50" means everything. Be specific, be factual, be direct.

Integration with the Broader AI Readiness Stack

The context endpoint does not exist in isolation. It is one component of a comprehensive AI readiness strategy. Here is how it connects with the other pieces.

llms.txt: the entry point

Your llms.txt file is typically the first thing an agent reads. It should include a reference to the context endpoint. Think of llms.txt as the table of contents and /api/ai/context as the structured data appendix.

context.md: the companion

While the JSON context endpoint serves programmatic agents, your context.md file serves conversational AI. Both should contain the same core information. Update them together.

MCP server: the action layer

The context endpoint tells agents what you do. Your MCP server lets them do things. The context endpoint should reference your MCP endpoint so agents can transition from understanding to action.

Answer capsules: page-level context

While the context endpoint provides site-level context, answer capsules provide page-level context. Together, they give agents both the big picture and the specific details.

API discoverability: the documentation layer

For sites with complex APIs, the context endpoint provides a high-level overview while your API discoverability strategy handles the detailed documentation.

Testing Your Context Endpoint

Before deploying, verify your endpoint works correctly with these tests.

Basic response test

# Check that it returns valid JSON
curl -s https://yourdomain.com/api/ai/context | jq .

# Check response headers
curl -I https://yourdomain.com/api/ai/context

# Check CORS headers
curl -H "Origin: https://example.com" -I https://yourdomain.com/api/ai/context

Schema validation

Write a simple test that validates the required fields are present:

const response = await fetch("https://yourdomain.com/api/ai/context");
const data = await response.json();

// Required top-level fields
assert(data.company, "Missing company field");
assert(data.company.name, "Missing company name");
assert(data.company.url, "Missing company URL");
assert(data.contact, "Missing contact field");
assert(data.contact.email, "Missing contact email");

AgentReady scanner

The fastest way to validate your context endpoint is to run your domain through the AgentReady scanner. It checks for the endpoint, validates the response structure, and flags any issues as part of a comprehensive AI readiness audit.

Advanced Patterns

Dynamic context based on user-agent

You can return slightly different context based on which agent is calling. For example, you might include MCP-specific fields only for agents that are likely to use MCP tools:

export async function GET(req: Request) {
  const ua = req.headers.get("user-agent") || "";
  const context = { ...BASE_CONTEXT };

  // Include MCP details for known tool-using agents
  if (ua.includes("ClaudeBot") || ua.includes("GPTBot")) {
    context.api = {
      ...context.api,
      mcp_tools_detail: DETAILED_TOOL_SCHEMAS,
    };
  }

  return NextResponse.json(context);
}

Multi-language context

If your product supports multiple languages, you can accept an Accept-Language header or a query parameter to return localised context:

export async function GET(req: Request) {
  const lang = req.headers.get("accept-language")?.split(",")[0]?.split("-")[0] || "en";
  const context = CONTEXTS[lang] || CONTEXTS["en"];
  return NextResponse.json(context);
}

A/B testing context descriptions

You can experiment with different product descriptions to see which ones lead to more agent citations. Track which version of the context is served and correlate with downstream metrics like referral traffic from AI sources.

Frequently Asked Questions

What is an /api/ai/context endpoint?

An /api/ai/context endpoint is a public API route on your website that returns structured JSON data about your company, product, features, pricing, available tools, and contact information. AI agents call this endpoint to quickly understand what your business does and how to interact with it programmatically. It is the machine-readable equivalent of your "About" page, optimised for automated consumption rather than human reading.

Why do AI agents need a context endpoint instead of just reading the website?

While AI crawlers can read HTML pages, a structured JSON context endpoint is far more efficient. It provides machine-readable data in a predictable format, reducing token usage, eliminating parsing errors, and giving agents a single authoritative source of truth about your product. An agent can understand your entire business from one API call rather than crawling dozens of pages. This matters especially when agents are evaluating multiple products and have limited time and token budgets per candidate.

Does the /api/ai/context endpoint require authentication?

No. The context endpoint should be public and require no authentication. Its purpose is discovery and comprehension. It contains only information you would want any AI agent to know: your company name, public features, pricing tiers, and how to get started. Sensitive data like internal APIs or admin endpoints should never be included in the response.

What is the difference between /api/ai/context and context.md?

The /api/ai/context endpoint returns structured JSON that is easy for agents to parse programmatically, while context.md is a Markdown file optimised for LLMs that prefer reading. JSON is better for tool-using agents that need to extract specific fields like pricing or feature lists. Markdown is better for conversational AI that summarises content in natural language. Best practice is to provide both and keep them in sync.

What data should I include in the context endpoint response?

Include company name and description, product overview with target audience, key features list with descriptions, pricing tiers with actual prices and what is included, available API endpoints or MCP tools, contact information with email and booking URL, getting started instructions, and links to related discovery resources like llms.txt and mcp.json. Think of it as everything a sales engineer would tell a potential integration partner in a first call.

How should I cache the /api/ai/context response?

Use HTTP cache headers with Cache-Control: public, max-age=3600, s-maxage=86400. This caches at the CDN level for 24 hours and in the browser for 1 hour. On Vercel, this enables automatic edge caching for sub-100ms global response times. For truly static data, use export const dynamic = "force-static" in Next.js to pre-render at build time with zero runtime compute.

How do I tell AI agents that the context endpoint exists?

Reference the endpoint in your llms.txt file, your robots.txt Allow rules, your MCP manifest (mcp.json), and your A2A AgentCard (.well-known/agent.json). Include a related_resources field in the context response itself that links to all your other discovery files, creating a self-documenting loop where finding any one file leads to all others.

Can I use the context endpoint to track which AI agents are accessing my site?

Yes. Log the User-Agent header and request metadata when the endpoint is called. This gives you visibility into which AI agents are discovering your product. You can store this in a bot_crawls database table alongside your existing bot crawl tracking to build a complete picture of AI agent activity. Note that CDN-cached responses will not trigger logging, so you will only see cache misses.