API Discoverability: Helping AI Agents Find Your Endpoints

Your website has API endpoints. Maybe it is a SaaS product with a public API, maybe it is a Next.js application with API routes for forms and search, or maybe you have built MCP server endpoints and an /api/ai/context endpoint as part of your agentic web stack. The question is: can AI agents find them?

Most websites inadvertently hide their API endpoints from AI agents. The default robots.txt configuration in many frameworks blocks all /api/ paths. API documentation lives behind authenticated developer portals. Endpoint URLs are buried in JavaScript code that crawlers cannot execute. The result is that AI agents can read your marketing pages but cannot interact with your product.

This is a missed opportunity. When AI agents can discover and understand your API endpoints, they can recommend your product to users, integrate your services into workflows, and drive traffic that converts at rates far exceeding traditional search. The agentic web is not just about publishing content for AI to read. It is about exposing capabilities for AI to use.

This guide covers every method for making your API endpoints discoverable: from discovery files like llms.txt and MCP manifests, to robots.txt configuration, self-documenting endpoints, OpenAPI specifications, and endpoint naming conventions. Use the AgentReady scanner to audit your current API discoverability automatically.

Why API discoverability matters for AI agents

Traditional API documentation was written for human developers who would read docs, generate API keys, and integrate your service into their applications over hours or days. AI agent integration happens in seconds. An agent discovers your endpoint, understands its capabilities, and makes a decision about whether to use it, all within a single conversation turn.

How AI agents find and evaluate APIs

When an AI agent encounters your website (either through a direct user query or through crawling), it follows a discovery chain:

1. Check robots.txt to understand what paths are accessible
2. Read llms.txt for a high-level overview of available APIs and tools
3. Fetch mcp.json for a machine-readable list of MCP tools
4. Call /api/ai/context for structured business data including endpoint URLs
5. Read .well-known/agent.json for A2A protocol capabilities
6. Check OpenAPI spec for detailed endpoint documentation

If your endpoints are not referenced in any of these files, the agent may never discover them. Even if the agent somehow finds an endpoint URL, it cannot use it effectively without understanding the request format, authentication requirements, and expected response structure.

The business impact of discoverable APIs

Discoverable APIs create new distribution channels for your product. When a user asks an AI assistant to "find a tool that can scan my website for AI readiness," the agent needs to discover services that offer that capability. If your API is discoverable and well-documented, the agent can recommend your product, explain its capabilities, and even help the user interact with it.

This is fundamentally different from traditional SEO. Traditional SEO gets your content in front of searchers. API discoverability gets your capabilities in front of agents that can recommend, compare, and integrate your services into user workflows.

Documenting APIs in llms.txt

The llms.txt file is the first place AI agents look for API information. It provides a plain-text overview of your site's capabilities, including API endpoints.

API section in llms.txt

Include a dedicated section for API endpoints in your llms.txt file. List each endpoint with its URL, method, purpose, and authentication requirements:

# Your Company

> One-line description of what your company does.

## API Endpoints

### Public (No Authentication)
- GET https://yourdomain.com/api/ai/context - Structured JSON about the company, products, and pricing
- GET https://yourdomain.com/api/mcp - MCP server info and available tools
- GET https://yourdomain.com/api/search?q={query} - Search site content

### Authenticated (Bearer Token)
- POST https://yourdomain.com/api/mcp - Execute MCP tools (requires API key)
- POST https://yourdomain.com/api/agent - A2A agent communication (JSON-RPC 2.0)

## Discovery Files
- llms.txt: https://yourdomain.com/llms.txt
- context.md: https://yourdomain.com/context.md
- mcp.json: https://yourdomain.com/mcp.json
- AgentCard: https://yourdomain.com/.well-known/agent.json
- OpenAPI: https://yourdomain.com/api/openapi.json

## Quick Start
curl https://yourdomain.com/api/ai/context
curl https://yourdomain.com/api/mcp
curl https://yourdomain.com/api/search?q=pricing

Why curl examples matter

Including curl examples in your llms.txt serves two purposes. First, AI agents can include these examples when recommending your API to users, making adoption frictionless. Second, the examples demonstrate that your endpoints are live and accessible, building confidence in your API as a reliable integration target.

The MCP manifest: mcp.json

The MCP manifest is the most structured way to document your API for AI agents. Placed at public/mcp.json, it provides a machine-readable description of your MCP server and all available tools.

Complete mcp.json example

{
  "name": "your-product",
  "description": "What your product does in one sentence",
  "endpoint": "https://yourdomain.com/api/mcp",
  "protocol": "mcp-http",
  "auth": {
    "type": "bearer",
    "header": "Authorization",
    "description": "API key from your dashboard settings"
  },
  "tools": [
    {
      "name": "get_services",
      "description": "Returns available services with pricing and descriptions",
      "scope": "public",
      "inputSchema": {
        "type": "object",
        "properties": {},
        "required": []
      }
    },
    {
      "name": "search_content",
      "description": "Search articles, guides, and documentation by keyword",
      "scope": "public",
      "inputSchema": {
        "type": "object",
        "properties": {
          "query": {
            "type": "string",
            "description": "Search query"
          },
          "limit": {
            "type": "number",
            "description": "Maximum results to return (default: 10)"
          }
        },
        "required": ["query"]
      }
    },
    {
      "name": "submit_enquiry",
      "description": "Submit a project enquiry with contact details and project description",
      "scope": "authenticated",
      "inputSchema": {
        "type": "object",
        "properties": {
          "name": { "type": "string" },
          "email": { "type": "string", "format": "email" },
          "company": { "type": "string" },
          "message": { "type": "string" }
        },
        "required": ["email", "message"]
      }
    }
  ],
  "resources": [
    {
      "name": "company_context",
      "description": "Structured JSON context about the company",
      "uri": "https://yourdomain.com/api/ai/context",
      "mimeType": "application/json"
    },
    {
      "name": "context_markdown",
      "description": "Rich context about the company",
      "uri": "https://yourdomain.com/context.md",
      "mimeType": "text/markdown"
    }
  ]
}

Tools versus resources

The MCP manifest distinguishes between tools and resources. Tools are actions that agents can invoke with parameters, like searching content or submitting a form. Resources are data that agents can read, like your context endpoint or documentation. Make sure to list both in your manifest.

Scope: public versus authenticated

Mark each tool with a scope field. Public tools do not require authentication and can be called by any agent. Authenticated tools require a bearer token or API key. This helps agents understand which tools they can use immediately and which require a registration step.

Configuring robots.txt for API access

The most common reason AI agents cannot find API endpoints is overly restrictive robots.txt rules. Many frameworks default to blocking all /api/ paths, which prevents AI crawlers from discovering your endpoints.

The default problem

Next.js, Django, Rails, and many other frameworks generate robots.txt rules that block API routes:

# Common default that blocks AI agent discovery
User-agent: *
Disallow: /api/

This single line prevents every AI crawler from accessing any of your API endpoints, including public ones like /api/ai/context and /api/mcp.

The correct configuration

Selectively allow public, read-only API paths while blocking authenticated and internal routes:

# Allow AI-specific public endpoints
User-agent: GPTBot
Allow: /api/ai/
Allow: /api/mcp
Allow: /api/agent
Allow: /api/search
Disallow: /api/admin/
Disallow: /api/auth/
Disallow: /api/webhooks/

User-agent: ClaudeBot
Allow: /api/ai/
Allow: /api/mcp
Allow: /api/agent
Allow: /api/search
Disallow: /api/admin/
Disallow: /api/auth/
Disallow: /api/webhooks/

User-agent: PerplexityBot
Allow: /api/ai/
Allow: /api/mcp
Allow: /api/agent
Allow: /api/search
Disallow: /api/admin/
Disallow: /api/auth/
Disallow: /api/webhooks/

The /api/ai/ convention

Grouping AI-specific endpoints under the /api/ai/ prefix makes robots.txt configuration cleaner. A single Allow: /api/ai/ rule opens up all your AI endpoints without exposing internal routes. This convention is becoming a de facto standard across the agentic web ecosystem.

Next.js implementation

In Next.js App Router, configure robots programmatically:

// app/robots.ts
import type { MetadataRoute } from "next";

export default function robots(): MetadataRoute.Robots {
  return {
    rules: [
      {
        userAgent: "*",
        allow: ["/"],
        disallow: ["/admin/", "/api/admin/", "/api/auth/", "/api/webhooks/"],
      },
      {
        userAgent: ["GPTBot", "ClaudeBot", "PerplexityBot", "Google-Extended"],
        allow: ["/", "/api/ai/", "/api/mcp", "/api/agent", "/api/search"],
        disallow: ["/admin/", "/api/admin/", "/api/auth/"],
      },
    ],
    sitemap: "https://yourdomain.com/sitemap.xml",
  };
}

Self-documenting endpoints

A self-documenting endpoint returns information about itself when called with a GET request, and performs its function on POST. This pattern makes endpoints discoverable without external documentation.

The MCP server pattern

Your MCP server endpoint should return server information on GET and handle tool execution on POST:

// app/api/mcp/route.ts
import { NextRequest, NextResponse } from "next/server";

// GET: Return server info (public, no auth)
export async function GET() {
  return NextResponse.json({
    name: "your-product",
    version: "1.0.0",
    description: "What this MCP server does",
    protocol: "mcp-http",
    tools: [
      {
        name: "get_services",
        description: "Returns available services",
        inputSchema: { type: "object", properties: {} },
      },
      {
        name: "search_content",
        description: "Search site content",
        inputSchema: {
          type: "object",
          properties: {
            query: { type: "string" },
          },
          required: ["query"],
        },
      },
    ],
    resources: [
      {
        name: "company_context",
        uri: "https://yourdomain.com/api/ai/context",
      },
    ],
  });
}

// POST: Execute tools (authenticated)
export async function POST(request: NextRequest) {
  // Validate auth
  const authHeader = request.headers.get("authorization");
  if (!authHeader?.startsWith("Bearer ")) {
    return NextResponse.json(
      { error: "Authentication required" },
      { status: 401 }
    );
  }

  // Handle tool execution...
}

The A2A agent endpoint pattern

Similarly, your A2A endpoint should return its AgentCard on GET:

// app/api/agent/route.ts
import { NextRequest, NextResponse } from "next/server";
import agentCard from "@/public/.well-known/agent.json";

// GET: Return AgentCard
export async function GET() {
  return NextResponse.json(agentCard, {
    headers: {
      "Access-Control-Allow-Origin": "*",
    },
  });
}

// OPTIONS: CORS preflight
export async function OPTIONS() {
  return new NextResponse(null, {
    status: 200,
    headers: {
      "Access-Control-Allow-Origin": "*",
      "Access-Control-Allow-Methods": "GET, POST, OPTIONS",
      "Access-Control-Allow-Headers": "Content-Type, Authorization",
    },
  });
}

// POST: Handle A2A tasks (JSON-RPC 2.0)
export async function POST(request: NextRequest) {
  // Handle incoming tasks...
}

The search endpoint pattern

Even utility endpoints like search can be self-documenting:

// app/api/search/route.ts
import { NextRequest, NextResponse } from "next/server";

export async function GET(request: NextRequest) {
  const query = request.nextUrl.searchParams.get("q");

  // If no query, return documentation
  if (!query) {
    return NextResponse.json({
      endpoint: "/api/search",
      method: "GET",
      description: "Search site content by keyword",
      parameters: {
        q: { type: "string", required: true, description: "Search query" },
        limit: { type: "number", required: false, default: 10 },
        type: { type: "string", required: false, enum: ["guide", "case-study", "page"] },
      },
      example: "/api/search?q=AI+agents&limit=5",
    });
  }

  // Perform search and return results...
}

OpenAPI and Swagger specifications

For APIs with multiple endpoints and complex request/response schemas, an OpenAPI specification provides the most comprehensive documentation. AI agents can read OpenAPI specs to understand every endpoint, parameter, and response format.

Generating an OpenAPI spec

Create a JSON or YAML OpenAPI specification and serve it at a predictable URL:

// app/api/openapi.json/route.ts
import { NextResponse } from "next/server";

const spec = {
  openapi: "3.0.0",
  info: {
    title: "Your Product API",
    version: "1.0.0",
    description: "API for interacting with Your Product",
    contact: {
      email: "api@yourdomain.com",
    },
  },
  servers: [
    {
      url: "https://yourdomain.com",
      description: "Production",
    },
  ],
  paths: {
    "/api/ai/context": {
      get: {
        summary: "Get company context",
        description: "Returns structured JSON about the company, products, and pricing",
        operationId: "getContext",
        tags: ["Discovery"],
        responses: {
          "200": {
            description: "Company context",
            content: {
              "application/json": {
                schema: {
                  type: "object",
                  properties: {
                    company: { type: "object" },
                    products: { type: "array" },
                    pricing: { type: "object" },
                  },
                },
              },
            },
          },
        },
      },
    },
    "/api/search": {
      get: {
        summary: "Search content",
        operationId: "searchContent",
        tags: ["Content"],
        parameters: [
          {
            name: "q",
            in: "query",
            required: true,
            schema: { type: "string" },
            description: "Search query",
          },
          {
            name: "limit",
            in: "query",
            required: false,
            schema: { type: "integer", default: 10 },
            description: "Maximum results",
          },
        ],
        responses: {
          "200": {
            description: "Search results",
            content: {
              "application/json": {
                schema: {
                  type: "object",
                  properties: {
                    results: { type: "array" },
                    total: { type: "integer" },
                  },
                },
              },
            },
          },
        },
      },
    },
  },
};

export async function GET() {
  return NextResponse.json(spec, {
    headers: {
      "Access-Control-Allow-Origin": "*",
      "Cache-Control": "public, max-age=86400",
    },
  });
}

When to use OpenAPI versus mcp.json

Criteria	mcp.json	OpenAPI
Primary audience	AI agents	Developers and AI agents
Complexity	Simple (tools + resources)	Comprehensive (full API spec)
Best for	2-10 MCP tools	Full REST APIs with many endpoints
Auth description	Basic (type + header)	Detailed (OAuth flows, scopes)
Response schemas	Not included	Full JSON Schema for every response
Ecosystem support	MCP clients	Swagger UI, Postman, every API tool

For most websites with a handful of MCP tools and public endpoints, mcp.json is sufficient. Add OpenAPI if you have a comprehensive public API with many endpoints, complex authentication, or if developer documentation is a priority.

Endpoint naming conventions for AI discoverability

How you name your endpoints directly affects whether AI agents can find and understand them. Follow these conventions for maximum discoverability.

Use descriptive, predictable paths

# Good: descriptive and predictable
/api/ai/context          # AI context endpoint
/api/mcp                 # MCP server
/api/agent               # A2A agent
/api/search              # Content search
/api/products            # Product listing
/api/pricing             # Pricing information

# Bad: opaque and unpredictable
/api/v2/ctx              # Abbreviated, unclear
/api/rpc                 # Too generic
/api/q                   # Meaningless
/api/internal/data-feed  # Internal naming leaked

Group AI endpoints under /api/ai/

Grouping AI-specific endpoints under a common prefix has three benefits:

• A single robots.txt rule (Allow: /api/ai/) opens all AI endpoints
• Agents can discover related endpoints by exploring the prefix
• Internal and AI-facing endpoints are clearly separated

/api/ai/context    # Structured company context
/api/ai/search     # AI-optimized search
/api/ai/recommend  # Product recommendations
/api/ai/pricing    # Machine-readable pricing

Use REST conventions

AI agents understand REST conventions. Use nouns for resources and HTTP methods for actions:

GET  /api/products           # List products
GET  /api/products/:id       # Get single product
POST /api/enquiries          # Create enquiry
GET  /api/search?q=keyword   # Search content

Avoid versioned API paths for public endpoints

For public discovery endpoints, avoid version prefixes like /api/v1/. AI agents check for conventional paths. If your context endpoint is at /api/v3/ai/context instead of /api/ai/context, agents may not find it. Use versioning internally but keep public discovery endpoints at standard paths.

The .well-known/agent.json AgentCard

The AgentCard is part of the Google A2A (Agent-to-Agent) protocol. Placed at public/.well-known/agent.json, it describes your agent's capabilities, skills, authentication requirements, and endpoint URL.

{
  "name": "Your Product Agent",
  "description": "What your agent does",
  "url": "https://yourdomain.com/api/agent",
  "provider": {
    "organization": "Your Company",
    "url": "https://yourdomain.com"
  },
  "version": "1.0.0",
  "capabilities": {
    "streaming": false,
    "pushNotifications": false
  },
  "authentication": {
    "schemes": ["bearer"]
  },
  "defaultInputModes": ["text"],
  "defaultOutputModes": ["text"],
  "skills": [
    {
      "id": "answer-questions",
      "name": "Answer Questions",
      "description": "Answers questions about the company, services, and pricing",
      "tags": ["information", "services", "pricing"],
      "examples": [
        "What services do you offer?",
        "How much does a website cost?",
        "Can you build an AI agent?"
      ]
    },
    {
      "id": "scan-website",
      "name": "Website Audit",
      "description": "Scans a website for AI agent readiness",
      "tags": ["audit", "scan", "AI readiness"],
      "examples": [
        "Scan example.com for AI readiness",
        "Check if my website is agent-ready"
      ]
    }
  ]
}

The AgentCard is particularly important for agent-to-agent discovery. When another AI agent wants to delegate a task to your agent, it reads the AgentCard to understand capabilities and determine whether your agent can handle the request.

The context.md file for API documentation

While llms.txt provides a brief overview, context.md (placed in public/) can include detailed API documentation in Markdown format that conversational AI agents read naturally.

# Your Company - Context

## API Access

### Public Endpoints (No Authentication)

**GET /api/ai/context**
Returns structured JSON describing the company, products, pricing, and available tools.
Response: `{ company: {...}, products: [...], pricing: {...}, tools: {...} }`

**GET /api/mcp**
Returns MCP server information and list of available tools.
Response: `{ name: "...", tools: [...], resources: [...] }`

**GET /api/search?q={query}**
Searches all site content including guides, case studies, and pages.
Parameters: `q` (required), `limit` (optional, default 10)
Response: `{ results: [...], total: number }`

### Authenticated Endpoints (Bearer Token)

**POST /api/mcp**
Execute MCP tools. Requires API key in Authorization header.
Body: `{ tool: "tool_name", params: {...} }`

**POST /api/agent**
A2A agent communication. JSON-RPC 2.0 protocol.
Body: `{ jsonrpc: "2.0", method: "tasks/send", params: {...} }`

### Getting an API Key
Contact hello@yourdomain.com for API access.

Connecting all discovery files together

The power of API discoverability comes from cross-referencing. Each discovery file should reference the others, creating a web of documentation that agents can navigate.

The complete cross-reference map

File	References
robots.txt	Allow rules for /api/ai/, /api/mcp, /api/agent; Sitemap URL
llms.txt	All endpoint URLs, context.md URL, mcp.json URL, curl examples
mcp.json	MCP endpoint URL, tool list, /api/ai/context as a resource
/api/ai/context	MCP endpoint, A2A endpoint, llms.txt URL, context.md URL
.well-known/agent.json	A2A endpoint URL, skills, auth requirements
context.md	Detailed endpoint docs, authentication guide, examples
OpenAPI spec	Full endpoint schemas, request/response formats

An agent's discovery journey

Here is how a well-configured site guides an agent from discovery to action:

1. Agent fetches /robots.txt, sees Allow: /api/ai/ and Allow: /api/mcp
2. Agent fetches /llms.txt, reads the API section, notes endpoint URLs and curl examples
3. Agent fetches /mcp.json, discovers 3 available tools with input schemas
4. Agent calls /api/ai/context, gets structured data about the company and pricing
5. Agent now has enough context to recommend the product accurately
6. If the user wants to interact, agent calls the MCP tool or A2A endpoint

At no point does the agent need to parse HTML, guess endpoint URLs, or navigate through a developer portal. The entire discovery-to-action journey happens through structured, machine-readable files.

Common mistakes that block API discoverability

Blocking all /api/ paths in robots.txt

This is the single most common mistake. The default robots.txt in many frameworks blocks /api/, which prevents AI crawlers from discovering public endpoints. Always explicitly allow your public AI endpoints.

Missing CORS headers on public endpoints

AI agents running in browsers (through WebMCP or extensions) need CORS headers to call your endpoints. Without Access-Control-Allow-Origin: * on public endpoints, browser-based agents will be blocked.

Returning HTML error pages for API routes

If your middleware redirects unauthenticated requests to a login page, API endpoints may return HTML instead of JSON. AI agents cannot parse HTML error pages. Ensure API routes always return JSON responses, even for errors:

// Middleware should skip API routes
export function middleware(request: NextRequest) {
  // Skip API routes
  if (request.nextUrl.pathname.startsWith("/api/")) {
    return NextResponse.next();
  }

  // Auth check for other routes...
}

Requiring authentication for discovery endpoints

Discovery endpoints like /api/ai/context, /api/mcp (GET), and /api/search should be public. If they require authentication, agents cannot discover your capabilities. Reserve authentication for write operations and sensitive data.

Using internal jargon in endpoint names

Endpoints named /api/v2/ctx, /api/internal/feed, or /api/rpc are not discoverable because agents cannot infer their purpose from the path. Use descriptive, standard names that follow common conventions.

Not updating discovery files when endpoints change

If you add a new API endpoint but do not update your llms.txt, mcp.json, and context endpoint to reference it, agents will not discover it. Treat discovery file updates as part of your deployment checklist.

Testing API discoverability

Automated scanning

Use the AgentReady scanner to automatically check:

• Whether robots.txt allows /api/ai/ and /api/mcp paths
• Whether llms.txt exists and lists API endpoints
• Whether mcp.json exists and has valid tool definitions
• Whether /api/ai/context returns valid JSON
• Whether .well-known/agent.json exists with valid AgentCard
• Whether endpoints have CORS headers

Manual verification

# Check robots.txt allows AI endpoints
curl -s https://yourdomain.com/robots.txt | grep -A2 "api"

# Verify llms.txt lists endpoints
curl -s https://yourdomain.com/llms.txt

# Fetch MCP manifest
curl -s https://yourdomain.com/mcp.json | jq .

# Test context endpoint
curl -s https://yourdomain.com/api/ai/context | jq .

# Check MCP server info
curl -s https://yourdomain.com/api/mcp | jq .

# Verify CORS headers
curl -I -H "Origin: https://example.com" \
  https://yourdomain.com/api/ai/context

# Test AgentCard
curl -s https://yourdomain.com/.well-known/agent.json | jq .

Implementation checklist

Follow this checklist to make your API endpoints fully discoverable by AI agents:

1. Update robots.txt to allow /api/ai/, /api/mcp, and /api/agent
2. Add API section to llms.txt with endpoint URLs and curl examples
3. Create public/mcp.json with tool and resource definitions
4. Implement /api/ai/context endpoint with structured business data
5. Make MCP and A2A endpoints self-documenting (return info on GET)
6. Add CORS headers to all public endpoints
7. Create public/.well-known/agent.json AgentCard
8. Document API endpoints in public/context.md
9. Use descriptive endpoint names following REST conventions
10. Group AI endpoints under /api/ai/ prefix
11. Ensure middleware does not redirect API routes to HTML pages
12. Run the AgentReady scanner to validate everything

Frequently Asked Questions

What is API discoverability for AI agents?

API discoverability is the practice of making your API endpoints findable and understandable by AI agents. This includes documenting endpoints in machine-readable formats like llms.txt, MCP manifests, and OpenAPI specifications, as well as allowing access through robots.txt rules and providing self-documenting endpoint responses. When your APIs are discoverable, AI agents can recommend, compare, and interact with your services programmatically.

How do AI agents discover API endpoints?

AI agents discover endpoints through multiple channels: llms.txt files that list API URLs with curl examples, mcp.json manifests that describe available MCP tools, /api/ai/context endpoints that return structured business data with endpoint URLs, robots.txt Allow rules for /api/ paths, .well-known/agent.json AgentCards for A2A discovery, and OpenAPI specifications for detailed endpoint documentation. Agents check these files in order of convention, building a complete picture of your API surface.

Should I allow AI crawlers to access my API routes?

Allow access to public, read-only API routes that provide information, such as /api/ai/context, /api/mcp (GET), and /api/search. Block access to authenticated endpoints, admin routes, internal webhooks, and write operations. Use specific Allow and Disallow directives in robots.txt rather than blanket-blocking all /api/ paths, which is the default in many frameworks and the most common cause of poor API discoverability.

What is the difference between mcp.json and OpenAPI for AI agents?

mcp.json is a lightweight manifest specifically designed for AI agent tool discovery. It lists available MCP tools with names, descriptions, input schemas, and scopes (public vs authenticated). OpenAPI (Swagger) is a comprehensive API specification standard that describes full request/response schemas, authentication flows, and endpoints. mcp.json is simpler and purpose-built for agents. OpenAPI is more detailed and serves both developers and agents. Use mcp.json for small tool sets and add OpenAPI for comprehensive public APIs.

How should I name API endpoints for AI agent discoverability?

Use descriptive, predictable names that follow REST conventions. Prefer /api/ai/context over /api/v2/ctx. Use nouns for resources (/api/products) and standard verbs for search (/api/search?q=keyword). Group AI-specific endpoints under /api/ai/ to simplify robots.txt configuration. Avoid abbreviations, internal jargon, or opaque identifiers that agents cannot interpret from the path alone.

What is a self-documenting API endpoint?

A self-documenting endpoint returns information about itself when called with a GET request without parameters or with an empty query. For example, your MCP endpoint at /api/mcp should return server info, available tools, and resource listings on GET, while handling tool execution on POST. Your search endpoint at /api/search should return parameter documentation when called without a query string. This allows agents to understand the endpoint without consulting external documentation.

How do I test whether AI agents can find my API endpoints?

Use the AgentReady scanner to automatically check endpoint discoverability across all your discovery files. Manually verify by fetching your llms.txt, mcp.json, and robots.txt to confirm endpoint URLs are listed and accessible. Test each public endpoint with curl to ensure it returns proper JSON with CORS headers. Check that your middleware does not redirect API routes to HTML login pages. Verify that cross-references between discovery files are accurate and up to date.