AI Agent Technologies, Standards & Platforms
AI agents interact with websites differently from human visitors. They need to access information, understand it reliably, discover available actions, interact with business systems, and increasingly transact directly with digital services.
We work with a broad range of web technologies, AI-agent standards, APIs, automation platforms, data systems, and engineering patterns to make those interactions reliable and commercially useful.
There is no fixed “AI agent stack.” The right solution depends on the customer journey, existing website architecture, business systems, security requirements, and commercial model. Wherever possible, we build on technology already in place rather than introducing unnecessary infrastructure.
Access & Control
Control how AI agents and crawlers access your website, identify who is making a request, and protect information or actions that should not be public.
Cloudflare AI Crawl Control & Bot Management
Identify and manage automated traffic, including AI crawlers, at the network edge.
Cloudflare CDN & WAF
Apply routing, caching, access policies, rate limits, and security rules before requests reach the application.
Web Bot Auth
Use cryptographically signed requests to verify supported bots and automated agents rather than relying only on easily imitated User-Agent strings.
robots.txt
Declare crawler access policies for website content.
OAuth
Authenticate applications and provide delegated access where an AI agent is acting on behalf of a user.
JWT — JSON Web Tokens
Represent authenticated identities and scoped authorization information between systems.
API Keys & Access Tokens
Provide controlled machine access to APIs and other non-public functionality.
Cloudflare Access
Protect applications and resources behind identity-aware access controls.
Supabase Authentication & Row-Level Security
Control authenticated access to application data and restrict which records individual users or services may access.
Engineering patterns: agent classification · User-Agent and IP-range identification · authentication · authorization · delegated user access · scoped tokens · role-based permissions · rate limits · quotas · least-privilege access
Information & Understanding
Make important business information easy for AI agents to retrieve, identify, interpret, and keep current.
Semantic HTML
Structure website content so that headings, navigation, tables, lists, forms, and other elements communicate their meaning clearly to machines as well as humans.
Server-Rendered & Static HTML
Expose important information directly in the page response rather than requiring complex browser interaction or JavaScript execution.
Machine-Facing HTML & Markdown
Provide simplified representations of selected content when the normal website experience is unnecessarily difficult for machine consumers.
JSON-LD
Embed structured, machine-readable information directly into web pages.
Schema.org Structured Data
Describe entities such as organizations, products, services, offers, events, articles, FAQs, and other website content using a shared vocabulary.
XML Sitemaps
Provide structured discovery of important website URLs.
llms.txt
Provide optional supplementary information intended to help AI systems discover relevant website resources.
Structured Feeds & APIs
Expose information that is dynamic, structured, or difficult to represent reliably through normal web pages.
CMS APIs & Webhooks
Keep website content, structured data, feeds, and agent-facing representations synchronized with the underlying source of truth.
Engineering patterns: single source of truth · authoritative data retrieval · structured factual content · server-side rendering · machine-readable representations · cache management · content synchronization · dynamic-data retrieval
Actions & Transactions
Allow AI agents to understand what they can do next and, where appropriate, perform those actions programmatically.
Examples include checking eligibility, requesting a quote, finding availability, registering, booking an appointment, creating a lead, or initiating a transaction.
REST APIs
Expose business information and actions through predictable programmatic interfaces.
MCP — Model Context Protocol
Expose tools and capabilities that compatible AI agents can discover and call, typically via a server you run.
WebMCP
A companion, browser-native standard that exposes callable tools directly inside a web page via JavaScript, so an in-browser agent can call them without a separate server or DOM scraping. A W3C Community Group draft backed by Chrome and Edge, with native browser support expected later in 2026.
A2A — Agent2Agent Protocol
Let agents built by different vendors discover each other and coordinate tasks directly — a different layer from MCP/WebMCP’s agent-to-tool model. Originated at Google, now governed by the Linux Foundation.
OpenAPI
Provide machine-readable specifications describing API endpoints, parameters, requests, and responses.
JSON Schema
Define structured input and output formats for machine interfaces.
JSON-LD Actions
Describe certain available actions through structured website data where appropriate.
Cloudflare Workers
Create lightweight API endpoints or wrappers around existing systems at the network edge.
Google Apps Script
Expose simple Google-centric workflows and business operations using services such as Google Sheets, Calendar, and Gmail.
SaaS & Business-System APIs
Connect agent actions directly to existing booking, CRM, commerce, payment, support, or other operational platforms.
Engineering patterns: callable actions · machine-readable parameters · input validation · structured JSON responses · machine-readable errors · HTTP status codes · explicit confirmation steps · request IDs · safe retries
Rules & Automation
Execute the business rules, calculations, decisions, and workflows behind an agent request.
An agent interface may be simple, while the operation behind it needs to determine eligibility, calculate a price, select a product, coordinate multiple applications, wait for an external process, or recover when something fails.
Cloudflare Workers
Run lightweight serverless business logic close to the website and expose it through HTTP APIs.
Google Apps Script
Implement simple business logic and automations around Google Workspace applications.
Supabase Edge Functions
Run serverless application logic alongside Supabase data and authentication.
Make
Build visual workflows connecting multiple SaaS applications and services.
Pipedream
Create API-driven and event-driven workflows connecting external systems.
Zapier
Automate business processes across commonly used SaaS applications.
Cloudflare Workflows
Orchestrate longer-running or multi-step serverless processes.
Queues & Asynchronous Jobs
Handle operations that should not remain inside a single synchronous web request.
Engineering patterns: deterministic business rules · validation · idempotency · retries · workflow orchestration · asynchronous processing · rate limiting · operation status · partial-failure handling · compensation and rollback
Where a deterministic rule can solve the problem reliably, we prefer deterministic software rather than introducing an LLM unnecessarily.
Data & Business Systems
Connect AI-agent interactions to the systems where business data, customers, bookings, transactions, and operational state actually live.
Data Platforms
Google Sheets & Google Apps Script
A lightweight option for simple, low-volume workflows where business users need direct access to the underlying data.
Cloudflare D1
Serverless SQL storage designed to integrate closely with Cloudflare Workers.
Supabase & PostgreSQL
Relational data infrastructure suitable for more complex applications, authentication, and structured business data.
Firebase & Firestore
Serverless document-oriented data storage with strong integration into the Google ecosystem.
Airtable
Business-user-friendly structured storage suitable for lightweight applications and operational workflows.
Business Systems
CRM Platforms
Store leads, customer interactions, qualification outcomes, and sales activity in the existing system of record.
Booking & Reservation Systems
Expose availability and create or manage reservations through existing booking infrastructure.
Commerce & Inventory Systems
Use authoritative product, availability, pricing, order, and inventory systems rather than creating parallel state.
Google Calendar
Read availability or create calendar events through lightweight workflows and integrations.
Notifications
Support transactional notifications, confirmations, approvals, and other human steps within an agent-driven workflow:
- Google Apps Script MailApp
- Postmark
- Resend
- SendGrid
- Amazon SES
Secrets & Credentials
Keep credentials available to the runtime without exposing them to visitors, agents, client-side code, logs, or source repositories:
- Cloudflare Worker Secrets
- Google Apps Script Properties
- Google Secret Manager
- Supabase Secrets
- Platform-managed environment variables and credential stores
Engineering patterns: transactions · concurrency control · state management · authoritative systems of record · human approval · notifications · secret management · retention controls · client-owned infrastructure
Monetization
Control the economic relationship between AI agents and valuable digital content, data, APIs, tools, and transactions.
AI agents may increasingly consume information or complete tasks without creating the page views, clicks, or conventional customer journeys on which many digital business models depend.
The appropriate response may be free access, restricted access, licensing, subscription entitlement, pay-per-use access, affiliate or transaction revenue, or a new machine-facing product.
RSL — Really Simple Licensing
Express machine-readable licensing, usage, and compensation terms for digital assets consumed by AI systems and automated clients.
HTTP 402 — Payment Required
An HTTP mechanism increasingly used to signal that a machine-accessible resource requires payment before access can continue.
x402
Support programmatic payment for APIs, content, or callable tools using an HTTP 402-based machine-payment flow.
MPP — Machine Payments Protocol
An emerging machine-payment protocol, formally proposed to the IETF, that uses HTTP 402 to support programmatic payments between agents and digital services across cards and stablecoins.
Cloudflare Pay Per Crawl
An AI Crawl Control capability for gating or charging AI crawler access to content over HTTP 402. The model is shifting from charging per fetch toward compensating publishers when their content is actually cited in an AI answer — an outcome-based direction still being piloted.
Stripe Billing & Payments
Support conventional subscriptions, payment processing, usage billing, and customer payment relationships.
UCP — Universal Commerce Protocol
An open standard for agentic commerce connecting businesses, platforms, AI agents, and payment providers across discovery, checkout, and related commerce flows.
AP2 — Agent Payments Protocol
An open protocol using cryptographically signed “mandates” to authorize agent-initiated payments across cards and stablecoins. Backed by a broad payments-industry coalition and governed through the FIDO Alliance.
ACP — Agentic Commerce Protocol
An open specification for programmatic commerce between buyers, AI agents, and merchants, including agent-driven checkout.
Paid APIs & MCP Tools
Package valuable information, calculations, data, or capabilities as directly consumable machine products.
Engineering patterns: machine-readable licensing · free / restricted / licensed / paid access · subscription entitlement · pay-per-request · usage metering · dynamic pricing · payment verification · affiliate / CPA tracking · transaction fees · revenue sharing · commercial attribution
Emerging agentic-payment, licensing, and commerce standards are evaluated based on ecosystem adoption and client requirements rather than introduced by default.
Monitoring & Accountability
Understand which agents are interacting with your digital assets, what they are doing, whether actions succeed, and what commercial outcomes those interactions produce.
CDN & Edge Logs
Capture requests before they reach the application and help distinguish different forms of automated traffic.
Cloudflare Bot & Crawl Analytics
Analyze crawler and automated-agent activity at the edge.
Google Sheets
Provide lightweight, human-readable operational and audit logs for simple workflows.
The same data stores described under Data & Business Systems — Cloudflare D1, Supabase/PostgreSQL, Firebase/Firestore, and Airtable — can double as audit trails, persisting interaction histories, request results, state transitions, and application events.
CRM, Booking & Commerce Histories
Use existing business systems as the authoritative record of leads, bookings, orders, and transactions.
Workflow Execution Histories
Use Make, Pipedream, Zapier, Cloudflare Workflows, and similar platforms to diagnose multi-step automation execution.
Payment & Settlement Records
Connect paid machine usage to actual commercial events.
Engineering patterns: agent attribution · request IDs · transaction IDs · external reference IDs · structured logs · state transitions · server-side event logging · usage measurement · conversion attribution · revenue attribution · failure monitoring · audit trails · privacy-aware logging
Technology Follows the Business Problem
Making a website ready for AI agents does not mean adding every available protocol or platform.
For some businesses, the highest-value improvement may simply be clearer HTML and structured information. Others may need live APIs, callable actions, business-system integration, authentication, or an agent-specific experience. Businesses with valuable digital content or data may also need to consider licensing, machine payments, or new agent-facing products.
The objective is to select the smallest practical technology set required to make the customer journey work reliably for both humans and AI agents.