AI Readiness

This page documents how we make our content accessible to AI systems: our bot access policy, structured discovery endpoints, and live monitoring data.

How AI systems discover our content

AI crawlers and LLM training pipelines discover PredictionTalk through multiple layers:

1. robots.txt

Our /robots.txt explicitly allows known AI crawlers to index the forum. We distinguish between crawlers we welcome (content indexers) and those we restrict (training scrapers).

2. llms.txt standard

We implement the emerging llms.txt standard — a plain-text file that tells AI assistants what our site is about, what content is available, and how to navigate it.

• /llms.txt — summary for AI assistants
• /llms-full.txt — full content index for LLM training

3. .well-known endpoints

Machine-readable metadata in standardized locations:

• /.well-known/ai-plugin.json — plugin manifest for AI tool use
• /.well-known/llms.txt — alternative discovery path

4. Structured data (Schema.org)

Every page includes JSON-LD structured data so AI systems can understand content type, author, date, and topic without reading the full HTML.

5. JSON API

Clean JSON endpoints for AI-friendly data consumption without HTML parsing overhead.

Bot access policy

We welcome AI crawlers that respect our content and contribute to knowledge discovery. Here is our policy for known bots:

llms.txt

We follow the llms.txt specification to help AI assistants understand our site structure and content.

What's in our llms.txt

• Site description — what PredictionTalk is and who it's for
• Key URLs — forum sections, API endpoints, important pages
• Content policy — licensing, attribution requirements
• Topics covered — prediction markets, probability, trading strategies

# Example from /llms.txt
# PredictionTalk — The community for prediction market traders
# https://predictiontalk.org
#
# PredictionTalk is a forum for traders on Polymarket, Kalshi,
# Manifold, and other prediction market platforms.
# ...

View full file: /llms.txt · /llms-full.txt

JSON API for AI consumption

Clean JSON endpoints that AI systems can query without HTML parsing:

Example response

{
  "site": "PredictionTalk",
  "url": "https://predictiontalk.org",
  "description": "Forum for prediction market traders",
  "topics": ["Polymarket", "Kalshi", "probability", "strategy"],
  "recent_discussions": [
    {
      "id": 123,
      "title": "Best strategies for binary options on Polymarket",
      "url": "https://predictiontalk.org/d/123",
      "tags": ["Strategy", "Polymarket"],
      "created_at": "2026-02-20T14:30:00Z"
    }
  ]
}

Structured data (Schema.org)

Every page on PredictionTalk includes JSON-LD structured data following Schema.org vocabulary, allowing AI systems and search engines to understand content without parsing HTML.

Implemented schemas

AI bot traffic (live monitoring)

We track all known AI crawler activity from nginx access logs, updated weekly every Monday.

What we monitor

• Total hits per bot — how often each AI crawler visits
• HTTP status distribution — 2xx (success), 3xx (redirect), 4xx (error)
• robots.txt compliance — are bots checking before crawling?
• Top crawled pages — which content AI bots find most interesting
• Week-over-week trends — growth or decline in AI visibility

AI readiness coverage

Status of all AI readiness features: