When I wired this site into the Fediverse, I made a commitment to open syndication. ActivityPub. RSS. Atom. JSON Feed. The works. That choice created an engineering problem I hadn't fully anticipated.
Bots are relentless.
Feed readers, aggregators, and crawlers poll syndication endpoints on aggressive schedules. Every poll hits the feed route. If that route queries a database, assembles XML, and returns it on every request, you are burning read capacity on string concatenation for machines that do not care about your CSS.
The web platform is also changing how it handles XML. Chrome is actively deprecating native XSLT support due to legacy codebase security concerns (detailed in WHATWG issue 11523 and their migration docs). This kills natively styled RSS feeds in the browser. Developers like Justin Jackson correctly point out that this creates real friction for independent creators who rely on readable feeds for audience growth.
My approach sidesteps XSLT entirely. Human readers get redirected to a dedicated HTML landing page (like my What is a Web Feed explainer). Bots and aggregators get the raw, unstyled XML they actually need.
The Architectural Problem
Serving raw XML sounds simple. It is not.
Feed readers poll aggressively. Some hit your endpoint every fifteen minutes. If you build a dynamic route that queries D1 or Postgres on every request, you are spending database reads on string concatenation for content that changes once a week. Maybe less.
Standard HTTP caching with Cache-Control: s-maxage handles this well for most teams. Cache the response at the CDN layer, let it expire on a timer. No complaints from me if that is your approach. It works.
I went a different direction because I wanted to eliminate the "thundering herd" entirely. When a CDN cache expires, the next request triggers a fresh database query. If fifty feed readers happen to poll in the same second after expiry, that is fifty identical queries racing to rebuild the same XML string. Not catastrophic. But avoidable.
The KV Fan-Out Alternative
The pattern I landed on uses eventual consistency. The database serves strictly as the source of truth for writes. Reads never touch it.
When a post is published, a background worker compiles the Atom, RSS, and JSON Feed payloads into finished strings. It pushes those strings directly into Cloudflare KV. Done.
The edge route does one thing: fetch a pre-compiled string from KV and return it. One key-value lookup. No SQL. No XML assembly. Single-digit millisecond response times regardless of how many bots are polling simultaneously.
You pay the compute cost once on publish. Every subsequent read is a key lookup that scales to any volume of concurrent readers without touching the database.
The Implementation
Below is a simplified, framework-agnostic implementation of this fan-out pattern using Cloudflare D1 and KV. The full, production-ready version is available as a three-file GitHub Gist.
1. The Schema
A flag in the database marks what gets syndicated.
-- D1 Schema for Syndicatable Nodes
CREATE TABLE IF NOT EXISTS nodes (
id TEXT PRIMARY KEY,
tenant_id TEXT NOT NULL,
slug TEXT NOT NULL,
title TEXT NOT NULL,
content TEXT,
include_in_feed INTEGER DEFAULT 0,
published_at DATETIME DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX idx_nodes_feed
ON nodes(tenant_id, include_in_feed, published_at DESC);
2. The Compiler Worker
This runs on a cron schedule or a publish webhook. It reads the database once, builds the XML, and writes the final string to KV.
export interface Env {
DB: D1Database;
CONTEXT_KV: KVNamespace;
}
export async function compileFeedToKV(
tenantId: string,
env: Env
) {
const { results } = await env.DB.prepare(
`SELECT id, slug, title, content, published_at
FROM nodes
WHERE tenant_id = ? AND include_in_feed = 1
ORDER BY published_at DESC LIMIT 50`
).bind(tenantId).all();
if (!results || results.length === 0) return;
const buildDate = new Date().toUTCString();
const itemsXml = results.map((item: any) => `
<item>
<title><![CDATA[${item.title}]]></title>
<link>https://example.com/post/${item.slug}</link>
<guid isPermaLink="false">${item.id}</guid>
<pubDate>${new Date(item.published_at).toUTCString()}</pubDate>
<description><![CDATA[${item.content}]]></description>
</item>
`).join('');
const rssXml = `<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0">
<channel>
<title>My Syndicated Feed</title>
<link>https://example.com</link>
<description>Latest updates</description>
<lastBuildDate>${buildDate}</lastBuildDate>
${itemsXml}
</channel>
</rss>`;
await env.CONTEXT_KV.put(
`syndication:${tenantId}:rss`,
rssXml,
{ metadata: { compiledAt: Date.now(), itemCount: results.length } }
);
}
3. The Edge Route
The API endpoint is completely decoupled from the database. It does not know SQL exists.
import type { APIRoute } from 'astro';
import { getCFEnvAsync } from '@pobal-os/platform/cf-env';
export const GET: APIRoute = async () => {
const env = await getCFEnvAsync<{ CONTEXT_KV?: KVNamespace }>();
const kv = env?.CONTEXT_KV;
const tenantId = 'my-tenant-id';
if (!kv) {
return new Response('Feed store unavailable.', { status: 503 });
}
const feedXml = await kv.get(`syndication:${tenantId}:rss`);
if (!feedXml) {
return new Response('Feed not yet compiled.', { status: 404 });
}
return new Response(feedXml, {
status: 200,
headers: {
'Content-Type': 'application/xml; charset=utf-8',
'Cache-Control': 'public, max-age=60',
},
});
};
Trade-offs
This is not a universal solution. It is a trade-off.
You gain zero database load on reads, predictable latency, and immunity to polling storms. You lose real-time freshness. There is a window between publish and compile where the feed is stale. For my use case, that window is seconds. For high-frequency publishing, it could matter.
Standard s-maxage caching is simpler, requires less infrastructure, and works well for most sites. If your feed generates fewer than a hundred requests per hour, the complexity of a KV fan-out is probably not justified. Pick the approach that fits your traffic pattern.
The full three-file implementation is available as a GitHub Gist at the end of this post.
