# Cloudflare Pages already keeps your preview deploys out of Google
I set up per-branch preview deployments on Cloudflare Pages and reached for an Astro noindex conditional to keep them out of search. I did not need it: Cloudflare already noindexes previews at the HTTP layer. A note on where infrastructure logic belongs.
- Author: Michael Mikus (https://mikus.io/about/)
- Published: 2026-08-06
- Category: Engineering
- Web page: https://mikus.io/blog/branch-previews-astro-cloudflare/
> The idea: Cloudflare Pages already adds a noindex header to every preview deployment, so you don't need application code to keep previews out of search. The wider lesson: don't solve an infrastructure problem inside your app.
I wanted to review every change to this site on a real URL before it went public. Not localhost. The actual rendered page, deployed, so I could catch the layout problems a dev server hides.
Cloudflare Pages gives you that: push a branch, get a preview deployment at its own URL. But those preview URLs are public, and my first thought was the obvious one. I don't want half-finished drafts turning up in Google. So I reached for an Astro conditional to stamp `noindex` on preview builds.
Then I checked whether I actually needed it. I didn't. Cloudflare already does it, one layer below my application, and the reason my clever conditional was pointless is the interesting part of this post.
## The setup: two systems, one job each
- **Cloudflare Pages** deploys. Production from `main`, a preview for the other branches it is configured to build. It deploys every branch that builds, good change or bad, because a preview is for looking at. It runs no checks beyond the build command.
- **GitHub Actions** is the quality gate. It runs format, type-check, and build on every pull request, and deploys nothing. It is what blocks a bad merge, once its job is a required status check in a branch protection rule or ruleset.
Connect the repo, set the build command (`pnpm build`) and output directory (`dist`), and choose the production branch (`main` here, but that is a project setting, not a rule). By default Cloudflare deploys your non-production branches as previews, and a pull request opened from the repository gets a preview URL posted on it. You can scope this to all branches, none, or a chosen set. Each preview gets an immutable hash URL and a per-branch alias.
That is the whole deploy configuration. Now the part I got wrong.
## The noindex I didn't need to write
Here is the code I was about to add to my base layout:
```astro
---
const isPreview =
process.env.CF_PAGES === '1' && process.env.CF_PAGES_BRANCH !== 'main';
---
{isPreview && }
```
It is not wrong. Cloudflare injects `CF_PAGES=1`, `CF_PAGES_BRANCH`, `CF_PAGES_URL`, and `CF_PAGES_COMMIT_SHA` into every Pages build, so at build time a static Astro site can tell it is a preview and emit the tag. The mechanism is real.
It is just unnecessary, because Cloudflare already handles this a layer down. From their docs:
> By default, every preview deployment generated by Cloudflare Pages includes the `X-Robots-Tag: noindex` HTTP response header.
You can watch it happen. Curl a preview URL and the production URL and compare headers:
```bash
$ curl -sI https://.my-project.pages.dev | grep -i robots
x-robots-tag: noindex
$ curl -sI https://my-project.pages.dev | grep -i robots
# (nothing)
```
Preview deployments carry `noindex`. Production does not. I had not written a line of code, and the exact behavior I wanted was already there.
## Why the header is the right place and my meta tag was the wrong one
There are two ways to tell a crawler to skip a page:
```html
```
```http
X-Robots-Tag: noindex
```
They mean the same thing to Google, but they live at different layers. The meta tag exists only inside HTML that you generated. The HTTP header applies to the response itself, every response, whatever your app rendered and whether or not it is even HTML.
That difference is the whole point. Cloudflare knows a deployment is a preview because _it_ created the preview. It does not need my application to re-derive that fact from an environment variable and act on it. The system that owns the fact applies the correct HTTP semantics, and my app never has to care.
That is the lesson worth keeping: **do not solve an infrastructure problem inside your application.** My `isPreview` check was pulling a fact that belongs to the deploy platform up into my page templates. It would only have covered HTML rendered through that layout, not images, PDFs, or any page that skips it. Cloudflare already answered the question, once, at the edge, for every response.
A static `_headers` file can also set `X-Robots-Tag` by hostname, using placeholders: `https://:project.pages.dev/*` for the production `pages.dev` address and `https://:version.:project.pages.dev/*` for previews, if you ever need indexing rules the defaults do not cover. For keeping previews out of search, you do not need it.
## The layers, in one picture
```text
git push / PR
│
┌────────────┴────────────┐
▼ ▼
GitHub Actions Cloudflare Pages
(quality gate) (deploy)
│ ┌────┴────┐
format / check / build ▼ ▼
│ main branch
▼ │ │
pass → merge ▼ ▼
production preview
│ │
indexable X-Robots-Tag:
noindex
```
Cloudflare deploys and decides indexing. GitHub Actions decides whether a commit is allowed to become production. Neither job belongs in my page templates.
## The one thing Cloudflare doesn't do for you
Cloudflare noindexes _previews_, but your _production_ `*.pages.dev` stays indexable. If you also serve the site on a custom domain, `my-project.pages.dev` and `mydomain.com` are now the same content at two addresses, which is a duplicate-content problem. The fix is not a preview conditional. It is a canonical URL on every page pointing at your real domain (this site sets one in its base layout), which search engines treat as a strong hint, or, to settle it, a Bulk Redirect from the `pages.dev` hostname to your domain. The `_redirects` file cannot match on hostname, so it cannot do this one.
And `noindex` is not access control. Anyone with a preview URL can still open it. If a draft must stay private, put the previews behind Cloudflare Access.
## The quality gate
The deploy side does none of the checking, so a separate workflow runs the same commands I run locally, and a red check blocks the merge once the job is marked as a required status check.
```yaml
name: CI
on:
push:
branches: [main]
pull_request:
concurrency:
group: ci-${{ github.ref }}
cancel-in-progress: true
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: pnpm/action-setup@v4
- uses: actions/setup-node@v4
with:
node-version: 22
cache: pnpm
- run: pnpm install --frozen-lockfile
- run: pnpm check-format # prettier --check .
- run: pnpm check # astro check
- run: pnpm build
```
`pnpm/action-setup` reads the pnpm version from the `packageManager` field in `package.json`; without that field, give it a `version`. `--frozen-lockfile` fails the install if the lockfile is out of date, so CI builds the exact tree I committed. The three checks are the same scripts in `package.json`, so `pnpm check-format && pnpm check && pnpm build` locally means green in CI. `cancel-in-progress` kills a superseded run, so a burst of commits only spends minutes on the last one.
## What I actually learned
I came in ready to write application code to keep previews out of Google, and the useful outcome was not writing it. Cloudflare deploys previews and marks them `noindex`. GitHub Actions gates the merge. The site sets a canonical URL. Three layers, each owning one job, and none of them my page templates.
The best infrastructure code is often the code you were about to write and then didn't have to.