Being found
Lesson 2 of 3.
Your reader rarely arrives at the home page. They arrive from a search engine, from a link a colleague pasted, from the field in your own header — or through an assistant that read your pages for them. Four doors, and this lesson opens all four.
The search inside the site
Every page carries a field in its header. It leads to /search/, a page the
build writes with the site: the list of every page, which a small script
filters as the reader types.
search: false removes the field and the page. Worth doing on a site of ten
pages, where the menu already shows everything.
The two files crawlers read
siteUrl: 'https://acme.example.com/docs',
That one field is what makes the rest possible — an address is what a crawler, a social network and a feed reader all need.
| File | Written when | What it holds |
|---|---|---|
sitemap.xml | siteUrl is set | Every page of every version, except a prerelease |
robots.txt | the site sits at the root of its domain | Where the sitemap is |
feed.xml | feed: true | The latest pages, announced in the head |
A version marked prerelease stays out of the sitemap and asks not to be
indexed. That is what stops a reader searching for your tool from landing on
the documentation of a version nobody can install yet.
A reader that is a program
llms: true,
Someone asks an assistant how to set up your tool. The assistant fetches your
pages, and what it gets is HTML: a header, a menu, a footer, and somewhere in
the middle the text it came for. llms: true hands it the text directly.
| File | Written where | What it holds |
|---|---|---|
llms.txt | root of the site | The project, then every page of the current version |
index.html.md | beside every page | The page as you wrote it, headed by its title and description |
The descriptions do the work here too. A page with no description is a bare
link in llms.txt, and the assistant has to open it to learn what it is about:
the field that writes the line under your search result writes this one.
It needs no siteUrl, and it stays off until you ask: whether programs should
read your pages is your call.
What a shared link looks like
A link pasted in a chat shows a title, a description and an image. All three come from what you already wrote:
Title and description from the frontmatter, image from socialImage.
The description of a page is not decoration: it is the sentence under the
title in a search result, and the one under the link in a chat. A page without
one is announced by its title alone.
socialImage is declared once, in the configuration, and needs siteUrl —
another machine fetches it, so the address has to be absolute.
Saying what a page is
Search engines read structured data. The build writes it from the frontmatter, and a page can say what kind of thing it is:
---
title: Install
description: One command sets up a project.
jsonld:
type: TechArticle
faq:
- question: Which Node version do I need?
answer: Version 22 or later.
---
The questions become a real FAQ block in the structured data — the kind a search engine can show folded under your result.
Weight, which nobody notices until it is bad
Three things happen at the build without being asked:
- Each image gets its dimensions, read from the file itself, so the page stops jumping while it loads.
- Every image but the first loads late, which is why the top of the page arrives first.
- The stylesheet is minified, and holds only the rules the pages use.
Nothing to configure. It is worth knowing because it explains a surprise: the first image of a page is deliberately not deferred, and that is not an oversight.
Tags, and what they are not
tags: [course, advanced, discovery]
They show under the page and travel into the structured data as keywords. There is no page per tag: they label, they do not navigate. A reader looking for everything on one subject uses the search field.
Check yourself
Your beta is in the sitemap. What did you forget?
prerelease: true on that version. Without it, nothing distinguishes it from
the version people should be reading.
No robots.txt was written and siteUrl is set. Why?
The site is served from a subfolder. A robots.txt only means something at the
root of a domain, so the build leaves it out rather than write a file that
would speak for addresses that are not yours.
An assistant quotes your page, but not the file its snippet shows. Why?
It read the Markdown copy, which holds the page as you wrote it: the Snippet
tag, not the file the build reads into it. Name the file and say what it does
in the text around the snippet — the reader of the HTML gains from it too.
The deployment guide covers the same ground from the host's side — what to serve, and what to check before it goes online.
DocPensieve