Skip to content
DocPensieve
Documentation menu

Being found

Module progress66%

Lesson 2 of 3.

Your reader rarely arrives at the home page. They arrive from a search engine, from a link a colleague pasted, from the field in your own header — or through an assistant that read your pages for them. Four doors, and this lesson opens all four.

The search inside the site

Every page carries a field in its header. It leads to /search/, a page the build writes with the site: the list of every page, which a small script filters as the reader types.

search: false removes the field and the page. Worth doing on a site of ten pages, where the menu already shows everything.

The two files crawlers read

siteUrl: 'https://acme.example.com/docs',

That one field is what makes the rest possible — an address is what a crawler, a social network and a feed reader all need.

FileWritten whenWhat it holds
sitemap.xmlsiteUrl is setEvery page of every version, except a prerelease
robots.txtthe site sits at the root of its domainWhere the sitemap is
feed.xmlfeed: trueThe latest pages, announced in the head

A version marked prerelease stays out of the sitemap and asks not to be indexed. That is what stops a reader searching for your tool from landing on the documentation of a version nobody can install yet.

A reader that is a program

llms: true,

Someone asks an assistant how to set up your tool. The assistant fetches your pages, and what it gets is HTML: a header, a menu, a footer, and somewhere in the middle the text it came for. llms: true hands it the text directly.

FileWritten whereWhat it holds
llms.txtroot of the siteThe project, then every page of the current version
index.html.mdbeside every pageThe page as you wrote it, headed by its title and description

The descriptions do the work here too. A page with no description is a bare link in llms.txt, and the assistant has to open it to learn what it is about: the field that writes the line under your search result writes this one.

It needs no siteUrl, and it stays off until you ask: whether programs should read your pages is your call.

A link pasted in a chat shows a title, a description and an image. All three come from what you already wrote:

The preview of a shared page
The preview

Title and description from the frontmatter, image from socialImage.

The description of a page is not decoration: it is the sentence under the title in a search result, and the one under the link in a chat. A page without one is announced by its title alone.

socialImage is declared once, in the configuration, and needs siteUrl — another machine fetches it, so the address has to be absolute.

Saying what a page is

Search engines read structured data. The build writes it from the frontmatter, and a page can say what kind of thing it is:

---
title: Install
description: One command sets up a project.
jsonld:
  type: TechArticle
  faq:
    - question: Which Node version do I need?
      answer: Version 22 or later.
---

The questions become a real FAQ block in the structured data — the kind a search engine can show folded under your result.

Weight, which nobody notices until it is bad

Scripts on a page
Stylesheets
100%
Images measured

Three things happen at the build without being asked:

  • Each image gets its dimensions, read from the file itself, so the page stops jumping while it loads.
  • Every image but the first loads late, which is why the top of the page arrives first.
  • The stylesheet is minified, and holds only the rules the pages use.

Nothing to configure. It is worth knowing because it explains a surprise: the first image of a page is deliberately not deferred, and that is not an oversight.

Tags, and what they are not

tags: [course, advanced, discovery]

They show under the page and travel into the structured data as keywords. There is no page per tag: they label, they do not navigate. A reader looking for everything on one subject uses the search field.

Check yourself

Your beta is in the sitemap. What did you forget?

prerelease: true on that version. Without it, nothing distinguishes it from the version people should be reading.

No robots.txt was written and siteUrl is set. Why?

The site is served from a subfolder. A robots.txt only means something at the root of a domain, so the build leaves it out rather than write a file that would speak for addresses that are not yours.

An assistant quotes your page, but not the file its snippet shows. Why?

It read the Markdown copy, which holds the page as you wrote it: the Snippet tag, not the file the build reads into it. Name the file and say what it does in the text around the snippet — the reader of the HTML gains from it too.

The deployment guide covers the same ground from the host's side — what to serve, and what to check before it goes online.