lock <- renv::lockfile_read()
lock$R$Version
#> [1] "4.6.1"
length(lock$Packages)
#> [1] 109Appendix C — How this book is built
The good-practice page near the end of the book (Chapter 13) makes a case for projects, relative paths, a locked set of package versions and a build anyone else can reproduce. The book itself follows the same arrangement, and this appendix describes how it is put together. Where the arrangement is awkward, or where a choice is a judgement rather than a rule, that is stated explicitly.
None of this is needed to work through the book; it is needed to rebuild the book, to fork it for another course, or to work out why a render failed.
C.1 The repository
r-intro/
├── _quarto.yml project and book configuration
├── _common.R sourced by every chapter, in a hidden chunk
├── _dependencies.R never sourced, never rendered; see below
├── .Rprofile one line, and renv wrote it
├── .gitignore short, and one line of it is load-bearing
├── .gitattributes keeps generated files out of diffs
├── LICENSE MIT
├── renv.lock exact versions of every package
├── renv/ activate.R, settings.json, and the library itself
├── index.qmd the front page
├── chapters/ the 14 chapters, one file each
│ └── setup.qmd … the first of them
├── appendices/ the 4 appendices, one file each
│ ├── solutions.qmd
│ ├── data.qmd
│ ├── build.qmd this page
│ └── colophon.qmd
├── R/nus_theme.R the ggplot2 theme and the colour palette
├── scss/nus.scss the site styling, layered on bootswatch litera
├── scripts/check_palette.R the palette's accessibility tests
├── scripts/publish.sh render, then push the site to gh-pages
├── scripts/stamp-build.sh records which commit produced the site
├── scripts/make-redirects.sh redirect stubs at the old flat page URLs
├── scripts/fix-footer-paths.sh footer logo paths, per page depth
├── scripts/check-pages-sync.sh reports whether the live site matches main
├── scripts/hooks/pre-push reminds you that pushing is not publishing
├── assets/ favicon, and the three footer logos
├── data/sg_covid_data.csv the dataset, all 23 kB of it
└── _freeze/ cached chunk output, committed on purposeThe rendered site is not in that list. Its absence is deliberate, and the publishing section below explains it. _freeze/ is generated and in version control anyway, which is unusual enough that it gets a section of its own too.
One file per chapter, and the chapter list in _quarto.yml is the only place the reading order is written down. Renaming a file means editing _quarto.yml, and forgetting to means the chapter silently disappears from the book while still sitting in the repository — an easy way to lose a page.
C.2 _quarto.yml, section by section
The whole build is one YAML file. It opens with a warning to whoever edits it next:
# Verified against Quarto 1.10.18. The project: and book: schemas are closed,
# so an unrecognised key there is a hard render failure — check
# /Applications/quarto/share/schema/definitions.yml before adding one.Under project: and book:, Quarto validates keys against a fixed schema. A typo is not ignored and does not produce a warning; the render stops. The schema files shipped with your Quarto installation are the authority, and they describe the version you are running.
C.2.1 project:
project:
type: book
execute-dir: project
post-render:
- scripts/stamp-build.shtype: book is what turns a folder of .qmd files into a numbered book with parts, appendices, previous and next links, and a single search index.
There is no output-dir, which is a decision rather than an oversight. Quarto therefore renders into its default _book/, and _book/ is gitignored, because the rendered site is not kept on this branch at all. What happens to it instead is the publishing section.
execute-dir: project sets R’s working directory to the project root for every chunk in every chapter. Without it, each chapter would execute in its own directory — chapters/ or appendices/ here — and the hidden setup chunk’s source("_common.R") would not find its file. Setting it means here::here("data", "sg_covid_data.csv") resolves identically from every chapter, including this one.
post-render: runs scripts/stamp-build.sh after the last page is written. It drops a small build-info.json into the output recording which commit the site was rendered from. That file has a job, and the job only makes sense alongside the rest of the publishing arrangement, so it is explained there too.
C.2.2 book:
Metadata first: title, subtitle, author, date: today with an explicit date-format, and a description that becomes the page’s meta description and the social preview text.
site-url and repo-url together are what make the “Edit this page”, “Report an issue” and “View source” links work. repo-actions: [edit, issue, source] turns all three on. They appear in the right-hand margin of every page, and they cost nothing to maintain. A student who spots a wrong number is two clicks from reporting it, and one of those clicks lands them in a text editor with the offending line in front of them.
repo-branch: main matters because Quarto builds the edit URL from it. Point it at a branch that does not exist and every edit link 404s, on every page, silently.
Then the reading conveniences: search: true, reader-mode: true (the button that collapses both sidebars), page-navigation: true and back-to-top-navigation: true.
The sidebar is docked rather than floating, collapsed to one level so the parts are visible but the sections are not. repo-url puts Quarto’s built-in source link in it, and repo-actions adds the per-page edit, issue and source links in the right margin; the one custom tool is a direct download link for the CSV, pointing at raw.githubusercontent.com rather than the repository page so it downloads the file instead of showing a preview of it.
C.2.3 chapters: and appendices:
chapters:
- index.qmd
- part: "Before you write any code"
chapters:
- chapters/setup.qmd
# ... parts II to V ...
- part: "Putting it together"
chapters:
- chapters/capstone.qmd
- chapters/good-practice.qmd
- chapters/next-steps.qmdThe parts are headings in the sidebar and nothing else; they carry no files of their own. index.qmd sits outside any part because a book’s front page is not a chapter.
appendices: is a separate list, and Quarto numbers it separately: chapters get 1, 2, 3, appendices get A, B, C, D. The crossref: appendix-title: "Appendix" setting at the bottom of the file is what makes a reference to one of them read “Appendix B” rather than just “B”.
C.2.5 format:
format:
html:
theme: [litera, scss/nus.scss]Bootswatch litera underneath, the project’s own SCSS on top. litera was chosen for a reason that has nothing to do with taste: it is one of the few bundled themes that does not fetch a web font from fonts.googleapis.com. The book therefore renders the same offline as it does on a fast connection, which matters for a course taught on university wifi.
The rest of the HTML block, in the order it appears:
toc: true with toc-depth: 3 and a custom toc-title. number-sections: true, so sections can be referenced by number. anchor-sections: true gives every heading a link target, which is what lets you point a student at one paragraph rather than a chapter. smooth-scroll: true is cosmetic.
code-copy: true puts a copy button on every code block. code-overflow: scroll gives a long line its own horizontal scrollbar rather than wrapping it, which is the less obvious choice of the two. This dataset has column names 49 characters long — retail_and_recreation_percent_change_from_baseline is one — and wrapping breaks them mid-identifier, so the name a student is meant to read and retype arrives in two halves.
code-link: true turns every function call in every code block into a link to that function’s documentation, so filter() in a chunk is clickable and lands on the dplyr reference page. It needs downlit and xml2 installed. Neither is called by name anywhere in the book, so both are in the lockfile only because _dependencies.R names them, which is a trap the renv section comes back to.
code-annotations: hover is how the numbered callouts in code blocks behave.
df-print: default is deliberate. Set it to kable and every data frame in the book renders as a tidy HTML table with proper borders, which looks better but looks nothing like what the student sees in their own console. A beginner comparing the book against their screen would then have to work out which difference matters, so the book’s data frames print exactly the way the console prints them.
Figures: fig-width: 7, fig-height: 4.2, fig-align: center, fig-format: svg. SVG because plot text stays sharp at any zoom and the files are usually smaller than PNG for this kind of chart. tbl-cap-location: bottom follows the usual convention of captions below tables and above figures.
C.2.6 execute: and why chapters should not override it
execute:
freeze: auto
echo: true
warning: true
message: trueWarnings and messages are on, globally and on purpose. read_csv()’s column specification, geom_histogram()’s note about picking 30 bins, and summarise()’s .groups notice are all messages, and the book teaches students to read all three. A chapter that sets message: false to tidy its output removes material the text refers to.
Because these are set globally, chapters should not set them again. Repeating #| echo: true on a chunk is harmless but noisy; repeating it inconsistently is how a chapter ends up with three different behaviours for no reason anybody can reconstruct later. The same goes for comment, collapse, and the figure dimensions, all of which are set once in the knitr: block at the bottom:
knitr:
opts_chunk:
comment: "#>"
collapse: true
dev: "svg"comment: "#>" prefixes every line of output. It means you can select a whole block, code and output together, paste it into your console, and have it run: the output lines are comments. collapse: true puts code and its output in one block rather than two.
Two per-chunk options are used, and only these two. #| eval: false for anything that must not run during a render, which means install.packages(), renv:: calls, setwd() demonstrations, View(). #| error: true for chunks that are supposed to fail, where the error message is what the reader is meant to see.
C.3 _common.R
Every chapter opens with the same four lines:
```{r}
#| label: setup
#| include: false
source("_common.R")
```include: false hides the chunk and its output. The reader never sees it.
There is a reason for the repetition. Quarto’s freeze machinery gives each chapter its own R session, which is what makes a single chapter re-renderable without re-running the whole book. A variable created in chapter 4 does not exist in chapter 5. So anything more than one chapter needs has to be defined in a file that every chapter sources, and _common.R is that file. It attaches the packages, sets the ggplot2 theme, defines read_covid() and covid_daily(), and runs the guard described in the next section.
The leading underscore is a Quarto convention: files and directories starting with _ are not treated as book content and are not copied to the output.
C.4 renv and the lockfile
renv records the exact version of every package the book was built with. It does this in renv.lock, a JSON file with one entry per package:
"here": {
"Package": "here",
"Version": "1.0.2",
"Source": "Repository",
...
"Repository": "CRAN"
},(Trimmed. Each real entry also carries the package’s title, licence, dependencies and maintainer.)
The lockfile is a manifest, not a copy of the packages. It is a few hundred kB of text that says which versions to go and get.
109 packages, for a book whose _common.R calls library() four times. Almost all of them are dependencies of dependencies, which is the honest reason to use a lockfile at all: that graph is far too large to keep track of by hand, and “install the tidyverse” resolves to a different set of versions every month.
The three commands you need:
renv::restore() reads the lockfile and installs exactly those versions into renv/library. This is the first thing to run in a fresh clone. It takes a few minutes once.
renv::status() compares what is installed against what is recorded, and tells you which way they disagree.
renv::snapshot() writes the current state back into the lockfile. Run it after adding a package, and commit the resulting diff along with the change that needed it.
The library itself is not in version control. renv writes and maintains renv/.gitignore for you:
library/
local/
cellar/
lock/
python/
sandbox/
staging/renv.lock, renv/activate.R and renv/settings.json are tracked. Those three plus the ignore rules are what makes a clean checkout reproducible without committing several hundred megabytes of compiled code.
.Rprofile is one line, and renv wrote it:
source("renv/activate.R")R runs .Rprofile from the working directory at startup. That line activates the project library, and it is the whole mechanism.
C.4.1 _dependencies.R
renv finds dependencies by scanning source files for library() calls. Its scanner automatically adds rmarkdown when it sees .Rmd files, but not for .qmd. This book has no .Rmd files, so without intervention renv::snapshot() leaves knitr and rmarkdown out of the lockfile entirely, and a clean checkout fails to render with an error that points nowhere near the cause.
_dependencies.R is a file that exists only to be scanned. It is never sourced and never rendered. It lists the packages the chapters use plus the ones the toolchain needs but nothing calls by name: knitr, rmarkdown, downlit, xml2, sessioninfo. If you add a package to a chapter, add it there too, or it will work on your machine and nowhere else.
C.4.2 Rendering from another project’s directory
Quarto decides which renv to activate by reading .Rprofile relative to the directory you invoked it from, not relative to the project you asked it to render. So this is fine:
cd ~/r-intro
quarto renderand this is not:
cd ~/sph5421-course # another R project, with its own .Rprofile and its own renv
quarto render ../r-introThe second one starts R in ~/sph5421-course, reads that project’s .Rprofile, activates that project’s library, and then renders this book against whatever versions happen to be sitting there. execute-dir: project moves the working directory afterwards, so here() and every relative path still work, and the render completes.
Nothing fails visibly: the versions in a neighbouring project are usually close enough that most of the book builds, and the problem surfaces three chapters later when a dplyr deprecation warning appears on a page that never had one.
So _common.R checks, before it does anything else:
local({
proj <- getwd()
while (!file.exists(file.path(proj, "_quarto.yml")) && dirname(proj) != proj) {
proj <- dirname(proj)
}
expected <- normalizePath(file.path(proj, "renv", "library"), mustWork = FALSE)
active <- normalizePath(.libPaths()[1], mustWork = FALSE)
if (!startsWith(active, expected)) {
stop(
"Wrong package library.\n",
" active: ", active, "\n",
" expected: ", expected, "/...\n",
"Run `quarto render` from the project root (", proj, ").",
call. = FALSE
)
}
})It walks up from the working directory until it finds _quarto.yml, works out where that project’s library ought to be, and compares it against the library R is actually using. When they disagree the render stops on the first chapter with:
Error: Wrong package library.
active: /Users/you/sph5421-course/renv/library/macos/R-4.6/aarch64-apple-darwin25.4.0
expected: /Users/you/r-intro/renv/library/...
Run `quarto render` from the project root (/Users/you/r-intro).The paths in the message will be your own, and the fix is always the same: cd into the project root and render from there.
Here is the library this page was built against:
.libPaths()[1]
#> [1] "/Users/smishra/Documents/GitHub/r-intro/renv/library/macos/R-4.4/aarch64-apple-darwin20"If that path does not contain r-intro/renv, the guard has a bug; please open an issue for that as well.
C.5 Freeze
freeze: auto means a chapter’s R code is executed when the chapter’s .qmd changes, and not otherwise. The output of every chunk is cached under _freeze/, and _freeze/ is committed.
The payoff is that anyone can clone the repository and rebuild the site without R, without renv, and without the several minutes of computation. It also means a one-word fix to chapter 9 does not re-execute chapters 1 through 8, which turns a two-minute render into a five-second one and makes previewing a small change practical.
The cost is a failure mode. Freeze watches the .qmd file, not the things it depends on. Edit _common.R, change the colour palette in R/nus_theme.R, or replace the CSV, and nothing re-runs. The book will render happily and every page will still show the old output. There is no warning, because from Quarto’s point of view nothing changed.
There is no command-line flag to force it. When you have changed something shared, delete the cache:
rm -rf _freeze/
quarto renderor delete one chapter’s directory under _freeze/chapters/ if you know that only one is affected. Doing this after any change to _common.R is a habit to build, and the reason _freeze/ is listed in .gitattributes as generated is that the resulting diff is enormous and carries no information anyone needs to read.
C.6 Publishing
main holds the source and nothing else. The rendered site lives on a branch of its own, gh-pages, which carries the built HTML and nothing else, and that is the branch GitHub Pages is configured to serve. Rendering locally puts the book in _book/, which is gitignored, so you can build and reread it as often as you like without git ever noticing.
Publishing is one command:
./scripts/publish.shIt renders the book and hands the result to quarto publish gh-pages --no-prompt, which pushes the output onto gh-pages. Then it checks what actually went out: that the expected number of pages exist, counting recursively because pages now render into _book/chapters/ and _book/appendices/ as well as the top level; that no page got stranded next to its source, which a failed render can now leave anywhere in the tree; that search.json is present, since a missing search index makes the search box return nothing rather than fail visibly; and that every solutions.html#sec-sol-… link in the book has an anchor waiting for it in appendices/solutions.html, with the back-links from each solution to its exercise checked the same way. A failed render announces itself; a dead cross-link does not, and this book cross-links every exercise to its worked answer, so the script checks them.
It refuses to start in three cases. If a quarto preview is running, because the preview watches the same files, races the render, and has left the output half-moved. If you are not on main. And if the working tree is dirty, which is the interesting one and needs the next section first.
C.6.1 What the split buys, and what it costs
The reason to keep the site off main is diffs: generated HTML is not source, and a re-render rewrites every page, so fixing two words in one chapter arrives as a commit touching ninety files, and git log on that chapter becomes something you scroll past rather than read. The problem stops being cosmetic as soon as a second person edits chapters, because HTML merges badly: two people who touched different chapters still both regenerated the whole site, and git offers you conflicts in dozens of files that cannot be resolved by reading them. With the output on its own branch, a chapter edit is a one-file diff, and a merge is a merge of prose.
The cost is real: the previous arrangement was better at one thing. When the site was committed alongside the source, forgetting to rebuild was impossible to miss: you edited a chapter and the working tree stayed dirty until you rendered, so staleness announced itself for free.
Split source from output and that goes away. main is clean whether or not you have ever published. You can edit a chapter, commit, push, watch GitHub show your change, and have the live site still serving last month’s text with nothing anywhere indicating it. The three small scripts below exist to replace the visibility the split removed.
scripts/stamp-build.sh runs as the post-render step and writes build-info.json into the output: the commit HEAD was at, whether the tree was dirty at the time, the build timestamp, and the Quarto version. Rendered HTML has no record of which revision of the source produced it, and this file supplies it.
A second post-render step, scripts/make-redirects.sh, writes a small stub page at each of the book’s old flat URLs, forwarding to the page’s current address under chapters/ or appendices/. Moving the sources into subdirectories changed every page’s URL, and the old addresses persist in slides and bookmarks. GitHub Pages has no server-side redirect mechanism, so a static stub at each old path is the only form the fix can take. The stubs are regenerated on every render, and the page-count check in publish.sh counts only the real pages.
scripts/check-pages-sync.sh reads that file off gh-pages and compares the commit in it against the tip of main. It exits 0 in sync, 1 when the site is stale, and 2 when it cannot tell — no gh-pages branch yet, no build-info.json in it, no network. --local skips the fetch and works with the refs you already have. publish.sh runs it at the end as its own last check.
One detail of that script deserves copying. It does not compare commit counts; it compares the files that changed since the publish against a list of things the rendered site is actually built from — the .qmd files, _quarto.yml, _common.R, R/, scss/, data/, assets/ — and if the only commits since were a README edit or a licence tweak, the site is still correct and it says so. A check that reported “stale” every time you fixed a typo in a comment would be ignored within a week, and an ignored check still leaves you believing something is being watched.
scripts/hooks/pre-push prints a reminder when the commits you are pushing touch anything on that list. It is not installed by default; enable it with:
git config core.hooksPath scripts/hooksIt prints the reminder and nothing more. It does not block the push, and that is deliberate: pushing source before publishing is a normal thing to do, and a hook that refused would be routed around with --no-verify the second time it got in the way.
That is where the dirty-tree rule in publish.sh comes back. Render with uncommitted changes and the stamp names a commit that does not contain what was rendered, so every check downstream is reasoning from a false statement — the sync check compares against source that was never published and reports agreement that is not there. stamp-build.sh does record "dirty": true when it happens, and the sync check treats that as stale outright rather than pretending, but refusing to publish from a dirty tree in the first place is cheaper than either.
C.6.2 Why there is no continuous integration
There is no GitHub Action here, and the reason has not changed. This book is built with Quarto 1.10.18, on one laptop, against a package library pinned by renv.lock. Any route that renders on a server adds a second Quarto at a second version, and a newer Quarto does not fail loudly on an older book. It produces slightly different HTML from the version the book was checked against, which is a category of bug you find weeks later on one page. Publishing from here keeps the render on the machine the versions are pinned to, and only the output travels.
C.6.3 .nojekyll, and why this repository does not have one
GitHub Pages runs Jekyll over your site by default, and Jekyll ignores every file and directory whose name begins with an underscore. Quarto’s HTML output is full of them. An empty .nojekyll at the root of the published site turns Jekyll off entirely, and without it the site deploys with no CSS at all, which looks exactly like a broken theme but is not one.
The published site has one. This repository does not, and there is nothing under project: resources: copying one into the output either, because Quarto writes that file itself during quarto publish and puts something in it: a deploy id, which it reads back from the live site afterwards to confirm that what is being served is the copy it just pushed. Ship your own .nojekyll and you overwrite that id, and the verification fails on a deploy that worked perfectly.
C.6.4 The *_files/ rule in .gitignore
# Chapters render to their source directory before Quarto moves them into
# _book, so a failed render can leave them behind. No leading slash: chapters
# and appendices now live in subdirectories, and _book/ is ignored separately,
# so unanchored is what catches the strays without touching the output.
*_files/Knitr writes a chapter’s figures into <chapter>_files/ next to the source, and Quarto moves them into the output directory at the end of the render. A render that dies partway through leaves those directories behind, and they are rubbish that should not be in git. Because chapters and appendices now live in subdirectories, a stray can sit at any depth — chapters/visualise_files/, not just ./visualise_files/ — so the pattern is written without a leading slash and matches at any depth.
That breadth is safe because _book/ is ignored on its own line above it: git does not descend into an ignored directory, so the pattern catches the strays without ever reaching the figure directories inside the rendered site. The distinction matters because the output still goes through git on its way out — that is what publishing to a branch is — and the failure mode from getting it wrong should be stated plainly: the book renders correctly on localhost, publishes, and the live site shows a broken-image icon in place of every plot, with nothing in the local build indicating the problem. When every chapter sat at the top level the same rule was written the other way around, as /*_files/ anchored to the repository root, because an unanchored pattern would also have matched _book/visualise_files/ along with the strays. The move into subdirectories flipped it.
.gitattributes finishes the job, marking the generated directory so it stays out of GitHub’s language statistics and out of diffs:
_freeze/** linguist-generated=true
_freeze/** -diffWithout it, GitHub reads the repository as mostly generated JSON and SVG rather than R, and re-running one chapter arrives as a pull request with thousands of changed lines that carry nothing to review.
C.7 Forking it
The licence permits forking the book; the rest of this section is the procedure.
git clone https://github.com/CERM-NUS/r-intro.git my-course
cd my-courseOpen r-intro.Rproj in RStudio. The .Rproj file keeps its old name after a clone, so rename it now if the old one will confuse you later. Then, in the R console:
renv::restore()Answer yes. The restore takes several minutes on a cold cache. When it finishes, from a terminal in that directory:
quarto previewThat builds the book and opens it in a browser with live reload: save a .qmd and the page updates. It is the right mode for writing. ./scripts/publish.sh is the one for shipping, and it will refuse to run while a preview is still going, so stop the preview first.
Four things to change before you publish your version:
_quarto.yml needs a new site-url, repo-url and probably repo-branch, or every edit link on your site will point at the original repository.
The logos in assets/logos/ have to go, or be replaced with your own. They are the trade marks of NUS, the Saw Swee Hock School of Public Health, CERM and the ML & GHN network, and the MIT licence on the text and code does not cover them. Remove the page-footer HTML block along with the files.
LICENSE should keep the original copyright line if you keep any of the original text, and add yours.
If you swap the dataset, _common.R is the only place that reads it, but data.qmd documents the columns of this file and will be wrong the moment you change it. Rewrite that appendix or delete it, since a reader who trusts it will be reading a description of a file that is no longer there.
Then commit your sources, push, and run ./scripts/publish.sh. That creates the gh-pages branch the first time it runs, and once it has, go to the repository’s Pages settings and set it to deploy from gh-pages. Until you do that last step the branch exists and nothing serves it.