Titus Coleman

Grove: a platform for self-hosted macOS GitHub Actions runners

How I built a platform that gives every macOS GitHub Actions job its own short-lived VM, with per-VM metrics and Packer image builds from a GUI.

The problem

macOS builds and tests need macOS, and macOS only runs on Apple hardware. GitHub offers hosted macOS runners, but teams often run their own Macs for more control over the hardware, the installed tools and the images builds run on.

Running your own brings its own problems:

  • State carries over between jobs. A runner that stays up keeps whatever the last job left behind: files, caches, keychain items, half-installed tools. Builds start to depend on that leftover state, and one job can see what another left. GitHub recommends ephemeral runners, which take one job and are then removed, for exactly this reason.
  • macOS VMs are scarce. Apple’s macOS license has long allowed only two macOS virtual machines per Mac, and Apple’s virtualization framework enforces it, so every host’s VM slots have to be scheduled with care.
  • Images drift. Machines set up by hand slowly stop matching each other, and rebuilding one means someone with shell access and time.

What did macOS CI look like at Vast before Grove? What broke, and who felt it?

What Grove does

Grove provisions an ephemeral VM per job on each macOS host, records per-VM time-series metrics, and drives Packer image builds from a GUI.

  • One VM per job. Each job gets a fresh VM started from the current image, and the VM is gone when the job ends, so nothing carries over to the next job.
  • Per-VM metrics. Recording each VM over time shows what every job actually used, which is what you need to size VMs and decide how many each host can run.
  • Image builds from a GUI. The image is defined in Packer, so it is rebuilt the same way every time, and engineers can start a build from the web app instead of logging in to a machine.
KubernetesHelmKubernetes, via HelmApplemacOS hostsjobVM per jobsign instart buildimagerecordsmetricsGitHub ActionsGitHub ActionsIcon-Resource/General-Resource/Res_User_48_LightGoogleEngineerGitHubRunner serviceTalks to the GitHub AppGrafanaGrafanaPer-VM metricsWeb appGo backend-for-frontendserving a Vite/TypeScriptfrontend over a REST APIGoogle Workspace sign-inmacOS hostVM for job 1VM for job 2Gone when the job endsPackerPackerVM image buildsIcon-Architecture/48/Arch_Amazon-Aurora_48Aurora PostgreSQLHosts, jobs, VMs, metricsJFrogArtifactory proxies the platform’s dependencies
KubernetesHelmKubernetes, via Helmjobsign inVM per jobmetricsbuildimagerecordsGitHub ActionsGitHub ActionsIcon-Resource/General-Resource/Res_User_48_LightGoogleEngineerGitHubRunner serviceGitHub AppWeb appGo + Vite/TSREST APIGoogle sign-inGrafanaGrafanaPer-VM metricsApplemacOS hostVM for job 1VM for job 2Gone when job endsPackerPackerImage buildsIcon-Architecture/48/Arch_Amazon-Aurora_48Aurora PostgreSQLHosts, jobs, VM lifecycle, metricsJFrogArtifactory proxies dependencies
Every macOS job gets its own short-lived VM, built from an image engineers can rebuild from the web app.

How it’s built

  • Data model: Aurora PostgreSQL tables for hosts, jobs, VM lifecycle and metrics. Why this shape?
  • API: the REST contract the frontend consumes.
  • Two containers, deployed separately: a Go backend-for-frontend serving a Vite/TypeScript frontend, and a runner service that talks to the GitHub App. Why split them?
  • Platform: Google Workspace OIDC sign-in, Artifactory as a dependency proxy, Helm deployment on Kubernetes.

Results

Only include outcomes you can state plainly.

Get in touch

Text me:

Open in new tab

Loading…