136 lines
7.3 KiB
Markdown
136 lines
7.3 KiB
Markdown
# Offline Gentoo base recovery
|
|
|
|
`hydra-system` creates and restores a matched Btrfs root snapshot and the installed
|
|
`EFI/HydraOS` tree. Boot the trusted Hydra live ISO and run it as root with the
|
|
installed root and ESP unmounted. This command does not operate from the installed
|
|
OS. An installed `/etc/hydra/live-mode/active` file alone is insufficient: the
|
|
command also checks the live boot arguments, overlay root and squashfs mount.
|
|
|
|
The supported installation has `@root`, `@home` and `@snapshots`. `/boot`, kernel
|
|
modules, the Portage package database and world file belong to `@root`. Nested
|
|
subvolumes beneath `@root` are refused. Boot and fstab must select the filesystem
|
|
UUID plus `subvol=@root`, never a numeric root subvolume ID. `@home` and VM disks
|
|
stored there survive restoration; files stored elsewhere in `@root` roll back.
|
|
|
|
The image keeps `/srv`, `/var/lib/machines` and `/var/lib/portables` as ordinary
|
|
directories so that systemd does not create nested subvolumes on first boot.
|
|
Assembly copies the installed vendor `home.conf`, `systemd-nspawn.conf` and
|
|
`portables.conf` into `/etc/tmpfiles.d`, changing only those three creation rules.
|
|
Other vendor rules and the separate `/home` policy are preserved. Image rebuilds
|
|
regenerate these overrides and reject unexpected vendor rules. After a Portage
|
|
systemd update, review the overrides against the new `/usr/lib/tmpfiles.d` files;
|
|
existing `/etc` overrides continue to take precedence. Recovery still refuses
|
|
unknown nested subvolumes rather than silently omitting their contents.
|
|
|
|
This includes `/var/lib/hydra-update`: a full root restoration also restores the
|
|
checkpoint's runtime update receipts and replay floor. The signed updater's
|
|
monotonic sequence guarantee applies to runtime apply/recovery/rollback, not to
|
|
replacing the entire root with an older checkpoint. Check the available signed
|
|
release again after base recovery.
|
|
|
|
Record the Btrfs filesystem UUID and ESP PARTUUID at installation. From the live
|
|
ISO, inspect the plan before executing it:
|
|
|
|
```sh
|
|
hydra-system checkpoint --root-uuid ROOT_UUID --esp-partuuid ESP_PARTUUID --name before-update
|
|
hydra-system checkpoint --root-uuid ROOT_UUID --esp-partuuid ESP_PARTUUID --name before-update \
|
|
--execute --confirm-root-uuid ROOT_UUID
|
|
hydra-system list --root-uuid ROOT_UUID
|
|
```
|
|
|
|
Commands emit JSON; the completed checkpoint has an `id` field. Boot the installed
|
|
system to perform ordinary Portage updates. To recover, boot the live ISO again:
|
|
|
|
```sh
|
|
hydra-system restore --root-uuid ROOT_UUID --checkpoint CHECKPOINT_ID
|
|
hydra-system restore --root-uuid ROOT_UUID --checkpoint CHECKPOINT_ID \
|
|
--execute --confirm-root-uuid ROOT_UUID
|
|
```
|
|
|
|
Plans and list mount targets read-only; Btrfs inspection disables log replay with
|
|
`rescue=nologreplay`, as supported by the packaged kernel.
|
|
Execution requires matching confirmation and verifies UUID/PARTUUID uniqueness,
|
|
ordinary partitions, inactive swap, no holders, and a single-device Btrfs layout.
|
|
A private mount namespace and a root-owned lock isolate the command's mounts.
|
|
No installed executable, network access or installed recovery program is needed.
|
|
|
|
Recovery restores only the recorded `EFI/HydraOS` directory. It does not change
|
|
partition tables, other EFI directories, `EFI/BOOT`, firmware entries or BootOrder.
|
|
Boot the restored system through its HydraOS firmware entry, or select
|
|
`EFI/HydraOS/grubx64.efi` in firmware. A whole-disk installation can also have a
|
|
removable fallback under `EFI/BOOT`; that fallback is deliberately not restored.
|
|
If an updated fallback can no longer load the restored root, select the owned
|
|
HydraOS loader explicitly. Automatic failed-boot recovery is not provided.
|
|
|
|
## Durable records and interrupted restores
|
|
|
|
The installer marker `/etc/hydra/storage.json` has exactly these fields:
|
|
|
|
```json
|
|
{
|
|
"format": 1,
|
|
"install_uuid": "installation UUID",
|
|
"filesystem": "btrfs",
|
|
"root_fs_uuid": "Btrfs UUID",
|
|
"root_partuuid": "root GPT partition UUID",
|
|
"esp_uuid": "FAT UUID (XXXX-XXXX)",
|
|
"esp_partuuid": "ESP GPT partition UUID",
|
|
"owned_efi_path": "EFI/HydraOS",
|
|
"root_subvolume": "@root"
|
|
}
|
|
```
|
|
|
|
At the Btrfs top level, completed checkpoints are stored as:
|
|
|
|
```text
|
|
@snapshots/<checkpoint-UUID>/root read-only Btrfs snapshot
|
|
@snapshots/<checkpoint-UUID>/efi/ owned EFI backup
|
|
@snapshots/<checkpoint-UUID>/checkpoint.json completed descriptor, published last
|
|
```
|
|
|
|
The descriptor fields are `format`, `id`, `name`, `created`, `storage`,
|
|
`source_root_uuid`, `snapshot_uuid` and `efi_manifest`. `format` is integer 1;
|
|
`created` is a UTC ISO timestamp; `storage` is the exact marker above. Each manifest
|
|
entry records a relative `path`, `kind` (`dir` or `file`), numeric `mode` and
|
|
`mtime_ns`; files additionally record `size` and SHA-256 `sha256`. Backup paths,
|
|
bytes and metadata are rechecked before every restore. Links, hardlinks, special
|
|
files, ambiguous FAT names and extra checkpoint files are refused. Metadata files
|
|
must be regular, singly linked, owned by root and not group/world writable.
|
|
|
|
`@snapshots/restore.json` is the durable active transaction. It has exactly
|
|
`format`, `id` (transaction UUID), `checkpoint`, `checkpoint_digest`, `storage`,
|
|
`old_uuid`, `new_uuid`, `prepared_root`, `failed_root` and `phase`. The digest is
|
|
SHA-256 of the descriptor's canonical JSON (sorted keys, compact separators).
|
|
Phases are `preparing`, `prepared`, `root-retained`, `root-installed` and
|
|
`efi-restored`. `new_uuid` starts null; it is recorded once the writable snapshot
|
|
exists. Snapshot parent UUID and marker checks establish the source if power was
|
|
lost before that UUID was journaled. Directory fsync and filesystem sync delimit
|
|
transitions.
|
|
|
|
A writable copy is prepared as `@restore-<transaction-UUID>`. The previous root is
|
|
retained as `@failed-<transaction-UUID>` before the prepared root becomes `@root`.
|
|
Resume verifies the actual subvolume UUIDs at all three paths and accepts only the
|
|
three valid rename arrangements. The recorded phase alone never decides which
|
|
root to keep. After EFI restoration and verification, the transaction is archived
|
|
as `@snapshots/restored-<transaction-UUID>.json`. No failed roots or checkpoints are
|
|
automatically deleted.
|
|
|
|
Btrfs root replacement and FAT file copying are not one atomic operation. Power
|
|
loss can leave the system temporarily unbootable. Rerun the **same checkpoint ID**
|
|
from the live ISO with the same explicit confirmation; recovery validates its
|
|
records and resumes. Do not manually rename or delete recovery roots or journal
|
|
files. Unexplained UUID/path combinations are refused for offline inspection.
|
|
An incomplete checkpoint without its completed descriptor is never restorable.
|
|
|
|
These snapshots remain on the original disk. They are not external backups, and
|
|
there is no promise of online application consistency: checkpoint only while the
|
|
installation is offline. The unit suite uses a persistent fake Btrfs backend and
|
|
fresh processes to exercise journal and rename interruptions. Real Btrfs, FAT,
|
|
UEFI, Portage and disk-only boot acceptance must be verified separately in a fresh
|
|
VM before release; unit tests do not establish those claims.
|
|
|
|
Btrfs's [subvolume documentation](https://btrfs.readthedocs.io/en/latest/btrfs-subvolume.html)
|
|
explains snapshot boundaries and read-only snapshots; the command checks the
|
|
[read-only property](https://btrfs.readthedocs.io/en/latest/btrfs-property.html)
|
|
explicitly instead of inferring it from mount options.
|