feat(recovery): add offline Btrfs base checkpoints and resumable restore
This commit is contained in:
Codex
@@ -0,0 +1,118 @@
|
||||
# Offline Gentoo base recovery
|
||||
|
||||
`hydra-system` creates and restores a matched Btrfs root snapshot and the installed
|
||||
`EFI/HydraOS` tree. Boot the trusted Hydra live ISO and run it as root with the
|
||||
installed root and ESP unmounted. This command does not operate from the installed
|
||||
OS. An installed `/etc/hydra/live-mode/active` file alone is insufficient: the
|
||||
command also checks the live boot arguments, overlay root and squashfs mount.
|
||||
|
||||
The supported installation has `@root`, `@home` and `@snapshots`. `/boot`, kernel
|
||||
modules, the Portage package database and world file belong to `@root`. Nested
|
||||
subvolumes beneath `@root` are refused. Boot and fstab must select the filesystem
|
||||
UUID plus `subvol=@root`, never a numeric root subvolume ID. `@home` and VM disks
|
||||
stored there survive restoration; files stored elsewhere in `@root` roll back.
|
||||
|
||||
Record the Btrfs filesystem UUID and ESP PARTUUID at installation. From the live
|
||||
ISO, inspect the plan before executing it:
|
||||
|
||||
```sh
|
||||
hydra-system checkpoint --root-uuid ROOT_UUID --esp-partuuid ESP_PARTUUID --name before-update
|
||||
hydra-system checkpoint --root-uuid ROOT_UUID --esp-partuuid ESP_PARTUUID --name before-update \
|
||||
--execute --confirm-root-uuid ROOT_UUID
|
||||
hydra-system list --root-uuid ROOT_UUID
|
||||
```
|
||||
|
||||
Commands emit JSON; the completed checkpoint has an `id` field. Boot the installed
|
||||
system to perform ordinary Portage updates. To recover, boot the live ISO again:
|
||||
|
||||
```sh
|
||||
hydra-system restore --root-uuid ROOT_UUID --checkpoint CHECKPOINT_ID
|
||||
hydra-system restore --root-uuid ROOT_UUID --checkpoint CHECKPOINT_ID \
|
||||
--execute --confirm-root-uuid ROOT_UUID
|
||||
```
|
||||
|
||||
Plans and list mount targets read-only; Btrfs inspection disables log replay.
|
||||
Execution requires matching confirmation and verifies UUID/PARTUUID uniqueness,
|
||||
ordinary partitions, inactive swap, no holders, and a single-device Btrfs layout.
|
||||
A private mount namespace and a root-owned lock isolate the command's mounts.
|
||||
No installed executable, network access or installed recovery program is needed.
|
||||
|
||||
Recovery restores only the recorded `EFI/HydraOS` directory. It does not change
|
||||
partition tables, other EFI directories, `EFI/BOOT`, firmware entries or BootOrder.
|
||||
Boot the restored system through its HydraOS firmware entry, or select
|
||||
`EFI/HydraOS/grubx64.efi` in firmware. A whole-disk installation can also have a
|
||||
removable fallback under `EFI/BOOT`; that fallback is deliberately not restored.
|
||||
If an updated fallback can no longer load the restored root, select the owned
|
||||
HydraOS loader explicitly. Automatic failed-boot recovery is not provided.
|
||||
|
||||
## Durable records and interrupted restores
|
||||
|
||||
The installer marker `/etc/hydra/storage.json` has exactly these fields:
|
||||
|
||||
```json
|
||||
{
|
||||
"format": 1,
|
||||
"install_uuid": "installation UUID",
|
||||
"filesystem": "btrfs",
|
||||
"root_fs_uuid": "Btrfs UUID",
|
||||
"root_partuuid": "root GPT partition UUID",
|
||||
"esp_uuid": "FAT UUID (XXXX-XXXX)",
|
||||
"esp_partuuid": "ESP GPT partition UUID",
|
||||
"owned_efi_path": "EFI/HydraOS",
|
||||
"root_subvolume": "@root"
|
||||
}
|
||||
```
|
||||
|
||||
At the Btrfs top level, completed checkpoints are stored as:
|
||||
|
||||
```text
|
||||
@snapshots/<checkpoint-UUID>/root read-only Btrfs snapshot
|
||||
@snapshots/<checkpoint-UUID>/efi/ owned EFI backup
|
||||
@snapshots/<checkpoint-UUID>/checkpoint.json completed descriptor, published last
|
||||
```
|
||||
|
||||
The descriptor fields are `format`, `id`, `name`, `created`, `storage`,
|
||||
`source_root_uuid`, `snapshot_uuid` and `efi_manifest`. `format` is integer 1;
|
||||
`created` is a UTC ISO timestamp; `storage` is the exact marker above. Each manifest
|
||||
entry records a relative `path`, `kind` (`dir` or `file`), numeric `mode` and
|
||||
`mtime_ns`; files additionally record `size` and SHA-256 `sha256`. Backup paths,
|
||||
bytes and metadata are rechecked before every restore. Links, hardlinks, special
|
||||
files, ambiguous FAT names and extra checkpoint files are refused. Metadata files
|
||||
must be regular, singly linked, owned by root and not group/world writable.
|
||||
|
||||
`@snapshots/restore.json` is the durable active transaction. It has exactly
|
||||
`format`, `id` (transaction UUID), `checkpoint`, `checkpoint_digest`, `storage`,
|
||||
`old_uuid`, `new_uuid`, `prepared_root`, `failed_root` and `phase`. The digest is
|
||||
SHA-256 of the descriptor's canonical JSON (sorted keys, compact separators).
|
||||
Phases are `preparing`, `prepared`, `root-retained`, `root-installed` and
|
||||
`efi-restored`. `new_uuid` starts null; it is recorded once the writable snapshot
|
||||
exists. Snapshot parent UUID and marker checks establish the source if power was
|
||||
lost before that UUID was journaled. Directory fsync and filesystem sync delimit
|
||||
transitions.
|
||||
|
||||
A writable copy is prepared as `@restore-<transaction-UUID>`. The previous root is
|
||||
retained as `@failed-<transaction-UUID>` before the prepared root becomes `@root`.
|
||||
Resume verifies the actual subvolume UUIDs at all three paths and accepts only the
|
||||
three valid rename arrangements. The recorded phase alone never decides which
|
||||
root to keep. After EFI restoration and verification, the transaction is archived
|
||||
as `@snapshots/restored-<transaction-UUID>.json`. No failed roots or checkpoints are
|
||||
automatically deleted.
|
||||
|
||||
Btrfs root replacement and FAT file copying are not one atomic operation. Power
|
||||
loss can leave the system temporarily unbootable. Rerun the **same checkpoint ID**
|
||||
from the live ISO with the same explicit confirmation; recovery validates its
|
||||
records and resumes. Do not manually rename or delete recovery roots or journal
|
||||
files. Unexplained UUID/path combinations are refused for offline inspection.
|
||||
An incomplete checkpoint without its completed descriptor is never restorable.
|
||||
|
||||
These snapshots remain on the original disk. They are not external backups, and
|
||||
there is no promise of online application consistency: checkpoint only while the
|
||||
installation is offline. The unit suite uses a persistent fake Btrfs backend and
|
||||
fresh processes to exercise journal and rename interruptions. Real Btrfs, FAT,
|
||||
UEFI, Portage and disk-only boot acceptance must be verified separately in a fresh
|
||||
VM before release; unit tests do not establish those claims.
|
||||
|
||||
Btrfs's [subvolume documentation](https://btrfs.readthedocs.io/en/latest/btrfs-subvolume.html)
|
||||
explains snapshot boundaries and read-only snapshots; the command checks the
|
||||
[read-only property](https://btrfs.readthedocs.io/en/latest/btrfs-property.html)
|
||||
explicitly instead of inferring it from mount options.
|
||||
Reference in New Issue
Block a user