Document terminal workflows and provide a backed-up guest runtime update

This commit is contained in:
Codex
2026-10-13 15:22:32 +03:00
parent c6099103b2
commit d2988bcecc
6 changed files with 540 additions and 12 deletions
+38 -5
View File
@@ -31,9 +31,37 @@ Build outputs, VM disks, runtime state and credentials stay outside Git.
## First use ## First use
Follow the [installation guide](docs/install.md) to install from the live ISO. Follow the [installation guide](docs/install.md) to install from the live ISO.
The initial empty graphical session opens a recovery terminal. Enter `:shell` The session opens a terminal VM menu. Choose a machine by number or name;
for an ordinary host shell; `sudo nmtui` configures host network connections. Enter opens the remembered selection, or the only machine. Opening a stopped
Guest operating systems are supplied by you: machine starts it; opening a running one reconnects its screen. Merely opening
the menu does not start every VM. `:shell` opens an ordinary host shell and
`:network` opens NetworkManager's connection menu. The existing VM disks and
command-line controls remain available.
From a Hydra shell, a direct installer URL can become a running VM in one
command. Replace the example URL with an actual ISO download link:
```sh
hydra run 'https://example.org/distribution.iso' --name work --memory 4096 --cpus 2
hydra open work
```
`run` also accepts a local ISO path. It refuses to replace an existing VM name;
use `open` for subsequent starts. Add `--no-view` for a headless launch. A
download-page URL returning HTML is rejected. Downloads show progress, can be
cancelled with Ctrl+C, and become reusable cache entries only after validation.
Supply the publisher's SHA-256 when available:
```sh
hydra download 'https://example.org/distribution.iso' --sha256 PUBLISHER_SHA256
```
The cache lives under `$XDG_CACHE_HOME/hydra/iso` (normally
`~/.cache/hydra/iso`). Without a publisher checksum, ISO format validation
does not establish publisher authenticity. New launches through this workflow
check available RAM with a 768 MiB host reserve; this cannot guarantee against
later memory pressure. In a nested test, leave RAM available on the outer host
too. The original lower-level commands remain useful for manual control:
```sh ```sh
hydra add work --iso /absolute/path/ubuntu.iso --memory 8192 --cpus 4 hydra add work --iso /absolute/path/ubuntu.iso --memory 8192 --cpus 4
@@ -43,7 +71,7 @@ hydra start experiments
exit exit
``` ```
Back at `hydra recovery>`, enter `:refresh` to show the newly created guests. Back in the terminal menu, select the new guest to open its screen.
Guests remain running while you switch, reconnect viewers or leave the host Guests remain running while you switch, reconnect viewers or leave the host
graphical session. Shut a guest down from inside its OS, or request an orderly graphical session. Shut a guest down from inside its OS, or request an orderly
shutdown with `hydra shutdown NAME`. Check `hydra status NAME` before detaching shutdown with `hydra shutdown NAME`. Check `hydra status NAME` before detaching
@@ -52,7 +80,7 @@ its installer with `hydra media NAME --detach` and starting it again.
| Shortcut | Action | | Shortcut | Action |
| --- | --- | | --- | --- |
| Ctrl+Alt+Shift+Left / Right | Switch guest workspaces | | Ctrl+Alt+Shift+Left / Right | Switch guest workspaces |
| Ctrl+Alt+Shift+Escape | Open local recovery | | Ctrl+Alt+Shift+Escape | Open the terminal VM menu |
| Ctrl+Alt+Shift+R | Release viewer input capture | | Ctrl+Alt+Shift+R | Release viewer input capture |
| Ctrl+Alt+Shift+U | Choose a USB device to redirect | | Ctrl+Alt+Shift+U | Choose a USB device to redirect |
| Brightness keys | Adjust host backlight | | Brightness keys | Adjust host backlight |
@@ -69,6 +97,11 @@ Other host desktops can use `hydra start NAME` and `hydra view NAME --windowed`
without entering the minimal labwc session. Ordinary Gentoo administration and without entering the minimal labwc session. Ordinary Gentoo administration and
Portage remain available through the recovery shell. Portage remain available through the recovery shell.
Fastfetch displays the **Hydra OS** system summary and its custom terminal
logo. It is included in the new image; the first published ISO predates this
interface update. See [terminal usage](docs/terminal-usage.md) for networking,
installation media and common recovery steps.
## Build the ISO ## Build the ISO
Follow the [isolated Gentoo build instructions](image/README.md). Outputs are Follow the [isolated Gentoo build instructions](image/README.md). Outputs are
+224
View File
@@ -0,0 +1,224 @@
#!/usr/bin/env python3
"""Build a self-contained, backed-up runtime update for an installed Hydra guest.
This updates explicitly listed program/branding files, never guest disks,
profiles, credentials, networking or the kernel. It is not a package manager.
"""
import argparse
import datetime
import hashlib
import json
import os
from pathlib import Path, PurePosixPath
import re
import shutil
import subprocess
import sys
import tempfile
import zipfile
class UpdateError(ValueError):
pass
EXACT = {'usr/lib/hydra/bin/hydra', 'etc/motd',
'etc/fastfetch/config.jsonc', 'usr/share/hydra/fastfetch.txt'}
LIMIT = 16 * 1024 * 1024
def allowed(name):
return (name in EXACT or
re.fullmatch(r'usr/lib/hydra/hydra/[a-z_]+\.py', name) is not None)
def digest(data):
return hashlib.sha256(data).hexdigest()
def build_bundle(source, output):
source, output = Path(source), Path(output)
files = {}
for module in sorted((source / 'hydra').glob('*.py')):
files['usr/lib/hydra/hydra/' + module.name] = (module.read_bytes(), 0o644)
launcher = source / 'bin/hydra'
if not launcher.is_file() or 'usr/lib/hydra/hydra/cli.py' not in files:
raise UpdateError('Source is missing the Hydra runtime')
files['usr/lib/hydra/bin/hydra'] = (launcher.read_bytes(), 0o755)
for name in EXACT - {'usr/lib/hydra/bin/hydra'}:
path = source / 'image/rootfs' / name
if path.is_file():
files[name] = (path.read_bytes(), 0o644)
commit = subprocess.run(['git', '-C', str(source), 'rev-parse', 'HEAD'],
text=True, capture_output=True, timeout=10)
manifest = {'format': 1, 'product': 'Hydra OS',
'source_commit': commit.stdout.strip() if commit.returncode == 0 else None,
'files': {name: {'sha256': digest(data), 'mode': mode}
for name, (data, mode) in sorted(files.items())}}
# Exclusive creation preserves previous artifacts, even on repeated builds.
output.parent.mkdir(parents=True, exist_ok=True)
try:
with output.open('xb') as stream:
stream.write(b'#!/usr/bin/env python3\n')
with zipfile.ZipFile(stream, 'w', zipfile.ZIP_DEFLATED) as bundle:
bundle.writestr('__main__.py', Path(__file__).read_bytes())
bundle.writestr('manifest.json', json.dumps(manifest, indent=2))
for name, (data, _) in sorted(files.items()):
if not allowed(name):
raise UpdateError('Unrecognized payload path: ' + name)
bundle.writestr('payload/' + name, data)
except BaseException:
# Do not remove a pre-existing output when exclusive creation failed.
if 'stream' in locals():
output.unlink(missing_ok=True)
raise
output.chmod(0o700)
return manifest
def checked_path(root, name):
relative = PurePosixPath(name)
if relative.is_absolute() or '..' in relative.parts:
raise UpdateError('Unsafe update path')
current = root
for part in relative.parts:
current = current / part
if current.is_symlink():
# /etc/os-release is the one intentional replacement of a known
# vendor link. os.replace replaces the link, never its target.
if (name == 'etc/os-release' and current == root / name and
str(current.readlink()) in {'../usr/lib/os-release', '/usr/lib/os-release'}):
checked_path(root, 'usr/lib/os-release')
return current
raise UpdateError('Symlink update destination refused: ' + str(current))
if current.exists() and not current.is_file():
raise UpdateError('Update destination is not a regular file: ' + str(current))
return current
def atomic_write(path, data, mode):
path.parent.mkdir(parents=True, exist_ok=True)
fd, temporary = tempfile.mkstemp(prefix='.hydra-update-', dir=path.parent)
try:
with os.fdopen(fd, 'wb') as stream:
stream.write(data)
stream.flush()
os.fsync(stream.fileno())
os.fchmod(stream.fileno(), mode)
os.replace(temporary, path)
finally:
Path(temporary).unlink(missing_ok=True)
def apply_bundle(bundle, root=Path('/'), *, execute=False):
root = Path(root).absolute()
if root.is_symlink() or root.resolve() != root:
raise UpdateError('Symlink target root refused')
marker = checked_path(root, 'usr/share/hydra/build-manifest.json')
if not marker.is_file() or not (root / 'usr/lib/hydra/hydra').is_dir():
raise UpdateError('Target is not an existing Hydra installation')
payload = {}
with zipfile.ZipFile(bundle) as archive:
names = archive.namelist()
if len(names) != len(set(names)) or sum(i.file_size for i in archive.infolist()) > LIMIT:
raise UpdateError('Invalid or oversized update bundle')
manifest = json.loads(archive.read('manifest.json'))
if manifest.get('format') != 1 or manifest.get('product') != 'Hydra OS' or not isinstance(manifest.get('files'), dict):
raise UpdateError('Invalid Hydra update manifest')
for name, entry in manifest['files'].items():
if not allowed(name) or entry.get('mode') not in {0o644, 0o755}:
raise UpdateError('Unrecognized update file: ' + name)
data = archive.read('payload/' + name)
if digest(data) != entry.get('sha256'):
raise UpdateError('Payload checksum mismatch: ' + name)
checked_path(root, name)
payload[name] = (data, entry['mode'])
if 'usr/lib/hydra/hydra/cli.py' not in payload:
raise UpdateError('Update has no Hydra CLI')
release = checked_path(root, 'etc/os-release')
vendor = checked_path(root, 'usr/lib/os-release')
current_release = vendor if release.is_symlink() else release
original = current_release.read_text() if current_release.is_file() else ''
retained = [line for line in original.splitlines()
if not re.match(r'^(NAME|PRETTY_NAME|ID|ID_LIKE)=', line)]
retained += ['NAME="Hydra OS"', 'PRETTY_NAME="Hydra OS"',
'ID=hydra-os', 'ID_LIKE=gentoo']
payload['etc/os-release'] = (('\n'.join(retained) + '\n').encode(), 0o644)
receipt_name = 'usr/share/hydra/runtime-update.json'
checked_path(root, receipt_name)
payload[receipt_name] = ((json.dumps(manifest, indent=2) + '\n').encode(), 0o644)
result = {'applied': False, 'source_commit': manifest.get('source_commit'),
'files': sorted(payload), 'backup': None}
if not execute:
return result
if root == Path('/') and os.geteuid() != 0:
raise UpdateError('Apply inside the Hydra guest with sudo')
backup_base = root / 'var/lib/hydra/updates'
# Validate ancestors without creating anything before all payload checks.
checked_path(root, 'var/lib/hydra/updates/.probe')
backup_base.mkdir(parents=True, exist_ok=True, mode=0o700)
stamp = datetime.datetime.now(datetime.timezone.utc).strftime('%Y%m%dT%H%M%SZ-')
backup = Path(tempfile.mkdtemp(prefix=stamp, dir=backup_base))
records = {}
for name in payload:
dest = checked_path(root, name)
if dest.exists() or dest.is_symlink():
saved = backup / name
saved.parent.mkdir(parents=True, exist_ok=True)
shutil.copy2(dest, saved, follow_symlinks=False)
records[name] = {'existed': True, 'mode': dest.lstat().st_mode & 0o777}
if dest.is_symlink():
records[name]['symlink'] = str(dest.readlink())
else:
records[name]['sha256'] = digest(dest.read_bytes())
else:
records[name] = {'existed': False}
(backup / 'backup.json').write_text(json.dumps(records, indent=2) + '\n')
changed = []
try:
for name, (data, mode) in payload.items():
atomic_write(checked_path(root, name), data, mode)
changed.append(name)
except BaseException:
for name in reversed(changed):
dest = checked_path(root, name)
original = records[name]
if 'symlink' in original:
dest.unlink()
dest.symlink_to(original['symlink'])
elif original['existed']:
atomic_write(dest, (backup / name).read_bytes(), original['mode'])
else:
dest.unlink()
raise
result.update(applied=True, backup=str(backup))
return result
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--build', type=Path, metavar='SOURCE')
parser.add_argument('--output', type=Path)
parser.add_argument('--root', type=Path, default=Path('/'), help='Installed Hydra root; staging roots are supported for verification')
parser.add_argument('--apply', action='store_true', help='Apply after verification and retain original files in a backup')
args = parser.parse_args()
try:
if args.build:
if not args.output or args.apply:
parser.error('--build requires --output and cannot be combined with --apply')
result = build_bundle(args.build, args.output)
else:
result = apply_bundle(Path(sys.argv[0]), args.root, execute=args.apply)
print(json.dumps(result, indent=2))
if args.apply:
print('Runtime updated. Log out of the Hydra graphical session and log in again to reload session code; running VMs are unchanged.')
if not (args.root / 'usr/bin/fastfetch').exists():
print('Fastfetch is not installed yet. Inside Hydra: sudo emerge --getbinpkg --noreplace app-misc/fastfetch')
return 0
except (UpdateError, OSError, ValueError, KeyError, zipfile.BadZipFile) as exc:
print('Hydra OS update: ' + str(exc), file=sys.stderr)
return 1
if __name__ == '__main__':
raise SystemExit(main())
+126
View File
@@ -0,0 +1,126 @@
# Using Hydra OS from the terminal
Hydra OS keeps the host small. Virtual machines have independent disks and
processes; closing a screen or the host menu does not power them off.
## Open a machine
The initial session shows the VM menu. Enter a number or a VM name. Enter alone
uses the last selection when it still exists, or the only configured VM. A
stopped VM starts; an already running one reconnects. Ctrl+Alt+Shift+Escape
opens the menu while working inside a VM, and Ctrl+Alt+Shift+Left/Right switch
between VM workspaces.
An ordinary terminal or SSH session can use:
```sh
hydra open work
hydra start work
hydra status work
hydra shutdown work
```
`open` requires a graphical display unless `--no-view` is supplied. `start`
and `shutdown` are the existing explicit lifecycle commands. Shutdown requests
an orderly guest poweroff; it does not promise instantaneous completion.
## Download and run an installer
Use a direct `.iso` download URL supplied by the distribution:
```sh
hydra run 'https://example.org/distribution.iso' --name work --memory 4096 --cpus 2
```
Replace the example URL. The command downloads a validated image, creates a
new profile, starts that VM and opens its screen. Existing names are refused
before downloads or disk changes; use `hydra open work` next time. The same
command accepts a local ISO path. Use `--no-view` to run without a viewer.
To download without creating a machine:
```sh
hydra download 'https://example.org/distribution.iso'
```
Add `--sha256` followed by the publisher's 64 hexadecimal digits to check the
download. Format validation alone checks that the file looks like an ISO,
not who published it. Validated files are cached under
`~/.cache/hydra/iso` (or `$XDG_CACHE_HOME/hydra/iso`). Ctrl+C cancels an active
download without publishing its partial file. HTTP is available for a local
test transfer; prefer the publisher's HTTPS endpoint for Internet downloads.
The guest OS installer still performs installation inside its virtual disk.
After it finishes, shut the guest down, detach the installer and reopen it:
```sh
hydra shutdown work
hydra status work
# Wait until status is stopped:
hydra media work --detach
hydra open work
```
This avoids accidentally booting the live ISO again. No guest disk is recreated.
## Network and host controls
The menu offers host shell and network actions. `sudo nmtui` controls the
physical host's Wi-Fi/Ethernet connections. Guests see virtual Ethernet and
receive Internet access through Hydra; they do not each select the Wi-Fi SSID.
When testing Hydra inside another VM, physical Wi-Fi remains controlled by the
outer host (currently NixOS).
The guest bridge uses `10.203.0.1/24`. A guest can SSH to `10.203.0.1` after
SSH service and authentication have been configured on Hydra. An SSH login is
not enabled automatically. Networking between Hydra and its guests does not
require Internet access. VPN clients can run in individual guests; a shared
host VPN requires routing guest traffic through that VPN explicitly.
## If something fails
- Download returns HTML: copy the actual ISO link, not its web download page.
- A checksum mismatch: do not boot that download; verify the source/checksum.
- A missing ISO: attach a valid image, or detach installation media after the
guest was installed and stopped.
- Insufficient RAM: stop an unused VM or choose a smaller guest allocation.
The launch check reserves 768 MiB on Hydra. A nested Hydra cannot see memory
pressure on the outer NixOS host; leave headroom there too.
- A viewer closes: reopen the VM. Closing a viewer does not erase its disk or
stop its running workloads.
- An unknown VM state: inspect the reported error/QEMU log. Hydra will not
blindly start a duplicate or signal a process whose identity it cannot verify.
Run `fastfetch` for the host summary and Hydra OS logo. Advanced administration
remains ordinary Gentoo administration from the host shell.
## Update an existing test installation
The first ISO predates this interface. A runtime bundle updates the Hydra
program and branding files while retaining the original image manifest and
backing up replaced files under `/var/lib/hydra/updates`. It leaves VM profiles,
disks, the kernel and network configuration in place. To build one from a
trusted checkout:
```sh
python3 dev/guest-update.py --build . --output /tmp/hydra-os-update.pyz
```
Copy the resulting file into Hydra, then inspect and apply it **inside Hydra**:
```sh
python3 hydra-os-update.pyz
sudo python3 hydra-os-update.pyz --apply
```
The first command only prints the file inventory. After applying, log out of
Hydra's graphical session and log in again so the long-running session process
loads the new code. Reopening a terminal alone cannot reload that process.
Running VMs remain independent of the graphical session.
This bundle contains the Fastfetch configuration and logo, but no executable
packages. If the old installation lacks Fastfetch, install it inside Hydra:
```sh
sudo emerge --getbinpkg --noreplace app-misc/fastfetch
```
+145
View File
@@ -0,0 +1,145 @@
"""An installed-guest update must be bounded, inspectable and reversible."""
import importlib.util
import json
from pathlib import Path
import subprocess
import sys
import tempfile
import unittest
from unittest.mock import patch
import zipfile
class GuestUpdateTests(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.script = Path(__file__).resolve().parents[1] / 'dev/guest-update.py'
def setUp(self):
self.assertTrue(self.script.is_file(), 'guest update helper must exist')
spec = importlib.util.spec_from_file_location('guest_update', self.script)
self.mod = importlib.util.module_from_spec(spec)
spec.loader.exec_module(self.mod)
self.tmp = tempfile.TemporaryDirectory()
self.addCleanup(self.tmp.cleanup)
self.base = Path(self.tmp.name)
self.source = self.base / 'source'
for name, value in {'hydra/__init__.py': '', 'hydra/cli.py': 'NEW = True\n',
'bin/hydra': '#!/usr/bin/python3\n',
'image/rootfs/etc/motd': 'Hydra OS\n',
'image/rootfs/usr/share/hydra/fastfetch.txt': 'hydra logo\n'}.items():
dest = self.source / name
dest.parent.mkdir(parents=True, exist_ok=True)
dest.write_text(value)
self.root = self.base / 'guest'
marker = self.root / 'usr/share/hydra/build-manifest.json'
marker.parent.mkdir(parents=True)
marker.write_text('{"original": true}\n')
self.old = self.root / 'usr/lib/hydra/hydra/cli.py'
self.old.parent.mkdir(parents=True)
self.old.write_text('OLD = True\n')
self.bundle = self.base / 'update.pyz'
def build(self):
return self.mod.build_bundle(self.source, self.bundle)
def test_dry_run_preserves_installed_files_and_user_disk(self):
self.build()
disk = self.root / 'home/user/.local/state/hydra/work/disk.qcow2'
disk.parent.mkdir(parents=True)
disk.write_bytes(b'user disk')
result = self.mod.apply_bundle(self.bundle, self.root)
self.assertFalse(result['applied'])
self.assertEqual(self.old.read_text(), 'OLD = True\n')
self.assertEqual(disk.read_bytes(), b'user disk')
self.assertFalse((self.root / 'var/lib/hydra/updates').exists())
def test_apply_saves_backup_and_preserves_original_image_evidence(self):
self.build()
result = self.mod.apply_bundle(self.bundle, self.root, execute=True)
self.assertTrue(result['applied'])
self.assertEqual(self.old.read_text(), 'NEW = True\n')
backup = Path(result['backup'])
self.assertEqual((backup / 'usr/lib/hydra/hydra/cli.py').read_text(), 'OLD = True\n')
self.assertEqual(json.loads((self.root / 'usr/share/hydra/build-manifest.json').read_text()), {'original': True})
self.assertTrue((self.root / 'usr/share/hydra/runtime-update.json').is_file())
self.assertEqual((self.root / 'usr/share/hydra/fastfetch.txt').read_text(), 'hydra logo\n')
def test_branding_preserves_vendor_release_and_backs_up_etc_symlink(self):
vendor = self.root / 'usr/lib/os-release'
vendor.parent.mkdir(parents=True, exist_ok=True)
vendor.write_text('NAME="Gentoo"\nID=gentoo\nVERSION_ID=2.18\n')
release = self.root / 'etc/os-release'
release.parent.mkdir(parents=True)
release.symlink_to('../usr/lib/os-release')
self.build()
result = self.mod.apply_bundle(self.bundle, self.root, execute=True)
self.assertFalse(release.is_symlink())
self.assertIn('PRETTY_NAME="Hydra OS"', release.read_text())
self.assertIn('ID_LIKE=gentoo', release.read_text())
self.assertIn('VERSION_ID=2.18', release.read_text())
self.assertEqual(vendor.read_text(), 'NAME="Gentoo"\nID=gentoo\nVERSION_ID=2.18\n')
saved = Path(result['backup']) / 'etc/os-release'
self.assertTrue(saved.is_symlink())
self.assertEqual(str(saved.readlink()), '../usr/lib/os-release')
def test_non_hydra_root_is_refused_before_changes(self):
self.build()
(self.root / 'usr/share/hydra/build-manifest.json').unlink()
with self.assertRaisesRegex(self.mod.UpdateError, 'Hydra'):
self.mod.apply_bundle(self.bundle, self.root, execute=True)
self.assertEqual(self.old.read_text(), 'OLD = True\n')
def test_modified_payload_is_refused_before_changes(self):
self.build()
other = self.base / 'tampered.pyz'
with zipfile.ZipFile(self.bundle) as src, zipfile.ZipFile(other, 'w') as dst:
for name in src.namelist():
data = b'CORRUPTED\n' if name == 'payload/usr/lib/hydra/hydra/cli.py' else src.read(name)
dst.writestr(name, data)
with self.assertRaisesRegex(self.mod.UpdateError, 'checksum'):
self.mod.apply_bundle(other, self.root, execute=True)
self.assertEqual(self.old.read_text(), 'OLD = True\n')
def test_symlink_destination_cannot_redirect_privileged_update(self):
self.build()
outside = self.base / 'outside'
outside.write_text('KEEP\n')
self.old.unlink()
self.old.symlink_to(outside)
with self.assertRaisesRegex(self.mod.UpdateError, '[Ss]ymlink'):
self.mod.apply_bundle(self.bundle, self.root, execute=True)
self.assertEqual(outside.read_text(), 'KEEP\n')
def test_bundle_never_overwrites_an_existing_output(self):
self.bundle.write_bytes(b'keep this artifact')
with self.assertRaises((FileExistsError, self.mod.UpdateError)):
self.build()
self.assertEqual(self.bundle.read_bytes(), b'keep this artifact')
def test_self_contained_bundle_applies_without_source_checkout(self):
self.build()
result = subprocess.run([sys.executable, str(self.bundle), '--root', str(self.root), '--apply'],
capture_output=True, text=True, timeout=10, cwd=self.base)
self.assertEqual(result.returncode, 0, result.stderr)
self.assertEqual(self.old.read_text(), 'NEW = True\n')
self.assertIn('Runtime updated', result.stdout)
def test_write_failure_rolls_back_files_already_replaced(self):
self.build()
actual_write = self.mod.atomic_write
calls = 0
def fail_once(*args):
nonlocal calls
calls += 1
if calls == 3:
raise OSError('injected full disk')
return actual_write(*args)
with patch.object(self.mod, 'atomic_write', side_effect=fail_once):
with self.assertRaisesRegex(OSError, 'full disk'):
self.mod.apply_bundle(self.bundle, self.root, execute=True)
self.assertEqual(self.old.read_text(), 'OLD = True\n')
self.assertFalse((self.root / 'etc/motd').exists())
self.assertFalse((self.root / 'usr/lib/hydra/bin/hydra').exists())