From 2a2a4552f7b48859127af2568d7cb97ddd1ef776 Mon Sep 17 00:00:00 2001 From: yaya Date: Tue, 6 Oct 2026 06:42:01 +0100 Subject: [PATCH 1/2] feat(apps): package DATUM with stable service discovery --- app-catalog/catalog.json | 15 ++++ apps/datum/README.md | 64 ++++++++++++++ apps/datum/manifest.yml | 84 +++++++++++++++++++ core/archipelago/src/fips/app_ports.rs | 1 + docker/datum/Dockerfile | 26 ++++++ docker/datum/configure.jq | 19 +++++ docker/datum/entrypoint.sh | 16 ++++ docker/datum/tests/test_config.py | 47 +++++++++++ .../public/assets/img/app-icons/datum.svg | 15 ++++ neode-ui/public/catalog.json | 15 ++++ .../appSession/generatedAppSessionConfig.ts | 2 + 11 files changed, 304 insertions(+) create mode 100644 apps/datum/README.md create mode 100644 apps/datum/manifest.yml create mode 100644 docker/datum/Dockerfile create mode 100644 docker/datum/configure.jq create mode 100644 docker/datum/entrypoint.sh create mode 100644 docker/datum/tests/test_config.py create mode 100644 neode-ui/public/assets/img/app-icons/datum.svg diff --git a/app-catalog/catalog.json b/app-catalog/catalog.json index b37c4146..67acb5e5 100644 --- a/app-catalog/catalog.json +++ b/app-catalog/catalog.json @@ -673,6 +673,21 @@ "tier": "optional", "icon": "/assets/img/app-icons/angor-green.png", "repoUrl": "https://github.com/hoytech/strfry" + }, + { + "id": "datum", + "author": "OCEAN contributors", + "requires": [ + "bitcoin-knots" + ], + "title": "DATUM", + "version": "0.4.1-beta.1", + "description": "Build Bitcoin mining templates on your own node and connect your miners to OCEAN through DATUM.", + "dockerImage": "localhost/archipelago-datum:0.4.1-beta.1", + "category": "bitcoin", + "tier": "optional", + "icon": "/assets/img/app-icons/datum.svg", + "repoUrl": "https://github.com/OCEAN-xyz/datum_gateway" } ] } diff --git a/apps/datum/README.md b/apps/datum/README.md new file mode 100644 index 00000000..656ad30a --- /dev/null +++ b/apps/datum/README.md @@ -0,0 +1,64 @@ +# DATUM on Archipelago + +Packages OCEAN DATUM v0.4.1beta, pinned to upstream commit +`5b061233a3d3323771b2be98e17f543e59346619`. The local build context must ship at +`/opt/archipelago/docker/datum`; no published registry image is assumed. + +## First launch + +Install a Bitcoin node and allow it to synchronize, then install DATUM. Open its +app tile and set your own Bitcoin payout address in DATUM's configuration page. +The initial address is deliberately empty: upstream keeps the UI available while +waiting for a valid address instead of mining to somebody else's address. +The admin username is `admin`. The generated password is stored on the node at +`/var/lib/archipelago/secrets/datum-admin-password`; retrieve it locally as the +node administrator. Do not put it in miner passwords or share it with miners. + +Point miners at `stratum+tcp://:23334`. Use a unique worker +name for every miner, following upstream's payout/worker naming rules: +https://github.com/OCEAN-xyz/datum_gateway/blob/v0.4.1beta/doc/usernames.md +The default is pooled mining only; loss of the pool connection stops mining +rather than silently switching to solo mining. DATUM's web UI reports template, +Bitcoin and pool readiness; an HTTP health check only proves the UI is alive. + +## Stable connections + +Gashboard connects inside `archy-net` to `http://datum:7152`, using Podman's DNS +alias. Never copy a container IP into either app's configuration. Bitcoin's DNS +name is resolved from `BITCOIN_HOST` on each start, and the shared RPC secret and +DATUM admin secret are refreshed without discarding the operator's settings. + +External miners connect to the **node**, not its container. Use a DHCP reservation +on your router and a LAN DNS name if the miner supports DNS. Some miners do not +support mDNS (`.local`); use the reserved LAN IP for those. Container DNS fixes +container recreation, while the reservation prevents the node's DHCP address +from moving. Neither setting requires host networking. + +Only Stratum is published directly. The admin UI is loopback-bound behind the +Archipelago app gate and retains DATUM's admin authentication. The backend uses +upstream's block notification polling fallback, so installing DATUM does not +rewrite or restart Bitcoin to add a `blocknotify` command. + +## Data and validation + +Settings live in `/var/lib/archipelago/datum/config.json` with mode 0600. Preserve +that directory and the platform secrets when uninstalling/reinstalling. + +Before catalog publication, validate install, setup, Bitcoin IBD and recovery, +accepted shares from a real miner, stop/start, container recreation, preserved-data +reinstall, backend restart and a controlled node reboot. Verify Gashboard recovers +after DATUM receives a different container address. These live-node checks are +separate from the local manifest/build checks and require a dedicated test node. + +## Local validation (2026-10-06) + +The pinned image builds on Linux/amd64. Its UI returns HTTP 200 while waiting +for setup, `/clients` rejects unauthenticated requests, and its config is 0600. +The container runs with read-only root, cap-drop ALL and no-new-privileges. +Three config regression tests cover empty first-run payout, preserved payout +policy (including explicit false settings), secret/DNS refresh and invalid input. +Gashboard successfully polls this image using digest authentication and reconnects +when its container IP changes. Manifest preflight and generated catalog drift +checks pass. The catalog entries in this branch are review candidates; no signed +catalog or image has been published. Real mining shares, rootless Podman and +actual-node lifecycle acceptance remain required before release. diff --git a/apps/datum/manifest.yml b/apps/datum/manifest.yml new file mode 100644 index 00000000..4c1addda --- /dev/null +++ b/apps/datum/manifest.yml @@ -0,0 +1,84 @@ +app: + id: datum + name: DATUM + version: 0.4.1-beta.1 + description: Build Bitcoin mining templates on your own node and connect your miners to OCEAN through DATUM. + upstream: + kind: github + repo: OCEAN-xyz/datum_gateway + container_name: datum + container: + build: + context: /opt/archipelago/docker/datum + dockerfile: Dockerfile + tag: localhost/archipelago-datum:0.4.1-beta.1 + network: archy-net + network_aliases: [datum] + data_uid: "1000:1000" + derived_env: + - key: BITCOIN_RPC_HOST + template: "{{BITCOIN_HOST}}" + generated_secrets: + - name: datum-admin-password + kind: hex32 + secret_env: + - key: BITCOIN_RPC_PASSWORD + secret_file: bitcoin-rpc-password + - key: DATUM_ADMIN_PASSWORD + secret_file: datum-admin-password + dependencies: + - app_id: bitcoin-knots + - storage: 1Gi + resources: + cpu_limit: 2 + memory_limit: 512Mi + disk_limit: 1Gi + security: + capabilities: [] + readonly_root: true + no_new_privileges: true + network_policy: isolated + ports: + - host: 7152 + container: 7152 + protocol: tcp + bind: 127.0.0.1 + auth: gated + - host: 23334 + container: 23334 + protocol: tcp + auth: none + auth_rationale: Stratum mining clients require a raw TCP connection and cannot complete a browser login. Payout worker names are handled by DATUM; the administration UI uses a separate gated port. + volumes: + - type: bind + source: /var/lib/archipelago/datum + target: /data + options: [rw] + - type: tmpfs + target: /tmp + tmpfs_options: rw,noexec,nosuid,size=16m + health_check: + type: http + endpoint: http://localhost:7152 + path: / + interval: 30s + timeout: 5s + retries: 3 + interfaces: + main: + name: DATUM Gateway + type: ui + port: 7152 + protocol: http + path: / + bitcoin_integration: + rpc_access: admin + sync_required: true + pruning_support: true + metadata: + icon: /assets/img/app-icons/datum.svg + category: bitcoin + tier: optional + repo: https://github.com/OCEAN-xyz/datum_gateway + launch: + open_in_new_tab: false diff --git a/core/archipelago/src/fips/app_ports.rs b/core/archipelago/src/fips/app_ports.rs index 21b122eb..71dd2bef 100644 --- a/core/archipelago/src/fips/app_ports.rs +++ b/core/archipelago/src/fips/app_ports.rs @@ -14,6 +14,7 @@ pub const APP_LAUNCH_PORTS: &[u16] = &[ 3002, 4080, 5180, + 7152, 7778, 8080, 8081, diff --git a/docker/datum/Dockerfile b/docker/datum/Dockerfile new file mode 100644 index 00000000..825e86d6 --- /dev/null +++ b/docker/datum/Dockerfile @@ -0,0 +1,26 @@ +FROM debian:bookworm-slim@sha256:7c7b2c966bc9ee8cedfeef67e0e279108992c77681fa595db4a9d65c06ccc587 AS build +RUN apt-get update && apt-get install -y --no-install-recommends \ + ca-certificates git build-essential cmake pkg-config libjansson-dev \ + libmicrohttpd-dev libsodium-dev libcurl4-openssl-dev \ + && rm -rf /var/lib/apt/lists/* +WORKDIR /src +# DATUM v0.4.1beta. Verify the commit as well as the tag. +RUN git init && git remote add origin https://github.com/OCEAN-xyz/datum_gateway.git \ + && git fetch --depth 1 origin refs/tags/v0.4.1beta \ + && git checkout --detach FETCH_HEAD \ + && test "$(git rev-parse HEAD)" = 5b061233a3d3323771b2be98e17f543e59346619 \ + && cmake -DCMAKE_BUILD_TYPE=Release . && make -j2 + +FROM debian:bookworm-slim@sha256:7c7b2c966bc9ee8cedfeef67e0e279108992c77681fa595db4a9d65c06ccc587 +RUN apt-get update && apt-get install -y --no-install-recommends \ + ca-certificates libjansson4 libmicrohttpd12 libsodium23 libcurl4 jq curl \ + && rm -rf /var/lib/apt/lists/* \ + && useradd --uid 1000 --create-home datum +WORKDIR /app +COPY --from=build /src/datum_gateway /app/datum_gateway +COPY --from=build /src/www /app/www +COPY entrypoint.sh /app/entrypoint.sh +COPY configure.jq /app/configure.jq +USER 1000:1000 +EXPOSE 7152 23334 +ENTRYPOINT ["sh", "/app/entrypoint.sh"] diff --git a/docker/datum/configure.jq b/docker/datum/configure.jq new file mode 100644 index 00000000..8841a62d --- /dev/null +++ b/docker/datum/configure.jq @@ -0,0 +1,19 @@ +if type != "object" then error("Datum config must be an object") else . end +| .bitcoind.rpcurl = ("http://" + env.BITCOIN_RPC_HOST + ":8332") +| .bitcoind.rpcuser = "archipelago" +| .bitcoind.rpcpassword = env.BITCOIN_RPC_PASSWORD +# Use upstream's getbestblockhash fallback; no host bitcoind hooks needed. +| .bitcoind.notify_fallback = true +| .stratum.listen_addr = "0.0.0.0" +| .stratum.listen_port = 23334 +| .mining.pool_address //= "" +| .mining.coinbase_tag_primary //= "DATUM Gateway" +| .mining.coinbase_tag_secondary //= "Archipelago" +| .api.listen_port = 7152 +| .api.admin_password = env.DATUM_ADMIN_PASSWORD +| .api.modify_conf = true +| .logger.log_to_console = true +| .logger.log_to_file = false +| if .datum.pool_pass_workers == null then .datum.pool_pass_workers = true else . end +| if .datum.pool_pass_full_users == null then .datum.pool_pass_full_users = true else . end +| if .datum.pooled_mining_only == null then .datum.pooled_mining_only = true else . end diff --git a/docker/datum/entrypoint.sh b/docker/datum/entrypoint.sh new file mode 100644 index 00000000..fb2e9089 --- /dev/null +++ b/docker/datum/entrypoint.sh @@ -0,0 +1,16 @@ +#!/bin/sh +set -eu +umask 077 +: "${BITCOIN_RPC_HOST:?Bitcoin host is required}" +: "${BITCOIN_RPC_PASSWORD:?Bitcoin RPC password is required}" +: "${DATUM_ADMIN_PASSWORD:?Datum admin password is required}" + +# Keep operator settings, including the payout address, across recreation. +# Refresh platform-owned credentials and DNS names on every container start. +config=/data/config.json +if [ ! -e "$config" ]; then printf '{}\n' > "$config"; fi +tmp=$(mktemp /data/config.json.XXXXXX) +trap 'rm -f "$tmp"' EXIT HUP INT TERM +jq -e -f /app/configure.jq "$config" > "$tmp" +mv "$tmp" "$config" +exec /app/datum_gateway --config "$config" diff --git a/docker/datum/tests/test_config.py b/docker/datum/tests/test_config.py new file mode 100644 index 00000000..5105634e --- /dev/null +++ b/docker/datum/tests/test_config.py @@ -0,0 +1,47 @@ +"""Config upgrades must preserve payout policy and reject broken input.""" +import json +import os +from pathlib import Path +import subprocess +import unittest + +FILTER = Path(__file__).resolve().parents[1] / 'configure.jq' + + +def configure(value, host='bitcoin-core', password='new-rpc'): + return subprocess.run(['jq', '-e', '-f', str(FILTER)], input=json.dumps(value), + text=True, capture_output=True, + env={**os.environ, 'BITCOIN_RPC_HOST': host, + 'BITCOIN_RPC_PASSWORD': password, + 'DATUM_ADMIN_PASSWORD': 'test-admin'}) + + +class ConfigTests(unittest.TestCase): + def test_first_run_has_no_borrowed_payout_address(self): + result = configure({}) + self.assertEqual(result.returncode, 0, result.stderr) + data = json.loads(result.stdout) + self.assertEqual(data['mining']['pool_address'], '') + self.assertTrue(data['datum']['pooled_mining_only']) + self.assertTrue(data['api']['modify_conf']) + + def test_restart_preserves_payout_and_explicit_false_settings(self): + original = {'mining': {'pool_address': 'operator-address'}, + 'datum': {'pool_pass_workers': False, 'pool_pass_full_users': False, + 'pooled_mining_only': False}, + 'bitcoind': {'rpcurl': 'http://old-ip:8332', 'rpcpassword': 'old'}} + result = configure(original, password='quotes"and\\slashes') + self.assertEqual(result.returncode, 0, result.stderr) + data = json.loads(result.stdout) + self.assertEqual(data['mining']['pool_address'], 'operator-address') + self.assertEqual(data['datum'], original['datum']) + self.assertEqual(data['bitcoind']['rpcurl'], 'http://bitcoin-core:8332') + self.assertEqual(data['bitcoind']['rpcpassword'], 'quotes"and\\slashes') + + def test_invalid_root_is_rejected(self): + for value in [None, [], 'broken', 1]: + self.assertNotEqual(configure(value).returncode, 0) + + +if __name__ == '__main__': + unittest.main() diff --git a/neode-ui/public/assets/img/app-icons/datum.svg b/neode-ui/public/assets/img/app-icons/datum.svg new file mode 100644 index 00000000..d2a53b17 --- /dev/null +++ b/neode-ui/public/assets/img/app-icons/datum.svg @@ -0,0 +1,15 @@ + + + + + + + + + + + + + + + diff --git a/neode-ui/public/catalog.json b/neode-ui/public/catalog.json index b37c4146..67acb5e5 100644 --- a/neode-ui/public/catalog.json +++ b/neode-ui/public/catalog.json @@ -673,6 +673,21 @@ "tier": "optional", "icon": "/assets/img/app-icons/angor-green.png", "repoUrl": "https://github.com/hoytech/strfry" + }, + { + "id": "datum", + "author": "OCEAN contributors", + "requires": [ + "bitcoin-knots" + ], + "title": "DATUM", + "version": "0.4.1-beta.1", + "description": "Build Bitcoin mining templates on your own node and connect your miners to OCEAN through DATUM.", + "dockerImage": "localhost/archipelago-datum:0.4.1-beta.1", + "category": "bitcoin", + "tier": "optional", + "icon": "/assets/img/app-icons/datum.svg", + "repoUrl": "https://github.com/OCEAN-xyz/datum_gateway" } ] } diff --git a/neode-ui/src/views/appSession/generatedAppSessionConfig.ts b/neode-ui/src/views/appSession/generatedAppSessionConfig.ts index b9f70e0e..e5af8327 100644 --- a/neode-ui/src/views/appSession/generatedAppSessionConfig.ts +++ b/neode-ui/src/views/appSession/generatedAppSessionConfig.ts @@ -10,6 +10,7 @@ export const GENERATED_APP_PORTS: Record = { "botfights": 9100, "btcpay-server": 23000, "cuprate-ui": 18091, + "datum": 7152, "electrs-ui": 50002, "electrumx": 50002, "fedimint": 8175, @@ -58,6 +59,7 @@ export const GENERATED_APP_TITLES: Record = { "core-lightning": "Core Lightning (CLN)", "cuprate": "Cuprate", "cuprate-ui": "Cuprate UI", + "datum": "DATUM", "electrs-ui": "Electrs UI", "electrumx": "ElectrumX", "fedimint": "Fedimint Guardian", From febdda968e171e8dd51491ee269fadec087a2aca Mon Sep 17 00:00:00 2001 From: yaya Date: Tue, 6 Oct 2026 07:25:24 +0100 Subject: [PATCH 2/2] docs(datum): record node deployment and DNS recovery checks --- apps/datum/README.md | 26 ++++++++++++++++++++++++-- 1 file changed, 24 insertions(+), 2 deletions(-) diff --git a/apps/datum/README.md b/apps/datum/README.md index 656ad30a..b063e98b 100644 --- a/apps/datum/README.md +++ b/apps/datum/README.md @@ -60,5 +60,27 @@ policy (including explicit false settings), secret/DNS refresh and invalid input Gashboard successfully polls this image using digest authentication and reconnects when its container IP changes. Manifest preflight and generated catalog drift checks pass. The catalog entries in this branch are review candidates; no signed -catalog or image has been published. Real mining shares, rootless Podman and -actual-node lifecycle acceptance remain required before release. +catalog or image has been published. Real mining shares and full lifecycle acceptance remain required before release. + +## Operator-authorized node deployment (2026-10-06) + +Installed as rootless Podman apps on archi-dev-box and yaya-server, with the +manifest and build context staged in the runtime payload. Both nodes report +DATUM healthy. Chromium verified the normalized My Apps icon, title, Launch +button, and embedded native UI with its Config navigation on both nodes. + +On yaya, a normal package restart recreated DATUM at 10.89.0.11 instead of +10.89.0.9. Its configuration checksum was unchanged, and the still-running +Gashboard resolved `datum` to the new address and resumed successful authenticated +polling. Existing app container IDs remained unchanged on both hosts. + +The payout address remains unset. HTTP health proves that setup is available, +not that Bitcoin/pool readiness or accepted mining shares have been established. +No node reboot, IBD experiment, preserved-data reinstall, signed catalog release, +or registry publication was performed in this deployment. + +The deployed platform drops manifest UI/icon metadata from its initial Installing +placeholder, briefly showing a disk-only app under Services. Once scanned, both +apps appear in My Apps with their declared icons and launch interfaces. A separate +platform fix is being prepared; do not claim the installation-placeholder issue +is fixed merely because the completed installation is displayed correctly.