0.0.002

ddat interop with dat js
cable interop with cabal js
ddat's frontend = dat-thang
cable fe = cable-thang
dillo support based on cel's dillo-dat
bootstrap node
xplatform support

Signed-off-by: divo <418@cryspace.space>

docs/beaker-networking.md

@@ -0,0 +1,109 @@
+# Beaker networking notes for `dat-gpt`
+
+These notes use the archived `beakerbrowser/beaker` 1.1.0 source and the
+included old `dillo-dat` plugin as references. Beaker was archived in 2022.
+Its later Hyperdrive network stack is useful for subsystem boundaries, but it
+is not proof of compatibility with the old Dat protocol or a replacement for
+the Dillo plugin's `hyperdrive` + `hyperdiscovery` path.
+
+## What Beaker separates
+
+1. **Name resolution.** URLs are parsed first. A 64-character key is used
+   directly; a domain requires a resolver. Beaker has a `dat-dns` based
+   resolver and local name/key maps. Its v1.1 HyperDNS implementation leaves
+   the remote DNS logic disabled, and its `dat://` handler calls a separate
+   `dat-dns` resolver. Name resolution therefore belongs above transport and
+   should return the archive key before opening feeds.
+2. **Drive sessions and persistence.** Beaker's later stack talks to a
+   long-lived Hyperspace process through `hyperdrive-daemon-client`. Sessions
+   are keyed by archive key plus version and read/write mode. The daemon owns
+   the storage and network handles; Beaker expires idle sessions and reconnects
+   them if the daemon restarts.
+3. **Announce versus lookup.** Beaker explicitly configures a drive with
+   `announce: true, lookup: true` when it should be hosted and discover peers.
+   Merely opening a drive and resolving its key are separate operations.
+4. **Archive reads.** The newer `hyper://` handler resolves the host to a key,
+   opens/loads the drive (which starts network lookup), checks out an optional
+   version, reads its manifest, resolves the path, and streams the selected
+   file from the drive. It supports `GET`/`HEAD`, byte ranges, redirects for
+   directory URLs without a trailing slash, manifest MIME overrides, and
+   content sniffing. A drive with version zero becomes a clear “no peers found
+   yet” response, so lookup-in-progress is distinct from a missing path.
+   The separate legacy `dat://` handler resolves through `dat-dns`, runs the
+   legacy conversion/download tool into temporary storage, and then serves
+   that filesystem through the same path resolver. That conversion route is
+   materially different from the native Hyperdrive read path.
+5. **Operational visibility.** Beaker has separate debug categories for the
+   application, Dat, DHT discovery, DNS discovery, and the Hypercore protocol.
+   That separation makes it possible to tell whether a failure is lookup,
+   connection, replication, or file resolution.
+
+## What applies to this C implementation
+
+- Preserve the existing `NETWORK_PROTOCOL` switch and `cesspool` spelling.
+  The default Cesspool build uses the V7 message format. `hypercore` selects
+  the old Dillo dependency's V6 channels, nonce-bearing feed handshake, and
+  XSalsa20 stream encryption. The `CSP0` UDP discovery packet format is still
+  separate from the old JS `hyperdiscovery` format.
+- Keep hostname resolution, discovery, authenticated peer sessions, feed
+  replication, and browser path handling as separate layers. The Dillo DPI
+  helper already consumes the URL/path and local archive layers; its cache
+  lookup is not yet a remote archive loader.
+- The archive session joins a key's topic, consumes topic-tagged Cesspool
+  announcements, carries peer feed channels, exchanges HAVE/WANT state,
+  requests blocks, and verifies/forwards signed proofs. The UDP discovery
+  packet format still needs old JS `hyperdiscovery` interoperability work;
+  Noise authentication and socket-to-session transport integration are
+  separate networking layers.
+- Beaker's daemon restart/session handling suggests a lifecycle boundary for
+  the C version: one archive session owns the discovery membership, peer set,
+  replication state, pending block requests, and close/retry behavior. Keep
+  this below both the CLI and Dillo DPI callers so they can share it.
+- `dat-dns` names should be resolved as a mapping to a key, not treated as a
+  second archive identity. Cache the mapping separately from archive content;
+  keep the resolved key visible for debugging and canonical `dat://<key>`
+  links.
+- `dat_dns_resolve` follows the old `dat-dns` URL-to-key contract. The bundled
+  Dillo plugin disables DNS-over-HTTPS and resolves through HTTPS
+  `/.well-known/dat`; the C resolver follows that same path after checking its
+  local alias map. It does not call the operating system's DNS TXT resolver.
+  The record contains a `dat://<64-hex-key>` line and optional `TTL=<seconds>`.
+  This lookup returns a key only; archive discovery and transfer remain
+  separate Cesspool/session operations.
+
+## Suggested implementation order
+
+1. Define an archive network-session API with explicit `open`, `announce`,
+   `lookup`, `peer-up`, `peer-down`, `request-block`, `block-arrived`, and
+   `close` transitions. Keep transport details behind the existing `cesspool`
+   boundary.
+2. Make one peer connection carry a real authenticated protocol handshake and
+   identify the metadata and content feeds. Do not start with browser DNS or
+   directory listing behavior.
+3. Connect remote metadata block requests to feed verification and archive
+   reads. A cold `dat://<key>/index.html` should be able to wait for and serve
+   the requested bytes, while earlier verified blocks remain cached.
+4. Add retry, peer removal, and session cleanup. Expose peer and missing-block
+   state through a diagnostic command before adding a dashboard.
+5. Add name resolution and any Beaker-style convenience behaviors only after
+   key-addressed remote reads work.
+
+## Source references
+
+- Beaker 1.1.0 `app/package.json`: `dat-dns`, `discovery-swarm`, `hyperspace`,
+  `hyperdrive-daemon-client`, and `pauls-dat-api2` dependencies.
+- `app/bg/hyper/daemon.js`: daemon setup/reconnect, per-drive sessions, peer
+  addresses, and session cleanup.
+- `app/bg/hyper/drives.js`: key/domain resolution, network configuration,
+  drive loading, manifest reads, and version checkouts.
+- `app/bg/hyper/dns.js`: local name/key map; remote DNS implementation is
+  commented out in this archived revision.
+- `app/bg/protocols/dat.js`: legacy `dat://` URL host resolution, local
+  legacy download/conversion to temporary storage, path resolution, MIME
+  selection, and response handling.
+- `app/bg/protocols/hyper.js`: native Hyperdrive network open, historic
+  checkout, path resolution, no-peer state, range reads, and browser response
+  behavior.
+- Beaker's repository README identifies version 1.1.0 as the last release and
+  the repository as archived; its debug namespaces include `dat`,
+  `bittorrent-dht`, `dns-discovery`, and `hypercore-protocol`.