80 lines
4.0 KiB
Markdown
80 lines
4.0 KiB
Markdown
|
# Scheme records at protocol boundaries
|
||
|
|
|
||
|
|
## Representation contract
|
||
|
|
|
||
|
|
The old Dat/Cabal implementations use JavaScript objects and JSON because they
|
||
|
|
run on Node. This implementation uses S7 Scheme values and tagged
|
||
|
|
S-expressions as its native application representation. Keep records,
|
||
|
|
configuration, and the C-to-client boundary in that form; translate to JSON
|
||
|
|
only at a compatibility edge whose existing peer format requires JSON. Dat
|
||
|
|
wire bytes and Cabal's canonical Hypercore JSON remain their established
|
||
|
|
protocol formats.
|
||
|
|
|
||
|
|
S7 is the application view for structured records that would otherwise be
|
||
|
|
assembled as JavaScript objects or JSON-shaped maps. New archive descriptions
|
||
|
|
are stored as `dat.scm` association lists. Existing wire codecs, feed files,
|
||
|
|
and peer protocol bytes retain their established formats; `dat.json` is not
|
||
|
|
written for newly created archives.
|
||
|
|
|
||
|
|
## Record form
|
||
|
|
|
||
|
|
Records are tagged lists with named association fields:
|
||
|
|
|
||
|
|
```scheme
|
||
|
|
(dat:wire-message
|
||
|
|
(channel . 1)
|
||
|
|
(type . 3)
|
||
|
|
(payload . #u8(...)))
|
||
|
|
```
|
||
|
|
|
||
|
|
`dat:record-ref` reads a named field and accepts an optional fallback.
|
||
|
|
`dat:record-set` returns a record with one field replaced or added. Names are
|
||
|
|
symbols; strings stay strings; protocol integers stay exact integers; binary
|
||
|
|
data stays an S7 byte-vector. Absence is represented by no field, not by
|
||
|
|
conflating an absent value with `#f` or an empty byte-vector.
|
||
|
|
|
||
|
|
## Wire boundary
|
||
|
|
|
||
|
|
`dat-wire->sexp` and `dat-sexp->wire` adapt the C `dat_wire` frame codec. The
|
||
|
|
frame record keeps the channel, type, and opaque payload bytes named and
|
||
|
|
inspectable. The active C codec remains authoritative for varints, framing,
|
||
|
|
message payloads, channel assignments, protocol identity, and encryption.
|
||
|
|
S-expressions are not sent to Dat peers and do not alter feed blocks or
|
||
|
|
signatures. Cable's local UI and room-event boundary uses S7 and
|
||
|
|
S-expressions inside the native C application; it does not start a Node
|
||
|
|
adapter.
|
||
|
|
|
||
|
|
Protocol-specific adapters can add named records such as `dat:feed-open` or
|
||
|
|
`dat:hyperdrive-stat` above this frame boundary. Each adapter should map
|
||
|
|
explicitly to the existing C type or legacy field numbers, preserve unknown
|
||
|
|
fields when forwarding is required, and keep byte strings as byte-vectors.
|
||
|
|
This allows Cesspool and test-only legacy Hypercore views to share Scheme-level record
|
||
|
|
operations while retaining their separate codecs and wire identities.
|
||
|
|
|
||
|
|
## Build and use
|
||
|
|
|
||
|
|
S7 is bundled and required by the default build. The build produces
|
||
|
|
`ddat-scheme` and links the runtime into the application library. With no
|
||
|
|
arguments it starts the REPL; `-e` evaluates an expression, `-l` loads a file,
|
||
|
|
`-I` adds a module directory, and `--repl` enters the REPL after scripts run. Dat's host
|
||
|
|
runtime registers the C bindings, assembles the module search path, and loads
|
||
|
|
`dat-init.scm`, `dat-record.scm`, `dat-wire.scm`, and `dat-json.scm` before
|
||
|
|
user code. `dat:json-parse` accepts strict JSON and returns tagged Scheme
|
||
|
|
objects/arrays while preserving JSON strings, booleans, numbers, and null.
|
||
|
|
`dat:json-stringify` maps those same tagged Scheme values back to JSON at
|
||
|
|
compatibility boundaries. Scheme/S-expression records remain the application
|
||
|
|
representation; JSON is emitted only where an existing peer format requires
|
||
|
|
it. The test-only Cabal harness owns Cabal's canonical JSON Hypercore encoding.
|
||
|
|
|
||
|
|
The runtime searches `DAT_SCHEME_PATH` (a platform-separated list),
|
||
|
|
`DAT_SCHEME_DIR`, the source-tree `scheme/` directory, and the configured
|
||
|
|
installed directory. Application modules map feature names such as
|
||
|
|
`dat:archive` to `dat/archive.scm`, load through `(dat:require 'dat:archive)`,
|
||
|
|
and finish with `(provide 'dat:archive)`. `dat:with-archive` closes an archive
|
||
|
|
even when its body exits through a Scheme condition. Archive values are S7
|
||
|
|
foreign objects: explicit close is idempotent, and garbage collection closes
|
||
|
|
an archive that the caller abandons.
|
||
|
|
|
||
|
|
This is the embedding boundary S7 expects: the host owns startup, load paths,
|
||
|
|
native types, resource lifetime, CLI policy, and application modules. The C
|
||
|
|
wire/feed implementations still own protocol bytes and on-disk formats.
|