0.0.002

ddat interop with dat js
cable interop with cabal js
ddat's frontend = dat-thang
cable fe = cable-thang
dillo support based on cel's dillo-dat
bootstrap node
xplatform support

Signed-off-by: divo <418@cryspace.space>

extra/dat-dpi/.gitignore

@@ -0,0 +1 @@
+/build/

extra/dat-dpi/LICENSE

@@ -0,0 +1,661 @@
+                    GNU AFFERO GENERAL PUBLIC LICENSE
+                       Version 3, 19 November 2007
+
+ Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
+ Everyone is permitted to copy and distribute verbatim copies
+ of this license document, but changing it is not allowed.
+
+                            Preamble
+
+  The GNU Affero General Public License is a free, copyleft license for
+software and other kinds of works, specifically designed to ensure
+cooperation with the community in the case of network server software.
+
+  The licenses for most software and other practical works are designed
+to take away your freedom to share and change the works.  By contrast,
+our General Public Licenses are intended to guarantee your freedom to
+share and change all versions of a program--to make sure it remains free
+software for all its users.
+
+  When we speak of free software, we are referring to freedom, not
+price.  Our General Public Licenses are designed to make sure that you
+have the freedom to distribute copies of free software (and charge for
+them if you wish), that you receive source code or can get it if you
+want it, that you can change the software or use pieces of it in new
+free programs, and that you know you can do these things.
+
+  Developers that use our General Public Licenses protect your rights
+with two steps: (1) assert copyright on the software, and (2) offer
+you this License which gives you legal permission to copy, distribute
+and/or modify the software.
+
+  A secondary benefit of defending all users' freedom is that
+improvements made in alternate versions of the program, if they
+receive widespread use, become available for other developers to
+incorporate.  Many developers of free software are heartened and
+encouraged by the resulting cooperation.  However, in the case of
+software used on network servers, this result may fail to come about.
+The GNU General Public License permits making a modified version and
+letting the public access it on a server without ever releasing its
+source code to the public.
+
+  The GNU Affero General Public License is designed specifically to
+ensure that, in such cases, the modified source code becomes available
+to the community.  It requires the operator of a network server to
+provide the source code of the modified version running there to the
+users of that server.  Therefore, public use of a modified version, on
+a publicly accessible server, gives the public access to the source
+code of the modified version.
+
+  An older license, called the Affero General Public License and
+published by Affero, was designed to accomplish similar goals.  This is
+a different license, not a version of the Affero GPL, but Affero has
+released a new version of the Affero GPL which permits relicensing under
+this license.
+
+  The precise terms and conditions for copying, distribution and
+modification follow.
+
+                       TERMS AND CONDITIONS
+
+  0. Definitions.
+
+  "This License" refers to version 3 of the GNU Affero General Public License.
+
+  "Copyright" also means copyright-like laws that apply to other kinds of
+works, such as semiconductor masks.
+
+  "The Program" refers to any copyrightable work licensed under this
+License.  Each licensee is addressed as "you".  "Licensees" and
+"recipients" may be individuals or organizations.
+
+  To "modify" a work means to copy from or adapt all or part of the work
+in a fashion requiring copyright permission, other than the making of an
+exact copy.  The resulting work is called a "modified version" of the
+earlier work or a work "based on" the earlier work.
+
+  A "covered work" means either the unmodified Program or a work based
+on the Program.
+
+  To "propagate" a work means to do anything with it that, without
+permission, would make you directly or secondarily liable for
+infringement under applicable copyright law, except executing it on a
+computer or modifying a private copy.  Propagation includes copying,
+distribution (with or without modification), making available to the
+public, and in some countries other activities as well.
+
+  To "convey" a work means any kind of propagation that enables other
+parties to make or receive copies.  Mere interaction with a user through
+a computer network, with no transfer of a copy, is not conveying.
+
+  An interactive user interface displays "Appropriate Legal Notices"
+to the extent that it includes a convenient and prominently visible
+feature that (1) displays an appropriate copyright notice, and (2)
+tells the user that there is no warranty for the work (except to the
+extent that warranties are provided), that licensees may convey the
+work under this License, and how to view a copy of this License.  If
+the interface presents a list of user commands or options, such as a
+menu, a prominent item in the list meets this criterion.
+
+  1. Source Code.
+
+  The "source code" for a work means the preferred form of the work
+for making modifications to it.  "Object code" means any non-source
+form of a work.
+
+  A "Standard Interface" means an interface that either is an official
+standard defined by a recognized standards body, or, in the case of
+interfaces specified for a particular programming language, one that
+is widely used among developers working in that language.
+
+  The "System Libraries" of an executable work include anything, other
+than the work as a whole, that (a) is included in the normal form of
+packaging a Major Component, but which is not part of that Major
+Component, and (b) serves only to enable use of the work with that
+Major Component, or to implement a Standard Interface for which an
+implementation is available to the public in source code form.  A
+"Major Component", in this context, means a major essential component
+(kernel, window system, and so on) of the specific operating system
+(if any) on which the executable work runs, or a compiler used to
+produce the work, or an object code interpreter used to run it.
+
+  The "Corresponding Source" for a work in object code form means all
+the source code needed to generate, install, and (for an executable
+work) run the object code and to modify the work, including scripts to
+control those activities.  However, it does not include the work's
+System Libraries, or general-purpose tools or generally available free
+programs which are used unmodified in performing those activities but
+which are not part of the work.  For example, Corresponding Source
+includes interface definition files associated with source files for
+the work, and the source code for shared libraries and dynamically
+linked subprograms that the work is specifically designed to require,
+such as by intimate data communication or control flow between those
+subprograms and other parts of the work.
+
+  The Corresponding Source need not include anything that users
+can regenerate automatically from other parts of the Corresponding
+Source.
+
+  The Corresponding Source for a work in source code form is that
+same work.
+
+  2. Basic Permissions.
+
+  All rights granted under this License are granted for the term of
+copyright on the Program, and are irrevocable provided the stated
+conditions are met.  This License explicitly affirms your unlimited
+permission to run the unmodified Program.  The output from running a
+covered work is covered by this License only if the output, given its
+content, constitutes a covered work.  This License acknowledges your
+rights of fair use or other equivalent, as provided by copyright law.
+
+  You may make, run and propagate covered works that you do not
+convey, without conditions so long as your license otherwise remains
+in force.  You may convey covered works to others for the sole purpose
+of having them make modifications exclusively for you, or provide you
+with facilities for running those works, provided that you comply with
+the terms of this License in conveying all material for which you do
+not control copyright.  Those thus making or running the covered works
+for you must do so exclusively on your behalf, under your direction
+and control, on terms that prohibit them from making any copies of
+your copyrighted material outside their relationship with you.
+
+  Conveying under any other circumstances is permitted solely under
+the conditions stated below.  Sublicensing is not allowed; section 10
+makes it unnecessary.
+
+  3. Protecting Users' Legal Rights From Anti-Circumvention Law.
+
+  No covered work shall be deemed part of an effective technological
+measure under any applicable law fulfilling obligations under article
+11 of the WIPO copyright treaty adopted on 20 December 1996, or
+similar laws prohibiting or restricting circumvention of such
+measures.
+
+  When you convey a covered work, you waive any legal power to forbid
+circumvention of technological measures to the extent such circumvention
+is effected by exercising rights under this License with respect to
+the covered work, and you disclaim any intention to limit operation or
+modification of the work as a means of enforcing, against the work's
+users, your or third parties' legal rights to forbid circumvention of
+technological measures.
+
+  4. Conveying Verbatim Copies.
+
+  You may convey verbatim copies of the Program's source code as you
+receive it, in any medium, provided that you conspicuously and
+appropriately publish on each copy an appropriate copyright notice;
+keep intact all notices stating that this License and any
+non-permissive terms added in accord with section 7 apply to the code;
+keep intact all notices of the absence of any warranty; and give all
+recipients a copy of this License along with the Program.
+
+  You may charge any price or no price for each copy that you convey,
+and you may offer support or warranty protection for a fee.
+
+  5. Conveying Modified Source Versions.
+
+  You may convey a work based on the Program, or the modifications to
+produce it from the Program, in the form of source code under the
+terms of section 4, provided that you also meet all of these conditions:
+
+    a) The work must carry prominent notices stating that you modified
+    it, and giving a relevant date.
+
+    b) The work must carry prominent notices stating that it is
+    released under this License and any conditions added under section
+    7.  This requirement modifies the requirement in section 4 to
+    "keep intact all notices".
+
+    c) You must license the entire work, as a whole, under this
+    License to anyone who comes into possession of a copy.  This
+    License will therefore apply, along with any applicable section 7
+    additional terms, to the whole of the work, and all its parts,
+    regardless of how they are packaged.  This License gives no
+    permission to license the work in any other way, but it does not
+    invalidate such permission if you have separately received it.
+
+    d) If the work has interactive user interfaces, each must display
+    Appropriate Legal Notices; however, if the Program has interactive
+    interfaces that do not display Appropriate Legal Notices, your
+    work need not make them do so.
+
+  A compilation of a covered work with other separate and independent
+works, which are not by their nature extensions of the covered work,
+and which are not combined with it such as to form a larger program,
+in or on a volume of a storage or distribution medium, is called an
+"aggregate" if the compilation and its resulting copyright are not
+used to limit the access or legal rights of the compilation's users
+beyond what the individual works permit.  Inclusion of a covered work
+in an aggregate does not cause this License to apply to the other
+parts of the aggregate.
+
+  6. Conveying Non-Source Forms.
+
+  You may convey a covered work in object code form under the terms
+of sections 4 and 5, provided that you also convey the
+machine-readable Corresponding Source under the terms of this License,
+in one of these ways:
+
+    a) Convey the object code in, or embodied in, a physical product
+    (including a physical distribution medium), accompanied by the
+    Corresponding Source fixed on a durable physical medium
+    customarily used for software interchange.
+
+    b) Convey the object code in, or embodied in, a physical product
+    (including a physical distribution medium), accompanied by a
+    written offer, valid for at least three years and valid for as
+    long as you offer spare parts or customer support for that product
+    model, to give anyone who possesses the object code either (1) a
+    copy of the Corresponding Source for all the software in the
+    product that is covered by this License, on a durable physical
+    medium customarily used for software interchange, for a price no
+    more than your reasonable cost of physically performing this
+    conveying of source, or (2) access to copy the
+    Corresponding Source from a network server at no charge.
+
+    c) Convey individual copies of the object code with a copy of the
+    written offer to provide the Corresponding Source.  This
+    alternative is allowed only occasionally and noncommercially, and
+    only if you received the object code with such an offer, in accord
+    with subsection 6b.
+
+    d) Convey the object code by offering access from a designated
+    place (gratis or for a charge), and offer equivalent access to the
+    Corresponding Source in the same way through the same place at no
+    further charge.  You need not require recipients to copy the
+    Corresponding Source along with the object code.  If the place to
+    copy the object code is a network server, the Corresponding Source
+    may be on a different server (operated by you or a third party)
+    that supports equivalent copying facilities, provided you maintain
+    clear directions next to the object code saying where to find the
+    Corresponding Source.  Regardless of what server hosts the
+    Corresponding Source, you remain obligated to ensure that it is
+    available for as long as needed to satisfy these requirements.
+
+    e) Convey the object code using peer-to-peer transmission, provided
+    you inform other peers where the object code and Corresponding
+    Source of the work are being offered to the general public at no
+    charge under subsection 6d.
+
+  A separable portion of the object code, whose source code is excluded
+from the Corresponding Source as a System Library, need not be
+included in conveying the object code work.
+
+  A "User Product" is either (1) a "consumer product", which means any
+tangible personal property which is normally used for personal, family,
+or household purposes, or (2) anything designed or sold for incorporation
+into a dwelling.  In determining whether a product is a consumer product,
+doubtful cases shall be resolved in favor of coverage.  For a particular
+product received by a particular user, "normally used" refers to a
+typical or common use of that class of product, regardless of the status
+of the particular user or of the way in which the particular user
+actually uses, or expects or is expected to use, the product.  A product
+is a consumer product regardless of whether the product has substantial
+commercial, industrial or non-consumer uses, unless such uses represent
+the only significant mode of use of the product.
+
+  "Installation Information" for a User Product means any methods,
+procedures, authorization keys, or other information required to install
+and execute modified versions of a covered work in that User Product from
+a modified version of its Corresponding Source.  The information must
+suffice to ensure that the continued functioning of the modified object
+code is in no case prevented or interfered with solely because
+modification has been made.
+
+  If you convey an object code work under this section in, or with, or
+specifically for use in, a User Product, and the conveying occurs as
+part of a transaction in which the right of possession and use of the
+User Product is transferred to the recipient in perpetuity or for a
+fixed term (regardless of how the transaction is characterized), the
+Corresponding Source conveyed under this section must be accompanied
+by the Installation Information.  But this requirement does not apply
+if neither you nor any third party retains the ability to install
+modified object code on the User Product (for example, the work has
+been installed in ROM).
+
+  The requirement to provide Installation Information does not include a
+requirement to continue to provide support service, warranty, or updates
+for a work that has been modified or installed by the recipient, or for
+the User Product in which it has been modified or installed.  Access to a
+network may be denied when the modification itself materially and
+adversely affects the operation of the network or violates the rules and
+protocols for communication across the network.
+
+  Corresponding Source conveyed, and Installation Information provided,
+in accord with this section must be in a format that is publicly
+documented (and with an implementation available to the public in
+source code form), and must require no special password or key for
+unpacking, reading or copying.
+
+  7. Additional Terms.
+
+  "Additional permissions" are terms that supplement the terms of this
+License by making exceptions from one or more of its conditions.
+Additional permissions that are applicable to the entire Program shall
+be treated as though they were included in this License, to the extent
+that they are valid under applicable law.  If additional permissions
+apply only to part of the Program, that part may be used separately
+under those permissions, but the entire Program remains governed by
+this License without regard to the additional permissions.
+
+  When you convey a copy of a covered work, you may at your option
+remove any additional permissions from that copy, or from any part of
+it.  (Additional permissions may be written to require their own
+removal in certain cases when you modify the work.)  You may place
+additional permissions on material, added by you to a covered work,
+for which you have or can give appropriate copyright permission.
+
+  Notwithstanding any other provision of this License, for material you
+add to a covered work, you may (if authorized by the copyright holders of
+that material) supplement the terms of this License with terms:
+
+    a) Disclaiming warranty or limiting liability differently from the
+    terms of sections 15 and 16 of this License; or
+
+    b) Requiring preservation of specified reasonable legal notices or
+    author attributions in that material or in the Appropriate Legal
+    Notices displayed by works containing it; or
+
+    c) Prohibiting misrepresentation of the origin of that material, or
+    requiring that modified versions of such material be marked in
+    reasonable ways as different from the original version; or
+
+    d) Limiting the use for publicity purposes of names of licensors or
+    authors of the material; or
+
+    e) Declining to grant rights under trademark law for use of some
+    trade names, trademarks, or service marks; or
+
+    f) Requiring indemnification of licensors and authors of that
+    material by anyone who conveys the material (or modified versions of
+    it) with contractual assumptions of liability to the recipient, for
+    any liability that these contractual assumptions directly impose on
+    those licensors and authors.
+
+  All other non-permissive additional terms are considered "further
+restrictions" within the meaning of section 10.  If the Program as you
+received it, or any part of it, contains a notice stating that it is
+governed by this License along with a term that is a further
+restriction, you may remove that term.  If a license document contains
+a further restriction but permits relicensing or conveying under this
+License, you may add to a covered work material governed by the terms
+of that license document, provided that the further restriction does
+not survive such relicensing or conveying.
+
+  If you add terms to a covered work in accord with this section, you
+must place, in the relevant source files, a statement of the
+additional terms that apply to those files, or a notice indicating
+where to find the applicable terms.
+
+  Additional terms, permissive or non-permissive, may be stated in the
+form of a separately written license, or stated as exceptions;
+the above requirements apply either way.
+
+  8. Termination.
+
+  You may not propagate or modify a covered work except as expressly
+provided under this License.  Any attempt otherwise to propagate or
+modify it is void, and will automatically terminate your rights under
+this License (including any patent licenses granted under the third
+paragraph of section 11).
+
+  However, if you cease all violation of this License, then your
+license from a particular copyright holder is reinstated (a)
+provisionally, unless and until the copyright holder explicitly and
+finally terminates your license, and (b) permanently, if the copyright
+holder fails to notify you of the violation by some reasonable means
+prior to 60 days after the cessation.
+
+  Moreover, your license from a particular copyright holder is
+reinstated permanently if the copyright holder notifies you of the
+violation by some reasonable means, this is the first time you have
+received notice of violation of this License (for any work) from that
+copyright holder, and you cure the violation prior to 30 days after
+your receipt of the notice.
+
+  Termination of your rights under this section does not terminate the
+licenses of parties who have received copies or rights from you under
+this License.  If your rights have been terminated and not permanently
+reinstated, you do not qualify to receive new licenses for the same
+material under section 10.
+
+  9. Acceptance Not Required for Having Copies.
+
+  You are not required to accept this License in order to receive or
+run a copy of the Program.  Ancillary propagation of a covered work
+occurring solely as a consequence of using peer-to-peer transmission
+to receive a copy likewise does not require acceptance.  However,
+nothing other than this License grants you permission to propagate or
+modify any covered work.  These actions infringe copyright if you do
+not accept this License.  Therefore, by modifying or propagating a
+covered work, you indicate your acceptance of this License to do so.
+
+  10. Automatic Licensing of Downstream Recipients.
+
+  Each time you convey a covered work, the recipient automatically
+receives a license from the original licensors, to run, modify and
+propagate that work, subject to this License.  You are not responsible
+for enforcing compliance by third parties with this License.
+
+  An "entity transaction" is a transaction transferring control of an
+organization, or substantially all assets of one, or subdividing an
+organization, or merging organizations.  If propagation of a covered
+work results from an entity transaction, each party to that
+transaction who receives a copy of the work also receives whatever
+licenses to the work the party's predecessor in interest had or could
+give under the previous paragraph, plus a right to possession of the
+Corresponding Source of the work from the predecessor in interest, if
+the predecessor has it or can get it with reasonable efforts.
+
+  You may not impose any further restrictions on the exercise of the
+rights granted or affirmed under this License.  For example, you may
+not impose a license fee, royalty, or other charge for exercise of
+rights granted under this License, and you may not initiate litigation
+(including a cross-claim or counterclaim in a lawsuit) alleging that
+any patent claim is infringed by making, using, selling, offering for
+sale, or importing the Program or any portion of it.
+
+  11. Patents.
+
+  A "contributor" is a copyright holder who authorizes use under this
+License of the Program or a work on which the Program is based.  The
+work thus licensed is called the contributor's "contributor version".
+
+  A contributor's "essential patent claims" are all patent claims
+owned or controlled by the contributor, whether already acquired or
+hereafter acquired, that would be infringed by some manner, permitted
+by this License, of making, using, or selling its contributor version,
+but do not include claims that would be infringed only as a
+consequence of further modification of the contributor version.  For
+purposes of this definition, "control" includes the right to grant
+patent sublicenses in a manner consistent with the requirements of
+this License.
+
+  Each contributor grants you a non-exclusive, worldwide, royalty-free
+patent license under the contributor's essential patent claims, to
+make, use, sell, offer for sale, import and otherwise run, modify and
+propagate the contents of its contributor version.
+
+  In the following three paragraphs, a "patent license" is any express
+agreement or commitment, however denominated, not to enforce a patent
+(such as an express permission to practice a patent or covenant not to
+sue for patent infringement).  To "grant" such a patent license to a
+party means to make such an agreement or commitment not to enforce a
+patent against the party.
+
+  If you convey a covered work, knowingly relying on a patent license,
+and the Corresponding Source of the work is not available for anyone
+to copy, free of charge and under the terms of this License, through a
+publicly available network server or other readily accessible means,
+then you must either (1) cause the Corresponding Source to be so
+available, or (2) arrange to deprive yourself of the benefit of the
+patent license for this particular work, or (3) arrange, in a manner
+consistent with the requirements of this License, to extend the patent
+license to downstream recipients.  "Knowingly relying" means you have
+actual knowledge that, but for the patent license, your conveying the
+covered work in a country, or your recipient's use of the covered work
+in a country, would infringe one or more identifiable patents in that
+country that you have reason to believe are valid.
+
+  If, pursuant to or in connection with a single transaction or
+arrangement, you convey, or propagate by procuring conveyance of, a
+covered work, and grant a patent license to some of the parties
+receiving the covered work authorizing them to use, propagate, modify
+or convey a specific copy of the covered work, then the patent license
+you grant is automatically extended to all recipients of the covered
+work and works based on it.
+
+  A patent license is "discriminatory" if it does not include within
+the scope of its coverage, prohibits the exercise of, or is
+conditioned on the non-exercise of one or more of the rights that are
+specifically granted under this License.  You may not convey a covered
+work if you are a party to an arrangement with a third party that is
+in the business of distributing software, under which you make payment
+to the third party based on the extent of your activity of conveying
+the work, and under which the third party grants, to any of the
+parties who would receive the covered work from you, a discriminatory
+patent license (a) in connection with copies of the covered work
+conveyed by you (or copies made from those copies), or (b) primarily
+for and in connection with specific products or compilations that
+contain the covered work, unless you entered into that arrangement,
+or that patent license was granted, prior to 28 March 2007.
+
+  Nothing in this License shall be construed as excluding or limiting
+any implied license or other defenses to infringement that may
+otherwise be available to you under applicable patent law.
+
+  12. No Surrender of Others' Freedom.
+
+  If conditions are imposed on you (whether by court order, agreement or
+otherwise) that contradict the conditions of this License, they do not
+excuse you from the conditions of this License.  If you cannot convey a
+covered work so as to satisfy simultaneously your obligations under this
+License and any other pertinent obligations, then as a consequence you may
+not convey it at all.  For example, if you agree to terms that obligate you
+to collect a royalty for further conveying from those to whom you convey
+the Program, the only way you could satisfy both those terms and this
+License would be to refrain entirely from conveying the Program.
+
+  13. Remote Network Interaction; Use with the GNU General Public License.
+
+  Notwithstanding any other provision of this License, if you modify the
+Program, your modified version must prominently offer all users
+interacting with it remotely through a computer network (if your version
+supports such interaction) an opportunity to receive the Corresponding
+Source of your version by providing access to the Corresponding Source
+from a network server at no charge, through some standard or customary
+means of facilitating copying of software.  This Corresponding Source
+shall include the Corresponding Source for any work covered by version 3
+of the GNU General Public License that is incorporated pursuant to the
+following paragraph.
+
+  Notwithstanding any other provision of this License, you have
+permission to link or combine any covered work with a work licensed
+under version 3 of the GNU General Public License into a single
+combined work, and to convey the resulting work.  The terms of this
+License will continue to apply to the part which is the covered work,
+but the work with which it is combined will remain governed by version
+3 of the GNU General Public License.
+
+  14. Revised Versions of this License.
+
+  The Free Software Foundation may publish revised and/or new versions of
+the GNU Affero General Public License from time to time.  Such new versions
+will be similar in spirit to the present version, but may differ in detail to
+address new problems or concerns.
+
+  Each version is given a distinguishing version number.  If the
+Program specifies that a certain numbered version of the GNU Affero General
+Public License "or any later version" applies to it, you have the
+option of following the terms and conditions either of that numbered
+version or of any later version published by the Free Software
+Foundation.  If the Program does not specify a version number of the
+GNU Affero General Public License, you may choose any version ever published
+by the Free Software Foundation.
+
+  If the Program specifies that a proxy can decide which future
+versions of the GNU Affero General Public License can be used, that proxy's
+public statement of acceptance of a version permanently authorizes you
+to choose that version for the Program.
+
+  Later license versions may give you additional or different
+permissions.  However, no additional obligations are imposed on any
+author or copyright holder as a result of your choosing to follow a
+later version.
+
+  15. Disclaimer of Warranty.
+
+  THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
+APPLICABLE LAW.  EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
+HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
+OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
+THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+PURPOSE.  THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
+IS WITH YOU.  SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
+ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
+
+  16. Limitation of Liability.
+
+  IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
+WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
+THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
+GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
+USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
+DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
+PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
+EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
+SUCH DAMAGES.
+
+  17. Interpretation of Sections 15 and 16.
+
+  If the disclaimer of warranty and limitation of liability provided
+above cannot be given local legal effect according to their terms,
+reviewing courts shall apply local law that most closely approximates
+an absolute waiver of all civil liability in connection with the
+Program, unless a warranty or assumption of liability accompanies a
+copy of the Program in return for a fee.
+
+                     END OF TERMS AND CONDITIONS
+
+            How to Apply These Terms to Your New Programs
+
+  If you develop a new program, and you want it to be of the greatest
+possible use to the public, the best way to achieve this is to make it
+free software which everyone can redistribute and change under these terms.
+
+  To do so, attach the following notices to the program.  It is safest
+to attach them to the start of each source file to most effectively
+state the exclusion of warranty; and each file should have at least
+the "copyright" line and a pointer to where the full notice is found.
+
+    <one line to give the program's name and a brief idea of what it does.>
+    Copyright (C) <year>  <name of author>
+
+    This program is free software: you can redistribute it and/or modify
+    it under the terms of the GNU Affero General Public License as published
+    by the Free Software Foundation, either version 3 of the License, or
+    (at your option) any later version.
+
+    This program is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU Affero General Public License for more details.
+
+    You should have received a copy of the GNU Affero General Public License
+    along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+Also add information on how to contact you by electronic and paper mail.
+
+  If your software can interact with users remotely through a computer
+network, you should also make sure that it provides a way for users to
+get its source.  For example, if your program is a web application, its
+interface could display a "Source" link that leads users to an archive
+of the code.  There are many ways you could offer source, and different
+solutions will be better for different programs; see section 13 for the
+specific requirements.
+
+  You should also get your employer (if you work as a programmer) or school,
+if any, to sign a "copyright disclaimer" for the program, if necessary.
+For more information on this, and how to apply and follow the GNU AGPL, see
+<http://www.gnu.org/licenses/>.

extra/dat-dpi/Makefile

@@ -0,0 +1,72 @@
+CC ?= cc
+AR ?= ar
+RANLIB ?= ranlib
+OBJCOPY ?= objcopy
+INSTALL ?= install
+PREFIX ?= /usr/local
+DPIDIR ?= $(PREFIX)/lib/dillo/dpi
+DESTDIR ?=
+BUILD ?= build
+CFLAGS ?= -O2 -std=c99 -Wall -Wextra -Wpedantic
+CPPFLAGS ?=
+LDFLAGS ?=
+
+PKG_CONFIG ?= pkg-config
+CURL_CFLAGS ?= $(shell $(PKG_CONFIG) --cflags libcurl 2>/dev/null)
+CURL_LIBS ?= $(shell $(PKG_CONFIG) --libs libcurl 2>/dev/null || echo -lcurl)
+SODIUM_CFLAGS ?= $(shell $(PKG_CONFIG) --cflags libsodium 2>/dev/null)
+SODIUM_LIBS ?= $(shell $(PKG_CONFIG) --libs libsodium 2>/dev/null || echo -lsodium)
+
+CPPFLAGS += -Iinc -DSLEEP_SALT -DDAT_BUILD_DIR=\"$(BUILD)\" -DDAT_NETWORK_PROTOCOL=\"$(shell tr -d '\r\n' < NETWORK_PROTOCOL)\" $(CURL_CFLAGS) $(SODIUM_CFLAGS)
+LIBS = $(SODIUM_LIBS) $(CURL_LIBS) -lm -ldl
+
+LIB_SOURCES = src/base32768.c src/cable.c src/cable_feed.c src/dat.c src/dat_browse.c src/dat_dns.c src/dat_fetch.c src/dat_walk.c src/dat_net.c src/dat_wire.c src/dat_session.c \
+	src/cesspool.c src/sleep.c src/sleep_tree.c src/sleep_bitfield.c \
+	src/sleep_proto.c src/sleep_crypto.c src/sleep_writer.c \
+	src/sleep_feed.c src/sleep_proof.c src/sleep_replication.c src/sleep_utp.c
+LIB_OBJECTS = $(patsubst src/%.c,$(BUILD)/%.o,$(LIB_SOURCES)) $(BUILD)/sleep_verify_lib.o
+DEPFILES = $(LIB_OBJECTS:.o=.d) $(BUILD)/s7.d
+
+.PHONY: all clean install install-user
+all: $(BUILD)/dat.dpi
+
+$(BUILD):
+	mkdir -p $@
+
+$(BUILD)/%.o: src/%.c | $(BUILD)
+	$(CC) $(CPPFLAGS) $(CFLAGS) -MMD -MP -c $< -o $@
+
+$(BUILD)/sleep_verify_lib.o: src/sleep_verify.c | $(BUILD)
+	$(CC) $(CPPFLAGS) $(CFLAGS) -DSLEEP_VERIFY_NO_MAIN -MMD -MP -c $< -o $@
+
+# S7 is used only as a parser for archive .scm metadata. Rename its optional
+# command-line entry point so the DPI owns the process main().
+$(BUILD)/s7.o: src/s7.c | $(BUILD)
+	$(CC) $(CPPFLAGS) $(CFLAGS) -w -MMD -MP -c $< -o $@
+
+$(BUILD)/dpi_s7.o: $(BUILD)/s7.o
+	$(OBJCOPY) --redefine-sym main=dat_dpi_s7_standalone_main $< $@
+
+$(BUILD)/libdatdpi.a: $(LIB_OBJECTS)
+	$(AR) rcs $@ $(LIB_OBJECTS)
+	$(RANLIB) $@
+
+$(BUILD)/dat.dpi: src/dat_dpi.c src/dat_dpi_icons.h $(BUILD)/libdatdpi.a $(BUILD)/dpi_s7.o
+	$(CC) $(CPPFLAGS) $(CFLAGS) $(LDFLAGS) src/dat_dpi.c $(BUILD)/libdatdpi.a $(BUILD)/dpi_s7.o $(LIBS) -o $@
+
+install: $(BUILD)/dat.dpi
+	$(INSTALL) -d "$(DESTDIR)$(DPIDIR)/dat"
+	$(INSTALL) -m 755 $(BUILD)/dat.dpi "$(DESTDIR)$(DPIDIR)/dat/dat.dpi"
+	$(INSTALL) -d "$(DESTDIR)$(PREFIX)/share/doc/dat-dpi"
+	$(INSTALL) -m 644 README.md "$(DESTDIR)$(PREFIX)/share/doc/dat-dpi/README.md"
+
+install-user: $(BUILD)/dat.dpi
+	$(INSTALL) -d "$(HOME)/.dillo/dpi/dat"
+	$(INSTALL) -m 755 $(BUILD)/dat.dpi "$(HOME)/.dillo/dpi/dat/dat.dpi"
+	@if ! test -f "$(HOME)/.dillo/dpidrc"; then printf '%s\n' 'dpi_dir=$(DPIDIR)' > "$(HOME)/.dillo/dpidrc"; fi
+	@if ! grep -q '^proto\.dat=' "$(HOME)/.dillo/dpidrc"; then printf '%s\n' 'proto.dat=dat/dat.dpi' >> "$(HOME)/.dillo/dpidrc"; fi
+
+clean:
+	rm -rf "$(BUILD)"
+
+-include $(DEPFILES)

extra/dat-dpi/NETWORK_PROTOCOL

@@ -0,0 +1 @@
+cesspool

extra/dat-dpi/README.md

@@ -0,0 +1,92 @@
+# DAT protocol plugin for Dillo
+
+This repository builds the standalone Dillo Protocol Interface (DPI) plugin for
+`dat://` archives. It is kept separate from the dat-thang application repository
+so it can be built, packaged, and proposed to Dillo as an independent plugin.
+
+The plugin implements the Dillo DPI server protocol and serves archive pages
+from the C implementation. Archive contents and metadata are read through SLEEP;
+`dat.scm` is parsed as S-expressions with the embedded S7 reader and is never
+evaluated as executable archive code. The networking identity is the Cesspool
+protocol token in `NETWORK_PROTOCOL`. Dat-DNS is the project's archive-name
+mapping and is not public DNS.
+
+## Build requirements
+
+- A C99 compiler, GNU Make, and binutils (`objcopy`)
+- `pkg-config`
+- libsodium and libcurl development headers and libraries
+- Dillo 3.0.5 or newer with its DPI daemon (`dpid`)
+
+On Debian-like systems, the build libraries are commonly provided by
+`libsodium-dev` and `libcurl4-openssl-dev`; package names vary by distribution.
+
+Build the plugin:
+
+```sh
+make
+```
+
+The result is `build/dat.dpi`. The build output is ignored by Git.
+
+## Install for one user
+
+```sh
+make install-user DPIDIR=/path/to/dillo/dpi
+```
+
+This installs `~/.dillo/dpi/dat/dat.dpi`. If `~/.dillo/dpidrc` does not exist, it creates one with `dpi_dir` set to the supplied `DPIDIR`; choose the system DPI directory used by your Dillo installation. It then adds this association to
+`~/.dillo/dpidrc` if it is not already present:
+
+```ini
+proto.dat=dat/dat.dpi
+```
+
+Restart Dillo's plugin daemon (`dpidc stop`, then open Dillo again), and navigate
+to a `dat://` archive address. User plugins are searched before the configured
+system DPI directory.
+
+## System install
+
+```sh
+sudo make install
+```
+
+By default, this installs to `/usr/local/lib/dillo/dpi/dat/dat.dpi`. Add the
+following line to `~/.dillo/dpidrc` (or the system `dpidrc` used by your Dillo
+installation), then restart `dpid`:
+
+```ini
+proto.dat=dat/dat.dpi
+```
+
+Override paths as needed, for example:
+
+```sh
+make install PREFIX=/usr DPIDIR=/usr/lib/dillo/dpi DESTDIR=/tmp/package-root
+```
+
+## Protocol/version behavior
+
+The plugin accepts legacy hexadecimal and compact Base32768 Dat keys. A `+N`
+version suffix selects the requested archive version, matching the versioned
+navigation supported by the C archive implementation. The plugin accesses
+archives through the configured Cesspool networking layer when content is not
+available locally.
+
+## Dillo plugin integration
+
+Dillo plugins are independent programs using the DPI protocol. Dillo searches
+for `.dpi` executables in user and system DPI directories and associates a URL
+scheme through `proto.<scheme>=<relative-plugin-path>` in `dpidrc`. See the Dillo
+User Manual's Plugins and Dpidrc sections for the browser-side configuration.
+
+The Dillo project requests plugin proposals as a repository link and short
+description. This repository is prepared for that review; it has no configured
+remote and has not been submitted upstream.
+
+## Source and license
+
+The C implementation and required headers are copied from the dat-thang source
+project. This repository carries the project's GNU AGPL v3 license. The bundled
+S7 source retains its upstream notices in `src/s7.c`.

extra/dat-dpi/inc/base32768.h

@@ -0,0 +1,17 @@
+#ifndef BASE32768_H
+#define BASE32768_H
+
+#include <stddef.h>
+#include <stdint.h>
+
+/* UTF-8 C-string form of qntm/base32768's BMP-safe Base32768 encoding. */
+int base32768_encode(const uint8_t *input, size_t input_length,
+		     char *output, size_t output_capacity);
+int base32768_decode(const char *input, uint8_t *output,
+		     size_t output_capacity, size_t *output_length);
+
+/* Enumerate the alphabet: bits=15 selects the 32,768 main characters;
+ * bits=7 selects the 128 possible final padding characters. */
+int base32768_codepoint_at(unsigned bits, uint32_t index, uint32_t *codepoint);
+
+#endif

extra/dat-dpi/inc/cable.h

@@ -0,0 +1,81 @@
+#ifndef CABLE_H
+#define CABLE_H
+
+#include <stddef.h>
+#include <stdint.h>
+#include "keys.h"
+#include "sleep_feed.h"
+
+#define CABLE_KEY_BYTES 32u
+#define CABLE_CHAT_TEXT_BYTES 768u
+#define CABLE_CHAT_WIRE_BYTES (4u + 8u + 2u + CABLE_CHAT_TEXT_BYTES)
+#define CABLE_CHAT_SEAL_BYTES (CABLE_CHAT_WIRE_BYTES + 48u)
+
+enum cable_address_scheme {
+	CABLE_ADDRESS_CABAL = 1,
+	CABLE_ADDRESS_CABLE = 2,
+	CABLE_ADDRESS_HYPER = 3
+};
+
+typedef struct cable_chat_message {
+	uint64_t timestamp_ms;
+	char text[CABLE_CHAT_TEXT_BYTES + 1u];
+	size_t text_length;
+} cable_chat_message_t;
+
+/* A Cabal user address is that user's persistent feed public key.  A room
+ * address may additionally carry a room moderation/admin key. */
+typedef struct cable_room_address {
+	uint8_t key[CABLE_KEY_BYTES];
+	uint8_t admin_key[CABLE_KEY_BYTES];
+	enum cable_address_scheme scheme;
+	int has_admin;
+} cable_room_address_t;
+
+/* The schemes share a 32-byte key representation; the caller decides whether
+ * it is a room key or a user feed key. hyper:// is a migration alias only. */
+int cable_address_parse(const char *address, uint8_t key[CABLE_KEY_BYTES],
+			enum cable_address_scheme *scheme);
+int cable_address_format(const uint8_t key[CABLE_KEY_BYTES],
+			enum cable_address_scheme scheme, char *out, size_t capacity);
+int cable_room_address_parse(const char *address, cable_room_address_t *room);
+int cable_room_address_format(const cable_room_address_t *room,
+			      enum cable_address_scheme scheme, char *out, size_t capacity);
+int cable_user_address_format(const uint8_t user_feed_key[CABLE_KEY_BYTES],
+			      enum cable_address_scheme scheme, char *out, size_t capacity);
+/* Cabal stores the local writer in a room's storage, so derive a per-room
+ * local feed-key path instead of sharing one global identity across rooms. */
+int cable_user_identity_path(const char *home,
+			     const uint8_t room_key[CABLE_KEY_BYTES],
+			     char *out, size_t capacity);
+/* Create a Cabal room signing key and persist it in the room directory. */
+int cable_room_create(const char *home, uint8_t room_key[CABLE_KEY_BYTES]);
+/* Store/load the user's local feed secret key. The public key is derived and
+ * checked on load. The file contains only the 64-byte Ed25519 secret key. */
+int cable_user_identity_load_or_create(const char *path, sleep_keypair_t *identity);
+/* Cabal stores messages as JSON blocks in each user's Hypercore feed. These
+ * helpers use the same Hypercore 7.x on-disk tree/signature layout. */
+int cable_user_feed_open(const char *home,
+			 const uint8_t room_key[CABLE_KEY_BYTES],
+			 sleep_keypair_t *identity, sleep_feed_t *feed);
+int cable_user_feed_append(const char *home,
+			   const uint8_t room_key[CABLE_KEY_BYTES],
+			   const uint8_t *record, size_t record_length,
+			   uint8_t user_key[CABLE_KEY_BYTES]);
+int cable_user_feed_publish(const char *home,
+			    const uint8_t room_key[CABLE_KEY_BYTES],
+			    const char *record_type, const char *nick,
+			    const char *channel, const char *text,
+			    uint8_t user_key[CABLE_KEY_BYTES]);
+int cable_chat_message_encode(const cable_chat_message_t *message,
+			      uint8_t *out, size_t capacity, size_t *written);
+int cable_chat_message_decode(cable_chat_message_t *message,
+			      const uint8_t *input, size_t length);
+int cable_chat_seal(const uint8_t recipient_ed25519[32],
+		    const uint8_t *message, size_t message_length,
+		    uint8_t *out, size_t capacity, size_t *written);
+int cable_chat_open(const sleep_keypair_t *identity,
+		   const uint8_t *sealed, size_t sealed_length,
+		   uint8_t *out, size_t capacity, size_t *written);
+
+#endif

extra/dat-dpi/inc/cable_media.h

@@ -0,0 +1,56 @@
+#ifndef CABLE_MEDIA_H
+#define CABLE_MEDIA_H
+
+#include <stddef.h>
+#include <stdint.h>
+
+#define CABLE_MEDIA_SESSION_BYTES 16u
+#define CABLE_MEDIA_HEADER_BYTES 42u
+#define CABLE_MEDIA_MAX_PACKET_BYTES 4096u
+#define CABLE_MEDIA_AAC_RATE 48000u
+#define CABLE_MEDIA_AAC_BITRATE 96000u
+#define CABLE_MEDIA_AAC_CHANNELS 1u
+#define CABLE_MEDIA_AAC_SAMPLES 1024u
+
+enum cable_media_kind {
+	CABLE_MEDIA_AUDIO_START = 1,
+	CABLE_MEDIA_AUDIO_FRAME = 2,
+	CABLE_MEDIA_AUDIO_STOP = 3,
+	CABLE_MEDIA_VIDEO_START = 4,
+	CABLE_MEDIA_VIDEO_FRAME = 5,
+	CABLE_MEDIA_VIDEO_STOP = 6
+};
+
+typedef struct cable_media_packet {
+	uint8_t kind;
+	uint8_t session[CABLE_MEDIA_SESSION_BYTES];
+	uint64_t sequence;
+	uint64_t timestamp_us;
+	const uint8_t *payload;
+	size_t payload_length;
+} cable_media_packet_t;
+
+int cable_media_packet_write(const cable_media_packet_t *packet,
+			     uint8_t *output, size_t capacity, size_t *written);
+int cable_media_packet_read(cable_media_packet_t *packet,
+			    const uint8_t *input, size_t length);
+
+typedef struct cable_aac_encoder { void *implementation; } cable_aac_encoder_t;
+typedef struct cable_aac_decoder { void *implementation; } cable_aac_decoder_t;
+typedef int (*cable_aac_frame_fn)(void *context, const uint8_t *adts,
+				  size_t length);
+typedef int (*cable_pcm_samples_fn)(void *context, const int16_t *samples,
+				    size_t sample_count);
+
+int cable_aac_encoder_open(cable_aac_encoder_t *encoder);
+int cable_aac_encoder_frame(cable_aac_encoder_t *encoder,
+			    const int16_t samples[CABLE_MEDIA_AAC_SAMPLES],
+			    cable_aac_frame_fn output, void *context);
+void cable_aac_encoder_close(cable_aac_encoder_t *encoder);
+int cable_aac_decoder_open(cable_aac_decoder_t *decoder);
+int cable_aac_decoder_frame(cable_aac_decoder_t *decoder,
+			    const uint8_t *adts, size_t length,
+			    cable_pcm_samples_fn output, void *context);
+void cable_aac_decoder_close(cable_aac_decoder_t *decoder);
+
+#endif

extra/dat-dpi/inc/cable_room.h

@@ -0,0 +1,30 @@
+#ifndef CABLE_ROOM_H
+#define CABLE_ROOM_H
+
+#include "cable.h"
+
+#define CABLE_ROOM_CHANNEL_BYTES 64u
+#define CABLE_ROOM_NICK_BYTES 64u
+
+enum cable_room_event_type {
+	CABLE_ROOM_PEER_JOIN = 1,
+	CABLE_ROOM_PEER_LEAVE = 2,
+	CABLE_ROOM_NICK = 3,
+	CABLE_ROOM_CHANNEL_JOIN = 4,
+	CABLE_ROOM_TEXT = 5
+};
+
+typedef struct cable_room_event {
+	uint8_t type;
+	uint64_t timestamp_ms;
+	uint8_t sender[CABLE_KEY_BYTES];
+	char channel[CABLE_ROOM_CHANNEL_BYTES + 1u];
+	char nick[CABLE_ROOM_NICK_BYTES + 1u];
+	char text[CABLE_CHAT_TEXT_BYTES + 1u];
+	size_t text_length;
+} cable_room_event_t;
+
+/* In-memory UI event only. Cable's network records are Cabal Core Hypercore
+ * values and never serialize this C structure. */
+
+#endif

extra/dat-dpi/inc/cable_video.h

@@ -0,0 +1,22 @@
+#ifndef CABLE_VIDEO_H
+#define CABLE_VIDEO_H
+
+#include <stddef.h>
+#include <stdint.h>
+
+typedef struct cable_video_source cable_video_source_t;
+
+/* Read an OBS MPEG-TS feed from a local UDP socket, normally port 5000.
+ * Pixels are copied as RGB24; a generation change means a new frame arrived. */
+cable_video_source_t *cable_video_source_open(const char *url);
+void cable_video_source_close(cable_video_source_t *source);
+int cable_video_source_frame(cable_video_source_t *source, uint8_t *rgb,
+			     size_t capacity, unsigned *width, unsigned *height,
+			     uint64_t *generation);
+/* Encode/decode one low-resolution JPEG frame for volatile Cable media. */
+int cable_video_encode_jpeg(const uint8_t *rgb, unsigned width, unsigned height,
+			    uint8_t *jpeg, size_t capacity, size_t *written);
+int cable_video_decode_jpeg(const uint8_t *jpeg, size_t length, uint8_t *rgb,
+			    size_t capacity, unsigned *width, unsigned *height);
+
+#endif

extra/dat-dpi/inc/cesspool.h

@@ -0,0 +1,172 @@
+#ifndef CESSPOOL_H
+#define CESSPOOL_H
+
+#include <stddef.h>
+#include <stdint.h>
+#include "keys.h"
+#include "dat_net.h"
+
+// bootstrap node
+// dht
+// discovery
+// lookup (key) ↳ lookup1
+// announce
+// ping
+// holepunch udp
+
+#define CESSPOOL_TOPIC_BYTES 32u
+#define CESSPOOL_PEER_BYTES 32u
+
+enum cesspool_message_type {
+	CESSPOOL_LOOKUP = 0,
+	CESSPOOL_ANNOUNCE = 1,
+	CESSPOOL_UNANNOUNCE = 2,
+	CESSPOOL_REPLY = 3,
+	CESSPOOL_PING = 4,
+	CESSPOOL_PONG = 5,
+	CESSPOOL_HOLEPUNCH = 6
+};
+
+typedef struct cesspool_message {
+	uint8_t type;
+	uint8_t topic[CESSPOOL_TOPIC_BYTES];
+	uint8_t peer_id[CESSPOOL_PEER_BYTES];
+	uint16_t port;
+	uint8_t payload[2048];
+	size_t payload_len;
+} cesspool_message_t;
+typedef struct cesspool_request {
+	uint32_t type;
+	uint16_t port;
+	uint8_t local_address[6];
+	size_t local_address_len;
+} cesspool_request_t;
+typedef struct cesspool_response {
+	uint8_t peers[1024];
+	size_t peers_len;
+	uint8_t local_peers[1024];
+	size_t local_peers_len;
+	/* Cesspool rendezvous records: IPv4(4), UDP port(2), peer key(32). */
+	uint8_t peer_info[2048];
+	size_t peer_info_len;
+} cesspool_response_t;
+
+#define CESSPOOL_MAX_BOOTSTRAP 16u
+#define CESSPOOL_MAX_PEERS 256u
+#define CESSPOOL_MAX_TOPICS 64u
+#define CESSPOOL_MAX_ANNOUNCEMENTS 1024u
+#define CESSPOOL_MAX_TRANSACTIONS 64u
+typedef struct cesspool_peer {
+	dat_peer_t address;
+	uint8_t public_key[32];
+	uint8_t discovery_topic[CESSPOOL_TOPIC_BYTES];
+	uint64_t last_seen;
+	unsigned failures;
+	int local;
+	int discovered;
+} cesspool_peer_t;
+typedef struct cesspool_topic {
+	uint8_t key[32];
+	uint16_t port;
+	int announcing;
+	int looking_up;
+	uint64_t refresh_at;
+} cesspool_topic_t;
+typedef struct cesspool_transaction {
+	uint16_t id;
+	uint8_t target[32];
+	uint8_t type;
+	uint64_t expires_at;
+	int active;
+} cesspool_transaction_t;
+typedef struct cesspool_announcement {
+	uint8_t topic[CESSPOOL_TOPIC_BYTES];
+	uint8_t peer_id[CESSPOOL_PEER_BYTES];
+	dat_peer_t address;
+	uint64_t last_seen;
+} cesspool_announcement_t;
+struct cesspool;
+typedef void (*cesspool_transport_packet_fn)(struct cesspool *pool,
+						 const dat_peer_t *source,
+						 const uint8_t *packet,
+						 size_t packet_len);
+typedef struct cesspool {
+	datnet_t net;
+	dat_peer_t bootstrap_nodes[CESSPOOL_MAX_BOOTSTRAP];
+	size_t bootstrap_count;
+	uint16_t port;
+	int ready;
+	int bootstrap_server;
+	cesspool_peer_t peers[CESSPOOL_MAX_PEERS];
+	size_t peer_count;
+	cesspool_topic_t topics[CESSPOOL_MAX_TOPICS];
+	size_t topic_count;
+	/* A peer may announce many topics; keep each membership independently. */
+	cesspool_announcement_t announcements[CESSPOOL_MAX_ANNOUNCEMENTS];
+	size_t announcement_count;
+	uint64_t bootstrap_last_seen;
+	uint64_t bootstrap_probe_at;
+	int bootstrap_reachable;
+	cesspool_transaction_t transactions[CESSPOOL_MAX_TRANSACTIONS];
+	uint16_t next_transaction;
+	void (*on_peer)(struct cesspool *, const cesspool_peer_t *);
+	void (*on_holepunch)(struct cesspool *, const dat_peer_t *);
+	cesspool_transport_packet_fn on_transport_packet;
+	void *transport_context;
+} cesspool_t;
+
+int cesspool_discovery_key(const uint8_t public_key[32], uint8_t topic[32]);
+int cesspool_encode(const cesspool_message_t *message, uint8_t *out,
+		    size_t capacity, size_t *written);
+int cesspool_decode(cesspool_message_t *message, const uint8_t *input,
+		    size_t input_len);
+int cesspool_request_encode(const cesspool_request_t *request, uint8_t *out,
+			    size_t capacity, size_t *written);
+int cesspool_request_decode(cesspool_request_t *request, const uint8_t *input,
+			    size_t input_len);
+int cesspool_response_encode(const cesspool_response_t *response, uint8_t *out,
+			     size_t capacity, size_t *written);
+int cesspool_response_decode(cesspool_response_t *response, const uint8_t *input,
+			     size_t input_len);
+int cesspool_process_response(cesspool_t *pool,
+			      const cesspool_response_t *response,
+			      const uint8_t local_prefix[2]);
+int cesspool_init(cesspool_t *pool, const uint8_t peer_id[32]);
+int cesspool_add_bootstrap(cesspool_t *pool, const char *host, uint16_t port);
+/* Load Cesspool bootstrap endpoints from DDAT_BOOTSTRAP and the per-user
+ * config/dat-gpt/bootstrap file. */
+int cesspool_configure_bootstraps(cesspool_t *pool);
+int cesspool_bootstrap_config_add(const char *endpoint);
+int cesspool_bootstrap_config_set(const char *endpoint);
+int cesspool_bootstrap_config_get(char *endpoint, size_t capacity);
+int cesspool_bootstrap_config_list(char *contents, size_t capacity);
+int cesspool_bootstrap(cesspool_t *pool);
+int cesspool_listen(cesspool_t *pool, const char *host, uint16_t port);
+int cesspool_set_bootstrap_server(cesspool_t *pool, int enabled);
+int cesspool_announce(cesspool_t *pool, const uint8_t topic[32], uint16_t port);
+int cesspool_lookup(cesspool_t *pool, const uint8_t topic[32]);
+int cesspool_unannounce(cesspool_t *pool, const uint8_t topic[32], uint16_t port);
+int cesspool_ping(cesspool_t *pool, const dat_peer_t *peer);
+int cesspool_holepunch(cesspool_t *pool, const dat_peer_t *peer);
+int cesspool_poll(cesspool_t *pool, int timeout_ms);
+int cesspool_add_peer(cesspool_t *pool, const cesspool_peer_t *peer);
+int cesspool_remove_peer(cesspool_t *pool, const uint8_t public_key[32]);
+cesspool_peer_t *cesspool_find_peer(cesspool_t *pool, const uint8_t public_key[32]);
+int cesspool_join(cesspool_t *pool, const uint8_t topic[32], uint16_t port,
+		  int announce, int lookup);
+int cesspool_leave(cesspool_t *pool, const uint8_t topic[32]);
+int cesspool_tick(cesspool_t *pool, uint64_t now_ms);
+size_t cesspool_closest(cesspool_t *pool, const uint8_t target[32],
+			cesspool_peer_t **out, size_t capacity);
+size_t cesspool_topic_peers(cesspool_t *pool,
+			    const uint8_t topic[CESSPOOL_TOPIC_BYTES],
+			    cesspool_peer_t **out, size_t capacity);
+size_t cesspool_announced_peer_count(const cesspool_t *pool);
+size_t cesspool_announced_topic_count(const cesspool_t *pool);
+int cesspool_transaction_begin(cesspool_t *pool, uint8_t type,
+			       const uint8_t target[32], uint64_t expires_at,
+			       uint16_t *transaction_id);
+int cesspool_transaction_finish(cesspool_t *pool, uint16_t transaction_id);
+int cesspool_destroy(cesspool_t *pool);
+
+#endif /* CESSPOOL_H */

extra/dat-dpi/inc/dat.h

@@ -0,0 +1,114 @@
+#ifndef DAT_H
+#define DAT_H
+
+#include <stdint.h>
+#include <stddef.h>
+#include "keys.h"
+#include "sleep_proto.h"
+
+#define DAT_KEY_BYTES 32u
+#define DAT_BLOCK_BYTES (64u * 1024u)
+#define DAT_URL_BYTES 71u
+#define DAT_LINK_BYTES 4096u
+
+typedef struct dat_archive {
+	char dat_path[1024];
+	uint8_t metadata_key[DAT_KEY_BYTES];
+	uint8_t content_key[DAT_KEY_BYTES];
+	sleep_keypair_t metadata_keys;
+	sleep_keypair_t content_keys;
+} dat_archive_t;
+typedef struct dat_link {
+	uint8_t key[DAT_KEY_BYTES];
+	uint64_t version;
+	int has_version;
+	char path[2048];
+} dat_link_t;
+typedef struct dat_checkout {
+	const dat_archive_t *archive;
+	uint64_t version;
+} dat_checkout_t;
+typedef struct dat_url {
+	char host[256];
+	char path[2048];
+	uint64_t version;
+	int has_version;
+	int wants_stat;
+	int force_directory_listing;
+} dat_url_t;
+typedef struct dat_path_info {
+	sleep_metadata_entry_t entry;
+	int exists;
+	int is_directory;
+} dat_path_info_t;
+typedef int (*dat_archive_change_fn)(const sleep_metadata_entry_t *entry,
+					     uint64_t version, void *context);
+typedef int (*dat_archive_readdir_fn)(const sleep_metadata_entry_t *entry,
+				      const char *name, void *context);
+
+/* Creates a new two-feed legacy-style archive skeleton under root/.dat. */
+int dat_archive_create(dat_archive_t *archive, const char *root);
+int dat_archive_open(dat_archive_t *archive, const char *root);
+int dat_archive_open_writable(dat_archive_t *archive, const char *root,
+			      const char *secret_directory);
+int dat_archive_save_secrets(const dat_archive_t *archive,
+			     const char *secret_directory);
+void dat_archive_close(dat_archive_t *archive);
+size_t dat_archive_entry_count(const dat_archive_t *archive);
+uint64_t dat_archive_version(const dat_archive_t *archive);
+int dat_archive_append_file(dat_archive_t *archive, const char *path,
+			    const uint8_t *data, size_t length,
+			    uint32_t mode, uint64_t mtime);
+int dat_archive_append_directory(dat_archive_t *archive, const char *path,
+				 uint32_t mode, uint64_t mtime);
+int dat_archive_append_symlink(dat_archive_t *archive, const char *path,
+			       const char *target, uint64_t mtime);
+int dat_archive_delete(dat_archive_t *archive, const char *path, uint64_t mtime);
+int dat_archive_read_entry(const dat_archive_t *archive, size_t index,
+			   sleep_metadata_entry_t *entry);
+int dat_archive_read_file(const dat_archive_t *archive,
+			  const sleep_metadata_entry_t *entry,
+			  uint8_t **data, size_t *length);
+int dat_archive_find(const dat_archive_t *archive, const char *path,
+		     sleep_metadata_entry_t *entry);
+int dat_archive_find_at(const dat_archive_t *archive, const char *path,
+			uint64_t version, sleep_metadata_entry_t *entry);
+int dat_checkout_init(dat_checkout_t *checkout, const dat_archive_t *archive,
+		      uint64_t version);
+int dat_checkout_find(const dat_checkout_t *checkout, const char *path,
+		      sleep_metadata_entry_t *entry);
+int dat_checkout_read(const dat_checkout_t *checkout,
+		      const sleep_metadata_entry_t *entry,
+		      uint8_t **data, size_t *length);
+int dat_checkout_export(const dat_checkout_t *checkout, const char *archive_path,
+			const char *filesystem_path);
+int dat_archive_changes(const dat_archive_t *archive, uint64_t from_version,
+		uint64_t to_version, dat_archive_change_fn callback, void *context);
+int dat_archive_stat_at(const dat_archive_t *archive, uint64_t version,
+			const char *path, dat_path_info_t *info);
+int dat_archive_readdir_at(const dat_archive_t *archive, uint64_t version,
+			   const char *path, dat_archive_readdir_fn callback,
+			   void *context);
+int dat_archive_resolve_at(const dat_archive_t *archive, uint64_t version,
+			   const char *path, int force_directory_listing,
+			   dat_path_info_t *info, int *serve_directory_listing);
+const char *dat_content_type(const char *path);
+int dat_url_parse(const char *text, dat_url_t *url);
+int dat_archive_import_path(dat_archive_t *archive, const char *filesystem_path,
+			    const char *archive_path);
+/* Imports only changed paths and records tombstones for paths removed locally. */
+int dat_archive_sync_path(dat_archive_t *archive, const char *filesystem_path,
+			  const char *archive_path);
+int dat_archive_export_path(const dat_archive_t *archive, const char *archive_path,
+			    const char *filesystem_path);
+int dat_archive_export_at(const dat_archive_t *archive, uint64_t version,
+			  const char *archive_path, const char *filesystem_path);
+int dat_key_to_url(const uint8_t key[DAT_KEY_BYTES], char url[DAT_URL_BYTES]);
+int dat_key_to_compact_url(const uint8_t key[DAT_KEY_BYTES], char url[DAT_URL_BYTES]);
+int dat_key_to_hex(const uint8_t key[DAT_KEY_BYTES], char hex[65]);
+int dat_url_to_key(const char *url, uint8_t key[DAT_KEY_BYTES]);
+int dat_link_format(const dat_link_t *link, char *out, size_t capacity);
+int dat_link_format_compact(const dat_link_t *link, char *out, size_t capacity);
+int dat_link_parse(const char *text, dat_link_t *link);
+
+#endif /* DAT_H */

extra/dat-dpi/inc/dat_dns.h

@@ -0,0 +1,27 @@
+#ifndef DAT_DNS_H
+#define DAT_DNS_H
+
+#include <stdint.h>
+#include "dat.h"
+
+typedef int (*dat_dns_name_iter_fn)(const char *hostname,
+				    const uint8_t key[DAT_KEY_BYTES], void *context);
+
+/* Resolve a legacy Dat DNS TXT record such as datkey=<64 hex digits>. */
+int dat_dns_resolve(const char *hostname, uint8_t key[DAT_KEY_BYTES],
+		    uint32_t *ttl_seconds);
+
+/* Local dat-dns aliases are stored in $HOME/.dat/dat-dns (or USERPROFILE on
+ * native Windows) and override DNS TXT. */
+int dat_dns_name_set(const char *hostname, const uint8_t key[DAT_KEY_BYTES]);
+int dat_dns_name_remove(const char *hostname);
+int dat_dns_name_resolve(const char *hostname, uint8_t key[DAT_KEY_BYTES]);
+int dat_dns_name_for_key(const uint8_t key[DAT_KEY_BYTES], char *hostname,
+			 size_t hostname_capacity);
+int dat_dns_name_foreach(dat_dns_name_iter_fn callback, void *context);
+
+/* Decode TXT record character strings from one DNS answer packet. */
+int dat_dns_parse_answer(const uint8_t *packet, unsigned packet_len,
+			uint8_t key[DAT_KEY_BYTES], uint32_t *ttl_seconds);
+
+#endif

extra/dat-dpi/inc/dat_fetch.h

@@ -0,0 +1,12 @@
+#ifndef DAT_FETCH_H
+#define DAT_FETCH_H
+
+#include <stdint.h>
+
+/* Fetch the public metadata and content feeds for a Dat key into root/.dat.
+ * The fetch uses the configured Cesspool network and persists verified sparse
+ * feed blocks so a later Dillo request can resume it. */
+int dat_fetch_archive(const char *root, const uint8_t metadata_key[32],
+		      unsigned timeout_ms);
+
+#endif

extra/dat-dpi/inc/dat_net.h

@@ -0,0 +1,36 @@
+#ifndef DAT_NET_H
+#define DAT_NET_H
+
+#include <stdint.h>
+#include <stddef.h>
+#include "dat_socket.h"
+
+#define DATNET_PORT 3282
+#define DATNET_MAX_PACKET 1400
+#define DATNET_TOPIC_BYTES 32
+#define DATNET_PEERID_BYTES 32
+
+typedef struct {
+	dat_socket_t fd;
+	struct sockaddr_storage addr;
+	socklen_t addrlen;
+	uint8_t id[DATNET_PEERID_BYTES];
+} dat_peer_t;
+
+typedef struct datnet {
+	dat_socket_t sock;
+	uint8_t topic[DATNET_TOPIC_BYTES];
+	uint8_t peerid[DATNET_PEERID_BYTES];
+	void *context;
+	void (*on_message)(struct datnet *net, dat_peer_t *peer, const uint8_t *msg, size_t len);
+} datnet_t;
+
+int datnet_init(datnet_t *net, const uint8_t topic[DATNET_TOPIC_BYTES],
+		const uint8_t peerid[DATNET_PEERID_BYTES]);
+int datnet_bind(datnet_t *net, const char *host, uint16_t port);
+int datnet_poll(datnet_t *net, int timeout_ms);
+int datnet_send(datnet_t *net, const dat_peer_t *peer,
+		const uint8_t *message, size_t length);
+int datnet_close(datnet_t *net);
+
+#endif /* DAT_NET_H */

extra/dat-dpi/inc/dat_paths.h

@@ -0,0 +1,48 @@
+#ifndef DAT_PATHS_H
+#define DAT_PATHS_H
+
+#include <errno.h>
+#include <stdio.h>
+#include <stdlib.h>
+#ifdef _WIN32
+#include <windows.h>
+#endif
+
+/* Return the user's home directory across POSIX shells and native Windows
+ * launches. Windows GUI starts commonly have USERPROFILE but no HOME. */
+static inline const char *dat_home_directory(void)
+{
+	const char *home = getenv("HOME");
+	if (home && *home) return home;
+#ifdef _WIN32
+	home = getenv("USERPROFILE");
+	if (home && *home) return home;
+	{
+		const char *drive = getenv("HOMEDRIVE");
+		const char *path = getenv("HOMEPATH");
+		static char combined[1024];
+		int n;
+		if (!drive || !*drive || !path || !*path) return NULL;
+		n = snprintf(combined, sizeof(combined), "%s%s", drive, path);
+		return n >= 0 && (size_t)n < sizeof(combined) ? combined : NULL;
+	}
+#else
+	return NULL;
+#endif
+}
+
+/* POSIX rename replaces an existing file; Windows rename does not. These
+ * state files are written to a sibling temporary file before replacement. */
+static inline int dat_replace_file(const char *temporary, const char *destination)
+{
+#ifdef _WIN32
+	if (MoveFileExA(temporary, destination,
+			MOVEFILE_REPLACE_EXISTING | MOVEFILE_WRITE_THROUGH)) return 0;
+	errno = EIO;
+	return -1;
+#else
+	return rename(temporary, destination);
+#endif
+}
+
+#endif

extra/dat-dpi/inc/dat_s7.h

@@ -0,0 +1,16 @@
+#ifndef DAT_S7_H
+#define DAT_S7_H
+#include "s7.h"
+#include "dat.h"
+typedef struct dat_scheme_runtime {
+	s7_scheme *scheme;
+} dat_scheme_runtime_t;
+
+int dat_s7_init(s7_scheme *scheme);
+int dat_scheme_runtime_init(dat_scheme_runtime_t *runtime);
+void dat_scheme_runtime_close(dat_scheme_runtime_t *runtime);
+int dat_scheme_write_archive_info(s7_scheme *scheme, const char *root,
+				  const dat_archive_t *archive,
+				  const char *title, const char *description,
+				  int replace);
+#endif

extra/dat-dpi/inc/dat_session.h

@@ -0,0 +1,151 @@
+#ifndef DAT_SESSION_H
+#define DAT_SESSION_H
+
+#include <stddef.h>
+#include <stdint.h>
+#include "cesspool.h"
+#include "dat.h"
+#include "dat_wire.h"
+#include "sleep_feed.h"
+#include "sleep_replication.h"
+
+#define DAT_SESSION_MAX_PEERS 64u
+#define DAT_SESSION_CAP_METADATA_ONLY (UINT64_C(1) << 63)
+
+enum dat_session_state {
+	DAT_SESSION_CLOSED = 0,
+	DAT_SESSION_OPEN = 1,
+	DAT_SESSION_DISCOVERING = 2,
+	DAT_SESSION_ACTIVE = 3,
+	DAT_SESSION_CLOSING = 4,
+	DAT_SESSION_FAILED = 5
+};
+
+enum dat_session_peer_state {
+	DAT_SESSION_PEER_DISCOVERED = 1,
+	DAT_SESSION_PEER_SECURE = 2,
+	DAT_SESSION_PEER_READY = 3,
+	DAT_SESSION_PEER_CLOSED = 4
+};
+
+enum dat_session_feed {
+	DAT_SESSION_METADATA = 0,
+	DAT_SESSION_CONTENT = 1,
+	DAT_SESSION_FEED_COUNT = 2
+};
+
+typedef struct dat_session_peer {
+	uint8_t peer_id[DAT_PEER_ID_BYTES];
+	uint8_t tx_nonce[24];
+	uint8_t rx_nonce[24];
+	uint8_t *rx_buffer;
+	size_t rx_buffer_len;
+	size_t rx_buffer_capacity;
+	uint64_t tx_offset;
+	uint64_t rx_offset;
+	uint8_t remote_feed_mask;
+	uint8_t local_feed_mask;
+	uint8_t remote_content_discovery[32];
+	int remote_content_pending;
+	dat_peer_t address;
+	uint64_t capabilities;
+	uint64_t payload_bytes_sent;
+	uint64_t payload_bytes_received;
+	int state;
+	int transport_authenticated;
+	int paused;
+	int tx_stream_encrypted;
+	int rx_stream_encrypted;
+	sleep_replication_t replication[DAT_SESSION_FEED_COUNT];
+} dat_session_peer_t;
+
+struct dat_session;
+typedef int (*dat_session_send_fn)(void *context,
+				   const uint8_t peer_id[DAT_PEER_ID_BYTES],
+				   const uint8_t *frame, size_t frame_len);
+typedef int (*dat_session_request_fn)(void *context,
+				      const uint8_t peer_id[DAT_PEER_ID_BYTES],
+				      enum dat_session_feed feed,
+				      const dat_request_t *request);
+/* Called when discovery yields a peer; establish an authenticated transport,
+ * then report success with dat_session_peer_secure(). */
+typedef void (*dat_session_peer_fn)(void *context,
+				    const dat_session_peer_t *peer);
+typedef int (*dat_session_poll_fn)(void *context, int timeout_ms);
+
+/*
+ * Session protocol state is separate from connection establishment. In the
+ * Cesspool path, call peer_secure only after the authenticated transport has
+ * bound this peer ID to the topic. In legacy Hypercore V6 mode, call it once
+ * the discovered TCP/uTP stream is connected; V6 performs its feed-key stream
+ * encryption inside the protocol itself.
+ */
+typedef struct dat_session {
+	int state;
+	int announcing;
+	int looking_up;
+	uint16_t port;
+	uint8_t archive_key[DAT_KEY_BYTES];
+	uint8_t discovery_topic[CESSPOOL_TOPIC_BYTES];
+	uint8_t peer_id[DAT_PEER_ID_BYTES];
+	uint8_t feed_keys[DAT_SESSION_FEED_COUNT][DAT_KEY_BYTES];
+	int content_key_known;
+	cesspool_t *cesspool;
+	sleep_feed_t *feeds[DAT_SESSION_FEED_COUNT];
+	dat_session_peer_t peers[DAT_SESSION_MAX_PEERS];
+	size_t peer_count;
+	dat_session_send_fn send;
+	dat_session_request_fn on_request;
+	dat_session_peer_fn on_peer;
+	void *peer_context;
+	dat_session_poll_fn poll_transport;
+	void *poll_context;
+	void *context;
+} dat_session_t;
+
+int dat_session_init(dat_session_t *session, cesspool_t *cesspool,
+		     const uint8_t archive_key[DAT_KEY_BYTES],
+		     const uint8_t peer_id[DAT_PEER_ID_BYTES], uint16_t port,
+		     sleep_feed_t *metadata_feed,
+		     const uint8_t metadata_key[DAT_KEY_BYTES],
+		     sleep_feed_t *content_feed,
+		     const uint8_t content_key[DAT_KEY_BYTES],
+		     dat_session_send_fn send, void *context);
+int dat_session_start(dat_session_t *session, int announce, int lookup);
+int dat_session_set_peer_callback(dat_session_t *session,
+				  dat_session_peer_fn callback, void *context);
+int dat_session_set_transport_poll(dat_session_t *session,
+				   dat_session_poll_fn poll, void *context);
+int dat_session_peer_discovered(dat_session_t *session,
+				const cesspool_peer_t *discovered_peer);
+int dat_session_peer_secure(dat_session_t *session,
+			    const uint8_t peer_id[DAT_PEER_ID_BYTES],
+			    int transport_authenticated);
+/* For a metadata-only archive import, install the content key learned from
+ * the verified metadata header and announce that feed to connected peers. */
+int dat_session_set_content_key(dat_session_t *session,
+				const uint8_t content_key[DAT_KEY_BYTES]);
+int dat_session_expect_content_key(dat_session_t *session);
+int dat_session_peer_down(dat_session_t *session,
+			  const uint8_t peer_id[DAT_PEER_ID_BYTES]);
+/* Pause this client's incoming block requests without dropping the peer. */
+int dat_session_peer_pause(dat_session_t *session,
+			   const uint8_t peer_id[DAT_PEER_ID_BYTES], int paused);
+/* Accept arbitrary transport chunks. Partial and coalesced wire frames are buffered. */
+int dat_session_receive(dat_session_t *session,
+			const uint8_t peer_id[DAT_PEER_ID_BYTES],
+			const uint8_t *frame, size_t frame_len);
+int dat_session_request_block(dat_session_t *session,
+			      const uint8_t peer_id[DAT_PEER_ID_BYTES],
+			      enum dat_session_feed feed, uint64_t index,
+			      uint64_t bytes);
+int dat_session_send_block(dat_session_t *session,
+			   const uint8_t peer_id[DAT_PEER_ID_BYTES],
+			   enum dat_session_feed feed, uint64_t index);
+int dat_session_tick(dat_session_t *session, uint64_t now_ms);
+int dat_session_poll(dat_session_t *session, int timeout_ms, uint64_t now_ms);
+int dat_session_close(dat_session_t *session);
+dat_session_peer_t *dat_session_find_peer(dat_session_t *session,
+					  const uint8_t peer_id[DAT_PEER_ID_BYTES]);
+
+#endif /* DAT_SESSION_H */

extra/dat-dpi/inc/dat_socket.h

@@ -0,0 +1,18 @@
+#ifndef DAT_SOCKET_H
+#define DAT_SOCKET_H
+
+#ifdef _WIN32
+#ifndef WIN32_LEAN_AND_MEAN
+#define WIN32_LEAN_AND_MEAN
+#endif
+#include <winsock2.h>
+#include <ws2tcpip.h>
+typedef SOCKET dat_socket_t;
+#define DAT_INVALID_SOCKET INVALID_SOCKET
+#else
+#include <sys/socket.h>
+typedef int dat_socket_t;
+#define DAT_INVALID_SOCKET (-1)
+#endif
+
+#endif

extra/dat-dpi/inc/dat_wire.h

@@ -0,0 +1,110 @@
+#ifndef DAT_WIRE_H
+#define DAT_WIRE_H
+#include <stddef.h>
+#include <stdint.h>
+
+enum dat_message_type {
+	DAT_MSG_FEED = 0,
+	DAT_MSG_HANDSHAKE = 1,
+	DAT_MSG_INFO = 2,
+	DAT_MSG_HAVE = 3,
+	DAT_MSG_UNHAVE = 4,
+	DAT_MSG_WANT = 5,
+	DAT_MSG_UNWANT = 6,
+	DAT_MSG_REQUEST = 7,
+	DAT_MSG_CANCEL = 8,
+	DAT_MSG_DATA = 9,
+	DAT_MSG_EXTENSION = 15
+};
+
+typedef struct dat_feed_open {
+	uint8_t discovery_key[32];
+	uint8_t nonce[24];
+	int has_nonce;
+} dat_feed_open_t;
+
+typedef struct dat_wire_message {
+	uint64_t channel;
+	uint64_t type;
+	const uint8_t *payload;
+	size_t payload_len;
+} dat_wire_message_t;
+
+#define DAT_PROTOCOL_BYTES 32u
+#define DAT_PEER_ID_BYTES 32u
+#ifndef DAT_NETWORK_PROTOCOL
+#define DAT_NETWORK_PROTOCOL "cesspool"
+#endif
+/*
+ * The exact protocol token lives in the repository-root NETWORK_PROTOCOL
+ * file. The Cesspool handshake carries it in `userData`; legacy Hypercore V6
+ * omits that application-owned field and uses the token to select its
+ * discovery namespace and wire mode. This checkout defaults to `cesspool`.
+ */
+typedef struct dat_handshake {
+	uint8_t protocol[DAT_PROTOCOL_BYTES];
+	uint8_t peer_id[DAT_PEER_ID_BYTES];
+	uint64_t capabilities;
+	int has_peer_id;
+} dat_handshake_t;
+int dat_handshake_init(dat_handshake_t *handshake,
+		       const uint8_t peer_id[DAT_PEER_ID_BYTES],
+		       uint64_t capabilities);
+int dat_handshake_protocol_matches(const dat_handshake_t *handshake);
+typedef struct dat_range { uint64_t start; uint64_t length; } dat_range_t;
+typedef struct dat_info { uint8_t uploading; uint8_t downloading; } dat_info_t;
+typedef struct dat_request { uint64_t index; uint64_t bytes; uint8_t want_hash; uint8_t want_nodes; } dat_request_t;
+#define DAT_DATA_MAX_NODES 128u
+typedef struct dat_data_node {
+	uint64_t index;
+	uint64_t length;
+	uint8_t hash[32];
+} dat_data_node_t;
+typedef struct dat_data {
+	uint64_t index;
+	const uint8_t *value;
+	size_t value_len;
+	const uint8_t *proof;
+	size_t proof_len;
+	dat_data_node_t nodes[DAT_DATA_MAX_NODES];
+	size_t node_count;
+	uint8_t signature[64];
+	int has_signature;
+} dat_data_t;
+typedef struct dat_cancel { uint64_t index; } dat_cancel_t;
+typedef struct dat_extension {
+	uint64_t id;
+	const uint8_t *message;
+	size_t message_len;
+} dat_extension_t;
+
+/* Hypercore frames are length-prefixed; headers pack channel and type. */
+int dat_wire_encode(const dat_wire_message_t *message, uint8_t *out,
+		    size_t capacity, size_t *written);
+int dat_wire_decode(dat_wire_message_t *message, const uint8_t *input,
+		    size_t input_len, size_t *consumed);
+int dat_handshake_encode(const dat_handshake_t *handshake, uint8_t *out,
+			 size_t capacity, size_t *written);
+int dat_handshake_decode(dat_handshake_t *handshake, const uint8_t *input,
+			 size_t input_len);
+int dat_feed_open_encode(const dat_feed_open_t *feed, uint8_t *out,
+			 size_t capacity, size_t *written);
+int dat_feed_open_decode(dat_feed_open_t *feed, const uint8_t *input,
+			 size_t input_len);
+int dat_range_encode(const dat_range_t *range, uint8_t *out, size_t capacity, size_t *written);
+int dat_range_decode(dat_range_t *range, const uint8_t *input, size_t input_len);
+/* WANT omits length to mean through the current end of a non-live feed. */
+int dat_want_decode(dat_range_t *range, const uint8_t *input, size_t input_len,
+		    uint64_t feed_length);
+int dat_info_encode(const dat_info_t *info, uint8_t *out, size_t capacity,
+		    size_t *written);
+int dat_info_decode(dat_info_t *info, const uint8_t *input, size_t input_len);
+int dat_request_encode(const dat_request_t *request, uint8_t *out, size_t capacity, size_t *written);
+int dat_request_decode(dat_request_t *request, const uint8_t *input, size_t input_len);
+int dat_data_encode(const dat_data_t *data, uint8_t *out, size_t capacity, size_t *written);
+int dat_data_decode(dat_data_t *data, const uint8_t *input, size_t input_len);
+int dat_cancel_encode(const dat_cancel_t *cancel, uint8_t *out, size_t capacity, size_t *written);
+int dat_cancel_decode(dat_cancel_t *cancel, const uint8_t *input, size_t input_len);
+int dat_extension_encode(const dat_extension_t *extension, uint8_t *out, size_t capacity, size_t *written);
+int dat_extension_decode(dat_extension_t *extension, const uint8_t *input, size_t input_len);
+#endif

extra/dat-dpi/inc/errno_compat.h

@@ -0,0 +1,12 @@
+#ifndef DATGPT_ERRNO_COMPAT_H
+#define DATGPT_ERRNO_COMPAT_H
+
+#include <errno.h>
+
+/* EKEYREJECTED is a Linux extension. Keep the key-authentication failure
+ * portable while preserving a useful access-denied error on other hosts. */
+#ifndef EKEYREJECTED
+#define EKEYREJECTED EACCES
+#endif
+
+#endif

extra/dat-dpi/inc/keys.h

@@ -0,0 +1,17 @@
+#ifndef KEYS_H
+#define KEYS_H
+
+#include <stdint.h>
+
+#define SLEEP_PUB_KEY_BYTES	32
+#define SLEEP_SECRET_KEY_BYTES	64
+#define SLEEP_SIGNATURE_BYTES	64
+#define SLEEP_BLAKE2B_BYTES	32
+
+typedef struct {
+	uint8_t pubkey[SLEEP_PUB_KEY_BYTES];
+	uint8_t seckey[SLEEP_SECRET_KEY_BYTES];
+} sleep_keypair_t;
+
+
+#endif /* KEYS_H */

extra/dat-dpi/inc/sleep.h

@@ -0,0 +1,127 @@
+// ॐ सूक्ष्म शरीर
+#ifndef SLEEP_H
+#define SLEEP_H
+// TODO implement SLEEP
+/*metadata.key
+metadata.signatures
+metadata.bitfield
+metadata.tree
+metadata.data
+content.key
+content.signatures
+content.bitfield
+content.tree*/
+#include "keys.h"
+#include <stdint.h>
+#include <stddef.h>
+// test
+#include <assert.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+int sleep_init(const char *root);
+
+#define SLEEP_HEADER_SIZE	32u
+
+#define SLEEP_MAGIC_BITFIELD 0x05025700u
+#define SLEEP_MAGIC_SIGNATURE 0x05025701u
+#define SLEEP_MAGIC_TREE 0x05025702u
+
+// TODO MAGIC_BITFIELD, MAGIC_SIG, MAGIC_TREE
+
+#define SLEEP_ALGO_BLAKE2B	"BLAKE2b"
+#define SLEEP_ALGO_ED25519	"Ed25519"
+
+#define SLEEP_TREE_ENTRY_SIZE	40u
+#define SLEEP_SIG_ENTRY_SIZE	64u
+#define SLEEP_BITFIELD_ENTRY_SIZE 3584u
+
+enum sleep_hash_type {
+	SLEEP_HASH_LEAF	= 0,
+	SLEEP_HASH_PARENT	= 1,
+	SLEEP_HASH_ROOT	= 2
+};
+
+// Wire
+#if defined(__GNUC__) || defined(__clang__) || defined(__MUSL__)
+	#define SLEEP_PACKED __attribute__((packed))
+#else
+	#define SLEEP_PACKED
+	#pragma pack(push,1) // pragma deez
+#endif
+
+
+typedef struct SLEEP_PACKED sleep_header {
+	uint32_t	magic_be;
+	uint8_t	version;
+	uint16_t	entry_size_be;
+	uint8_t	algo_len;
+	char	algo[24];
+} sleep_header_t;
+// TODO assert must be 32 bytes
+
+// 32 byte hash + 8 byte BE len
+typedef struct SLEEP_PACKED sleep_tree_entry {
+	uint8_t hash[32];
+	uint64_t length_be;
+} sleep_tree_entry_t;
+// TODO assert
+
+// 64 bytes
+typedef struct SLEEP_PACKED sleep_sig_entry {
+	uint8_t sig[64];
+} sleep_sig_entry_t;
+// TODO assert
+
+// 3584-byte historical record. The final 256 bytes are opaque for now.
+typedef struct SLEEP_PACKED sleep_bitfield_entry {
+	uint8_t data[1024];
+	uint8_t tree[2048];
+	uint8_t index[256];
+	uint8_t reserved[256];
+} sleep_bitfield_entry_t;
+// TODO DOTO ODOT assert it
+
+#if !defined(__GNUC__) && !defined(__clang__)
+  #pragma pack(pop)
+#endif
+
+// TODO BE INT into and out of buffers **
+
+// crypto helpers
+#ifdef SLEEP_SALT
+int sleep_gen_ed25519(uint8_t pubkey[32], uint8_t seckey[64]);
+// sign
+int sleep_sign_ed25519(const uint8_t *msg, size_t msglen,
+		       const uint8_t seckey[64], uint8_t sig_out[64]);
+// verify
+int sleep_verify_ed25519(const uint8_t *msg, size_t msglen,
+			 const uint8_t pubkey[32], const uint8_t sig[64]);
+// discovery
+int sleep_compute_discovery_key(const uint8_t pubkey[32], uint8_t out32[32]);
+#endif /* SLEEP_SALT */
+
+// function decs
+uint64_t sleep_tree_index(uint64_t depth, uint64_t offset);
+uint64_t sleep_tree_depth(uint64_t index);
+uint64_t sleep_tree_offset(uint64_t index, uint64_t depth);
+uint64_t sleep_tree_parent(uint64_t index);
+int sleep_tree_left_child(uint64_t index, uint64_t *out);
+int sleep_tree_right_child(uint64_t index, uint64_t *out);
+int sleep_tree_sibling(uint64_t index, uint64_t *out);
+uint64_t sleep_tree_left_span(uint64_t index);
+uint64_t sleep_tree_right_span(uint64_t index);
+void sleep_tree_spans(uint64_t index, uint64_t *left, uint64_t *right);
+size_t sleep_tree_count(uint64_t index);
+size_t sleep_tree_count_leaves(uint64_t index);
+size_t sleep_tree_full_roots(uint64_t index, uint64_t *out, size_t capacity);
+int sleep_tree_hash_block(const uint8_t *data, size_t len, uint8_t out_hash[32]);
+int sleep_tree_hash_parent(const uint8_t left[32], const uint8_t right[32],
+                           uint64_t length, uint8_t out_hash[32]);
+
+#ifdef __cplusplus
+}
+#endif
+#endif /* SLEEP_H */

extra/dat-dpi/inc/sleep_bitfield.h

@@ -0,0 +1,27 @@
+#ifndef SLEEP_BITFIELD_H
+#define SLEEP_BITFIELD_H
+#include <stddef.h>
+#include <stdint.h>
+
+#define SLEEP_BITFIELD_DATA_BYTES 1024u
+#define SLEEP_BITFIELD_TREE_BYTES 2048u
+#define SLEEP_BITFIELD_INDEX_BYTES 256u
+#define SLEEP_BITFIELD_RESERVED_BYTES 256u
+
+typedef struct sleep_bitfield {
+	uint8_t data[SLEEP_BITFIELD_DATA_BYTES];
+	uint8_t tree[SLEEP_BITFIELD_TREE_BYTES];
+	uint8_t index[SLEEP_BITFIELD_INDEX_BYTES];
+	uint8_t reserved[SLEEP_BITFIELD_RESERVED_BYTES];
+} sleep_bitfield_t;
+
+void sleep_bitfield_clear(sleep_bitfield_t *field);
+int sleep_bitfield_get_data(const sleep_bitfield_t *field, size_t bit);
+int sleep_bitfield_get_tree(const sleep_bitfield_t *field, size_t bit);
+int sleep_bitfield_set_data(sleep_bitfield_t *field, size_t bit, int present);
+int sleep_bitfield_set_tree(sleep_bitfield_t *field, size_t bit, int present);
+void sleep_bitfield_reindex(sleep_bitfield_t *field);
+size_t sleep_bitfield_next_data(const sleep_bitfield_t *field, size_t start, int present);
+int sleep_bitfield_encode(const sleep_bitfield_t *field, uint8_t *out, size_t capacity);
+int sleep_bitfield_decode(sleep_bitfield_t *field, const uint8_t *input, size_t length);
+#endif

extra/dat-dpi/inc/sleep_crypto.h

@@ -0,0 +1,29 @@
+#ifndef SLEEP_CRYPTO_H
+#define SLEEP_CRYPTO_H
+
+#include <stddef.h>
+#include <stdint.h>
+#include "keys.h"
+#define SLEEP_ROOT_RECORD_BYTES 48u
+
+int sleep_crypto_init(void);
+int sleep_stream_xsalsa20_xor(uint8_t *out, const uint8_t *in, size_t length,
+			     const uint8_t nonce[24], const uint8_t key[32],
+			     uint64_t stream_offset);
+int sleep_gen_keypair(sleep_keypair_t *kp);
+int sleep_save_pubkey(const char *path, const uint8_t pubkey[SLEEP_PUB_KEY_BYTES]);
+int sleep_load_pubkey(const char *path, uint8_t pubkey[SLEEP_PUB_KEY_BYTES]);
+int sleep_save_keypair(const char *path, const sleep_keypair_t *keypair);
+int sleep_load_keypair(const char *path, sleep_keypair_t *keypair);
+
+int sleep_blake2b(uint8_t out[SLEEP_BLAKE2B_BYTES], const uint8_t *in, size_t inlen);
+
+int sleep_sign(uint8_t sig[SLEEP_SIGNATURE_BYTES], const uint8_t *msg, size_t msglen, const sleep_keypair_t *kp);
+
+int sleep_verify(const uint8_t sig[SLEEP_SIGNATURE_BYTES], const uint8_t *msg, size_t msglen, const uint8_t pubkey[SLEEP_PUB_KEY_BYTES]);
+
+int sleep_sign_roots(uint8_t sig[SLEEP_SIGNATURE_BYTES], const uint8_t *roots, size_t nroots, const uint64_t *indices, const uint64_t *lengths, const sleep_keypair_t *kp);
+
+int sleep_verify_roots(const uint8_t sig[SLEEP_SIGNATURE_BYTES], const uint8_t *roots, size_t nroots, const uint64_t *indices, const uint64_t *lengths, const uint8_t pub[SLEEP_PUB_KEY_BYTES]);
+
+#endif /* SLEEP_CRYPTO_H */

extra/dat-dpi/inc/sleep_feed.h

@@ -0,0 +1,43 @@
+#ifndef SLEEP_FEED_H
+#define SLEEP_FEED_H
+#include <stddef.h>
+#include <stdint.h>
+#include "keys.h"
+
+typedef struct sleep_feed {
+	char directory[1024];
+	char name[64];
+	uint8_t **blocks;
+	size_t *lengths;
+	uint8_t **proofs;
+	size_t *proof_lengths;
+	size_t length;
+	size_t capacity;
+	sleep_keypair_t keypair;
+	int writable;
+	int dirty;
+	int remote_only;
+	int persist_remote;
+} sleep_feed_t;
+
+int sleep_feed_open(sleep_feed_t *feed, const char *directory, const char *name,
+		    const sleep_keypair_t *keypair);
+/* Opens an in-memory, read-only feed replica using only its public key. */
+int sleep_feed_open_remote(sleep_feed_t *feed, const char *directory,
+			   const char *name, const uint8_t public_key[32]);
+/* Merge verified cached remote blocks and persist subsequent received blocks. */
+int sleep_feed_load_remote(sleep_feed_t *feed, const uint8_t public_key[32]);
+size_t sleep_feed_length(const sleep_feed_t *feed);
+int sleep_feed_read(const sleep_feed_t *feed, size_t index, uint8_t **data, size_t *length);
+int sleep_feed_append(sleep_feed_t *feed, const uint8_t *data, size_t length);
+/* Keeps already verified remote blocks and proofs in this session's memory. */
+int sleep_feed_append_remote(sleep_feed_t *feed, size_t index,
+			     const uint8_t *data, size_t length,
+			     const uint8_t *proof, size_t proof_length);
+int sleep_feed_truncate(sleep_feed_t *feed, size_t length);
+int sleep_feed_has(const sleep_feed_t *feed, size_t index);
+int sleep_feed_byte_offset(const sleep_feed_t *feed, size_t index,
+			   uint64_t *offset);
+int sleep_feed_flush(sleep_feed_t *feed);
+void sleep_feed_close(sleep_feed_t *feed);
+#endif

extra/dat-dpi/inc/sleep_proof.h

@@ -0,0 +1,43 @@
+#ifndef SLEEP_PROOF_H
+#define SLEEP_PROOF_H
+#include <stddef.h>
+#include <stdint.h>
+#include "sleep_feed.h"
+
+#define SLEEP_PROOF_HASH_BYTES 32u
+#define SLEEP_PROOF_MAX_NODES 64u
+typedef struct sleep_proof_node {
+	uint64_t index;
+	uint64_t length;
+	uint8_t hash[SLEEP_PROOF_HASH_BYTES];
+} sleep_proof_node_t;
+typedef struct sleep_proof {
+	uint64_t block;
+	uint64_t block_length;
+	uint8_t block_hash[SLEEP_PROOF_HASH_BYTES];
+	sleep_proof_node_t nodes[SLEEP_PROOF_MAX_NODES];
+	size_t node_count;
+	sleep_proof_node_t roots[SLEEP_PROOF_MAX_NODES];
+	size_t root_count;
+	uint8_t signature[64];
+	int has_signature;
+} sleep_proof_t;
+
+void sleep_proof_init(sleep_proof_t *proof, uint64_t block);
+int sleep_proof_add_node(sleep_proof_t *proof, uint64_t index,
+			 uint64_t length, const uint8_t hash[32]);
+int sleep_proof_add_root(sleep_proof_t *proof, uint64_t index,
+			 uint64_t length, const uint8_t hash[32]);
+int sleep_proof_create(const sleep_feed_t *feed, uint64_t block,
+		       sleep_proof_t *proof);
+int sleep_proof_from_nodes(uint64_t block, const uint8_t *value,
+			   size_t value_len, const sleep_proof_node_t *nodes,
+			   size_t node_count, const uint8_t signature[64],
+			   uint64_t feed_length, sleep_proof_t *proof);
+int sleep_proof_encode(const sleep_proof_t *proof, uint8_t *out,
+		       size_t capacity, size_t *written);
+int sleep_proof_decode(sleep_proof_t *proof, const uint8_t *input,
+		       size_t input_len);
+int sleep_proof_verify(const sleep_proof_t *proof, const uint8_t *block,
+		       size_t block_len, const uint8_t public_key[32]);
+#endif

extra/dat-dpi/inc/sleep_proto.h

@@ -0,0 +1,26 @@
+#ifndef SLEEP_PROTO_H
+#define SLEEP_PROTO_H
+#include <stddef.h>
+#include <stdint.h>
+#define SLEEP_PROTO_KEY_BYTES 32u
+typedef struct sleep_metadata_header { char type[32]; uint8_t content[32]; size_t content_len; } sleep_metadata_header_t;
+enum sleep_entry_type { SLEEP_ENTRY_FILE=1, SLEEP_ENTRY_DIRECTORY=2, SLEEP_ENTRY_SYMLINK=3, SLEEP_ENTRY_DELETE=4 };
+typedef struct sleep_metadata_entry {
+	char path[1024];
+	uint32_t type;
+	uint32_t mode;
+	uint64_t size;
+	uint64_t mtime;
+	uint64_t block_offset;
+	uint64_t block_count;
+	char linkname[1024];
+} sleep_metadata_entry_t;
+size_t sleep_varint_size(uint64_t value);
+int sleep_varint_encode(uint64_t value, uint8_t *out, size_t capacity, size_t *written);
+int sleep_varint_decode(const uint8_t *input, size_t input_len, uint64_t *value, size_t *consumed);
+int sleep_metadata_header_encode(const sleep_metadata_header_t *header, uint8_t *out, size_t capacity, size_t *written);
+int sleep_metadata_header_decode(sleep_metadata_header_t *header, const uint8_t *input, size_t input_len);
+int sleep_write_metadata_header(const char *path, const uint8_t content_pubkey[32]);
+int sleep_metadata_entry_encode(const sleep_metadata_entry_t *entry, uint8_t *out, size_t capacity, size_t *written);
+int sleep_metadata_entry_decode(sleep_metadata_entry_t *entry, const uint8_t *input, size_t input_len);
+#endif

extra/dat-dpi/inc/sleep_replication.h

@@ -0,0 +1,44 @@
+#ifndef SLEEP_REPLICATION_H
+#define SLEEP_REPLICATION_H
+#include <stddef.h>
+#include <stdint.h>
+#include "sleep_feed.h"
+#include "sleep_bitfield.h"
+#include "sleep_proof.h"
+
+#define SLEEP_REPLICATION_QUEUE 128u
+typedef struct sleep_replication_request { uint64_t index; uint64_t bytes; int hash; int nodes; } sleep_replication_request_t;
+typedef struct sleep_replication {
+	sleep_feed_t *feed;
+	uint64_t channel;
+	uint64_t remote_length;
+	sleep_bitfield_t remote_bitfield;
+	sleep_replication_request_t requests[SLEEP_REPLICATION_QUEUE];
+	size_t request_head, request_count;
+	int live, uploading, downloading;
+} sleep_replication_t;
+
+int sleep_replication_init(sleep_replication_t *replication,
+			   sleep_feed_t *feed, uint64_t channel);
+int sleep_replication_have(sleep_replication_t *replication,
+			   uint64_t start, uint64_t length);
+int sleep_replication_unhave(sleep_replication_t *replication,
+			     uint64_t start, uint64_t length);
+int sleep_replication_want(sleep_replication_t *replication,
+			   uint64_t start, uint64_t length);
+int sleep_replication_request(sleep_replication_t *replication,
+			      const sleep_replication_request_t *request);
+int sleep_replication_next(sleep_replication_t *replication,
+			   sleep_replication_request_t *request);
+int sleep_replication_complete(sleep_replication_t *replication,
+			      uint64_t index);
+int sleep_replication_is_requested(const sleep_replication_t *replication,
+				   uint64_t index);
+int sleep_replication_unwant(sleep_replication_t *replication,
+			     uint64_t start, uint64_t length);
+void sleep_replication_clear_requests(sleep_replication_t *replication);
+int sleep_replication_receive(sleep_replication_t *replication,
+			      const uint8_t *block, size_t block_len,
+			      const sleep_proof_t *proof,
+			      const uint8_t feed_public_key[32]);
+#endif

extra/dat-dpi/inc/sleep_utp.h

@@ -0,0 +1,39 @@
+#ifndef SLEEP_UTP_H
+#define SLEEP_UTP_H
+#include <stddef.h>
+#include <stdint.h>
+#include "dat_socket.h"
+
+#define SLEEP_UTP_MTU 1200u
+#define SLEEP_UTP_WINDOW 64u
+enum sleep_utp_state { SLEEP_UTP_CLOSED, SLEEP_UTP_SYN_SENT, SLEEP_UTP_SYN_RECV, SLEEP_UTP_CONNECTED, SLEEP_UTP_FIN_SENT, SLEEP_UTP_RESET };
+enum sleep_utp_packet_type { SLEEP_UTP_DATA, SLEEP_UTP_FIN, SLEEP_UTP_STATE, SLEEP_UTP_RESET_PACKET, SLEEP_UTP_SYN };
+typedef struct sleep_utp_packet { uint8_t type, version; uint16_t connection_id; uint32_t timestamp, timestamp_diff, window_size; uint16_t sequence, acknowledgement; uint32_t selective_ack; int has_selective_ack; const uint8_t *payload; size_t payload_len; } sleep_utp_packet_t;
+typedef struct sleep_utp_slot { uint16_t sequence; uint8_t data[SLEEP_UTP_MTU]; size_t length; uint64_t sent_at; unsigned retries; int used; } sleep_utp_slot_t;
+typedef struct sleep_utp {
+	dat_socket_t socket;
+	struct sockaddr_storage remote;
+	socklen_t remote_len;
+	uint16_t send_id, receive_id, sequence, acknowledgement;
+	uint32_t receive_window, send_window;
+	enum sleep_utp_state state;
+	uint64_t retransmit_at;
+	unsigned syn_retries;
+	uint32_t congestion_window, slow_start_threshold;
+	uint32_t round_trip_time, round_trip_variance;
+	size_t bytes_in_flight;
+	sleep_utp_slot_t unacked[SLEEP_UTP_WINDOW];
+	sleep_utp_slot_t reorder[SLEEP_UTP_WINDOW];
+	void *context;
+	void (*on_data)(struct sleep_utp *, const uint8_t *, size_t);
+} sleep_utp_t;
+int sleep_utp_init(sleep_utp_t *utp, dat_socket_t udp_socket);
+int sleep_utp_connect(sleep_utp_t *utp, const struct sockaddr *remote, socklen_t remote_len);
+int sleep_utp_accept(sleep_utp_t *utp, const sleep_utp_packet_t *syn, const struct sockaddr *remote, socklen_t remote_len);
+int sleep_utp_packet_encode(const sleep_utp_packet_t *packet, uint8_t *out, size_t capacity, size_t *written);
+int sleep_utp_packet_decode(sleep_utp_packet_t *packet, const uint8_t *input, size_t input_len);
+int sleep_utp_input(sleep_utp_t *utp, const uint8_t *packet, size_t packet_len);
+int sleep_utp_send(sleep_utp_t *utp, const uint8_t *data, size_t length);
+int sleep_utp_tick(sleep_utp_t *utp, uint64_t now_ms);
+int sleep_utp_close(sleep_utp_t *utp);
+#endif

extra/dat-dpi/inc/sleep_verify.h

@@ -0,0 +1,6 @@
+#ifndef SLEEP_VERIFY_H
+#define SLEEP_VERIFY_H
+
+int sleep_verify_dat(const char *dat_dir);
+
+#endif

extra/dat-dpi/inc/sleep_writer.h

@@ -0,0 +1,18 @@
+#ifndef SLEEP_WRITER_H
+#define SLEEP_WRITER_H
+#include <stddef.h>
+#include <stdint.h>
+#include "sleep.h"
+#ifdef __cplusplus
+extern "C" {
+#endif
+int sleep_write_feed(const char *feed_dir,
+                     const char *feed_name,
+                     const uint8_t *const *blocks,
+                     const size_t *lengths,
+                     size_t nblocks,
+                     const sleep_keypair_t *keypair);
+#ifdef __cplusplus
+}
+#endif
+#endif

extra/dat-dpi/inc/win_compat.h

@@ -0,0 +1,80 @@
+#ifndef DATGPT_WIN_COMPAT_H
+#define DATGPT_WIN_COMPAT_H
+
+/* Small POSIX spellings used by the shared C sources on native Windows. */
+#ifdef _WIN32
+#include <io.h>
+#include <direct.h>
+#include <process.h>
+#include <fcntl.h>
+#include <errno.h>
+#include <sys/stat.h>
+#include <string.h>
+#include <winsock2.h>
+#include <ws2tcpip.h>
+#include <windows.h>
+#ifndef mkdir
+#define mkdir(path, mode) _mkdir(path)
+#endif
+#ifndef fsync
+#define fsync(fd) _commit(fd)
+#endif
+#ifndef getpid
+#define getpid _getpid
+#endif
+#ifndef O_CLOEXEC
+#define O_CLOEXEC 0
+#endif
+#ifndef O_NOFOLLOW
+#define O_NOFOLLOW 0
+#endif
+#ifndef EKEYREJECTED
+#define EKEYREJECTED EACCES
+#endif
+#ifndef S_ISLNK
+#define S_ISLNK(mode) (((mode) & S_IFMT) == S_IFLNK)
+#endif
+#ifndef S_IFLNK
+#define S_IFLNK 0120000
+#endif
+#ifndef fchmod
+#define fchmod(fd, mode) ((void)(fd), (void)(mode), 0)
+#endif
+#ifndef strcasecmp
+#define strcasecmp _stricmp
+#endif
+#ifndef strncasecmp
+#define strncasecmp _strnicmp
+#endif
+static inline char *dat_win_getenv_compat(const char *name)
+{
+	char *value = getenv(name);
+	if (!value && name && !strcmp(name, "HOME")) value = getenv("USERPROFILE");
+	return value;
+}
+#define getenv dat_win_getenv_compat
+static inline int dat_win_lstat(const char *path, struct stat *info)
+{
+	DWORD attributes;
+	if (stat(path, info)) return -1;
+	attributes = GetFileAttributesA(path);
+	if (attributes != INVALID_FILE_ATTRIBUTES && (attributes & FILE_ATTRIBUTE_REPARSE_POINT))
+		info->st_mode = (info->st_mode & ~S_IFMT) | S_IFLNK;
+	return 0;
+}
+#ifndef lstat
+#define lstat dat_win_lstat
+#endif
+static inline int dat_win_readlink(const char *path, char *target, size_t capacity)
+{ (void)path; (void)target; (void)capacity; errno = ENOTSUP; return -1; }
+static inline int dat_win_symlink(const char *target, const char *path)
+{ (void)target; (void)path; errno = ENOTSUP; return -1; }
+#ifndef readlink
+#define readlink dat_win_readlink
+#endif
+#ifndef symlink
+#define symlink dat_win_symlink
+#endif
+#endif
+
+#endif

extra/dat-dpi/inc/win_dirent.h

@@ -0,0 +1,53 @@
+#ifndef DATGPT_WIN_DIRENT_H
+#define DATGPT_WIN_DIRENT_H
+#ifdef _WIN32
+#include <windows.h>
+#include <stdlib.h>
+#include <stdio.h>
+#include <string.h>
+#include <errno.h>
+#define NAME_MAX MAX_PATH
+typedef struct dirent { char d_name[MAX_PATH]; } struct_dirent_t;
+typedef struct dat_win_dir {
+	HANDLE handle;
+	WIN32_FIND_DATAA data;
+	int first;
+	struct dirent entry;
+} DIR;
+static DIR *dat_opendir(const char *path)
+{
+	DIR *dir = (DIR *)calloc(1, sizeof(*dir));
+	char pattern[MAX_PATH * 2];
+	int n;
+	if (!dir) return NULL;
+	n = snprintf(pattern, sizeof(pattern), "%s\\*", path);
+	if (n < 0 || (size_t)n >= sizeof(pattern)) { free(dir); errno = ENAMETOOLONG; return NULL; }
+	dir->handle = FindFirstFileA(pattern, &dir->data);
+	if (dir->handle == INVALID_HANDLE_VALUE) { free(dir); errno = ENOENT; return NULL; }
+	dir->first = 1;
+	return dir;
+}
+static struct dirent *dat_readdir(DIR *dir)
+{
+	if (!dir) return NULL;
+	if (dir->first) dir->first = 0;
+	else if (!FindNextFileA(dir->handle, &dir->data)) return NULL;
+	strncpy(dir->entry.d_name, dir->data.cFileName, sizeof(dir->entry.d_name) - 1);
+	dir->entry.d_name[sizeof(dir->entry.d_name) - 1] = 0;
+	return &dir->entry;
+}
+static int dat_closedir(DIR *dir)
+{
+	int rc;
+	if (!dir) { errno = EINVAL; return -1; }
+	rc = FindClose(dir->handle) ? 0 : -1;
+	free(dir);
+	return rc;
+}
+#define opendir dat_opendir
+#define readdir dat_readdir
+#define closedir dat_closedir
+#else
+#include <dirent.h>
+#endif
+#endif

extra/dat-dpi/inc/win_fnmatch.h

@@ -0,0 +1,24 @@
+#ifndef DATGPT_WIN_FNMATCH_H
+#define DATGPT_WIN_FNMATCH_H
+#ifdef _WIN32
+static int dat_win_fnmatch(const char *pattern, const char *text)
+{
+	while (*pattern) {
+		if (*pattern == '*') {
+			while (*pattern == '*') pattern++;
+			if (!*pattern) return 0;
+			do { if (dat_win_fnmatch(pattern, text) == 0) return 0; } while (*text++);
+			return 1;
+		}
+		if (!*text) return 1;
+		if (*pattern == '?') { pattern++; text++; continue; }
+		if (*pattern != *text) return 1;
+		pattern++; text++;
+	}
+	return *text ? 1 : 0;
+}
+#define fnmatch(pattern, text, flags) ((void)(flags), dat_win_fnmatch((pattern), (text)))
+#else
+#include <fnmatch.h>
+#endif
+#endif

extra/dat-dpi/src/base32768.c

@@ -0,0 +1,231 @@
+/*
+ * Base32768 alphabet and bit-packing ported from qntm/base32768.
+ * Upstream implementation: https://github.com/qntm/base32768
+ * Copyright (c) 2021 qntm. Used under the MIT license; see
+ * third_party/base32768.LICENSE.
+ */
+#include "base32768.h"
+#include <errno.h>
+#include <string.h>
+
+typedef struct base_range {
+	uint32_t first, last, offset;
+} base_range_t;
+
+#define BASE32768_MAX_RANGES 128u
+static base_range_t repertoire[2][BASE32768_MAX_RANGES];
+static size_t repertoire_count[2];
+static uint32_t repertoire_size[2];
+static int repertoire_ready;
+static int repertoire_error;
+
+/* The pairs encode inclusive ranges. Keep in sync with upstream's pairStrings. */
+static const char *const pair_strings[2] = {
+	"ҠҿԀԟڀڿݠޟ߀ߟကဟႠႿᄀᅟᆀᆟᇠሿበቿዠዿጠጿᎠᏟᐠᙟᚠᛟកសᠠᡟᣀᣟᦀᦟ᧠᧿ᨠᨿᯀᯟᰀᰟᴀᴟ⇠⇿⋀⋟⍀⏟␀␟─❟➀➿⠀⥿⦠⦿⨠⩟⪀⪿⫠⭟ⰀⰟⲀⳟⴀⴟⵀⵟ⺠⻟㇀㇟㐀䶟䷀龿ꀀꑿ꒠꒿ꔀꗿꙀꙟꚠꛟ꜀ꝟꞀꞟꡀꡟ",
+	"ƀƟɀʟ"
+};
+
+static int utf8_next(const unsigned char *s, size_t length, size_t *offset,
+		     uint32_t *codepoint)
+{
+	unsigned char a, b, c, d;
+	if (*offset >= length) return -1;
+	a = s[(*offset)++];
+	if (a < 0x80) { *codepoint = a; return 0; }
+	if (a >= 0xc2 && a <= 0xdf) {
+		if (*offset >= length) return -1;
+		b = s[(*offset)++];
+		if ((b & 0xc0) != 0x80) return -1;
+		*codepoint = ((uint32_t)(a & 0x1f) << 6) | (b & 0x3f);
+		return 0;
+	}
+	if (a >= 0xe0 && a <= 0xef) {
+		if (length - *offset < 2) return -1;
+		b = s[(*offset)++]; c = s[(*offset)++];
+		if ((b & 0xc0) != 0x80 || (c & 0xc0) != 0x80 ||
+		    (a == 0xe0 && b < 0xa0) || (a == 0xed && b >= 0xa0)) return -1;
+		*codepoint = ((uint32_t)(a & 0x0f) << 12) |
+			((uint32_t)(b & 0x3f) << 6) | (c & 0x3f);
+		return 0;
+	}
+	if (a >= 0xf0 && a <= 0xf4) {
+		if (length - *offset < 3) return -1;
+		b = s[(*offset)++]; c = s[(*offset)++]; d = s[(*offset)++];
+		if ((b & 0xc0) != 0x80 || (c & 0xc0) != 0x80 || (d & 0xc0) != 0x80 ||
+		    (a == 0xf0 && b < 0x90) || (a == 0xf4 && b >= 0x90)) return -1;
+		*codepoint = ((uint32_t)(a & 7) << 18) | ((uint32_t)(b & 0x3f) << 12) |
+			((uint32_t)(c & 0x3f) << 6) | (d & 0x3f);
+		return 0;
+	}
+	return -1;
+}
+
+static size_t utf8_put(uint32_t cp, char *out)
+{
+	if (cp < 0x80) { out[0] = (char)cp; return 1; }
+	if (cp < 0x800) {
+		out[0] = (char)(0xc0 | (cp >> 6));
+		out[1] = (char)(0x80 | (cp & 0x3f)); return 2;
+	}
+	if (cp < 0x10000) {
+		out[0] = (char)(0xe0 | (cp >> 12));
+		out[1] = (char)(0x80 | ((cp >> 6) & 0x3f));
+		out[2] = (char)(0x80 | (cp & 0x3f)); return 3;
+	}
+	out[0] = (char)(0xf0 | (cp >> 18));
+	out[1] = (char)(0x80 | ((cp >> 12) & 0x3f));
+	out[2] = (char)(0x80 | ((cp >> 6) & 0x3f));
+	out[3] = (char)(0x80 | (cp & 0x3f)); return 4;
+}
+
+static int initialize_repertoire(void)
+{
+	size_t row;
+	if (repertoire_ready) return repertoire_error ? -1 : 0;
+	repertoire_ready = 1;
+	for (row = 0; row < 2; row++) {
+		const char *text = pair_strings[row];
+		size_t length = strlen(text), offset = 0;
+		uint32_t points[BASE32768_MAX_RANGES * 2u];
+		size_t point_count = 0, i;
+		while (offset < length) {
+			if (point_count >= sizeof(points) / sizeof(points[0]) ||
+			    utf8_next((const unsigned char *)text, length, &offset,
+				      &points[point_count])) {
+				repertoire_error = 1; return -1;
+			}
+			point_count++;
+		}
+		if (point_count % 2 || point_count / 2 > BASE32768_MAX_RANGES) {
+			repertoire_error = 1; return -1;
+		}
+		for (i = 0; i < point_count; i += 2) {
+			base_range_t *range = &repertoire[row][repertoire_count[row]++];
+			if (points[i] > points[i + 1]) { repertoire_error = 1; return -1; }
+			range->first = points[i]; range->last = points[i + 1];
+			range->offset = repertoire_size[row];
+			repertoire_size[row] += points[i + 1] - points[i] + 1;
+		}
+	}
+	if (repertoire_size[0] != 32768u || repertoire_size[1] != 128u) {
+		repertoire_error = 1; return -1;
+	}
+	return 0;
+}
+
+static int index_to_codepoint(unsigned bits, uint32_t index, uint32_t *out)
+{
+	size_t row = bits == 15 ? 0 : 1, i;
+	if (initialize_repertoire()) return -1;
+	for (i = 0; i < repertoire_count[row]; i++) {
+		const base_range_t *range = &repertoire[row][i];
+		uint32_t count = range->last - range->first + 1;
+		if (index >= range->offset && index - range->offset < count) {
+			*out = range->first + index - range->offset;
+			return 0;
+		}
+	}
+	return -1;
+}
+
+int base32768_codepoint_at(unsigned bits, uint32_t index, uint32_t *codepoint)
+{
+	if (!codepoint || (bits == 15 && index >= 32768u) ||
+	    (bits == 7 && index >= 128u) || (bits != 15 && bits != 7)) return -1;
+	return index_to_codepoint(bits, index, codepoint);
+}
+
+static int codepoint_to_index(uint32_t cp, unsigned *bits, uint32_t *index)
+{
+	size_t row, i;
+	if (initialize_repertoire()) return -1;
+	for (row = 0; row < 2; row++) for (i = 0; i < repertoire_count[row]; i++) {
+		const base_range_t *range = &repertoire[row][i];
+		if (cp >= range->first && cp <= range->last) {
+			*bits = row == 0 ? 15u : 7u;
+			*index = range->offset + cp - range->first;
+			return 0;
+		}
+	}
+	return -1;
+}
+
+int base32768_encode(const uint8_t *input, size_t input_length,
+		     char *output, size_t output_capacity)
+{
+	size_t i, used = 0;
+	uint32_t value = 0;
+	unsigned value_bits = 0;
+	if ((!input && input_length) || !output || !output_capacity) { errno = EINVAL; return -1; }
+	if (initialize_repertoire()) { errno = EPROTO; return -1; }
+	for (i = 0; i < input_length; i++) {
+		int bit;
+		for (bit = 7; bit >= 0; bit--) {
+			uint32_t cp;
+			char encoded[4];
+			size_t n;
+			value = (value << 1) | ((input[i] >> bit) & 1u);
+			if (++value_bits == 15) {
+				if (index_to_codepoint(15, value, &cp)) { errno = EPROTO; return -1; }
+				n = utf8_put(cp, encoded);
+				if (used + n >= output_capacity) { errno = ENOSPC; return -1; }
+				memcpy(output + used, encoded, n); used += n;
+				value = 0; value_bits = 0;
+			}
+		}
+	}
+	if (value_bits) {
+		unsigned target_bits = value_bits >= 8 ? 15u : 7u;
+		uint32_t cp;
+		char encoded[4];
+		size_t n;
+		while (value_bits < target_bits) {
+			value = (value << 1) | 1u;
+			value_bits++;
+		}
+		if (index_to_codepoint(target_bits, value, &cp)) { errno = EPROTO; return -1; }
+		n = utf8_put(cp, encoded);
+		if (used + n >= output_capacity) { errno = ENOSPC; return -1; }
+		memcpy(output + used, encoded, n); used += n;
+	}
+	output[used] = '\0';
+	return 0;
+}
+
+int base32768_decode(const char *input, uint8_t *output,
+		     size_t output_capacity, size_t *output_length)
+{
+	size_t length, offset = 0, used = 0, i = 0, total_characters = 0, scan = 0;
+	unsigned pending_bits = 0;
+	uint32_t pending = 0;
+	if (!input || (!output && output_capacity) || !output_length) { errno = EINVAL; return -1; }
+	*output_length = 0;
+	length = strlen(input);
+	if (initialize_repertoire()) { errno = EPROTO; return -1; }
+	while (scan < length) {
+		uint32_t ignored;
+		if (utf8_next((const unsigned char *)input, length, &scan, &ignored)) {
+			errno = EILSEQ; return -1;
+		}
+		total_characters++;
+	}
+	while (offset < length) {
+		uint32_t cp, value;
+		unsigned bits, bit;
+		if (utf8_next((const unsigned char *)input, length, &offset, &cp) ||
+		    codepoint_to_index(cp, &bits, &value)) { errno = EILSEQ; return -1; }
+		if (bits == 7 && i + 1 != total_characters) { errno = EINVAL; return -1; }
+		for (bit = bits; bit > 0; bit--) {
+			pending = (pending << 1) | ((value >> (bit - 1)) & 1u);
+			if (++pending_bits == 8) {
+				if (used >= output_capacity) { errno = ENOSPC; return -1; }
+				output[used++] = (uint8_t)pending;
+				pending = 0; pending_bits = 0;
+			}
+		}
+		i++;
+	}
+	if (pending != ((1u << pending_bits) - 1u)) { errno = EINVAL; return -1; }
+	*output_length = used;
+	return 0;
+}

extra/dat-dpi/src/cable.c

@@ -0,0 +1,200 @@
+#define _POSIX_C_SOURCE 200809L
+#include "cable.h"
+#include "sleep_crypto.h"
+#include <errno.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <fcntl.h>
+#include <sys/stat.h>
+#include <unistd.h>
+#include <sodium.h>
+
+static int hex_value(char c)
+{
+	if(c>='0'&&c<='9')return c-'0';
+	if(c>='a'&&c<='f')return c-'a'+10;
+	if(c>='A'&&c<='F')return c-'A'+10;
+	return -1;
+}
+
+int cable_address_parse(const char *address,uint8_t key[32],enum cable_address_scheme *scheme)
+{
+	const char *hex=address; enum cable_address_scheme parsed=0; size_t i;
+	if(!address||!key){errno=EINVAL;return -1;}
+	if(!strncmp(hex,"cabal://",8)){parsed=CABLE_ADDRESS_CABAL;hex+=8;}
+	else if(!strncmp(hex,"cable://",8)){parsed=CABLE_ADDRESS_CABLE;hex+=8;}
+	else if(!strncmp(hex,"hyper://",8)){parsed=CABLE_ADDRESS_HYPER;hex+=8;}
+	if(strlen(hex)!=64){errno=EINVAL;return -1;}
+	for(i=0;i<32;i++){int hi=hex_value(hex[i*2]),lo=hex_value(hex[i*2+1]);if(hi<0||lo<0){errno=EINVAL;return -1;}key[i]=(uint8_t)((hi<<4)|lo);}
+	if(scheme)*scheme=parsed?parsed:CABLE_ADDRESS_CABAL;
+	return 0;
+}
+
+int cable_address_format(const uint8_t key[32],enum cable_address_scheme scheme,char *out,size_t cap)
+{
+	static const char digits[]="0123456789abcdef";const char *prefix;size_t i;
+	if(!key||!out){errno=EINVAL;return -1;}
+	switch(scheme){case CABLE_ADDRESS_CABAL:prefix="cabal://";break;case CABLE_ADDRESS_CABLE:prefix="cable://";break;case CABLE_ADDRESS_HYPER:prefix="hyper://";break;default:errno=EINVAL;return -1;}
+	if(cap<73){errno=ENOSPC;return -1;}
+	memcpy(out,prefix,8);for(i=0;i<32;i++){out[8+i*2]=digits[key[i]>>4];out[9+i*2]=digits[key[i]&15];}out[72]='\0';return 0;
+}
+
+int cable_room_address_parse(const char *address,cable_room_address_t *room)
+{
+	const char *query; char key_text[73]; size_t key_len;
+	if(!address||!room){errno=EINVAL;return -1;}
+	memset(room,0,sizeof(*room));
+	query=strchr(address,'?');
+	key_len=query?(size_t)(query-address):strlen(address);
+	if(key_len>=sizeof(key_text)){errno=EINVAL;return -1;}
+	memcpy(key_text,address,key_len);key_text[key_len]='\0';
+	if(cable_address_parse(key_text,room->key,&room->scheme))return -1;
+	if(query){
+		static const char prefix[]="?admin=";
+		if(strncmp(query,prefix,sizeof(prefix)-1)){errno=EINVAL;return -1;}
+		if(cable_address_parse(query+sizeof(prefix)-1,room->admin_key,NULL))return -1;
+		room->has_admin=1;
+	}
+	return 0;
+}
+
+int cable_room_address_format(const cable_room_address_t *room,enum cable_address_scheme scheme,char *out,size_t cap)
+{
+	char key[73],admin[73];
+	if(!room||!out){errno=EINVAL;return -1;}
+	if(cable_address_format(room->key,scheme,key,sizeof(key)))return -1;
+	if(!room->has_admin){if(cap<sizeof(key)){errno=ENOSPC;return -1;}memcpy(out,key,sizeof(key));return 0;}
+	if(cable_address_format(room->admin_key,CABLE_ADDRESS_CABAL,admin,sizeof(admin)))return -1;
+	if(cap<8+64+7+64+1){errno=ENOSPC;return -1;}
+	memcpy(out,key,72);memcpy(out+72,"?admin=",7);memcpy(out+79,admin+8,64);out[143]='\0';return 0;
+}
+
+int cable_user_address_format(const uint8_t key[32],enum cable_address_scheme scheme,char *out,size_t cap)
+{
+	return cable_address_format(key,scheme,out,cap);
+}
+
+int cable_user_identity_path(const char *home,const uint8_t room_key[32],char *out,size_t cap)
+{
+	static const char digits[]="0123456789abcdef";
+	char base[1024],rooms[1056],roomdir[1120];size_t i;int n;
+	if(!home||!home[0]||!room_key||!out){errno=EINVAL;return -1;}
+	n=snprintf(base,sizeof(base),"%s/.cable",home);
+	if(n<0||(size_t)n>=sizeof(base)){errno=ENAMETOOLONG;return -1;}
+	if(mkdir(base,0700)<0&&errno!=EEXIST)return -1;
+	n=snprintf(rooms,sizeof(rooms),"%s/rooms",base);
+	if(n<0||(size_t)n>=sizeof(rooms)){errno=ENAMETOOLONG;return -1;}
+	if(mkdir(rooms,0700)<0&&errno!=EEXIST)return -1;
+	if(strlen(rooms)+1+64>=sizeof(roomdir)){errno=ENAMETOOLONG;return -1;}
+	strcpy(roomdir,rooms);strcat(roomdir,"/");
+	for(i=0;i<32;i++){roomdir[strlen(rooms)+1+i*2]=digits[room_key[i]>>4];roomdir[strlen(rooms)+2+i*2]=digits[room_key[i]&15];}
+	roomdir[strlen(rooms)+65]='\0';
+	if(mkdir(roomdir,0700)<0&&errno!=EEXIST)return -1;
+	n=snprintf(out,cap,"%s/user-feed.key",roomdir);
+	if(n<0||(size_t)n>=cap){errno=ENAMETOOLONG;return -1;}
+	return 0;
+}
+
+int cable_room_create(const char *home,uint8_t room_key[32])
+{
+	sleep_keypair_t room; char feed_path[1200], secret_path[1300];
+	size_t offset=0; int fd=-1, saved=0, rc=-1; ssize_t wrote;
+	if(!home||!room_key){errno=EINVAL;return -1;}
+	secret_path[0]='\0';
+	memset(&room,0,sizeof(room));
+	if(sleep_crypto_init()||sleep_gen_keypair(&room))goto done;
+	if(cable_user_identity_path(home,room.pubkey,feed_path,sizeof(feed_path)))goto done;
+	{
+		static const char suffix[]="/user-feed.key";
+		size_t n=strlen(feed_path);
+		if(n<sizeof(suffix)-1||strcmp(feed_path+n-(sizeof(suffix)-1),suffix)){errno=EINVAL;goto done;}
+		feed_path[n-(sizeof(suffix)-1)]='\0';
+	}
+	if(snprintf(secret_path,sizeof(secret_path),"%s/room-secret.key",feed_path) >= (int)sizeof(secret_path)) { errno=ENAMETOOLONG; goto done; }
+	fd=open(secret_path,O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC|O_NOFOLLOW,0600);
+	if(fd<0)goto done;
+	while(offset<sizeof(room.seckey)){
+		wrote=write(fd,room.seckey+offset,sizeof(room.seckey)-offset);
+		if(wrote<0&&errno==EINTR)continue;
+		if(wrote<=0){if(!wrote)errno=EIO;goto done;}
+		offset+=(size_t)wrote;
+	}
+	if(fsync(fd))goto done;
+	if(close(fd)){fd=-1;goto done;} fd=-1;
+	memcpy(room_key,room.pubkey,sizeof(room.pubkey)); rc=0;
+done:
+	saved=errno;
+	if(fd>=0)close(fd);
+	if(rc&&secret_path[0])unlink(secret_path);
+	sodium_memzero(&room,sizeof(room));
+	if(rc)errno=saved;
+	return rc;
+}
+
+int cable_user_identity_load_or_create(const char *path,sleep_keypair_t *identity)
+{
+	int fd; struct stat st; char parent[1024]; char *slash; ssize_t n; uint8_t check[32];
+	if(!path||!identity||strlen(path)>=sizeof(parent)){errno=EINVAL;return -1;}
+	if(sleep_crypto_init())return -1;
+	strcpy(parent,path);slash=strrchr(parent,'/');
+	if(slash){*slash='\0';if(parent[0]&&mkdir(parent,0700)<0&&errno!=EEXIST)return -1;}
+	fd=open(path,O_RDONLY|O_CLOEXEC|O_NOFOLLOW);
+	if(fd>=0){
+		if(fstat(fd,&st)||!S_ISREG(st.st_mode)||st.st_size!=crypto_sign_SECRETKEYBYTES){close(fd);errno=EINVAL;return -1;}
+		if((st.st_mode&077)!=0&&fchmod(fd,0600)){int saved=errno;close(fd);errno=saved;return -1;}
+		n=read(fd,identity->seckey,crypto_sign_SECRETKEYBYTES);close(fd);
+		if(n!=crypto_sign_SECRETKEYBYTES||crypto_sign_ed25519_sk_to_pk(identity->pubkey,identity->seckey)!=0){errno=EIO;return -1;}
+		if(crypto_sign_ed25519_sk_to_pk(check,identity->seckey)!=0||sodium_memcmp(check,identity->pubkey,32)){errno=EINVAL;return -1;}
+		return 0;
+	}
+	if(errno!=ENOENT)return -1;
+	if(sleep_gen_keypair(identity))return -1;
+	fd=open(path,O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC|O_NOFOLLOW,0600);
+	if(fd<0){if(errno==EEXIST)return cable_user_identity_load_or_create(path,identity);return -1;}
+	n=write(fd,identity->seckey,crypto_sign_SECRETKEYBYTES);
+	if(n!=crypto_sign_SECRETKEYBYTES){int saved=errno?errno:EIO;close(fd);unlink(path);errno=saved;return -1;}
+	if(fsync(fd)){int saved=errno;close(fd);unlink(path);errno=saved;return -1;}
+	if(close(fd)){int saved=errno;unlink(path);errno=saved;return -1;}
+	return 0;
+}
+
+static void put64be(uint8_t *p,uint64_t v){int i;for(i=0;i<8;i++)p[i]=(uint8_t)(v>>(56-8*i));}
+static uint64_t get64be(const uint8_t *p){uint64_t v=0;int i;for(i=0;i<8;i++)v=(v<<8)|p[i];return v;}
+
+int cable_chat_message_encode(const cable_chat_message_t *m,uint8_t *out,size_t cap,size_t *written)
+{
+	size_t total;
+	if(!m||!out||!written||m->text_length>CABLE_CHAT_TEXT_BYTES||strlen(m->text)!=m->text_length){errno=EINVAL;return -1;}
+	total=14+m->text_length;if(cap<total){errno=ENOSPC;return -1;}
+	memcpy(out,"CBL1",4);put64be(out+4,m->timestamp_ms);out[12]=(uint8_t)(m->text_length>>8);out[13]=(uint8_t)m->text_length;
+	memcpy(out+14,m->text,m->text_length);*written=total;return 0;
+}
+
+int cable_chat_message_decode(cable_chat_message_t *m,const uint8_t *in,size_t n)
+{
+	size_t text_len;
+	if(!m||!in||n<14||memcmp(in,"CBL1",4)){errno=EPROTO;return -1;}
+	text_len=((size_t)in[12]<<8)|in[13];if(text_len>CABLE_CHAT_TEXT_BYTES||n!=14+text_len||memchr(in+14,0,text_len)){errno=EPROTO;return -1;}
+	memset(m,0,sizeof(*m));m->timestamp_ms=get64be(in+4);memcpy(m->text,in+14,text_len);m->text_length=text_len;return 0;
+}
+
+int cable_chat_seal(const uint8_t recipient[32],const uint8_t *message,size_t n,uint8_t *out,size_t cap,size_t *written)
+{
+	uint8_t curve[crypto_box_PUBLICKEYBYTES];
+	if(!recipient||(!message&&n)||!out||!written){errno=EINVAL;return -1;}
+	if(cap<n+crypto_box_SEALBYTES){errno=ENOSPC;return -1;}
+	if(crypto_sign_ed25519_pk_to_curve25519(curve,recipient)!=0){errno=EINVAL;return -1;}
+	if(crypto_box_seal(out,message,(unsigned long long)n,curve)!=0){errno=EIO;return -1;}
+	*written=n+crypto_box_SEALBYTES;return 0;
+}
+
+int cable_chat_open(const sleep_keypair_t *identity,const uint8_t *sealed,size_t n,uint8_t *out,size_t cap,size_t *written)
+{
+	uint8_t curve_pk[crypto_box_PUBLICKEYBYTES],curve_sk[crypto_box_SECRETKEYBYTES];size_t plain;
+	if(!identity||!sealed||!out||!written||n<crypto_box_SEALBYTES){errno=EINVAL;return -1;}
+	plain=n-crypto_box_SEALBYTES;if(cap<plain){errno=ENOSPC;return -1;}
+	if(crypto_sign_ed25519_pk_to_curve25519(curve_pk,identity->pubkey)!=0||crypto_sign_ed25519_sk_to_curve25519(curve_sk,identity->seckey)!=0){errno=EINVAL;return -1;}
+	if(crypto_box_seal_open(out,sealed,(unsigned long long)n,curve_pk,curve_sk)!=0){errno=EACCES;return -1;}
+	sodium_memzero(curve_sk,sizeof(curve_sk));*written=plain;return 0;
+}

extra/dat-dpi/src/cable_feed.c

@@ -0,0 +1,205 @@
+#define _POSIX_C_SOURCE 200809L
+#include "cable.h"
+#include "cable_room.h"
+#include "sleep_crypto.h"
+#include <errno.h>
+#include <stdio.h>
+#include <string.h>
+#include <time.h>
+#include <sys/stat.h>
+#include <sodium.h>
+#include <unistd.h>
+
+static int ensure_link(const char *directory, const char *link_name,
+		       const char *target)
+{
+	char path[1400], observed[1024];
+	ssize_t length;
+	struct stat info;
+	int n = snprintf(path, sizeof(path), "%s/%s", directory, link_name);
+	if (n < 0 || (size_t)n >= sizeof(path)) { errno = ENAMETOOLONG; return -1; }
+	if (symlink(target, path) == 0) return 0;
+	if (errno != EEXIST) return -1;
+	if (lstat(path, &info) || !S_ISLNK(info.st_mode)) { errno = EEXIST; return -1; }
+	length = readlink(path, observed, sizeof(observed) - 1);
+	if (length < 0) return -1;
+	observed[length] = '\0';
+	if (strcmp(observed, target)) { errno = EEXIST; return -1; }
+	return 0;
+}
+
+static int ensure_localname(const char *directory)
+{
+	char path[1400], contents[16];
+	FILE *file;
+	size_t length;
+	int n = snprintf(path, sizeof(path), "%s/localname", directory);
+	if (n < 0 || (size_t)n >= sizeof(path)) { errno = ENAMETOOLONG; return -1; }
+	file = fopen(path, "rb");
+	if (file) {
+		length = fread(contents, 1, sizeof(contents), file);
+		if (ferror(file) || fclose(file)) return -1;
+		if (length != 5 || memcmp(contents, "local", 5)) { errno = EEXIST; return -1; }
+		return 0;
+	}
+	if (errno != ENOENT) return -1;
+	file = fopen(path, "wbx");
+	if (!file) return -1;
+	if (fwrite("local", 1, 5, file) != 5 || fflush(file)) {
+		int saved = errno ? errno : EIO;
+		fclose(file); errno = saved; return -1;
+	}
+	if (fclose(file)) return -1;
+	return 0;
+}
+
+int cable_user_feed_open(const char *home, const uint8_t room_key[32],
+			 sleep_keypair_t *identity, sleep_feed_t *feed)
+{
+	char identity_path[1200], room_dir[1200], feed_dir[1240];
+	char *slash;
+	int n;
+	if (!home || !room_key || !identity || !feed) { errno = EINVAL; return -1; }
+	if (cable_user_identity_path(home, room_key, identity_path,
+				     sizeof(identity_path)) ||
+	    cable_user_identity_load_or_create(identity_path, identity)) return -1;
+	slash = strrchr(identity_path, '/');
+	if (!slash) { errno = EINVAL; goto fail_identity; }
+	*slash = '\0';
+	n = snprintf(room_dir, sizeof(room_dir), "%s/0", identity_path);
+	if (n < 0 || (size_t)n >= sizeof(room_dir)) { errno = ENAMETOOLONG; goto fail_identity; }
+	if (mkdir(room_dir, 0700) < 0 && errno != EEXIST) goto fail_identity;
+	n = snprintf(feed_dir, sizeof(feed_dir), "%s", room_dir);
+	if (n < 0 || (size_t)n >= sizeof(feed_dir)) { errno = ENAMETOOLONG; goto fail_identity; }
+	/* Kappa Core 3 / multifeed 3 stores the local writer in room/0 and
+	 * addresses its storage by the bare Hypercore names. Keep the SLEEP
+	 * filename convention as symlinked aliases over the same Hypercore files. */
+	if (ensure_link(feed_dir, "feed.tree", "tree") ||
+	    ensure_link(feed_dir, "feed.signatures", "signatures") ||
+	    ensure_link(feed_dir, "feed.bitfield", "bitfield") ||
+	    ensure_link(feed_dir, "feed.data", "data") ||
+	    ensure_link(feed_dir, "feed.key", "key") ||
+	    ensure_link(feed_dir, "secret_key", "../user-feed.key") ||
+	    ensure_localname(feed_dir)) goto fail_identity;
+	if (sleep_feed_open(feed, feed_dir, "feed", identity)) goto fail_identity;
+	return 0;
+fail_identity:
+	sodium_memzero(identity, sizeof(*identity));
+	return -1;
+}
+
+int cable_user_feed_append(const char *home, const uint8_t room_key[32],
+			   const uint8_t *record, size_t record_length,
+			   uint8_t user_key[32])
+{
+	sleep_keypair_t identity;
+	sleep_feed_t feed;
+	int rc = -1;
+	memset(&identity, 0, sizeof(identity));
+	memset(&feed, 0, sizeof(feed));
+	if (!record || !record_length || !user_key) { errno = EINVAL; return -1; }
+	if (cable_user_feed_open(home, room_key, &identity, &feed)) return -1;
+	if (sleep_feed_append(&feed, record, record_length) || sleep_feed_flush(&feed)) goto done;
+	memcpy(user_key, identity.pubkey, sizeof(identity.pubkey));
+	rc = 0;
+done:
+	sleep_feed_close(&feed);
+	sodium_memzero(&identity, sizeof(identity));
+	return rc;
+}
+
+static int append_json_text(char *out, size_t capacity, size_t *offset,
+			    const char *text)
+{
+	const unsigned char *p = (const unsigned char *)text;
+	if (*offset >= capacity) { errno = ENOSPC; return -1; }
+	out[(*offset)++] = '"';
+	for (; *p; p++) {
+		const char *escaped = NULL;
+		char control[7];
+		switch (*p) {
+		case '"': escaped = "\\\""; break;
+		case '\\': escaped = "\\\\"; break;
+		case '\b': escaped = "\\b"; break;
+		case '\f': escaped = "\\f"; break;
+		case '\n': escaped = "\\n"; break;
+		case '\r': escaped = "\\r"; break;
+		case '\t': escaped = "\\t"; break;
+		default: break;
+		}
+		if (*p < 0x20 && !escaped) {
+			snprintf(control, sizeof(control), "\\u%04x", *p);
+			escaped = control;
+		}
+		if (escaped) {
+			size_t n = strlen(escaped);
+			if (n > capacity - *offset) { errno = ENOSPC; return -1; }
+			memcpy(out + *offset, escaped, n); *offset += n;
+		} else {
+			if (*offset >= capacity) { errno = ENOSPC; return -1; }
+			out[(*offset)++] = (char)*p;
+		}
+	}
+	if (*offset >= capacity) { errno = ENOSPC; return -1; }
+	out[(*offset)++] = '"';
+	return 0;
+}
+
+int cable_user_feed_publish(const char *home, const uint8_t room_key[32],
+			    const char *record_type, const char *nick,
+			    const char *channel, const char *text,
+			    uint8_t user_key[32])
+{
+	char *record;
+	size_t capacity, offset = 0;
+	struct timespec now;
+	unsigned long long timestamp;
+	int n, rc = -1;
+	if (!home || !room_key || !record_type || !user_key ||
+	    (strcmp(record_type, "about") && strcmp(record_type, "chat/text")) ||
+	    (!strcmp(record_type, "about") && !nick) ||
+	    (!strcmp(record_type, "chat/text") && (!channel || !text))) {
+		errno = EINVAL; return -1;
+	}
+	if (strlen(record_type) > 16 || (nick && strlen(nick) > CABLE_CHAT_TEXT_BYTES) ||
+	    (channel && strlen(channel) > CABLE_ROOM_CHANNEL_BYTES) ||
+	    (text && strlen(text) > CABLE_CHAT_TEXT_BYTES)) { errno = EMSGSIZE; return -1; }
+	capacity = 512 + strlen(record_type) * 6 + (nick ? strlen(nick) * 6 : 0) +
+		(channel ? strlen(channel) * 6 : 0) + (text ? strlen(text) * 6 : 0);
+	record = malloc(capacity);
+	if (!record) return -1;
+	if (clock_gettime(CLOCK_REALTIME, &now)) goto done;
+	timestamp = (unsigned long long)now.tv_sec * 1000u +
+		(unsigned long long)now.tv_nsec / 1000000u;
+	n = snprintf(record, capacity, "{\"type\":");
+	if (n < 0 || (size_t)n >= capacity) { errno = ENOSPC; goto done; }
+	offset = (size_t)n;
+	if (append_json_text(record, capacity, &offset, record_type)) goto done;
+	n = snprintf(record + offset, capacity - offset, ",\"content\":{");
+	if (n < 0 || (size_t)n >= capacity - offset) { errno = ENOSPC; goto done; }
+	offset += (size_t)n;
+	if (!strcmp(record_type, "about")) {
+		n = snprintf(record + offset, capacity - offset, "\"name\":");
+		if (n < 0 || (size_t)n >= capacity - offset) { errno = ENOSPC; goto done; }
+		offset += (size_t)n;
+		if (append_json_text(record, capacity, &offset, nick)) goto done;
+	} else {
+		n = snprintf(record + offset, capacity - offset, "\"channel\":");
+		if (n < 0 || (size_t)n >= capacity - offset) { errno = ENOSPC; goto done; }
+		offset += (size_t)n;
+		if (append_json_text(record, capacity, &offset, channel)) goto done;
+		n = snprintf(record + offset, capacity - offset, ",\"text\":");
+		if (n < 0 || (size_t)n >= capacity - offset) { errno = ENOSPC; goto done; }
+		offset += (size_t)n;
+		if (append_json_text(record, capacity, &offset, text)) goto done;
+	}
+	n = snprintf(record + offset, capacity - offset,
+		     "},\"timestamp\":%llu}", timestamp);
+	if (n < 0 || (size_t)n >= capacity - offset) { errno = ENOSPC; goto done; }
+	offset += (size_t)n;
+	rc = cable_user_feed_append(home, room_key, (const uint8_t *)record,
+				    offset, user_key);
+done:
+	free(record);
+	return rc;
+}

extra/dat-dpi/src/cesspool.c

@@ -0,0 +1,748 @@
+#define _POSIX_C_SOURCE 200112L
+// hyperswarm dht
+#include "cesspool.h"
+#include "dat_wire.h"
+#include "sleep_proto.h"
+#include <errno.h>
+#include <string.h>
+#ifdef _WIN32
+#include <winsock2.h>
+#include <ws2tcpip.h>
+#else
+#include <arpa/inet.h>
+#include <netdb.h>
+#include <netinet/in.h>
+#endif
+#include <time.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <sys/stat.h>
+#ifndef _WIN32
+#include <unistd.h>
+#endif
+#ifdef SLEEP_SALT
+#include <sodium.h>
+#endif
+
+#define CESSPOOL_PACKET_BYTES 72u
+#define CESSPOOL_MAX_RENDEZVOUS_PEERS 34u
+#define CESSPOOL_PEER_TTL_MS 600000u
+#define CESSPOOL_BOOTSTRAP_PROBE_MS 5000u
+#define CESSPOOL_BOOTSTRAP_TIMEOUT_MS 15000u
+static uint64_t monotonic_ms(void)
+{
+	struct timespec now;
+	if (clock_gettime(CLOCK_MONOTONIC, &now) != 0) return 0;
+	return (uint64_t)now.tv_sec * 1000u + (uint64_t)now.tv_nsec / 1000000u;
+}
+static int pb_var(uint8_t tag,uint64_t value,uint8_t*out,size_t cap,size_t*off){size_t n;if(*off>=cap){errno=ENOSPC;return-1;}out[(*off)++]=tag;if(sleep_varint_encode(value,out+*off,cap-*off,&n))return-1;*off+=n;return 0;}
+static int pb_bytes(uint8_t tag,const uint8_t*data,size_t len,uint8_t*out,size_t cap,size_t*off){size_t n;if(*off>=cap){errno=ENOSPC;return-1;}out[(*off)++]=tag;if(sleep_varint_encode(len,out+*off,cap-*off,&n))return-1;*off+=n;if(len>cap-*off){errno=ENOSPC;return-1;}memcpy(out+*off,data,len);*off+=len;return 0;}
+
+int cesspool_request_encode(const cesspool_request_t*r,uint8_t*out,size_t cap,size_t*written){size_t off=0;if(!r||!out){errno=EINVAL;return-1;}if(pb_var(0x08,r->type,out,cap,&off))return-1;if(r->port&&pb_var(0x10,r->port,out,cap,&off))return-1;if(r->local_address_len&&pb_bytes(0x1a,r->local_address,r->local_address_len,out,cap,&off))return-1;if(written)*written=off;return 0;}
+int cesspool_request_decode(cesspool_request_t*r,const uint8_t*in,size_t len){size_t off=0,n;uint64_t v;if(!r||!in){errno=EINVAL;return-1;}memset(r,0,sizeof(*r));while(off<len){uint8_t tag=in[off++];if(tag==0x08||tag==0x10){if(sleep_varint_decode(in+off,len-off,&v,&n))return-1;off+=n;if(tag==0x08)r->type=(uint32_t)v;else r->port=(uint16_t)v;}else if(tag==0x1a){if(sleep_varint_decode(in+off,len-off,&v,&n))return-1;off+=n;if(v>sizeof(r->local_address)||v>len-off){errno=EPROTO;return-1;}memcpy(r->local_address,in+off,(size_t)v);r->local_address_len=(size_t)v;off+=(size_t)v;}else{errno=EPROTO;return-1;}}return 0;}
+int cesspool_response_encode(const cesspool_response_t *r, uint8_t *out,
+			     size_t cap, size_t *written)
+{
+	size_t off = 0;
+	if (!r || !out) { errno = EINVAL; return -1; }
+	if (r->peers_len && pb_bytes(0x0a, r->peers, r->peers_len, out, cap, &off)) return -1;
+	if (r->local_peers_len && pb_bytes(0x12, r->local_peers, r->local_peers_len, out, cap, &off)) return -1;
+	if (r->peer_info_len && pb_bytes(0x1a, r->peer_info, r->peer_info_len, out, cap, &off)) return -1;
+	if (written) *written = off;
+	return 0;
+}
+int cesspool_response_decode(cesspool_response_t *r, const uint8_t *in, size_t len)
+{
+	size_t off = 0, n;
+	uint64_t v;
+	if (!r || !in) { errno = EINVAL; return -1; }
+	memset(r, 0, sizeof(*r));
+	while (off < len) {
+		uint8_t tag = in[off++], *dst;
+		size_t *dstlen, capacity;
+		if (sleep_varint_decode(in + off, len - off, &v, &n)) return -1;
+		off += n;
+		if (v > len - off) { errno = EPROTO; return -1; }
+		if (tag == 0x0a) { dst = r->peers; dstlen = &r->peers_len; capacity = sizeof(r->peers); }
+		else if (tag == 0x12) { dst = r->local_peers; dstlen = &r->local_peers_len; capacity = sizeof(r->local_peers); }
+		else if (tag == 0x1a) { dst = r->peer_info; dstlen = &r->peer_info_len; capacity = sizeof(r->peer_info); }
+		else { off += (size_t)v; continue; }
+		if (v > capacity) { errno = EOVERFLOW; return -1; }
+		memcpy(dst, in + off, (size_t)v);
+		*dstlen = (size_t)v;
+		off += (size_t)v;
+	}
+	return 0;
+}
+int cesspool_process_response(cesspool_t*p,const cesspool_response_t*r,const uint8_t prefix[2]){size_t off;cesspool_peer_t peer;struct sockaddr_in*address;if(!p||!r){errno=EINVAL;return-1;}if(r->peers_len%6||r->local_peers_len%4){errno=EPROTO;return-1;}for(off=0;off<r->peers_len;off+=6){memset(&peer,0,sizeof(peer));address=(struct sockaddr_in*)&peer.address.addr;address->sin_family=AF_INET;memcpy(&address->sin_addr,r->peers+off,4);memcpy(&address->sin_port,r->peers+off+4,2);peer.address.addrlen=sizeof(*address);memcpy(peer.public_key,r->peers+off,6);cesspool_add_peer(p,&peer);}if(prefix)for(off=0;off<r->local_peers_len;off+=4){memset(&peer,0,sizeof(peer));address=(struct sockaddr_in*)&peer.address.addr;address->sin_family=AF_INET;((uint8_t*)&address->sin_addr)[0]=prefix[0];((uint8_t*)&address->sin_addr)[1]=prefix[1];((uint8_t*)&address->sin_addr)[2]=r->local_peers[off];((uint8_t*)&address->sin_addr)[3]=r->local_peers[off+1];memcpy(&address->sin_port,r->local_peers+off+2,2);peer.address.addrlen=sizeof(*address);peer.local=1;memcpy(peer.public_key,&address->sin_addr,4);memcpy(peer.public_key+4,&address->sin_port,2);cesspool_add_peer(p,&peer);}return 0;}
+
+int cesspool_discovery_key(const uint8_t key[32], uint8_t topic[32])
+{
+	if (!key || !topic) { errno = EINVAL; return -1; }
+#ifdef SLEEP_SALT
+	/* The root NETWORK_PROTOCOL token selects the discovery namespace.
+	 * With "hypercore", this matches legacy Dat's discoveryKey derivation. */
+	return crypto_generichash(topic, 32,
+				  (const unsigned char *)DAT_NETWORK_PROTOCOL,
+				  strlen(DAT_NETWORK_PROTOCOL), key, 32);
+#else
+	memcpy(topic, key, 32);
+	return 0;
+#endif
+}
+
+int cesspool_encode(const cesspool_message_t *m, uint8_t *out,
+		    size_t capacity, size_t *written)
+{
+	size_t total = CESSPOOL_PACKET_BYTES;
+	if (!m || !out) { errno = EINVAL; return -1; }
+	if (m->type > CESSPOOL_HOLEPUNCH) { errno = EINVAL; return -1; }
+	if (m->type == CESSPOOL_REPLY) {
+		if (capacity < CESSPOOL_PACKET_BYTES + 2 ||
+		    m->payload_len > UINT16_MAX ||
+		    m->payload_len > capacity - CESSPOOL_PACKET_BYTES - 2) {
+			errno = ENOSPC; return -1;
+		}
+		total += 2 + m->payload_len;
+	}
+	if (capacity < total) { errno = ENOSPC; return -1; }
+	memcpy(out, "CSP0", 4);
+	out[4] = m->type;
+	out[5] = m->type == CESSPOOL_REPLY ? 1 : 0;
+	out[6] = (uint8_t)(m->port >> 8);
+	out[7] = (uint8_t)m->port;
+	memcpy(out + 8, m->topic, 32);
+	memcpy(out + 40, m->peer_id, 32);
+	if (m->type == CESSPOOL_REPLY) {
+		out[72] = (uint8_t)(m->payload_len >> 8);
+		out[73] = (uint8_t)m->payload_len;
+		memcpy(out + 74, m->payload, m->payload_len);
+	}
+	if (written) *written = total;
+	return 0;
+}
+
+int cesspool_decode(cesspool_message_t *m, const uint8_t *in, size_t len)
+{
+	if (!m || !in) { errno = EINVAL; return -1; }
+	if (len < CESSPOOL_PACKET_BYTES || memcmp(in, "CSP0", 4) ||
+	    in[4] > CESSPOOL_HOLEPUNCH) {
+		errno = EPROTO;
+		return -1;
+	}
+	memset(m, 0, sizeof(*m));
+	m->type = in[4];
+	m->port = ((uint16_t)in[6] << 8) | in[7];
+	memcpy(m->topic, in + 8, 32);
+	memcpy(m->peer_id, in + 40, 32);
+	if (m->type == CESSPOOL_REPLY) {
+		if (in[5] != 1 || len < CESSPOOL_PACKET_BYTES + 2) { errno = EPROTO; return -1; }
+		m->payload_len = ((size_t)in[72] << 8) | in[73];
+		if (m->payload_len > sizeof(m->payload) || len != CESSPOOL_PACKET_BYTES + 2 + m->payload_len) { errno = EPROTO; return -1; }
+		memcpy(m->payload, in + 74, m->payload_len);
+	} else if (in[5] != 0 || len != CESSPOOL_PACKET_BYTES) { errno = EPROTO; return -1; }
+	return 0;
+}
+
+static int send_message(cesspool_t *pool,const dat_peer_t *peer,uint8_t type,const uint8_t topic[32],uint16_t port)
+{
+	cesspool_message_t message;uint8_t packet[DATNET_MAX_PACKET];size_t len;
+	memset(&message,0,sizeof(message));message.type=type;message.port=port;if(topic)memcpy(message.topic,topic,32);memcpy(message.peer_id,pool->net.peerid,32);
+	if(cesspool_encode(&message,packet,sizeof(packet),&len))return-1;
+	return datnet_send(&pool->net,peer,packet,len);
+}
+static int send_reply(cesspool_t *pool, const dat_peer_t *peer,
+		      const uint8_t topic[32], const cesspool_response_t *response)
+{
+	cesspool_message_t message;
+	uint8_t packet[DATNET_MAX_PACKET];
+	size_t payload_len, packet_len;
+	memset(&message, 0, sizeof(message));
+	message.type = CESSPOOL_REPLY;
+	memcpy(message.topic, topic, CESSPOOL_TOPIC_BYTES);
+	memcpy(message.peer_id, pool->net.peerid, CESSPOOL_PEER_BYTES);
+	if (cesspool_response_encode(response, message.payload,
+				     sizeof(message.payload), &payload_len)) return -1;
+	message.payload_len = payload_len;
+	if (cesspool_encode(&message, packet, sizeof(packet), &packet_len)) return -1;
+	return datnet_send(&pool->net, peer, packet, packet_len);
+}
+static int send_bootstrap(cesspool_t *pool,uint8_t type,const uint8_t topic[32],uint16_t port)
+{
+	static const uint16_t lan_ports[] = {DATNET_PORT,3000,3002,3004,2001,2003,2005};
+	size_t i;int sent=0,lan_sent=0;
+	if (!pool) { errno = EINVAL; return -1; }
+	for(i=0;i<pool->bootstrap_count;i++)
+		if(send_message(pool,&pool->bootstrap_nodes[i],type,topic,port)==0)sent++;
+	/* Always keep zero-configuration LAN discovery active, even when a
+	 * rendezvous node is configured. Routers do not forward these broadcasts;
+	 * this complements Internet discovery and lets same-LAN peers connect
+	 * without requiring them to share bootstrap configuration. */
+	for (i = 0; i < sizeof(lan_ports) / sizeof(lan_ports[0]); i++) {
+		dat_peer_t target;
+		struct sockaddr_in *address;
+		memset(&target, 0, sizeof(target));
+		address = (struct sockaddr_in *)&target.addr;
+		address->sin_family = AF_INET;
+		address->sin_addr.s_addr = htonl(INADDR_BROADCAST);
+		address->sin_port = htons(lan_ports[i]);
+		target.addrlen = sizeof(*address);
+		if (send_message(pool, &target, type, topic, port) == 0) lan_sent++;
+	}
+	if (pool->bootstrap_count) return (sent || lan_sent) ? 0 : -1;
+	return 0; /* LAN broadcasts are best-effort, never a join failure. */
+}
+static int has_topic(const cesspool_t *pool,const uint8_t topic[32],int announcing)
+{
+	size_t i;
+	for(i=0;i<pool->topic_count;i++)if(!memcmp(pool->topics[i].key,topic,32)&&(!announcing||pool->topics[i].announcing))return 1;
+	return 0;
+}
+static int same_peer_address(const dat_peer_t *a, const dat_peer_t *b)
+{
+	const struct sockaddr_in *aa, *bb;
+	if (!a || !b || a->addrlen < (socklen_t)sizeof(struct sockaddr_in) ||
+	    b->addrlen < (socklen_t)sizeof(struct sockaddr_in)) return 0;
+	aa = (const struct sockaddr_in *)&a->addr;
+	bb = (const struct sockaddr_in *)&b->addr;
+	return aa->sin_family == AF_INET && bb->sin_family == AF_INET &&
+	       aa->sin_addr.s_addr == bb->sin_addr.s_addr && aa->sin_port == bb->sin_port;
+}
+
+static int announcement_add(cesspool_t *pool, const uint8_t topic[32],
+			    const uint8_t peer_id[32], const dat_peer_t *address,
+			    uint64_t now)
+{
+	size_t i;
+	for (i = 0; i < pool->announcement_count; i++) {
+		cesspool_announcement_t *entry = &pool->announcements[i];
+		if (!memcmp(entry->topic, topic, 32) && !memcmp(entry->peer_id, peer_id, 32)) {
+			entry->address = *address;
+			entry->last_seen = now;
+			return 0;
+		}
+	}
+	if (pool->announcement_count >= CESSPOOL_MAX_ANNOUNCEMENTS) {
+		errno = ENOSPC;
+		return -1;
+	}
+	{
+		cesspool_announcement_t *entry = &pool->announcements[pool->announcement_count++];
+		memcpy(entry->topic, topic, 32);
+		memcpy(entry->peer_id, peer_id, 32);
+		entry->address = *address;
+		entry->last_seen = now;
+	}
+	return 0;
+}
+
+static void announcement_remove(cesspool_t *pool, const uint8_t topic[32],
+				const uint8_t peer_id[32])
+{
+	size_t i = 0;
+	while (i < pool->announcement_count) {
+		cesspool_announcement_t *entry = &pool->announcements[i];
+		if (!memcmp(entry->topic, topic, 32) && !memcmp(entry->peer_id, peer_id, 32)) {
+			pool->announcements[i] = pool->announcements[--pool->announcement_count];
+			continue;
+		}
+		i++;
+	}
+}
+
+static int peer_has_announcement(const cesspool_t *pool, const uint8_t peer_id[32])
+{
+	size_t i;
+	for (i = 0; i < pool->announcement_count; i++)
+		if (!memcmp(pool->announcements[i].peer_id, peer_id, 32)) return 1;
+	return 0;
+}
+
+static int send_rendezvous_reply(cesspool_t *pool, dat_peer_t *requester,
+				 const uint8_t topic[32])
+{
+	cesspool_response_t response;
+	size_t i;
+	memset(&response, 0, sizeof(response));
+	for (i = 0; i < pool->announcement_count; i++) {
+		const cesspool_announcement_t *peer = &pool->announcements[i];
+		const struct sockaddr_in *address;
+		uint8_t *record;
+		if (memcmp(peer->topic, topic, 32) || same_peer_address(&peer->address, requester) ||
+		    peer->address.addrlen < (socklen_t)sizeof(struct sockaddr_in)) continue;
+		address = (const struct sockaddr_in *)&peer->address.addr;
+		if (address->sin_family != AF_INET ||
+		    response.peer_info_len / 38 >= CESSPOOL_MAX_RENDEZVOUS_PEERS ||
+		    response.peer_info_len + 38 > sizeof(response.peer_info)) continue;
+		record = response.peer_info + response.peer_info_len;
+		memcpy(record, &address->sin_addr, 4);
+		memcpy(record + 4, &address->sin_port, 2);
+		memcpy(record + 6, peer->peer_id, 32);
+		response.peer_info_len += 38;
+	}
+	return send_reply(pool, requester, topic, &response);
+}
+
+static int receive_rendezvous_reply(cesspool_t *pool,
+				    const cesspool_message_t *message)
+{
+	cesspool_response_t response;
+	size_t i;
+	if (cesspool_response_decode(&response, message->payload, message->payload_len)) return -1;
+	if (response.peer_info_len % 38) { errno = EPROTO; return -1; }
+	for (i = 0; i < response.peer_info_len; i += 38) {
+		const uint8_t *record = response.peer_info + i;
+		cesspool_peer_t peer;
+		struct sockaddr_in *address;
+		memset(&peer, 0, sizeof(peer));
+		address = (struct sockaddr_in *)&peer.address.addr;
+		address->sin_family = AF_INET;
+		memcpy(&address->sin_addr, record, 4);
+		memcpy(&address->sin_port, record + 4, 2);
+		peer.address.addrlen = sizeof(*address);
+		memcpy(peer.public_key, record + 6, 32);
+		memcpy(peer.discovery_topic, message->topic, 32);
+		peer.discovered = 1;
+		peer.last_seen = monotonic_ms();
+		if (announcement_add(pool, message->topic, peer.public_key,
+				    &peer.address, peer.last_seen)) continue;
+		if (cesspool_add_peer(pool, &peer)) return -1;
+	}
+	return 0;
+}
+
+static void cesspool_on_message(datnet_t *net,dat_peer_t *source,const uint8_t *packet,size_t length)
+{
+	cesspool_t *pool=(cesspool_t *)net->context;
+	cesspool_message_t message;
+	cesspool_peer_t peer;
+	if(!pool)return;
+	if(cesspool_decode(&message,packet,length)){
+		if(pool->on_transport_packet)pool->on_transport_packet(pool,source,packet,length);
+		return;
+	}
+	if(message.type>CESSPOOL_HOLEPUNCH||!memcmp(message.peer_id,net->peerid,32))return;
+	if (message.type == CESSPOOL_REPLY) {
+		size_t i;
+		int from_bootstrap = 0;
+		for (i = 0; i < pool->bootstrap_count; i++)
+			if (same_peer_address(&pool->bootstrap_nodes[i], source)) {
+				from_bootstrap = 1;
+				break;
+			}
+		if (!from_bootstrap) return;
+		/* A malformed datagram is not proof that the configured rendezvous
+		 * endpoint is speaking Cesspool, so validate before refreshing online. */
+		if (receive_rendezvous_reply(pool, &message)) return;
+		pool->bootstrap_last_seen = monotonic_ms();
+		pool->bootstrap_reachable = 1;
+		return;
+	}
+	if (message.type == CESSPOOL_PONG) {
+		size_t i;
+		for (i = 0; i < pool->bootstrap_count; i++) {
+			if (same_peer_address(&pool->bootstrap_nodes[i], source)) {
+				pool->bootstrap_last_seen = monotonic_ms();
+				pool->bootstrap_reachable = 1;
+				break;
+			}
+		}
+		return;
+	}
+	memset(&peer,0,sizeof(peer));peer.address=*source;memcpy(peer.public_key,message.peer_id,32);
+	peer.last_seen = monotonic_ms();
+	if (message.type == CESSPOOL_ANNOUNCE) {
+		peer.discovered = 1;
+		memcpy(peer.discovery_topic, message.topic, sizeof(peer.discovery_topic));
+		if (announcement_add(pool, message.topic, message.peer_id, source,
+				    peer.last_seen)) return;
+	}
+	cesspool_add_peer(pool,&peer);
+	if (message.type == CESSPOOL_UNANNOUNCE) {
+		cesspool_peer_t *known = cesspool_find_peer(pool, message.peer_id);
+		announcement_remove(pool, message.topic, message.peer_id);
+		if (known) known->discovered = peer_has_announcement(pool, message.peer_id);
+	} else if(message.type==CESSPOOL_PING)send_message(pool,source,CESSPOOL_PONG,NULL,pool->port);
+	else if(message.type==CESSPOOL_LOOKUP&&pool->bootstrap_server)
+		(void)send_rendezvous_reply(pool, source, message.topic);
+	else if(message.type==CESSPOOL_LOOKUP&&has_topic(pool,message.topic,1))
+		send_message(pool,source,CESSPOOL_ANNOUNCE,message.topic,pool->port);
+	else if(message.type==CESSPOOL_HOLEPUNCH&&pool->on_holepunch)pool->on_holepunch(pool,source);
+}
+int cesspool_init(cesspool_t *pool,const uint8_t peer_id[32])
+{
+	uint8_t empty[32]={0};if(!pool||!peer_id){errno=EINVAL;return-1;}memset(pool,0,sizeof(*pool));if(datnet_init(&pool->net,empty,peer_id))return-1;pool->net.context=pool;pool->net.on_message=cesspool_on_message;return 0;
+}
+int cesspool_add_bootstrap(cesspool_t *pool,const char *host,uint16_t port)
+{
+	struct addrinfo hints, *resolved = NULL;
+	struct sockaddr_in *address;
+	dat_peer_t *peer;
+	if(!pool||!host||!port||pool->bootstrap_count>=CESSPOOL_MAX_BOOTSTRAP){errno=EINVAL;return-1;}
+	memset(&hints,0,sizeof(hints));hints.ai_family=AF_INET;hints.ai_socktype=SOCK_DGRAM;
+	if(getaddrinfo(host,NULL,&hints,&resolved)!=0||!resolved){errno=EHOSTUNREACH;return-1;}
+	peer=&pool->bootstrap_nodes[pool->bootstrap_count];memset(peer,0,sizeof(*peer));
+	memcpy(&peer->addr,resolved->ai_addr,resolved->ai_addrlen);peer->addrlen=(socklen_t)resolved->ai_addrlen;
+	address=(struct sockaddr_in*)&peer->addr;address->sin_port=htons(port);
+	freeaddrinfo(resolved);
+	{
+		size_t i;
+		for (i = 0; i < pool->bootstrap_count; i++)
+			if (same_peer_address(&pool->bootstrap_nodes[i], peer)) return 0;
+	}
+	if (pool->bootstrap_count >= CESSPOOL_MAX_BOOTSTRAP) { errno = ENOSPC; return -1; }
+	pool->bootstrap_count++;return 0;
+}
+static int add_endpoint_list(cesspool_t *pool, const char *setting)
+{
+	char list[2048], *item, *next;
+	int added = 0;
+	if (!setting || !*setting) return 0;
+	if (strlen(setting) >= sizeof(list)) { errno = EOVERFLOW; return -1; }
+	strcpy(list, setting);
+	for (item = list; item && *item; item = next) {
+		char *separator, *end;
+		unsigned long port;
+		next = strpbrk(item, ",;\r\n");
+		if (next) *next++ = '\0';
+		separator = strrchr(item, ':');
+		if (!separator || separator == item || !separator[1]) continue;
+		*separator++ = '\0';
+		port = strtoul(separator, &end, 10);
+		if (*end || port == 0 || port > 65535) continue;
+		if (cesspool_add_bootstrap(pool, item, (uint16_t)port) != 0) return -1;
+		added++;
+	}
+	return added;
+}
+
+static int bootstrap_config_path(char *path, size_t capacity, int make_dirs)
+{
+	const char *home = getenv("HOME");
+	const char *config = getenv("XDG_CONFIG_HOME");
+	char directory[1024], config_directory[1024];
+	if (!home || !*home) home = getenv("USERPROFILE");
+#ifdef _WIN32
+	if ((!home || !*home) && getenv("HOMEDRIVE") && getenv("HOMEPATH")) {
+		static char combined[1024];
+		if (snprintf(combined, sizeof(combined), "%s%s", getenv("HOMEDRIVE"), getenv("HOMEPATH"))
+		    < (int)sizeof(combined)) home = combined;
+	}
+#endif
+	if (!home || !*home) home = ".";
+	if (!config || !*config) {
+		if (snprintf(config_directory, sizeof(config_directory), "%s/.config", home) >= (int)sizeof(config_directory)) {
+			errno = ENAMETOOLONG; return -1;
+		}
+		config = config_directory;
+	}
+	if (snprintf(directory, sizeof(directory), "%s/dat-gpt", config) >= (int)sizeof(directory) ||
+	    snprintf(path, capacity, "%s/bootstrap", directory) >= (int)capacity) {
+		errno = ENAMETOOLONG; return -1;
+	}
+	if (make_dirs) {
+		char parent[1024];
+		if (snprintf(parent, sizeof(parent), "%s", config) >= (int)sizeof(parent)) { errno = ENAMETOOLONG; return -1; }
+		if (mkdir(parent, 0700) && errno != EEXIST) return -1;
+		if (mkdir(directory, 0700) && errno != EEXIST) return -1;
+	}
+	return 0;
+}
+
+int cesspool_configure_bootstraps(cesspool_t *pool)
+{
+	char path[1200], line[512];
+	FILE *file;
+	int count = 0, loaded;
+	if (!pool) { errno = EINVAL; return -1; }
+	loaded = add_endpoint_list(pool, getenv("DDAT_BOOTSTRAP"));
+	if (loaded < 0) return -1;
+	count += loaded;
+	loaded = add_endpoint_list(pool, getenv("CABLE_BOOTSTRAP"));
+	if (loaded < 0) return -1;
+	count += loaded;
+	if (bootstrap_config_path(path, sizeof(path), 0)) return count;
+	file = fopen(path, "r");
+	if (!file) return errno == ENOENT ? count : -1;
+	while (fgets(line, sizeof(line), file)) {
+		line[strcspn(line, "\r\n#")] = '\0';
+		loaded = add_endpoint_list(pool, line);
+		if (loaded < 0) { fclose(file); return -1; }
+		count += loaded;
+	}
+	if (ferror(file)) { fclose(file); return -1; }
+	fclose(file);
+	return count;
+}
+
+int cesspool_bootstrap_config_add(const char *endpoint)
+{
+	char path[1200], copy[512], *separator, *end;
+	unsigned long port;
+	FILE *file;
+	if (!endpoint || strlen(endpoint) >= sizeof(copy)) { errno = EINVAL; return -1; }
+	strcpy(copy, endpoint);
+	separator = strrchr(copy, ':');
+	if (!separator || separator == copy || !separator[1]) { errno = EINVAL; return -1; }
+	*separator++ = '\0'; port = strtoul(separator, &end, 10);
+	if (*end || port == 0 || port > 65535) { errno = EINVAL; return -1; }
+	if (bootstrap_config_path(path, sizeof(path), 1)) return -1;
+	file = fopen(path, "a");
+	if (!file) return -1;
+	{
+		int failed = fprintf(file, "%s:%lu\n", copy, port) < 0;
+		if (fclose(file)) failed = 1;
+		return failed ? -1 : 0;
+	}
+}
+
+int cesspool_bootstrap_config_set(const char *endpoint)
+{
+	char path[1200], temporary[1240], copy[512], *separator, *end;
+	unsigned long port;
+	FILE *file;
+	int n, failed;
+	if (!endpoint || strlen(endpoint) >= sizeof(copy)) { errno = EINVAL; return -1; }
+	strcpy(copy, endpoint);
+	separator = strrchr(copy, ':');
+	if (!separator || separator == copy || !separator[1]) { errno = EINVAL; return -1; }
+	*separator++ = '\0';
+	port = strtoul(separator, &end, 10);
+	if (*end || port == 0 || port > 65535) { errno = EINVAL; return -1; }
+	if (bootstrap_config_path(path, sizeof(path), 1)) return -1;
+	n = snprintf(temporary, sizeof(temporary), "%s.tmp", path);
+	if (n < 0 || (size_t)n >= sizeof(temporary)) { errno = ENAMETOOLONG; return -1; }
+	file = fopen(temporary, "w");
+	if (!file) return -1;
+	failed = fprintf(file, "%s:%lu\n", copy, port) < 0;
+	if (fclose(file)) failed = 1;
+	if (failed || chmod(temporary, S_IRUSR | S_IWUSR)) {
+		int saved = errno;
+		(void)remove(temporary);
+		errno = saved;
+		return -1;
+	}
+#ifdef _WIN32
+	if (!MoveFileExA(temporary, path, MOVEFILE_REPLACE_EXISTING | MOVEFILE_WRITE_THROUGH)) {
+		(void)remove(temporary);
+		errno = EIO;
+		return -1;
+	}
+#else
+	if (rename(temporary, path)) {
+		int saved = errno;
+		(void)remove(temporary);
+		errno = saved;
+		return -1;
+	}
+#endif
+	return 0;
+}
+
+int cesspool_bootstrap_config_get(char *endpoint, size_t capacity)
+{
+	char path[1200], line[512];
+	FILE *file;
+	size_t length;
+	if (!endpoint || !capacity) { errno = EINVAL; return -1; }
+	endpoint[0] = '\0';
+	if (bootstrap_config_path(path, sizeof(path), 0)) return -1;
+	file = fopen(path, "r");
+	if (!file) return -1;
+	while (fgets(line, sizeof(line), file)) {
+		line[strcspn(line, "\r\n#")] = '\0';
+		if (!line[0]) continue;
+		length = strlen(line);
+		if (length >= capacity) { fclose(file); errno = ENOSPC; return -1; }
+		memcpy(endpoint, line, length + 1);
+		fclose(file);
+		return 0;
+	}
+	fclose(file);
+	errno = ENOENT;
+	return -1;
+}
+
+int cesspool_bootstrap_config_list(char *contents, size_t capacity)
+{
+	char path[1200];
+	FILE *file;
+	size_t length;
+	if (!contents || !capacity) { errno = EINVAL; return -1; }
+	contents[0] = '\0';
+	if (bootstrap_config_path(path, sizeof(path), 0)) return -1;
+	file = fopen(path, "r");
+	if (!file) return -1;
+	length = fread(contents, 1, capacity - 1, file);
+	if (ferror(file)) { int saved = errno; fclose(file); errno = saved; return -1; }
+	if (!feof(file)) { fclose(file); errno = ENOSPC; return -1; }
+	contents[length] = '\0';
+	return fclose(file) == 0 ? 0 : -1;
+}
+int cesspool_bootstrap(cesspool_t *pool){if(!pool||!pool->bootstrap_count){errno=EINVAL;return-1;}return send_bootstrap(pool,CESSPOOL_PING,NULL,pool->port);}
+int cesspool_listen(cesspool_t *pool,const char *host,uint16_t port){
+	static const uint16_t defaults[]={DATNET_PORT,3000,3002,3004,2001,2003,2005};
+	struct sockaddr_in bound;socklen_t length=sizeof(bound);size_t i,count;
+	if(!pool){errno=EINVAL;return-1;}
+	count=port?1:sizeof(defaults)/sizeof(defaults[0]);
+	for(i=0;i<count;i++){
+		uint16_t candidate=port?port:defaults[i];
+		if(datnet_bind(&pool->net,host,candidate)==0)break;
+		if(port||errno!=EADDRINUSE)return-1;
+	}
+	if(i==count){errno=EADDRINUSE;return-1;}
+	if(getsockname(pool->net.sock,(struct sockaddr*)&bound,&length)){datnet_close(&pool->net);return-1;}
+	pool->port=ntohs(bound.sin_port);pool->ready=1;return 0;
+}
+int cesspool_set_bootstrap_server(cesspool_t *pool, int enabled)
+{
+	if (!pool || !pool->ready) { errno = EINVAL; return -1; }
+	pool->bootstrap_server = !!enabled;
+	return 0;
+}
+int cesspool_announce(cesspool_t *pool,const uint8_t topic[32],uint16_t port){if(!pool||!topic){errno=EINVAL;return-1;}return send_bootstrap(pool,CESSPOOL_ANNOUNCE,topic,port?port:pool->port);}
+int cesspool_lookup(cesspool_t *pool,const uint8_t topic[32]){if(!pool||!topic){errno=EINVAL;return-1;}return send_bootstrap(pool,CESSPOOL_LOOKUP,topic,0);}
+int cesspool_unannounce(cesspool_t *pool,const uint8_t topic[32],uint16_t port){if(!pool||!topic){errno=EINVAL;return-1;}return send_bootstrap(pool,CESSPOOL_UNANNOUNCE,topic,port);}
+int cesspool_ping(cesspool_t *pool,const dat_peer_t *peer){if(!pool||!peer){errno=EINVAL;return-1;}return send_message(pool,peer,CESSPOOL_PING,NULL,pool->port);}
+int cesspool_holepunch(cesspool_t *pool,const dat_peer_t *peer){if(!pool||!peer){errno=EINVAL;return-1;}return send_message(pool,peer,CESSPOOL_HOLEPUNCH,NULL,pool->port);}
+int cesspool_poll(cesspool_t *pool,int timeout_ms){if(!pool){errno=EINVAL;return-1;}return datnet_poll(&pool->net,timeout_ms);}
+cesspool_peer_t *cesspool_find_peer(cesspool_t*p,const uint8_t key[32]){size_t i;if(!p||!key)return NULL;for(i=0;i<p->peer_count;i++)if(!memcmp(p->peers[i].public_key,key,32))return&p->peers[i];return NULL;}
+int cesspool_add_peer(cesspool_t*p,const cesspool_peer_t*peer){cesspool_peer_t*old;int newly_discovered;if(!p||!peer){errno=EINVAL;return-1;}old=cesspool_find_peer(p,peer->public_key);if(old){newly_discovered=peer->discovered&&(!old->discovered||memcmp(peer->discovery_topic,old->discovery_topic,32));{cesspool_peer_t merged=*peer;if(!peer->discovered&&old->discovered){merged.discovered=1;memcpy(merged.discovery_topic,old->discovery_topic,32);}*old=merged;}if(newly_discovered&&p->on_peer)p->on_peer(p,old);return 0;}if(p->peer_count>=CESSPOOL_MAX_PEERS){errno=ENOSPC;return-1;}p->peers[p->peer_count++]=*peer;if(peer->discovered&&p->on_peer)p->on_peer(p,peer);return 0;}
+int cesspool_remove_peer(cesspool_t*p,const uint8_t key[32]){size_t i;if(!p||!key){errno=EINVAL;return-1;}for(i=0;i<p->peer_count;i++)if(!memcmp(p->peers[i].public_key,key,32)){p->peers[i]=p->peers[--p->peer_count];return 0;}errno=ENOENT;return-1;}
+int cesspool_join(cesspool_t *p, const uint8_t topic[32], uint16_t port,
+		 int announce, int lookup)
+{
+	cesspool_topic_t entry;
+	int announced_now = 0;
+	if (!p || !topic || (!announce && !lookup)) { errno = EINVAL; return -1; }
+	if (p->topic_count >= CESSPOOL_MAX_TOPICS) { errno = ENOSPC; return -1; }
+	memset(&entry, 0, sizeof(entry));
+	memcpy(entry.key, topic, CESSPOOL_TOPIC_BYTES);
+	entry.port = port ? port : p->port;
+	entry.announcing = !!announce;
+	entry.looking_up = !!lookup;
+	/* No configured rendezvous falls back to best-effort LAN broadcast. */
+	if (announce) {
+		if (cesspool_announce(p, topic, entry.port)) return -1;
+		announced_now = 1;
+	}
+	if (lookup && cesspool_lookup(p, topic)) {
+		if (announced_now) (void)cesspool_unannounce(p, topic, entry.port);
+		return -1;
+	}
+	p->topics[p->topic_count++] = entry;
+	return 0;
+}
+int cesspool_leave(cesspool_t*p,const uint8_t topic[32]){size_t i;if(!p||!topic){errno=EINVAL;return-1;}for(i=0;i<p->topic_count;i++)if(!memcmp(p->topics[i].key,topic,32)){if(p->topics[i].announcing)cesspool_unannounce(p,topic,p->topics[i].port);p->topics[i]=p->topics[--p->topic_count];return 0;}errno=ENOENT;return-1;}
+int cesspool_tick(cesspool_t *p, uint64_t now)
+{
+	size_t i = 0;
+	uint64_t wall_now;
+	if (!p) { errno = EINVAL; return -1; }
+	for (i = 0; i < p->topic_count; i++) if (now >= p->topics[i].refresh_at) {
+		if (p->topics[i].announcing) (void)cesspool_announce(p, p->topics[i].key, p->topics[i].port);
+		if (p->topics[i].looking_up) (void)cesspool_lookup(p, p->topics[i].key);
+		p->topics[i].refresh_at = now + 240000u;
+	}
+	if (!p->bootstrap_server && p->bootstrap_count && now >= p->bootstrap_probe_at) {
+		int probed = 0;
+		for (i = 0; i < p->topic_count; i++) if (p->topics[i].looking_up) {
+			(void)cesspool_lookup(p, p->topics[i].key);
+			probed = 1;
+		}
+		if (!probed) (void)cesspool_bootstrap(p);
+		p->bootstrap_probe_at = now + CESSPOOL_BOOTSTRAP_PROBE_MS;
+	}
+	for (i = 0; i < CESSPOOL_MAX_TRANSACTIONS; i++)
+		if (p->transactions[i].active && now >= p->transactions[i].expires_at)
+			p->transactions[i].active = 0;
+	i = 0;
+	wall_now = monotonic_ms();
+	if (p->bootstrap_count && p->bootstrap_last_seen && wall_now >= p->bootstrap_last_seen)
+		p->bootstrap_reachable = wall_now - p->bootstrap_last_seen < CESSPOOL_BOOTSTRAP_TIMEOUT_MS;
+	else if (p->bootstrap_count) p->bootstrap_reachable = 0;
+	while (i < p->announcement_count) {
+		cesspool_announcement_t *entry = &p->announcements[i];
+		if (!entry->last_seen || (wall_now >= entry->last_seen &&
+		    wall_now - entry->last_seen >= CESSPOOL_PEER_TTL_MS)) {
+			p->announcements[i] = p->announcements[--p->announcement_count];
+			continue;
+		}
+		i++;
+	}
+	/* Client peer rows are a view of live topic announcements. Without
+	 * reconciling this bit after pruning, a peer remains "discovered" forever
+	 * even though its rendezvous record has expired. */
+	for (i = 0; i < p->peer_count; i++) {
+		cesspool_peer_t *peer = &p->peers[i];
+		size_t announcement;
+		peer->discovered = 0;
+		for (announcement = 0; announcement < p->announcement_count; announcement++) {
+			const cesspool_announcement_t *entry = &p->announcements[announcement];
+			if (memcmp(peer->public_key, entry->peer_id, CESSPOOL_PEER_BYTES)) continue;
+			peer->discovered = 1;
+			memcpy(peer->discovery_topic, entry->topic, CESSPOOL_TOPIC_BYTES);
+			if (entry->last_seen > peer->last_seen) peer->last_seen = entry->last_seen;
+			break;
+		}
+	}
+	if (p->bootstrap_server) {
+		i = 0;
+		while (i < p->peer_count) {
+			cesspool_peer_t *peer = &p->peers[i];
+			if (!peer->last_seen || (wall_now >= peer->last_seen &&
+			    wall_now - peer->last_seen >= CESSPOOL_PEER_TTL_MS)) {
+				p->peers[i] = p->peers[--p->peer_count];
+				continue;
+			}
+			i++;
+		}
+	}
+	return 0;
+}
+static int distance_less(const uint8_t*a,const uint8_t*b,const uint8_t target[32]){size_t i;for(i=0;i<32;i++){uint8_t da=a[i]^target[i],db=b[i]^target[i];if(da!=db)return da<db;}return 0;}
+size_t cesspool_closest(cesspool_t*p,const uint8_t target[32],cesspool_peer_t**out,size_t cap){size_t i,j,n=0;if(!p||!target||(!out&&cap))return 0;for(i=0;i<p->peer_count;i++){if(n<cap)out[n++]=&p->peers[i];else if(cap&&distance_less(p->peers[i].public_key,out[n-1]->public_key,target))out[n-1]=&p->peers[i];for(j=n;j>1&&distance_less(out[j-1]->public_key,out[j-2]->public_key,target);j--){cesspool_peer_t*t=out[j-1];out[j-1]=out[j-2];out[j-2]=t;}}return n;}
+size_t cesspool_topic_peers(cesspool_t *p, const uint8_t topic[32],
+			    cesspool_peer_t **out, size_t cap)
+{
+	size_t i, n = 0;
+	if (!p || !topic || (!out && cap)) return 0;
+	for (i = 0; i < p->announcement_count && n < cap; i++) {
+		cesspool_announcement_t *entry = &p->announcements[i];
+		cesspool_peer_t *peer;
+		size_t j;
+		if (memcmp(entry->topic, topic, 32)) continue;
+		for (j = 0; j < n; j++) if (!memcmp(out[j]->public_key, entry->peer_id, 32)) break;
+		if (j < n) continue;
+		peer = cesspool_find_peer(p, entry->peer_id);
+		if (peer) out[n++] = peer;
+	}
+	return n;
+}
+
+size_t cesspool_announced_peer_count(const cesspool_t *p)
+{
+	size_t i, j, count = 0;
+	if (!p) return 0;
+	for (i = 0; i < p->announcement_count; i++) {
+		for (j = 0; j < i; j++)
+			if (!memcmp(p->announcements[i].peer_id, p->announcements[j].peer_id, 32)) break;
+		if (j == i) count++;
+	}
+	return count;
+}
+
+size_t cesspool_announced_topic_count(const cesspool_t *p)
+{
+	size_t i, j, count = 0;
+	if (!p) return 0;
+	for (i = 0; i < p->announcement_count; i++) {
+		for (j = 0; j < i; j++)
+			if (!memcmp(p->announcements[i].topic, p->announcements[j].topic, 32)) break;
+		if (j == i) count++;
+	}
+	return count;
+}
+int cesspool_transaction_begin(cesspool_t*p,uint8_t type,const uint8_t target[32],uint64_t expires,uint16_t*out){size_t i,j,attempt;uint16_t candidate;if(!p||!target||!out){errno=EINVAL;return-1;}for(i=0;i<CESSPOOL_MAX_TRANSACTIONS;i++)if(!p->transactions[i].active){for(attempt=0;attempt<UINT16_MAX;attempt++){candidate=++p->next_transaction;if(!candidate)candidate=++p->next_transaction;for(j=0;j<CESSPOOL_MAX_TRANSACTIONS;j++)if(p->transactions[j].active&&p->transactions[j].id==candidate)break;if(j==CESSPOOL_MAX_TRANSACTIONS){cesspool_transaction_t*t=&p->transactions[i];memset(t,0,sizeof(*t));t->active=1;t->type=type;t->id=candidate;t->expires_at=expires;memcpy(t->target,target,32);*out=t->id;return 0;}}errno=ENOSPC;return-1;}errno=ENOSPC;return-1;}
+int cesspool_transaction_finish(cesspool_t*p,uint16_t id){size_t i;if(!p){errno=EINVAL;return-1;}for(i=0;i<CESSPOOL_MAX_TRANSACTIONS;i++)if(p->transactions[i].active&&p->transactions[i].id==id){p->transactions[i].active=0;return 0;}errno=ENOENT;return-1;}
+int cesspool_destroy(cesspool_t *pool){if(!pool){errno=EINVAL;return-1;}pool->ready=0;return datnet_close(&pool->net);}

extra/dat-dpi/src/dat.c

@@ -0,0 +1,235 @@
+#include "dat.h"
+#include "base32768.h"
+#include "sleep.h"
+#include "sleep_crypto.h"
+#include "sleep_proto.h"
+#include "sleep_writer.h"
+#include "sleep_feed.h"
+#include "errno_compat.h"
+
+#include <errno.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <unistd.h>
+#include <sodium.h>
+
+int dat_key_to_hex(const uint8_t key[DAT_KEY_BYTES], char hex[65])
+{
+	static const char digits[] = "0123456789abcdef";
+	size_t i;
+	if (!key || !hex) { errno = EINVAL; return -1; }
+	for (i = 0; i < DAT_KEY_BYTES; i++) {
+		hex[i * 2] = digits[key[i] >> 4];
+		hex[i * 2 + 1] = digits[key[i] & 15];
+	}
+	hex[64] = '\0';
+	return 0;
+}
+
+int dat_key_to_url(const uint8_t key[DAT_KEY_BYTES], char url[DAT_URL_BYTES])
+{
+	if (!key || !url) { errno = EINVAL; return -1; }
+	memcpy(url, "dat://", 6);
+	if (dat_key_to_hex(key, url + 6)) return -1;
+	return 0;
+}
+int dat_key_to_compact_url(const uint8_t key[DAT_KEY_BYTES], char url[DAT_URL_BYTES])
+{
+	if (!key || !url) { errno = EINVAL; return -1; }
+	memcpy(url, "dat://", 6);
+	if (base32768_encode(key, DAT_KEY_BYTES, url + 6, DAT_URL_BYTES - 6)) return -1;
+	return 0;
+}
+static int hex_value(char c){if(c>='0'&&c<='9')return c-'0';if(c>='a'&&c<='f')return c-'a'+10;if(c>='A'&&c<='F')return c-'A'+10;return-1;}
+int dat_url_to_key(const char *url, uint8_t key[DAT_KEY_BYTES])
+{
+	size_t i, length, decoded_length = 0;
+	int hi, lo;
+	if (!url || !key) { errno = EINVAL; return -1; }
+	if (!strncmp(url, "dat://", 6)) url += 6;
+	length = strlen(url);
+	if (length == 64) {
+		for (i = 0; i < DAT_KEY_BYTES; i++) {
+			hi = hex_value(url[i * 2]); lo = hex_value(url[i * 2 + 1]);
+			if (hi < 0 || lo < 0) { errno = EINVAL; return -1; }
+			key[i] = (uint8_t)((hi << 4) | lo);
+		}
+		return 0;
+	}
+	if (base32768_decode(url, key, DAT_KEY_BYTES, &decoded_length) ||
+	    decoded_length != DAT_KEY_BYTES) { errno = EINVAL; return -1; }
+	return 0;
+}
+static int dat_link_format_with(const dat_link_t*l,char*out,size_t cap,int compact){char base[DAT_URL_BYTES];int n;if(!l||!out){errno=EINVAL;return-1;}if((compact?dat_key_to_compact_url(l->key,base):dat_key_to_url(l->key,base)))return-1;if(l->has_version)n=snprintf(out,cap,"%s+%llu%s%s",base,(unsigned long long)l->version,*l->path&&l->path[0]!='/'?"/":"",l->path);else n=snprintf(out,cap,"%s%s%s",base,*l->path&&l->path[0]!='/'?"/":"",l->path);if(n<0||(size_t)n>=cap){errno=ENOSPC;return-1;}return 0;}
+int dat_link_format(const dat_link_t*l,char*out,size_t cap){return dat_link_format_with(l,out,cap,0);}
+int dat_link_format_compact(const dat_link_t*l,char*out,size_t cap){return dat_link_format_with(l,out,cap,1);}
+int dat_link_parse(const char *text, dat_link_t *link)
+{
+	const char *p, *suffix, *plus;
+	char encoded[DAT_URL_BYTES], version[32];
+	size_t key_length, n;
+	if (!text || !link) { errno = EINVAL; return -1; }
+	memset(link, 0, sizeof(*link));
+	p = !strncmp(text, "dat://", 6) ? text + 6 : text;
+	suffix = strpbrk(p, "/+?#");
+	key_length = suffix ? (size_t)(suffix - p) : strlen(p);
+	if (!key_length || key_length >= sizeof(encoded)) { errno = EINVAL; return -1; }
+	memcpy(encoded, p, key_length); encoded[key_length] = '\0';
+	if (dat_url_to_key(encoded, link->key)) return -1;
+	suffix = p + key_length;
+	plus = *suffix == '+' ? suffix : NULL;
+	if (plus) {
+		const char *end = plus + 1;
+		while (*end >= '0' && *end <= '9') end++;
+		n = (size_t)(end - (plus + 1));
+		if (!n || n >= sizeof(version)) { errno = EINVAL; return -1; }
+		memcpy(version, plus + 1, n); version[n] = '\0';
+		link->version = (uint64_t)strtoull(version, NULL, 10);
+		link->has_version = 1; suffix = end;
+	}
+	if (*suffix && *suffix != '/') { errno = EINVAL; return -1; }
+	if (*suffix == '/') suffix++;
+	if (strlen(suffix) >= sizeof(link->path)) { errno = ENAMETOOLONG; return -1; }
+	strcpy(link->path, suffix);
+	return 0;
+}
+
+int dat_archive_create(dat_archive_t *archive, const char *root)
+{
+	sleep_keypair_t metadata_keys, content_keys;
+	sleep_metadata_header_t header;
+	uint8_t metadata_block[128], empty = 0;
+	const uint8_t *metadata_blocks[1], *content_blocks[1];
+	size_t metadata_lengths[1], content_lengths[1] = { 0 };
+	size_t metadata_len;
+	int n;
+
+	if (!archive || !root || !*root) { errno = EINVAL; return -1; }
+	memset(archive, 0, sizeof(*archive));
+	n = snprintf(archive->dat_path, sizeof(archive->dat_path), "%s/.dat", root);
+	if (n < 0 || (size_t)n >= sizeof(archive->dat_path)) { errno = ENAMETOOLONG; return -1; }
+	if (sleep_init(root) || sleep_crypto_init() || sleep_gen_keypair(&metadata_keys) || sleep_gen_keypair(&content_keys)) return -1;
+
+	memset(&header, 0, sizeof(header));
+	strcpy(header.type, "hyperdrive");
+	memcpy(header.content, content_keys.pubkey, 32); header.content_len = 32;
+	if (sleep_metadata_header_encode(&header, metadata_block, sizeof(metadata_block), &metadata_len)) return -1;
+
+	metadata_blocks[0] = metadata_block; metadata_lengths[0] = metadata_len;
+	content_blocks[0] = &empty;
+	if (sleep_write_feed(archive->dat_path, "metadata", metadata_blocks, metadata_lengths, 1, &metadata_keys)) return -1;
+	if (sleep_write_feed(archive->dat_path, "content", content_blocks, content_lengths, 1, &content_keys)) return -1;
+	memcpy(archive->metadata_key, metadata_keys.pubkey, 32);
+	memcpy(archive->content_key, content_keys.pubkey, 32);
+	archive->metadata_keys = metadata_keys;
+	archive->content_keys = content_keys;
+	return 0;
+}
+
+int dat_archive_open(dat_archive_t*a,const char*root){char path[1200];int n;if(!a||!root){errno=EINVAL;return-1;}memset(a,0,sizeof(*a));n=snprintf(a->dat_path,sizeof(a->dat_path),"%s/.dat",root);if(n<0||(size_t)n>=sizeof(a->dat_path)){errno=ENAMETOOLONG;return-1;}snprintf(path,sizeof(path),"%s/metadata.key",a->dat_path);if(sleep_load_pubkey(path,a->metadata_key))return-1;snprintf(path,sizeof(path),"%s/content.key",a->dat_path);return sleep_load_pubkey(path,a->content_key);}
+/* Dat Desktop's dat-secret-storage indexes each Hypercore secret by the
+ * discovery key and stores the raw 64-byte Ed25519 secret at that path. */
+static int feed_secret_path(char *out,size_t cap,const char *root,const uint8_t pub[32])
+{
+	uint8_t disc[32]; char hex[65], shard[1200]; int n;
+	if(sleep_compute_discovery_key(pub,disc)||dat_key_to_hex(disc,hex))return -1;
+	n=snprintf(shard,sizeof(shard),"%s/%c%c",root,hex[0],hex[1]);
+	if(n<0||(size_t)n>=sizeof(shard)){errno=ENAMETOOLONG;return -1;}
+	n=snprintf(out,cap,"%s/%s",shard,hex+2);
+	if(n<0||(size_t)n>=cap){errno=ENAMETOOLONG;return -1;} return 0;
+}
+
+static int load_feed_secret(const char *root,const uint8_t pub[32],sleep_keypair_t *kp)
+{
+	char path[1400]; FILE *f; uint8_t raw[sizeof(*kp)]; size_t n; int saved;
+	if(feed_secret_path(path,sizeof(path),root,pub))return -1;
+	f=fopen(path,"rb");
+	if(f){n=fread(raw,1,sizeof(raw),f);if(ferror(f)){saved=errno;fclose(f);errno=saved;return -1;}fclose(f);
+		if(n==SLEEP_SECRET_KEY_BYTES){memcpy(kp->seckey,raw,n);if(crypto_sign_ed25519_sk_to_pk(kp->pubkey,kp->seckey)!=0||memcmp(kp->pubkey,pub,32)){errno=EKEYREJECTED;return -1;}return 0;}
+		errno=EINVAL;return -1;
+	}
+	return -1;
+}
+
+static int save_feed_secret(const char *root,const sleep_keypair_t *kp)
+{
+	char path[1400], parent[1400], *slash; FILE *f; int fd, saved;
+	if(feed_secret_path(path,sizeof(path),root,kp->pubkey))return -1;
+	strcpy(parent,path); slash=strrchr(parent,'/'); if(!slash){errno=EINVAL;return -1;} *slash='\0';
+	if(mkdir(parent,0700)&&errno!=EEXIST)return -1;
+	f=fopen(path,"rb"); if(f){uint8_t raw[sizeof(*kp)];size_t n=fread(raw,1,sizeof(raw),f);fclose(f);
+		if(n==SLEEP_SECRET_KEY_BYTES&&!memcmp(raw,kp->seckey,SLEEP_SECRET_KEY_BYTES))return 0;
+		errno=EEXIST;return -1;
+	}
+	fd=open(path,O_WRONLY|O_CREAT|O_EXCL,0600);if(fd<0){if(errno==EEXIST)return save_feed_secret(root,kp);return -1;}
+	if(write(fd,kp->seckey,SLEEP_SECRET_KEY_BYTES)!=(ssize_t)SLEEP_SECRET_KEY_BYTES){saved=errno;close(fd);unlink(path);errno=saved?saved:EIO;return -1;}
+	if(close(fd)){saved=errno;unlink(path);errno=saved;return -1;}return 0;
+}
+
+int dat_archive_open_writable(dat_archive_t*a,const char*root,const char*secrets)
+{
+	if(dat_archive_open(a,root))return -1;
+	if(!secrets){errno=EINVAL;return -1;}
+	if(load_feed_secret(secrets,a->metadata_key,&a->metadata_keys)||
+	   load_feed_secret(secrets,a->content_key,&a->content_keys))return -1;
+	return 0;
+}
+
+int dat_archive_save_secrets(const dat_archive_t*a,const char*secrets)
+{
+	if(!a||!secrets){errno=EINVAL;return -1;}
+	if(mkdir(secrets,0700)&&errno!=EEXIST)return -1;
+	if(save_feed_secret(secrets,&a->metadata_keys))return -1;
+	return save_feed_secret(secrets,&a->content_keys);
+}
+void dat_archive_close(dat_archive_t*a){if(a)memset(a,0,sizeof(*a));}
+size_t dat_archive_entry_count(const dat_archive_t*a){sleep_feed_t f;size_t n=0;if(a&&sleep_feed_open(&f,a->dat_path,"metadata",NULL)==0){n=sleep_feed_length(&f);sleep_feed_close(&f);}return n?n-1:0;}
+uint64_t dat_archive_version(const dat_archive_t*a){return (uint64_t)dat_archive_entry_count(a);}
+
+int dat_archive_append_file(dat_archive_t *a,const char *path,const uint8_t *data,size_t length,uint32_t mode,uint64_t mtime)
+{
+	sleep_feed_t content,metadata;sleep_metadata_entry_t entry;uint8_t encoded[2300];size_t encoded_len,block,off=0,count=0,part;
+	if(!a||!path||(!data&&length)||strlen(path)>=sizeof(entry.path)){errno=EINVAL;return-1;}
+	if(sleep_feed_open(&content,a->dat_path,"content",&a->content_keys))return-1;
+	block=sleep_feed_length(&content);do{part=length-off>DAT_BLOCK_BYTES?DAT_BLOCK_BYTES:length-off;if(sleep_feed_append(&content,data?data+off:NULL,part)){sleep_feed_close(&content);return-1;}off+=part;count++;}while(off<length);if(sleep_feed_flush(&content)){sleep_feed_close(&content);return-1;}sleep_feed_close(&content);
+	memset(&entry,0,sizeof(entry));strcpy(entry.path,path);entry.type=SLEEP_ENTRY_FILE;entry.mode=mode;entry.size=length;entry.mtime=mtime;entry.block_offset=block;entry.block_count=count;
+	if(sleep_metadata_entry_encode(&entry,encoded,sizeof(encoded),&encoded_len))return-1;
+	if(sleep_feed_open(&metadata,a->dat_path,"metadata",&a->metadata_keys))return-1;
+	if(sleep_feed_append(&metadata,encoded,encoded_len)||sleep_feed_flush(&metadata)){sleep_feed_close(&metadata);return-1;}sleep_feed_close(&metadata);return 0;
+}
+int dat_archive_read_entry(const dat_archive_t *a,size_t index,sleep_metadata_entry_t *entry)
+{
+	sleep_feed_t f;uint8_t *data;size_t len;int rc;if(!a||!entry){errno=EINVAL;return-1;}if(sleep_feed_open(&f,a->dat_path,"metadata",NULL))return-1;
+	/* Metadata block zero is the archive header. */ rc=sleep_feed_read(&f,index+1,&data,&len);sleep_feed_close(&f);if(rc)return-1;rc=sleep_metadata_entry_decode(entry,data,len);free(data);return rc;
+}
+int dat_archive_read_file(const dat_archive_t *a,const sleep_metadata_entry_t *entry,uint8_t **data,size_t *length)
+{
+	sleep_feed_t f;uint8_t*out,*block;size_t total=0,n,i;if(!a||!entry||!data||!length||entry->type!=SLEEP_ENTRY_FILE){errno=EINVAL;return-1;}if(entry->size>SIZE_MAX){errno=EOVERFLOW;return-1;}out=malloc(entry->size?(size_t)entry->size:1);if(!out)return-1;if(sleep_feed_open(&f,a->dat_path,"content",NULL)){free(out);return-1;}for(i=0;i<entry->block_count;i++){if(sleep_feed_read(&f,(size_t)entry->block_offset+i,&block,&n)||n>(size_t)entry->size-total){sleep_feed_close(&f);free(out);return-1;}if(n)memcpy(out+total,block,n);total+=n;free(block);}sleep_feed_close(&f);if(total!=(size_t)entry->size){free(out);errno=EPROTO;return-1;}*data=out;*length=total;return 0;
+}
+
+static int append_metadata_only(dat_archive_t *a,const sleep_metadata_entry_t *entry)
+{
+	sleep_feed_t metadata;uint8_t encoded[2300];size_t len;int rc;
+	if(sleep_metadata_entry_encode(entry,encoded,sizeof(encoded),&len))return-1;
+	if(sleep_feed_open(&metadata,a->dat_path,"metadata",&a->metadata_keys))return-1;
+	rc=sleep_feed_append(&metadata,encoded,len);if(!rc)rc=sleep_feed_flush(&metadata);sleep_feed_close(&metadata);return rc;
+}
+int dat_archive_append_directory(dat_archive_t *a,const char *path,uint32_t mode,uint64_t mtime)
+{
+	sleep_metadata_entry_t e;if(!a||!path||strlen(path)>=sizeof(e.path)){errno=EINVAL;return-1;}memset(&e,0,sizeof(e));strcpy(e.path,path);e.type=SLEEP_ENTRY_DIRECTORY;e.mode=mode;e.mtime=mtime;return append_metadata_only(a,&e);
+}
+int dat_archive_append_symlink(dat_archive_t *a,const char *path,const char *target,uint64_t mtime)
+{
+	sleep_metadata_entry_t e;if(!a||!path||!target||strlen(path)>=sizeof(e.path)||strlen(target)>=sizeof(e.linkname)){errno=EINVAL;return-1;}memset(&e,0,sizeof(e));strcpy(e.path,path);strcpy(e.linkname,target);e.type=SLEEP_ENTRY_SYMLINK;e.mtime=mtime;return append_metadata_only(a,&e);
+}
+int dat_archive_delete(dat_archive_t*a,const char*path,uint64_t mtime){sleep_metadata_entry_t e;if(!a||!path||strlen(path)>=sizeof(e.path)){errno=EINVAL;return-1;}memset(&e,0,sizeof(e));strcpy(e.path,path);e.type=SLEEP_ENTRY_DELETE;e.mtime=mtime;return append_metadata_only(a,&e);}
+int dat_archive_find(const dat_archive_t*a,const char*path,sleep_metadata_entry_t*entry){size_t n,i;if(!a||!path||!entry){errno=EINVAL;return-1;}n=dat_archive_entry_count(a);for(i=n;i>0;i--){if(dat_archive_read_entry(a,i-1,entry)==0&&!strcmp(entry->path,path)){if(entry->type==SLEEP_ENTRY_DELETE){errno=ENOENT;return-1;}return 0;}}errno=ENOENT;return-1;}
+int dat_archive_find_at(const dat_archive_t*a,const char*path,uint64_t version,sleep_metadata_entry_t*entry){size_t n,i;if(!a||!path||!entry){errno=EINVAL;return-1;}n=dat_archive_entry_count(a);if(version>n){errno=ERANGE;return-1;}for(i=(size_t)version;i>0;i--){if(dat_archive_read_entry(a,i-1,entry)==0&&!strcmp(entry->path,path)){if(entry->type==SLEEP_ENTRY_DELETE){errno=ENOENT;return-1;}return 0;}}errno=ENOENT;return-1;}
+int dat_checkout_init(dat_checkout_t*c,const dat_archive_t*a,uint64_t version){if(!c||!a||version>dat_archive_version(a)){errno=ERANGE;return-1;}c->archive=a;c->version=version;return 0;}
+int dat_checkout_find(const dat_checkout_t*c,const char*path,sleep_metadata_entry_t*entry){if(!c){errno=EINVAL;return-1;}return dat_archive_find_at(c->archive,path,c->version,entry);}
+int dat_checkout_read(const dat_checkout_t*c,const sleep_metadata_entry_t*entry,uint8_t**data,size_t*length){if(!c){errno=EINVAL;return-1;}return dat_archive_read_file(c->archive,entry,data,length);}
+int dat_checkout_export(const dat_checkout_t*c,const char*archive_path,const char*filesystem_path){if(!c){errno=EINVAL;return-1;}return dat_archive_export_at(c->archive,c->version,archive_path,filesystem_path);}
+int dat_archive_changes(const dat_archive_t*a,uint64_t from,uint64_t to,dat_archive_change_fn callback,void*context){sleep_metadata_entry_t entry;uint64_t version,last;if(!a||!callback){errno=EINVAL;return-1;}last=dat_archive_version(a);if(from>to||to>last){errno=ERANGE;return-1;}for(version=from+1;version<=to;version++){if(dat_archive_read_entry(a,(size_t)(version-1),&entry)||callback(&entry,version,context))return-1;}return 0;}

extra/dat-dpi/src/dat_browse.c

@@ -0,0 +1,314 @@
+#define _POSIX_C_SOURCE 200809L
+#include "dat.h"
+
+#include <ctype.h>
+#include <errno.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <strings.h>
+
+static int normalize_url_path(const char *src, char *dst, size_t capacity)
+{
+	char decoded[2048];
+	size_t i = 0, decoded_len = 0, out = 1, segments[1024], count = 0;
+	int trailing_slash;
+	if (!src || !dst || capacity < 2 || strlen(src) >= sizeof(decoded)) {
+		errno = ENAMETOOLONG;
+		return -1;
+	}
+	while (src[i]) {
+		unsigned char c = (unsigned char)src[i++];
+		if (c == '%') {
+			int hi, lo;
+			if (!src[i] || !src[i + 1]) { errno = EINVAL; return -1; }
+			hi = isdigit((unsigned char)src[i]) ? src[i] - '0' :
+				(tolower((unsigned char)src[i]) >= 'a' && tolower((unsigned char)src[i]) <= 'f' ? tolower((unsigned char)src[i]) - 'a' + 10 : -1);
+			lo = isdigit((unsigned char)src[i + 1]) ? src[i + 1] - '0' :
+				(tolower((unsigned char)src[i + 1]) >= 'a' && tolower((unsigned char)src[i + 1]) <= 'f' ? tolower((unsigned char)src[i + 1]) - 'a' + 10 : -1);
+			if (hi < 0 || lo < 0) { errno = EINVAL; return -1; }
+			c = (unsigned char)((hi << 4) | lo); i += 2;
+		}
+		if (!c || c == '\\') { errno = EINVAL; return -1; }
+		decoded[decoded_len++] = (char)c;
+	}
+	decoded[decoded_len] = '\0';
+	trailing_slash = decoded_len == 0 || decoded[decoded_len - 1] == '/';
+	dst[0] = '/';
+	i = 0;
+	while (i < decoded_len) {
+		size_t start, n;
+		while (i < decoded_len && decoded[i] == '/') i++;
+		start = i;
+		while (i < decoded_len && decoded[i] != '/') i++;
+		n = i - start;
+		if (!n) break;
+		if (n == 1 && decoded[start] == '.') { trailing_slash = 1; continue; }
+		if (n == 2 && decoded[start] == '.' && decoded[start + 1] == '.') {
+			trailing_slash = 1;
+			if (count) out = segments[--count];
+			continue;
+		}
+		trailing_slash = 0;
+		if (count >= sizeof(segments) / sizeof(segments[0]) ||
+		    out + n + 2 > capacity) { errno = ENAMETOOLONG; return -1; }
+		segments[count++] = out;
+		if (out > 1) dst[out++] = '/';
+		memcpy(dst + out, decoded + start, n);
+		out += n;
+	}
+	if (trailing_slash && out > 1) dst[out++] = '/';
+	if (out >= capacity) { errno = ENAMETOOLONG; return -1; }
+	dst[out] = '\0';
+	return 0;
+}
+
+static int parse_decimal(const char *s, uint64_t *value)
+{
+	char *end;
+	unsigned long long v;
+	if (!s || !*s || !value) return -1;
+	errno = 0; v = strtoull(s, &end, 10);
+	if (errno || *end) return -1;
+	*value = (uint64_t)v;
+	return 0;
+}
+
+int dat_url_parse(const char *text, dat_url_t *url)
+{
+	const char *authority, *path, *query, *end;
+	size_t host_len, path_len;
+	char query_copy[512], raw_path[2048], *item, *save;
+	char *plus;
+	if (!text || !url || strncmp(text, "dat://", 6)) { errno = EINVAL; return -1; }
+	memset(url, 0, sizeof(*url));
+	authority = text + 6;
+	path = strpbrk(authority, "/?");
+	end = path ? path : authority + strlen(authority);
+	host_len = (size_t)(end - authority);
+	if (!host_len || host_len >= sizeof(url->host)) { errno = EINVAL; return -1; }
+	memcpy(url->host, authority, host_len); url->host[host_len] = 0;
+	plus = strchr(url->host, '+');
+	if (plus) {
+		if (parse_decimal(plus + 1, &url->version)) { errno = EINVAL; return -1; }
+		*plus = 0; url->has_version = 1;
+		if (!url->host[0]) { errno = EINVAL; return -1; }
+	}
+	query = (*end == '?') ? end : strchr(end, '?');
+	if (*end == '/') {
+		path_len = query ? (size_t)(query - end) : strlen(end);
+		if (path_len >= sizeof(raw_path)) { errno = ENAMETOOLONG; return -1; }
+		memcpy(raw_path, end, path_len); raw_path[path_len] = 0;
+	} else { strcpy(raw_path, "/"); path_len = 1; }
+	if (path_len >= 2 && raw_path[path_len - 1] == '/' && raw_path[path_len - 2] == '/')
+		url->force_directory_listing = 1;
+	if (normalize_url_path(raw_path, url->path, sizeof(url->path))) return -1;
+	if (query) {
+		if (strlen(query + 1) >= sizeof(query_copy)) { errno = EOVERFLOW; return -1; }
+		strcpy(query_copy, query + 1);
+		for (item = strtok_r(query_copy, "&", &save); item; item = strtok_r(NULL, "&", &save)) {
+			char *equals = strchr(item, '=');
+			if (equals) *equals++ = 0;
+			if (!strcmp(item, "version")) {
+				if (parse_decimal(equals, &url->version)) { errno = EINVAL; return -1; }
+				if (url->has_version && url->version != (uint64_t)strtoull(equals, NULL, 10)) { errno = EINVAL; return -1; }
+				url->has_version = 1;
+			} else if (!strcmp(item, "stat")) {
+				url->wants_stat = !equals || strcmp(equals, "0") != 0;
+			}
+		}
+	}
+	return 0;
+}
+
+static int canonical_entry_path(const char *input, char *out, size_t capacity)
+{
+	const char *p = input;
+	size_t n;
+	while (*p == '/') p++;
+	n = strlen(p);
+	while (n && p[n - 1] == '/') n--;
+	if (n >= capacity) { errno = ENAMETOOLONG; return -1; }
+	memcpy(out, p, n); out[n] = 0;
+	return 0;
+}
+
+static int find_normalized(const dat_archive_t *archive, uint64_t version,
+			   const char *path, sleep_metadata_entry_t *entry)
+{
+	size_t i;
+	for (i = (size_t)version; i > 0; i--) {
+		sleep_metadata_entry_t current;
+		const char *candidate;
+		if (dat_archive_read_entry(archive, i - 1, &current)) return -1;
+		candidate = current.path; while (*candidate == '/') candidate++;
+		if (strcmp(candidate, path)) continue;
+		if (current.type == SLEEP_ENTRY_DELETE) { errno = ENOENT; return -1; }
+		*entry = current;
+		return 0;
+	}
+	errno = ENOENT;
+	return -1;
+}
+
+int dat_archive_stat_at(const dat_archive_t *archive, uint64_t version,
+			const char *path, dat_path_info_t *info)
+{
+	char wanted[2048];
+	size_t i;
+	if (!archive || !path || !info || canonical_entry_path(path, wanted, sizeof(wanted))) { errno = EINVAL; return -1; }
+	if (version > dat_archive_version(archive)) { errno = ERANGE; return -1; }
+	memset(info, 0, sizeof(*info));
+	if (find_normalized(archive, version, wanted, &info->entry) == 0) {
+		info->exists = 1;
+		info->is_directory = info->entry.type == SLEEP_ENTRY_DIRECTORY;
+		return 0;
+	}
+	if (errno != ENOENT) return -1;
+	for (i = 0; i < (size_t)version; i++) {
+		sleep_metadata_entry_t e, latest;
+		const char *candidate;
+		size_t prefix = strlen(wanted);
+		if (dat_archive_read_entry(archive, i, &e)) return -1;
+		candidate = e.path; while (*candidate == '/') candidate++;
+		if (prefix && (strncmp(candidate, wanted, prefix) || candidate[prefix] != '/')) continue;
+		if (!prefix && !*candidate) continue;
+		if (find_normalized(archive, version, candidate, &latest)) {
+			if (errno == ENOENT) continue;
+			return -1;
+		}
+		if (latest.type == SLEEP_ENTRY_DELETE) continue;
+		memset(info, 0, sizeof(*info));
+		strcpy(info->entry.path, wanted);
+		info->entry.type = SLEEP_ENTRY_DIRECTORY;
+		info->exists = 1; info->is_directory = 1;
+		return 0;
+	}
+	errno = ENOENT;
+	return -1;
+}
+
+int dat_archive_readdir_at(const dat_archive_t *archive, uint64_t version,
+			   const char *path, dat_archive_readdir_fn callback,
+			   void *context)
+{
+	char wanted[2048];
+	size_t i, prefix, seen_count = 0;
+	char **seen;
+	dat_path_info_t info;
+	if (!archive || !path || !callback || canonical_entry_path(path, wanted, sizeof(wanted))) { errno = EINVAL; return -1; }
+	if (dat_archive_stat_at(archive, version, wanted, &info) || !info.is_directory) { if (!errno) errno = ENOTDIR; return -1; }
+	prefix = strlen(wanted);
+	seen = calloc(version ? (size_t)version : 1, sizeof(*seen));
+	if (!seen) return -1;
+	for (i = 0; i < (size_t)version; i++) {
+		sleep_metadata_entry_t e, latest;
+		const char *candidate, *relative, *slash;
+		char name[1024], child[2048];
+		size_t child_len;
+		if (dat_archive_read_entry(archive, i, &e)) { free(seen); return -1; }
+		candidate = e.path; while (*candidate == '/') candidate++;
+		if (prefix) {
+			if (strncmp(candidate, wanted, prefix) || candidate[prefix] != '/') continue;
+			relative = candidate + prefix + 1;
+		} else relative = candidate;
+		if (!*relative) continue;
+		slash = strchr(relative, '/');
+		child_len = slash ? (size_t)(slash - relative) : strlen(relative);
+		if (!child_len || child_len >= sizeof(name)) continue;
+		memcpy(name, relative, child_len); name[child_len] = 0;
+		{
+			size_t s;
+			for (s = 0; s < seen_count; s++) if (!strcmp(seen[s], name)) break;
+			if (s < seen_count) continue;
+		}
+		if (prefix) snprintf(child, sizeof(child), "%s/%s", wanted, name);
+		else snprintf(child, sizeof(child), "%s", name);
+		if (find_normalized(archive, version, child, &latest)) {
+			if (errno == ENOENT) {
+				/* A descendant may imply a directory even without an explicit record. */
+				if (!slash || dat_archive_stat_at(archive, version, child, &info)) continue;
+				latest = info.entry;
+			} else { free(seen); return -1; }
+		}
+		if (latest.type == SLEEP_ENTRY_DELETE) continue;
+		seen[seen_count] = strdup(name);
+		if (!seen[seen_count]) { while (seen_count) free(seen[--seen_count]); free(seen); return -1; }
+		seen_count++;
+		if (callback(&latest, name, context)) { while (seen_count) free(seen[--seen_count]); free(seen); return -1; }
+	}
+	while (seen_count) free(seen[--seen_count]);
+	free(seen);
+	return 0;
+}
+
+static int lookup_index(const dat_archive_t *archive, uint64_t version,
+			const char *path, dat_path_info_t *info)
+{
+	char index_path[2048];
+	int n;
+	if (*path) n = snprintf(index_path, sizeof(index_path), "%s/index.html", path);
+	else n = snprintf(index_path, sizeof(index_path), "index.html");
+	if (n < 0 || (size_t)n >= sizeof(index_path)) { errno = ENAMETOOLONG; return -1; }
+	if (dat_archive_stat_at(archive, version, index_path, info)) return -1;
+	return info->entry.type == SLEEP_ENTRY_FILE ? 0 : -1;
+}
+
+int dat_archive_resolve_at(const dat_archive_t *archive, uint64_t version,
+			   const char *path, int force_listing,
+			   dat_path_info_t *info, int *serve_listing)
+{
+	char normalized[2048], html_path[2048];
+	const char *p;
+	size_t n;
+	int trailing_slash, is_directory;
+	if (!archive || !path || !info || !serve_listing || canonical_entry_path(path, normalized, sizeof(normalized))) { errno = EINVAL; return -1; }
+	p = path + strlen(path);
+	trailing_slash = p > path && p[-1] == '/';
+	*serve_listing = 0;
+	if (dat_archive_stat_at(archive, version, normalized, info)) {
+		/* Hyperdrive resolves /name to /name.html when the direct path is absent. */
+		if (errno != ENOENT || force_listing) return -1;
+		n = strlen(normalized);
+		if (n + sizeof(".html") > sizeof(html_path)) { errno = ENAMETOOLONG; return -1; }
+		snprintf(html_path, sizeof(html_path), "%s.html", normalized);
+		if (!dat_archive_stat_at(archive, version, html_path, info) &&
+		    info->entry.type == SLEEP_ENTRY_FILE) return 0;
+		errno = ENOENT;
+		return -1;
+	}
+	is_directory = info->is_directory;
+	if (force_listing) {
+		if (!is_directory) { errno = ENOTDIR; return -1; }
+		*serve_listing = 1; return 0;
+	}
+	if (trailing_slash || is_directory) {
+		if (!lookup_index(archive, version, normalized, info)) return 0;
+		if (!is_directory) { errno = ENOENT; return -1; }
+		*serve_listing = 1; return 0;
+	}
+	if (info->entry.type == SLEEP_ENTRY_FILE) return 0;
+	n = strlen(normalized);
+	if (n + sizeof(".html") > sizeof(html_path)) { errno = ENAMETOOLONG; return -1; }
+	snprintf(html_path, sizeof(html_path), "%s.html", normalized);
+	if (!dat_archive_stat_at(archive, version, html_path, info) && info->entry.type == SLEEP_ENTRY_FILE) return 0;
+	errno = ENOENT;
+	return -1;
+}
+
+const char *dat_content_type(const char *path)
+{
+	const char *dot = strrchr(path ? path : "", '.');
+	if (!dot) return "text/plain";
+	dot++;
+	if (!strcasecmp(dot, "html") || !strcasecmp(dot, "htm")) return "text/html";
+	if (!strcasecmp(dot, "css")) return "text/css";
+	if (!strcasecmp(dot, "js")) return "application/javascript";
+	if (!strcasecmp(dot, "json")) return "application/json";
+	if (!strcasecmp(dot, "svg")) return "image/svg+xml";
+	if (!strcasecmp(dot, "png")) return "image/png";
+	if (!strcasecmp(dot, "gif")) return "image/gif";
+	if (!strcasecmp(dot, "jpg") || !strcasecmp(dot, "jpeg")) return "image/jpeg";
+	if (!strcasecmp(dot, "webp")) return "image/webp";
+	return "text/plain";
+}

extra/dat-dpi/src/dat_dns.c

@@ -0,0 +1,421 @@
+#define _DEFAULT_SOURCE
+#include "dat_dns.h"
+#include "dat_paths.h"
+
+#include <ctype.h>
+#include <errno.h>
+#include <limits.h>
+#include <stdio.h>
+#include <curl/curl.h>
+#include <stdlib.h>
+#include <string.h>
+#include <strings.h>
+#include <sys/stat.h>
+#include <unistd.h>
+#include <time.h>
+
+#define DAT_DNS_PACKET_MAX 65536
+#define DAT_DNS_TXT_MAX 4096
+#define DAT_DNS_CACHE_SIZE 32
+
+typedef struct dat_dns_cache_entry {
+	char hostname[256];
+	uint8_t key[DAT_KEY_BYTES];
+	uint64_t expires_at;
+	int active;
+} dat_dns_cache_entry_t;
+
+static dat_dns_cache_entry_t dns_cache[DAT_DNS_CACHE_SIZE];
+static size_t dns_cache_next;
+static int curl_initialized;
+static int hex_value(unsigned char c);
+
+typedef struct dat_dns_body {
+	char text[4096];
+	size_t length;
+	int overflow;
+} dat_dns_body_t;
+
+static size_t receive_dat_dns_body(char *data, size_t size, size_t count, void *context)
+{
+	dat_dns_body_t *body = (dat_dns_body_t *)context;
+	size_t bytes = size * count;
+	if (bytes > sizeof(body->text) - body->length - 1) { body->overflow = 1; return 0; }
+	memcpy(body->text + body->length, data, bytes);
+	body->length += bytes;
+	body->text[body->length] = '\0';
+	return bytes;
+}
+
+static int parse_wellknown_record(const char *body, size_t length,
+				  uint8_t key[DAT_KEY_BYTES], uint32_t *ttl_out)
+{
+	const char *line_end, *ttl_line;
+	size_t key_length, i;
+	uint32_t ttl = 3600;
+	if (!body || !key) { errno = EINVAL; return -1; }
+	line_end = memchr(body, '\n', length);
+	if (!line_end) line_end = body + length;
+	if ((size_t)(line_end - body) < 70 || strncasecmp(body, "dat://", 6)) { errno = EPROTO; return -1; }
+	key_length = (size_t)(line_end - body) - 6;
+	if (key_length < 64) { errno = EPROTO; return -1; }
+	for (i = 0; i < DAT_KEY_BYTES; i++) {
+		int hi = hex_value((unsigned char)body[6 + i * 2]);
+		int lo = hex_value((unsigned char)body[7 + i * 2]);
+		if (hi < 0 || lo < 0) { errno = EPROTO; return -1; }
+		key[i] = (uint8_t)((hi << 4) | lo);
+	}
+	if (key_length > 64 && body[70] != '+' && body[70] != '/' && body[70] != '?') { errno = EPROTO; return -1; }
+	ttl_line = line_end < body + length ? line_end + 1 : NULL;
+	if (ttl_line && !strncasecmp(ttl_line, "ttl=", 4)) {
+		char *end;
+		unsigned long parsed = strtoul(ttl_line + 4, &end, 10);
+		if (end != ttl_line + 4 && (*end == '\n' || *end == '\r' || *end == '\0'))
+			ttl = parsed > 604800ul ? 604800u : (uint32_t)parsed;
+	}
+	if (ttl_out) *ttl_out = ttl;
+	return 0;
+}
+
+static int hex_value(unsigned char c)
+{
+	if (c >= '0' && c <= '9') return c - '0';
+	if (c >= 'a' && c <= 'f') return c - 'a' + 10;
+	if (c >= 'A' && c <= 'F') return c - 'A' + 10;
+	return -1;
+}
+
+static int normalize_hostname(const char *input, char output[256])
+{
+	size_t n, label_start = 0, i;
+	if (!input || !*input) { errno = EINVAL; return -1; }
+	n = strlen(input);
+	if (n && input[n - 1] == '.') n--;
+	if (!n || n > 253) { errno = EINVAL; return -1; }
+	for (i = 0; i < n; i++) {
+		unsigned char c = (unsigned char)input[i];
+		if (c == '.') {
+			size_t label_length = i - label_start;
+			if (!label_length || label_length > 63 || input[label_start] == '-' || input[i - 1] == '-') {
+				errno = EINVAL; return -1;
+			}
+			output[i] = '.';
+			label_start = i + 1;
+		} else if (isalnum(c) || c == '-') output[i] = (char)tolower(c);
+		else { errno = EINVAL; return -1; }
+	}
+	if (label_start == n || n - label_start > 63 || input[label_start] == '-' || input[n - 1] == '-') {
+		errno = EINVAL; return -1;
+	}
+	output[n] = '\0';
+	return 0;
+}
+
+static int names_file(char *path, size_t capacity, int create)
+{
+	const char *home = dat_home_directory();
+	char directory[1024];
+	int n;
+	if (!home || !*home) { errno = ENOENT; return -1; }
+	n = snprintf(directory, sizeof(directory), "%s/.dat", home);
+	if (n < 0 || (size_t)n >= sizeof(directory)) { errno = ENAMETOOLONG; return -1; }
+	if (create && mkdir(directory, 0700) && errno != EEXIST) return -1;
+	n = snprintf(path, capacity, "%s/dat-dns", directory);
+	if (n < 0 || (size_t)n >= capacity) { errno = ENAMETOOLONG; return -1; }
+	return 0;
+}
+
+static int parse_name_line(char *line, char hostname[256], uint8_t key[DAT_KEY_BYTES])
+{
+	char name[256], hex[65];
+	size_t i;
+	if (sscanf(line, "%255s %64s", name, hex) != 2 || strlen(hex) != 64 ||
+	    normalize_hostname(name, hostname)) return -1;
+	for (i = 0; i < DAT_KEY_BYTES; i++) {
+		int hi = hex_value((unsigned char)hex[i * 2]);
+		int lo = hex_value((unsigned char)hex[i * 2 + 1]);
+		if (hi < 0 || lo < 0) return -1;
+		key[i] = (uint8_t)((hi << 4) | lo);
+	}
+	return 0;
+}
+
+int dat_dns_name_foreach(dat_dns_name_iter_fn callback, void *context)
+{
+	char path[1200], line[400], hostname[256];
+	uint8_t key[DAT_KEY_BYTES];
+	FILE *file;
+	if (!callback || names_file(path, sizeof(path), 0)) { if (!callback) errno = EINVAL; return -1; }
+	file = fopen(path, "r");
+	if (!file) return errno == ENOENT ? 0 : -1;
+	while (fgets(line, sizeof(line), file)) {
+		if (!parse_name_line(line, hostname, key) && callback(hostname, key, context)) {
+			fclose(file); return 1;
+		}
+	}
+	if (ferror(file)) { fclose(file); return -1; }
+	fclose(file);
+	return 0;
+}
+
+typedef struct name_lookup_context {
+	const uint8_t *key;
+	char *hostname;
+	size_t capacity;
+	int found;
+} name_lookup_context_t;
+
+static int find_name_for_key(const char *name, const uint8_t key[DAT_KEY_BYTES], void *context)
+{
+	name_lookup_context_t *lookup = (name_lookup_context_t *)context;
+	if (!memcmp(lookup->key, key, DAT_KEY_BYTES)) {
+		if (strlen(name) >= lookup->capacity) { errno = ENOSPC; return 1; }
+		strcpy(lookup->hostname, name);
+		lookup->found = 1;
+	}
+	return 0;
+}
+
+int dat_dns_name_for_key(const uint8_t key[DAT_KEY_BYTES], char *hostname,
+			 size_t hostname_capacity)
+{
+	name_lookup_context_t lookup;
+	if (!key || !hostname || !hostname_capacity) { errno = EINVAL; return -1; }
+	lookup.key = key; lookup.hostname = hostname; lookup.capacity = hostname_capacity; lookup.found = 0;
+	if (dat_dns_name_foreach(find_name_for_key, &lookup) < 0) return -1;
+	if (!lookup.found) { errno = ENOENT; return -1; }
+	return 0;
+}
+
+typedef struct name_resolve_context {
+	const char *hostname;
+	uint8_t *key;
+	int found;
+} name_resolve_context_t;
+
+static int find_key_for_name(const char *name, const uint8_t key[DAT_KEY_BYTES], void *context)
+{
+	name_resolve_context_t *lookup = (name_resolve_context_t *)context;
+	if (!strcasecmp(lookup->hostname, name)) {
+		memcpy(lookup->key, key, DAT_KEY_BYTES);
+		lookup->found = 1;
+		return 1;
+	}
+	return 0;
+}
+
+int dat_dns_name_resolve(const char *hostname, uint8_t key[DAT_KEY_BYTES])
+{
+	char normalized[256];
+	name_resolve_context_t lookup;
+	if (!key || normalize_hostname(hostname, normalized)) { if (!key) errno = EINVAL; return -1; }
+	lookup.hostname = normalized; lookup.key = key; lookup.found = 0;
+	if (dat_dns_name_foreach(find_key_for_name, &lookup) < 0) return -1;
+	if (!lookup.found) { errno = ENOENT; return -1; }
+	return 0;
+}
+
+static int update_name(const char *hostname, const uint8_t *key, int remove_alias)
+{
+	char normalized[256], path[1200], temporary[1250], line[400], old_name[256];
+	uint8_t old_key[DAT_KEY_BYTES];
+	FILE *input = NULL, *output = NULL;
+	char hex[65];
+	int found = 0, n, result = -1;
+	if (normalize_hostname(hostname, normalized) || (!remove_alias && !key)) return -1;
+	if (names_file(path, sizeof(path), 1)) return -1;
+	n = snprintf(temporary, sizeof(temporary), "%s.tmp", path);
+	if (n < 0 || (size_t)n >= sizeof(temporary)) { errno = ENAMETOOLONG; return -1; }
+	input = fopen(path, "r");
+	if (!input && errno != ENOENT) return -1;
+	output = fopen(temporary, "w");
+	if (!output) { if (input) fclose(input); return -1; }
+	while (input && fgets(line, sizeof(line), input)) {
+		if (!parse_name_line(line, old_name, old_key) && !strcasecmp(old_name, normalized)) {
+			found = 1;
+			if (!remove_alias) {
+				if (dat_key_to_hex(key, hex) || fprintf(output, "%s %s\n", normalized, hex) < 0)
+					goto done;
+			}
+		} else if (fputs(line, output) == EOF) goto done;
+	}
+	if (input && ferror(input)) goto done;
+	if (!remove_alias && !found) {
+		if (dat_key_to_hex(key, hex) || fprintf(output, "%s %s\n", normalized, hex) < 0) goto done;
+	}
+	if (fclose(output)) { output = NULL; goto done; }
+	output = NULL;
+	if (input && fclose(input)) { input = NULL; goto done; }
+	input = NULL;
+	if (remove_alias && !found) { errno = ENOENT; goto done; }
+	if (dat_replace_file(temporary, path)) goto done;
+	result = 0;
+done:
+	if (input) fclose(input);
+	if (output) fclose(output);
+	if (result) unlink(temporary);
+	return result;
+}
+
+int dat_dns_name_set(const char *hostname, const uint8_t key[DAT_KEY_BYTES])
+{
+	return update_name(hostname, key, 0);
+}
+
+int dat_dns_name_remove(const char *hostname)
+{
+	return update_name(hostname, NULL, 1);
+}
+
+static int parse_datkey(const char *txt, size_t length, uint8_t key[32])
+{
+	size_t i;
+	const char *value = NULL;
+	for (i = 0; i + 7 <= length; i++) {
+		if ((txt[i] == 'd' || txt[i] == 'D') &&
+		    (txt[i + 1] == 'a' || txt[i + 1] == 'A') &&
+		    (txt[i + 2] == 't' || txt[i + 2] == 'T') &&
+		    (txt[i + 3] == 'k' || txt[i + 3] == 'K') &&
+		    (txt[i + 4] == 'e' || txt[i + 4] == 'E') &&
+		    (txt[i + 5] == 'y' || txt[i + 5] == 'Y') && txt[i + 6] == '=') {
+			value = txt + i + 7;
+			break;
+		}
+	}
+	if (!value || (size_t)(txt + length - value) < 64) return 0;
+	for (i = 0; i < 32; i++) {
+		int hi = hex_value((unsigned char)value[i * 2]);
+		int lo = hex_value((unsigned char)value[i * 2 + 1]);
+		if (hi < 0 || lo < 0) return 0;
+		key[i] = (uint8_t)((hi << 4) | lo);
+	}
+	if ((size_t)(txt + length - value) > 64 && value[64] != '&' && value[64] != ';' &&
+	    value[64] != ' ' && value[64] != '"' && value[64] != '\'')
+		return 0;
+	return 1;
+}
+
+int dat_dns_parse_answer(const uint8_t *packet, unsigned packet_len,
+			uint8_t key[DAT_KEY_BYTES], uint32_t *ttl_seconds)
+{
+	unsigned offset = 12, questions, answers, record_index;
+	if (!packet || packet_len < 12 || !key) { errno = EINVAL; return -1; }
+	questions = ((unsigned)packet[4] << 8) | packet[5];
+	answers = ((unsigned)packet[6] << 8) | packet[7];
+	for (record_index = 0; record_index < questions; record_index++) {
+		unsigned hops = 0, cursor = offset;
+		int jumped = 0;
+		for (;;) {
+			unsigned label;
+			if (cursor >= packet_len || ++hops > packet_len) { errno = EPROTO; return -1; }
+			label = packet[cursor++];
+			if ((label & 0xc0u) == 0xc0u) {
+				if (cursor >= packet_len) { errno = EPROTO; return -1; }
+				if (!jumped) offset = cursor + 1;
+				cursor = ((label & 0x3fu) << 8) | packet[cursor]; jumped = 1; continue;
+			}
+			if (label & 0xc0u) { errno = EPROTO; return -1; }
+			if (!label) { if (!jumped) offset = cursor; break; }
+			if (label > 63 || label > packet_len - cursor) { errno = EPROTO; return -1; }
+			cursor += label;
+		}
+		if (packet_len - offset < 4) { errno = EPROTO; return -1; }
+		offset += 4;
+	}
+	for (record_index = 0; record_index < answers; record_index++) {
+		unsigned hops = 0, cursor = offset, type, data_length, end;
+		int jumped = 0;
+		uint32_t ttl;
+		for (;;) {
+			unsigned label;
+			if (cursor >= packet_len || ++hops > packet_len) { errno = EPROTO; return -1; }
+			label = packet[cursor++];
+			if ((label & 0xc0u) == 0xc0u) {
+				if (cursor >= packet_len) { errno = EPROTO; return -1; }
+				if (!jumped) offset = cursor + 1;
+				cursor = ((label & 0x3fu) << 8) | packet[cursor]; jumped = 1; continue;
+			}
+			if (label & 0xc0u) { errno = EPROTO; return -1; }
+			if (!label) { if (!jumped) offset = cursor; break; }
+			if (label > 63 || label > packet_len - cursor) { errno = EPROTO; return -1; }
+			cursor += label;
+		}
+		if (packet_len - offset < 10) { errno = EPROTO; return -1; }
+		type = ((unsigned)packet[offset] << 8) | packet[offset + 1];
+		ttl = ((uint32_t)packet[offset + 4] << 24) | ((uint32_t)packet[offset + 5] << 16) |
+		      ((uint32_t)packet[offset + 6] << 8) | packet[offset + 7];
+		data_length = ((unsigned)packet[offset + 8] << 8) | packet[offset + 9];
+		offset += 10;
+		if (data_length > packet_len - offset) { errno = EPROTO; return -1; }
+		end = offset + data_length;
+		if (type == 16) {
+			char text[DAT_DNS_TXT_MAX];
+			size_t used = 0;
+			while (offset < end) {
+				unsigned part = packet[offset++];
+				if (part > end - offset || used + part >= sizeof(text)) { errno = EPROTO; return -1; }
+				memcpy(text + used, packet + offset, part); used += part; offset += part;
+			}
+			if (parse_datkey(text, used, key)) { if (ttl_seconds) *ttl_seconds = ttl; return 0; }
+		} else offset = end;
+	}
+	errno = ENOENT;
+	return -1;
+}
+
+int dat_dns_resolve(const char *hostname, uint8_t key[DAT_KEY_BYTES],
+		    uint32_t *ttl_seconds)
+{
+	size_t i;
+	uint64_t now;
+	uint32_t ttl = 0;
+	char normalized[256], request_url[320];
+	if (!hostname || !*hostname || !key) { errno = EINVAL; return -1; }
+	if (normalize_hostname(hostname, normalized)) return -1;
+	if (dat_dns_name_resolve(normalized, key) == 0) {
+		if (ttl_seconds) *ttl_seconds = 0;
+		return 0;
+	}
+	now = (uint64_t)time(NULL);
+	for (i = 0; i < DAT_DNS_CACHE_SIZE; i++) {
+		if (dns_cache[i].active && dns_cache[i].expires_at <= now)
+			dns_cache[i].active = 0;
+		if (dns_cache[i].active && !strcasecmp(normalized, dns_cache[i].hostname)) {
+			memcpy(key, dns_cache[i].key, DAT_KEY_BYTES);
+			if (ttl_seconds) *ttl_seconds = (uint32_t)(dns_cache[i].expires_at - now);
+			return 0;
+		}
+	}
+	{
+		CURL *curl;
+		CURLcode result;
+		long status = 0;
+		dat_dns_body_t body;
+		int url_length = snprintf(request_url, sizeof(request_url), "https://%s/.well-known/dat", normalized);
+		if (url_length < 0 || (size_t)url_length >= sizeof(request_url)) { errno = ENAMETOOLONG; return -1; }
+		if (!curl_initialized) { if (curl_global_init(CURL_GLOBAL_DEFAULT) != CURLE_OK) { errno = EIO; return -1; } curl_initialized = 1; }
+		memset(&body, 0, sizeof(body));
+		curl = curl_easy_init();
+		if (!curl) { errno = ENOMEM; return -1; }
+		curl_easy_setopt(curl, CURLOPT_URL, request_url);
+		curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 0L);
+		curl_easy_setopt(curl, CURLOPT_CONNECTTIMEOUT_MS, 2000L);
+		curl_easy_setopt(curl, CURLOPT_TIMEOUT_MS, 2000L);
+		curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, receive_dat_dns_body);
+		curl_easy_setopt(curl, CURLOPT_WRITEDATA, &body);
+		result = curl_easy_perform(curl);
+		if (result == CURLE_OK) curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, &status);
+		curl_easy_cleanup(curl);
+		if (result != CURLE_OK || status != 200 || body.overflow) { errno = EHOSTUNREACH; return -1; }
+		if (parse_wellknown_record(body.text, body.length, key, &ttl)) return -1;
+	}
+	if (ttl_seconds) *ttl_seconds = ttl;
+	if (ttl) {
+		dat_dns_cache_entry_t *entry = &dns_cache[dns_cache_next++ % DAT_DNS_CACHE_SIZE];
+		strcpy(entry->hostname, hostname);
+		memcpy(entry->key, key, DAT_KEY_BYTES);
+		entry->expires_at = now + ttl;
+		entry->active = 1;
+	}
+	return 0;
+}

extra/dat-dpi/src/dat_dpi.c

@@ -0,0 +1,758 @@
+#define _POSIX_C_SOURCE 200809L
+#include "dat.h"
+#include "dat_dns.h"
+#include "dat_fetch.h"
+#include "dat_paths.h"
+#include "dat_dpi_icons.h"
+#include "s7.h"
+#include "sleep_feed.h"
+
+#include <ctype.h>
+#include <errno.h>
+#include <locale.h>
+#include <stdarg.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <strings.h>
+#include <sys/socket.h>
+#include <sys/stat.h>
+#include <time.h>
+#include <unistd.h>
+
+#define TAG_CAP 8192
+#define DPI_ARCHIVES_MAX 128
+
+typedef struct {
+	char key[65];
+	char root[1200];
+	char title[256];
+	char description[512];
+	uint64_t version;
+	int used;
+} dpi_archive_t;
+
+static dpi_archive_t open_archives[DPI_ARCHIVES_MAX];
+static s7_scheme *metadata_scheme;
+static int quit_requested;
+
+typedef struct {
+	char *data;
+	size_t length;
+	size_t capacity;
+} body_buffer_t;
+
+static int body_appendf(body_buffer_t *body, const char *format, ...)
+{
+	va_list args, copy;
+	int needed;
+	size_t capacity;
+	char *data;
+	va_start(args, format);
+	va_copy(copy, args);
+	needed = vsnprintf(NULL, 0, format, copy);
+	va_end(copy);
+	if (needed < 0) { va_end(args); return -1; }
+	if ((size_t)needed > SIZE_MAX - body->length - 1) { va_end(args); errno = EOVERFLOW; return -1; }
+	if (body->length + (size_t)needed + 1 > body->capacity) {
+		capacity = body->capacity ? body->capacity : 512;
+		while (capacity < body->length + (size_t)needed + 1) {
+			if (capacity > SIZE_MAX / 2) { capacity = body->length + (size_t)needed + 1; break; }
+			capacity *= 2;
+		}
+		data = realloc(body->data, capacity);
+		if (!data) { va_end(args); return -1; }
+		body->data = data;
+		body->capacity = capacity;
+	}
+	vsnprintf(body->data + body->length, body->capacity - body->length, format, args);
+	va_end(args);
+	body->length += (size_t)needed;
+	return 0;
+}
+
+static void send_page(const char *url, const char *mime, const void *body, size_t length)
+{
+	const char *p;
+	printf("<cmd='start_send_page' url='");
+	for (p = url; *p; p++) { putchar(*p); if (*p == '\'') putchar(*p); }
+	printf("' '>\nContent-Type: %s\r\n\r\n", mime);
+	if (length) fwrite(body, 1, length, stdout);
+	fflush(stdout);
+}
+
+static int send_file(const char *url, const char *mime, const dat_archive_t *archive,
+		     const sleep_metadata_entry_t *entry)
+{
+	sleep_feed_t feed;
+	uint64_t block;
+	size_t sent = 0;
+	const char *p;
+	if (entry->size > SIZE_MAX) { errno = EOVERFLOW; return -1; }
+	if (sleep_feed_open(&feed, archive->dat_path, "content", NULL)) return -1;
+	for (block = 0; block < entry->block_count; block++) {
+		if (!sleep_feed_has(&feed, (size_t)entry->block_offset + (size_t)block)) {
+			sleep_feed_close(&feed); errno = ENOENT; return -1;
+		}
+	}
+	fputs("<cmd='start_send_page' url='", stdout);
+	for (p = url; *p; p++) { putchar(*p); if (*p == '\'') putchar(*p); }
+	printf("' '>\nContent-Type: %s\r\n\r\n", mime);
+	fflush(stdout);
+	for (block = 0; block < entry->block_count; block++) {
+		uint8_t *data = NULL;
+		size_t length = 0;
+		if (sleep_feed_read(&feed, (size_t)entry->block_offset + (size_t)block,
+				    &data, &length) || length > (size_t)entry->size - sent) {
+			free(data); sleep_feed_close(&feed); errno = EIO; return -1;
+		}
+		if (length && fwrite(data, 1, length, stdout) != length) {
+			free(data); sleep_feed_close(&feed); errno = EIO; return -1;
+		}
+		sent += length;
+		free(data);
+	}
+	sleep_feed_close(&feed);
+	fflush(stdout);
+	if (sent != (size_t)entry->size) { errno = EPROTO; return -1; }
+	return ferror(stdout) ? -1 : 0;
+}
+
+static void send_status(const char *message)
+{
+	const char *p;
+	fputs("<cmd='send_status_message' msg='", stdout);
+	for (p = message; *p; p++) { putchar(*p); if (*p == '\'') putchar(*p); }
+	fputs("' '>", stdout);
+	fflush(stdout);
+}
+
+static void send_reload(const char *url)
+{
+	const char *p;
+	fputs("<cmd='reload_request' url='", stdout);
+	for (p = url; *p; p++) { putchar(*p); if (*p == '\'') putchar(*p); }
+	fputs("' '>", stdout);
+	fflush(stdout);
+}
+
+static int html_append(body_buffer_t *body, const char *text)
+{
+	const unsigned char *p = (const unsigned char *)(text ? text : "");
+	for (; *p; p++) {
+		const char *escape = NULL;
+		switch (*p) {
+		case '&': escape = "&amp;"; break;
+		case '<': escape = "&lt;"; break;
+		case '>': escape = "&gt;"; break;
+		case '\"': escape = "&quot;"; break;
+		case '\'': escape = "&#39;"; break;
+		default: break;
+		}
+		if (escape) { if (body_appendf(body, "%s", escape)) return -1; }
+		else if (body_appendf(body, "%c", *p)) return -1;
+	}
+	return 0;
+}
+
+static int url_append_path(body_buffer_t *body, const char *path)
+{
+	static const char hex[] = "0123456789ABCDEF";
+	const unsigned char *p = (const unsigned char *)path;
+	for (; *p; p++) {
+		if ((*p >= 'a' && *p <= 'z') || (*p >= 'A' && *p <= 'Z') ||
+		    (*p >= '0' && *p <= '9') || strchr("/-_.~", *p)) {
+			if (body_appendf(body, "%c", *p)) return -1;
+		} else if (body_appendf(body, "%%%c%c", hex[*p >> 4], hex[*p & 15])) return -1;
+	}
+	return 0;
+}
+
+/* Dillo DPIP tags end in space, quote, greater-than. */
+static int read_tag(FILE *in, char *tag, size_t cap)
+{
+	int c, quote = 0;
+	size_t n = 0;
+	while ((c = fgetc(in)) != EOF) {
+		if (!n && isspace((unsigned char)c)) continue;
+		if (n + 1 >= cap) { errno = EOVERFLOW; return -1; }
+		tag[n++] = (char)c;
+		if (c == '\'') {
+			if (quote) {
+				c = fgetc(in);
+				if (c == '\'') {
+					if (n + 1 >= cap) { errno = EOVERFLOW; return -1; }
+					tag[n++] = (char)c;
+					continue;
+				}
+				if (c != EOF) ungetc(c, in);
+			}
+			quote = !quote;
+		}
+		if (n >= 2 && tag[n - 2] == '\'' && tag[n - 1] == '>') {
+			tag[n] = 0;
+			return 1;
+		}
+	}
+	return n ? -1 : 0;
+}
+
+static char *attribute(const char *tag, const char *name)
+{
+	char needle[64];
+	const char *p, *start, *end;
+	char *out;
+	size_t n, used = 0;
+	if (snprintf(needle, sizeof(needle), " %s='", name) >= (int)sizeof(needle)) return NULL;
+	p = strstr(tag, needle);
+	if (p) start = p + strlen(needle);
+	else if (tag[0] == '<' && !strncmp(tag + 1, needle + 1, strlen(needle) - 1))
+		start = tag + strlen(needle);
+	else return NULL;
+	end = start;
+	while (*end) {
+		if (*end == '\'' && end[1] != '\'') break;
+		if (*end == '\'' && end[1] == '\'') end += 2;
+		else end++;
+	}
+	if (!*end) return NULL;
+	n = (size_t)(end - start);
+	out = malloc(n + 1);
+	if (!out) return NULL;
+	while (start < end) {
+		out[used++] = *start;
+		if (*start == '\'' && start + 1 < end && start[1] == '\'') start++;
+		start++;
+	}
+	out[used] = 0;
+	return out;
+}
+
+static int authenticate(const char *tag)
+{
+	char path[1024], line[512], secret[128];
+	FILE *f;
+	char *msg;
+	int ok = 0;
+	const char *home = dat_home_directory();
+	if (!home || snprintf(path, sizeof(path), "%s/.dillo/dpid_comm_keys", home) >= (int)sizeof(path)) return 0;
+	f = fopen(path, "r");
+	if (!f) return 0;
+	if (fgets(line, sizeof(line), f) && sscanf(line, "%*u %127s", secret) == 1) {
+		msg = attribute(tag, "msg");
+		if (msg && strlen(msg) == strlen(secret)) ok = memcmp(msg, secret, strlen(secret)) == 0;
+		free(msg);
+	}
+	fclose(f);
+	return ok;
+}
+
+static s7_pointer manifest_field(s7_pointer record, const char *name)
+{
+	s7_pointer fields = record;
+	while (s7_is_pair(fields)) {
+		s7_pointer field = s7_car(fields), key, values;
+		if (s7_is_pair(field)) {
+			key = s7_car(field);
+			values = s7_cdr(field);
+			if (s7_is_symbol(key) && !strcmp(s7_symbol_name(key), name) &&
+			    s7_is_pair(values)) return s7_car(values);
+		}
+		fields = s7_cdr(fields);
+	}
+	return NULL;
+}
+
+static void read_manifest(const dat_archive_t *archive, uint64_t version,
+			  dpi_archive_t *record)
+{
+	dat_path_info_t info;
+	int listing;
+	uint8_t *data = NULL;
+	size_t length = 0;
+	if (!metadata_scheme) metadata_scheme = s7_init();
+	if (metadata_scheme &&
+	    dat_archive_resolve_at(archive, version, "/dat.scm", 0, &info, &listing) == 0 &&
+	    !listing && info.entry.type == SLEEP_ENTRY_FILE &&
+	    dat_archive_read_file(archive, &info.entry, &data, &length) == 0 &&
+	    length < 1024u * 1024u && !memchr(data, 0, length)) {
+		char *source = malloc(length + 1);
+		if (source) {
+			s7_pointer port, parsed, title, description;
+			memcpy(source, data, length); source[length] = '\0';
+			port = s7_open_input_string(metadata_scheme, source);
+			parsed = port ? s7_read(metadata_scheme, port) : NULL;
+			if (port) s7_close_input_port(metadata_scheme, port);
+			title = parsed ? manifest_field(parsed, "title") : NULL;
+			description = parsed ? manifest_field(parsed, "description") : NULL;
+			if (title && s7_is_string(title)) snprintf(record->title, sizeof(record->title), "%s", s7_string(title));
+			if (description && s7_is_string(description)) snprintf(record->description, sizeof(record->description), "%s", s7_string(description));
+			free(source);
+		}
+	}
+	free(data);
+	if (!record->title[0] || !record->description[0]) {
+		uint8_t *cname = NULL;
+		size_t cname_length = 0;
+		if (dat_archive_resolve_at(archive, version, "/CNAME", 0, &info, &listing) == 0 &&
+		    !listing && info.entry.type == SLEEP_ENTRY_FILE &&
+		    dat_archive_read_file(archive, &info.entry, &cname, &cname_length) == 0 &&
+		    cname_length && cname_length < sizeof(record->title)) {
+			while (cname_length && (cname[cname_length - 1] == '\n' || cname[cname_length - 1] == '\r')) cname_length--;
+			if (!record->title[0]) { memcpy(record->title, cname, cname_length); record->title[cname_length] = '\0'; }
+		}
+		free(cname);
+	}
+}
+
+static dpi_archive_t *remember_archive(const dat_archive_t *archive, const char *keyhex,
+				      const char *root, uint64_t version)
+{
+	size_t i;
+	dpi_archive_t *record = NULL;
+	for (i = 0; i < DPI_ARCHIVES_MAX; i++) {
+		if (open_archives[i].used && !strcmp(open_archives[i].key, keyhex)) {
+			record = &open_archives[i]; break;
+		}
+		if (!open_archives[i].used && !record) record = &open_archives[i];
+	}
+	if (!record) return NULL;
+	if (!record->used) {
+		memset(record, 0, sizeof(*record));
+		snprintf(record->key, sizeof(record->key), "%s", keyhex);
+		snprintf(record->root, sizeof(record->root), "%s", root);
+		snprintf(record->title, sizeof(record->title), "%s", keyhex);
+		record->used = 1;
+	}
+	record->version = version;
+	read_manifest(archive, version, record);
+	return record;
+}
+
+static int parse_query_value(const char *url, const char *name, char *out, size_t capacity)
+{
+	const char *query = strchr(url, '?');
+	size_t name_length = strlen(name);
+	if (!query) return 0;
+	query++;
+	while (*query) {
+		const char *end = strchr(query, '&');
+		size_t part_length = end ? (size_t)(end - query) : strlen(query);
+		const char *equals = memchr(query, '=', part_length);
+		if (equals && (size_t)(equals - query) == name_length && !memcmp(query, name, name_length)) {
+			size_t value_length = part_length - name_length - 1;
+			if (value_length >= capacity) { errno = ENAMETOOLONG; return -1; }
+			memcpy(out, equals + 1, value_length); out[value_length] = '\0';
+			return 1;
+		}
+		if (!end) break;
+		query = end + 1;
+	}
+	return 0;
+}
+
+static int error_page(const char *url, int status, const char *message)
+{
+	const char *p;
+	(void)status;
+	printf("<cmd='start_send_page' url='");
+	for (p = url; *p; p++) { putchar(*p); if (*p == '\'') putchar(*p); }
+	printf("' '>\nContent-Type: text/plain\r\n\r\n%s\n", message);
+	fflush(stdout);
+	return ferror(stdout) ? -1 : 0;
+}
+
+typedef struct {
+	sleep_metadata_entry_t entry;
+	char name[1024];
+	int type;
+} listing_context_t;
+
+typedef struct {
+	listing_context_t *entries;
+	size_t count;
+	size_t capacity;
+} listing_accumulator_t;
+
+static int list_entry(const sleep_metadata_entry_t *entry, const char *name, void *context)
+{
+	listing_accumulator_t *list = context;
+	listing_context_t *next;
+	if (list->count == list->capacity) {
+		size_t capacity = list->capacity ? list->capacity * 2 : 16;
+		next = realloc(list->entries, capacity * sizeof(*next));
+		if (!next) return -1;
+		list->entries = next; list->capacity = capacity;
+	}
+	next = &list->entries[list->count++];
+	memset(next, 0, sizeof(*next));
+	next->entry = *entry;
+	next->type = entry->type == SLEEP_ENTRY_DIRECTORY ? 3 : 2;
+	if (strlen(name) >= sizeof(next->name)) { errno = ENAMETOOLONG; return -1; }
+	strcpy(next->name, name);
+	return 0;
+}
+
+static int compare_listing(const void *left, const void *right)
+{
+	const listing_context_t *a = left, *b = right;
+	if (a->type != b->type) return b->type - a->type;
+	return -strcoll(a->name, b->name);
+}
+
+static int append_listing_row(body_buffer_t *body, const dat_url_t *url,
+			      const listing_context_t *entry, const char *link_path)
+{
+	const char *icon = entry->type == 1 ? dat_dpi_icon_updir :
+		entry->type == 3 ? dat_dpi_icon_directory : dat_dpi_icon_file;
+	char date[64] = "";
+	if (entry->entry.mtime) {
+		time_t seconds = (time_t)(entry->entry.mtime / 1000u);
+		struct tm *tm = gmtime(&seconds);
+		if (tm) strftime(date, sizeof(date), "%Y-%m-%d %H:%M:%S", tm);
+	}
+	if (body_appendf(body, "<tr><td><img alt=\"%s\" src=\"data:image/png;base64,%s\"></td><td><a href=\"dat://",
+		entry->type == 1 ? "up" : entry->type == 2 ? "directory" : "file", icon)) return -1;
+	if (html_append(body, url->host)) return -1;
+	if (url->has_version && body_appendf(body, "+%llu", (unsigned long long)url->version)) return -1;
+	if (url_append_path(body, link_path) || body_appendf(body, "\">") ||
+	    html_append(body, entry->name)) return -1;
+	if (entry->type == 3 && body_appendf(body, "/")) return -1;
+	if (body_appendf(body, "</a></td><td>")) return -1;
+	if (*date && html_append(body, date)) return -1;
+	if (body_appendf(body, "</td><td align=\"right\">")) return -1;
+	if (entry->type == 2 && body_appendf(body, "%llu", (unsigned long long)entry->entry.size)) return -1;
+	return body_appendf(body, "</td></tr>\n");
+}
+
+static const char *dpi_content_type(const char *pathname)
+{
+	const char *dot = strrchr(pathname, '.');
+	if (!dot) return "text/plain";
+	dot++;
+	if (!strcmp(dot, "htm") || !strcmp(dot, "html")) return "text/html";
+	if (!strcmp(dot, "png")) return "image/png";
+	if (!strcmp(dot, "gif")) return "image/gif";
+	if (!strcmp(dot, "jpg")) return "image/jpeg";
+	return "text/plain";
+}
+
+static int serve_not_found(const char *url)
+{
+	static const char page[] = "<h3>Not found</h3>";
+	send_page(url, "text/html", page, sizeof(page) - 1);
+	return -1;
+}
+
+static int serve_timeout(const char *url)
+{
+	static const char page[] = "<h3>Timed out</h3>";
+	send_page(url, "text/html", page, sizeof(page) - 1);
+	return -1;
+}
+
+static int serve_directory(const char *url_text, const dat_url_t *url,
+			   const dat_archive_t *archive, uint64_t version)
+{
+	listing_accumulator_t entries = {0};
+	listing_context_t parent;
+	body_buffer_t body = {0};
+	char parent_path[2048];
+	size_t i, n;
+	int failed = 0;
+	if (dat_archive_readdir_at(archive, version, url->path, list_entry, &entries)) {
+		free(entries.entries);
+		return serve_not_found(url_text);
+	}
+	qsort(entries.entries, entries.count, sizeof(*entries.entries), compare_listing);
+	if (body_appendf(&body, "<!doctype html><html><head><title>")) failed = 1;
+	if (!failed && html_append(&body, url->path)) failed = 1;
+	if (!failed && body_appendf(&body, "</title></head><body><h4>")) failed = 1;
+	if (!failed && html_append(&body, url->path)) failed = 1;
+	if (!failed && body_appendf(&body, "</h4><table>")) failed = 1;
+	if (strcmp(url->path, "/")) {
+		memset(&parent, 0, sizeof(parent));
+		parent.type = 1;
+		strcpy(parent.name, "..");
+		strcpy(parent_path, url->path);
+		n = strlen(parent_path);
+		while (n && parent_path[n - 1] == '/') parent_path[--n] = '\0';
+		while (n && parent_path[n - 1] != '/') parent_path[--n] = '\0';
+		if (!n) strcpy(parent_path, "/");
+		else parent_path[n] = '\0';
+		if (!failed && append_listing_row(&body, url, &parent, parent_path)) failed = 1;
+	}
+	for (i = 0; !failed && i < entries.count; i++) {
+		body_buffer_t href = {0};
+		const listing_context_t *entry = &entries.entries[i];
+		if (body_appendf(&href, "%s", url->path) ||
+		    (href.length && href.data[href.length - 1] != '/' && body_appendf(&href, "/")) ||
+		    body_appendf(&href, "%s%s", entry->name, entry->type == 3 ? "/" : "") ||
+		    append_listing_row(&body, url, entry, href.data)) failed = 1;
+		free(href.data);
+	}
+	if (!failed && body_appendf(&body, "</table></body></html>")) failed = 1;
+	if (failed) {
+		free(body.data); free(entries.entries);
+		return error_page(url_text, 500, "Unable to build directory listing");
+	}
+	send_page(url_text, "text/html", body.data, body.length);
+	free(body.data); free(entries.entries);
+	return 0;
+}
+
+static int json_append_string(body_buffer_t *body, const char *text)
+{
+	const unsigned char *p = (const unsigned char *)(text ? text : "");
+	if (body_appendf(body, "\"")) return -1;
+	for (; *p; p++) {
+		if (*p == '\"' || *p == '\\') { if (body_appendf(body, "\\%c", *p)) return -1; }
+		else if (*p < 0x20) { if (body_appendf(body, "\\u%04x", *p)) return -1; }
+		else if (body_appendf(body, "%c", *p)) return -1;
+	}
+	return body_appendf(body, "\"");
+}
+
+static int serve(const char *url_text)
+{
+	dat_url_t url;
+	uint8_t key[DAT_KEY_BYTES];
+	const char *home = dat_home_directory();
+	char root[1200], keyhex[65], fetch_error[512];
+	dat_archive_t archive;
+	dat_path_info_t info;
+	int listing = 0;
+	uint64_t version;
+	if (dat_url_parse(url_text, &url)) {
+		error_page(url_text, 400, "Invalid dat:// URL"); return -1;
+	}
+	send_status("resolving name");
+	if (dat_url_to_key(url.host, key) && dat_dns_resolve(url.host, key, NULL)) {
+		error_page(url_text, 404, "Dat hostname has no usable datkey TXT record"); return -1;
+	}
+	if (!home) { error_page(url_text, 500, "User home directory is unavailable"); return -1; }
+	if (dat_key_to_hex(key, keyhex)) { error_page(url_text, 500, "Unable to format archive key"); return -1; }
+	if (snprintf(root, sizeof(root), "%s/.dillo/dat/archives/%.2s/%s",
+		home, keyhex, keyhex + 2) >= (int)sizeof(root)) {
+		error_page(url_text, 400, "Dillo archive cache path is too long"); return -1;
+	}
+	if (dat_archive_open(&archive, root)) {
+		send_status("getting archive");
+		if (dat_fetch_archive(root, key, 10000u)) {
+			int saved = errno;
+			if (saved == ETIMEDOUT) return serve_timeout(url_text);
+			const char *reason = saved == ENETUNREACH ? "No Cesspool bootstrap is configured" :
+				saved == EHOSTUNREACH ? "Cesspool bootstrap did not answer" :
+				saved == ENOENT ? "No peer is announcing this archive key" :
+				strerror(saved);
+			snprintf(fetch_error, sizeof(fetch_error), "Cesspool archive fetch failed: %s", reason);
+			return error_page(url_text, 404, fetch_error);
+		}
+		if (dat_archive_open(&archive, root)) {
+			snprintf(fetch_error, sizeof(fetch_error), "Fetched archive cannot be opened: %s", strerror(errno));
+			return error_page(url_text, 404, fetch_error);
+		}
+	}
+	version = url.has_version ? url.version : dat_archive_version(&archive);
+	if (version > dat_archive_version(&archive)) {
+		dat_archive_close(&archive); return serve_not_found(url_text);
+	}
+	if (dat_archive_resolve_at(&archive, version, url.path,
+		url.force_directory_listing, &info, &listing)) {
+		int saved = errno;
+		dat_archive_close(&archive);
+		if (saved == ETIMEDOUT || saved == EAGAIN) return serve_timeout(url_text);
+		return serve_not_found(url_text);
+	}
+	(void)remember_archive(&archive, keyhex, root, version);
+	send_status("serving file");
+	if (url.wants_stat) {
+		body_buffer_t stat = {0};
+		if (body_appendf(&stat, "{\n  \"path\": ") || json_append_string(&stat, url.path) ||
+		    body_appendf(&stat, ",\n  \"directory\": %s,\n  \"size\": %llu,\n  \"mtime\": %llu\n}\n",
+			info.is_directory ? "true" : "false",
+			(unsigned long long)info.entry.size, (unsigned long long)info.entry.mtime)) {
+			free(stat.data); dat_archive_close(&archive); return error_page(url_text, 500, "Stat response is too large");
+		}
+		send_page(url_text, "text/plain", stat.data, stat.length);
+		free(stat.data);
+	} else if (listing) {
+		serve_directory(url_text, &url, &archive, version);
+	} else {
+		if (info.entry.type != SLEEP_ENTRY_FILE) {
+			dat_archive_close(&archive); return serve_not_found(url_text);
+		}
+		if (send_file(url_text, dpi_content_type(info.entry.path), &archive, &info.entry)) {
+			dat_archive_close(&archive); return error_page(url_text, 500, "File content unavailable");
+		}
+	}
+	dat_archive_close(&archive);
+	fflush(stdout);
+	return ferror(stdout) ? -1 : 0;
+}
+
+static int serve_dashboard(const char *url)
+{
+	char action[32], key[80], reload[2048];
+	int has_action, has_key, has_reload;
+	body_buffer_t body = {0};
+	size_t i, count = 0;
+	if (parse_query_value(url, "unswarm", action, sizeof(action)) > 0) {
+		has_key = parse_query_value(url, "key", key, sizeof(key));
+		has_reload = parse_query_value(url, "reload", reload, sizeof(reload));
+		if (has_key > 0) {
+			for (i = 0; i < DPI_ARCHIVES_MAX; i++)
+				if (open_archives[i].used && !strcmp(open_archives[i].key, key))
+					memset(&open_archives[i], 0, sizeof(open_archives[i]));
+			send_status("unswarming archive");
+			send_reload(has_reload > 0 ? reload : "dpi:/dat/");
+			return 0;
+		}
+	}
+	has_action = parse_query_value(url, "quit", action, sizeof(action));
+	if (has_action > 0) {
+		static const char quitting[] = "Quitting";
+		send_page(url, "text/plain", quitting, sizeof(quitting) - 1);
+		quit_requested = 1;
+		return 0;
+	}
+	if (body_appendf(&body, "<!doctype html><html><head><title>Dat</title></head><body>")) goto fail;
+	if (body_appendf(&body, "<form action=\"dpi:/dat/\"><input type=\"submit\" name=\"quit\" value=\"Quit\"></form><h3>Archives</h3>")) goto fail;
+	for (i = 0; i < DPI_ARCHIVES_MAX; i++) if (open_archives[i].used) count++;
+	if (!count && body_appendf(&body, "<p>None</p>")) goto fail;
+	for (i = 0; i < DPI_ARCHIVES_MAX; i++) {
+		dpi_archive_t *archive = &open_archives[i];
+		if (!archive->used) continue;
+		if (body_appendf(&body, "<p><a href=\"dat://%s\">", archive->key) ||
+		    html_append(&body, archive->title[0] ? archive->title : archive->key) ||
+		    body_appendf(&body, "</a>")) goto fail;
+		if (archive->description[0] &&
+		    (body_appendf(&body, " - ") || html_append(&body, archive->description))) goto fail;
+		if (body_appendf(&body, "<br>version <a href=\"dat://%s+%llu\">%llu</a></p>",
+			archive->key, (unsigned long long)archive->version,
+			(unsigned long long)archive->version) ||
+		    body_appendf(&body, "<blockquote><h3>Peers</h3>None")) goto fail;
+		if (body_appendf(&body, "<form action=\"dpi:/dat/\"><input type=\"hidden\" name=\"key\" value=\"%s\"><input type=\"hidden\" name=\"reload\" value=\"dpi:/dat/\"><input type=\"submit\" name=\"unswarm\" value=\"unswarm\"></form></blockquote>", archive->key)) goto fail;
+	}
+	if (body_appendf(&body, "</body></html>")) goto fail;
+	send_page(url, "text/html", body.data, body.length);
+	free(body.data);
+	return 0;
+fail:
+	free(body.data);
+	return error_page(url, 500, "Unable to build Dat dashboard");
+}
+
+static int serve_internal(const char *url)
+{
+	const char *path = strstr(url, "://");
+	const char *query;
+	char decoded[2048];
+	size_t n;
+	if (!strncmp(url, "dpi:/", 5)) path = url + 4;
+	else if (path) path = strchr(path + 3, '/');
+	if (!path) path = "/";
+	query = strchr(path, '?');
+	n = query ? (size_t)(query - path) : strlen(path);
+	if (n >= sizeof(decoded)) return error_page(url, 400, "Internal DPI path is too long");
+	memcpy(decoded, path, n); decoded[n] = '\0';
+	if (!strcmp(decoded, "/dat/")) return serve_dashboard(url);
+	return serve_not_found(url);
+}
+
+static int serve_stdio_requests(void)
+{
+	char tag[TAG_CAP];
+	int result = read_tag(stdin, tag, sizeof(tag));
+	if (result <= 0 || !authenticate(tag)) return 1;
+	while (!quit_requested) {
+		char *cmd, *url;
+		result = read_tag(stdin, tag, sizeof(tag));
+		if (result <= 0) break;
+		cmd = attribute(tag, "cmd");
+		url = attribute(tag, "url");
+		if (cmd && !strcmp(cmd, "DpiBye")) { free(cmd); free(url); break; }
+		if (cmd && !strcmp(cmd, "open_url") && url) {
+			if (!strncmp(url, "dat:", 4)) serve(url);
+			else if (!strncmp(url, "dpi:/", 5)) serve_internal(url);
+			else error_page(url, 404, "Not found");
+		}
+		free(cmd);
+		free(url);
+	}
+	return 0;
+}
+
+static int serve_socket_requests(void)
+{
+	int listener = dup(STDIN_FILENO), stopped = 0;
+	if (listener < 0) return -1;
+	while (!stopped) {
+		int client = accept(listener, NULL, NULL);
+		int saved_in, saved_out, stop_after_request = 0;
+		char tag[TAG_CAP];
+		if (client < 0) {
+			if (errno == EINTR) continue;
+			close(listener);
+			return -1;
+		}
+		saved_in = dup(STDIN_FILENO);
+		saved_out = dup(STDOUT_FILENO);
+		if (saved_in < 0 || saved_out < 0 || dup2(client, STDIN_FILENO) < 0 ||
+		    dup2(client, STDOUT_FILENO) < 0) {
+			if (saved_in >= 0) close(saved_in);
+			if (saved_out >= 0) close(saved_out);
+			close(client);
+			close(listener);
+			return -1;
+		}
+		clearerr(stdin);
+		clearerr(stdout);
+		if (read_tag(stdin, tag, sizeof(tag)) > 0 && authenticate(tag)) {
+			while (!stop_after_request) {
+				char *cmd, *url;
+				int result = read_tag(stdin, tag, sizeof(tag));
+				if (result <= 0) break;
+				cmd = attribute(tag, "cmd");
+				url = attribute(tag, "url");
+				if (cmd && !strcmp(cmd, "DpiBye")) {
+					stopped = 1;
+					stop_after_request = 1;
+				} else if (cmd && !strcmp(cmd, "open_url") && url) {
+					if (!strncmp(url, "dat:", 4)) serve(url);
+					else if (!strncmp(url, "dpi:/", 5)) serve_internal(url);
+					else error_page(url, 404, "Not found");
+					stop_after_request = 1;
+				}
+				free(cmd);
+				free(url);
+			}
+		}
+		fflush(stdout);
+		/* The historical dillo-dat plugin ends the accepted socket per page. */
+		dup2(saved_in, STDIN_FILENO);
+		dup2(saved_out, STDOUT_FILENO);
+		close(saved_in);
+		close(saved_out);
+		close(client);
+		clearerr(stdin);
+		clearerr(stdout);
+	}
+	close(listener);
+	return 0;
+}
+
+int main(int argc, char **argv)
+{
+	int result;
+	(void)setlocale(LC_COLLATE, "");
+	setvbuf(stdin, NULL, _IONBF, 0);
+	setvbuf(stdout, NULL, _IONBF, 0);
+	metadata_scheme = s7_init();
+	if (argc == 2 && !strcmp(argv[1], "--stdio-test"))
+		result = serve_stdio_requests();
+	else
+		result = serve_socket_requests();
+	if (metadata_scheme) s7_free(metadata_scheme);
+	return result < 0 ? 1 : result;
+}

extra/dat-dpi/src/dat_dpi_icons.h

@@ -0,0 +1,33 @@
+#ifndef DAT_DPI_ICONS_H
+#define DAT_DPI_ICONS_H
+/* PNG data URIs copied from the bundled historical dillo-dat DPI. */
+static const char dat_dpi_icon_updir[] =
+    "iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAKxJ"
+    "REFUeNpi/P//PwMlgImBQjAMDGBBF2BkZISz09LSwCE8a9YsuCBGoIMEkDEMJCUl/b90+QoYg9i41LNgc1ZycvL/hMQkhgcPH4H5"
+    "iUnJIJf9nzt3LiNBL2RkZPwPj4hk4BMUYuDh44MEFDMLQ0xsHAMrKyvIJYyEwuDLiuXLeP7+/Qv3EihcmJmZGZiYmL5gqEcPFKBi"
+    "AyDFjCPQ/wLVX8BrwGhSJh0ABBgAsetR5KBfw9EAAAAASUVORK5CYII="
+    ;
+static const char dat_dpi_icon_directory[] =
+    "iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAN1wAADdcBQiibeAAAABl0"
+    "RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAAAAXdEVYdEF1dGhvcgBMYXBvIENhbGFtYW5kcmVp35EaKgAAACl0RVh0"
+    "RGVzY3JpcHRpb24AQmFzZWQgb2YgSmFrdWIgU3RlaW5lciBkZXNpZ26ghAVzAAABbElEQVQ4jaWQO0tDQRCFz2x2A8YHQoogaKFW"
+    "2qSysbATsdAIWgrWlhIFBRvLoFhZW/gb0vgPRBAStEgExZA2VR7X3Nw7MxY3BhUjCU6zMOz5zrcL/HPo/HDzREFnZMj1tgoI1FPm"
+    "/ePL/M2fgNxRxltaXh8xxkCEoSIQYQQdH6XHO6/T8ZePL/PFfgBLCifCqJQfesswDNBoNhAEnQQRFXLZjV+qAefiRQsAba/e27MI"
+    "Wl4Ta1t7SE3N9lVXEVxfnaYtyJjS0z04DCMlF8fK6jaSyRQatUpfwFhypvsEUrOze4CxiUmoAlBF4LfwXq/1DUcG3UJhRmJ0HI1a"
+    "9c/AzxGOAAYApEsbCiBfAMrDA5T5nwb8zYCHN/j8RABQFYAINGgYgEhUamPGKLOQiyciCFH3NABRdFsFqhoVqUJV4bebiBmjNmZd"
+    "8eW5kJ6bXxhUAADw9lpWY12BLrKZRWNjt0EYTA8DsM5Vw7a/9gEhN65EVGzVRQAAAABJRU5ErkJggg="
+    ;
+static const char dat_dpi_icon_file[] =
+    "iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAN1wAADdcBQiibeAAAABl0"
+    "RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAAAASdEVYdFRpdGxlAFBhcGVyIFNoZWV0c7mvkfkAAAAXdEVYdEF1dGhv"
+    "cgBMYXBvIENhbGFtYW5kcmVp35EaKgAAACd0RVh0RGVzY3JpcHRpb24Ad2l0aCBhIEhVR0UgaGVscCBmcm9tIEpha3VihlQHswAA"
+    "AhNJREFUOI11kstqU1EUhr91ctI2A2uTNsRaOxDEkeILiIgTL6CCAx+iUnTSgQPBRxAFSxWhA8XiBQst7aQjUV+kMWlzOaeJVZvs"
+    "y3JwctK0wQWLvQabb/3/v7eoKuubqzdFZMk5PwuKqqIKoAB/Qba8d8/v3b2/xfFSVVbXPpWbUUO990Pd7Xa0Uv2paxurf1Y+vnuc"
+    "wA87AOh0OjP5iQL7v/dptWOacZ1ao0plZ5vdepV2q8Wt67dzxanik7fvlxcGBQQAxlgAqpUK5e0KO5Ua9d2IuNlmL/pFuVwhCAKu"
+    "XrmWGx0Ze/pm+dXlFBAmAANAYSqPcy5p73DO4pwjE8OHzyuMZXNcvHAp9/3H1wXgWx9gjQGURi3CWjuU01S+xMkTBbxYgiCQg4OD"
+    "Gy9ePsvMzz1yfQUKTBTGcc7iVVHv8T5V4hhhFJExzp09z8bmesarzwIpINkaN1s454YUpCWBkC706gcysEkG+clxnPNo7y/0PsMh"
+    "QHoAa1CvwyFCQBAoipBcFY4eyWCtxTt/FCBAHO3h7P8tZMIMpeI0xlh8z+pABkLpVBG0J1UGVKQKVBARrDH9rAaeERq1iG63298Y"
+    "hiFnZmf63rWXiTEGd9wCwOmZaUTkaA8ooJfpEEBEqnEcTRcKk//1n1a73QIkMtZ0EluqzD98cCfMhoum2y2pgpI84fEZlGx2pG6M"
+    "mVtafP0F4B+wR1eZMTEGTgAAAABJRU5ErkJggg=="
+    ;
+#endif

extra/dat-dpi/src/dat_fetch.c

@@ -0,0 +1,617 @@
+#define _POSIX_C_SOURCE 200809L
+#include "dat_fetch.h"
+#include "dat.h"
+#include "dat_paths.h"
+#include "cesspool.h"
+#include "dat_session.h"
+#include "sleep_crypto.h"
+#include "sleep_utp.h"
+
+#include <sodium.h>
+#include <errno.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/stat.h>
+#include <time.h>
+#ifndef _WIN32
+#include <arpa/inet.h>
+#include <netinet/in.h>
+#include <unistd.h>
+#else
+#include <direct.h>
+#include <io.h>
+#include <winsock2.h>
+#include <ws2tcpip.h>
+#define mkdir(path, mode) _mkdir(path)
+#define chmod(path, mode) 0
+#endif
+
+#define FETCH_HELLO_BYTES 132u
+#define FETCH_ACK_BYTES 100u
+#define FETCH_RECORD_LIMIT (4u * 1024u * 1024u)
+
+typedef struct fetch_client fetch_client_t;
+typedef struct fetch_transport {
+	sleep_utp_t utp;
+	fetch_client_t *owner;
+	uint8_t peer_id[32];
+	uint8_t local_nonce[32], remote_nonce[32];
+	uint8_t auth[FETCH_HELLO_BYTES + FETCH_ACK_BYTES];
+	size_t auth_length;
+	uint8_t *tx, *rx;
+	size_t tx_length, tx_offset, tx_capacity;
+	size_t rx_length, rx_capacity;
+	int used, authenticated, hello_sent, hello_received;
+	int ack_sent, ack_received, waiting_incoming;
+} fetch_transport_t;
+
+struct fetch_client {
+	cesspool_t pool;
+	sleep_keypair_t identity;
+	sleep_feed_t metadata, content;
+	dat_session_t session;
+	uint8_t metadata_key[32], content_key[32], topic[32];
+	fetch_transport_t transports[DAT_SESSION_MAX_PEERS];
+	int pool_ready, session_ready, content_known, failed;
+	int failure_errno;
+};
+
+static uint64_t fetch_now_ms(void)
+{
+	struct timespec ts;
+	if (clock_gettime(CLOCK_MONOTONIC, &ts)) return 0;
+	return (uint64_t)ts.tv_sec * 1000u + (uint64_t)ts.tv_nsec / 1000000u;
+}
+
+static int ensure_dir(const char *path)
+{
+	if (mkdir(path, 0700) == 0 || errno == EEXIST) return 0;
+	return -1;
+}
+
+static int ensure_dir_tree(const char *path)
+{
+	char copy[1200];
+	size_t i, length;
+	if (!path || !*path) { errno = EINVAL; return -1; }
+	length = strlen(path);
+	if (length >= sizeof(copy)) { errno = ENAMETOOLONG; return -1; }
+	memcpy(copy, path, length + 1);
+	for (i = 0; i < length; i++) {
+		if (copy[i] != '/' && copy[i] != '\\') continue;
+		if (i == 0 || (i == 2 && copy[1] == ':')) continue;
+		{
+			char saved = copy[i];
+			copy[i] = '\0';
+			if (*copy && ensure_dir(copy)) return -1;
+			copy[i] = saved;
+		}
+	}
+	return ensure_dir(copy);
+}
+
+/* Cached public keys identify the archive. Never replace a mismatched key
+ * when a later request reuses this cache directory. */
+static int persist_pubkey(const char *path, const uint8_t key[32])
+{
+	uint8_t existing[32];
+	if (sleep_load_pubkey(path, existing) == 0) {
+		if (memcmp(existing, key, sizeof(existing))) { errno = EACCES; return -1; }
+		return 0;
+	}
+	if (errno != ENOENT) return -1;
+	return sleep_save_pubkey(path, key);
+}
+
+static int load_identity(sleep_keypair_t *identity)
+{
+	const char *home = dat_home_directory();
+	char directory[1100], path[1200];
+	int n;
+	if (!home || !*home) { errno = ENOENT; return -1; }
+	n = snprintf(directory, sizeof(directory), "%s/.dat", home);
+	if (n < 0 || (size_t)n >= sizeof(directory) || ensure_dir(directory)) return -1;
+	n = snprintf(path, sizeof(path), "%s/dillo-dat.network-key", directory);
+	if (n < 0 || (size_t)n >= sizeof(path)) { errno = ENAMETOOLONG; return -1; }
+	if (sleep_load_keypair(path, identity) == 0) return 0;
+	if (errno != ENOENT || sleep_gen_keypair(identity) ||
+	    sleep_save_keypair(path, identity) || chmod(path, 0600)) return -1;
+	return 0;
+}
+
+static fetch_transport_t *transport_by_id(fetch_client_t *client,
+						 const uint8_t peer_id[32])
+{
+	size_t i;
+	for (i = 0; i < DAT_SESSION_MAX_PEERS; i++)
+		if (client->transports[i].used &&
+		    !memcmp(client->transports[i].peer_id, peer_id, 32))
+			return &client->transports[i];
+	return NULL;
+}
+
+static fetch_transport_t *transport_create(fetch_client_t *client,
+						  const uint8_t peer_id[32])
+{
+	size_t i;
+	fetch_transport_t *transport = transport_by_id(client, peer_id);
+	if (transport) return transport;
+	for (i = 0; i < DAT_SESSION_MAX_PEERS; i++) {
+		transport = &client->transports[i];
+		if (transport->used) continue;
+		memset(transport, 0, sizeof(*transport));
+		if (sleep_utp_init(&transport->utp, client->pool.net.sock)) return NULL;
+		transport->used = 1;
+		transport->owner = client;
+		memcpy(transport->peer_id, peer_id, 32);
+		transport->utp.context = transport;
+		return transport;
+	}
+	errno = ENOSPC;
+	return NULL;
+}
+
+static int flush_transport(fetch_transport_t *transport)
+{
+	while (transport->tx_offset < transport->tx_length &&
+	       transport->utp.state == SLEEP_UTP_CONNECTED) {
+		size_t amount = transport->tx_length - transport->tx_offset;
+		if (amount > SLEEP_UTP_MTU) amount = SLEEP_UTP_MTU;
+		if (sleep_utp_send(&transport->utp, transport->tx + transport->tx_offset, amount)) {
+			if (errno == EAGAIN || errno == ENOBUFS) return 0;
+			return -1;
+		}
+		transport->tx_offset += amount;
+	}
+	if (transport->tx_offset == transport->tx_length)
+		transport->tx_offset = transport->tx_length = 0;
+	return 0;
+}
+
+static int queue_transport(fetch_transport_t *transport, const uint8_t *bytes, size_t length)
+{
+	size_t queued, needed, capacity;
+	uint8_t *next;
+	if (transport->tx_offset && transport->tx_length + length > transport->tx_capacity) {
+		memmove(transport->tx, transport->tx + transport->tx_offset,
+			transport->tx_length - transport->tx_offset);
+		transport->tx_length -= transport->tx_offset;
+		transport->tx_offset = 0;
+	}
+	queued = transport->tx_length - transport->tx_offset;
+	if (length > FETCH_RECORD_LIMIT - queued) { errno = ENOBUFS; return -1; }
+	needed = transport->tx_length + length;
+	if (needed > transport->tx_capacity) {
+		capacity = transport->tx_capacity ? transport->tx_capacity : 4096;
+		while (capacity < needed) {
+			if (capacity >= FETCH_RECORD_LIMIT / 2) { capacity = FETCH_RECORD_LIMIT; break; }
+			capacity *= 2;
+		}
+		next = realloc(transport->tx, capacity);
+		if (!next) return -1;
+		transport->tx = next;
+		transport->tx_capacity = capacity;
+	}
+	memcpy(transport->tx + transport->tx_length, bytes, length);
+	transport->tx_length += length;
+	return flush_transport(transport);
+}
+
+static int send_session_frame(void *context, const uint8_t peer_id[32],
+			      const uint8_t *frame, size_t frame_len)
+{
+	fetch_client_t *client = context;
+	fetch_transport_t *transport = transport_by_id(client, peer_id);
+	uint8_t *record;
+	size_t prefix, written;
+	int rc;
+	if (!transport || !transport->authenticated) { errno = ENOTCONN; return -1; }
+	if (frame_len > FETCH_RECORD_LIMIT) { errno = EMSGSIZE; return -1; }
+	prefix = 1 + sleep_varint_size(frame_len);
+	record = malloc(prefix + frame_len);
+	if (!record) return -1;
+	record[0] = 0;
+	if (sleep_varint_encode(frame_len, record + 1, prefix - 1, &written)) { free(record); return -1; }
+	memcpy(record + prefix, frame, frame_len);
+	rc = queue_transport(transport, record, prefix + frame_len);
+	free(record);
+	return rc;
+}
+
+static int send_hello(fetch_transport_t *transport)
+{
+	uint8_t hello[FETCH_HELLO_BYTES];
+	if (transport->hello_sent || transport->utp.state != SLEEP_UTP_CONNECTED) return 0;
+	memcpy(hello, "THH0", 4);
+	memcpy(hello + 4, transport->owner->identity.pubkey, 32);
+	randombytes_buf(transport->local_nonce, sizeof(transport->local_nonce));
+	memcpy(hello + 36, transport->local_nonce, 32);
+	if (sleep_sign(hello + 68, hello, 68, &transport->owner->identity) ||
+	    sleep_utp_send(&transport->utp, hello, sizeof(hello))) return -1;
+	transport->hello_sent = 1;
+	return 0;
+}
+
+static int send_ack(fetch_transport_t *transport)
+{
+	uint8_t ack[FETCH_ACK_BYTES], transcript[132];
+	if (transport->ack_sent) return 0;
+	memcpy(ack, "THA0", 4);
+	memcpy(ack + 4, transport->remote_nonce, 32);
+	memcpy(transcript, ack, 36);
+	memcpy(transcript + 36, transport->local_nonce, 32);
+	memcpy(transcript + 68, transport->owner->identity.pubkey, 32);
+	memcpy(transcript + 100, transport->peer_id, 32);
+	if (sleep_sign(ack + 36, transcript, sizeof(transcript), &transport->owner->identity) ||
+	    sleep_utp_send(&transport->utp, ack, sizeof(ack))) return -1;
+	transport->ack_sent = 1;
+	return 0;
+}
+
+static int activate_content(fetch_client_t *client)
+{
+	sleep_metadata_header_t header;
+	uint8_t *block = NULL;
+	size_t length = 0;
+	char key_path[1200];
+	int n;
+	if (client->content_known || !sleep_feed_has(&client->metadata, 0)) return 0;
+	if (sleep_feed_read(&client->metadata, 0, &block, &length)) return -1;
+	if (sleep_metadata_header_decode(&header, block, length) ||
+	    strcmp(header.type, "hyperdrive") || header.content_len != 32) {
+		free(block); errno = EPROTO; return -1;
+	}
+	free(block);
+	if (sleep_feed_load_remote(&client->content, header.content) ||
+	    dat_session_set_content_key(&client->session, header.content)) return -1;
+	n = snprintf(key_path, sizeof(key_path), "%s/content.key", client->metadata.directory);
+	if (n < 0 || (size_t)n >= sizeof(key_path) || persist_pubkey(key_path, header.content)) return -1;
+	memcpy(client->content_key, header.content, 32);
+	client->content_known = 1;
+	return 0;
+}
+
+static int receive_records(fetch_transport_t *transport, const uint8_t *bytes, size_t length)
+{
+	fetch_client_t *client = transport->owner;
+	size_t need, capacity;
+	uint8_t *next;
+	if (length > FETCH_RECORD_LIMIT - transport->rx_length) { errno = ENOBUFS; return -1; }
+	need = transport->rx_length + length;
+	if (need > transport->rx_capacity) {
+		capacity = transport->rx_capacity ? transport->rx_capacity : 4096;
+		while (capacity < need) capacity = capacity > FETCH_RECORD_LIMIT / 2 ? FETCH_RECORD_LIMIT : capacity * 2;
+		next = realloc(transport->rx, capacity);
+		if (!next) return -1;
+		transport->rx = next;
+		transport->rx_capacity = capacity;
+	}
+	if (length) memcpy(transport->rx + transport->rx_length, bytes, length);
+	transport->rx_length += length;
+	for (;;) {
+		uint64_t frame_len;
+		size_t varint_len, prefix;
+		int saved_errno;
+		if (transport->rx_length < 2) return 0;
+		if (transport->rx[0] != 0) { errno = EPROTO; return -1; }
+		errno = 0;
+		if (sleep_varint_decode(transport->rx + 1, transport->rx_length - 1,
+					&frame_len, &varint_len)) {
+			saved_errno = errno;
+			if (saved_errno == EAGAIN) return 0;
+			errno = saved_errno; return -1;
+		}
+		if (frame_len > FETCH_RECORD_LIMIT) { errno = EMSGSIZE; return -1; }
+		prefix = 1 + varint_len;
+		if (frame_len > transport->rx_length - prefix) return 0;
+		if (dat_session_receive(&client->session, transport->peer_id,
+					transport->rx + prefix, (size_t)frame_len) ||
+		    activate_content(client)) return -1;
+		memmove(transport->rx, transport->rx + prefix + (size_t)frame_len,
+			transport->rx_length - prefix - (size_t)frame_len);
+		transport->rx_length -= prefix + (size_t)frame_len;
+	}
+}
+
+static void on_utp_data(sleep_utp_t *utp, const uint8_t *bytes, size_t length)
+{
+	fetch_transport_t *transport = utp ? utp->context : NULL;
+	if (!transport || !transport->used) return;
+	if (transport->authenticated) {
+		if (receive_records(transport, bytes, length)) {
+			transport->owner->failed = 1;
+			transport->owner->failure_errno = errno ? errno : EPROTO;
+		}
+		return;
+	}
+	if (length > sizeof(transport->auth) - transport->auth_length) {
+		transport->owner->failed = 1; transport->owner->failure_errno = EMSGSIZE; return;
+	}
+	memcpy(transport->auth + transport->auth_length, bytes, length);
+	transport->auth_length += length;
+	for (;;) {
+		if (!transport->hello_received && transport->auth_length >= FETCH_HELLO_BYTES) {
+			const uint8_t *hello = transport->auth;
+			if (memcmp(hello, "THH0", 4) || memcmp(hello + 4, transport->peer_id, 32) ||
+			    sleep_verify(hello + 68, hello, 68, transport->peer_id)) {
+				transport->owner->failed = 1; transport->owner->failure_errno = EACCES; return;
+			}
+			memcpy(transport->remote_nonce, hello + 36, 32);
+			transport->hello_received = 1;
+			memmove(transport->auth, transport->auth + FETCH_HELLO_BYTES,
+				transport->auth_length - FETCH_HELLO_BYTES);
+			transport->auth_length -= FETCH_HELLO_BYTES;
+			if (send_ack(transport)) { transport->owner->failed = 1; transport->owner->failure_errno = errno; return; }
+			continue;
+		}
+		if (transport->hello_received && !transport->ack_received &&
+		    transport->auth_length >= FETCH_ACK_BYTES) {
+			const uint8_t *ack = transport->auth;
+			uint8_t transcript[132];
+			if (memcmp(ack, "THA0", 4) || memcmp(ack + 4, transport->local_nonce, 32)) {
+				transport->owner->failed = 1; transport->owner->failure_errno = EPROTO; return;
+			}
+			memcpy(transcript, ack, 36);
+			memcpy(transcript + 36, transport->remote_nonce, 32);
+			memcpy(transcript + 68, transport->peer_id, 32);
+			memcpy(transcript + 100, transport->owner->identity.pubkey, 32);
+			if (sleep_verify(ack + 36, transcript, sizeof(transcript), transport->peer_id)) {
+				transport->owner->failed = 1; transport->owner->failure_errno = EACCES; return;
+			}
+			transport->ack_received = 1;
+			memmove(transport->auth, transport->auth + FETCH_ACK_BYTES,
+				transport->auth_length - FETCH_ACK_BYTES);
+			transport->auth_length -= FETCH_ACK_BYTES;
+			transport->authenticated = 1;
+			if (dat_session_peer_secure(&transport->owner->session, transport->peer_id, 1)) {
+				transport->owner->failed = 1; transport->owner->failure_errno = errno; return;
+			}
+			continue;
+		}
+		if (transport->authenticated && transport->auth_length) {
+			if (receive_records(transport, transport->auth, transport->auth_length)) {
+				transport->owner->failed = 1; transport->owner->failure_errno = errno; return;
+			}
+			transport->auth_length = 0;
+		}
+		return;
+	}
+}
+
+static int same_endpoint(const struct sockaddr_storage *a, const struct sockaddr_storage *b)
+{
+	const struct sockaddr_in *aa = (const struct sockaddr_in *)a;
+	const struct sockaddr_in *bb = (const struct sockaddr_in *)b;
+	return aa->sin_family == AF_INET && bb->sin_family == AF_INET &&
+		aa->sin_port == bb->sin_port && aa->sin_addr.s_addr == bb->sin_addr.s_addr;
+}
+
+static void send_hello_if_ready(fetch_transport_t *transport)
+{
+	if (transport->utp.state == SLEEP_UTP_CONNECTED && send_hello(transport)) {
+		transport->owner->failed = 1;
+		transport->owner->failure_errno = errno;
+	}
+}
+
+static void transport_packet(cesspool_t *pool, const dat_peer_t *source,
+			     const uint8_t *bytes, size_t length)
+{
+	fetch_client_t *client = pool ? pool->transport_context : NULL;
+	sleep_utp_packet_t packet;
+	fetch_transport_t *transport = NULL;
+	cesspool_peer_t *known = NULL;
+	size_t i;
+	if (!client || sleep_utp_packet_decode(&packet, bytes, length)) return;
+	if (packet.type == SLEEP_UTP_SYN) {
+		for (i = 0; i < pool->peer_count; i++)
+			if (pool->peers[i].discovered &&
+			    !memcmp(pool->peers[i].discovery_topic, client->topic, 32) &&
+			    same_endpoint(&pool->peers[i].address.addr, &source->addr)) {
+				known = &pool->peers[i]; break;
+			}
+		if (!known || memcmp(client->identity.pubkey, known->public_key, 32) < 0) return;
+		transport = transport_by_id(client, known->public_key);
+		if (!transport) transport = transport_create(client, known->public_key);
+		if (!transport || transport->utp.state == SLEEP_UTP_CONNECTED) return;
+		transport->waiting_incoming = 0;
+		transport->utp.on_data = on_utp_data;
+		if (sleep_utp_accept(&transport->utp, &packet,
+				     (const struct sockaddr *)&source->addr, source->addrlen)) return;
+		send_hello_if_ready(transport);
+		return;
+	}
+	for (i = 0; i < DAT_SESSION_MAX_PEERS; i++) {
+		fetch_transport_t *candidate = &client->transports[i];
+		if (candidate->used && candidate->utp.receive_id == packet.connection_id &&
+		    same_endpoint(&candidate->utp.remote, &source->addr)) { transport = candidate; break; }
+	}
+	if (!transport) return;
+	if (sleep_utp_input(&transport->utp, bytes, length)) return;
+	send_hello_if_ready(transport);
+}
+
+static void peer_discovered(void *context, const dat_session_peer_t *peer)
+{
+	fetch_client_t *client = context;
+	fetch_transport_t *transport = transport_create(client, peer->peer_id);
+	if (!transport) return;
+	transport->utp.on_data = on_utp_data;
+	if (memcmp(client->identity.pubkey, peer->peer_id, 32) > 0) {
+		transport->waiting_incoming = 1;
+		return;
+	}
+	if (sleep_utp_connect(&transport->utp,
+			      (const struct sockaddr *)&peer->address.addr,
+			      peer->address.addrlen)) {
+		transport->owner->failed = 1;
+		transport->owner->failure_errno = errno;
+	}
+}
+
+static int open_remote_feed(sleep_feed_t *feed, const char *directory,
+			    const char *name, const uint8_t key[32])
+{
+	if (sleep_feed_open(feed, directory, name, NULL) != 0) {
+		if (errno != ENOENT) return -1;
+		return sleep_feed_open_remote(feed, directory, name, key);
+	}
+	if (sleep_feed_load_remote(feed, key)) { sleep_feed_close(feed); return -1; }
+	feed->persist_remote = 1;
+	memcpy(feed->keypair.pubkey, key, 32);
+	return 0;
+}
+
+static int feed_complete(const sleep_feed_t *feed, const sleep_replication_t *replication)
+{
+	uint64_t i;
+	if (!replication->remote_length || replication->remote_length > (uint64_t)SLEEP_BITFIELD_DATA_BYTES * 8u) return 0;
+	for (i = 0; i < replication->remote_length; i++) if (!sleep_feed_has(feed, (size_t)i)) return 0;
+	return 1;
+}
+
+static int archive_complete(const fetch_client_t *client)
+{
+	size_t p;
+	for (p = 0; p < client->session.peer_count; p++) {
+		const dat_session_peer_t *peer = &client->session.peers[p];
+		if (peer->state != DAT_SESSION_PEER_READY) continue;
+		if (!feed_complete(&client->metadata, &peer->replication[DAT_SESSION_METADATA])) continue;
+		if (!client->content_known ||
+		    !feed_complete(&client->content, &peer->replication[DAT_SESSION_CONTENT])) continue;
+		return 1;
+	}
+	return 0;
+}
+
+static void cleanup(fetch_client_t *client)
+{
+	size_t i;
+	if (client->session_ready) (void)dat_session_close(&client->session);
+	if (client->pool_ready) (void)cesspool_destroy(&client->pool);
+	for (i = 0; i < DAT_SESSION_MAX_PEERS; i++) {
+		fetch_transport_t *transport = &client->transports[i];
+		if (transport->used) (void)sleep_utp_close(&transport->utp);
+		free(transport->tx); free(transport->rx);
+	}
+	sleep_feed_close(&client->metadata);
+	sleep_feed_close(&client->content);
+}
+
+int dat_fetch_archive(const char *root, const uint8_t metadata_key[32],
+		      unsigned timeout_ms)
+{
+	fetch_client_t *client;
+	char datdir[1200], path[1400];
+	uint8_t unknown_content[32] = {0};
+	uint16_t port = DATNET_PORT;
+	uint64_t deadline;
+	int n, rc = -1, bootstrap_count;
+	if (!root || !metadata_key) { errno = EINVAL; return -1; }
+	client = calloc(1, sizeof(*client));
+	if (!client) return -1;
+	memcpy(client->metadata_key, metadata_key, 32);
+	if (sleep_crypto_init() || load_identity(&client->identity)) goto done;
+	if (ensure_dir_tree(root)) goto done;
+	n = snprintf(datdir, sizeof(datdir), "%s/.dat", root);
+	if (n < 0 || (size_t)n >= sizeof(datdir) || ensure_dir(datdir)) goto done;
+	n = snprintf(path, sizeof(path), "%s/metadata.key", datdir);
+	if (n < 0 || (size_t)n >= sizeof(path) || persist_pubkey(path, metadata_key)) goto done;
+	if (open_remote_feed(&client->metadata, datdir, "metadata", metadata_key) ||
+	    sleep_feed_open_remote(&client->content, datdir, "content", unknown_content)) goto done;
+	n = snprintf(path, sizeof(path), "%s/content.key", datdir);
+	if (n < 0 || (size_t)n >= sizeof(path)) { errno = ENAMETOOLONG; goto done; }
+	if (sleep_load_pubkey(path, client->content_key) == 0) {
+		if (sleep_feed_load_remote(&client->content, client->content_key)) goto done;
+		client->content.persist_remote = 1;
+		memcpy(client->content.keypair.pubkey, client->content_key, 32);
+		client->content_known = 1;
+	} else if (errno != ENOENT) goto done;
+	if (cesspool_init(&client->pool, client->identity.pubkey)) goto done;
+	client->pool_ready = 1;
+	client->pool.on_transport_packet = transport_packet;
+	client->pool.transport_context = client;
+	if (cesspool_listen(&client->pool, "0.0.0.0", port)) {
+		if (errno != EADDRINUSE || cesspool_listen(&client->pool, "0.0.0.0", 0)) goto done;
+	}
+	bootstrap_count = cesspool_configure_bootstraps(&client->pool);
+	if (bootstrap_count < 0) goto done;
+	if (cesspool_discovery_key(metadata_key, client->topic) ||
+	    dat_session_init(&client->session, &client->pool, metadata_key,
+			     client->identity.pubkey, client->pool.port,
+			     &client->metadata, metadata_key, &client->content,
+			     client->content_known ? client->content_key : unknown_content,
+			     send_session_frame, client)) goto done;
+	client->session_ready = 1;
+	if (!client->content_known && dat_session_expect_content_key(&client->session)) goto done;
+	if (dat_session_set_peer_callback(&client->session, peer_discovered, client) ||
+	    dat_session_start(&client->session, 1, 1)) goto done;
+	deadline = fetch_now_ms() + (timeout_ms ? timeout_ms : 30000u);
+	while (fetch_now_ms() < deadline) {
+		size_t i;
+		if (client->failed) { errno = client->failure_errno ? client->failure_errno : EIO; goto done; }
+		if (dat_session_tick(&client->session, fetch_now_ms()) < 0 && errno != EINTR) goto done;
+		for (i = 0; i < DAT_SESSION_MAX_PEERS; i++) {
+			fetch_transport_t *transport = &client->transports[i];
+			if (!transport->used) continue;
+			if (transport->waiting_incoming && transport->utp.state == SLEEP_UTP_CLOSED) continue;
+			if (sleep_utp_tick(&transport->utp, fetch_now_ms()) != 0 ||
+			    transport->utp.state == SLEEP_UTP_RESET ||
+			    transport->utp.state == SLEEP_UTP_CLOSED) {
+				transport->used = 0;
+				continue;
+			}
+			if (flush_transport(transport)) goto done;
+		}
+		if (archive_complete(client)) { rc = 0; goto done; }
+		{
+			struct timespec pause = {0, 20000000};
+			nanosleep(&pause, NULL);
+		}
+	}
+	if (!bootstrap_count && !client->session.peer_count) errno = ENETUNREACH;
+	else if (bootstrap_count && !client->pool.bootstrap_reachable) errno = EHOSTUNREACH;
+	else if (!client->session.peer_count) errno = ENOENT;
+	else {
+		size_t i, secure = 0, ready = 0, active_utp = 0, auth = 0;
+		for (i = 0; i < client->session.peer_count; i++) {
+			if (client->session.peers[i].state >= DAT_SESSION_PEER_SECURE &&
+			    client->session.peers[i].state != DAT_SESSION_PEER_CLOSED) secure++;
+			if (client->session.peers[i].state == DAT_SESSION_PEER_READY) ready++;
+		}
+		for (i = 0; i < DAT_SESSION_MAX_PEERS; i++) if (client->transports[i].used) {
+			active_utp += client->transports[i].utp.state == SLEEP_UTP_CONNECTED;
+			auth += client->transports[i].authenticated;
+		}
+		fprintf(stderr, "dat-fetch: bootstrap reachable; peers=%lu secure=%lu ready=%lu utp=%lu authenticated=%lu\n",
+			(unsigned long)client->session.peer_count, (unsigned long)secure,
+			(unsigned long)ready, (unsigned long)active_utp, (unsigned long)auth);
+		for (i = 0; i < client->session.peer_count; i++) {
+			const dat_session_peer_t *peer = &client->session.peers[i];
+			const struct sockaddr_in *address = (const struct sockaddr_in *)&peer->address.addr;
+			char host[INET_ADDRSTRLEN] = "?";
+			char peer_hex[65];
+			size_t j;
+			fetch_transport_t *transport = transport_by_id(client, peer->peer_id);
+			for (j = 0; j < 32; j++) snprintf(peer_hex + j * 2, 3, "%02x", peer->peer_id[j]);
+			if (address->sin_family == AF_INET)
+				(void)inet_ntop(AF_INET, &address->sin_addr, host, sizeof(host));
+			fprintf(stderr, "dat-fetch: peer=%s state=%d endpoint=%s:%u local-id-order=%d waiting-incoming=%d utp-state=%d\n",
+				peer_hex, peer->state, host, (unsigned)ntohs(address->sin_port),
+				memcmp(client->identity.pubkey, peer->peer_id, 32),
+				transport ? transport->waiting_incoming : 0,
+				transport ? transport->utp.state : SLEEP_UTP_CLOSED);
+		}
+		errno = ETIMEDOUT;
+	}
+done:
+	if (rc != 0 && !errno) errno = EIO;
+	{
+		int saved = errno;
+		cleanup(client);
+		free(client);
+		errno = saved;
+	}
+	return rc;
+}

extra/dat-dpi/src/dat_net.c

@@ -0,0 +1,166 @@
+#define _POSIX_C_SOURCE 200112L
+#include "dat_net.h"
+#include <errno.h>
+#include <string.h>
+#ifdef _WIN32
+#include <windows.h>
+#ifndef SIO_UDP_CONNRESET
+#define SIO_UDP_CONNRESET _WSAIOW(IOC_VENDOR, 12)
+#endif
+static int datnet_winsock_ready;
+#else
+#include <netdb.h>
+#include <unistd.h>
+#include <arpa/inet.h>
+#include <netinet/in.h>
+#include <poll.h>
+#endif
+
+#ifdef _WIN32
+static int datnet_winsock_errno(int error)
+{
+	switch (error) {
+	case WSAEINTR: return EINTR;
+	case WSAEACCES: return EACCES;
+	case WSAEADDRINUSE: return EADDRINUSE;
+	case WSAEADDRNOTAVAIL: return EADDRNOTAVAIL;
+	case WSAENETDOWN: return ENETDOWN;
+	case WSAENETUNREACH: return ENETUNREACH;
+	case WSAEHOSTUNREACH: return EHOSTUNREACH;
+	case WSAECONNRESET: return ECONNRESET;
+	case WSAECONNREFUSED: return ECONNREFUSED;
+	case WSAETIMEDOUT: return ETIMEDOUT;
+	case WSAEMSGSIZE: return EMSGSIZE;
+	case WSAENOBUFS: return ENOBUFS;
+	case WSAEWOULDBLOCK: return EAGAIN;
+	default: return EIO;
+	}
+}
+#endif
+
+int datnet_init(datnet_t *net, const uint8_t topic[32], const uint8_t peerid[32]) {
+	if (!net || !topic || !peerid) { errno = EINVAL; return -1; }
+#ifdef _WIN32
+	if (!datnet_winsock_ready) { WSADATA data; if (WSAStartup(MAKEWORD(2,2), &data)) { errno = ENETDOWN; return -1; } datnet_winsock_ready = 1; }
+#endif
+	memset(net, 0, sizeof(*net)); net->sock = DAT_INVALID_SOCKET; memcpy(net->topic, topic, 32); memcpy(net->peerid, peerid, 32); return 0;
+}
+int datnet_bind(datnet_t *net, const char *host, uint16_t port) {
+	struct sockaddr_in address; int broadcast = 1;
+	if (!net) { errno = EINVAL; return -1; }
+	memset(&address, 0, sizeof(address)); address.sin_family = AF_INET; address.sin_port = htons(port ? port : DATNET_PORT);
+	if (!host || !*host || !strcmp(host, "*")) address.sin_addr.s_addr = htonl(INADDR_ANY);
+	else {
+		struct addrinfo hints, *resolved = NULL;
+		memset(&hints, 0, sizeof(hints)); hints.ai_family = AF_INET; hints.ai_socktype = SOCK_DGRAM;
+		if (getaddrinfo(host, NULL, &hints, &resolved) != 0 || !resolved) {
+			errno = EADDRNOTAVAIL; return -1;
+		}
+		address.sin_addr = ((struct sockaddr_in *)resolved->ai_addr)->sin_addr;
+		freeaddrinfo(resolved);
+	}
+	net->sock = socket(AF_INET, SOCK_DGRAM, 0);
+	if (net->sock == DAT_INVALID_SOCKET) {
+#ifdef _WIN32
+		errno = ENETDOWN;
+#endif
+		return -1;
+	}
+#ifdef _WIN32
+	if (setsockopt(net->sock, SOL_SOCKET, SO_BROADCAST, (const char *)&broadcast, sizeof(broadcast))) {
+		int error = WSAGetLastError();
+		closesocket(net->sock); net->sock = DAT_INVALID_SOCKET;
+		errno = datnet_winsock_errno(error);
+		return -1;
+	}
+	/* A peer that is not listening can send an ICMP port-unreachable response.
+	 * Winsock reports that asynchronously as WSAECONNRESET on a UDP socket;
+	 * that is a peer/network condition, not a dead local socket. */
+	{
+		BOOL disable_udp_connreset = FALSE;
+		DWORD returned = 0;
+		(void)WSAIoctl(net->sock, SIO_UDP_CONNRESET, &disable_udp_connreset,
+			       sizeof(disable_udp_connreset), NULL, 0, &returned, NULL, NULL);
+	}
+	if (bind(net->sock, (struct sockaddr *)&address, sizeof(address))) {
+		int error = WSAGetLastError();
+		closesocket(net->sock); net->sock = DAT_INVALID_SOCKET;
+		errno = error == WSAEADDRINUSE ? EADDRINUSE :
+			error == WSAEACCES ? EACCES : EIO;
+		return -1;
+	}
+#else
+	if (setsockopt(net->sock, SOL_SOCKET, SO_BROADCAST, &broadcast, sizeof(broadcast))) {
+		int saved = errno;
+		close(net->sock); net->sock = DAT_INVALID_SOCKET; errno = saved;
+		return -1;
+	}
+	if (bind(net->sock, (struct sockaddr *)&address, sizeof(address))) { close(net->sock); net->sock = DAT_INVALID_SOCKET; return -1; }
+#endif
+	return 0;
+}
+int datnet_poll(datnet_t *net, int timeout_ms) {
+	uint8_t packet[DATNET_MAX_PACKET], peer_id[DATNET_PEERID_BYTES]; struct sockaddr_storage source; socklen_t source_len = sizeof(source); ssize_t n; dat_peer_t peer;
+	if (!net || net->sock == DAT_INVALID_SOCKET) { errno = EINVAL; return -1; }
+	memcpy(peer_id, net->peerid, sizeof(peer_id));
+#ifdef _WIN32
+	{ WSAPOLLFD pfd; int ready; pfd.fd = net->sock; pfd.events = POLLRDNORM; pfd.revents = 0;
+	  ready = WSAPoll(&pfd, 1, timeout_ms);
+	  if (ready == SOCKET_ERROR) { errno = datnet_winsock_errno(WSAGetLastError()); return -1; }
+	  if (ready == 0) return 0;
+	  if (pfd.revents & (POLLHUP | POLLNVAL)) { errno = EIO; return -1; }
+	  if (pfd.revents & POLLERR) {
+		int socket_error = 0, error_length = sizeof(socket_error);
+		if (getsockopt(net->sock, SOL_SOCKET, SO_ERROR, (char *)&socket_error,
+			       &error_length) == SOCKET_ERROR) {
+			errno = datnet_winsock_errno(WSAGetLastError());
+			return -1;
+		}
+		if (socket_error == WSAECONNRESET || socket_error == WSAECONNREFUSED ||
+		    socket_error == WSAEHOSTUNREACH || socket_error == WSAENETUNREACH)
+			return 0;
+		if (socket_error) { errno = datnet_winsock_errno(socket_error); return -1; }
+	  }
+	  if (!(pfd.revents & POLLRDNORM)) return 0; }
+#else
+	{ struct pollfd pfd; int ready; pfd.fd = net->sock; pfd.events = POLLIN; pfd.revents = 0;
+	  ready = poll(&pfd, 1, timeout_ms);
+	  if (ready < 0) return -1;
+	  if (ready == 0) return 0;
+	  if (pfd.revents & (POLLERR | POLLHUP | POLLNVAL)) { errno = EIO; return -1; }
+	  if (!(pfd.revents & POLLIN)) return 0; }
+#endif
+#ifdef _WIN32
+	{ int source_length = (int)source_len; n = recvfrom(net->sock, (char *)packet, sizeof(packet), 0, (struct sockaddr *)&source, &source_length); source_len = (socklen_t)source_length;
+	  if (n < 0) { errno = datnet_winsock_errno(WSAGetLastError()); return -1; } }
+#else
+	n = recvfrom(net->sock, packet, sizeof(packet), 0, (struct sockaddr *)&source, &source_len); if (n < 0) return -1;
+#endif
+	if (n < 0) return -1;
+	memset(&peer, 0, sizeof(peer)); peer.fd = net->sock; peer.addr = source; peer.addrlen = source_len;
+	/* Cesspool decodes discovery records here; other datagrams are passed to
+	 * its authenticated peer-transport hook. */
+	if (net->on_message) net->on_message(net, &peer, packet, (size_t)n);
+	/* The identity bound to this UDP socket is immutable. Keep it intact even
+	 * if a protocol callback uses an aliased view while handling the packet. */
+	memcpy(net->peerid, peer_id, sizeof(peer_id));
+	return 1;
+}
+int datnet_send(datnet_t *net, const dat_peer_t *peer, const uint8_t *message, size_t length) {
+	ssize_t n; if (!net || !peer || net->sock == DAT_INVALID_SOCKET || (!message && length) || length > DATNET_MAX_PACKET) { errno = EINVAL; return -1; }
+#ifdef _WIN32
+	n = sendto(net->sock, (const char *)message, (int)length, 0, (const struct sockaddr *)&peer->addr, (int)peer->addrlen);
+	if (n == SOCKET_ERROR) { errno = datnet_winsock_errno(WSAGetLastError()); return -1; }
+#else
+	n = sendto(net->sock, message, length, 0, (const struct sockaddr *)&peer->addr, peer->addrlen);
+	if (n < 0) return -1;
+#endif
+	return n == (ssize_t)length ? 0 : -1;
+}
+int datnet_close(datnet_t *net) { int rc = 0; if (!net) { errno = EINVAL; return -1; } if (net->sock != DAT_INVALID_SOCKET) {
+#ifdef _WIN32
+	rc = closesocket(net->sock);
+#else
+	rc = close(net->sock);
+#endif
+	} net->sock = DAT_INVALID_SOCKET; return rc; }

extra/dat-dpi/src/dat_session.c

@@ -0,0 +1,884 @@
+#include "dat_session.h"
+#include "errno_compat.h"
+
+#include "sleep.h"
+#include "sleep_crypto.h"
+#include "sleep_proof.h"
+#include <errno.h>
+#include <stdlib.h>
+#include <string.h>
+#ifdef DAT_LEGACY_HYPERCORE
+#include <sodium.h>
+#endif
+
+#define DAT_SESSION_FRAME_BYTES (128u * 1024u)
+
+#ifdef DAT_LEGACY_HYPERCORE
+static const uint64_t feed_channels[DAT_SESSION_FEED_COUNT] = {0, 1};
+#else
+static const uint64_t feed_channels[DAT_SESSION_FEED_COUNT] = {1, 2};
+#endif
+
+static int valid_feed(enum dat_session_feed feed)
+{
+	return feed >= DAT_SESSION_METADATA && feed < DAT_SESSION_FEED_COUNT;
+}
+
+static int feed_for_channel(uint64_t channel, enum dat_session_feed *feed)
+{
+	size_t i;
+	for (i = 0; i < DAT_SESSION_FEED_COUNT; i++) if (feed_channels[i] == channel) {
+		*feed = (enum dat_session_feed)i;
+		return 0;
+	}
+	errno = EPROTO;
+	return -1;
+}
+
+static int discover_known_peers(dat_session_t *session)
+{
+	cesspool_peer_t *peers[CESSPOOL_MAX_PEERS];
+	size_t count, i;
+	count = cesspool_topic_peers(session->cesspool, session->discovery_topic,
+				     peers, CESSPOOL_MAX_PEERS);
+	for (i = 0; i < count; i++)
+		if (dat_session_peer_discovered(session, peers[i])) return -1;
+	return 0;
+}
+
+dat_session_peer_t *dat_session_find_peer(dat_session_t *session,
+					  const uint8_t peer_id[DAT_PEER_ID_BYTES])
+{
+	size_t i;
+	if (!session || !peer_id) return NULL;
+	for (i = 0; i < session->peer_count; i++)
+		if (!memcmp(session->peers[i].peer_id, peer_id, DAT_PEER_ID_BYTES))
+			return &session->peers[i];
+	return NULL;
+}
+
+int dat_session_init(dat_session_t *session, cesspool_t *pool,
+		     const uint8_t archive_key[DAT_KEY_BYTES],
+		     const uint8_t peer_id[DAT_PEER_ID_BYTES], uint16_t port,
+		     sleep_feed_t *metadata_feed,
+		     const uint8_t metadata_key[DAT_KEY_BYTES],
+		     sleep_feed_t *content_feed,
+		     const uint8_t content_key[DAT_KEY_BYTES],
+		     dat_session_send_fn send, void *context)
+{
+	if (!session || !pool || !archive_key || !peer_id || !metadata_feed ||
+	    !metadata_key || !content_feed || !content_key || !send) {
+		errno = EINVAL;
+		return -1;
+	}
+	memset(session, 0, sizeof(*session));
+	memcpy(session->archive_key, archive_key, DAT_KEY_BYTES);
+	memcpy(session->peer_id, peer_id, DAT_PEER_ID_BYTES);
+	memcpy(session->feed_keys[DAT_SESSION_METADATA], metadata_key, DAT_KEY_BYTES);
+	memcpy(session->feed_keys[DAT_SESSION_CONTENT], content_key, DAT_KEY_BYTES);
+	session->content_key_known = 1;
+	if (cesspool_discovery_key(archive_key, session->discovery_topic)) return -1;
+	session->cesspool = pool;
+	session->feeds[DAT_SESSION_METADATA] = metadata_feed;
+	session->feeds[DAT_SESSION_CONTENT] = content_feed;
+	session->send = send;
+	session->context = context;
+	session->port = port;
+	if (session->port == 0) session->port = pool->port;
+	session->state = DAT_SESSION_OPEN;
+	return 0;
+}
+
+int dat_session_start(dat_session_t *session, int announce, int lookup)
+{
+	if (!session || session->state != DAT_SESSION_OPEN || (!announce && !lookup)) {
+		errno = EINVAL;
+		return -1;
+	}
+	session->announcing = !!announce;
+	session->looking_up = !!lookup;
+	if (cesspool_join(session->cesspool, session->discovery_topic,
+			  session->port, announce, lookup)) {
+		(void)cesspool_leave(session->cesspool, session->discovery_topic);
+		session->state = DAT_SESSION_FAILED;
+		return -1;
+	}
+	session->state = DAT_SESSION_DISCOVERING;
+	return discover_known_peers(session);
+}
+
+int dat_session_peer_discovered(dat_session_t *session,
+				const cesspool_peer_t *discovered_peer)
+{
+	dat_session_peer_t *peer;
+	if (!session || !discovered_peer || (session->state != DAT_SESSION_DISCOVERING &&
+	    session->state != DAT_SESSION_ACTIVE)) { errno = EINVAL; return -1; }
+	peer = dat_session_find_peer(session, discovered_peer->public_key);
+	if (peer) {
+		int retry = peer->state == DAT_SESSION_PEER_CLOSED;
+		peer->address = discovered_peer->address;
+		if (retry) {
+			peer->state = DAT_SESSION_PEER_DISCOVERED;
+			peer->transport_authenticated = 0;
+			free(peer->rx_buffer);
+			peer->rx_buffer = NULL;
+			peer->rx_buffer_len = peer->rx_buffer_capacity = 0;
+			peer->tx_offset = peer->rx_offset = 0;
+			peer->remote_feed_mask = 0;
+			peer->local_feed_mask = 0;
+			peer->remote_content_pending = 0;
+			memset(peer->remote_content_discovery, 0,
+			       sizeof(peer->remote_content_discovery));
+			peer->tx_stream_encrypted = peer->rx_stream_encrypted = 0;
+			if (session->on_peer) session->on_peer(session->peer_context, peer);
+		}
+		return 0;
+	}
+	if (session->peer_count >= DAT_SESSION_MAX_PEERS) { errno = ENOSPC; return -1; }
+	peer = &session->peers[session->peer_count];
+	memset(peer, 0, sizeof(*peer));
+	memcpy(peer->peer_id, discovered_peer->public_key, DAT_PEER_ID_BYTES);
+	peer->address = discovered_peer->address;
+	peer->state = DAT_SESSION_PEER_DISCOVERED;
+	if (sleep_replication_init(&peer->replication[DAT_SESSION_METADATA],
+				   session->feeds[DAT_SESSION_METADATA],
+				   feed_channels[DAT_SESSION_METADATA]) ||
+	    sleep_replication_init(&peer->replication[DAT_SESSION_CONTENT],
+				   session->feeds[DAT_SESSION_CONTENT],
+				   feed_channels[DAT_SESSION_CONTENT])) return -1;
+	session->peer_count++;
+	if (session->on_peer) session->on_peer(session->peer_context, peer);
+	return 0;
+}
+
+int dat_session_set_peer_callback(dat_session_t *session,
+				  dat_session_peer_fn callback, void *context)
+{
+	size_t i;
+	if (!session || session->state == DAT_SESSION_CLOSED ||
+	    session->state == DAT_SESSION_CLOSING) { errno = EINVAL; return -1; }
+	session->on_peer = callback;
+	session->peer_context = context;
+	if (callback)
+		for (i = 0; i < session->peer_count; i++)
+			if (session->peers[i].state == DAT_SESSION_PEER_DISCOVERED)
+				callback(context, &session->peers[i]);
+	return 0;
+}
+
+int dat_session_set_transport_poll(dat_session_t *session,
+				   dat_session_poll_fn poll, void *context)
+{
+	if (!session || session->state == DAT_SESSION_CLOSED ||
+	    session->state == DAT_SESSION_CLOSING) { errno = EINVAL; return -1; }
+	session->poll_transport = poll;
+	session->poll_context = context;
+	return 0;
+}
+
+static int send_frame(dat_session_t *session, dat_session_peer_t *peer,
+		      uint64_t channel, uint64_t type,
+		      const uint8_t *payload, size_t payload_len)
+{
+	dat_wire_message_t message;
+	uint8_t frame[DAT_SESSION_FRAME_BYTES];
+	size_t frame_len;
+	int send_result;
+	message.channel = channel;
+	message.type = type;
+	message.payload = payload;
+	message.payload_len = payload_len;
+	if (dat_wire_encode(&message, frame, sizeof(frame), &frame_len)) return -1;
+#ifdef DAT_LEGACY_HYPERCORE
+	if (peer->tx_stream_encrypted) {
+		if (sleep_stream_xsalsa20_xor(frame, frame, frame_len,
+					      peer->tx_nonce,
+					      session->feed_keys[DAT_SESSION_METADATA],
+					      peer->tx_offset)) return -1;
+		if (peer->tx_offset > UINT64_MAX - frame_len) { errno = EOVERFLOW; return -1; }
+	}
+#endif
+	send_result = session->send(session->context, peer->peer_id, frame, frame_len);
+#ifdef DAT_LEGACY_HYPERCORE
+	if (!send_result && peer->tx_stream_encrypted) peer->tx_offset += frame_len;
+#endif
+	return send_result;
+}
+
+static int send_have_range(dat_session_t *session, dat_session_peer_t *peer,
+			   enum dat_session_feed feed, uint64_t start,
+			   uint64_t length)
+{
+	dat_range_t range = {start, length};
+	uint8_t payload[32];
+	size_t payload_len;
+	if (!range.length) return 0;
+	if (dat_range_encode(&range, payload, sizeof(payload), &payload_len)) return -1;
+	return send_frame(session, peer, feed_channels[feed], DAT_MSG_HAVE,
+			  payload, payload_len);
+}
+
+static int send_have(dat_session_t *session, dat_session_peer_t *peer,
+		     enum dat_session_feed feed)
+{
+	size_t index, length = sleep_feed_length(session->feeds[feed]);
+	size_t run_start = 0;
+	int in_run = 0;
+	if (length > SLEEP_BITFIELD_DATA_BYTES * 8u)
+		length = SLEEP_BITFIELD_DATA_BYTES * 8u;
+	for (index = 0; index < length; index++) {
+		int have = sleep_feed_has(session->feeds[feed], index);
+		if (have && !in_run) { run_start = index; in_run = 1; }
+		if (!have && in_run) {
+			if (send_have_range(session, peer, feed, run_start, index - run_start)) return -1;
+			in_run = 0;
+		}
+	}
+	if (in_run) return send_have_range(session, peer, feed, run_start, length - run_start);
+	return 0;
+}
+
+static int answer_want(dat_session_t *session, dat_session_peer_t *peer,
+		       enum dat_session_feed feed, const dat_range_t *range)
+{
+	uint64_t i, end, run_start = 0;
+	int in_run = 0;
+	uint64_t feed_length = sleep_feed_length(session->feeds[feed]);
+	if (range->start > UINT64_MAX - range->length) {
+		errno = EOVERFLOW;
+		return -1;
+	}
+	end = range->start + range->length;
+	if (end > feed_length) end = feed_length;
+	if (end > SLEEP_BITFIELD_DATA_BYTES * 8u) end = SLEEP_BITFIELD_DATA_BYTES * 8u;
+	for (i = range->start; i < end; i++) {
+		int have = i <= SIZE_MAX && sleep_feed_has(session->feeds[feed], (size_t)i);
+		if (have && !in_run) { run_start = i; in_run = 1; }
+		if (!have && in_run) {
+			if (send_have_range(session, peer, feed, run_start, i - run_start)) return -1;
+			in_run = 0;
+		}
+	}
+	if (in_run && send_have_range(session, peer, feed, run_start, end - run_start)) return -1;
+	return 0;
+}
+
+static int announce_received_block(dat_session_t *session,
+				   enum dat_session_feed feed, uint64_t index)
+{
+	dat_range_t range = {index, 1};
+	uint8_t payload[32];
+	size_t payload_len, i;
+	if (dat_range_encode(&range, payload, sizeof(payload), &payload_len)) return -1;
+	for (i = 0; i < session->peer_count; i++) {
+		dat_session_peer_t *peer = &session->peers[i];
+		if (peer->state == DAT_SESSION_PEER_READY &&
+		    (peer->remote_feed_mask & (uint8_t)(1u << feed)) &&
+		    send_frame(session, peer, feed_channels[feed], DAT_MSG_HAVE,
+			       payload, payload_len)) return -1;
+	}
+	return 0;
+}
+
+static int pump_replication(dat_session_t *session, dat_session_peer_t *peer,
+			    enum dat_session_feed feed)
+{
+	sleep_replication_t *replication = &peer->replication[feed];
+	size_t index, limit;
+	if (peer->state != DAT_SESSION_PEER_READY || peer->paused || !replication->uploading) return 0;
+	limit = replication->remote_length > SLEEP_BITFIELD_DATA_BYTES * 8u ?
+		SLEEP_BITFIELD_DATA_BYTES * 8u : (size_t)replication->remote_length;
+	for (index = 0; index < limit; index++) {
+		int requested, available;
+		if (sleep_feed_has(session->feeds[feed], index)) continue;
+		available = sleep_bitfield_get_data(&replication->remote_bitfield, index);
+		if (available <= 0) continue;
+		requested = sleep_replication_is_requested(replication, index);
+		if (requested < 0) return -1;
+		if (requested) continue;
+		if (dat_session_request_block(session, peer->peer_id, feed, index, 0)) {
+			if (errno == ENOSPC) return 0;
+			return -1;
+		}
+	}
+	return 0;
+}
+
+int dat_session_peer_secure(dat_session_t *session,
+			    const uint8_t peer_id[DAT_PEER_ID_BYTES], int authenticated)
+{
+	dat_session_peer_t *peer;
+	dat_handshake_t handshake;
+	dat_feed_open_t feed_open;
+	uint8_t payload[128];
+	size_t payload_len;
+	if (!session || !peer_id || !authenticated) { errno = EACCES; return -1; }
+	peer = dat_session_find_peer(session, peer_id);
+	if (!peer || peer->state != DAT_SESSION_PEER_DISCOVERED) { errno = EINVAL; return -1; }
+	peer->transport_authenticated = 1;
+	peer->state = DAT_SESSION_PEER_SECURE;
+#ifdef DAT_LEGACY_HYPERCORE
+	memset(&feed_open, 0, sizeof(feed_open));
+	if (sleep_crypto_init()) goto fail;
+	if (sleep_compute_discovery_key(session->feed_keys[DAT_SESSION_METADATA],
+					 feed_open.discovery_key)) goto fail;
+	feed_open.has_nonce = 1;
+	randombytes_buf(feed_open.nonce, sizeof(feed_open.nonce));
+	memcpy(peer->tx_nonce, feed_open.nonce, sizeof(peer->tx_nonce));
+	if (dat_feed_open_encode(&feed_open, payload, sizeof(payload), &payload_len) ||
+	    send_frame(session, peer, feed_channels[DAT_SESSION_METADATA], DAT_MSG_FEED,
+		       payload, payload_len)) goto fail;
+	peer->local_feed_mask |= (uint8_t)(1u << DAT_SESSION_METADATA);
+	peer->tx_stream_encrypted = 1;
+	if (dat_handshake_init(&handshake, session->peer_id,
+			       session->content_key_known ? 0 : DAT_SESSION_CAP_METADATA_ONLY) ||
+	    dat_handshake_encode(&handshake, payload, sizeof(payload), &payload_len) ||
+	    send_frame(session, peer, 0, DAT_MSG_HANDSHAKE, payload, payload_len)) goto fail;
+	memset(&feed_open, 0, sizeof(feed_open));
+	if (session->content_key_known &&
+	    (sleep_compute_discovery_key(session->feed_keys[DAT_SESSION_CONTENT],
+					 feed_open.discovery_key) ||
+	    dat_feed_open_encode(&feed_open, payload, sizeof(payload), &payload_len) ||
+	    send_frame(session, peer, feed_channels[DAT_SESSION_CONTENT], DAT_MSG_FEED,
+		       payload, payload_len))) goto fail;
+	if (session->content_key_known) peer->local_feed_mask |= (uint8_t)(1u << DAT_SESSION_CONTENT);
+	return 0;
+#else
+	memset(&feed_open, 0, sizeof(feed_open));
+	if (sleep_compute_discovery_key(session->feed_keys[DAT_SESSION_METADATA],
+					 feed_open.discovery_key) ||
+	    dat_feed_open_encode(&feed_open, payload, sizeof(payload), &payload_len) ||
+	    send_frame(session, peer, feed_channels[DAT_SESSION_METADATA],
+		       DAT_MSG_FEED, payload, payload_len)) goto fail;
+	peer->local_feed_mask |= (uint8_t)(1u << DAT_SESSION_METADATA);
+	if (session->content_key_known &&
+	    (sleep_compute_discovery_key(session->feed_keys[DAT_SESSION_CONTENT],
+					 feed_open.discovery_key) ||
+	    dat_feed_open_encode(&feed_open, payload, sizeof(payload), &payload_len) ||
+	    send_frame(session, peer, feed_channels[DAT_SESSION_CONTENT],
+		       DAT_MSG_FEED, payload, payload_len))) goto fail;
+	if (session->content_key_known) peer->local_feed_mask |= (uint8_t)(1u << DAT_SESSION_CONTENT);
+	if (dat_handshake_init(&handshake, session->peer_id,
+			       session->content_key_known ? 0 : DAT_SESSION_CAP_METADATA_ONLY) ||
+	    dat_handshake_encode(&handshake, payload, sizeof(payload), &payload_len) ||
+	    send_frame(session, peer, 0, DAT_MSG_HANDSHAKE, payload, payload_len)) goto fail;
+	return 0;
+#endif
+
+fail:
+	peer->state = DAT_SESSION_PEER_CLOSED;
+	peer->transport_authenticated = 0;
+	peer->tx_stream_encrypted = peer->rx_stream_encrypted = 0;
+	peer->tx_offset = peer->rx_offset = 0;
+	free(peer->rx_buffer);
+	peer->rx_buffer = NULL;
+	peer->rx_buffer_len = peer->rx_buffer_capacity = 0;
+	return -1;
+}
+
+int dat_session_expect_content_key(dat_session_t *session)
+{
+	if (!session || session->state != DAT_SESSION_OPEN) { errno = EINVAL; return -1; }
+	memset(session->feed_keys[DAT_SESSION_CONTENT], 0, DAT_KEY_BYTES);
+	session->content_key_known = 0;
+	return 0;
+}
+
+int dat_session_set_content_key(dat_session_t *session,
+				const uint8_t content_key[DAT_KEY_BYTES])
+{
+	uint8_t expected[32];
+	size_t i;
+	if (!session || !content_key || session->state == DAT_SESSION_CLOSED ||
+	    session->state == DAT_SESSION_CLOSING) { errno = EINVAL; return -1; }
+	if (session->content_key_known) {
+		if (!memcmp(session->feed_keys[DAT_SESSION_CONTENT], content_key, DAT_KEY_BYTES)) return 0;
+		errno = EKEYREJECTED;
+		return -1;
+	}
+	if (sleep_compute_discovery_key(content_key, expected)) return -1;
+	for (i = 0; i < session->peer_count; i++) {
+		dat_session_peer_t *peer = &session->peers[i];
+		if (peer->remote_content_pending &&
+		    memcmp(peer->remote_content_discovery, expected, sizeof(expected))) {
+			errno = EKEYREJECTED;
+			return -1;
+		}
+	}
+	memcpy(session->feed_keys[DAT_SESSION_CONTENT], content_key, DAT_KEY_BYTES);
+	memcpy(session->feeds[DAT_SESSION_CONTENT]->keypair.pubkey, content_key, DAT_KEY_BYTES);
+	session->content_key_known = 1;
+	for (i = 0; i < session->peer_count; i++) {
+		dat_session_peer_t *peer = &session->peers[i];
+		dat_feed_open_t feed_open;
+		uint8_t payload[128];
+		size_t payload_len;
+		peer->remote_content_pending = 0;
+		if (peer->state != DAT_SESSION_PEER_READY && peer->state != DAT_SESSION_PEER_SECURE) continue;
+		if (!(peer->local_feed_mask & (uint8_t)(1u << DAT_SESSION_CONTENT))) {
+			memset(&feed_open, 0, sizeof(feed_open));
+			if (sleep_compute_discovery_key(content_key, feed_open.discovery_key) ||
+			    dat_feed_open_encode(&feed_open, payload, sizeof(payload), &payload_len) ||
+			    send_frame(session, peer, feed_channels[DAT_SESSION_CONTENT], DAT_MSG_FEED,
+				       payload, payload_len)) return -1;
+			peer->local_feed_mask |= (uint8_t)(1u << DAT_SESSION_CONTENT);
+		}
+		if (peer->state == DAT_SESSION_PEER_READY &&
+		    (peer->remote_feed_mask & (uint8_t)(1u << DAT_SESSION_CONTENT)) &&
+		    send_have(session, peer, DAT_SESSION_CONTENT)) return -1;
+	}
+	return 0;
+}
+
+static int receive_data(dat_session_t *session, dat_session_peer_t *peer,
+			      enum dat_session_feed feed, const uint8_t *payload,
+			      size_t payload_len)
+{
+	dat_data_t data;
+	sleep_proof_t proof;
+	sleep_feed_t *local;
+	uint8_t *old_data = NULL;
+	uint8_t proof_cache[8192];
+	const uint8_t *cached_proof;
+	size_t old_len = 0;
+	size_t cached_proof_len;
+	size_t i;
+	if (!valid_feed(feed) || dat_data_decode(&data, payload, payload_len)) return -1;
+	local = session->feeds[feed];
+	if (data.proof_len) {
+		if (sleep_proof_decode(&proof, data.proof, data.proof_len)) return -1;
+	} else {
+		sleep_proof_node_t nodes[DAT_DATA_MAX_NODES];
+		uint64_t remote_length = peer->replication[feed].remote_length;
+		if (!data.has_signature || !remote_length || data.node_count > DAT_DATA_MAX_NODES) {
+			errno = EPROTO;
+			return -1;
+		}
+		for (i = 0; i < data.node_count; i++) {
+			nodes[i].index = data.nodes[i].index;
+			nodes[i].length = data.nodes[i].length;
+			memcpy(nodes[i].hash, data.nodes[i].hash, 32);
+		}
+		if (sleep_proof_from_nodes(data.index, data.value, data.value_len,
+					   nodes, data.node_count, data.signature,
+					   remote_length, &proof)) return -1;
+	}
+	if (proof.block != data.index || !proof.has_signature ||
+	    sleep_proof_verify(&proof, data.value, data.value_len,
+			       session->feed_keys[feed])) {
+		errno = EBADMSG;
+		return -1;
+	}
+	cached_proof = data.proof;
+	cached_proof_len = data.proof_len;
+	if (!cached_proof_len) {
+		if (sleep_proof_encode(&proof, proof_cache, sizeof(proof_cache),
+				       &cached_proof_len)) return -1;
+		cached_proof = proof_cache;
+	}
+	if (data.index >= SLEEP_BITFIELD_DATA_BYTES * 8u ||
+	    (peer->replication[feed].remote_length &&
+	     data.index >= peer->replication[feed].remote_length)) {
+		errno = ERANGE;
+		return -1;
+	}
+	if (peer->paused)
+		return sleep_replication_complete(&peer->replication[feed], data.index);
+	if (sleep_feed_has(local, (size_t)data.index)) {
+		if (sleep_feed_read(local, (size_t)data.index, &old_data, &old_len)) return -1;
+		if (old_len != data.value_len || memcmp(old_data, data.value, old_len)) {
+			free(old_data); errno = EBADMSG; return -1;
+		}
+		free(old_data);
+		peer->payload_bytes_received += data.value_len;
+		if (sleep_replication_complete(&peer->replication[feed], data.index)) return -1;
+		return pump_replication(session, peer, feed);
+	}
+	if (sleep_feed_append_remote(local, (size_t)data.index, data.value,
+				     data.value_len, cached_proof,
+				     cached_proof_len)) return -1;
+	peer->payload_bytes_received += data.value_len;
+	if (sleep_replication_complete(&peer->replication[feed], data.index)) return -1;
+	if (announce_received_block(session, feed, data.index)) return -1;
+	return pump_replication(session, peer, feed);
+}
+
+static int dispatch_request(dat_session_t *session, dat_session_peer_t *peer,
+			    enum dat_session_feed feed,
+			    const dat_request_t *request)
+{
+	if (session->on_request)
+		return session->on_request(session->context, peer->peer_id, feed, request);
+	return dat_session_send_block(session, peer->peer_id, feed, request->index);
+}
+
+static int dat_session_process_frame(dat_session_t *session,
+				     dat_session_peer_t *peer,
+				     const uint8_t *frame, size_t frame_len)
+{
+	dat_wire_message_t message;
+	size_t consumed;
+	if (dat_wire_decode(&message, frame, frame_len, &consumed) || consumed != frame_len) return -1;
+	if (message.type == DAT_MSG_FEED) {
+		dat_feed_open_t remote_feed;
+		enum dat_session_feed feed;
+		uint8_t expected[32];
+		if ((peer->state != DAT_SESSION_PEER_SECURE && peer->state != DAT_SESSION_PEER_READY) ||
+		    feed_for_channel(message.channel, &feed) ||
+		    (peer->remote_feed_mask & (uint8_t)(1u << feed)) ||
+		    dat_feed_open_decode(&remote_feed, message.payload, message.payload_len)) {
+			errno = EKEYREJECTED;
+			return -1;
+		}
+		if (feed == DAT_SESSION_CONTENT && !session->content_key_known) {
+			memcpy(peer->remote_content_discovery, remote_feed.discovery_key, 32);
+			peer->remote_content_pending = 1;
+		} else if (sleep_compute_discovery_key(session->feed_keys[feed], expected) ||
+			   memcmp(remote_feed.discovery_key, expected, sizeof(expected))) {
+			errno = EKEYREJECTED;
+			return -1;
+		}
+#ifdef DAT_LEGACY_HYPERCORE
+		if (feed == DAT_SESSION_METADATA && message.channel == 0) {
+			if (!remote_feed.has_nonce || peer->rx_stream_encrypted) { errno = EPROTO; return -1; }
+			memcpy(peer->rx_nonce, remote_feed.nonce, sizeof(peer->rx_nonce));
+			peer->rx_offset = 0;
+			peer->rx_stream_encrypted = 1;
+		}
+#endif
+		peer->remote_feed_mask |= (uint8_t)(1u << feed);
+		if (peer->state == DAT_SESSION_PEER_READY)
+			return send_have(session, peer, feed);
+		return 0;
+	}
+	if (message.type == DAT_MSG_HANDSHAKE) {
+		dat_handshake_t remote;
+		if (message.channel != 0 || peer->state != DAT_SESSION_PEER_SECURE ||
+		    !(peer->remote_feed_mask & (uint8_t)(1u << DAT_SESSION_METADATA)) ||
+		    dat_handshake_decode(&remote, message.payload, message.payload_len) ||
+		    dat_handshake_protocol_matches(&remote) != 1
+#ifndef DAT_LEGACY_HYPERCORE
+		    || (remote.has_peer_id && memcmp(remote.peer_id, peer->peer_id,
+						       DAT_PEER_ID_BYTES))
+#endif
+		    ) {
+			errno = EPROTO;
+			return -1;
+		}
+		#ifndef DAT_LEGACY_HYPERCORE
+		if (!(remote.capabilities & DAT_SESSION_CAP_METADATA_ONLY) &&
+		    peer->remote_feed_mask != (uint8_t)((1u << DAT_SESSION_FEED_COUNT) - 1u)) {
+			errno = EPROTO;
+			return -1;
+		}
+		if ((remote.capabilities & DAT_SESSION_CAP_METADATA_ONLY) &&
+		    peer->remote_feed_mask != (uint8_t)(1u << DAT_SESSION_METADATA)) {
+			errno = EPROTO;
+			return -1;
+		}
+		#endif
+		peer->capabilities = remote.capabilities;
+		peer->state = DAT_SESSION_PEER_READY;
+		session->state = DAT_SESSION_ACTIVE;
+		for (size_t feed = 0; feed < DAT_SESSION_FEED_COUNT; feed++)
+			if ((peer->remote_feed_mask & (uint8_t)(1u << feed)) &&
+			    send_have(session, peer, (enum dat_session_feed)feed)) return -1;
+		return 0;
+	}
+	{
+		enum dat_session_feed feed;
+		if (peer->state != DAT_SESSION_PEER_READY || feed_for_channel(message.channel, &feed)) {
+		errno = EPROTO;
+		return -1;
+		}
+		if (!(peer->remote_feed_mask & (uint8_t)(1u << feed))) {
+			errno = EPROTO;
+			return -1;
+		}
+		if (message.type == DAT_MSG_INFO) {
+			dat_info_t info;
+			if (dat_info_decode(&info, message.payload, message.payload_len)) return -1;
+			peer->replication[feed].uploading = info.uploading;
+			peer->replication[feed].downloading = info.downloading;
+			return pump_replication(session, peer, feed);
+		}
+		if (message.type == DAT_MSG_UNHAVE) {
+			dat_range_t range;
+			if (dat_range_decode(&range, message.payload, message.payload_len)) return -1;
+			if (sleep_replication_unhave(&peer->replication[feed], range.start,
+						     range.length)) return -1;
+			return sleep_replication_unwant(&peer->replication[feed], range.start,
+						 range.length);
+		}
+		if (message.type == DAT_MSG_HAVE || message.type == DAT_MSG_WANT) {
+		dat_range_t range;
+		if (message.type == DAT_MSG_HAVE) {
+			if (dat_range_decode(&range, message.payload, message.payload_len)) return -1;
+			if (sleep_replication_have(&peer->replication[feed],
+						   range.start, range.length)) return -1;
+			return pump_replication(session, peer, feed);
+		}
+		if (dat_want_decode(&range, message.payload, message.payload_len,
+				    sleep_feed_length(session->feeds[feed]))) return -1;
+		return answer_want(session, peer, feed, &range);
+	}
+	if (message.type == DAT_MSG_UNWANT) return 0;
+	if (message.type == DAT_MSG_REQUEST) {
+		dat_request_t request;
+		if (dat_request_decode(&request, message.payload, message.payload_len)) return -1;
+		return dispatch_request(session, peer, feed, &request);
+	}
+	if (message.type == DAT_MSG_DATA)
+		return receive_data(session, peer, feed, message.payload, message.payload_len);
+	if (message.type == DAT_MSG_CANCEL) return 0;
+	errno = EPROTONOSUPPORT;
+	return -1;
+	}
+}
+
+static int dat_session_receive_buffered(dat_session_t *session,
+					dat_session_peer_t *peer,
+					const uint8_t *bytes, size_t length)
+{
+	if (length > DAT_SESSION_FRAME_BYTES * 2u ||
+	    peer->rx_buffer_len > DAT_SESSION_FRAME_BYTES * 2u - length) {
+		errno = EMSGSIZE;
+		return -1;
+	}
+	if (peer->rx_buffer_capacity < peer->rx_buffer_len + length) {
+		size_t capacity = peer->rx_buffer_len + length;
+		uint8_t *next;
+		if (capacity < 4096) capacity = 4096;
+		next = realloc(peer->rx_buffer, capacity);
+		if (!next) return -1;
+		peer->rx_buffer = next;
+		peer->rx_buffer_capacity = capacity;
+	}
+#ifdef DAT_LEGACY_HYPERCORE
+	if (peer->rx_stream_encrypted) {
+		if (sleep_stream_xsalsa20_xor(peer->rx_buffer + peer->rx_buffer_len,
+					      bytes, length, peer->rx_nonce,
+					      session->feed_keys[DAT_SESSION_METADATA],
+					      peer->rx_offset)) return -1;
+		peer->rx_offset += length;
+	} else if (length) {
+		memcpy(peer->rx_buffer + peer->rx_buffer_len, bytes, length);
+	}
+#else
+	if (length) memcpy(peer->rx_buffer + peer->rx_buffer_len, bytes, length);
+#endif
+	peer->rx_buffer_len += length;
+	while (peer->rx_buffer_len) {
+		uint64_t body_len;
+		size_t prefix, total;
+	#ifdef DAT_LEGACY_HYPERCORE
+		int was_encrypted = peer->rx_stream_encrypted;
+	#endif
+		if (sleep_varint_decode(peer->rx_buffer, peer->rx_buffer_len,
+					&body_len, &prefix)) {
+			if (errno == EAGAIN) return 0;
+			return -1;
+		}
+		if (!body_len || body_len > DAT_SESSION_FRAME_BYTES ||
+		    body_len > SIZE_MAX - prefix) { errno = EMSGSIZE; return -1; }
+		total = prefix + (size_t)body_len;
+		if (peer->rx_buffer_len < total) return 0;
+		if (dat_session_process_frame(session, peer, peer->rx_buffer, total)) return -1;
+		peer->rx_buffer_len -= total;
+		if (peer->rx_buffer_len)
+			memmove(peer->rx_buffer, peer->rx_buffer + total, peer->rx_buffer_len);
+#ifdef DAT_LEGACY_HYPERCORE
+		/* Feed(0) turns encryption on mid-read; cipher the buffered tail from offset zero. */
+		if (!was_encrypted && peer->rx_stream_encrypted && peer->rx_buffer_len) {
+			if (sleep_stream_xsalsa20_xor(peer->rx_buffer, peer->rx_buffer,
+						      peer->rx_buffer_len, peer->rx_nonce,
+						      session->feed_keys[DAT_SESSION_METADATA], 0)) return -1;
+			peer->rx_offset = peer->rx_buffer_len;
+		}
+#endif
+	}
+	return 0;
+}
+
+int dat_session_receive(dat_session_t *session,
+			const uint8_t peer_id[DAT_PEER_ID_BYTES],
+			const uint8_t *bytes, size_t length)
+{
+	dat_session_peer_t *peer;
+	if (!session || !peer_id || (!bytes && length)) { errno = EINVAL; return -1; }
+	peer = dat_session_find_peer(session, peer_id);
+	if (!peer || !peer->transport_authenticated) { errno = EACCES; return -1; }
+	return dat_session_receive_buffered(session, peer, bytes, length);
+}
+
+int dat_session_request_block(dat_session_t *session,
+			      const uint8_t peer_id[DAT_PEER_ID_BYTES],
+			      enum dat_session_feed feed, uint64_t index,
+			      uint64_t bytes)
+{
+	dat_session_peer_t *peer;
+	sleep_replication_request_t queued;
+	dat_request_t request;
+	uint8_t payload[64];
+	size_t payload_len;
+	int already_requested;
+	if (!session || !peer_id || !valid_feed(feed)) { errno = EINVAL; return -1; }
+	if (index >= SLEEP_BITFIELD_DATA_BYTES * 8u) { errno = ERANGE; return -1; }
+	peer = dat_session_find_peer(session, peer_id);
+	if (!peer || peer->state != DAT_SESSION_PEER_READY ||
+	    !(peer->remote_feed_mask & (uint8_t)(1u << feed))) { errno = ENOTCONN; return -1; }
+	if (!peer->replication[feed].uploading) { errno = EACCES; return -1; }
+	already_requested = sleep_replication_is_requested(&peer->replication[feed], index);
+	if (already_requested < 0) return -1;
+	if (already_requested) return 0;
+	queued.index = index; queued.bytes = bytes; queued.hash = 1; queued.nodes = 1;
+	if (sleep_replication_request(&peer->replication[feed], &queued)) return -1;
+	request.index = index; request.bytes = bytes; request.want_hash = 1; request.want_nodes = 1;
+	if (dat_request_encode(&request, payload, sizeof(payload), &payload_len) ||
+	    send_frame(session, peer, feed_channels[feed], DAT_MSG_REQUEST, payload, payload_len)) {
+		(void)sleep_replication_complete(&peer->replication[feed], index);
+		return -1;
+	}
+	return 0;
+}
+
+int dat_session_send_block(dat_session_t *session,
+			   const uint8_t peer_id[DAT_PEER_ID_BYTES],
+			   enum dat_session_feed feed, uint64_t index)
+{
+	dat_session_peer_t *peer;
+	sleep_proof_t proof;
+	dat_data_t data;
+	uint8_t *block = NULL, *proof_bytes = NULL, *payload = NULL;
+	size_t block_len = 0, proof_len = 0, payload_len = 0;
+	int rc = -1;
+	if (!session || !peer_id || !valid_feed(feed)) { errno = EINVAL; return -1; }
+	peer = dat_session_find_peer(session, peer_id);
+	if (!peer || peer->state != DAT_SESSION_PEER_READY ||
+	    !(peer->remote_feed_mask & (uint8_t)(1u << feed))) { errno = ENOTCONN; return -1; }
+	if (!peer->replication[feed].downloading) { errno = EACCES; return -1; }
+	if (sleep_feed_read(session->feeds[feed], (size_t)index, &block, &block_len) ||
+	    sleep_proof_create(session->feeds[feed], index, &proof)) goto done;
+	proof_bytes = malloc(51 + (proof.node_count + proof.root_count) * 48 + 64);
+	if (!proof_bytes) goto done;
+	if (sleep_proof_encode(&proof, proof_bytes,
+			       51 + (proof.node_count + proof.root_count) * 48 + 64,
+			       &proof_len)) goto done;
+	data.index = index;
+	data.value = block;
+	data.value_len = block_len;
+	data.proof = proof_bytes;
+	data.proof_len = proof_len;
+	data.node_count = 0;
+	data.has_signature = proof.has_signature;
+	memcpy(data.signature, proof.signature, sizeof(data.signature));
+	{
+		size_t i;
+		for (i = 0; i < proof.node_count + proof.root_count; i++) {
+			const sleep_proof_node_t *node = i < proof.node_count ?
+				&proof.nodes[i] : &proof.roots[i - proof.node_count];
+			data.nodes[i].index = node->index;
+			data.nodes[i].length = node->length;
+			memcpy(data.nodes[i].hash, node->hash, 32);
+		}
+		data.node_count = proof.node_count + proof.root_count;
+	}
+	if (block_len > SIZE_MAX - proof_len - data.node_count * 64 - 256) {
+		errno = EOVERFLOW;
+		goto done;
+	}
+	payload = malloc(block_len + proof_len + data.node_count * 64 + 256);
+	if (!payload) goto done;
+	if (dat_data_encode(&data, payload,
+			    block_len + proof_len + data.node_count * 64 + 256,
+			    &payload_len)) goto done;
+	rc = send_frame(session, peer, feed_channels[feed], DAT_MSG_DATA, payload, payload_len);
+	if (!rc) peer->payload_bytes_sent += block_len;
+done:
+	free(block); free(proof_bytes); free(payload);
+	return rc;
+}
+
+int dat_session_peer_down(dat_session_t *session, const uint8_t peer_id[DAT_PEER_ID_BYTES])
+{
+	dat_session_peer_t *peer;
+	size_t i;
+	int active = 0;
+	if (!session || !peer_id) { errno = EINVAL; return -1; }
+	peer = dat_session_find_peer(session, peer_id);
+	if (!peer) { errno = ENOENT; return -1; }
+	peer->state = DAT_SESSION_PEER_CLOSED;
+	peer->transport_authenticated = 0;
+	free(peer->rx_buffer);
+	peer->rx_buffer = NULL;
+	peer->rx_buffer_len = peer->rx_buffer_capacity = 0;
+	for (i = 0; i < DAT_SESSION_FEED_COUNT; i++)
+		sleep_replication_clear_requests(&peer->replication[i]);
+	for (i = 0; i < session->peer_count; i++)
+		if (session->peers[i].state == DAT_SESSION_PEER_READY) active++;
+	if (!active) session->state = DAT_SESSION_DISCOVERING;
+	return 0;
+}
+
+int dat_session_peer_pause(dat_session_t *session,
+			   const uint8_t peer_id[DAT_PEER_ID_BYTES], int paused)
+{
+	dat_session_peer_t *peer;
+	size_t feed;
+	if (!session || !peer_id) { errno = EINVAL; return -1; }
+	peer = dat_session_find_peer(session, peer_id);
+	if (!peer || peer->state != DAT_SESSION_PEER_READY) { errno = ENOTCONN; return -1; }
+	peer->paused = !!paused;
+	if (peer->paused) {
+		for (feed = 0; feed < DAT_SESSION_FEED_COUNT; feed++)
+			sleep_replication_clear_requests(&peer->replication[feed]);
+		return 0;
+	}
+	for (feed = 0; feed < DAT_SESSION_FEED_COUNT; feed++)
+		if (pump_replication(session, peer, (enum dat_session_feed)feed)) return -1;
+	return 0;
+}
+
+int dat_session_tick(dat_session_t *session, uint64_t now_ms)
+{
+	return dat_session_poll(session, 0, now_ms);
+}
+
+int dat_session_poll(dat_session_t *session, int timeout_ms, uint64_t now_ms)
+{
+	int received, wait_ms;
+	if (!session || (session->state != DAT_SESSION_DISCOVERING &&
+	    session->state != DAT_SESSION_ACTIVE) || timeout_ms < 0) { errno = EINVAL; return -1; }
+	if (session->poll_transport &&
+	    session->poll_transport(session->poll_context, timeout_ms) < 0) return -1;
+	wait_ms = session->poll_transport ? 0 : timeout_ms;
+	received = cesspool_poll(session->cesspool, wait_ms);
+	if (received < 0) return -1;
+	while (received > 0) {
+		received = cesspool_poll(session->cesspool, 0);
+		if (received < 0) return -1;
+	}
+	if (cesspool_tick(session->cesspool, now_ms)) return -1;
+	if (discover_known_peers(session)) return -1;
+	return 0;
+}
+
+int dat_session_close(dat_session_t *session)
+{
+	size_t i;
+	int rc = 0;
+	if (!session || session->state == DAT_SESSION_CLOSED) { errno = EINVAL; return -1; }
+	session->state = DAT_SESSION_CLOSING;
+	for (i = 0; i < session->peer_count; i++) {
+		session->peers[i].state = DAT_SESSION_PEER_CLOSED;
+		session->peers[i].transport_authenticated = 0;
+		free(session->peers[i].rx_buffer);
+		session->peers[i].rx_buffer = NULL;
+		session->peers[i].rx_buffer_len = session->peers[i].rx_buffer_capacity = 0;
+	}
+	if (session->announcing || session->looking_up) {
+		if (cesspool_leave(session->cesspool, session->discovery_topic)) rc = -1;
+	}
+	session->state = DAT_SESSION_CLOSED;
+	return rc;
+}

extra/dat-dpi/src/dat_walk.c

@@ -0,0 +1,207 @@
+#define _POSIX_C_SOURCE 200809L
+#include "dat.h"
+#include "win_dirent.h"
+#include <errno.h>
+#include "win_fnmatch.h"
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+#include <time.h>
+#include <unistd.h>
+
+static int join_path(char*out,size_t cap,const char*a,const char*b){int n=snprintf(out,cap,"%s%s%s",a,*a&&a[strlen(a)-1]!='/'?"/":"",b);if(n<0||(size_t)n>=cap){errno=ENAMETOOLONG;return-1;}return 0;}
+static int matches_archive_path(const char *path, const char *root);
+typedef struct sync_context { char patterns[128][256]; size_t pattern_count; } sync_context_t;
+static uint64_t stat_mtime_ms(const struct stat *st){
+#if defined(_WIN32)
+	if(st->st_mtime<0)return 0;
+	return (uint64_t)st->st_mtime*1000u;
+#elif defined(__APPLE__)
+	if(st->st_mtimespec.tv_sec<0)return 0;
+	return (uint64_t)st->st_mtimespec.tv_sec*1000u+(uint64_t)st->st_mtimespec.tv_nsec/1000000u;
+#else
+	if(st->st_mtim.tv_sec<0)return 0;
+	return (uint64_t)st->st_mtim.tv_sec*1000u+(uint64_t)st->st_mtim.tv_nsec/1000000u;
+#endif
+}
+static uint64_t now_ms(void){struct timespec ts;if(clock_gettime(CLOCK_REALTIME,&ts))return (uint64_t)time(NULL)*1000u;return (uint64_t)ts.tv_sec*1000u+(uint64_t)ts.tv_nsec/1000000u;}
+static int import_file(dat_archive_t*a,const char*disk,const char*name,const struct stat*st){FILE*f;uint8_t*data;size_t len=(size_t)st->st_size;if(st->st_size<0||(uint64_t)st->st_size>SIZE_MAX){errno=EOVERFLOW;return-1;}data=malloc(len?len:1);if(!data)return-1;f=fopen(disk,"rb");if(!f){free(data);return-1;}if(len&&fread(data,1,len,f)!=len){fclose(f);free(data);errno=EIO;return-1;}fclose(f);if(dat_archive_append_file(a,name,data,len,(uint32_t)st->st_mode,stat_mtime_ms(st))){free(data);return-1;}free(data);return 0;}
+static int import_one(dat_archive_t*a,const char*disk,const char*name){struct stat st;if(lstat(disk,&st))return-1;if(S_ISREG(st.st_mode))return import_file(a,disk,name,&st);if(S_ISLNK(st.st_mode)){char target[1024];ssize_t n=readlink(disk,target,sizeof(target)-1);if(n<0)return-1;target[n]=0;return dat_archive_append_symlink(a,name,target,stat_mtime_ms(&st));}if(S_ISDIR(st.st_mode)){DIR*dir;struct dirent*item;int rc=0;if(*name&&dat_archive_append_directory(a,name,(uint32_t)st.st_mode,stat_mtime_ms(&st)))return-1;dir=opendir(disk);if(!dir)return-1;while((item=readdir(dir))){char child_disk[2048],child_name[2048];if(!strcmp(item->d_name,".")||!strcmp(item->d_name,"..")||!strcmp(item->d_name,".dat"))continue;if(join_path(child_disk,sizeof(child_disk),disk,item->d_name)||join_path(child_name,sizeof(child_name),name,item->d_name)||import_one(a,child_disk,child_name)){rc=-1;break;}}closedir(dir);return rc;}return 0;}
+int dat_archive_import_path(dat_archive_t*a,const char*filesystem_path,const char*archive_path){if(!a||!filesystem_path||!archive_path){errno=EINVAL;return-1;}while(*archive_path=='/')archive_path++;return import_one(a,filesystem_path,archive_path);}
+
+static int same_file(const dat_archive_t *a, const char *disk,
+			     const struct stat *st, const sleep_metadata_entry_t *entry)
+{
+	FILE *f;
+	uint8_t *old, buffer[8192];
+	size_t old_len, n, off = 0;
+	if (entry->type != SLEEP_ENTRY_FILE || entry->size != (uint64_t)st->st_size ||
+	    entry->mode != (uint32_t)st->st_mode || entry->mtime != stat_mtime_ms(st))
+		return 0;
+	if (dat_archive_read_file(a, entry, &old, &old_len)) return -1;
+	f = fopen(disk, "rb");
+	if (!f) { free(old); return -1; }
+	while ((n = fread(buffer, 1, sizeof(buffer), f)) != 0) {
+		if (n > old_len - off || memcmp(buffer, old + off, n)) {
+			fclose(f); free(old); return 0;
+		}
+		off += n;
+	}
+	if (ferror(f)) { fclose(f); free(old); errno = EIO; return -1; }
+	fclose(f);
+	free(old);
+	return off == old_len;
+}
+
+static int ignored_path(const sync_context_t *context, const char *name, const char *basename)
+{
+	size_t i;
+	if (*name && basename[0] == '.') return 1;
+	for (i = 0; i < context->pattern_count; i++) {
+		const char *pattern = context->patterns[i];
+		size_t n = strlen(pattern);
+		if (pattern[0] == '/') pattern++;
+		if (n && context->patterns[i][n - 1] == '/' && !strncmp(name, pattern, strlen(pattern))) return 1;
+		if (!fnmatch(pattern, name, 0)) return 1;
+	}
+	return 0;
+}
+
+static int sync_one(dat_archive_t *a, const char *disk, const char *name, const sync_context_t *context)
+{
+	struct stat st;
+	sleep_metadata_entry_t previous;
+	int found;
+	if (lstat(disk, &st)) return -1;
+	found = dat_archive_find(a, name, &previous) == 0;
+	if (!found && errno != ENOENT) return -1;
+	if (S_ISREG(st.st_mode)) {
+		int equal = found ? same_file(a, disk, &st, &previous) : 0;
+		if (equal < 0) return -1;
+		return equal ? 0 : import_file(a, disk, name, &st);
+	}
+	if (S_ISLNK(st.st_mode)) {
+		char target[1024]; ssize_t n = readlink(disk, target, sizeof(target) - 1);
+		if (n < 0) return -1;
+		target[n] = 0;
+		if (found && previous.type == SLEEP_ENTRY_SYMLINK && previous.mtime == stat_mtime_ms(&st) && !strcmp(previous.linkname, target)) return 0;
+		return dat_archive_append_symlink(a, name, target, stat_mtime_ms(&st));
+	}
+	if (S_ISDIR(st.st_mode)) {
+		DIR *dir; struct dirent *item;
+		if (*name && (!found || previous.type != SLEEP_ENTRY_DIRECTORY || previous.mode != (uint32_t)st.st_mode || previous.mtime != stat_mtime_ms(&st)) && dat_archive_append_directory(a, name, (uint32_t)st.st_mode, stat_mtime_ms(&st))) return -1;
+		dir = opendir(disk);
+		if (!dir) return -1;
+		while ((item = readdir(dir))) {
+			char child_disk[2048], child_name[2048];
+			if (!strcmp(item->d_name, ".") || !strcmp(item->d_name, "..")) continue;
+			if (join_path(child_disk, sizeof(child_disk), disk, item->d_name) || join_path(child_name, sizeof(child_name), name, item->d_name)) { closedir(dir); return -1; }
+			if (ignored_path(context, child_name, item->d_name)) continue;
+			if (sync_one(a, child_disk, child_name, context)) { closedir(dir); return -1; }
+		}
+		closedir(dir);
+	}
+	return 0;
+}
+
+static int load_ignore_file(const char *root, sync_context_t *context)
+{
+	char path[1200], line[256];
+	FILE *f;
+	if (join_path(path, sizeof(path), root, ".datignore")) return -1;
+	f = fopen(path, "r");
+	if (!f) return errno == ENOENT ? 0 : -1;
+	while (fgets(line, sizeof(line), f)) {
+		size_t n = strcspn(line, "\r\n");
+		line[n] = 0;
+		if (!line[0] || line[0] == '#') continue;
+		if (context->pattern_count >= sizeof(context->patterns) / sizeof(context->patterns[0])) { fclose(f); errno = EOVERFLOW; return -1; }
+		strcpy(context->patterns[context->pattern_count++], line);
+	}
+	if (ferror(f)) { fclose(f); errno = EIO; return -1; }
+	return fclose(f);
+}
+
+static int path_missing(const char *filesystem_path, const char *archive_path, const char *entry_path)
+{
+	char disk[4096]; const char *relative = entry_path; size_t prefix; struct stat st;
+	while (*relative == '/') relative++;
+	prefix = strlen(archive_path);
+	if (prefix) { if (!strcmp(relative, archive_path)) relative = ""; else relative += prefix + 1; }
+	if (join_path(disk, sizeof(disk), filesystem_path, relative)) return -1;
+	if (!lstat(disk, &st)) return 0;
+	return errno == ENOENT ? 1 : -1;
+}
+
+int dat_archive_sync_path(dat_archive_t *a, const char *filesystem_path, const char *archive_path)
+{
+	size_t count, i, j;
+	sync_context_t context;
+	if (!a || !filesystem_path || !archive_path) { errno = EINVAL; return -1; }
+	while (*archive_path == '/') archive_path++;
+	memset(&context, 0, sizeof(context));
+	if (load_ignore_file(filesystem_path, &context) || sync_one(a, filesystem_path, archive_path, &context)) return -1;
+	count = dat_archive_entry_count(a);
+	for (i = 0; i < count; i++) {
+		sleep_metadata_entry_t entry, later; int newest = 1, missing;
+		if (dat_archive_read_entry(a, i, &entry)) return -1;
+		if (!matches_archive_path(entry.path, archive_path)) continue;
+		for (j = i + 1; j < count; j++) { if (dat_archive_read_entry(a, j, &later)) return -1; if (!strcmp(entry.path, later.path)) { newest = 0; break; } }
+		if (!newest || entry.type == SLEEP_ENTRY_DELETE) continue;
+		missing = path_missing(filesystem_path, archive_path, entry.path);
+		if (missing < 0) return -1;
+		if (missing && dat_archive_delete(a, entry.path, now_ms())) return -1;
+	}
+	return 0;
+}
+
+static int export_entry(const dat_archive_t*a,const sleep_metadata_entry_t*e,const char*path){if(e->type==SLEEP_ENTRY_DIRECTORY){if(mkdir(path,(mode_t)e->mode)&&errno!=EEXIST)return-1;return 0;}if(e->type==SLEEP_ENTRY_SYMLINK)return symlink(e->linkname,path);if(e->type==SLEEP_ENTRY_FILE){uint8_t*data;size_t len;FILE*f;if(dat_archive_read_file(a,e,&data,&len))return-1;f=fopen(path,"wb");if(!f){free(data);return-1;}if(len&&fwrite(data,1,len,f)!=len){fclose(f);free(data);errno=EIO;return-1;}free(data);return fclose(f);}return 0;}
+static int matches_archive_path(const char *path, const char *root)
+{
+	size_t n;
+	while (*path == '/') path++;
+	while (*root == '/') root++;
+	n = strlen(root);
+	return !n || !strcmp(path, root) || (!strncmp(path, root, n) && path[n] == '/');
+}
+
+int dat_archive_export_at(const dat_archive_t*a,uint64_t version,const char*archive_path,const char*filesystem_path)
+{
+	sleep_metadata_entry_t e, later;
+	size_t count,i,j,prefix;
+	if(!a||!archive_path||!filesystem_path){errno=EINVAL;return-1;}
+	while(*archive_path=='/')archive_path++;
+	count=dat_archive_entry_count(a);
+	if(version>count){errno=ERANGE;return-1;}
+	prefix=strlen(archive_path);
+	for(i=0;i<(size_t)version;i++){
+		char suffix[2048],disk[4096];
+		int current=1;
+		if(dat_archive_read_entry(a,i,&e))return-1;
+		if(!matches_archive_path(e.path,archive_path))continue;
+		for(j=i+1;j<(size_t)version;j++){
+			if(dat_archive_read_entry(a,j,&later))return-1;
+			if(!strcmp(later.path,e.path)){current=0;break;}
+		}
+		if(!current||e.type==SLEEP_ENTRY_DELETE)continue;
+		{
+			const char *entry_path=e.path;
+			while(*entry_path=='/')entry_path++;
+			if(!prefix)sprintf(suffix,"%s",entry_path);
+			else if(!strcmp(entry_path,archive_path))suffix[0]=0;
+			else strcpy(suffix,entry_path+prefix+1);
+		}
+		if(*suffix){if(join_path(disk,sizeof(disk),filesystem_path,suffix))return-1;}
+		else{if(strlen(filesystem_path)>=sizeof(disk)){errno=ENAMETOOLONG;return-1;}strcpy(disk,filesystem_path);}
+		if(export_entry(a,&e,disk))return-1;
+	}
+	return 0;
+}
+
+int dat_archive_export_path(const dat_archive_t*a,const char*archive_path,const char*filesystem_path)
+{
+	if(!a){errno=EINVAL;return-1;}
+	return dat_archive_export_at(a,dat_archive_version(a),archive_path,filesystem_path);
+}

extra/dat-dpi/src/dat_wire.c

@@ -0,0 +1,421 @@
+#include "dat_wire.h"
+#include "sleep_proto.h"
+#include <errno.h>
+#include <string.h>
+
+static int put_varint(uint64_t value, uint8_t *out, size_t cap, size_t *off)
+{
+	size_t n;
+	if (*off > cap || sleep_varint_encode(value, out + *off, cap - *off, &n)) return -1;
+	*off += n;
+	return 0;
+}
+
+static int put_bytes(uint32_t field, const uint8_t *data, size_t len,
+		     uint8_t *out, size_t cap, size_t *off)
+{
+	if ((!data && len) || put_varint(((uint64_t)field << 3) | 2, out, cap, off) ||
+	    put_varint(len, out, cap, off)) return -1;
+	if (len > cap - *off) { errno = ENOSPC; return -1; }
+	if (len) memcpy(out + *off, data, len);
+	*off += len;
+	return 0;
+}
+
+static int put_uint(uint32_t field, uint64_t value,
+		    uint8_t *out, size_t cap, size_t *off)
+{
+	return put_varint((uint64_t)field << 3, out, cap, off) ||
+		put_varint(value, out, cap, off) ? -1 : 0;
+}
+
+/* Returns one field. Unknown protobuf wire types are skipped by read_field. */
+typedef struct proto_field {
+	uint32_t number;
+	uint8_t wire;
+	uint64_t value;
+	const uint8_t *bytes;
+	size_t length;
+} proto_field_t;
+
+static int read_field(const uint8_t *in, size_t len, size_t *off,
+		      proto_field_t *field)
+{
+	uint64_t tag, value;
+	size_t n;
+	if (*off >= len) return 0;
+	if (sleep_varint_decode(in + *off, len - *off, &tag, &n)) return -1;
+	*off += n;
+	field->number = (uint32_t)(tag >> 3);
+	field->wire = (uint8_t)(tag & 7);
+	field->bytes = NULL;
+	field->length = 0;
+	field->value = 0;
+	if (!field->number) { errno = EPROTO; return -1; }
+	if (field->wire == 0) {
+		if (sleep_varint_decode(in + *off, len - *off, &value, &n)) return -1;
+		*off += n;
+		field->value = value;
+	} else if (field->wire == 2) {
+		if (sleep_varint_decode(in + *off, len - *off, &value, &n)) return -1;
+		*off += n;
+		if (value > len - *off) { errno = EPROTO; return -1; }
+		field->bytes = in + *off;
+		field->length = (size_t)value;
+		*off += (size_t)value;
+	} else if (field->wire == 1 || field->wire == 5) {
+		size_t skip = field->wire == 1 ? 8 : 4;
+		if (skip > len - *off) { errno = EPROTO; return -1; }
+		*off += skip;
+	} else { errno = EPROTO; return -1; }
+	return 1;
+}
+
+int dat_wire_encode(const dat_wire_message_t *m, uint8_t *out, size_t cap,
+		    size_t *written)
+{
+	uint8_t header[10];
+	size_t hlen, prefix, total;
+	uint64_t packed;
+	if (!m || !out || (!m->payload && m->payload_len) || m->type > 15 ||
+	    m->channel > (UINT64_MAX >> 4)) { errno = EINVAL; return -1; }
+	packed = (m->channel << 4) | m->type;
+	if (sleep_varint_encode(packed, header, sizeof(header), &hlen)) return -1;
+	if (m->payload_len > SIZE_MAX - hlen) { errno = EOVERFLOW; return -1; }
+	total = hlen + m->payload_len;
+	if (sleep_varint_encode(total, out, cap, &prefix)) return -1;
+	if (total > cap - prefix) { errno = ENOSPC; return -1; }
+	memcpy(out + prefix, header, hlen);
+	if (m->payload_len) memcpy(out + prefix + hlen, m->payload, m->payload_len);
+	if (written) *written = prefix + total;
+	return 0;
+}
+
+int dat_wire_decode(dat_wire_message_t *m, const uint8_t *in, size_t len,
+		    size_t *consumed)
+{
+	uint64_t frame, packed;
+	size_t prefix, hlen;
+	if (!m || !in) { errno = EINVAL; return -1; }
+	if (sleep_varint_decode(in, len, &frame, &prefix)) return -1;
+	if (frame > len - prefix || frame > SIZE_MAX) { errno = EAGAIN; return -1; }
+	if (!frame || sleep_varint_decode(in + prefix, (size_t)frame, &packed, &hlen)) return -1;
+	m->channel = packed >> 4;
+	m->type = packed & 15;
+	m->payload = in + prefix + hlen;
+	m->payload_len = (size_t)frame - hlen;
+	if (consumed) *consumed = prefix + (size_t)frame;
+	return 0;
+}
+
+int dat_handshake_init(dat_handshake_t *h, const uint8_t peer_id[32],
+		       uint64_t capabilities)
+{
+	size_t n;
+	if (!h || !peer_id) { errno = EINVAL; return -1; }
+	n = strlen(DAT_NETWORK_PROTOCOL);
+	if (!n || n > DAT_PROTOCOL_BYTES) { errno = EINVAL; return -1; }
+	memset(h, 0, sizeof(*h));
+	memcpy(h->protocol, DAT_NETWORK_PROTOCOL, n);
+	memcpy(h->peer_id, peer_id, 32);
+	h->capabilities = capabilities;
+	h->has_peer_id = 1;
+	return 0;
+}
+
+int dat_handshake_protocol_matches(const dat_handshake_t *h)
+{
+	size_t n, i;
+	if (!h) { errno = EINVAL; return -1; }
+	n = strlen(DAT_NETWORK_PROTOCOL);
+	if (!n || n > DAT_PROTOCOL_BYTES) { errno = EINVAL; return -1; }
+	if (memcmp(h->protocol, DAT_NETWORK_PROTOCOL, n)) { errno = EPROTONOSUPPORT; return 0; }
+	for (i = n; i < DAT_PROTOCOL_BYTES; i++) if (h->protocol[i]) {
+		errno = EPROTONOSUPPORT;
+		return 0;
+	}
+	return 1;
+}
+
+int dat_handshake_encode(const dat_handshake_t *h, uint8_t *out, size_t cap,
+			 size_t *written)
+{
+	size_t off = 0;
+#ifndef DAT_LEGACY_HYPERCORE
+	size_t protocol_len = strlen(DAT_NETWORK_PROTOCOL);
+#endif
+	if (!h || !out || dat_handshake_protocol_matches(h) != 1) { errno = EINVAL; return -1; }
+	if ((h->has_peer_id && put_bytes(1, h->peer_id, 32, out, cap, &off)) ||
+	    (h->capabilities && put_uint(2, h->capabilities, out, cap, &off))) return -1;
+#ifndef DAT_LEGACY_HYPERCORE
+	if (put_bytes(3, (const uint8_t *)DAT_NETWORK_PROTOCOL, protocol_len,
+		      out, cap, &off)) return -1;
+#endif
+	if (written) *written = off;
+	return 0;
+}
+
+int dat_handshake_decode(dat_handshake_t *h, const uint8_t *in, size_t len)
+{
+	size_t off = 0;
+	proto_field_t f;
+	int rc;
+	if (!h || !in) { errno = EINVAL; return -1; }
+	memset(h, 0, sizeof(*h));
+	while ((rc = read_field(in, len, &off, &f)) > 0) {
+		if (f.number == 1 && f.wire == 2) {
+			if (f.length == sizeof(h->peer_id)) {
+				memcpy(h->peer_id, f.bytes, f.length);
+				h->has_peer_id = 1;
+			}
+		} else if (f.number == 2 && f.wire == 0) h->capabilities = f.value;
+#ifndef DAT_LEGACY_HYPERCORE
+		else if (f.number == 3 && f.wire == 2) {
+			if (f.length > sizeof(h->protocol)) { errno = EPROTO; return -1; }
+			memcpy(h->protocol, f.bytes, f.length);
+		}
+#endif
+	}
+	if (rc < 0) { errno = EPROTO; return -1; }
+	/* Hypercore v6 userData is application data, not a protocol identity. */
+	memcpy(h->protocol, DAT_NETWORK_PROTOCOL, strlen(DAT_NETWORK_PROTOCOL));
+	return 0;
+}
+
+int dat_feed_open_encode(const dat_feed_open_t *feed, uint8_t *out,
+			 size_t cap, size_t *written)
+{
+	size_t off = 0;
+	if (!feed || !out) { errno = EINVAL; return -1; }
+	if (put_bytes(1, feed->discovery_key, 32, out, cap, &off) ||
+	    (feed->has_nonce && put_bytes(2, feed->nonce, sizeof(feed->nonce), out, cap, &off))) return -1;
+	if (written) *written = off;
+	return 0;
+}
+
+int dat_feed_open_decode(dat_feed_open_t *feed, const uint8_t *in, size_t len)
+{
+	size_t off = 0;
+	proto_field_t f;
+	int rc, found = 0;
+	if (!feed || !in) { errno = EINVAL; return -1; }
+	memset(feed, 0, sizeof(*feed));
+	while ((rc = read_field(in, len, &off, &f)) > 0) {
+		if ((f.number == 1 || f.number == 2) && f.wire == 2) {
+			if ((f.number == 1 && f.length != 32) ||
+			    (f.number == 2 && f.length != sizeof(feed->nonce))) { errno = EPROTO; return -1; }
+			if (f.number == 1) { memcpy(feed->discovery_key, f.bytes, 32); found = 1; }
+			else { memcpy(feed->nonce, f.bytes, sizeof(feed->nonce)); feed->has_nonce = 1; }
+		}
+	}
+	if (rc < 0 || !found) { errno = EPROTO; return -1; }
+	return 0;
+}
+
+int dat_range_encode(const dat_range_t *r, uint8_t *out, size_t cap, size_t *written)
+{
+	size_t off = 0;
+	if (!r || !out) { errno = EINVAL; return -1; }
+	if (put_uint(1, r->start, out, cap, &off) || put_uint(2, r->length, out, cap, &off)) return -1;
+	if (written) *written = off;
+	return 0;
+}
+
+int dat_range_decode(dat_range_t *r, const uint8_t *in, size_t len)
+{
+	size_t off = 0;
+	proto_field_t f;
+	int rc, start = 0;
+	if (!r || !in) { errno = EINVAL; return -1; }
+	r->start = 0; r->length = 1;
+	while ((rc = read_field(in, len, &off, &f)) > 0) if (f.wire == 0) {
+		if (f.number == 1) { r->start = f.value; start = 1; }
+		if (f.number == 2) r->length = f.value;
+	}
+	if (rc < 0 || !start) { errno = EPROTO; return -1; }
+	return 0;
+}
+
+int dat_want_decode(dat_range_t *r, const uint8_t *in, size_t len,
+		    uint64_t feed_length)
+{
+	size_t off = 0;
+	proto_field_t f;
+	int rc, have_start = 0, have_length = 0;
+	if (!r || !in) { errno = EINVAL; return -1; }
+	r->start = 0;
+	r->length = 0;
+	while ((rc = read_field(in, len, &off, &f)) > 0) if (f.wire == 0) {
+		if (f.number == 1) { r->start = f.value; have_start = 1; }
+		else if (f.number == 2) { r->length = f.value; have_length = 1; }
+	}
+	if (rc < 0 || !have_start) { errno = EPROTO; return -1; }
+	if (!have_length && r->start < feed_length) r->length = feed_length - r->start;
+	return 0;
+}
+
+int dat_info_encode(const dat_info_t *info, uint8_t *out, size_t cap,
+		    size_t *written)
+{
+	size_t off = 0;
+	if (!info || !out) { errno = EINVAL; return -1; }
+	if ((!info->uploading && put_uint(1, 0, out, cap, &off)) ||
+	    (!info->downloading && put_uint(2, 0, out, cap, &off))) return -1;
+	if (written) *written = off;
+	return 0;
+}
+
+int dat_info_decode(dat_info_t *info, const uint8_t *in, size_t len)
+{
+	size_t off = 0;
+	proto_field_t f;
+	int rc;
+	if (!info || (!in && len)) { errno = EINVAL; return -1; }
+	info->uploading = 1;
+	info->downloading = 1;
+	while ((rc = read_field(in, len, &off, &f)) > 0) if (f.wire == 0) {
+		if (f.number == 1) info->uploading = !!f.value;
+		else if (f.number == 2) info->downloading = !!f.value;
+	}
+	if (rc < 0) { errno = EPROTO; return -1; }
+	return 0;
+}
+
+int dat_request_encode(const dat_request_t *r, uint8_t *out, size_t cap, size_t *written)
+{
+	size_t off = 0;
+	if (!r || !out) { errno = EINVAL; return -1; }
+	if (put_uint(1, r->index, out, cap, &off) ||
+	    (r->bytes && put_uint(2, r->bytes, out, cap, &off)) ||
+	    (r->want_hash && put_uint(3, 1, out, cap, &off)) ||
+	    (r->want_nodes && put_uint(4, 64, out, cap, &off))) return -1;
+	if (written) *written = off;
+	return 0;
+}
+
+int dat_request_decode(dat_request_t *r, const uint8_t *in, size_t len)
+{
+	size_t off = 0;
+	proto_field_t f;
+	int rc, found = 0;
+	if (!r || !in) { errno = EINVAL; return -1; }
+	memset(r, 0, sizeof(*r));
+	while ((rc = read_field(in, len, &off, &f)) > 0) if (f.wire == 0) {
+		if (f.number == 1) { r->index = f.value; found = 1; }
+		else if (f.number == 2) r->bytes = f.value;
+		else if (f.number == 3) r->want_hash = !!f.value;
+		else if (f.number == 4) r->want_nodes = !!f.value;
+	}
+	if (rc < 0 || !found) { errno = EPROTO; return -1; }
+	return 0;
+}
+
+static int encode_node(const dat_data_node_t *node, uint8_t *out,
+		       size_t cap, size_t *written)
+{
+	size_t off = 0;
+	if (put_uint(1, node->index, out, cap, &off) ||
+	    put_bytes(2, node->hash, 32, out, cap, &off) ||
+	    put_uint(3, node->length, out, cap, &off)) return -1;
+	*written = off;
+	return 0;
+}
+
+static int decode_node(dat_data_node_t *node, const uint8_t *in, size_t len)
+{
+	size_t off = 0;
+	proto_field_t f;
+	int rc, have_index = 0, have_hash = 0, have_length = 0;
+	while ((rc = read_field(in, len, &off, &f)) > 0) {
+		if (f.number == 1 && f.wire == 0) { node->index = f.value; have_index = 1; }
+		else if (f.number == 2 && f.wire == 2 && f.length == 32) { memcpy(node->hash, f.bytes, 32); have_hash = 1; }
+		else if (f.number == 3 && f.wire == 0) { node->length = f.value; have_length = 1; }
+	}
+	if (rc < 0 || !have_index || !have_hash || !have_length) { errno = EPROTO; return -1; }
+	return 0;
+}
+
+int dat_data_encode(const dat_data_t *d, uint8_t *out, size_t cap, size_t *written)
+{
+	size_t off = 0, i, node_len;
+	uint8_t node[64];
+	if (!d || !out || (!d->value && d->value_len) || (!d->proof && d->proof_len) ||
+	    d->node_count > DAT_DATA_MAX_NODES) { errno = EINVAL; return -1; }
+	if (put_uint(1, d->index, out, cap, &off) ||
+	    put_bytes(2, d->value, d->value_len, out, cap, &off)) return -1;
+	for (i = 0; i < d->node_count; i++) {
+		if (encode_node(&d->nodes[i], node, sizeof(node), &node_len) ||
+		    put_bytes(3, node, node_len, out, cap, &off)) return -1;
+	}
+	if (d->has_signature && put_bytes(4, d->signature, sizeof(d->signature), out, cap, &off)) return -1;
+	/* Field 5 carries C's complete multi-root proof; older protobuf readers ignore it. */
+	if (d->proof_len && put_bytes(5, d->proof, d->proof_len, out, cap, &off)) return -1;
+	if (written) *written = off;
+	return 0;
+}
+
+int dat_data_decode(dat_data_t *d, const uint8_t *in, size_t len)
+{
+	size_t off = 0;
+	proto_field_t f;
+	int rc, have_index = 0;
+	if (!d || !in) { errno = EINVAL; return -1; }
+	memset(d, 0, sizeof(*d));
+	while ((rc = read_field(in, len, &off, &f)) > 0) {
+		if (f.number == 1 && f.wire == 0) { d->index = f.value; have_index = 1; }
+		else if (f.number == 2 && f.wire == 2) { d->value = f.bytes; d->value_len = f.length; }
+		else if (f.number == 3 && f.wire == 2) {
+			if (d->node_count >= DAT_DATA_MAX_NODES ||
+			    decode_node(&d->nodes[d->node_count], f.bytes, f.length)) return -1;
+			d->node_count++;
+		} else if (f.number == 4 && f.wire == 2) {
+			if (f.length != 64) { errno = EPROTO; return -1; }
+			memcpy(d->signature, f.bytes, 64); d->has_signature = 1;
+		} else if (f.number == 5 && f.wire == 2) { d->proof = f.bytes; d->proof_len = f.length; }
+	}
+	if (rc < 0 || !have_index) { errno = EPROTO; return -1; }
+	return 0;
+}
+
+int dat_cancel_encode(const dat_cancel_t *c, uint8_t *out, size_t cap, size_t *written)
+{
+	size_t off = 0;
+	if (!c || !out) { errno = EINVAL; return -1; }
+	if (put_uint(1, c->index, out, cap, &off)) return -1;
+	if (written) *written = off;
+	return 0;
+}
+
+int dat_cancel_decode(dat_cancel_t *c, const uint8_t *in, size_t len)
+{
+	dat_request_t r;
+	if (!c || dat_request_decode(&r, in, len)) return -1;
+	c->index = r.index;
+	return 0;
+}
+
+int dat_extension_encode(const dat_extension_t *e, uint8_t *out, size_t cap,
+			 size_t *written)
+{
+	size_t off = 0;
+	if (!e || !out || (!e->message && e->message_len)) { errno = EINVAL; return -1; }
+	if (put_varint(e->id, out, cap, &off)) return -1;
+	if (e->message_len > cap - off) { errno = ENOSPC; return -1; }
+	if (e->message_len) memcpy(out + off, e->message, e->message_len);
+	off += e->message_len;
+	if (written) *written = off;
+	return 0;
+}
+
+int dat_extension_decode(dat_extension_t *e, const uint8_t *in, size_t len)
+{
+	uint64_t id;
+	size_t n, off = 0;
+	if (!e || !in || sleep_varint_decode(in, len, &id, &n)) return -1;
+	off = n;
+	memset(e, 0, sizeof(*e));
+	e->id = id;
+	e->message = in + off;
+	e->message_len = len - off;
+	return 0;
+}

extra/dat-dpi/src/sleep.c

@@ -0,0 +1,24 @@
+#include "sleep.h"
+#include <errno.h>
+#include <stdio.h>
+#include <string.h>
+#include <sys/stat.h>
+#ifdef _WIN32
+#include <direct.h>
+#define DAT_MKDIR(path) _mkdir(path)
+#else
+#define DAT_MKDIR(path) mkdir(path, 0700)
+#endif
+
+static int make_directory(const char *path) {
+	if (DAT_MKDIR(path) == 0 || errno == EEXIST) return 0;
+	return -1;
+}
+
+int sleep_init(const char *root) {
+	char path[1024]; int n;
+	if (!root || !*root) { errno = EINVAL; return -1; }
+	n = snprintf(path, sizeof(path), "%s/.dat", root);
+	if (n < 0 || (size_t)n >= sizeof(path)) { errno = ENAMETOOLONG; return -1; }
+	return make_directory(path);
+}

extra/dat-dpi/src/sleep_bitfield.c

@@ -0,0 +1,34 @@
+#include "sleep_bitfield.h"
+#include <errno.h>
+#include <string.h>
+
+static int bit_get(const uint8_t *bits, size_t bytes, size_t bit) {
+	if (bit >= bytes * 8) { errno = ERANGE; return -1; }
+	return (bits[bit >> 3] >> (7 - (bit & 7))) & 1;
+}
+static int bit_set(uint8_t *bits, size_t bytes, size_t bit, int value) {
+	uint8_t mask; if (bit >= bytes * 8) { errno = ERANGE; return -1; }
+	mask = (uint8_t)(1u << (7 - (bit & 7))); if (value) bits[bit >> 3] |= mask; else bits[bit >> 3] &= (uint8_t)~mask; return 0;
+}
+static unsigned byte_state(uint8_t v) { return v == 0 ? 0u : (v == 0xff ? 3u : 2u); }
+void sleep_bitfield_clear(sleep_bitfield_t *f) { if (f) memset(f, 0, sizeof(*f)); }
+int sleep_bitfield_get_data(const sleep_bitfield_t *f, size_t bit) { if (!f) { errno = EINVAL; return -1; } return bit_get(f->data, sizeof(f->data), bit); }
+int sleep_bitfield_get_tree(const sleep_bitfield_t *f, size_t bit) { if (!f) { errno = EINVAL; return -1; } return bit_get(f->tree, sizeof(f->tree), bit); }
+void sleep_bitfield_reindex(sleep_bitfield_t *f) {
+	size_t pair; if (!f) return; memset(f->index, 0, sizeof(f->index));
+	for (pair = 0; pair < 512; pair++) { unsigned tuple = byte_state(f->data[pair*2]) | (byte_state(f->data[pair*2+1]) << 2); size_t byte = pair >> 1;
+		f->index[byte] |= (uint8_t)(tuple << ((pair & 1) ? 4 : 0)); }
+}
+int sleep_bitfield_set_data(sleep_bitfield_t *f, size_t bit, int present) { if (!f) { errno = EINVAL; return -1; } if (bit_set(f->data, sizeof(f->data), bit, present)) return -1; sleep_bitfield_reindex(f); return 0; }
+int sleep_bitfield_set_tree(sleep_bitfield_t *f, size_t bit, int present) { if (!f) { errno = EINVAL; return -1; } return bit_set(f->tree, sizeof(f->tree), bit, present); }
+size_t sleep_bitfield_next_data(const sleep_bitfield_t *f, size_t start, int present) {
+	size_t bit; if (!f) return SIZE_MAX; for (bit = start; bit < sizeof(f->data)*8; bit++) if (bit_get(f->data, sizeof(f->data), bit) == !!present) return bit; return SIZE_MAX;
+}
+int sleep_bitfield_encode(const sleep_bitfield_t *f, uint8_t *out, size_t cap) {
+	if (!f || !out) { errno = EINVAL; return -1; } if (cap < sizeof(*f)) { errno = ENOSPC; return -1; }
+	memcpy(out, f->data, 1024); memcpy(out+1024, f->tree, 2048); memcpy(out+3072, f->index, 256); memcpy(out+3328, f->reserved, 256); return 0;
+}
+int sleep_bitfield_decode(sleep_bitfield_t *f, const uint8_t *in, size_t len) {
+	if (!f || !in) { errno = EINVAL; return -1; } if (len < sizeof(*f)) { errno = EPROTO; return -1; }
+	memcpy(f->data, in, 1024); memcpy(f->tree, in+1024, 2048); memcpy(f->index, in+3072, 256); memcpy(f->reserved, in+3328, 256); return 0;
+}

extra/dat-dpi/src/sleep_crypto.c

@@ -0,0 +1,116 @@
+#include "sleep_crypto.h"
+#include <errno.h>
+#include <stdlib.h>
+#include <sodium.h>
+#include <stdio.h>
+#include <string.h>
+
+static void put64be(uint8_t *p, uint64_t v) {
+	unsigned int i;
+	for (i = 0; i < 8; i++) p[i] = (uint8_t)(v >> (56 - i * 8));
+}
+
+static int encode_roots(uint8_t **out, size_t *out_len, const uint8_t *roots,
+		 size_t nroots, const uint64_t *indices, const uint64_t *lengths) {
+	size_t i, len;
+	uint8_t *p;
+	if (!out || !out_len || (nroots && (!roots || !indices || !lengths))) { errno = EINVAL; return -1; }
+	if (nroots > (SIZE_MAX - 1) / SLEEP_ROOT_RECORD_BYTES) { errno = EOVERFLOW; return -1; }
+	len = 1 + nroots * SLEEP_ROOT_RECORD_BYTES;
+	p = malloc(len);
+	if (!p) return -1;
+	p[0] = 2; /* legacy SLEEP root domain */
+	for (i = 0; i < nroots; i++) {
+		memcpy(p + 1 + i * 48, roots + i * 32, 32);
+		put64be(p + 33 + i * 48, indices[i]);
+		put64be(p + 41 + i * 48, lengths[i]);
+	}
+	*out = p; *out_len = len; return 0;
+}
+
+int sleep_crypto_init(void) { return sodium_init() < 0 ? -1 : 0; }
+
+int sleep_stream_xsalsa20_xor(uint8_t *out, const uint8_t *in, size_t length,
+			     const uint8_t nonce[24], const uint8_t key[32],
+			     uint64_t stream_offset)
+{
+	uint8_t *aligned;
+	size_t skip = (size_t)(stream_offset & 63u), total;
+	uint64_t counter = stream_offset >> 6;
+	if ((!out && length) || (!in && length) || !nonce || !key ||
+	    length > SIZE_MAX - skip) { errno = EINVAL; return -1; }
+	total = skip + length;
+	if (!total) return 0;
+	if (counter > UINT64_MAX - (total / 64u + !!(total % 64u))) { errno = EOVERFLOW; return -1; }
+	aligned = calloc(total, 1);
+	if (!aligned) return -1;
+	if (length) memcpy(aligned + skip, in, length);
+	if (crypto_stream_xsalsa20_xor_ic(aligned, aligned, total, nonce, counter, key) != 0) {
+		free(aligned); errno = EIO; return -1;
+	}
+	if (length) memmove(out, aligned + skip, length);
+	sodium_memzero(aligned, total);
+	free(aligned);
+	return 0;
+}
+
+int sleep_compute_discovery_key(const uint8_t pubkey[32], uint8_t out[32])
+{
+	static const uint8_t domain[] = "hypercore";
+	if (!pubkey || !out) { errno = EINVAL; return -1; }
+	/* Dat discovery keys are BLAKE2b(key=feed public key, message="hypercore"). */
+	return crypto_generichash(out, 32, domain, sizeof(domain) - 1, pubkey, 32);
+}
+
+int sleep_gen_keypair(sleep_keypair_t *kp) {
+	if (!kp) { errno = EINVAL; return -1; }
+	if (crypto_sign_keypair(kp->pubkey, kp->seckey) !=0)
+		return -1;
+	return 0;
+}
+
+int sleep_save_pubkey(const char *path, const uint8_t pubkey[SLEEP_PUB_KEY_BYTES]) {
+	FILE *f = fopen(path, "wb");
+	size_t written;
+	if (!f) return -1;
+	written = fwrite(pubkey, 1, SLEEP_PUB_KEY_BYTES, f);
+	fclose(f);
+	return (written == SLEEP_PUB_KEY_BYTES) ? 0 : -1;
+}
+
+int sleep_load_pubkey(const char *path, uint8_t pubkey[SLEEP_PUB_KEY_BYTES]) {
+	FILE *f = fopen(path, "rb");
+	if (!f) return -1;
+	size_t read = fread(pubkey, 1, SLEEP_PUB_KEY_BYTES, f);
+	fclose(f);
+	return (read == SLEEP_PUB_KEY_BYTES) ? 0 : -1;
+}
+
+int sleep_save_keypair(const char *path,const sleep_keypair_t*kp){FILE*f;size_t n;if(!path||!kp){errno=EINVAL;return-1;}f=fopen(path,"wb");if(!f)return-1;n=fwrite(kp,1,sizeof(*kp),f);if(fclose(f))return-1;return n==sizeof(*kp)?0:-1;}
+int sleep_load_keypair(const char *path,sleep_keypair_t*kp){FILE*f;size_t n;if(!path||!kp){errno=EINVAL;return-1;}f=fopen(path,"rb");if(!f)return-1;n=fread(kp,1,sizeof(*kp),f);fclose(f);return n==sizeof(*kp)?0:-1;}
+
+int sleep_blake2b(uint8_t out[SLEEP_BLAKE2B_BYTES], const uint8_t *in, size_t inlen) {
+	return crypto_generichash(out, SLEEP_BLAKE2B_BYTES, in, inlen, NULL, 0);
+}
+
+int sleep_sign(uint8_t sig[SLEEP_SIGNATURE_BYTES], const uint8_t *msg, size_t msglen, const sleep_keypair_t *kp) {
+	return crypto_sign_detached(sig, NULL, msg, msglen, kp->seckey);
+}
+
+int sleep_verify(const uint8_t sig[SLEEP_SIGNATURE_BYTES], const uint8_t *msg, size_t msglen, const uint8_t pubkey[SLEEP_PUB_KEY_BYTES]) {
+	return crypto_sign_verify_detached(sig, msg, msglen, pubkey);
+}
+
+int sleep_sign_roots(uint8_t sig[SLEEP_SIGNATURE_BYTES], const uint8_t *roots, size_t nroots, const uint64_t *indices, const uint64_t *lengths, const sleep_keypair_t *kp) {
+	uint8_t *message, digest[32]; size_t len; int rc;
+	if (encode_roots(&message, &len, roots, nroots, indices, lengths)) return -1;
+	rc = sleep_blake2b(digest, message, len); free(message);
+	return rc ? rc : sleep_sign(sig, digest, sizeof(digest), kp);
+}
+
+int sleep_verify_roots(const uint8_t sig[SLEEP_SIGNATURE_BYTES], const uint8_t *roots, size_t nroots, const uint64_t *indices, const uint64_t *lengths, const uint8_t pub[SLEEP_PUB_KEY_BYTES]) {
+	uint8_t *message, digest[32]; size_t len; int rc;
+	if (encode_roots(&message, &len, roots, nroots, indices, lengths)) return -1;
+	rc = sleep_blake2b(digest, message, len); free(message);
+	return rc ? rc : sleep_verify(sig, digest, sizeof(digest), pub);
+}

extra/dat-dpi/src/sleep_feed.c

@@ -0,0 +1,131 @@
+#define _POSIX_C_SOURCE 200809L
+#include "sleep_feed.h"
+#include "sleep.h"
+#include "sleep_crypto.h"
+#include "sleep_writer.h"
+#include "sleep_bitfield.h"
+#include "sleep_proof.h"
+#include <errno.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+
+static uint64_t get64be(const uint8_t *p) { uint64_t v=0; unsigned i; for(i=0;i<8;i++) v=(v<<8)|p[i]; return v; }
+static uint32_t get32be(const uint8_t *p) { return ((uint32_t)p[0]<<24)|((uint32_t)p[1]<<16)|((uint32_t)p[2]<<8)|p[3]; }
+static void put64be(uint8_t *p,uint64_t v){unsigned i;for(i=0;i<8;i++)p[i]=(uint8_t)(v>>(56-i*8));}
+static void put32be(uint8_t *p,uint32_t v){p[0]=(uint8_t)(v>>24);p[1]=(uint8_t)(v>>16);p[2]=(uint8_t)(v>>8);p[3]=(uint8_t)v;}
+static int path_for(char *out, size_t cap, const sleep_feed_t *f, const char *suffix) { int n=snprintf(out,cap,"%s/%s.%s",f->directory,f->name,suffix); if(n<0||(size_t)n>=cap){errno=ENAMETOOLONG;return -1;}return 0; }
+static int read_file(const char *path, uint8_t **out, size_t *len) {
+	FILE *fp=fopen(path,"rb"); long n; uint8_t *p; if(!fp) return -1;
+	if(fseek(fp,0,SEEK_END)||(n=ftell(fp))<0||fseek(fp,0,SEEK_SET)){fclose(fp);return -1;}
+	p=malloc(n?(size_t)n:1); if(!p){fclose(fp);return -1;} if(n&&fread(p,1,(size_t)n,fp)!=(size_t)n){free(p);fclose(fp);errno=EIO;return -1;}
+	fclose(fp);*out=p;*len=(size_t)n;return 0;
+}
+static int reserve_feed(sleep_feed_t *f,size_t needed,int with_proofs){
+	size_t capacity=f->capacity?f->capacity:8,old=f->capacity;
+	uint8_t **blocks,**proofs;
+	size_t *lengths,*proof_lengths;
+	if(needed<=f->capacity&&(!with_proofs||f->proofs))return 0;
+	while(capacity<needed){if(capacity>SIZE_MAX/2){errno=EOVERFLOW;return-1;}capacity*=2;}
+	if(capacity>SIZE_MAX/sizeof(*f->blocks)||capacity>SIZE_MAX/sizeof(*f->lengths)){errno=EOVERFLOW;return-1;}
+	blocks=realloc(f->blocks,capacity*sizeof(*blocks));if(!blocks)return-1;f->blocks=blocks;
+	lengths=realloc(f->lengths,capacity*sizeof(*lengths));if(!lengths)return-1;f->lengths=lengths;
+	memset(f->blocks+old,0,(capacity-old)*sizeof(*f->blocks));
+	memset(f->lengths+old,0,(capacity-old)*sizeof(*f->lengths));
+	if(with_proofs||f->proofs){
+		proofs=realloc(f->proofs,capacity*sizeof(*proofs));if(!proofs)return-1;f->proofs=proofs;
+		proof_lengths=realloc(f->proof_lengths,capacity*sizeof(*proof_lengths));if(!proof_lengths)return-1;f->proof_lengths=proof_lengths;
+		memset(f->proofs+old,0,(capacity-old)*sizeof(*f->proofs));
+		memset(f->proof_lengths+old,0,(capacity-old)*sizeof(*f->proof_lengths));
+	}
+	f->capacity=capacity;return 0;
+}
+static int append_remote_record(sleep_feed_t *f,size_t index,const uint8_t *data,size_t len,const uint8_t *proof,size_t proof_len){
+	static const uint8_t magic[8]={'S','F','E','E','D','R','1',0};
+	char path[1200];uint8_t header[40],record[16];FILE *fp;long end;
+	if(len>UINT32_MAX||proof_len>UINT32_MAX||path_for(path,sizeof(path),f,"remote"))return-1;
+	fp=fopen(path,"ab+");if(!fp)return-1;
+	if(fseek(fp,0,SEEK_END)||(end=ftell(fp))<0){fclose(fp);return-1;}
+	if(end==0){memcpy(header,magic,8);memcpy(header+8,f->keypair.pubkey,32);if(fwrite(header,1,sizeof(header),fp)!=sizeof(header)){fclose(fp);errno=EIO;return-1;}}
+	put64be(record,(uint64_t)index);put32be(record+8,(uint32_t)len);put32be(record+12,(uint32_t)proof_len);
+	if(fwrite(record,1,sizeof(record),fp)!=sizeof(record)||(len&&fwrite(data,1,len,fp)!=len)||(proof_len&&fwrite(proof,1,proof_len,fp)!=proof_len)||fflush(fp)||fsync(fileno(fp))){fclose(fp);errno=EIO;return-1;}
+	return fclose(fp);
+}
+int sleep_feed_open(sleep_feed_t *f,const char *dir,const char *name,const sleep_keypair_t *kp) {
+	char path[1200]; uint8_t *tree=NULL,*data=NULL; size_t tree_len=0,data_len=0,count,i,off=0;
+	if(!f||!dir||!name){errno=EINVAL;return -1;} memset(f,0,sizeof(*f));
+	if(strlen(dir)>=sizeof(f->directory)||strlen(name)>=sizeof(f->name)){errno=ENAMETOOLONG;return -1;} strcpy(f->directory,dir);strcpy(f->name,name);
+	if(kp){f->keypair=*kp;f->writable=1;}
+	if(path_for(path,sizeof(path),f,"tree")||read_file(path,&tree,&tree_len)) {
+		if(errno==ENOENT&&kp)return 0;
+		if(errno==ENOENT&&!kp){
+			char key_path[1200]; uint8_t public_key[32]; int n;
+			n=snprintf(key_path,sizeof(key_path),"%s/%s.key",dir,name);
+			if(n<0||(size_t)n>=sizeof(key_path)){errno=ENAMETOOLONG;return-1;}
+			if(sleep_load_pubkey(key_path,public_key))return-1;
+			return sleep_feed_open_remote(f,dir,name,public_key);
+		}
+		return -1;
+	}
+	if(tree_len<32||(tree_len-32)%40){free(tree);errno=EPROTO;return -1;} count=((tree_len-32)/40+1)/2;
+	if(path_for(path,sizeof(path),f,"data")||read_file(path,&data,&data_len)){free(tree);return -1;}
+	f->blocks=calloc(count,sizeof(*f->blocks));f->lengths=calloc(count,sizeof(*f->lengths));f->capacity=count;if(count&&(!f->blocks||!f->lengths))goto fail;
+	for(i=0;i<count;i++){uint64_t n=get64be(tree+32+i*80+32);if(n>SIZE_MAX||n>data_len-off){errno=EPROTO;goto fail;}f->lengths[i]=(size_t)n;f->blocks[i]=malloc(n?(size_t)n:1);if(!f->blocks[i])goto fail;if(n)memcpy(f->blocks[i],data+off,(size_t)n);off+=(size_t)n;f->length++;}
+	free(tree);free(data);return 0;
+fail: free(tree);free(data);sleep_feed_close(f);return -1;
+}
+int sleep_feed_open_remote(sleep_feed_t *f,const char *dir,const char *name,const uint8_t public_key[32]) {
+	static const uint8_t magic[8]={'S','F','E','E','D','R','1',0};
+	char path[1200];uint8_t *bytes=NULL;size_t length=0,off=40;
+	if(!f||!dir||!name||!public_key){errno=EINVAL;return-1;}
+	if(strlen(dir)>=sizeof(f->directory)||strlen(name)>=sizeof(f->name)){errno=ENAMETOOLONG;return-1;}
+	memset(f,0,sizeof(*f));strcpy(f->directory,dir);strcpy(f->name,name);
+	memcpy(f->keypair.pubkey,public_key,32);f->remote_only=1;f->writable=0;f->persist_remote=1;
+	if(path_for(path,sizeof(path),f,"remote")||read_file(path,&bytes,&length)){
+		if(errno==ENOENT)return 0;
+		sleep_feed_close(f);return-1;
+	}
+	if(length<40||memcmp(bytes,magic,8)||memcmp(bytes+8,public_key,32))goto corrupt;
+	f->persist_remote=0;
+	while(off<length){
+		uint64_t index;uint32_t data_len,proof_len;sleep_proof_t proof;
+		if(length-off<16)goto corrupt;
+		index=get64be(bytes+off);data_len=get32be(bytes+off+8);proof_len=get32be(bytes+off+12);off+=16;
+		if(index>=SLEEP_BITFIELD_DATA_BYTES*8u||!proof_len||data_len>length-off||proof_len>length-off-data_len)goto corrupt;
+		if(sleep_proof_decode(&proof,bytes+off+data_len,proof_len)||proof.block!=index||
+		   sleep_proof_verify(&proof,bytes+off,data_len,public_key))goto corrupt;
+		if(sleep_feed_append_remote(f,(size_t)index,bytes+off,data_len,
+					    bytes+off+data_len,proof_len))goto corrupt;
+		off+=(size_t)data_len+proof_len;
+	}
+	f->persist_remote=1;free(bytes);return 0;
+corrupt:
+	free(bytes);sleep_feed_close(f);errno=EBADMSG;return-1;
+}
+int sleep_feed_load_remote(sleep_feed_t *f,const uint8_t public_key[32]) {
+	sleep_feed_t cached;
+	int persist,rc=0;
+	size_t i;
+	if(!f||!public_key||f->writable){errno=EINVAL;return-1;}
+	if(sleep_feed_open_remote(&cached,f->directory,f->name,public_key))return-1;
+	persist=f->persist_remote;
+	f->persist_remote=0;
+	memcpy(f->keypair.pubkey,public_key,32);
+	for(i=0;i<cached.length;i++)if(cached.blocks[i]&&!sleep_feed_has(f,i)){
+		if(sleep_feed_append_remote(f,i,cached.blocks[i],cached.lengths[i],
+					    cached.proofs[i],cached.proof_lengths[i])){rc=-1;break;}
+	}
+	f->persist_remote=persist;
+	sleep_feed_close(&cached);
+	return rc;
+}
+size_t sleep_feed_length(const sleep_feed_t *f){return f?f->length:0;}
+int sleep_feed_read(const sleep_feed_t *f,size_t index,uint8_t **out,size_t *len){uint8_t *p;if(!f||!out||!len||index>=f->length){errno=EINVAL;return -1;}if(!f->blocks[index]){errno=ENODATA;return-1;}p=malloc(f->lengths[index]?f->lengths[index]:1);if(!p)return -1;if(f->lengths[index])memcpy(p,f->blocks[index],f->lengths[index]);*out=p;*len=f->lengths[index];return 0;}
+int sleep_feed_append(sleep_feed_t *f,const uint8_t *data,size_t len){uint8_t *copy;if(!f||!f->writable||(!data&&len)){errno=EINVAL;return-1;}if(reserve_feed(f,f->length+1,0))return-1;copy=malloc(len?len:1);if(!copy)return-1;if(len)memcpy(copy,data,len);f->blocks[f->length]=copy;f->lengths[f->length]=len;f->length++;f->dirty=1;return 0;}
+int sleep_feed_append_remote(sleep_feed_t *f,size_t index,const uint8_t *data,size_t len,const uint8_t *proof,size_t proof_len){uint8_t *copy,*proof_copy;if(!f||f->writable||index>=SLEEP_BITFIELD_DATA_BYTES*8u||(!data&&len)||!proof||!proof_len){errno=EINVAL;return-1;}if(index<f->length&&f->blocks[index]){errno=EEXIST;return-1;}if(reserve_feed(f,index+1,1))return-1;copy=malloc(len?len:1);proof_copy=malloc(proof_len);if(!copy||!proof_copy){free(copy);free(proof_copy);return-1;}if(len)memcpy(copy,data,len);memcpy(proof_copy,proof,proof_len);if(f->persist_remote&&append_remote_record(f,index,data,len,proof,proof_len)){free(copy);free(proof_copy);return-1;}f->blocks[index]=copy;f->lengths[index]=len;f->proofs[index]=proof_copy;f->proof_lengths[index]=proof_len;if(f->length<=index)f->length=index+1;f->remote_only=1;return 0;}
+int sleep_feed_truncate(sleep_feed_t*f,size_t length){size_t i;if(!f||!f->writable||length>f->length){errno=EINVAL;return-1;}for(i=length;i<f->length;i++)free(f->blocks[i]);f->length=length;f->dirty=1;return 0;}
+int sleep_feed_has(const sleep_feed_t*f,size_t index){return f&&index<f->length&&f->blocks[index]!=NULL;}
+int sleep_feed_byte_offset(const sleep_feed_t*f,size_t index,uint64_t*out){size_t i;uint64_t n=0;if(!f||!out||index>f->length){errno=EINVAL;return-1;}for(i=0;i<index;i++){if(!f->blocks[i]){errno=ENODATA;return-1;}if(UINT64_MAX-n<f->lengths[i]){errno=EOVERFLOW;return-1;}n+=f->lengths[i];}*out=n;return 0;}
+int sleep_feed_flush(sleep_feed_t *f){if(!f||!f->writable){errno=EINVAL;return-1;}if(!f->dirty)return 0;if(sleep_write_feed(f->directory,f->name,(const uint8_t *const*)f->blocks,f->lengths,f->length,&f->keypair))return-1;f->dirty=0;return 0;}
+void sleep_feed_close(sleep_feed_t *f){size_t i;if(!f)return;for(i=0;i<f->length;i++){free(f->blocks?f->blocks[i]:NULL);free(f->proofs?f->proofs[i]:NULL);}free(f->blocks);free(f->lengths);free(f->proofs);free(f->proof_lengths);memset(f,0,sizeof(*f));}

extra/dat-dpi/src/sleep_proof.c

@@ -0,0 +1,302 @@
+#define _POSIX_C_SOURCE 200809L
+#include "sleep_proof.h"
+
+#include "sleep.h"
+#include "sleep_crypto.h"
+#include <errno.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#define PROOF_FIXED_BYTES 51u
+#define PROOF_NODE_BYTES 48u
+
+static void put64(uint8_t *p, uint64_t value)
+{
+	unsigned i;
+	for (i = 0; i < 8; i++) p[i] = (uint8_t)(value >> (56 - i * 8));
+}
+
+static uint64_t get64(const uint8_t *p)
+{
+	uint64_t value = 0;
+	unsigned i;
+	for (i = 0; i < 8; i++) value = (value << 8) | p[i];
+	return value;
+}
+
+static int read_file(const char *path, uint8_t **data, size_t *length)
+{
+	FILE *file;
+	long size;
+	uint8_t *buffer;
+	file = fopen(path, "rb");
+	if (!file) return -1;
+	if (fseek(file, 0, SEEK_END) || (size = ftell(file)) < 0 ||
+	    fseek(file, 0, SEEK_SET)) { fclose(file); return -1; }
+	buffer = malloc(size ? (size_t)size : 1);
+	if (!buffer) { fclose(file); return -1; }
+	if (size && fread(buffer, 1, (size_t)size, file) != (size_t)size) {
+		free(buffer); fclose(file); errno = EIO; return -1;
+	}
+	fclose(file);
+	*data = buffer;
+	*length = (size_t)size;
+	return 0;
+}
+
+static int add_node(sleep_proof_node_t *nodes, size_t *count,
+		    uint64_t index, const uint8_t *entry)
+{
+	sleep_proof_node_t *node;
+	if (*count >= SLEEP_PROOF_MAX_NODES) { errno = EOVERFLOW; return -1; }
+	node = &nodes[(*count)++];
+	memcpy(node->hash, entry, 32);
+	node->index = index;
+	node->length = get64(entry + 32);
+	return 0;
+}
+
+void sleep_proof_init(sleep_proof_t *proof, uint64_t block)
+{
+	if (proof) { memset(proof, 0, sizeof(*proof)); proof->block = block; }
+}
+
+int sleep_proof_add_node(sleep_proof_t *proof, uint64_t index,
+			 uint64_t length, const uint8_t hash[32])
+{
+	sleep_proof_node_t *node;
+	if (!proof || !hash || proof->node_count >= SLEEP_PROOF_MAX_NODES) {
+		errno = EINVAL; return -1;
+	}
+	node = &proof->nodes[proof->node_count++];
+	node->index = index; node->length = length; memcpy(node->hash, hash, 32);
+	return 0;
+}
+
+int sleep_proof_add_root(sleep_proof_t *proof, uint64_t index,
+			 uint64_t length, const uint8_t hash[32])
+{
+	sleep_proof_node_t *root;
+	if (!proof || !hash || proof->root_count >= SLEEP_PROOF_MAX_NODES) {
+		errno = EINVAL; return -1;
+	}
+	root = &proof->roots[proof->root_count++];
+	root->index = index; root->length = length; memcpy(root->hash, hash, 32);
+	return 0;
+}
+
+int sleep_proof_create(const sleep_feed_t *feed, uint64_t block,
+		       sleep_proof_t *proof)
+{
+	char tree_path[1200], sig_path[1200];
+	uint8_t *tree = NULL, *signatures = NULL;
+	size_t tree_len = 0, sig_len = 0, node_count, root_count, i;
+	uint64_t root_indexes[SLEEP_PROOF_MAX_NODES], index;
+	const uint8_t *leaf;
+	int is_root;
+	if (!feed || !proof || !feed->length || block >= feed->length || feed->dirty) {
+		errno = feed && feed->dirty ? EBUSY : EINVAL;
+		return -1;
+	}
+	/* A read-only replica can forward the publisher's original signed proof. */
+	if (feed->proofs && feed->proof_lengths && feed->proofs[block])
+		return sleep_proof_decode(proof, feed->proofs[block], feed->proof_lengths[block]);
+	if (snprintf(tree_path, sizeof(tree_path), "%s/%s.tree", feed->directory,
+		     feed->name) >= (int)sizeof(tree_path) ||
+	    snprintf(sig_path, sizeof(sig_path), "%s/%s.signatures", feed->directory,
+		     feed->name) >= (int)sizeof(sig_path)) { errno = ENAMETOOLONG; return -1; }
+	if (read_file(tree_path, &tree, &tree_len) ||
+	    read_file(sig_path, &signatures, &sig_len)) goto fail;
+	if (tree_len < SLEEP_HEADER_SIZE || (tree_len - SLEEP_HEADER_SIZE) % SLEEP_TREE_ENTRY_SIZE ||
+	    sig_len < SLEEP_HEADER_SIZE || (sig_len - SLEEP_HEADER_SIZE) / SLEEP_SIG_ENTRY_SIZE < feed->length) {
+		errno = EPROTO; goto fail;
+	}
+	node_count = (tree_len - SLEEP_HEADER_SIZE) / SLEEP_TREE_ENTRY_SIZE;
+	root_count = sleep_tree_full_roots((uint64_t)feed->length * 2,
+					   root_indexes, SLEEP_PROOF_MAX_NODES);
+	if (!root_count || root_count > SLEEP_PROOF_MAX_NODES || block > UINT64_MAX / 2) {
+		errno = EOVERFLOW; goto fail;
+	}
+	sleep_proof_init(proof, block);
+	index = block * 2;
+	if (index >= node_count) { errno = EPROTO; goto fail; }
+	leaf = tree + SLEEP_HEADER_SIZE + (size_t)index * SLEEP_TREE_ENTRY_SIZE;
+	proof->block_length = get64(leaf + 32);
+	memcpy(proof->block_hash, leaf, 32);
+	for (;;) {
+		is_root = 0;
+		for (i = 0; i < root_count; i++) if (root_indexes[i] == index) is_root = 1;
+		if (is_root) break;
+		{
+			uint64_t sibling;
+			const uint8_t *entry;
+			if (sleep_tree_sibling(index, &sibling) || sibling >= node_count) {
+				errno = EPROTO; goto fail;
+			}
+			entry = tree + SLEEP_HEADER_SIZE + (size_t)sibling * SLEEP_TREE_ENTRY_SIZE;
+			if (add_node(proof->nodes, &proof->node_count, sibling, entry) ||
+			    sleep_tree_parent(index) == UINT64_MAX) goto fail;
+			index = sleep_tree_parent(index);
+		}
+	}
+	for (i = 0; i < root_count; i++) {
+		uint64_t root = root_indexes[i];
+		const uint8_t *entry;
+		if (root >= node_count) { errno = EPROTO; goto fail; }
+		entry = tree + SLEEP_HEADER_SIZE + (size_t)root * SLEEP_TREE_ENTRY_SIZE;
+		if (add_node(proof->roots, &proof->root_count, root, entry)) goto fail;
+	}
+	memcpy(proof->signature,
+	       signatures + SLEEP_HEADER_SIZE + (feed->length - 1) * SLEEP_SIG_ENTRY_SIZE,
+	       sizeof(proof->signature));
+	proof->has_signature = 1;
+	free(tree); free(signatures);
+	return 0;
+fail:
+	free(tree); free(signatures); memset(proof, 0, sizeof(*proof));
+	return -1;
+}
+
+int sleep_proof_from_nodes(uint64_t block, const uint8_t *value,
+			   size_t value_len, const sleep_proof_node_t *nodes,
+			   size_t node_count, const uint8_t signature[64],
+			   uint64_t feed_length, sleep_proof_t *proof)
+{
+	uint64_t root_indexes[SLEEP_PROOF_MAX_NODES], index;
+	size_t root_count, i, j;
+	int is_root;
+	if (!proof || (!value && value_len) || !signature ||
+	    (node_count && !nodes) || !feed_length || block >= feed_length ||
+	    feed_length > UINT64_MAX / 2 || block > UINT64_MAX / 2) {
+		errno = EINVAL;
+		return -1;
+	}
+	root_count = sleep_tree_full_roots(feed_length * 2, root_indexes,
+					   SLEEP_PROOF_MAX_NODES);
+	if (!root_count || root_count > SLEEP_PROOF_MAX_NODES) { errno = EOVERFLOW; return -1; }
+	sleep_proof_init(proof, block);
+	proof->block_length = value_len;
+	if (sleep_tree_hash_block(value, value_len, proof->block_hash)) return -1;
+	index = block * 2;
+	for (;;) {
+		is_root = 0;
+		for (i = 0; i < root_count; i++) if (root_indexes[i] == index) is_root = 1;
+		if (is_root) break;
+		{
+			uint64_t sibling;
+			if (sleep_tree_sibling(index, &sibling)) goto malformed;
+			for (j = 0; j < node_count; j++) if (nodes[j].index == sibling) break;
+			if (j == node_count || sleep_proof_add_node(proof, sibling,
+							 nodes[j].length, nodes[j].hash)) goto malformed;
+			index = sleep_tree_parent(index);
+			if (index == UINT64_MAX) goto malformed;
+		}
+	}
+	for (i = 0; i < root_count; i++) {
+		for (j = 0; j < node_count; j++) if (nodes[j].index == root_indexes[i]) break;
+		if (j == node_count || sleep_proof_add_root(proof, nodes[j].index,
+							   nodes[j].length, nodes[j].hash)) goto malformed;
+	}
+	memcpy(proof->signature, signature, 64);
+	proof->has_signature = 1;
+	return 0;
+malformed:
+	memset(proof, 0, sizeof(*proof));
+	errno = EBADMSG;
+	return -1;
+}
+
+static void encode_node(uint8_t *out, const sleep_proof_node_t *node)
+{
+	memcpy(out, node->hash, 32); put64(out + 32, node->index); put64(out + 40, node->length);
+}
+
+int sleep_proof_encode(const sleep_proof_t *proof, uint8_t *out,
+		       size_t capacity, size_t *written)
+{
+	size_t need, off = 0, i;
+	if (!proof || !out || proof->node_count > SLEEP_PROOF_MAX_NODES ||
+	    proof->root_count > SLEEP_PROOF_MAX_NODES) { errno = EINVAL; return -1; }
+	need = PROOF_FIXED_BYTES + (proof->node_count + proof->root_count) * PROOF_NODE_BYTES +
+		(proof->has_signature ? 64 : 0);
+	if (capacity < need) { errno = ENOSPC; return -1; }
+	put64(out + off, proof->block); off += 8;
+	put64(out + off, proof->block_length); off += 8;
+	memcpy(out + off, proof->block_hash, 32); off += 32;
+	out[off++] = (uint8_t)proof->node_count;
+	out[off++] = (uint8_t)!!proof->has_signature;
+	out[off++] = (uint8_t)proof->root_count;
+	for (i = 0; i < proof->node_count; i++, off += PROOF_NODE_BYTES) encode_node(out + off, &proof->nodes[i]);
+	for (i = 0; i < proof->root_count; i++, off += PROOF_NODE_BYTES) encode_node(out + off, &proof->roots[i]);
+	if (proof->has_signature) { memcpy(out + off, proof->signature, 64); off += 64; }
+	if (written) *written = off;
+	return 0;
+}
+
+static void decode_node(sleep_proof_node_t *node, const uint8_t *in)
+{
+	memcpy(node->hash, in, 32); node->index = get64(in + 32); node->length = get64(in + 40);
+}
+
+int sleep_proof_decode(sleep_proof_t *proof, const uint8_t *input, size_t input_len)
+{
+	size_t off = 0, i, need;
+	if (!proof || !input || input_len < PROOF_FIXED_BYTES) { errno = EPROTO; return -1; }
+	sleep_proof_init(proof, get64(input)); off = 8;
+	proof->block_length = get64(input + off); off += 8;
+	memcpy(proof->block_hash, input + off, 32); off += 32;
+	proof->node_count = input[off++];
+	proof->has_signature = !!input[off++];
+	proof->root_count = input[off++];
+	if (proof->node_count > SLEEP_PROOF_MAX_NODES || proof->root_count > SLEEP_PROOF_MAX_NODES) {
+		errno = EPROTO; return -1;
+	}
+	need = PROOF_FIXED_BYTES + (proof->node_count + proof->root_count) * PROOF_NODE_BYTES +
+		(proof->has_signature ? 64 : 0);
+	if (input_len != need) { errno = EPROTO; return -1; }
+	for (i = 0; i < proof->node_count; i++, off += PROOF_NODE_BYTES) decode_node(&proof->nodes[i], input + off);
+	for (i = 0; i < proof->root_count; i++, off += PROOF_NODE_BYTES) decode_node(&proof->roots[i], input + off);
+	if (proof->has_signature) memcpy(proof->signature, input + off, 64);
+	return 0;
+}
+
+int sleep_proof_verify(const sleep_proof_t *proof, const uint8_t *block,
+		       size_t block_len, const uint8_t public_key[32])
+{
+	uint8_t hash[32], parent[32], root_hashes[SLEEP_PROOF_MAX_NODES * 32];
+	uint64_t indices[SLEEP_PROOF_MAX_NODES], lengths[SLEEP_PROOF_MAX_NODES];
+	uint64_t index, length;
+	size_t i, r;
+	int root_found = 0;
+	if (!proof || (!block && block_len) || !public_key || !proof->has_signature ||
+	    !proof->root_count || proof->root_count > SLEEP_PROOF_MAX_NODES) { errno = EINVAL; return -1; }
+	if (proof->block > UINT64_MAX / 2 || proof->block_length != block_len ||
+	    sleep_tree_hash_block(block, block_len, hash) || memcmp(hash, proof->block_hash, 32)) {
+		errno = EBADMSG; return -1;
+	}
+	index = proof->block * 2;
+	length = block_len;
+	for (i = 0; i < proof->node_count; i++) {
+		const sleep_proof_node_t *node = &proof->nodes[i];
+		uint64_t sibling;
+		if (sleep_tree_sibling(index, &sibling) || node->index != sibling ||
+		    UINT64_MAX - length < node->length) { errno = EBADMSG; return -1; }
+		if (sibling < index) {
+			if (sleep_tree_hash_parent(node->hash, hash, node->length + length, parent)) return -1;
+		} else if (sleep_tree_hash_parent(hash, node->hash, length + node->length, parent)) return -1;
+		memcpy(hash, parent, 32); length += node->length; index = sleep_tree_parent(index);
+	}
+	for (r = 0; r < proof->root_count; r++) {
+		const sleep_proof_node_t *root = &proof->roots[r];
+		indices[r] = root->index; lengths[r] = root->length;
+		memcpy(root_hashes + r * 32, root->hash, 32);
+		if (root->index == index && root->length == length && !memcmp(root->hash, hash, 32)) root_found = 1;
+		if (r && proof->roots[r - 1].index >= root->index) { errno = EBADMSG; return -1; }
+	}
+	if (!root_found || sleep_verify_roots(proof->signature, root_hashes,
+						proof->root_count, indices, lengths, public_key)) {
+		errno = EBADMSG; return -1;
+	}
+	return 0;
+}

extra/dat-dpi/src/sleep_proto.c

@@ -0,0 +1,58 @@
+#include "sleep_proto.h"
+#include <errno.h>
+#include <stdio.h>
+#include <string.h>
+static size_t bounded_len(const char *s,size_t cap){size_t n=0;while(n<cap&&s[n])n++;return n;}
+
+size_t sleep_varint_size(uint64_t v) { size_t n = 1; while (v >= 128) { v >>= 7; n++; } return n; }
+int sleep_varint_encode(uint64_t v, uint8_t *out, size_t cap, size_t *written) {
+	size_t need = sleep_varint_size(v), i = 0; if (!out || cap < need) { errno = out ? ENOSPC : EINVAL; return -1; }
+	while (v >= 128) { out[i++] = (uint8_t)((v & 127) | 128); v >>= 7; } out[i++] = (uint8_t)v;
+	if (written) *written = i;
+	return 0;
+}
+int sleep_varint_decode(const uint8_t *in, size_t len, uint64_t *value, size_t *used) {
+	uint64_t v = 0; size_t i; if (!in || !value) { errno = EINVAL; return -1; }
+	for (i = 0; i < len && i < 10; i++) { uint8_t b = in[i]; if (i == 9 && b > 1) { errno = EOVERFLOW; return -1; }
+		v |= (uint64_t)(b & 127) << (i * 7); if (!(b & 128)) { *value = v; if (used) *used = i + 1; return 0; } }
+	errno = len < 10 ? EAGAIN : EOVERFLOW; return -1;
+}
+static int put_field(uint8_t tag, const uint8_t *data, size_t len, uint8_t *out, size_t cap, size_t *off) {
+	size_t n; if (*off >= cap) { errno = ENOSPC; return -1; } out[(*off)++] = tag;
+	if (sleep_varint_encode(len, out + *off, cap - *off, &n)) return -1;
+	*off += n;
+	if (len > cap - *off) { errno = ENOSPC; return -1; } memcpy(out + *off, data, len); *off += len; return 0;
+}
+int sleep_metadata_header_encode(const sleep_metadata_header_t *h, uint8_t *out, size_t cap, size_t *written) {
+	size_t off = 0, type_len; if (!h || !out || h->content_len > 32) { errno = EINVAL; return -1; } type_len = strlen(h->type);
+	if (put_field(0x0a, (const uint8_t *)h->type, type_len, out, cap, &off) || put_field(0x12, h->content, h->content_len, out, cap, &off)) return -1;
+	if (written) *written = off;
+	return 0;
+}
+int sleep_metadata_header_decode(sleep_metadata_header_t *h, const uint8_t *in, size_t len) {
+	size_t off = 0; if (!h || (!in && len)) { errno = EINVAL; return -1; } memset(h, 0, sizeof(*h));
+	while (off < len) { uint8_t tag = in[off++]; uint64_t n64; size_t n, copy;
+		if (sleep_varint_decode(in + off, len - off, &n64, &n)) return -1;
+		off += n;
+		if (n64 > len - off) { errno = EPROTO; return -1; }
+		if (tag == 0x0a) { copy = n64 < sizeof(h->type)-1 ? (size_t)n64 : sizeof(h->type)-1; memcpy(h->type, in+off, copy); }
+		else if (tag == 0x12) { if (n64 > 32) { errno = EOVERFLOW; return -1; } memcpy(h->content, in+off, (size_t)n64); h->content_len = (size_t)n64; }
+		off += (size_t)n64; }
+	return 0;
+}
+int sleep_write_metadata_header(const char *path, const uint8_t key[32]) {
+	sleep_metadata_header_t h; uint8_t buf[128]; size_t len; FILE *f; if (!path || !key) { errno = EINVAL; return -1; }
+	memset(&h, 0, sizeof(h)); strcpy(h.type, "hyperdrive"); memcpy(h.content, key, 32); h.content_len = 32;
+	if (sleep_metadata_header_encode(&h, buf, sizeof(buf), &len)) return -1;
+	f = fopen(path, "wb");
+	if (!f) return -1;
+	if (fwrite(buf, 1, len, f) != len) { fclose(f); errno = EIO; return -1; } return fclose(f);
+}
+
+static int put_uint_field(uint8_t tag,uint64_t value,uint8_t *out,size_t cap,size_t *off){size_t n;if(*off>=cap){errno=ENOSPC;return-1;}out[(*off)++]=tag;if(sleep_varint_encode(value,out+*off,cap-*off,&n))return-1;*off+=n;return 0;}
+int sleep_metadata_entry_encode(const sleep_metadata_entry_t *e,uint8_t *out,size_t cap,size_t *written){size_t off=0,path_len,link_len;if(!e||!out){errno=EINVAL;return-1;}path_len=bounded_len(e->path,sizeof(e->path));link_len=bounded_len(e->linkname,sizeof(e->linkname));
+	if(put_field(0x0a,(const uint8_t*)e->path,path_len,out,cap,&off)||put_uint_field(0x10,e->type,out,cap,&off)||put_uint_field(0x18,e->mode,out,cap,&off)||put_uint_field(0x20,e->size,out,cap,&off)||put_uint_field(0x28,e->mtime,out,cap,&off)||put_uint_field(0x30,e->block_offset,out,cap,&off)||put_uint_field(0x38,e->block_count,out,cap,&off))return-1;
+	if(link_len&&put_field(0x42,(const uint8_t*)e->linkname,link_len,out,cap,&off))return-1;
+	if(written)*written=off;
+	return 0;}
+int sleep_metadata_entry_decode(sleep_metadata_entry_t *e,const uint8_t *in,size_t len){size_t off=0;if(!e||(!in&&len)){errno=EINVAL;return-1;}memset(e,0,sizeof(*e));while(off<len){uint8_t tag=in[off++];uint64_t value;size_t n;if((tag&7)==2){size_t copy;if(sleep_varint_decode(in+off,len-off,&value,&n))return-1;off+=n;if(value>len-off){errno=EPROTO;return-1;}if(tag==0x0a){copy=value<sizeof(e->path)-1?(size_t)value:sizeof(e->path)-1;memcpy(e->path,in+off,copy);}else if(tag==0x42){copy=value<sizeof(e->linkname)-1?(size_t)value:sizeof(e->linkname)-1;memcpy(e->linkname,in+off,copy);}off+=(size_t)value;}else if((tag&7)==0){if(sleep_varint_decode(in+off,len-off,&value,&n))return-1;off+=n;switch(tag){case 0x10:e->type=(uint32_t)value;break;case 0x18:e->mode=(uint32_t)value;break;case 0x20:e->size=value;break;case 0x28:e->mtime=value;break;case 0x30:e->block_offset=value;break;case 0x38:e->block_count=value;break;default:break;}}else{errno=EPROTO;return-1;}}return 0;}

extra/dat-dpi/src/sleep_replication.c

@@ -0,0 +1,160 @@
+#include "sleep_replication.h"
+#include <errno.h>
+#include <stdlib.h>
+#include <string.h>
+
+int sleep_replication_init(sleep_replication_t *r, sleep_feed_t *feed,
+			   uint64_t channel)
+{
+	if (!r || !feed) { errno = EINVAL; return -1; }
+	memset(r, 0, sizeof(*r));
+	r->feed = feed;
+	r->channel = channel;
+	r->live = 1;
+	r->uploading = 1;
+	r->downloading = 1;
+	return 0;
+}
+
+int sleep_replication_have(sleep_replication_t *r, uint64_t start,
+			   uint64_t length)
+{
+	uint64_t i, end;
+	if (!r || start > UINT64_MAX - length) { errno = EINVAL; return -1; }
+	end = start + length;
+	for (i = start; i < end && i < SLEEP_BITFIELD_DATA_BYTES * 8u; i++)
+		if (sleep_bitfield_set_data(&r->remote_bitfield, (size_t)i, 1)) return -1;
+	if (end > r->remote_length) r->remote_length = end;
+	return 0;
+}
+
+int sleep_replication_unhave(sleep_replication_t *r, uint64_t start,
+			     uint64_t length)
+{
+	uint64_t i, end;
+	size_t last = SIZE_MAX;
+	if (!r || start > UINT64_MAX - length) { errno = EINVAL; return -1; }
+	end = start + length;
+	for (i = start; i < end && i < SLEEP_BITFIELD_DATA_BYTES * 8u; i++)
+		if (sleep_bitfield_set_data(&r->remote_bitfield, (size_t)i, 0)) return -1;
+	for (i = 0; i < SLEEP_BITFIELD_DATA_BYTES * 8u; i++)
+		if (sleep_bitfield_get_data(&r->remote_bitfield, (size_t)i) == 1) last = (size_t)i;
+	r->remote_length = last == SIZE_MAX ? 0 : (uint64_t)last + 1;
+	return 0;
+}
+
+int sleep_replication_request(sleep_replication_t *r,
+			      const sleep_replication_request_t *q)
+{
+	size_t i, tail;
+	if (!r || !q) { errno = EINVAL; return -1; }
+	for (i = 0; i < r->request_count; i++) {
+		size_t slot = (r->request_head + i) % SLEEP_REPLICATION_QUEUE;
+		if (r->requests[slot].index == q->index) return 0;
+	}
+	if (r->request_count >= SLEEP_REPLICATION_QUEUE) { errno = ENOSPC; return -1; }
+	tail = (r->request_head + r->request_count) % SLEEP_REPLICATION_QUEUE;
+	r->requests[tail] = *q;
+	r->request_count++;
+	return 0;
+}
+
+int sleep_replication_want(sleep_replication_t *r, uint64_t start,
+			   uint64_t length)
+{
+	uint64_t i;
+	size_t additions = 0;
+	if (!r || start > UINT64_MAX - length) { errno = EINVAL; return -1; }
+	if (length > SLEEP_REPLICATION_QUEUE) { errno = ENOSPC; return -1; }
+	for (i = start; i < start + length; i++) {
+		if (!sleep_replication_is_requested(r, i)) additions++;
+	}
+	if (additions > SLEEP_REPLICATION_QUEUE - r->request_count) {
+		errno = ENOSPC;
+		return -1;
+	}
+	for (i = start; i < start + length; i++) {
+		sleep_replication_request_t q = {i, 0, 1, 1};
+		if (sleep_replication_request(r, &q)) return -1;
+	}
+	return 0;
+}
+
+int sleep_replication_next(sleep_replication_t *r,
+			   sleep_replication_request_t *q)
+{
+	if (!r || !q) { errno = EINVAL; return -1; }
+	if (!r->request_count) return 0;
+	*q = r->requests[r->request_head];
+	r->request_head = (r->request_head + 1) % SLEEP_REPLICATION_QUEUE;
+	r->request_count--;
+	return 1;
+}
+
+int sleep_replication_complete(sleep_replication_t *r, uint64_t index)
+{
+	sleep_replication_request_t kept[SLEEP_REPLICATION_QUEUE];
+	size_t i, count = 0;
+	if (!r) { errno = EINVAL; return -1; }
+	for (i = 0; i < r->request_count; i++) {
+		size_t slot = (r->request_head + i) % SLEEP_REPLICATION_QUEUE;
+		if (r->requests[slot].index != index) kept[count++] = r->requests[slot];
+	}
+	memcpy(r->requests, kept, count * sizeof(*kept));
+	r->request_head = 0;
+	r->request_count = count;
+	return 0;
+}
+
+int sleep_replication_is_requested(const sleep_replication_t *r, uint64_t index)
+{
+	size_t i;
+	if (!r) { errno = EINVAL; return -1; }
+	for (i = 0; i < r->request_count; i++) {
+		size_t slot = (r->request_head + i) % SLEEP_REPLICATION_QUEUE;
+		if (r->requests[slot].index == index) return 1;
+	}
+	return 0;
+}
+
+int sleep_replication_unwant(sleep_replication_t *r, uint64_t start,
+			     uint64_t length)
+{
+	uint64_t i;
+	if (!r || start > UINT64_MAX - length || length > SLEEP_REPLICATION_QUEUE) { errno = EINVAL; return -1; }
+	for (i = start; i < start + length; i++)
+		if (sleep_replication_complete(r, i)) return -1;
+	return 0;
+}
+
+void sleep_replication_clear_requests(sleep_replication_t *r)
+{
+	if (!r) return;
+	r->request_head = 0;
+	r->request_count = 0;
+}
+
+int sleep_replication_receive(sleep_replication_t *r, const uint8_t *block,
+			      size_t len, const sleep_proof_t *proof,
+			      const uint8_t public_key[32])
+{
+	uint8_t encoded[8192];
+	uint8_t *old = NULL;
+	size_t encoded_len, old_len;
+	if (!r || !r->feed || (!block && len) || !proof || !public_key) {
+		errno = EINVAL;
+		return -1;
+	}
+	if (proof->block >= SLEEP_BITFIELD_DATA_BYTES * 8u ||
+	    sleep_proof_verify(proof, block, len, public_key)) return -1;
+	if (sleep_proof_encode(proof, encoded, sizeof(encoded), &encoded_len) ||
+	    (sleep_feed_has(r->feed, (size_t)proof->block) &&
+	     sleep_feed_read(r->feed, (size_t)proof->block, &old, &old_len))) return -1;
+	if (old) {
+		int same = old_len == len && !memcmp(old, block, len);
+		free(old);
+		if (!same) { errno = EBADMSG; return -1; }
+	} else if (sleep_feed_append_remote(r->feed, (size_t)proof->block, block, len,
+					    encoded, encoded_len)) return -1;
+	return sleep_replication_have(r, proof->block, 1);
+}

extra/dat-dpi/src/sleep_tree.c

@@ -0,0 +1,287 @@
+/*
+ * SLEEP Merkle tree.
+ *
+ * The on-disk tree is flat-tree/in-order notation:
+ *
+ *          3
+ *       1     5
+ *      0 2   4 6
+ *
+ * even indexes = leaves
+ * odd indexes  = parent nodes
+ *
+ * SLEEP adds domain-separated LEAF/PARENT/ROOT hashing and stores
+ * aggregate byte lengths in every tree entry.
+ */
+#include "sleep.h"
+
+#include <errno.h>
+#include <stdint.h>
+#include <stdlib.h>
+#include <string.h>
+
+#ifdef SLEEP_SALT
+#include <sodium.h>
+#endif
+
+static unsigned int sleep_tree_ctz64(uint64_t x)
+{
+#if defined(__GNUC__) || defined(__clang__)
+	if (x == 0)
+		return 64;
+	return (unsigned int)__builtin_ctzll(x);
+#else
+	unsigned int n = 0;
+
+	if (x == 0)
+		return 64;
+
+	while ((x & 1ULL) == 0) {
+		x >>= 1;
+		n++;
+	}
+
+	return n;
+#endif
+}
+
+uint64_t sleep_tree_index(uint64_t depth, uint64_t offset)
+{
+	if (depth >= 63) {
+		errno = EOVERFLOW;
+		return UINT64_MAX;
+	}
+
+	return ((1ULL + (2ULL * offset)) << depth) - 1ULL;
+}
+
+uint64_t sleep_tree_depth(uint64_t index)
+{
+	return sleep_tree_ctz64(index + 1ULL);
+}
+
+uint64_t sleep_tree_offset(uint64_t index, uint64_t depth)
+{
+	if ((index & 1ULL) == 0)
+		return index / 2ULL;
+
+	if (depth == UINT64_MAX)
+		depth = sleep_tree_depth(index);
+
+	return (((index + 1ULL) >> depth) - 1ULL) / 2ULL;
+}
+
+uint64_t sleep_tree_parent(uint64_t index)
+{
+	uint64_t depth = sleep_tree_depth(index);
+	uint64_t offset = sleep_tree_offset(index, depth);
+
+	return sleep_tree_index(depth + 1ULL, offset >> 1);
+}
+
+int sleep_tree_sibling(uint64_t index, uint64_t *out)
+{
+	uint64_t depth;
+	uint64_t offset;
+
+	if (out == NULL) {
+		errno = EINVAL;
+		return -1;
+	}
+
+	depth = sleep_tree_depth(index);
+	offset = sleep_tree_offset(index, depth);
+
+	*out = sleep_tree_index(depth, offset & 1ULL ? offset - 1ULL
+	                                             : offset + 1ULL);
+	return 0;
+}
+
+int sleep_tree_left_child(uint64_t index, uint64_t *out)
+{
+	uint64_t depth;
+	uint64_t offset;
+
+	if (out == NULL) {
+		errno = EINVAL;
+		return -1;
+	}
+
+	if ((index & 1ULL) == 0) {
+		errno = EINVAL;
+		return -1;
+	}
+
+	depth = sleep_tree_depth(index);
+	if (depth == 0) {
+		errno = EINVAL;
+		return -1;
+	}
+
+	offset = sleep_tree_offset(index, depth);
+	*out = sleep_tree_index(depth - 1ULL, offset * 2ULL);
+	return 0;
+}
+
+int sleep_tree_right_child(uint64_t index, uint64_t *out)
+{
+	uint64_t depth;
+	uint64_t offset;
+
+	if (out == NULL) {
+		errno = EINVAL;
+		return -1;
+	}
+
+	if ((index & 1ULL) == 0) {
+		errno = EINVAL;
+		return -1;
+	}
+
+	depth = sleep_tree_depth(index);
+	if (depth == 0) {
+		errno = EINVAL;
+		return -1;
+	}
+
+	offset = sleep_tree_offset(index, depth);
+	*out = sleep_tree_index(depth - 1ULL, offset * 2ULL + 1ULL);
+	return 0;
+}
+
+uint64_t sleep_tree_left_span(uint64_t index)
+{
+	uint64_t depth;
+
+	if ((index & 1ULL) == 0)
+		return index;
+
+	depth = sleep_tree_depth(index);
+	return sleep_tree_offset(index, depth) * (1ULL << (depth + 1ULL));
+}
+
+uint64_t sleep_tree_right_span(uint64_t index)
+{
+	uint64_t depth;
+
+	if ((index & 1ULL) == 0)
+		return index;
+
+	depth = sleep_tree_depth(index);
+	return (sleep_tree_offset(index, depth) + 1ULL) *
+	       (1ULL << (depth + 1ULL)) - 2ULL;
+}
+
+void sleep_tree_spans(uint64_t index, uint64_t *left, uint64_t *right)
+{
+	if (left != NULL)
+		*left = sleep_tree_left_span(index);
+	if (right != NULL)
+		*right = sleep_tree_right_span(index);
+}
+
+size_t sleep_tree_count(uint64_t index)
+{
+	uint64_t depth;
+
+	if ((index & 1ULL) == 0)
+		return 1;
+
+	depth = sleep_tree_depth(index);
+	if (depth >= sizeof(size_t) * 8 - 1)
+		return SIZE_MAX;
+
+	return ((size_t)1 << (depth + 1ULL)) - 1;
+}
+
+size_t sleep_tree_count_leaves(uint64_t index)
+{
+	return (sleep_tree_count(index) + 1) / 2;
+}
+
+size_t sleep_tree_full_roots(uint64_t index, uint64_t *out, size_t capacity)
+{
+	uint64_t blocks;
+	uint64_t offset = 0;
+	uint64_t factor = 1;
+	size_t count = 0;
+
+	if (index & 1ULL) {
+		errno = EINVAL;
+		return 0;
+	}
+
+	/* mafintosh/flat-tree fullRoots(index) works on index / 2. */
+	blocks = index / 2ULL;
+
+	while (blocks != 0) {
+		while (factor <= UINT64_MAX / 2ULL && factor * 2ULL <= blocks)
+			factor *= 2ULL;
+
+		if (out != NULL && count < capacity)
+			out[count] = offset + factor - 1ULL;
+
+		count++;
+		offset += 2ULL * factor;
+		blocks -= factor;
+		factor = 1;
+	}
+
+	return count;
+}
+
+#ifdef SLEEP_SALT
+int sleep_tree_hash_block(const uint8_t *data, size_t len,
+                          uint8_t out_hash[32])
+{
+	uint8_t header[9];
+	crypto_generichash_state state;
+
+	if (out_hash == NULL || (data == NULL && len != 0)) {
+		errno = EINVAL;
+		return -1;
+	}
+
+	header[0] = SLEEP_HASH_LEAF;
+	for (int i = 0; i < 8; i++)
+		header[1 + i] = (uint8_t)((uint64_t)len >> (56 - i * 8));
+
+	if (crypto_generichash_init(&state, NULL, 0, 32) != 0)
+		return -1;
+	if (crypto_generichash_update(&state, header, sizeof(header)) != 0)
+		return -1;
+	if (len != 0 && crypto_generichash_update(&state, data, len) != 0)
+		return -1;
+
+	return crypto_generichash_final(&state, out_hash, 32);
+}
+
+int sleep_tree_hash_parent(const uint8_t left[32],
+                           const uint8_t right[32],
+                           uint64_t length,
+                           uint8_t out_hash[32])
+{
+	uint8_t header[9];
+	crypto_generichash_state state;
+
+	if (left == NULL || right == NULL || out_hash == NULL) {
+		errno = EINVAL;
+		return -1;
+	}
+
+	header[0] = SLEEP_HASH_PARENT;
+	for (int i = 0; i < 8; i++)
+		header[1 + i] = (uint8_t)(length >> (56 - i * 8));
+
+	if (crypto_generichash_init(&state, NULL, 0, 32) != 0)
+		return -1;
+	if (crypto_generichash_update(&state, header, sizeof(header)) != 0)
+		return -1;
+	if (crypto_generichash_update(&state, left, 32) != 0)
+		return -1;
+	if (crypto_generichash_update(&state, right, 32) != 0)
+		return -1;
+
+	return crypto_generichash_final(&state, out_hash, 32);
+}
+#endif

extra/dat-dpi/src/sleep_utp.c

@@ -0,0 +1,114 @@
+// hyperswarm dht
+#include "sleep_utp.h"
+#include <errno.h>
+#include <stdlib.h>
+#include <string.h>
+#include "dat_socket.h"
+static void put16(uint8_t*p,uint16_t v){p[0]=(uint8_t)(v>>8);p[1]=(uint8_t)v;}static void put32(uint8_t*p,uint32_t v){p[0]=(uint8_t)(v>>24);p[1]=(uint8_t)(v>>16);p[2]=(uint8_t)(v>>8);p[3]=(uint8_t)v;}static uint16_t get16(const uint8_t*p){return((uint16_t)p[0]<<8)|p[1];}static uint32_t get32(const uint8_t*p){return((uint32_t)p[0]<<24)|((uint32_t)p[1]<<16)|((uint32_t)p[2]<<8)|p[3];}
+int sleep_utp_init(sleep_utp_t*u,dat_socket_t fd){if(!u||fd==DAT_INVALID_SOCKET){errno=EINVAL;return-1;}memset(u,0,sizeof(*u));u->socket=fd;u->state=SLEEP_UTP_CLOSED;u->receive_window=1024*1024;u->send_window=SLEEP_UTP_WINDOW*SLEEP_UTP_MTU;u->congestion_window=2*SLEEP_UTP_MTU;u->slow_start_threshold=32*SLEEP_UTP_MTU;u->round_trip_time=500;u->round_trip_variance=250;u->send_id=(uint16_t)rand();u->receive_id=u->send_id+1;return 0;}
+int sleep_utp_packet_encode(const sleep_utp_packet_t*p,uint8_t*out,size_t cap,size_t*written){size_t off=20;if(!p||!out||(!p->payload&&p->payload_len)){errno=EINVAL;return-1;}if(p->type>SLEEP_UTP_SYN||p->version!=1){errno=EPROTO;return-1;}if(p->type==SLEEP_UTP_DATA&&p->payload_len>SLEEP_UTP_MTU){errno=EMSGSIZE;return-1;}if(cap<20+(p->has_selective_ack?6:0)+p->payload_len){errno=ENOSPC;return-1;}out[0]=(uint8_t)((p->type<<4)|(p->version&15));out[1]=p->has_selective_ack?1:0;put16(out+2,p->connection_id);put32(out+4,p->timestamp);put32(out+8,p->timestamp_diff);put32(out+12,p->window_size);put16(out+16,p->sequence);put16(out+18,p->acknowledgement);if(p->has_selective_ack){out[off++]=0;out[off++]=4;put32(out+off,p->selective_ack);off+=4;}if(p->payload_len)memcpy(out+off,p->payload,p->payload_len);if(written)*written=off+p->payload_len;return 0;}
+int sleep_utp_packet_decode(sleep_utp_packet_t *p, const uint8_t *in, size_t len)
+{
+	size_t off = 20;
+	uint8_t ext;
+	if (!p || !in || len < 20) { errno = EPROTO; return -1; }
+	memset(p, 0, sizeof(*p));
+	p->type = in[0] >> 4;
+	p->version = in[0] & 15;
+	if (p->type > SLEEP_UTP_SYN || p->version != 1) { errno = EPROTO; return -1; }
+	ext = in[1];
+	p->connection_id = get16(in + 2);
+	p->timestamp = get32(in + 4);
+	p->timestamp_diff = get32(in + 8);
+	p->window_size = get32(in + 12);
+	p->sequence = get16(in + 16);
+	p->acknowledgement = get16(in + 18);
+	while (ext) {
+		uint8_t next, n;
+		if (len - off < 2) { errno = EPROTO; return -1; }
+		next = in[off++];
+		n = in[off++];
+		if (n > len - off) { errno = EPROTO; return -1; }
+		if (ext == 1) {
+			if (n != 4 || p->has_selective_ack) { errno = EPROTO; return -1; }
+			p->has_selective_ack = 1;
+			p->selective_ack = get32(in + off);
+		}
+		off += n;
+		ext = next;
+	}
+	p->payload = in + off;
+	p->payload_len = len - off;
+	if ((p->type == SLEEP_UTP_DATA && p->payload_len > SLEEP_UTP_MTU) ||
+	    (p->type != SLEEP_UTP_DATA && p->payload_len != 0)) {
+		errno = p->type == SLEEP_UTP_DATA ? EMSGSIZE : EPROTO;
+		return -1;
+	}
+	return 0;
+}
+static int emit_sequence(sleep_utp_t*u,uint8_t type,const uint8_t*data,size_t len,uint16_t sequence,int advance,int track){sleep_utp_packet_t p;uint8_t packet[26+SLEEP_UTP_MTU];size_t n,i;sleep_utp_slot_t*tracked=NULL;if(track&&type==SLEEP_UTP_DATA){tracked=&u->unacked[sequence%SLEEP_UTP_WINDOW];if(tracked->used){errno=EAGAIN;return-1;}}memset(&p,0,sizeof(p));p.type=type;p.version=1;p.connection_id=u->send_id;p.window_size=u->receive_window;p.sequence=sequence;p.acknowledgement=u->acknowledgement;if(type==SLEEP_UTP_STATE){for(i=0;i<32;i++){uint16_t seq=(uint16_t)(u->acknowledgement+2+i);sleep_utp_slot_t*s=&u->reorder[seq%SLEEP_UTP_WINDOW];if(s->used&&s->sequence==seq)p.selective_ack|=(uint32_t)1<<i;}p.has_selective_ack=p.selective_ack!=0;}p.payload=data;p.payload_len=len;if(sleep_utp_packet_encode(&p,packet,sizeof(packet),&n))return-1;if(sendto(u->socket,packet,n,0,(struct sockaddr*)&u->remote,u->remote_len)!=(ssize_t)n)return-1;if(advance)u->sequence++;if(tracked){tracked->sequence=sequence;tracked->length=len;if(len)memcpy(tracked->data,data,len);tracked->used=1;tracked->sent_at=0;tracked->retries=0;u->bytes_in_flight+=len;}return 0;}
+static int emit(sleep_utp_t*u,uint8_t type,const uint8_t*data,size_t len){int advance=type==SLEEP_UTP_DATA||type==SLEEP_UTP_SYN||type==SLEEP_UTP_FIN;return emit_sequence(u,type,data,len,u->sequence,advance,type==SLEEP_UTP_DATA);}
+int sleep_utp_connect(sleep_utp_t*u,const struct sockaddr*r,socklen_t n){if(!u||!r||n>sizeof(u->remote)){errno=EINVAL;return-1;}memcpy(&u->remote,r,n);u->remote_len=n;u->state=SLEEP_UTP_SYN_SENT;u->retransmit_at=0;u->syn_retries=0;return emit(u,SLEEP_UTP_SYN,NULL,0);}
+int sleep_utp_accept(sleep_utp_t*u,const sleep_utp_packet_t*syn,const struct sockaddr*r,socklen_t n){if(!u||!syn||!r||n>sizeof(u->remote)){errno=EINVAL;return-1;}if(syn->type!=SLEEP_UTP_SYN||syn->version!=1){errno=EPROTO;return-1;}memcpy(&u->remote,r,n);u->remote_len=n;u->receive_id=syn->connection_id;u->send_id=(uint16_t)(syn->connection_id+1);u->sequence=(uint16_t)(syn->sequence+1);u->acknowledgement=syn->sequence;u->state=SLEEP_UTP_CONNECTED;return emit(u,SLEEP_UTP_STATE,NULL,0);}
+int sleep_utp_input(sleep_utp_t *u, const uint8_t *buf, size_t len)
+{
+	sleep_utp_packet_t packet;
+	size_t i;
+	if (!u) { errno = EINVAL; return -1; }
+	if (sleep_utp_packet_decode(&packet, buf, len)) return -1;
+	if (packet.version != 1 || packet.type > SLEEP_UTP_SYN ||
+	    packet.connection_id != u->receive_id) { errno = EPROTO; return -1; }
+	u->send_window = packet.window_size;
+	for (i = 0; i < SLEEP_UTP_WINDOW; i++) {
+		sleep_utp_slot_t *slot = &u->unacked[i];
+		uint16_t behind;
+		int acknowledged;
+		if (!slot->used) continue;
+		behind = (uint16_t)(packet.acknowledgement - slot->sequence);
+		acknowledged = behind < 0x8000;
+		if (!acknowledged && packet.has_selective_ack) {
+			uint16_t ahead = (uint16_t)(slot->sequence - (uint16_t)(packet.acknowledgement + 2));
+			if (ahead < 32 && (packet.selective_ack & ((uint32_t)1 << ahead))) acknowledged = 1;
+		}
+		if (!acknowledged) continue;
+		u->bytes_in_flight = slot->length > u->bytes_in_flight ? 0 : u->bytes_in_flight - slot->length;
+		slot->used = 0;
+		if (u->congestion_window < u->slow_start_threshold) u->congestion_window += SLEEP_UTP_MTU;
+		else u->congestion_window += SLEEP_UTP_MTU * SLEEP_UTP_MTU / u->congestion_window;
+	}
+	if (packet.type == SLEEP_UTP_STATE && u->state == SLEEP_UTP_SYN_SENT) {
+		u->state = SLEEP_UTP_CONNECTED;
+		return 0;
+	}
+	if (packet.type == SLEEP_UTP_DATA) {
+		uint16_t distance = (uint16_t)(packet.sequence - (uint16_t)(u->acknowledgement + 1));
+		sleep_utp_slot_t *slot;
+		if (distance >= 0x8000 || distance >= SLEEP_UTP_WINDOW)
+			return emit(u, SLEEP_UTP_STATE, NULL, 0);
+		slot = &u->reorder[packet.sequence % SLEEP_UTP_WINDOW];
+		if (!slot->used) {
+			slot->sequence = packet.sequence;
+			slot->length = packet.payload_len;
+			if (packet.payload_len) memcpy(slot->data, packet.payload, packet.payload_len);
+			slot->used = 1;
+		}
+		for (;;) {
+			uint16_t next = (uint16_t)(u->acknowledgement + 1);
+			slot = &u->reorder[next % SLEEP_UTP_WINDOW];
+			if (!slot->used || slot->sequence != next) break;
+			u->acknowledgement = next;
+			if (u->on_data) u->on_data(u, slot->data, slot->length);
+			slot->used = 0;
+		}
+		return emit(u, SLEEP_UTP_STATE, NULL, 0);
+	}
+	if (packet.type == SLEEP_UTP_FIN) {
+		u->state = SLEEP_UTP_CLOSED;
+		return emit(u, SLEEP_UTP_STATE, NULL, 0);
+	}
+	if (packet.type == SLEEP_UTP_RESET_PACKET) u->state = SLEEP_UTP_RESET;
+	return 0;
+}
+int sleep_utp_send(sleep_utp_t*u,const uint8_t*data,size_t len){size_t off=0,part,slots=0,i,needed;if(!u||u->state!=SLEEP_UTP_CONNECTED||(!data&&len)){errno=ENOTCONN;return-1;}needed=(len+SLEEP_UTP_MTU-1)/SLEEP_UTP_MTU;for(i=0;i<SLEEP_UTP_WINDOW;i++)if(!u->unacked[i].used)slots++;if(needed>slots||u->bytes_in_flight>u->congestion_window||len>u->congestion_window-u->bytes_in_flight||len>u->send_window){errno=EAGAIN;return-1;}while(off<len){part=len-off>SLEEP_UTP_MTU?SLEEP_UTP_MTU:len-off;if(emit(u,SLEEP_UTP_DATA,data+off,part))return-1;off+=part;}return 0;}
+int sleep_utp_tick(sleep_utp_t*u,uint64_t now){size_t i;if(!u){errno=EINVAL;return-1;}if(u->state==SLEEP_UTP_SYN_SENT){if(!u->retransmit_at){u->retransmit_at=now+250;}else if(now>=u->retransmit_at){if(u->syn_retries>=8){u->state=SLEEP_UTP_RESET;errno=ETIMEDOUT;return-1;}if(emit_sequence(u,SLEEP_UTP_SYN,NULL,0,(uint16_t)(u->sequence-1),0,0))return-1;u->syn_retries++;u->retransmit_at=now+250;}}for(i=0;i<SLEEP_UTP_WINDOW;i++){sleep_utp_slot_t*s=&u->unacked[i];if(!s->used){continue;}if(!s->sent_at){s->sent_at=now;continue;}if(now-s->sent_at>=u->round_trip_time+4*u->round_trip_variance){u->slow_start_threshold=u->congestion_window/2;u->congestion_window=SLEEP_UTP_MTU;s->sent_at=now;s->retries++;if(s->retries>8){u->state=SLEEP_UTP_RESET;errno=ETIMEDOUT;return-1;}if(emit_sequence(u,SLEEP_UTP_DATA,s->data,s->length,s->sequence,0,0))return-1;}}return 0;}
+int sleep_utp_close(sleep_utp_t*u){if(!u){errno=EINVAL;return-1;}if(u->state==SLEEP_UTP_CONNECTED){u->state=SLEEP_UTP_FIN_SENT;return emit(u,SLEEP_UTP_FIN,NULL,0);}u->state=SLEEP_UTP_CLOSED;return 0;}

extra/dat-dpi/src/sleep_verify.c

@@ -0,0 +1,589 @@
+#include "sleep.h"
+#include "sleep_verify.h"
+
+#include <errno.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#ifdef SLEEP_SALT
+#include <sodium.h>
+#endif
+
+
+static uint16_t
+be16(const uint8_t *p)
+{
+	return ((uint16_t)p[0] << 8) | p[1];
+}
+
+
+static uint32_t
+be32(const uint8_t *p)
+{
+	return ((uint32_t)p[0] << 24) |
+	       ((uint32_t)p[1] << 16) |
+	       ((uint32_t)p[2] << 8) |
+	       p[3];
+}
+
+
+static uint64_t
+be64(const uint8_t *p)
+{
+	return ((uint64_t)p[0] << 56) |
+	       ((uint64_t)p[1] << 48) |
+	       ((uint64_t)p[2] << 40) |
+	       ((uint64_t)p[3] << 32) |
+	       ((uint64_t)p[4] << 24) |
+	       ((uint64_t)p[5] << 16) |
+	       ((uint64_t)p[6] << 8) |
+	       p[7];
+}
+
+
+static int
+load_file(const char *path,
+	  uint8_t **buf,
+	  size_t *len)
+{
+	FILE *f;
+	long size;
+	uint8_t *p;
+
+	f = fopen(path, "rb");
+
+	if (!f) {
+		fprintf(stderr,
+			"sleep-verify: %s: %s\n",
+			path,
+			strerror(errno));
+		return -1;
+	}
+
+	if (fseek(f, 0, SEEK_END) != 0 ||
+	    (size = ftell(f)) < 0 ||
+	    fseek(f, 0, SEEK_SET) != 0) {
+		fclose(f);
+		return -1;
+	}
+
+	p = malloc(size ? (size_t)size : 1);
+
+	if (!p) {
+		fclose(f);
+		errno = ENOMEM;
+		return -1;
+	}
+
+	if (size &&
+	    fread(p, 1, (size_t)size, f) != (size_t)size) {
+		free(p);
+		fclose(f);
+		errno = EIO;
+		return -1;
+	}
+
+	fclose(f);
+
+	*buf = p;
+	*len = (size_t)size;
+
+	return 0;
+}
+
+
+static int
+check_header(const uint8_t *buf,
+	     size_t len,
+	     uint32_t magic,
+	     uint16_t entry_size,
+	     const char *algo)
+{
+	size_t algo_len;
+
+	if (len < SLEEP_HEADER_SIZE)
+		return -1;
+
+	if (be32(buf) != magic ||
+	    buf[4] != 0 ||
+	    be16(buf + 5) != entry_size)
+		return -1;
+
+	algo_len = buf[7];
+
+	if (algo_len != strlen(algo))
+		return -1;
+
+	return memcmp(buf + 8, algo, algo_len) == 0 ? 0 : -1;
+}
+
+
+static int
+verify_tree_structure(const sleep_tree_entry_t *tree,
+		      size_t node_count)
+{
+	size_t i;
+
+	for (i = 1; i < node_count; i += 2) {
+		uint64_t left;
+		uint64_t right;
+		uint64_t left_len;
+		uint64_t right_len;
+		uint64_t length;
+		uint8_t hash[32];
+
+		if (sleep_tree_left_child(i, &left) != 0 ||
+		    sleep_tree_right_child(i, &right) != 0)
+			return -1;
+
+		/*
+		 * This parent belongs to a subtree that does not
+		 * completely exist yet.
+		 */
+		if (right >= node_count) {
+			static const uint8_t zero[40];
+
+			if (memcmp(&tree[i],
+				   zero,
+				   sizeof(tree[i])) != 0) {
+				fprintf(stderr,
+					"  tree: nonzero incomplete node %zu\n",
+					i);
+				return -1;
+			}
+
+			continue;
+		}
+
+		left_len =
+			be64((const uint8_t *)&tree[left].length_be);
+
+		right_len =
+			be64((const uint8_t *)&tree[right].length_be);
+
+		if (UINT64_MAX - left_len < right_len)
+			return -1;
+
+		length = left_len + right_len;
+
+		if (sleep_tree_hash_parent(tree[left].hash,
+					   tree[right].hash,
+					   length,
+					   hash) != 0)
+			return -1;
+
+		if (memcmp(hash, tree[i].hash, 32) != 0 ||
+		    be64((const uint8_t *)&tree[i].length_be) != length) {
+			fprintf(stderr,
+				"  tree: parent mismatch at node %zu\n",
+				i);
+			return -1;
+		}
+	}
+
+	return 0;
+}
+
+
+static int
+verify_data_leaves(const uint8_t *data,
+		   size_t data_len,
+		   const sleep_tree_entry_t *tree,
+		   size_t node_count)
+{
+	size_t leaf_count = (node_count + 1) / 2;
+	size_t offset = 0;
+	size_t i;
+
+	for (i = 0; i < leaf_count; i++) {
+		size_t node = i * 2;
+		uint64_t len =
+			be64((const uint8_t *)&tree[node].length_be);
+		uint8_t hash[32];
+
+		if (len > data_len - offset) {
+			fprintf(stderr,
+				"  data: leaf %zu exceeds metadata.data\n",
+				i);
+			return -1;
+		}
+
+		if (sleep_tree_hash_block(data + offset,
+					  (size_t)len,
+					  hash) != 0)
+			return -1;
+
+		if (memcmp(hash, tree[node].hash, 32) != 0) {
+			fprintf(stderr,
+				"  data: leaf hash mismatch at %zu\n",
+				i);
+			return -1;
+		}
+
+		offset += (size_t)len;
+	}
+
+	if (offset != data_len) {
+		fprintf(stderr,
+			"  data: tree consumes %zu bytes, file is %zu\n",
+			offset,
+			data_len);
+		return -1;
+	}
+
+	return 0;
+}
+
+
+#ifdef SLEEP_SALT
+
+static int
+verify_signatures(const uint8_t *signatures,
+		  size_t signatures_len,
+		  const uint8_t pubkey[32],
+		  const sleep_tree_entry_t *tree,
+		  size_t node_count,
+		  size_t feed_length)
+{
+	size_t sequence;
+	uint64_t roots[64];
+	uint8_t message[1 + 64 * 48];
+	uint8_t digest[32];
+
+	if (signatures_len < SLEEP_HEADER_SIZE ||
+	    (signatures_len - SLEEP_HEADER_SIZE) %
+	    SLEEP_SIG_ENTRY_SIZE != 0)
+		return -1;
+
+	if ((signatures_len - SLEEP_HEADER_SIZE) /
+	    SLEEP_SIG_ENTRY_SIZE != feed_length)
+		return -1;
+
+	for (sequence = 1;
+	     sequence <= feed_length;
+	     sequence++) {
+		const uint8_t *signature = signatures + SLEEP_HEADER_SIZE +
+			(sequence - 1) * SLEEP_SIG_ENTRY_SIZE;
+		size_t root_count;
+		size_t off = 0;
+		size_t r;
+		size_t signature_byte;
+		int signature_present = 0;
+
+		for (signature_byte = 0;
+		     signature_byte < SLEEP_SIG_ENTRY_SIZE;
+		     signature_byte++)
+			signature_present |= signature[signature_byte] != 0;
+
+		/* Sparse content feeds retain empty signature slots. */
+		if (!signature_present)
+			continue;
+
+		root_count =
+			sleep_tree_full_roots((uint64_t)sequence * 2,
+					      roots,
+					      64);
+
+		message[off++] = SLEEP_HASH_ROOT;
+
+		for (r = 0; r < root_count; r++) {
+			uint64_t index = roots[r];
+			uint64_t length;
+
+			if (index >= node_count)
+				return -1;
+
+			memcpy(message + off,
+			       tree[index].hash,
+			       32);
+			off += 32;
+
+			message[off++] = (uint8_t)(index >> 56);
+			message[off++] = (uint8_t)(index >> 48);
+			message[off++] = (uint8_t)(index >> 40);
+			message[off++] = (uint8_t)(index >> 32);
+			message[off++] = (uint8_t)(index >> 24);
+			message[off++] = (uint8_t)(index >> 16);
+			message[off++] = (uint8_t)(index >> 8);
+			message[off++] = (uint8_t)index;
+
+			length =
+				be64((const uint8_t *)
+				     &tree[index].length_be);
+
+			message[off++] = (uint8_t)(length >> 56);
+			message[off++] = (uint8_t)(length >> 48);
+			message[off++] = (uint8_t)(length >> 40);
+			message[off++] = (uint8_t)(length >> 32);
+			message[off++] = (uint8_t)(length >> 24);
+			message[off++] = (uint8_t)(length >> 16);
+			message[off++] = (uint8_t)(length >> 8);
+			message[off++] = (uint8_t)length;
+		}
+
+		if (crypto_generichash(digest,
+				       sizeof(digest),
+				       message,
+				       off,
+				       NULL,
+				       0) != 0)
+			return -1;
+
+		if (crypto_sign_verify_detached(
+			signature,
+			digest,
+			sizeof(digest),
+			pubkey) != 0) {
+			fprintf(stderr,
+				"  signatures: failed at sequence %zu\n",
+				sequence);
+			return -1;
+		}
+	}
+
+	return 0;
+}
+
+#endif
+
+
+static int
+verify_feed(const char *dir,
+	    const char *name,
+	    int verify_data,
+	    size_t *out_length)
+{
+	char path[4096];
+
+	uint8_t *tree_file = NULL;
+	uint8_t *sig_file = NULL;
+	uint8_t *key_file = NULL;
+	uint8_t *data_file = NULL;
+	uint8_t *bitfield = NULL;
+
+	size_t tree_len = 0;
+	size_t sig_len = 0;
+	size_t key_len = 0;
+	size_t data_len = 0;
+	size_t bitfield_len = 0;
+
+	size_t node_count;
+	size_t feed_length;
+
+	int rc = -1;
+
+	snprintf(path,
+		 sizeof(path),
+		 "%s/%s.tree",
+		 dir,
+		 name);
+
+	if (load_file(path, &tree_file, &tree_len) != 0)
+		goto done;
+
+	if (check_header(tree_file,
+			 tree_len,
+			 SLEEP_MAGIC_TREE,
+			 SLEEP_TREE_ENTRY_SIZE,
+			 SLEEP_ALGO_BLAKE2B) != 0) {
+		fprintf(stderr, "  tree: invalid header\n");
+		goto done;
+	}
+
+	if ((tree_len - SLEEP_HEADER_SIZE) %
+	    sizeof(sleep_tree_entry_t) != 0)
+		goto done;
+
+	node_count =
+		(tree_len - SLEEP_HEADER_SIZE) /
+		sizeof(sleep_tree_entry_t);
+
+	if (!node_count || !(node_count & 1))
+		goto done;
+
+	feed_length = (node_count + 1) / 2;
+
+	if (verify_tree_structure(
+		(const sleep_tree_entry_t *)(tree_file +
+					     SLEEP_HEADER_SIZE),
+		node_count) != 0)
+		goto done;
+
+	snprintf(path,
+		 sizeof(path),
+		 "%s/%s.key",
+		 dir,
+		 name);
+
+	if (load_file(path, &key_file, &key_len) != 0 ||
+	    key_len != 32)
+		goto done;
+
+	snprintf(path,
+		 sizeof(path),
+		 "%s/%s.signatures",
+		 dir,
+		 name);
+
+	if (load_file(path, &sig_file, &sig_len) != 0)
+		goto done;
+
+	if (check_header(sig_file,
+			 sig_len,
+			 SLEEP_MAGIC_SIGNATURE,
+			 SLEEP_SIG_ENTRY_SIZE,
+			 SLEEP_ALGO_ED25519) != 0)
+		goto done;
+
+#ifdef SLEEP_SALT
+
+	if (verify_signatures(
+		sig_file,
+		sig_len,
+		key_file,
+		(const sleep_tree_entry_t *)(tree_file +
+					     SLEEP_HEADER_SIZE),
+		node_count,
+		feed_length) != 0)
+		goto done;
+
+#else
+
+	fprintf(stderr,
+		"  signatures: build with SLEEP_SALT\n");
+	goto done;
+
+#endif
+
+	snprintf(path,
+		 sizeof(path),
+		 "%s/%s.bitfield",
+		 dir,
+		 name);
+
+	if (load_file(path,
+		      &bitfield,
+		      &bitfield_len) != 0)
+		goto done;
+
+	if (bitfield_len < SLEEP_HEADER_SIZE ||
+	    be32(bitfield) != SLEEP_MAGIC_BITFIELD ||
+	    bitfield[4] != 0)
+		goto done;
+
+	/*
+	 * The SLEEP V2 document says 3328 here.
+	 *
+	 * The supplied historical Dat fixture says 3584.
+	 * We deliberately accept both until the extra 256 bytes
+	 * in the historical format have been identified.
+	 */
+	if (be16(bitfield + 5) != SLEEP_BITFIELD_ENTRY_SIZE &&
+	    be16(bitfield + 5) != 3584)
+		goto done;
+
+	if (verify_data) {
+		snprintf(path,
+			 sizeof(path),
+			 "%s/%s.data",
+			 dir,
+			 name);
+
+		if (load_file(path,
+			      &data_file,
+			      &data_len) != 0)
+			goto done;
+
+		if (verify_data_leaves(
+			data_file,
+			data_len,
+			(const sleep_tree_entry_t *)(tree_file +
+						     SLEEP_HEADER_SIZE),
+			node_count) != 0)
+			goto done;
+	}
+
+	if (out_length)
+		*out_length = feed_length;
+
+	rc = 0;
+
+done:
+	free(tree_file);
+	free(sig_file);
+	free(key_file);
+	free(data_file);
+	free(bitfield);
+
+	return rc;
+}
+
+
+int
+sleep_verify_dat(const char *dir)
+{
+	size_t metadata_length;
+	size_t content_length;
+
+	printf("sleep-verify: %s\n", dir);
+
+	printf("metadata: ");
+
+	if (verify_feed(dir,
+			"metadata",
+			1,
+			&metadata_length) != 0) {
+		printf("FAIL\n");
+		return -1;
+	}
+
+	printf("OK (%zu entries)\n", metadata_length);
+
+	printf("content:  ");
+
+	if (verify_feed(dir,
+			"content",
+			0,
+			&content_length) != 0) {
+		printf("FAIL\n");
+		return -1;
+	}
+
+	printf("OK (%zu entries)\n", content_length);
+
+	printf("sleep-verify: PASS\n");
+
+	return 0;
+}
+
+
+#ifndef SLEEP_VERIFY_NO_MAIN
+int
+main(int argc, char **argv)
+{
+	if (argc != 2) {
+		fprintf(stderr,
+			"usage: %s .dat\n",
+			argv[0]);
+		return 2;
+	}
+
+#ifdef SLEEP_SALT
+
+	if (sodium_init() < 0)
+		return 1;
+
+#else
+
+	fprintf(stderr,
+		"sleep-verify: built without SLEEP_SALT\n");
+
+	return 1;
+
+#endif
+
+	return sleep_verify_dat(argv[1]) == 0 ? 0 : 1;
+}
+#endif

extra/dat-dpi/src/sleep_writer.c

@@ -0,0 +1,530 @@
+#include "sleep_writer.h"
+#include "sleep.h"
+
+#include <errno.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#ifdef SLEEP_SALT
+#include <sodium.h>
+#endif
+
+#define SLEEP_MAGIC_BITFIELD 0x05025700u
+#define SLEEP_MAGIC_SIGNATURE 0x05025701u
+#define SLEEP_MAGIC_TREE 0x05025702u
+
+#define SLEEP_BITFIELD_SIZE 3584u
+#define SLEEP_DATA_BITS 8192u
+#define SLEEP_TREE_BITS 16384u
+#define SLEEP_INDEX_BYTES 256u
+
+#define SLEEP_HASH_BYTES 32u
+#define SLEEP_STACK_MAX 64u
+
+typedef struct writer_node {
+	uint8_t hash[SLEEP_HASH_BYTES];
+	uint64_t length;
+	uint64_t index;
+	uint64_t depth;
+} writer_node_t;
+
+typedef struct writer_peak {
+	uint64_t index;
+	uint64_t depth;
+} writer_peak_t;
+
+static void put_u16be(uint8_t *p, uint16_t v)
+{
+	p[0] = (uint8_t)(v >> 8);
+	p[1] = (uint8_t)v;
+}
+
+static void put_u32be(uint8_t *p, uint32_t v)
+{
+	p[0] = (uint8_t)(v >> 24);
+	p[1] = (uint8_t)(v >> 16);
+	p[2] = (uint8_t)(v >> 8);
+	p[3] = (uint8_t)v;
+}
+
+static void put_u64be(uint8_t *p, uint64_t v)
+{
+	p[0] = (uint8_t)(v >> 56);
+	p[1] = (uint8_t)(v >> 48);
+	p[2] = (uint8_t)(v >> 40);
+	p[3] = (uint8_t)(v >> 32);
+	p[4] = (uint8_t)(v >> 24);
+	p[5] = (uint8_t)(v >> 16);
+	p[6] = (uint8_t)(v >> 8);
+	p[7] = (uint8_t)v;
+}
+
+static int make_path(char *out, size_t out_size,
+                     const char *dir, const char *name,
+                     const char *suffix)
+{
+	int n = snprintf(out, out_size, "%s/%s%s", dir, name, suffix);
+	if (n < 0 || (size_t)n >= out_size) {
+		errno = ENAMETOOLONG;
+		return -1;
+	}
+	return 0;
+}
+
+static int write_header(FILE *f, uint32_t magic,
+                        uint16_t entry_size, const char *algo)
+{
+	uint8_t header[SLEEP_HEADER_SIZE];
+	size_t len = algo ? strlen(algo) : 0;
+
+	if (len > 24) {
+		errno = EINVAL;
+		return -1;
+	}
+
+	memset(header, 0, sizeof(header));
+	put_u32be(header, magic);
+	put_u16be(header + 5, entry_size);
+	header[7] = (uint8_t)len;
+	if (len)
+		memcpy(header + 8, algo, len);
+
+	if (fwrite(header, 1, sizeof(header), f) != sizeof(header)) {
+		errno = EIO;
+		return -1;
+	}
+	return 0;
+}
+
+static int hash_leaf(const uint8_t *data, size_t len,
+                     uint8_t out[32])
+{
+#ifdef SLEEP_SALT
+	uint8_t head[9];
+	crypto_generichash_state st;
+
+	head[0] = 0;
+	put_u64be(head + 1, (uint64_t)len);
+
+	if (crypto_generichash_init(&st, NULL, 0, 32) != 0)
+		return -1;
+	if (crypto_generichash_update(&st, head, sizeof(head)) != 0)
+		return -1;
+	if (len && crypto_generichash_update(&st, data, len) != 0)
+		return -1;
+	return crypto_generichash_final(&st, out, 32);
+#else
+	(void)data; (void)len; (void)out;
+	errno = ENOSYS;
+	return -1;
+#endif
+}
+
+static int hash_parent(const writer_node_t *left,
+                       const writer_node_t *right,
+                       uint8_t out[32])
+{
+#ifdef SLEEP_SALT
+	uint8_t head[9];
+	crypto_generichash_state st;
+
+	head[0] = 1;
+	put_u64be(head + 1, left->length + right->length);
+
+	if (crypto_generichash_init(&st, NULL, 0, 32) != 0)
+		return -1;
+	if (crypto_generichash_update(&st, head, sizeof(head)) != 0)
+		return -1;
+	if (crypto_generichash_update(&st, left->hash, 32) != 0)
+		return -1;
+	if (crypto_generichash_update(&st, right->hash, 32) != 0)
+		return -1;
+	return crypto_generichash_final(&st, out, 32);
+#else
+	(void)left; (void)right; (void)out;
+	errno = ENOSYS;
+	return -1;
+#endif
+}
+
+static int tree_build(const uint8_t *const *blocks,
+                      const size_t *lengths,
+                      size_t nblocks,
+                      writer_node_t **out_nodes,
+                      size_t *out_count)
+{
+	writer_node_t *nodes;
+	writer_peak_t stack[SLEEP_STACK_MAX];
+	size_t stack_len = 0;
+	size_t node_count;
+	size_t i;
+
+	if (nblocks > SIZE_MAX / 2 + 1) {
+		errno = EOVERFLOW;
+		return -1;
+	}
+
+	node_count = nblocks ? nblocks * 2 - 1 : 0;
+	nodes = node_count ? calloc(node_count, sizeof(*nodes)) : NULL;
+	if (node_count && !nodes) {
+		errno = ENOMEM;
+		return -1;
+	}
+
+	for (i = 0; i < nblocks; i++) {
+		writer_node_t *leaf;
+		uint64_t leaf_index = (uint64_t)(i * 2);
+
+		leaf = &nodes[leaf_index];
+		leaf->index = leaf_index;
+		leaf->depth = 0;
+		leaf->length = lengths[i];
+
+		if (hash_leaf(blocks[i], lengths[i], leaf->hash) != 0)
+			goto fail;
+
+		stack[stack_len].index = leaf_index;
+		stack[stack_len].depth = 0;
+		stack_len++;
+
+		while (stack_len >= 2 &&
+		       stack[stack_len - 1].depth == stack[stack_len - 2].depth) {
+			writer_peak_t right = stack[stack_len - 1];
+			writer_peak_t left = stack[stack_len - 2];
+			writer_node_t *ln = &nodes[left.index];
+			writer_node_t *rn = &nodes[right.index];
+			uint64_t parent_index;
+			writer_node_t *parent;
+
+			stack_len -= 2;
+			parent_index = sleep_tree_parent(left.index);
+			if (parent_index >= node_count) {
+				errno = EINVAL;
+				goto fail;
+			}
+
+			parent = &nodes[parent_index];
+			parent->index = parent_index;
+			parent->depth = left.depth + 1;
+			parent->length = ln->length + rn->length;
+
+			if (hash_parent(ln, rn, parent->hash) != 0)
+				goto fail;
+
+			stack[stack_len].index = parent_index;
+			stack[stack_len].depth = parent->depth;
+			stack_len++;
+		}
+	}
+
+	*out_nodes = nodes;
+	*out_count = node_count;
+	return 0;
+
+fail:
+	free(nodes);
+	return -1;
+}
+
+static int write_tree(const char *path,
+                      const writer_node_t *nodes, size_t node_count)
+{
+	FILE *f;
+	size_t i;
+
+	f = fopen(path, "wb");
+	if (!f)
+		return -1;
+
+	if (write_header(f, SLEEP_MAGIC_TREE,
+	                 SLEEP_TREE_ENTRY_SIZE, SLEEP_ALGO_BLAKE2B) != 0)
+		goto fail;
+
+	for (i = 0; i < node_count; i++) {
+		uint8_t entry[40];
+		memcpy(entry, nodes[i].hash, 32);
+		put_u64be(entry + 32, nodes[i].length);
+		if (fwrite(entry, 1, sizeof(entry), f) != sizeof(entry)) {
+			errno = EIO;
+			goto fail;
+		}
+	}
+
+	return fclose(f) == 0 ? 0 : -1;
+fail:
+	fclose(f);
+	return -1;
+}
+
+static int sign_state(const writer_node_t *nodes,
+                      size_t block_count,
+                      const sleep_keypair_t *keypair,
+                      uint8_t signature[64])
+{
+#ifdef SLEEP_SALT
+	uint64_t root_indexes[64];
+	size_t root_count;
+	size_t i;
+	size_t msg_len;
+	uint8_t *msg;
+	uint8_t digest[32];
+	crypto_generichash_state st;
+
+	/* The Dat/SLEEP root set is fullRoots(2 * length). */
+	root_count = sleep_tree_full_roots((uint64_t)(2 * block_count),
+	                                   root_indexes, 64);
+
+	if (root_count > (SIZE_MAX - 1) / 48) {
+		errno = EOVERFLOW;
+		return -1;
+	}
+	msg_len = 1 + root_count * 48;
+	msg = malloc(msg_len);
+	if (!msg) {
+		errno = ENOMEM;
+		return -1;
+	}
+
+	msg[0] = 2;
+	for (i = 0; i < root_count; i++) {
+		const writer_node_t *root = &nodes[root_indexes[i]];
+		size_t off = 1 + i * 48;
+
+		memcpy(msg + off, root->hash, 32);
+		put_u64be(msg + off + 32, root->index);
+		put_u64be(msg + off + 40, root->length);
+	}
+
+	if (crypto_generichash_init(&st, NULL, 0, 32) != 0)
+		goto fail;
+	if (crypto_generichash_update(&st, msg, msg_len) != 0)
+		goto fail;
+	if (crypto_generichash_final(&st, digest, 32) != 0)
+		goto fail;
+
+	free(msg);
+
+	if (crypto_sign_detached(signature, NULL,
+	                         digest, sizeof(digest),
+	                         keypair->seckey) != 0)
+		return -1;
+
+	return 0;
+
+fail:
+	free(msg);
+	return -1;
+#else
+	(void)nodes; (void)block_count; (void)keypair; (void)signature;
+	errno = ENOSYS;
+	return -1;
+#endif
+}
+
+static int write_signatures(const char *path,
+                            const writer_node_t *nodes,
+                            size_t nblocks,
+                            const sleep_keypair_t *keypair)
+{
+	FILE *f;
+	size_t length;
+
+	f = fopen(path, "wb");
+	if (!f)
+		return -1;
+
+	if (write_header(f, SLEEP_MAGIC_SIGNATURE,
+	                 SLEEP_SIG_ENTRY_SIZE, SLEEP_ALGO_ED25519) != 0)
+		goto fail;
+
+	for (length = 1; length <= nblocks; length++) {
+		uint8_t sig[64];
+
+		if (sign_state(nodes, length, keypair, sig) != 0)
+			goto fail;
+		if (fwrite(sig, 1, sizeof(sig), f) != sizeof(sig)) {
+			errno = EIO;
+			goto fail;
+		}
+	}
+
+	return fclose(f) == 0 ? 0 : -1;
+fail:
+	fclose(f);
+	return -1;
+}
+
+static void set_bit(uint8_t *bits, size_t bit)
+{
+	bits[bit >> 3] |= (uint8_t)(1u << (7 - (bit & 7)));
+}
+
+static unsigned int byte_state(uint8_t v)
+{
+	if (v == 0x00)
+		return 0;
+	if (v == 0xff)
+		return 3;
+	return 2;
+}
+
+static void build_index(uint8_t index[SLEEP_INDEX_BYTES],
+                        const uint8_t data[1024])
+{
+	size_t pair;
+
+	memset(index, 0, SLEEP_INDEX_BYTES);
+
+	/* Four index bits describe every two bytes of the data bitfield. */
+	for (pair = 0; pair < 512; pair++) {
+		unsigned int a = byte_state(data[pair * 2]);
+		unsigned int b = byte_state(data[pair * 2 + 1]);
+		unsigned int tuple = a | (b << 2);
+		size_t byte = pair >> 1;
+		index[byte] |= (uint8_t)(tuple << ((pair & 1) ? 4 : 0));
+	}
+}
+
+static int write_bitfield(const char *path,
+                          size_t nblocks,
+                          size_t node_count)
+{
+	uint8_t entry[SLEEP_BITFIELD_SIZE];
+	uint8_t *data = entry;
+	uint8_t *tree = entry + 1024;
+	uint8_t *index = entry + 3072;
+	FILE *f;
+	size_t i;
+
+	memset(entry, 0, sizeof(entry));
+
+	if (nblocks > SLEEP_DATA_BITS || node_count > SLEEP_TREE_BITS) {
+		errno = EOVERFLOW;
+		return -1;
+	}
+
+	for (i = 0; i < nblocks; i++)
+		set_bit(data, i);
+
+	for (i = 0; i < node_count; i++)
+		set_bit(tree, i);
+
+	build_index(index, data);
+
+	f = fopen(path, "wb");
+	if (!f)
+		return -1;
+
+	if (write_header(f, SLEEP_MAGIC_BITFIELD,
+	                 SLEEP_BITFIELD_SIZE, NULL) != 0)
+		goto fail;
+	if (fwrite(entry, 1, sizeof(entry), f) != sizeof(entry)) {
+		errno = EIO;
+		goto fail;
+	}
+
+	return fclose(f) == 0 ? 0 : -1;
+fail:
+	fclose(f);
+	return -1;
+}
+
+static int write_data(const char *path,
+                      const uint8_t *const *blocks,
+                      const size_t *lengths,
+                      size_t nblocks)
+{
+	FILE *f;
+	size_t i;
+
+	f = fopen(path, "wb");
+	if (!f)
+		return -1;
+
+	for (i = 0; i < nblocks; i++) {
+		if (lengths[i] &&
+		    fwrite(blocks[i], 1, lengths[i], f) != lengths[i]) {
+			errno = EIO;
+			goto fail;
+		}
+	}
+
+	return fclose(f) == 0 ? 0 : -1;
+fail:
+	fclose(f);
+	return -1;
+}
+
+static int write_key(const char *path, const sleep_keypair_t *keypair)
+{
+	FILE *f = fopen(path, "wb");
+	if (!f)
+		return -1;
+	if (fwrite(keypair->pubkey, 1, 32, f) != 32) {
+		errno = EIO;
+		fclose(f);
+		return -1;
+	}
+	return fclose(f) == 0 ? 0 : -1;
+}
+
+int sleep_write_feed(const char *feed_dir,
+                     const char *feed_name,
+                     const uint8_t *const *blocks,
+                     const size_t *lengths,
+                     size_t nblocks,
+                     const sleep_keypair_t *keypair)
+{
+	char path[1024];
+	writer_node_t *nodes = NULL;
+	size_t node_count = 0, i;
+	int rc = -1;
+
+	if (!feed_dir || !feed_name || !keypair ||
+	    !feed_name[0] || !strcmp(feed_name, ".") || !strcmp(feed_name, "..") ||
+	    strchr(feed_name, '/') || strchr(feed_name, '\\') ||
+	    (nblocks && (!blocks || !lengths))) {
+		errno = EINVAL;
+		return -1;
+	}
+	for (i = 0; i < nblocks; i++) {
+		if (lengths[i] && !blocks[i]) { errno = EINVAL; return -1; }
+	}
+
+	if (make_path(path, sizeof(path), feed_dir, feed_name, ".tree") != 0)
+		return -1;
+
+	if (tree_build(blocks, lengths, nblocks, &nodes, &node_count) != 0)
+		return -1;
+
+	if (write_tree(path, nodes, node_count) != 0)
+		goto done;
+
+	if (make_path(path, sizeof(path), feed_dir, feed_name, ".signatures") != 0)
+		goto done;
+	if (write_signatures(path, nodes, nblocks, keypair) != 0)
+		goto done;
+
+	if (make_path(path, sizeof(path), feed_dir, feed_name, ".key") != 0)
+		goto done;
+	if (write_key(path, keypair) != 0)
+		goto done;
+
+	if (make_path(path, sizeof(path), feed_dir, feed_name, ".bitfield") != 0)
+		goto done;
+	if (write_bitfield(path, nblocks, node_count) != 0)
+		goto done;
+
+	if (make_path(path, sizeof(path), feed_dir, feed_name, ".data") != 0)
+		goto done;
+	if (write_data(path, blocks, lengths, nblocks) != 0)
+		goto done;
+
+	rc = 0;
+
+done:
+	free(nodes);
+	return rc;
+}

extra/sleep_verify.c

@@ -0,0 +1,576 @@
+#include "sleep.h"
+#include "sleep_verify.h"
+
+#include <errno.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#ifdef SLEEP_SALT
+#include <sodium.h>
+#endif
+
+
+static uint16_t
+be16(const uint8_t *p)
+{
+	return ((uint16_t)p[0] << 8) | p[1];
+}
+
+
+static uint32_t
+be32(const uint8_t *p)
+{
+	return ((uint32_t)p[0] << 24) |
+	       ((uint32_t)p[1] << 16) |
+	       ((uint32_t)p[2] << 8) |
+	       p[3];
+}
+
+
+static uint64_t
+be64(const uint8_t *p)
+{
+	return ((uint64_t)p[0] << 56) |
+	       ((uint64_t)p[1] << 48) |
+	       ((uint64_t)p[2] << 40) |
+	       ((uint64_t)p[3] << 32) |
+	       ((uint64_t)p[4] << 24) |
+	       ((uint64_t)p[5] << 16) |
+	       ((uint64_t)p[6] << 8) |
+	       p[7];
+}
+
+
+static int
+load_file(const char *path,
+	  uint8_t **buf,
+	  size_t *len)
+{
+	FILE *f;
+	long size;
+	uint8_t *p;
+
+	f = fopen(path, "rb");
+
+	if (!f) {
+		fprintf(stderr,
+			"sleep-verify: %s: %s\n",
+			path,
+			strerror(errno));
+		return -1;
+	}
+
+	if (fseek(f, 0, SEEK_END) != 0 ||
+	    (size = ftell(f)) < 0 ||
+	    fseek(f, 0, SEEK_SET) != 0) {
+		fclose(f);
+		return -1;
+	}
+
+	p = malloc(size ? (size_t)size : 1);
+
+	if (!p) {
+		fclose(f);
+		errno = ENOMEM;
+		return -1;
+	}
+
+	if (size &&
+	    fread(p, 1, (size_t)size, f) != (size_t)size) {
+		free(p);
+		fclose(f);
+		errno = EIO;
+		return -1;
+	}
+
+	fclose(f);
+
+	*buf = p;
+	*len = (size_t)size;
+
+	return 0;
+}
+
+
+static int
+check_header(const uint8_t *buf,
+	     size_t len,
+	     uint32_t magic,
+	     uint16_t entry_size,
+	     const char *algo)
+{
+	size_t algo_len;
+
+	if (len < SLEEP_HEADER_SIZE)
+		return -1;
+
+	if (be32(buf) != magic ||
+	    buf[4] != 0 ||
+	    be16(buf + 5) != entry_size)
+		return -1;
+
+	algo_len = buf[7];
+
+	if (algo_len != strlen(algo))
+		return -1;
+
+	return memcmp(buf + 8, algo, algo_len) == 0 ? 0 : -1;
+}
+
+
+static int
+verify_tree_structure(const sleep_tree_entry_t *tree,
+		      size_t node_count)
+{
+	size_t i;
+
+	for (i = 1; i < node_count; i += 2) {
+		uint64_t left;
+		uint64_t right;
+		uint64_t left_len;
+		uint64_t right_len;
+		uint64_t length;
+		uint8_t hash[32];
+
+		if (!sleep_tree_left_child(i, &left) ||
+		    !sleep_tree_right_child(i, &right))
+			return -1;
+
+		/*
+		 * This parent belongs to a subtree that does not
+		 * completely exist yet.
+		 */
+		if (right >= node_count) {
+			static const uint8_t zero[40];
+
+			if (memcmp(&tree[i],
+				   zero,
+				   sizeof(tree[i])) != 0) {
+				fprintf(stderr,
+					"  tree: nonzero incomplete node %zu\n",
+					i);
+				return -1;
+			}
+
+			continue;
+		}
+
+		left_len =
+			be64((const uint8_t *)&tree[left].length_be);
+
+		right_len =
+			be64((const uint8_t *)&tree[right].length_be);
+
+		if (UINT64_MAX - left_len < right_len)
+			return -1;
+
+		length = left_len + right_len;
+
+		if (sleep_tree_hash_parent(tree[left].hash,
+					   tree[right].hash,
+					   length,
+					   hash) != 0)
+			return -1;
+
+		if (memcmp(hash, tree[i].hash, 32) != 0 ||
+		    be64((const uint8_t *)&tree[i].length_be) != length) {
+			fprintf(stderr,
+				"  tree: parent mismatch at node %zu\n",
+				i);
+			return -1;
+		}
+	}
+
+	return 0;
+}
+
+
+static int
+verify_data_leaves(const uint8_t *data,
+		   size_t data_len,
+		   const sleep_tree_entry_t *tree,
+		   size_t node_count)
+{
+	size_t leaf_count = (node_count + 1) / 2;
+	size_t offset = 0;
+	size_t i;
+
+	for (i = 0; i < leaf_count; i++) {
+		size_t node = i * 2;
+		uint64_t len =
+			be64((const uint8_t *)&tree[node].length_be);
+		uint8_t hash[32];
+
+		if (len > data_len - offset) {
+			fprintf(stderr,
+				"  data: leaf %zu exceeds metadata.data\n",
+				i);
+			return -1;
+		}
+
+		if (sleep_tree_hash_block(data + offset,
+					  (size_t)len,
+					  hash) != 0)
+			return -1;
+
+		if (memcmp(hash, tree[node].hash, 32) != 0) {
+			fprintf(stderr,
+				"  data: leaf hash mismatch at %zu\n",
+				i);
+			return -1;
+		}
+
+		offset += (size_t)len;
+	}
+
+	if (offset != data_len) {
+		fprintf(stderr,
+			"  data: tree consumes %zu bytes, file is %zu\n",
+			offset,
+			data_len);
+		return -1;
+	}
+
+	return 0;
+}
+
+
+#ifdef SLEEP_SALT
+
+static int
+verify_signatures(const uint8_t *signatures,
+		  size_t signatures_len,
+		  const uint8_t pubkey[32],
+		  const sleep_tree_entry_t *tree,
+		  size_t node_count,
+		  size_t feed_length)
+{
+	size_t sequence;
+	uint64_t roots[64];
+	uint8_t message[1 + 64 * 48];
+	uint8_t digest[32];
+
+	if (signatures_len < SLEEP_HEADER_SIZE ||
+	    (signatures_len - SLEEP_HEADER_SIZE) %
+	    SLEEP_SIG_ENTRY_SIZE != 0)
+		return -1;
+
+	if ((signatures_len - SLEEP_HEADER_SIZE) /
+	    SLEEP_SIG_ENTRY_SIZE != feed_length)
+		return -1;
+
+	for (sequence = 1;
+	     sequence <= feed_length;
+	     sequence++) {
+		size_t root_count;
+		size_t off = 0;
+		size_t r;
+
+		root_count =
+			sleep_tree_full_roots(sequence,
+					      roots,
+					      64);
+
+		message[off++] = SLEEP_HASH_ROOT;
+
+		for (r = 0; r < root_count; r++) {
+			uint64_t index = roots[r];
+			uint64_t length;
+
+			if (index >= node_count)
+				return -1;
+
+			memcpy(message + off,
+			       tree[index].hash,
+			       32);
+			off += 32;
+
+			message[off++] = (uint8_t)(index >> 56);
+			message[off++] = (uint8_t)(index >> 48);
+			message[off++] = (uint8_t)(index >> 40);
+			message[off++] = (uint8_t)(index >> 32);
+			message[off++] = (uint8_t)(index >> 24);
+			message[off++] = (uint8_t)(index >> 16);
+			message[off++] = (uint8_t)(index >> 8);
+			message[off++] = (uint8_t)index;
+
+			length =
+				be64((const uint8_t *)
+				     &tree[index].length_be);
+
+			message[off++] = (uint8_t)(length >> 56);
+			message[off++] = (uint8_t)(length >> 48);
+			message[off++] = (uint8_t)(length >> 40);
+			message[off++] = (uint8_t)(length >> 32);
+			message[off++] = (uint8_t)(length >> 24);
+			message[off++] = (uint8_t)(length >> 16);
+			message[off++] = (uint8_t)(length >> 8);
+			message[off++] = (uint8_t)length;
+		}
+
+		if (crypto_generichash(digest,
+				       sizeof(digest),
+				       message,
+				       off,
+				       NULL,
+				       0) != 0)
+			return -1;
+
+		if (crypto_sign_verify_detached(
+			signatures +
+			SLEEP_HEADER_SIZE +
+			(sequence - 1) * SLEEP_SIG_ENTRY_SIZE,
+			digest,
+			sizeof(digest),
+			pubkey) != 0) {
+			fprintf(stderr,
+				"  signatures: failed at sequence %zu\n",
+				sequence);
+			return -1;
+		}
+	}
+
+	return 0;
+}
+
+#endif
+
+
+static int
+verify_feed(const char *dir,
+	    const char *name,
+	    int verify_data,
+	    size_t *out_length)
+{
+	char path[4096];
+
+	uint8_t *tree_file = NULL;
+	uint8_t *sig_file = NULL;
+	uint8_t *key_file = NULL;
+	uint8_t *data_file = NULL;
+	uint8_t *bitfield = NULL;
+
+	size_t tree_len = 0;
+	size_t sig_len = 0;
+	size_t key_len = 0;
+	size_t data_len = 0;
+	size_t bitfield_len = 0;
+
+	size_t node_count;
+	size_t feed_length;
+
+	int rc = -1;
+
+	snprintf(path,
+		 sizeof(path),
+		 "%s/%s.tree",
+		 dir,
+		 name);
+
+	if (load_file(path, &tree_file, &tree_len) != 0)
+		goto done;
+
+	if (check_header(tree_file,
+			 tree_len,
+			 SLEEP_MAGIC_TREE,
+			 SLEEP_TREE_ENTRY_SIZE,
+			 SLEEP_ALGO_BLAKE2B) != 0) {
+		fprintf(stderr, "  tree: invalid header\n");
+		goto done;
+	}
+
+	if ((tree_len - SLEEP_HEADER_SIZE) %
+	    sizeof(sleep_tree_entry_t) != 0)
+		goto done;
+
+	node_count =
+		(tree_len - SLEEP_HEADER_SIZE) /
+		sizeof(sleep_tree_entry_t);
+
+	if (!node_count || !(node_count & 1))
+		goto done;
+
+	feed_length = (node_count + 1) / 2;
+
+	if (verify_tree_structure(
+		(const sleep_tree_entry_t *)(tree_file +
+					     SLEEP_HEADER_SIZE),
+		node_count) != 0)
+		goto done;
+
+	snprintf(path,
+		 sizeof(path),
+		 "%s/%s.key",
+		 dir,
+		 name);
+
+	if (load_file(path, &key_file, &key_len) != 0 ||
+	    key_len != 32)
+		goto done;
+
+	snprintf(path,
+		 sizeof(path),
+		 "%s/%s.signatures",
+		 dir,
+		 name);
+
+	if (load_file(path, &sig_file, &sig_len) != 0)
+		goto done;
+
+	if (check_header(sig_file,
+			 sig_len,
+			 SLEEP_MAGIC_SIGNATURE,
+			 SLEEP_SIG_ENTRY_SIZE,
+			 SLEEP_ALGO_ED25519) != 0)
+		goto done;
+
+#ifdef SLEEP_SALT
+
+	if (verify_signatures(
+		sig_file,
+		sig_len,
+		key_file,
+		(const sleep_tree_entry_t *)(tree_file +
+					     SLEEP_HEADER_SIZE),
+		node_count,
+		feed_length) != 0)
+		goto done;
+
+#else
+
+	fprintf(stderr,
+		"  signatures: build with SLEEP_SALT\n");
+	goto done;
+
+#endif
+
+	snprintf(path,
+		 sizeof(path),
+		 "%s/%s.bitfield",
+		 dir,
+		 name);
+
+	if (load_file(path,
+		      &bitfield,
+		      &bitfield_len) != 0)
+		goto done;
+
+	if (bitfield_len < SLEEP_HEADER_SIZE ||
+	    be32(bitfield) != SLEEP_MAGIC_BITFIELD ||
+	    bitfield[4] != 0)
+		goto done;
+
+	/*
+	 * The SLEEP V2 document says 3328 here.
+	 *
+	 * The supplied historical Dat fixture says 3584.
+	 * We deliberately accept both until the extra 256 bytes
+	 * in the historical format have been identified.
+	 */
+	if (be16(bitfield + 5) != SLEEP_BITFIELD_ENTRY_SIZE &&
+	    be16(bitfield + 5) != 3584)
+		goto done;
+
+	if (verify_data) {
+		snprintf(path,
+			 sizeof(path),
+			 "%s/%s.data",
+			 dir,
+			 name);
+
+		if (load_file(path,
+			      &data_file,
+			      &data_len) != 0)
+			goto done;
+
+		if (verify_data_leaves(
+			data_file,
+			data_len,
+			(const sleep_tree_entry_t *)(tree_file +
+						     SLEEP_HEADER_SIZE),
+			node_count) != 0)
+			goto done;
+	}
+
+	if (out_length)
+		*out_length = feed_length;
+
+	rc = 0;
+
+done:
+	free(tree_file);
+	free(sig_file);
+	free(key_file);
+	free(data_file);
+	free(bitfield);
+
+	return rc;
+}
+
+
+int
+sleep_verify_dat(const char *dir)
+{
+	size_t metadata_length;
+	size_t content_length;
+
+	printf("sleep-verify: %s\n", dir);
+
+	printf("metadata: ");
+
+	if (verify_feed(dir,
+			"metadata",
+			1,
+			&metadata_length) != 0) {
+		printf("FAIL\n");
+		return -1;
+	}
+
+	printf("OK (%zu entries)\n", metadata_length);
+
+	printf("content:  ");
+
+	if (verify_feed(dir,
+			"content",
+			0,
+			&content_length) != 0) {
+		printf("FAIL\n");
+		return -1;
+	}
+
+	printf("OK (%zu entries)\n", content_length);
+
+	printf("sleep-verify: PASS\n");
+
+	return 0;
+}
+
+
+int
+main(int argc, char **argv)
+{
+	if (argc != 2) {
+		fprintf(stderr,
+			"usage: %s .dat\n",
+			argv[0]);
+		return 2;
+	}
+
+#ifdef SLEEP_SALT
+
+	if (sodium_init() < 0)
+		return 1;
+
+#else
+
+	fprintf(stderr,
+		"sleep-verify: built without SLEEP_SALT\n");
+
+	return 1;
+
+#endif
+
+	return sleep_verify_dat(argv[1]) == 0 ? 0 : 1;
+}

extra/sleep_verify.h

@@ -0,0 +1,6 @@
+#ifndef SLEEP_VERIFY_H
+#define SLEEP_VERIFY_H
+
+int sleep_verify_dat(const char *dat_dir);
+
+#endif

extra/sleep_writer_pass/README.txt

@@ -0,0 +1,16 @@
+SLEEP writer pass
+
+Files:
+  inc/sleep_writer.h
+  src/sleep_writer.c
+  src/sleep_tree.c
+
+Important change from the previous writer draft:
+  SLEEP/Dat root sets for a feed of N blocks use fullRoots(2*N), not fullRoots(2*(N-1)).
+  Thus N=83 produces roots 63, 143, 161, 164, matching the historical fixture's signatures.
+
+The writer emits one signature for every feed length, 1..N.
+
+The bitfield writer follows the documented 3328-byte SLEEP V2 layout. The historical fixture supplied with the project has a 3584-byte bitfield entry; the extra 256 bytes remain deliberately unresolved rather than guessed.
+
+The code requires SLEEP_SALT/libsodium for cryptographic operation.

extra/sleep_writer_pass/inc/sleep_writer.h

@@ -0,0 +1,18 @@
+#ifndef SLEEP_WRITER_H
+#define SLEEP_WRITER_H
+#include <stddef.h>
+#include <stdint.h>
+#include "sleep.h"
+#ifdef __cplusplus
+extern "C" {
+#endif
+int sleep_write_feed(const char *feed_dir,
+                     const char *feed_name,
+                     const uint8_t *const *blocks,
+                     const size_t *lengths,
+                     size_t nblocks,
+                     const sleep_keypair_t *keypair);
+#ifdef __cplusplus
+}
+#endif
+#endif

extra/sleep_writer_pass/src/sleep_tree.c

@@ -0,0 +1,272 @@
+#include "sleep.h"
+
+#include <errno.h>
+#include <stdint.h>
+#include <stdlib.h>
+#include <string.h>
+
+#ifdef SLEEP_SALT
+#include <sodium.h>
+#endif
+
+static unsigned int sleep_tree_ctz64(uint64_t x)
+{
+#if defined(__GNUC__) || defined(__clang__)
+	if (x == 0)
+		return 64;
+	return (unsigned int)__builtin_ctzll(x);
+#else
+	unsigned int n = 0;
+
+	if (x == 0)
+		return 64;
+
+	while ((x & 1ULL) == 0) {
+		x >>= 1;
+		n++;
+	}
+
+	return n;
+#endif
+}
+
+uint64_t sleep_tree_index(uint64_t depth, uint64_t offset)
+{
+	if (depth >= 63) {
+		errno = EOVERFLOW;
+		return UINT64_MAX;
+	}
+
+	return ((1ULL + (2ULL * offset)) << depth) - 1ULL;
+}
+
+uint64_t sleep_tree_depth(uint64_t index)
+{
+	return sleep_tree_ctz64(index + 1ULL);
+}
+
+uint64_t sleep_tree_offset(uint64_t index, uint64_t depth)
+{
+	if ((index & 1ULL) == 0)
+		return index / 2ULL;
+
+	if (depth == UINT64_MAX)
+		depth = sleep_tree_depth(index);
+
+	return (((index + 1ULL) >> depth) - 1ULL) / 2ULL;
+}
+
+uint64_t sleep_tree_parent(uint64_t index)
+{
+	uint64_t depth = sleep_tree_depth(index);
+	uint64_t offset = sleep_tree_offset(index, depth);
+
+	return sleep_tree_index(depth + 1ULL, offset >> 1);
+}
+
+int sleep_tree_sibling(uint64_t index, uint64_t *out)
+{
+	uint64_t depth;
+	uint64_t offset;
+
+	if (out == NULL) {
+		errno = EINVAL;
+		return -1;
+	}
+
+	depth = sleep_tree_depth(index);
+	offset = sleep_tree_offset(index, depth);
+
+	*out = sleep_tree_index(depth, offset & 1ULL ? offset - 1ULL
+	                                             : offset + 1ULL);
+	return 0;
+}
+
+int sleep_tree_left_child(uint64_t index, uint64_t *out)
+{
+	uint64_t depth;
+	uint64_t offset;
+
+	if (out == NULL) {
+		errno = EINVAL;
+		return -1;
+	}
+
+	if ((index & 1ULL) == 0) {
+		errno = EINVAL;
+		return -1;
+	}
+
+	depth = sleep_tree_depth(index);
+	if (depth == 0) {
+		errno = EINVAL;
+		return -1;
+	}
+
+	offset = sleep_tree_offset(index, depth);
+	*out = sleep_tree_index(depth - 1ULL, offset * 2ULL);
+	return 0;
+}
+
+int sleep_tree_right_child(uint64_t index, uint64_t *out)
+{
+	uint64_t depth;
+	uint64_t offset;
+
+	if (out == NULL) {
+		errno = EINVAL;
+		return -1;
+	}
+
+	if ((index & 1ULL) == 0) {
+		errno = EINVAL;
+		return -1;
+	}
+
+	depth = sleep_tree_depth(index);
+	if (depth == 0) {
+		errno = EINVAL;
+		return -1;
+	}
+
+	offset = sleep_tree_offset(index, depth);
+	*out = sleep_tree_index(depth - 1ULL, offset * 2ULL + 1ULL);
+	return 0;
+}
+
+uint64_t sleep_tree_left_span(uint64_t index)
+{
+	uint64_t depth;
+
+	if ((index & 1ULL) == 0)
+		return index;
+
+	depth = sleep_tree_depth(index);
+	return sleep_tree_offset(index, depth) * (1ULL << (depth + 1ULL));
+}
+
+uint64_t sleep_tree_right_span(uint64_t index)
+{
+	uint64_t depth;
+
+	if ((index & 1ULL) == 0)
+		return index;
+
+	depth = sleep_tree_depth(index);
+	return (sleep_tree_offset(index, depth) + 1ULL) *
+	       (1ULL << (depth + 1ULL)) - 2ULL;
+}
+
+void sleep_tree_spans(uint64_t index, uint64_t *left, uint64_t *right)
+{
+	if (left != NULL)
+		*left = sleep_tree_left_span(index);
+	if (right != NULL)
+		*right = sleep_tree_right_span(index);
+}
+
+size_t sleep_tree_count(uint64_t index)
+{
+	uint64_t depth;
+
+	if ((index & 1ULL) == 0)
+		return 1;
+
+	depth = sleep_tree_depth(index);
+	if (depth >= sizeof(size_t) * 8 - 1)
+		return SIZE_MAX;
+
+	return ((size_t)1 << (depth + 1ULL)) - 1;
+}
+
+size_t sleep_tree_count_leaves(uint64_t index)
+{
+	return (sleep_tree_count(index) + 1) / 2;
+}
+
+size_t sleep_tree_full_roots(uint64_t index, uint64_t *out, size_t capacity)
+{
+	uint64_t blocks;
+	uint64_t offset = 0;
+	uint64_t factor = 1;
+	size_t count = 0;
+
+	if (index & 1ULL) {
+		errno = EINVAL;
+		return 0;
+	}
+
+	/* mafintosh/flat-tree fullRoots(index) works on index / 2. */
+	blocks = index / 2ULL;
+
+	while (blocks != 0) {
+		while (factor <= UINT64_MAX / 2ULL && factor * 2ULL <= blocks)
+			factor *= 2ULL;
+
+		if (out != NULL && count < capacity)
+			out[count] = offset + factor - 1ULL;
+
+		count++;
+		offset += 2ULL * factor;
+		blocks -= factor;
+		factor = 1;
+	}
+
+	return count;
+}
+
+#ifdef SLEEP_SALT
+int sleep_tree_hash_block(const uint8_t *data, size_t len,
+                          uint8_t out_hash[32])
+{
+	uint8_t header[9];
+	crypto_generichash_state state;
+
+	if (out_hash == NULL || (data == NULL && len != 0)) {
+		errno = EINVAL;
+		return -1;
+	}
+
+	header[0] = SLEEP_HASH_LEAF;
+	for (int i = 0; i < 8; i++)
+		header[1 + i] = (uint8_t)((uint64_t)len >> (56 - i * 8));
+
+	if (crypto_generichash_init(&state, NULL, 0, 32) != 0)
+		return -1;
+	if (crypto_generichash_update(&state, header, sizeof(header)) != 0)
+		return -1;
+	if (len != 0 && crypto_generichash_update(&state, data, len) != 0)
+		return -1;
+
+	return crypto_generichash_final(&state, out_hash, 32);
+}
+
+int sleep_tree_hash_parent(const uint8_t left[32],
+                           const uint8_t right[32],
+                           uint64_t length,
+                           uint8_t out_hash[32])
+{
+	uint8_t header[9];
+	crypto_generichash_state state;
+
+	if (left == NULL || right == NULL || out_hash == NULL) {
+		errno = EINVAL;
+		return -1;
+	}
+
+	header[0] = SLEEP_HASH_PARENT;
+	for (int i = 0; i < 8; i++)
+		header[1 + i] = (uint8_t)(length >> (56 - i * 8));
+
+	if (crypto_generichash_init(&state, NULL, 0, 32) != 0)
+		return -1;
+	if (crypto_generichash_update(&state, header, sizeof(header)) != 0)
+		return -1;
+	if (crypto_generichash_update(&state, left, 32) != 0)
+		return -1;
+	if (crypto_generichash_update(&state, right, 32) != 0)
+		return -1;
+
+	return crypto_generichash_final(&state, out_hash, 32);
+}
+#endif

extra/sleep_writer_pass/src/sleep_writer.c

@@ -0,0 +1,530 @@
+#include "sleep_writer.h"
+#include "sleep.h"
+
+#include <errno.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#ifdef SLEEP_SALT
+#include <sodium.h>
+#endif
+
+#define SLEEP_MAGIC_BITFIELD 0x05025700u
+#define SLEEP_MAGIC_SIGNATURE 0x05025701u
+#define SLEEP_MAGIC_TREE 0x05025702u
+
+#define SLEEP_BITFIELD_SIZE 3328u
+#define SLEEP_DATA_BITS 8192u
+#define SLEEP_TREE_BITS 16384u
+#define SLEEP_INDEX_BYTES 256u
+
+#define SLEEP_HASH_BYTES 32u
+#define SLEEP_STACK_MAX 64u
+
+typedef struct writer_node {
+	uint8_t hash[SLEEP_HASH_BYTES];
+	uint64_t length;
+	uint64_t index;
+	uint64_t depth;
+} writer_node_t;
+
+typedef struct writer_peak {
+	uint64_t index;
+	uint64_t depth;
+} writer_peak_t;
+
+static void put_u16be(uint8_t *p, uint16_t v)
+{
+	p[0] = (uint8_t)(v >> 8);
+	p[1] = (uint8_t)v;
+}
+
+static void put_u32be(uint8_t *p, uint32_t v)
+{
+	p[0] = (uint8_t)(v >> 24);
+	p[1] = (uint8_t)(v >> 16);
+	p[2] = (uint8_t)(v >> 8);
+	p[3] = (uint8_t)v;
+}
+
+static void put_u64be(uint8_t *p, uint64_t v)
+{
+	p[0] = (uint8_t)(v >> 56);
+	p[1] = (uint8_t)(v >> 48);
+	p[2] = (uint8_t)(v >> 40);
+	p[3] = (uint8_t)(v >> 32);
+	p[4] = (uint8_t)(v >> 24);
+	p[5] = (uint8_t)(v >> 16);
+	p[6] = (uint8_t)(v >> 8);
+	p[7] = (uint8_t)v;
+}
+
+static int make_path(char *out, size_t out_size,
+                     const char *dir, const char *name,
+                     const char *suffix)
+{
+	int n = snprintf(out, out_size, "%s/%s%s", dir, name, suffix);
+	if (n < 0 || (size_t)n >= out_size) {
+		errno = ENAMETOOLONG;
+		return -1;
+	}
+	return 0;
+}
+
+static int write_header(FILE *f, uint32_t magic,
+                        uint16_t entry_size, const char *algo)
+{
+	uint8_t header[SLEEP_HEADER_SIZE];
+	size_t len = algo ? strlen(algo) : 0;
+
+	if (len > 24) {
+		errno = EINVAL;
+		return -1;
+	}
+
+	memset(header, 0, sizeof(header));
+	put_u32be(header, magic);
+	put_u16be(header + 5, entry_size);
+	header[7] = (uint8_t)len;
+	if (len)
+		memcpy(header + 8, algo, len);
+
+	if (fwrite(header, 1, sizeof(header), f) != sizeof(header)) {
+		errno = EIO;
+		return -1;
+	}
+	return 0;
+}
+
+static int hash_leaf(const uint8_t *data, size_t len,
+                     uint8_t out[32])
+{
+#ifdef SLEEP_SALT
+	uint8_t head[9];
+	crypto_generichash_state st;
+
+	head[0] = 0;
+	put_u64be(head + 1, (uint64_t)len);
+
+	if (crypto_generichash_init(&st, NULL, 0, 32) != 0)
+		return -1;
+	if (crypto_generichash_update(&st, head, sizeof(head)) != 0)
+		return -1;
+	if (len && crypto_generichash_update(&st, data, len) != 0)
+		return -1;
+	return crypto_generichash_final(&st, out, 32);
+#else
+	(void)data; (void)len; (void)out;
+	errno = ENOSYS;
+	return -1;
+#endif
+}
+
+static int hash_parent(const writer_node_t *left,
+                       const writer_node_t *right,
+                       uint8_t out[32])
+{
+#ifdef SLEEP_SALT
+	uint8_t head[9];
+	crypto_generichash_state st;
+
+	head[0] = 1;
+	put_u64be(head + 1, left->length + right->length);
+
+	if (crypto_generichash_init(&st, NULL, 0, 32) != 0)
+		return -1;
+	if (crypto_generichash_update(&st, head, sizeof(head)) != 0)
+		return -1;
+	if (crypto_generichash_update(&st, left->hash, 32) != 0)
+		return -1;
+	if (crypto_generichash_update(&st, right->hash, 32) != 0)
+		return -1;
+	return crypto_generichash_final(&st, out, 32);
+#else
+	(void)left; (void)right; (void)out;
+	errno = ENOSYS;
+	return -1;
+#endif
+}
+
+static int tree_build(const uint8_t *const *blocks,
+                      const size_t *lengths,
+                      size_t nblocks,
+                      writer_node_t **out_nodes,
+                      size_t *out_count)
+{
+	writer_node_t *nodes;
+	writer_peak_t stack[SLEEP_STACK_MAX];
+	size_t stack_len = 0;
+	size_t node_count;
+	size_t i;
+
+	if (nblocks > (SIZE_MAX + 1) / 2) {
+		errno = EOVERFLOW;
+		return -1;
+	}
+
+	node_count = nblocks ? nblocks * 2 - 1 : 0;
+	nodes = node_count ? calloc(node_count, sizeof(*nodes)) : NULL;
+	if (node_count && !nodes) {
+		errno = ENOMEM;
+		return -1;
+	}
+
+	for (i = 0; i < nblocks; i++) {
+		writer_node_t *leaf;
+		uint64_t leaf_index = (uint64_t)(i * 2);
+
+		leaf = &nodes[leaf_index];
+		leaf->index = leaf_index;
+		leaf->depth = 0;
+		leaf->length = lengths[i];
+
+		if (hash_leaf(blocks[i], lengths[i], leaf->hash) != 0)
+			goto fail;
+
+		stack[stack_len].index = leaf_index;
+		stack[stack_len].depth = 0;
+		stack_len++;
+
+		while (stack_len >= 2 &&
+		       stack[stack_len - 1].depth == stack[stack_len - 2].depth) {
+			writer_peak_t right = stack[stack_len - 1];
+			writer_peak_t left = stack[stack_len - 2];
+			writer_node_t *ln = &nodes[left.index];
+			writer_node_t *rn = &nodes[right.index];
+			uint64_t parent_index;
+			writer_node_t *parent;
+
+			stack_len -= 2;
+			parent_index = sleep_tree_parent(left.index);
+			if (parent_index >= node_count) {
+				errno = EINVAL;
+				goto fail;
+			}
+
+			parent = &nodes[parent_index];
+			parent->index = parent_index;
+			parent->depth = left.depth + 1;
+			parent->length = ln->length + rn->length;
+
+			if (hash_parent(ln, rn, parent->hash) != 0)
+				goto fail;
+
+			stack[stack_len].index = parent_index;
+			stack[stack_len].depth = parent->depth;
+			stack_len++;
+		}
+	}
+
+	*out_nodes = nodes;
+	*out_count = node_count;
+	return 0;
+
+fail:
+	free(nodes);
+	return -1;
+}
+
+static int write_tree(const char *path,
+                      const writer_node_t *nodes, size_t node_count)
+{
+	FILE *f;
+	size_t i;
+
+	f = fopen(path, "wb");
+	if (!f)
+		return -1;
+
+	if (write_header(f, SLEEP_MAGIC_TREE,
+	                 SLEEP_TREE_ENTRY_SIZE, SLEEP_ALGO_BLAKE2B) != 0)
+		goto fail;
+
+	for (i = 0; i < node_count; i++) {
+		uint8_t entry[40];
+		memcpy(entry, nodes[i].hash, 32);
+		put_u64be(entry + 32, nodes[i].length);
+		if (fwrite(entry, 1, sizeof(entry), f) != sizeof(entry)) {
+			errno = EIO;
+			goto fail;
+		}
+	}
+
+	return fclose(f) == 0 ? 0 : -1;
+fail:
+	fclose(f);
+	return -1;
+}
+
+static int sign_state(const writer_node_t *nodes,
+                      size_t block_count,
+                      const sleep_keypair_t *keypair,
+                      uint8_t signature[64])
+{
+#ifdef SLEEP_SALT
+	uint64_t root_indexes[64];
+	size_t root_count;
+	size_t i;
+	size_t msg_len;
+	uint8_t *msg;
+	uint8_t digest[32];
+	crypto_generichash_state st;
+
+	/* The Dat/SLEEP root set is fullRoots(2 * length). */
+	root_count = sleep_tree_full_roots((uint64_t)(2 * block_count),
+	                                   root_indexes, 64);
+
+	if (root_count > (SIZE_MAX - 1) / 48) {
+		errno = EOVERFLOW;
+		return -1;
+	}
+	msg_len = 1 + root_count * 48;
+	msg = malloc(msg_len);
+	if (!msg) {
+		errno = ENOMEM;
+		return -1;
+	}
+
+	msg[0] = 2;
+	for (i = 0; i < root_count; i++) {
+		const writer_node_t *root = &nodes[root_indexes[i]];
+		size_t off = 1 + i * 48;
+
+		memcpy(msg + off, root->hash, 32);
+		put_u64be(msg + off + 32, root->index);
+		put_u64be(msg + off + 40, root->length);
+	}
+
+	if (crypto_generichash_init(&st, NULL, 0, 32) != 0)
+		goto fail;
+	if (crypto_generichash_update(&st, msg, msg_len) != 0)
+		goto fail;
+	if (crypto_generichash_final(&st, digest, 32) != 0)
+		goto fail;
+
+	free(msg);
+
+	if (crypto_sign_detached(signature, NULL,
+	                         digest, sizeof(digest),
+	                         keypair->seckey) != 0)
+		return -1;
+
+	return 0;
+
+fail:
+	free(msg);
+	return -1;
+#else
+	(void)nodes; (void)block_count; (void)keypair; (void)signature;
+	errno = ENOSYS;
+	return -1;
+#endif
+}
+
+static int write_signatures(const char *path,
+                            const writer_node_t *nodes,
+                            size_t nblocks,
+                            const sleep_keypair_t *keypair)
+{
+	FILE *f;
+	size_t length;
+
+	f = fopen(path, "wb");
+	if (!f)
+		return -1;
+
+	if (write_header(f, SLEEP_MAGIC_SIGNATURE,
+	                 SLEEP_SIG_ENTRY_SIZE, SLEEP_ALGO_ED25519) != 0)
+		goto fail;
+
+	for (length = 1; length <= nblocks; length++) {
+		uint8_t sig[64];
+
+		if (sign_state(nodes, length, keypair, sig) != 0)
+			goto fail;
+		if (fwrite(sig, 1, sizeof(sig), f) != sizeof(sig)) {
+			errno = EIO;
+			goto fail;
+		}
+	}
+
+	return fclose(f) == 0 ? 0 : -1;
+fail:
+	fclose(f);
+	return -1;
+}
+
+static void set_bit(uint8_t *bits, size_t bit)
+{
+	bits[bit >> 3] |= (uint8_t)(1u << (bit & 7));
+}
+
+static unsigned int byte_state(uint8_t v)
+{
+	if (v == 0x00)
+		return 0;
+	if (v == 0xff)
+		return 3;
+	return 2;
+}
+
+static void build_index(uint8_t index[SLEEP_INDEX_BYTES],
+                        const uint8_t data[1024])
+{
+	size_t pair;
+
+	memset(index, 0, SLEEP_INDEX_BYTES);
+
+	/* Four index bits describe every two bytes of the data bitfield. */
+	for (pair = 0; pair < 512; pair++) {
+		unsigned int a = byte_state(data[pair * 2]);
+		unsigned int b = byte_state(data[pair * 2 + 1]);
+		unsigned int tuple = a | (b << 2);
+		size_t nibble = pair;
+		size_t byte = nibble >> 1;
+
+		if ((nibble & 1) == 0)
+			index[byte] |= (uint8_t)tuple;
+		else
+			index[byte] |= (uint8_t)(tuple << 4);
+	}
+}
+
+static int write_bitfield(const char *path,
+                          size_t nblocks,
+                          size_t node_count)
+{
+	uint8_t entry[SLEEP_BITFIELD_SIZE];
+	uint8_t *data = entry;
+	uint8_t *tree = entry + 1024;
+	uint8_t *index = entry + 3072;
+	FILE *f;
+	size_t i;
+
+	memset(entry, 0, sizeof(entry));
+
+	if (nblocks > SLEEP_DATA_BITS || node_count > SLEEP_TREE_BITS) {
+		errno = EOVERFLOW;
+		return -1;
+	}
+
+	for (i = 0; i < nblocks; i++)
+		set_bit(data, i);
+
+	for (i = 0; i < node_count; i++)
+		set_bit(tree, i);
+
+	build_index(index, data);
+
+	f = fopen(path, "wb");
+	if (!f)
+		return -1;
+
+	if (write_header(f, SLEEP_MAGIC_BITFIELD,
+	                 SLEEP_BITFIELD_SIZE, NULL) != 0)
+		goto fail;
+	if (fwrite(entry, 1, sizeof(entry), f) != sizeof(entry)) {
+		errno = EIO;
+		goto fail;
+	}
+
+	return fclose(f) == 0 ? 0 : -1;
+fail:
+	fclose(f);
+	return -1;
+}
+
+static int write_data(const char *path,
+                      const uint8_t *const *blocks,
+                      const size_t *lengths,
+                      size_t nblocks)
+{
+	FILE *f;
+	size_t i;
+
+	f = fopen(path, "wb");
+	if (!f)
+		return -1;
+
+	for (i = 0; i < nblocks; i++) {
+		if (lengths[i] &&
+		    fwrite(blocks[i], 1, lengths[i], f) != lengths[i]) {
+			errno = EIO;
+			goto fail;
+		}
+	}
+
+	return fclose(f) == 0 ? 0 : -1;
+fail:
+	fclose(f);
+	return -1;
+}
+
+static int write_key(const char *path, const sleep_keypair_t *keypair)
+{
+	FILE *f = fopen(path, "wb");
+	if (!f)
+		return -1;
+	if (fwrite(keypair->pubkey, 1, 32, f) != 32) {
+		errno = EIO;
+		fclose(f);
+		return -1;
+	}
+	return fclose(f) == 0 ? 0 : -1;
+}
+
+int sleep_write_feed(const char *feed_dir,
+                     const char *feed_name,
+                     const uint8_t *const *blocks,
+                     const size_t *lengths,
+                     size_t nblocks,
+                     const sleep_keypair_t *keypair)
+{
+	char path[1024];
+	writer_node_t *nodes = NULL;
+	size_t node_count = 0;
+	int rc = -1;
+
+	if (!feed_dir || !feed_name || !keypair ||
+	    (nblocks && (!blocks || !lengths))) {
+		errno = EINVAL;
+		return -1;
+	}
+
+	if (make_path(path, sizeof(path), feed_dir, feed_name, ".tree") != 0)
+		return -1;
+
+	if (tree_build(blocks, lengths, nblocks, &nodes, &node_count) != 0)
+		return -1;
+
+	if (write_tree(path, nodes, node_count) != 0)
+		goto done;
+
+	if (make_path(path, sizeof(path), feed_dir, feed_name, ".signatures") != 0)
+		goto done;
+	if (write_signatures(path, nodes, nblocks, keypair) != 0)
+		goto done;
+
+	if (make_path(path, sizeof(path), feed_dir, feed_name, ".key") != 0)
+		goto done;
+	if (write_key(path, keypair) != 0)
+		goto done;
+
+	if (make_path(path, sizeof(path), feed_dir, feed_name, ".bitfield") != 0)
+		goto done;
+	if (write_bitfield(path, nblocks, node_count) != 0)
+		goto done;
+
+	if (make_path(path, sizeof(path), feed_dir, feed_name, ".data") != 0)
+		goto done;
+	if (write_data(path, blocks, lengths, nblocks) != 0)
+		goto done;
+
+	rc = 0;
+
+done:
+	free(nodes);
+	return rc;
+}