From 06d2a0be68fde9700e1f61bc8637d29b3d1e46c9 Mon Sep 17 00:00:00 2001 From: Brian Warner Date: Wed, 25 May 2016 20:58:53 -0700 Subject: [PATCH] update docs --- README.md | 65 ++++++++++--------------- docs/api.md | 138 ++++++++++------------------------------------------ 2 files changed, 53 insertions(+), 150 deletions(-) diff --git a/README.md b/README.md index 2189cf0..58a25cb 100644 --- a/README.md +++ b/README.md @@ -53,8 +53,10 @@ delivered and used. If this does not feel strong enough, users can turn on additional verification that doesn't depend upon the secrecy of the channel. The notion of a "magic wormhole" comes from the image of two distant wizards -speaking the same phrase at the same time, and causing a connection to be -established between them. Transferring files securely should be that easy. +speaking the same enchanted phrase at the same time, and causing a mystical +connection to pop into existence between them. The wizards then throw books +into the wormhole and they fall out the other side. Transferring files +securely should be that easy. ## Design @@ -76,27 +78,22 @@ of success. ## Timing -At present, the two clients must be run within about 3 minutes of each other, -as they will stop waiting after that time. This makes the tool most useful -for people who are having a real-time conversation already, and want to -graduate to a secure connection. - -Future releases should increase that to several hours. This will enable a -mode in which two humans can decide on a code phrase offline, by choosing a -channel number and a few random words, and then go back home to their -computers later and begin the wormhole process. (This mode is already -supported, but is not currently easy to use because the two users must type -the phrases within three minutes of each other). +The program does not have any built-in timeouts, however it is expected that +both clients will be run within an hour or so of each other. This makes the +tool most useful for people who are having a real-time conversation already, +and want to graduate to a secure connection. Both clients must be left +running until the transfer has finished. ## Relays -The wormhole library requires a "Rendezvous Server": a simple relay that -delivers messages from one client to another. This allows the wormhole codes -to omit IP addresses and port numbers. The URL of a public server is baked -into the library for use as a default, and will be freely available until -volume or abuse makes it infeasible to support. Applications which desire -more reliability can easily run their own relay and configure their clients -to use it instead. Code for the Rendezvous Server is included in the library. +The wormhole library requires a "Rendezvous Server": a simple WebSocket-based +relay that delivers messages from one client to another. This allows the +wormhole codes to omit IP addresses and port numbers. The URL of a public +server is baked into the library for use as a default, and will be freely +available until volume or abuse makes it infeasible to support. Applications +which desire more reliability can easily run their own relay and configure +their clients to use it instead. Code for the Rendezvous Server is included +in the library. The file-transfer commands also use a "Transit Relay", which is another simple server that glues together two inbound TCP connections and transfers @@ -127,34 +124,26 @@ Both commands accept: ## Library The `wormhole` module makes it possible for other applications to use these -code-protected channels. This includes blocking/synchronous support and -async/Twisted support, both for a symmetric scheme. The main module is named -`wormhole.blocking.transcribe`, to reflect that it is for -synchronous/blocking code, and uses a PAKE mode whereby one user transcribes -their code to the other. (internal names may change in the future). +code-protected channels. This includes Twisted support, and (in the future) +will include blocking/synchronous support too. See docs/api.md for details. -The file-transfer tools use a second module named -`wormhole.blocking.transit`, which provides an encrypted record-pipe. It -knows how to use the Transit Relay as well as direct connections, and -attempts them all in parallel. `TransitSender` and `TransitReceiver` are -distinct, although once the connection is established, data can flow in -either direction. All data is encrypted (using nacl/libsodium "secretbox") -using a key derived from the PAKE phase. See -`src/wormhole/scripts/cmd_send.py` for examples. +The file-transfer tools use a second module named `wormhole.transit`, which +provides an encrypted record-pipe. It knows how to use the Transit Relay as +well as direct connections, and attempts them all in parallel. +`TransitSender` and `TransitReceiver` are distinct, although once the +connection is established, data can flow in either direction. All data is +encrypted (using nacl/libsodium "secretbox") using a key derived from the +PAKE phase. See `src/wormhole/cli/cmd_send.py` for examples. ## License, Compatibility This library is released under the MIT license, see LICENSE for details. This library is compatible with python2.7, 3.3, 3.4, and 3.5 . It is probably -compatible with py2.6, but the latest Twisted (15.5.0) is not. The +compatible with py2.6, but the latest Twisted (>=15.5.0) is not. The (daemonizing) 'wormhole server start' command does not yet work with py3, but will in the future once Twisted itself is finished being ported. -This package depends upon the SPAKE2, pynacl, requests, and argparse -libraries. To run a relay server, use the async support, or run the unit -tests, you must also install Twisted. - #### footnotes diff --git a/docs/api.md b/docs/api.md index 2efb436..2d95323 100644 --- a/docs/api.md +++ b/docs/api.md @@ -60,69 +60,28 @@ to be available, then returns it. The Wormhole is not meant as a long-term communication channel, but some protocols work better if they can exchange an initial pair of messages (perhaps offering some set of negotiable capabilities), and then follow up with a second pair (to reveal the results -of the negotiation). Another use case is for an ACK that gets sent at the end -of a file transfer: the Wormhole is held open until the Transit object -reports completion, and the last message is a hash of the file contents to -prove it was received correctly. +of the negotiation). Note: the application developer must be careful to avoid deadlocks (if both sides want to `get()`, somebody has to `send()` first). -When both sides are done, they must call `close()`, to let the library know -that the connection is complete and it can deallocate the channel. If you -forget to call `close()`, the server will not free the channel, and other -users will suffer longer invitation codes as a result. To encourage -`close()`, the library will log an error if a Wormhole object is destroyed -before being closed. - -To make it easier to call `close()`, the blocking Wormhole objects can be -used as a context manager. Just put your code in the body of a `with -wormhole(ARGS) as w:` statement, and `close()` will automatically be called -when the block exits (either successfully or due to an exception). - -## Examples - -The synchronous+blocking flow looks like this: - -```python -from wormhole.blocking.transcribe import wormhole -from wormhole.public_relay import RENDEZVOUS_RELAY -mydata = b"initiator's data" -with wormhole(u"appid", RENDEZVOUS_RELAY) as i: - code = i.get_code() - print("Invitation Code: %s" % code) - i.send(mydata) - theirdata = i.get() - print("Their data: %s" % theirdata.decode("ascii")) -``` - -```python -import sys -from wormhole.blocking.transcribe import wormhole -from wormhole.public_relay import RENDEZVOUS_RELAY -mydata = b"receiver's data" -code = sys.argv[1] -with wormhole(u"appid", RENDEZVOUS_RELAY) as r: - r.set_code(code) - r.send(mydata) - theirdata = r.get() - print("Their data: %s" % theirdata.decode("ascii")) -``` +When both sides are done, they must call `close()`, to flush all pending +`send()` calls, deallocate the channel, and close the websocket connection. ## Twisted -The Twisted-friendly flow looks like this: +The Twisted-friendly flow looks like this (note that passing reactor= is how +you get a non-blocking Wormhole): ```python from twisted.internet import reactor from wormhole.public_relay import RENDEZVOUS_RELAY -from wormhole.twisted.transcribe import wormhole -outbound_message = b"outbound data" +from wormhole import wormhole w1 = wormhole(u"appid", RENDEZVOUS_RELAY, reactor) d = w1.get_code() def _got_code(code): print "Invitation Code:", code - return w1.send(outbound_message) + return w1.send(b"outbound data") d.addCallback(_got_code) d.addCallback(lambda _: w1.get()) def _got(inbound_message): @@ -156,15 +115,19 @@ the rest of the protocol to proceed. If they do not match, then the two programs are not talking to each other (they may both be talking to a man-in-the-middle attacker), and the protocol should be abandoned. -To retrieve the verifier, you call `w.get_verifier()` before any calls to -`send()/get()`. Turn this into hex or Base64 to print it, or render it as -ASCII-art, etc. Once the users are convinced that `get_verifier()` from both -sides are the same, call `send()/get()` to continue the protocol. If you call -`send()/get()` before `get_verifier()`, it will perform the complete protocol -without pausing. +To retrieve the verifier, you call `d=w.verify()` before any calls to +`send()/get()`. The Deferred will not fire until internal key-confirmation +has taken place (meaning the two sides have exchanged their initial PAKE +messages, and the wormhole codes matched), so `verify()` is also a good way +to detect typos or mistakes entering the code. The Deferred will errback with +wormhole.WrongPasswordError if the codes did not match, or it will callback +with the verifier bytes if they did match. -The Twisted form of `get_verifier()` returns a Deferred that fires with the -verifier bytes. +Once retrieved, you can turn this into hex or Base64 to print it, or render +it as ASCII-art, etc. Once the users are convinced that `verify()` from both +sides are the same, call `send()/get()` to continue the protocol. If you call +`send()/get()` before `verify()`, it will perform the complete protocol +without pausing. ## Generating the Invitation Code @@ -176,8 +139,8 @@ randomly-generated selection from the PGP wordlist, providing a default of 16 bits of entropy. The initiating program should display this code to the user, who should transcribe it to the receiving user, who gives it to the Receiver object by calling `set_code()`. The receiving program can also use -`input_code_with_completion()` to use a readline-based input function: this -offers tab completion of allocated channel-ids and known codewords. +`input_code()` to use a readline-based input function: this offers tab +completion of allocated channel-ids and known codewords. Alternatively, the human users can agree upon an invitation code themselves, and provide it to both programs later (both sides call `set_code()`). They @@ -212,59 +175,10 @@ the other. This must be the same for both clients, and is generally baked-in to the application source code or default config. This library includes the URL of a public relay run by the author. -Application developers can use this one, or they can run their own (see -src/wormhole/servers/relay.py) and configure their clients to use it instead. -This URL is passed as a unicode string. - -## Polling and Shutdown - -TODO: this is mostly imaginary - -The reactor-based (Twisted-style) forms of these objects need to establish -TCP connections, re-establish them if they are lost, and sometimes (for -transports that don't support long-running connections) poll for new -messages. They may also time out eventually. Longer delays mean less network -traffic, but higher latency. - -These timers should be matched to the expectations, and expected behavior, of -your users. In a file-transfer application, where the users are sitting next -to each other, it is appropriate to poll very frequently (perhaps every -500ms) for a few minutes, then give up. In an email-like messaging program -where the introduction is establishing a long-term relationship, and the -program can store any outgoing messages until the connection is established, -it is probably better to poll once a minute for the first few minutes, then -back off to once an hour, and not give up for several days. - -The `schedule=` constructor argument establishes the polling schedule. It -should contain a sorted list of (when, interval) tuples (both floats). At -`when` seconds after the first `start()` call, the polling interval will be -set to `interval`. - -The `timeout=` argument provides a hard timeout. After this many seconds, the -sync will be abandoned, and all callbacks will errback with a TimeoutError. - -Both have defaults suitable for face-to-face realtime setup environments. - -## Serialization - -TODO: only the Twisted form supports serialization so far - -You may not be able to hold the Wormhole object in memory for the whole sync -process: maybe you allow it to wait for several days, but the program will be -restarted during that time. To support this, you can persist the state of the -object by calling `data = w.serialize()`, which will return a printable -bytestring (the JSON-encoding of a small dictionary). To restore, use `w = -wormhole_from_serialized(data, reactor)`. - -There is exactly one point at which you can serialize the wormhole: *after* -establishing the invitation code, but before waiting for `get_verifier()` or -`get()`, or calling `send()`. If you are creating a new invitation code, the -correct time is during the callback fired by `get_code()`. If you are -accepting a pre-generated code, the time is just after calling `set_code()`. - -To properly checkpoint the process, you should store the first message -(returned by `start()`) next to the serialized wormhole instance, so you can -re-send it if necessary. +Application developers can use this one, or they can run their own (see the +`wormhole-server` command and the `src/wormhole/server/` directory) and +configure their clients to use it instead. This URL is passed as a unicode +string. ## Bytes, Strings, Unicode, and Python 3 @@ -283,4 +197,4 @@ in python3): * transit URLs * transit connection hints (e.g. "host:port") * application identifier -* derived-key "purpose" string: `w.derive_key(PURPOSE)` +* derived-key "purpose" string: `w.derive_key(PURPOSE, LENGTH)`