open.source/magic-wormhole
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

update docs

Browse Source
This commit is contained in:
Brian Warner 2016-05-25 20:58:53 -07:00
parent 708bcf36d4
commit 06d2a0be68
2 changed files with 53 additions and 150 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

65
README.md
View File

@ -53,8 +53,10 @@ delivered and used. If this does not feel strong enough, users can turn on
additional verification that doesn't depend upon the secrecy of the channel. additional verification that doesn't depend upon the secrecy of the channel.
The notion of a "magic wormhole" comes from the image of two distant wizards The notion of a "magic wormhole" comes from the image of two distant wizards
speaking the same phrase at the same time, and causing a connection to be speaking the same enchanted phrase at the same time, and causing a mystical
established between them. Transferring files securely should be that easy. connection to pop into existence between them. The wizards then throw books
into the wormhole and they fall out the other side. Transferring files
securely should be that easy.
## Design ## Design
@ -76,27 +78,22 @@ of success.
## Timing ## Timing
At present, the two clients must be run within about 3 minutes of each other, The program does not have any built-in timeouts, however it is expected that
as they will stop waiting after that time. This makes the tool most useful both clients will be run within an hour or so of each other. This makes the
for people who are having a real-time conversation already, and want to tool most useful for people who are having a real-time conversation already,
graduate to a secure connection. and want to graduate to a secure connection. Both clients must be left
running until the transfer has finished.
Future releases should increase that to several hours. This will enable a
mode in which two humans can decide on a code phrase offline, by choosing a
channel number and a few random words, and then go back home to their
computers later and begin the wormhole process. (This mode is already
supported, but is not currently easy to use because the two users must type
the phrases within three minutes of each other).
## Relays ## Relays
The wormhole library requires a "Rendezvous Server": a simple relay that The wormhole library requires a "Rendezvous Server": a simple WebSocket-based
delivers messages from one client to another. This allows the wormhole codes relay that delivers messages from one client to another. This allows the
to omit IP addresses and port numbers. The URL of a public server is baked wormhole codes to omit IP addresses and port numbers. The URL of a public
into the library for use as a default, and will be freely available until server is baked into the library for use as a default, and will be freely
volume or abuse makes it infeasible to support. Applications which desire available until volume or abuse makes it infeasible to support. Applications
more reliability can easily run their own relay and configure their clients which desire more reliability can easily run their own relay and configure
to use it instead. Code for the Rendezvous Server is included in the library. their clients to use it instead. Code for the Rendezvous Server is included
in the library.
The file-transfer commands also use a "Transit Relay", which is another The file-transfer commands also use a "Transit Relay", which is another
simple server that glues together two inbound TCP connections and transfers simple server that glues together two inbound TCP connections and transfers
@ -127,34 +124,26 @@ Both commands accept:
## Library ## Library
The `wormhole` module makes it possible for other applications to use these The `wormhole` module makes it possible for other applications to use these
code-protected channels. This includes blocking/synchronous support and code-protected channels. This includes Twisted support, and (in the future)
async/Twisted support, both for a symmetric scheme. The main module is named will include blocking/synchronous support too. See docs/api.md for details.
`wormhole.blocking.transcribe`, to reflect that it is for
synchronous/blocking code, and uses a PAKE mode whereby one user transcribes
their code to the other. (internal names may change in the future).
The file-transfer tools use a second module named The file-transfer tools use a second module named `wormhole.transit`, which
`wormhole.blocking.transit`, which provides an encrypted record-pipe. It provides an encrypted record-pipe. It knows how to use the Transit Relay as
knows how to use the Transit Relay as well as direct connections, and well as direct connections, and attempts them all in parallel.
attempts them all in parallel. `TransitSender` and `TransitReceiver` are `TransitSender` and `TransitReceiver` are distinct, although once the
distinct, although once the connection is established, data can flow in connection is established, data can flow in either direction. All data is
either direction. All data is encrypted (using nacl/libsodium "secretbox") encrypted (using nacl/libsodium "secretbox") using a key derived from the
using a key derived from the PAKE phase. See PAKE phase. See `src/wormhole/cli/cmd_send.py` for examples.
`src/wormhole/scripts/cmd_send.py` for examples.
## License, Compatibility ## License, Compatibility
This library is released under the MIT license, see LICENSE for details. This library is released under the MIT license, see LICENSE for details.
This library is compatible with python2.7, 3.3, 3.4, and 3.5 . It is probably This library is compatible with python2.7, 3.3, 3.4, and 3.5 . It is probably
compatible with py2.6, but the latest Twisted (15.5.0) is not. The compatible with py2.6, but the latest Twisted (>=15.5.0) is not. The
(daemonizing) 'wormhole server start' command does not yet work with py3, but (daemonizing) 'wormhole server start' command does not yet work with py3, but
will in the future once Twisted itself is finished being ported. will in the future once Twisted itself is finished being ported.
This package depends upon the SPAKE2, pynacl, requests, and argparse
libraries. To run a relay server, use the async support, or run the unit
tests, you must also install Twisted.
#### footnotes #### footnotes

138
docs/api.md
View File

@ -60,69 +60,28 @@ to be available, then returns it. The Wormhole is not meant as a long-term
communication channel, but some protocols work better if they can exchange an communication channel, but some protocols work better if they can exchange an
initial pair of messages (perhaps offering some set of negotiable initial pair of messages (perhaps offering some set of negotiable
capabilities), and then follow up with a second pair (to reveal the results capabilities), and then follow up with a second pair (to reveal the results
of the negotiation). Another use case is for an ACK that gets sent at the end of the negotiation).
of a file transfer: the Wormhole is held open until the Transit object
reports completion, and the last message is a hash of the file contents to
prove it was received correctly.
Note: the application developer must be careful to avoid deadlocks (if both Note: the application developer must be careful to avoid deadlocks (if both
sides want to `get()`, somebody has to `send()` first). sides want to `get()`, somebody has to `send()` first).
When both sides are done, they must call `close()`, to let the library know When both sides are done, they must call `close()`, to flush all pending
that the connection is complete and it can deallocate the channel. If you `send()` calls, deallocate the channel, and close the websocket connection.
forget to call `close()`, the server will not free the channel, and other
users will suffer longer invitation codes as a result. To encourage
`close()`, the library will log an error if a Wormhole object is destroyed
before being closed.
To make it easier to call `close()`, the blocking Wormhole objects can be
used as a context manager. Just put your code in the body of a `with
wormhole(ARGS) as w:` statement, and `close()` will automatically be called
when the block exits (either successfully or due to an exception).
## Examples
The synchronous+blocking flow looks like this:
```python
from wormhole.blocking.transcribe import wormhole
from wormhole.public_relay import RENDEZVOUS_RELAY
mydata = b"initiator's data"
with wormhole(u"appid", RENDEZVOUS_RELAY) as i:
code = i.get_code()
print("Invitation Code: %s" % code)
i.send(mydata)
theirdata = i.get()
print("Their data: %s" % theirdata.decode("ascii"))
```
```python
import sys
from wormhole.blocking.transcribe import wormhole
from wormhole.public_relay import RENDEZVOUS_RELAY
mydata = b"receiver's data"
code = sys.argv[1]
with wormhole(u"appid", RENDEZVOUS_RELAY) as r:
r.set_code(code)
r.send(mydata)
theirdata = r.get()
print("Their data: %s" % theirdata.decode("ascii"))
```
## Twisted ## Twisted
The Twisted-friendly flow looks like this: The Twisted-friendly flow looks like this (note that passing reactor= is how
you get a non-blocking Wormhole):
```python ```python
from twisted.internet import reactor from twisted.internet import reactor
from wormhole.public_relay import RENDEZVOUS_RELAY from wormhole.public_relay import RENDEZVOUS_RELAY
from wormhole.twisted.transcribe import wormhole from wormhole import wormhole
outbound_message = b"outbound data"
w1 = wormhole(u"appid", RENDEZVOUS_RELAY, reactor) w1 = wormhole(u"appid", RENDEZVOUS_RELAY, reactor)
d = w1.get_code() d = w1.get_code()
def _got_code(code): def _got_code(code):
print "Invitation Code:", code print "Invitation Code:", code
return w1.send(outbound_message) return w1.send(b"outbound data")
d.addCallback(_got_code) d.addCallback(_got_code)
d.addCallback(lambda _: w1.get()) d.addCallback(lambda _: w1.get())
def _got(inbound_message): def _got(inbound_message):
@ -156,15 +115,19 @@ the rest of the protocol to proceed. If they do not match, then the two
programs are not talking to each other (they may both be talking to a programs are not talking to each other (they may both be talking to a
man-in-the-middle attacker), and the protocol should be abandoned. man-in-the-middle attacker), and the protocol should be abandoned.
To retrieve the verifier, you call `w.get_verifier()` before any calls to To retrieve the verifier, you call `d=w.verify()` before any calls to
`send()/get()`. Turn this into hex or Base64 to print it, or render it as `send()/get()`. The Deferred will not fire until internal key-confirmation
ASCII-art, etc. Once the users are convinced that `get_verifier()` from both has taken place (meaning the two sides have exchanged their initial PAKE
sides are the same, call `send()/get()` to continue the protocol. If you call messages, and the wormhole codes matched), so `verify()` is also a good way
`send()/get()` before `get_verifier()`, it will perform the complete protocol to detect typos or mistakes entering the code. The Deferred will errback with
without pausing. wormhole.WrongPasswordError if the codes did not match, or it will callback
with the verifier bytes if they did match.
The Twisted form of `get_verifier()` returns a Deferred that fires with the Once retrieved, you can turn this into hex or Base64 to print it, or render
verifier bytes. it as ASCII-art, etc. Once the users are convinced that `verify()` from both
sides are the same, call `send()/get()` to continue the protocol. If you call
`send()/get()` before `verify()`, it will perform the complete protocol
without pausing.
## Generating the Invitation Code ## Generating the Invitation Code
@ -176,8 +139,8 @@ randomly-generated selection from the PGP wordlist, providing a default of 16
bits of entropy. The initiating program should display this code to the user, bits of entropy. The initiating program should display this code to the user,
who should transcribe it to the receiving user, who gives it to the Receiver who should transcribe it to the receiving user, who gives it to the Receiver
object by calling `set_code()`. The receiving program can also use object by calling `set_code()`. The receiving program can also use
`input_code_with_completion()` to use a readline-based input function: this `input_code()` to use a readline-based input function: this offers tab
offers tab completion of allocated channel-ids and known codewords. completion of allocated channel-ids and known codewords.
Alternatively, the human users can agree upon an invitation code themselves, Alternatively, the human users can agree upon an invitation code themselves,
and provide it to both programs later (both sides call `set_code()`). They and provide it to both programs later (both sides call `set_code()`). They
@ -212,59 +175,10 @@ the other. This must be the same for both clients, and is generally baked-in
to the application source code or default config. to the application source code or default config.
This library includes the URL of a public relay run by the author. This library includes the URL of a public relay run by the author.
Application developers can use this one, or they can run their own (see Application developers can use this one, or they can run their own (see the
src/wormhole/servers/relay.py) and configure their clients to use it instead. `wormhole-server` command and the `src/wormhole/server/` directory) and
This URL is passed as a unicode string. configure their clients to use it instead. This URL is passed as a unicode
string.
## Polling and Shutdown
TODO: this is mostly imaginary
The reactor-based (Twisted-style) forms of these objects need to establish
TCP connections, re-establish them if they are lost, and sometimes (for
transports that don't support long-running connections) poll for new
messages. They may also time out eventually. Longer delays mean less network
traffic, but higher latency.
These timers should be matched to the expectations, and expected behavior, of
your users. In a file-transfer application, where the users are sitting next
to each other, it is appropriate to poll very frequently (perhaps every
500ms) for a few minutes, then give up. In an email-like messaging program
where the introduction is establishing a long-term relationship, and the
program can store any outgoing messages until the connection is established,
it is probably better to poll once a minute for the first few minutes, then
back off to once an hour, and not give up for several days.
The `schedule=` constructor argument establishes the polling schedule. It
should contain a sorted list of (when, interval) tuples (both floats). At
`when` seconds after the first `start()` call, the polling interval will be
set to `interval`.
The `timeout=` argument provides a hard timeout. After this many seconds, the
sync will be abandoned, and all callbacks will errback with a TimeoutError.
Both have defaults suitable for face-to-face realtime setup environments.
## Serialization
TODO: only the Twisted form supports serialization so far
You may not be able to hold the Wormhole object in memory for the whole sync
process: maybe you allow it to wait for several days, but the program will be
restarted during that time. To support this, you can persist the state of the
object by calling `data = w.serialize()`, which will return a printable
bytestring (the JSON-encoding of a small dictionary). To restore, use `w =
wormhole_from_serialized(data, reactor)`.
There is exactly one point at which you can serialize the wormhole: *after*
establishing the invitation code, but before waiting for `get_verifier()` or
`get()`, or calling `send()`. If you are creating a new invitation code, the
correct time is during the callback fired by `get_code()`. If you are
accepting a pre-generated code, the time is just after calling `set_code()`.
To properly checkpoint the process, you should store the first message
(returned by `start()`) next to the serialized wormhole instance, so you can
re-send it if necessary.
## Bytes, Strings, Unicode, and Python 3 ## Bytes, Strings, Unicode, and Python 3
@ -283,4 +197,4 @@ in python3):
* transit URLs * transit URLs
* transit connection hints (e.g. "host:port") * transit connection hints (e.g. "host:port")
* application identifier * application identifier
* derived-key "purpose" string: `w.derive_key(PURPOSE)` * derived-key "purpose" string: `w.derive_key(PURPOSE, LENGTH)`