open.source/magic-wormhole
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

update docs

Browse Source
This commit is contained in:
Brian Warner 2016-05-25 20:58:53 -07:00
parent 708bcf36d4
commit 06d2a0be68
2 changed files with 53 additions and 150 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

65
README.md
View File

@ -53,8 +53,10 @@ delivered and used. If this does not feel strong enough, users can turn on
additional verification that doesn't depend upon the secrecy of the channel.
The notion of a "magic wormhole" comes from the image of two distant wizards
speaking the same phrase at the same time, and causing a connection to be
established between them. Transferring files securely should be that easy.
speaking the same enchanted phrase at the same time, and causing a mystical
connection to pop into existence between them. The wizards then throw books
into the wormhole and they fall out the other side. Transferring files
securely should be that easy.
## Design
@ -76,27 +78,22 @@ of success.
## Timing
At present, the two clients must be run within about 3 minutes of each other,
as they will stop waiting after that time. This makes the tool most useful
for people who are having a real-time conversation already, and want to
graduate to a secure connection.
Future releases should increase that to several hours. This will enable a
mode in which two humans can decide on a code phrase offline, by choosing a
channel number and a few random words, and then go back home to their
computers later and begin the wormhole process. (This mode is already
supported, but is not currently easy to use because the two users must type
the phrases within three minutes of each other).
The program does not have any built-in timeouts, however it is expected that
both clients will be run within an hour or so of each other. This makes the
tool most useful for people who are having a real-time conversation already,
and want to graduate to a secure connection. Both clients must be left
running until the transfer has finished.
## Relays
The wormhole library requires a "Rendezvous Server": a simple relay that
delivers messages from one client to another. This allows the wormhole codes
to omit IP addresses and port numbers. The URL of a public server is baked
into the library for use as a default, and will be freely available until
volume or abuse makes it infeasible to support. Applications which desire
more reliability can easily run their own relay and configure their clients
to use it instead. Code for the Rendezvous Server is included in the library.
The wormhole library requires a "Rendezvous Server": a simple WebSocket-based
relay that delivers messages from one client to another. This allows the
wormhole codes to omit IP addresses and port numbers. The URL of a public
server is baked into the library for use as a default, and will be freely
available until volume or abuse makes it infeasible to support. Applications
which desire more reliability can easily run their own relay and configure
their clients to use it instead. Code for the Rendezvous Server is included
in the library.
The file-transfer commands also use a "Transit Relay", which is another
simple server that glues together two inbound TCP connections and transfers
@ -127,34 +124,26 @@ Both commands accept:
## Library
The `wormhole` module makes it possible for other applications to use these
code-protected channels. This includes blocking/synchronous support and
async/Twisted support, both for a symmetric scheme. The main module is named
`wormhole.blocking.transcribe`, to reflect that it is for
synchronous/blocking code, and uses a PAKE mode whereby one user transcribes
their code to the other. (internal names may change in the future).
code-protected channels. This includes Twisted support, and (in the future)
will include blocking/synchronous support too. See docs/api.md for details.
The file-transfer tools use a second module named
`wormhole.blocking.transit`, which provides an encrypted record-pipe. It
knows how to use the Transit Relay as well as direct connections, and
attempts them all in parallel. `TransitSender` and `TransitReceiver` are
distinct, although once the connection is established, data can flow in
either direction. All data is encrypted (using nacl/libsodium "secretbox")
using a key derived from the PAKE phase. See
`src/wormhole/scripts/cmd_send.py` for examples.
The file-transfer tools use a second module named `wormhole.transit`, which
provides an encrypted record-pipe. It knows how to use the Transit Relay as
well as direct connections, and attempts them all in parallel.
`TransitSender` and `TransitReceiver` are distinct, although once the
connection is established, data can flow in either direction. All data is
encrypted (using nacl/libsodium "secretbox") using a key derived from the
PAKE phase. See `src/wormhole/cli/cmd_send.py` for examples.
## License, Compatibility
This library is released under the MIT license, see LICENSE for details.
This library is compatible with python2.7, 3.3, 3.4, and 3.5 . It is probably
compatible with py2.6, but the latest Twisted (15.5.0) is not. The
compatible with py2.6, but the latest Twisted (>=15.5.0) is not. The
(daemonizing) 'wormhole server start' command does not yet work with py3, but
will in the future once Twisted itself is finished being ported.
This package depends upon the SPAKE2, pynacl, requests, and argparse
libraries. To run a relay server, use the async support, or run the unit
tests, you must also install Twisted.
#### footnotes

138
docs/api.md
View File

@ -60,69 +60,28 @@ to be available, then returns it. The Wormhole is not meant as a long-term
communication channel, but some protocols work better if they can exchange an
initial pair of messages (perhaps offering some set of negotiable
capabilities), and then follow up with a second pair (to reveal the results
of the negotiation). Another use case is for an ACK that gets sent at the end
of a file transfer: the Wormhole is held open until the Transit object
reports completion, and the last message is a hash of the file contents to
prove it was received correctly.
of the negotiation).
Note: the application developer must be careful to avoid deadlocks (if both
sides want to `get()`, somebody has to `send()` first).
When both sides are done, they must call `close()`, to let the library know
that the connection is complete and it can deallocate the channel. If you
forget to call `close()`, the server will not free the channel, and other
users will suffer longer invitation codes as a result. To encourage
`close()`, the library will log an error if a Wormhole object is destroyed
before being closed.
To make it easier to call `close()`, the blocking Wormhole objects can be
used as a context manager. Just put your code in the body of a `with
wormhole(ARGS) as w:` statement, and `close()` will automatically be called
when the block exits (either successfully or due to an exception).
## Examples
The synchronous+blocking flow looks like this:
```python
from wormhole.blocking.transcribe import wormhole
from wormhole.public_relay import RENDEZVOUS_RELAY
mydata = b"initiator's data"
with wormhole(u"appid", RENDEZVOUS_RELAY) as i:
code = i.get_code()
print("Invitation Code: %s" % code)
i.send(mydata)
theirdata = i.get()
print("Their data: %s" % theirdata.decode("ascii"))
```
```python
import sys
from wormhole.blocking.transcribe import wormhole
from wormhole.public_relay import RENDEZVOUS_RELAY
mydata = b"receiver's data"
code = sys.argv[1]
with wormhole(u"appid", RENDEZVOUS_RELAY) as r:
r.set_code(code)
r.send(mydata)
theirdata = r.get()
print("Their data: %s" % theirdata.decode("ascii"))
```
When both sides are done, they must call `close()`, to flush all pending
`send()` calls, deallocate the channel, and close the websocket connection.
## Twisted
The Twisted-friendly flow looks like this:
The Twisted-friendly flow looks like this (note that passing reactor= is how
you get a non-blocking Wormhole):
```python
from twisted.internet import reactor
from wormhole.public_relay import RENDEZVOUS_RELAY
from wormhole.twisted.transcribe import wormhole
outbound_message = b"outbound data"
from wormhole import wormhole
w1 = wormhole(u"appid", RENDEZVOUS_RELAY, reactor)
d = w1.get_code()
def _got_code(code):
print "Invitation Code:", code
return w1.send(outbound_message)
return w1.send(b"outbound data")
d.addCallback(_got_code)
d.addCallback(lambda _: w1.get())
def _got(inbound_message):
@ -156,15 +115,19 @@ the rest of the protocol to proceed. If they do not match, then the two
programs are not talking to each other (they may both be talking to a
man-in-the-middle attacker), and the protocol should be abandoned.
To retrieve the verifier, you call `w.get_verifier()` before any calls to
`send()/get()`. Turn this into hex or Base64 to print it, or render it as
ASCII-art, etc. Once the users are convinced that `get_verifier()` from both
sides are the same, call `send()/get()` to continue the protocol. If you call
`send()/get()` before `get_verifier()`, it will perform the complete protocol
without pausing.
To retrieve the verifier, you call `d=w.verify()` before any calls to
`send()/get()`. The Deferred will not fire until internal key-confirmation
has taken place (meaning the two sides have exchanged their initial PAKE
messages, and the wormhole codes matched), so `verify()` is also a good way
to detect typos or mistakes entering the code. The Deferred will errback with
wormhole.WrongPasswordError if the codes did not match, or it will callback
with the verifier bytes if they did match.
The Twisted form of `get_verifier()` returns a Deferred that fires with the
verifier bytes.
Once retrieved, you can turn this into hex or Base64 to print it, or render
it as ASCII-art, etc. Once the users are convinced that `verify()` from both
sides are the same, call `send()/get()` to continue the protocol. If you call
`send()/get()` before `verify()`, it will perform the complete protocol
without pausing.
## Generating the Invitation Code
@ -176,8 +139,8 @@ randomly-generated selection from the PGP wordlist, providing a default of 16
bits of entropy. The initiating program should display this code to the user,
who should transcribe it to the receiving user, who gives it to the Receiver
object by calling `set_code()`. The receiving program can also use
`input_code_with_completion()` to use a readline-based input function: this
offers tab completion of allocated channel-ids and known codewords.
`input_code()` to use a readline-based input function: this offers tab
completion of allocated channel-ids and known codewords.
Alternatively, the human users can agree upon an invitation code themselves,
and provide it to both programs later (both sides call `set_code()`). They
@ -212,59 +175,10 @@ the other. This must be the same for both clients, and is generally baked-in
to the application source code or default config.
This library includes the URL of a public relay run by the author.
Application developers can use this one, or they can run their own (see
src/wormhole/servers/relay.py) and configure their clients to use it instead.
This URL is passed as a unicode string.
## Polling and Shutdown
TODO: this is mostly imaginary
The reactor-based (Twisted-style) forms of these objects need to establish
TCP connections, re-establish them if they are lost, and sometimes (for
transports that don't support long-running connections) poll for new
messages. They may also time out eventually. Longer delays mean less network
traffic, but higher latency.
These timers should be matched to the expectations, and expected behavior, of
your users. In a file-transfer application, where the users are sitting next
to each other, it is appropriate to poll very frequently (perhaps every
500ms) for a few minutes, then give up. In an email-like messaging program
where the introduction is establishing a long-term relationship, and the
program can store any outgoing messages until the connection is established,
it is probably better to poll once a minute for the first few minutes, then
back off to once an hour, and not give up for several days.
The `schedule=` constructor argument establishes the polling schedule. It
should contain a sorted list of (when, interval) tuples (both floats). At
`when` seconds after the first `start()` call, the polling interval will be
set to `interval`.
The `timeout=` argument provides a hard timeout. After this many seconds, the
sync will be abandoned, and all callbacks will errback with a TimeoutError.
Both have defaults suitable for face-to-face realtime setup environments.
## Serialization
TODO: only the Twisted form supports serialization so far
You may not be able to hold the Wormhole object in memory for the whole sync
process: maybe you allow it to wait for several days, but the program will be
restarted during that time. To support this, you can persist the state of the
object by calling `data = w.serialize()`, which will return a printable
bytestring (the JSON-encoding of a small dictionary). To restore, use `w =
wormhole_from_serialized(data, reactor)`.
There is exactly one point at which you can serialize the wormhole: *after*
establishing the invitation code, but before waiting for `get_verifier()` or
`get()`, or calling `send()`. If you are creating a new invitation code, the
correct time is during the callback fired by `get_code()`. If you are
accepting a pre-generated code, the time is just after calling `set_code()`.
To properly checkpoint the process, you should store the first message
(returned by `start()`) next to the serialized wormhole instance, so you can
re-send it if necessary.
Application developers can use this one, or they can run their own (see the
`wormhole-server` command and the `src/wormhole/server/` directory) and
configure their clients to use it instead. This URL is passed as a unicode
string.
## Bytes, Strings, Unicode, and Python 3
@ -283,4 +197,4 @@ in python3):
* transit URLs
* transit connection hints (e.g. "host:port")
* application identifier
* derived-key "purpose" string: `w.derive_key(PURPOSE)`
* derived-key "purpose" string: `w.derive_key(PURPOSE, LENGTH)`