2017-11-12 23:12:36 +00:00
|
|
|
# The Magic-Wormhole API
|
2015-02-10 09:39:17 +00:00
|
|
|
|
2017-03-06 18:49:11 +00:00
|
|
|
This library provides a mechanism to securely transfer small amounts
|
2015-02-10 09:39:17 +00:00
|
|
|
of data between two computers. Both machines must be connected to the
|
|
|
|
internet, but they do not need to have public IP addresses or know how to
|
|
|
|
contact each other ahead of time.
|
|
|
|
|
2018-11-03 14:38:58 +00:00
|
|
|
Security and connectivity is provided by means of a "wormhole code": a short
|
2017-03-06 18:49:11 +00:00
|
|
|
string that is transcribed from one machine to the other by the users at the
|
|
|
|
keyboard. This works in conjunction with a baked-in "rendezvous server" that
|
|
|
|
relays information from one machine to the other.
|
2015-02-10 09:39:17 +00:00
|
|
|
|
2016-05-12 22:42:40 +00:00
|
|
|
The "Wormhole" object provides a secure record pipe between any two programs
|
|
|
|
that use the same wormhole code (and are configured with the same application
|
|
|
|
ID and rendezvous server). Each side can send multiple messages to the other,
|
|
|
|
but the encrypted data for all messages must pass through (and be temporarily
|
|
|
|
stored on) the rendezvous server, which is a shared resource. For this
|
|
|
|
reason, larger data (including bulk file transfers) should use the Transit
|
2017-05-15 01:52:20 +00:00
|
|
|
class instead. The Wormhole can be used to create a Transit object for this
|
|
|
|
purpose. In the future, Transit will be deprecated, and this functionality
|
|
|
|
will be incorporated directly as a "dilated wormhole".
|
2016-05-12 22:42:40 +00:00
|
|
|
|
2017-03-06 18:49:11 +00:00
|
|
|
A quick example:
|
2016-05-12 22:42:40 +00:00
|
|
|
|
2017-03-06 18:49:11 +00:00
|
|
|
```python
|
|
|
|
import wormhole
|
|
|
|
from twisted.internet.defer import inlineCallbacks
|
|
|
|
|
|
|
|
@inlineCallbacks
|
|
|
|
def go():
|
|
|
|
w = wormhole.create(appid, relay_url, reactor)
|
2017-06-26 10:58:41 +00:00
|
|
|
w.allocate_code()
|
2017-05-12 18:54:33 +00:00
|
|
|
code = yield w.get_code()
|
2017-03-06 18:49:11 +00:00
|
|
|
print "code:", code
|
2017-05-12 18:54:33 +00:00
|
|
|
w.send_message(b"outbound data")
|
|
|
|
inbound = yield w.get_message()
|
2017-03-06 18:49:11 +00:00
|
|
|
yield w.close()
|
|
|
|
```
|
2016-05-12 22:42:40 +00:00
|
|
|
|
2017-03-06 18:49:11 +00:00
|
|
|
## Modes
|
2016-05-12 22:42:40 +00:00
|
|
|
|
2017-03-06 18:49:11 +00:00
|
|
|
The API comes in two flavors: Delegated and Deferred. Controlling the
|
|
|
|
Wormhole and sending data is identical in both, but they differ in how
|
|
|
|
inbound data and events are delivered to the application.
|
2015-02-10 09:39:17 +00:00
|
|
|
|
2017-03-06 18:49:11 +00:00
|
|
|
In Delegated mode, the Wormhole is given a "delegate" object, on which
|
|
|
|
certain methods will be called when information is available (e.g. when the
|
|
|
|
code is established, or when data messages are received). In Deferred mode,
|
|
|
|
the Wormhole object has methods which return Deferreds that will fire at
|
|
|
|
these same times.
|
2015-03-26 00:02:57 +00:00
|
|
|
|
2017-03-06 18:49:11 +00:00
|
|
|
Delegated mode:
|
2015-02-10 09:39:17 +00:00
|
|
|
|
|
|
|
```python
|
2017-03-06 18:49:11 +00:00
|
|
|
class MyDelegate:
|
|
|
|
def wormhole_got_code(self, code):
|
|
|
|
print("code: %s" % code)
|
2017-05-12 18:54:33 +00:00
|
|
|
def wormhole_got_message(self, msg): # called for each message
|
|
|
|
print("got data, %d bytes" % len(msg))
|
2017-03-06 18:49:11 +00:00
|
|
|
|
|
|
|
w = wormhole.create(appid, relay_url, reactor, delegate=MyDelegate())
|
2017-06-26 10:58:41 +00:00
|
|
|
w.allocate_code()
|
2015-02-10 09:39:17 +00:00
|
|
|
```
|
|
|
|
|
2017-03-06 18:49:11 +00:00
|
|
|
Deferred mode:
|
2015-06-21 02:03:10 +00:00
|
|
|
|
2015-02-10 09:39:17 +00:00
|
|
|
```python
|
2017-03-06 18:49:11 +00:00
|
|
|
w = wormhole.create(appid, relay_url, reactor)
|
2017-06-26 10:58:41 +00:00
|
|
|
w.allocate_code()
|
2017-03-06 18:49:11 +00:00
|
|
|
def print_code(code):
|
|
|
|
print("code: %s" % code)
|
2017-05-12 18:54:33 +00:00
|
|
|
w.get_code().addCallback(print_code)
|
|
|
|
def received(msg):
|
|
|
|
print("got data, %d bytes" % len(msg))
|
|
|
|
w.get_message().addCallback(received) # gets exactly one message
|
2015-02-10 09:39:17 +00:00
|
|
|
```
|
|
|
|
|
|
|
|
## Application Identifier
|
|
|
|
|
2015-03-26 00:02:57 +00:00
|
|
|
Applications using this library must provide an "application identifier", a
|
2015-10-07 00:02:52 +00:00
|
|
|
simple string that distinguishes one application from another. To ensure
|
|
|
|
uniqueness, use a domain name. To use multiple apps for a single domain,
|
|
|
|
append a URL-like slash and path, like `example.com/app1`. This string must
|
|
|
|
be the same on both clients, otherwise they will not see each other. The
|
|
|
|
invitation codes are scoped to the app-id. Note that the app-id must be
|
|
|
|
unicode, not bytes, so on python2 use `u"appid"`.
|
2015-02-10 09:39:17 +00:00
|
|
|
|
|
|
|
Distinct app-ids reduce the size of the connection-id numbers. If fewer than
|
2016-05-12 22:42:40 +00:00
|
|
|
ten Wormholes are active for a given app-id, the connection-id will only need
|
|
|
|
to contain a single digit, even if some other app-id is currently using
|
2015-02-10 09:39:17 +00:00
|
|
|
thousands of concurrent sessions.
|
|
|
|
|
2017-03-06 18:49:11 +00:00
|
|
|
## Rendezvous Servers
|
2015-03-26 00:02:57 +00:00
|
|
|
|
2017-03-06 18:49:11 +00:00
|
|
|
The library depends upon a "rendezvous server", which is a service (on a
|
2015-03-26 00:02:57 +00:00
|
|
|
public IP address) that delivers small encrypted messages from one client to
|
|
|
|
the other. This must be the same for both clients, and is generally baked-in
|
|
|
|
to the application source code or default config.
|
2015-02-10 09:39:17 +00:00
|
|
|
|
2017-03-06 18:49:11 +00:00
|
|
|
This library includes the URL of a public rendezvous server run by the
|
|
|
|
author. Application developers can use this one, or they can run their own
|
2020-09-18 15:10:48 +00:00
|
|
|
(see the [warner/magic-wormhole-mailbox-server
|
|
|
|
](https://github.com/warner/magic-wormhole-mailbox-server) repository)
|
2018-03-07 08:25:53 +00:00
|
|
|
and configure their clients to use it instead. The URL of the public
|
2021-08-25 19:19:21 +00:00
|
|
|
rendezvous server is passed as a unicode string. Note that because the server
|
2018-03-07 08:25:53 +00:00
|
|
|
actually speaks WebSockets, the URL starts with `ws:` instead of `http:`.
|
2017-03-06 18:49:11 +00:00
|
|
|
|
|
|
|
## Wormhole Parameters
|
|
|
|
|
|
|
|
All wormholes must be created with at least three parameters:
|
|
|
|
|
|
|
|
* `appid`: a (unicode) string
|
|
|
|
* `relay_url`: a (unicode) string
|
|
|
|
* `reactor`: the Twisted reactor object
|
|
|
|
|
|
|
|
In addition to these three, the `wormhole.create()` function takes several
|
|
|
|
optional arguments:
|
|
|
|
|
|
|
|
* `delegate`: provide a Delegate object to enable "delegated mode", or pass
|
|
|
|
None (the default) to get "deferred mode"
|
|
|
|
* `journal`: provide a Journal object to enable journaled mode. See
|
|
|
|
journal.md for details. Note that journals only work with delegated mode,
|
|
|
|
not with deferred mode.
|
|
|
|
* `tor_manager`: to enable Tor support, create a `wormhole.TorManager`
|
|
|
|
instance and pass it here. This will hide the client's IP address by
|
|
|
|
proxying all connections (rendezvous and transit) through Tor. It also
|
|
|
|
enables connecting to Onion-service transit hints, and (in the future) will
|
|
|
|
enable the creation of Onion-services for transit purposes.
|
|
|
|
* `timing`: this accepts a DebugTiming instance, mostly for internal
|
|
|
|
diagnostic purposes, to record the transmit/receive timestamps for all
|
|
|
|
messages. The `wormhole --dump-timing=` feature uses this to build a
|
|
|
|
JSON-format data bundle, and the `misc/dump-timing.py` tool can build a
|
|
|
|
scrollable timing diagram from these bundles.
|
|
|
|
* `welcome_handler`: this is a function that will be called when the
|
|
|
|
Rendezvous Server's "welcome" message is received. It is used to display
|
|
|
|
important server messages in an application-specific way.
|
2017-04-06 19:46:42 +00:00
|
|
|
* `versions`: this can accept a dictionary (JSON-encodable) of data that will
|
|
|
|
be made available to the peer via the `got_version` event. This data is
|
|
|
|
delivered before any data messages, and can be used to indicate peer
|
2017-03-06 18:49:11 +00:00
|
|
|
capabilities.
|
|
|
|
|
|
|
|
## Code Management
|
|
|
|
|
|
|
|
Each wormhole connection is defined by a shared secret "wormhole code". These
|
2018-05-29 06:42:04 +00:00
|
|
|
codes can be created by humans offline (by picking a unique number and some
|
|
|
|
secret words), but are more commonly generated by asking the library to make
|
|
|
|
one. In the "bin/wormhole" file-transfer tool, the default behavior is for
|
|
|
|
the sender's program to create the code, and for the receiver to type it in.
|
2017-03-06 18:49:11 +00:00
|
|
|
|
|
|
|
The code is a (unicode) string in the form `NNN-code-words`. The numeric
|
|
|
|
"NNN" prefix is the "channel id" or "nameplate", and is a short integer
|
|
|
|
allocated by talking to the rendezvous server. The rest is a
|
|
|
|
randomly-generated selection from the PGP wordlist, providing a default of 16
|
|
|
|
bits of entropy. The initiating program should display this code to the user,
|
|
|
|
who should transcribe it to the receiving user, who gives it to their local
|
|
|
|
Wormhole object by calling `set_code()`. The receiving program can also use
|
2017-03-09 10:05:36 +00:00
|
|
|
`input_code()` to use a readline-based input function: this offers tab
|
2017-03-06 18:49:11 +00:00
|
|
|
completion of allocated channel-ids and known codewords.
|
|
|
|
|
|
|
|
The Wormhole object has three APIs for generating or accepting a code:
|
|
|
|
|
2017-06-26 10:58:41 +00:00
|
|
|
* `w.allocate_code(length=2)`: this contacts the Rendezvous Server, allocates
|
2017-03-06 18:49:11 +00:00
|
|
|
a short numeric nameplate, chooses a configurable number of random words,
|
|
|
|
then assembles them into the code
|
2018-05-29 06:42:04 +00:00
|
|
|
* `w.set_code(code)`: this accepts the complete code as an argument
|
2017-03-09 10:05:36 +00:00
|
|
|
* `helper = w.input_code()`: this facilitates interactive entry of the code,
|
2017-03-06 18:49:11 +00:00
|
|
|
with tab-completion. The helper object has methods to return a list of
|
|
|
|
viable completions for whatever portion of the code has been entered so
|
|
|
|
far. A convenience wrapper is provided to attach this to the `rlcompleter`
|
|
|
|
function of libreadline.
|
|
|
|
|
2017-05-12 18:54:33 +00:00
|
|
|
No matter which mode is used, the `w.get_code()` Deferred (or
|
2017-03-06 18:49:11 +00:00
|
|
|
`delegate.wormhole_got_code(code)` callback) will fire when the code is
|
2017-06-26 10:58:41 +00:00
|
|
|
known. `get_code` is clearly necessary for `allocate_code`, since there's no
|
2017-03-06 18:49:11 +00:00
|
|
|
other way to learn what code was created, but it may be useful in other modes
|
|
|
|
for consistency.
|
|
|
|
|
|
|
|
The code-entry Helper object has the following API:
|
|
|
|
|
2017-03-12 17:38:48 +00:00
|
|
|
* `refresh_nameplates()`: requests an updated list of nameplates from the
|
2017-03-11 09:03:05 +00:00
|
|
|
Rendezvous Server. These form the first portion of the wormhole code (e.g.
|
|
|
|
"4" in "4-purple-sausages"). Note that they are unicode strings (so "4",
|
|
|
|
not 4). The Helper will get the response in the background, and calls to
|
2017-03-12 17:38:48 +00:00
|
|
|
`get_nameplate_completions()` after the response will use the new list.
|
2017-03-19 16:35:05 +00:00
|
|
|
Calling this after `h.choose_nameplate` will raise
|
|
|
|
`AlreadyChoseNameplateError`.
|
2017-04-06 01:26:28 +00:00
|
|
|
* `matches = h.get_nameplate_completions(prefix)`: returns (synchronously) a
|
|
|
|
set of completions for the given nameplate prefix, along with the hyphen
|
|
|
|
that always follows the nameplate (and separates the nameplate from the
|
|
|
|
rest of the code). For example, if the server reports nameplates 1, 12, 13,
|
|
|
|
24, and 170 are in use, `get_nameplate_completions("1")` will return
|
|
|
|
`{"1-", "12-", "13-", "170-"}`. You may want to sort these before
|
|
|
|
displaying them to the user. Raises `AlreadyChoseNameplateError` if called
|
|
|
|
after `h.choose_nameplate`.
|
2017-03-19 16:35:05 +00:00
|
|
|
* `h.choose_nameplate(nameplate)`: accepts a string with the chosen
|
|
|
|
nameplate. May only be called once, after which
|
|
|
|
`AlreadyChoseNameplateError` is raised. (in this future, this might
|
|
|
|
return a Deferred that fires (with None) when the nameplate's wordlist is
|
|
|
|
known (which happens after the nameplate is claimed, requiring a roundtrip
|
|
|
|
to the server)).
|
2017-04-06 01:26:28 +00:00
|
|
|
* `d = h.when_wordlist_is_available()`: return a Deferred that fires (with
|
|
|
|
None) when the wordlist is known. This can be used to block a readline
|
|
|
|
frontend which has just called `h.choose_nameplate()` until the resulting
|
|
|
|
wordlist is known, which can improve the tab-completion behavior.
|
|
|
|
* `matches = h.get_word_completions(prefix)`: return (synchronously) a set of
|
|
|
|
completions for the given words prefix. This will include a trailing hyphen
|
|
|
|
if more words are expected. The possible completions depend upon the
|
|
|
|
wordlist in use for the previously-claimed nameplate, so calling this
|
|
|
|
before `choose_nameplate` will raise `MustChooseNameplateFirstError`.
|
2017-03-19 16:35:05 +00:00
|
|
|
Calling this after `h.choose_words()` will raise `AlreadyChoseWordsError`.
|
2017-04-06 01:26:28 +00:00
|
|
|
Given a prefix like "su", this returns a set of strings which are potential
|
|
|
|
matches (e.g. `{"supportive-", "surrender-", "suspicious-"}`. The prefix
|
|
|
|
should not include the nameplate, but *should* include whatever words and
|
|
|
|
hyphens have been typed so far (the default wordlist uses alternate lists,
|
|
|
|
where even numbered words have three syllables, and odd numbered words have
|
|
|
|
two, so the completions depend upon how many words are present, not just
|
|
|
|
the partial last word). E.g. `get_word_completions("pr")` will return
|
|
|
|
`{"processor-", "provincial-", "proximate-"}`, while
|
|
|
|
`get_word_completions("opulent-pr")` will return `{"opulent-preclude",
|
|
|
|
"opulent-prefer", "opulent-preshrunk", "opulent-printer",
|
|
|
|
"opulent-prowler"}` (note the lack of a trailing hyphen, because the
|
|
|
|
wordlist is expecting a code of length two). If the wordlist is not yet
|
|
|
|
known, this returns an empty set. All return values will
|
2018-11-03 14:38:58 +00:00
|
|
|
`.startswith(prefix)`. The frontend is responsible for sorting the results
|
2017-04-06 01:26:28 +00:00
|
|
|
before display.
|
2017-03-12 17:38:48 +00:00
|
|
|
* `h.choose_words(words)`: call this when the user is finished typing in the
|
2017-03-11 09:03:05 +00:00
|
|
|
code. It does not return anything, but will cause the Wormhole's
|
2017-05-12 18:54:33 +00:00
|
|
|
`w.get_code()` (or corresponding delegate) to fire, and triggers the
|
2017-03-11 09:03:05 +00:00
|
|
|
wormhole connection process. This accepts a string like "purple-sausages",
|
2017-03-12 17:38:48 +00:00
|
|
|
without the nameplate. It must be called after `h.choose_nameplate()` or
|
2017-03-19 16:35:05 +00:00
|
|
|
`MustChooseNameplateFirstError` will be raised. May only be called once,
|
|
|
|
after which `AlreadyChoseWordsError` is raised.
|
2017-03-06 18:49:11 +00:00
|
|
|
|
2017-04-05 02:51:59 +00:00
|
|
|
The `input_with_completion` wrapper is a function that knows how to use the
|
|
|
|
code-entry helper to do tab completion of wormhole codes:
|
2017-03-06 18:49:11 +00:00
|
|
|
|
|
|
|
```python
|
2017-04-05 02:51:59 +00:00
|
|
|
from wormhole import create, input_with_completion
|
2017-03-06 18:49:11 +00:00
|
|
|
w = create(appid, relay_url, reactor)
|
2017-04-05 02:51:59 +00:00
|
|
|
input_with_completion("Wormhole code:", w.input_code(), reactor)
|
2017-05-12 18:54:33 +00:00
|
|
|
d = w.get_code()
|
2017-03-06 18:49:11 +00:00
|
|
|
```
|
|
|
|
|
2017-04-05 02:51:59 +00:00
|
|
|
This helper runs python's (raw) `input()` function inside a thread, since
|
|
|
|
`input()` normally blocks.
|
2017-03-06 18:49:11 +00:00
|
|
|
|
|
|
|
The two machines participating in the wormhole setup are not distinguished:
|
|
|
|
it doesn't matter which one goes first, and both use the same Wormhole
|
2017-06-26 10:58:41 +00:00
|
|
|
constructor function. However if `w.allocate_code()` is used, only one side
|
2017-03-06 18:49:11 +00:00
|
|
|
should use it.
|
|
|
|
|
2017-07-04 17:50:21 +00:00
|
|
|
Providing an invalid nameplate (which is easily caused by cut-and-paste
|
|
|
|
errors that include an extra space at the beginning, or which copy the words
|
|
|
|
but not the number) will raise a `KeyFormatError`, either in
|
|
|
|
`w.set_code(code)` or in `h.choose_nameplate()`.
|
|
|
|
|
2017-03-06 18:49:11 +00:00
|
|
|
## Offline Codes
|
|
|
|
|
|
|
|
In most situations, the "sending" or "initiating" side will call
|
2017-06-26 10:58:41 +00:00
|
|
|
`w.allocate_code()` and display the resulting code. The sending human reads
|
2017-03-06 18:49:11 +00:00
|
|
|
it and speaks, types, performs charades, or otherwise transmits the code to
|
|
|
|
the receiving human. The receiving human then types it into the receiving
|
|
|
|
computer, where it either calls `w.set_code()` (if the code is passed in via
|
2017-03-09 10:05:36 +00:00
|
|
|
argv) or `w.input_code()` (for interactive entry).
|
2017-03-06 18:49:11 +00:00
|
|
|
|
|
|
|
Usually one machine generates the code, and a pair of humans transcribes it
|
2017-06-26 10:58:41 +00:00
|
|
|
to the second machine (so `w.allocate_code()` on one side, and `w.set_code()`
|
2017-03-09 10:05:36 +00:00
|
|
|
or `w.input_code()` on the other). But it is also possible for the humans to
|
2017-03-06 18:49:11 +00:00
|
|
|
generate the code offline, perhaps at a face-to-face meeting, and then take
|
|
|
|
the code back to their computers. In this case, `w.set_code()` will be used
|
|
|
|
on both sides. It is unlikely that the humans will restrict themselves to a
|
|
|
|
pre-established wordlist when manually generating codes, so the completion
|
2017-03-09 10:05:36 +00:00
|
|
|
feature of `w.input_code()` is not helpful.
|
2017-03-06 18:49:11 +00:00
|
|
|
|
|
|
|
When the humans create an invitation code out-of-band, they are responsible
|
|
|
|
for choosing an unused channel-ID (simply picking a random 3-or-more digit
|
|
|
|
number is probably enough), and some random words. Dice, coin flips, shuffled
|
|
|
|
cards, or repeated sampling of a high-resolution stopwatch are all useful
|
|
|
|
techniques. The invitation code uses the same format either way: channel-ID,
|
|
|
|
a hyphen, and an arbitrary string. There is no need to encode the sampled
|
|
|
|
random values (e.g. by using the Diceware wordlist) unless that makes it
|
|
|
|
easier to transcribe: e.g. rolling 6 dice could result in a code like
|
|
|
|
"913-166532", and flipping 16 coins could result in "123-HTTHHHTTHTTHHTHH".
|
|
|
|
|
2017-04-03 21:23:03 +00:00
|
|
|
## Welcome Messages
|
|
|
|
|
|
|
|
The first message sent by the rendezvous server is a "welcome" message (a
|
2017-05-12 18:54:33 +00:00
|
|
|
dictionary). This is sent as soon as the client connects to the server,
|
|
|
|
generally before the code is established. Clients should use
|
|
|
|
`d=get_welcome()` to get and process the `motd` key (and maybe
|
|
|
|
`current_cli_version`) inside the welcome message.
|
2017-04-03 21:23:03 +00:00
|
|
|
|
|
|
|
The welcome message serves three main purposes:
|
|
|
|
|
|
|
|
* notify users about important server changes, such as CAPTCHA requirements
|
|
|
|
driven by overload, or donation requests
|
|
|
|
* enable future protocol negotiation between clients and the server
|
|
|
|
* advise users of the CLI tools (`wormhole send`) to upgrade to a new version
|
|
|
|
|
|
|
|
There are three keys currently defined for the welcome message, all of which
|
|
|
|
are optional (the welcome message omits "error" and "motd" unless the server
|
|
|
|
operator needs to signal a problem).
|
|
|
|
|
|
|
|
* `motd`: if this key is present, it will be a string with embedded newlines.
|
|
|
|
The client should display this string to the user, including a note that it
|
|
|
|
comes from the magic-wormhole Rendezvous Server and that server's URL.
|
|
|
|
* `error`: if present, the server has decided it cannot service this client.
|
|
|
|
The string will be wrapped in a `WelcomeError` (which is a subclass of
|
|
|
|
`WormholeError`), and all API calls will signal errors (pending Deferreds
|
|
|
|
will errback). The rendezvous connection will be closed.
|
|
|
|
* `current_cli_version`: if present, the server is advising instances of the
|
|
|
|
CLI tools (the `wormhole` command included in the python distribution) that
|
|
|
|
there is a newer release available, thus users should upgrade if they can,
|
|
|
|
because more features will be available if both clients are running the
|
|
|
|
same version. The CLI tools compare this string against their `__version__`
|
|
|
|
and can print a short message to stderr if an upgrade is warranted.
|
|
|
|
|
|
|
|
There is currently no facility in the server to actually send `motd`, but a
|
|
|
|
static `error` string can be included by running the server with
|
|
|
|
`--signal-error=MESSAGE`.
|
|
|
|
|
|
|
|
The main idea of `error` is to allow the server to cleanly inform the client
|
|
|
|
about some necessary action it didn't take. The server currently sends the
|
|
|
|
welcome message as soon as the client connects (even before it receives the
|
|
|
|
"claim" request), but a future server could wait for a required client
|
|
|
|
message and signal an error (via the Welcome message) if it didn't see this
|
|
|
|
extra message before the CLAIM arrived.
|
|
|
|
|
|
|
|
This could enable changes to the protocol, e.g. requiring a CAPTCHA or
|
|
|
|
proof-of-work token when the server is under DoS attack. The new server would
|
|
|
|
send the current requirements in an initial message (which old clients would
|
|
|
|
ignore). New clients would be required to send the token before their "claim"
|
|
|
|
message. If the server sees "claim" before "token", it knows that the client
|
|
|
|
is too old to know about this protocol, and it could send a "welcome" with an
|
|
|
|
`error` field containing instructions (explaining to the user that the server
|
|
|
|
is under attack, and they must either upgrade to a client that can speak the
|
|
|
|
new protocol, or wait until the attack has passed). Either case is better
|
|
|
|
than an opaque exception later when the required message fails to arrive.
|
|
|
|
|
|
|
|
(Note that the server can also send an explicit ERROR message at any time,
|
|
|
|
and the client should react with a ServerError. Versions 0.9.2 and earlier of
|
|
|
|
the library did not pay attention to the ERROR message, hence the server
|
|
|
|
should deliver errors in a WELCOME message if at all possible)
|
|
|
|
|
|
|
|
The `error` field is handled internally by the Wormhole object. The other
|
2017-05-12 18:54:33 +00:00
|
|
|
fields can be processed by application, by using `d=w.get_welcome()`. The
|
|
|
|
Deferred will fire with the full welcome dictionary, so any other keys that a
|
|
|
|
future server might send will be available to it.
|
|
|
|
|
|
|
|
Applications which need to display `motd` or an upgrade message, and wish to
|
|
|
|
do so before using stdin/stdout for interactive code entry (`w.input_code()`)
|
|
|
|
should wait for `get_welcome()` before starting the entry process:
|
|
|
|
|
|
|
|
```python
|
|
|
|
@inlineCallbacks
|
|
|
|
def go():
|
|
|
|
w = wormhole.create(appid, relay_url, reactor)
|
|
|
|
welcome = yield w.get_welcome()
|
|
|
|
if "motd" in welcome: print welcome["motd"]
|
|
|
|
input_with_completion("Wormhole code:", w.input_code(), reactor)
|
|
|
|
...
|
|
|
|
```
|
|
|
|
|
|
|
|
## Verifier
|
|
|
|
|
|
|
|
For extra protection against guessing attacks, Wormhole can provide a
|
|
|
|
"Verifier". This is a moderate-length series of bytes (a SHA256 hash) that is
|
|
|
|
derived from the supposedly-shared session key. If desired, both sides can
|
|
|
|
display this value, and the humans can manually compare them before allowing
|
|
|
|
the rest of the protocol to proceed. If they do not match, then the two
|
|
|
|
programs are not talking to each other (they may both be talking to a
|
|
|
|
man-in-the-middle attacker), and the protocol should be abandoned.
|
|
|
|
|
|
|
|
Deferred-mode applications can wait for `d=w.get_verifier()`: the Deferred
|
|
|
|
it returns will fire with the verifier. You can turn this into hex or Base64
|
|
|
|
to print it, or render it as ASCII-art, etc.
|
|
|
|
|
|
|
|
Asking the wormhole object for the verifier does not affect the flow of the
|
|
|
|
protocol. To benefit from verification, applications must refrain from
|
|
|
|
sending any data (with `w.send_message(data)`) until after the verifiers are
|
|
|
|
approved by the user. In addition, applications must queue or otherwise
|
|
|
|
ignore incoming (received) messages until that point. However once the
|
|
|
|
verifiers are confirmed, previously-received messages can be considered valid
|
|
|
|
and processed as usual.
|
2017-04-03 21:23:03 +00:00
|
|
|
|
2017-03-06 18:49:11 +00:00
|
|
|
## Events
|
|
|
|
|
|
|
|
As the wormhole connection is established, several events may be dispatched
|
|
|
|
to the application. In Delegated mode, these are dispatched by calling
|
|
|
|
functions on the delegate object. In Deferred mode, the application retrieves
|
|
|
|
Deferred objects from the wormhole, and event dispatch is performed by firing
|
|
|
|
those Deferreds.
|
|
|
|
|
2017-05-12 18:54:33 +00:00
|
|
|
Most applications will only use `code`, `received`, and `close`.
|
|
|
|
|
|
|
|
* code (`code = yield w.get_code()` / `dg.wormhole_got_code(code)`): fired
|
2017-06-26 10:58:41 +00:00
|
|
|
when the wormhole code is established, either after `w.allocate_code()`
|
2017-05-12 18:54:33 +00:00
|
|
|
finishes the generation process, or when the Input Helper returned by
|
|
|
|
`w.input_code()` has been told `h.set_words()`, or immediately after
|
|
|
|
`w.set_code(code)` is called. This is most useful after calling
|
2017-06-26 10:58:41 +00:00
|
|
|
`w.allocate_code()`, to show the generated code to the user so they can
|
2017-05-12 18:54:33 +00:00
|
|
|
transcribe it to their peer.
|
|
|
|
* key (`yield w.get_unverified_key()` /
|
|
|
|
`dg.wormhole_got_unverified_key(key)`): fired (with the raw master SPAKE2
|
|
|
|
key) when the key-exchange process has completed and a purported shared key
|
|
|
|
is established. At this point we do not know that anyone else actually
|
|
|
|
shares this key: the peer may have used the wrong code, or may have
|
|
|
|
disappeared altogether. To wait for proof that the key is shared, wait for
|
|
|
|
`get_verifier` instead. This event is really only useful for detecting that
|
|
|
|
the initiating peer has disconnected after leaving the initial PAKE
|
add w.when_key(), fix w.when_verified() to fire later
Previously, w.when_verified() was documented to fire only after a valid
encrypted message was received, but in fact it fired as soon as the shared
key was derived (before any encrypted messages are seen, so no actual
"verification" could occur yet).
This fixes that, and also adds a new w.when_key() API call which fires at the
earlier point. Having something which fires early is useful for the CLI
commands that want to print a pacifier message when the peer is responding
slowly. In particular it helps detect the case where 'wormhole send' has quit
early (after depositing the PAKE message on the server, but before the
receiver has started). In this case, the receiver will compute the shared
key, but then wait forever hoping for a VERSION that will never come. By
starting a timer when w.when_key() fires, and cancelling it when
w.when_verified() fires, we have a good place to tell the user that something
is taking longer than it should have.
This shifts responsibility for notifying Boss.got_verifier, out of Key and
into Receive, since Receive is what notices the first valid encrypted
message. It also shifts the Boss's ordering expectations: it now receives
B.happy() before B.got_verifier(), and consequently got_verifier ought to
arrive in the S2_happy state rather than S1_lonely.
2017-04-07 01:27:41 +00:00
|
|
|
message, to display a pacifying message to the user.
|
2017-05-12 18:54:33 +00:00
|
|
|
* verifier (`verifier = yield w.get_verifier()` /
|
|
|
|
`dg.wormhole_got_verifier(verifier)`: fired when the key-exchange process
|
|
|
|
has completed and a valid VERSION message has arrived. The "verifier" is a
|
|
|
|
byte string with a hash of the shared session key; clients can compare them
|
2017-03-07 11:34:36 +00:00
|
|
|
(probably as hex) to ensure that they're really talking to each other, and
|
2017-05-12 18:54:33 +00:00
|
|
|
not to a man-in-the-middle. When `get_verifier` happens, this side knows
|
2017-03-07 11:34:36 +00:00
|
|
|
that *someone* has used the correct wormhole code; if someone used the
|
|
|
|
wrong code, the VERSION message cannot be decrypted, and the wormhole will
|
|
|
|
be closed instead.
|
2017-05-12 18:54:33 +00:00
|
|
|
* versions (`versions = yield w.get_versions()` /
|
|
|
|
`dg.wormhole_got_versions(versions)`: fired when the VERSION message
|
|
|
|
arrives from the peer. This fires just after `verified`, but delivers the
|
|
|
|
"app_versions" data (as passed into `wormhole.create(versions=)`) instead
|
|
|
|
of the verifier string. This is mostly a hack to make room for
|
|
|
|
forwards-compatible changes to the CLI file-transfer protocol, which sends
|
|
|
|
a request in the first message (rather than merely sending the abilities of
|
|
|
|
each side).
|
|
|
|
* received (`yield w.get_message()` / `dg.wormhole_got_message(msg)`: fired
|
2017-03-06 18:49:11 +00:00
|
|
|
each time a data message arrives from the peer, with the bytestring that
|
2017-05-12 18:54:33 +00:00
|
|
|
the peer passed into `w.send_message(msg)`. This is the primary
|
|
|
|
data-transfer API.
|
2017-03-06 18:49:11 +00:00
|
|
|
* closed (`yield w.close()` / `dg.wormhole_closed(result)`: fired when
|
|
|
|
`w.close()` has finished shutting down the wormhole, which means all
|
|
|
|
nameplates and mailboxes have been deallocated, and the WebSocket
|
|
|
|
connection has been closed. This also fires if an internal error occurs
|
|
|
|
(specifically WrongPasswordError, which indicates that an invalid encrypted
|
|
|
|
message was received), which also shuts everything down. The `result` value
|
|
|
|
is an exception (or Failure) object if the wormhole closed badly, or a
|
|
|
|
string like "happy" if it had no problems before shutdown.
|
|
|
|
|
|
|
|
## Sending Data
|
|
|
|
|
|
|
|
The main purpose of a Wormhole is to send data. At any point after
|
2017-05-12 18:54:33 +00:00
|
|
|
construction, callers can invoke `w.send_message(msg)`. This will queue the
|
|
|
|
message if necessary, but (if all goes well) will eventually result in the
|
|
|
|
peer getting a `received` event and the data being delivered to the
|
|
|
|
application.
|
2017-03-06 18:49:11 +00:00
|
|
|
|
2017-05-12 18:54:33 +00:00
|
|
|
Since Wormhole provides an ordered record pipe, each call to `w.send_message`
|
|
|
|
will result in exactly one `received` event on the far side. Records are not
|
2017-03-06 18:49:11 +00:00
|
|
|
split, merged, dropped, or reordered.
|
|
|
|
|
2017-05-12 18:54:33 +00:00
|
|
|
Each side can do an arbitrary number of `send_message()` calls. The Wormhole
|
|
|
|
is not meant as a long-term communication channel, but some protocols work
|
|
|
|
better if they can exchange an initial pair of messages (perhaps offering
|
|
|
|
some set of negotiable capabilities), and then follow up with a second pair
|
|
|
|
(to reveal the results of the negotiation). The Rendezvous Server does not
|
|
|
|
currently enforce any particular limits on number of messages, size of
|
|
|
|
messages, or rate of transmission, but in general clients are expected to
|
|
|
|
send fewer than a dozen messages, of no more than perhaps 20kB in size
|
|
|
|
(remember that all these messages are temporarily stored in a SQLite database
|
|
|
|
on the server). A future version of the protocol may make these limits more
|
|
|
|
explicit, and will allow clients to ask for greater capacity when they
|
|
|
|
connect (probably by passing additional "mailbox attribute" parameters with
|
|
|
|
the `allocate`/`claim`/`open` messages).
|
2017-03-06 18:49:11 +00:00
|
|
|
|
|
|
|
For bulk data transfer, see "transit.md", or the "Dilation" section below.
|
|
|
|
|
|
|
|
## Closing
|
|
|
|
|
|
|
|
When the application is done with the wormhole, it should call `w.close()`,
|
|
|
|
and wait for a `closed` event. This ensures that all server-side resources
|
|
|
|
are released (allowing the nameplate to be re-used by some other client), and
|
|
|
|
all network sockets are shut down.
|
|
|
|
|
|
|
|
In Deferred mode, this just means waiting for the Deferred returned by
|
|
|
|
`w.close()` to fire. In Delegated mode, this means calling `w.close()` (which
|
|
|
|
doesn't return anything) and waiting for the delegate's `wormhole_closed()`
|
|
|
|
method to be called.
|
|
|
|
|
2017-04-06 20:22:15 +00:00
|
|
|
`w.close()` will errback (with some form of `WormholeError`) if anything went
|
|
|
|
wrong with the process, such as:
|
|
|
|
|
|
|
|
* `WelcomeError`: the server told us to signal an error, probably because the
|
|
|
|
client is too old understand some new protocol feature
|
|
|
|
* `ServerError`: the server rejected something we did
|
|
|
|
* `LonelyError`: we didn't hear from the other side, so no key was
|
|
|
|
established
|
|
|
|
* `WrongPasswordError`: we received at least one incorrectly-encrypted
|
|
|
|
message. This probably indicates that the other side used a different
|
|
|
|
wormhole code than we did, perhaps because of a typo, or maybe an attacker
|
|
|
|
tried to guess your code and failed.
|
|
|
|
|
|
|
|
If the wormhole was happy at the time it was closed, the `w.close()` Deferred
|
|
|
|
will callback (probably with the string "happy", but this may change in the
|
|
|
|
future).
|
|
|
|
|
2017-03-07 07:45:56 +00:00
|
|
|
## Serialization
|
|
|
|
|
2017-04-06 20:02:27 +00:00
|
|
|
(NOTE: this section is speculative: this code has not yet been written)
|
2017-03-07 07:45:56 +00:00
|
|
|
|
|
|
|
Wormhole objects can be serialized. This can be useful for apps which save
|
|
|
|
their own state before shutdown, and restore it when they next start up
|
|
|
|
again.
|
|
|
|
|
|
|
|
|
|
|
|
The `w.serialize()` method returns a dictionary which can be JSON encoded
|
|
|
|
into a unicode string (most applications will probably want to UTF-8 -encode
|
|
|
|
this into a bytestring before saving on disk somewhere).
|
|
|
|
|
|
|
|
To restore a Wormhole, call `wormhole.from_serialized(data, reactor,
|
|
|
|
delegate)`. This will return a wormhole in roughly the same state as was
|
|
|
|
serialized (of course all the network connections will be disconnected).
|
|
|
|
|
|
|
|
Serialization only works for delegated-mode wormholes (since Deferreds point
|
|
|
|
at functions, which cannot be serialized easily). It also only works for
|
|
|
|
"non-dilated" wormholes (see below).
|
|
|
|
|
|
|
|
To ensure correct behavior, serialization should probably only be done in
|
|
|
|
"journaled mode". See journal.md for details.
|
|
|
|
|
|
|
|
If you use serialization, be careful to never use the same partial wormhole
|
|
|
|
object twice.
|
2015-02-11 00:50:32 +00:00
|
|
|
|
2017-03-07 07:45:56 +00:00
|
|
|
## Dilation
|
|
|
|
|
2018-12-24 19:33:41 +00:00
|
|
|
(NOTE: this API is still in development)
|
|
|
|
|
2018-07-01 21:44:45 +00:00
|
|
|
To send bulk data, or anything more than a handful of messages, a Wormhole
|
|
|
|
can be "dilated" into a form that uses a direct TCP connection between the
|
|
|
|
two endpoints.
|
2017-03-07 07:45:56 +00:00
|
|
|
|
|
|
|
All wormholes start out "undilated". In this state, all messages are queued
|
|
|
|
on the Rendezvous Server for the lifetime of the wormhole, and server-imposed
|
|
|
|
number/size/rate limits apply. Calling `w.dilate()` initiates the dilation
|
2018-07-01 21:44:45 +00:00
|
|
|
process, and eventually yields a set of Endpoints. Once dilated, the usual
|
|
|
|
`.send_message()`/`.get_message()` APIs are disabled (TODO: really?), and
|
|
|
|
these endpoints can be used to establish multiple (encrypted) "subchannel"
|
|
|
|
connections to the other side.
|
|
|
|
|
|
|
|
Each subchannel behaves like a regular Twisted `ITransport`, so they can be
|
|
|
|
glued to the Protocol instance of your choice. They also implement the
|
|
|
|
IConsumer/IProducer interfaces.
|
|
|
|
|
|
|
|
These subchannels are *durable*: as long as the processes on both sides keep
|
|
|
|
running, the subchannel will survive the network connection being dropped.
|
|
|
|
For example, a file transfer can be started from a laptop, then while it is
|
|
|
|
running, the laptop can be closed, moved to a new wifi network, opened back
|
|
|
|
up, and the transfer will resume from the new IP address.
|
2017-03-07 07:45:56 +00:00
|
|
|
|
|
|
|
What's good about a non-dilated wormhole?:
|
|
|
|
|
|
|
|
* setup is faster: no delay while it tries to make a direct connection
|
|
|
|
* works with "journaled mode", allowing progress to be made even when both
|
|
|
|
sides are never online at the same time, by serializing the wormhole
|
|
|
|
|
|
|
|
What's good about dilated wormholes?:
|
|
|
|
|
|
|
|
* they support bulk data transfer
|
2017-04-06 20:02:27 +00:00
|
|
|
* you get flow control (backpressure), and IProducer/IConsumer
|
2017-03-07 07:45:56 +00:00
|
|
|
* throughput is faster: no store-and-forward step
|
|
|
|
|
|
|
|
Use non-dilated wormholes when your application only needs to exchange a
|
|
|
|
couple of messages, for example to set up public keys or provision access
|
2017-05-12 18:54:33 +00:00
|
|
|
tokens. Use a dilated wormhole to move files.
|
2017-03-07 07:45:56 +00:00
|
|
|
|
2018-07-01 21:44:45 +00:00
|
|
|
Dilated wormholes can provide multiple "subchannels": these are multiplexed
|
|
|
|
through the single (encrypted) TCP connection. Each subchannel is a separate
|
|
|
|
stream (offering IProducer/IConsumer for flow control), and is opened and
|
|
|
|
closed independently. A special "control channel" is available to both sides
|
|
|
|
so they can coordinate how they use the subchannels.
|
|
|
|
|
|
|
|
The `d = w.dilate()` Deferred fires with a triple of Endpoints:
|
|
|
|
|
|
|
|
```python
|
|
|
|
d = w.dilate()
|
|
|
|
def _dilated(res):
|
|
|
|
(control_channel_ep, subchannel_client_ep, subchannel_server_ep) = res
|
|
|
|
d.addCallback(_dilated)
|
|
|
|
```
|
|
|
|
|
|
|
|
The `control_channel_ep` endpoint is a client-style endpoint, so both sides
|
|
|
|
will connect to it with `ep.connect(factory)`. This endpoint is single-use:
|
|
|
|
calling `.connect()` a second time will fail. The control channel is
|
|
|
|
symmetric: it doesn't matter which side is the application-level
|
|
|
|
client/server or initiator/responder, if the application even has such
|
|
|
|
concepts. The two applications can use the control channel to negotiate who
|
|
|
|
goes first, if necessary.
|
|
|
|
|
|
|
|
The subchannel endpoints are *not* symmetric: for each subchannel, one side
|
|
|
|
must listen as a server, and the other must connect as a client. Subchannels
|
|
|
|
can be established by either side at any time. This supports e.g.
|
|
|
|
bidirectional file transfer, where either user of a GUI app can drop files
|
|
|
|
into the "wormhole" whenever they like.
|
|
|
|
|
|
|
|
The `subchannel_client_ep` on one side is used to connect to the other side's
|
|
|
|
`subchannel_server_ep`, and vice versa. The client endpoint is reusable. The
|
|
|
|
server endpoint is single-use: `.listen(factory)` may only be called once.
|
|
|
|
|
|
|
|
Applications are under no obligation to use subchannels: for many use cases,
|
|
|
|
the control channel is enough.
|
|
|
|
|
|
|
|
To use subchannels, once the wormhole is dilated and the endpoints are
|
|
|
|
available, the listening-side application should attach a listener to the
|
|
|
|
`subchannel_server_ep` endpoint:
|
|
|
|
|
|
|
|
```python
|
|
|
|
def _dilated(res):
|
|
|
|
(control_channel_ep, subchannel_client_ep, subchannel_server_ep) = res
|
|
|
|
f = Factory(MyListeningProtocol)
|
|
|
|
subchannel_server_ep.listen(f)
|
|
|
|
```
|
|
|
|
|
|
|
|
When the connecting-side application wants to connect to that listening
|
|
|
|
protocol, it should use `.connect()` with a suitable connecting protocol
|
|
|
|
factory:
|
|
|
|
|
|
|
|
```python
|
|
|
|
def _connect():
|
|
|
|
f = Factory(MyConnectingProtocol)
|
|
|
|
subchannel_client_ep.connect(f)
|
|
|
|
```
|
|
|
|
|
|
|
|
For a bidirectional file-transfer application, both sides will establish a
|
|
|
|
listening protocol. Later, if/when the user drops a file on the application
|
|
|
|
window, that side will initiate a connection, use the resulting subchannel to
|
|
|
|
transfer the single file, and then close the subchannel.
|
|
|
|
|
|
|
|
```python
|
|
|
|
def FileSendingProtocol(internet.Protocol):
|
|
|
|
def __init__(self, metadata, filename):
|
|
|
|
self.file_metadata = metadata
|
|
|
|
self.file_name = filename
|
|
|
|
def connectionMade(self):
|
|
|
|
self.transport.write(self.file_metadata)
|
|
|
|
sender = protocols.basic.FileSender()
|
|
|
|
f = open(self.file_name,"rb")
|
|
|
|
d = sender.beginFileTransfer(f, self.transport)
|
|
|
|
d.addBoth(self._done, f)
|
|
|
|
def _done(res, f):
|
|
|
|
self.transport.loseConnection()
|
|
|
|
f.close()
|
|
|
|
def _send(metadata, filename):
|
2018-12-24 19:33:41 +00:00
|
|
|
f = protocol.ClientCreator(reactor,
|
2018-07-01 21:44:45 +00:00
|
|
|
FileSendingProtocol, metadata, filename)
|
|
|
|
subchannel_client_ep.connect(f)
|
|
|
|
def FileReceivingProtocol(internet.Protocol):
|
|
|
|
state = INITIAL
|
|
|
|
def dataReceived(self, data):
|
|
|
|
if state == INITIAL:
|
|
|
|
self.state = DATA
|
|
|
|
metadata = parse(data)
|
|
|
|
self.f = open(metadata.filename, "wb")
|
|
|
|
else:
|
|
|
|
# local file writes are blocking, so don't bother with IConsumer
|
|
|
|
self.f.write(data)
|
|
|
|
def connectionLost(self, reason):
|
|
|
|
self.f.close()
|
|
|
|
def _dilated(res):
|
|
|
|
(control_channel_ep, subchannel_client_ep, subchannel_server_ep) = res
|
|
|
|
f = Factory(FileReceivingProtocol)
|
|
|
|
subchannel_server_ep.listen(f)
|
|
|
|
```
|
2017-04-06 20:02:27 +00:00
|
|
|
|
|
|
|
## Bytes, Strings, Unicode, and Python 3
|
|
|
|
|
|
|
|
All cryptographically-sensitive parameters are passed as bytes ("str" in
|
|
|
|
python2, "bytes" in python3):
|
|
|
|
|
|
|
|
* verifier string
|
|
|
|
* data in/out
|
|
|
|
* transit records in/out
|
|
|
|
|
|
|
|
Other (human-facing) values are always unicode ("unicode" in python2, "str"
|
|
|
|
in python3):
|
|
|
|
|
|
|
|
* wormhole code
|
|
|
|
* relay URL
|
|
|
|
* transit URLs
|
|
|
|
* transit connection hints (e.g. "host:port")
|
|
|
|
* application identifier
|
|
|
|
* derived-key "purpose" string: `w.derive_key(PURPOSE, LENGTH)`
|
|
|
|
|
|
|
|
## Full API list
|
|
|
|
|
2017-05-12 18:54:33 +00:00
|
|
|
action | Deferred-Mode | Delegated-Mode
|
|
|
|
------------------ | ------------------ | --------------
|
|
|
|
. | d=w.get_welcome() | dg.wormhole_got_welcome(welcome)
|
2017-06-26 10:58:41 +00:00
|
|
|
w.allocate_code() | |
|
2017-05-12 18:54:33 +00:00
|
|
|
h=w.input_code() | |
|
2017-06-26 10:58:41 +00:00
|
|
|
w.set_code(code) | |
|
2017-05-12 18:54:33 +00:00
|
|
|
. | d=w.get_code() | dg.wormhole_got_code(code)
|
|
|
|
. | d=w.get_unverified_key() | dg.wormhole_got_unverified_key(key)
|
|
|
|
. | d=w.get_verifier() | dg.wormhole_got_verifier(verifier)
|
|
|
|
. | d=w.get_versions() | dg.wormhole_got_versions(versions)
|
|
|
|
key=w.derive_key(purpose, length) | |
|
|
|
|
w.send_message(msg) | |
|
|
|
|
. | d=w.get_message() | dg.wormhole_got_message(msg)
|
|
|
|
w.close() | | dg.wormhole_closed(result)
|
|
|
|
. | d=w.close() |
|