This is a very large branch that replaces many aspects of the wormhole
protocol. Clients that use code before this change (including the 0.7.6
release) will not be able to talk to clients after this change. They
won't even be able to talk to the relay.
Things that have changed:
* The server protocol has changed. A new public relay has been set up,
which listens on a different port.
* The blocking (non-Twisted) implementation has been removed. It will
return, built on top of the Twisted falvor, using the Crochet library.
* Persistence (state = wormhole.serialize()) has been removed. It will
return, in a form that works better for constantly-evolving Wormholes.
* API changes:
* 'from wormhole.wormhole import wormhole', rather than from
wormhole.twisted.transcribe (this is likely to change further)
* Create the Wormhole with the wormhole() function, rather than the
Wormhole() class constructor. You *must* pass reactor= to get a
Twisted-flavor wormhole (omitting reactor= will, in the future, give
you a blocking-flavor wormhole).
* w.get() and w.send(data), instead of w.get_data(phase) and
w.send_data(data, phase). Wormhole is now a sequential record pipe,
rather than a named-record channel. Internally, these APIs produce
numbered phases.
* verifier = yield w.verify(), instead of get_verifier(). The new
verify() defers until the connection has received the
key-confirmation message, and will errback with WrongPasswordError
if that message doesn't match.
* w.derive_key(purpose, length) now requires a length, instead of
defaulting to the NaCl SecretBox key size.
* w.close() now always defers until all outbound messages have been
delivered to the relay server, and the connection has closed. It
always returns a Deferred. Application code should close() before
calling os.exit(), to make sure your ACKs have been delivered.
* Any errors (WrongPasswordError, websocket dropped early) will cause
all pending Deferreds to errback, the nameplate and mailbox will be
released, and the websocket connection will be closed. w.close() is
still the right thing to call after an error, as it will defer until
the connection is finally dropped.
* The Wormhole object starts working as soon as wormhole() is called,
rather than waiting until an API method is invoked.
* There are more opportunities for parallelism, which should avoid a few
roundtrips and make things faster.
* We now use SPAKE2-0.7, which changes the key-derivation function to
one that hopefully matches a proposed SJCL implementation, enabling
future interoperability between python and javascript clients.
* We derive per-message keys differently, to prevent a particular kind
of reflection attack that was mitigated differently before.
* The server now manages "nameplates" and "mailboxes" separately (the
old server/protocol didn't make a distinction). A "nameplate" is a
channel with a short name (the number from the wormhole code) and
which only contains one value (a pointer to a mailbox). A "mailbox"
has a long random name and contains the usual queue of messages being
sent from one client to the other. This lets us drop the nameplate as
soon as the second side has connected (and switches to the mailbox),
so long file transfers don't hog the short wormhole codes for longer
than necessary.
* There is room for "nameplate attributes", which will (in the future)
be used to indicate the wordlist being used for the wormhole code,
allowing tab-completion for alternate wordlists, including languages
other than english.
* The new expectation is that nameplates and mailboxes will be deleted
if nobody is connected to them for a while (although this is not yet
implemented in the server). Applications which need extended offline
persistent channels will be able to ask for them when claiming the
nameplate.
Previously the encryption key used for "phase messages" (anything sent
from one side to the other, protected by the shared PAKE-generated
session key) was derived just from the session key and the phase name.
The two sides would use the same key for their first message (but with
random, thus different, nonces).
This uses the sending side's string (a random 5-byte/10-character hex
string) in the derivation process too, so the two sides use different
keys. This gives us an easy way to reject reflected messages. We already
ignore messages that claim to use a "side" which matches our own (to
ignore server echoes of our own outbound messages). With this change, an
attacker (or the server) can't swap in the payload of an outbound
message, change the "side" to make it look like a peer message, and then
let us decrypt it correctly.
It also changes the derivation function to combine the phase and side
values safely. This didn't matter much when we only had one
externally-provided string, but with two, there's an opportunity for
format confusion if they were combined with a simple delimiter. Now we
hash both values before concatenating them.
This breaks interoperability with clients from before this change. They
will always get WrongPasswordErrors.
* To avoid an incompatible patch that landed in Twisted trunk after the
16.1.1 release, autobahn pinned their requirement on Twisted to
be <=16.1.1 . However Twisted reverted the patch before making a
release. The new 16.2.0 is fine. Since autobahn has this pin, and
since pip doesn't do full dependency resolution, I must add the pin
too, so that 'pip install magic-wormhole' can work. I plan to remove
this pin as soon as autobahn does the same upstream.
https://github.com/crossbario/autobahn-python/issues/680
* A previous version of autobahn had a bug where it tried to import
something that wasn't actually depended upon, exposed by having pynacl
installed. Installing 'pytrie' manually fixed it. This doesn't seem to
be a problem anymore, so I'm removing the manual dependency.
* add "released" ack-response for "release" command, to sync w.close()
* move websocket URL to root
* relayurl= should now be a "ws://" URL
* many tests pass (except for test_twisted, which will be removed, and
test_scripts)
* still moving integration tests from test_twisted to
test_wormhole.Wormholes
This made sense for ServerSentEvent channels (which has no purpose once
the channel was gone), but not so much for websockets. And it prevented
testing duplicate-close.
Pass in a handle and a pair of functions, rather than an object with two
well-known methods. This should make it easier to subscribe to multiple
channels in the future.
