This will allow a future peer to figure out what transit modes we can
and cannot do, and thus avoid spinning up expensive modes that we won't
be able to use (e.g. WebRTC).
The new autobahn release removes the Twisted pin, so we no longer need
one. Depend on the new autobahn to flush out any environments that are
still using the old one (and thus can't use the current Twisted).
Thanks https://github.com/crossbario/autobahn-python/issues/680 !
This enhances the ACK that wormhole-receive returns when it finishes
receiving all the data to be a dictionary. The dict includes the SHA256
hash of everything it received, and the sender checks this for a match
before declaring the transfer to be a success. This guards against data
being shuffled somehow during transit.
This better reflects the purpose of the message. Key confirmation is a
side-effect.
This patch only changes the "phase:" name and the key-derivation string.
A subsequent patch will modify the function and variable names to match.
The file-send protocol now sends a "hints-v1" key in the "transit"
message, which contains a list of JSON data structures that describe the
connection hints (a mixture of direct, tor, and relay hints, for now).
Previously the direct/tor and relay hints were sent in different keys,
and all were sent as strings like "tcp:hostname:1234" which had to be
parsed by the recipient.
The new structures include a version string, to make it easier to add
new types in the future. Transit logs+ignores hints it cannot
understand.
In the future, both sides should expect to receive "transit" messages at
any time, and they will add to the list of hints that they should try.
For now, each side only sends a single transit message, before they send
the offer (sender) or answer (receiver).
This moves us slowly towards a file-transfer protocol that exchanges
multiple messages, with a single offer (sender->receiver) and
answer (receiver->sender), and one or more connection hint messages (in
either direction) that appear gradually over time as connection
providers come online.
At present the protocol still expects the whole hint list to be present
in the offer/answer message.
This should enable forwards-compatibility with clients which send extra
data, like a pre-PAKE "auxdata" message that hints we should spin up a
tor client (because they can connect to it) while we're waiting for the
user to type in the wormhole code.
This is a very large branch that replaces many aspects of the wormhole
protocol. Clients that use code before this change (including the 0.7.6
release) will not be able to talk to clients after this change. They
won't even be able to talk to the relay.
Things that have changed:
* The server protocol has changed. A new public relay has been set up,
which listens on a different port.
* The blocking (non-Twisted) implementation has been removed. It will
return, built on top of the Twisted falvor, using the Crochet library.
* Persistence (state = wormhole.serialize()) has been removed. It will
return, in a form that works better for constantly-evolving Wormholes.
* API changes:
* 'from wormhole.wormhole import wormhole', rather than from
wormhole.twisted.transcribe (this is likely to change further)
* Create the Wormhole with the wormhole() function, rather than the
Wormhole() class constructor. You *must* pass reactor= to get a
Twisted-flavor wormhole (omitting reactor= will, in the future, give
you a blocking-flavor wormhole).
* w.get() and w.send(data), instead of w.get_data(phase) and
w.send_data(data, phase). Wormhole is now a sequential record pipe,
rather than a named-record channel. Internally, these APIs produce
numbered phases.
* verifier = yield w.verify(), instead of get_verifier(). The new
verify() defers until the connection has received the
key-confirmation message, and will errback with WrongPasswordError
if that message doesn't match.
* w.derive_key(purpose, length) now requires a length, instead of
defaulting to the NaCl SecretBox key size.
* w.close() now always defers until all outbound messages have been
delivered to the relay server, and the connection has closed. It
always returns a Deferred. Application code should close() before
calling os.exit(), to make sure your ACKs have been delivered.
* Any errors (WrongPasswordError, websocket dropped early) will cause
all pending Deferreds to errback, the nameplate and mailbox will be
released, and the websocket connection will be closed. w.close() is
still the right thing to call after an error, as it will defer until
the connection is finally dropped.
* The Wormhole object starts working as soon as wormhole() is called,
rather than waiting until an API method is invoked.
* There are more opportunities for parallelism, which should avoid a few
roundtrips and make things faster.
* We now use SPAKE2-0.7, which changes the key-derivation function to
one that hopefully matches a proposed SJCL implementation, enabling
future interoperability between python and javascript clients.
* We derive per-message keys differently, to prevent a particular kind
of reflection attack that was mitigated differently before.
* The server now manages "nameplates" and "mailboxes" separately (the
old server/protocol didn't make a distinction). A "nameplate" is a
channel with a short name (the number from the wormhole code) and
which only contains one value (a pointer to a mailbox). A "mailbox"
has a long random name and contains the usual queue of messages being
sent from one client to the other. This lets us drop the nameplate as
soon as the second side has connected (and switches to the mailbox),
so long file transfers don't hog the short wormhole codes for longer
than necessary.
* There is room for "nameplate attributes", which will (in the future)
be used to indicate the wordlist being used for the wormhole code,
allowing tab-completion for alternate wordlists, including languages
other than english.
* The new expectation is that nameplates and mailboxes will be deleted
if nobody is connected to them for a while (although this is not yet
implemented in the server). Applications which need extended offline
persistent channels will be able to ask for them when claiming the
nameplate.
Previously the encryption key used for "phase messages" (anything sent
from one side to the other, protected by the shared PAKE-generated
session key) was derived just from the session key and the phase name.
The two sides would use the same key for their first message (but with
random, thus different, nonces).
This uses the sending side's string (a random 5-byte/10-character hex
string) in the derivation process too, so the two sides use different
keys. This gives us an easy way to reject reflected messages. We already
ignore messages that claim to use a "side" which matches our own (to
ignore server echoes of our own outbound messages). With this change, an
attacker (or the server) can't swap in the payload of an outbound
message, change the "side" to make it look like a peer message, and then
let us decrypt it correctly.
It also changes the derivation function to combine the phase and side
values safely. This didn't matter much when we only had one
externally-provided string, but with two, there's an opportunity for
format confusion if they were combined with a simple delimiter. Now we
hash both values before concatenating them.
This breaks interoperability with clients from before this change. They
will always get WrongPasswordErrors.
