get things from one computer to another, safely

Go to file

Brian Warner 1db485d493 Antithesis: merge branch 'send-multiple' (no code changes) At PyCon 2007, Robert "r0ml" Lefkowitz gave a keynote comparing the rise of actual-paper literacy (the development of whitespace, punctuation, sentences, pages, bookmarks, an index, argumentative forms, forensics, rhetoric) with the rise of computer-language literacy (macros, multicharacter variable names, loops, comments, OOP, reusable code, collaborative review). He pointed out that many classical written techniques do not yet have analogues in our programming practices, citing "antithesis" as one such tool. In writing, Antithesis is where you lay out the opposite of the idea you really want to convey, to explain what's wrong with it. By including antithesis, you can capture some valuable knowledge, and might anticipate (and head off) future "but what about X" arguments. This branch documents a wrong turn: an API that I thought would be a good idea, but which turned out to not be worth it. Rather than discarding the branch entirely, I decided to merge the history (but not the changes) into trunk, so I don't lose the decision-making process or the implementation. The impetus for this feature was the unfortunate extra round trip introduced when I added "confirmation" messages in `3220014`. Confirmation messages were necessary to avoid a hang when "wormhole receive" was given the wrong codephrase. The previous messages flow was: * sender->receiver: PAKE1 * receiver->sender: PAKE2 * sender->receiver: DATA * receiver->sender: ACK Both sides compute a key when they hear the other's PAKE message, but if the wormhole codes are different, they will compute different keys. When they discover this, they should raise a WrongPasswordError to notify their users. But when exactly does this happen? The receiver learns about this when they hear the DATA message, and (before commit `d1cf1c6`) would hang up immediately, before allowing the application code to send any ACK. As a result, the sender never sees the ACK (which would be mis-encrypted, and thus reveal that the codes were different), and waits forever. Adding confirmations to the flow gives us: * sender->receiver: PAKE1 * receiver->sender: PAKE2 * sender->receiver: CONFIRM1 * sender->receiver: DATA * receiver->sender: CONFIRM2 * receiver->sender: ACK Both sides send a CONFIRM message as soon as they hear the other's PAKE message, before computing a shared key or returning control to application code. The receiver's CONFIRM2 goes out before it processes DATA. A moment later, in the same function call, the receiver gets a decrypt error on the DATA message and aborts the connection. However the sender will see CONFIRM2 arrive, tries (and fails) to validate it, and can abort the connection itself, giving the "wormhole send" user a clear error message (WrongPasswordError). The sender is now sending two messages in close succession: CONFIRM1 and DATA. Both are sent in response to the incoming PAKE2 message, and in an ideal world both would be sent in the same round trip. In the hopes of achieving this, I spent quite a bit of time changing the architecture on both client and server sides, and improving the server API: * POST to the server would accept multiple messages, not just one * the EventSource "watch" API could deliver multiple messages in a single line Those changes worked, however when I finally came to change the sender to put both messages in a single call, I found that I could not: the messages come from very different places. The CONFIRM1 is sent just after waiting for (and receiving) PAKE2, in `_get_key()`. The DATA message it sent after getting the key, in `send_data()`. Despite both happening in the same turn of the event loop (or, equivalently, in the same stack frame), the Wormhole API would have to be unpleasantly changed to make it possible for both messages to go out together. In particular, `_get_key()` is called from both `send_data()` (which sends DATA) and `get_verifier()` (which deliberately does not). The least-bad approach I could come up with was to have CONFIRM1 be accumulated in a Nagle-like queue until the caller allowed all messages to be sent. In the end I decided it wasn't worth the complexity. Sufficiently motivated senders can manually pipeline the two messages without explicit API support (there's no reason an async sender must wait for CONFIRM1 to be delivered before sending DATA down the same wire). And receivers don't really need their "watch" (EventSource) API to deliver batches of messages instead of single ones: apps should treat messages as an unordered set anyways. I also realized that the prioritization aspect of the new "get_first_of" API was unnecessary: any client that wants a CONFIRM message for key confirmation would be just as well served by any DATA message (either can be used for key-confirmation): the important property is that we accept CONFIRM in addition to a DATA, because in some error cases we'll never see the DATA (ACK). So, having watched the reasons for these changes crumble to the ground, I decided to not land them. But the lessons learned in the process were still valuable, so I'm including this branch in the mainline history even though the actual code changes were abandoned.		2015-11-23 16:29:40 -08:00
docs	api.md: fix typo	2015-11-12 09:30:48 -08:00
src/wormhole	test_server: rename some functions	2015-11-22 18:06:42 -08:00
.gitattributes	initial setup.py, versioneer-ification	2015-02-10 00:48:19 -08:00
.gitignore	relay: add database, not used yet	2015-05-04 18:24:23 -07:00
.travis.yml	travis: we're now py3.3-compatible too	2015-09-28 16:32:49 -07:00
LICENSE	Initial commit	2015-02-10 00:38:44 -08:00
MANIFEST.in	add docs for 0.3.0	2015-06-24 00:26:03 -07:00
NEWS.md	add 'wormhole send --text -' to read message from stdin	2015-11-11 17:27:26 -08:00
README.md	Updated README.md with Installation instruction	2015-10-12 17:44:36 -07:00
setup.cfg	setup.cfg: make a py2/py3 "universal" wheel	2015-10-07 17:13:39 -07:00
setup.py	make blocking/send-text work on py3, add dependency on 'six'	2015-09-28 00:24:36 -07:00
versioneer.py	upgrade to versioneer-0.15, fixes 'setup.py develop'	2015-05-31 16:39:39 -07:00

README.md

Magic Wormhole

Get things from one computer to another, safely.

This package provides a library and a command-line tool named wormhole, which makes it possible to get short pieces of text (and arbitrary-sized files) from one computer to another. The two endpoints are identified by using identical "wormhole codes": in general, the sending machine generates and displays the code, which must then be typed into the receiving machine.

The codes are short and human-pronounceable, using a phonetically-distinct wordlist. The receiving side offers tab-completion on the codewords, so usually only a few characters must be typed. Wormhole codes are single-use and do not need to be memorized.

Installation

$ pip install magic-wormhole

Motivation

Moving a file to a friend's machine, when the humans can speak to each other but the computers cannot
Delivering a properly-random password to a new user via the phone
Supplying an SSH public key for future login use

Copying files onto a USB stick requires physical proximity, and is uncomfortable for transferring long-term secrets because flash memory is hard to erase. Copying files with ssh/scp is fine, but requires previous arrangements and an account on the target machine, and how do you bootstrap the account? Copying files through email first requires transcribing an email address in the opposite direction, and is even worse for secrets, because email is unencrypted. Copying files through encrypted email requires bootstrapping a GPG key as well as an email address. Copying files through Dropbox is not secure against the Dropbox server and results in a large URL that must be transcribed. Using a URL shortener adds an extra step and reveals the URL to the shortening service.

Many common use cases start with a human-mediated communication channel, such as IRC, IM, email, a phone call, or a face-to-face conversation. Some of these are basically secret, or are "secret enough" to last until the code is delivered and used. If this does not feel strong enough, users can turn on additional verification that doesn't depend upon the secrecy of the channel.

The notion of a "magic wormhole" comes from the image of two distant wizards speaking the same phrase at the same time, and causing a connection to be established between them. Transferring files securely should be that easy.

Design

The wormhole tool uses PAKE "Password-Authenticated Key Exchange", a family of cryptographic algorithms that uses a short low-entropy password to establish a strong high-entropy shared key. This key can then be used to encrypt data. wormhole uses the SPAKE2 algorithm, due to Abdalla and Pointcheval1.

PAKE effectively trades off interaction against offline attacks. The only way for a network attacker to learn the shared key is to perform a man-in-the-middle attack during the initial connection attempt, and to correctly guess the code being used by both sides. Their chance of doing this is inversely proportional to the entropy of the wormhole code. The default (which can be changed) uses 16-bit codes, so for each use of the tool, an attacker gets a 1-in-65536 chance of success. As such, users can expect to see many error messages before the attacker has a reasonable chance of success.

Timing

At present, the two clients must be run within about 3 minutes of each other, as they will stop waiting after that time. This makes the tool most useful for people who are having a real-time conversation already, and want to graduate to a secure connection.

Future releases should increase that to several hours. This will enable a mode in which two humans can decide on a code phrase offline, by choosing a channel number and a few random words, and then go back home to their computers later and begin the wormhole process. (This mode is already supported, but is not currently easy to use because the two users must type the phrases within three minutes of each other).

Relays

The wormhole library requires a "Rendezvous Server": a simple relay that delivers messages from one client to another. This allows the wormhole codes to omit IP addresses and port numbers. The URL of a public server is baked into the library for use as a default, and will be freely available until volume or abuse makes it infeasible to support. Applications which desire more reliability can easily run their own relay and configure their clients to use it instead. Code for the Rendezvous Server is included in the library.

The file-transfer commands also use a "Transit Relay", which is another simple server that glues together two inbound TCP connections and transfers data on each to the other. The wormhole send file mode shares the IP addresses of each client with the other (inside the encrypted message), and both clients first attempt to connect directly. If this fails, they fall back to using the transit relay. As before, the host/port of a public server is baked into the library, and should be sufficient to handle moderate traffic.

The protocol includes provisions to deliver notices and error messages to clients: if either relay must be shut down, these channels will be used to provide information about alternatives.

CLI tool

wormhole send --text TEXT
wormhole send FILENAME
wormhole receive

Both commands accept:

--relay-url URL : override the rendezvous server URL
--transit-helper tcp:HOST:PORT: override the Transit Relay
--code-length WORDS: use more or fewer than 2 words for the code
--verify : add extra verification

Library

The wormhole module makes it possible for other applications to use these code-protected channels. This includes blocking/synchronous support and async/Twisted support, both for a symmetric scheme. The main module is named wormhole.blocking.transcribe, to reflect that it is for synchronous/blocking code, and uses a PAKE mode whereby one user transcribes their code to the other. (internal names may change in the future).

The file-transfer tools use a second module named wormhole.blocking.transit, which provides an encrypted record-pipe. It knows how to use the Transit Relay as well as direct connections, and attempts them all in parallel. TransitSender and TransitReceiver are distinct, although once the connection is established, data can flow in either direction. All data is encrypted (using nacl/libsodium "secretbox") using a key derived from the PAKE phase. See src/wormhole/scripts/cmd_send.py for examples.

License, Compatibility

This library is released under the MIT license, see LICENSE for details.

This library is compatible with python2.6, 2.7, 3.3, 3.4, and 3.5 . The async support does not yet work with py3, but will in the future once Twisted itself is finished being ported.

This package depends upon the SPAKE2, pynacl, requests, and argparse libraries. To run a relay server, use the async support, or run the unit tests, you must also install Twisted.