Commit Graph

231 Commits

Author SHA1 Message Date
meejah
e16b53817e Refactor to use Click 2016-06-22 01:11:07 -06:00
meejah
07b4067727 unit-test for relay request, and fix Data() handling 2016-06-20 16:32:07 -07:00
laharah
150caedf73 u"" cleanup on test_scripts.py 2016-06-04 14:09:19 -07:00
laharah
bb970dafc0 u"" cleanup on test_server.py 2016-06-04 14:09:19 -07:00
laharah
2dd76f57c0 u"" cleanup on test_transit.py 2016-06-04 14:09:19 -07:00
laharah
aff13e9121 u"" cleanup on test_util.py 2016-06-04 14:09:19 -07:00
laharah
8fa40d3cd5 u"" cleanup on test_wormhole.py 2016-06-04 14:09:19 -07:00
laharah
6a73d50fdd added unicode_literals import to all apropriate modules
bug in twisted serverFromString prevents test.common and transit
from using unicode properly should revisit if twisted gets patched
2016-06-04 12:42:59 -07:00
Brian Warner
e13b95660b test_transit: tolerate a localhost-only system
e.g. Appveyor . Refs #16.
2016-06-03 21:46:56 -07:00
higs4281
55f2fcc3a7 remove commented line 2016-06-03 18:18:34 -07:00
higs4281
eaed2f0a12 Prevent transmit from suggesting 127.0.0.1
The test runs the listener locally, which can turn up 127.0.0.1.
The added code in transmit stops the hint.
2016-06-03 18:18:34 -07:00
Brian Warner
26445c3fc4 test_scripts: fix on windows 2016-06-03 16:43:22 -07:00
Brian Warner
48cc85e88c add file-mode checks to directory test
This new test failed before fixing _extract_file, and now it passes.
2016-06-03 16:05:51 -07:00
Brian Warner
71512809a9 extract _extract_file, add test 2016-06-03 15:38:49 -07:00
Chris Wolfe
0ad8df3750 flush the correct error 2016-06-02 15:21:31 -07:00
Chris Wolfe
bc7dea8bab add simpler exception message, assert that it is being used 2016-06-02 15:21:31 -07:00
Chris Wolfe
d4d3320277 add failing test, error 2016-06-02 15:21:31 -07:00
Brian Warner
00277c22cf allow MOTD to be displayed multiple times
(one displayed message per received welcome["motd"])

There's not much value in prohibiting the server from sending multiple
MOTD messages, and it would prevent us from using it to display a "your
client is using an old API, please upgrade" message after having already
sent a regular "please donate" MOTD message. (We could send a second
welcome message with ["error"] to kill the client, but ["motd"] is the
most convenient way to deliver a non-fatal warning).
2016-05-28 19:19:22 -07:00
Brian Warner
52e5cbd690 INCOMPATIBILITY: send "current_cli_version", not "current_version"
The reasoning is that this string is only ever likely to refer to the
version of the primary/initial client (the CLI application, written in
Python, that you get with "pip install magic-wormhole"). When there are
other implementations, with unrelated versions, they should obviously
not pay attention to a warning about the other implementation being out
of date.
2016-05-28 19:11:27 -07:00
Brian Warner
0b53094927 INCOMPATIBILITY: send pake message as dict, not raw bytes
This gives us room in the future to put other keys there, like one which
says we want to use Noise for the phase-message encryption instead of
our current HKDF scheme.
2016-05-28 18:30:36 -07:00
Brian Warner
3850c164f7 move utility functions to util.py, add new ones 2016-05-28 18:13:32 -07:00
Brian Warner
5fe7d320ab oops, pass blur_usage to apps properly, add tests 2016-05-27 18:44:41 -07:00
Brian Warner
eebc9ebd54 rewrite pruning, add full tests
Apparently it was broken: the first time the LoopingCall fired, it would
throw an exception, and never try again. Now it should be fixed.
2016-05-27 18:42:17 -07:00
Brian Warner
905ec05696 fix welcome-error signalling
The handler was calling _signal_error() incorrectly.
2016-05-26 15:37:24 -07:00
Brian Warner
19702e01ac make send pay attention to receiver errors correctly
Errors like "Refusing to overwrite existing file" would leave the sender
hanging, because it was looking for the "error" key in the wrong place.
2016-05-25 22:44:18 -07:00
Brian Warner
01ec5fb2a6 rename test_transit_twisted.py, drop the twisted 2016-05-25 19:21:54 -07:00
Brian Warner
75877effef finish renaming "confirm" to "version" 2016-05-25 19:13:56 -07:00
Brian Warner
dcdddc9d60 INCOMPATIBILITY: rename "confirm" phase to "version"
This better reflects the purpose of the message. Key confirmation is a
side-effect.

This patch only changes the "phase:" name and the key-derivation string.
A subsequent patch will modify the function and variable names to match.
2016-05-25 19:13:56 -07:00
Brian Warner
7f43561a50 INCOMPATIBILITY: change "confirm" message to include version dict
This gives the two Wormholes a way to signal capabilities to each other,
before the applications start sending their own messages.
2016-05-25 18:27:37 -07:00
Brian Warner
e1c488247f remove redundant test 2016-05-25 18:27:24 -07:00
Brian Warner
5553729a87 w.verify() now stalls until confirmation message is checked
If it succeeds, you get back the verifier string, which can be compared
against the other side. If it fails, the wormhole code didn't match.
2016-05-25 18:05:02 -07:00
Brian Warner
a251e58967 oops, fix tests to use /v1 too 2016-05-25 00:34:13 -07:00
Brian Warner
f2050c5fd4 internal renaming: hint objects vs hint structs 2016-05-25 00:23:12 -07:00
Brian Warner
7aa55e6b65 INCOMPATIBILITY: deliver hints as JSON, not strings
The file-send protocol now sends a "hints-v1" key in the "transit"
message, which contains a list of JSON data structures that describe the
connection hints (a mixture of direct, tor, and relay hints, for now).
Previously the direct/tor and relay hints were sent in different keys,
and all were sent as strings like "tcp:hostname:1234" which had to be
parsed by the recipient.

The new structures include a version string, to make it easier to add
new types in the future. Transit logs+ignores hints it cannot
understand.
2016-05-25 00:11:17 -07:00
Brian Warner
1a9e565fc3 cmd_send: turn into a self-contained class
This will make it easier to maintain state.
2016-05-24 17:44:41 -07:00
Brian Warner
96f25ec7a2 rename phase1 to offer/answer
rearrange cmd_send a bit to do less before the Wormhole is built
2016-05-24 16:41:13 -07:00
Brian Warner
0e1a4dd513 move wormhole.twisted.transit up to wormhole.transit 2016-05-24 16:22:37 -07:00
Brian Warner
783cd984f6 add 'coverage' environment for tox 2016-05-24 16:03:39 -07:00
Brian Warner
30ab940034 INCOMPATIBLE: change derivation of phase keys to include side
Previously the encryption key used for "phase messages" (anything sent
from one side to the other, protected by the shared PAKE-generated
session key) was derived just from the session key and the phase name.
The two sides would use the same key for their first message (but with
random, thus different, nonces).

This uses the sending side's string (a random 5-byte/10-character hex
string) in the derivation process too, so the two sides use different
keys. This gives us an easy way to reject reflected messages. We already
ignore messages that claim to use a "side" which matches our own (to
ignore server echoes of our own outbound messages). With this change, an
attacker (or the server) can't swap in the payload of an outbound
message, change the "side" to make it look like a peer message, and then
let us decrypt it correctly.

It also changes the derivation function to combine the phase and side
values safely. This didn't matter much when we only had one
externally-provided string, but with two, there's an opportunity for
format confusion if they were combined with a simple delimiter. Now we
hash both values before concatenating them.

This breaks interoperability with clients from before this change. They
will always get WrongPasswordErrors.
2016-05-24 13:47:15 -07:00
Brian Warner
3a062eaa26 bring scripts and tests up to date
* use wormhole instead of transcribe.py
* send() no longer waits
* get_verifier -> verify
* derive_key demands a length
2016-05-24 00:00:44 -07:00
Brian Warner
9bd5afe7df make close() always wait 2016-05-23 23:59:49 -07:00
Brian Warner
e11a6f8243 new connection management, test_wormhole passes 2016-05-23 22:53:26 -07:00
Brian Warner
7bcefa78e6 remove test_twisted, now in test_wormhole 2016-05-23 22:53:26 -07:00
Brian Warner
528092dd97 improve error signalling 2016-05-23 00:14:39 -07:00
Brian Warner
c88d6937c2 close(wait=True): wait for connection to be dropped 2016-05-22 18:45:50 -07:00
Brian Warner
c10fd98167 many tests working
* add "released" ack-response for "release" command, to sync w.close()
* move websocket URL to root
* relayurl= should now be a "ws://" URL
* many tests pass (except for test_twisted, which will be removed, and
  test_scripts)
* still moving integration tests from test_twisted to
  test_wormhole.Wormholes
2016-05-22 18:40:44 -07:00
Brian Warner
3da52b0a3e add 'mock', building out test_wormhole 2016-05-22 11:31:00 -07:00
Brian Warner
0ee56e12b0 change 'list' protocol, make room for nameplate attributes 2016-05-22 11:01:44 -07:00
Brian Warner
181ef04a91 break out more message components, use SidedMessage 2016-05-20 16:39:59 -07:00
Brian Warner
3b86571de3 fix py3 2016-05-20 12:12:07 -07:00
Brian Warner
390cd08b53 better command/response names: allocate+allocated, claim+claimed 2016-05-20 11:35:30 -07:00
Brian Warner
6c5b517ad1 hush 2016-05-20 11:10:17 -07:00
Brian Warner
ce06d379d9 remove old tests 2016-05-20 11:09:45 -07:00
Brian Warner
0a14901e94 full coverage of websocket 2016-05-20 11:08:10 -07:00
Brian Warner
f044ef0efa tests almost good 2016-05-19 23:50:22 -07:00
Brian Warner
335ed00cb7 build out tests 2016-05-19 19:55:11 -07:00
Brian Warner
2ea5d96290 Channels don't need "welcome" anymore 2016-05-17 17:35:44 -07:00
Brian Warner
5dd91c7311 test too-many-allocate, allocate+claim 2016-05-13 00:46:12 -07:00
Brian Warner
c4c0cf71eb add test 2016-05-13 00:43:59 -07:00
Brian Warner
1198977e06 SCHEMA CHANGE: channelids are now strs, not ints
This will enable the use of large randomly-generated hex or base32
channelids, for post-startup or resumed-connection channels.
2016-05-13 00:37:53 -07:00
Brian Warner
c14e982ae7 rendezvous: allow multiple channels per connection 2016-05-12 18:01:56 -07:00
Brian Warner
85dc0fd41b change server API: "release" instead of "deallocate" 2016-05-12 17:46:15 -07:00
Brian Warner
a34fb2a98b remove plain-HTTP (non-WebSocket) rendezvous server 2016-05-12 16:56:19 -07:00
Brian Warner
104ef44d53 provide wormhole() as a function, rather than a class constructor
You must always provide a reactor= argument. In the future, omitting the
reactor= argument is how you ask for a blocking Wormhole.
2016-05-12 16:45:54 -07:00
Brian Warner
d0ef53fc4d remove phase= from the Wormhole API
Phase are now implicit and numbered.
2016-05-12 16:45:54 -07:00
Brian Warner
501af4b4ec rename send_data/get_data to just send/get 2016-05-12 16:45:54 -07:00
Brian Warner
49785008bb remove blocking implementation: it will return
It will return as a crochet-based wrapper around the Twisted
implementation.
2016-05-12 16:45:54 -07:00
Brian Warner
5530c33185 rdv_ws: send acks for each message
but only if the client is modern enough to include "id" in the message,
which lets us avoid sending acks to an 0.7.5 client (which would cause
them to abort, they don't like unrecognized server messages).

The acks let the client learn the server_rx time of messages that
terminate on the server, like "allocate" and "claim".
2016-05-06 18:51:28 -07:00
Brian Warner
644c7c6840 DB schema change: add/store/return msgid
This enables dump-timing to correlate sender logs with receiver logs.
2016-05-06 18:29:58 -07:00
Brian Warner
8a80242532 rdv_ws: deliver "server_tx", not "sent"
Update twisted/transcribe.py to accept it, update tests.
2016-05-06 18:29:58 -07:00
Brian Warner
d2dfc325d8 deliver stored server_rx to clients
This enables the "dump-timing" tools to display more information.
2016-05-06 18:29:58 -07:00
Brian Warner
fe2dfc1a35 DB schema change: rename 'when' to 'server_rx' 2016-05-06 18:29:58 -07:00
Brian Warner
b70c2f8868 Make get_verifier() wait for _confirm to arrive
This improves the error behavior when --verify is used but there's a
WrongPasswordError: the mismatch is detected before the verifiers are
displayed or confirmation is requested.

It requires that the far end sends a "_confirm" message, which was
introduced in release 0.6.0. Use with older versions (if it doesn't
break for other reasons) will cause a hang.

This patch also deletes test_twisted.Basic.test_verifier_mismatch, since
both sides now detect this on their own. It changes
test_wrong_password() too, since we might now notice the error during
send_data (previously we'd only see it in get_data).
2016-04-25 19:01:55 -07:00
Brian Warner
2f6caa04d3 test_scripts.Cleanup: new test
This asserts that we deallocate the rendezvous channel, even when we
terminate early due to errors.
2016-04-25 17:53:00 -07:00
Brian Warner
34116c7b1f CLI: document and return correct errors
Also clean up test_scripts.PregeneratedCode:

* fetch results from both sides at the same time
* only check rc when using a subprocess, since the direct call doesn't
  use rc=0 anymore
* no need to cancel the other side's Deferred when one errors
* provide more information if stderr was non-empty
2016-04-25 17:52:59 -07:00
Brian Warner
4eaf88d7d2 cmd: remove _twisted suffix from function names
No longer necessary now that all commands are twisted-based.
2016-04-24 22:49:18 -07:00
Brian Warner
86edf96412 switch to tqdm for nicer CLI progress bars 2016-04-24 12:04:05 -07:00
Brian Warner
b2c3f49038 unsplit txwormhole step 2: edit files 2016-04-20 19:18:41 -07:00
Brian Warner
7c45f3d472 unsplit wormhole_server step 2: edit files 2016-04-20 19:15:33 -07:00
Brian Warner
85b6264826 unsplit cli step 2: edit files 2016-04-20 19:15:33 -07:00
Brian Warner
7a5a99a1a8 unsplit tests step 3: modify files 2016-04-20 18:54:10 -07:00
Brian Warner
fc33ea41ff unsplit tests step 2: delete test_load 2016-04-20 18:51:03 -07:00
Brian Warner
c890a850aa unsplit tests step 1: move files 2016-04-20 18:44:33 -07:00
Brian Warner
95706c752c move tests step 1: rename files 2016-04-18 16:45:03 -07:00
Brian Warner
36e63e2347 split wormhole_cli step 2: fix imports, setup.py 2016-04-18 16:43:18 -07:00
Brian Warner
1688da74d9 split txwormhole step 2: fix imports 2016-04-18 16:41:56 -07:00
Brian Warner
87266f1292 split wormhole_server step 3: fix tests 2016-04-18 16:18:51 -07:00
Brian Warner
1a455c05f5 remove wormhole.twisted.util port-allocation functions 2016-04-18 16:16:27 -07:00
Brian Warner
db137c26e5 remove blocking.transit 2016-04-18 15:52:29 -07:00
Brian Warner
4e937c2100 rename cmd_send_twisted to cmd_send, same for cmd_receive 2016-04-18 15:52:29 -07:00
Brian Warner
a1033b06a3 merge send_common into cmd_send_twisted 2016-04-18 15:52:29 -07:00
Brian Warner
8c67a98259 merge test_transit.py into test_server.py 2016-04-18 15:52:29 -07:00
Brian Warner
68b22bec97 remove --twisted, stop using blocking-flavor scripts 2016-04-18 15:52:18 -07:00
Brian Warner
049fac01db tolerate trailing slash on "wormhole send dirname/"
Previously, the trailing slash would cause the receiving side to get an
empty-named directory.
2016-03-24 08:46:29 -07:00
Brian Warner
626732b730 relay_server: include 'sent' timestamp with each message
This will be used by client-side timeline instrumentation, to guide
performance improvements.
2016-03-03 17:52:59 -08:00
Brian Warner
418fe9419e twisted: use persistent connections, slight speedup
This uses a single TCP connection to the relay server for all
requests (although it probably uses a second one for the downstream
EventSource feed). This should squeeze out some of the round-trip times.
2016-03-03 13:39:09 -08:00
Brian Warner
df2384bea2 twisted.transit: move FileConsumer into RecordPipe
This adds an expected= argument to Connection.connectConsumer(), which
then returns a Deferred that fires when enough bytes have been written
to the consumer. It also adds Connection.writeToFile(), a helper method
that writes bytes to a filehandle.
2016-03-02 00:48:43 -08:00
Brian Warner
7234e25897 twisted.transit: handle multiple records in one chunk
I made the classic dataReceived() mistake, and exited the function after
delivering the first record. Keep at it until there are no complete
records left.
2016-03-02 00:48:43 -08:00
Brian Warner
8d82726c51 add DebugTiming object, --dump-timing= option
This writes timeline data to a .json file, which can be examined later
to find likely candidates for optimization.
2016-03-01 18:23:06 -08:00
Brian Warner
84def8a54b add some inlineCallbacks for simplicity
This control flow was getting too hairy.
2016-03-01 18:22:03 -08:00