open.source/magic-wormhole
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
495 Commits 1 Branch 0 Tags 2.9 MiB
644c7c6840
Commit Graph

396 Commits

Author SHA1 Message Date
Brian Warner
558f01818f server: log websocket URL 2016-04-20 11:14:52 -07:00
Brian Warner
88b78d1c46 fix comment: bodies are hex, not base64 2016-04-20 11:13:40 -07:00
Brian Warner
92c4ad19bc rendezvous_websocket: fix Error 2016-04-20 11:12:50 -07:00
Brian Warner
aa8f714a48 websocket: change 'all-channelids' to 'channelids' 
Yeah, names are easier to work with when they're also valid identifiers.
 2016-04-18 23:49:11 -07:00
Brian Warner
aec0615d66 Add WebSocket-based rendezvous protocol frontend 
The websocket lives on a Resource of the main rendezvous web site, and
the websocket URL is derived from the main "relay_url", so there's no
extra port to allocate, and no extra service to shut down.
 2016-04-18 18:03:26 -07:00
Brian Warner
efc23bd078 Rendezvous: change internal API 
Deliver not-yet-JSONed objects to listeners (both in broadcast_message
and as the "catch-up" responses to add_listener). Also make the (web)
frontend responsible for adding "sent" timestamps. This all makes
rendezvous.py less web-centric.
 2016-04-18 17:40:12 -07:00
Brian Warner
c3bd9e936e split rendezvous server into web, nonweb files 
Also rename files/classes from "relay" to "rendezvous".
 2016-04-18 17:40:08 -07:00
Brian Warner
3b215c106a use "hkdf" from PyPI instead of wormhole.hkdf 2016-04-18 16:49:29 -07:00
Brian Warner
95706c752c move tests step 1: rename files 2016-04-18 16:45:03 -07:00
Brian Warner
17613ce8c1 move hkdf out of util/, remove now-empty directory 2016-04-18 16:45:03 -07:00
Brian Warner
36e63e2347 split wormhole_cli step 2: fix imports, setup.py 2016-04-18 16:43:18 -07:00
Brian Warner
2e7bb1a8f1 split wormhole_cli step 1: move files 2016-04-18 16:43:18 -07:00
Brian Warner
1688da74d9 split txwormhole step 2: fix imports 2016-04-18 16:41:56 -07:00
Brian Warner
323175ddfe split txwormhole step 1: move files 2016-04-18 16:41:52 -07:00
Brian Warner
0b162af09e move ipaddrs.py from wormhole.util to wormhole.twisted 2016-04-18 16:24:13 -07:00
Brian Warner
87266f1292 split wormhole_server step 3: fix tests 2016-04-18 16:18:51 -07:00
Brian Warner
190ce2c027 split wormhole_server step 2: new files, fix imports 
remove server commands from old src/wormhole/scripts/runner.py
 2016-04-18 16:18:47 -07:00
Brian Warner
bde4696a8b split wormhole_server step 1: move files 2016-04-18 16:18:45 -07:00
Brian Warner
1a455c05f5 remove wormhole.twisted.util port-allocation functions 2016-04-18 16:16:27 -07:00
Brian Warner
e20026488a remove unused observer.py 2016-04-18 16:16:27 -07:00
Brian Warner
a8446d2bc5 merge transit_common into twisted.transit 2016-04-18 16:16:27 -07:00
Brian Warner
db137c26e5 remove blocking.transit 2016-04-18 15:52:29 -07:00
Brian Warner
4e937c2100 rename cmd_send_twisted to cmd_send, same for cmd_receive 2016-04-18 15:52:29 -07:00
Brian Warner
a1033b06a3 merge send_common into cmd_send_twisted 2016-04-18 15:52:29 -07:00
Brian Warner
9b53bb96c6 merge cmd_receive_blocking into cmd_receive_twisted 2016-04-18 15:52:29 -07:00
Brian Warner
8c67a98259 merge test_transit.py into test_server.py 2016-04-18 15:52:29 -07:00
Brian Warner
68b22bec97 remove --twisted, stop using blocking-flavor scripts 2016-04-18 15:52:18 -07:00
Brian Warner
94a3be91b2 add tab-completion to twisted-style input_code() 2016-04-15 17:27:32 -07:00
Brian Warner
589226f076 tor: add comments, let it pick its own control port 2016-03-28 02:26:11 -07:00
Brian Warner
618706681a record time spent importing code 2016-03-28 02:25:40 -07:00
Brian Warner
9630ab9aae find-tor: record more detailed timings 2016-03-28 02:25:32 -07:00
Brian Warner
12c4c51fd8 record tor-launch time in DebugTiming 2016-03-28 02:25:01 -07:00
Brian Warner
01ed9902de add --tor support 2016-03-28 02:25:01 -07:00
Brian Warner
ed6e5ff169 get a TorManager working 2016-03-28 02:25:01 -07:00
Brian Warner
049fac01db tolerate trailing slash on "wormhole send dirname/" 
Previously, the trailing slash would cause the receiving side to get an
empty-named directory.
 2016-03-24 08:46:29 -07:00
Brian Warner
c5415495c0 dump-timing: store server-sent time too 
Adjust dump-timing to ignore the extra data, for now. Also do some
general instrumentation cleanup.
 2016-03-03 18:03:27 -08:00
Brian Warner
aaf4e70a33 remove obsolete TODO 2016-03-03 17:56:15 -08:00
Brian Warner
626732b730 relay_server: include 'sent' timestamp with each message 
This will be used by client-side timeline instrumentation, to guide
performance improvements.
 2016-03-03 17:52:59 -08:00
Brian Warner
e342236b7d relay_server: internal rearrangement 
Construct the response in the render_* method, rather than
add_messages(), to prepare for adding server timestamps. Use
json_response() everywhere.
 2016-03-03 16:57:44 -08:00
Brian Warner
9d7b9dd8d2 add --no-listen, for debugging 2016-03-03 14:52:14 -08:00
Brian Warner
418fe9419e twisted: use persistent connections, slight speedup 
This uses a single TCP connection to the relay server for all
requests (although it probably uses a second one for the downstream
EventSource feed). This should squeeze out some of the round-trip times.
 2016-03-03 13:39:09 -08:00
Brian Warner
4f5e037d39 cmd-twisted: report exceptions better 
The previous scheme would swallow the real traceback, making debugging
difficult.
 2016-03-03 12:33:26 -08:00
Brian Warner
08a5e6043a twisted.transit: tolerate the lack of a listener 
In Tor mode, there is no local listener. Eventually we'll add an Onion
Service listener, but it's not guaranteed that we can run one.
 2016-03-03 12:32:26 -08:00
Brian Warner
8d92b40139 twisted.transcribe: use same Agent for ReconnectingEventSource 
This will be especially important when we add Tor support.
 2016-03-03 12:29:57 -08:00
Brian Warner
be2c66a5f5 transit: ignore errors on sockets we were about to close anyways 2016-03-02 14:28:04 -08:00
Brian Warner
26f512fba4 add --twisted to use the twisted CLI flavor 2016-03-02 01:07:37 -08:00
Brian Warner
6d3d0c1cb3 unify synchronous calling of twisted CLI commands 2016-03-02 00:51:21 -08:00
Brian Warner
4d405c8cef cmd_send_twisted: fix display message ordering 
By creating the ProgressingFileSender too early, it printed a progress
message before the "Sending" line.
 2016-03-02 00:50:22 -08:00
Brian Warner
df2384bea2 twisted.transit: move FileConsumer into RecordPipe 
This adds an expected= argument to Connection.connectConsumer(), which
then returns a Deferred that fires when enough bytes have been written
to the consumer. It also adds Connection.writeToFile(), a helper method
that writes bytes to a filehandle.
 2016-03-02 00:48:43 -08:00
Brian Warner
7234e25897 twisted.transit: handle multiple records in one chunk 
I made the classic dataReceived() mistake, and exited the function after
delivering the first record. Keep at it until there are no complete
records left.
 2016-03-02 00:48:43 -08:00
Brian Warner
8d82726c51 add DebugTiming object, --dump-timing= option 
This writes timeline data to a .json file, which can be examined later
to find likely candidates for optimization.
 2016-03-01 18:23:06 -08:00
Brian Warner
84def8a54b add some inlineCallbacks for simplicity 
This control flow was getting too hairy.
 2016-03-01 18:22:03 -08:00
Brian Warner
fd143caded tests.ServerBase: print message when threadpool is not idle 
The previous commits improve test failures by dropping relay connections
at shutdown, and flunking a test quickly when one client fails but the
other one hangs.

If that doesn't work (say, some client has a time.sleep(), or other
stall that isn't affected by the relay shutdown), we'll be left with an
active thread holding that hanging client.

This patch adds a check to wormhole.test.common.ServerBase.tearDown that
looks for active threads, waits a second (after stopService), then
checks the threadpool again. If the threadpool is empty, everything is
fine. If not, it prints a message (to stdout) to inform the impatient
user why the test is probably hanging.
 2016-03-01 17:07:48 -08:00
Brian Warner
b1dae14e6d test_scripts: handle hangs in one client 
When test_scripts ran two clients at the same time, an error in one
could leave the other hanging (in a thread). One Deferred would errback,
the other would hang. Tests wait on one Deferred at a time, so if we're
unlucky and were waiting on the hanging Deferred (instead of the
erroring one), we'll wait forever, or at least until the default test
timeout of 180 seconds.

This adds an errback to notice when either client has errored, and
cancels the other Deferred, so it doesn't matter which one we wait upon
first.
 2016-03-01 17:07:43 -08:00
Brian Warner
3fc3a563bf relay_server: disconnect clients upon shutdown 2016-03-01 17:07:37 -08:00
Brian Warner
0771aae7c7 server: Relay wasn't pruning channels 
I forgot to hook it up to the service parent, so the timer was never
started.
 2016-02-29 08:59:53 -08:00
Brian Warner
c225d57f8c cmd_receive_twisted.py: twisted-based implementation 2016-02-29 07:31:34 -08:00
Brian Warner
01064325a2 cmd_receive: refactor (slight message changes) 
This prepares the way for a twisted-based implementation.
 2016-02-29 07:30:40 -08:00
Brian Warner
6654efb429 move describe() from Transit to RecordPipe 2016-02-28 01:42:46 -08:00
Brian Warner
1903c58248 ipaddrs: return 127.0.0.1 if everything else fails 
This helps the windows tests where ipaddrs currently fails entirely.
 2016-02-27 17:37:12 -08:00
Brian Warner
ca06e95bbd test_blocking: avoid using real hints 2016-02-27 17:29:04 -08:00
Brian Warner
8067acbf82 cmd_receive: close the file before renaming 
I'm always forgetting this one. It's more obvious on windows.
 2016-02-27 17:20:43 -08:00
Brian Warner
ed1eb06ae8 blocking/transit: use descriptive error message 2016-02-27 14:48:12 -08:00
Brian Warner
1ff0792b32 test_scripts: prep for windows compatibility 
newlines, os.mkfifo errors, JSON type differences
 2016-02-27 14:46:38 -08:00
Brian Warner
14dcfeed73 tolerate lack of readline at runtime 
'readline' is part of the python stdlib, so declaring a dependency on it
doesn't help. It doesn't exist on windows, and the pypi 'readline'
module doesn't work on windows. So instead, just attempt to import
readline, and if that fails, fall back to a non-completion flavor.
 2016-02-27 14:16:58 -08:00
Brian Warner
b7319f90c8 windows "route.exe" is not absolute, and that's ok 2016-02-27 14:15:36 -08:00
Brian Warner
dbba482c62 cmd_receive_blocking.py: refactor 2016-02-17 21:35:53 -08:00
Brian Warner
e6fba34570 start on twisted receive: move old files into new homes 2016-02-17 19:25:19 -08:00
Brian Warner
7f056ca075 send-twisted: set transit key before sending phase1 
This ensures that we'll be ready for them. Previously there was a race
between us revealing the direct hints to the peer, and us setting the
transit key (thus allowing us to check inbound handshake requests). The
Transit instance didn't handle the race, causing errors to be thrown
when the other side connected quickly.
 2016-02-17 19:02:36 -08:00
Brian Warner
aa27bfd32c add twisted form of sender 
Currently this is only invokable from tests.
 2016-02-17 19:02:36 -08:00
Brian Warner
7ceffd783a add more assertions around transit_key 2016-02-17 19:02:35 -08:00
Brian Warner
3ffceff9d5 send-blocking: set transit key before sending phase1 
This ensures that we'll be ready for them. Previously there was a race
between us revealing the direct hints to the peer, and us setting the
transit key (thus allowing us to check inbound handshake requests). The
Transit instance handles this race (with an interlock on the transit
key), but it's still nicer to do it cleanly.

This exposed a new race in Transit, where the inbound connection would
complete before transit.connect() had been called. The previous commit
adds an interlock to wait for that too. Until this change, the transit
key lock was covering that one up.
 2016-02-17 19:02:35 -08:00
Brian Warner
142f3fc154 send-blocking: don't create Transit unless we need it 2016-02-17 19:02:35 -08:00
Brian Warner
dba42aff01 blocking transit: tolerate inbound connections before connect() starts 2016-02-17 19:02:35 -08:00
Brian Warner
369854b4cc test build_phase1_data 2016-02-17 18:07:35 -08:00
Brian Warner
79decea9ea rearrange scripts: make room for twisted 2016-02-17 16:26:03 -08:00
Brian Warner
d36d1cb063 runner.py: new approach to dispatching 2016-02-17 16:03:30 -08:00
Brian Warner
3b447df0f4 move argparser out to a separate module 2016-02-17 15:55:11 -08:00
Brian Warner
5d572137f4 improve test error message 2016-02-17 14:00:03 -08:00
Brian Warner
fd7d1b97d4 test scripts in a thread, not a subprocess, mostly 2016-02-17 14:00:01 -08:00
Brian Warner
903129f4a2 add --hide-progress, mostly for tests 2016-02-17 13:58:41 -08:00
Brian Warner
c5b2800a3e runner: strictly use cwd/stdout/stderr from 'args' 
This will make it easier to test the scripts in a controlled fashion.
 2016-02-17 13:53:20 -08:00
Brian Warner
e2f3bebe38 allow --relay-helper="" to disable relay 
test_scripts now uses this to avoid accidentally using a relay
 2016-02-17 13:53:18 -08:00
Brian Warner
d14d35f3cd cmd_send_blocking: close the transit when we're done 2016-02-17 13:52:26 -08:00
Brian Warner
5e928ac9f0 rewrite ProgressPrinter as a class, add tests 2016-02-17 12:46:12 -08:00
Brian Warner
00833a4bde test_scripts: factor out common test code 2016-02-17 10:22:31 -08:00
Brian Warner
99ff75259a catch 'send' of non-file/non-directory 2016-02-17 09:36:31 -08:00
Brian Warner
8079340bff more rearranging 2016-02-15 21:59:18 -08:00
Brian Warner
512c7c0a59 cmd_send: rearrange in preparation for twisted version 2016-02-15 21:40:57 -08:00
Brian Warner
a235b507c8 twisted.transit: implement producer/consumer flow control 2016-02-15 21:23:20 -08:00
Brian Warner
763d72f582 twisted.transit: implement Deferred-based receive_record() 2016-02-15 11:40:21 -08:00
Brian Warner
fb1461fa8c add twisted.transit, with tests 2016-02-14 17:57:09 -08:00
Brian Warner
7212e9e9f4 transit: split common code out to a new file 
This will be shared between blocking.transit and the soon-to-land
twisted.transit .
 2016-02-14 17:51:46 -08:00
Brian Warner
36cc0e2bca tests: don't use network 
Some tests failed to override --transit-helper, which meant they
intermittently talked to the real transit server (briefly, before
deciding the local+direct connection was better).
 2016-02-12 17:11:06 -08:00
Brian Warner
7ded461342 show-usage: give more room for "rendezvous-lonely" 2016-01-14 14:30:08 -08:00
Brian Warner
bd912a73ee Transit: blur the file sizes too 2016-01-12 16:32:38 -08:00
Brian Warner
35c9e29eb3 transit handshake: wait for newline, not just expected string 2016-01-12 15:04:52 -08:00
Brian Warner
9ed39be346 don't log HTTP requests when blur-usage is on 2015-12-04 17:35:56 -08:00
Brian Warner
1ab66d2fd0 privacy: only store coarse timestamps in the usage table 2015-12-03 21:15:19 -08:00
Brian Warner
ee86059b0a transit_server: clean up logging slightly 2015-12-03 20:44:34 -08:00