open.source/magic-wormhole
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
832 Commits 1 Branch 0 Tags 2.9 MiB
2b049ac7c6
Commit Graph

627 Commits

Author SHA1 Message Date
Brian Warner
fb2ffe1963 add --signal-error to restart too 2016-05-24 12:59:02 -07:00
Brian Warner
a2b88dbf61 server: add --signal-error CLI argument 2016-05-24 12:46:42 -07:00
Brian Warner
1ef6218b5b remove old twisted/transcribe.py, now just wormhole.py 2016-05-24 00:01:22 -07:00
Brian Warner
3a062eaa26 bring scripts and tests up to date 
* use wormhole instead of transcribe.py
* send() no longer waits
* get_verifier -> verify
* derive_key demands a length
 2016-05-24 00:00:44 -07:00
Brian Warner
e2aa43d0a9 transit: expose desired key length 2016-05-24 00:00:21 -07:00
Brian Warner
2c64805ea1 fix input_code 2016-05-24 00:00:04 -07:00
Brian Warner
9bd5afe7df make close() always wait 2016-05-23 23:59:49 -07:00
Brian Warner
e11a6f8243 new connection management, test_wormhole passes 2016-05-23 22:53:26 -07:00
Brian Warner
7bcefa78e6 remove test_twisted, now in test_wormhole 2016-05-23 22:53:26 -07:00
Brian Warner
528092dd97 improve error signalling 2016-05-23 00:14:39 -07:00
Brian Warner
c88d6937c2 close(wait=True): wait for connection to be dropped 2016-05-22 18:45:50 -07:00
Brian Warner
c10fd98167 many tests working 
* add "released" ack-response for "release" command, to sync w.close()
* move websocket URL to root
* relayurl= should now be a "ws://" URL
* many tests pass (except for test_twisted, which will be removed, and
  test_scripts)
* still moving integration tests from test_twisted to
  test_wormhole.Wormholes
 2016-05-22 18:40:44 -07:00
Brian Warner
3da52b0a3e add 'mock', building out test_wormhole 2016-05-22 11:31:00 -07:00
Brian Warner
0ee56e12b0 change 'list' protocol, make room for nameplate attributes 2016-05-22 11:01:44 -07:00
Brian Warner
53bbcc33f6 new file, state-machine based 2016-05-20 18:49:20 -07:00
Brian Warner
181ef04a91 break out more message components, use SidedMessage 2016-05-20 16:39:59 -07:00
Brian Warner
05aa5ca76e WIP Wormhole 2016-05-20 13:51:05 -07:00
Brian Warner
3b86571de3 fix py3 2016-05-20 12:12:07 -07:00
Brian Warner
390cd08b53 better command/response names: allocate+allocated, claim+claimed 2016-05-20 11:35:30 -07:00
Brian Warner
6c5b517ad1 hush 2016-05-20 11:10:17 -07:00
Brian Warner
ce06d379d9 remove old tests 2016-05-20 11:09:45 -07:00
Brian Warner
0a14901e94 full coverage of websocket 2016-05-20 11:08:10 -07:00
Brian Warner
399efb374c don't close websocket when mailbox is deleted 
This made sense for ServerSentEvent channels (which has no purpose once
the channel was gone), but not so much for websockets. And it prevented
testing duplicate-close.
 2016-05-20 11:07:21 -07:00
Brian Warner
f044ef0efa tests almost good 2016-05-19 23:50:22 -07:00
Brian Warner
335ed00cb7 build out tests 2016-05-19 19:55:11 -07:00
Brian Warner
e39a8291e3 checkpointing: server roughed out 2016-05-19 18:09:17 -07:00
Brian Warner
0e72422ffa WIP 2016-05-19 14:18:49 -07:00
Brian Warner
5994eb11d4 WIP new proto 2016-05-18 00:16:46 -07:00
Brian Warner
2ea5d96290 Channels don't need "welcome" anymore 2016-05-17 17:35:44 -07:00
Brian Warner
a74b1b1e3a WIP: new server protocol 2016-05-16 22:04:25 -07:00
Brian Warner
5dd91c7311 test too-many-allocate, allocate+claim 2016-05-13 00:46:12 -07:00
Brian Warner
c4c0cf71eb add test 2016-05-13 00:43:59 -07:00
Brian Warner
1198977e06 SCHEMA CHANGE: channelids are now strs, not ints 
This will enable the use of large randomly-generated hex or base32
channelids, for post-startup or resumed-connection channels.
 2016-05-13 00:37:53 -07:00
Brian Warner
c14e982ae7 rendezvous: allow multiple channels per connection 2016-05-12 18:01:56 -07:00
Brian Warner
31491bb939 update docs 2016-05-12 17:48:26 -07:00
Brian Warner
85dc0fd41b change server API: "release" instead of "deallocate" 2016-05-12 17:46:15 -07:00
Brian Warner
2c2cf29564 update comment: sent -> server_tx 2016-05-12 17:12:04 -07:00
Brian Warner
bdc9066c23 rendezvous: change add_listener signature 
Pass in a handle and a pair of functions, rather than an object with two
well-known methods. This should make it easier to subscribe to multiple
channels in the future.
 2016-05-12 17:03:57 -07:00
Brian Warner
a34fb2a98b remove plain-HTTP (non-WebSocket) rendezvous server 2016-05-12 16:56:19 -07:00
Brian Warner
104ef44d53 provide wormhole() as a function, rather than a class constructor 
You must always provide a reactor= argument. In the future, omitting the
reactor= argument is how you ask for a blocking Wormhole.
 2016-05-12 16:45:54 -07:00
Brian Warner
d87aba40e4 rename _confirm message to just "confirm" 2016-05-12 16:45:54 -07:00
Brian Warner
d0ef53fc4d remove phase= from the Wormhole API 
Phase are now implicit and numbered.
 2016-05-12 16:45:54 -07:00
Brian Warner
501af4b4ec rename send_data/get_data to just send/get 2016-05-12 16:45:54 -07:00
Brian Warner
49785008bb remove blocking implementation: it will return 
It will return as a crochet-based wrapper around the Twisted
implementation.
 2016-05-12 16:45:54 -07:00
Brian Warner
5501a6bf1c simplify timing, add msgid 2016-05-06 18:53:28 -07:00
Brian Warner
5530c33185 rdv_ws: send acks for each message 
but only if the client is modern enough to include "id" in the message,
which lets us avoid sending acks to an 0.7.5 client (which would cause
them to abort, they don't like unrecognized server messages).

The acks let the client learn the server_rx time of messages that
terminate on the server, like "allocate" and "claim".
 2016-05-06 18:51:28 -07:00
Brian Warner
644c7c6840 DB schema change: add/store/return msgid 
This enables dump-timing to correlate sender logs with receiver logs.
 2016-05-06 18:29:58 -07:00
Brian Warner
8a80242532 rdv_ws: deliver "server_tx", not "sent" 
Update twisted/transcribe.py to accept it, update tests.
 2016-05-06 18:29:58 -07:00
Brian Warner
d2dfc325d8 deliver stored server_rx to clients 
This enables the "dump-timing" tools to display more information.
 2016-05-06 18:29:58 -07:00
Brian Warner
959ab4baca rdv_ws: get server_rx at a better place 2016-05-06 18:29:58 -07:00
Brian Warner
d9ba55621b Channel.add_message(server_rx=): add new arg 2016-05-06 18:29:58 -07:00
Brian Warner
fe2dfc1a35 DB schema change: rename 'when' to 'server_rx' 2016-05-06 18:29:58 -07:00
Brian Warner
cd9f6e4377 ignore unknown rendezvous messages 
I should have added this earlier, so 0.7.5 would be tolerant of new
server messages.
 2016-05-06 18:29:19 -07:00
Brian Warner
a671982ab0 rdv: add comment 2016-05-05 19:13:05 -07:00
Brian Warner
58e95cff0d rdv_ws: avoid magic method-name concatenation 2016-05-05 19:11:21 -07:00
Brian Warner
a086effd52 cmd_send: time import of tor_manager too 2016-05-05 19:10:01 -07:00
Brian Warner
cfbd418898 rendezvous: minor refactoring 2016-05-05 18:21:06 -07:00
Brian Warner
8e664fdfcd timing.py: record finish(when) correctly 2016-04-30 15:48:43 -07:00
Brian Warner
24e52c0320 rewrite timing instrumentation: use context managers 2016-04-29 14:27:29 -07:00
Brian Warner
b70c2f8868 Make get_verifier() wait for _confirm to arrive 
This improves the error behavior when --verify is used but there's a
WrongPasswordError: the mismatch is detected before the verifiers are
displayed or confirmation is requested.

It requires that the far end sends a "_confirm" message, which was
introduced in release 0.6.0. Use with older versions (if it doesn't
break for other reasons) will cause a hang.

This patch also deletes test_twisted.Basic.test_verifier_mismatch, since
both sides now detect this on their own. It changes
test_wrong_password() too, since we might now notice the error during
send_data (previously we'd only see it in get_data).
 2016-04-25 19:01:55 -07:00
Brian Warner
c017de5e4b cosmetic cleanups to error messages 2016-04-25 18:24:39 -07:00
Brian Warner
2a6767ee6f cmd_receive: internal error-handling cleanups 2016-04-25 18:24:18 -07:00
Brian Warner
a181a3fb35 WrongPasswordError: display docstring in str() 2016-04-25 18:16:08 -07:00
Brian Warner
2f6caa04d3 test_scripts.Cleanup: new test 
This asserts that we deallocate the rendezvous channel, even when we
terminate early due to errors.
 2016-04-25 17:53:00 -07:00
Brian Warner
7e8bfe314d Call w.close() exactly once, in both success and error cases. 
One downside is that we keep the wormhole channel allocated longer (we
have to finish the file transfer before we can deallocate it, which
could take a while for large files). Maybe we can fix this in the
future.
 2016-04-25 17:53:00 -07:00
Brian Warner
34116c7b1f CLI: document and return correct errors 
Also clean up test_scripts.PregeneratedCode:

* fetch results from both sides at the same time
* only check rc when using a subprocess, since the direct call doesn't
  use rc=0 anymore
* no need to cancel the other side's Deferred when one errors
* provide more information if stderr was non-empty
 2016-04-25 17:52:59 -07:00
Brian Warner
e4a19748db CLI: don't catch+wrap WrongPasswordError 
Also remove .explain, now that we no longer wrap it in TransferError
 2016-04-25 17:52:59 -07:00
Brian Warner
b96b052c96 start on cleanups: error catching in CLI dispatch 2016-04-25 17:52:59 -07:00
Brian Warner
a4c1ba9e4e transcribe: stop automatically doing close() on error 
And provide a close() that can live at the end of a Deferred chain, so
callers can do d.addBoth(w.close).

I like auto-close-on-error in general, but I'm removing it so I can
clean up the error-handling pathways. It will probably come back later.
The constraint is that it must be possible to wait on the return
Deferred that close() gives you (to synchronize tests, or keep the CLI
program running long enough to deallocate the channel) even if something
else (and error handler) called close() earlier. This will require
either a OneShotObserverList, or keeping a "deallocated" Deferred around
in case more callers want to wait on it later.
 2016-04-25 17:50:52 -07:00
Brian Warner
8d0bcf9f82 transcribe._sleep: make it usable from deallocate() 
If we're closing because of an error, we need to sleep through the old
error, to be able to wait for the "deallocated" message. This might want
to be different: maybe clear the error first, or store the errors in a
list and sleep until a second error happens.
 2016-04-25 17:34:57 -07:00
Brian Warner
4eaf88d7d2 cmd: remove _twisted suffix from function names 
No longer necessary now that all commands are twisted-based.
 2016-04-24 22:49:18 -07:00
Brian Warner
b0da2a9ac7 cmd_send: fold helper routines back into main function 
These were split out to make the blocking- and twisted- based
implementations share some code, but now that we're down to just
Twisted, it's clearer to merge them back in.
 2016-04-24 22:46:21 -07:00
Brian Warner
35639dafed cmd_send: rearrange functions, no code changes 2016-04-24 22:40:25 -07:00
Brian Warner
754cabbdd8 Fail better when input-code is interrupted 
Hitting Control-C (which sends SIGINT) while we're waiting in the
readline-based input_code() function didn't shut down the process
properly: the reactor would wait for the readline thread to exit, which
wouldn't happen until it finished getting a code, which requires the
user to hit Return. I haven't found a good way to force the thread to
exit, or to synthetically inject a newline into stdin. So my compromise
is to tell the user that they need to hit Return to finish interrupting
the command.

See the _warn_readline() function for a list of other potential
approaches.
 2016-04-24 22:20:27 -07:00
Brian Warner
e8d3689a3a runner: display TransferErrors better 
These are known failures (like file-not-found, transfer rejected, etc).
Display the error message, but not a traceback.
 2016-04-24 12:10:51 -07:00
Brian Warner
86edf96412 switch to tqdm for nicer CLI progress bars 2016-04-24 12:04:05 -07:00
Brian Warner
16c6c0977e progress: make it easier to change display width 2016-04-23 22:43:42 -07:00
Brian Warner
1e7c714453 CLI: don't hide errors so much 2016-04-20 22:36:43 -07:00
Brian Warner
1b16127cad fix --tor 
* twisted/transcribe: call correct tor_manager method
* tor_manager: remove now-unused web agent endpoint factory
 2016-04-20 22:34:58 -07:00
Brian Warner
b2c3f49038 unsplit txwormhole step 2: edit files 2016-04-20 19:18:41 -07:00
Brian Warner
0ae8463331 unsplit txwormhole step 1: move files 2016-04-20 19:15:33 -07:00
Brian Warner
7c45f3d472 unsplit wormhole_server step 2: edit files 2016-04-20 19:15:33 -07:00
Brian Warner
ec13404fca unsplit wormhole_server step 1: move files 2016-04-20 19:15:33 -07:00
Brian Warner
85b6264826 unsplit cli step 2: edit files 2016-04-20 19:15:33 -07:00
Brian Warner
a3c836bcd3 unsplit cli step 1: move files 2016-04-20 18:54:36 -07:00
Brian Warner
7a5a99a1a8 unsplit tests step 3: modify files 2016-04-20 18:54:10 -07:00
Brian Warner
fc33ea41ff unsplit tests step 2: delete test_load 2016-04-20 18:51:03 -07:00
Brian Warner
c890a850aa unsplit tests step 1: move files 2016-04-20 18:44:33 -07:00
Brian Warner
3b215c106a use "hkdf" from PyPI instead of wormhole.hkdf 2016-04-18 16:49:29 -07:00
Brian Warner
95706c752c move tests step 1: rename files 2016-04-18 16:45:03 -07:00
Brian Warner
17613ce8c1 move hkdf out of util/, remove now-empty directory 2016-04-18 16:45:03 -07:00
Brian Warner
36e63e2347 split wormhole_cli step 2: fix imports, setup.py 2016-04-18 16:43:18 -07:00
Brian Warner
2e7bb1a8f1 split wormhole_cli step 1: move files 2016-04-18 16:43:18 -07:00
Brian Warner
1688da74d9 split txwormhole step 2: fix imports 2016-04-18 16:41:56 -07:00
Brian Warner
323175ddfe split txwormhole step 1: move files 2016-04-18 16:41:52 -07:00
Brian Warner
0b162af09e move ipaddrs.py from wormhole.util to wormhole.twisted 2016-04-18 16:24:13 -07:00
Brian Warner
87266f1292 split wormhole_server step 3: fix tests 2016-04-18 16:18:51 -07:00
Brian Warner
190ce2c027 split wormhole_server step 2: new files, fix imports 
remove server commands from old src/wormhole/scripts/runner.py
 2016-04-18 16:18:47 -07:00
Brian Warner
bde4696a8b split wormhole_server step 1: move files 2016-04-18 16:18:45 -07:00
Brian Warner
1a455c05f5 remove wormhole.twisted.util port-allocation functions 2016-04-18 16:16:27 -07:00
Brian Warner
e20026488a remove unused observer.py 2016-04-18 16:16:27 -07:00
Brian Warner
a8446d2bc5 merge transit_common into twisted.transit 2016-04-18 16:16:27 -07:00
Brian Warner
db137c26e5 remove blocking.transit 2016-04-18 15:52:29 -07:00
Brian Warner
4e937c2100 rename cmd_send_twisted to cmd_send, same for cmd_receive 2016-04-18 15:52:29 -07:00
Brian Warner
a1033b06a3 merge send_common into cmd_send_twisted 2016-04-18 15:52:29 -07:00
Brian Warner
9b53bb96c6 merge cmd_receive_blocking into cmd_receive_twisted 2016-04-18 15:52:29 -07:00
Brian Warner
8c67a98259 merge test_transit.py into test_server.py 2016-04-18 15:52:29 -07:00
Brian Warner
68b22bec97 remove --twisted, stop using blocking-flavor scripts 2016-04-18 15:52:18 -07:00
Brian Warner
94a3be91b2 add tab-completion to twisted-style input_code() 2016-04-15 17:27:32 -07:00
Brian Warner
589226f076 tor: add comments, let it pick its own control port 2016-03-28 02:26:11 -07:00
Brian Warner
618706681a record time spent importing code 2016-03-28 02:25:40 -07:00
Brian Warner
9630ab9aae find-tor: record more detailed timings 2016-03-28 02:25:32 -07:00
Brian Warner
12c4c51fd8 record tor-launch time in DebugTiming 2016-03-28 02:25:01 -07:00
Brian Warner
01ed9902de add --tor support 2016-03-28 02:25:01 -07:00
Brian Warner
ed6e5ff169 get a TorManager working 2016-03-28 02:25:01 -07:00
Brian Warner
049fac01db tolerate trailing slash on "wormhole send dirname/" 
Previously, the trailing slash would cause the receiving side to get an
empty-named directory.
 2016-03-24 08:46:29 -07:00
Brian Warner
c5415495c0 dump-timing: store server-sent time too 
Adjust dump-timing to ignore the extra data, for now. Also do some
general instrumentation cleanup.
 2016-03-03 18:03:27 -08:00
Brian Warner
aaf4e70a33 remove obsolete TODO 2016-03-03 17:56:15 -08:00
Brian Warner
626732b730 relay_server: include 'sent' timestamp with each message 
This will be used by client-side timeline instrumentation, to guide
performance improvements.
 2016-03-03 17:52:59 -08:00
Brian Warner
e342236b7d relay_server: internal rearrangement 
Construct the response in the render_* method, rather than
add_messages(), to prepare for adding server timestamps. Use
json_response() everywhere.
 2016-03-03 16:57:44 -08:00
Brian Warner
9d7b9dd8d2 add --no-listen, for debugging 2016-03-03 14:52:14 -08:00
Brian Warner
418fe9419e twisted: use persistent connections, slight speedup 
This uses a single TCP connection to the relay server for all
requests (although it probably uses a second one for the downstream
EventSource feed). This should squeeze out some of the round-trip times.
 2016-03-03 13:39:09 -08:00
Brian Warner
4f5e037d39 cmd-twisted: report exceptions better 
The previous scheme would swallow the real traceback, making debugging
difficult.
 2016-03-03 12:33:26 -08:00
Brian Warner
08a5e6043a twisted.transit: tolerate the lack of a listener 
In Tor mode, there is no local listener. Eventually we'll add an Onion
Service listener, but it's not guaranteed that we can run one.
 2016-03-03 12:32:26 -08:00
Brian Warner
8d92b40139 twisted.transcribe: use same Agent for ReconnectingEventSource 
This will be especially important when we add Tor support.
 2016-03-03 12:29:57 -08:00
Brian Warner
be2c66a5f5 transit: ignore errors on sockets we were about to close anyways 2016-03-02 14:28:04 -08:00
Brian Warner
26f512fba4 add --twisted to use the twisted CLI flavor 2016-03-02 01:07:37 -08:00
Brian Warner
6d3d0c1cb3 unify synchronous calling of twisted CLI commands 2016-03-02 00:51:21 -08:00
Brian Warner
4d405c8cef cmd_send_twisted: fix display message ordering 
By creating the ProgressingFileSender too early, it printed a progress
message before the "Sending" line.
 2016-03-02 00:50:22 -08:00
Brian Warner
df2384bea2 twisted.transit: move FileConsumer into RecordPipe 
This adds an expected= argument to Connection.connectConsumer(), which
then returns a Deferred that fires when enough bytes have been written
to the consumer. It also adds Connection.writeToFile(), a helper method
that writes bytes to a filehandle.
 2016-03-02 00:48:43 -08:00
Brian Warner
7234e25897 twisted.transit: handle multiple records in one chunk 
I made the classic dataReceived() mistake, and exited the function after
delivering the first record. Keep at it until there are no complete
records left.
 2016-03-02 00:48:43 -08:00
Brian Warner
8d82726c51 add DebugTiming object, --dump-timing= option 
This writes timeline data to a .json file, which can be examined later
to find likely candidates for optimization.
 2016-03-01 18:23:06 -08:00
Brian Warner
84def8a54b add some inlineCallbacks for simplicity 
This control flow was getting too hairy.
 2016-03-01 18:22:03 -08:00
Brian Warner
fd143caded tests.ServerBase: print message when threadpool is not idle 
The previous commits improve test failures by dropping relay connections
at shutdown, and flunking a test quickly when one client fails but the
other one hangs.

If that doesn't work (say, some client has a time.sleep(), or other
stall that isn't affected by the relay shutdown), we'll be left with an
active thread holding that hanging client.

This patch adds a check to wormhole.test.common.ServerBase.tearDown that
looks for active threads, waits a second (after stopService), then
checks the threadpool again. If the threadpool is empty, everything is
fine. If not, it prints a message (to stdout) to inform the impatient
user why the test is probably hanging.
 2016-03-01 17:07:48 -08:00
Brian Warner
b1dae14e6d test_scripts: handle hangs in one client 
When test_scripts ran two clients at the same time, an error in one
could leave the other hanging (in a thread). One Deferred would errback,
the other would hang. Tests wait on one Deferred at a time, so if we're
unlucky and were waiting on the hanging Deferred (instead of the
erroring one), we'll wait forever, or at least until the default test
timeout of 180 seconds.

This adds an errback to notice when either client has errored, and
cancels the other Deferred, so it doesn't matter which one we wait upon
first.
 2016-03-01 17:07:43 -08:00
Brian Warner
3fc3a563bf relay_server: disconnect clients upon shutdown 2016-03-01 17:07:37 -08:00
Brian Warner
0771aae7c7 server: Relay wasn't pruning channels 
I forgot to hook it up to the service parent, so the timer was never
started.
 2016-02-29 08:59:53 -08:00
Brian Warner
c225d57f8c cmd_receive_twisted.py: twisted-based implementation 2016-02-29 07:31:34 -08:00
Brian Warner
01064325a2 cmd_receive: refactor (slight message changes) 
This prepares the way for a twisted-based implementation.
 2016-02-29 07:30:40 -08:00
Brian Warner
6654efb429 move describe() from Transit to RecordPipe 2016-02-28 01:42:46 -08:00
Brian Warner
1903c58248 ipaddrs: return 127.0.0.1 if everything else fails 
This helps the windows tests where ipaddrs currently fails entirely.
 2016-02-27 17:37:12 -08:00
Brian Warner
ca06e95bbd test_blocking: avoid using real hints 2016-02-27 17:29:04 -08:00
Brian Warner
8067acbf82 cmd_receive: close the file before renaming 
I'm always forgetting this one. It's more obvious on windows.
 2016-02-27 17:20:43 -08:00
Brian Warner
ed1eb06ae8 blocking/transit: use descriptive error message 2016-02-27 14:48:12 -08:00
Brian Warner
1ff0792b32 test_scripts: prep for windows compatibility 
newlines, os.mkfifo errors, JSON type differences
 2016-02-27 14:46:38 -08:00
Brian Warner
14dcfeed73 tolerate lack of readline at runtime 
'readline' is part of the python stdlib, so declaring a dependency on it
doesn't help. It doesn't exist on windows, and the pypi 'readline'
module doesn't work on windows. So instead, just attempt to import
readline, and if that fails, fall back to a non-completion flavor.
 2016-02-27 14:16:58 -08:00
Brian Warner
b7319f90c8 windows "route.exe" is not absolute, and that's ok 2016-02-27 14:15:36 -08:00
Brian Warner
dbba482c62 cmd_receive_blocking.py: refactor 2016-02-17 21:35:53 -08:00
Brian Warner
e6fba34570 start on twisted receive: move old files into new homes 2016-02-17 19:25:19 -08:00
Brian Warner
7f056ca075 send-twisted: set transit key before sending phase1 
This ensures that we'll be ready for them. Previously there was a race
between us revealing the direct hints to the peer, and us setting the
transit key (thus allowing us to check inbound handshake requests). The
Transit instance didn't handle the race, causing errors to be thrown
when the other side connected quickly.
 2016-02-17 19:02:36 -08:00
Brian Warner
aa27bfd32c add twisted form of sender 
Currently this is only invokable from tests.
 2016-02-17 19:02:36 -08:00
Brian Warner
7ceffd783a add more assertions around transit_key 2016-02-17 19:02:35 -08:00
Brian Warner
3ffceff9d5 send-blocking: set transit key before sending phase1 
This ensures that we'll be ready for them. Previously there was a race
between us revealing the direct hints to the peer, and us setting the
transit key (thus allowing us to check inbound handshake requests). The
Transit instance handles this race (with an interlock on the transit
key), but it's still nicer to do it cleanly.

This exposed a new race in Transit, where the inbound connection would
complete before transit.connect() had been called. The previous commit
adds an interlock to wait for that too. Until this change, the transit
key lock was covering that one up.
 2016-02-17 19:02:35 -08:00
Brian Warner
142f3fc154 send-blocking: don't create Transit unless we need it 2016-02-17 19:02:35 -08:00
Brian Warner
dba42aff01 blocking transit: tolerate inbound connections before connect() starts 2016-02-17 19:02:35 -08:00
Brian Warner
369854b4cc test build_phase1_data 2016-02-17 18:07:35 -08:00
Brian Warner
79decea9ea rearrange scripts: make room for twisted 2016-02-17 16:26:03 -08:00
Brian Warner
d36d1cb063 runner.py: new approach to dispatching 2016-02-17 16:03:30 -08:00
Brian Warner
3b447df0f4 move argparser out to a separate module 2016-02-17 15:55:11 -08:00
Brian Warner
5d572137f4 improve test error message 2016-02-17 14:00:03 -08:00
Brian Warner
fd7d1b97d4 test scripts in a thread, not a subprocess, mostly 2016-02-17 14:00:01 -08:00
Brian Warner
903129f4a2 add --hide-progress, mostly for tests 2016-02-17 13:58:41 -08:00
Brian Warner
c5b2800a3e runner: strictly use cwd/stdout/stderr from 'args' 
This will make it easier to test the scripts in a controlled fashion.
 2016-02-17 13:53:20 -08:00
Brian Warner
e2f3bebe38 allow --relay-helper="" to disable relay 
test_scripts now uses this to avoid accidentally using a relay
 2016-02-17 13:53:18 -08:00
Brian Warner
d14d35f3cd cmd_send_blocking: close the transit when we're done 2016-02-17 13:52:26 -08:00
Brian Warner
5e928ac9f0 rewrite ProgressPrinter as a class, add tests 2016-02-17 12:46:12 -08:00
Brian Warner
00833a4bde test_scripts: factor out common test code 2016-02-17 10:22:31 -08:00
Brian Warner
99ff75259a catch 'send' of non-file/non-directory 2016-02-17 09:36:31 -08:00
Brian Warner
8079340bff more rearranging 2016-02-15 21:59:18 -08:00
Brian Warner
512c7c0a59 cmd_send: rearrange in preparation for twisted version 2016-02-15 21:40:57 -08:00
Brian Warner
a235b507c8 twisted.transit: implement producer/consumer flow control 2016-02-15 21:23:20 -08:00
Brian Warner
763d72f582 twisted.transit: implement Deferred-based receive_record() 2016-02-15 11:40:21 -08:00
Brian Warner
fb1461fa8c add twisted.transit, with tests 2016-02-14 17:57:09 -08:00
Brian Warner
7212e9e9f4 transit: split common code out to a new file 
This will be shared between blocking.transit and the soon-to-land
twisted.transit .
 2016-02-14 17:51:46 -08:00
Brian Warner
36cc0e2bca tests: don't use network 
Some tests failed to override --transit-helper, which meant they
intermittently talked to the real transit server (briefly, before
deciding the local+direct connection was better).
 2016-02-12 17:11:06 -08:00
Brian Warner
7ded461342 show-usage: give more room for "rendezvous-lonely" 2016-01-14 14:30:08 -08:00
Brian Warner
bd912a73ee Transit: blur the file sizes too 2016-01-12 16:32:38 -08:00
Brian Warner
35c9e29eb3 transit handshake: wait for newline, not just expected string 2016-01-12 15:04:52 -08:00
Brian Warner
9ed39be346 don't log HTTP requests when blur-usage is on 2015-12-04 17:35:56 -08:00
Brian Warner
1ab66d2fd0 privacy: only store coarse timestamps in the usage table 2015-12-03 21:15:19 -08:00
Brian Warner
ee86059b0a transit_server: clean up logging slightly 2015-12-03 20:44:34 -08:00
Brian Warner
6c88396f14 CLI: make 'wormhole server usage' show transit too 2015-12-03 20:15:29 -08:00
Brian Warner
909cdfa3dc track transit usage in DB 2015-12-03 19:45:34 -08:00
Brian Warner
a3656c162b schema change: prep usage table for including transit too 2015-12-03 19:44:47 -08:00
Brian Warner
fb493da8c7 transit_server.py: underscore-ify private attributes 2015-12-03 19:25:40 -08:00
Brian Warner
6369c17595 test_transit: use better error in Accumulator 
apparently this was broken before, but never got triggered
 2015-12-03 19:25:40 -08:00
Brian Warner
568903ac48 add test for Transit (blocking only) 2015-12-03 16:24:52 -06:00
Brian Warner
229c702ccc transit: fix py3 2015-12-03 16:23:12 -06:00
Brian Warner
95d0e68cf2 transit: avoid near-infinite loop upon connector error 
Now we will never try any hint more than once. Previously we'd hit the
relay hint over and over until the timeout fired.
 2015-12-03 16:22:03 -06:00
Brian Warner
11f806a316 transit_relay: add Producer/Consumer flow control 
This limits the buffering to about 10MB (per connection*direction).
Previously, if the sender had more bandwidth than the receiver, the
transit relay would buffer the entire file. With this change, the sender
will be throttled to match the receiver's downstream speed.
 2015-12-03 15:07:47 -06:00
Brian Warner
c103441648 test_transit: fix py3.3, no %s in bytestrings 2015-12-02 03:02:50 -06:00
Brian Warner
8240d9e910 add test of transit server, make it work under py3 
Also have transit send logs to log.msg, not stdout.
 2015-12-02 01:47:52 -06:00
Brian Warner
80603aaa32 finish py3/async support, needs Twisted >= 15.5.0 
The latest Twisted fixes the web.Agent code we need for proper async
support. There's still a daemonization bug that prevents 'wormhole
server start' from succeeding (it hangs).
 2015-12-01 00:15:24 -06:00
Brian Warner
5b46df133f implement (and test) --output-file for receive-file 2015-11-29 01:40:25 -06:00
Brian Warner
3a343f9895 test send/receive directory 2015-11-29 01:33:15 -06:00
Brian Warner
6fefcde061 add send/receive of whole directories 2015-11-25 01:53:04 -06:00
Brian Warner
6958241f3f preliminary refactoring 2015-11-25 01:47:24 -06:00
Brian Warner
1428507909 refactor cmd_receive.py, split accept_file() to a separate function 2015-11-25 01:40:17 -06:00
Brian Warner
a96f29d01f wormhole server show-usage / tail-usage 
Use a separate "tail-usage" command instead of "show-usage -f". Make
both work on py3 too.
 2015-11-24 13:20:58 -08:00
Brian Warner
0c36fad720 add "wormhole server usage" to dump usage DB 2015-11-24 13:03:53 -08:00
Brian Warner
1d6c3d1f96 clients: use "watch" endpoint, not "get" 2015-11-23 16:50:54 -08:00
Brian Warner
2318c94169 test_server: error properly when EventSource fails 2015-11-23 16:49:46 -08:00
Brian Warner
c482c248ff server: add "watch" endpoint, deprecate non-ES "get" 
I'm planning to leave non-EventSource "/get" in until after 0.6.0, then
remove it. I think it's cleaner for the logs to have the two
forms (EventSource and immediate) use different URLs.
 2015-11-23 16:43:25 -08:00
Brian Warner
82cdadae80 check welcome message 'send' too 2015-11-23 16:33:09 -08:00
Brian Warner
badf5168ef test_server: rename some functions 2015-11-22 18:06:42 -08:00
Brian Warner
5e0a7e4c93 remove stale copy of server URLs 2015-11-22 17:44:25 -08:00
Brian Warner
00bb816d11 remove close-on-error from derive_key(), for now 
In the twisted-style code, the close_on_error() decorator forces the
return value to be a Deferred, which is all wrong for internal uses of
derive_key() (verification string and confirmation message). It might be
useful to have a synchronous form of close_on_error(), but since the
actual close() is async, that's not very straightforward.

So for now, tolerate unclosed Wormhole objects when someone calls
derive_key() too early, or with a non-unicode type string.
 2015-11-19 17:11:27 -08:00
Brian Warner
22a1ce2eda add close-with-mood-on-error to twisted style too 2015-11-19 17:08:21 -08:00
Brian Warner
9827a2e50c add twisted/blocking interop test 2015-11-19 16:21:10 -08:00
Brian Warner
fd9a62e8ff change confirmation message: must be different on each side 
The previous same-message-for-both-sides approach failed, because the
Channel filters out duplicates.
 2015-11-19 16:06:30 -08:00
Brian Warner
1ad001bbc3 WIP: test that we tolerate missing key-confirmation messages 2015-11-16 18:25:28 -08:00
Brian Warner
6b57d7d05d check key-confirmation messages, if present 2015-11-16 18:24:39 -08:00
Brian Warner
3220014605 send key-confirmation message upon receipt of PAKE 
This will allow the first peer to detect a wrong password even though
the second peer bails before sending something encrypted.
 2015-11-16 17:02:02 -08:00
Brian Warner
4ad7342459 make self.channel internal (twisted) 2015-11-16 16:54:00 -08:00
Brian Warner
ae2a6c6a05 add Channel.get_first_of() 
This allows the Wormhole code to wait for multiple messages, which will
be useful for getting Confirmation messages soon.
 2015-11-16 16:47:52 -08:00
Brian Warner
b709a45891 get_data/set_data: reserve _ for internal uses 2015-11-16 16:20:00 -08:00
Brian Warner
6956f35e9a receive: fetch channel list before completion, to get welcome message 2015-11-15 10:53:13 -08:00
Brian Warner
47d3eee6fe server: treat missing moods (from older clients) as "quiet" 2015-11-15 10:34:40 -08:00
Brian Warner
4f0dde9529 server: summarize transfers, store in DB 2015-11-15 10:34:29 -08:00
Brian Warner
26c7008c23 DB: use 'messages' to track allocations, not 'allocations' 
This removes the 'allocations' table entirely, and cleans up the way we
prune old messages. This should make it easier to summarize each
connection (for usage stats) when it gets deallocated, as well as making
pruning more reliable.
 2015-11-13 18:24:36 -08:00
Brian Warner
bb97729a23 server: more refactoring 
flattening some attribute access paths
 2015-11-13 18:20:47 -08:00
Brian Warner
0b9f858761 server: internal refactoring 2015-11-13 18:12:47 -08:00
Brian Warner
101c800237 deallocate: ignore all 'requests' exceptions 2015-11-12 10:11:30 -08:00
Brian Warner
2e393c145e make default mood "happy", change other mood names 2015-11-12 09:31:03 -08:00
Brian Warner
80beb20631 make blocking.Wormhole into a context manager 2015-11-11 21:59:16 -08:00
Brian Warner
a881d6055f auto-close Channel (with a "mood") upon server or crypto error 2015-11-11 21:54:45 -08:00
Brian Warner
0748647049 allow multiple close() calls, throw error when using a closed Wormhole 2015-11-11 18:17:52 -08:00
Brian Warner
3daef13ac0 indent commands: no functional changes 2015-11-11 18:11:53 -08:00
Brian Warner
cb5ad8ced1 Use exception for Timeout, not return value 2015-11-11 18:01:22 -08:00
Brian Warner
6de677c1df use timeouts for allocate and list_channels too 2015-11-11 18:00:06 -08:00
Brian Warner
fa3be3523d pass timeouts down 2015-11-11 17:56:08 -08:00
Brian Warner
07686f3de7 make self.channel internal 2015-11-11 17:52:13 -08:00
Brian Warner
9ead3f48a8 fix 'wormhole server restart' 
It got broken when I added --no-daemon to 'server start'.
 2015-11-11 17:30:31 -08:00
Brian Warner
ec51adb713 add 'wormhole send --text -' to read message from stdin 2015-11-11 17:27:26 -08:00
Brian Warner
b83062701d server: give old 0.4.0 senders a "you must upgrade" error 
Without this, old senders will throw a messy 404 traceback when talking
to a modern server.

Unfortunately 0.4.0 receivers don't make API calls in the right order,
so they throw a 404 before seeing our "you need to upgrade" message.
 2015-11-11 17:06:33 -08:00
Brian Warner
595a0e5845 server API: include "mood" when closing the channel 
This will be used as a simple unverifiable counter of success/failure.
 2015-10-08 18:35:26 -07:00
Brian Warner
0a6ab83bc5 test distinctness of encryption nonces 2015-10-07 16:31:45 -07:00
Brian Warner
385762b36d demo.py: make it easier to modify for local testing 2015-10-07 16:31:45 -07:00
Brian Warner
df3aee2a86 fix EventSource (server and parsers) 
I was really confused about the Server-Sent Events syntax. This new one
is compatible with actual web browsers and the spec:

 http://www.w3.org/TR/eventsource/
 2015-10-07 16:31:45 -07:00
Brian Warner
e77b39313a use TypeError for type errors, not UsageError 2015-10-06 20:52:47 -07:00
Brian Warner
ce236ae70c test_blocking: stop aliasing Wormhole 2015-10-06 20:44:32 -07:00
Brian Warner
bf43dae2ad add multiple phases, change key-derivation strings 
Because of the key-derivation change, clients will not be compatible
across this commit.
 2015-10-06 20:39:20 -07:00
Brian Warner
d0a7da3a63 twisted/demo.py: fix receive, code must be unicode 2015-10-06 20:33:17 -07:00
Brian Warner
6e8a1d8adb codes.py: fix input-with-completion on py3 
The input() function returns str(bytes) on py2, and str(unicode) on py3.
We want unicode in both cases.
 2015-10-06 20:04:56 -07:00
Brian Warner
35768d6738 wormhole/invitation code is now unicode 2015-10-06 19:42:10 -07:00
Brian Warner
7f6410812c transit URL, transit hints are now unicode 2015-10-06 19:29:59 -07:00
Brian Warner
574d5f2314 scope channelids to the appid, change API and DB schema 
This requires a DB delete/recreate when upgrading. It changes the server
protocol, and app IDs, so clients cannot interoperate with each other
across this change, nor with the server. Flag day for everyone!

Now apps do not share channel IDs, so a lot of usage of app1 will not
cause the wormhole codes for app2 to get longer.
 2015-10-06 19:21:53 -07:00
Brian Warner
8692bd2cd7 eventsource.py: use py3-style print 2015-10-06 19:21:53 -07:00
Brian Warner
9e1a00cbd9 appid and derive_key(purpose) are now unicode 2015-10-06 19:21:53 -07:00
Brian Warner
9ba7de6e1e relay-url is now unicode 2015-10-06 19:19:39 -07:00