Commit Graph

379 Commits

Author SHA1 Message Date
Brian Warner
5a018c23f4 dump-timing: try to display the server-sent time usefully
Not really satisfied yet.
2016-03-03 18:06:26 -08:00
Brian Warner
c5415495c0 dump-timing: store server-sent time too
Adjust dump-timing to ignore the extra data, for now. Also do some
general instrumentation cleanup.
2016-03-03 18:03:27 -08:00
Brian Warner
aaf4e70a33 remove obsolete TODO 2016-03-03 17:56:15 -08:00
Brian Warner
626732b730 relay_server: include 'sent' timestamp with each message
This will be used by client-side timeline instrumentation, to guide
performance improvements.
2016-03-03 17:52:59 -08:00
Brian Warner
e342236b7d relay_server: internal rearrangement
Construct the response in the render_* method, rather than
add_messages(), to prepare for adding server timestamps. Use
json_response() everywhere.
2016-03-03 16:57:44 -08:00
Brian Warner
9d7b9dd8d2 add --no-listen, for debugging 2016-03-03 14:52:14 -08:00
Brian Warner
fdf96b07fe dump-timing: show elapsed time on click 2016-03-03 14:45:47 -08:00
Brian Warner
418fe9419e twisted: use persistent connections, slight speedup
This uses a single TCP connection to the relay server for all
requests (although it probably uses a second one for the downstream
EventSource feed). This should squeeze out some of the round-trip times.
2016-03-03 13:39:09 -08:00
Brian Warner
4f5e037d39 cmd-twisted: report exceptions better
The previous scheme would swallow the real traceback, making debugging
difficult.
2016-03-03 12:33:26 -08:00
Brian Warner
08a5e6043a twisted.transit: tolerate the lack of a listener
In Tor mode, there is no local listener. Eventually we'll add an Onion
Service listener, but it's not guaranteed that we can run one.
2016-03-03 12:32:26 -08:00
Brian Warner
8d92b40139 twisted.transcribe: use same Agent for ReconnectingEventSource
This will be especially important when we add Tor support.
2016-03-03 12:29:57 -08:00
Brian Warner
ca99784cd5 coveragerc: omit _version.py 2016-03-02 15:00:51 -08:00
Brian Warner
91269d18b1 README: Add codecov.io badge 2016-03-02 14:42:19 -08:00
Brian Warner
be2c66a5f5 transit: ignore errors on sockets we were about to close anyways 2016-03-02 14:28:04 -08:00
Brian Warner
60079a9fcf travis: add codecov, stop using tox, more
This runs tests directly (with 'coverage run' and 'trial'), rather than
running them inside tox, because that was proving difficult.

It upgrades pip first (the travis builders have an older version), to
use and cache wheels for speed. Travis is now configured to retain the
pip cache between runs.

This installs twisted from trunk (as tox was doing), to work around a
bug in the current twisted-15.5.0 release that should be resolved soon.

Travis is told to ignore any branch named WIP-, so I can transfer
branches between computers without causing red builds.

A coverage report is displayed before uploading to codecov.

The wormhole package is installed as --editable, to make the source
filenames in the coverage data match what I get at home. This also
appears necessary to get codecov to display the data (maybe it ignores
data for files outside the starting directory?).
2016-03-02 14:10:59 -08:00
Brian Warner
0e5286c7d3 .coveragerc: better config 2016-03-02 13:35:43 -08:00
Brian Warner
d86dc05b5f tox: remove 'meta', do pyflakes/smoketest in each build 2016-03-02 13:35:12 -08:00
Brian Warner
26f512fba4 add --twisted to use the twisted CLI flavor 2016-03-02 01:07:37 -08:00
Brian Warner
6d3d0c1cb3 unify synchronous calling of twisted CLI commands 2016-03-02 00:51:21 -08:00
Brian Warner
4d405c8cef cmd_send_twisted: fix display message ordering
By creating the ProgressingFileSender too early, it printed a progress
message before the "Sending" line.
2016-03-02 00:50:22 -08:00
Brian Warner
df2384bea2 twisted.transit: move FileConsumer into RecordPipe
This adds an expected= argument to Connection.connectConsumer(), which
then returns a Deferred that fires when enough bytes have been written
to the consumer. It also adds Connection.writeToFile(), a helper method
that writes bytes to a filehandle.
2016-03-02 00:48:43 -08:00
Brian Warner
7234e25897 twisted.transit: handle multiple records in one chunk
I made the classic dataReceived() mistake, and exited the function after
delivering the first record. Keep at it until there are no complete
records left.
2016-03-02 00:48:43 -08:00
Brian Warner
84749dd8b3 add misc/dump-timing.py too, to visualize the timeline 2016-03-01 18:23:19 -08:00
Brian Warner
8d82726c51 add DebugTiming object, --dump-timing= option
This writes timeline data to a .json file, which can be examined later
to find likely candidates for optimization.
2016-03-01 18:23:06 -08:00
Brian Warner
84def8a54b add some inlineCallbacks for simplicity
This control flow was getting too hairy.
2016-03-01 18:22:03 -08:00
Brian Warner
fd143caded tests.ServerBase: print message when threadpool is not idle
The previous commits improve test failures by dropping relay connections
at shutdown, and flunking a test quickly when one client fails but the
other one hangs.

If that doesn't work (say, some client has a time.sleep(), or other
stall that isn't affected by the relay shutdown), we'll be left with an
active thread holding that hanging client.

This patch adds a check to wormhole.test.common.ServerBase.tearDown that
looks for active threads, waits a second (after stopService), then
checks the threadpool again. If the threadpool is empty, everything is
fine. If not, it prints a message (to stdout) to inform the impatient
user why the test is probably hanging.
2016-03-01 17:07:48 -08:00
Brian Warner
b1dae14e6d test_scripts: handle hangs in one client
When test_scripts ran two clients at the same time, an error in one
could leave the other hanging (in a thread). One Deferred would errback,
the other would hang. Tests wait on one Deferred at a time, so if we're
unlucky and were waiting on the hanging Deferred (instead of the
erroring one), we'll wait forever, or at least until the default test
timeout of 180 seconds.

This adds an errback to notice when either client has errored, and
cancels the other Deferred, so it doesn't matter which one we wait upon
first.
2016-03-01 17:07:43 -08:00
Brian Warner
3fc3a563bf relay_server: disconnect clients upon shutdown 2016-03-01 17:07:37 -08:00
Brian Warner
0771aae7c7 server: Relay wasn't pruning channels
I forgot to hook it up to the service parent, so the timer was never
started.
2016-02-29 08:59:53 -08:00
Brian Warner
9293709e97 add NEWS for 0.6.3 release 2016-02-29 08:10:13 -08:00
Brian Warner
25ee48e989 add .coveragerc 2016-02-29 07:48:57 -08:00
Brian Warner
c225d57f8c cmd_receive_twisted.py: twisted-based implementation 2016-02-29 07:31:34 -08:00
Brian Warner
01064325a2 cmd_receive: refactor (slight message changes)
This prepares the way for a twisted-based implementation.
2016-02-29 07:30:40 -08:00
Brian Warner
6654efb429 move describe() from Transit to RecordPipe 2016-02-28 01:42:46 -08:00
Brian Warner
1903c58248 ipaddrs: return 127.0.0.1 if everything else fails
This helps the windows tests where ipaddrs currently fails entirely.
2016-02-27 17:37:12 -08:00
Brian Warner
ca06e95bbd test_blocking: avoid using real hints 2016-02-27 17:29:04 -08:00
Brian Warner
8067acbf82 cmd_receive: close the file before renaming
I'm always forgetting this one. It's more obvious on windows.
2016-02-27 17:20:43 -08:00
Brian Warner
f28f294138 first attempt at windows builds, with appveyor.com
The .appveyor.yml tells the tox.ini to install pypiwin32 directly,
rather than through a "twisted[windows]" extra. I wasn't able to get the
"extra" syntax to work properly under appyeyor, even though it worked ok
for me locally.

We use an extra "meta" tox environment to run "wormhole --version". This
confirms that versioneer is getting the right value, since we're now
running it from an installed sdist, rather than a checked-out git source
tree.
2016-02-27 16:11:02 -08:00
Brian Warner
ed1eb06ae8 blocking/transit: use descriptive error message 2016-02-27 14:48:12 -08:00
Brian Warner
1ff0792b32 test_scripts: prep for windows compatibility
newlines, os.mkfifo errors, JSON type differences
2016-02-27 14:46:38 -08:00
Brian Warner
cc9b2c574b tox: temporary workaround for twisted bug
Install twisted from git trunk until a release after twisted-15.5.0.
2016-02-27 14:43:33 -08:00
Brian Warner
101ac4c1af switch to tox for running tests 2016-02-27 14:39:56 -08:00
Brian Warner
14dcfeed73 tolerate lack of readline at runtime
'readline' is part of the python stdlib, so declaring a dependency on it
doesn't help. It doesn't exist on windows, and the pypi 'readline'
module doesn't work on windows. So instead, just attempt to import
readline, and if that fails, fall back to a non-completion flavor.
2016-02-27 14:16:58 -08:00
Brian Warner
b7319f90c8 windows "route.exe" is not absolute, and that's ok 2016-02-27 14:15:36 -08:00
Brian Warner
dbba482c62 cmd_receive_blocking.py: refactor 2016-02-17 21:35:53 -08:00
Brian Warner
e6fba34570 start on twisted receive: move old files into new homes 2016-02-17 19:25:19 -08:00
Brian Warner
7f056ca075 send-twisted: set transit key before sending phase1
This ensures that we'll be ready for them. Previously there was a race
between us revealing the direct hints to the peer, and us setting the
transit key (thus allowing us to check inbound handshake requests). The
Transit instance didn't handle the race, causing errors to be thrown
when the other side connected quickly.
2016-02-17 19:02:36 -08:00
Brian Warner
aa27bfd32c add twisted form of sender
Currently this is only invokable from tests.
2016-02-17 19:02:36 -08:00
Brian Warner
7ceffd783a add more assertions around transit_key 2016-02-17 19:02:35 -08:00
Brian Warner
3ffceff9d5 send-blocking: set transit key before sending phase1
This ensures that we'll be ready for them. Previously there was a race
between us revealing the direct hints to the peer, and us setting the
transit key (thus allowing us to check inbound handshake requests). The
Transit instance handles this race (with an interlock on the transit
key), but it's still nicer to do it cleanly.

This exposed a new race in Transit, where the inbound connection would
complete before transit.connect() had been called. The previous commit
adds an interlock to wait for that too. Until this change, the transit
key lock was covering that one up.
2016-02-17 19:02:35 -08:00