Commit Graph

761 Commits

Author SHA1 Message Date
Brian Warner
899e539cb1 cmd_ssh: ~ is not always /home
On OS-X, in particular, ~foo expands to /Users/foo . We delegate
everything to os.path.expanduser(), which should know the local
convention.
2016-08-15 17:35:38 -07:00
Brian Warner
75d362f60a add some basic tests, doesn't cover everything 2016-08-15 17:35:34 -07:00
Brian Warner
d057b91371 xfer_util: work on py2+py3
wormhole.send takes bytes, but the utility functions take strings. So
encode the JSON blob before sending, and decode it on the way back out.
2016-08-15 17:35:34 -07:00
meejah
fe29b3130b 'wormhole ssh' cleanups
- move to 'wormhole ssh' group with accept/invite subcommands
- change names of methods
- check for permissions
- use --user option (instead of --auth-file)
- move implementation to cmd_ssh.py
- if multiple public-keys, ask user
2016-08-15 00:30:39 -06:00
meejah
069b76485b Add 'wormhole ssh-add' and 'wormhole ssh-send' commands 2016-08-14 19:59:20 -06:00
meejah
026c8fd093 Print proper tracebacks when inlineCallbacks + yield involved 2016-08-14 16:50:29 -06:00
meejah
2a332471d3 add pyflakes to 'dev' extra 2016-08-14 16:49:34 -06:00
meejah
afa123abae make tests work on pypy 2016-08-04 15:57:01 -04:00
David Reid
6aa8a47f9a Use StreamServerEndpointService (available since Twisted 10.2) 2016-08-04 00:23:02 -07:00
Brian Warner
dc6416a257 fix stats-writing bug on py3
refs #67
2016-08-01 16:31:05 -07:00
Brian Warner
094a121f3a Merge branch 'pr70' 2016-08-01 15:32:27 -07:00
Daniel Kahn Gillmor
eb46656244 wormhole-server needs a hyphen 2016-07-31 19:50:28 -04:00
Daniel Kahn Gillmor
2fafdf34ee help bad spelers
Some of us can never remember the old ditty:

   i before e, except after c
   or when sounding like "a"
   as in neighbor or weigh.

Perhaps magic wormhole can coddle us in our misorthography :)
2016-07-31 18:55:42 -04:00
Brian Warner
5f76885d78 munin: add transit_alltime, minor cleanups 2016-07-28 17:52:03 -07:00
Brian Warner
0ba59136a0 upgrade to versioneer-0.16 2016-07-27 19:08:12 -07:00
Brian Warner
21719b6f9c MANIFEST.in: include the correct munin plugins 2016-07-27 19:04:01 -07:00
Brian Warner
27ae3b722c munin: improve 'events' ordering, add new plugin 2016-07-27 18:43:10 -07:00
Brian Warner
890b47cad2 NEWS: update for upcoming 0.8.1 release 2016-07-27 18:31:24 -07:00
Brian Warner
5542545165 CLI: move most top-level args down into the subcommand
So instead of "wormhole --verify send", use "wormhole send --verify".

The full set of arguments that were moved down:

* --code-length=
* --verify
* --hide-progress
* --no-listen
* --tor

The following remain as top-level arguments (which should appear after
"wormhole" and before the subcommand):

* --relay-url=
* --transit-helper=
* --dump-timing=
* --version
2016-07-27 17:56:03 -07:00
Brian Warner
894e00bea1 README: add link to PyCon video 2016-07-27 16:53:18 -07:00
Brian Warner
3b03593b9c README: add example
closes #65
2016-07-27 16:51:01 -07:00
Brian Warner
e6b38ecacc cli: start factoring out args into common sets 2016-07-23 17:54:29 -07:00
Brian Warner
37b8ccbe4f cli: mostly use kwargs/setattr 2016-07-22 19:13:59 -07:00
Brian Warner
1a82846578 cli.Config: remove common attributes
The values set by the base Config constructor could mask Click parsers
that weren't supplying defaults properly, or which were using different
defaults.
2016-07-14 22:37:35 -06:00
Brian Warner
52ef00b46b CLI: refactor to make testing easier
When tests need a Config object, they now call a function which invokes
Click with a mocked-out go() function, and grabs the Config object
before actually doing anything with it.
2016-07-14 22:34:10 -06:00
Brian Warner
cdb5c19010 websockets: turn on autoPingInterval to expire stale connections
With this, both clients and servers will send a PING at least once every
minute, and will drop connections that haven't seen any traffic for 10
minutes.

This should help keep NAT table entries alive, and will drop connections
that are no longer viable because their NAT entries have expired.

closes #60
2016-07-03 21:51:56 -07:00
Brian Warner
fc9a6f7d99 tox: turn on 'usedevelop'
This should fix the coverage data's filenames: previously they were like
".tox/coverage/lib/python2.7/site-packages/wormhole/foo.py", now they
should be "src/wormhole/foo.py".
2016-07-01 09:41:57 -07:00
Brian Warner
0f6315b3d7 travis: re-enable codecov.io uploads 2016-07-01 09:28:15 -07:00
Brian Warner
bb47c09ebd travis: disable pypy/pypy3
nevermind.. it appears that travis's pypy (2.6-ish) is too old for
PyNaCl to work, and their pypy3 (3.0-3.2ish) is too. Revisit this when
their images get updated.
2016-06-30 16:14:11 -07:00
Brian Warner
f9f6ed31c9 travis: enable pypy, pypy3 2016-06-30 16:10:36 -07:00
Brian Warner
4978be6b90 args: fix --no-listen, --tor
Also add tests to check that argv is being parsed properly, and to check
the defaults.

fixes #59
2016-06-27 14:40:51 -07:00
Brian Warner
0f8b98dec5 DB: log schema upgrades 2016-06-26 18:04:00 -07:00
Brian Warner
de8efdb9f9 cmd_send: wait for ack from close()
Without this, the sender drops the connection before the "close" message
has made it to the server, which leaves the mailbox hanging until it
expires. It still lives in a 'd.addBoth()' slot, so it gets closed even
if some error occurrs, but we wait for it's Deferred to fire in both
success and failure cases.
2016-06-26 17:55:28 -07:00
Brian Warner
1d3b22a0b4 server: hard-code --stats-file=stats.json
We already hard-code 'relay.sqlite', so I don't see a lot of value in
making making the stats file configurable too. That said, if it makes
life easier for packagers (e.g. start-stop-daemon or systemd wanting
these files to go into /var/run/something/ , and if it isn't sufficient
to just use /var/run/something/ as the CWD), I'd accept a patch to
add it back.
2016-06-26 17:55:28 -07:00
Brian Warner
f005d8a9ce Merge branch 'new-prune'
This changes the DB schema and rewrites the expiration/pruning
algorithm. The previous code had several bugs which failed to clean up
nameplates, mailboxes, and messages when clients didn't explicitly close
them before disconnecting (and sometimes even when they did).

The new code runs an expiration loop every 10 minutes, and prunes
anything that is more than 11 minutes old. Clients with a connected
listener (a websocket that has open()ed a Mailbox) update the "in-use"
timestamp at the beginning of the loop. As a result, nameplates and
mailboxes should be pruned within between 10 and 20 minutes after the
last activity.

Clients are expected to reconnect within 9 minutes of a connection being
lost. The current release does not survive a dropped connection, but a
future one will. By raising the 10min/11min settings for specific
applications, a "Persistent Wormhole" mode will tolerate even longer
periods of being offline.

The server now has an option to write stats to a file at the end of each
expiration loop, and the munin plugins have been rewritten to use this
file instead of reading the database directly. The file includes a
timestamp so the plugins can avoid using stale data.

When this new server is run against an old (v2) database, it should
automatically upgrade it to the new v3 schema. The nameplate and mailbox
tables will be erased, but the usage (history) will be preserved.
2016-06-26 17:50:15 -07:00
Brian Warner
930ed2c089 server: log get_stats time
The DB queries this uses aren't particularly efficient, and when the
time it takes to run starts to become a problem, we should do an
optimization pass.
2016-06-26 17:49:36 -07:00
Brian Warner
45d46643ca transit: rename stats keys 2016-06-26 17:49:36 -07:00
Brian Warner
f950161642 rendezvous: rename stats keys for consistency 2016-06-26 17:49:36 -07:00
Brian Warner
df96f2e590 rewrite munin plugins 2016-06-26 17:49:36 -07:00
Brian Warner
638adc71ec server: add transit to stats.json 2016-06-26 17:49:35 -07:00
Brian Warner
cd934fdfde server: rename stats.json keys 2016-06-26 17:49:35 -07:00
Brian Warner
31be12a4be misc/dump-stats.py: tool to show stats.json validity
Run like 'watch python misc/dump-stats.py' against a 'wormhole-server
start --stats-file=stats.json'
2016-06-26 11:31:54 -07:00
Brian Warner
44bc99e987 get_stats: update comments 2016-06-26 11:25:25 -07:00
Brian Warner
f192db5412 track+report mailbox.for_nameplate in usage
This counts the number of "standalone" mailboxes we create, which
happens when a client does open() without first using a nameplate. The
current client doesn't do this, but future clients might.
2016-06-26 11:19:53 -07:00
Brian Warner
db8511c700 schema: add mailbox_usage.for_nameplate too 2016-06-26 11:18:48 -07:00
Brian Warner
bae020d5a7 DB: add mailboxes.for_nameplate
Also add a few indexes to make get_stats() less expensive
2016-06-26 11:01:52 -07:00
Brian Warner
8eb1396f31 server: delete stats file at reboot
This makes it easier to discover an exception in the stats-gathering
timer, since you get a missing stats.json instead of a stale one.
2016-06-26 11:01:52 -07:00
Brian Warner
7b1d4cbcee more stats: count since last reboot 2016-06-26 11:01:52 -07:00
Brian Warner
9e29898207 Rendezvous: add many stats 2016-06-26 11:01:48 -07:00
Brian Warner
62f9a4d702 move stats-file up to RelayServer
This moves responsibility for the periodic prune-everything Timer up to
RelayServer too. That way we can be sure the stats are dumped
immediately after prune, and we can incorporate stats from Transit as
well.
2016-06-25 11:43:13 -07:00