Commit Graph

660 Commits

Author SHA1 Message Date
Brian Warner
f2e011bc9c Add --appid to override the APPID, for subprocess-based wrappers.
Tools which use `wormhole send` under the hood should use a distinct
--appid= (setting the same URL-shaped value on both sides, starting with a
domain name related to the tool and/or its author), so wormhole codes used by
those tools won't compete for short channelids with other tools, or the
default text/file/directory-sending tool.

Closes #113
2016-12-22 15:44:13 -05:00
Brian Warner
57c0894774 cmd_send: flush args.stdout, in case it's different 2016-12-17 17:32:12 -08:00
Joey Hess
8bec79289a send: flush stdout after displaying code
Fixes https://github.com/warner/magic-wormhole/issues/108

Apparently python defaults to buffering stdout when not outputting to
the terminal. It would be better IMHO if every line of stdout were
autoflushed, but based on
http://stackoverflow.com/questions/107705/disable-output-buffering
there's not a really good way to accomplish that in python.
2016-12-17 17:20:45 -04:00
Brian Warner
30af04d245 test_scripts: rename mode to be "slow-text" 2016-12-16 01:33:45 -08:00
Brian Warner
face9423dd internals: ensure _API_establish_key is not called twice 2016-12-16 01:33:17 -08:00
laharah
2d1f474c60 rebased and fixed merge error 2016-12-16 01:06:21 -08:00
laharah
6bdaaf368c Moved slow connection notification to stderr
added test for slow connection notifications
2016-12-16 01:06:21 -08:00
laharah
01318d1130 added notification to cmd_receive plus reactor cleanup
Added a try finally block around verify to ensure notification
deferreds are being cleaned out of the reactor
2016-12-16 01:03:01 -08:00
laharah
a2ab1863cf added the new key waiter to be cleaned up on error in wormhole
Also, set new print statment to print to args.stdout
2016-12-16 01:03:01 -08:00
laharah
f786031f40 basic key established hook added to CLI send 2016-12-16 01:03:01 -08:00
laharah
680b01b3fc added api hooks to get notification about key excange
added more coverage to establish key API hook
2016-12-16 01:03:01 -08:00
laharah
33fa6f6ede tests for new establish_key wormhole hook
defered should behave similarly to the `verify` hook
2016-12-16 01:03:01 -08:00
Brian Warner
5c751eb3ed fix tests on windows 2016-12-15 19:52:26 -08:00
Brian Warner
f3e1aab3a1 reject transfers when there isn't enough disk space available
closes #91

Also tweaks an error message: don't say "refusing to clobber pre-existing
file FOO" when we don't check that it's actually a file. Just say "..
pre-existing 'FOO'".
2016-12-15 19:52:26 -08:00
Brian Warner
b57928431a add utility to estimate free disk space 2016-12-15 19:52:26 -08:00
Brian Warner
2596c58e4a ipaddrs: import function from latest Tahoe
I'm hoping this will help with Cygwin (#13), by removing/bypassing the
assert(os.path.isabs) check.
2016-12-10 13:06:27 -08:00
Brian Warner
58f20d79a7 update to versioneer-0.17
This should fix the pyflakes error when running tox on an unpacked sdist.
2016-12-10 10:36:59 -08:00
Brian Warner
f0bfcd7ca4 fix some py2/py3-isms picked up by mypy
These happened to work, but it's probably best to get the types right.
2016-12-08 23:47:53 -08:00
Brian Warner
582cebfb5f rx: don't reveal file-already-exists, just reject transfer
closes #74
2016-12-08 16:59:54 -08:00
Brian Warner
ab972785aa CLI: don't emit traceback for TransferErrors
But do print the message contained therein

refs #74
2016-12-08 16:59:52 -08:00
Brian Warner
58c7639635 CLI: exit with rc=1 upon errors 2016-12-08 16:59:02 -08:00
Antoine Beaupré
e9cd5b5d60 fix capitalization output to match humanize 2016-11-17 11:36:00 -05:00
Antoine Beaupré
342bebbd0e use humanize library instead of custom implementation 2016-11-11 22:01:21 -05:00
Antoine Beaupré
047af4b27d use human-readable sizes more broadly
there was a function to "abbreviate" sizes, but it was somewhat
unclear and incomplete. reuse the sizeof_fmt_* set of functions from
the borg backup project (MIT licensed) to implement a more complete
and flexible display that will scale up to the Yottabyte and
beyond. it also supports non-IEC units (like "kibibyte", AKA 1024
bytes) if you fancy that stuff.

this is a workaround for #91: it allows users to better see the size
of the file that will be transfered.

*some* places are still kept in bytes, most notably when receive fails
to receive all bytes ("got %d bytes, wanted %d") because we may want
more clarity there.

text transfers also use the "bytes" suffix (instead of "B") because it
will commonly not reach beyond the KiB range.

note that the test suite only covers decimal (non-IEC) prefix, but it
is assumed to be sufficient to be considered correct.
2016-11-09 15:14:01 -05:00
Brian Warner
0004315431 transit: tolerate non-ascii bad handshake
I think somebody was port-scanning the server (or pointed some
non-wormhole client at it), and caused some exceptions in the logs.
These are still bad handshakes, but should be logged normally instead of
throwing exceptions.
2016-08-22 23:13:04 -07:00
Tom Lowenthal
9dd1d738e1 Fix small CLI typo
The output for verified copies asks the user to run `wormhole --verify receive`, but the correct argument order is `wormhole receive --verify`.
2016-08-15 17:39:52 -07:00
Brian Warner
f449466f4f update tests 2016-08-15 17:36:55 -07:00
Brian Warner
899e539cb1 cmd_ssh: ~ is not always /home
On OS-X, in particular, ~foo expands to /Users/foo . We delegate
everything to os.path.expanduser(), which should know the local
convention.
2016-08-15 17:35:38 -07:00
Brian Warner
75d362f60a add some basic tests, doesn't cover everything 2016-08-15 17:35:34 -07:00
Brian Warner
d057b91371 xfer_util: work on py2+py3
wormhole.send takes bytes, but the utility functions take strings. So
encode the JSON blob before sending, and decode it on the way back out.
2016-08-15 17:35:34 -07:00
meejah
fe29b3130b 'wormhole ssh' cleanups
- move to 'wormhole ssh' group with accept/invite subcommands
- change names of methods
- check for permissions
- use --user option (instead of --auth-file)
- move implementation to cmd_ssh.py
- if multiple public-keys, ask user
2016-08-15 00:30:39 -06:00
meejah
069b76485b Add 'wormhole ssh-add' and 'wormhole ssh-send' commands 2016-08-14 19:59:20 -06:00
meejah
026c8fd093 Print proper tracebacks when inlineCallbacks + yield involved 2016-08-14 16:50:29 -06:00
meejah
afa123abae make tests work on pypy 2016-08-04 15:57:01 -04:00
David Reid
6aa8a47f9a Use StreamServerEndpointService (available since Twisted 10.2) 2016-08-04 00:23:02 -07:00
Brian Warner
dc6416a257 fix stats-writing bug on py3
refs #67
2016-08-01 16:31:05 -07:00
Daniel Kahn Gillmor
2fafdf34ee help bad spelers
Some of us can never remember the old ditty:

   i before e, except after c
   or when sounding like "a"
   as in neighbor or weigh.

Perhaps magic wormhole can coddle us in our misorthography :)
2016-07-31 18:55:42 -04:00
Brian Warner
0ba59136a0 upgrade to versioneer-0.16 2016-07-27 19:08:12 -07:00
Brian Warner
5542545165 CLI: move most top-level args down into the subcommand
So instead of "wormhole --verify send", use "wormhole send --verify".

The full set of arguments that were moved down:

* --code-length=
* --verify
* --hide-progress
* --no-listen
* --tor

The following remain as top-level arguments (which should appear after
"wormhole" and before the subcommand):

* --relay-url=
* --transit-helper=
* --dump-timing=
* --version
2016-07-27 17:56:03 -07:00
Brian Warner
e6b38ecacc cli: start factoring out args into common sets 2016-07-23 17:54:29 -07:00
Brian Warner
37b8ccbe4f cli: mostly use kwargs/setattr 2016-07-22 19:13:59 -07:00
Brian Warner
1a82846578 cli.Config: remove common attributes
The values set by the base Config constructor could mask Click parsers
that weren't supplying defaults properly, or which were using different
defaults.
2016-07-14 22:37:35 -06:00
Brian Warner
52ef00b46b CLI: refactor to make testing easier
When tests need a Config object, they now call a function which invokes
Click with a mocked-out go() function, and grabs the Config object
before actually doing anything with it.
2016-07-14 22:34:10 -06:00
Brian Warner
cdb5c19010 websockets: turn on autoPingInterval to expire stale connections
With this, both clients and servers will send a PING at least once every
minute, and will drop connections that haven't seen any traffic for 10
minutes.

This should help keep NAT table entries alive, and will drop connections
that are no longer viable because their NAT entries have expired.

closes #60
2016-07-03 21:51:56 -07:00
Brian Warner
4978be6b90 args: fix --no-listen, --tor
Also add tests to check that argv is being parsed properly, and to check
the defaults.

fixes #59
2016-06-27 14:40:51 -07:00
Brian Warner
0f8b98dec5 DB: log schema upgrades 2016-06-26 18:04:00 -07:00
Brian Warner
de8efdb9f9 cmd_send: wait for ack from close()
Without this, the sender drops the connection before the "close" message
has made it to the server, which leaves the mailbox hanging until it
expires. It still lives in a 'd.addBoth()' slot, so it gets closed even
if some error occurrs, but we wait for it's Deferred to fire in both
success and failure cases.
2016-06-26 17:55:28 -07:00
Brian Warner
1d3b22a0b4 server: hard-code --stats-file=stats.json
We already hard-code 'relay.sqlite', so I don't see a lot of value in
making making the stats file configurable too. That said, if it makes
life easier for packagers (e.g. start-stop-daemon or systemd wanting
these files to go into /var/run/something/ , and if it isn't sufficient
to just use /var/run/something/ as the CWD), I'd accept a patch to
add it back.
2016-06-26 17:55:28 -07:00
Brian Warner
930ed2c089 server: log get_stats time
The DB queries this uses aren't particularly efficient, and when the
time it takes to run starts to become a problem, we should do an
optimization pass.
2016-06-26 17:49:36 -07:00
Brian Warner
45d46643ca transit: rename stats keys 2016-06-26 17:49:36 -07:00
Brian Warner
f950161642 rendezvous: rename stats keys for consistency 2016-06-26 17:49:36 -07:00
Brian Warner
638adc71ec server: add transit to stats.json 2016-06-26 17:49:35 -07:00
Brian Warner
cd934fdfde server: rename stats.json keys 2016-06-26 17:49:35 -07:00
Brian Warner
44bc99e987 get_stats: update comments 2016-06-26 11:25:25 -07:00
Brian Warner
f192db5412 track+report mailbox.for_nameplate in usage
This counts the number of "standalone" mailboxes we create, which
happens when a client does open() without first using a nameplate. The
current client doesn't do this, but future clients might.
2016-06-26 11:19:53 -07:00
Brian Warner
db8511c700 schema: add mailbox_usage.for_nameplate too 2016-06-26 11:18:48 -07:00
Brian Warner
bae020d5a7 DB: add mailboxes.for_nameplate
Also add a few indexes to make get_stats() less expensive
2016-06-26 11:01:52 -07:00
Brian Warner
8eb1396f31 server: delete stats file at reboot
This makes it easier to discover an exception in the stats-gathering
timer, since you get a missing stats.json instead of a stale one.
2016-06-26 11:01:52 -07:00
Brian Warner
7b1d4cbcee more stats: count since last reboot 2016-06-26 11:01:52 -07:00
Brian Warner
9e29898207 Rendezvous: add many stats 2016-06-26 11:01:48 -07:00
Brian Warner
62f9a4d702 move stats-file up to RelayServer
This moves responsibility for the periodic prune-everything Timer up to
RelayServer too. That way we can be sure the stats are dumped
immediately after prune, and we can incorporate stats from Transit as
well.
2016-06-25 11:43:13 -07:00
Brian Warner
7f389dc76e server: add --stats-file= scaffolding
no actual stats yet
2016-06-25 11:05:28 -07:00
Brian Warner
cfcd74db4b remove listener on close too
This was failing to clean up after WrongPassword disconnects.
2016-06-24 18:57:49 -07:00
Brian Warner
6a2cbf9014 server: remove listeners on disconnect
This wasn't happening before, so channels were staying alive until
reboot.
2016-06-24 18:48:28 -07:00
Brian Warner
ffb1a9b9c9 change pruning algorithm
The new approach runs every 10 minutes and keeps a
nameplate/mailbox/messages "channel" alive if the mailbox has been
updated within 11 minutes, or if there has been an attached listener
within that time.

Also remove the "nameplates.updated" column. Now we only track "updated"
timestamps on the "mailboxes" table, and a new mailbox will preserve any
attached nameplate.
2016-06-24 17:35:23 -07:00
Brian Warner
6176500cf4 factor out Mailbox._touch 2016-06-24 16:35:30 -07:00
Brian Warner
0158df9b63 stop using is_active(), or pruning inactive apps
Unless/until people start writing new applications (with different
app-ids), this code is unlikely to get used very much, and the code is
simpler without it.
2016-06-24 16:31:00 -07:00
Brian Warner
08443ee288 rename Rendezvous.prune() to prune_all_apps
so grep can distinguish it from Mailbox.prune()
2016-06-24 16:30:55 -07:00
Brian Warner
b8521e81b1 add comment about new pruning plan 2016-06-24 16:26:42 -07:00
Brian Warner
3c0f832bdc test that mailbox timestamp is updated correctly 2016-06-24 16:25:44 -07:00
Brian Warner
550b9e3c94 db: add v2->v3 upgrader 2016-06-24 16:19:39 -07:00
Brian Warner
404925d314 server: mailbox row should always exist
This makes the nameplate's "mailbox_id" into a foreign-key.
2016-06-24 16:02:37 -07:00
Brian Warner
41f229de87 use 'mailbox_sides' instead of cols in 'mailboxes' 2016-06-24 00:01:51 -07:00
Brian Warner
6c725e4a86 db: use 'nameplate_sides' instead of cols in 'nameplates' 2016-06-23 18:27:26 -07:00
Brian Warner
355ece3e47 db: turn on foreign keys 2016-06-23 18:27:26 -07:00
Brian Warner
74a1902fcd schema: add v3, rename nameplates.id to .name 2016-06-23 18:27:26 -07:00
Brian Warner
6d00576ee4 server-stop: ignore missing twistd.pid
I changed my mind, it's actually easier if 'wormhole-server stop' (and
'restart') does *not* throw an error when there wasn't already a server
running in that directory. Specifically that lets me use 'restart' as an
idempotent "make sure a server is running" command.
2016-06-23 18:25:58 -07:00
Brian Warner
fa29789769 test_scripts: re-enable tests on py3
These weren't running because Click complained about an ASCII locale
when running under py3, which triggered an error check that was there to
detect broken virtualenvs, skipping those tests.

The fix appears to be to force the en_US.UTF-8 locale when running the
wormhole program in a subprocess.
2016-06-22 19:28:17 -07:00
Brian Warner
177aac2d84 test upgrader, badly
This adds a test for database upgrades, which I developed on a branch
that added a new DB schema (v3) and an upgrader to match, but then I
changed my mind about the schema and removed that part. The test will be
useful some time in the future when I change the schema in a small
enough way that I bother to write an upgrader for the change. For now,
the test is disabled.

In addition, the upgrader test is kind of lame. I'd really prefer to
assert that the upgraded schema is identical to the schema of a
brand-new (latest-version) database, but ALTER TABLE doesn't quite work
that way (comments are omitted, and the order of the columns is slightly
different).

This also adds database.dump_db() for the tests.
2016-06-22 19:05:52 -07:00
Brian Warner
7250deb94c add schema upgrades 2016-06-22 19:05:48 -07:00
Brian Warner
b613ac1b3b cli.py: add comment 2016-06-22 13:15:07 -07:00
Brian Warner
60b40c2de2 server/cli.py: remove timing instrumentation
I don't anticipate ever caring how long it takes to import code on the
server side. Only the client's performance is interesting to me.
2016-06-22 13:12:45 -07:00
Brian Warner
d8c2c1249e server/cli.py: fix port numbers
There was some vestigal server-cli code (leftover in the client-side
wormhole.cli.cli_args) that used port 3000/3001, and it accidentally got
used for the new Click-based parser, rather than the actual server-cli
code (in wormhole.server.cli_args) that uses port 4000/4001. This
changes the port numbers to match (everything uses 4000/4001 these days,
to avoid confusing interactions with the old 0.7.6 server that might
still be listening on the old ports).
2016-06-22 13:12:34 -07:00
Brian Warner
940aec7570 cli.py: improve parallelism between send+receive 2016-06-22 13:11:43 -07:00
Brian Warner
b1f994e939 transit.py: improve error 2016-06-22 13:10:46 -07:00
meejah
7fab6b3dff Change UsageError -> InternalError, use click.UsageError for human-visible errors 2016-06-22 02:14:34 -06:00
meejah
6b29d62d78 typo 2016-06-22 01:29:13 -06:00
meejah
03594a2d0e consistent return usage 2016-06-22 01:28:53 -06:00
meejah
ad6200c474 resolve XXX comments 2016-06-22 01:27:29 -06:00
meejah
c4cd4db30e Correct exit code when no twistd.pid file 2016-06-22 01:25:47 -06:00
meejah
e16b53817e Refactor to use Click 2016-06-22 01:11:07 -06:00
Brian Warner
d89fbd69dd sketching out a click-based CLI parser 2016-06-22 01:02:09 -06:00
meejah
07b4067727 unit-test for relay request, and fix Data() handling 2016-06-20 16:32:07 -07:00
laharah
2a056059c0 fix for wormhole-server count-* functions
broken when we switched to dictfactory on db
2016-06-04 16:50:56 -07:00
laharah
150caedf73 u"" cleanup on test_scripts.py 2016-06-04 14:09:19 -07:00
laharah
bb970dafc0 u"" cleanup on test_server.py 2016-06-04 14:09:19 -07:00
laharah
2dd76f57c0 u"" cleanup on test_transit.py 2016-06-04 14:09:19 -07:00
laharah
aff13e9121 u"" cleanup on test_util.py 2016-06-04 14:09:19 -07:00
laharah
8fa40d3cd5 u"" cleanup on test_wormhole.py 2016-06-04 14:09:19 -07:00
laharah
255fb12f88 `u"" cleanup on cmd_usage.py 2016-06-04 14:09:19 -07:00