Commit Graph

720 Commits

Author SHA1 Message Date
Brian Warner
31be12a4be misc/dump-stats.py: tool to show stats.json validity
Run like 'watch python misc/dump-stats.py' against a 'wormhole-server
start --stats-file=stats.json'
2016-06-26 11:31:54 -07:00
Brian Warner
44bc99e987 get_stats: update comments 2016-06-26 11:25:25 -07:00
Brian Warner
f192db5412 track+report mailbox.for_nameplate in usage
This counts the number of "standalone" mailboxes we create, which
happens when a client does open() without first using a nameplate. The
current client doesn't do this, but future clients might.
2016-06-26 11:19:53 -07:00
Brian Warner
db8511c700 schema: add mailbox_usage.for_nameplate too 2016-06-26 11:18:48 -07:00
Brian Warner
bae020d5a7 DB: add mailboxes.for_nameplate
Also add a few indexes to make get_stats() less expensive
2016-06-26 11:01:52 -07:00
Brian Warner
8eb1396f31 server: delete stats file at reboot
This makes it easier to discover an exception in the stats-gathering
timer, since you get a missing stats.json instead of a stale one.
2016-06-26 11:01:52 -07:00
Brian Warner
7b1d4cbcee more stats: count since last reboot 2016-06-26 11:01:52 -07:00
Brian Warner
9e29898207 Rendezvous: add many stats 2016-06-26 11:01:48 -07:00
Brian Warner
62f9a4d702 move stats-file up to RelayServer
This moves responsibility for the periodic prune-everything Timer up to
RelayServer too. That way we can be sure the stats are dumped
immediately after prune, and we can incorporate stats from Transit as
well.
2016-06-25 11:43:13 -07:00
Brian Warner
7f389dc76e server: add --stats-file= scaffolding
no actual stats yet
2016-06-25 11:05:28 -07:00
Brian Warner
cfcd74db4b remove listener on close too
This was failing to clean up after WrongPassword disconnects.
2016-06-24 18:57:49 -07:00
Brian Warner
6a2cbf9014 server: remove listeners on disconnect
This wasn't happening before, so channels were staying alive until
reboot.
2016-06-24 18:48:28 -07:00
Brian Warner
ffb1a9b9c9 change pruning algorithm
The new approach runs every 10 minutes and keeps a
nameplate/mailbox/messages "channel" alive if the mailbox has been
updated within 11 minutes, or if there has been an attached listener
within that time.

Also remove the "nameplates.updated" column. Now we only track "updated"
timestamps on the "mailboxes" table, and a new mailbox will preserve any
attached nameplate.
2016-06-24 17:35:23 -07:00
Brian Warner
6176500cf4 factor out Mailbox._touch 2016-06-24 16:35:30 -07:00
Brian Warner
0158df9b63 stop using is_active(), or pruning inactive apps
Unless/until people start writing new applications (with different
app-ids), this code is unlikely to get used very much, and the code is
simpler without it.
2016-06-24 16:31:00 -07:00
Brian Warner
08443ee288 rename Rendezvous.prune() to prune_all_apps
so grep can distinguish it from Mailbox.prune()
2016-06-24 16:30:55 -07:00
Brian Warner
b8521e81b1 add comment about new pruning plan 2016-06-24 16:26:42 -07:00
Brian Warner
3c0f832bdc test that mailbox timestamp is updated correctly 2016-06-24 16:25:44 -07:00
Brian Warner
550b9e3c94 db: add v2->v3 upgrader 2016-06-24 16:19:39 -07:00
Brian Warner
404925d314 server: mailbox row should always exist
This makes the nameplate's "mailbox_id" into a foreign-key.
2016-06-24 16:02:37 -07:00
Brian Warner
41f229de87 use 'mailbox_sides' instead of cols in 'mailboxes' 2016-06-24 00:01:51 -07:00
Brian Warner
6c725e4a86 db: use 'nameplate_sides' instead of cols in 'nameplates' 2016-06-23 18:27:26 -07:00
Brian Warner
355ece3e47 db: turn on foreign keys 2016-06-23 18:27:26 -07:00
Brian Warner
74a1902fcd schema: add v3, rename nameplates.id to .name 2016-06-23 18:27:26 -07:00
Brian Warner
6d00576ee4 server-stop: ignore missing twistd.pid
I changed my mind, it's actually easier if 'wormhole-server stop' (and
'restart') does *not* throw an error when there wasn't already a server
running in that directory. Specifically that lets me use 'restart' as an
idempotent "make sure a server is running" command.
2016-06-23 18:25:58 -07:00
Brian Warner
f0c32617d0 README: mention libsodium-dev package
pynacl-1.0.1 has a bug, on systems which have the sodium runtime library
installed (/usr/lib/libsodium.so, as provided by a package like
"libsodium13"), but not the development headers (/usr/include/sodium.h,
as provided by "libsodium-dev"). The pynacl setup.py tries to detect
whether a system libsodium can be used (instead of the bundled copy), by
using cffi to load libsodium.so and check the version strings. However
this test doesn't check that sodium.h is available, which is needed to
build the glue code.

The long-term fix is to change pynacl, either to improve the test (to
look for a header file, which sounds tricky), or to always use the
bundled sodium unless explicitly told otherwise (removing the test
altogether). The short term fix is to tell magic-wormhole builders to
install libsodium-dev if they run into this error, or use the
environment-variable override when installing.

closes #10

refs https://github.com/pyca/pynacl/issues/184
2016-06-23 11:31:24 -07:00
Brian Warner
fa29789769 test_scripts: re-enable tests on py3
These weren't running because Click complained about an ASCII locale
when running under py3, which triggered an error check that was there to
detect broken virtualenvs, skipping those tests.

The fix appears to be to force the en_US.UTF-8 locale when running the
wormhole program in a subprocess.
2016-06-22 19:28:17 -07:00
Brian Warner
177aac2d84 test upgrader, badly
This adds a test for database upgrades, which I developed on a branch
that added a new DB schema (v3) and an upgrader to match, but then I
changed my mind about the schema and removed that part. The test will be
useful some time in the future when I change the schema in a small
enough way that I bother to write an upgrader for the change. For now,
the test is disabled.

In addition, the upgrader test is kind of lame. I'd really prefer to
assert that the upgraded schema is identical to the schema of a
brand-new (latest-version) database, but ALTER TABLE doesn't quite work
that way (comments are omitted, and the order of the columns is slightly
different).

This also adds database.dump_db() for the tests.
2016-06-22 19:05:52 -07:00
Brian Warner
7250deb94c add schema upgrades 2016-06-22 19:05:48 -07:00
Brian Warner
38ebc0d8a4 Merge branch 'pr47' 2016-06-22 13:23:35 -07:00
Brian Warner
b613ac1b3b cli.py: add comment 2016-06-22 13:15:07 -07:00
Brian Warner
60b40c2de2 server/cli.py: remove timing instrumentation
I don't anticipate ever caring how long it takes to import code on the
server side. Only the client's performance is interesting to me.
2016-06-22 13:12:45 -07:00
Brian Warner
d8c2c1249e server/cli.py: fix port numbers
There was some vestigal server-cli code (leftover in the client-side
wormhole.cli.cli_args) that used port 3000/3001, and it accidentally got
used for the new Click-based parser, rather than the actual server-cli
code (in wormhole.server.cli_args) that uses port 4000/4001. This
changes the port numbers to match (everything uses 4000/4001 these days,
to avoid confusing interactions with the old 0.7.6 server that might
still be listening on the old ports).
2016-06-22 13:12:34 -07:00
Brian Warner
940aec7570 cli.py: improve parallelism between send+receive 2016-06-22 13:11:43 -07:00
Brian Warner
b1f994e939 transit.py: improve error 2016-06-22 13:10:46 -07:00
meejah
e11c7496a9 add 'dev' extra, with mock + tox, and a pypy environment 2016-06-22 02:14:34 -06:00
meejah
7fab6b3dff Change UsageError -> InternalError, use click.UsageError for human-visible errors 2016-06-22 02:14:34 -06:00
meejah
f32cd46e2c remove argparse, whitespace-ify requirements 2016-06-22 01:30:15 -06:00
meejah
6b29d62d78 typo 2016-06-22 01:29:13 -06:00
meejah
03594a2d0e consistent return usage 2016-06-22 01:28:53 -06:00
meejah
ad6200c474 resolve XXX comments 2016-06-22 01:27:29 -06:00
meejah
c4cd4db30e Correct exit code when no twistd.pid file 2016-06-22 01:25:47 -06:00
meejah
e16b53817e Refactor to use Click 2016-06-22 01:11:07 -06:00
Brian Warner
d89fbd69dd sketching out a click-based CLI parser 2016-06-22 01:02:09 -06:00
meejah
07b4067727 unit-test for relay request, and fix Data() handling 2016-06-20 16:32:07 -07:00
Francois Marier
1170f196b3 Add missing install to Debian instructions 2016-06-06 19:01:55 -07:00
laharah
2a056059c0 fix for wormhole-server count-* functions
broken when we switched to dictfactory on db
2016-06-04 16:50:56 -07:00
Brian Warner
7042a01369 Merge 'pr44': remove u"" marker from most files
refs #30
2016-06-04 15:13:21 -07:00
laharah
150caedf73 u"" cleanup on test_scripts.py 2016-06-04 14:09:19 -07:00
laharah
bb970dafc0 u"" cleanup on test_server.py 2016-06-04 14:09:19 -07:00