replace --log-stdout with --log-fd=

This commit is contained in:
Brian Warner 2017-11-07 20:35:29 -06:00
parent ff48518e37
commit 65b2192e89
2 changed files with 24 additions and 14 deletions

View File

@ -12,12 +12,14 @@ timestamps or exact transfer sizes. The ``--blur-usage=`` option enables
this, and it takes an integer value (in seconds) to specify the desired time
window.
## Logging JSON to stdout
## Logging JSON Upon Each Connection
If --log-stdout is provided, a line will be written to stdout after each
connection is done. This line will be a complete JSON object (starting with
``{``, ending with ``}\n``, and containing no internal newlines). The keys
will be:
If --log-fd is provided, a line will be written to the given (numeric) file
descriptor after each connection is done. These events could be delivered to
a comprehensive logging system like XXX for offline analysis.
Each line will be a complete JSON object (starting with ``{``, ending with
``}\n``, and containing no internal newlines). The keys will be:
* ``started``: number, seconds since epoch
* ``total_time``: number, seconds from open to last close
@ -30,7 +32,11 @@ means a second matching side never appeared (and thus ``waiting_time`` will
be null). ``errory`` means the first side gave an invalid handshake.
If --blur-usage= is provided, then ``started`` will be rounded to the given
time interval, and ``total_bytes`` will be rounded as well.
time interval, and ``total_bytes`` will be rounded to a fixed set of buckets:
* file sizes less than 1MB: rounded to the next largest multiple of 10kB
* less than 1GB: multiple of 1MB
* 1GB or larger: multiple of 100MB
## Usage Database
@ -77,5 +83,9 @@ the ``current`` table will be updated at least once every 5 minutes.
If daemonized by twistd, the server will write ``twistd.pid`` and
``twistd.log`` files as usual. By default ``twistd.log`` will only contain
startup, shutdown, and exception messages. Adding --log-stdout will add
per-connection JSON lines to ``twistd.log``.
startup, shutdown, and exception messages.
Setting ``--log-fd=1`` (file descriptor 1 is always stdout) will cause the
per-connection JSON lines to be interleaved with any messages sent to
Twisted's logging system. It may be better to use a different file
descriptor.

View File

@ -1,4 +1,4 @@
import sys
import os
from . import transit_server
from twisted.internet import reactor
from twisted.python import usage
@ -14,15 +14,13 @@ glues the two TCP sockets together.
"""
class Options(usage.Options):
synopsis = "[--port=] [--log-stdout] [--blur-usage=] [--usage-db=]"
synopsis = "[--port=] [--log-fd] [--blur-usage=] [--usage-db=]"
longdesc = LONGDESC
optFlags = {
("log-stdout", None, "write JSON usage logs to stdout"),
}
optParameters = [
("port", "p", "tcp:4001", "endpoint to listen on"),
("blur-usage", None, None, "blur timestamps and data sizes in logs"),
("log-fd", None, None, "write JSON usage logs to this file descriptor"),
("usage-db", None, None, "record usage data (SQLite)"),
]
@ -32,7 +30,9 @@ class Options(usage.Options):
def makeService(config, reactor=reactor):
ep = endpoints.serverFromString(reactor, config["port"]) # to listen
log_file = sys.stdout if config["log-stdout"] else None
log_file = (os.fdopen(int(config["log-fd"]), "w")
if config["log-fd"] is not None
else None)
f = transit_server.Transit(blur_usage=config["blur-usage"],
log_file=log_file,
usage_db=config["usage-db"])