160 Commits

Author	SHA1	Message	Date
Noah Levitt	4bb3556709	implement enforcement of Warcprox-Meta header block rules; includes automated tests	2016-05-10 23:11:47 +00:00
Noah Levitt	6f10e2708d	disable tor test to give travis build a chance to pass tests (waiting on https://github.com/travis-ci/apt-package-whitelist/issues/1753)	2016-04-06 19:39:28 -07:00
Noah Levitt	2c65ff89fa	add license headers	2016-04-06 19:37:55 -07:00
Noah Levitt	42a81d8f8f	fix bug where two warc-payload-digest headers were written to revisit records	2016-03-15 06:27:21 +00:00
Noah Levitt	4bb7e043d4	wait longer for stats to be updated in test_limits(), now that rethinkdb stats are pushed only every 2.0 seconds	2016-01-26 18:47:08 -08:00
Noah Levitt	00dc9eed84	new option --onion-tor-socks-proxy, host:port of tor socks proxy, used only to connect to .onion sites	2016-01-26 18:47:08 -08:00
Noah Levitt	18cc818cf0	more timing tweaks to make sure tests pass, improved logging etc	2016-01-26 18:47:08 -08:00
Noah Levitt	e67c7be5bc	service registry init	2016-01-26 18:47:08 -08:00
Noah Levitt	95e611a5d0	update stats in RethinkDb asynchronously, since profiling shows this to be a bottleneck in WarcWriterThread (which in turn makes it a bottleneck for the whole app)	2016-01-26 18:47:08 -08:00
Noah Levitt	4c380dcc41	move tests out of installed package dir	2016-01-26 18:47:08 -08:00