107 Commits

Author	SHA1	Message	Date
Noah Levitt	42a81d8f8f	fix bug where two warc-payload-digest headers were written to revisit records	2016-03-15 06:27:21 +00:00
Noah Levitt	4bb7e043d4	wait longer for stats to be updated in test_limits(), now that rethinkdb stats are pushed only every 2.0 seconds	2016-01-26 18:47:08 -08:00
Noah Levitt	00dc9eed84	new option --onion-tor-socks-proxy, host:port of tor socks proxy, used only to connect to .onion sites	2016-01-26 18:47:08 -08:00
Noah Levitt	18cc818cf0	more timing tweaks to make sure tests pass, improved logging etc	2016-01-26 18:47:08 -08:00
Noah Levitt	e67c7be5bc	service registry init	2016-01-26 18:47:08 -08:00
Noah Levitt	95e611a5d0	update stats in RethinkDb asynchronously, since profiling shows this to be a bottleneck in WarcWriterThread (which in turn makes it a bottleneck for the whole app)	2016-01-26 18:47:08 -08:00
Noah Levitt	4c380dcc41	move tests out of installed package dir	2016-01-26 18:47:08 -08:00