Noah Levitt fd81190517 refactor the multithreaded warc writing ...

main functional change is that only as man warc files are created as are
needed to keep up with the throughput

2018-02-07 15:48:43 -08:00

..

__init__.py

move tests out of installed package dir

2016-01-26 18:47:08 -08:00

conftest.py

change rethinkdb-related command line options to use "rethinkdb urls" (parser just added to doublethink) to reduce the proliferation of rethinkdb options, and add --rethinkdb-trough-db-url option

2017-10-11 12:06:19 -07:00

Dockerfile

get all the tests to pass with ./tests/run-tests.sh

2017-10-13 15:54:05 -07:00

run-tests.sh

Merge branch 'master' into trough-dedup

2017-11-02 16:34:52 -07:00

run-trough.sh

update travis-ci trough deployment

2017-11-03 14:21:39 -07:00

single-threaded-proxy.py

fix payload digest by pulling calculation up one level where content has already been transfer-decoded

2017-11-10 17:18:22 -08:00

test_dedup.py

Update CdxServerDedup unit test

2018-01-15 20:58:36 +00:00

test_ensure_rethinkdb_tables.py

fix warcprox-ensure-rethinkdb-tables and add tests

2017-11-28 10:38:38 -08:00

test_queue.py

add seconds_behind to service registry and status api, which is the length of time the next url to be written to warc has been waiting in the queue

2017-03-30 15:54:19 -07:00

test_warcprox.py

refactor the multithreaded warc writing

2018-02-07 15:48:43 -08:00

test_writer.py

refactor the multithreaded warc writing

2018-02-07 15:48:43 -08:00