Noah Levitt b762d6468b just one should_dedup() for trough dedup ...

fixes failing test and clarifies things

2018-05-16 14:25:01 -07:00

..

__init__.py

cap the number of urls queued for warc writing

2018-04-11 22:29:50 +00:00

bigtable.py

we want to save all captures to the big "captures"

2018-05-15 15:33:52 -07:00

controller.py

default to 100 proxy threads, 1 warc writer thread

2018-04-12 12:31:04 -07:00

crawl_log.py

fix payload digest by pulling calculation up one level where content has already been transfer-decoded

2017-11-10 17:18:22 -08:00

dedup.py

just one should_dedup() for trough dedup

2018-05-16 14:25:01 -07:00

main.py

Add hidden CLI option --dedup-only-with-bucket

2018-05-04 20:50:54 +00:00

mitmproxy.py

default to 100 proxy threads, 1 warc writer thread

2018-04-12 12:31:04 -07:00

playback.py

tests are passing

2018-01-15 14:49:13 -08:00

stats.py

use batch postfetch processor for stats

2018-01-17 14:58:52 -08:00

trough.py

make sure to send utf-8 to trough

2018-02-28 11:34:47 -08:00

warc.py

Add Warc-Truncated: lenght header

2018-02-10 21:30:56 +00:00

warcproxy.py

support "captures-bucket" for backward compatibility

2018-05-09 15:43:39 -07:00

writer.py

more logging

2018-04-04 15:36:46 -07:00

writerthread.py

default to 100 proxy threads, 1 warc writer thread

2018-04-12 12:31:04 -07:00