Noah Levitt 0031091d4f

Merge pull request #99 from vbanos/blackout_period ...

New --blackout-period option to skip writing redundant revisits to WARC

2018-08-03 17:27:42 -07:00

..

__init__.py

some logging improvements

2018-07-18 19:25:43 -05:00

bigtable.py

we want to save all captures to the big "captures"

2018-05-15 15:33:52 -07:00

controller.py

default to 100 proxy threads, 1 warc writer thread

2018-04-12 12:31:04 -07:00

crawl_log.py

record request method in crawl log if not GET

2018-07-17 13:47:52 -05:00

dedup.py

dumb mistake

2018-07-18 20:10:30 -05:00

main.py

Merge pull request #99 from vbanos/blackout_period

2018-08-03 17:27:42 -07:00

mitmproxy.py

default to 100 proxy threads, 1 warc writer thread

2018-04-12 12:31:04 -07:00

playback.py

tests are passing

2018-01-15 14:49:13 -08:00

stats.py

add another "wait" to fix failing test

2018-05-29 13:08:34 -07:00

trough.py

make sure to send utf-8 to trough

2018-02-28 11:34:47 -08:00

warc.py

Add Warc-Truncated: lenght header

2018-02-10 21:30:56 +00:00

warcproxy.py

fix bug in limits enforcement

2018-05-29 12:18:51 -07:00

writer.py

some logging improvements

2018-07-18 19:25:43 -05:00

writerthread.py

Merge pull request #99 from vbanos/blackout_period

2018-08-03 17:27:42 -07:00