Noah Levitt 4b505c524b new flag dedup_ok and warcprox-meta field dedup-ok which can be used to prevent deduplication against particular entries rethinkdb big captures table

2017-01-13 17:29:05 -08:00

..

__init__.py

idn support, at least for domain limits (getting a segfault in tests on mac however, let's see what happens on travis-ci)

2016-06-29 15:54:40 -05:00

bigtable.py

new flag dedup_ok and warcprox-meta field dedup-ok which can be used to prevent deduplication against particular entries rethinkdb big captures table

2017-01-13 17:29:05 -08:00

controller.py

check if already started/stopped in WarcproxController.{start,shutdown}, fix bugs

2016-06-27 14:36:06 -05:00

dedup.py

add a start() method to the two classes that save data to rethinkdb periodically in batches, instead of starting the timer in __init__

2016-06-16 00:04:59 +00:00

dump_anydbm.py

convert command-line executables to entry_points console_scripts, best practice according to Python Packaging Authority (eases testing, etc)

2016-06-27 14:46:42 -05:00

kafkafeed.py

keep trying to connect to kafka and don't let connection failure interfere with other warcprox operations

2016-09-07 13:43:01 -07:00

main.py

add --method-filter option

2016-11-15 23:26:13 +11:00

mitmproxy.py

handle case of unlimited resource limits and cap max_threads at 5000

2016-10-20 17:31:52 -07:00

playback.py

fix renamed overridden method name in subclass

2016-05-10 17:55:18 +00:00

stats.py

switching from host limits to domain limits, which apply in aggregate to the host and subdomains

2016-06-29 14:56:14 -05:00

warc.py

add license headers

2016-04-06 19:37:55 -07:00

warcproxy.py

refactor some general mitm proxy stuff into mitmproxy.py

2016-10-19 15:32:58 -07:00

writer.py

add license headers

2016-04-06 19:37:55 -07:00

writerthread.py

two different measures of size in the big captures table, record_length and wire_bytes

2016-11-21 15:17:50 -08:00