Noah Levitt ca7625b18d set via header on request and response, record request via in warc (because it is sent to the remote site), do not record response via in warc (because it is not sent by the remote site)

2017-04-28 11:07:33 -07:00

..

__init__.py

add seconds_behind to service registry and status api, which is the length of time the next url to be written to warc has been waiting in the queue

2017-03-30 15:54:19 -07:00

bigtable.py

damn it dude get it right

2017-03-15 12:38:38 -07:00

controller.py

test choosing port 0 which means, let the system choose one for me, and fix a bug in service registry reporting of the port

2017-04-17 11:45:37 -07:00

dedup.py

rethinkstuff -> doublethink

2017-03-02 15:06:26 -08:00

dump_anydbm.py

convert command-line executables to entry_points console_scripts, best practice according to Python Packaging Authority (eases testing, etc)

2016-06-27 14:46:42 -05:00

kafkafeed.py

keep trying to connect to kafka and don't let connection failure interfere with other warcprox operations

2016-09-07 13:43:01 -07:00

main.py

make --queue-size and --max-threads hidden options work

2017-04-11 16:29:57 -07:00

mitmproxy.py

set via header on request and response, record request via in warc (because it is sent to the remote site), do not record response via in warc (because it is not sent by the remote site)

2017-04-28 11:07:33 -07:00

playback.py

WARCPROX_WRITE_RECORD is exempt from method filter

2017-02-01 15:30:22 -08:00

stats.py

use urlcanon library for canonicalization, surtification, scope match rules

2017-03-15 09:33:50 -07:00

warc.py

add license headers

2016-04-06 19:37:55 -07:00

warcproxy.py

add another field to status api and service registry, "threads", the size of the proxy server thread pool

2017-03-30 16:18:50 -07:00

writer.py

add license headers

2016-04-06 19:37:55 -07:00

writerthread.py

fix some obsolete calls

2017-04-17 11:00:43 -07:00