Barbara Miller b67f1ad0f3 add logging

2021-12-06 17:29:27 -08:00

..

__init__.py

increase trough dedup batch window

2021-12-06 17:29:02 -08:00

bigtable.py

WIP (untested): handle multiple dedup-buckets, rw or ro

2019-05-30 19:27:46 -07:00

controller.py

fix logging config which trough interfered with

2020-01-07 15:19:03 -08:00

crawl_log.py

Checking for content type header consiting of only empty spaces and removing spaces from exception messages in json section

2021-03-29 22:22:19 +00:00

dedup.py

add logging

2021-12-06 17:29:27 -08:00

main.py

Add port to custom WARC filename vars

2020-01-08 21:19:48 +00:00

mitmproxy.py

Expanding logging to handle DNS failures, print error message to crawl log info, and report cached connection errors.

2020-07-22 21:36:39 +00:00

playback.py

Add bad_hostnames_ports in PlaybackProxy

2019-05-09 15:57:01 +00:00

stats.py

Moving more variables from RecordedUrl to RequiredUrl

2020-01-04 01:41:28 +00:00

warc.py

do not write incorrect warc-payload-digest to...

2019-05-02 14:25:29 -07:00

warcproxy.py

set failed_url.timestamp

2020-09-30 11:47:17 -07:00

writer.py

Add port to custom WARC filename vars

2020-01-08 21:19:48 +00:00

writerthread.py

Moving more variables from RecordedUrl to RequiredUrl

2020-01-04 01:41:28 +00:00