Adam Miller
|
caf7f3b30f
|
Merge branch 'qa' of github.com:internetarchive/warcprox into qa
|
2022-03-24 21:41:19 +00:00 |
|
Adam Miller
|
28bec1bb14
|
Merge branch 'adds-hop-path-logging' into qa
|
2022-03-24 21:41:11 +00:00 |
|
Adam Miller
|
5ae1291e37
|
Refactor of hop path referer logic
|
2022-03-24 21:40:55 +00:00 |
|
Barbara Miller
|
a614df69fd
|
Merge branch 'qa' of github.com:internetarchive/warcprox into qa
|
2022-03-03 18:49:24 -08:00 |
|
Barbara Miller
|
e48a8dda05
|
Merge branch 'increase_batch_sec' into qa
|
2022-03-03 18:47:04 -08:00 |
|
Barbara Miller
|
05daafa19e
|
increase MIN_BATCH_SEC, MAX_BATCH_SEC
|
2022-03-03 18:46:20 -08:00 |
|
Adam Miller
|
c8563b9407
|
Merge branch 'adds-hop-path-logging' into qa
|
2022-03-04 02:02:18 +00:00 |
|
Adam Miller
|
ade2373711
|
Fixing referer on request with null hop path
|
2022-03-04 02:01:55 +00:00 |
|
Adam Miller
|
60bd2ea2bd
|
Merge branch 'adds-hop-path-logging' into qa
|
2022-03-03 00:19:00 +00:00 |
|
Adam Miller
|
3a234d0cec
|
Refactor hop_path metadata
|
2022-03-03 00:18:16 +00:00 |
|
Adam Miller
|
dea2d1c8fa
|
Merge pull request #168 from internetarchive/adds-hop-path-logging
Adds hop path logging
|
2022-02-09 10:55:12 -08:00 |
|
Adam Miller
|
366ed5155f
|
Merge branch 'master' into adds-hop-path-logging
|
2022-02-09 18:18:32 +00:00 |
|
Barbara Miller
|
c027659001
|
Merge pull request #167 from galgeek/WT-31
fix logging buglet iii
|
2021-12-29 12:14:56 -08:00 |
|
Barbara Miller
|
6ccd72b8e3
|
Merge branch 'WT-31' into qa
|
2021-12-29 12:06:40 -08:00 |
|
Barbara Miller
|
9e8ea5bb45
|
fix logging buglet iii
|
2021-12-29 12:06:18 -08:00 |
|
Barbara Miller
|
5fd22a0809
|
Merge branch 'WT-31' into qa
|
2021-12-29 11:57:40 -08:00 |
|
Barbara Miller
|
a66a5157c7
|
bump qa version too
|
2021-12-29 11:57:35 -08:00 |
|
Barbara Miller
|
bc3d1e6d00
|
fix logging buglet ii
|
2021-12-29 11:55:39 -08:00 |
|
Barbara Miller
|
6b372e2f3f
|
Merge pull request #166 from galgeek/WT-31
fix logging buglet
|
2021-12-29 11:04:03 -08:00 |
|
Barbara Miller
|
cff2b19745
|
Merge branch 'WT-31' into qa
|
2021-12-29 10:25:30 -08:00 |
|
Barbara Miller
|
5d8fbf7038
|
fix logging buglet
|
2021-12-29 10:25:04 -08:00 |
|
Barbara Miller
|
a969430b37
|
Merge pull request #163 from internetarchive/idna2_10
idna==2.10
|
2021-12-28 13:50:23 -08:00 |
|
Barbara Miller
|
aeecb6515f
|
bump version
|
2021-12-28 11:58:30 -08:00 |
|
Adam Miller
|
e1eddb8fa7
|
Merge pull request #165 from galgeek/WT-31
in-batch dedup
|
2021-12-28 11:52:41 -08:00 |
|
Barbara Miller
|
48f48c34cd
|
Merge branch 'WT-31' into qa
|
2021-12-16 18:45:00 -08:00 |
|
Barbara Miller
|
d7aec77597
|
faster, likely
|
2021-12-16 18:36:00 -08:00 |
|
Barbara Miller
|
6e65b5ff55
|
Merge branch 'WT-31' into qa
|
2021-12-09 12:20:09 -08:00 |
|
Barbara Miller
|
bcaf293081
|
better logging
|
2021-12-09 12:19:45 -08:00 |
|
Barbara Miller
|
1d3e3b3671
|
Merge branch 'WT-31' into qa
|
2021-12-08 11:04:27 -08:00 |
|
Barbara Miller
|
7d4c8dcb4e
|
recorded_url.do_not_archive = True
|
2021-12-08 11:04:09 -08:00 |
|
Barbara Miller
|
69529e5845
|
Merge branch 'WT-31' into qa
|
2021-12-06 20:33:37 -08:00 |
|
Barbara Miller
|
da089e0a92
|
bytes not str
|
2021-12-06 20:33:16 -08:00 |
|
Barbara Miller
|
2ceb0f69f1
|
Merge branch 'WT-31' into qa
|
2021-12-06 19:43:37 -08:00 |
|
Barbara Miller
|
3eeccd0016
|
more hash_plus_url
|
2021-12-06 19:43:27 -08:00 |
|
Barbara Miller
|
a8944ddea3
|
Merge branch 'WT-31' into qa
|
2021-12-06 19:33:32 -08:00 |
|
Barbara Miller
|
5e5a74f204
|
str, not object
|
2021-12-06 19:33:10 -08:00 |
|
Barbara Miller
|
533234162e
|
str, not object
|
2021-12-06 19:32:35 -08:00 |
|
Barbara Miller
|
85bb6ff437
|
Merge branch 'WT-31' into qa
|
2021-12-06 17:30:25 -08:00 |
|
Barbara Miller
|
b67f1ad0f3
|
add logging
|
2021-12-06 17:29:27 -08:00 |
|
Barbara Miller
|
e6a1a7dd7e
|
increase trough dedup batch window
|
2021-12-06 17:29:02 -08:00 |
|
Barbara Miller
|
4e7a4c3eae
|
Merge branch 'WT-31' into qa
|
2021-12-02 11:46:58 -08:00 |
|
Barbara Miller
|
e744075913
|
python 3.5 version, mostly
|
2021-12-02 11:46:39 -08:00 |
|
Barbara Miller
|
16412d64dc
|
Merge branch 'WT-31' into qa
|
2021-12-02 11:18:44 -08:00 |
|
Barbara Miller
|
1476bfec8c
|
discard batch hash+url match
|
2021-12-02 11:17:59 -08:00 |
|
Adam Miller
|
b57ec9c589
|
Check warcprox meta headers for hop information necessary to record a hop path if provided
|
2021-08-31 17:09:06 +00:00 |
|
Barbara Miller
|
bab938f080
|
Merge branch 'idna2_10' into qa
|
2021-04-27 10:28:25 -07:00 |
|
Barbara Miller
|
e61099ff5f
|
idna==2.10
|
2021-04-27 10:26:45 -07:00 |
|
Barbara Miller
|
0e23a31a31
|
Merge pull request #161 from internetarchive/fixes-malformed-crawl-log-lines
Checking for content type header consiting of only empty spaces and r…
|
2021-04-21 15:31:17 -07:00 |
|
Barbara Miller
|
f782f8a985
|
Merge pull request #162 from internetarchive/fixes-malformed-crawl-log-lines
Fixes malformed crawl log lines
|
2021-04-01 12:19:03 -07:00 |
|
Adam Miller
|
7f406b7942
|
Trying to fix tests that only fail during ci
|
2021-04-01 00:01:47 +00:00 |
|