backup/warcprox
1
0
Fork 0
mirror of https://github.com/internetarchive/warcprox.git synced 2025-01-18 13:22:09 +01:00
Code Issues Projects Releases Activity

Merge branch 'dedup-fixes' into qa

Browse Source
This commit is contained in:
Barbara Miller 2019-06-20 14:55:45 -07:00
commit fa7d5e9326
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
README.rst
View File

@ -90,7 +90,7 @@ for deduplication works similarly to deduplication by `Heritrix
a. Write ``response`` record with full payload
b. Store new entry in deduplication database (can be disabled, see
`Warcprox-Meta HTTP request header <api.rst#warcprox-meta-http-request-header>`_
`Warcprox-Meta HTTP request header <api.rst#warcprox-meta-http-request-header>`_)
The deduplication database is partitioned into different "buckets". URLs are
deduplicated only against other captures in the same bucket. If specified, the