backup/warcprox
1
0
Fork 0
mirror of https://github.com/internetarchive/warcprox.git synced 2025-01-18 13:22:09 +01:00
Code Issues Projects Releases Activity

handle case where warc record id is missing

Browse Source 
... from trough dedup. Not sure why this error happened but we shouldn't
need that field anyway.
This commit is contained in:
Noah Levitt 2018-01-19 14:38:54 -08:00
parent 4b53c10132
commit 57abab100c
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
warcprox/dedup.py
View File

@ -419,7 +419,7 @@ class TroughDedupDb(DedupDb):
len(digest_keys), len(results))
assert len(results) >= 0 and len(results) <= len(digest_keys)
for result in results:
result['id'] = result['id'].encode('ascii')
result['id'] = result.get('id') and result['id'].encode('ascii')
result['url'] = result['url'].encode('ascii')
result['date'] = result['date'].encode('ascii')
result['digest_key'] = result['digest_key'].encode('ascii')