10 Commits

Author	SHA1	Message	Date
Ilya Kreymer	a474335501	fix missing param, typo	2014-02-24 19:42:37 +00:00
Ilya Kreymer	ef062fee7b	cdx: add prototype support for redis cdx source (need testing)	2014-02-24 11:05:48 -08:00
Ilya Kreymer	d8d7435d77	add zipnum location reloading support ... default to 10 min interval #17	2014-02-22 16:49:37 -08:00
Ilya Kreymer	8e840ccaaf	zipnum first version! #17 ... split binsearch further into binsearch and linearsearch components reading blocks one at a time currently, due to zlib decompress limitations fix bufferedreader.readline() and fileloader bugs	2014-02-22 10:50:03 -08:00
Ilya Kreymer	a56cbcf62e	binsearch: add range based matching via iter_range() ... support for: exact, prefix, host, domain match types	2014-02-20 21:21:12 -08:00
Ilya Kreymer	ff428ed43e	exclusions: add AllAllowPerms and refactor exclusions interface ... add TestExclusionPerms and a sample exclusion integration test refactor cdx server init params into **kwargs convert all cdx params to use camelCase	2014-02-19 20:20:31 -08:00
Ilya Kreymer	a09dec4b3e	cdx: add domain-specific rules at cdx layer for custom canonicalization! ... and 'fuzzy' matching when not found handled via cdxdomainspecific.py BaseCDXServer contains a canonicalizer object and a fuzzy query canonicalizer abstracted to seperate class (in canonicalizer.py) clean up cdx related exceptions default rules read from cdx/rules.yaml filename configurable via 'domain_specific_rules' setting in config.yaml fix typo in pywb/rewrite	2014-02-18 14:56:13 -08:00
Ilya Kreymer	28187b34d3	fix typos in remotecdxserver, url-agnostic dedup ... when raising new exception, pass traceback of original also!	2014-02-17 14:52:13 -08:00
Ilya Kreymer	abea504b04	cleanup cdx server config, refactored such that ... a cdx server need implement a single interface: load_cdx(self, **params) CDXServer and RemoteCDXServer distinct classes in cdxserver.py utility function cdxserver.create_cdx_server() to create appropriate server based on input	2014-02-17 13:58:02 -08:00
Ilya Kreymer	5345459298	pywb 0.2! ... move to distinct packages: pywb.utils, pywb.cdx, pywb.warc, pywb.util, pywb.rewrite! each package will have its own README and tests shared sample_data and install	2014-02-17 10:01:09 -08:00