Ilya Kreymer
|
63f44cbc26
|
wombat: bump version to 2.11
|
2016-03-20 22:54:27 -07:00 |
|
Ilya Kreymer
|
8302353209
|
wombat: fix typo in window.Crypto check!
|
2016-03-20 22:52:19 -07:00 |
|
Ilya Kreymer
|
4b8bbb7b40
|
wombat: fix typo in window.Crypto check!
|
2016-03-20 22:51:39 -07:00 |
|
Ilya Kreymer
|
ad66f9e11a
|
Merge branch '0.11.4' into develop
|
2016-03-20 22:44:55 -07:00 |
|
Ilya Kreymer
|
c570f76ae4
|
update changelist for 0.11.4
|
2016-03-20 22:43:44 -07:00 |
|
Ilya Kreymer
|
d2390ee809
|
wombat: add check for window.crypto, generate 32-bit values
|
2016-03-20 22:43:05 -07:00 |
|
Ilya Kreymer
|
4b37d13c34
|
tests/gevent: skip gevent monkey patch when running tests
|
2016-03-20 22:23:07 -07:00 |
|
Ilya Kreymer
|
aa0113147a
|
Merge branch '0.11.4' into develop
|
2016-03-20 18:51:23 -07:00 |
|
Ilya Kreymer
|
08cb02c644
|
wombat: override window.crypto.getRandomValues() with seeded random
|
2016-03-20 18:50:45 -07:00 |
|
Ilya Kreymer
|
8e90ea6576
|
gevent/uwsgi config: explicitly monkey patch in pywb.apps.wayback as early monkey patch
not available until uwsgi 2.1 is released!
|
2016-03-20 18:49:59 -07:00 |
|
Ilya Kreymer
|
4cf935abd1
|
directory agg: add CacheDirectoryAggregator to cache file listing, rescan dir only if changed
|
2016-03-19 20:34:09 -07:00 |
|
Ilya Kreymer
|
f5ee3c7bca
|
inputreq: add reconstruct_request() to return a bytestring of the request, add test for inputreq
|
2016-03-19 20:32:37 -07:00 |
|
Ilya Kreymer
|
c96e419341
|
recorder: ensure filename is also tracked by the indexer, add tests
for redis file mapping
|
2016-03-19 10:24:28 -07:00 |
|
Ilya Kreymer
|
3452cf39e0
|
recorder: use more general MultiFileWARCWriter, supporting both keeping file open
and one-warc-per record use cases
|
2016-03-18 21:40:41 -07:00 |
|
Ilya Kreymer
|
e81457df5f
|
rename WARCRecorder -> WARCWriter, add optional max_size to single warc recorder
per-record recorder combines http response/req into single file
|
2016-03-18 19:49:14 -07:00 |
|
Ilya Kreymer
|
b64be0dff1
|
recorder: add tests for single file writer, including file locking
dedup policy: support customizable dedup/skip/write policy plugins and add tests
|
2016-03-18 15:28:24 -07:00 |
|
Ilya Kreymer
|
cba8e4ee3a
|
filters: more functional filter impl for header exclusion
|
2016-03-17 18:22:26 -07:00 |
|
Ilya Kreymer
|
58e8c709aa
|
docker: add initial docker-compose, webagg Dockerfile
|
2016-03-16 18:42:15 -07:00 |
|
Ilya Kreymer
|
fe0f8ed1d8
|
Merge branch '0.11.3' into develop
|
2016-03-16 14:38:49 -07:00 |
|
Ilya Kreymer
|
cd65ddafbc
|
bump version to 0.11.3
|
2016-03-16 14:29:42 -07:00 |
|
Ilya Kreymer
|
f962418c1f
|
html rewrite typo: ensure rw_mod is set for meta content rewrite
|
2016-03-16 14:27:55 -07:00 |
|
Ilya Kreymer
|
c26660e20f
|
cookies: use httplib headers pair list instead of requests headers dict to avoid 'set-cookie' headers being concatenated, as that messes up parsing in 3.5.1
|
2016-03-16 09:47:55 -07:00 |
|
Ilya Kreymer
|
ef5860901f
|
warc record loader: if no content-length is specified on WARC record (as opposed to error or invalid), leave stream alone, don't force size of 0 and 204
|
2016-03-13 17:56:37 -07:00 |
|
Ilya Kreymer
|
8dc59ef6bd
|
webagg: add test for live server config
|
2016-03-13 16:53:39 -07:00 |
|
Ilya Kreymer
|
06978bd8d2
|
recorder: check for empty input stream (support for direct proxy?)
|
2016-03-13 11:17:52 -07:00 |
|
Ilya Kreymer
|
709d2b1ea2
|
reorg: move StreamIter to utils
|
2016-03-12 23:29:23 -08:00 |
|
Ilya Kreymer
|
7a828017d1
|
recorder: clean up logging, ReadFullyStream moves to utils, get_request_uri to inputreq
|
2016-03-12 22:18:01 -08:00 |
|
Ilya Kreymer
|
49b6ae78a8
|
live loader: remove liverec (doesn't work well with gevent), use regular requests
instead of overriden version.
reconstruct header block from httplib header pairs list
move ReadFullyStream to utils
|
2016-03-12 22:15:24 -08:00 |
|
Ilya Kreymer
|
9adb8da3b7
|
recorder: add support for filtering collections to record by regex (default: .*)
add support for excluding certain headers when writing WARCs
tests: add first batch of tests for recorder, using live upstream server
|
2016-03-11 11:12:25 -08:00 |
|
Ilya Kreymer
|
2003925b75
|
setup: fix pywb py3 version to 0.30.0, add coverage for recorder
|
2016-03-11 11:11:43 -08:00 |
|
Ilya Kreymer
|
3b3e190cf4
|
testing: use test mixins for class-scope temp directory, live server creation
use processes instead of threads for live server
|
2016-03-11 11:10:22 -08:00 |
|
Ilya Kreymer
|
2051785e6b
|
statusandheaders: add to_str() method with 'exclude_list' to support converting to str with certain headers
excluded. also supported by to_bytes()
|
2016-03-11 11:02:13 -08:00 |
|
Ilya Kreymer
|
46d013ab19
|
test redis: minor tweak to use @patch for fakeredis mock
|
2016-03-10 21:35:01 -08:00 |
|
Ilya Kreymer
|
c309637a3a
|
tests: webagg test tweaks, create TempDirTests for sharing tests that require a temp dir
|
2016-03-10 16:04:27 -08:00 |
|
Ilya Kreymer
|
7b847311d5
|
dir agg: include filename in dir source name
|
2016-03-10 15:51:01 -08:00 |
|
Ilya Kreymer
|
3f734e1c98
|
tests: remove 3.2, fix auto_index test assert
|
2016-03-10 13:07:57 -08:00 |
|
Ilya Kreymer
|
42aa12f9ae
|
test py3.2 also
|
2016-03-10 12:55:36 -08:00 |
|
Ilya Kreymer
|
34cc3ccacb
|
versions and readme: update version to 0.30.0, update README with python 2 and 3 support
|
2016-03-10 12:51:14 -08:00 |
|
Ilya Kreymer
|
0f6e3da127
|
cdx: tests: add tests for comparison ops
|
2016-03-10 12:47:36 -08:00 |
|
Ilya Kreymer
|
e5ca9bf601
|
Merge branch 'master' into py3
|
2016-03-10 10:53:30 -08:00 |
|
Ilya Kreymer
|
effd618bb3
|
tests: add parse_comment test for html_rewriter
|
2016-03-10 10:10:51 -08:00 |
|
Ilya Kreymer
|
8ae692d630
|
Merge pull request #172 from machawk1/patch-3
Spelling: Quick comment fix
|
2016-03-10 09:52:25 -08:00 |
|
Ilya Kreymer
|
a25bb5e238
|
Merge pull request #166 from machawk1/patch-2
Fixed misspelling
|
2016-03-10 09:51:56 -08:00 |
|
Ilya Kreymer
|
12ecb29a01
|
tweak CHANGES
|
2016-03-10 09:48:28 -08:00 |
|
Ilya Kreymer
|
2f67b78023
|
Update CHANGES for 0.11.2
|
2016-03-10 09:47:12 -08:00 |
|
Ilya Kreymer
|
c1bdeac92b
|
redis: fix redis key lookup, add tests for zrangebylex with new fakeredis
|
2016-03-09 18:33:04 -08:00 |
|
Ilya Kreymer
|
31fb2f926f
|
add recorder app, initial pass!
|
2016-03-09 14:33:36 -08:00 |
|
Ilya Kreymer
|
1499f0e611
|
add shared README.rst and coverage
|
2016-03-09 14:33:11 -08:00 |
|
Ilya Kreymer
|
34386578a5
|
shared setup: move webagg test to webagg/test
|
2016-03-09 14:29:14 -08:00 |
|
Ilya Kreymer
|
0198bf1213
|
Merge branch 'develop' into py3
|
2016-03-09 07:29:54 -08:00 |
|