backup/pywb
1
0
Fork 0
mirror of https://github.com/webrecorder/pywb.git synced 2025-03-15 16:14:48 +01:00
Code Issues Projects Releases Activity
1,656 Commits 56 Branches 63 Tags
Commit Graph

1656 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Ilya Kreymer
b8a1cf5081 bump version to 0.11.4 2016-03-20 23:36:42 -07:00
Ilya Kreymer
33f60efb96 tweak CHANGES 2016-03-20 23:35:58 -07:00
Ilya Kreymer
f014c32650 tweak CHANGES 2016-03-20 23:33:57 -07:00
Ilya Kreymer
dfda481b19 Merge branch '0.11.4' into develop 2016-03-20 23:19:16 -07:00
Ilya Kreymer
1a458d6b26 gevent: set env var GEVENT_MONKEY_PATCH=1 in uwsgi.ini to enable monkey-patching explicitly in pywb.apps.wayback 2016-03-20 23:18:11 -07:00
Ilya Kreymer
fff94bb526 Merge branch '0.11.4' into develop 2016-03-20 22:54:44 -07:00
Ilya Kreymer
63f44cbc26 wombat: bump version to 2.11 2016-03-20 22:54:27 -07:00
Ilya Kreymer
8302353209 wombat: fix typo in window.Crypto check! 2016-03-20 22:52:19 -07:00
Ilya Kreymer
4b8bbb7b40 wombat: fix typo in window.Crypto check! 2016-03-20 22:51:39 -07:00
Ilya Kreymer
ad66f9e11a Merge branch '0.11.4' into develop 2016-03-20 22:44:55 -07:00
Ilya Kreymer
c570f76ae4 update changelist for 0.11.4 2016-03-20 22:43:44 -07:00
Ilya Kreymer
d2390ee809 wombat: add check for window.crypto, generate 32-bit values 2016-03-20 22:43:05 -07:00
Ilya Kreymer
4b37d13c34 tests/gevent: skip gevent monkey patch when running tests 2016-03-20 22:23:07 -07:00
Ilya Kreymer
aa0113147a Merge branch '0.11.4' into develop 2016-03-20 18:51:23 -07:00
Ilya Kreymer
08cb02c644 wombat: override window.crypto.getRandomValues() with seeded random 2016-03-20 18:50:45 -07:00
Ilya Kreymer
8e90ea6576 gevent/uwsgi config: explicitly monkey patch in pywb.apps.wayback as early monkey patch 
not available until uwsgi 2.1 is released!
 2016-03-20 18:49:59 -07:00
Ilya Kreymer
4cf935abd1 directory agg: add CacheDirectoryAggregator to cache file listing, rescan dir only if changed 2016-03-19 20:34:09 -07:00
Ilya Kreymer
f5ee3c7bca inputreq: add reconstruct_request() to return a bytestring of the request, add test for inputreq 2016-03-19 20:32:37 -07:00
Ilya Kreymer
c96e419341 recorder: ensure filename is also tracked by the indexer, add tests 
for redis file mapping
 2016-03-19 10:24:28 -07:00
Ilya Kreymer
3452cf39e0 recorder: use more general MultiFileWARCWriter, supporting both keeping file open 
and one-warc-per record use cases
 2016-03-18 21:40:41 -07:00
Ilya Kreymer
e81457df5f rename WARCRecorder -> WARCWriter, add optional max_size to single warc recorder 
per-record recorder combines http response/req into single file
 2016-03-18 19:49:14 -07:00
Ilya Kreymer
b64be0dff1 recorder: add tests for single file writer, including file locking 
dedup policy: support customizable dedup/skip/write policy plugins and add tests
 2016-03-18 15:28:24 -07:00
Ilya Kreymer
cba8e4ee3a filters: more functional filter impl for header exclusion 2016-03-17 18:22:26 -07:00
Ilya Kreymer
58e8c709aa docker: add initial docker-compose, webagg Dockerfile 2016-03-16 18:42:15 -07:00
Ilya Kreymer
fe0f8ed1d8 Merge branch '0.11.3' into develop 2016-03-16 14:38:49 -07:00
Ilya Kreymer
cd65ddafbc bump version to 0.11.3 2016-03-16 14:29:42 -07:00
Ilya Kreymer
f962418c1f html rewrite typo: ensure rw_mod is set for meta content rewrite 2016-03-16 14:27:55 -07:00
Ilya Kreymer
c26660e20f cookies: use httplib headers pair list instead of requests headers dict to avoid 'set-cookie' headers being concatenated, as that messes up parsing in 3.5.1 2016-03-16 09:47:55 -07:00
Ilya Kreymer
ef5860901f warc record loader: if no content-length is specified on WARC record (as opposed to error or invalid), leave stream alone, don't force size of 0 and 204 2016-03-13 17:56:37 -07:00
Ilya Kreymer
8dc59ef6bd webagg: add test for live server config 2016-03-13 16:53:39 -07:00
Ilya Kreymer
06978bd8d2 recorder: check for empty input stream (support for direct proxy?) 2016-03-13 11:17:52 -07:00
Ilya Kreymer
709d2b1ea2 reorg: move StreamIter to utils 2016-03-12 23:29:23 -08:00
Ilya Kreymer
7a828017d1 recorder: clean up logging, ReadFullyStream moves to utils, get_request_uri to inputreq 2016-03-12 22:18:01 -08:00
Ilya Kreymer
49b6ae78a8 live loader: remove liverec (doesn't work well with gevent), use regular requests 
instead of overriden version.
reconstruct header block from httplib header pairs list
move ReadFullyStream to utils
 2016-03-12 22:15:24 -08:00
Ilya Kreymer
9adb8da3b7 recorder: add support for filtering collections to record by regex (default: .*) 
add support for excluding certain headers when writing WARCs
tests: add first batch of tests for recorder, using live upstream server
 2016-03-11 11:12:25 -08:00
Ilya Kreymer
2003925b75 setup: fix pywb py3 version to 0.30.0, add coverage for recorder 2016-03-11 11:11:43 -08:00
Ilya Kreymer
3b3e190cf4 testing: use test mixins for class-scope temp directory, live server creation 
use processes instead of threads for live server
 2016-03-11 11:10:22 -08:00
Ilya Kreymer
2051785e6b statusandheaders: add to_str() method with 'exclude_list' to support converting to str with certain headers 
excluded. also supported by to_bytes()
 2016-03-11 11:02:13 -08:00
Ilya Kreymer
46d013ab19 test redis: minor tweak to use @patch for fakeredis mock 2016-03-10 21:35:01 -08:00
Ilya Kreymer
c309637a3a tests: webagg test tweaks, create TempDirTests for sharing tests that require a temp dir 2016-03-10 16:04:27 -08:00
Ilya Kreymer
7b847311d5 dir agg: include filename in dir source name 2016-03-10 15:51:01 -08:00
Ilya Kreymer
3f734e1c98 tests: remove 3.2, fix auto_index test assert 2016-03-10 13:07:57 -08:00
Ilya Kreymer
42aa12f9ae test py3.2 also 2016-03-10 12:55:36 -08:00
Ilya Kreymer
34cc3ccacb versions and readme: update version to 0.30.0, update README with python 2 and 3 support 2016-03-10 12:51:14 -08:00
Ilya Kreymer
0f6e3da127 cdx: tests: add tests for comparison ops 2016-03-10 12:47:36 -08:00
Ilya Kreymer
e5ca9bf601 Merge branch 'master' into py3 2016-03-10 10:53:30 -08:00
Ilya Kreymer
effd618bb3 tests: add parse_comment test for html_rewriter 2016-03-10 10:10:51 -08:00
Ilya Kreymer
8ae692d630 Merge pull request #172 from machawk1/patch-3 
Spelling: Quick comment fix
 2016-03-10 09:52:25 -08:00
Ilya Kreymer
a25bb5e238 Merge pull request #166 from machawk1/patch-2 
Fixed misspelling
 2016-03-10 09:51:56 -08:00
Ilya Kreymer
12ecb29a01 tweak CHANGES 2016-03-10 09:48:28 -08:00