Ilya Kreymer
4303ce4ecb
warc indexing: better handling of records with content-length to small, read first line to get to warc end (fixes indexing of warc in ikreymer/webarchiveplayer#14 )
2015-11-26 00:47:15 -08:00
Ilya Kreymer
2922801b7c
test: rewrite_live: pass Accepnt-Encoding: identity to disable gzip, simplified version of fix in #151
2015-11-26 00:34:45 -08:00
Ilya Kreymer
6c76675fe7
setup: fix setup.py again
2015-11-04 16:33:19 -08:00
Ilya Kreymer
c107dd3af1
setup: fix 'setup.py test' with new setuptools changes, fix surt==0.2
2015-11-04 16:16:10 -08:00
Ilya Kreymer
d98c1f6cf7
memento/api: add a new /collinfo.json end-point, enabled with 'enable_coll_info' config setting, which returns
...
the value fo collinfo.json template. Default template returns an entry for each handler route,
including the route path (id), title (name) and memento timegate and timemap paths, to be used with
an aggregator. Using a custom 'info_json' template can specify a different collinfo template, alternative to #69 (local aggregation)
Closes #146
2015-11-04 15:36:44 -08:00
Ilya Kreymer
dc74b14af0
iframe scrolling: use webkit scrollbars for better scrolling iframes due to osx chrome iframe bug
...
scrollbar now fully functional, though some artifacts still appear #93
2015-10-31 12:06:01 -07:00
Ilya Kreymer
bd2b5181a0
tests: add new tests for redis-based cache, #145
2015-10-30 13:18:58 -07:00
Ilya Kreymer
3132bfa7f4
cache: add a simple RedisCache implementation (alongside local and uwsgi)
...
proxy_ip_resolver: add option to use RedisCache if redis_cache_key set in config
proxy_ip_resolver: add 'delete' option to delete ip from cache, closes #145
2015-10-30 13:15:07 -07:00
Ilya Kreymer
16cf997a07
proxy: stick with http1.0 as not really supporting 1.1, #143
2015-10-26 15:25:00 -07:00
Ilya Kreymer
eeb35ea3b4
proxy: add ProxyRouter wrapper to check for content-length and, if missing, perform full buffering (http1.0) or chunked encoding (http1.1) (separate from replay view buffering)
...
add tests for buffering and chunked encoding, fixes #143 , also tests no banner url-rewrite only proxy related to #142
2015-10-25 18:02:51 -07:00
Ilya Kreymer
0c96591c49
proxy: change HttpsUrlRewriter to SchemeOnlyUrlRewriter, which fixes http->https or https->http to match
...
the scheme of the current page.
url-rewrite-only mode: add uo_ mod and use that to rewrite only urls (no banner, no client side rewrite)
addresses #142
2015-10-24 15:10:30 -07:00
Ilya Kreymer
979fcaeda3
tests: fix mock YoutubeDLWrapper after refactor, #141
2015-10-23 12:19:15 -07:00
Ilya Kreymer
39e824cb3a
live rewite proxy: decouple having http/https proxy from recording,
...
move youtubedl wrapper calls, metadata add calls to live rewrite proxy class for easier extension
closes #141 also improves #136
2015-10-23 11:57:12 -07:00
Ilya Kreymer
c7224ecceb
tests: use proxy str directly (imrpove test cov)
2015-10-23 11:54:16 -07:00
Ilya Kreymer
4ba4521b56
tests: use random port instead of 8080 for cli test to avoid conflicts with running services
2015-10-23 11:53:28 -07:00
Ilya Kreymer
e37636de84
cdxindexer: if latest ujson (with forward slash not-escaping) is available, use that when indexing, closes #140
...
tests: update indexer CDXJ tests to be order-independent
travis: install ujson for testing
2015-10-22 17:46:05 -07:00
Ilya Kreymer
c003a96618
Merge pull request #139 from jcushman/webencodings
...
Use webencodings to encode head_insert_str.
2015-10-22 16:23:47 -07:00
Ilya Kreymer
3318402fc3
bump version to 0.11.0
2015-10-22 15:40:36 -07:00
Jack Cushman
633eb31f57
Use webencodings to encode head_insert_str.
2015-10-22 16:40:59 -04:00
Ilya Kreymer
6035a9dc82
Update CHANGES for 0.10.10
0.10.10
2015-10-21 17:50:26 -07:00
Ilya Kreymer
b8b473bf19
cdxindexer: use ujson if it is available
2015-10-21 15:28:26 -07:00
Ilya Kreymer
f3c87c1786
wombat: additional checks for cross-frame top frame access, #137
2015-10-21 10:54:48 -07:00
Ilya Kreymer
9e1447c448
rewrite: strip spaces when rewriting urls in html, closes #134
2015-10-20 12:59:07 -07:00
Ilya Kreymer
75085ad91b
loaders: fix loader inits, don't inherit from BlockLoader #135
2015-10-20 10:33:24 -07:00
Ilya Kreymer
94095e452a
loaders: refactor BlockLoader to use an extensible dict of loaders
...
individual HttpLoader, LocalFileLoader and S3Loader supported by default
Loaders created via BlockLoader also cached for reuse, closes #135
2015-10-19 11:59:35 -07:00
Ilya Kreymer
4ea8e5d535
rewrite: Make LiveRewriter class overridable in config, request method extensible in LiveRewriter, closes #136
2015-10-19 11:46:27 -07:00
Ilya Kreymer
8b68119994
wombat: use wb_rel_prefix instead of wbinfo.coll for rel prefix path
2015-10-17 08:53:01 -07:00
Ilya Kreymer
93045fb39f
rules: fuzzy rule for fastly..
2015-10-16 09:43:22 -07:00
Ilya Kreymer
06999791de
warc: when iterating over WARC, don't stop after first empty gzip records, unless really at the end..
...
add test for post + minimal error
2015-10-16 08:48:59 -07:00
Ilya Kreymer
925b23f8a8
rewrite: guard against invalid encoding in html charset= and default to utf-8 if specified encoding fails, related to hypothesis/via#53
2015-10-15 13:58:15 -07:00
Ilya Kreymer
e249f300e3
tests refactor! init pywb once per module, instead of once per test
...
refactor common init pattern to server_mock for now (can add fixtures also)
2015-10-14 20:34:46 -07:00
Ilya Kreymer
31dd946114
live rewrite: redir query requests to latest replay in live rewrite handler
2015-10-14 20:33:59 -07:00
Ilya Kreymer
42091fd36e
wombat: fix typo which caused rewrite_style() to be skipped unless style started with url prefix!
2015-10-14 17:36:33 -07:00
Ilya Kreymer
e2b0f5c9cb
bump version to 0.10.10
2015-10-14 17:35:17 -07:00
Ilya Kreymer
b612c584de
tests: test fixes for windows
2015-10-13 21:36:27 -07:00
Ilya Kreymer
d39ac71531
bump version to 0.10.9.1 for wombat scheme-rel url fix
0.10.9.1
2015-10-12 22:18:19 -07:00
Ilya Kreymer
edbfd500a4
wombat: fix rel '/' rewrite which incorrectly handles rel scheme '//' urls
2015-10-12 22:13:26 -07:00
Ilya Kreymer
4c2e7b89b6
More 0.10.9 CHANGES update
0.10.9
2015-10-12 00:31:34 -07:00
Ilya Kreymer
fd90857dc2
README tweaks
2015-10-12 00:11:18 -07:00
Ilya Kreymer
c4ba08d3c1
Update README with more info about proxy collection and timestamp selection
2015-10-12 00:09:02 -07:00
Ilya Kreymer
78788fa744
Fixes in CHANGES for 0.10.9
2015-10-11 23:55:28 -07:00
Ilya Kreymer
57a35347c1
Update CHANGES for 0.10.9
2015-10-11 23:52:03 -07:00
Ilya Kreymer
6efff4cd8f
rules: cleanup, remove obsolete rules
2015-10-11 23:50:38 -07:00
Ilya Kreymer
8b7516708d
wombat: improved rewrite when running at root: fixes for extract_orig and /-rel rewrite
...
move init to be before proxy check
2015-10-11 23:16:15 -07:00
Ilya Kreymer
bd664ac79d
Update README to 0.10.9, remove gratipay links
2015-10-11 21:13:48 -07:00
Ilya Kreymer
4dfe187174
proxy improvements:
...
use proxy_magic path to get video info to ensure video info, addresses #106
video info: ensure vi_ replay has CORS support to support serving from magic path
proxy & wombat improvements: set replay_top to window.top and avoid causing cross-domain errors
2015-10-11 21:03:30 -07:00
Ilya Kreymer
db4fbe79ec
tests: add test for BufferedReader 'deflate' (w/o gzip header)
2015-10-11 17:47:19 -07:00
Ilya Kreymer
6f7bd8c291
proxy resolvers: add tests for ip-based resolver
...
cache: default cache returns empty instead of raise KeyError on invalid key, to be consistent with uwsgi
2015-10-11 17:46:12 -07:00
Ilya Kreymer
a3a8b777d2
cdx: don't warn on .loc files, zipnum: add newline to page info response
2015-10-07 17:16:39 -07:00
Ilya Kreymer
a0878e6998
tests: fix regex for 2.6, fix live example
2015-10-07 11:59:31 -07:00