1
0
mirror of https://github.com/webrecorder/pywb.git synced 2025-03-15 08:04:49 +01:00

13 Commits

Author SHA1 Message Date
Ilya Kreymer
772993ba53 Adaptive Streaming Improvements (#236)
* adaptive rewrite improvements:
- Add 'application/vnd.apple.mpegurl' as HLS type in rules.yaml and default_rewriter.py
- Support setting max resolution and max bandwidth to choose, defaults to 480x854 and 200000 respectively
- LiveWebLoader provides a get_custom_metadata for specifying WARC-JSON-Metadata header, per mime type (TODO: support customization via rules)
- When filtering, first limiting by resolution (if set), then by bandwidth (if set), otherwise default to max bandwidth
- Max resoluton/max bandwidth stored in WARC record under WARC-JSON-Metadata as 'adaptive_max_resolution' and 'adaptive_max_bandwidth' to ensure replayability. If absent, choose absolute max in manifest to be backwards compatible
- Add sample HLS and DASH manifests for testing, with and without max resolution/bandwidth settings.
2017-09-06 23:23:39 -07:00
Ilya Kreymer
06b1134be5 aggregator: support 'invert_sources' option to exclude source list, rather than include
can be set explicitly or via '!' on the sources list
tests: test invert sources
filters: include params to skip_response() filter
warc headers: change headers for recording from other source to: WARC-Source-URI and WARC-Created-Date
2017-06-01 07:45:02 -07:00
Ilya Kreymer
d24868db7a tests: add MementoOverrideTests as a reusable class, convert memento_agg tests to use class,
handlers: add saved link header data for memento tests for handlers
2016-11-15 14:24:34 -08:00
Ilya Kreymer
6b4b038471 refactor: fix pywb.webagg package paths
all webagg tests working!
move testdata cdxj into sample_archive, remove rest (duplicates) #200
2016-11-08 14:30:09 -08:00
Ilya Kreymer
1fb6e9b5fa rewrite: url rewriter: don't rewrite relative urls, only those that start with scheme, / or contain ../ #195
update tests to reflect this new behavior
2016-09-14 13:04:46 -07:00
Ilya Kreymer
d457223555 tests: add brotli compression test #184 2016-06-16 00:00:47 -04:00
Ilya Kreymer
33f247582f rewrite: HTMLRewriter should insert head_insert at end of stream, if it hasn't
been inserted by the end (and if there was some content written -- don't insert for 0-length responses)
Addresses missing head insert if only head tags are present and no head, as per hypothesis/via#9
2015-05-14 22:32:08 -07:00
Ilya Kreymer
e806a33289 add unclosed script sample 2015-04-01 07:13:51 -07:00
Ilya Kreymer
5a11714b41 rewrite: refactor JS rewriters into seperate mixins, allowing for
link only, location only, and link + location JS rewriters.
location-only rewriter is new
js_rewrite_location options: all, location, urls (for now)
2014-12-07 21:09:37 -08:00
Ilya Kreymer
71e8ada57d rewrite: add test for banner-only mode, rewriting w/o a head using local
'sample_no_head' file.
query.html: use client side rewriting for calendar dates
rewrite: remove unused decode stuff
2014-08-04 20:45:02 -07:00
Ilya Kreymer
1317b2b10f route selection via proxy auth!
refactor poute request parsing to happen in the actual router class instead of in the route
in proxy mode, add support for picking a route via proxy-auth
improve test for 'top' rewriting
2014-07-10 21:54:23 -07:00
Ilya Kreymer
8d3d326c9e tests: add pathresolver tests for RedisResolver and PathIndexResolver 2014-04-02 11:41:20 -07:00
Ilya Kreymer
5a41f59f39 new unified config system, via rules.yaml!
contains configs for cdx canon, fuzzy matching and rewriting!
rewriting: ability to add custom regexs per domain
also, ability to toggle js rewriting and custom rewriting file
(default is wombat.js)
2014-02-26 18:02:01 -08:00