1
0
mirror of https://github.com/webrecorder/pywb.git synced 2025-03-15 16:14:48 +01:00

737 Commits

Author SHA1 Message Date
Ilya Kreymer
1a91f514c5 Merge branch 'develop' into video 2014-11-06 22:13:44 -08:00
Ilya Kreymer
49e98e0cdc archiveiterator/cdxindexer: cleaner load path for compressed and
uncompressed, ability to distinguish between chunked and non-chunked
warcs/arcs
Raise error for non-chunked gzip warcs as they can not be indexed for
replay, addressing #48
add 'bad' non-chunked gzip file for testing, using custom ext
2014-11-06 01:32:42 -08:00
Ilya Kreymer
044792f99f bump version to 0.6.5! 2014-11-06 01:28:56 -08:00
Ilya Kreymer
cc776b65bd update README.rst 0.6.4 2014-11-06 00:35:36 -08:00
Ilya Kreymer
71a8abe9c3 Merge branch 'develop' for 0.6.4 2014-11-06 00:34:32 -08:00
Ilya Kreymer
f6053a977b Update changes for 0.6.4 2014-11-05 21:59:54 -08:00
Ilya Kreymer
00121aa165 statusandheaders parsing: properly skip multiline bad headers (missing
header name and ':'), fixes #49
2014-11-05 20:26:23 -08:00
Ilya Kreymer
c6a2c83b66 rangecache: always bound range, set default bound of 16384
wombat: work on date override, disable for now
head_insert: check for wombat not being inited to avoid undef error
2014-11-05 10:55:46 -08:00
Ilya Kreymer
88f553dce7 video work: live rewrite pings proxy with full rewrite, proxies direct
range request
reorg rangecache to support is_range() check, yt-specific logic
(experimental)
wombat: add date override (experimental)
bump tentative version to 0.7.0!
yt replays work with native player! (though still issues remain)
2014-11-04 22:11:25 -08:00
Ilya Kreymer
fea48fd27a Merge branch 'develop' into video 2014-11-04 12:19:58 -08:00
Ilya Kreymer
e4bcef1c8b rewrite: default HTMLParser entityref and charref are treated as plain
data for HTMLRewriter, since they are never rewritten, and to avoid
semicolon ambiguity, since no way to determine if there is a ; or not
at end. Addresses #43
2014-11-04 12:14:00 -08:00
Ilya Kreymer
5e4b830fa7 cdx: ensure cdx file is closed when iterator is done, since cdx files
are opened per-lookup, related to #45
2014-11-04 09:42:53 -08:00
Ilya Kreymer
72aa921ce5 video: work on domain-specific range cache rewrites 2014-11-04 08:44:45 -08:00
Ilya Kreymer
703ec0eb5e banner insert: extra check for document.body before adding banner 2014-11-02 16:11:03 -08:00
Ilya Kreymer
7aac3aa2dd rewrite: add support for srcset rewriting for img tag 2014-11-02 16:10:38 -08:00
Ilya Kreymer
ced94d289d wombat: add generic setAttribute override for href and src, Image object
override, src prop rewrite
2014-11-02 16:09:56 -08:00
Ilya Kreymer
09e461c58c Merge branch 'video' of https://github.com/ikreymer/pywb into video 2014-11-01 18:24:52 -07:00
Ilya Kreymer
1aac5a9f15 cache: move cache wrappers to seperate cache.py in framework from
proxy_resolvers
range cache: and buffering cache for serving range requests, intended
for videos but not only. full response cached in temp file and range
requests served from cache, still experimental
need to add deletion.
youtube_dl: wrap youtube-dl import due to youtube-dl HTMLParser regex
bug
tests: add test for vi_ handler
2014-11-01 15:41:01 -07:00
Ilya Kreymer
07bcf9fbfe vidrw: work on yt rewrite 2014-11-01 15:41:01 -07:00
Ilya Kreymer
50cae15663 wombat: add wombat class to window 2014-11-01 15:41:00 -07:00
Ilya Kreymer
2396d80dc4 setup: add youtube_dl as dependency
rewrite: add call to _live_rewrite_headers to get correct headers for
proxy on video info
2014-11-01 15:41:00 -07:00
Ilya Kreymer
5b9dcba15f video: add video rewriting use vidrw client side and youtube-dl on the server
add vi_ modifier:
-on record, gets video_info from youtube-dl, sends to proxy,
if any, via PUTMETA to create metadata record
-on playback, fetches special metadata record with video info and
returns to client as json
-vidrw script: fetches video info, if any, and attempts to replace
iframe and embed tags (so far) which are videos
wombat: export extract_url function, fix spaces and use object instance
semantics
2014-11-01 15:41:00 -07:00
Ilya Kreymer
a3b931b45e regex rewrite: fix js regex (dashes), add additional test case 2014-11-01 15:39:51 -07:00
Ilya Kreymer
841fd3f7b4 warc: add ability to set read block size (def 16384) in archiveiterator 2014-11-01 13:29:37 -07:00
Ilya Kreymer
5be65f2945 rules: better rule def, cleanup spacing 2014-10-30 00:10:39 -07:00
Ilya Kreymer
f14f37d5b1 tests: use httpbin for redirect tests 2014-10-29 09:47:32 -07:00
Ilya Kreymer
3d4526f994 wombat: add wombat class to window 2014-10-28 11:46:16 -07:00
Ilya Kreymer
d7b1bc8151 setup: add youtube_dl as dependency
rewrite: add call to _live_rewrite_headers to get correct headers for
proxy on video info
2014-10-28 11:11:15 -07:00
Ilya Kreymer
fb85570974 video: add video rewriting use vidrw client side and youtube-dl on the server
add vi_ modifier:
-on record, gets video_info from youtube-dl, sends to proxy,
if any, via PUTMETA to create metadata record
-on playback, fetches special metadata record with video info and
returns to client as json
-vidrw script: fetches video info, if any, and attempts to replace
iframe and embed tags (so far) which are videos
wombat: export extract_url function, fix spaces and use object instance
semantics
2014-10-28 10:36:48 -07:00
Ilya Kreymer
61ce53a0e0 warc/cdx: include metadata and resource records in default cdx index
emit 200 and 204 responses for metadata and resource, though write '-'
to cdx (for compatibility for now)
include content-length in resource/metadata records
2014-10-28 10:29:50 -07:00
Ilya Kreymer
c9273ee5ed rewrite: add 'deprefix' support to remove wburl prefix from any query
params
2014-10-26 12:12:37 -07:00
Ilya Kreymer
037cf35eb8 wsgi_wrapper: check for str before decoding err msg 2014-10-25 11:42:44 -07:00
Ilya Kreymer
8441b54192 head_insert: add mod to wombat 2014-10-24 14:13:59 -07:00
Ilya Kreymer
67e94d13f4 handlers/wombat: pass in mod to wombat, ability to customize modifier
for embeds
2014-10-24 12:45:41 -07:00
Ilya Kreymer
9b64194342 bump version to 0.6.4 2014-10-24 12:44:52 -07:00
Ilya Kreymer
a4f9138cb4 update README link 2014-10-21 19:29:27 -07:00
Ilya Kreymer
f394e26cf1 update CHANGES.rst 2014-10-21 19:21:15 -07:00
Ilya Kreymer
05995ad9cf Merge branch 'master' into develop, just README changes 2014-10-21 19:09:31 -07:00
Ilya Kreymer
e8d3965269 pep8 style fixes, remove unused methods 2014-10-21 19:06:16 -07:00
Ilya Kreymer
0a1c053507 Add badge 2014-10-19 08:33:26 -07:00
Ilya Kreymer
dfae25da01 Update README with News! 2014-10-19 08:32:11 -07:00
Ilya Kreymer
1a78fffa22 refactor handlers: simplify handling methods: handle_request() called
for all requests, handle_query() only for url query/calendar, and
handle_replay() only for replay. Improves extensibility of the handling
path
2014-10-19 00:33:32 -07:00
Ilya Kreymer
d99f7f996c urlrewriter refactor: replace get_abs_url and get_timestamp_url with
get_new_ur() which just calls wburl.to_str and applies rewriter prefix
allows creating a new wburl with any component(s) changed
2014-10-19 00:24:00 -07:00
Ilya Kreymer
d01275335b bump version to 0.6.3 2014-10-19 00:19:07 -07:00
Ilya Kreymer
c9c9e9d7ed Add Gratipay link 2014-10-18 17:00:33 -07:00
Ilya Kreymer
e4befd0d85 update README.rst 0.6.2 2014-10-18 15:27:58 -07:00
Ilya Kreymer
729320393a update license statement in js files with github link 2014-10-18 15:18:40 -07:00
Ilya Kreymer
268861b2ea Update README with UI Customization info 2014-10-18 15:14:43 -07:00
Ilya Kreymer
b7d23e4736 Update CHANGES.rst with latest 2014-10-18 14:51:21 -07:00
Ilya Kreymer
7f378c9aab move wb.css include into banner.html for easier overridability 2014-10-18 12:40:02 -07:00