Ilya Kreymer
f21f4fb1ba
Merge branch 'develop' into video
2014-12-01 09:10:08 -08:00
Ilya Kreymer
dffa0bcbad
video work: vidrw fixes to support object, fallback to flash
...
rangecache: create dir on first use only
2014-12-01 09:09:23 -08:00
Ilya Kreymer
c996e70a6e
wburl: detect and decode partially encoded schemes in url, such as http%3A//,
...
https%A2F2F// before handling further
add additional tests for wburl
2014-11-29 11:13:57 -08:00
Ilya Kreymer
2f5be9ed9f
video: work on extracting video from <object> tags, fallback to
...
flowplayer if html5 player fails
2014-11-29 11:04:18 -08:00
Ilya Kreymer
7a14e167e1
replay: vi_ handling: properly replace any protocol with metadata:// for
...
video info lookup
2014-11-29 11:03:40 -08:00
Ilya Kreymer
87d791eba8
html rewrite: rewrite param value only if start with http
2014-11-29 11:03:09 -08:00
Ilya Kreymer
3e3a74619f
various fixes: wombat: add Date.UTC and Date.parse
...
rewrite: support vi_ https -> metadata
video: fallback to vi_ call on current page
remove debug logging
2014-11-25 00:21:28 -08:00
Ilya Kreymer
4b7186a62e
wombat: adjust date for timezone!
2014-11-24 13:46:50 -08:00
Ilya Kreymer
c10df57e07
rules: add support for customizing matchType prefix, adding multiple
...
filters
2014-11-24 11:10:49 -08:00
Ilya Kreymer
d3ef47342c
Merge branch 'develop' into video
2014-11-23 18:58:31 -08:00
Ilya Kreymer
d7eb40af20
rewrite: properly rewrite scheme relative JS-escaped urls:
...
'\/\/example.com', '\\/\\/example.com/', treat same as '//example.com'
adding http: prefix
2014-11-23 18:56:49 -08:00
Ilya Kreymer
f4431c0872
wombat: add extra check for correct scheme for ajax open
2014-11-23 18:44:11 -08:00
Ilya Kreymer
fcb90fde86
rules: work on yt rules
2014-11-23 18:39:58 -08:00
Ilya Kreymer
36887f565a
video: more rewriting work, support: html audio if audio format,
...
fallback to flash (flowplayer) if html video/audio playback has failed
update setup.py to support nested include structure
2014-11-22 18:39:30 -08:00
Ilya Kreymer
0d191b338f
rules: fix rules typo
2014-11-22 18:39:17 -08:00
Ilya Kreymer
0eafd06c42
rangecache: use tmp dir for cache and delete on exit
2014-11-22 18:38:57 -08:00
Ilya Kreymer
550d78a181
add flowplayer 3.2 for flash fallback
2014-11-22 18:37:52 -08:00
Ilya Kreymer
b8b8c30573
cookie_rewriter: add tests for exact cookie rewriter
2014-11-13 09:43:50 -08:00
Ilya Kreymer
20070e95b6
cookie_rewriter: add 'exact' cookie rewriter which never changes the
...
path/domain
2014-11-13 09:24:34 -08:00
Ilya Kreymer
388f31e08f
rewrite: don't rewrite rel=canonical links, need to make rewriting more
...
configurable (#50 )
2014-11-11 15:34:14 -08:00
Ilya Kreymer
9929737a8e
rangecache: don't redirect when using range header, don't cache non-200
...
responses
2014-11-06 22:14:41 -08:00
Ilya Kreymer
1a91f514c5
Merge branch 'develop' into video
2014-11-06 22:13:44 -08:00
Ilya Kreymer
49e98e0cdc
archiveiterator/cdxindexer: cleaner load path for compressed and
...
uncompressed, ability to distinguish between chunked and non-chunked
warcs/arcs
Raise error for non-chunked gzip warcs as they can not be indexed for
replay, addressing #48
add 'bad' non-chunked gzip file for testing, using custom ext
2014-11-06 01:32:42 -08:00
Ilya Kreymer
044792f99f
bump version to 0.6.5!
2014-11-06 01:28:56 -08:00
Ilya Kreymer
cc776b65bd
update README.rst
2014-11-06 00:35:36 -08:00
Ilya Kreymer
71a8abe9c3
Merge branch 'develop' for 0.6.4
2014-11-06 00:34:32 -08:00
Ilya Kreymer
f6053a977b
Update changes for 0.6.4
2014-11-05 21:59:54 -08:00
Ilya Kreymer
00121aa165
statusandheaders parsing: properly skip multiline bad headers (missing
...
header name and ':'), fixes #49
2014-11-05 20:26:23 -08:00
Ilya Kreymer
c6a2c83b66
rangecache: always bound range, set default bound of 16384
...
wombat: work on date override, disable for now
head_insert: check for wombat not being inited to avoid undef error
2014-11-05 10:55:46 -08:00
Ilya Kreymer
88f553dce7
video work: live rewrite pings proxy with full rewrite, proxies direct
...
range request
reorg rangecache to support is_range() check, yt-specific logic
(experimental)
wombat: add date override (experimental)
bump tentative version to 0.7.0!
yt replays work with native player! (though still issues remain)
2014-11-04 22:11:25 -08:00
Ilya Kreymer
fea48fd27a
Merge branch 'develop' into video
2014-11-04 12:19:58 -08:00
Ilya Kreymer
e4bcef1c8b
rewrite: default HTMLParser entityref and charref are treated as plain
...
data for HTMLRewriter, since they are never rewritten, and to avoid
semicolon ambiguity, since no way to determine if there is a ; or not
at end. Addresses #43
2014-11-04 12:14:00 -08:00
Ilya Kreymer
5e4b830fa7
cdx: ensure cdx file is closed when iterator is done, since cdx files
...
are opened per-lookup, related to #45
2014-11-04 09:42:53 -08:00
Ilya Kreymer
72aa921ce5
video: work on domain-specific range cache rewrites
2014-11-04 08:44:45 -08:00
Ilya Kreymer
703ec0eb5e
banner insert: extra check for document.body before adding banner
2014-11-02 16:11:03 -08:00
Ilya Kreymer
7aac3aa2dd
rewrite: add support for srcset rewriting for img tag
2014-11-02 16:10:38 -08:00
Ilya Kreymer
ced94d289d
wombat: add generic setAttribute override for href and src, Image object
...
override, src prop rewrite
2014-11-02 16:09:56 -08:00
Ilya Kreymer
09e461c58c
Merge branch 'video' of https://github.com/ikreymer/pywb into video
2014-11-01 18:24:52 -07:00
Ilya Kreymer
1aac5a9f15
cache: move cache wrappers to seperate cache.py in framework from
...
proxy_resolvers
range cache: and buffering cache for serving range requests, intended
for videos but not only. full response cached in temp file and range
requests served from cache, still experimental
need to add deletion.
youtube_dl: wrap youtube-dl import due to youtube-dl HTMLParser regex
bug
tests: add test for vi_ handler
2014-11-01 15:41:01 -07:00
Ilya Kreymer
07bcf9fbfe
vidrw: work on yt rewrite
2014-11-01 15:41:01 -07:00
Ilya Kreymer
50cae15663
wombat: add wombat class to window
2014-11-01 15:41:00 -07:00
Ilya Kreymer
2396d80dc4
setup: add youtube_dl as dependency
...
rewrite: add call to _live_rewrite_headers to get correct headers for
proxy on video info
2014-11-01 15:41:00 -07:00
Ilya Kreymer
5b9dcba15f
video: add video rewriting use vidrw client side and youtube-dl on the server
...
add vi_ modifier:
-on record, gets video_info from youtube-dl, sends to proxy,
if any, via PUTMETA to create metadata record
-on playback, fetches special metadata record with video info and
returns to client as json
-vidrw script: fetches video info, if any, and attempts to replace
iframe and embed tags (so far) which are videos
wombat: export extract_url function, fix spaces and use object instance
semantics
2014-11-01 15:41:00 -07:00
Ilya Kreymer
a3b931b45e
regex rewrite: fix js regex (dashes), add additional test case
2014-11-01 15:39:51 -07:00
Ilya Kreymer
841fd3f7b4
warc: add ability to set read block size (def 16384) in archiveiterator
2014-11-01 13:29:37 -07:00
Ilya Kreymer
5be65f2945
rules: better rule def, cleanup spacing
2014-10-30 00:10:39 -07:00
Ilya Kreymer
f14f37d5b1
tests: use httpbin for redirect tests
2014-10-29 09:47:32 -07:00
Ilya Kreymer
3d4526f994
wombat: add wombat class to window
2014-10-28 11:46:16 -07:00
Ilya Kreymer
d7b1bc8151
setup: add youtube_dl as dependency
...
rewrite: add call to _live_rewrite_headers to get correct headers for
proxy on video info
2014-10-28 11:11:15 -07:00
Ilya Kreymer
fb85570974
video: add video rewriting use vidrw client side and youtube-dl on the server
...
add vi_ modifier:
-on record, gets video_info from youtube-dl, sends to proxy,
if any, via PUTMETA to create metadata record
-on playback, fetches special metadata record with video info and
returns to client as json
-vidrw script: fetches video info, if any, and attempts to replace
iframe and embed tags (so far) which are videos
wombat: export extract_url function, fix spaces and use object instance
semantics
2014-10-28 10:36:48 -07:00
