Ilya Kreymer
2396d80dc4
setup: add youtube_dl as dependency
...
rewrite: add call to _live_rewrite_headers to get correct headers for
proxy on video info
2014-11-01 15:41:00 -07:00
Ilya Kreymer
5b9dcba15f
video: add video rewriting use vidrw client side and youtube-dl on the server
...
add vi_ modifier:
-on record, gets video_info from youtube-dl, sends to proxy,
if any, via PUTMETA to create metadata record
-on playback, fetches special metadata record with video info and
returns to client as json
-vidrw script: fetches video info, if any, and attempts to replace
iframe and embed tags (so far) which are videos
wombat: export extract_url function, fix spaces and use object instance
semantics
2014-11-01 15:41:00 -07:00
Ilya Kreymer
a3b931b45e
regex rewrite: fix js regex (dashes), add additional test case
2014-11-01 15:39:51 -07:00
Ilya Kreymer
841fd3f7b4
warc: add ability to set read block size (def 16384) in archiveiterator
2014-11-01 13:29:37 -07:00
Ilya Kreymer
5be65f2945
rules: better rule def, cleanup spacing
2014-10-30 00:10:39 -07:00
Ilya Kreymer
f14f37d5b1
tests: use httpbin for redirect tests
2014-10-29 09:47:32 -07:00
Ilya Kreymer
3d4526f994
wombat: add wombat class to window
2014-10-28 11:46:16 -07:00
Ilya Kreymer
d7b1bc8151
setup: add youtube_dl as dependency
...
rewrite: add call to _live_rewrite_headers to get correct headers for
proxy on video info
2014-10-28 11:11:15 -07:00
Ilya Kreymer
fb85570974
video: add video rewriting use vidrw client side and youtube-dl on the server
...
add vi_ modifier:
-on record, gets video_info from youtube-dl, sends to proxy,
if any, via PUTMETA to create metadata record
-on playback, fetches special metadata record with video info and
returns to client as json
-vidrw script: fetches video info, if any, and attempts to replace
iframe and embed tags (so far) which are videos
wombat: export extract_url function, fix spaces and use object instance
semantics
2014-10-28 10:36:48 -07:00
Ilya Kreymer
61ce53a0e0
warc/cdx: include metadata and resource records in default cdx index
...
emit 200 and 204 responses for metadata and resource, though write '-'
to cdx (for compatibility for now)
include content-length in resource/metadata records
2014-10-28 10:29:50 -07:00
Ilya Kreymer
c9273ee5ed
rewrite: add 'deprefix' support to remove wburl prefix from any query
...
params
2014-10-26 12:12:37 -07:00
Ilya Kreymer
037cf35eb8
wsgi_wrapper: check for str before decoding err msg
2014-10-25 11:42:44 -07:00
Ilya Kreymer
8441b54192
head_insert: add mod to wombat
2014-10-24 14:13:59 -07:00
Ilya Kreymer
67e94d13f4
handlers/wombat: pass in mod to wombat, ability to customize modifier
...
for embeds
2014-10-24 12:45:41 -07:00
Ilya Kreymer
9b64194342
bump version to 0.6.4
2014-10-24 12:44:52 -07:00
Ilya Kreymer
a4f9138cb4
update README link
2014-10-21 19:29:27 -07:00
Ilya Kreymer
f394e26cf1
update CHANGES.rst
2014-10-21 19:21:15 -07:00
Ilya Kreymer
05995ad9cf
Merge branch 'master' into develop, just README changes
2014-10-21 19:09:31 -07:00
Ilya Kreymer
e8d3965269
pep8 style fixes, remove unused methods
2014-10-21 19:06:16 -07:00
Ilya Kreymer
0a1c053507
Add badge
2014-10-19 08:33:26 -07:00
Ilya Kreymer
dfae25da01
Update README with News!
2014-10-19 08:32:11 -07:00
Ilya Kreymer
1a78fffa22
refactor handlers: simplify handling methods: handle_request() called
...
for all requests, handle_query() only for url query/calendar, and
handle_replay() only for replay. Improves extensibility of the handling
path
2014-10-19 00:33:32 -07:00
Ilya Kreymer
d99f7f996c
urlrewriter refactor: replace get_abs_url and get_timestamp_url with
...
get_new_ur() which just calls wburl.to_str and applies rewriter prefix
allows creating a new wburl with any component(s) changed
2014-10-19 00:24:00 -07:00
Ilya Kreymer
d01275335b
bump version to 0.6.3
2014-10-19 00:19:07 -07:00
Ilya Kreymer
c9c9e9d7ed
Add Gratipay link
2014-10-18 17:00:33 -07:00
Ilya Kreymer
e4befd0d85
update README.rst
0.6.2
2014-10-18 15:27:58 -07:00
Ilya Kreymer
729320393a
update license statement in js files with github link
2014-10-18 15:18:40 -07:00
Ilya Kreymer
268861b2ea
Update README with UI Customization info
2014-10-18 15:14:43 -07:00
Ilya Kreymer
b7d23e4736
Update CHANGES.rst with latest
2014-10-18 14:51:21 -07:00
Ilya Kreymer
7f378c9aab
move wb.css include into banner.html for easier overridability
2014-10-18 12:40:02 -07:00
Ilya Kreymer
4a1cc46fa3
framed replay: invert framed replay paradigm, replay always uses
...
canonical, no-modifier archival url (instead of mp_).
When using frames, the page redirects to a 'tf_' page, which then uses
replaceHistory() to change url back to canonical form.
memento: support for framed replay, include memento headers in top frame
bump version to 0.6.2
2014-10-18 11:21:07 -07:00
Ilya Kreymer
b99dcb41f0
banner: support rel and abs paths for banner_html, relative to current
...
dir or system absolute
2014-10-17 09:24:16 -07:00
Ilya Kreymer
cede54f0c1
self-redir: remove referrer-based self-redirect check, as it may be
...
triggered incorrectly during refresh.. (will need to investigate more if
there's an edge-case to test against)
2014-10-17 08:54:03 -07:00
Ilya Kreymer
1c23e12c06
banner: fixes for framed replay with new default banner
2014-10-17 08:40:57 -07:00
Ilya Kreymer
0efa2dc0ad
rewrite/banner: add a seperate 'banner_html' setting which allows
...
overriding just the banner (and not the entire head_insert). Setting
banner_html: False will disable the banner, or setting to a custom
template will insert that template. Default template loads
default_banner.js which does the actual initialization.
2014-10-17 08:28:06 -07:00
Ilya Kreymer
b7a098a9a7
update rules for additional sites
2014-10-17 08:27:56 -07:00
Ilya Kreymer
614938479b
jinaj2 replay: use ChoiceLoader to properly load either local file
...
system or package templates
2014-10-16 20:33:17 -07:00
Ilya Kreymer
aecc847ec1
rewrite: seperate stream_to_gen and text_rewriting_stream_to_gen
...
The regular stream_to_gen is much simpler and specifically for
binary/unrewritten content. text_rewriting_stream_to_gen() performs
rewriting. Use fixed buffer of 16384 for read size, allows for better
steaming when using live rewrite
2014-10-16 20:13:53 -07:00
Ilya Kreymer
50bf7d2634
rewrite: move extract_client_cookie to utils for access at rewrite
...
root cookie_rewriter: keep max-age
add csrf token copying (experimental)
update tests
2014-10-12 03:07:54 -07:00
Ilya Kreymer
498a864441
rewriting: support setting cookie_scope at collection level
...
js rewriting: add custom url rewrite option to per-url rewrite rules
2014-10-06 10:14:45 -07:00
Ilya Kreymer
f1b3f8c76f
cookie rewriter work: ability to set a custom 'root scope' rewriter,
...
which sets the path of all cookies to pywb root.
Option to enable per url-prefix in rules, still more testing, other
options needed
2014-09-30 12:42:11 -07:00
Ilya Kreymer
7feb0893eb
rewrite: add 'application/json' to a seperate 'json' regex rewriter type (rewrite links only, no
...
http), can be customized via rules
wombat: add rewrite_style for rewriting style attrs
query: don't include any filter in latest, custom filter can be used
without any other filters
tests: fix typos in tests
2014-09-30 10:57:25 -07:00
Ilya Kreymer
00efe33870
Merge branch 'master' into develop
2014-09-22 21:15:18 -07:00
Ilya Kreymer
319b8124be
cdxobject: add ability to create empty CDXObject(), add tests for
...
CDXObject/IDXObject checking for supported and unsupported number of
fields
2014-09-22 21:12:25 -07:00
Ilya Kreymer
3aa70985a7
README: memento line tweak
2014-09-22 21:11:43 -07:00
Ilya Kreymer
eef1f1042a
update README to mention memento protocol support
2014-09-22 21:10:10 -07:00
Ilya Kreymer
e2f8594ea7
rules: add [?&] prefix to query match, use {0} instead of {} for 2.6
...
compatibility
2014-09-21 20:04:51 -07:00
Ilya Kreymer
4c5a7d6bcd
rules: use yaml lists in fuzzy rules, update CHANGES.rst
2014-09-21 19:48:14 -07:00
Ilya Kreymer
ec27ccfbb6
fuzzy match rules: to simplify custom fuzzy match use cases, add support
...
for matching fuzzy match query params as a list
2014-09-21 14:46:10 -07:00
Ilya Kreymer
7ac98fbfe2
cookie rewriter: use relative path for cookie path rewriting, pass
...
relative path to urlrewriter
rules: add more rules
2014-09-21 13:23:19 -07:00