backup/pywb
1
0
Fork 0
mirror of https://github.com/webrecorder/pywb.git synced 2025-03-15 16:14:48 +01:00
Code Issues Projects Releases Activity
867 Commits 56 Branches 63 Tags
Commit Graph

867 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Ilya Kreymer
2396d80dc4 setup: add youtube_dl as dependency 
rewrite: add call to _live_rewrite_headers to get correct headers for
proxy on video info
 2014-11-01 15:41:00 -07:00
Ilya Kreymer
5b9dcba15f video: add video rewriting use vidrw client side and youtube-dl on the server 
add vi_ modifier:
-on record, gets video_info from youtube-dl, sends to proxy,
if any, via PUTMETA to create metadata record
-on playback, fetches special metadata record with video info and
returns to client as json
-vidrw script: fetches video info, if any, and attempts to replace
iframe and embed tags (so far) which are videos
wombat: export extract_url function, fix spaces and use object instance
semantics
 2014-11-01 15:41:00 -07:00
Ilya Kreymer
a3b931b45e regex rewrite: fix js regex (dashes), add additional test case 2014-11-01 15:39:51 -07:00
Ilya Kreymer
841fd3f7b4 warc: add ability to set read block size (def 16384) in archiveiterator 2014-11-01 13:29:37 -07:00
Ilya Kreymer
5be65f2945 rules: better rule def, cleanup spacing 2014-10-30 00:10:39 -07:00
Ilya Kreymer
f14f37d5b1 tests: use httpbin for redirect tests 2014-10-29 09:47:32 -07:00
Ilya Kreymer
3d4526f994 wombat: add wombat class to window 2014-10-28 11:46:16 -07:00
Ilya Kreymer
d7b1bc8151 setup: add youtube_dl as dependency 
rewrite: add call to _live_rewrite_headers to get correct headers for
proxy on video info
 2014-10-28 11:11:15 -07:00
Ilya Kreymer
fb85570974 video: add video rewriting use vidrw client side and youtube-dl on the server 
add vi_ modifier:
-on record, gets video_info from youtube-dl, sends to proxy,
if any, via PUTMETA to create metadata record
-on playback, fetches special metadata record with video info and
returns to client as json
-vidrw script: fetches video info, if any, and attempts to replace
iframe and embed tags (so far) which are videos
wombat: export extract_url function, fix spaces and use object instance
semantics
 2014-10-28 10:36:48 -07:00
Ilya Kreymer
61ce53a0e0 warc/cdx: include metadata and resource records in default cdx index 
emit 200 and 204 responses for metadata and resource, though write '-'
to cdx (for compatibility for now)
include content-length in resource/metadata records
 2014-10-28 10:29:50 -07:00
Ilya Kreymer
c9273ee5ed rewrite: add 'deprefix' support to remove wburl prefix from any query 
params
 2014-10-26 12:12:37 -07:00
Ilya Kreymer
037cf35eb8 wsgi_wrapper: check for str before decoding err msg 2014-10-25 11:42:44 -07:00
Ilya Kreymer
8441b54192 head_insert: add mod to wombat 2014-10-24 14:13:59 -07:00
Ilya Kreymer
67e94d13f4 handlers/wombat: pass in mod to wombat, ability to customize modifier 
for embeds
 2014-10-24 12:45:41 -07:00
Ilya Kreymer
9b64194342 bump version to 0.6.4 2014-10-24 12:44:52 -07:00
Ilya Kreymer
a4f9138cb4 update README link 2014-10-21 19:29:27 -07:00
Ilya Kreymer
f394e26cf1 update CHANGES.rst 2014-10-21 19:21:15 -07:00
Ilya Kreymer
05995ad9cf Merge branch 'master' into develop, just README changes 2014-10-21 19:09:31 -07:00
Ilya Kreymer
e8d3965269 pep8 style fixes, remove unused methods 2014-10-21 19:06:16 -07:00
Ilya Kreymer
0a1c053507 Add badge 2014-10-19 08:33:26 -07:00
Ilya Kreymer
dfae25da01 Update README with News! 2014-10-19 08:32:11 -07:00
Ilya Kreymer
1a78fffa22 refactor handlers: simplify handling methods: handle_request() called 
for all requests, handle_query() only for url query/calendar, and
handle_replay() only for replay. Improves extensibility of the handling
path
 2014-10-19 00:33:32 -07:00
Ilya Kreymer
d99f7f996c urlrewriter refactor: replace get_abs_url and get_timestamp_url with 
get_new_ur() which just calls wburl.to_str and applies rewriter prefix
allows creating a new wburl with any component(s) changed
 2014-10-19 00:24:00 -07:00
Ilya Kreymer
d01275335b bump version to 0.6.3 2014-10-19 00:19:07 -07:00
Ilya Kreymer
c9c9e9d7ed Add Gratipay link 2014-10-18 17:00:33 -07:00
Ilya Kreymer
e4befd0d85 update README.rst 0.6.2 2014-10-18 15:27:58 -07:00
Ilya Kreymer
729320393a update license statement in js files with github link 2014-10-18 15:18:40 -07:00
Ilya Kreymer
268861b2ea Update README with UI Customization info 2014-10-18 15:14:43 -07:00
Ilya Kreymer
b7d23e4736 Update CHANGES.rst with latest 2014-10-18 14:51:21 -07:00
Ilya Kreymer
7f378c9aab move wb.css include into banner.html for easier overridability 2014-10-18 12:40:02 -07:00
Ilya Kreymer
4a1cc46fa3 framed replay: invert framed replay paradigm, replay always uses 
canonical, no-modifier archival url (instead of mp_).
When using frames, the page redirects to a 'tf_' page, which then uses
replaceHistory() to change url back to canonical form.
memento: support for framed replay, include memento headers in top frame
bump version to 0.6.2
 2014-10-18 11:21:07 -07:00
Ilya Kreymer
b99dcb41f0 banner: support rel and abs paths for banner_html, relative to current 
dir or system absolute
 2014-10-17 09:24:16 -07:00
Ilya Kreymer
cede54f0c1 self-redir: remove referrer-based self-redirect check, as it may be 
triggered incorrectly during refresh.. (will need to investigate more if
there's an edge-case to test against)
 2014-10-17 08:54:03 -07:00
Ilya Kreymer
1c23e12c06 banner: fixes for framed replay with new default banner 2014-10-17 08:40:57 -07:00
Ilya Kreymer
0efa2dc0ad rewrite/banner: add a seperate 'banner_html' setting which allows 
overriding just the banner (and not the entire head_insert). Setting
banner_html: False will disable the banner, or setting to a custom
template will insert that template. Default template loads
default_banner.js which does the actual initialization.
 2014-10-17 08:28:06 -07:00
Ilya Kreymer
b7a098a9a7 update rules for additional sites 2014-10-17 08:27:56 -07:00
Ilya Kreymer
614938479b jinaj2 replay: use ChoiceLoader to properly load either local file 
system or package templates
 2014-10-16 20:33:17 -07:00
Ilya Kreymer
aecc847ec1 rewrite: seperate stream_to_gen and text_rewriting_stream_to_gen 
The regular stream_to_gen is much simpler and specifically for
binary/unrewritten content. text_rewriting_stream_to_gen() performs
rewriting. Use fixed buffer of 16384 for read size, allows for better
steaming when using live rewrite
 2014-10-16 20:13:53 -07:00
Ilya Kreymer
50bf7d2634 rewrite: move extract_client_cookie to utils for access at rewrite 
root cookie_rewriter: keep max-age
add csrf token copying (experimental)
update tests
 2014-10-12 03:07:54 -07:00
Ilya Kreymer
498a864441 rewriting: support setting cookie_scope at collection level 
js rewriting: add custom url rewrite option to per-url rewrite rules
 2014-10-06 10:14:45 -07:00
Ilya Kreymer
f1b3f8c76f cookie rewriter work: ability to set a custom 'root scope' rewriter, 
which sets the path of all cookies to pywb root.
Option to enable per url-prefix in rules, still more testing, other
options needed
 2014-09-30 12:42:11 -07:00
Ilya Kreymer
7feb0893eb rewrite: add 'application/json' to a seperate 'json' regex rewriter type (rewrite links only, no 
http), can be customized via rules
wombat: add rewrite_style for rewriting style attrs
query: don't include any filter in latest, custom filter can be used
without any other filters
tests: fix typos in tests
 2014-09-30 10:57:25 -07:00
Ilya Kreymer
00efe33870 Merge branch 'master' into develop 2014-09-22 21:15:18 -07:00
Ilya Kreymer
319b8124be cdxobject: add ability to create empty CDXObject(), add tests for 
CDXObject/IDXObject checking for supported and unsupported number of
fields
 2014-09-22 21:12:25 -07:00
Ilya Kreymer
3aa70985a7 README: memento line tweak 2014-09-22 21:11:43 -07:00
Ilya Kreymer
eef1f1042a update README to mention memento protocol support 2014-09-22 21:10:10 -07:00
Ilya Kreymer
e2f8594ea7 rules: add [?&] prefix to query match, use {0} instead of {} for 2.6 
compatibility
 2014-09-21 20:04:51 -07:00
Ilya Kreymer
4c5a7d6bcd rules: use yaml lists in fuzzy rules, update CHANGES.rst 2014-09-21 19:48:14 -07:00
Ilya Kreymer
ec27ccfbb6 fuzzy match rules: to simplify custom fuzzy match use cases, add support 
for matching fuzzy match query params as a list
 2014-09-21 14:46:10 -07:00
Ilya Kreymer
7ac98fbfe2 cookie rewriter: use relative path for cookie path rewriting, pass 
relative path to urlrewriter
rules: add more rules
 2014-09-21 13:23:19 -07:00