backup/pywb
1
0
Fork 0
mirror of https://github.com/webrecorder/pywb.git synced 2025-03-15 16:14:48 +01:00
Code Issues Projects Releases Activity
1,295 Commits 56 Branches 63 Tags
Commit Graph

1295 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Ilya Kreymer
d2e17bf505 vidrw: formatting, special case for ustream flash 2015-07-04 00:41:00 -07:00
Ilya Kreymer
9333ebc843 rules: tweak better twitter rules, more limited custom rules, hopefully fix inline video 2015-07-03 11:53:45 -07:00
Ilya Kreymer
76ebabd3f2 rewrite rules: none or location-only rule should apply to json as well 2015-07-03 11:53:29 -07:00
Ilya Kreymer
a71e3209ae wombat 2.5! use more protoype property overrides 
- WB_wombat_location now a property, defaults to _WB_wombat_location or location on base object
- Use base lookupGetter/lookupSetter when overriding properties
- Added Element.prototype.baseURI and document.baseURI override, href and src attribute overrides
- Added insertAdjacentHTML() override
- Refactored seperate iframe_init_wombat() for all wombat setup on new iframes
- Added contentWindow and contentDocument access overrides for ensuring iframe_init_wombat() is called on iframe
 2015-07-03 11:51:11 -07:00
Ilya Kreymer
5d1c7a0430 bump version to 0.10.2-dev 2015-07-03 11:46:20 -07:00
Ilya Kreymer
4d7d40454d youtube-dl: adjust coverage as using mock youtube-dl wrapper #118 0.10.1 2015-06-27 23:18:05 -07:00
Ilya Kreymer
080587516b youtube-dl tests: use mock youtube-dl info for tests 2015-06-27 20:46:55 -07:00
Ilya Kreymer
4e37bf0a4a update changes for 0.10.1 2015-06-27 18:31:16 -07:00
Ilya Kreymer
a0b5189c67 youtube-dl: set no cover on youtube-dl import error paths #118 2015-06-27 16:34:41 -07:00
Ilya Kreymer
f0359877f0 youtube-dl: remove from dependency, installation is optional. Return 404 if attempting live 
proxy of videos and youtube-dl is not available (the only use case).
HTTPParser wrapping logic no longer needed in latest versions
Modify tests to only run if youtube-dl is installed in cases where it is not available #118
 2015-06-27 16:11:59 -07:00
Ilya Kreymer
2d0c526053 post handling: when reading post data in extract_post_query(), add optional buffer_stream which would hold the original POST 
data. This is necessary to override the `wsgi.input` to allow the post data to be read again via a fallback handler, even
after reading POST query data in replay handler, addresses #117
 2015-06-25 15:58:58 -07:00
Ilya Kreymer
06fcc89de6 readers: support 'content-encoding: deflate' using different zlib decompression options 
support default and alt settings for attempting to decompress deflate stream
tests: add tests with httpbin.org/deflate Fixes #115
 2015-06-24 13:11:33 -07:00
Ilya Kreymer
69f6354934 fix typo in rules 2015-06-18 02:49:26 -04:00
Ilya Kreymer
aa80532987 rules: actual disqus fixes.. 2015-06-18 02:40:11 -04:00
Ilya Kreymer
07c2093020 rules: disqus comments work 2015-06-18 02:33:03 -04:00
Ilya Kreymer
dea1917932 bump version to 0.10.1 2015-06-18 02:25:58 -04:00
Ilya Kreymer
f616a57e9e bump version to 0.10.0 for release! 2015-06-05 20:14:48 -07:00
Ilya Kreymer
4714f4ad31 wombat: added extra null check for contentWindow 2015-06-05 19:49:05 -07:00
Ilya Kreymer
d25572da86 Update CHANGES for 0.10.0 2015-06-05 19:16:55 -07:00
Ilya Kreymer
20b42afa4e jinja2 include: use custom RelEnvironment overriding join_path() to make includes relative to current file, to allow 
for per-collection includes to be used more easily #108
 2015-06-05 18:38:08 -07:00
Ilya Kreymer
dfeccbe62d tests: header rewriter tests for custom cache header settings, #110 2015-06-05 17:48:05 -07:00
Ilya Kreymer
27212488e3 tests: zipnum: better test coverage for incorrect idx or loc files, add invalid sample files zipnum-bad{.idx, .loc}, #112 2015-06-05 17:46:45 -07:00
Ilya Kreymer
2b9e1b97c3 rules: disable tw rewrite rule as it was page reloads 2015-06-04 17:31:16 -07:00
Ilya Kreymer
f80be17392 buffering: when buffered_replay is enabled, only buffer responses that do not have a content-length 
(eg. rewritten text content) and only buffer up to buffer_max_size (default 16384), and stream the remainder.
if the response has a content-length already, no buffering is performed #111
 2015-05-29 19:40:25 -07:00
Ilya Kreymer
15c2ddbfcf header rewriter: cache options: use 'rewrite_opts.http_cache' to set caching headers options, #110 
'pass': passthrough original cache headers unrewritten
None (default): rewrite cache headers and don't add anything else
N: set cache-control max-age: N and corresponding expires
N=0: set cache-control: no-cache; no-store
 2015-05-29 12:53:29 -07:00
Ilya Kreymer
bb250cafbc zipnum: add query arg to location resolver 2015-05-29 12:52:35 -07:00
Ilya Kreymer
c8980c3f8f query_handler: pass wbrequest.coll as 'coll' param to cdx query automatically 2015-05-29 11:51:16 -07:00
Ilya Kreymer
f26f74ec84 bump version to 0.10.0 2015-05-29 11:48:50 -07:00
Ilya Kreymer
a51b2936f3 zipnum: fix bug with urls in last block not being accessible. when iter_range() fails, if check to see if last_line == end_line, 
and if so, check if start_line should also be end_line #112
support non-linenumbered idx files w/o pagination queries
add new zipnum-sample to test cdx lines in last block (previous sample had only one line in last block except the first)
 2015-05-29 11:46:00 -07:00
Ilya Kreymer
d104c03135 wombat: check coll prefix w/o mod or timestamp 2015-05-26 18:27:35 -07:00
Ilya Kreymer
07d6031d3e wombat: check for dropped collection and add back to avoid refer-relative redirect check on server 2015-05-26 18:16:17 -07:00
Ilya Kreymer
c4dad56681 rules: add custom js for resizing poster on twitter video images 2015-05-26 15:15:11 -07:00
Ilya Kreymer
ce8da00b89 wombat: wteak history override to be more consistent 
add exported 'watch_elem' func to be used by rules for custom ops
 2015-05-26 15:14:12 -07:00
Ilya Kreymer
ee20ac66d6 rules: tw video player rules, disable rewriting 
rewrite: tweak location rule
wombat: add getAttribute() override, but disabled for now
store default getAttribute()/setAttribute() to refer internally
 2015-05-25 17:52:03 -07:00
Ilya Kreymer
6c97fe1d44 vidrw: support livestream playlist, support for generic extractor 2015-05-25 17:48:25 -07:00
Ilya Kreymer
0a606ce558 cdxindexing: store arbitrary json metadata from WARC-Json-Metadata field (experimental) 2015-05-24 20:17:10 -07:00
Ilya Kreymer
b1c9503a9d rewrite: insert head-insert after <html>, <head> and before any other tags (if head is missing) 
previously was being inserted after other head tags #109
 2015-05-24 20:17:10 -07:00
Ilya Kreymer
af37b99e80 wombat: additional fixes/testing: for about:blank/empty iframes, initialize WB_wombat_location, document.WB_wombat_location and WB_wombat_top immediately. 
disable redundant rewrites
vidrw: check for null parent node, fix bug with double-add! don't rewrite added elements
 2015-05-24 20:17:04 -07:00
Ilya Kreymer
adb9448f27 rules: improved rules for googleplus! 2015-05-22 18:45:50 -07:00
Ilya Kreymer
d5b92dbb3c rules: update rules for yt comments 2015-05-21 17:20:40 -07:00
Ilya Kreymer
179f11198b fuzzy match: look at first occurence, not last of match seperator 
rules: add new rule for yt comments
 2015-05-21 23:52:09 +00:00
Ilya Kreymer
35e2e535bb def banner: ensure banner element isn't rewritten! 2015-05-21 12:24:16 -07:00
Ilya Kreymer
a929e96433 wombat: add rewrite_elem() back to main init_dom_override(), check if already overwritten 
createElement(): add optional skip arg
 2015-05-21 12:11:08 -07:00
Ilya Kreymer
b7e27ba1a8 tests: update tests for keeping scheme-relative, scheme-relative after rewrite #101 
remove tests for document.cookie, document.referrer and document.domain rewrite, as this is now handled client-side
 2015-05-21 11:38:06 -07:00
Ilya Kreymer
690106bcb4 wombat: more refactoring! enable http/src observer by default, add skip_createElement override 
implement document.cookie, document.referrer and document.domain as property overrides instead of WB_wombat rewrites
when a new iframe is loaded, ensure the *document* is reinited with wombat, even if window already has wombat settings
 2015-05-21 11:26:54 -07:00
Ilya Kreymer
4983bf4425 rewrite: keep relative scheme after all, to work where with: scheme + "//..." constructions, #101 2015-05-21 11:26:54 -07:00
Ilya Kreymer
9912a31523 wombat: add prototype-level override for innerHTML and outerHTML 2015-05-21 11:26:54 -07:00
Ilya Kreymer
4e1be5c275 wombat work: add createElement() override, use current protocol instead of original url protocol 
refactor init_dom_override() to only check children for fragments, add innerHTML override
 2015-05-21 11:26:54 -07:00
Ilya Kreymer
058b25ec5a wombat: test with href overrides 2015-05-21 11:26:54 -07:00
Ilya Kreymer
c5a5d45a58 wombat: experimenting with initializing wombat on iframe init directly, rather than waiting for injected init... 2015-05-21 11:26:54 -07:00