Ilya Kreymer
a0878e6998
tests: fix regex for 2.6, fix live example
2015-10-07 11:59:31 -07:00
Ilya Kreymer
c3aab1514c
query/cdx: support from and to cdx query arguments, support ranged calendar query,
...
eg. /[from]*[to]/[url] or /[from]-[to]/[url], with both from and to optional, closes #130
exposes lower and upper bound timestamps in timeutils, pad_timestamp
2015-10-07 10:44:12 -07:00
Ilya Kreymer
78ba91ff63
rewrite: fixes for attr override, add test for 'integrity' override
2015-10-05 23:26:47 -07:00
Ilya Kreymer
9a19265786
rewrite: disable 'integrity' (as well as crossorigin) attr due to rewriting, which modifies the resource
2015-10-05 23:04:56 -07:00
Ilya Kreymer
cb67c172ed
rewrite: html rewriter can accept optional url for initial base url of page
2015-10-02 14:01:33 -07:00
Ilya Kreymer
e9c4c2d4db
rewrite: add an explicit <base> if trailing slash missing after host. Ensure when existing <base> is rewritten,
...
a trailing slash is added (before query or anchor) closes #129
2015-09-14 19:22:49 -07:00
Ilya Kreymer
eb2162a477
rewrite: attr unescape attr entities only if url starts with 'http', use unescaped if any exceptions escaping
2015-09-10 16:03:00 -07:00
Ilya Kreymer
8aa3ea7d3d
rewrite: rewrite long-deprecated "background" element in all elems
2015-09-03 15:32:07 -07:00
Ilya Kreymer
a4d10ca562
rewrite: ensure entity-encoded urls in attrs are decoded before url rewriting
2015-09-02 05:13:56 -07:00
Ilya Kreymer
3ef5d98bc6
bump to 0.10.7-dev
...
wombat: use Document.prototpe for cookie getter and setter if not found on
document object, add no-op setter for document.domain
tests: fix test to avoid including line ending
2015-08-09 09:25:02 -07:00
Ilya Kreymer
2b3dbd12e2
rewrite cookie test: disable secure and httponly test as 2.6 doesn't handle these!
2015-08-07 19:36:07 -07:00
Ilya Kreymer
b532056c6e
cookie_rewriter: ensure cookie paths are always relative
...
cookie_rewriter tests: add cookie rewriter tests for secure, httponly
html_rewriter tests: add <base> rel and abs rewrite tests
no cover for waitress as its not used by default
2015-08-07 19:36:07 -07:00
Ilya Kreymer
6bf6a02868
tests: add explicit 'js_rewrite_location: all' rule for testing all-rewrite (as not default anymore)
2015-08-07 12:02:48 -07:00
Ilya Kreymer
a3c8698cc3
rewrite: disable server-side url rewriting in JS by default! now handled by client-side rewriting
2015-08-07 11:37:43 -07:00
Ilya Kreymer
eeff79461a
rewrite: allow '\' in JS url host part (for escaped slashes)
...
tests: update test to reflect full 'top' rewriting
2015-08-05 11:58:44 -07:00
Ilya Kreymer
ef9fa9ec5c
rewrite: don't assume window.top is the top replay frame, refactor to find top replay frame (window.__WB_replay_top) and top frame window.__WB_top_frame, for framed mode)
...
make top -> WB_wombat_top rewriting more general, use Object property override to return __WB_replay_top or default to regular top if not window
fixes #125
2015-08-05 10:10:10 -07:00
Ilya Kreymer
191ebe1132
url_rewriter: check for rel_prefix when 'rebasing' url_rewriter #123
2015-07-31 01:57:17 -07:00
Ilya Kreymer
a2a2a054a0
rewrite: rewrite rel urls to rel urls, both server and wombat side #123
2015-07-31 01:57:17 -07:00
Ilya Kreymer
43716f5f1d
rewrite: add HostScopeCookieRewriter and make it default -- set cookie scope to rewritten host
...
wombat: ensure rewrite html is using a string
bump to 0.10.5!
2015-07-30 23:06:34 -07:00
Ilya Kreymer
cee3c8cb61
new wombat! refactor of rewriting:
...
- use defineProperty overrides on element prototypes
- postMessage() rework: store actual origin with helper function __WB_pmw(window), from
server side rewrite
- Use window.URL (or external jsurl script) to override all properties of HTMLAnchorElement,
override getAttribute() to return original
- rename window -> $wbwindow
2015-07-30 15:10:13 -07:00
Ilya Kreymer
0b4ceb9cde
rewrite: if removing content-encoding, also remove the content-length as it will need to be recomputed!
...
proxy: for proxy mode, must buffer fully so that content-length can be added (may add chunked encoding later)
2015-07-28 14:23:50 -07:00
Ilya Kreymer
dcbe32b742
regex rewrite: don't match quoted location for rewrite
2015-07-21 11:46:19 -07:00
Ilya Kreymer
0f4d779b00
html rewrite: add 'force_html_decl' option, which if set in rewrite_opts, can be used to force an HTML decl, eg. <!DOCTYPE html> if a default one was not provided
2015-07-18 18:23:57 -07:00
Ilya Kreymer
9b08ca9005
vidrw: ensure iframe replacement does get rewritten!
...
regex rewrite: include '==top?' for wombat rewrite
rewrite css: if js_ modifier on text/css, treat as css
2015-07-18 12:59:20 -07:00
Ilya Kreymer
53b86798c7
url_rewrite: don't rewrite urls starting with '{'
2015-07-07 22:39:24 -07:00
Ilya Kreymer
c38d0c2aaf
html attr rewrite: rewrite any " -> " in html attributes
2015-07-07 14:43:15 -07:00
Ilya Kreymer
93d49ae24b
rewrite deprefix: improve query deprefix to also test url-encoded params, closes #119
2015-07-06 18:19:01 -07:00
Ilya Kreymer
76ebabd3f2
rewrite rules: none or location-only rule should apply to json as well
2015-07-03 11:53:29 -07:00
Ilya Kreymer
06fcc89de6
readers: support 'content-encoding: deflate' using different zlib decompression options
...
support default and alt settings for attempting to decompress deflate stream
tests: add tests with httpbin.org/deflate Fixes #115
2015-06-24 13:11:33 -07:00
Ilya Kreymer
dfeccbe62d
tests: header rewriter tests for custom cache header settings, #110
2015-06-05 17:48:05 -07:00
Ilya Kreymer
15c2ddbfcf
header rewriter: cache options: use 'rewrite_opts.http_cache' to set caching headers options, #110
...
'pass': passthrough original cache headers unrewritten
None (default): rewrite cache headers and don't add anything else
N: set cache-control max-age: N and corresponding expires
N=0: set cache-control: no-cache; no-store
2015-05-29 12:53:29 -07:00
Ilya Kreymer
ee20ac66d6
rules: tw video player rules, disable rewriting
...
rewrite: tweak location rule
wombat: add getAttribute() override, but disabled for now
store default getAttribute()/setAttribute() to refer internally
2015-05-25 17:52:03 -07:00
Ilya Kreymer
b1c9503a9d
rewrite: insert head-insert after <html>, <head> and before any other tags (if head is missing)
...
previously was being inserted after other head tags #109
2015-05-24 20:17:10 -07:00
Ilya Kreymer
b7e27ba1a8
tests: update tests for keeping scheme-relative, scheme-relative after rewrite #101
...
remove tests for document.cookie, document.referrer and document.domain rewrite, as this is now handled client-side
2015-05-21 11:38:06 -07:00
Ilya Kreymer
690106bcb4
wombat: more refactoring! enable http/src observer by default, add skip_createElement override
...
implement document.cookie, document.referrer and document.domain as property overrides instead of WB_wombat rewrites
when a new iframe is loaded, ensure the *document* is reinited with wombat, even if window already has wombat settings
2015-05-21 11:26:54 -07:00
Ilya Kreymer
4983bf4425
rewrite: keep relative scheme after all, to work where with: scheme + "//..." constructions, #101
2015-05-21 11:26:54 -07:00
Ilya Kreymer
0223ac0489
rewrite: top rewrite: avoid rewriting 'top('
2015-05-14 22:32:10 -07:00
Ilya Kreymer
e94b239d84
rewrite: when rewriting scheme-relative urls, if adding an absolute prefix, use the scheme of the prefix
...
otherwise, keep relative scheme #101
2015-05-14 22:32:09 -07:00
Ilya Kreymer
1b9ef4e325
html_rewriter: handle parse_comments by rewriting as html, not as js, should address ikreymer/pywb-webrecorder#7
2015-05-14 22:32:09 -07:00
Ilya Kreymer
99ff29e283
js regex rewrite: scheme-rel rewrite must be preceded by a quote no semicolon, to avoid rewriting ;//comment; as url
...
add rewrite tests
2015-05-14 22:32:08 -07:00
Ilya Kreymer
33f247582f
rewrite: HTMLRewriter should insert head_insert at end of stream, if it hasn't
...
been inserted by the end (and if there was some content written -- don't insert for 0-length responses)
Addresses missing head insert if only head tags are present and no head, as per hypothesis/via#9
2015-05-14 22:32:08 -07:00
Ilya Kreymer
fbb9de264a
rewrite: cookie: add 'removeall' cookie rewriter which removes all the cookies!
2015-04-14 18:04:51 -07:00
Ilya Kreymer
756b80cac4
rewrite: remove urlrewriter optimization which uses wburl.original_url, as its
...
not realiable (%-encoding, no trailing slash issues) and not really needed.. rename original_url -> _original_url, should not be used
directly.
add test for rewriting when base url has no trailing slash
2015-04-13 12:51:38 -07:00
Ilya Kreymer
9f1ae42d1c
rewrite: security: by default, disable file:// and local access unless explicitly allowed!
...
enable for tests with remote_only=False param
2015-04-13 09:51:12 -07:00
Ilya Kreymer
b21e288063
wburl to_uri: when parsing scheme, also for first '?' in addition to '/' to catch any irregular urls (to be ignored)
2015-04-11 10:21:52 -07:00
Ilya Kreymer
4bf1dd4cad
urlrewriter: simply startswith check by using tuple
2015-04-10 14:46:46 -07:00
Ilya Kreymer
1286e9da4c
wburl test: better test for bad domain
2015-04-04 17:31:38 -07:00
Ilya Kreymer
1844597889
wburl to_uri: catch error on idna encode with very invalid urls
2015-04-04 12:51:49 -07:00
Ilya Kreymer
bd21480db9
framed replay: add supporting for 'inverting' frame and replay modifiers,
...
setting default mod to be top-frame and inner frame to be 'mp_' #92
can enable this mode by setting framed_replay: inverse instead of true
modifiers passed to client side script via wbinfo as well
2015-04-01 10:13:56 -07:00
Ilya Kreymer
c378cb5188
rewrite: check for closed before any use of readline() (2.6 may throw if closed),
...
only use readline() if line alignment needed (non-html), related to #86 work
2015-04-01 07:54:17 -07:00
