1
0
mirror of https://github.com/webrecorder/pywb.git synced 2025-04-01 11:41:27 +02:00

908 Commits

Author SHA1 Message Date
Ilya Kreymer
79d5ec2b2d statusheaders: when not verifying protocol line, avoid indexerror when no space in first line, add tests 2015-12-18 21:46:00 -08:00
Ilya Kreymer
f6a4f80e47 bump to 0.11.1 2015-12-18 21:27:03 -08:00
Ilya Kreymer
79b35a64b3 bump wombat version to 2.10 2015-12-13 00:31:01 -08:00
Ilya Kreymer
381f350917 proxy: switching not available for ip resolver either
tests: update tests for auth and ip resolver to check that proxy magic is not set
2015-12-12 22:59:32 -08:00
Ilya Kreymer
e9b11fcbf2 proxy: default to cookie resolver, which allows switching collections and datetime, instead of auth resolver
auth resolver can be used by setting 'cookie_resolver: false' explicitly. when using auth resolver,
don't set proxy magic path as switching collections or datetime is not possible with auth resolver
closes #160
2015-12-12 21:58:12 -08:00
Ilya Kreymer
64fe49d703 wombat: add CSSStyleSheet.prototype href override
karma tests: change name to just 'pywb'
2015-12-11 17:33:22 -08:00
Ilya Kreymer
467ea0f68b Merge branch 'client-side-tests' into develop 2015-12-10 12:50:40 -08:00
Ilya Kreymer
9b8b4d8388 tests/typo fix: add tests for truncated record detection (see: ikreymer/webarchiveplayer#14) fix typo, closes #161 2015-12-10 12:31:58 -08:00
Robert Knight
83a33e0541 Resolve relative canonical paths if rewriting is disabled
For Via, we want rel=canonical links to resolve to the same
absolute URL as it did on the original page.

For absolute URLs, no rewriting is necessary. If the original
rel=canonical URL was relative however, it needs to be resolved
relative to the original URL.

See https://github.com/hypothesis/via/issues/65 for context.
2015-12-10 08:31:50 +00:00
Robert Knight
34721a6742 Override HTMLAnchorElement.prototype.toString()
The values returned by getting the 'href' and 'toString'
properties of an anchor element should be the same.

This inconsistency broke the URL polyfill in
https://github.com/inexorabletash/polyfill under Microsoft Edge
2015-12-02 20:09:09 +00:00
Ilya Kreymer
7a0680fb35 memento: for not found timemap query, return empty timemap, instead of html query error page, closes #158 2015-11-30 09:40:07 -08:00
Robert Knight
977647cea9 Override the baseURI property on the correct object
baseURI is a property of Node.prototype
in current browsers and should be overridden there,
except for Safari where it cannot be overridden at all.
2015-11-26 17:09:26 +00:00
Robert Knight
1484b06da6 Avoid changing the writability of the 'href' attr of <base>
Wombat overrides document.baseURI and <base>.href in order
to return the original URL rather than the proxied URL.

The <base>.href override however ended up making a writable
attribute read-only, which could trigger script errors
in strict-mode JS.

Fix this by avoiding replacing the setter for a DOM property
if no replacement setter is provided.

Fixes an error loading Hypothesis under Microsoft Edge.
2015-11-26 13:54:47 +00:00
Robert Knight
f2fdbcc511 Lookup baseURI getter on Node.prototype, not Node
Fix regression in baseURI override, spotted by the Karma
tests.

The getter should be looked up on Node.prototype, not Node.
2015-11-26 10:00:09 +00:00
Ilya Kreymer
cd2390333a Merge pull request #153 from robertknight/safari-dom-fixes
Fix Wombat in Safari 9
2015-11-26 01:15:30 -08:00
Ilya Kreymer
4303ce4ecb warc indexing: better handling of records with content-length to small, read first line to get to warc end (fixes indexing of warc in ikreymer/webarchiveplayer#14) 2015-11-26 00:47:15 -08:00
Ilya Kreymer
2922801b7c test: rewrite_live: pass Accepnt-Encoding: identity to disable gzip, simplified version of fix in #151 2015-11-26 00:34:45 -08:00
Robert Knight
5208117299 Fix Wombat in Safari 9
In Safari 9, Object.getOwnPropertyDescriptor(domObject.prototype, prop)
returns descriptors which have the correct structure but undefined
getters and are marked as unconfigurable.

See https://bugs.webkit.org/show_bug.cgi?id=49739#c19
for details. The getters cannot be retrieved via obj.__lookupGetter__()
either.

 * Resolve the issue by skipping overrides for DOM properties
   where the property is not configurable, or the original getter for
   a property could not be retrieved.

 * Lookup the 'baseURI' property on the correct prototype (Node,
   not document)

This fix also resolves the problem where accesses to document.baseURI
on Edge would fail.
2015-11-25 16:52:11 +00:00
Ilya Kreymer
d98c1f6cf7 memento/api: add a new /collinfo.json end-point, enabled with 'enable_coll_info' config setting, which returns
the value fo collinfo.json template. Default template returns an entry for each handler route,
including the route path (id), title (name) and memento timegate and timemap paths, to be used with
an aggregator. Using a custom 'info_json' template can specify a different collinfo template, alternative to #69 (local aggregation)
Closes #146
2015-11-04 15:36:44 -08:00
Ilya Kreymer
dc74b14af0 iframe scrolling: use webkit scrollbars for better scrolling iframes due to osx chrome iframe bug
scrollbar now fully functional, though some artifacts still appear #93
2015-10-31 12:06:01 -07:00
Ilya Kreymer
3132bfa7f4 cache: add a simple RedisCache implementation (alongside local and uwsgi)
proxy_ip_resolver: add option to use RedisCache if redis_cache_key set in config
proxy_ip_resolver: add 'delete' option to delete ip from cache, closes #145
2015-10-30 13:15:07 -07:00
Ilya Kreymer
16cf997a07 proxy: stick with http1.0 as not really supporting 1.1, #143 2015-10-26 15:25:00 -07:00
Ilya Kreymer
eeb35ea3b4 proxy: add ProxyRouter wrapper to check for content-length and, if missing, perform full buffering (http1.0) or chunked encoding (http1.1) (separate from replay view buffering)
add tests for buffering and chunked encoding, fixes #143, also tests no banner url-rewrite only proxy related to #142
2015-10-25 18:02:51 -07:00
Ilya Kreymer
0c96591c49 proxy: change HttpsUrlRewriter to SchemeOnlyUrlRewriter, which fixes http->https or https->http to match
the scheme of the current page.
url-rewrite-only mode: add uo_ mod and use that to rewrite only urls (no banner, no client side rewrite)
addresses #142
2015-10-24 15:10:30 -07:00
Ilya Kreymer
979fcaeda3 tests: fix mock YoutubeDLWrapper after refactor, #141 2015-10-23 12:19:15 -07:00
Ilya Kreymer
39e824cb3a live rewite proxy: decouple having http/https proxy from recording,
move youtubedl wrapper calls, metadata add calls to live rewrite proxy class for easier extension
closes #141 also improves #136
2015-10-23 11:57:12 -07:00
Ilya Kreymer
4ba4521b56 tests: use random port instead of 8080 for cli test to avoid conflicts with running services 2015-10-23 11:53:28 -07:00
Ilya Kreymer
e37636de84 cdxindexer: if latest ujson (with forward slash not-escaping) is available, use that when indexing, closes #140
tests: update indexer CDXJ tests to be order-independent
travis: install ujson for testing
2015-10-22 17:46:05 -07:00
Ilya Kreymer
c003a96618 Merge pull request #139 from jcushman/webencodings
Use webencodings to encode head_insert_str.
2015-10-22 16:23:47 -07:00
Ilya Kreymer
3318402fc3 bump version to 0.11.0 2015-10-22 15:40:36 -07:00
Jack Cushman
633eb31f57 Use webencodings to encode head_insert_str. 2015-10-22 16:40:59 -04:00
Ilya Kreymer
b8b473bf19 cdxindexer: use ujson if it is available 2015-10-21 15:28:26 -07:00
Ilya Kreymer
f3c87c1786 wombat: additional checks for cross-frame top frame access, #137 2015-10-21 10:54:48 -07:00
Ilya Kreymer
9e1447c448 rewrite: strip spaces when rewriting urls in html, closes #134 2015-10-20 12:59:07 -07:00
Ilya Kreymer
75085ad91b loaders: fix loader inits, don't inherit from BlockLoader #135 2015-10-20 10:33:24 -07:00
Ilya Kreymer
94095e452a loaders: refactor BlockLoader to use an extensible dict of loaders
individual HttpLoader, LocalFileLoader and S3Loader supported by default
Loaders created via BlockLoader also cached for reuse, closes #135
2015-10-19 11:59:35 -07:00
Ilya Kreymer
4ea8e5d535 rewrite: Make LiveRewriter class overridable in config, request method extensible in LiveRewriter, closes #136 2015-10-19 11:46:27 -07:00
Ilya Kreymer
8b68119994 wombat: use wb_rel_prefix instead of wbinfo.coll for rel prefix path 2015-10-17 08:53:01 -07:00
Ilya Kreymer
93045fb39f rules: fuzzy rule for fastly.. 2015-10-16 09:43:22 -07:00
Ilya Kreymer
06999791de warc: when iterating over WARC, don't stop after first empty gzip records, unless really at the end..
add test for post + minimal error
2015-10-16 08:48:59 -07:00
Ilya Kreymer
925b23f8a8 rewrite: guard against invalid encoding in html charset= and default to utf-8 if specified encoding fails, related to hypothesis/via#53 2015-10-15 13:58:15 -07:00
Ilya Kreymer
31dd946114 live rewrite: redir query requests to latest replay in live rewrite handler 2015-10-14 20:33:59 -07:00
Ilya Kreymer
42091fd36e wombat: fix typo which caused rewrite_style() to be skipped unless style started with url prefix! 2015-10-14 17:36:33 -07:00
Ilya Kreymer
e2b0f5c9cb bump version to 0.10.10 2015-10-14 17:35:17 -07:00
Ilya Kreymer
b612c584de tests: test fixes for windows 2015-10-13 21:36:27 -07:00
Ilya Kreymer
d39ac71531 bump version to 0.10.9.1 for wombat scheme-rel url fix 2015-10-12 22:18:19 -07:00
Ilya Kreymer
edbfd500a4 wombat: fix rel '/' rewrite which incorrectly handles rel scheme '//' urls 2015-10-12 22:13:26 -07:00
Ilya Kreymer
6efff4cd8f rules: cleanup, remove obsolete rules 2015-10-11 23:50:38 -07:00
Ilya Kreymer
8b7516708d wombat: improved rewrite when running at root: fixes for extract_orig and /-rel rewrite
move init to be before proxy check
2015-10-11 23:16:15 -07:00
Ilya Kreymer
4dfe187174 proxy improvements:
use proxy_magic path to get video info to ensure video info, addresses #106
video info: ensure vi_ replay has CORS support to support serving from magic path
proxy & wombat improvements: set replay_top to window.top and avoid causing cross-domain errors
2015-10-11 21:03:30 -07:00