backup/pywb
1
0
Fork 0
mirror of https://github.com/webrecorder/pywb.git synced 2025-03-20 02:39:13 +01:00
Code Issues Projects Releases Activity
792 Commits 56 Branches 63 Tags
Commit Graph

581 Commits

Author SHA1 Message Date
Ilya Kreymer
78812c8085 rewrite: more conservative change, only rewrite the X-Forwarded-Proto 
header for now, #57
 2015-02-04 15:17:23 -08:00
Ilya Kreymer
cdb3dcc3d2 rewrite_live: don't forward via or https_x headers, only standard (for 
now) possible fix for #57
 2015-02-04 14:19:37 -08:00
Ilya Kreymer
40fba3c27b cdx-indexer: minor cleanup, add custom writer override to 
write_multi_cdx_index
 2015-02-04 11:17:26 -08:00
Ilya Kreymer
c47d3ca925 wombat: add mutation observers, addressing #71 and maybe #67 
rules: fix regex for yt, add rx for wikimedia
 2015-02-03 11:19:41 -08:00
Ilya Kreymer
734ee4471b frame ui: pass timestamp to frame banner, fix typo in html 
banner: allow overriding of banner id by returning custom id
 2015-02-02 09:41:49 -08:00
Ilya Kreymer
29c6a36dac cdx api query: pass query timestamp mod to index query via 'query_closest' 
field, to avoid confusion with 'closest'
 2015-01-31 17:45:46 -08:00
Ilya Kreymer
55426e7619 memento: fix headers to be more consistent for framed replay. when using 
frames, outer frames 'mirrors' mementos of the inner frame to be
discoverable by client side memento tools, tracked via #70
 2015-01-29 22:27:15 -08:00
Ilya Kreymer
757345d317 replay api: make ReplayView overridable in WBHandler subclass, 
allow custom content loader callable
 2015-01-29 20:10:41 -08:00
Ilya Kreymer
7e017fd85e rewrite fixes: don't rewrite window.parent as it is overridable directly 
html rewriter: ensure style is rewritten for all elements, add test!
wombat: cleanup and additional checks for assign(), setAttribute()
 2015-01-29 20:08:00 -08:00
Ilya Kreymer
043ad5c860 wombat: improve createElementNS override to set prototype, just assign 
window.parent directly
 2015-01-29 10:13:32 -08:00
Ilya Kreymer
bf3d256a51 rewrite: add css-in-js rewrite rule for wikimedia, tracking via #67 for 
perhaps a more general solution
 2015-01-28 09:20:42 -08:00
Ilya Kreymer
ccedb2d60e regex_rewrite: add 'parent' rewrite in addition to 'top' for frames, add 
WB_wombat_parent to wombat, add test for WB_wombat_parent
 2015-01-27 19:57:56 -08:00
Ilya Kreymer
976decb3f1 wombat: ensure document.write override handles elements that go into 
head as well as body
 2015-01-27 18:02:14 -08:00
Ilya Kreymer
695245d9e8 wburl idn: more complete support for idn urls (#66) 
add distinct to_iri() and to_uri() functions in WbUrl
internal representation is always as ascii uri
for rewriting, defaults to iri representation unless
'rewrite_ascii_only_urls' is set to true per collection
add wbrequest.get_url() to get url as either iri or uri to be passed
to templates
 2015-01-26 11:07:59 -08:00
Ilya Kreymer
edff3f17fb wburl: convert %-encoded hostnames or unicode urls to punycode for 
better IDN support (#66)
 2015-01-26 11:07:58 -08:00
Ilya Kreymer
38e3bbbaef templates: add new 'not_found.html' template, which will be called for any missing replay request 
instead of default error.html
'not_found_html' settable in the config per collection, as per #65
for not found index query, still use query.html but add condition to check for 0 results
add more query and replay not found
remove unused conditional (for search_view -- always exists)
 2015-01-24 12:32:50 -08:00
Ilya Kreymer
80fd47ba3e add rules for vine (#62) 2015-01-22 16:45:09 -05:00
Ilya Kreymer
c9b2e3e69e wombat 2.2 improvements: 
* for postMessage, add receive message overrides which uses original origin
to fix message passing tests that check for origin

* for createElementNS, ensure that the namespace url is not rewritten

* add equals_any() method, add "poster" attr to attr rewriting list

(solves several issues for vine replay, #62)
 2015-01-22 16:43:52 -05:00
Ilya Kreymer
48b7751f80 bump version to 0.7.6 
jinja2: allow adding multiple packages to search path
 2015-01-19 21:54:11 -05:00
Ilya Kreymer
43805c67ef view: fix format_ts, use existing utc timestamp_to_sec conversion for %s 2015-01-12 00:28:06 -08:00
Ilya Kreymer
ac525b0937 tests: add tests for extract_post_query() 
add test for HttpsUrlRewriter, remove unnecessary check in
bufferedreader
 2015-01-11 23:54:29 -08:00
Ilya Kreymer
8449647c5f wbexception: remove unused status in WbException, set default error for 
any uncaught exception to 500, instead of 400
 2015-01-11 23:53:34 -08:00
Ilya Kreymer
7610d9deb7 views: cleanup view filters, remove obsolete, add tests for format_ts 
and is_wb_handler
 2015-01-11 23:02:48 -08:00
Ilya Kreymer
db75bda736 file open() pass: convert all read and write to ensure binary 'b' flag is set (#56) 2015-01-11 18:54:11 -08:00
Ilya Kreymer
fb4bf817f7 rangecache: use 'b' for file open 2015-01-11 18:34:32 -08:00
Ilya Kreymer
14657fbe15 certauth: fix max cert duration to avoid int overflow 2015-01-11 15:04:19 -08:00
Ilya Kreymer
7ae0ff86d2 test certauth: fix paths 2015-01-11 13:10:14 -08:00
Ilya Kreymer
cf0a21509b loaders: add to_file_url() for converting between filename and file://, 
used in live rewrite and tests
 2015-01-11 13:05:48 -08:00
Ilya Kreymer
ba853a4eae fixes for windows: convert url to file with pathname2url, use 'b' for 
reading warcs, don't use %s for timestamp conversion (not portable)
(#56)
 2015-01-10 20:59:23 -08:00
Ilya Kreymer
7f52ecdca9 tests: fix indexing test, remove extra space/print 2015-01-10 15:36:53 -08:00
Ilya Kreymer
d5c22e3649 test loaders: fix file:// prefix 2015-01-10 15:27:45 -08:00
Ilya Kreymer
1eb0f96f92 windows support work: fix loaders to use pathname2url to convert to 
file:/// url, use urlopen to open file paths
fix some tests to use universal line breaks
 2015-01-10 14:06:15 -08:00
Ilya Kreymer
06b2ea498e setup: set zipSafe to true to allow bundling egg in installer 2015-01-09 22:57:21 -08:00
Ilya Kreymer
205aeca4a1 bump version to 0.7.3 
rewrite: add additional tags for client side src rewrite, add missing
tags to server-side html rewrite
 2015-01-04 17:32:58 -08:00
Ilya Kreymer
d9c5345d3c rewrite: add support for Cookie request header rewrite to support sites 
which require a cookie to be set. req_cookie_rewrite directive can be
set in rules.yaml per url prefix with a list of match/replace regexs
 2015-01-03 12:51:09 -08:00
Ilya Kreymer
df94e17305 video: yt: switch on timeout only if not started.. 2014-12-28 23:29:13 -08:00
Ilya Kreymer
1684c14cda bump version to 0.7.2 
video: disable yt DASH for better proxy and replay (experiment)
 2014-12-28 16:34:48 -08:00
Ilya Kreymer
a76bf79b83 html_rewriter: add explicit <video>, <audio> tags to html_rewriter tag 
list
 2014-12-26 18:15:49 -08:00
Ilya Kreymer
8d6845a552 fuzzy match: add support for specifying regex and args seperately for 
fuzzy_lookup match
 2014-12-26 14:29:51 -08:00
Ilya Kreymer
ffb702ce03 rewrite: content detection for specific case: if content type is html and mod type is css 
or js, peek stream to determine actual type. Addresses #31 in part.
Fix typo in wb_frame.js
 2014-12-26 13:08:35 -08:00
Ilya Kreymer
8f57ce622d Improved top rewriting, addressing #54 2014-12-26 13:06:33 -08:00
Ilya Kreymer
4c08a6a064 video work: improved yt handling: 
- disable yt using yt api, for forced html/flash, diable on load
- use yt error event to detect error
- better fallback on recorded video
use seperate cache for range and video info tracking
fix yt rules query to account for & and ?
 2014-12-26 13:02:47 -08:00
Ilya Kreymer
ca17410056 video: better extension extraction for explicit video info (use format 
if ext is unknown)
improved support for _pywbvid=html, forces generic html player over yt
player, better deletion of ytplayer
 2014-12-24 18:13:12 -08:00
Ilya Kreymer
ad5a43db76 replay redirect: ensure no timestamp redirect when range request is 
present, alter test to include inexact timestamp
 2014-12-23 21:19:39 -08:00
Ilya Kreymer
181c18a1b8 pep8 pass: fix spacing, line length, issues 
also remove references to obsolete cached_replay, hostnames in pywb_init
 2014-12-23 15:14:03 -08:00
Ilya Kreymer
51919ed1e7 replay: make range cache available by default in replay_views since its 
inited on first use. remove
separate subclass. 'enable_ranges' can be set to false to disable range
cache altogether
improve tests
 2014-12-23 14:34:59 -08:00
Ilya Kreymer
c28304fd90 tests rangecache: added integration tests for range support via range 
cache, using enable_cache option
 2014-12-23 11:09:19 -08:00
Ilya Kreymer
a9892f531f proxy testing: refactored test server thread into ServerThreadRunner 
class which runs a server in a seperate thread.. used by http/s proxies
as well, as mock live server proxy
add test for live rewrite with proxy, covering simple case as well as
video
 2014-12-23 11:07:47 -08:00
Ilya Kreymer
a8b4041716 live rewrite: proxy setup refactor: ignore_proxy flag, pass proxy during constructor only 2014-12-22 21:58:07 -08:00
Ilya Kreymer
b54e4c1c06 tests: add more tests for cookie, html and rewrite_live crsf 2014-12-22 20:34:18 -08:00