backup/pywb
1
0
Fork 0
mirror of https://github.com/webrecorder/pywb.git synced 2025-03-15 16:14:48 +01:00
Code Issues Projects Releases Activity
1,584 Commits 56 Branches 63 Tags
Commit Graph

1584 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Ilya Kreymer
896f81fd1c Add README.rst 2016-03-03 12:09:17 -08:00
Ilya Kreymer
ed1d3555c3 rename rezag -> webagg 
rename aggindexsource -> aggregator
 2016-03-03 11:55:43 -08:00
Ilya Kreymer
98830147b5 add memento headers to all response loaders, use BaseLoader base class, update tests 
for memento headers
 2016-03-03 11:04:28 -08:00
Ilya Kreymer
65e969a492 errors and timeouts reported back to the user via ResErrors header 
add new /index, /resource access point system
 2016-03-02 18:13:13 -08:00
Ilya Kreymer
1f3763d02c misc fixes: add route listing, more not found tests, timemap use file:// with ranges 2016-03-01 14:46:05 -08:00
Ilya Kreymer
008e5284b1 seperate iter_sources from list_sources api 
all errors returned as json block with error msg
tests for not found, invalid errors
 2016-02-29 12:34:06 -08:00
Ilya Kreymer
68090d00c1 add routing setup via app.py 
add full test suite for handlers and responseloaders, as well as timeouts
 2016-02-28 14:33:08 -08:00
Ilya Kreymer
c88c5f4cca add new package setup! 
add tests and testdata, splitting mem and dir agg tests
 2016-02-26 18:25:10 -08:00
Ilya Kreymer
fc5d7cc7cd rewrite: add rewriting of <meta> content="" attribute if it is a url 2016-02-25 18:49:31 -08:00
Ilya Kreymer
8fc789cc8f rewrite: leave out charset in top-frame and don't modify it in replay frame 
to allow browser to detect best charset, as it would on original page if it is absent)
see #170 for details
 2016-02-25 18:25:53 -08:00
Ilya Kreymer
c76aa17b78 wb.js: pad timestamp to 14 digits 2016-02-25 18:25:28 -08:00
Ilya Kreymer
e6361c58ac bump version to 0.11.2 2016-02-25 18:15:29 -08:00
Ilya Kreymer
398e8f1a77 inputrequest: add input request handling (direct wsgi headers) or as a prepared post request 
add timemap link output
rename source_name -> source
 2016-02-24 14:22:29 -08:00
Ilya Kreymer
20bd9d118b travis: remove --use-mirrors 2016-02-23 18:39:27 -08:00
Ilya Kreymer
1d5b23413f proxy: ensure proxy cert download sets content length 
proxy options: 'use_default_coll' must specify exact default coll
(otherwise a random coll is chosen, as ordering is not defined)
travis: add py3.4, py3.5!
 2016-02-23 18:09:09 -08:00
Ilya Kreymer
cebd6b6239 rewrite: fix rewriting encoding -- for best rewriting, keep strategy of encoding 
insert to match page, then using latin-1 for rewriting. support for non-ascii
based encoding still needed
 2016-02-23 18:07:34 -08:00
Ilya Kreymer
3a584a1ec3 py3: all tests pass, at last! 
but not yet py2... need to resolve encoding in rewriting issues
 2016-02-23 13:26:53 -08:00
Ilya Kreymer
0dff388e4e cdx: CDXQuery takes params dict not **params 
CDXObject comparison using to_json()
 2016-02-23 01:36:39 -08:00
Ilya Kreymer
57991fd0cf cdx: ensure url param required check is performed on init 2016-02-22 13:59:07 -08:00
Ilya Kreymer
af7c876263 cdx: ensure CDXQuery computes key and end_key automatically 
key and end_key encoded as utf-8 by default
 2016-02-22 13:39:47 -08:00
Ilya Kreymer
1a0b2fba17 add aggregate index source and tests! 2016-02-22 13:30:12 -08:00
Ilya Kreymer
7513011cac path resolvers: add PathResolverMapper for converting paths to resolvers, 
ResolvingLoader takes a list of resolvers, not paths (to allow for custom overrides)
ResolvingLoader and ArcWarcRecordLoader support 'no_record_parse' on load to not parse http headers from stream
 2016-02-19 22:33:38 -08:00
Ilya Kreymer
37198767ed add utils, responseloader and liverec 2016-02-19 17:27:19 -08:00
Ilya Kreymer
baa02add69 add indexloader and tests, including file, redis, remote cdx, memento, and live sources 2016-02-19 17:25:54 -08:00
Ilya Kreymer
bd841b91a9 more python 3 support work -- pywb.cdx, pywb.warc tests succeed 
most relative imports replaced with absolute
 2016-02-18 21:26:40 -08:00
Ilya Kreymer
b7008920de fix setup.py typo 2016-02-16 16:14:10 -08:00
Ilya Kreymer
3c85f7b7ac py3: make pywb.utils work with python 3! 2016-02-16 14:52:20 -08:00
Mat Kelly
50dab0bc98 Fixed misspelling 2016-01-12 18:21:05 -05:00
Ilya Kreymer
7cf81935e1 Update CHANGES for 0.11.1 2015-12-29 23:03:51 -08:00
Ilya Kreymer
d1c0bfac10 warc indexing: refactor to add create_payload_buffer() which can be overriden in custom iterators to create a file-like object 
that will receive write() calls to buffer the payload when indexing. Default implementation does not buffer the payload
 2015-12-29 17:01:25 -08:00
Ilya Kreymer
98843a2551 wombat: call reload() on actual location, possible fix related to #164 2015-12-29 16:17:39 -08:00
Ilya Kreymer
1e54f8c8fa proxy: add tests for proxy-mode 'Pywb-Rewrite-Prefix' header which adds optional prefix to proxy mode rewrites.. ensures such rewrites always absolute to include the prefix 2015-12-29 16:10:23 -08:00
Ilya Kreymer
a25096968a proxy: ip resolver: show 500 error if incorrect coll preconfigured for ip-based settings (todo: make it configurable?) 2015-12-29 14:53:50 -08:00
Ilya Kreymer
ba19ff1cd5 proxy: add custom rewrite prefix in proxy mode with 'pywb-proxy-rewrite: prefix' header 2015-12-23 23:14:47 -08:00
Ilya Kreymer
0cf6b40af9 wombat: add option to def_prop() to make overriden property enumerable, make WombatLocation and other loc overrides enumerable, fixes #163 2015-12-18 21:46:50 -08:00
Ilya Kreymer
79d5ec2b2d statusheaders: when not verifying protocol line, avoid indexerror when no space in first line, add tests 2015-12-18 21:46:00 -08:00
Ilya Kreymer
f6a4f80e47 bump to 0.11.1 2015-12-18 21:27:03 -08:00
Ilya Kreymer
6e58fe7a40 CHANGES.rst clarification 0.11.0 2015-12-13 00:40:01 -08:00
Ilya Kreymer
7d4c76f308 CHANGES.rst fixes 2015-12-13 00:37:01 -08:00
Ilya Kreymer
da01f107c7 Update CHANGES for 0.11.0 2015-12-13 00:31:38 -08:00
Ilya Kreymer
79b35a64b3 bump wombat version to 2.10 2015-12-13 00:31:01 -08:00
Ilya Kreymer
381f350917 proxy: switching not available for ip resolver either 
tests: update tests for auth and ip resolver to check that proxy magic is not set
 2015-12-12 22:59:32 -08:00
Ilya Kreymer
e9b11fcbf2 proxy: default to cookie resolver, which allows switching collections and datetime, instead of auth resolver 
auth resolver can be used by setting 'cookie_resolver: false' explicitly. when using auth resolver,
don't set proxy magic path as switching collections or datetime is not possible with auth resolver
closes #160
 2015-12-12 21:58:12 -08:00
Ilya Kreymer
64fe49d703 wombat: add CSSStyleSheet.prototype href override 
karma tests: change name to just 'pywb'
 2015-12-11 17:33:22 -08:00
Ilya Kreymer
467ea0f68b Merge branch 'client-side-tests' into develop 2015-12-10 12:50:40 -08:00
Ilya Kreymer
9b8b4d8388 tests/typo fix: add tests for truncated record detection (see: ikreymer/webarchiveplayer#14) fix typo, closes #161 2015-12-10 12:31:58 -08:00
Ilya Kreymer
3e3d7e4d87 Merge pull request #159 from robertknight/anchor-override-tostring 
wombat: Override HTMLAnchorElement.prototype.toString()
 2015-12-10 11:34:56 -08:00
Ilya Kreymer
b12a24e990 Merge pull request #162 from robertknight/resolve_relative_rel_canon 
rewrite: Resolve relative canonical paths if rewriting is disabled
 2015-12-10 11:33:15 -08:00
Robert Knight
83a33e0541 Resolve relative canonical paths if rewriting is disabled 
For Via, we want rel=canonical links to resolve to the same
absolute URL as it did on the original page.

For absolute URLs, no rewriting is necessary. If the original
rel=canonical URL was relative however, it needs to be resolved
relative to the original URL.

See https://github.com/hypothesis/via/issues/65 for context.
 2015-12-10 08:31:50 +00:00
Robert Knight
34721a6742 Override HTMLAnchorElement.prototype.toString() 
The values returned by getting the 'href' and 'toString'
properties of an anchor element should be the same.

This inconsistency broke the URL polyfill in
https://github.com/inexorabletash/polyfill under Microsoft Edge
 2015-12-02 20:09:09 +00:00