backup/pywb
1
0
Fork 0
mirror of https://github.com/webrecorder/pywb.git synced 2025-04-02 12:06:14 +02:00
Code Issues Projects Releases Activity
1,249 Commits 57 Branches 63 Tags
Commit Graph

140 Commits

Author SHA1 Message Date
Ilya Kreymer
ae35d92dde fix typo in certauth 2014-07-26 15:27:02 -07:00
Ilya Kreymer
5beb831ae9 wbrequestresponse: update doctest 2014-07-26 14:27:31 -07:00
Ilya Kreymer
1464e89c41 wbresponse: always include Content-Length for text_response 2014-07-26 14:24:28 -07:00
Ilya Kreymer
eca3cf5fbf https proxy: add ca generator! 
support uwsgi, gunicorn and ref
better handling of 407, other error responses in response to CONNECT
 2014-07-26 13:24:53 -07:00
Ilya Kreymer
739f23da9e https proxy support, CONNECT verb handling (uwsgi only) 2014-07-26 09:48:44 -07:00
Ilya Kreymer
4d31c17d4c archivalrouter: make SCRIPT_NAME key lookup default to '', addresses #39 2014-07-23 15:31:30 -07:00
Ilya Kreymer
7c57345363 proxy: add 'unaltered_replay' option to proxy_options to replay 
all content unaltered (no rewriting html, no banner, no wombat)
use 'proxy_options' instead of 'routing_options', add additional
tests for proxy mode
 2014-07-21 16:42:14 -07:00
Ilya Kreymer
a2973b04e7 wbrequest: add options dictionary to store misc request options 2014-07-21 14:02:31 -07:00
Ilya Kreymer
fa813bdd19 pep8 cleanup pass 2014-07-20 18:26:16 -07:00
Ilya Kreymer
b785cd6f08 memento: use mp_ modifier to support memento with frame or non-frame replay 
change memento test to use frame replay
 2014-07-20 15:43:39 -07:00
Ilya Kreymer
96fcaab521 live-rewrite-server: add ability to specify http/https proxy for live fetching 
(for example, for use with a recording proxy)
 2014-07-19 14:43:28 -07:00
Ilya Kreymer
1b1a1f8115 proxy: add 'proxy_coll_select' config which will require a proxy-auth to select a collection for proxy mode. 
Otherwise, defaults to first available collection, though proxy-auth can still be sent to specify different collection
 2014-07-14 19:12:30 -07:00
Ilya Kreymer
1317b2b10f route selection via proxy auth! 
refactor poute request parsing to happen in the actual router class instead of in the route
in proxy mode, add support for picking a route via proxy-auth
improve test for 'top' rewriting
 2014-07-10 21:54:23 -07:00
Ilya Kreymer
88d3e94b36 fixes for pep8, name fixes 2014-06-15 11:57:48 -07:00
Ilya Kreymer
0d3f663ef1 rewrite: disable refer-redirect in case of POST, handle request w/o redirect 
(can't use 307 because of FF)
 2014-06-13 16:23:11 -07:00
Ilya Kreymer
e2349a74e2 replay: better POST support via post query append! 
record_loader can optionally parse 'request' records
archiveindexer has -a flag to write all records ('request' included),
-p flag to append post query
post-test.warc.gz and cdx
POST redirects using 307
 2014-06-10 19:21:46 -07:00
Ilya Kreymer
52040127b3 update wombat.js to latest 
rewrite live: add another rewrite live header,
use 307 for archival referer based redirects
 2014-05-30 11:03:22 -07:00
Ilya Kreymer
89da165467 exceptions: add optional url param to WbException, move handler_exception() 
into WSGIApp for easier customization
 2014-05-13 01:54:12 -07:00
Ilya Kreymer
53f0cb540f url rewriter: add optional 'full prefix', check and don't rewrite urls 
if starting with prefix or full prefix
wbrequest: if no scheme present (shouldn't happen with wsgi) default to http
 2014-04-24 10:44:08 -07:00
Ilya Kreymer
85593696fa remove rfc3987 validation, was rejecting valid urls 
add extract_referer_wburl_str() to extract WbUrl str, if any,
from the referrer. Use that for live_rewrite_handler to override
default referrer
 2014-04-15 16:38:53 -07:00
Ilya Kreymer
19f2df4717 refactor: 
- move is_identity(), is_embed() to wburl from wbrequest
- add is_mainpage() predicate
- add create_template() to each J2TemplateView to create itself
- add HeadInsertView to create a reusable head insert for
RewriteContent
- add 'mp_' as modifier for frames mode to be used as possible
  modifier with HTMLRewriter
 2014-04-09 15:49:55 -07:00
Ilya Kreymer
80f2da9548 refactor: move configs/config.yaml to root again 
remove cdx-server specific config, instead make cdx server api-only
path configurable from regular config
 2014-04-02 21:26:53 -07:00
Ilya Kreymer
91184426b7 test coverage pass: 
refactor and cleanup to improve coverage for corner cases
 2014-04-02 13:16:54 -07:00
Ilya Kreymer
99eadb3d4f update package paths 2014-03-28 11:57:13 -07:00
Ilya Kreymer
093d8310e5 config: move config files to ./configs/ 
PYWB_CONFIG_FILE setting overrides passed in config
 2014-03-27 14:31:27 -07:00
Ilya Kreymer
c6c9fe680a memento: add original link to timemap #10 2014-03-24 14:57:41 -07:00
Ilya Kreymer
2a605652c6 add memento timemap support (for archival mode only) 
add timemap Link headers to timegate and memento responses
timemap accessible via /timemap/*/ path
 2014-03-24 14:00:06 -07:00
Ilya Kreymer
10c84d8354 embed rewriting: add 'em_' flag for all regex-based rewrites 
(js, css, xml) to be able to distinguish between embeds and non-embeds
more conclusively
wbrequest: add is_embed(), is_identity() properties
update tests
don't insert html banner if detected as an embed
 2014-03-17 19:36:25 -07:00
Ilya Kreymer
bfffac45b0 remove reference to deleted file wbexceptions.py 2014-03-14 11:22:50 -07:00
Ilya Kreymer
14a12f95b2 pep8 fixes, improve docs for proxy 
move CaptureException into replay_views
 2014-03-14 11:02:03 -07:00
Ilya Kreymer
a1ab54c340 first pass at memento support #10! 
memento support enabled by default, togglable via 'enable_memento' config property
supporting timegate and memento apis, no timemap yet
supporting pattern 2.3 for archival and pattern 1.3 for proxy modes
also:
simplify exception hierarchy a bit more, move down to utils
make WbRequest and WbResponse extensible with mixins (eg for memento)
 2014-03-14 10:46:20 -07:00
Ilya Kreymer
e384425d48 proxy cleanup: move HttpsUrlRewriter to url_rewriter module, 
move strip_scheme to replay_views where it is used
regex rewriters: use url rewriter for rewriting http:// in JS,
instead of just prefix, to support custom rewriters (such as
https->http rewriter in proxy mode)
 2014-03-09 14:21:32 -07:00
Ilya Kreymer
541c076b77 setup: add cli scripts for wayback, cdx-server 
fix logging of app name, make most logging debug
 2014-03-08 15:09:53 -08:00
Ilya Kreymer
e3618871c8 proxy: support setting hostname via env variable 2014-03-07 11:42:09 -08:00
Ilya Kreymer
a60ab1f118 routing/proxy: pass in hostpaths to proxy routing 
add PYWB_HOST_NAME env var to allow overriding default hostname
add request_hostname jinja filter
 2014-03-07 10:29:11 -08:00
Ilya Kreymer
cc22448cc5 fixes for 2.6 and pypy 2014-03-04 19:11:17 -08:00
Ilya Kreymer
202f6101e0 coverage work! add additional test for wsgi_wrappers 
additional test for zipnum bad location
for now, not testing cli interfaces which depend on opt params
 2014-03-04 16:13:49 -08:00
Ilya Kreymer
2d4ae62fbe - cdx handler refactoring: factor out CDXHandler and init to 
seperate cdx_handler module
- Make wsgi app a class, add port as an optional field in wsgi app
and router. (not required to be specified)
 2014-03-03 10:35:57 -08:00
Ilya Kreymer
0bf651c2e3 add cdx_server app! 
port wsgi cdx server tests to test new app!
move base handlers to basehandlers in framework pkg
(remove werkzeug dependency)
 2014-03-02 23:41:44 -08:00
Ilya Kreymer
f0a0976038 more refactoring! 
create 'framework' subpackage for general purpose components!
contains routing, request/response, exceptions and wsgi wrappers
update framework package for pep8
dsrules: using load_config_yaml() (pushed to utils)
to init default config
 2014-03-02 21:42:05 -08:00