1
0
mirror of https://github.com/webrecorder/pywb.git synced 2025-03-28 00:25:21 +01:00

20 Commits

Author SHA1 Message Date
Ilya Kreymer
304ddbec84 Support for new UI, as per #16
* Refactor views class to support more Jinja2 views (J2Template)
* Add a home page, collection search page, and error pages, all optional
* all exceptions appear on error page
* wbrequest supports a request with an empty or / wb_url
2014-01-31 10:04:21 -08:00
Ilya Kreymer
6de794a4e1 style fixes: convert camelCase func and var names to 'not_camel_case'
WbHtml -> HTMLRewriter
ArchivalUrl -> WbUrl
2014-01-28 19:37:37 -08:00
Ilya Kreymer
8732499dd5 - cdx server bootstrap configured, #12
- pywb_init module inits from ./test directory

misc:
- router has lookahead for '/'
- dechunk even for transparent/binary
- 'text' query mode displays cdx
2014-01-27 21:46:38 -08:00
Ilya Kreymer
b237b144ff further refactor steaming of responses related to #13: always create a generator from
response stream, and if buffering, read entire generator into temp buffer
remove duplicate reading logic
2014-01-22 17:55:55 -08:00
Ilya Kreymer
7722014a96 Cleanup rewrite interfaces to address #13
All rewriters can support either buffered or streaming mode.
In buffered mode, the full text content is written into a buffer
and served with a Content-Length
in streaming mode, text is streamed as it is rewritten and
no Content-Length is written
Default is to stream the response
2014-01-22 14:03:41 -08:00
Ilya Kreymer
9ff3fc300b Fix #5, bringing back customParams optional params sent to cdx server
Rename archivalrouter.MatchRegex -> archivalrouter.Route, supporting regex/prefix matching
add redir_to_exact to turn off redirect to exact timestamp in RewritingReplayHandler
update README
2014-01-20 10:50:06 -08:00
Ilya Kreymer
80b2585d22 Should resolve #4 -- supports pywb running as a non-root app
* Instead of relying on REQUEST_URI, pywb constructs a
REL_REQUEST_URI, from PATH_INFO + QUERY_STRING.
SCRIPT_NAME auto-added to prefix
* MatchPrefix is now superceded by MatchRegex, which
can match a plain string -- collId defaults to the full match
* Added optional archivalurl_class to router to allow for customized
ArchivalUrl implementations to be specified
* run.sh can test on a non-root mountpoint, eg. ./run.sh "/approot"
2014-01-19 21:13:48 -08:00
Ilya Kreymer
354040a7e0 support for url-agnostic dedup, eg loading payload from a different url
than the revisit
2014-01-19 12:31:19 -08:00
Ilya Kreymer
7ce6d0d22b first pass on html rendering via jinja, support for query (cdx) rendering 2014-01-17 16:24:36 -08:00
Ilya Kreymer
bcc9588c00 * archivalrouter: to take a list of handlers,
currently MatchPrefix and MatchRegex. handler returns a single response
(no chaining for now)
* rewriting: don't rewrite anchor only urls
* perf: add a very basic profiler in WBHandler for testing
2014-01-16 20:33:51 -08:00
Ilya Kreymer
2357f108a3 rename rewriters
header_rewriter added!
support for encoding detection
various fixes
xmlrewriter
2014-01-03 13:03:03 -08:00
Ilya Kreymer
d9930322f1 support utf-8 (so far)
support protocol-agnostic prefix //
failedFile list for warc loading
2013-12-31 00:18:12 +00:00
Ilya Kreymer
a84ec2abc7 first iteration of archival mode working w/ banner insertion!! 2013-12-28 17:39:43 -08:00
Ilya Kreymer
16f458d5ec archiveloader: Support for loading warc/arc records using hanzo parser (for record header parsing only)
ReplayHandler: load replay from query response, find best option
basic support for matching url, checking self-redirects!
2013-12-28 05:00:06 -08:00
Ilya Kreymer
072befe3c8 archivalrouter: support handler chaining, using call convention and pass prev response 2013-12-20 15:10:12 -08:00
Ilya Kreymer
4cf4bf3bbb add wburlrewriter, ReferRedirect uses the rewriter
more refactoring, ReferRedirect moved into archivalrouter module
wbrequest: parses from uri directly, keeps track of wburl and prefix
2013-12-20 14:54:41 -08:00
Ilya Kreymer
0a2b16407d better exception handling, specific status codes for exceptions,
detect access control and not found exceptions more consistently
2013-12-19 12:06:47 -08:00
Ilya Kreymer
c8d2271e8a archiveurl: add support for url_query, format modifier for more unit tests
archivalrouter: flesh out router seperately
indexreader: RemoteCDXServer reader
unit tests for req/resp
wbapp -- cdx output for query, urlquery, replay and latest_replay!
2013-12-18 18:52:52 -08:00
Ilya Kreymer
5d42cc0cac rename aurl -> archiveurl, add default scheme, test for empty url 2013-12-13 15:43:07 -08:00
Ilya Kreymer
27b35f31e8 add basic wsgi app for parsing archivalurls, fallback on a referrer based redirect 2013-12-13 15:20:13 -08:00