Ilya Kreymer
00a7691f69
add optional filters to default Route
...
add examples to config.yaml and test_config.yaml and integration test
per route config is inherited globally if only name is set
2014-02-06 17:28:08 -08:00
Ilya Kreymer
d347b4952b
don't mask raised exceptions, to address #23
2014-02-05 13:21:57 -08:00
Ilya Kreymer
1a1aa814d0
first pass at simple http proxy! #8
...
* proxy router for handling only proxy
* proxy/archival router for handling both archival and proxy mode,
togglable with 'enable_http_proxy' setting in config
* supports only most recent capture playback -- no support for
selecting replay date/calendar view yet
* not testable with WebTest -- need better way to unit test proxy mode
2014-02-05 13:08:10 -08:00
Ilya Kreymer
848dc6d000
add new test_config.yaml!
2014-02-05 10:17:06 -08:00
Ilya Kreymer
3168b80cfa
improve docs for config.yaml, group all ui settings together
...
create seperate test_config.yaml for testing
rename ArchivalRequestRouter -> ArchivalRouter for consistency
2014-02-05 10:10:33 -08:00
ikreymer
94f8f080c4
Merge pull request #22 from rajbot/master
...
Add Vagrantfile
2014-02-04 13:19:34 -08:00
rajbot
bad88e298f
Clarify vagrant section of the README
2014-02-04 13:12:25 -08:00
rajbot
e3dd09f8fc
Add Vagrantfile
2014-02-04 13:06:32 -08:00
rajbot
e7b0b2319c
Remove trailing whitespace.
2014-02-04 13:05:30 -08:00
Ilya Kreymer
6388a78162
refactor: replay_views to support cleaner inheritance, no longer
...
wrapping previous WbResponse
overhaul yaml config to be much simpler, move best resolver and
best index reader to respective classes
add config_utils for sharing config, standard non-yaml config
provides defaults for testing
fix bug in query.html
2014-02-03 09:24:40 -08:00
ikreymer
b6846c54e0
Merge pull request #21 from ikreymer/wburl-drop-slash
...
refactor WbUrl and UrlRewriter to drop requirement for having a WbUrl start with /
2014-02-01 19:47:18 -08:00
Ilya Kreymer
bdef00cb8d
refactor WbUrl and UrlRewriter to drop requirement for having a WbUrl start with /
...
Changes WbUrl forms:
/2013/im_/example.com -> 2013/im_/example.com
/*/example.com -> */example.com
/example.com -> example.com
* also simplify scheme-agnostic url (//) handling by just eating up extra
slashes
* add additional doctests on route, with and w/o custom SCRIPT_NAME
2014-02-01 18:20:23 -08:00
Ilya Kreymer
9f258fa64c
fix up cdx server query interface
...
supports /cdx?url=... and other params including
filter=<regex>
collapse_time=<0-14>
resolve_revisits=<true|false>
reverse=<true|false>
closest=<timestamp>
2014-02-01 14:47:07 -08:00
Ilya Kreymer
b685772b96
fixup loading from archive, add LimitReader to ensure record length is respected
...
rename FileReader -> FileLoader, HttpReader -> HttpLoader
loaders create 'readers', which support read()/readline()
2014-02-01 14:02:53 -08:00
Ilya Kreymer
d9c4e5cba4
make RemoteCDXServer api conform to LocalCDXServer api, addressing #19
2014-02-01 13:19:30 -08:00
Ilya Kreymer
86a093d164
support cdx server query at (/cdx in default config)
...
also enable /echo_env and /echo_req debug handlers
2014-02-01 00:43:24 -08:00
Ilya Kreymer
f00ac826cf
fix typo in setup.py
2014-02-01 00:15:31 -08:00
Ilya Kreymer
2f5ffb3a88
switch test framework to use py.test instead of nose
2014-02-01 00:12:11 -08:00
Ilya Kreymer
6d2c8286ca
render_response has option to pass in statuscode
2014-01-31 19:45:01 -08:00
Ilya Kreymer
44ef14b022
add first integration tests with WebTest!
...
covers home page, search page, replay, calendar, redirect + replay, cdx
2014-01-31 19:41:44 -08:00
Ilya Kreymer
bd94e3c656
fix replay_resolvers tests, don't use abs paths!
2014-01-31 10:33:47 -08:00
Ilya Kreymer
304ddbec84
Support for new UI, as per #16
...
* Refactor views class to support more Jinja2 views (J2Template)
* Add a home page, collection search page, and error pages, all optional
* all exceptions appear on error page
* wbrequest supports a request with an empty or / wb_url
2014-01-31 10:04:21 -08:00
Ilya Kreymer
57fe9515db
- support for running uwsgi with virtualenv
...
- text changes in banner
- some info about testing in README
2014-01-29 17:23:19 -08:00
Ilya Kreymer
467d880681
update README
2014-01-29 15:15:39 -08:00
Ilya Kreymer
53eb5072ec
more README tweaks
2014-01-29 15:12:57 -08:00
Ilya Kreymer
28618c69c6
update query.html, listing unique timestamps
...
update README
2014-01-29 15:07:45 -08:00
Ilya Kreymer
e7b70ae496
fix links in README
2014-01-29 12:08:51 -08:00
Ilya Kreymer
f45234f39b
README tweaks
2014-01-29 12:07:33 -08:00
Ilya Kreymer
a6cfe9a87b
update README.md
2014-01-29 12:01:03 -08:00
Ilya Kreymer
937fc7229e
update README, fix typo
2014-01-29 02:12:58 -08:00
Ilya Kreymer
9cde058ccf
check for osx uwsgi path and use that, otherwise run 'uwsgi'
2014-01-29 02:12:54 -08:00
Ilya Kreymer
84ffec9b8d
update README.md
2014-01-29 01:52:30 -08:00
Ilya Kreymer
eb9cef9e28
update README.md for beta!!!
2014-01-29 01:36:31 -08:00
Ilya Kreymer
7a20d26d5f
support non-surt ordered cdx
...
add unsurt() util func and surt_ordered init param to LocalCDXServer
test make_best_resolver()
2014-01-29 00:58:37 -08:00
Ilya Kreymer
9a3449dfd5
add pyyaml to dependency
2014-01-29 00:04:54 -08:00
Ilya Kreymer
411e7fe8a3
cleanup pywb_init, work on documenting config.yaml!
2014-01-29 00:03:24 -08:00
Ilya Kreymer
43a46b373d
move sample/test data to ./sample_archive/warcs and ./sample_archive/cdx
...
pywb_init now driven by config.yaml! (#14 )
Not yet supporting customized handlers, views, etc...
2014-01-28 22:03:01 -08:00
Ilya Kreymer
35f7cb0477
new-feature: support jinja2 template generated banner
...
template receives cdx and wbrequest
default template inserts capture time into banner
2014-01-28 20:18:47 -08:00
Ilya Kreymer
6de794a4e1
style fixes: convert camelCase func and var names to 'not_camel_case'
...
WbHtml -> HTMLRewriter
ArchivalUrl -> WbUrl
2014-01-28 19:37:37 -08:00
Ilya Kreymer
c0f8edf517
more refactoring: seperate top-level handlers (WBHandler) from
...
views (html, text)
Add CDXHandler for interfacing with cdx server directly, #12
2014-01-28 17:23:44 -08:00
Ilya Kreymer
1a234f2953
refactor: remove intermediate query object.
...
rename query -> views
wbhandler queries index, replayer and renders via view
new feature: 'cdx_' modifier can be used to render cdx from any request
2014-01-28 16:41:19 -08:00
Ilya Kreymer
a83d527702
add surt to dependency list
2014-01-27 22:07:27 -08:00
Ilya Kreymer
a6458b056f
some tweaks on transfer-encoding: always remove and serve unchunked
...
(should allow front-end serve can rechunk as needed)
2014-01-27 22:05:49 -08:00
Ilya Kreymer
8732499dd5
- cdx server bootstrap configured, #12
...
- pywb_init module inits from ./test directory
misc:
- router has lookahead for '/'
- dechunk even for transparent/binary
- 'text' query mode displays cdx
2014-01-27 21:46:38 -08:00
Ilya Kreymer
c55bdf0e1f
-binsearch: add tests, support both prefix and exact loading, for #11
...
-cdx server first pass for #12 : implement cdx parsing and transforming
-operations supported: merge sort, regex filter, resolve revisits, closest sort, reverse sort,
timestamp collapse
timestamp parsing utils
2014-01-27 17:02:48 -08:00
Ilya Kreymer
e1b669fdea
improved customization: can setup pywb_init.pywb_config() config,
...
or specify custom init module <initmodule>.py_config() by
setting PYWB_INIT=<initmodule>
fix run.sh to support testing with custom mount point
2014-01-24 12:25:27 -08:00
Ilya Kreymer
44f68158a9
update README and comments
2014-01-24 01:17:18 -08:00
Ilya Kreymer
1033feb2f8
use sample settings if driver file not found
2014-01-24 00:59:15 -08:00
Ilya Kreymer
391f3bf81d
remove pycdx_server pkg for now, move binsearch into pywb package,
...
update setup.py
2014-01-24 00:54:48 -08:00
Ilya Kreymer
03b6938b9c
referer fallback: check for non empty SCRIPT_NAME when parsing referrer
2014-01-24 00:53:55 -08:00