Ilya Kreymer
e58a63a9fe
setup: add openssl as a req
2014-07-26 14:35:52 -07:00
Ilya Kreymer
739f23da9e
https proxy support, CONNECT verb handling (uwsgi only)
2014-07-26 09:48:44 -07:00
Ilya Kreymer
22c210131a
bump version to 0.5.1
2014-07-23 15:31:38 -07:00
Ilya Kreymer
ca405ef179
update version to 0.5.0, update CHANGELIST and README
2014-07-20 16:58:28 -07:00
Ilya Kreymer
02326a2b12
bump dev version to 0.4.8
2014-07-07 17:02:28 -07:00
Ilya Kreymer
46b16c61d5
update changelist, version to 0.4.7
2014-07-01 16:15:25 -07:00
Ilya Kreymer
694b97e67f
archive indexing: Refactor, split into ArchiveIterator generic iteration and cdx-indexer,
...
which writes out CDX specifically
recordloader: always load request, limit stream before headers are loaded
2014-06-19 13:37:42 -07:00
Ilya Kreymer
ac3efec4bc
update develop to 0.4.6
...
improved regex for top -> WB_wombat_top rewriting
2014-06-16 15:57:22 -07:00
Ilya Kreymer
f26b0ddbe4
update setup.py version
2014-06-15 12:35:20 -07:00
Ilya Kreymer
028cdaa22e
bump version to 0.4.1
2014-06-05 14:10:30 -07:00
Ilya Kreymer
871cc26fa4
rewrite: add optional cookie_rewriter, created by urlrewriter and called from header_rewriter
...
cookie_rewriter works correctly with a concatenated set-cookie list, returns a list of rewritten 'set-cookie' headers
rewrite_live: add proxying of Host, Origin, additional headers
split header rewriter tests into test_header_rewriter, add test_cookie_rewriter
bump version to 0.4.0!
2014-05-13 17:07:41 -07:00
Ilya Kreymer
85593696fa
remove rfc3987 validation, was rejecting valid urls
...
add extract_referer_wburl_str() to extract WbUrl str, if any,
from the referrer. Use that for live_rewrite_handler to override
default referrer
2014-04-15 16:38:53 -07:00
Ilya Kreymer
bfc2e63793
live rewriter: integrate handler with rewrite_live.py module,
...
clean up css, add unit and integration tests
clean up cli server now known as 'live-rewrite-server', which performs live rewrite using
iframe paradigm
2014-04-09 15:49:55 -07:00
Ilya Kreymer
d8c20a59cf
update to version 0.3.1
2014-04-06 11:46:43 -07:00
Ilya Kreymer
b0b0adb043
refactor: rename pywb.core -> pywb.webapp
...
move perms/test/test_perms_policy -> tests/perms_fixture
for rules file, use single DEFAULT_RULES_FILE import
2014-04-04 10:09:26 -07:00
Ilya Kreymer
80f2da9548
refactor: move configs/config.yaml to root again
...
remove cdx-server specific config, instead make cdx server api-only
path configurable from regular config
2014-04-02 21:26:53 -07:00
Ilya Kreymer
90f4833df3
add cli interface for archiveindexer expose as 'cdx-indexer'
...
add tests for cli interface
additional tests for statusheaders
2014-04-02 10:36:55 -07:00
Ilya Kreymer
9700004dc8
move configs to pywb package as package data
2014-03-28 11:53:59 -07:00
Ilya Kreymer
83e07442f0
add configs to datadirs
2014-03-28 10:54:37 -07:00
Ilya Kreymer
2c74ea9f23
fuzzy match: make filter string optionally overridable
...
setup.py: unset PYWB_CONFIG_ENV
2014-03-27 21:43:30 -07:00
Ilya Kreymer
5847087aae
add fakeredis mock, test for RedisCDXSource
2014-03-25 11:02:32 -07:00
Ilya Kreymer
10c84d8354
embed rewriting: add 'em_' flag for all regex-based rewrites
...
(js, css, xml) to be able to distinguish between embeds and non-embeds
more conclusively
wbrequest: add is_embed(), is_identity() properties
update tests
don't insert html banner if detected as an embed
2014-03-17 19:36:25 -07:00
Ilya Kreymer
6461af030b
refactoring: clean up handlers and replay_views for pep8
...
use BlockLoader().load for StaticHandler static file resolving
update static paths to point to pywb/static instead of static
2014-03-14 18:17:22 -07:00
Ilya Kreymer
29ecadee54
update README, fix setup.py typo
2014-03-12 18:35:21 -07:00
Ilya Kreymer
3222f3ee08
update setup, remove markdown readme
2014-03-12 17:57:54 -07:00
Ilya Kreymer
1c85aebbf0
fix setup.py
2014-03-10 19:19:41 -07:00
Ilya Kreymer
fe9eaea006
update setup.py classifiers
2014-03-10 19:11:19 -07:00
Ilya Kreymer
541c076b77
setup: add cli scripts for wayback, cdx-server
...
fix logging of app name, make most logging debug
2014-03-08 15:09:53 -08:00
Ilya Kreymer
40b7a8e921
move pytest args to pytest.ini
2014-03-08 09:30:56 -08:00
Ilya Kreymer
daf868fd61
README tweaks
...
update setup.py to support setup.py test!
.travis.yml uses python setup.py test
2014-03-05 11:19:26 -08:00
Ilya Kreymer
577c74be49
cdx: move perms related handling to pywb.perms package, support
...
custom processing ops, of which perms is a specific type
add lazy_ops test to ensure all cdx processing ops are lazy
perms: set up a 'perms policy' factory and perms policy implementation
perms policy setting results in a custom processing op
update tests to work with new config
IndexReader handles both cdx server + perms policy
2014-03-03 18:27:04 -08:00
Ilya Kreymer
0bf651c2e3
add cdx_server app!
...
port wsgi cdx server tests to test new app!
move base handlers to basehandlers in framework pkg
(remove werkzeug dependency)
2014-03-02 23:41:44 -08:00
Ilya Kreymer
f1acad53fc
wsgi wrapper reorg!
...
support pluggable wsgi apps
utils: BlockLoader() supports loading from package
exceptions: base WbException moved to utils
2014-03-02 19:26:06 -08:00
Ilya Kreymer
19f86305bf
update pkg-reorg with changes from master, including
...
CDXQuery configuration
2014-03-02 00:26:29 -08:00
Ilya Kreymer
355fa32600
cdx: refactor to create seperate CDXQuery object for wrapping
...
params passed to load_cdx()
2014-03-01 08:41:24 -08:00
Ilya Kreymer
af9cabdc72
Merge branch 'cdx-server' of git://github.com/kngenie/pywb into kngenie-cdx-server
...
Kengie's cdx server refactoring and wsgi improvements
2014-02-28 15:28:41 -08:00
Ilya Kreymer
502666fd3d
Merge branch 'just-a-cleanup' of git://github.com/kngenie/pywb into kngenie-just-a-cleanup
...
cleanup setup.py indentation
2014-02-28 12:23:48 -08:00
Kenji Nagahashi
1f65eff828
Merge remote-tracking branch 'origin/master' into cdx-server
...
Conflicts:
pywb/cdx/cdxdomainspecific.py
pywb/cdx/cdxserver.py
pywb/cdx/test/cdxserver_test.py
setup.py
tests/test_integration.py
2014-02-28 19:47:24 +00:00
Ilya Kreymer
c084b45298
Merge master into pkg-reorg
2014-02-28 10:25:36 -08:00
Ilya Kreymer
bff39626b5
add first set of zipnum tests #17
...
still need to test timed reload, multi sources
2014-02-27 12:33:11 -08:00
Ilya Kreymer
22f1f78fca
cdx: clean up filters, add '~' modifier for contains
...
rules: fix regex to be lazy not greedy, turn off unneeded custom
canonicalizer (need tests for custom canon)
cleanup fuzzy match query
fix data package in setup.py
2014-02-27 18:22:10 +00:00
Ilya Kreymer
453ab678ed
refactor domain specific rules:
...
- head insert callback passed in with rule, up to template
to handle additional inserts based on rule properties
- ability to pass in custom rules config to both cdx server
and content rewriter
- move canonicalize to utils pkg
- add wombat, modify wb.js to remove wombat-related settings
2014-02-26 22:04:37 -08:00
Ilya Kreymer
5a41f59f39
new unified config system, via rules.yaml!
...
contains configs for cdx canon, fuzzy matching and rewriting!
rewriting: ability to add custom regexs per domain
also, ability to toggle js rewriting and custom rewriting file
(default is wombat.js)
2014-02-26 18:02:01 -08:00
Ilya Kreymer
51d61a8738
package reorg!
...
split up remaining parts of pywb root pkg
into core, dispatch and bootstrap
2014-02-24 03:00:01 -08:00
Kenji Nagahashi
79eb3be44f
rewrite wsgi_cdxserver with werkzeug
...
use pkg_resources instead of pkgutil because pkgutil breaks with auto-reload.
add --port command line option.
2014-02-20 09:58:08 +00:00
Kenji Nagahashi
d0229b6b2d
cleanup setup.py indent for ease of add/remove things. also use find_package().
2014-02-19 23:37:44 +00:00
Ilya Kreymer
abea504b04
cleanup cdx server config, refactored such that
...
a cdx server need implement a single interface:
load_cdx(self, **params)
CDXServer and RemoteCDXServer distinct classes in cdxserver.py
utility function cdxserver.create_cdx_server() to create
appropriate server based on input
2014-02-17 13:58:02 -08:00
Ilya Kreymer
5345459298
pywb 0.2!
...
move to distinct packages: pywb.utils, pywb.cdx, pywb.warc, pywb.util, pywb.rewrite!
each package will have its own README and tests
shared sample_data and install
2014-02-17 10:01:09 -08:00
Ilya Kreymer
2528ee0a7c
refactoring of binsearch and cdxserver into seperate packages
...
also move complicated doctests and integration tests to tests/
2014-02-12 13:16:07 -08:00
Ilya Kreymer
2a828fab32
add sample data to dist egg to allow tests against the installed package
2014-02-09 12:06:35 -08:00