backup/pywb
1
0
Fork 0
mirror of https://github.com/webrecorder/pywb.git synced 2025-03-19 02:09:38 +01:00
Code Issues Projects Releases Activity
435 Commits 56 Branches 63 Tags
Commit Graph

61 Commits

Author SHA1 Message Date
Ilya Kreymer
e58a63a9fe setup: add openssl as a req 2014-07-26 14:35:52 -07:00
Ilya Kreymer
739f23da9e https proxy support, CONNECT verb handling (uwsgi only) 2014-07-26 09:48:44 -07:00
Ilya Kreymer
22c210131a bump version to 0.5.1 2014-07-23 15:31:38 -07:00
Ilya Kreymer
ca405ef179 update version to 0.5.0, update CHANGELIST and README 2014-07-20 16:58:28 -07:00
Ilya Kreymer
02326a2b12 bump dev version to 0.4.8 2014-07-07 17:02:28 -07:00
Ilya Kreymer
46b16c61d5 update changelist, version to 0.4.7 2014-07-01 16:15:25 -07:00
Ilya Kreymer
694b97e67f archive indexing: Refactor, split into ArchiveIterator generic iteration and cdx-indexer, 
which writes out CDX specifically
recordloader: always load request, limit stream before headers are loaded
 2014-06-19 13:37:42 -07:00
Ilya Kreymer
ac3efec4bc update develop to 0.4.6 
improved regex for top -> WB_wombat_top rewriting
 2014-06-16 15:57:22 -07:00
Ilya Kreymer
f26b0ddbe4 update setup.py version 2014-06-15 12:35:20 -07:00
Ilya Kreymer
028cdaa22e bump version to 0.4.1 2014-06-05 14:10:30 -07:00
Ilya Kreymer
871cc26fa4 rewrite: add optional cookie_rewriter, created by urlrewriter and called from header_rewriter 
cookie_rewriter works correctly with a concatenated set-cookie list, returns a list of rewritten 'set-cookie' headers
rewrite_live: add proxying of Host, Origin, additional headers
split header rewriter tests into test_header_rewriter, add test_cookie_rewriter
bump version to 0.4.0!
 2014-05-13 17:07:41 -07:00
Ilya Kreymer
85593696fa remove rfc3987 validation, was rejecting valid urls 
add extract_referer_wburl_str() to extract WbUrl str, if any,
from the referrer. Use that for live_rewrite_handler to override
default referrer
 2014-04-15 16:38:53 -07:00
Ilya Kreymer
bfc2e63793 live rewriter: integrate handler with rewrite_live.py module, 
clean up css, add unit and integration tests
clean up cli server now known as 'live-rewrite-server', which performs live rewrite using
iframe paradigm
 2014-04-09 15:49:55 -07:00
Ilya Kreymer
d8c20a59cf update to version 0.3.1 2014-04-06 11:46:43 -07:00
Ilya Kreymer
b0b0adb043 refactor: rename pywb.core -> pywb.webapp 
move perms/test/test_perms_policy -> tests/perms_fixture
for rules file, use single DEFAULT_RULES_FILE import
 2014-04-04 10:09:26 -07:00
Ilya Kreymer
80f2da9548 refactor: move configs/config.yaml to root again 
remove cdx-server specific config, instead make cdx server api-only
path configurable from regular config
 2014-04-02 21:26:53 -07:00
Ilya Kreymer
90f4833df3 add cli interface for archiveindexer expose as 'cdx-indexer' 
add tests for cli interface
additional tests for statusheaders
 2014-04-02 10:36:55 -07:00
Ilya Kreymer
9700004dc8 move configs to pywb package as package data 2014-03-28 11:53:59 -07:00
Ilya Kreymer
83e07442f0 add configs to datadirs 2014-03-28 10:54:37 -07:00
Ilya Kreymer
2c74ea9f23 fuzzy match: make filter string optionally overridable 
setup.py: unset PYWB_CONFIG_ENV
 2014-03-27 21:43:30 -07:00
Ilya Kreymer
5847087aae add fakeredis mock, test for RedisCDXSource 2014-03-25 11:02:32 -07:00
Ilya Kreymer
10c84d8354 embed rewriting: add 'em_' flag for all regex-based rewrites 
(js, css, xml) to be able to distinguish between embeds and non-embeds
more conclusively
wbrequest: add is_embed(), is_identity() properties
update tests
don't insert html banner if detected as an embed
 2014-03-17 19:36:25 -07:00
Ilya Kreymer
6461af030b refactoring: clean up handlers and replay_views for pep8 
use BlockLoader().load for StaticHandler static file resolving
update static paths to point to pywb/static instead of static
 2014-03-14 18:17:22 -07:00
Ilya Kreymer
29ecadee54 update README, fix setup.py typo 2014-03-12 18:35:21 -07:00
Ilya Kreymer
3222f3ee08 update setup, remove markdown readme 2014-03-12 17:57:54 -07:00
Ilya Kreymer
1c85aebbf0 fix setup.py 2014-03-10 19:19:41 -07:00
Ilya Kreymer
fe9eaea006 update setup.py classifiers 2014-03-10 19:11:19 -07:00
Ilya Kreymer
541c076b77 setup: add cli scripts for wayback, cdx-server 
fix logging of app name, make most logging debug
 2014-03-08 15:09:53 -08:00
Ilya Kreymer
40b7a8e921 move pytest args to pytest.ini 2014-03-08 09:30:56 -08:00
Ilya Kreymer
daf868fd61 README tweaks 
update setup.py to support setup.py test!
.travis.yml uses python setup.py test
 2014-03-05 11:19:26 -08:00
Ilya Kreymer
577c74be49 cdx: move perms related handling to pywb.perms package, support 
custom processing ops, of which perms is a specific type
add lazy_ops test to ensure all cdx processing ops are lazy

perms: set up a 'perms policy' factory and perms policy implementation
perms policy setting results in a custom processing op
update tests to work with new config
IndexReader handles both cdx server + perms policy
 2014-03-03 18:27:04 -08:00
Ilya Kreymer
0bf651c2e3 add cdx_server app! 
port wsgi cdx server tests to test new app!
move base handlers to basehandlers in framework pkg
(remove werkzeug dependency)
 2014-03-02 23:41:44 -08:00
Ilya Kreymer
f1acad53fc wsgi wrapper reorg! 
support pluggable wsgi apps
utils: BlockLoader() supports loading from package
exceptions: base WbException moved to utils
 2014-03-02 19:26:06 -08:00
Ilya Kreymer
19f86305bf update pkg-reorg with changes from master, including 
CDXQuery configuration
 2014-03-02 00:26:29 -08:00
Ilya Kreymer
355fa32600 cdx: refactor to create seperate CDXQuery object for wrapping 
params passed to load_cdx()
 2014-03-01 08:41:24 -08:00
Ilya Kreymer
af9cabdc72 Merge branch 'cdx-server' of git://github.com/kngenie/pywb into kngenie-cdx-server 
Kengie's cdx server refactoring and wsgi improvements
 2014-02-28 15:28:41 -08:00
Ilya Kreymer
502666fd3d Merge branch 'just-a-cleanup' of git://github.com/kngenie/pywb into kngenie-just-a-cleanup 
cleanup setup.py indentation
 2014-02-28 12:23:48 -08:00
Kenji Nagahashi
1f65eff828 Merge remote-tracking branch 'origin/master' into cdx-server 
Conflicts:
	pywb/cdx/cdxdomainspecific.py
	pywb/cdx/cdxserver.py
	pywb/cdx/test/cdxserver_test.py
	setup.py
	tests/test_integration.py
 2014-02-28 19:47:24 +00:00
Ilya Kreymer
c084b45298 Merge master into pkg-reorg 2014-02-28 10:25:36 -08:00
Ilya Kreymer
bff39626b5 add first set of zipnum tests #17 
still need to test timed reload, multi sources
 2014-02-27 12:33:11 -08:00
Ilya Kreymer
22f1f78fca cdx: clean up filters, add '~' modifier for contains 
rules: fix regex to be lazy not greedy, turn off unneeded custom
canonicalizer (need tests for custom canon)
cleanup fuzzy match query
fix data package in setup.py
 2014-02-27 18:22:10 +00:00
Ilya Kreymer
453ab678ed refactor domain specific rules: 
- head insert callback passed in with rule, up to template
to handle additional inserts based on rule properties
- ability to pass in custom rules config to both cdx server
and content rewriter
- move canonicalize to utils pkg
- add wombat, modify wb.js to remove wombat-related settings
 2014-02-26 22:04:37 -08:00
Ilya Kreymer
5a41f59f39 new unified config system, via rules.yaml! 
contains configs for cdx canon, fuzzy matching and rewriting!
rewriting: ability to add custom regexs per domain
also, ability to toggle js rewriting and custom rewriting file
(default is wombat.js)
 2014-02-26 18:02:01 -08:00
Ilya Kreymer
51d61a8738 package reorg! 
split up remaining parts of pywb root pkg
into core, dispatch and bootstrap
 2014-02-24 03:00:01 -08:00
Kenji Nagahashi
79eb3be44f rewrite wsgi_cdxserver with werkzeug 
use pkg_resources instead of pkgutil because pkgutil breaks with auto-reload.
add --port command line option.
 2014-02-20 09:58:08 +00:00
Kenji Nagahashi
d0229b6b2d cleanup setup.py indent for ease of add/remove things. also use find_package(). 2014-02-19 23:37:44 +00:00
Ilya Kreymer
abea504b04 cleanup cdx server config, refactored such that 
a cdx server need implement a single interface:
load_cdx(self, **params)

CDXServer and RemoteCDXServer distinct classes in cdxserver.py
utility function cdxserver.create_cdx_server() to create
appropriate server based on input
 2014-02-17 13:58:02 -08:00
Ilya Kreymer
5345459298 pywb 0.2! 
move to distinct packages: pywb.utils, pywb.cdx, pywb.warc, pywb.util, pywb.rewrite!
each package will have its own README and tests
shared sample_data and install
 2014-02-17 10:01:09 -08:00
Ilya Kreymer
2528ee0a7c refactoring of binsearch and cdxserver into seperate packages 
also move complicated doctests and integration tests to tests/
 2014-02-12 13:16:07 -08:00
Ilya Kreymer
2a828fab32 add sample data to dist egg to allow tests against the installed package 2014-02-09 12:06:35 -08:00