From 4db661a09e255d2d4dad84fca412d7a822572ddb Mon Sep 17 00:00:00 2001 From: Ilya Kreymer Date: Fri, 17 Apr 2015 11:01:54 -0700 Subject: [PATCH] update README and CHANGES for 0.9.5! --- CHANGES.rst | 44 ++++++++++++++++++++++++++++++++++++++++++++ README.rst | 20 ++++++++++---------- 2 files changed, 54 insertions(+), 10 deletions(-) diff --git a/CHANGES.rst b/CHANGES.rst index 45f58cfb..8f0194b1 100644 --- a/CHANGES.rst +++ b/CHANGES.rst @@ -1,3 +1,47 @@ +pywb 0.9.5 changelist +~~~~~~~~~~~~~~~~~~~~~ + +* s3 loading: support ``s3://`` scheme in block loader, allowing for loading index and archive files from s3. ``boto`` library must be installed seperately + via ``pip install boto``. Attempt default boto auth path, and if that fails, attempt anonymous s3 connection. + +* Wombat Customizations: Pass ``rewrite_opts.client`` settings from ``config.yaml`` directly to wombat as json. Allow for customizing wombat as needed. + Currently supported: ``no_rewrite_prefixes`` list for skipping rewrite of certain path, and ``skip_dom``, ``skip_setAttribute`` and ``skip_postmessage`` + for disabling these overrides, respectively. + +* Revamp template setup: All templates now use shared env, which is created on first use or can be explicitly set (if embedding) + via ``J2TemplateView.init_shared_env()`` call. Support for specifiying a base env, as well as custom template lookup paths also provided + +* Template lookup paths can also be set via config options ``templates_dirs``. The default list is: ``templates``, ``.``, ``/`` in that order. + +* Embedding improvements: move custom env (``REL_REQUEST_URI`` setup) into routers, should be able to call router created by ``create_wb_router()`` directly + with WSGI enviorn and receive a callable response. + +* Embedding improvements: If set, contents of ``environ['pywb.template_params']`` dictionary are added directly to Jinja context, allowing for custom template +params to be passed to pywb jinja templates. + +* Root collection support: Can specify a route with `''` which will be the root collection. Fix routing paths to ensure root collection is checked last. + +* Customization: support custom route_class for cdx server and pass wbrequest to ``not_found_html`` error handlers. + +* Manager: Validate collection names to start with word char and contain alphanum or dash only. + +* CLI refactor: easier to create custom cli apps and pass params, inherit shared params. ``live-rewrite-server`` uses new system and accepts `-f` and `-x` +params along with standard. Also runs on ``/live/`` path by default. See ``live-rewrite-server -h`` for more details. + +* Add ``removeall`` cookie rewriter, which, removes all cookies (from replay)! + +* Security: disable file:// altogether for live rewrite path. + +* Fuzzy match: better support for custom replace string >1 character: leave string, and strip remainder before fuzzy query. + +* Urlrewriter and wburl fixes for various corner cases + +* Rangecache: use url as key if digest not present + +* Framed replay: attempt to mitigate chrome OS X scrolling issue by disabling `-webkit-transform: none` in framed mode. Improves scrolling + capability but not yet consistent (a chrome bug). + + pywb 0.9.3 changelist ~~~~~~~~~~~~~~~~~~~~~ diff --git a/README.rst b/README.rst index c4fa5181..f336c285 100644 --- a/README.rst +++ b/README.rst @@ -1,4 +1,4 @@ -PyWb 0.9.4-dev +PyWb 0.9.5-dev ============== .. image:: https://travis-ci.org/ikreymer/pywb.svg?branch=develop @@ -65,7 +65,7 @@ Point your browser to ``http://localhost:8080/my_coll//`` where ```` i If all worked well, you should see your archived version of ````. Congrats, you are now running your own web archive! -`A more detailed tutorial is available on the wiki `_ . +`A more detailed tutorial is available on the wiki `_ Using Existing Web Archive Collections @@ -140,7 +140,7 @@ To run with the bundled sample and test suite, you'll need to clone pywb locally To run tests on your system, you may run ``python setup.py test`` -(A few tests require the optional ``pyopenssl`` package and are skipped if the package is not installed) +(The HTTPS proxy tests require the optional ``certauth`` package and are skipped if the package is not installed) Additional Samples and Other Projects @@ -175,14 +175,14 @@ number of useful command-line and web server tools. The tools should be availabl See ``wb-manager --help`` for an up-to-date listing of commands and options. -* ``live-rewrite-server`` -- a demo live rewriting web server which accepts requests using wayback machine url format at ``/rewrite/`` path, eg, ``/rewrite/http://example.com/`` and applies the same url rewriting rules as are used for archived content. +* ``live-rewrite-server`` -- a demo live rewriting web server which accepts requests using wayback machine url format at ``/live/`` path, eg, ``/live/http://example.com/`` and applies the same url rewriting rules as are used for archived content. This is useful for checking how live content will appear when archived before actually creating any archive files, or for recording data. - The `webrecorder.io `_ service is built using this tool. + The `webrecorder.io `_ service extends upon this functionality. -* ``cdx-indexer`` -- a command-line tool for creating CDX indexs from WARC and ARC files. Supports SURT and - non-SURT based cdx files and optional sorting. See ``cdx-indexer -h`` for all options. - for all options. +* ``cdx-indexer`` -- a command-line tool for manually creating CDX indexs from WARC and ARC files. Supports SURT and + non-SURT based cdx files, optional sorting, and several formats. See ``cdx-indexer -h`` for all options. Using ``wb-manager`` is recommended + for higher-level collection file management, but this tool can be used for any custom indexing needs. * ``cdx-server`` -- a CDX API only server which returns a responses about CDX captures in bulk. See `CDX Server API `_ @@ -204,7 +204,7 @@ to serve live content, and inject customized code into any web page on the fly. For example, the `pywb-webrecorder `_ demonstrates a way to use pywb live web rewriting together with a recording proxy (warcprox) to record content while browsing. -The `via.hypothes.is `_ project provides an example of using pywb to inject annotations into any live web page. +The `via.hypothes.is `_ project provides an example of using pywb to inject annotations into any live web page. Running in HTTP/HTTPS Proxy Mode @@ -261,7 +261,7 @@ Additional Reference - The `wiki `_ will have additional technical documentation about various aspects of pywb -- The sample config.yaml file, although not required, will provide a listing of various advanced configuration options: +- The sample ``config.yaml`` file, although not required, provides a listing of various advanced configuration options: `config.yaml `_