2.3 Changelist + Docs Update (#487)

* docs: update changelist and add docs about new wombat

* update to latest wombat

* update wombat, fix pytest cmdline in setup

CHANGES.rst

@@ -1,3 +1,25 @@
+pywb 2.3.x changelist
+~~~~~~~~~~~~~~~~~~~~~
+
+* Wombat Improvements and modularization:
+    - Client-side rewriting and auto-fetch systems moved to https://github.com/webrecorder/wombat
+    - Module-based setup and full testing for wombat
+    - Continuous auto-fetch up to 20 requests (#484)
+
+* Replay / Fidelity Improvements (#451):
+    - Introduced a new server-side rewriter, JSWorkerRewriter, that handles rewriting JS workers and service-workers
+    - Improvements to JSOP Rewriter to handle empty query (#475)
+    - Improvements to postMessage rewriting, override `eval(` while preserving scope (#475)
+    - Fixes to ``this`` proxy rewrite to include ``, this``
+
+* Misc Changes:
+    - Improved handling of open http connections and file handles (#463)
+    - Fixes for latest urllib3, not verifying SSL certs (#467), (#469)
+    - Better logging for invalid cdxlines and cookies (#477), (#478)
+    - Fix warning in yaml.load (#472)
+    - Index invalid form-data as binary (#471)
+
+
 pywb 2.2.20190410 changelist
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 

docs/manual/configuring.rst

@@ -382,6 +382,8 @@ The detected urls are loaded in the background using a web worker while the user
 
 To enable this functionality, add ``--enable-auto-fetch`` to the command-line or ``enable_auto_fetch: true`` to the root of the ``config.yaml``
 
+The auto-fetch system is provided as part of the :ref:`wombat`
+
 
 Auto-Indexing Mode
 ------------------
@@ -408,6 +410,23 @@ The auto-indexing mode can also be enabled via command-line by running ``wayback
 (If running pywb with uWSGI in multi-process mode, the auto-indexing is only run in a single worker to avoid race conditions and duplicate indexing)
 
 
+.. _wombat:
+
+Client-Side Rewriting System (wombat.js)
+----------------------------------------
+
+In addition to server-side rewriting, pywb includes a Javascript client-rewriting system.
+
+This system intercepts network traffic and emulates the correct JS environment expected by a replayed page.
+
+The auto-fetch system is also implemented as part of wombat.
+
+Wombat was integrated into pywb upto 2.2.x. Starting with 2.3, wombat has been spun off into its own
+standalone JS module.
+
+For more information on wombat.js and client-side rewriting, see the `wombat README <https://github.com/webrecorder/wombat/blob/master/README.md>`_
+
+
 .. _https-proxy:
 
 HTTP/S Proxy Mode

docs/manual/rewriter.rst

@@ -7,6 +7,8 @@ pywb includes a sophisticated server and client-side rewriting systems, includin
 configuration for domain and content-specific rewriting rules, fuzzy index matching for replay,
 and a thorough client-side JS rewriting system.
 
+With pywb 2.3.0, the client-side rewriting system exists in a separate module at `https://github.com/webrecorder/wombat``
+
 
 URL Rewriting
 -------------

docs/manual/usage.rst

@@ -20,7 +20,7 @@ and introduces many new features, including:
 
 * Flexible rewriting system with pluggable rewriters for different content-types.
 
-* Significantly improved client-side rewriting to handle most modern web sites.
+* Significantly improved :ref:`wombat` to handle most modern web sites.
 
 * Improved 'calendar' query UI, grouping results by year and month, and updated replay banner.
 

setup.py

@@ -29,7 +29,7 @@ class PyTest(TestCommand):
         import os
         os.environ.pop('PYWB_CONFIG_FILE', None)
         cmdline = '--cov-config .coveragerc --cov pywb'
-        cmdline += ' -v --doctest-module ./pywb/ tests/'
+        cmdline += ' -v --doctest-modules ./pywb/ tests/'
 
         errcode = pytest.main(cmdline.split(' '))
 

wombat

@@ -1 +1 @@
-Subproject commit 2e6eb7a0c3772da61ff24f07bca7e69ccf346db4
+Subproject commit 5ad8ce852fcda08ce420af114c709116cd78b84e