* rewrite:
- don't rewrite xml in proxy mode / html-insert only mode
- ajax: if sec-fetch-mode is set to non-navigate, also treat as 'ajax'
* ci: build python 3.8, ignore 2.7 failures
* reqs: use released ujson for extra_reqs
* hmac: add digestmod, fix for py3.8
* py3.7 fixes:
- add __repr__ to WBException for consistent output in py3.7
- don't raise StopIteration in generator, just return
* ci: add py3.7 builds to travis and appveyor, (don't include in integration test suite for now)
* recoder fix: ensure Transfer-Encoding header is not passed through by RecorderApp,
as may result in duplicate Transfer-Encoding in py2.7, fixes#432
* html rewriter fixes:
- html detection: allow for UTF-8 BOM when detecting if text is html
- html decl parsing: modify base parser regex to allow IE conditional declaration to also
end with -->, eg. support '<![endif]-->' in addition to '<![endif]>', fixes#425
* travis: add allow failure for integration tests (for now)
update pip and setuptools when running install.sh found in .travis
use xenial
removed trailing dash
only run webrecorder-tests using chrome and firefox
only run webrecorder-tests using pywbtest and chrometest marker expression
- Split wombat and auto-fetch worker into two files (proxy mode and non-proxy mode)
- Renamed preservationWorker to autoFetchWorker in order to better convey what it does
- Root config file control over including wombat and auto-fetch worker in proxy or non-proxy mode
- Added additional proxy mode + auto-fetch worker only route for fetching the auto-fetch worker code nicely for CORS
- templateview: add 'tobool' formatter to more cleanly format python bools to JS 'true'/'false'
- proxy options: config and command line:
'use_auto_fetch_worker' and '--proxy-with-auto-fetch'
'use_wombat' and '--proxy-with-wombat'
- head_insert.html: only include wombat in proxy mode when use_wombat or use_auto_fetch_worker are set.
- wombatProxyMode.js: slimmed down wombat for proxy mode only including auto-fetch support.
- more consistent naming: rename 'preserveWorker' and 'autoArchive' to 'auto-fetch'
Updated tests:
- test_wbrequestresponse.py: added tests covering constructor defaults, _init_derived, options_response, json_response, encode_stream, text_stream
- test_auto_colls.py: fixed broken test test_more_custom_templates, reason using ujson now not json so spacing was off
- test_proxy.py: updated existing tests to reflect splitting wombat into proxy and non-proxy mode, added tests covering auto-fetch worker specific endpoints in proxy mode
removed duplicate addons key in .travis.yml
- test_cli.py: updated to properly test the cli with these changes
added ultrajon dep to tests_require in setup.py to reflect its usage by wbrequestresponse.py
Fully documented:
- cli.py
- frontendapp.py
- templateview.py
- wbrequestresponse.py
Removed duplicate addons key in .travis.yml
Added ultrajson dependency to tests_require in setup.py to reflect its usage by wbrequestresponse.py
Fixes#371
New integration tests using webrecorder-tests:
- WR_TEST=true is set for integration test run (only run with py3.6, excluded for py2.7, 3.5)
- Added .travis directory that includes two scripts: install.sh and test.sh.
- install.sh handles all installation and test.sh handles running of unit or integration tests
- sudo: true required to run headless chrome
setup.py just loads from requirements.txt
Dockerfile pip installs requirements, then extra requirements for improved cacheing
travis runs setup install, then installs extra requirements
proxy options: 'use_default_coll' must specify exact default coll
(otherwise a random coll is chosen, as ordering is not defined)
travis: add py3.4, py3.5!
Rather than specifying the username and encrypted
key in the Travis config, we just enable Sauce Connect
and require SAUCE_USERNAME and SAUCE_ACCESS_KEY env vars
to be set.
This is so that the Karma tests have the same env vars
available to them which they can use to check whether
to run against Sauce Labs or not.
For some Karma tests, we can run them faster by running
against a local browser in the Travis CI instance rather
than Sauce Labs.
This is also useful for:
- Contributors wishing to run
the Travis tests against their own forks and have not
set up Sauce Labs credentials.
- Running Karma tests against Pull Request builds where
Sauce Connect is not available.
This gives us the capability to test the client-side JS
rewriting against a suite of different browsers on different
platforms.
Note that getting Karma running on Travis CI requires some
additional configuration documented at
github.com/karma-runner/karma-sauce-launcher/issues/73
which is not made clear in the original documentation.
In order to run the Karma tests, the Sauce Labs username
and access keys need to be passed in via the SAUCE_USERNAME
and SAUCE_ACCESS_KEYS env vars.
* Add a Karma configuration for unit/integration tests
for the client-side pywb code.
* Add an integration test suite which creates an <iframe> loads
the client-side rewriting code (wombat.js) in it and
then executes a test script.
Since wombat.js monkey-patches the DOM and the exact behavior
of DOM objects varies between browsers, which we want to test,
the suite does not mock the DOM but instead runs
a set of tests in an isolated environment against
the DOM.
* Add Travis config to run the Karma tests
https proxy support contingest on openssl installation
if not installed and https use is enabled, a warning is printed
tests: updated tests to skip if openssl not installed
(make https post to only run in 2.7, as there are issues with in 2.6)
