* rewrite: add 'ir_' mod to support header only url-rewriting with no content rewriting
* tests: add tests for ir_ to test that content is identical to id_, but Location headers are rewritten with ir_ modifier.
* tests cleanup:
- move test requirements to test_requirements.txt to share between setup.py and tox.ini
- README: update to recommend using 'tox --current-env' for running tests locally
- replaces #741
* test tweaks:
- don't require i18n to import locmanager, instead set flag on load (to avoid breaking tox / pytest)
- don't add werkzeug to test requirements
- if a revisit is of a redirect (3xx response) and revisit has http headers, return
the http headers with empty payload -- don't bother loading the original record
builds on changes in #751
- cleanup redirect revisit tests from #751
* Enable translation for the remaining strings on the search results page
* Use toLocaleString() to format timestamps also for search results without matchType
* revisit loading fix for revisit records with http headers:
- if revisit record has http headers, always use those headers
- otherwise, continue to use http headers from payload record
- parse headers of http and payload records on initial lookup, to simplify loading
- tests: add test for loading revisit records with different urls, different headers but same payload
- fix for sul-dlss/was-pywb#64
* also bump version to 2.6.8
If a CDXJ entry has a status that is an int that can cause problems in
multiple places in pywb. This change ensures that int status lines are
converted to str.
Currently error messages display on a single line that can be difficult
to scroll. This updates the CSS slightly to allow the message to spread
over multiple lines if needed.
I was playing back a YouTube video and noticed that the playback worked
fine with using uwsgi/pywb directly but failed when using nginx. I think
a very long HTTP Link header was causing nginx to hang up. I increased
the uwsgi_buffer_size to 8k and the problem went away. Maybe this will
save someone else some time if it is increased?
https://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_buffer_size
* post append query: fix json parsing of lists to be identical to cdxj-indexer
if json parsing errors occur, log to stderr
fixes#709 in a better way
* update CHANGES.rst
* rewrite: add missing wordbreak to eval regex to avoid false positives, eg. '_eval' from being rewritten!
* dependencies: bump gevent to 21.12.0
* inputrequest: remove unnecessary print
* bump version to 2.6.7, update CHANGES for 2.6.7
* js rewriting: default to moden js-proxy based rewriting by default, use legacy rewriting only if browsers are older than minimum, as suggested in #707
* user-agent detection: use ua_parser for user-agent detection instead of obsolete werkzeug.useragent, which also did not support browsers >=100
* tests: additional tests for rewriting with various user-agents, defaulting to new-style rewriting for unknown browsers
* dockerfile: Update Dockerfile to use py3.8
* tests: skip s3 tests dependent on commoncrawl data (for now, need better s3 tests).
* bump to 2.6.6, update CHANGES
- when 'redirect_to_exact' is enabled, the top-frame expects a redirect for top-frame, however, live mode does not result in redirect to top-frame, so render live top-frame same as before
- tests: ensure top-frame loads correctly for live mode with redirect_to_exact enabled
- tests: fix webenact index tests
There are small typos in:
- pywb/utils/test/test_binsearch.py
- pywb/warcserver/resource/responseloader.py
- pywb/warcserver/resource/test/test_pathresolvers.py
Fixes:
- Should read `length` rather than `lenghth`.
- Should read `equals` rather than `eqauls`.
- Should read `assume` rather than `asume`.
The top two access types end in a ".". The final two do not. This simply adds periods to the third and fourth list items to make punctuation consistent among the bullets.
CHANGES: update changes for 2.6.3
location rewrite: pass 'arguments' to rewrite func to guard against rewriting local 'location' in some circumstances, partial fix for #684
ci: add automated docker push on new v-* tag
* template/custom env var fix:
- ensure pywb.host_prefix, pywb.app_prefix and pywb.static_prefix set for all requests via prepare_env()
- ensure X-Forwarded-Proto is accounted for in pywb.host_prefix
- call prepare_env() in handle_request(), and also in rewriterapp (in case using a different front-end app).
* update wombat to 3.3.6 (includes partial fix for #684)
* bump version to 2.6.3
* rules: add custom twitter video rewriting to capture non-chunked twitter video (max bitrate of 5000000)
* autoescaping regression fix: don't escape URL in frame_insert.html, use as is
* html rewriting:
- don't rewrite 'data-' attributes, no longer necessary for best fidelity
- do rewrite <link rel='alternate'> as main page (mp_)
- update html rewriting test
* feature: support customizing the static path used in pywb via 'static_prefix' config option (defaults to 'static')
* update to latest wombat (3.3.4)
* bump to 2.6.1, update CHANGES for 2.6.1
* eval fix: instead of rewriting to 'WB_wombat_eval', rewrite to 'self.eval' for non-top-level eval
the wombat object will handle rewriting the eval arg on 'self.eval'
tighten rewriting for top-level 'eval', add additional tests
part of fix for #663
* rewrite wrap: add extra {, } to avoid collisions, as suggested in webrecorder/wombat#72
eval rewrite: exclude ',eval' as more likely than not causing a false positive, as per #643
* update to latest wombat 3.3.0 with corresponding fixes
* add support for custom data being added via 'PUT /<coll>/record' when in recording mode and 'enable_put_custom_record: true' set in 'recorder' config
- url specified via 'url' query arg and content type via request Content-Type
- update docs for put custom record options
* bump version to 2.6.0b4
