1
0
mirror of https://github.com/webrecorder/pywb.git synced 2025-03-14 15:53:28 +01:00

196 Commits

Author SHA1 Message Date
Sara Tavares
c441d83435
chore(typos): fix typos across codebase (#811)
Co-authored-by: stavares843 <stavares843@users.noreply.github.com>
2023-02-15 13:04:20 -05:00
Ilya Kreymer
4a3e7ddff7 update CHANGES for 2.7.3 2023-02-02 16:22:24 -08:00
Ilya Kreymer
02288db81c
bump wombat to 3.4.4 (#808) 2023-02-02 16:21:02 -08:00
Ilya Kreymer
03f9708d8d CHANGELIST: Update changelist for 2.7.3 2023-02-01 17:44:50 -08:00
Tessa Walsh
0758e81b62
Add default_locale fix to 2.7.3 changelog 2023-01-31 15:02:48 -05:00
Tessa Walsh
9bc8a2e1ef
Modify search template buttons (#801)
* Modify search page button colors

* Rename Clear Options to Reset and reset URL as well

* Add search improvements to CHANGES
2023-01-25 13:33:18 -05:00
Tessa Walsh
cdab280669 Add 2.7.3 changes 2023-01-19 11:30:42 -05:00
Tessa Walsh
3c94da04a2
2.7.2 patch release (#787)
* Fix 2.7.1 regressions

* Bump to 2.7.2

* fix redirect-to-exact false:
- check if current loaded timestamp is the same as to-redirected to timestamp, and avoid reload

* additional ui fixes:
- location bar: reload with current timestamp, instead of going to calendar
- ensure calendar popup on replay view is scrollable
- 'Live' mode fixes: don't cache live cdx entry, don't add timestamp when navigating in live mode without timestamp
- remember timeline view toggle on replay
- title: add 'Archived Page: ' prefix to document.title, consistent with old version
- ensure 'Archived Page: ' text is localizable
- ui: change ',' to '|' on capture display

* update CHANGES for 2.7.2

Co-authored-by: Ilya Kreymer <ikreymer@gmail.com>
2022-12-08 16:35:39 -08:00
Tessa Walsh
2d19b6b18d
Merge 2.7.1 development branch (#785)
* Add locale-dependent handling of first day of week

The Intl.Locale is a proposed standard not yet supported by Firefox so
in Firefox the first day of week will default to Monday (as specified
in ISO-8601).

* Set top frame document title when Vue updates

* Update template guide for 2.7

* Drop Python 3.6 and add 3.10 in test CI

* Allow either JS mimetype in test_add_static

* Add convenience build script for Vue UI

* Add build flag to docker compose example

* Fix Vue app issue with redirect_to_exact: false

Fixes #779

Undated URLs were resulting in a broken calendar and timeline in the
Vue app when redirect_to_exact was set to false. This was due to
TopFrameView using the current datetime if no timestamp was included,
which caused a failed snapshot lookup in the Vue app.

This commit changes the default timestamp in TopFrameView to None and
adds additional logic in the Vue app to use the last snapshot's
timestamp as the default if one is not present to match the snapshot
that pywb loads by default under the same conditions.

* Add filter instead of submitting form when pressing enter in the filtering expression field

* Make filter expressions translatable

* Add missing tooltip strings to vue_loc

* Add changelog

* Bump version to 2.7.1

* Use empty string as default template timestamp

* Bump wombat to 3.3.13

Co-authored-by: Jonas Linde <jonasjlinde@gmail.com>
2022-12-07 18:16:18 -08:00
Tessa Walsh
138e2b284d Add 2.7.0 to changelog 2022-11-23 14:54:50 -05:00
Ilya Kreymer
1dedc46dce
ci: automated pypi publish on release (#776)
- dependencies: wombat 3.3.11
- ci: pypi publish for pywb
- update CHANGES for 2.6.9 release
2022-11-18 17:56:24 -08:00
Ilya Kreymer
6e7a8b1e59 CHANGES: Update changelist for 2.6.8
README: Remove unused appveyor badge (fixes #757)
2022-08-31 19:01:35 -07:00
Ilya Kreymer
4f44c2ec98
Post query json parse fix (#711)
* post append query: fix json parsing of lists to be identical to cdxj-indexer
if json parsing errors occur, log to stderr
fixes #709 in a better way

* update CHANGES.rst
2022-04-14 21:30:52 -07:00
Ilya Kreymer
09f7084aa1
pywb 2.6.7 (#710)
* rewrite: add missing wordbreak to eval regex to avoid false positives, eg. '_eval' from being rewritten!

* dependencies: bump gevent to 21.12.0

* inputrequest: remove unnecessary print

* bump version to 2.6.7, update CHANGES for 2.6.7
2022-04-14 20:21:24 -07:00
Ilya Kreymer
403167fbe0
User-Agent Detection Fix + New-Style rewriting on by default + Dependency Update (2.6.6) (#708)
* js rewriting: default to moden js-proxy based rewriting by default, use legacy rewriting only if browsers are older than minimum, as suggested in #707 
* user-agent detection: use ua_parser for user-agent detection instead of obsolete werkzeug.useragent, which also did not support browsers >=100
* tests: additional tests for rewriting with various user-agents, defaulting to new-style rewriting for unknown browsers
* dockerfile: Update Dockerfile to use py3.8
* tests: skip s3 tests dependent on commoncrawl data (for now, need better s3 tests).
* bump to 2.6.6, update CHANGES
2022-04-11 14:51:11 -07:00
Ilya Kreymer
42445562da
dependency fix (#697)
* add dependency bound (markupsafe<2.1.0)
* bump to 2.6.5
2022-02-20 16:36:28 -08:00
Ilya Kreymer
0f05dbde55 CHANGES: update changelist for 2.6.4 release 2022-01-25 23:19:23 -08:00
Ilya Kreymer
6bde8fd8c4 wombat.js: rebuild wombat.js to 3.3.6 (was not properly rebuilt previously), alternative fix to #690
update CHANGES
bump to 2.6.4
2022-01-19 18:35:39 -08:00
Ilya Kreymer
7ff789f1a8 CHANGES: fix typos in changelist 2021-12-22 17:42:19 -08:00
Ilya Kreymer
de9b9310d4
Additional fixes for 2.6.3 (#689)
CHANGES: update changes for 2.6.3

location rewrite: pass 'arguments' to rewrite func to guard against rewriting local 'location' in some circumstances, partial fix for #684

ci: add automated docker push on new v-* tag
2021-12-22 17:26:45 -08:00
Ilya Kreymer
e64e58f040
2.6.2 fix (#682)
2.6.2 release:
* fix for regression caused by 2.6.1, invalid static path #681
* add missing base.css
2021-11-12 17:51:34 -08:00
Ilya Kreymer
a6be76642a
2.6.1 Release Work (#679)
* rules: add custom twitter video rewriting to capture non-chunked twitter video (max bitrate of 5000000)

* autoescaping regression fix: don't escape URL in frame_insert.html, use as is

* html rewriting:
- don't rewrite 'data-' attributes, no longer necessary for best fidelity
- do rewrite <link rel='alternate'> as main page (mp_)
- update html rewriting test

* feature: support customizing the static path used in pywb via 'static_prefix' config option (defaults to 'static')

* update to latest wombat (3.3.4)

* bump to 2.6.1, update CHANGES for 2.6.1
2021-11-11 22:30:54 -08:00
Ilya Kreymer
96de80f83e update CHANGES for 2.6.0 release!
README: update for 2.6, add links to guides!
bump version to 2.6.0
2021-08-11 19:00:54 -07:00
Ilya Kreymer
342007244b update CHANGES.rst for 2.6.0b4
update wombat to latest
2021-07-18 17:13:12 -07:00
Ilya Kreymer
81308780ec
version display: add -V/--version flag to wb-manager and wayback/pywb commands to display version and exit (#654)
update CHANGES
comment out default locales in config.yaml
only show warning for installing i18n extra when locales actually specified in config

bump to 2.6.0b3
2021-06-24 11:28:48 -07:00
Ilya Kreymer
cff2a9efc5
more locale fixes: (#653)
* more locale fixes:
- fix running wb-manager w/o i18n dependencies
- dependencies: move babel to extra_requires, show warning if locale used or 'wb-manager i18n' called and i18n are not installed
- not found page: don't language switch header banner on nested content frame
2021-06-18 14:58:21 -07:00
Ilya Kreymer
3ca765f847 add autoescapding disable to banner.html
update CHANGES
bump version to 2.6.0b2
2021-06-17 17:40:15 -07:00
Ilya Kreymer
f7bd84cdac
Localization / doc fixes (#650)
* localization / doc fixes:
- add missing header.html
- docs: support 'i18n' extra, mention in docs
- use 'default_locale' for html lang tag
- access control docs: fix documentation for adding user with acl command

* localization: add compile_catalog after extract as well to simplify updates for identity (en) locale

* ui: 
- include locale in home page collection listing
- keep locale on error page home link

* autoescape:
- ensure jinja2 templates are autoescaped to prevent xss issues (thanks @sebastian-nagel for suggested fix)
- ensure banner inserts are not double-escaped
- update tests for template autoescaping

* update CHANGES.rst

* bump version to 2.6.0b1
2021-06-14 17:09:00 -07:00
Ilya Kreymer
12fcc87962
Localization Support (#647)
* add localization utilities:
- add locmanager to support extract, update, remove, list using pybabel
- add po2csv/csv2po conversion with translate-utils
- docs: add localization.rst to manual!

* add language switch header (via header.html) to all pages if more than one locale is present.

* localization: wrap more text strings in templates in existing templates

* docs:
- document `wb-manager i18n` commands
- mention `<html lang>` setting
- include csv example
- add info about adding localizable text in templates

* add localization to CHANGES
2021-06-09 13:12:53 -07:00
Ilya Kreymer
d95b79a8ab CHANGES: update changelist for 2.6.0b0
bump version to 2.6.0b0
2021-06-09 12:20:47 -07:00
Ilya Kreymer
94f6273a91
Update CHANGES.rst for 2.5.0! (#612) 2021-01-26 19:01:44 -08:00
Ilya Kreymer
69654fd013 update CHANGES for 2.5.0! 2021-01-26 18:54:37 -08:00
Ilya Kreymer
9e09bcd2a7
Docs Update: OpenWayback -> pywb Transition Guide (#588)
* docs work on OpenWayback -> pywb transition, part 1

* docs: add config change examples, exclusions and deploy recommendations

* update with path index example

* update terms with collection info

* docs update:
- add zipnum examples to owb-to-pywb config transition
- add working docker compose examples for nginx subdirectory, apache subdirectory and outback cdx deployment in ./sample-deploy
- update usage and owb-to-pywb deployment docs with updated subdiretory deployment info + sample-deploy links

* tweak exclusion info, deploy title

* add missing filee uwsgi_subdir.ini

* Docs: fix typos and clarifications from review (thanks @ldko!)

Co-authored-by: Lauren Ko <lauren.ko@unt.edu>

* docs: explain that existing cdx can be added to outbackcdx, explain reindexing is optional

* docs: elaborate on docker-compose examples

* minor tweaks

* update to latest wombat 3.0.2
* update CHANGES.rst

* bump version to 2.5.0 for release

Co-authored-by: Lauren Ko <lauren.ko@unt.edu>
2020-12-04 18:40:58 -08:00
Ilya Kreymer
bb1c2a3ec9
Fix logo (#575)
* pypi fixes: fix README to use logo w/o raw to work on pypi, resize logo

* update changelist, use abs path
2020-07-10 20:46:18 -07:00
Ilya Kreymer
9b8c187b3a
2.4.2 Develop->Master (#572)
* ensure that the RemoteCDXIndexSource also adds a 'matchType=' param, fix for ukwa-pywb/ukwa#57

* 2.4.2 fixes:
- cdxindexer: don't treat first param as output, require '-o <output>' instead, update tests
- cleanup: move url-polyfill.min.js to correct static dir, addresses #571
- update to latest wombat
- move logo to ./pywb/static, fix README path
- tests: update indexing tests for cdx-indexer fix
- bump version to 2.4.2
- Fix link in access-control docs to use RST instead of MD syntax (#568) (by @machawk1)
2020-07-10 20:22:58 -07:00
Ilya Kreymer
94b7fdcf97 minor fix: timegate check: allow timegate content check from #564 to be ignored if 'no_timegate_check' option is set (for use with derived classes)
bump version to 2.4.1
2020-06-08 17:12:18 -07:00
Ilya Kreymer
47e87ef387 CHANGES: bump version and update changelist for 2.4.0 2020-06-08 15:03:55 -07:00
Ilya Kreymer
fed3263ac6
Docs: Fix access controls and ui customizations docs links (#513)
* docs: ensure docs added to access controls, fix typos

* begin changelist for 2.4.0
2019-10-31 16:56:36 -07:00
Ilya Kreymer
56e7c78ea3
SOCKS Proxy Improvements (#504)
* https over socks fix: fix issue with https url handling by using 'adapter.proxy_manager_for()' instead of 'adapter.get_connection' to get proxy manager, which create connection indirectly (parallel to no-proxy path).
- simplify socks config, avoiding global monkey-patch, as requests/urllib3 now support socks proxy directly and do not require patching global socket.
- add SOCKS_DISABLE env dynamically disabling socks proxy
2019-08-29 11:59:45 -07:00
John Berlin
295f67e675 auto-fetch/wombat: updated wombat submodule to current master for 2.3.5 release (#503)
general auto-fetch improvements: 
- Fixed issue that caused HTTP 404 errors to happen when parsing <link> stylesheet hrefs as sheets (webrecorder/wombat#11)
- Ensured that auto-fetch requests made are cached by the browser (webrecorder/wombat#13 & webrecorder/wombat#15)
- Ensured that the request made by the backing web worker when in proxy mode are not blocked by CORS (webrecorder/wombat#13 & webrecorder/wombat#15)

updated changelist and bumped version to 2.3.5
2019-08-28 11:35:18 -07:00
Ilya Kreymer
cf5aceb4f5
HTML Unescape Improvements (#500)
* html-unescape fix:
- unescape any url that contains '&#' as it may be html-encoded
- unescape css blocks that contain '&#' as well, as they may contain css urls that need rewriting
* misc fixes:
- Update CHANGES
- Update to latest wombat
- Update reqs to surt 0.3.1, fix tests
2019-08-22 18:35:32 -07:00
Ilya Kreymer
bdf4a26807
cookie cache fix: don't cache headers for service workers generally (#499)
update CHANGELIST for 2.3.4
2019-08-20 14:54:23 -04:00
Ilya Kreymer
e79c657255
New Feature: support for autoFetch of urls deemed as pages by history api (pywb part) (#497)
* auto-fetch page fetch support:
- check for X-Wombat-History-Page header to indicate page url
- set title from X-Wombat-History-Title header, and attempt to parse <title> from response
- update auto-fetch workers in wombat
- update changelist, bump to 2.3.4
2019-08-12 13:34:33 -07:00
Ilya Kreymer
bf9284fec5
proxy mode HTMLInsertOnlyRewriter: (#496)
- insert head-insert before first tag that is not <html> or <head> insert before
- addresses issue with rewriting pages that have no <head> tag (already handled in full rewriter)
- tests: add tests for HTMLInsertOnlyRewriter
- bump version to 2.3.3, update changelist
2019-08-03 11:24:50 -07:00
Ilya Kreymer
42089e237b update CHANGELIST and version for 2.3.2 release 2019-08-01 16:23:31 -07:00
Ilya Kreymer
d4518ae557 update to latest wombat 3.0.0, fix issue with parent override (webrecorder/wombat#3)
bump version to 2.3.1
2019-07-10 18:09:22 -07:00
Ilya Kreymer
a4027c7904
Switch back to Semver for 2.3.0 (#488)
versioning: switch back to semver for 2.3.0, manual version updates
- rename update-version.sh -> update-tag.sh to push tag for existing versions
- bump version to 2.3.0 for release
2019-07-09 19:29:52 -07:00
Ilya Kreymer
11610f6e04
2.3 Changelist + Docs Update (#487)
* docs: update changelist and add docs about new wombat

* update to latest wombat

* update wombat, fix pytest cmdline in setup
2019-07-09 17:50:57 -07:00
Ilya Kreymer
77f8bb6476 CHANGES: update changelist
bump version to 2.2.20190410
2019-04-10 11:17:33 -07:00
Ilya Kreymer
9448f4fe45 release: update changelist for 2.2.20190311
docs: fix typos
2019-03-11 16:40:53 -07:00