backup/pywb
1
0
Fork 0
mirror of https://github.com/webrecorder/pywb.git synced 2025-03-14 15:53:28 +01:00
Code Issues Projects Releases Activity

Merge 2.7.1 development branch (#785)

Browse Source 
* Add locale-dependent handling of first day of week

The Intl.Locale is a proposed standard not yet supported by Firefox so
in Firefox the first day of week will default to Monday (as specified
in ISO-8601).

* Set top frame document title when Vue updates

* Update template guide for 2.7

* Drop Python 3.6 and add 3.10 in test CI

* Allow either JS mimetype in test_add_static

* Add convenience build script for Vue UI

* Add build flag to docker compose example

* Fix Vue app issue with redirect_to_exact: false

Fixes #779

Undated URLs were resulting in a broken calendar and timeline in the
Vue app when redirect_to_exact was set to false. This was due to
TopFrameView using the current datetime if no timestamp was included,
which caused a failed snapshot lookup in the Vue app.

This commit changes the default timestamp in TopFrameView to None and
adds additional logic in the Vue app to use the last snapshot's
timestamp as the default if one is not present to match the snapshot
that pywb loads by default under the same conditions.

* Add filter instead of submitting form when pressing enter in the filtering expression field

* Make filter expressions translatable

* Add missing tooltip strings to vue_loc

* Add changelog

* Bump version to 2.7.1

* Use empty string as default template timestamp

* Bump wombat to 3.3.13

Co-authored-by: Jonas Linde <jonasjlinde@gmail.com>
This commit is contained in:
Tessa Walsh 2022-12-07 21:16:18 -05:00 committed by GitHub
parent 6cc9cdc3ad
commit 2d19b6b18d
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
19 changed files with 168 additions and 80 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.github/workflows/ci.yaml vendored
View File

@ -8,7 +8,7 @@ jobs:
strategy:
max-parallel: 3
matrix:
python-version: [3.6, 3.7, 3.8, 3.9]
python-version: ['3.7', '3.8', '3.9', '3.10']
steps:
- name: checkout

10
CHANGES.rst
View File

@ -1,3 +1,13 @@
pywb 2.7.1 changelist
~~~~~~~~~~~~~~~~~~~~~
* Add locale-dependent handling of first day of week by @krakan in https://github.com/webrecorder/pywb/pull/781
* Make filter expressions translatable by @krakan in https://github.com/webrecorder/pywb/pull/783
* Add title to top frame in framed replay
* Add missing tooltip translation strings
* Fix calendar and timeline rendering for replay URLs without a timestamp
* Update template documentation
pywb 2.7.0 changelist
~~~~~~~~~~~~~~~~~~~~~

8
CONTRIBUTING.md
View File

@ -51,13 +51,11 @@ The first time you run this command, it make take some time to build.
Changes to the [Vue](https://vuejs.org/) frontend components require rebuilding the Vue bundle (`pywb/static/vue/vueui.js`) to take effect. After making changes to one or more Vue components, you can rebuild the static bundle and view the changes in your development environment like so:
```bash
cd pywb/vueui
yarn run build
cd ../..
docker compose up -d --force-recreate
./build-vue-ui.sh
docker compose up -d --build --force-recreate
```
Changes that modify pywb's Python dependencies or the operating system may require rebuilding the container:
Changes that modify pywb's Python dependencies or the operating system also require rebuilding the container:
```bash
docker compose up -d --build --force-recreate

6
build-vue-ui.sh Executable file
View File

@ -0,0 +1,6 @@
#!/bin/bash
CURR_DIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )
cd $CURR_DIR/pywb/vueui/
yarn run build

28
docs/manual/template-guide.rst
View File

@ -48,8 +48,7 @@ Base Templates (and supporting templates)
File: ``base.html``
This template includes the HTML added to all other pages, replay and non-replay. Shared JS and CSS includes can be added here.
For theming all pywb UI, it may be useful to modify this template.
This template includes the HTML added to all pages other than framed replay. Shared JS and CSS includes meant for pages other than framed replay can be added here.
To customize the default pywb UI across multiple pages, the following additional templates
can also be overriden:
@ -61,7 +60,7 @@ can also be overriden:
* ``footer.html`` -- Template for adding content as the "footer" of the ``<body>`` tag of the ``base`` template
Note: The default pywb ``head.html`` and ``footer.html`` are currently blank. They can be populated to customize the rendering, add analytics, etc... as needed.
Note: The default pywb ``head.html`` and ``footer.html`` are currently blank. They can be populated to customize the rendering, add analytics, etc... as needed. Content such as styles or JS code (for example for analytics) must be added to the ``frame_insert.html`` template as well (details on that template below) to also be included in framed replay.
The ``base.html`` template also provides five blocks that can be supplied by templates that extend it.
@ -172,9 +171,7 @@ Banner Template
File: ``banner.html``
This template is used to render the banner and is used both in framed replay and frameless replay.
In framed replay, the template is only rendered in the top/outer frame, while in frameless replay, it is added to every page.
This template is used to render the banner for framed replay. It is rendered only rendered in the top/outer frame.
Template variables:
@ -192,7 +189,17 @@ Template variables:
* ``{{ ui }}`` - an optional ``ui`` dictionary from ``config.yaml``, if any.
The default banner creates the UI dynamically in JavaScript using Vue.
The default banner creates the UI dynamically in JavaScript using Vue in the ``frame_insert.html`` template.
Custom Banner Template
^^^^^^^^^^^^^^^^^^^^^^
File: ``custom_banner.html``
This template can be used to render a custom banner for frameless replay. It is blank by default.
In frameless replay, the content of this template is injected into the ``head_insert.html`` template to render the banner.
Head Insert Template
@ -204,7 +211,7 @@ This template represents the HTML injected into every replay page to add support
This template is part of the core pywb replay, and modifying this template is not recommended.
For customizing the banner, modify the ``banner.html`` template instead.
For customizing the banner, modify the ``banner.html`` (framed replay) or ``custom_banner.html`` (frameless replay) template instead.
Top Frame Template
@ -221,16 +228,21 @@ This template is responsible for creating the iframe that will render the conten
This template only renders the banner and is designed *not* to set the encoding to allow the browser to 'detect' the encoding for the containing iframe.
For this reason, the template should only contain ASCII text, and %-encode any non-ASCII characters.
Content such as analytics code that is desired in the top frame of framed replay pages should be added to this template.
Template variables:
* ``{{ url }}`` - the URL being replayed.
* ``{{ timestamp }}`` - the timestamp being replayed, e.g. ``20211226`` in ``http://localhost:8080/pywb/20211226/mp_/https://example.com/``
* ``{{ wb_url }}`` - A complete ``WbUrl`` object, which contains the ``url``, ``timestamp`` and ``mod`` properties, representing the replay url.
* ``{{ wb_prefix }}`` - the collection prefix, e.g. ``http://localhost:8080/pywb/``
* ``{{ is_proxy }}`` - set to true if page is being loaded via an HTTP/S proxy (checks if WSGI env has ``wsgiprox.proxy_host`` set)
* ``{{ ui }}`` - an optional ``ui`` dictionary from ``config.yaml``, if any.
.. _custom-top-frame:

3
pywb/rewrite/templateview.py
View File

@ -405,10 +405,9 @@ class TopFrameView(BaseInsertView):
embed_url = wb_url.to_str(mod=replay_mod)
timestamp = ''
if wb_url.timestamp:
timestamp = wb_url.timestamp
else:
timestamp = timestamp_now()
is_proxy = 'wsgiprox.proxy_host' in env

13
pywb/static/query.js
View File

@ -57,14 +57,6 @@ function RenderCalendar(init) {
};
// regex for extracting the filter constraints and filter mods to human explanation
this.filterRE = /filter([^a-z]+)([a-z]+):(.+)/i;
this.filterMods = {
'=': 'Contains',
'==': 'Matches Exactly',
'=~': 'Matches Regex',
'=!': 'Does Not Contains',
'=!=': 'Is Not',
'=!~': 'Does Not Begins With'
};
this.text = init.text;
this.versionString = null;
}
@ -433,7 +425,6 @@ RenderCalendar.prototype.createContainers = function() {
return;
}
// create the advanced results query info DOM structure
var forString = ' for ';
var forElems;
if (this.queryInfo.searchParams.matchType) {
@ -503,7 +494,7 @@ RenderCalendar.prototype.createContainers = function() {
{
tag: 'p',
className: 'text-center mb-0 mt-1',
innerText: 'Filtering by'
innerText: filteringBy
},
{
tag: 'ul',
@ -950,7 +941,7 @@ RenderCalendar.prototype.niceFilterDisplay = function() {
filterList.push({
tag: 'li',
className: 'list-group-item',
innerText: match[2] + ' ' + this.filterMods[match[1]] + ' ' + match[3]
innerText: match[2] + ' ' + filterMods[match[1]] + ' "' + match[3] + '"'
});
}
}

51
pywb/static/search.js
View File

@ -1,14 +1,6 @@
var dtRE = /^\d{4,14}$/;
var didSetWasValidated = false;
var showBadDateTimeClass = 'show-optional-bad-input';
var filterMods = {
'=': 'Contains',
'==': 'Matches Exactly',
'=~': 'Matches Regex',
'=!': 'Does Not Contains',
'=!=': 'Is Not',
'=!~': 'Does Not Begins With'
};
var elemIds = {
filtering: {
@ -65,7 +57,7 @@ function makeCheckDateRangeChecker(dtInputId, dtBadNotice) {
function createAndAddNoFilter(filterList) {
var nothing = document.createElement('li');
nothing.innerText = 'No Filter';
nothing.innerText = noFilter;
nothing.id = elemIds.filtering.nothing;
filterList.appendChild(nothing);
}
@ -78,19 +70,24 @@ function addFilter(event) {
if (!expr) return;
var filterExpr = 'filter' + modifier + by + ':' + expr;
var filterList = document.getElementById(elemIds.filtering.list);
var previousFilters = filterList.children;
for (var i = 0; i < previousFilters.length; ++i) {
var filterData = previousFilters[i].dataset;
if (filterData && filterData.filter && filterData.filter == filterExpr) return;
}
var filterNothing = document.getElementById(elemIds.filtering.nothing);
if (filterNothing) {
filterList.removeChild(filterNothing);
}
var li = document.createElement('li');
li.innerText =
'By ' +
by[0].toUpperCase() +
by.substr(1) +
' ' +
filterMods[modifier] +
' ' +
expr;
' "' +
expr +
'"';
li.dataset.filter = filterExpr;
var nukeButton = document.createElement('button');
nukeButton.type = 'button';
@ -110,6 +107,7 @@ function addFilter(event) {
};
li.appendChild(nukeButton);
filterList.appendChild(li);
return true;
}
function clearFilters(event) {
@ -166,6 +164,17 @@ function validateFields(form) {
}
}
function submitForm(event, form, searchURLInput) {
event.preventDefault();
event.stopPropagation();
var url = searchURLInput.value;
if (!url) {
validateFields(form);
return;
}
performQuery(url);
}
$(document).ready(function() {
$('[data-toggle="tooltip"]').tooltip({
container: 'body',
@ -184,16 +193,18 @@ $(document).ready(function() {
var searchURLInput = document.getElementById(elemIds.url);
var form = document.getElementById(elemIds.form);
form.addEventListener('submit', function(event) {
event.preventDefault();
event.stopPropagation();
var url = searchURLInput.value;
if (!url) {
validateFields(form);
return;
}
performQuery(url);
submitForm(event, form, searchURLInput);
});
document.getElementById(elemIds.advancedOptions).onclick = function() {
validateFields(form);
}
var filteringExpression = document.getElementById(elemIds.filtering.expression);
filteringExpression.addEventListener("keypress", function(event) {
if (event.key === "Enter") {
event.preventDefault();
if (! addFilter()) {
submitForm(event, form, searchURLInput);
}
}
});
});

43
pywb/static/vue/vueui.js
View File

File diff suppressed because one or more lines are too long

2
pywb/static/wombat.js
View File

File diff suppressed because one or more lines are too long

10
pywb/templates/query.html
View File

@ -71,6 +71,16 @@
},
};
var filterMods = {
'=': "{{ _('Contains') }}",
'==': "{{ _('Matches Exactly') }}",
'=~': "{{ _('Matches Regex') }}",
'=!': "{{ _('Does Not Contain') }}",
'=!=': "{{ _('Is Not') }}",
'=!~': "{{ _('Does Not Begin With') }}"
};
var filteringBy = "{{ _('Filtering by') }}";
var forString = " {{ _('for') }} ";
var renderCal = new RenderCalendar({ prefix: "{{ prefix }}", staticPrefix: "{{ static_prefix }}", text: text });
renderCal.init();
</script>

20
pywb/templates/search.html
View File

@ -3,8 +3,18 @@
{% block head %}
{{ super() }}
<script>
// TODO: cleanup
window.wb_prefix = "{{ wb_prefix }}";
var filterMods = {
'=': "{{ _('Contains') }}",
'==': "{{ _('Matches Exactly') }}",
'=~': "{{ _('Matches Regex') }}",
'=!': "{{ _('Does Not Contain') }}",
'=!=': "{{ _('Is Not') }}",
'=!~': "{{ _('Does Not Begin With') }}"
};
var noFilter = "{{ _('No Filter') }}";
// TODO: cleanup
window.wb_prefix = "{{ wb_prefix }}";
</script>
<script src="{{ static_prefix }}/search.js"></script>
{% endblock %}
@ -24,14 +34,14 @@ window.wb_prefix = "{{ wb_prefix }}";
<label for="search-url" class="lead" aria-label="Search For Col">
{% set coll_title = metadata.title if metadata and metadata.title else coll %}
{% autoescape false %}
{% trans %}Search the {{ coll_title }} collection by url: {% endtrans %}
{% trans %}Search the {{ coll_title }} collection by url:{% endtrans %}
{% endautoescape %}
</label>
<input aria-label="url" aria-required="true" class="form-control form-control-lg" id="search-url"
name="search" placeholder="{{ _('Enter a URL to search for') }}"
title="{{ _('Enter a URL to search for') }}" type="search" required/>
<div class="invalid-feedback">
{% trans %}'Please enter a URL{% endtrans %}
{% trans %}Please enter a URL{% endtrans %}
</div>
</div>
</div>
@ -131,7 +141,7 @@ window.wb_prefix = "{{ wb_prefix }}";
<option value="=~">{% trans %}Matches Regex{% endtrans %}</option>
<option value="=!">{% trans %}Does Not Contain{% endtrans %}</option>
<option value="=!=">{% trans %}Is Not{% endtrans %}</option>
<option value="=!~">{% trans %}Does Not Begins With{% endtrans %}</option>
<option value="=!~">{% trans %}Does Not Begin With{% endtrans %}</option>
</select>
</div>
<div class="row">

11
pywb/templates/vue_loc.html
View File

@ -43,10 +43,13 @@
sat_long: "{{ _Q('Saturday') }}",
sun_long: "{{ _Q('Sunday') }}",
"All-time": "{{ _Q('All-time') }}",
"show timeline":"{{ _Q('show timeline') }}",
"hide timeline":"{{ _Q('hide timeline') }}",
"show calendar":"{{ _Q('show calendar') }}",
"hide calendar":"{{ _Q('hide calendar') }}",
"Show timeline":"{{ _Q('Show timeline') }}",
"Hide timeline":"{{ _Q('Hide timeline') }}",
"Show calendar":"{{ _Q('Show calendar') }}",
"Hide calendar":"{{ _Q('Hide calendar') }}",
"Previous capture":"{{ _Q('Previous capture') }}",
"Next capture":"{{ _Q('Next capture') }}",
"Select language":"{{ _Q('Select language') }}",
"View capture on {date}":"{{ _Q('View capture on {date}') }}",
"{count} capture":"{{ _Q('{count} capture') }}",
"{count} captures":"{{ _Q('{count} captures') }}",

2
pywb/version.py
View File

@ -1,4 +1,4 @@
__version__ = '2.7.0'
__version__ = '2.7.1'
if __name__ == '__main__':
print(__version__)

8
pywb/vueui/src/App.vue
View File

@ -192,6 +192,10 @@ export default {
// when the user navigates there via browser back/forward buttons
addEventListener('unload', (event) => { });
},
updated: function() {
// set top frame title equal to value pulled from replay frame
document.title = this.config.title;
},
computed: {
sessionStorageUrlKey() {
// remove http(s), www and trailing slash
@ -326,7 +330,9 @@ export default {
this.config.url = view.url;
let periodToChangeTo = this.currentPeriod.findByFullId(snapshot.getFullId());
this.gotoPeriod(periodToChangeTo, false /* onlyZoomToPeriod */);
if (periodToChangeTo) {
this.gotoPeriod(periodToChangeTo, false /* onlyZoomToPeriod */);
}
},
setTimelineView() {
this.showTimelineView = !this.showTimelineView;

4
pywb/vueui/src/components/CalendarMonth.vue
View File

@ -126,8 +126,8 @@ export default {
const days = [];
// Get days in month, and days in the complete weeks before first day and after last day
const [firstDay, lastDay] = this.month.getChildrenRange();
const daysBeforeFirst = (new Date(this.year.id, this.month.id-1, firstDay)).getDay();
const daysAfterLastDay = (6 - (new Date(this.year.id, this.month.id-1, lastDay)).getDay());
const daysBeforeFirst = (7 + (new Date(this.year.id, this.month.id-1, firstDay)).getDay() - PywbI18N.firstDayOfWeek) % 7;
const daysAfterLastDay = (6 - (new Date(this.year.id, this.month.id-1, lastDay)).getDay() + PywbI18N.firstDayOfWeek) % 7;
for(let i=0; i<daysBeforeFirst; i++) {
days.push(null);
}

6
pywb/vueui/src/i18n.js
View File

@ -3,12 +3,15 @@ export class PywbI18N {
static getLocale() { // get via public static method
return PywbI18N.#locale;
}
static firstDayOfWeek = 1;
static init = (locale, config) => {
if (PywbI18N.instance) {
throw new Error('cannot instantiate PywbI18N twice');
}
PywbI18N.#locale = locale;
PywbI18N.instance = new PywbI18N(config);
let intlLocale = new Intl.Locale(PywbI18N.getLocale());
if ('weekInfo' in intlLocale) PywbI18N.firstDayOfWeek = intlLocale.weekInfo.firstDay % 7;
}
// PywbI18N expects from the i18n string source to receive months SHORT and LONG names in the config like this:
@ -36,7 +39,8 @@ export class PywbI18N {
return decodeURIComponent(this.config[id+'_'+type])
}
getWeekDays(type='long') {
return ['sun', 'mon', 'tue', 'wed', 'thu', 'fri', 'sat'].map(d => this.getWeekDay(d, type));
let weekDays = ['sun', 'mon', 'tue', 'wed', 'thu', 'fri', 'sat'];
return weekDays.concat(weekDays).slice(PywbI18N.firstDayOfWeek, PywbI18N.firstDayOfWeek + 7).map(d => this.getWeekDay(d, type));
}
getText(id, embeddedVariableStrings=null) {
const translated = decodeURIComponent(this.config[id] || id);

19
pywb/vueui/src/index.js
View File

@ -39,6 +39,7 @@ class CDXLoader {
}
let queryURL;
let isQueryURL = window.location.href.indexOf("*") > -1 ? true : false;
// query form *?=url...
if (window.location.href.indexOf("*?") > 0) {
@ -61,7 +62,7 @@ class CDXLoader {
this.app = this.initApp({logoImg, navbarBackground, navbarColor, navbarLightButtons, url, allLocales});
this.loadCDX(queryURL).then((cdxList) => {
this.setAppData(cdxList, timestamp ? {url, timestamp}:null);
this.setAppData(cdxList, url, isQueryURL, timestamp);
});
}
@ -100,18 +101,26 @@ class CDXLoader {
params.set("url", url);
params.set("output", "json");
const queryURL = this.prefix + "cdx?" + params.toString();
let isQueryURL = window.location.href.indexOf("*") > -1 ? true : false;
const cdxList = await this.loadCDX(queryURL);
this.setAppData(cdxList, {url, timestamp});
this.setAppData(cdxList, url, isQueryURL, timestamp);
}
setAppData(cdxList, snapshot=null) {
setAppData(cdxList, url, isQueryURL, timestamp="") {
this.app.setData(new PywbData(cdxList));
if (snapshot) {
// if this is a capture but we don't have a timestamp (e.g. if redirect_to_exact is false)
// set the timestamp to the latest capture
if ((!timestamp) && (!isQueryURL)) {
const lastSnapshot = cdxList[cdxList.length - 1];
timestamp = lastSnapshot.timestamp;
}
if (timestamp) {
this.app.hideBannerUtilities();
this.app.setSnapshot(snapshot);
this.app.setSnapshot({url, timestamp});
}
}

2
tests/test_auto_colls.py
View File

@ -203,7 +203,7 @@ class TestManagedColls(CollsDirMixin, BaseConfigTest):
resp = self.testapp.get('/static/_/test/abc.js')
assert resp.status_int == 200
assert resp.content_type == 'application/javascript'
assert resp.content_type in ('application/javascript', 'text/javascript')
resp.charset = 'utf-8'
assert '/* Some JS File */' in resp.text