Commit Graph - pywb - Source code and issue tracker for Open Eggbert

backup/pywb

Fork 0

mirror of https://github.com/webrecorder/pywb.git synced 2025-03-15 08:04:49 +01:00

Commit Graph

Select branches

Hide Pull Requests

0.1

0.11.3

0.11.4

0.11.5

0.2.0

0.3.0

0.33.2

0.4.0

0.4.5

0.4.7

0.5.4

0.6.0

0.6.6

0.7.7

0.8.3

2.7.0

2.7.4-work

add-logo

auto-scalar-proto

bump-dockerfile-python

ci-test

cond_rules

dependabot/npm_and_yarn/pywb/vueui/braces-3.0.3

dependabot/npm_and_yarn/pywb/vueui/semver-5.7.2

dependabot/npm_and_yarn/pywb/vueui/word-wrap-1.2.4

dev/issue-674-module-rewriting

develop-new

develop-with-wombat

disable-query-fuzzy-match

docs

embeds-url-scheme

final-2.4.0-fixes

fix-travis

history-own-window

hls-native-fix

issue-859-json-post-values

issue-924-client-side-playback

issue-sec-12-path-traversal

lists

main

new-ui-work

new-ui-work-i18n

patch-work

record-history

record-history-new

redis-key-log

remote-cdx-prefix

scheme-rel-change-ts

seq-filters

sub-coll

timegate-check-opt

ukwa-merge-even-master

ukwa-pywb-terms-of-use-popup

update-wombat

warcio-1.5.3

werkzeug-version

#1

#102

#127

#133

#133

#139

#149

#149

#150

#150

#151

#151

#152

#153

#154

#155

#155

#156

#157

#159

#162

#166

#172

#183

#19

#19

#194

#2

#203

#209

#21

#212

#215

#215

#218

#22

#220

#221

#224

#225

#226

#227

#228

#229

#230

#231

#232

#233

#234

#235

#236

#237

#240

#241

#242

#243

#244

#245

#246

#247

#248

#25

#250

#251

#252

#253

#254

#255

#256

#257

#258

#258

#259

#26

#260

#261

#262

#263

#264

#265

#266

#268

#269

#27

#272

#272

#274

#276

#277

#278

#279

#28

#280

#281

#282

#285

#286

#287

#287

#288

#29

#3

#30

#30

#303

#303

#304

#306

#308

#313

#315

#317

#318

#32

#320

#320

#321

#323

#323

#325

#327

#328

#329

#330

#331

#332

#334

#335

#336

#337

#338

#339

#340

#341

#345

#346

#347

#348

#349

#349

#350

#351

#352

#353

#354

#355

#356

#357

#357

#358

#358

#359

#360

#361

#362

#363

#364

#365

#366

#367

#368

#369

#372

#373

#374

#375

#376

#377

#378

#379

#380

#381

#382

#383

#384

#384

#385

#386

#387

#389

#392

#394

#395

#396

#397

#398

#400

#401

#402

#403

#404

#409

#409

#415

#418

#420

#421

#423

#426

#427

#428

#430

#431

#436

#437

#438

#440

#441

#442

#443

#444

#445

#446

#447

#448

#450

#451

#453

#454

#456

#457

#460

#463

#468

#469

#470

#472

#473

#475

#477

#478

#479

#480

#481

#482

#483

#484

#486

#487

#488

#489

#490

#491

#493

#494

#495

#496

#497

#498

#499

#5

#5

#500

#503

#504

#507

#509

#511

#512

#513

#514

#515

#516

#517

#518

#519

#520

#523

#524

#529

#53

#530

#531

#534

#536

#540

#543

#544

#557

#559

#561

#563

#564

#565

#566

#568

#572

#574

#575

#583

#586

#586

#587

#587

#588

#590

#592

#596

#597

#599

#6

#600

#603

#604

#606

#609

#610

#611

#612

#613

#617

#620

#621

#623

#624

#625

#626

#629

#630

#631

#633

#634

#635

#636

#637

#640

#642

#647

#649

#650

#651

#653

#654

#655

#657

#659

#661

#662

#664

#666

#668

#669

#670

#675

#679

#682

#683

#687

#688

#689

#690

#691

#692

#697

#7

#701

#705

#706

#708

#710

#711

#712

#716

#721

#722

#723

#725

#726

#730

#732

#733

#737

#738

#739

#741

#742

#746

#747

#748

#749

#751

#752

#753

#754

#755

#758

#759

#760

#761

#763

#767

#770

#772

#774

#775

#776

#777

#780

#781

#783

#785

#786

#787

#791

#793

#794

#796

#797

#800

#801

#805

#806

#808

#809

#810

#810

#811

#812

#812

#813

#814

#815

#816

#819

#820

#823

#839

#841

#841

#845

#855

#855

#856

#856

#864

#866

#868

#871

#873

#874

#877

#877

#882

#888

#889

#891

#892

#893

#895

#896

#899

#900

#901

#902

#904

#905

#911

#911

#912

#914

#914

#916

#916

#917

#917

#920

#926

#926

#927

#928

#928

#98

0.10.1

0.10.10

0.10.5

0.10.8

0.10.9

0.10.9.1

0.11.0

0.11.4

0.2.2

0.3.0

0.32.0

0.4.0

0.5.0

0.6.0

0.6.2

0.6.4

0.6.5

0.7.0

0.7.1

0.7.5

0.8.0

0.9.0

0.9.5

2.4.0-beta

v-2.2.20190227

v-2.2.20190311

v-2.2.20190410

v-2.3.0

v-2.3.1

v-2.4.0

v-2.4.0-beta

v-2.4.0-rc0

v-2.4.0-rc1

v-2.4.0-rc2

v-2.4.0-rc5

v-2.4.0-rc6-test

v-2.4.0-rc7

v-2.4.0rc4

v-2.4.1

v-2.4.2

v-2.4.2-beta

v-2.5.0

v-2.6.0

v-2.6.0b0

v-2.6.0b1

v-2.6.1

v-2.6.2

v-2.6.3

v-2.6.3b0

v-2.6.4

v-2.6.6

v-2.6.7

v-2.6.8

v-2.6.9

v-2.7.0

v-2.7.1

v-2.7.2

v-2.7.3

v-2.7.4

v-2.8.0

v-2.8.1

v-2.8.2

v-2.8.3

138aed3ddd change version to 0.9.0b1 Ilya Kreymer 2015-03-16 19:15:00 -07:00
2f6780a576 rename for 0.9.0: rename default templates package from ui/* templates to templates/* rename default subdirs: warcs -> archive, cdx -> indexes Ilya Kreymer 2015-03-16 18:48:09 -07:00
19b8650891 manager: templates: add collections manager (#74) commands for adding, removing and listing available ui templates. Support for both collection and shared templates. confirmation for overwrite/remove updated full template list in default_config and added tests Ilya Kreymer 2015-03-16 16:53:24 -07:00
3d53fdde9e cleanup: remove unused __str__ from Handlers / Route, not as useful anymore Ilya Kreymer 2015-03-15 22:55:23 -07:00
be5139b635 fix tests for coll listing, #78 config override: when loading from coll-specific config.yaml, resolve relative paths to that collection, not to root #55 Ilya Kreymer 2015-03-15 22:23:08 -07:00
30454abb6b metadata: add support for user-defined per-collection metadata! #78 metadata stored in wbrequest.user_metadata and available to all templates Ilya Kreymer 2015-03-15 21:20:00 -07:00
b417b47835 collections manager: support for merge when adding warc, explicit --index-warcs option to index and merge instead of reindexing whole dir, #74 additional testing for recursive indexing, index merge timeutils: add timestamp20_now() function Ilya Kreymer 2015-03-14 14:56:15 -07:00
759d151551 tests: add test for directory auto collection loader, collection manager and new 6-field minimal cdx format Ilya Kreymer 2015-03-13 19:53:50 -07:00
1ba24de357 Merge branch 'develop' into config-work Ilya Kreymer 2015-03-13 11:53:27 -07:00
b4b92482ad Merge branch 'develop' for 0.8.3 0.8.3 Ilya Kreymer 2015-03-13 11:06:52 -07:00
b2ce3feb80 readme fix Ilya Kreymer 2015-03-13 11:05:32 -07:00
3e3794d4dc Update CHANGES for 0.8.3 Ilya Kreymer 2015-03-13 11:04:37 -07:00
24021fcd57 html rewrite: add trailing slash for <base> tag rewrite if url is a scheme://host with no path component #77 cleanup: remove unused code path for tags with no rewriting -- all tags now checked for dynamic attrs which may need rewriting update tests, including live rewrite test dependent on live site (FB) Ilya Kreymer 2015-03-13 10:53:57 -07:00
fe1683da56 indexing: for minimal index, use a single -m flag to create a 6 field index. minimal index also skips parsing contents of warc/arc records altogether add cli docs for minimal index, tracked via #75 Ilya Kreymer 2015-03-07 11:56:17 -08:00
499e21233e statusandheaders: make protocol check case-insensitive, eg. accept HTTP/1.0 and http/1.0 for better compatibility Ilya Kreymer 2015-03-07 11:37:06 -08:00
5aa497dc68 Merge branch 'develop' into config-work Ilya Kreymer 2015-03-06 21:09:21 -08:00
1fb631870b wb_frame: fix extra slash typo in replaced frame url Ilya Kreymer 2015-03-05 17:04:44 -08:00
f2d7bd074a bump version to 0.8.3 cookie rewrite: remove 'secure' flag if present Ilya Kreymer 2015-03-05 16:18:56 -08:00
1eadd35598 Merge branch 'develop' for 0.8.2 Ilya Kreymer 2015-02-28 09:05:09 -08:00
6c8cb806d9 update 0.8.2 changelist, minor fixes Ilya Kreymer 2015-02-28 09:04:15 -08:00
48eab2662d cdx indexer: refactor indexer into mixins for differnt formats for easier customization Ilya Kreymer 2015-02-25 16:45:47 -08:00
ee1fabf600 config fix: check for existance of root 'collections dir', #55 Ilya Kreymer 2015-02-25 13:49:39 -08:00
11c8cc92f3 add beta to README Ilya Kreymer 2015-02-25 13:33:42 -08:00
671f45f69f cdx indexing: wrap record iterator global functions in class DefaultRecordIter to allow for better extensibility add 'minimal' option to skip digest/mime/status extraction only include minimal data (url+timestamp) cdx-indexer: add -6 option to create 6-field index Ilya Kreymer 2015-02-25 13:31:37 -08:00
1d4c54deaa frames ui: update frames to use <!DOCTYPE html>, improved css and html5 compatibility Ilya Kreymer 2015-02-25 13:25:05 -08:00
60f33412ff collections manager: add new collections manager, first pass #74 add cli 'wb-manager' tool very preliminary, needs testing still Ilya Kreymer 2015-02-25 13:19:20 -08:00
69613a0e25 tests: disable 'invalid config' test as its no longer applicable, fix default banner to just 'banner.html' Ilya Kreymer 2015-02-25 13:18:32 -08:00
5c67782a2c config system: some fixes for auto-init, add trailing '/' for dir paths, #55 Ilya Kreymer 2015-02-25 13:15:48 -08:00
7c60bf17f7 bump version to 0.9.0-beta! Ilya Kreymer 2015-02-24 16:54:49 -08:00
e39d6e207c config & collections: auto static path and templates working! #55 Ilya Kreymer 2015-02-24 14:32:51 -08:00
a932235f85 Merge branch 'develop' into config-work Ilya Kreymer 2015-02-24 10:40:58 -08:00
cb857df125 memento: fix MementoTimemapView to have consistent signature with other query views Ilya Kreymer 2015-02-24 10:35:49 -08:00
39824711f0 memento tweak: ensure rel=memento link for timegate uses exact in Location (cdx original) as opposed to url from request Ilya Kreymer 2015-02-23 23:21:39 -08:00
435fa390ed config system: initial work on automated directory-convention based config! config.yaml file now optional, add default_config.yaml which for default settings #55 Ilya Kreymer 2015-02-23 21:59:41 -08:00
5d80d2d891 replay: change strip_scheme() to strip_scheme_www() to also strip away www. prefix for self-redirect checking, #73 Ilya Kreymer 2015-02-22 22:51:35 -08:00
83f8d7d29b bump version to 0.8.2 Ilya Kreymer 2015-02-22 22:51:23 -08:00
de40e2920a update README for 0.8.1 Ilya Kreymer 2015-02-21 14:29:40 -08:00
7989c06ea4 Add webarchiveplayer link to README Ilya Kreymer 2015-02-21 14:28:04 -08:00
80da0e91da update CHANGELIST for 0.8.1 Ilya Kreymer 2015-02-21 14:13:35 -08:00
9f838241c7 wb.js bug fix: use only window.__orig_parent and not window.parent, as window.parent overriden. window instead of window.self Ilya Kreymer 2015-02-21 12:34:35 -08:00
c0ff596c68 tests: add tests for recursive cdx indexing, #64 cross-platform: store rel filename path as '/', but convert to os.path.sep when resolving to full path as prefix Ilya Kreymer 2015-02-20 13:56:35 -08:00
8d52be4c44 live proxy: enable ssl validation for live proxy, was initially disabled for testing, should be on by default! Ilya Kreymer 2015-02-20 13:22:21 -08:00
1646c90cd0 cdxindexer: add -r option to support recursive indexing when input is a directory. filename field in cdx contains relative path including subdir, eg. subdir/file.warc.gz related to #64 Ilya Kreymer 2015-02-20 02:34:09 -08:00
adeb8bfb27 bump version to 0.8.1, (fix blank spacing in changelist) Ilya Kreymer 2015-02-20 02:02:34 -08:00
cb6aebf06d Merge CHANGES.rst from 'develop' 0.8.0 Ilya Kreymer 2015-02-19 01:29:22 -08:00
bf203a2dc6 Merge branch 'develop' of https://github.com/ikreymer/pywb into develop Ilya Kreymer 2015-02-19 01:29:03 -08:00
121e1df3c9 README: update branch config to master Ilya Kreymer 2015-02-19 01:26:55 -08:00
824587bd90 A few more CHANGES.rst tweaks Ilya Kreymer 2015-02-19 01:24:52 -08:00
26df8d7784 remove debug logging and spaces Ilya Kreymer 2015-02-19 01:17:31 -08:00
0ddc490b8d Update CHANGELIST for 0.8.0! Ilya Kreymer 2015-02-19 01:16:25 -08:00
80dcb6ff27 rewrite: improvements to non-exact replay mode, redir_to_exact option set to false frames: add request_ts to wbinfo and use that as the timestamp in the top-frame. for exact replay, request_ts == timestamp for latest replay / no timestamp / memento timegate, redirect to current time instead of time of last capture, while serving last capture. timeutils: add timestamp_now() function to return timestamp of current datetime Add extra tests for this mode Tracked via #72 Ilya Kreymer 2015-02-17 17:47:30 -08:00
9623f95439 memento: add rel="memento" header to timegate as well, improve memento test, clearly differntiate between timegate redirect and intermediate resource redirect, related to #70 Ilya Kreymer 2015-02-16 09:59:03 -08:00
c4d5dd4690 rewrite: optimize / sanity, only %-encode urls that are actually idna-encoded, otherwise return as is, #66 Ilya Kreymer 2015-02-15 10:34:56 -08:00
afe49a91f4 rewrite: more fixes for IDN #66 - add _do_percent_encode field to wburl itself defaults to true, may be disabled with 'punycode_links' remove wbrequest and urlrewriter from get_url path, simply call wb_url.get_url() to get properly formatted url Ilya Kreymer 2015-02-14 20:55:36 -08:00
f9452bf48e rewrite: refactor IDN support: instead of returning IRI, return utf-8 %-encoded url remove support for returning IRI, as that requires detecting charset, instead just use %-encoded form and let browser decode. Should address #66 Ilya Kreymer 2015-02-14 17:26:39 -08:00
79cfdd6a08 framework/urlrewriter: allow overriding UrlRewriter with optional urlrewriter_class param, easier to override create_rebased_rewriter() with custom rewriter as well Ilya Kreymer 2015-02-12 10:34:04 -08:00
dcf3688dc3 wombat: also override frameElement when changing window.parent for top-level replay frame Ilya Kreymer 2015-02-11 19:26:45 -08:00
0b72bfe911 add 'none' js regex rewriter, which does not rewrite urls or location regexs add test for none rewriter in test rule Ilya Kreymer 2015-02-11 15:01:29 -08:00
f068186e37 wombat: replace window.self -> window for clarity Ilya Kreymer 2015-02-11 15:01:04 -08:00
78bd89b4cb rewrite: simplify deprefix, url already unquoted now so remove extra unquote Ilya Kreymer 2015-02-11 14:28:45 -08:00
4e7f95081f url_rewriter: catch exception when encoding to utf-8, may not be properly encoded, in which case treat as bytes Ilya Kreymer 2015-02-10 15:05:15 -08:00
90aba00ca0 not_found: catch NotFoundException from any part of handle_request, not just indexing.. allows for more flexible usage with cdx iterators that are lazily evaluated on replay Ilya Kreymer 2015-02-10 15:03:21 -08:00
148651680a wombat fix: use __orig_parent when referencing top-frame, since window.parent is being overriden Ilya Kreymer 2015-02-10 15:02:08 -08:00
78ae86b6b6 Merge branch 'master' for 0.7.8 into develop Ilya Kreymer 2015-02-05 08:45:55 -08:00
384e68c84b bump version to 0.7.8 for latest fix Ilya Kreymer 2015-02-04 21:46:57 -08:00
cc144fdead rewrite: add basic test for X-Forwarded-Proto #57 Ilya Kreymer 2015-02-04 21:44:18 -08:00
78812c8085 rewrite: more conservative change, only rewrite the X-Forwarded-Proto header for now, #57 Ilya Kreymer 2015-02-04 15:17:23 -08:00
cdb3dcc3d2 rewrite_live: don't forward via or https_x headers, only standard (for now) possible fix for #57 Ilya Kreymer 2015-02-04 14:19:37 -08:00
40fba3c27b cdx-indexer: minor cleanup, add custom writer override to write_multi_cdx_index 0.7.7 Ilya Kreymer 2015-02-04 11:17:26 -08:00
ef98716bd8 bump version to 0.7.7 in prep for release Ilya Kreymer 2015-02-03 11:23:12 -08:00
c47d3ca925 wombat: add mutation observers, addressing #71 and maybe #67 rules: fix regex for yt, add rx for wikimedia Ilya Kreymer 2015-02-03 11:14:06 -08:00
734ee4471b frame ui: pass timestamp to frame banner, fix typo in html banner: allow overriding of banner id by returning custom id Ilya Kreymer 2015-02-02 09:41:49 -08:00
29c6a36dac cdx api query: pass query timestamp mod to index query via 'query_closest' field, to avoid confusion with 'closest' Ilya Kreymer 2015-01-31 17:45:46 -08:00
55426e7619 memento: fix headers to be more consistent for framed replay. when using frames, outer frames 'mirrors' mementos of the inner frame to be discoverable by client side memento tools, tracked via #70 Ilya Kreymer 2015-01-29 22:27:15 -08:00
757345d317 replay api: make ReplayView overridable in WBHandler subclass, allow custom content loader callable Ilya Kreymer 2015-01-29 20:10:41 -08:00
7e017fd85e rewrite fixes: don't rewrite window.parent as it is overridable directly html rewriter: ensure style is rewritten for all elements, add test! wombat: cleanup and additional checks for assign(), setAttribute() Ilya Kreymer 2015-01-29 20:08:00 -08:00
043ad5c860 wombat: improve createElementNS override to set prototype, just assign window.parent directly Ilya Kreymer 2015-01-29 10:13:32 -08:00
bf3d256a51 rewrite: add css-in-js rewrite rule for wikimedia, tracking via #67 for perhaps a more general solution Ilya Kreymer 2015-01-28 09:20:42 -08:00
ccedb2d60e regex_rewrite: add 'parent' rewrite in addition to 'top' for frames, add WB_wombat_parent to wombat, add test for WB_wombat_parent Ilya Kreymer 2015-01-27 19:57:56 -08:00
976decb3f1 wombat: ensure document.write override handles elements that go into head as well as body Ilya Kreymer 2015-01-27 18:02:14 -08:00
59630c08f6 bump version to 0.8.0! Ilya Kreymer 2015-01-26 11:07:30 -08:00
695245d9e8 wburl idn: more complete support for idn urls (#66) add distinct to_iri() and to_uri() functions in WbUrl internal representation is always as ascii uri for rewriting, defaults to iri representation unless 'rewrite_ascii_only_urls' is set to true per collection add wbrequest.get_url() to get url as either iri or uri to be passed to templates Ilya Kreymer 2015-01-26 09:52:04 -08:00
edff3f17fb wburl: convert %-encoded hostnames or unicode urls to punycode for better IDN support (#66) Ilya Kreymer 2015-01-25 12:21:50 -08:00
933343fa01 update README for 0.7.2 master Ilya Kreymer 2015-01-03 13:51:47 -08:00
8b5a6be956 Merge branch 'develop' for 0.7.6 Ilya Kreymer 2015-01-26 10:38:35 -08:00
8567b3fa76 CHANGELIST tweaks Ilya Kreymer 2015-01-26 10:37:51 -08:00
5acd1164ab update CHANGELIST for 0.7.6 Ilya Kreymer 2015-01-26 10:31:24 -08:00
38e3bbbaef templates: add new 'not_found.html' template, which will be called for any missing replay request instead of default error.html 'not_found_html' settable in the config per collection, as per #65 for not found index query, still use query.html but add condition to check for 0 results add more query and replay not found remove unused conditional (for search_view -- always exists) Ilya Kreymer 2015-01-24 12:32:50 -08:00
80fd47ba3e add rules for vine (#62) Ilya Kreymer 2015-01-22 16:45:09 -05:00
c9b2e3e69e wombat 2.2 improvements: * for postMessage, add receive message overrides which uses original origin to fix message passing tests that check for origin Ilya Kreymer 2015-01-22 16:42:52 -05:00
48b7751f80 bump version to 0.7.6 jinja2: allow adding multiple packages to search path Ilya Kreymer 2015-01-19 21:54:11 -05:00
c935aa5ec9 Merge branch 'develop' for 0.7.5 0.7.5 Ilya Kreymer 2015-01-12 00:50:16 -08:00
71d9e58d7c fixup changes for 0.7.5 Ilya Kreymer 2015-01-12 00:38:51 -08:00
43805c67ef view: fix format_ts, use existing utc timestamp_to_sec conversion for %s Ilya Kreymer 2015-01-12 00:28:06 -08:00
7ece05d022 bump version to 0.7.5 update CHANGES fix .gitattributes to use standard flags Ilya Kreymer 2015-01-12 00:09:02 -08:00
ac525b0937 tests: add tests for extract_post_query() add test for HttpsUrlRewriter, remove unnecessary check in bufferedreader Ilya Kreymer 2015-01-11 23:54:29 -08:00
8449647c5f wbexception: remove unused status in WbException, set default error for any uncaught exception to 500, instead of 400 Ilya Kreymer 2015-01-11 23:53:34 -08:00
7610d9deb7 views: cleanup view filters, remove obsolete, add tests for format_ts and is_wb_handler Ilya Kreymer 2015-01-11 23:02:48 -08:00
438f9c3e5c git: add gitattributes to ensure consistent line endings for warc, arc and cdx Ilya Kreymer 2015-01-11 19:09:01 -08:00
db75bda736 file open() pass: convert all read and write to ensure binary 'b' flag is set (#56) Ilya Kreymer 2015-01-11 18:53:47 -08:00