backup/pywb
1
0
Fork 0
mirror of https://github.com/webrecorder/pywb.git synced 2025-03-15 16:14:48 +01:00
Code Issues Projects Releases Activity
1,131 Commits 56 Branches 63 Tags
Commit Graph

1131 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Ilya Kreymer
be5139b635 fix tests for coll listing, #78 
config override: when loading from coll-specific config.yaml, resolve
relative paths to that collection, not to root #55
 2015-03-15 22:23:08 -07:00
Ilya Kreymer
30454abb6b metadata: add support for user-defined per-collection metadata! #78 
metadata stored in wbrequest.user_metadata and available to all templates

collections manager: refactor to use subparsers, add list collections and set metadata commands
update tests for new commands
index template: use user metadata title for collections listing
search template: display all metadata and title, if available
 2015-03-15 21:24:15 -07:00
Ilya Kreymer
b417b47835 collections manager: support for merge when adding warc, explicit --index-warcs 
option to index and merge instead of reindexing whole dir, #74
additional testing for recursive indexing, index merge
timeutils: add timestamp20_now() function
 2015-03-14 14:56:15 -07:00
Ilya Kreymer
759d151551 tests: add test for directory auto collection loader, 
collection manager and new 6-field minimal cdx format
 2015-03-13 19:53:50 -07:00
Ilya Kreymer
1ba24de357 Merge branch 'develop' into config-work 2015-03-13 11:53:27 -07:00
Ilya Kreymer
b4b92482ad Merge branch 'develop' for 0.8.3 2015-03-13 11:06:52 -07:00
Ilya Kreymer
b2ce3feb80 readme fix 2015-03-13 11:05:32 -07:00
Ilya Kreymer
3e3794d4dc Update CHANGES for 0.8.3 2015-03-13 11:04:37 -07:00
Ilya Kreymer
24021fcd57 html rewrite: add trailing slash for <base> tag rewrite if url is a scheme://host 
with no path component #77
cleanup: remove unused code path for tags with no rewriting -- all tags
now checked for dynamic attrs which may need rewriting
update tests, including live rewrite test dependent on live site (FB)
 2015-03-13 10:53:57 -07:00
Ilya Kreymer
fe1683da56 indexing: for minimal index, use a single -m flag to create a 6 field index. 
minimal index also skips parsing contents of warc/arc records altogether
add cli docs for minimal index, tracked via #75
 2015-03-07 11:56:17 -08:00
Ilya Kreymer
499e21233e statusandheaders: make protocol check case-insensitive, eg. accept HTTP/1.0 and http/1.0 for better compatibility 2015-03-07 11:37:06 -08:00
Ilya Kreymer
5aa497dc68 Merge branch 'develop' into config-work 2015-03-06 21:09:21 -08:00
Ilya Kreymer
1fb631870b wb_frame: fix extra slash typo in replaced frame url 2015-03-05 17:04:44 -08:00
Ilya Kreymer
f2d7bd074a bump version to 0.8.3 
cookie rewrite: remove 'secure' flag if present
 2015-03-05 16:18:56 -08:00
Ilya Kreymer
1eadd35598 Merge branch 'develop' for 0.8.2 2015-02-28 09:05:09 -08:00
Ilya Kreymer
6c8cb806d9 update 0.8.2 changelist, minor fixes 2015-02-28 09:04:15 -08:00
Ilya Kreymer
48eab2662d cdx indexer: refactor indexer into mixins for differnt formats for easier customization 2015-02-25 16:45:47 -08:00
Ilya Kreymer
ee1fabf600 config fix: check for existance of root 'collections dir', #55 2015-02-25 13:51:12 -08:00
Ilya Kreymer
11c8cc92f3 add beta to README 2015-02-25 13:33:42 -08:00
Ilya Kreymer
671f45f69f cdx indexing: wrap record iterator global functions in class DefaultRecordIter to allow for better extensibility 
add 'minimal' option to skip digest/mime/status extraction only include minimal data (url+timestamp)
cdx-indexer: add -6 option to create 6-field index
 2015-02-25 13:31:37 -08:00
Ilya Kreymer
1d4c54deaa frames ui: update frames to use <!DOCTYPE html>, improved css and html5 compatibility 2015-02-25 13:25:05 -08:00
Ilya Kreymer
60f33412ff collections manager: add new collections manager, first pass #74 
add cli 'wb-manager' tool
very preliminary, needs testing still
 2015-02-25 13:19:20 -08:00
Ilya Kreymer
69613a0e25 tests: disable 'invalid config' test as its no longer applicable, fix default banner to just 'banner.html' 2015-02-25 13:18:32 -08:00
Ilya Kreymer
5c67782a2c config system: some fixes for auto-init, add trailing '/' for dir paths, #55 2015-02-25 13:15:48 -08:00
Ilya Kreymer
7c60bf17f7 bump version to 0.9.0-beta! 2015-02-24 16:54:49 -08:00
Ilya Kreymer
e39d6e207c config & collections: auto static path and templates working! #55 2015-02-24 14:32:51 -08:00
Ilya Kreymer
a932235f85 Merge branch 'develop' into config-work 2015-02-24 10:40:58 -08:00
Ilya Kreymer
cb857df125 memento: fix MementoTimemapView to have consistent signature with other query views 2015-02-24 10:35:49 -08:00
Ilya Kreymer
39824711f0 memento tweak: ensure rel=memento link for timegate uses exact in Location (cdx original) as opposed to url from request 2015-02-23 23:21:39 -08:00
Ilya Kreymer
435fa390ed config system: initial work on automated directory-convention based config! 
config.yaml file now optional, add default_config.yaml which for default settings #55
 2015-02-23 21:59:41 -08:00
Ilya Kreymer
5d80d2d891 replay: change strip_scheme() to strip_scheme_www() to also strip away www. prefix for self-redirect checking, #73 2015-02-22 22:51:35 -08:00
Ilya Kreymer
83f8d7d29b bump version to 0.8.2 2015-02-22 22:51:23 -08:00
Ilya Kreymer
de40e2920a update README for 0.8.1 2015-02-21 14:29:40 -08:00
Ilya Kreymer
7989c06ea4 Add webarchiveplayer link to README 2015-02-21 14:28:04 -08:00
Ilya Kreymer
80da0e91da update CHANGELIST for 0.8.1 2015-02-21 14:13:35 -08:00
Ilya Kreymer
9f838241c7 wb.js bug fix: use only window.__orig_parent and not window.parent, as window.parent overriden. window instead of window.self 2015-02-21 12:34:35 -08:00
Ilya Kreymer
c0ff596c68 tests: add tests for recursive cdx indexing, #64 
cross-platform: store rel filename path as '/', but convert to os.path.sep
when resolving to full path as prefix
 2015-02-20 13:56:35 -08:00
Ilya Kreymer
8d52be4c44 live proxy: enable ssl validation for live proxy, was initially disabled for testing, should be on by default! 2015-02-20 13:22:21 -08:00
Ilya Kreymer
1646c90cd0 cdxindexer: add -r option to support recursive indexing when input is a directory. 
filename field in cdx contains relative path including subdir, eg. subdir/file.warc.gz
related to #64
 2015-02-20 02:40:32 -08:00
Ilya Kreymer
adeb8bfb27 bump version to 0.8.1, (fix blank spacing in changelist) 2015-02-20 02:02:34 -08:00
Ilya Kreymer
cb6aebf06d Merge CHANGES.rst from 'develop' 0.8.0 2015-02-19 01:29:22 -08:00
Ilya Kreymer
bf203a2dc6 Merge branch 'develop' of https://github.com/ikreymer/pywb into develop 2015-02-19 01:29:03 -08:00
Ilya Kreymer
121e1df3c9 README: update branch config to master 2015-02-19 01:26:55 -08:00
Ilya Kreymer
824587bd90 A few more CHANGES.rst tweaks 2015-02-19 01:24:52 -08:00
Ilya Kreymer
26df8d7784 remove debug logging and spaces 2015-02-19 01:17:31 -08:00
Ilya Kreymer
0ddc490b8d Update CHANGELIST for 0.8.0! 2015-02-19 01:16:25 -08:00
Ilya Kreymer
80dcb6ff27 rewrite: improvements to non-exact replay mode, redir_to_exact option set to false 
frames: add request_ts to wbinfo and use that as the timestamp in the top-frame. for exact replay, request_ts == timestamp
for latest replay / no timestamp / memento timegate, redirect to current time instead of time of last capture, while serving
last capture.
timeutils: add timestamp_now() function to return timestamp of current datetime
Add extra tests for this mode
Tracked via #72
 2015-02-17 17:51:45 -08:00
Ilya Kreymer
9623f95439 memento: add rel="memento" header to timegate as well, improve memento test, clearly differntiate between 
timegate redirect and intermediate resource redirect, related to #70
 2015-02-16 09:59:03 -08:00
Ilya Kreymer
c4d5dd4690 rewrite: optimize / sanity, only %-encode urls that are actually idna-encoded, 
otherwise return as is, #66
 2015-02-15 10:34:56 -08:00
Ilya Kreymer
afe49a91f4 rewrite: more fixes for IDN #66 - add _do_percent_encode field to wburl itself 
defaults to true, may be disabled with 'punycode_links'
remove wbrequest and urlrewriter from get_url path, simply call wb_url.get_url() to get properly formatted url
 2015-02-14 20:55:36 -08:00