Ilya Kreymer
fc9d659b5d
loaders: switch BlockLoader to use requests instead of urliib2
2015-03-28 16:41:52 -07:00
Ilya Kreymer
f3a066f58b
cdx-server query & zipnum: fixes for showNumPages query:
...
- if query contained in <1 secondary index block, must read first line of cdx to determine if any matches
- if no matches, don't throw 404 exception but always return json info with 0 pages
2015-03-28 16:15:24 -07:00
Ilya Kreymer
313a2efeac
bump version to 0.9.3-dev
2015-03-28 16:12:28 -07:00
Ilya Kreymer
c3a108b169
minor readme tweaks
2015-03-27 09:31:17 -07:00
Ilya Kreymer
d2be90d4a1
test case tweak
2015-03-27 08:56:43 -07:00
Ilya Kreymer
41487dd9d4
update changelist for 0.9.2
...
cdx: include match type in cdx query error
2015-03-27 07:58:51 -07:00
Ilya Kreymer
8d686a4a98
README typos fix
2015-03-26 19:56:09 -07:00
Ilya Kreymer
6bbbb51f6e
manager: relax template requirements, allow any collection template to also be added to shared dir
2015-03-26 19:40:43 -07:00
Ilya Kreymer
753300d5ed
manager: use absolute path when adding warcs, ( #84 )
2015-03-26 19:18:55 -07:00
Ilya Kreymer
6ce75f80f5
replay: remove restricting to provided http Content-Length (in addition to record content-length) as it may be incorrect for variety of reasons
2015-03-26 17:12:38 -07:00
Ilya Kreymer
0a4e97baa1
revisit resolving: if cdx digest is missing, attempt to resolve revisits based on url + timestamp only, if warc-refers-to-target-uri and warc-refers-to-date are available, even if warc-refers-to-target-uri == target-uri (see #88 for more info)
2015-03-26 14:20:08 -07:00
Ilya Kreymer
85082e46bf
cdxj: ensure revisit resolve is skipped if the digest is missing, as may be case in cdxj ( #85 )
2015-03-26 11:11:10 -07:00
Ilya Kreymer
2dbde35d74
bump to version to 0.9.2
2015-03-26 09:14:27 -07:00
Ilya Kreymer
cf4b5c50dd
more README.rst fixes
2015-03-25 22:08:53 -07:00
Ilya Kreymer
e8b6a1af88
README typo fixes
2015-03-25 21:52:38 -07:00
Ilya Kreymer
1cfe73c9db
zipnum: fix block count off-by-1 error in showNumPages query
2015-03-25 20:43:59 -07:00
Ilya Kreymer
72ddb54f82
Minor README tweaks
2015-03-25 15:01:12 -07:00
Ilya Kreymer
3efbfaa8c8
pywb_init: simplify DictChain usage, remove unused methods
2015-03-25 13:30:16 -07:00
Ilya Kreymer
f808f34ba7
Update CHANGES for 0.9.1
2015-03-25 12:16:26 -07:00
Ilya Kreymer
0e8b305adc
Update README to 0.9.1, add cdx api link, fix typo
2015-03-25 12:06:05 -07:00
Ilya Kreymer
15d1aea5ec
Update README, improve existing collection instructions.
2015-03-25 12:02:57 -07:00
Ilya Kreymer
a6c24c2882
autoindex: undo stop/join call for indexing, breaks os x unit test.. (autoindex test may need more improvements on windows)
2015-03-25 11:09:17 -07:00
Ilya Kreymer
90eee03cdb
fixes for windows:
...
indexing: ensure '/' always written to cdx
autoindex: improved test case, ensure threads exit with join
style: fix long lines
2015-03-25 10:56:53 -07:00
Ilya Kreymer
a7307a6d98
pywb_init: auto-collections init: inherit shared archive_paths, if any are set in main config.yaml
2015-03-25 09:36:00 -07:00
Ilya Kreymer
6a3ca566db
zipnum: cleanup shared location resolution, in addition .loc file,
...
support a prefix resolver, where can be a regex replacement on the index path
(default is unchanged index path) (#83 )
2015-03-25 09:07:54 -07:00
Ilya Kreymer
1a8211d752
cdx server: add simplified matchType notation, using host* for prefix and *.host for domain matchType
...
(#34 )
2015-03-24 19:49:54 -07:00
Ilya Kreymer
2af5a25009
zipnum: support for pagination api! #34 and #83 . cdx server now bounded by pageSize (default 10 blocks),
...
showNumPages=true returns json indicating num pages, page=N can be set to page number 0-numPages - 1
loaders: add read_last_line() to read last line of a seekable file, used to read last line of index file when
at end
tests: additional test for binsearch boundary conditions
zipnum: secondary index output supports json also
2015-03-24 18:56:13 -07:00
Ilya Kreymer
872607c07d
README: move new features towards the top
2015-03-24 10:56:56 -07:00
Ilya Kreymer
3dd600c530
wombat: improve document.write override to write each elem at a time for body as well as head, #82
2015-03-24 10:46:10 -07:00
Ilya Kreymer
e5f321e32f
bump version to 0.9.1 for further dev
2015-03-23 20:21:09 -07:00
Ilya Kreymer
57be9ca7bc
tweak CHANGES.rst and INSTALL.rst for release
0.9.0
2015-03-23 17:38:22 -07:00
Ilya Kreymer
cda9f435a3
update README for final 0.9.0 release
2015-03-23 17:34:16 -07:00
Ilya Kreymer
c93501e16d
more changes.rst updates
2015-03-23 16:29:18 -07:00
Ilya Kreymer
500a441ea9
README tweaks and edits from Dragan (@despens)
2015-03-23 16:16:16 -07:00
Ilya Kreymer
ec7a29a3ba
static paths: ensure consistent renaming of static/default -> static/__pywb for bundled static path
2015-03-23 16:15:37 -07:00
Ilya Kreymer
5b4d12eb05
wombat: fix wombat_location.href assign when url is already rewritten, compare against current url not passed in url
...
fixes ikreymer/pywb-webrecorder#9
2015-03-23 16:12:58 -07:00
Ilya Kreymer
5020a09004
more CHANGES.rst updates
2015-03-23 15:43:05 -07:00
Ilya Kreymer
4aa6512b05
rewrite: fix WbUrl parsing for urls that start with a digit, eg. 1234.example.com
...
split latest replay url from timestamped replay regex
add additional rewrite tests
2015-03-23 15:38:10 -07:00
Ilya Kreymer
6acac67d3c
rewrite: fix js rewrite again to ensure '// comments' are not rewritten as scheme-rel urls
...
add tests
2015-03-23 11:49:24 -07:00
Ilya Kreymer
bf0996c27a
uwsgi: run with gevent loop by default, install gevent in run script
2015-03-23 11:05:17 -07:00
Ilya Kreymer
da7532a1f8
wb-manager: rename 'migrate' to 'cdx-convert' for clarity
2015-03-23 11:05:02 -07:00
Ilya Kreymer
0faa6aac3e
setup: set version in pywb __init__.py
2015-03-23 11:04:41 -07:00
Ilya Kreymer
ced0ed208e
Update CHANGELIST for 0.9.0
2015-03-23 10:48:58 -07:00
Ilya Kreymer
7681b4a634
Update INSTALL.rst
2015-03-23 10:36:37 -07:00
Ilya Kreymer
317a6c6e8e
Update INSTALL.rst
2015-03-23 10:31:59 -07:00
Ilya Kreymer
6d879c10bb
README work
2015-03-23 10:18:46 -07:00
Ilya Kreymer
4cfeb6d958
More README tweaks
2015-03-23 10:15:33 -07:00
Ilya Kreymer
e2623ed149
Update README.rst for latest update
2015-03-23 09:52:07 -07:00
Ilya Kreymer
df76bc3500
cli: change cdx-server and live-rewrite-server to go through shared cli
...
entry point
2015-03-23 09:08:09 -07:00
Ilya Kreymer
ae363ad368
autoindex and cli: add autoindex to cli with 'wayback -a' option, #81
2015-03-22 23:03:39 -07:00