Jack Cushman
4622a6ca52
Return recorded_url from _proxy_request.
2015-10-23 15:15:45 -04:00
Noah Levitt
67f2ceb717
make sure timestamp17(), which is part of warc name, always returns a 17 digit timestamp (even if millisecond part is <100)
2015-07-17 13:31:04 -07:00
Noah Levitt
8dfcf0401c
bump up socket timeout setting on connection to remote server, and send appropriate error 504 on timeout
2015-06-30 17:45:19 -07:00
Noah Levitt
b07f194c63
send requested hostname to remote server if python ssl version supports SNI, fixes ssl handshake error for some servers
2015-06-30 17:38:45 -07:00
Noah Levitt
1abe98c99b
Merge pull request #12 from ikreymer/dev.use-certauth-pkg
...
remove certauth.py and use the seperate certauth package release
2015-03-30 17:48:54 -07:00
Ilya Kreymer
c045369dcd
change 'get_cert_for_host' -> 'cert_for_host'
2015-03-30 15:46:31 -07:00
Ilya Kreymer
574f1f3f52
remove certauth.py and use the seperate certauth package release
2015-03-30 09:32:10 -07:00
Noah Levitt
965853f4ab
add payload digest header to revisit records
2015-03-26 15:17:46 -07:00
Noah Levitt
0eb2917e50
update tox and travis config for supported python versions 2.7 and 3.4
2015-03-18 16:36:24 -07:00
Noah Levitt
016749a822
bump version since api has changed as a result of reorganization
2015-03-18 16:33:07 -07:00
Noah Levitt
5f84b061f3
make it work with python 2.7 again
2015-03-18 16:29:44 -07:00
Noah Levitt
1e3dd0b910
swallow request headers that don't make sense to send on to the destination, i.e. most hop-by-hop headers; parse and save Warcprox-Meta header (nothing is done with it yet)
2014-11-20 03:26:42 -08:00
Noah Levitt
a2c25d4242
split into even more source files
2014-11-20 00:04:43 -08:00
Noah Levitt
9b8ffbbb51
separate WarcWriter and WarcWriterThread
2014-11-15 04:47:26 -08:00
Noah Levitt
b34edf8fb1
split into multiple files
2014-11-15 03:20:05 -08:00
Noah Levitt
e8438dc8ad
Merge pull request #10 from ikreymer/dev.fix-sync
...
check if 'sync' method exists before calling it
2014-10-28 13:48:53 -07:00
Ilya Kreymer
a139465512
check if 'sync' method actually exists before calling -- anydbm does not have sync()
...
method
2014-10-28 12:49:02 -07:00
Noah Levitt
a0a5ef2355
fix formatting of Install section again so it looks right on pypi! (test with rst2html)
2014-08-08 12:53:16 -07:00
Noah Levitt
b2479d39f6
fix formatting of Install section so it looks right on github
2014-08-08 12:29:12 -07:00
Noah Levitt
9562338d01
add Install section to readme, update the --help dump
2014-08-08 12:22:33 -07:00
Noah Levitt
16f21b2e76
https://github.com/internetarchive/warcprox/issues/9 record warcprox version in warcinfo metadata, and add --version command line option
2014-08-08 12:10:45 -07:00
Noah Levitt
b434e33fdd
bump version number for updated submission to pypi
2014-08-05 19:04:07 -07:00
Noah Levitt
7b66f27758
since ndbm creates different files on different platforms, glob them all and delete them
2014-08-01 17:40:34 -07:00
Noah Levitt
1cdc013c75
some debugging to try to figure out what the hell is up with tox saying OSError: [Errno 2] No such file or directory: /tmp/tmpnz51j6.db
2014-08-01 17:32:16 -07:00
Noah Levitt
111c678cee
add python3.4 to travis, tox test list; remove apt-get install python3.3-gdbm from travis configuration to fix travis error "Unable to locate package python3.3-gdbm"
2014-08-01 16:43:00 -07:00
Noah Levitt
ccbe3522c5
timestamps in utc!
2014-08-01 16:00:53 -07:00
Noah Levitt
e79cdb84cb
set x509 cert version correctly fixes problem with firefox 31; set_version(2) really means version 3, because 0 is understood to mean version 1 (wtf)
2014-08-01 12:35:34 -07:00
Eldon
5d09aea67d
Merge pull request #6 from jcushman/master
...
If gdbm is not available, fall back to anydbm.
2014-01-31 08:08:01 -08:00
Jack Cushman
4488c04e5e
If gdbm is not available, fall back to anydbm.
2014-01-30 19:07:05 -05:00
Noah Levitt
37bd3b75a6
Merge pull request #5 from khawley/master
...
updated setup.py to use pytest (for compatilibity with dump-anydbm tests)
2014-01-20 14:25:42 -08:00
Kelsey Hawley
c0fbd61507
changed the way I was retrieving the python version
2014-01-17 16:20:16 -08:00
Kelsey Hawley
a87a5dd972
updated test to directly use the specified py version & access the file path to dump-anydbm directly. Also added some more helpful print error statements
2014-01-17 15:35:25 -08:00
Kelsey Hawley
ae3a039d95
updated setup.py to use pytest (for compatilibity with dump-anydbm tests)
2014-01-17 12:13:39 -08:00
Noah Levitt
2ab44d4adf
maybe travis build can use tox
2014-01-07 14:10:39 -08:00
Noah Levitt
62e58caea5
.travis.yml - use pip to install test dependencies
2014-01-07 14:00:46 -08:00
Noah Levitt
ad30585cbf
fix travis build?
2014-01-07 12:04:52 -08:00
Noah Levitt
8213c47969
py.test
2014-01-07 11:54:20 -08:00
Kelsey Karin Hawley
e3116482d4
Merge pull request #4 from nlevitt/test_dump-anydbm
...
small fixes to kelsey's tests
2014-01-07 11:14:08 -08:00
Noah Levitt
f69ec424fb
minor cleanup
2014-01-06 17:22:49 -08:00
Noah Levitt
b4a7a7fa5f
remove warcprox from deps, because it downloads and installs from pypi instead of using local checkout
2014-01-06 17:22:36 -08:00
Noah Levitt
e89fba4332
logic fix
2014-01-06 17:22:00 -08:00
Kelsey Hawley
b6ea681c2b
changed file creation and deletion to use temporaryfile. Still needed to use os to delete the 'extra' files that ndbm & dumbdbm created. Also did not explicitly state the file name in checking the output statements, as now they are random everytime.
2014-01-02 18:18:46 -08:00
Kelsey Hawley
1b69aea7ed
removed the string splicing and replaced with one clear assert statement based on the script output for each test. simplifies and clarifies the test
2014-01-02 17:05:45 -08:00
Kelsey Hawley
4b0ab0ff72
updated file to PEP 8, as editor was complaining, and tabs are generally bad
2014-01-02 16:29:15 -08:00
Kelsey Hawley
39fb1c75ba
added py27 & py32 back into envlist for tox to run against
2013-12-31 13:47:50 -08:00
Kelsey Hawley
e6606c67fa
committing updates to tox.ini file for use with tox
2013-12-31 12:34:09 -08:00
Kelsey Hawley
d643be1c8c
moved dump-anydbm test file to be in the existing test folder, as proximity to dump-anydbm script is not necessary
2013-12-20 14:01:42 -08:00
Kelsey Hawley
12e00e3060
Merge branch 'master' of https://github.com/internetarchive/warcprox
...
* 'master' of https://github.com/internetarchive/warcprox :
oops, adding missing comma
add some classifiers
since we depend on warctools trunk now, update the readme, and update the version number, so we can push latest to pypi
2013-12-20 13:54:15 -08:00
Kelsey Hawley
6a129fed34
corrected the import of python 3 modules. Updated print statements to function form to work correctly with python 2 & 3. Also decoded the subprocess output into unicode string for splicing and processing of keys
2013-12-20 13:51:34 -08:00
Kelsey Hawley
48a2db509f
updated string printing to be consistent between python 2 & 3. Previously, python 3 would print the db key & values as byte code, while python 2 inherently ignores this when printing. Now the stdout is the same in both versions.
2013-12-20 13:47:41 -08:00
