11 Commits

Author SHA1 Message Date
Vangelis Banos
47ea3110be Yet another unit test fix 2018-01-10 20:55:31 +00:00
Vangelis Banos
b2c47142de Change the writer unit test
To be able to run in github.
2018-01-10 20:38:06 +00:00
Vangelis Banos
e737a30ec1 fix github problem with unit test 2018-01-10 19:29:22 +00:00
Vangelis Banos
deddd4f850 Another fix for the unit test 2018-01-10 18:52:59 +00:00
Vangelis Banos
9d789cdae8 Fix writer unit test 2018-01-10 18:41:56 +00:00
Vangelis Banos
d2ce61aec9 Add WarcWriter warc_filename unit test
Use custom ``warc_filename`` option and check that the created WARC
filename follows the defined pattern.
2018-01-09 12:54:42 +00:00
Noah Levitt
9784c91459 test for special warc prefix "-" which means "do not archive" 2017-12-21 14:31:54 -08:00
Vangelis Banos
c087cc7a2e Improve test_writer tests
Check also that locking succeeds after the writer closes the WARC file.

Remove parametrize from ``test_warc_writer_locking``, test only for the
``no_warc_open_suffix=True`` option.

Change `1` to `OBTAINED LOCK` and `0` to `FAILED TO OBTAIN LOCK` in
``lock_file`` method.
2017-11-01 17:50:46 +00:00
vbanos
25c0accc3c Swap fcntl.flock with fcntl.lockf
On Linux, `fcntl.flock` is implemented with `flock(2)`, and
`fcntl.lockf` is implemented with `fcntl(2)` — they are not compatible.
Java `lock()` appears to be `fcntl(2)`. So, other Java programs working
with these files work correctly only with `fcntl.lockf`.
`warcprox` MUST use `fcntl.lockf`
2017-10-28 21:13:23 +03:00
vbanos
eda3da1db7 Unit test fix for Python2 compatibility 2017-10-28 15:32:04 +03:00
vbanos
3132856912 Test WarcWriter file locking when no_warc_open_suffix=True
Add unit test for ``WarcWriter`` which open a different process and
tries to lock the WARC file created by ``WarcWriter`` to check that
locking works.
2017-10-28 14:36:16 +03:00