9 Commits

Author	SHA1	Message	Date
Vangelis Banos	66b4c35322	Remove unused imports	2017-09-24 11:15:30 +00:00
Noah Levitt	2c65ff89fa	add license headers	2016-04-06 19:37:55 -07:00
Noah Levitt	42a81d8f8f	fix bug where two warc-payload-digest headers were written to revisit records	2016-03-15 06:27:21 +00:00
Noah Levitt	a41c426b0a	giving up on using git revision in version number :( latest issue is when installing a package that calls git to compute a version number, but cwd is some other git project, you get the wrong thing	2016-01-26 18:47:08 -08:00
Noah Levitt	686a297f98	fixes to let screenshot recordss be saved in big capture tables for wayback playback	2016-01-26 18:47:08 -08:00
Noah Levitt	b30218027e	get "mimetype" (without ;params) from content-type in one place in RecordedUrl, and also note host and duration (time spent serving request)	2016-01-26 18:47:08 -08:00
Noah Levitt	ab4e90c4b8	make warc-date follow warc spec "timestamp shall represent the instant that data capture for record creation began"	2016-01-26 18:47:08 -08:00
Noah Levitt	e66dc3a9fb	rethinkdb dedup	2016-01-26 18:46:13 -08:00
Noah Levitt	274a2f6b1d	refactor warc writing, deduplication for somewhat cleaner separation of concerns	2016-01-26 18:45:36 -08:00