barely starting to flesh out warcprox-meta section

This commit is contained in:
Noah Levitt 2018-05-25 10:33:45 -07:00
parent b562170403
commit 02e96188c3

57
api.rst
View File

@ -1,7 +1,9 @@
warcprox API warcprox API
************ ************
Means of Interacting with warcprox over http, aside from simply proxying urls. Means of interacting with warcprox over http, aside from simply proxying urls.
.. contents::
``/status`` url ``/status`` url
=============== ===============
@ -132,16 +134,49 @@ remote server, and also does not write it in the warc request record.
Warcprox-Meta: {} Warcprox-Meta: {}
- warc-prefix Warcprox-Meta fields
- stats -------------------
- buckets
- dedup-bucket ``warc-prefix`` (string)
- blocks ~~~~~~~~~~~~~~~~~~~~~~~~
- limits Specifies a warc filename prefix. Warcprox will write the warc record for this
- soft-limits capture, if any, to a warc named accordingly.
- metadata
- accept Example::
- dedup-ok # deprecate?
Warcprox-Meta: {"warc-prefix": "special-warc"}
``stats`` (dictionary)
~~~~~~~~~~~~~~~~~~~~~~
* buckets
Example::
Warcprox-Meta: {"stats":{"buckets":["my-stats-bucket","all-the-stats"]}}
``dedup-bucket`` (string)
~~~~~~~~~~~~~~~~~~~~~~~~~
Specifies the deduplication bucket. For more information about deduplication
see `<readme.rst#deduplication>`_.
Example::
Warcprox-Meta: {"dedup-bucket":"my-dedup-bucket"}
``blocks``
~~~~~~~~~~
``limits``
~~~~~~~~~~
``soft-limits``
~~~~~~~~~~~~~~~
``metadata`` (dictionary)
~~~~~~~~~~~~~~~~~~~~~~~~~
``accept``
~~~~~~~~~~
Brozzler knows about ``warcprox-meta``. For information on configuring Brozzler knows about ``warcprox-meta``. For information on configuring
``warcprox-meta`` in brozzler, see https://github.com/internetarchive/brozzler/blob/master/job-conf.rst#warcprox-meta ``warcprox-meta`` in brozzler, see https://github.com/internetarchive/brozzler/blob/master/job-conf.rst#warcprox-meta