mirror of
https://github.com/internetarchive/warcprox.git
synced 2025-01-18 13:22:09 +01:00
barely starting to flesh out warcprox-meta section
This commit is contained in:
parent
b562170403
commit
02e96188c3
57
api.rst
57
api.rst
@ -1,7 +1,9 @@
|
|||||||
warcprox API
|
warcprox API
|
||||||
************
|
************
|
||||||
|
|
||||||
Means of Interacting with warcprox over http, aside from simply proxying urls.
|
Means of interacting with warcprox over http, aside from simply proxying urls.
|
||||||
|
|
||||||
|
.. contents::
|
||||||
|
|
||||||
``/status`` url
|
``/status`` url
|
||||||
===============
|
===============
|
||||||
@ -132,16 +134,49 @@ remote server, and also does not write it in the warc request record.
|
|||||||
|
|
||||||
Warcprox-Meta: {}
|
Warcprox-Meta: {}
|
||||||
|
|
||||||
- warc-prefix
|
Warcprox-Meta fields
|
||||||
- stats
|
-------------------
|
||||||
- buckets
|
|
||||||
- dedup-bucket
|
``warc-prefix`` (string)
|
||||||
- blocks
|
~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
- limits
|
Specifies a warc filename prefix. Warcprox will write the warc record for this
|
||||||
- soft-limits
|
capture, if any, to a warc named accordingly.
|
||||||
- metadata
|
|
||||||
- accept
|
Example::
|
||||||
- dedup-ok # deprecate?
|
|
||||||
|
Warcprox-Meta: {"warc-prefix": "special-warc"}
|
||||||
|
|
||||||
|
``stats`` (dictionary)
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
* buckets
|
||||||
|
|
||||||
|
Example::
|
||||||
|
|
||||||
|
Warcprox-Meta: {"stats":{"buckets":["my-stats-bucket","all-the-stats"]}}
|
||||||
|
|
||||||
|
``dedup-bucket`` (string)
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
Specifies the deduplication bucket. For more information about deduplication
|
||||||
|
see `<readme.rst#deduplication>`_.
|
||||||
|
|
||||||
|
Example::
|
||||||
|
|
||||||
|
Warcprox-Meta: {"dedup-bucket":"my-dedup-bucket"}
|
||||||
|
|
||||||
|
``blocks``
|
||||||
|
~~~~~~~~~~
|
||||||
|
|
||||||
|
``limits``
|
||||||
|
~~~~~~~~~~
|
||||||
|
|
||||||
|
``soft-limits``
|
||||||
|
~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
``metadata`` (dictionary)
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
``accept``
|
||||||
|
~~~~~~~~~~
|
||||||
|
|
||||||
Brozzler knows about ``warcprox-meta``. For information on configuring
|
Brozzler knows about ``warcprox-meta``. For information on configuring
|
||||||
``warcprox-meta`` in brozzler, see https://github.com/internetarchive/brozzler/blob/master/job-conf.rst#warcprox-meta
|
``warcprox-meta`` in brozzler, see https://github.com/internetarchive/brozzler/blob/master/job-conf.rst#warcprox-meta
|
||||||
|
Loading…
x
Reference in New Issue
Block a user