The recorder component acts a proxy component, intercepting requests to and response from the :ref:`warcserver` and recording them
to a WARC file on disk.
The recorder uses the :class:`pywb.recorder.multifilewarcwriter.MultiFileWARCWriter` which extends the base :class:`warcio.warcwriter.WARCWriter` from :mod:`warcio` and provides support for:
* appending to multiple WARC files at once
* WARC 'rollover' based on maximum size idle time
* indexing (CDXJ) on write
Many of the features of the Recorder are created for use with Webrecorder project, although the core recorder is used to provide
a basic recording via ``/record/`` endpoint. (See: :ref:`recording-mode`)
Deduplication Filters
---------------------
The core recorder class provides for optional deduplication using the :class:`pywb.recorder.redisindexer.WritableRedisIndexer` class which requires Redis to store the index, and can be used to either:
* write duplicates responses.
* write ``revisit`` records.
* ignore duplicates and don't write to WARC.
Custom Filtering
----------------
The recorder filter system also includes a filtering system to allow for not writing certain requests and responses.
Filters include:
* Skipping by regex applied to source (``Warcserver-Source-Coll`` header from Warcserver)
* Skipping if ``Recorder-Skip: 1`` header is provided
* Skipping if ``Range`` request header is provided
* Filtering out certain HTTP headers, for example, http-only cookies
