* master:
bump version number after PR #72
Fix SOCKS connection error
Improve Connection Pool
Reduce the PoolManager num_pools size and fix bugs
bump dev version after PR #75
bump dev version number
Fix ListenerPostfetchProcessor typo
Configurable tmp file max memory size
Address unit test failure in Python 3.4
a minimal example
Extra connection evaluation before putting it back to the pool
Fix typo
Remove whitespace
Remote server connection pool
Set connection pool maxsize to 6 (borrowing from browser behavior).
Set num_pools to `max_threads / 6` but set a minimum of 200 for the cases
that we use a very low number of `max_threads`.
Remove `connection_is_fine` variable from connection code.
Fix http headers bug introduced in the previous commit.
Define PoolManager num_pools size as `max(max_threads, 500)` and reduce
each pool size from 100 to 30. The aim is to limit the total number of
open connections.
Fix remote SOCKS connection typo.
Now that we reuse remote connections, its better NOT to remove the
`keep-alive` request header. We need to send it to the remote host to make it
keep the connection open if possible.
We use `tempfile.SpooledTemporaryFile(max_size=512*1024)` to keep
recorded data before writing them to WARC.
Data are kept in memory when they are smaller than `max_size`, else they
are written to disk.
We add option `--tmp-file-max-memory-size` to make this configurable.
A higher value means less /tmp disk I/O and higher overall performance but
also increased memory usage.
Use urllib3 connection pooling to improve remote server connection
speed. Our aim is to reuse socket connections to the same target hosts when
possible.
Initialize a `urllib3.PoolManager` in `SingleThreadedWarcProxy` and use
it in `MitmProxyHandler` to connect to remote servers.
Socket read / write and ssl / socks code is exactly the same, only the
connection management changes.
Use arbitratry settings: pool_size=2000 and maxsize=100 (number of
connections per host) for now. Maybe we can come up with better values in the
future.
* trough-utf8:
make sure to send utf-8 to trough
bump dev version after revert
Revert "Merge pull request #67 from vbanos/update-ssl-ciphers"
bump dev version number after PR merge
Generate wildcard certs to reduce the number of certs generated
