mirror of
https://github.com/internetarchive/warcprox.git
synced 2025-01-18 13:22:09 +01:00
The last approach was not good, timeout of 0.1 seconds was too short. A bunch of stuff has to happen in the timeout period inside of rethinkdb.connect(). It doesn't offer a way to set only the socket timeout. Even a timeout of 0.5 seconds results in a noticeable error rate. The new approach is to put a server in the penalty box for 5 minutes when it errors. While the server is in the penalty box, we don't try to connect to it, unless all the servers are in the penalty box, in which case we try the server that errored least recently.