backup/warcprox
1
0
Fork 0
mirror of https://github.com/internetarchive/warcprox.git synced 2025-01-18 13:22:09 +01:00
Code Issues Projects Releases Activity

connect quickly when a server is down

Browse Source 
Another tweak to that end. We have observed that when a rethinkdb server
is offline, an attempt to connect to it takes a second or two to time
out. On the other hand, if the host is up but the port is not open
(rethinkdb is not running or something like that), the connection
failure happens very quickly.

To achieve good performance in case a rethinkdb server is down, we are
now setting a timeout on the connect() call. The timeout starts at
0.1 sec, for quick retry, and backs off up to 10 sec in case of repeated
failures.
This commit is contained in:
Noah Levitt 2018-11-01 19:17:50 +00:00
parent 59c8e7b8cd
commit 8ec2d853b6
1 changed files with 5 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
doublethink/rethinker.py
View File

@ -123,13 +123,15 @@ class Rethinker(object):
try:
try:
host, port = server.split(':')
return r.connect(host=host, port=port)
return r.connect(
host=host, port=port, timeout=max(0.1, retry_wait))
except ValueError:
return r.connect(host=server)
return r.connect(host=server, timeout=max(0.1, retry_wait))
except Exception as e:
self.logger.warn(
'will keep trying after failure connecting to '
'rethinkdb server at %s: %s', server, e)
'rethinkdb server at %s: %s (sleeping for %s sec)',
server, e, retry_wait)
time.sleep(retry_wait)
retry_wait = min(retry_wait * 2, 10.0)