backup/warcprox
1
0
Fork 0
mirror of https://github.com/internetarchive/warcprox.git synced 2025-01-18 13:22:09 +01:00
Code Issues Projects Releases Activity

tweak some docs

Browse Source
This commit is contained in:
Noah Levitt 2017-05-17 12:28:48 -07:00
parent 03e9d4eeef
commit d33695e40b
1 changed files with 18 additions and 22 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

40
doublethink/services.py
View File

@ -31,24 +31,24 @@ class ServiceRegistry(object):
by calling `heartbeat(status_info)` periodically. by calling `heartbeat(status_info)` periodically.
`status_info` is a dict and must have at least the fields 'role', 'load', `status_info` is a dict and must have at least the fields 'role', 'load',
and 'ttl'. Certain other fields are populated automatically and 'ttl'. Certain other fields are populated automatically as in the
as in the example below. In addition, services may set arbitrary other example below. In addition, services may set arbitrary other fields.
fields.
Some information about required fields: Some information about required fields:
'role': The role of the service. `healthy_service()` and 'role': The role of the service. `healthy_service()` and
`healthy_services()` look up services using this field. `healthy_services()` look up services using this field.
'ttl': Specifies the expected time between heartbeats. If a service's 'ttl': If a service's last heartbeat was more than 'ttl' seconds ago, it
last heartbeat was more than `ttl` seconds ago, it is considered is considered to be "down". `healthy_services()` and
to be "down". `healthy_services()` and `healthy_service()` never `healthy_service()` never return entries for services that are
return entries for services that are considered "down". considered "down". A sensible convention is to heartbeat 3 times per
'ttl', that is, every `ttl/3` seconds.
'load': An arbitrary numeric value. It is up to each service to populate 'load': An arbitrary numeric value. It is up to each service to populate
this field in a way that makes sense to the particular service. this field in a way that makes sense to the particular service.
`healthy_service(role)` returns the service with the lowest load `healthy_service(role)` returns the service with the lowest load
for the supplied role. Thus load values need to be comparable to for the supplied role. Thus load values need to be comparable to
within the context of a single service, but comparing loads of within the context of a single service, but comparing loads of
services of different roles does not necessarily make any sense. services of different roles might not make any sense.
About the 'id' field: About the 'id' field:
@ -72,7 +72,7 @@ class ServiceRegistry(object):
'id': 'd0bed0be-d000-d000-f00d-abeefface0ff' # generated by rethinkdb if not supplied 'id': 'd0bed0be-d000-d000-f00d-abeefface0ff' # generated by rethinkdb if not supplied
'role': 'brozzler-worker', 'role': 'brozzler-worker',
'load': 0.5, # load score 'load': 0.5, # load score
'ttl': 20.0, 'ttl': 60.0,
'host': 'wbgrp-svc999.us.archive.org', # set in svcreg.heartbeat() as a fallback 'host': 'wbgrp-svc999.us.archive.org', # set in svcreg.heartbeat() as a fallback
'pid': 1234, # set in svcreg.heartbeat() as a fallback 'pid': 1234, # set in svcreg.heartbeat() as a fallback
'first_heartbeat': '2015-10-30T03:39:40.080814', # set in svcreg.heartbeat() 'first_heartbeat': '2015-10-30T03:39:40.080814', # set in svcreg.heartbeat()
@ -121,9 +121,8 @@ class ServiceRegistry(object):
service service
`status_info` must have at least the fields 'role', 'load', and `status_info` must have at least the fields 'role', 'load', and
'ttl'. Some additional fields are populated 'ttl'. Some additional fields are populated automatically by this
automatically by this method. If the field 'id' is absent, it will be method. If the field 'id' is absent, it will be generated by rethinkdb.
generated by rethinkdb.
See the ServiceRegistry class-level documentation for more information See the ServiceRegistry class-level documentation for more information
about the various fields. about the various fields.
@ -134,8 +133,7 @@ class ServiceRegistry(object):
Raises: Raises:
Exception: if `status_info` is missing a required field, or a Exception: if `status_info` is missing a required field, or a
`status_info['ttl']` is not a number greater `status_info['ttl']` is not a number greater than zero
than zero
''' '''
for field in 'role', 'ttl', 'load': for field in 'role', 'ttl', 'load':
if not field in status_info: if not field in status_info:
@ -226,12 +224,10 @@ class ServiceRegistry(object):
candidate['id'] = role candidate['id'] = role
if not 'ttl' in candidate: if not 'ttl' in candidate:
raise Exception( raise Exception("candidate is missing required field 'ttl'")
"candidate is missing required field "
"'ttl'")
val = candidate['ttl'] val = candidate['ttl']
if not (isinstance(val, float) or isinstance(val, int)) or val <= 0: if not (isinstance(val, float) or isinstance(val, int)) or val <= 0:
raise Exception('"ttl" must be a number > 0') raise Exception("'ttl' must be a number > 0")
candidate['first_heartbeat'] = now candidate['first_heartbeat'] = now
candidate['last_heartbeat'] = now candidate['last_heartbeat'] = now
@ -268,8 +264,8 @@ class ServiceRegistry(object):
''' '''
Find least loaded healthy service in the registry. Find least loaded healthy service in the registry.
A service is considered healthy if its 'last_heartbeat' is in the last A service is considered healthy if its 'last_heartbeat' was less than
`ttl` seconds. 'ttl' seconds ago
Args: Args:
role (str): role name role (str): role name
@ -290,8 +286,8 @@ class ServiceRegistry(object):
''' '''
Look up healthy services in the registry. Look up healthy services in the registry.
A service is considered healthy if its `last_heartbeat` is in the last A service is considered healthy if its 'last_heartbeat' was less than
`ttl` seconds. 'ttl' seconds ago
Args: Args:
role (str, optional): role name role (str, optional): role name