score.init

Introduction

This module provides helper functions that will allow implementing our initialization guidelines conveniently. It provides an auto-initializer for a list of score-modules and several functions supporting the initialization process itself.

Configuration

score.init.init(confdict, *, overrides={}, init_logging=True, finalize=True)

This function automates the process of initializing all other modules. It will operate on given confdict, which is expected to be a configparser.ConfigParser object or a 2-dimensional dict mapping names of modules to their respective confdicts. The recommended way of acquiring such a confdict is through parse_config_file(), but any 2-dimensional dict is fine.

The confdict should also contain the configuration for this module, which interprets the configuration key modules (which should be accessible as confdict['score.init']['modules']):

modules

A list of module names that shall be initialized. If this value is missing, you will end up with an empty ConfiguredScore object.

The provided overrides will be integrated into the actual confdict prior to initialization. While the confdict is assumed to be retrieved from external resources (like a configuration file), this parameter aims to make programmatic adjustment of the configuration a bit easier.

The final parameter init_logging makes sure python’s own logging facility is initialized with the provided configuration, too.

This function returns a ConfiguredScore object.

score.init.init_from_file(file, *, overrides={}, init_logging=True)

Reads configuration from given file using config.parse_config_file() and initializes score using init(). See the documentation of init() for a description of all keyword arguments.

score.init.parse_config_file(file, *, recurse=True, return_configparser=False)

Reads a configuration file and returns a nested dict.

The main feature of this function is the support for “adjustment files”, i.e. files that do not actually define all values, but define deviations from another file. The function will collect the set of all values by recursing into the files defined in the initial file. This whole feature can be disabled, though, by passing a falsy value as the recurse argument, in which case the function just behaves like a configparser with configparser.ExtendedInterpolation.

If the recurse value is left at its default, the parsing process is at follows:

• The file is parsed using the configparser.ExtendedInterpolation.

• If there is no section score.init, or that section does not have a key based_on, the parsed configuration is returned as-is. At this point the function was just used to parse the given file as a stand-alone file.

• If a base file, as described earlier, was configured, that file is parsed first. The current file is regarded as an “adjustment file”, which mangles the configuration provided in the base file.

• The function now iterates on all key/value pairs of the adjustment file and updates the base configuration in the following manner:

  • If the adjustment value is “<delete>”, the value in the original configuration is removed. If this leaves the section in the original file empty, the section is removed as well.

  • If the adjustment value starts with the string “<diff>”, it is considered to contain a value in pseudo-diff format operating on the base value. The accepted format for this mode is explained later.

  • If the adjustment value is of the form “<replace:regex:replacement>”, the regular expression given as regex is applied on the base value and the first occurrence is replaced with the value given as replacement. If one wants to replace all occurrences, it is possible to do so providing the flag “g” as last parameter: “<replace:regex:replacement:g>”.

    The colons used in the example can be replaced with any other character, so the same rule could have been written as “<replace/regex/replacement>”.

    It is further possible to chain multiple replace actions.

  • Otherwise the value is considered to be the replacement for the value in the base configuration.

• The updated configuration is the return value of the function.

Example with the following base file called “app.conf”:

[score.init]
modules =
    score.ctx
    score.db
    score.es

[score.db]
base = fuf.db.base.Storable
sqlalchemy.url = sqlite:///${here}/database.sqlite3
destroyable = true

The next file is intended to adjust the above configuration to the local environment:

[score.init]
based_on = app.conf
modules = <diff>
    -score.es

[score.db]
sqlalchemy.url =
    <replace:database:app>
    <replace:\.sqlite3$:.db>
destroyable = <delete>

The resulting configuration will behave as if the input file looked like this:

[score.init]
based_on = app.conf
modules =
    score.ctx
    score.db

[score.db]
base = fuf.db.base.Storable
sqlalchemy.url = sqlite:///${here}/app.db

The custom diff format of this function works without line numbers and just consists of removals (lines with a leading dash), additions (leading plus sign) and anchors (leading space or no character at all).

The only issue with this format that has no line numbers is the question where to insert the additions. The solution to this problem is: right after the last anchor, or, if there was no anchor, at the beginning of the string.

Also note that all removals also act as anchors for this purpose.

Here are some examples demonstrating the above:

|foo     |+baz      |baz
|bar  +  |      =>  |foo
                    |bar

|foo     | bar      |foo
|bar  +  |+baz  =>  |bar
                    |baz

|foo     | foo      |foo
|bar  +  |+baz  =>  |baz
                    |bar

|foo     |-bar      |foo
|bar  +  |+baz  =>  |baz

score.init.init_logging_from_file(file)

Just the part of init_from_file() that would initialize logging.

class score.init.ConfiguredScore(confdict, modules, dependency_aliases)

The return value of init(). Contains the resulting ConfiguredModule of every initialized module as a member.

class score.init.ConfiguredModule(module)

The return value of an init function. This class is abstract and modules must create sub-classes containing their respective configuration.

class score.init.DependencySolver

A simple helper for resolving module interdependencies. Basic usage:

solver = DependencySolver()
solver.add_dependency('a', 'b')  # a depends on b
solver.add_dependency('b', 'c')  # b depends on c
solver.solve()  # returns all modules in the order in which they
                # should be initialized: ['c', 'b', 'a']

Exceptions

class score.init.InitializationError(module, *args, **kwargs)

Base class for exceptions to raise when the initialization of a module fails.

class score.init.DependencyLoop(loop, *, module='score.init')

Thrown if a dependency loop was detected during a call to init() or ConfiguredModule._finalize().

class score.init.ConfigurationError(module, *args, **kwargs)

The exception to raise when the initialization of a module failed due to a bogus configuration.

Helper functions

score.init.parse_bool(value)

Converts a string value to a boolean. This function will accept the same strings as the default configuration of python’s configparser module.

score.init.parse_list(value)

Converts a string value to a corresponding list of strings. Substrings are assumed to be delimited by newline characters.

score.init.parse_host_port(value, fallback=None)

Extracts a host and a port definition from given value. Valid values are:

• hostname
• hostname:port

The return value will be a 2-tuple containing the hostname and the port. If the given value is empty, or contains no port definition, these values can be dropped in from a give fallback value, which can have the same format as defined for the first parameter (a str), or the same format as the return value (a tuple).

The following call would return ('example.com', 5109):

>>> parse_host_port('example.com', 'localhost:5109')

score.init.parse_datetime(value)

Returns the given value as a datetime without timezone information.

The following formats are currently supported:

• YYYY-MM-DD HH:II
• YYYY-MM-DD HH:II:SS
• YYYY-MM-DD HH:II:SS.microsecond
• timestamp

score.init.parse_time_interval(value)

Converts a human readable time interval string to a float in seconds.

>>> parse_time_interval('3s')
3.0
>>> parse_time_interval('5 milliseconds')
0.005
>>> parse_time_interval('1 minute')
60.0
>>> parse_time_interval('2 hours')
7200.0
>>> parse_time_interval('365days')
31536000.0

score.init.parse_dotted_path(value)

Converts a dotted python path to the denoted object. The following will return the randint() function from the random module, for example:

parse_dotted_path('random.randint')

score.init.parse_call(value, args=(), kwargs={})

Parses a string containing a function call or an object construction. The given value is expected to call the path to a python object (as interpreted by parse_dotted_path()), followed by an opening parenthesis, arguments and keywords separated by commas, and a closing parenthesis.

This will look a lot like real python code, but the actual invocation will be enriched with the args and kwargs given to this function. If this function is invoked like the following …

>>> parse_call('foo.Test(3, ovr=b)', (1, 2), kwargs={'ovr': 'c', 'bar': 4})

… it will invoke and return the result of the following code:

>>> foo.Test(1, 2, '3', ovr='b', bar=4)

Note that all arguments (and keyword arguments) in the value string will be passed as strings!

score.init.parse_object(confdict, key, args=(), kwargs={})

Creates an object from a confdict. This function either expects a string accepted by parse_call(), or a more verbose and flexible configuration. If the given value in the confdict contains an opening parenthesis, it is assumed to be in the terse format, in which case it will be parsed by parse_call.

In any other case, the function assumes that the confdict contains a class name under the specified key, which will be parsed using parse_dotted_path(). The function will then extract all other keys from confdict that start with the same key, process them and pass the resulting dict to the class’s contructor as keyword arguments. It is possible to provide additional arguments to the constructor as args.

The aforementioned processing phase will replace all multi-line values with arrays using parse_list().

For example, the following configuration:

versionmanager = score.webassets.versioning.Mercurial
versionmanager.folder = /usr/share/versionmanager
versionmanager.repos =
    /var/www/project
    /var/www/library1
    /var/www/library2

will invoke the constructor like this:

Mercurial(folder="/usr/share/versionmanager", repos=[
    "/var/www/project",
    "/var/www/library1",
    "/var/www/library2",
])

score.init.parse_json(value)

Converts a string value to a python object. Currently this function is just a wrapper around json.loads().

score.init.extract_conf(configuration, prefix, defaults={})

This function can be used to extract confdict values with a given prefix. When called with the prefix spam., for example, it will return all values in the confdict that start with that string.

If a defaults dict is present, it will be used as the base for the return value.

>>> defaults = {
...   'eggs': 'Spam and eggs',
... }
>>> conf = {
...   'spam.eggs': 'Eggs with Spam!',
...   'spam.bacon.eggs': 'Spam, bacon and eggs',
...   'bacon.spam': 'Bacon and Spam'
... }
>>> extract_conf(conf, 'spam.', defaults)
{'eggs': 'Eggs with Spam!', 'bacon.eggs': 'Spam, bacon and eggs'}

score.init.init_cache_folder(confdict, key, autopurge=False)

Initializes the cache folder described in the confdict with the given key. This function will make sure that the folder exists and is writable and return its absolute path.

If autopurge is True, it will further write the whole confdict into a file called __conf__ in the folder to detect changes to the confdict. If the function thus detects a confdict change during the next initialization, it will delete the contents of the folder, assuming that its contents have become obsolete.