Dynamic settings

Those settings are stored in the database and can be dynamically changed by API request.

They must comply with three goals:

  1. being dynamic

  2. use the module marshmallow to set the type of parameters

  3. can export a JSON used by the UI to create a form dynamically.

Being dynamic

Each Flask request is independent. The settings are loaded at the beginning in a @before_request.

As they won’t change much, find a way with Redis to store the settings with a system to notify Flask to update them if needed

Using marshmallow

Using its schema and fields to automatically check if a setting’s dictionary is ok.

Export a JSON to create a form.

The goal here is that the UI doesn’t need to hardcode the settings. Meaning, to add a setting you only need to modify the backend. SOGo has its own way to create such a JSON.

Parameter setting

Each setting will be translated like this:

{
    "name": "SOGO_D_SIEVE_PORT",
    "data_type": "number",
    "default": 4190,
    "required": false,
    "constraints": {
            "max_inclusive": 65535,
            "min_inclusive": 1
            },
    "depends": "SOGO_D_MAIL_FILTERING_TYPE%%%equal%%%sieve",
}

name

A string, with the "code" name of the settings. The UI has to handle the translation itself.

data_type

A string representing the type of the setting. Values can be:

  • "bool": this setting is a boolean, expects JSON value 'true' or 'false'.

  • "str": this setting is a string.

  • "secret": this setting is a string of a password or secret, it should be hidden in the UI.

  • "email": this setting is a string of an email, meaning it expects one @ and at least one dot after.

  • "url": this setting is a string url, the parameter constraints prefixes give details about the schemes.

  • "number": this setting is an integer. #TODO for now we don’t have any float in the settings, but maybe already add a difference for the future.

    • "dict": this setting expects a dictionary\[string, string\]

  • "list[X]": this setting is a list of type X, X can be any of the above. All items of the list must be of the same type.

default

Default value of the setting. It is of type data_type. If there is no default value, this param will be null.

required

Boolean, True if this setting is required.

if a setting has a dependency (see: depends) and required is True. That means the setting is only required if the dependency is satisfied. If not, the setting is not required even if this param is True

constraints

A dict with details about constraints on this setting. If there are no constraints, the value will be null. Available constraints are:

  • "choice": is a list with available values for this setting, any other value will be rejected.

  • "prefixes": is a list of mandatory prefixes for this setting

Example: "prefixes": ["http://", "https://"] means the setting value must be https://www.sogo.nu/ or http://www.sogo.nu/

  • min, max: an integer with the minimum or maximum (exclusive) value of this number

  • min_inclusive, max_inclusive: idem but inclusive

  • len_min, len_max: in case of string, the minimum or maximum length (inclusive)

depends

Some settings only make sense if another setting has a certain value. For example, OpenID settings are only needed if the authentication type is set to 'openid'.

depends value represents that. It’s a string with this format "{var_name}%%%equal%%%{var_value}" It means that the current setting requires var_name to be equal to var_value

In the example at the start of the page, it means SOGO_D_SIEVE_PORT is only shown/useful if SOGO_D_MAIL_FILTERING_TYPE = "sieve"

Dependencies can be nested, look at these three settings:

{
    "constraints": null,
    "data_type": "bool",
    "default": null,
    "depends": null,
    "name": "US_CAN_AUTH",
    "required": true
}
 
{
    "constraints": null,
    "data_type": "str",
    "default": null,
    "depends": "US_CAN_AUTH%%%equal%%%True",
    "name": "US_PWD_ALGO",
    "required": false
}
 
{
    "constraints": null,
    "data_type": "str",
    "default": null,
    "depends": "US_PWD_ALGO%%%equal%%%sym-aes-128-cbc",
    "name": "US_SIM_KEY_VALUE",
    "required": false
}

  • US_PWD_ALGO is only useful if US_CAN_AUTH = true.

  • US_SIM_KEY_VALUE is only useful if US_PWD_ALGO = "sym-aes-128-cbc".

  • Technically US_SIM_KEY_VALUE is also only useful if US_CAN_AUTH = true but this dependency is not shown in US_SIM_KEY_VALUE dictionary. It is inherited from US_PWD_ALGO.

Parameter Category

To avoid having all parameters in one block, SOGo sorts them by category. A category contains a list of parameters. To do that, there is a class called SogoSchema that inherits marshmallow.Schema. This class adds attributes useful to create those categories and also to create the parameter setting above.

The category will be present in the database. If a setting changes category, do an update_script for the next version.
SogoSchema is defined here app/config/settings/SogoSchema.py

SogoSchema attributes

subparent

A string, this is the name of the category

dependencies

It is a dictionary to tell the dependencies of the parameters

    dependencies = {
        "PARAM1": ("PARAM2", "value")
    }

This example means PARAM1 needs PARAM2 to be equal to "value" to be needed.

will be translated as depends = "PARAM2%%%equal%%%value" in PARAM1 dict

is_required

marshmallow.field already has an argument required.

param = field.String(required=True)

But a parameter with a dependency can’t use that as it won’t be in the dictionary if its dependency is not satisfied. is_required is a set with all parameters with dependency that are required. The SOGoSchema validator will use this to check if the data is correct.

Example: US_LDAP_HOSTNAME is the parameter with the LDAP hostname. It depends on US_TYPE = 'ldap'.
If the user source is ldap, US_LDAP_HOSTNAME is required. But if we do this:

US_LDAP_HOSTNAME = field.String(required=True)

And if US_TYPE = 'sql', meaning US_LDAP_HOSTNAME will be absent, SOGoSchema will raise an error because US_LDAP_HOSTNAME is missing.

For cases like this, we add US_LDAP_HOSTNAME to is_required.

is_secret

Simply a set with parameter names that are secrets as marshmallow doesn’t handle this case.

is_duplicable

Some categories can have one or several blocks of themselves. is_duplicable is a bool to indicate that. Categories that are duplicable will be lists themselves in the database

Example, there can be several user sources. The Schema for user source’s parameters has is_duplicable=True

is_uid

For a duplicable category, there is always a unique identifier parameter to distinguish themselves. Besides adding the field parameter, also put the name in is_uid which is a string?

validate_required_with_dependency

This method uses the wrapper marshmallow.validates_schema meaning it will also run when the schema is validating data. It will simply check parameters that have a dependency and are required in that case.

Creating the dynamic form

All the magic happens in app/config/settings/DynamicFormSettings.py

It takes a SogoSchema and creates the adequate dictionary with two keys:

  • name of the category (subparent): its value is a list of all parameters

  • "is_duplicable": true or false if this category is duplicable

this dynamic form is only used by the UI to create the correct data form. It does not contain the real values of these parameters 
{
    "is_duplicable": true,
    "subparent/category": [
        {
            "name": "SOGO_D_SIEVE_PORT",
            "data_type": "number",
            "default": 4190,
            "required": false,
            "constraints": {
                    "max_inclusive": 65535,
                    "min_inclusive": 1
                    },
            "depends": "SOGO_D_MAIL_FILTERING_TYPE%%%equal%%%sieve",
        },
        "...",
        "and all others parameters in dict...",
    ]
}

The final dict returned by the API will also contain the type of settings, domain or system, like this:

{
    "domain": [
        {
            "is_duplicable": true,
            "subparent/category": [
            {
                "name": "SOGO_D_SIEVE_PORT",
                "data_type": "number",
                "default": 4190,
                "required": false,
                "constraints": {
                        "max_inclusive": 65535,
                        "min_inclusive": 1
                        },
                "depends": "SOGO_D_MAIL_FILTERING_TYPE%%%equal%%%sieve",
            },
            "...",
            "and all others parameters in dict...",
            ]
        },
        {
            "other_category": "here"
        }
    ],
    "system": [
        "idem": "also a list of category dict"
    ]
}