Playbooks, host vars, group vars

pdo debian sw eng

This is part of a series of posts on ideas for an ansible-like provisioning system, implemented in Transilience.

Host variables

Ansible allows to specify per-host variables, and I like that. Let's try to model a host as a dataclass:

@dataclass
class Host:
    """
    A host to be provisioned.
    """
    name: str
    type: str = "Mitogen"
    args: Dict[str, Any] = field(default_factory=dict)

    def _make_system(self) -> System:
        cls = getattr(transilience.system, self.type)
        return cls(self.name, **self.args)

This should have enough information to create a connection to the host, and can be subclassed to add host-specific dataclass fields.

Host variables can then be provided as default constructor arguments when instantiating Roles:

    # Add host/group variables to role constructor args
    host_fields = {f.name: f for f in fields(host)}
    for field in fields(role_cls):
        if field.name in host_fields:
            role_kwargs.setdefault(field.name, getattr(host, field.name))

    role = role_cls(**role_kwargs)

Group variables

It looks like I can model groups and group variables by using dataclasses as mixins:

@dataclass
class Webserver:
    server_name: str = "www.example.org"

@dataclass
class Srv1(Webserver):
    ...

Doing things like filtering all hosts that are members of a given group can be done with a simple isinstance or issubclass test.

Playbooks

So far Transilience is executing on one host at a time, and Ansible can execute on a whole host inventory.

Since the most part of running a playbook is I/O bound, we can parallelize hosts using threads, without worrying too much about the performance impact of GIL.

Let's introduce a Playbook class as the main entry point for a playbook:

class Playbook:
    def setup_logging(self):
        ...

    def make_argparser(self):
        description = inspect.getdoc(self)
        if not description:
            description = "Provision systems"

        parser = argparse.ArgumentParser(description=description)
        parser.add_argument("-v", "--verbose", action="store_true",
                            help="verbose output")
        parser.add_argument("--debug", action="store_true",
                            help="verbose output")
        return parser

    def hosts(self) -> Sequence[Host]:
        """
        Generate a sequence with all the systems on which the playbook needs to run
        """
        return ()

    def start(self, runner: Runner):
        """
        Start the playbook on the given runner.

        This method is called once for each system returned by systems()
        """
        raise NotImplementedError(f"{self.__class__.__name__}.start is not implemented")

    def main(self):
        parser = self.make_argparser()
        self.args = parser.parse_args()
        self.setup_logging()

        # Start all the runners in separate threads
        threads = []
        for host in self.hosts():
            runner = Runner(host)
            self.start(runner)
            t = threading.Thread(target=runner.main)
            threads.append(t)
            t.start()

        # Wait for all threads to complete
        for t in threads:
            t.join()

And an actual playbook will now look like something like this:

from dataclasses import dataclass
import sys
from transilience import Playbook, Host


@dataclass
class MyServer(Host):
    srv_root: str = "/srv"
    site_admin: str = "enrico@enricozini.org"


class VPS(Playbook):
    """
    Provision my VPS
    """

    def hosts(self):
        yield MyServer(name="server", args={
            "method": "ssh",
            "hostname": "host.example.org",
            "username": "root",
        })

    def start(self, runner):
        runner.add_role("fail2ban")
        runner.add_role("prosody")
        runner.add_role(
                "mailserver",
                postmaster="enrico",
                myhostname="mail.example.org",
                aliases={...})


if __name__ == "__main__":
    sys.exit(VPS().main())

It looks quite straightforward to me, works on any number of hosts, and has a proper command line interface:

./provision  --help
usage: provision [-h] [-v] [--debug]

Provision my VPS

optional arguments:
  -h, --help     show this help message and exit
  -v, --verbose  verbose output
  --debug        verbose output

Next step: check mode!