Inventory file

In YAML format, this type of file is used to group together the information required to carry out one or more operations. Inventory is not mandatory in an automation process, but becomes so in a multi-host/user environment.

You can organize your inventories as you see fit, but the most important thing is that the operation must have access to them. Inventory files can be encrypted with SOPS, (more details in this chapter).

This part uses the “run” command, Full details on this command.

Do you need an inventory file?

The easiest way is to find out why an inventory file will become necessary in a production process.

Password-protected SSH keys

An “OP1” operation is commonly performed on “server1”, and executed as follows :

automation run -h "server1" -op "OP1" -sshpk "private SSH key place"

“OP1” becomes necessary on ‘server2’, I add host ‘server2’ to ‘-h’ parameters

automation run -h "server1 server2" -op "OP1" -sshpk "private SSH key place"

The Boss decides that the SSH private key used to connect to the servers “server1” and “server2” must be password-protected, so we need to provide automation-cli with the secret to decrypt the SSH private key. :

automation run -h "server1 server2" -op "OP1" -sshpk "private SSH key place" -sshpass "supersecret"

Working in shared space, the Boss realizes that the bash command history shows the password “in the clear”. Bob recommends the use of “magic space” (strongly discouraged in teams), but half the sysadmins forget it. Bob suggests writing the secret in a file, and the command becomes :

automation run -h "server1 server2" -op "OP1" -sshpk "private SSH key place" -sshpass "$(cat ./supersecretfile)"

Each sysadmin now creates its own file, so the secret is now scattered all over the control node. The solution is to store this secret and the location of the key in a single file to be used by all sysadmins.

automation run -h "server1 server2" -op "OP1" -i "/inventoryFile.yaml"

Bob even decides to add this file to the Git repository containing all the operations.
The Boss consults the Git repository and notices that the secret, “in the clear”, is now visible to all those who have access to the repository!
Bob hadn’t yet had time to read the “Secrets” part of this documentation. He converts the inventory file into an encrypted SOPS file, changes the secret and uploads it to the Git repository.
The Boss sees that the file is encrypted in the Git repository, and can’t decrypt it, so he reverts to the previous version and tries to connect with the secret “in the clear”, but Bob has done his job…

Custom settings

Syntax

By passing the “-i” switch to the various commands supporting it, the value of this switch is the name of an inventory file, including the “.yaml” extension.

This path can be :

Related to execution path : ./inventory.yaml.
Absolute path : /directory1/directory2/inventory.yaml.
Related to OPSDirectory : “inventory.yaml” without ./ ou /, e.g. : If the value of the OPS environment variable is equal to $(pwd)/OPSDirectory then the final path will be the concatenation of the value of OPS and the file : $(pwd)/OPSDirectory/inventory.yaml.

For a “bash” shortcut to the “Home directory” of a server user (~/), do not enclose the value, e.g. : -i ~/mydirectory/inventory.yaml.

The syntax for reading the value of an inventory attribute is :

‘#inv.attribut.otherattribut’, also supports iteration on arrays : ‘#inv.attribut[0].otherattribut’.
also supports environment variables : ‘#inv.$USER.attribut’. The environment taken into account is that of the control node at runtime. These values are never resolved on the host.

When the value of an inventory attribute is empty or has not been found :

The environment variable is required: this causes a runtime error.
Environment variable not required: the environment variable is defined with an empty value, e.g. TEST=“”. This makes it easy to manage the definition of optional variables for bash. Remember, the final shell executed on the host includes the command: “set -u”, which implies that any variables used must first have been declared.

You can also specify an attribute representing an object, which will be converted into a JSON string that can be manipulated with ‘jq’ in your shell.

Inventory example :

servers:
  one:
    ipAddress: "xxxx"
  two:
    ipAddress: "xxxx"

If you specify -e MYVAR='#inv.servers', the extracted value will be a single-line JSON structure :

{ "one": { "ipAddress": "xxxx" }, "two": { "ipAddress": "xxxx" } }

which can be converted (in your shell) by ‘jq’ to array :

echo "${MYVAR}" | jq 'map(select(.))|.[]'

Result obtained with ‘jq :

[{ "one": { "ipAddress": "xxxx" } }, { "two": { "ipAddress": "xxxx" } }]

Example

An “OP1” operation is commonly performed on “server1”, and executed as follows :

automation run -h "server1" -op "OP1"

“OP1” becomes necessary on ‘server2’, I add the host ‘server2’ to the ‘-h’ parameter

automation run -h "server1 server2" -op "OP1"

The operation ends with an error for “server2”. After analysis, one of the files required has a different name than “server2”, and the command used in this operation includes a parameter to specify the location of this file.
Creating an inventory in this form :

server1:
  appfile: "/var/lib/1.conf"
server2:
  appfile: "/var/lib/5.conf"

This inventory shows the exact location of the required file on each server.
Let’s arrange the operation.
The operation is of the “script” type (run.sh), with the following contents :

#!/usr/bin/env bash
mycommand

According to the documentation, “mycommand” includes the “-fileplace” parameter to indicate the path of the required file, so the script becomes :

#!/usr/bin/env bash
mycommand -fileplace ${FILEPLACE}

I’ve added the “FILEPLACE” operation parameter, of type ‘string’, whose function specifies the location of the required file. This parameter becomes “required” for execution. Let’s modify the operation manifest :

comment: "operation OP1"
scripts:
  - "run.sh"
parameters:
  required:
    FILEPLACE:
      type: string
      comment: "file place [full path]"

Let’s run the operation, providing the inventory file and the value of the “FILEPLACE” variable :

automation run -h "server1 server2" -op "OP1" -i "/inventoryFile.yaml" -e FILEPLACE='#inv.$host.appfile'

How does it work?
- automation-cli detects a variable required by the operation and provided on the command line (“FILEPLACE”) and also detects the “#inv.” shortcut.
- automation-cli” deduces the use of an inventory file,
  - is it specified? Yes.
  - Does it exist? Yes.
  - The operation can be started simultaneously on each host.
  - On preparation, “automation-cli” substitutes “$host” with the host on which the operation is performed (“server1” or “server2”).
    - For “server1”, the value of “FILEPLACE” becomes “server1.appfile”,
    - reading the inventory, does the ‘server1.appfile’ attribute exist? Yes.
    - Its value replaces the original: “FILEPLACE=‘server1.appfile’ => FILEPLACE=”/var/lib/1.conf”, if the attribute does not exist in the inventory file, the operation is terminated with an error without even attempting execution.
    - The operation is executed, the operation does not end with an error.

Create an inventory file

The command to create an inventory file is: “cinv”, the “-age” option is recommended, this allows you to create a file encrypted with SOPS.

automation-cli cinv "[inventory yaml full path]" -age "[Your public age key]"

Exemple :

automation-cli cinv "./inventory.yaml" -age "agexxxxxxxx"

Predefined YAML attributes in the internal template

Nom	Type	Fonction
sshpk	string	path (absolute/relative) of SSH private key used to connect to hosts
sshpass	string	SSH private key password
serverGroups.all	Array	list of hosts as called by the -h option

Example of inventory

Host resolution by IP address

This scheme enables “automation-cli” to resolve host names into IP addresses. The operating mechanism is simple: the user defines the host(s) with the “-h” parameter. The “serverGroups” attribute is a tree structure, which can be browsed to resolve the IP address associated with a host. All “array” attributes are groups, used to organize hosts. If the final host is not referenced by an IP address, “automation-cli” will use the DNS resolution mechanism.

sshpk: "./sshkeys/maintenance"
sshpass: "mysupersecret"
serverGroups:
  all:
    - sgbdr
    - redis
    - localhost
  controlnode:
    - localhost
  sgbdr:
    - postgresql1
    - postgresql2
    - postgresql3
  redis:
    - redis1
    - redis2
    - redis3
  redis1: 172.28.10.1
  redis2: 172.28.10.2
  redis3: 172.28.10.3
  postgresql1: 172.28.50.1
  postgresql2: 172.28.50.1
  postgresql3: 172.28.50.1
  localhost: 127.0.0.1

Hosts are resolved by traversing the tree, with the last value acting as the “Endpoint”. Let’s take the ‘postgresql3’ server as an example. Run :

automation run -h "postgresql3" -op "OP1" -i "/inventoryFile.yaml"

Execution reports will indicate that the operation was performed on ‘postgresql3’, SSH connections will be made to ‘172.28.50.1’. Note that execution reports always show the association ‘host requested by user’ and ‘host used for SSH connection’.