Configbackup with Ansible + Git

Tired of scrolling through massive directories on a file server, flooded with config files? Easy to implement ‘backup’ solution at day one (IOS archive?), but often based on debatable, unreliable protocols like TFTP, combined with great challenges in tracking changes on day two.

So, this alternative approach takes Ansible to fetch running configs from network devices and stores them in a Git enabled directory for future use. Git brings build in versioning to the table, so there is only the latest config per device visable, plus the possibility to reliably track changes over the whole network.
It uses ansible ios modules to gather the configuration of Cisco IOS devices, but can of course be changed / complemented with other vendor modules. Success or failures for every device will be reported via syslog to an external receiver.

Git preperations

Install Git at your automation host.

$ sudo yum install git
[...]
$ git --version
git version 1.8.3.1

Create a directory to store the configuration files and initialize a new git repository in this directory.

$ mkdir configs
$ cd configs
$ git init
Initialized empty Git repository in /home/nwmichl/configs/.git/
$ cd ..

Ansible playbook

Ansible should be installed already, otherwise have a look at the Ansible Installation page. In most cases (CentOS/RHEL) a simple

$ sudo yum install epel-release
$ sudo yum install ansible

gets the job done.

The complete playbook ios_cfgback.yml can be downloaded from github. You might need to install wget first or use curl with the same URL to copy/paste local on your host.

$ sudo yum install wget
$ wget https://raw.githubusercontent.com/NWMichl/configbackup/master/ios_cfgback.yml

Ok, now let’s take this playbook apart and have a look at all these different tasks. No surprises with regard to the header, just remember to alter the vars section according to your needs:

---
- name: CONFIG BACKUP TO LOCAL DIRECTORY
  hosts: all
  connection: network_cli
  ignore_errors: yes
  gather_facts: no

  vars:
    config_path: /home/nwmichl/configs
    syslog_host: 192.168.1.12
    timestamp_line_identifier: "! Last configuration"

Errors have to be ignored, because we don’t want the playbook to terminate halfway in the middle of the night – moreover, the ‘Git check’ task has to fail sometimes on purpose.

Task – GATHER FACTS VIA SSH

Uses the ios_facts module to gather all config facts, including the running config, which makes total sense because nobody has ever forgotten to copy run start after a change via CLI, right? The task is only executed, if the network_os equals ios. So if you herd a multi vendor environment, just add additional tasks with a corresponding conditional in terms of the ansible_network_os .

Task – SAVE CONFIG TO FILE

The full running config of each device can be accessed via the variable ansible_net_config and will be stored to a file named <hostname>.txt in the directory specified in config_path .

Tasks – SYSLOG MESSAGE

Depending on the value of ios_facts_result.failure and save_result.failure there will be a notice or error syslog message send to the receiver specified as syslog_host. This is a very important step, as you want to run this playbook unattended, maybe after working hours. So there has to be a way to get information about failures and syslog seems to be the obvious choice. But as you own your destiny aka playbook, you are also able to add email notifications or Slack messages for instance.

Task – REMOVE CONFIG LINES WITH TIMESTAMPS

This task sanitizes the config file from lines with timestamps like ‘Last configuration change …’. The goal is to version only actual changes to the config, not a random ‘copy run start’ by a human or a stupid config mgmt tool.

Task – GIT – ADD ALL (NEW) FILES AND CHECK IF WORKING DIRECTORY IS DIRTY

Ansible has no native git add and commit module so far, so we have to break out to the shell to add all files to staging, including new ones, and test if there has been any changes.

Task – GIT – COMMIT/PUSH ONLY IF WORKING DIRECTORY IS DIRTY

If there are any changes in the working directory, this task commits with a timestamp to the master branch. If you use a central repo like GitHub/Lab for all your config files, just uncomment the line ‘git push -f origin master’ to push your local repository.

Tasks – SYSLOG MESSAGES GIT

Even for the Git part of the playbook there are three tasks to generate helpful syslog messages – just in case.

Watch this asciinema recording of a playbook run, followed by a change to one device and another playbook run to show the different behavior.

Cronjob

To schedule a daily backup job (3:00 am) at your automation host, just generate a user crontab and enter the following line:

$ crontab -e 

0 3 * * * ansible-playbook ios_cfgback.yml

If you’re not used to vim editor:

• i enters editing/insert mode
• <esc> leaves editing/insert mode
• : x <enter> saves the file and exits vim

The device credentials can be hard coded as clear text ansible_user / ansible_password in the /etc/ansible/hosts file, but shouldn’t – and certainly not in a production environment. Please use ssh keys or Ansible vault instead!

At this point everything’s ready for prime time. The files in the configs directory reflect the state of the last change based on the playbook run at 3:00am every day. You can use shell commands like git log -p or git blame <file> to follow changes in the local repository.

Using a GitHub repository

And here is how to bind your local git repository to a central web based one, like GitHub. An on-prem hosted instance of GitLab would be another way, especially for security sensitive environments.
You need an account on GitHub and create a new private repository. It makes sense to use the directory name of your config files, in my case: github.com/NWMichl/configs

Pushing a repository via https to GitHub usually needs to manually type in user credentials, which is a no go for automation purposes. The git credential.helper stores username and password of your GitHub account as clear text in a file located in the home directory instead, once typed in for the first time. You might want to switch to a ssh connection later and use public/private keys to push to the GitHub repo.

$ cd configs
$ git remote add origin https://github.com/NWMichl/configs.git
$ git config credential.helper store
$ git push -f origin master
Username for 'https://github.com': nwmichl
Password for 'https://nwmichl@github.com':
Everything up-to-date

Summary

We use a similar playbook with great success in production, including some enhancements like multi platform, email notification, summarization of all faults and a check if files are readable (compliance requirement). And by the way: I think this config backup use case makes up a good introduction to Git because it leaves concepts like pull/clone and branching besides – for now.