Tired of scrolling through massive directories on a file server, flooded with config files? Easy to implement ‘backup’ solution at day one (IOS archive?), but often based on debatable, unreliable protocols like TFTP, combined with great challenges in tracking changes on day two.
So, this alternative approach takes Ansible to fetch running configs from network devices and stores them in a Git enabled directory for future use. Git brings build in versioning to the table, so there is only the latest config per device visable, plus the possibility to reliably track changes over the whole network.
It uses ansible ios modules to gather the configuration of Cisco IOS devices, but can of course be changed / complemented with other vendor modules. Success or failures for every device will be reported via syslog to an external receiver.
Install Git at your automation host.
$ sudo yum install git [...] $ git --version git version 22.214.171.124
Create a directory to store the configuration files and initialize a new git repository in this directory.
$ mkdir configs $ cd configs $ git init Initialized empty Git repository in /home/nwmichl/configs/.git/ $ cd ..
Ansible should be installed already, otherwise have a look at the Ansible Installation page. In most cases (CentOS/RHEL) a simple
$ sudo yum install epel-release $ sudo yum install ansible
gets the job done.
The complete playbook ios_cfgback.yml can be downloaded from github. You might need to install wget first or use curl with the same URL to copy/paste local on your host.
$ sudo yum install wget $ wget https://raw.githubusercontent.com/NWMichl/configbackup/master/ios_cfgback.yml
Ok, now let’s take this playbook apart and have a look at all these different tasks. No surprises with regard to the header, just remember to alter the vars section according to your needs:
--- - name: CONFIG BACKUP TO LOCAL DIRECTORY hosts: all connection: network_cli ignore_errors: yes gather_facts: no vars: config_path: /home/nwmichl/configs syslog_host: 192.168.1.12 timestamp_line_identifier: "! Last configuration"
Errors have to be ignored, because we don’t want the playbook to terminate halfway in the middle of the night – moreover, the ‘Git check’ task has to fail sometimes on purpose.
Task – GATHER FACTS VIA SSH
Uses the ios_facts module to gather all config facts, including the running config, which makes total sense because nobody has ever forgotten to copy run start after a change via CLI, right? The task is only executed, if the network_os equals ios. So if you herd a multi vendor environment, just add additional tasks with a corresponding conditional in terms of the ansible_network_os .
Task – SAVE CONFIG TO FILE
The full running config of each device can be accessed via the variable ansible_net_config and will be stored to a file named <hostname>.txt in the directory specified in config_path .
Tasks – SYSLOG MESSAGE
Depending on the value of ios_facts_result.failure and save_result.failure there will be a notice or error syslog message send to the receiver specified as syslog_host. This is a very important step, as you want to run this playbook unattended, maybe after working hours. So there has to be a way to get information about failures and syslog seems to be the obvious choice. But as you own your destiny aka playbook, you are also able to add email notifications or Slack messages for instance.
Task – REMOVE CONFIG LINES WITH TIMESTAMPS
This task sanitizes the config file from lines with timestamps like ‘Last configuration change …’. The goal is to version only actual changes to the config, not a random ‘copy run start’ by a human or a stupid config mgmt tool.
Task – GIT – ADD ALL (NEW) FILES AND CHECK IF WORKING DIRECTORY IS DIRTY
Ansible has no native git add and commit module so far, so we have to break out to the shell to add all files to staging, including new ones, and test if there has been any changes.
Task – GIT – COMMIT/PUSH ONLY IF WORKING DIRECTORY IS DIRTY
If there are any changes in the working directory, this task commits with a timestamp to the master branch. If you use a central repo like GitHub/Lab for all your config files, just uncomment the line ‘git push -f origin master’ to push your local repository.
Tasks – SYSLOG MESSAGES GIT
Even for the Git part of the playbook there are three tasks to generate helpful syslog messages – just in case.
Watch this asciinema recording of a playbook run, followed by a change to one device and another playbook run to show the different behavior.
To schedule a daily backup job (3:00 am) at your automation host, just generate a user crontab and enter the following line:
$ crontab -e 0 3 * * * ansible-playbook ios_cfgback.yml
If you’re not used to vim editor:
- i enters editing/insert mode
- <esc> leaves editing/insert mode
- : x <enter> saves the file and exits vim
The device credentials can be hard coded as clear text ansible_user / ansible_password in the /etc/ansible/hosts file, but shouldn’t – and certainly not in a production environment. Please use ssh keys or Ansible vault instead!
At this point everything’s ready for prime time. The files in the configs directory reflect the state of the last change based on the playbook run at 3:00am every day. You can use shell commands like git log -p or git blame <file> to follow changes in the local repository.
Using a GitHub repository
And here is how to bind your local git repository to a central web based one, like GitHub. An on-prem hosted instance of GitLab would be another way, especially for security sensitive environments.
You need an account on GitHub and create a new private repository. It makes sense to use the directory name of your config files, in my case: github.com/NWMichl/configs
Pushing a repository via https to GitHub usually needs to manually type in user credentials, which is a no go for automation purposes. The git credential.helper stores username and password of your GitHub account as clear text in a file located in the home directory instead, once typed in for the first time. You might want to switch to a ssh connection later and use public/private keys to push to the GitHub repo.
$ cd configs $ git remote add origin https://github.com/NWMichl/configs.git $ git config credential.helper store $ git push -f origin master Username for 'https://github.com': nwmichl Password for 'https://firstname.lastname@example.org': Everything up-to-date
We use a similar playbook with great success in production, including some enhancements like multi platform, email notification, summarization of all faults and a check if files are readable (compliance requirement). And by the way: I think this config backup use case makes up a good introduction to Git because it leaves concepts like pull/clone and branching besides – for now.