Python's ConfigParser and server checkouts

Welcome Page
Technical articles

Weekend Checkout Challenge
My client had created a script for their weekend checkout, but there was too much output for efficient comparison. After a while, the script was not run regularly. My solution was to use Python and its ConfigParser module to parse the script’s output and compare key data for differences. Over time, this script was leveraged to track additional settings.

Since existing scripts were leveraged, the data continued to be gathered in BASH. The existing BASH scripts were improved to create an additional output file and Python was used for comparisons.  An additional document is planned about how to gather the data using Python.

The Big Picture
Tracking changes to UNIX servers is significantly more difficult in large environments than it should be. Well-meaning systems administrators might change settings to match a standard, not realizing they are causing a Monday-morning production issue. Often patching and other changes are handled by UNIX administrators several steps removed from those supporting the applications. Critical software or OS components may not come back up after patching or reboots. Patching may permanently change settings. Waivers for certain settings may be granted for a small number of servers by the security team, but then changed anyway.

Unless the DevOps or application support team can track settings over time and quickly ascertain changes, both the discover and resolution of these issues will be greatly reduced.


ConfigParser
ConfigParser is designed for reading configuration files. Section headings are separated by headings within [ ]. The setting names are separated from the values with a : (colon).

In Python version 3.x, ConfigParser was renamed configparser. ConfigParser is a standard module and can simply be imported by one of the following commands (depending on the version of Python):
- import ConfigParser # Python version 2.x
- import configparser # Python version 2.x  

Preparing the Data
If you already have shell scripts that collect the necessary data, there is no need to convert them. You can add some echo/print statements to create an additional output file. 

ConfigParser's get method uses both the section headings and the setting names. This allows us to break up the input data file into sections that make most sense. For example, you may not want to gather data that will not be used in a simple comparison, like uptime. You can create a section called “[do not compare]” for this data. This is how you use ConfigParser's get method to read the data from the data file:
uptime = serverdata.get('do not compare', 'uptime')

Notice that in the get method, the first parameter is the text between the [ ] and the second parameter is the test to the left of the colon.

Here is an example of a file that ConfigParser can use:

[do not compare]
uptime :  5 days
date time run : 20180514_0633

[for comparison]
hostname : server1
address : 10.60.3.65
os : Linux
os version :  6.2


Note that on the left side of the colon, the text must all be lower case.

The following Python code can be used to read the data (assuming the filename is server1.before).

serverdata = configparser.ConfigParser()
serverdata.read("server1.before")

# data we are not comparing
uptime = serverdata.get('do not compare', 'uptime')
as_of = serverdata.get('do not compare', 'date time run')

# data we are comparing
hostname = serverdata.get('for comparison', 'hostname')
address = serverdata.get('for comparison', 'address')
os = serverdata.get('for comparison', 'os')

# print statements to test data:
print("The hostname is ", hostname)
print("date / time run is ", as_of)
print("IP address is ", address)
print("uptime is ", uptime)
print("Operating system is ", os)


Server Comparisons
To make the comparison work, a file must be generated before the work and one after.

A good way use Python to compare the data is to generate a series of lists, each list with two sets of information, the first before before the work was done and one for after the work was done. For example, operating system version information can be put into os_ver[] with os_ver[0] being the “before” information, and os_ver[1] being the “after” data. When os_ver[0] does not equal os_ver[1], you can make note of that change, send an email, put it in a web page, et cetera.

The Python script will loop through the two files for each server. Early in the script you can create the os_ver list:
over_ver = []

Then, when you loop through the input files you can append the data:
os_ver.append(serverdata.get('for comparison', 'os version'))

Some things being checked will have different urgencies. If some processes are not running, that may be urgent, but other processes are less critical. When you make your comparisons, you can respond differently. For example if os_ver[0] is not equal to os_ver[1], you might treat that as not urgent, while if certain processes are not running, you might want to send an alert.

Alternatively, you can create two lists:
after = [ ]
before = [ ]

This will allow you to use the append method in a more simple loop, however, it might be more difficult to maintain as more settings are being tracked by your script.

 

Suggestions for Future Learning

This information will be in the forthcoming eBook Up To Speed with Python. The ETA for the first edition is August 1, 2018.

ConfigParser documentation for Python 3:  https://docs.python.org/3/library/configparser.html


Welcome Page
Technical articles