Why Network Engineers Should Learn Programming

Because Microsoft Excel is not a text editor. Seriously.

This is a followup to the previous post, inspired by Ethan Banks of Packet Pushers. If you do operational networking at all, you deal with text files all the time: logs, debug output, configuration files, command line diagnostics, and more. I'm constantly amazed when I see people open Word or Excel to do their text editing, often one keystroke at a time.

The number one reason to learn basic programming is to automate that stuff. Personally, I use a combination of traditional Unix shell tools and Python to get the job done, but you could probably do it all with one or the other.

There are lots of other reasons to learn programming too, many of which will be discussed on an upcoming Packet Pushers episode. But if you don't believe any of those, this one alone makes it worth the effort.

Step away from the spreadsheet. Do it now.

Quick Thoughts on Learning Python

I was scheduled to be a guest on an upcoming episode of the Packet Pushers podcast, on the topic of Python for network engineers. Unfortunately due to bad luck I'm not going to be able to make the recording. Here are some quick thoughts on learning Python. If you're already an expert programmer you already know how to learn languages, so this post isn't for you.

Scenario 1: You've coded in another language, but you're not an expert.
I would start with the basic Python class at Google Code. It's targeted specifically at people who know basic programming skills in some other language. It was perfect for me; I went through the exercises and was able to quickly start writing simple, useful Python scripts.

Scenario 2: You don't know how to write code at all.
Start with the Udacity CS101 class if you like guided learning, or Learn Python the Hard Way if you prefer books. Be prepared to spend a lot of time on either. It's not easy the first time around.

After you've gotten through one of those two scenarios, do the following:

1. Spend time browsing the documentation for the Python Standard Library. Python is a large language, and chances are there's something in the standard library that will help you meet your goals. If you find yourself writing a lot of lines of code to accomplish something fairly simple, look harder. I recommend skimming the documentation for every module, then looking more carefully at the ones that interest you.
2. Matt Harrison's books on basic and intermediate Python are excellent. I recommend buying them and reading them.
3. Jeff Knupp's Writing Idiomatic Python is really good as you gain skills. It's a bit rough around the edges, but it will help you avoid common beginner mistakes and is well worth the read.
4. Practice a lot. I enjoy the math puzzles on Project Euler. These are not Python specific, but their structure makes them well suited to quick problem solving in any language. Beware -- it's very addictive!

Using Bro DNS Logging for Network Management

I was recently asked if someone in our desktop support group could get alerted when certain laptops connected to the corporate network. We have a lot of employees who work at industrial locations and rarely connect their machines to our internal networks, so the support group likes to take those rare opportunities to do management tasks that aren't otherwise automated.

The two mechanisms that came to mind for alerting on these events are DHCP address assignment, and DNS autoregistration. While we do send DHCP logs to a central archive, the process of alerting on a frequently changing list of hostnames would be somewhat cumbersome. I have been looking for ways to use Bro for network management tasks, so this seemed like a natural use case.

We already had Bro instances monitoring DNS traffic for two of our central DNS servers. I don't fully understand how Windows DNS autoregistration works, but from looking at the Bro logs, it appears that the DHCP server sends a DNS SOA query to the central DNS servers containing the hostname of the device to which it assigns a lease.

I wanted to learn how to use the input framework in Bro 2.2, so I wrote the following script and loaded it from local.bro:

https://gist.github.com/jayswan/8321141

This raises a Bro notice whenever one of the hostnames in the hostnames.txt file is seen in a DNS SOA query. I then set up local.bro and broctl to email this notice type to the correct person.

This works for now, but I'd love to hear from any more experienced Bro programmers about better ways to do it.

New Year Resolution: Code Cleanup

I enjoyed Ethan Banks' post on New Year's Thoughts: Start with Documentation, so I thought I'd write about what I'm doing this week: code cleanup. Over the last couple of years I've written a decent amount of code to automate mundane network management tasks. As quick one-off hacks have turned into things that I actually depend on, I've noticed a lot of ugliness that I want to fix.

Everything here is assuming Python as the language of choice:

• For scripts that send email, check to make sure the list of mail receivers is up-to-date.
• Look for those nasty embedded IP addresses and replace them with DNS names.
• Change from old-style open(FILE)/close(FILE) constructs to with open(FILE) as f constructs.
• Get rid of "pickles" for persistent structured data storage. Pickles are a form of native Python object serialization that are quick and convenient, but have a lot of potential problems. I've mostly used Python's native SQLite3 library to store data in simple persistent databases, but occasionally I just use plain text files.
• Look for repetitive code in the main script logic and try to move it into functions or classes where possible. For example, I had several scripts that were building email reports via clunky string concatenation, so I created a Report class that knows how to append to itself and do simple formatting.
• Remove unused module imports (that were usually there for debugging).
• Standardize module imports, declaration of constants, etc.
• Add more comments!
• Remove old code that was commented out for whatever reason.
• Look for places where I was creating huge lists in memory and try to figure out a way to reduce memory consumption with generators.

Things I haven't done, but probably should:

• Change to new-style string formatting.
• Write some tests, particularly for log parsers.
• Migrate from optparse to argparse for handling CLI flags.

Things I haven't done, and probably never will:

•  Migrate to Python 3.