The spanner in the works
–Rory Nolan, Director of TADASoft–
Configuration errors in network device backups can create huge IT disaster recovery problems. By Rory Nolan.
Picture the scene; it is 17:00hrs on a weekday and you have been alerted that your firewall has crashed. Thankfully, you have a spare on-site and you manage to have that up and running quickly in factory default setting. How do you recover the config from the firewall that has just crashed? Who is actually responsible for taking regular configuration backups from the network devices? If backups are actually done, where are they stored, do you have access to retrieve them, how old are they?
Thankfully, Network automation has come a long way, embrace it. I bet your file servers are already covered and more often these days, bare metal desktop PCs can be recovered rapidly. Around the world, network managers are obsessed with ensuring that the data is always available and rightly so: downtime costs money and unnecessary personal grief which we can all do without.
When it comes to taking regular backups from the network devices (firewalls, routers, switches), unfortunately this is somewhat of an overlooked area. Humans normally play the part of the backup agent when it comes to backing up the network devices. Humans make errors and that’s the problem in a nutshell.
With all the networking qualifications in the world, one config line error on a network device can cripple the network. When a disaster happens, panic strikes. Humans actually tend to make more errors when under stress; which causes more downtime and more stress. Automated tools don’t put humans out of jobs; they actually help humans, letting them focus on other core tasks. Automated tools increase continuity and help your network stay compliant as well as increasing productivity.
Did you know that industry analysts report that 40 – 60 percent of network problems are due to improper configuration changes? That frightens me to be frank.
If you think you have this area covered by writing your own scripts, think again. The day of the guru in the corner writing scripts to look after certain tasks is over in my opinion. What organization nowadays can rely on one or two people with scripting knowledge to hold the network together? Could you imagine the network crashing and having to explain to the CTO and/or shareholders that the reason you lost x,y,z in downtime/money was simply because ‘the guru’ was on holiday and he/she was the only one who knew how to do the task!
It may sound like I am contradicting myself, but I actually compliment organizations when I hear they have scripts in place to automatically take the configurations files from network devices. Why? They recognise that taking backups is important and they have taken a punt at solving the problem. Humans make errors though and when the guru is on annual leave and someone needs to restore a Check Point firewall and Nokia box, who does it? The answer is the third party consultant @ £1,200 a day who if you’re lucky, arrives on-site the next day in an attempt to put back a half decent firewall rule set. It turns out the last known working backup was done one month ago but it is stored on a server which the guru uses for his projects. When you eventually find out how to get hold of the configuration file, it is out of date so you spend hours putting each rule set back in. What a waste of time, money and energy.
There are numerous companies on the market that offer solutions for network device backup. Some specialise in managing one vendor, take CiscoWorks for example, a Cisco only solution. Others have a solution that will backup & restore configs from multi-vendors, for example, Restorepoint from TADASoft. Check Point, Cisco, Juniper Networks, Brocade, HP Procurve, Blue Coat, f5 Networks, Barracuda, Alcatel, ConSentry, Bloxx, Nokia, Nortel, Riverbed, NetApp, Fortinet, Proofpoint, Watchguard, there all covered in one cost-effective secure appliance.
As it stands right now, networks are getting larger and far more complex. Let’s be honest, the day of the human making manual backups is gone. The day of the guru writing scripts to combat the global spam problem is well and truly gone. It is far more economical to have a dedicated solution in place to solve a dedicated problem. Don’t be naive and believe you can handle it all yourself, be smart and automate the network more often.
Network automation is nothing to be scared of. Without it life would be far more complicated and stressful and networks would cripple a lot more.
Network automation is your friend!