[CALUG] School system seeks help on backup strategy

[Ray Lischner]

The Howard County Public School System's Technology Advisory Committee
will be meeting on Wednesday evening. The main topic of discussion will
be on HCPSS backup strategy, tools, etc. If anyone is able to read the
following brief and provide some helpful comments, no matter how brief,
I would greatly appreciate it. I will collect them and take them to the
TAC meeting. If anyone wants to attend the meeting in person, it will
be 7-9 PM at the HCPSS Board of Ed building. (Contact Ray Lischner,
lisch at tempest-sw.com if you want more information.) Thank you.

Technology Advisory Committee
Technology Brief – Backup Strategies
November 2007

I. Introduction

Data loss is inevitable.  We can all tell stories about our own
experiences with data loss; whether it is a file being lost because of
a hard drive failure, an electrical outage or even a natural disaster.
We can also tell stories of user error that hopefully make us chuckle,
not cry.

The HCPSS is not immune to any of these and other data-loss disasters. 
It currently has over 50 central office servers, over 70 school-based
administrative servers and almost 100 instructional servers.  The
amount of mission-critical data it stores—which includes student,
administrative, and financial data—is approximately 6 terabytes (TB). 

It is anticipated that the storage requirements for data will increase
by at least a factor of two in the next five years as HCPSS continues
to roll out additional servers and deploy new data- intensive
applications.  

II. Current Status

Protecting the integrity and availability of data is a top priority for
the HCPSS, but there are some challenges with our current strategy.

* The HCPSS currently lacks a unified backup system.  
  - Daily backups do occur for mission critical systems, but the
    methodology and tools are not standardized.
 * The tape-based storage methods used today are prone to media and
   mechanical errors and are slow and unreliable when recovering backups
   is a necessity.
 * Tape as a backup medium is cost prohibitive due to archiving
   requirements, on site storage, and off-site transport of media.  
 * The amount of backup data that can be sent to data center locations
   within the allotted backup window is hindered by the volume of data
   transmitted and WAN bandwidth limits. 
 * Verification and testing of existing backup processes is staff
   intensive.


III. Requirements

An ideal solution to protect and reproduce mission-critical data would
consist of a backup and recovery strategy that would:  
 * Centralize backups
 * Ensure the integrity and recoverability of daily backups
 * Consume small amounts of storage media and bandwidth so that
   incremental backups can take place 24/7
 * Automate backups to reduce reliance on staff-intensive manual
  processes 
 * Provide tools to monitor backup procedures
 * Establish a system that is scalable and focuses on simplified
   management and cost efficiency
 * Supports backups of Windows, Macintosh and Linux/UNIX-based data
   along with SQL and Oracle databases
 * Provide rapid and reliable recoveries, including single file restore
   capabilities
 * Offer an easy-to-use graphical user interface (GUI)
 * Includes the capability to replicate to multiple sites for disaster
   recovery options
 * Improve Recovery Time Objectives (RTO) and Recovery Point Objectives
   (RPO), with an initial focus on the most mission-critical file and
   application data

Keeping the requirements listed above in mind, what strategies and best
practices should HCPSS pursue to achieve process maturity and
excellence in data backup and recovery management?  

Questions to consider:

1.What is the appropriate technique/strategy to protect the data?
 * Disk-based?
   - Disk-to-Disk backup combined with data de-duplication?
   - Virtual Tape Libraries (VTL)?
 * Replication or snapshot?
 * Imaging solutions?
 * Continuous data protection software?
 * Backup services?

2.What are available tools for the technique/strategy? What is their
industry adoption/maturity?

3.How does the technique/strategy protect itself from data loss and
corruption (how “bullet proof” is it)?

3.How scalable is it to handle the increasing rate and number of changes
in mission-critical data?

4.How does it handle storage provisioning to automate change (moves,
adds, deletes) and to reclaim storage capacity?

5.How does it make backup and recovery easier and less costly in terms
of:
 * Storage capacity?
 * WAN bandwidth?
 * Staff hours managing tape/disk administration, monitoring and
   availability?
 * Hardware/software requirements?
 * Architecture and minimized complexity?


IV. Technology Advisory Committee’s Recommendations

There are a multitude of different strategies for handling backups.  The
HCPSS staff has been researching this issue and needs to determine if
we are on the correct course.  

Our TAC members come from a variety of different environments
(education, business, government, higher ed), and we would like to
capitalize on their experience to bring to our attention strategies and
tools that we may not have considered.

To accomplish this, a volunteer from the TAC will be asked to gather all
of the TAC members’ ideas, information and research and present the
results as a recommendation at the next TAC meeting (scheduled for
8:00am – 9:30am - December 12, 2007).
--
Ray Lischner