Frustration and the general grinding of teeth from Backup administrators has been heard for many years since NDMP was first introduced. Performance issues, vendor lock-in and slow restore times are all common complaints when using NDMP. Organisations are having to deal with more unstructured data than ever before and with the added focus on cloud the pain of typical legacy approaches to backing up NAS are really being felt. Network Data Management Protocol (NDMP) has been the standard approach for protecting NAS data, but an increasing number of enterprises are exploring new approaches to protect their data as their environments grow and the challenges and limitations associated with NDMP are realised.
The History of NDMP
Version one of NDMP was invented and developed in 1995 by NAS pioneer NetApp and PDC who collaborated to solve an issue that was increasingly challenging NAS users and the inability to reliably protect their data. Prior to NDMP, NAS platforms such as NetApp Filers were being backed up by having backup servers mount NAS shares and then moving the backup data to locally attached tape devices or a networked tape library (Direct Backup). This method was rife with issues ranging from compromised performance based on the need to read every file over a specific interface (POIX), to bottlenecks being created by sending data over a single mount point, and managing multiple devices.
NDMP is a protocol that defines a common architecture for the way heterogeneous file servers on a network are backed up. The protocol was designed to allow the creation of a common agent used by the central back-up application to back up different file servers running different platforms and platform versions. With NDMP, network congestion is minimised because the data path and control path are separated. Back up can occur locally–from file servers direct to tape drives, while management can occur from a central location. NDMP is an open standard protocol promoted and supported by server vendors, back-up software vendors, and back-up device vendors.
Benefits of NDMP included
- Allowed Data Protection vendors to avoid writing specific agents for each NAS vendors’ solution. NAS vendors would then build integration that followed the NDMP standard
- Backup traffic is able to be offloaded to local or fibre channel-attached devices, addressing the bottlenecks from streaming data across slow networks
- Delivered the ability for centralised data management where a single instance of a backup solution could create and manage data protection for many NAS systems
Challenges with NAS Backup and NDMP
One of the undeniable facts about NAS is that today data Volumes have grown exponentially. In the past it was primarily about standalone storage servers however today there is petabyte level Scale-out NAS. Additionally you were unable to put agents onto a NAS system as they were sold as appliances with proprietary operating systems (such as ONTAP) – Files systems on a NAS system can grow very large and if backups are required to be restored, file systems could be required to be rebuilt which if they have grown very large this would take a long time.
NDMP did make things easier for a period and addressed some of the issues of traffic issues with Direct backup, however today many issues still remain. NDMP still does not cater for interoperability between NAS vendors so you are unable to relocate data to another file system, leaving you stuck with one particular NAS vendor. NDMP is primarily suited to file data however it does not work well with database applications. Furthermore it is restricted to a limited number of incremental backups before you are required to make a full backup. This is because NDMP was designed for use with tape and the limit with incremental is due to the fact they are a complex process.
Is there a better way? NetApp SnapDiff v3 and SnapMirror Cloud
Today, NetApp are helping customers address the legacy issues with NAS backups with the introduction of the latest version of SnapDiff, v3 (SnapDiff NetApp’s API). The new SnapDiff v3 backup architecture helps NetApp customers efficiently backup up NAS data both on premise and Cloud object storage. This new architecture is a highly efficient method for backing up NAS Devices. NetApp SnapShot and SnapMirror technologies have been utilised for many years by NetApp customers to provide an efficient method to create snapshot copies and transfer between ONTAP systems. SnapDiff v3 uses the same principle to determine the changed data between ONTAP Snapshot copies and then transfer to a backup target.
SnapDifff v3 take advantage of two key technologies in SnapDiff and SnapMirror Cloud. SnapDiff provides the list of changed files between two snapshot copies and SnapMirror Cloud which transfers Snapshot copies from the ONTAP system to a target such as an object storage target. This new architecture offers benefits such as Block-level incremental, Storage efficiency preservation, Forever incremental backup and granular file recovery to name a few.
The Answer to your NDMP challenges – ProLion RestoreManager and NetApp SnapDiff v3
NetApp are working with selected partners to integrate solutions for the new SnapDiff v3 backup architecture. As one of those chosen partners ProLion have introduced this integration into our RestoreManager solution. As a NetApp technical alliance partner ProLion have been producing innovative solutions for NetApp technology for many years. With well over five hundred organisations globally utilising ProLion solutions we pride ourselves in making a difference for our customers. ProLion have extensive experience with SnapMirror Cloud and we are the vendor of choice for customers wanting to drive innovation in the way they backup their ONTAP systems.
ProLion RestoreManager creates a central, online file index of every NetApp snapshot, giving you a single catalogue-based view into your files. You can search snapshots using multiple criteria and restore files and folders right from within RestoreManager with a single click. The integration of SnapDiff V3 into ProLion RestoreManager software will allow NetApp customers to keep a detailed file catalogue. The catalogue is created by utilising the NetApp SnapDiff technology to efficiently identify changes between snapshots, whilst conserve the efficiencies of incremental backups. NetApp users will be able to quickly restore to a granular level both from object storage in the Cloud and local backup targets.