Production Data Copy Options on the Public Cloud

The Use Case

Often, a copy of Production data (i.e. in a database) is needed over in a non production environment (in the public cloud). This would entail moving data between a PRODUCTION VPC / VNET  and a non production VPC / VNET. This post assumes that the source data is in a production database (either on premises or on the cloud) and the target is also another database (again, eitehr cloud hosted or on premises database).

 

Some Scoping Questions

1. The first question to ask is the ‘recovery time’ or the replication time required to replicate the data.
2. The second question to ask is the security constraints around the movement of the data.

Based on these answers, common solutions for replicating production data including

1. Backup and Restore (of the actual filesystem to a cloud blob storage service)
2. Snapshots of Production Data (of the Server containing the database)
3. Using Shared Volumes (e.g. with a Netapp type of appliance).

Backup and Restore (using a blog storage service on the public cloud) 

Backups generate copies of your data files. This could take anywhere from minutes to days.

Snapshots – A Quicker form of backups

Snapshot SNAP to an instantaneous picture of your file system at a point in time. A snapshot is used typically, to restore the server back to the point in time.

Most standard databases (SQL Server, Oracle…) allow for manual snapshotting at any time.

Snapshots versus Backups

Backups can be stored in a different location from the original data whereas a snapshot, can only exist in the same location as the original data.

Snapshots by themselves are not backups. However, they are an essential part of the backup process (part of the data movement process to a backup file).

Using Shared Volumes (e.g. with a Netapp type of appliance).

The appliance would need to reside in it’s own subnet, which might be in the production VNET/VPC. However, the advantage of such an appliance is that it provides enhanced export options – i.e. additional security controls over which environment to export / copy volumes to.

The export time / recovery time would be in the minutes as opposed to hours.