GCP backups – Some ways to transfer on premises data to Google Cloud

Three common ways to get data up onto a storage bucket in Google Cloud include:

1. 1. Using the Console (Storage Browser)
   2. Using gsUtil
   3. Using storage transfer service (needs agent installed locally)

Storage Browser – For simple, quick file (or folder) uploads to a GCP storage bucket.

gsUtil – Ideal for a large number of files to be uploaded (use gsUtil):

If you have a large number of files to upload you can use the gsutil -m option, to perform a parallel (multi-threaded/multi-processing) copy. To recursively copy subdirectories, use the -R flag of the cp command. For example, to copy files including subdirectories from a local directory named top-level-dir to a bucket, you can use:

Large SIZED single file

Large single file uploads are more susceptible to timeouts and failures.

To successfully transfer large files (> 1 tb), You can either use something called the Resumable Upload feature of GCP storage OR Google’s Storage Transfer Service

Storage Transfer Service

The cost is 4 cents per GB (approximately $40 per TB). A 10 B transfer would cost your $400.

CommVault and 3rd party\\\ tooling

CommVault will use an agentless method to perform backups from your on premises environment to a GCP storage bucket. It provisions a VM to serve as a VSA proxy (this manages all the backup and restore operations). This VM’s commvault software also comes with a web console to manage the entire transfer process.

 Licensing Mode for Commvault’s VSA based backup

Licensing is based on one or more of the following: CPU sockets, VM counts (VMs), Operating Instances (OIs), or Front-End TB Capacity (FETs).

For example, on-premises physical environments are best suited for OIs or FETs, whereas virtualized and public cloud environments are more suitable for VM counts.

Backup snapshots on Premises?

Always try and take on-premises snapshots using your backup tool (Veritas etc.). And store those snapshots both on premises and as a second backup – in cloud storage.

Summary

There’s 3 native GCP methods and one semi-native (using 3rd party tools) to manage backups to GCP.  Depending on workload sizes and the level of backup needed (backups of backup etc.), one of these should work for your backup needs.

Need assistance with your GCP or AWS migrations or security audits?