Migrate data to cloud with AzCopy
What is AzCopy?
- AzCopy is a command-line tool from Microsoft designed to quickly and efficiently transfer data to and from Azure Blob Storage, Azure Files, and Azure Table Storage. It’s widely used in scenarios where performance, automation, and scalability are required for large data transfers.
Scenario:
A company is moving
its website’s media files (images, videos, backups) from on-premises servers to
Azure Blob Storage. With AzCopy, you can quickly upload terabytes of data while
optimizing for bandwidth and transfer performance.
Task 1:
Sign in to Azure Portal
Go to the
Azure portal by using URL https://portal.azure.com.
Task 2:
Create a Storage account
In the search
box at the top of the Azure portal, enter storage account. Select Storage
accounts from the search results.
On the Basic tab of the Create a Storage account
page, enter or select the following values:
- Resource group : Select rg_eastus
- Instance details :
- Storage account name : Enter a globally unique name
example: whizstorageak
- Region : Select East US
- Performance: Select Standard
- Redundancy : Select Geo-redundant storage (GRS)
Now, go to the Advanced tab, under Hierarchical
Namespace, Enable Hierarchical namespace to Enabled.
Click on Review + Create and then click on Create.
After a few minutes your deployment will be completed.
Now, click on Go to resource. On the left side panel,
select Containers and click on + Container.
On the New container tab, enter the following details
and click on Create:
Task 3:
Download AzCopy and upload your data to Blob storage
In this task firstly we will install Azcopy in our system
after which we will upload sample files to our storage account with the help of
it. Once the file will be uploaded we will be able to access the uploaded files
directly through our container on our Azure portal.
- Let us
download AzCopy on our system from the below link:
https://aka.ms/downloadazcopy-v10-windows
2.
Once it is downloaded, you will
see it is a zip file. Extract the folder. You will see
the AzCopy Application.
3.
Now, Click on the address bar
and copy the path.
4.
Open the Start Menu, search
for Command
Prompt and Select
it.
5. Navigate to the azcopy folder using the following command cd <path-you-copied>
6.
Now, enter the following
command to authenticate and login to use AzCopy.
7.
Now, go to your web browser and
open the link given in the output and then enter the given code when prompted.
8.
Now, go to your web browser and
open the link given in the output and then enter the given code when prompted.
9. Now, when you go back to the command prompt you will see Login
succeeded.
10.
Now, enter the following
command to upload all files in a folder to Blob storage using AzCopy. Make sure
to replace your local folder path in which your data is present, storage
account name and container name.
azcopy copy "<local-folder-path>" "https://<storage-account
name>.blob.core.windows.net/<container-name>" --recursive=true
11.
Go to your Azure portal, search for
your storage account and select your container. You will notice that all your
files from your local folder have been uploaded to your blob storage.
Task 4:
Modify the data for test purposes
In this task, we will upload a new
file in our source directory with the help of azcopy.
1. You can use AzCopy to upload files based on their last-modified
time.To try this, you need to modify or create new files in your source
directory for test purposes. Then, use the AzCopy sync command.
Note : Make sure that you upload a
new file in your local folder before running this command to see the correct
result.
azcopy sync "<local-folder-path>" "https://<storage-account-name>.blob.core.windows.net/<container-name>" --recursive=true
2.
Now, go to your portal and in your container you will see the
newly added file.
Task 5: Create a scheduled
task
1.
You can create a scheduled task that runs an AzCopy command
script. The script identifies and uploads new on-premises data to cloud storage
at a specific time interval.
2.
Open Notepad or any text editor
on your computer and enter the following contents in it. Make sure to replace
the placeholders with your folder path, storage account name and container
name. Save your file as script.bat.
3.
Now, on the command prompt, run the following command to create
a scheduled task:
schtasks /CREATE /SC minute /MO 5 /TN "AzCopy Script" /TR
C:\sync-script.bat
4.
To validate that the scheduled task runs correctly, create new
files in your local folder. Run the following command:
azcopy
sync "C:\Users\Amit.Kumar\Downloads\azwindowsvm"
"https://whizstorageak.blob.core.windows.net/democontainer"
--recursive=true
