Nisarg Upadhyay
Azure blob storage home page

Use AzCopy to upload data to Azure Blob Storage

February 19, 2020 by

AzCopy is a command-line tool that is used to upload and download blobs/files from or to the Azure Blob Storage. In this article, I am going to explain how we can use it to create a new container on Azure blob storage and upload the data from the local machine to the Azure blob storage.

This command-line utility is not required to be installed on the workstation. You can download the appropriate executables from the below location.

The executable files are compressed in .zip (Windows) or .tar (Linux). You can download and unzip the executables. In this article, I am going to explain the following:

  1. Generate a shared access signature to connect to azure blob storage
  2. Create a container on the Azure blob storage
  3. Upload files on the container
  4. Upload the entire directory on the container
  5. Upload specific files on the container

Generate a shared access signature (SAS)

First, let us create a container on the Azure blob storage. To connect to the Azure blob storage, we must provide authorization credentials. To provide authorization credentials, you can use any of the following:

  1. Azure active directory
  2. Shared access signature token (SAS token)

In this article, I have used the shared access signature (SAS) token. To generate the SAS token, the first login to the Azure portal navigate to the storage account resource group Click on Shared access signature. See the following image:

Azure blob storage home page

On the Shared access signature page, click on “Generate SAS and connection string.” A connection string and SAS token will be generated. See the following image:

Generate a shared access signature (SAS) token

Now, to run the AzCopy command, we will use the SAS token. The SAS token will be appended at the end of the azure blob container URL (Storage account connection string). Following is the example of the blob storage URL appended with the SAS token:

“<storage_account_name>.<blob>.core.windows.net/<container_name>?<SAS token>”

Create an Azure blob container

Using AzCopy command, I can create a BLOB container in the azure storage account. Following is the syntax:

Example:

For example, I want to create a container named “MyFirstBLOBContainer.” To do that, execute the following command:

The screenshot of the command:

Container has been created on Azure blob storage

You can verify that the new container has been created in the storage account. On the Azure portal, navigate to the Azure blob storage account Open containers.

Container can be viewed on Azure blob storage

We can see in the above image, a new container named “MyFirstBLOBContainer” has been created.

Upload a file on the container

Using AzCopy, I can upload the files from the source to the destination. It supports the following directions:

  1. local machine<-> Azure Blob (SAS or OAuth authentication)
  2. local machine <-> Azure Files (Share/directory SAS authentication)
  3. Azure Blob (SAS or public) -> Azure Blob (SAS or OAuth authentication)
  4. Azure Blob (SAS or public) -> Azure Files (SAS)
  5. Azure Files (SAS) -> Azure Files (SAS)

You can read more about the copy command here.

The syntax to upload a file is following:

Example:

Once the container has been created, let us upload the data to the container. I have created a CSV file named “CountryRegion.csv” in the directory “C:\CSVFiles“. I want to upload it to the BLOB container “MyFirstBLOBContainer.” As I mentioned, for authorization, we are going to use SAS. Execute the following command to upload the file:

The screenshot of the command:

The file has been uploded

To view the uploaded file on the Azure portal, navigate to the Azure blob storage account Open containers open “MyFirstBLOBContainer.”

View a file in a container

As you can see in the above image, the file has been uploaded.

Upload the directory on the container

Using AzCopy command, we can upload the directory and all the files within the directory to the Azure blob storage container. The command creates a directory with the same name on the container and uploads the files.

Following is the syntax:

Instead of uploading the entire directory, if you want to upload the content of the directory, then you can specify a wildcard symbol in the command(*).

Following is the syntax:

Example:

Suppose, I want to upload a “CSVFiles” directory to the azure blob container “ MyFirstBLOBContainer.” To do that, execute the following command.

The screenshot of the command:

The directory has been uploded

A new directory named “CSVFiles” will be created on “MyFirstBLOBContainer.” You can view the files login to the Azure portal open storage account Open containers open MyFirstBLOBContainer.

Viewing the directory on the Azure blob storage container

Now, to upload all the files within the “Adventureworks2014-install-files” directory to the “Adventureworks2014” directory on the storage container, execute the following command.

The screenshot of the command.

All files within the directory have been uploaded

To view the files, login to the Azure portal open storage account open container open Adventureworks2014 directory.

View files in Azure blob storage container

Upload specific files on the container

Using AzCopy command, we can upload specific files to the blob storage container. To do that, we must use – -include-path option and separate individual files using a semi-colon(;). Following is the syntax:

Example:

For example, I want to upload three CSV files named “ProductCategory.csv“, “Shift.csv“,” ShoppingCartItem.csv” to the “MyFirstBLOBContainer.” To do that, execute the following command.

The screenshot of the command:

Specific files have been uploaded.

To view these files, login to the Azure portal open the Azure blob storage account Open containers open MyFirstBLOBContainer.

View files in Azure blob storage container

Summary

In this article, I have explained the AzCopy command and how it can be used to create new containers and upload files on the Azure Blob Storage containers.

Nisarg Upadhyay
Latest posts by Nisarg Upadhyay (see all)
SQL Azure

About Nisarg Upadhyay

Nisarg Upadhyay is a SQL Server Database Administrator and Microsoft certified professional who has more than 8 years of experience with SQL Server administration and 2 years with Oracle 10g database administration. He has expertise in database design, performance tuning, backup and recovery, HA and DR setup, database migrations and upgrades. He has completed the B.Tech from Ganpat University. He can be reached on nisargupadhyay87@outlook.com

10,006 Views