Friday, May 21, 2021

Migrate Data from a Cosmos DB Azure Table API

If you need to migrate data from or into Azure Cosmos DB you can use Microsoft’s data migration tool to do this. The tool is versatile but the documentation isn't providing all the answers. Specifically, in a scenario when you need to migrate data from Cosmos DB instance configured as an Azure Table API to a JSON file, you can use the tool, but the settings you need to provide are not obvious. Here are the settings which worked for me for its two main tabs: Source Information and Target Information (you would see them if you run the tool dtui.exe):

Source Information

  1. Prepare data for assembling a connection string:

    1. Grab the value of Azure Table Endpoint from Overview page, for example: https://name-of-your-cosmos-db-account.table.cosmos.azure.com:443/

    2. Modify this URL, replacing table.cosmos.azure.com with documents.azure.com

    3. Grab the value of PRIMARY KEY from Connection String page

    4. Grab the name of the root node on the Data Explorer page, this will be your database name.

2. Assemble the connection string as follows:

AccountEndpoint=https://name-of-your-cosmos-db-account.documents.azure.com:443/;AccountKey=primary-key-goes-here;Database=Your-DB-Name

3. Expand the root node (Your-DB-Name) and take a note of a table you want to export, for example My-Table-Name

4. Fill in the form on the Source Information tab:

Import from: Azure Cosmos DB

Connection String: use connection string created in step 2. Click Verify button, it should work.

Collection: My-Table-Name

Other fields: leave them to defaults, or you can optionally specify a query to limit the export

5. Click Next button to configure Target Information

Target Information

  1. Export To: JSON file

  2. Choose Local File radio button option, specify path, optionally select Prettify JSON.

  3. Click Next to complete the wizard and run through the export.

The JSON file should be saved in the directory you specified, or if you didn’t - in the folder you have started the data management tool from.

UPDATE: importing from JSON into Azure Table storage in Cosmos DB also works. Same manipulations with connection string as described above for exporting scenario are needed. In addition, depending on your situation you may want to fill out extra parameters describing whether to regenerate Ids or not, etc. The tutorial on using the data migration tool covers these well.