AzCopy#

If the DPR wishes to use AzCopy, work with them to come up with an appropriate ingress procedure, which might look something like:

  1. Using a secure out-of-band communication method (such as a secure email or messaging service), the data provider sends Trusted Research Environments Service Area (TRESA) a URL where the data can be downloaded.

    • This might be another cloud storage provider (eg. an AWS S3 bucket or Google Cloud Storage bucket), an SFTP server or a password-protected https server.

  2. The Trusted Research Environments Service Area (TRESA) use AzCopy to copy the data to the storage account.

Ingress from Google Cloud with AzCopy#

Note

This method is used by the EDoN data wrangling project

  1. Ask the DPR to create a service account key and send this, along with the <bucket name> and <directory name> via the secure egress email service to trustedresearch@turing.ac.uk (or your own email)

    • Note: you’ll need to request approval for your own account when attempting to access a secure egress email from the shared mailbox; you might want to inform the DPR of this

  2. Go to the Data Safe Haven System Manager instructions for ingress (make sure you are reading the version of the docs appropriate to the deployed SRE) and follow the guide on how to generate a SAS URL

  3. Complete the transfer with AZCopy:

    azcopy copy 'https://storage.cloud.google.com/<bucket name>/<directory name>' 'https://<SHM ID><SRE ID>data<hash>.blob.core.windows.net/ingress/<folder name><SAS URL>'

    Note

    It may be possible to do this without a SAS url, via authenticating your admin account with AzCopy.