(perform_data_ingress_ase)= # Data ingress with Azure Storage Explorer Ingress with Azure Storage Explorer should preferably be done by the data provider. If this is not possible, the {ref}`system administrators ` can do so, provided they can have access to the if they have access to the data. Either way, it will be useful for TRESA members to [download](https://azure.microsoft.com/en-us/products/storage/storage-explorer/) ASE and sign in with their Azure account. If the TRESA already have access to the data, skip to {ref}`perform_data_ingress_tresa`. Otherwise do the following: 1. Go to the [Data Safe Haven System Manager instructions](https://data-safe-haven.readthedocs.io/en/v4.2.2/roles/system_manager/manage_data.html#data-ingress) for ingress (make sure you are reading the version of the docs appropriate to the deployed SRE) and follow the guide on how to generate a SAS URL 2. Send the SAS URL via the secure email service to the {ref}`role_project_team`'s chosen [DPR](https://data-safe-haven.readthedocs.io/en/v4.2.2/roles/data_provider_representative/index.html) (see the signed TRE Data Ingress Process Form) as per the Data Safe Haven System Manager instructions, but also include a link to the below {ref}`perform_data_ingress_dpr` instructions to upload the data via Azure Storage Explorer for the DPR to follow 3. You can check that ingress was successful by viewing the contents of the ingress container in ASE (or in the Azure Portal) (perform_data_ingress_dpr)= ## Perform data ingress as a Data Provider The Safe Haven is built on the Microsoft Azure cloud platform. The most convenient way of safely transferring data is to use [Azure Storage Explorer](https://azure.microsoft.com/en-us/features/storage-explorer/). You will not need log-in credentials, as the Turing trusted research team will provide a short-lived secure access token which will let you upload data. ### Prerequisites :::{important} - You must be able to receive a secure email. We recommend the [Egress secure email](https://www.egress.com/) service, which is free to setup for receiving secure emails. - You must know the public IP address(es) that are used by the people in your organisation who will be uploading the data. Talk to your IT team if you're not sure what these are. ::: The {ref}`system administrator ` will have requested your IP address(es) and once you have provided this they will respond by sending a {ref}`secure email ` back to you. This will contain the secure access token, which has **write** and **list** privileges, allowing the uploader to: - upload files - verify that files are fully uploaded - remove or overwrite outdated files :::{attention} The secure access token does **not** permit files to be downloaded. This provides additional protection in case the token is accidentally leaked. In the event that the token is leaked, inform a {ref}`system administrator ` who can revoke it. ::: :::{danger} Whilst the connection between your computers and our repository is one way – you can only send data, not retrieve it – if a malicious actor were to get hold of the link, they could poison your data. ::: ### Uploading your data 1. Open [Azure Storage Explorer](https://azure.microsoft.com/en-us/features/storage-explorer/) 2. Click the socket image on the left hand side 3. On `Select Resource`, choose `Blob container` :::{image} ../images/step3.png :alt: Azure Storage Explorer connection step 3 :align: center ::: 4. On `Select Connection Method`, choose `Shared access signature URL (SAS)` and hit `Next` :::{image} ../images/step4.png :alt: Azure Storage Explorer connection step 4 :align: center ::: 5. On `Enter Connection Info`: - Set the `Display name` to `ingress` (or choose an appropriate name) - Copy the SAS URL that the {ref}`system administrator ` sent you via secure email into the `Blob container SAS URL` box and hit `Next` :::{image} ../images/step5.png :alt: Azure Storage Explorer connection step 5 :align: center ::: 6. On the `Summary` page: - Ensure the permissions include `Write` & `List` (if not, you will be unable to upload data and should contact the administrator who sent you the token) - Hit `Connect` 7. On the left hand side, the connection should show up under `Local & Attached > Storage Accounts > (Attached Containers) > Blob Containers`->`ingress (SAS)` :::{image} ../images/step7.png :alt: Azure Storage Explorer connection step 7 :align: center ::: 8. You should now be able to upload data to the Safe Haven by clicking the `Upload` button, completing the ingress process 9. Alert the {ref}`system administrators ` if ingress was successful by emailing `trustedresearch@turing.ac.uk`, or ask for help if something went wrong ::::{note} Since you were not given read permissions, it's expected that you will receive the following warning when uploading a file. Click `Yes`. :::{image} ../images/warning.png :alt: Azure Storage Explorer warning :align: center ::: :::: (perform_data_ingress_tresa)= ## Perform data ingress (TRESA) In some cases (for example a Tier 0/1 SRE), the data provider or {ref}`role_project_team` may send the data directly to the {ref}`system administrators ` to peform the upload, in which case this can be done as follows: 1. Open [Azure Storage Explorer](https://azure.microsoft.com/en-us/features/storage-explorer/) 2. Click the person icon in the top left, then "Add an account..." :::{image} ../images/dt_step1.png :alt: Azure Storage Explorer deployment team connection step 1 :align: center ::: 3. Connect to a Subscription and click through (select "Azure") :::{image} ../images/dt_step2.png :alt: Azure Storage Explorer deployment team connection step 2 :align: center ::: 4. A browser page will be launched and you will be asked to choose your account. Choose the account that has access to the [Prod] Safe Haven Management V4 subscription (or the most recent SHM subscription at the Turing) :::{image} ../images/dt_step3.png :alt: Azure Storage Explorer deployment team connection step 3 :align: center ::: 5. After you are connected to the account. Click the list icon (above the person icon) in the top left. You will be shown a list of Subscriptions you have access to, which should include the Prod SHM subscription. Click the dropdown for this subscription :::{image} ../images/dt_step4.png :alt: Azure Storage Explorer deployment team connection step 4 :align: center ::: 6. Click the "Storage Accounts" dropdown, which should reveal the storage for all the SREs in the SHM. For the SRE you wish to carry out ingress, click the dropdown for `data` which should reveal the Blob Containers, including ingress. Click ingress :::{image} ../images/dt_step5.png :alt: Azure Storage Explorer deployment team connection step 5 :align: center ::: 7. Opening the ingress container will reveal the existing contents, and allow you to add new data with the Upload button :::{image} ../images/dt_step6.png :alt: Azure Storage Explorer deployment team connection step 6 :align: center :::