Platform R: Setting up an AWS S3 Collection on Globus

Globus is a specialized managed transfer service designed to move massive (ranging from 10s of GBs to 10s of TBs) datasets directly between storage environments. Globus acts as a secure high-speed bridge that coordinates the direct movement of files from the Amazon S3 cloud to our local secured drives. Globus isn’t a storage space, but instead transfers data between collections (storage locations that can be accessed by Globus - similar to mapping a drive to your computer). Globus provides instructions for setting up Amazon Web Services S3 Connectors here.

This document is based on setting up our transfer of RNAseq data from Tempus to a University Restricted Drive. Because the Tempus AWS S3 bucket is highly restricted most of the Globus instructions linked above do not apply. These instructions assume that you have your account set up on Globus, the AWS credentials (access key and secret key) for the bucket you’re setting up, and the directory where your data is located in the AWS S3 bucket (if applicable).

1. To set up a Guest Collection, go to wisc-awsrestricted. First you will need to add your AWS credentials. Go to the Credentials tab and select the option to add a key and buckets.
   
   If you hover over the + where the red arrow is pointing, you’ll see Add a Key and Buckets. Click on the + to get the following popup:
   
   Fill in your AWS access key, secret key, and the name of the S3 bucket.
2. After setting up your credentials, go to the Collections tab and click on Add Guest Collection:
   
   After selecting Add Guest Collection the following screen will come up.
   
   Fill in the directory where your data is located on the AWS S3 bucket. This includes the bucket name.
   
   After entering the directory, click Browse; a green checkmark (see the red arrow in the image above) indicates the directory was found.
3. Select your new collection - you can see the overview, set permissions, etc.
   
4. Now that everything is set up, you should be able to work with this collection like the others. You should also be able to see it listed with the other collections you have access to. If you don’t see it when you click on Collections in the left-hand side-bar, also check Shareable By You.
   
5. For instructions on how to initiate the transfer, Globus has an excellent step-by-step guide.

Searching for Users When Setting up Permissions

When searching for a user to share access, the auto-fill can behave oddly due to how many addresses are in the list. Best practice is just to copy/paste the person’s email address. Ensure this is the email address their Globus account is set up on. But if you find yourself needing to search, here is an example here to prepare you.

1. After you have entered a few letters of the person’s name or email address, a list of options will come up (as expected, but usually not useful at first).
   
2. After a few more letters, you may end up with “No results found.”
   
3. If you keep typing it will eventually have enough to find who you are looking for.