Topics Map > Platform X
Topics Map > User Guides

Platform X: Using Globus to share data with UW-Madison Study Team for the Neighborhoods Project

Using Globus to share data with UW-Madison Study Team for the Neighborhoods Project

Globus is a popular platform for accessing, transferring, and sharing research data, including data covered by HIPAA and other sensitive data. Neighborhoods study sites  will use the Globus tool to upload specified study data. Data uploaded with Globus will be stored in secure cloud storage in the Microsoft Azure platform. The UW-Madison Neighborhoods team will access and process the data after it has been uploaded to Globus.

 

Create a Globus Account

Follow these steps if you do not already have a Globus account. If you already have a Globus account, please send the name of the study site with which you work and the email address you used for your Globus account to platformx-support@mailplus.wisc.edu.

Check to see if your institution is integrated with Globus for single sign on. 

  1. Visit the Globus Log In page and check if your organization is in the organization drop down.
    1. Check for your organization
  2. If your organization is NOT integrated with Globus for single sign on, go to the 'Create an Account With a Globus ID' section.
  3. If your organization is integrated with Globus for single sign on, go to the 'Create an Account With your Organization ID' section.

Create an Account With a Globus ID

  1. Open the following link in your web browser https://www.globusid.org/create?viewlocale=en_US
  2. Complete the Create a Globus ID form, then click Create ID
    1. Create Globus account
  3. You should be brought to this page next page asking you to verify your email address. Additionally, you should receive a verification email from Globus containing a Verification code. Copy the verification code from the email into the ‘Verification Code’ field on the web page, then click the ‘Verify’ button. 
    1. Create Globus account
    2. Confirmation email
  4. Continue through the following screens until you are brought to the Globus Dashboard
    1. Globus File Manager
  5. Once your account is created, please send the the name of your organization and the email address you used to sign up and your Globus username (e.g. joecavanaughdemo@globusid.org) to platformx-support@mailplus.wisc.edu. This is required to finish set up of your account.

Create an Account With your Organization ID

  1. Open the following link in your web browser https://auth.globus.org/p/login?client_id=89ba3e72-768f-4ddb-952d-e0bb7305e2c7&scope=urn%3Aglobus%3Aauth%3Ascope%3Aauth.globus.org%3Aview_identities+urn%3Aglobus%3Aauth%3Ascope%3Anexus.api.globus.org%3Agroups+urn%3Aglobus%3Aauth%3Ascope%3Atransfer.api.globus.org%3Aall+urn%3Aglobus%3Aauth%3Ascope%3Asearch.api.globus.org%3Asearch&response_type=token&redirect_uri=%2Fv2%2Foauth2%2Fauthorize%3Fclient_id%3D89ba3e72-768f-4ddb-952d-e0bb7305e2c7%26scope%3Durn%253Aglobus%253Aauth%253Ascope%253Aauth.globus.org%253Aview_identities%2520urn%253Aglobus%253Aauth%253Ascope%253Anexus.api.globus.org%253Agroups%2520urn%253Aglobus%253Aauth%253Ascope%253Atransfer.api.globus.org%253Aall%2520urn%253Aglobus%253Aauth%253Ascope%253Asearch.api.globus.org%253Asearch%26response_type%3Dtoken%26redirect_uri%3Dhttps%253A%252F%252Fapp.globus.org%252Flogin%26redirect_name%3DGlobus%2520Web%2520App%26state%3Do67ulzhgiri%26viewlocale%3Den_US&redirect_name=Globus+Web+App&viewlocale=en_US
  2. Select your organization from the drop down, then click the 'Continue' button.
    1. Select organization
  3. Complete sign in with your organizational ID, then continue through the following screens until you are brought to the Globus Dashboard
    1. Globus Dashboard

Get Access to Site-Specific Guest Collection

  1. After you’ve created your account and sent the email address and Globus ID you used to sign up to platformx-support@mailplus.wisc.edu, the UW-Madison team will invite you to your site-specific guest collection. You will receive an email like the below screenshot. Click on the hyperlink after “Use this URL to access the share:”. This should open a page in your web browser.
    1. Globus email notification
    2. Alternatively, you can find your site-specific guest collection by going to the ‘File Manager’ tab in Globus, click in the ‘Collection’ search box, Click the Search and begin typing “px-neighborhoods”. You should see and be able to click on a collection with your Institution’s name (e.g. PX - Neighborhoods – Indiana University ), though you may see collections for other sites too.

Upload Data

There are different methods you can use to upload the data.

  • If your organization is already using Globus and has Globus Connect Server set up, please email platformx-support@mailplus.wisc.edu to discuss steps establish a connection.
  • If your organization does NOT have Globus Connect Server set up:
    • If the data is less than 10 GB, follow the below instructions for Uploading Data using the Globus Web portal.
    • If the data is 10 GB or larger, follow the below instructions for either Uploading Data using the Globus Connect Personal or Uploading Data using the  Globus Command Line Interface (CLI).

Uploading Data using the Globus Web portal

  1. Navigate to your site-specific guest collection.
    1. Site specific guest collection
  2. Click the Upload button on the right.
      1. If you are prompted for Authentication/Consent, click Continue. You need to grant access to Globus access to your collection. Click Allow. If you have already granted this access, you can skip this step.
        1. Upload data to Globus
        2. Upload data to Globus
            1. If you had to grant access and redirected to the File Manager, click the Upload link on the right again.
  3. Click Select Files to Upload. This will bring up a file browser. Navigate to the files you want to upload and click Open. This should upload the files to your collection. If successful, you should see a green checkbox next to the files in the Uploads pop-up in the bottom left and the files should appear in the middle of the screen.
    1. Upload
    2. Upload pop up

Uploading Data using the Globus Connect Personal

Globus Connect Personal allows you to setup a Globus endpoint on a laptop or workstation in order to upload or download data from Globus via GridFTP. 

  1. Follow the 'How to Install, Configure, and Uninstall Globus Connect Personal' steps in the below link that matches the operating system of your computer.
    1.  How to Install, Configure, and Uninstall Globus Connect Personal for Windows
    2. How to Install, Configure, and Uninstall Globus Connect Personal for Mac OS X
    3. How to Install, Configure, and Uninstall Globus Connect Personal for Linux
  2. Put the data you will upload in a folder that is accessible to the Globus Personal client
    1. Accessible folders
    2. Files for upload
  3. Open the File Manager tab in the Globus Web portal and find the GCP collection you created in the previous step. Then click the 'Transfer or Sync to...' link on the right side of the screen.
    1. GCP collection
  4. You should see an additional pane open on screen. Search for the appropriate guest collection in the right pane ('PX - Neighborhoods - Indiana University' in the below example), select the files you want to upload, then click the 'Start' button.
    1. Upload file
    2. You should see a popup letting you know the transfer has started.
      1. Transfer request submitted
  5. Navigate to the Activity tab on the left side of the screen to monitor the status of the upload
    1. Note: You must leave your computer on and connected to the internet for the upload to complete.
    2. Activity tab

Uploading Data using the Globus Command Line Interface (CLI)

These instructions are meant for a non-GUI interface. The example given is on Ubuntu Linux. The source endpoint is named playground-01 and the destination endpoint is Neighborhoods - Px Stg - Indiana University. When using these direction you will need to use a different endpoint and the appropriate destination endpoint for your study site.

Requirements

  1. Have a Globus Account and access to your Neighborhoods study site Globus collection.
  2. Install Globus Command Line Interface (CLI) via pip or pipx. Instructions can be found on the official Globus documentation website.
  3. Install Globus Personal Connect for your OS (Windows, Mac OS X, or Linux) following the instructions in the official Globus documentation website.

Helpful Documentation

Globus Command Line Interface (CLI) Reference

Example Workflow

  1. Login to Globus CLI

    CLI Login
  2. On the machine that has access to the data you want to send determine local end point ID and save it as a variable by issuing the following commands:

    ~$: globus endpoint local-id
    ~$: 0c11eff0-0000-0000-0000-000000000000
    ~$: export source_ep=0c11eff0-0000-0000-0000-000000000000
  3. Determine the destination endpoint ID. From the CLI do a endpoint search for the display name of the destination endpoint

    ~$ globus endpoint search "Neighborhoods - Px Stg - Indiana University"
    ~$ 5ab282d6-0000-0000-0000-0fc4eedd9695
    ~$ export dest_ep=5ab282d6-0000-0000-0000-0fc4eedd9695

  4. Transfer files

    ~$ globus transfer $source_ep:/path/to/data $dest_ep:/Request_living/imaging/ 

  5. By default you will get an email to your Globus registration email with details of the transfer. You can also check the status of your transfer from the Globus web interface “File Manager”. The files that you uploaded will show up in the YYYMMDD_file_transfer.log with success/failure messages.

Additional Support

If you have any issues or questions, please email platformx-support@mailplus.wisc.edu.



Keywords:Neighborhoods, Globus   Doc ID:119956
Owner:Joe C.Group:SMPH Research Informatics
Created:2022-07-27 17:06 CDTUpdated:2023-07-17 10:38 CDT
Sites:SMPH Research Informatics
Feedback:  0   0