Topics Map > Platform X
Topics Map > User Guides
Platform X: Using Globus to share data with UW-Madison Study Team for the Neighborhoods Project
Using Globus to share data with UW-Madison Study Team for the Neighborhoods Project
Globus is a popular platform for accessing, transferring, and sharing research data, including data covered by HIPAA and other sensitive data. Neighborhoods study sites will use the Globus tool to upload specified study data. Data uploaded with Globus will be stored in secure cloud storage in the Microsoft Azure platform. The UW-Madison Neighborhoods team will access and process the data after it has been uploaded to Globus.
Create a Globus Account
Check to see if your institution is integrated with Globus for single sign on.
- Visit the Globus Log In page and check if your organization is in the organization drop down.
- If your organization is NOT integrated with Globus for single sign on, go to the 'Create an Account With a Globus ID' section.
- If your organization is integrated with Globus for single sign on, go to the 'Create an Account With your Organization ID' section.
Create an Account With a Globus ID
- Open the following link in your web browser https://www.globusid.org/create?viewlocale=en_US
- Complete the Create a Globus ID form, then click Create ID
- You should be brought to this page next page asking you to verify your email address. Additionally, you should receive a verification email from Globus containing a Verification code. Copy the verification code from the email into the ‘Verification Code’ field on the web page, then click the ‘Verify’ button.
- Continue through the following screens until you are brought to the Globus Dashboard
- Once your account is created, please send the the name of your organization and the email address you used to sign up and your Globus username (e.g. joecavanaughdemo@globusid.org) to platformx-support@mailplus.wisc.edu. This is required to finish set up of your account.
Create an Account With your Organization ID
- Open the following link in your web browser https://auth.globus.org/p/login?client_id=89ba3e72-768f-4ddb-952d-e0bb7305e2c7&scope=urn%3Aglobus%3Aauth%3Ascope%3Aauth.globus.org%3Aview_identities+urn%3Aglobus%3Aauth%3Ascope%3Anexus.api.globus.org%3Agroups+urn%3Aglobus%3Aauth%3Ascope%3Atransfer.api.globus.org%3Aall+urn%3Aglobus%3Aauth%3Ascope%3Asearch.api.globus.org%3Asearch&response_type=token&redirect_uri=%2Fv2%2Foauth2%2Fauthorize%3Fclient_id%3D89ba3e72-768f-4ddb-952d-e0bb7305e2c7%26scope%3Durn%253Aglobus%253Aauth%253Ascope%253Aauth.globus.org%253Aview_identities%2520urn%253Aglobus%253Aauth%253Ascope%253Anexus.api.globus.org%253Agroups%2520urn%253Aglobus%253Aauth%253Ascope%253Atransfer.api.globus.org%253Aall%2520urn%253Aglobus%253Aauth%253Ascope%253Asearch.api.globus.org%253Asearch%26response_type%3Dtoken%26redirect_uri%3Dhttps%253A%252F%252Fapp.globus.org%252Flogin%26redirect_name%3DGlobus%2520Web%2520App%26state%3Do67ulzhgiri%26viewlocale%3Den_US&redirect_name=Globus+Web+App&viewlocale=en_US
- Select your organization from the drop down, then click the 'Continue' button.
- Complete sign in with your organizational ID, then continue through the following screens until you are brought to the Globus Dashboard
Get Access to Site-Specific Guest Collection
- After you’ve created your account and sent the email address and Globus ID you used to sign up to platformx-support@mailplus.wisc.edu, the UW-Madison team will invite you to your site-specific guest collection. You will receive an email like the below screenshot. Click on the hyperlink after “Use this URL to access the share:”. This should open a page in your web browser.
- Alternatively, you can find your site-specific guest collection by going to the ‘File Manager’ tab in Globus, click in the ‘Collection’ search box, Click the Search and begin typing “px-neighborhoods”. You should see and be able to click on a collection with your Institution’s name (e.g. PX - Neighborhoods – Indiana University ), though you may see collections for other sites too.
Upload Data
There are different methods you can use to upload the data.
- If your organization is already using Globus and has Globus Connect Server set up, please email platformx-support@mailplus.wisc.edu to discuss steps establish a connection.
- If your organization does NOT have Globus Connect Server set up:
- If the data is less than 10 GB, follow the below instructions for Uploading Data using the Globus Web portal.
- If the data is 10 GB or larger, follow the below instructions for either Uploading Data using the Globus Connect Personal or Uploading Data using the Globus Command Line Interface (CLI).
Uploading Data using the Globus Web portal
- Navigate to your site-specific guest collection.
- Click the Upload button on the right.
-
- If you are prompted for Authentication/Consent, click Continue. You need to grant access to Globus access to your collection. Click Allow. If you have already granted this access, you can skip this step.
-
- If you had to grant access and redirected to the File Manager, click the Upload link on the right again.
-
- If you are prompted for Authentication/Consent, click Continue. You need to grant access to Globus access to your collection. Click Allow. If you have already granted this access, you can skip this step.
-
- Click Select Files to Upload. This will bring up a file browser. Navigate to the files you want to upload and click Open. This should upload the files to your collection. If successful, you should see a green checkbox next to the files in the Uploads pop-up in the bottom left and the files should appear in the middle of the screen.
Uploading Data using the Globus Connect Personal
Globus Connect Personal allows you to setup a Globus endpoint on a laptop or workstation in order to upload or download data from Globus via GridFTP.
- Follow the 'How to Install, Configure, and Uninstall Globus Connect Personal' steps in the below link that matches the operating system of your computer.
- Put the data you will upload in a folder that is accessible to the Globus Personal client
- Open the File Manager tab in the Globus Web portal and find the GCP collection you created in the previous step. Then click the 'Transfer or Sync to...' link on the right side of the screen.
- You should see an additional pane open on screen. Search for the appropriate guest collection in the right pane ('PX - Neighborhoods - Indiana University' in the below example), select the files you want to upload, then click the 'Start' button.
- You should see a popup letting you know the transfer has started.
- Navigate to the Activity tab on the left side of the screen to monitor the status of the upload
- Note: You must leave your computer on and connected to the internet for the upload to complete.
Uploading Data using the Globus Command Line Interface (CLI)
These instructions are meant for a non-GUI interface. The example given is on Ubuntu Linux. The source endpoint is named playground-01 and the destination endpoint is Neighborhoods - Px Stg - Indiana University. When using these direction you will need to use a different endpoint and the appropriate destination endpoint for your study site.
Requirements
- Have a Globus Account and access to your Neighborhoods study site Globus collection.
- Install Globus Command Line Interface (CLI) via pip or pipx. Instructions can be found on the official Globus documentation website.
- Install Globus Personal Connect for your OS (Windows, Mac OS X, or Linux) following the instructions in the official Globus documentation website.
Helpful Documentation
Globus Command Line Interface (CLI) Reference
Example Workflow
- Login to Globus CLI
- On the machine that has access to the data you want to send determine local end point ID and save it as a variable by issuing the following commands:
~$: globus endpoint local-id
~$: 0c11eff0-0000-0000-0000-000000000000
~$: export source_ep=0c11eff0-0000-0000-0000-000000000000
- Determine the destination endpoint ID. From the CLI do a endpoint search for the display name of the destination endpoint
~$ globus endpoint search "Neighborhoods - Px Stg - Indiana University"
~$ 5ab282d6-0000-0000-0000-0fc4eedd9695
~$ export dest_ep=5ab282d6-0000-0000-0000-0fc4eedd9695
- Transfer files
~$ globus transfer $source_ep:/path/to/data $dest_ep:/Request_living/imaging/
- By default you will get an email to your Globus registration email with details of the transfer. You can also check the status of your transfer from the Globus web interface “File Manager”. The files that you uploaded will show up in the YYYMMDD_file_transfer.log with success/failure messages.
Additional Support
If you have any issues or questions, please email platformx-support@mailplus.wisc.edu.