Topics Map > Research Object Storage (S3)

Research Object Storage (S3): Accessing & Transferring Data

Table of Contents:

 

Access key and secret

Before you access your account, you will need to ensure that you have your access key and secret.  The access key and secret are similar to a username and password, but they are not tied to a person.  In fact, anyone with the access key and secret will have full access to your account.

The access key and secret were emailed to you via a OneTimeSecret link when you signed up for your account.  Once you click the OneTimeSecret link, it will reveal your access key and secret.  Record this information in a secure location such as LastPass. 

If you did not receive an access key or if you have lost your access key, please contact your departmental IT for assistance (or use another support request method).


I have my access key... now what?

Once you receive your access key, you may wonder "so... where do I use it?".  Research Object Storage (S3) doesn't have a "website" where you input the access key to "login" to your account.  Instead, you will use your access key to authenticate with one of the access methods listed below:

Access methods

You can access & transfer files to and from Research Object Storage (S3) a few different ways, including via:

Note: Research Object Storage (S3) cannot be mapped to a network drive on your computer.


Accessing and transferring via Cyberduck

What is Cyberduck and how do I get it?

Cyberduck is a free, open-source storage browser client for Mac and Windows.  It will allow you to see, download, and upload files to your account as well as create, delete, and configure buckets. 

You may download the client by visiting https://cyberduck.io/ and clicking the "Download" button in the upper right-hand corner of the webpage.  Next, select between the "Cyberduck for Windows" and "Cyberduck for macOS" options, double-click the downloaded .exe file, and follow the prompts for installing.

Accessing via Cyberduck

In order to access your buckets, you will need to setup a connection.

Setting up your connection for the first time

  1. If you are connecting to buckets in the Campus-only access pool, ensure that you are physically on campus or are connected to the campus VPN.

  2. Launch the Cyberduck client

  3. Select the "Open Connection" button

  4. In the new popup window labeled "Open Connection", open the first drop-down and select "Amazon S3".
    • Note:  the reason we are selecting Amazon S3 is because Research Object Storage (S3) is based on the AWS S3 protocol.  It is not, however, an AWS service.

  5. In the box labeled "Server", change the field to be one of the following:
    • Campus-only buckets:   "campus.s3.wisc.edu"
    • Web buckets:   "web.s3.wisc.edu"
    • Note:  for an overview of Campus-only and Web access pools, see "Access Pools (i.e. Bucket "Gateways")".

  6. In the Access Key ID box, type in your Access Key (you received this via the One-Time Secret email you received when you first got your account).

  7. In the Secret Access Key box, type in your Secret (you received this via the One-Time Secret email as well)

  8. Leave all other information the same and select the "Connect" button.

Image:  Setting up a connection

Accessing your account after the initial setup

Once you have setup the initial connection, you will want to bookmark you bucket so that you can access it again in the future without having to complete the connection setup steps again.

Bookmarking a bucket

  1. Select the bucket / object you wish to bookmark. (Note:  you can bookmark an entire bucket or an object within the bucket such as a file)

  2. Select the "Bookmark" button option in the upper menu.  Select "New Bookmark" from drop-down menu.

  3. In the new popup window, type a meaningful label for the bucket in the "Nickname" box.

  4. Input your secret in the box labeled "Secret Access Key"

  5. Click the "X" icon in the upper right-hand corner of the popup window.  (Note: the information will be saved automatically and there is no "save" button)

  6. To access the bookmark in the future, select it from the main content area or under the "Quick Connect" drop-down in the toolbar.

Image:  Setting up a connection

Accessing your default bucket

When your account was created, we provided you with a pre-made "default" bucket that had a set of default configurations applied to it.

To access this bucket via Cyberduck:

  1. Connect to the campus access pool (campus.s3.wisc.edu)
  2. Double-click the bucket with the following naming convention:  <netid>-bucket-01 (please substitute your own netid)

Note: For information about your default bucket, including configuration settings, see Bucket Creation & Configuration: Configuration settings for "default" bucket.

Creating new buckets

 See Bucket Creation & Configuration:  Creating a new bucket for instructions.

Creating folders within a bucket

You may create a file structure within a bucket to organize your files by project or other schema.

To create a folder: 

  1. Double click on the name of the bucket you wish to create a folder in.

  2. Right-click on a blank section of the file browsing pane and click "New Folder".

  3. Type the name of the folder you wish to create and click "Create".

Transferring files to/from a bucket

You may transfer data into your bucket/folder by dragging and dropping the files directly into the file browsing window or by uploading using the app controls.

  1. Navigate into the bucket and folder where you would like to transfer files into.
  2. Select the Upload icon in the upper menu.

  3. Select the files your wish to move into the folder using the resulting file browser window.

Deleting files and folders

Select the files and/or folders you wish to delete and hit the "delete" key on your keyboard or right-click and select "Delete".

Accessing and Transferring via Python / S3 API

See Research Object Storage (S3) - Python / S3 API Guide for instructions on connecting to and transferring data to/from your Research Object Storage (S3) via the AWS SDK for Python.  Note that while Research Object Storage (S3) is not an AWS service, it is based on the AWS S3 protocol and many (but not all) of the functions in the SDK work with this service.

Accessing via a shared HTTPS links

Files/folders within Research Object Storage (S3) buckets can be set to be accessible via a web link (i.e. an HTTPS link).  This means you can provide someone a URL and they can access the file/folder using a web browser.  To access via this method, the file/folder will need to be configured to be "public". 

See Bucket Creation & Configuration:  "Everyone" Permission (i.e. enabling access without an access key) for more information about making files and buckets accessible via weblinks.

Accessing and transferring FAQ

Can I access using my NetID?

No.  Research Object Storage (S3) does not support netid access. 

Can I map Research Object Storage (S3) to a network drive on my computer?

No, this service does not support mapped network drives.

Can I add collaborators via Manifest / Active Directory?

No.  Unlike with ResearchDrive, you cannot add/remove access via Manifest / Active Directory because Research Object Storage (S3) cannot be integrated with these services. 

How do I remove access?

To remove access to Research Object Storage, you must deactivate the access key being used to access the account.  To deactivate an access key, please contact researchdrive@wisc.edu.



Keywords:
"Research Object Storage (S3)" 
Doc ID:
134390
Owned by:
Casey S. in UW-Madison Research Data
Created:
2024-01-18
Updated:
2024-08-04
Sites:
UW-Madison Research Data