Note:  This document uses specialized terminology.  See the Glossary of Terms for definitions.

• Overview of Research Object Storage (S3)
• Requesting an account
• 50TB no cost storage subsidy and overage risk
• Requesting support 
• Understanding your access key and secret
• Accessing & transferring data
• Working with collaborators
• Creating & configuring buckets
• Firewall rules
• Versioning / backup
  • Enabling/disabling versioning

Overview of Research Object Storage (S3)

Research Object Storage (S3) is an object file storage solution for Faculty PIs, permanent PIs, and their research group members.  It is hosted on premise by UW-Madison and is not a public cloud application, such as Amazon Web Services (although it is based on the AWS S3 protocol).  

It is useful for: 

• Data archiving
• Backup software target (Commvault, IBM Storage Protect, Synology NAS, Oracle, etc)
• Web applications - Lots of things know how to speak S3 natively
• Static web content
• Big data analysis
• Instrumentation/IoT data
• and more!

See Research Object Storage (S3) FAQ for more details about this service.

Requesting an account

If you DO have an existing ResearchDrive account:

If you already have an existing ResearchDrive account, you (or one of your designees) should complete the Research Object Storage (S3) Request Form.

If you DO NOT have an existing ResearchDrive account:

You must have a ResearchDrive account in order to request a Research Object Storage (S3) account.  ResearchDrive is a separate storage service that offers 25TB of no cost storage to eligible researchers.  If you do not have a ResearchDrive account, you may request one by filling out the ResearchDrive Request Form.  We will not be able to process Research Object Storage (S3) account requests for individuals without existing ResearchDrive accounts.

Once you receive the welcome email for your ResearchDrive account, you may request a Research Object Storage (S3) account by completing the Research Object Storage (S3) Request Form. 

50TB no cost storage subsidy and overage risk

Eligible researchers receive a 50TB no cost cost storage subsidy provided through campus support.  When your account was created, we provided you with a pre-made "default" bucket that has a 50TB storage cap applied to it, meaning you cannot upload more than 50TB (unless you request additional fully billed space).  This minimizes the risk that you will be billed for storage.

If you create additional buckets beyond the default bucket, those bucket will not have a storage cap set and you could exceed your 50TB no cost storage subsidy.  Any storage beyond the subsidy will be billed at the current rate.  To minimize the risk of exceeding your 50TB storage subsidy, please store all data within the default bucket.

To learn more about your default bucket, see Bucket Creation and Configuration:  Configuration settings for "default" bucket.

To learn about other risks and considerations associated with this service, see Research Object Storage (S3) - Warnings and important considerations.

To learn more about storage units, see ResearchDrive - Understanding Storage Units GB vs. GiB.

Requesting support 

Please see Research Object Storage (S3) - Requesting Support for details on support options.

Understanding your access key and secret

To access your account, you don't use your netid.  Instead, you will use an access key and secret.  Anyone with the access key and secret will have full access to your account so only share with trusted collaborators when absolutely necessary.  The access key and secret functions similarly to a username and password, but they are not tied to a person.  Access keys can be used with several access methods, but please note that there isn't a "website" where you login to view your account.

See Accessing & Transferring Data:  Access key and secret for more information about your access key.

Accessing & transferring data

See Accessing & Transferring Data for instructions.

Working with collaborators

In general, Research Object Storage (S3) isn’t the ideal solution for collaboration.  If you need to collaborate with others, we recommend using ResearchDrive. 

If you choose to share access to your Research Object Storage (S3) account with others, it can be done using a shared access key.  Anyone with your account's access key will have full access to your account. 

Another option for public sharing is by way of public web links via the "Everyone" permission option, which would grant anyone access to the files on your account.   

Note:  Unlike with ResearchDrive, you cannot add/remove collaborator access via Manifest / Active Directory because Research Object Storage (S3) cannot be integrated with these services.

Creating & configuring buckets

Research Object Storage (S3) utilizes buckets, which are “containers” for objects (i.e. files and folders) stored in your account.  Your data is stored in buckets, and you can have more than one bucket.  When your account was provisioned, a bucket with default configurations was created where you can store your data.  For most users, this default bucket will be all you need.  However, some users may need to create additional buckets and/or change settings or permissions for existing buckets.

See Bucket Creation & Configuration for more information.

Firewall rules

See Bucket Creation & Configuration: Access pools (i.e. Campus-only vs. web buckets) for more information about who can access each type of bucket.

Versioning / backup

Unlike with ResearchDrive, data stored on Research Object Storage (S3) is not automatically backed up daily and does not support snapshots. 

Instead, this service offers a feature called "versioning". A "version" is a copy of file that is generated every time the file is uploaded, edited, or deleted.  This allows you to recover accidentally deleted files, providing you have not deleted the whole bucket or the older versions of files themselves.  Older versions of files can be deleted by the user.  Versions are assigned a lifecycle, which refers to how long a previous version is kept before it is automatically deleted.  

Versioning is turned “on” for the default bucket we created for you and assigned a lifecycle of 30 days (this refers to how long the system will keep version files). 

Any additional buckets you create will NOT have versioning turned on by default.  You will need to enable versioning for any buckets you create.  The default version lifecycle for any bucket you create will be "forever", meaning they will never be deleted by the system.

Warning:  older versions of files consume storage space and count against your space usage.  Additionally, if you accidentally delete your bucket or the version files, you will not be able to recover a file version.

Enabling/disabling versioning

See Bucket Creation & Configuration:  Versioning/backup for more instructions.