Object Store: store large quantities of data
Object Store provides a general purpose online storage service that may be used for storing large quantities of data. The service is ideal for storing various types of data, and has the ability to grow without restriction, while your data remains quickly and easily accessible.
What is object storage?
Our Object Store service is a distributed and highly scalable data storage architecture that manages data as objects, whereas traditional file system storage organizes data in a hierarchy of folders and files, whilst block storage manages data as blocks within sectors and tracks. Object stores have a very flat structure of containers and the objects they contain. The advantage of this is that object storage systems can scale up far-beyond the capabilities of file-based storage systems.
Scalability, durability and availability
Object Store provides an all-round online and highly-available cloud storage system that is characterized by its scalability, durability and availability. It is ideal for storing various types of data that can grow to very high volumes.
Object storage is particularly well suited for:
- Scale-out storage
- Storing large amounts of static data (many files)
- Media streaming – audio/video
- Files with the need for associated meta-data (for example for policy driven data management)
- Backup, archiving & data retention
- A national storage facility for research publication repositories – (figshare, Dataverse)
- Online data storage within an iRODS deployment.
Access protocols
Object Store is based on OpenStack SWIFT that, apart from its own native protocol, also provides the general S3 protocol. The S3 REST API was developed by Amazon for providing access to their cloud storage system. This opens up a wealth of possibilities and lots of tooling is available for this storage system type. Apache Spark and Elasticsearch are able to interact with S3 storage and there are also client-side tools such as Cyberduck and NextCloud, which provide graphic user interfaces for data handling and management, or Rclone, which is an rsync-like sync tool for cloud storage systems.
Access with your existing account
If you are already using other SURF services and have an account, you can access the Object Store service using the same account credentials. If you want to use this possibility, please contact our helpdesk.
It is also possible to archive your data with our Data Archive service and the Object Store offers a fast network connection directly to this archive. SURF does not make any automatic backups of data located in Object Store. Although data is stored multiple times which considerately minimizes the risk of loss, you are personally responsible for making backups of files.
Customized solutions
SURF can support implementations of customized solutions tailored to local institutions, such as hosting NextCloud which is a web-browser based interface offering a simple and easy to use work environment for research teams working collaboratively with external parties and with large amounts of data.
Rates
You can find the rates for this service in the SURF Services and Rates brochure (PDF)
Support and consultancy
As a user of SURF's Object Store you can always approach us for assistance. Our helpdesk can be reached by telephone or e-mail, or you can schedule a personal appointment. Please submit your questions or problems via the servicedesk portal, via helpdesk@surfsara.nl or by telephone on +31-208001400. The helpdesk is available during office hours (9:00-17:00).
If you are looking for specific advice about how to best deploy our Object Store for your research, please feel free to contact our consultancy service.