Data Processing & Data Management Technologies within the Lab

Endless amounts of data are available nowadays. But how can you process, analyse and (re)use them safely and securely? The projects in this Labs theme explore these aspects.

Labs thema data technologies


Everyone is using their own service to synchronise and share files. This is no different in research. But researchers also want to share files with colleagues who use another sync-and-share service. In the European project CS3MESH4EOSC we link a large number of these services.


In recent years various organisations worldwide have developed on-premise sync-and-share services: services that allow you to store, share, and synchronise files online. SURF developed SURFdrive and SURF Research Drive. A community has since arisen around these services: Cloud Services for Synchronisation and Sharing (CS3). A number of organisations in CS3, including SURF, have decided to jointly design and develop their services further and to build an e-infrastructure (mesh) of sync-and-share services. This e-infrastructure will be part of the European Open Science Cloud (EOSC). Hence the name CS3MESH4EOSC.

Why do we run this project?

Sharing files with people outside your organisation

Sync-and-share services nowadays allow you to do much more than just synchronise and share files. They are often integrated with other services such as environments for Jupyter notebooks, services for publishing and archiving data, collaborative editing, and so on. For example, from your sync-and-share environment you can not only edit your data directly with Jupyter notebooks, but also share those notebooks.

All those features have so far been available to users within one sync-and-share service. But both education and research do not stop at the boundaries of the organisation. Users also want to be able to share these services and data with people outside the organisation who use their own sync-and-share service.

Connecting sync-and-share services

In CS3MESH4EOSC we therefore want to integrate sync-and-share services and link them together in a federated infrastructure, with the aim that:

  • users of different sync-and-share services can share data and applications
  • sync-and-share services are linked to services that make data FAIR (Findable, Accessible, Interoperable, Reusable)
  • enable federated data analysis and distribution of data. 
    Federated means that a trust relationship is used: on the basis of a trust relationship between sync-and-share services, authenticated users of one service can access data within the other service.

Initially, the following sync-and-share services will be involved: SURFdrive, Research Drive, CERNBox, PSNCBox, CloudSTOR, Sciebo, owncloud@CESNET, SWITCHdrive and ScienceData.

What are the main activities?

  • We develop software for linking sync-and-share services with other services, and for linking sync-and-share services with each other via the Open Cloud Mesh standard. The vendors Owncloud, Nextcloud and Seafile support this standard.
  • We are creating a federated infrastructure. SURF is responsible for this. This includes support, quality of service, monitoring, security, governance structure, and operational procedures.
  • We are working on the further development of applications. This involves Jupyter notebooks, collaborative editing and integration with archiving services and on-demand data transfers. SURF is responsible for the latter. On-demand data transfers enable users to send data from their own sync-and-share service to other sync-and-share systems or storage infrastructures. This is particularly relevant for the more demanding applications of sync-and-share.

Who do we collaborate with?

In this project we are collaborating with:

  • CERN
  • PSNC
  • Trust-IT
  • Joint Research Centre
  • Cubbit
  • Westfälische Wilhelms Universität
  • Ailleron
  • Danmark Tekniske Universitet