Second call for research pilots on public cloud (AWS, Microsoft Azure)
The well-known top 3 cloud providers Amazon Web Services (AWS), Microsoft Azure and Google Cloud are some of the largest companies of the world. Together they have revenues which are more than half of the Dutch GDP. They have a vast number of resources in terms of hardware, man power and services. Furthermore, they offer many innovative services in a user-friendly manner. For example, AWS at the moment offers more than 200 services that can easily be combined to form complex architectures.
The grand scale at which the public clouds operate allows them to offer these services in a cost-efficient manner. Users can benefit from this as well. Their pay-as-you-go model can be used to reduce costs dramatically by cleverly selecting services that minimize running costs and switch of the services when not in use. As such, we see great potential in using these hyperscale capabilities for research purposes.
The challenges of using the public cloud are well known. One of these risks is that the companies behind it become too powerful and customers find themselves in a vendor lock-in. Having an exit strategy ready is, in our opinion, a prerequisite for going to the cloud. Security is another one of these challenges. Security in public clouds, like in any architecture, requires a focus directly from the design phase onwards.
Call for pilots
The public clouds provide a great opportunity for research as many of these new services and their speed of development can accelerate research. Also, we feel that the only way to cope with the afore mentioned challenges, is to start using public clouds. Therefore, we started a first public cloud call last year that resulted in 4 successful projects. As a result, we have decided to organize a second call for researchers who share our interest in the public cloud and the great opportunities it can bring.
This call is an open call for which projects can sign up throughout the year. The maximum number of projects depends on the available human and financial resources. We are most experienced developing solutions on AWS, but we are also interested in developing on Microsoft Azure as a lot of SURF research members use the latter. SURF will provide design and implementation support (for a maximum of 160 hours per selected project) and the public cloud resources are subsidized by NWO (fair use principle).
Data intensive projects
This call is meant for pilot projects. We will help to develop a Proof of Concept (PoC) for your research environment in the public cloud and advise you on how to continue after the funded pilot project. All viable projects should have a need for data intensive processing, possibly in combination with data analytics, data storage, data sharing or streaming data. Past and present examples of the SDA group are solutions to process the data of IoT-devices, traffic data, large amounts of Twitter data or a machine learning project, but we are open to explore other application areas as well. From your description of your research, it has to be clear that you have a data intensive project.
How to write a proposal
Do you like what you read? Write a proposal that describes your project and that meets our description:
- The project is a PoC in which we demonstrate a component or service of a public cloud that can be used by your research project. The proposal has to describe your research and the high-level requirements for the service you want to develop. Do not forget to describe the data-intensive component of your project and the application area at the research section.
- The project can take up to 3 months maximum. Give a rough planning in the proposal.
- Sufficient participation of the applicant: the applicant should have an idea and preferably has own resources with at least basic cloud skills to actively co-develop with us. The proposal should describe your available resources (number of individuals, available man power, skillsets and available time) and explain how you would like to collaborate with us.
- We help creating components or services, but will not maintain them after the project. If you have an idea, describe your vision on how to continue the components or service after the project.
- The total budget for public cloud resources is approximately 5K per project. To meet budget restraints, we may develop the PoC in a scaled-down version. Describe if you can your estimated need for compute and storage during the project such that it is possible to derive cost from it.
- Use of AWS or MS Azure public cloud standard-, managed - or serverless services. If you have an idea, describe the services you want to use.
- Projects will start as soon as it has been approved and resources are available.
- Preferably the components or services of the PoC results are reusable. Describe if applicable which parts of your research environment might be of interest to other researchers.
- You accept and will actively collaborate in the creation of a use case after completion of the pilot project which SURF may use to demonstrate value of public clouds among Dutch and international researchers.
Furthermore, the same criteria apply as for small NWO Computing Time on National Computer facilities applications. Researchers may submit an application if they are employed (i.e. hold a salaried position) at one of the following organizations:
- Universities established in the Kingdom of the Netherlands;
- University Medical Centers;
- NWO and KNAW institutes;
- Dutch universities of applied sciences;
- the Netherlands Cancer Institute;
- the Max Planck Institute for Psycholinguistics in Nijmegen;
- the DUBBLE beam line at the ESRF in Grenoble;
- the Princess Máxima Centre for paediatric oncology;
- NCB Naturalis;
- the institutes participating in the SURF Cooperative: KNMI, RIVM, TNO, the National Archives, the National Library, University of Humanistic Studies and the Police Academy.
Note: applications from researchers with a temporary position may need a signature as guarantee from a supervising staff member with a permanent contract. By signing the application, the supervising staff member declares that they are responsible for the awarded computing time after the expiration date of the project.
Procedure of selecting the projects
Projects will be approved if they meet the above described criteria and financial and human resources are available.
If you are interested, please send your idea to firstname.lastname@example.org
Expertise at SURF
The Scalable Data Analytics Group at SURF uses cloud-native solutions to process and analyze (streaming) big data for research and data science. The group has certified AWS solution architects and has expertise and experience with open-source tools such as Docker containers, Kubernetes (container orchestration), Apache Kafka and Spark. With these and other tools, we build platforms to process, store and/or analyze (streaming) data. These solutions allow for maximum portability and can either be deployed on SURF’s own private cloud or on any of the public clouds.