How digital heritage was migrated to an open and transparent Kubernetes platform

Scanned war diaries and historical photographs, videogames and digital art: digital heritage matters to a wide audience. Archives, museums, libraries, and other organizations that manage digital heritage collections collaborate within the Network for Digital Heritage (NDE) toward a shared goal: making history digitally accessible to everyone.

Containers and Kubernetes for easier migration 

Together with developers from CLARIAH, the research infrastructure for humanities, NDE began using containers and Kubernetes as early as 2020. A container is a layer around an application – or part of it – that ensures it can run anywhere. Kubernetes is a largely open-source platform that automatically deploys, manages, and scales large numbers of containers to the required capacity. 

This architecture creates a separation between applications and the underlying infrastructure. As a result, containers can be moved to another infrastructure provider, reducing the risk of vendor lock-in. The team developed the platform for NDE services on the Kubernetes infrastructure of an American provider, which was very advanced in supporting container technology at the time. 

More than 200 cultural heritage institutions in the Netherlands have now joined NDE’s services, ranging from the Municipal Museum Het Hannemahuis to the Maastricht University Library. Meijers explains: “An archive is a completely different type of institution than a library. Each has its own objectives and very different internal processes.” As a result, a highly fragmented application landscape had emerged. 

NDE aimed to standardise and therefore focused the collaboration on the publication layer that unlocks diverse source materials. In this way, source holders do not need to adapt their internal information processes, while heritage researchers can still easily find and connect the information. 

Migrating to a largely open-source environment at SURF 

NDE developed the Dataset Register, which functions as a guide or catalog showing where heritage datasets can be found. It also set up the Terminology Network, enabling searches across decentralised terminology sources. Meijers explains: “This allows us to connect information, which is exactly what we are working toward with the National Strategy for Digital Heritage.” 

Although container technology makes it easier to migrate applications, De Boer still encountered challenges. “SURF handled some things slightly differently—and often better. That’s why we had to make significant changes, including to our infrastructure code. But these were all improvements.” He greatly benefited from SURF’s support. “Even our collaboration with SURF was approached in an open-source way, using GitHub repositories, so others can also benefit from our experiences.”

Kubernetes infrastructure continues to evolve 

De Boer sees that SURF is actively working on new additions to its Kubernetes infrastructure. For example, he would welcome improved backup facilities and greater insight into resource usage, including related alerts. “SURF is already working on some of my wishes.” 

Meijers also sees opportunities to make greater use of SURF services for NDE, such as automating reliable access management through Identity & Access Management. An access broker – such as the one SURF is developing within the SSHOC-NL programme – can provide controlled access to collections that are only partially available due to regulations like GDPR or copyright. These kinds of additions help researchers find reliable, secure, and as complete as possible answers to their questions on the NDE platform. 

Tekst: Thijs Doorenbosch 
Foto’s: Sicco van Grieken