Presentation Information
[cMFDCOEL-03]Making the analytics travel: Iliad application packages from local twins to EDITO
*Marco Oliveira1 (1. INESC TEC (Portugal))
Many ocean Digital Twin applications begin as standalone analytical models, research scripts or locally deployed workflows configured for a specific computing environment. These implementations produce valuable scientific results, although they are frequently difficult to discover, reproduce, transfer between infrastructures, integrate into operational systems, replicate, or deploy in other Digital Twin infrastructures
The ILIAD project addressed this challenge by extending the concept of the Application Package defined in the OGC Best Practice (BP) for Earth Observation Application Packages. That BP specifies how applications can be described, packaged with their dependencies, and deployed on exploitation platforms, using technologies such as the Common Workflow Language, containers, and the OGC API – Processes. Building on this foundation, ILIAD refined the packaging guidelines for ocean and Digital Twin workflows and developed a standards-based processing service layer supporting the full application lifecycle.
The resulting architecture combines portable workflow descriptions and containerised tools with registries and execution services. OGC API – Records supports the registration, discovery and retrieval of Application Packages, while Processes API supports their deployment and execution as interoperable processing services. The same packaged workflow can therefore be prepared for execution across personal computers, cloud systems, clusters and HPC environments.
ILIAD also explored interoperability with broader European Digital Twin infrastructures. A translation tool was developed to transform suitable CWL-based packages, facilitating their deployment as processes on platforms such as EDITO. Several ILIAD workflows were translated and deployed using this approach, enabling the use of the European infrastructure as the computing backbone for Local DTs, while also revealing practical constraints related to workflow complexity, storage models, and infrastructure-specific requirements.
The main contribution is a practical transition pathway from research code to an operational, reusable service. The presentation will discuss architectural decisions, implementation experiences and remaining challenges related to portability, reproducibility, security, infrastructure heterogeneity and operational sustainability.
Acknowledgements: This work has received funding from the European Union's Horizon Europe Research and Innovation Programme under the Grant Agreement No. 101136903, through the project INESCTEC.OCEAN - Centre of Excellence in Ocean Research and Engineering (HORIZON-WIDERA-2023-ACCESS-01 - Teaming for Excellence) and National Funds from the Portuguese Republic.
The ILIAD project addressed this challenge by extending the concept of the Application Package defined in the OGC Best Practice (BP) for Earth Observation Application Packages. That BP specifies how applications can be described, packaged with their dependencies, and deployed on exploitation platforms, using technologies such as the Common Workflow Language, containers, and the OGC API – Processes. Building on this foundation, ILIAD refined the packaging guidelines for ocean and Digital Twin workflows and developed a standards-based processing service layer supporting the full application lifecycle.
The resulting architecture combines portable workflow descriptions and containerised tools with registries and execution services. OGC API – Records supports the registration, discovery and retrieval of Application Packages, while Processes API supports their deployment and execution as interoperable processing services. The same packaged workflow can therefore be prepared for execution across personal computers, cloud systems, clusters and HPC environments.
ILIAD also explored interoperability with broader European Digital Twin infrastructures. A translation tool was developed to transform suitable CWL-based packages, facilitating their deployment as processes on platforms such as EDITO. Several ILIAD workflows were translated and deployed using this approach, enabling the use of the European infrastructure as the computing backbone for Local DTs, while also revealing practical constraints related to workflow complexity, storage models, and infrastructure-specific requirements.
The main contribution is a practical transition pathway from research code to an operational, reusable service. The presentation will discuss architectural decisions, implementation experiences and remaining challenges related to portability, reproducibility, security, infrastructure heterogeneity and operational sustainability.
Acknowledgements: This work has received funding from the European Union's Horizon Europe Research and Innovation Programme under the Grant Agreement No. 101136903, through the project INESCTEC.OCEAN - Centre of Excellence in Ocean Research and Engineering (HORIZON-WIDERA-2023-ACCESS-01 - Teaming for Excellence) and National Funds from the Portuguese Republic.
Comment
To browse or post comments, you must log in.Log in