From June 11th to 12th 2025, the European Synchrotron Radiation Facility (ESRF) in Grenoble hosted the OSTrails Hackathon: Building Assessment‑IF & SKG‑IF Solutions Together, a two-day collaborative event that brought together OSTrails’ tool developers, domain experts, and open science advocates, as well as experts from GraspOS project. The aim was to co-develop and refine the core components of the OSTrails Interoperability Reference Architecture, focusing on the Assessment Interoperability Framework (Assessment‑IF, previously FAIR‑IF) and the Scientific Knowledge Graph Interoperability Framework (SKG‑IF).

Key takeaways

• A common REST API specification for FAIR assessment operations was refined
• Benchmarks were defined in terms of authorship, scope, and execution
• The distinction between Benchmarks and Algorithms was formalised
• Metadata needed for describing a Benchmark and Algorithm (basic) has been discussed
• A mock DMP Evaluation walk-through showed how even staged scenarios align with Assessment‑IF
• Developers contributed real-world dataset examples and tested the SKG‑IF API
• The SKG-IF extension process was refined, and documentation improvements were proposed
• Collaboration with GraspOS and Athena RC ensured alignment with community needs and RDA SKG‑IF WG

Sharpening the Assessment‑IF

For the Assessment‑IF, the hackathon centred on two of its essential elements: API and Benchmarks. 

API: the meeting described a first version of the Assessment-IF REST API, refining the operations for interchanging metadata and results that result from FAIR assessment tools. 

Benchmarks: These represent community expectations for how digital objects should behave and serve as the foundation for assessing conformance and quality. Until now, Benchmarks were acknowledged as necessary, but their practical definition, authorship, and implementation had not been formalised. This event addressed that by:

• Defining how Benchmark are authored and tied to user or community expectations

• Clarifying how Benchmarks are executed and how user feedback is produced

• Distinguishing Benchmarks from the Scoring Algorithms that implement them

A key example was the mock-up of the DMP Evaluation Service, showcasing how real-world scenarios for evaluating machine-actionable Data Management Plans (maDMPs) fully align with the structure and semantics of the Assessment‑IF and are compliant with the DMP Common Standard (DCS). Specifically, the example demonstrated how the evaluation can be applied across different stages of a DMP’s lifecycle, confirming the framework’s robustness and flexibility.

Advancing the SKG‑IF

The SKG‑IF provides a shared framework for exchanging structured metadata across a wide array of research entities, including publications, datasets, software, organisations, and more. Sessions focused on advancing the SKG-IF through standardisation of a common API and refinement of the model’s extension mechanism. These sessions involved close collaboration with the GraspOS project and colleagues from the Athena Research Center, reinforcing OSTrails' alignment with current implementations and the RDA SKG‑IF Working Group. With the first public release of the SKG‑IF API specification scheduled for late 2025, this hackathon served as a critical milestone in its development.

Key outcomes included:

• Real-world testing of the SKG‑IF API in pilot implementations.

• Contributions of community-driven dataset examples to the official SKG‑IF GitHub repository.

• Review and refinement of the SKG‑IF extension process, leading to planned improvements in documentation and onboarding materials.

• Strengthened collaboration with GraspOS and Athena RC to ensure SKG‑IF remains aligned with evolving community and infrastructure needs

Importance for OSTrails 
The hackathon was a key checkpoint for validating and refining core elements of the OSTrails Interoperability Reference Architecture. For Assessment‑IF, it confirmed that the architecture is generic and flexible enough to support multiple assessment across different domains. Clarifying the roles of Benchmarks and Algorithms also improved internal coherence and communication across teams.

On the SKG‑IF side, the event enabled early pilot testing of the API specification and strengthened the approach for community-driven extensions. Collaboration with GraspOS and Athena RC helped ensure the work remains grounded in real-world needs and existing international efforts.

Why it matters for the broader Open Science community 
Across the research landscape, tools for FAIR assessment and metadata exchange often lack common structures, terms, or protocols. This fragmentation limits interoperability and reduces transparency.

The harmonisation work advanced during this hackathon, including shared terminology, standardised APIs, and modular design, lays the foundation for:

• Greater compatibility between assessment tools and their results

• Seamless metadata integration across infrastructures

• Stronger alignment between technical systems and researcher needs

Together, these efforts support a more open, interoperable, and scalable research ecosystem.

Impressions 
The in-person format created space for rapid iteration, immediate feedback, and deep technical exchange. From architecture discussions to hands-on testing, the setting proved highly productive.

“Sometimes a few hours of discussion are more important than a few days of coding.”

-Renaud Duyme, ESRF

Looking ahead 
The hackathons solidified core design decisions for both Assessment‑IF and SKG‑IF. For Assessment-IF, the API now provides a consistent way to discover and run assessments, and the benchmark framework has been formally defined, with pilot data to be added before implementation across use cases. For SKG-IF, the API has been tested in real-world scenarios, and its extension mechanism is progressing toward wider adoption.

At Open Repositories 2025, held from June 15th to 18th in Chicago, Illinois, Maximilian Moser (TU Wien) presented OSTrails and its ongoing work to make FAIR and reproducible science more interoperable across research services. Under the conference theme “Twenty Years of Progress, a Future of Possibilities”, the OSTrails poster highlighted the project's efforts to connect isolated systems into cohesive workflows that reduce researcher workload and support long-term sustainability.

“OSTrails is making it easier for different research services to work together by providing standardised APIs. This reduces manual effort for researchers and helps repository teams build more sustainable, vendor-neutral infrastructures.”

- Maximilian Moser, TU Wien, at OR2025

The Poster at a Glance

TU Wien, the technical lead for OSTrails and one of the project's 24 national and thematic pilot institutions, showcased a poster titled “TU Wien & OSTrails: Connecting services”. It focused on how service-agnostic APIs and interoperability specifications developed within the project are enabling more seamless interactions between tools for Data Management Plans (DMPs), Scientific Knowledge Graphs (SKGs), and FAIR assessment.

The poster featured a working pilot: an integration between TU Wien’s InvenioRDM-based research data repository (TUWRD) with the DAMAP Data Management Planning tool, implemented as part of the OSTrails Austrian pilot. This allows researchers to report dataset reuse in their DMPs directly from the repository interface, replacing a previously manual and fragmented workflow.

A limited proof of concept for integration between a DMP platform and a research data repository has already been presented at OR2023 and completed at TU Wien in 2024. Similar tests are now scaled across different services and research settings to achieve more standardized proof of concepts across the project’s pilots.

Why It Matters

The OSTrails Interoperability Frameworks aim to overcome fragmentation by defining modular, vendor-neutral APIs that can be implemented across various platforms. This approach:

• Simplifies how researchers interact with multiple tools

• Reduces reliance on specific service providers

• Supports a more sustainable and collaborative infrastructure ecosystem

Infrastructure providers and developers can reuse these standardised building blocks. This lowers integration costs and improves metadata quality, traceability, and FAIRness.

 Audience Engagement

The poster was designed to engage both repository managers and developers. The presentation aligned particularly well with the OR2025 sub-themes of “Sustainability and Preservation” and “Community”, highlighting how open and flexible technical approaches can foster inclusive, future-ready research infrastructure.

Visitors particularly appreciated the connected services (DMP Platform, SKGs, FAIR Assessment Tool) and were interested in future use-cases pertaining to their own repositories. Moreover, they enjoyed Maximilian’s tablet-drawn illustrations on the poster!

Visitors engage with the poster as Suvini Lai (TU Wien) offers project specific insights. Photo by Maximilian Moser.

Looking Ahead

OSTrails offers practical, standards-based solutions to long-standing interoperability challenges, helping to turn open science principles into everyday practice. Rather than creating new platforms, it builds bridges between existing ones, empowering institutions to incrementally improve their workflows.

Once the OSTrails interoperability framework is finalised, the Austrian OSTrails pilot will implement it in its participating institutions.

- Written by Suvini Lai (TU Wien)

More about OSTrails pilots
InvenioRDM | Damap DMP Tool
Integration preview on Zenodo

Over the past semester, OSTrails launched the first part of its Interoperability Webinar Series spotlighting the development of Interoperability Frameworks (IFs) under the OSTrails Reference Architecture.

Key takeaways 

• Early access to FAIR-IF, DMP-IF, and SKG-IF specifications to prepare tools and workflows for integration.

• Practical examples of embedding frameworks into widely used platforms for EOSC alignment.

• Networking with EOSC, RDA, and related initiatives to align with interoperability standards.

• Exchange of insights on APIs, benchmarks, and extension mechanisms for real-world application.

• Direct input into framework development through use cases and feedback.

Nearly 300 participants joined the sessions on the Assessment-IF (previously FAIR‑IF), the Data Management Plan IF (DMP-IF), and the Scientific Knowledge Graphs IF (SKG-IF), representing a diverse mix of data stewards, librarians, repository managers, software developers, researchers, and open science coordinators from universities, research infrastructures, and international organisations. They came to learn how these efforts are enabling seamless collaboration and information exchange across research data management (RDM) tools and communities.

Main message.

In the context of the Open Science agenda, interoperability of research practices, tools, and infrastructures is no longer optional. It is essential for research to function at a scale across domains and communities. Achieving this, however, requires more than technical fixes — it demands alignment, shared standards, and collective effort.

OSTrails is working with the community to align practices, clarify responsibilities, and define common interfaces between tools and services. The immediate goal is technical compatibility, ultimately supporting a research ecosystem where infrastructures can interact by design, policies are easier to implement, and knowledge flows more openly across systems.

Session Highlights

Assessments IF (previously FAIR-IF): Mark Wilkinson (UPM), WP3 lead (Assessment tools & services), introduced the Assessment-IF, developed to address the lack of harmonisation across assessment tools and the inconsistent interpretation of FAIR principles. The framework supports interoperability and is designed to be embedded in DMP platforms to provide real-time, standardised feedback.

Components of the FAIR Assessment landscape, as presented during the webinar.

“We want to design and publish metrics and tests for a wider range of digital objects beyond data and including domain specific assessments.”

- Mark Wilkinson

In the discussion, participants showed strong interest in the upcoming API specifications that will enable tools to interact and harmonise outputs, as well as in approaches for describing algorithms as community judgments over benchmarks. There was also significant engagement around governance models involving diverse stakeholders — including domain experts, coders, and FAIR specialists — in the validation of metrics and tests.

DMP-IF: Marek Suchánek (CTU), WP2 (Plan-Track-Assess Alignment Implementation) lead, presented DMP-IF as a solution to fragmentation and lack of machine-actionability in current data management plans. Building on the RDA DMP Common Standard, the framework adds an application profile and a common API. This enables platforms to integrate with FAIR assessment tools, repositories, and SKGs in a uniform way while adapting the standard to European research needs. In doing so, it supports policy compliance and accommodates diverse research outputs.Insights on DMP-IF presentation.

“We are building on top of the DMP Common Standard from RDA by creating an application profile and API specification for DMP platforms, enabling interoperability, extensibility, and alignment with real-world needs such as policy compliance and support for diverse research outputs.”

- Marek Suchánek

During the discussion, participants highlighted how the framework can connect data repositories and DMP platforms in both directions — either fetching DMPs or pushing updates, such as dataset publication events, via the planned API. They emphasised its tool-agnostic design and its potential to reduce duplication through automation, ensure policy compliance, and support diverse outputs, including software management plans. Questions also covered reusing example DMPs, integrating with OSTrails DMP platforms and ensuring compatibility across systems.

SKG-IF: Andrea Mannocci (CNR), co-chair of the SKG-IF RDA WG, introduced the framework and its role in enabling cross-disciplinary research through semantic and technical interoperability. Building on the RDA model, SKG-IF provides a common data model, API specifications, and an extension mechanism to align and integrate knowledge graphs across systems, addressing the fragmentation caused by isolated, non-aligned graphs.

Overview of OSTrails contribution to the SKG-IF core model.

“The SKG-IF isn’t magic. There’s no central implementation or governing body that operates a service or infrastructure to enable dialogue between different SKGs. That might sound reductive, but in essence, SKG-IF is a set of guidelines. Anyone who wants their SKG to be interoperable with others and compliant with SKG-IF must follow these guidelines.”

- Andrea Mannocci

In the discussion, participants explored how communities — both established and emerging — can contribute requirements to SKG-IF extensions, either through the RDA Working Group or dedicated extension projects. Questions focused on the level of technical expertise needed to develop extensions and the importance of practical guidance and real-world examples. Interest also centred on keeping the extension process accessible to non-experts, providing templates, and maintaining strong links between community-driven extensions and the core model.

Why was it important for OSTrails? 

The webinar series provided OSTrails with a high-impact opportunity to present its core technical developments and engage with the research community and EOSC landscape. The events helped validate the project’s direction, and practical feedback surfaced for several topics, ranging from inconsistencies in FAIR metrics to technical needs for DMP platform integration and SKG alignment. These insights are now shaping the ongoing refinement of each of the three frameworks to better support real-world use cases.

Why was it important for the Research Community in general?

The series enabled the research community to engage directly with the emerging frameworks and provide input early in their development. It also fostered alignment with ongoing international and EOSC-related initiatives, such as the RDA DMP Common Standard and SKG-IF Working Group and FAIR Metrics and Digital Objects Task Force, and complementary projects like FAIRCORE4EOSC and FAIR-IMPACT. These connections ensure that OSTrails developments build on widely adopted standards, support machine-actionable outputs, enable integration with established platforms, and address shared priorities such as policy compliance, persistent identifiers, and cross-platform interoperability.

Conclusions

The OSTrails webinars highlighted a shared need across the research landscape for more aligned, automated, and interoperable infrastructures. Each framework is helping to bridge gaps, whether in FAIR evaluation, data management automation, or SKG integration, and their development is being guided by community input. As they evolve, the IFs are poised to support a more open, trustworthy, and connected research ecosystem.

Further Resources

Explore the full webinar series and access resources (recordings and slides) here.

Discover the documentation for the OSTrails Interoperability Architecture and its three Interoperability Frameworks here.

The recent maDMP API Hackathon, held on 27 May in Vienna, marked a significant milestone in the journey towards making research data management (RDM) more efficient and machine-actionable. This collaborative event brought together experts with a clear objective: to standardise the way that Data Management Planning platforms interact in an automated way with other RDM services.

Key takeaways from the event

• The "Common maDMP API - preliminary design" is officially closed following the hackathon, providing a stable foundation for implementation.

• A common CRUD (Create, Retrieve, Update, Delete) operation was firmly established for each entity within the API.

• Decisions were made regarding consistent HTTP status codes and error handling.

• It was agreed that each tool will handle its own authentication and authorisation mechanisms, offering flexibility, though OpenID Connect (OIDC) was recommended for future common ground.

• DMP versioning will not be supported in the API.

What was the main message?

In this context, the hackathon served as a catalyst for finalising critical design discussions on the topic, leading to a successful outcome: the establishment of a stable, common blueprint for a machine-actionable Data Management Plan (maDMP) API. This API will primarily enable the seamless creation, modification, updating, filtering, and even potential deletion of DMPs, forming the essential baseline for future interoperability of data.

Why is it important for OSTrails?

For OSTrails, this hackathon is a crucial step towards its mission of delivering the Commons to Plan-Track-Assess research in European Open Science Cloud (EOSC). OSTrails aims to streamline FAIRness, interconnectivity, and machine-actionability across all phases of research. Data Management Plans are a core pillar of the OSTrails project, and a standardised maDMP API is vital for achieving its goal of co-developing and testing fit-for-purpose solutions for FAIRness and machine-actionability across diverse domains and national settings. Building on the OSTrails Reference Architecture, which provides the foundational blueprint for interoperable research tools and ensures alignment across all OSTrails pillars (FAIR, SKG, and DMP), the finalised API directly supports making Data Management Plans more robust and integrated within the broader research ecosystem.

Why is it important for Open Science in general?

This API is important for Open Science because it facilitates the machine-actionability of Data Management Plans, moving them beyond static documents to dynamic, integrable entities. This advancement is key to enhancing the FAIR principles (Findable, Accessible, Interoperable, Reusable) for research data. By standardising how DMPs interact across different tools and platforms, the API fosters greater interoperability within EOSC. This means better governance, tracking, and assessment of research data, ultimately contributing to a more efficient and transparent research landscape globally.

Impressions

The hackathon was characterised by lively discussions and a strong collaborative spirit, especially when tackling complex topics. One of the key outcomes was the successful conclusion of ongoing negotiations around the API design, culminating in the closure of the Common maDMP API - preliminary design document, now considered stable. This outcome reflected a focused and productive working environment, with participants working effectively together to reach consensus on critical technical matters.

Conclusions

The hackathon marks a major leap forward in making Data Management Plans truly machine-actionable and interoperable. The consensus reached and the finalised preliminary design provide a solid foundation for DMP providers to begin implementing the API. This collaborative effort is essential for building a robust infrastructure that supports researchers, institutions, and funders in creating a cohesive, machine-actionable Open Science ecosystem aligned with FAIR-by-design principles and the broader EOSC vision. The event also underscores the power of community-driven development in addressing complex technical challenges for the benefit of the wider research community.