On 12 June, the OSTrails Horizon Europe project launched the first in its series of maDMP Evaluation Webinars, aiming to improve data management practices through the adoption of machine-actionable Data Management Plans (maDMPs). Attended by more than 80 participants, the first webinar focused on inviting funders to explore new potentials, discuss evaluation challenges, and directly shape the OSTrails maDMP Evaluation Framework.
Key Takeaways
- Funders want practical tools that help evaluate DMPs based on clarity, feasibility, and policy alignment, not just completeness.
- Machine-actionable DMPs are powerful, but they must be designed and evaluated with care. Undoubtedly because they make it easier to link planning with implementation, reuse, and assessment.
- The evaluation framework fills a gap by translating high-level guidance (e.g. Science Europe) into structured, testable dimensions. This can help align DMPs with funder goals and research needs.
- Community feedback is essential to ensure the framework supports real workflows and diverse policy contexts. Rather than enforcing universal openness, Open Science infrastructures should support pluralistic and ethical openness that is tailored to different contexts.
- There is a strong interest in using the evaluation service dimensions once it moves beyond beta.
The central message was that DMPs should no longer be treated as static documents but rather as active and shared components of research workflows.
The session was led by the OSTrails Horizon Europe pilot, which has been working with funders and research communities to redesign how DMPs are created, reviewed, and assessed. It introduced the three core areas of OSTrails’ work with DMPs:
- Conceptual: Defining the structure, content, and principles of DMPs, including the evaluation framework and the underlying policy and community requirements.
- Technical: Developing the specifications, standards, and interoperability mechanisms that enable DMPs to be machine-actionable and integrated with other research services.
- Operational: Implementing and testing DMP concepts and technical solutions through platforms, pilots, and real-world use cases, ensuring they work in practice and can be adopted by different stakeholders.
As a concrete example, the pilot presented a proposed two-layer Horizon Europe DMP model separating declarative content (e.g. intent, policy) from implementation details (e.g. storage, preservation, access), to better support assessment and improve clarity of roles and responsibilities.
Building on this foundation, participants were introduced to the draft DMP Evaluation Framework, designed to support semi-automated assessments that are flexible enough to adapt to different funder templates and emerging needs. The framework consists of dimensions and metrics that go beyond completeness to address aspects like feasibility and clarity of data practices. This represents a significant shift towards tracking the implementation of data practices over time, rather than merely recording intentions. For OSTrails, this work is fundamental.
The webinar also featured a collaboration with the TIER2 project, which focuses on enhancing the reproducibility aspects available in DMPs. Together, we shared community insights and demonstrated the importance of engaging with stakeholders early on and often.
The session invited funders to contribute feedback through a short survey, reinforcing the participatory ethos that underpins the OSTrails approach. Moreover, it demonstrated that the OSTrails approach (combining machine-actionable formats, shared APIs, and layered planning) is technically feasible and ready to be integrated into funder workflows and DMP platforms. This confirmation is essential as OSTrails prepares for broader adoption across EOSC-related infrastructures used in its 24 pilots.
Participating research funders found the session highly relevant and timely. Many expressed strong interest in the evaluation approach and welcomed the opportunity to shape its development. There was clear demand for the DMP evaluation service, with several attendees noting they were eager to use it once it moves from beta to full production.
As OSTrails continues its work, this webinar series will provide an ongoing opportunity to include more voices in the conversation, including not only funders, but also researchers, institutions and community organisations.
We thank all attendees and invite continued engagement through upcoming sessions and surveys!
