Multiple beamlines and experimental methodologies are developed in parallel at Sirius, which continues to expand with new stations and experimental capabilities. In this context, with finite engineering and operations resources and a diverse user community, it is essential to have a common language to answer, objectively, three questions: what is ready, for whom, and under which constraints.
To that end, LNLS adopts a multi-level assessment framework that connects scientific maturity and technical evidence when discussing an experiment that is available. The Experimental Validation Level (EVL) describes the maturity of a specific experimental methodology on a beamline (or station). In parallel, the Technical Validation Level (TVL) describes the maturity of the technical dependencies (beam, optics, detectors, mechanics, controls, processing, infrastructure, and sample preparation) based on key performance parameters (KPPs). In practice, EVL determines whether a method can reliably produce science, and for which user profiles, while TVL highlights the technical bottlenecks that must be addressed for the methodology to advance safely and robustly.
EVL is a maturity scale with the following levels:
EVL-1 | Theoretical concept. There is a solid technical and scientific basis, supported by calculations and simulations, indicating that the technique can generate meaningful data even before a practical demonstration.
EVL-2 | Experimental proof of concept. Controlled tests validate the fundamentals of the method, not necessarily yet under full beamline conditions. Integration with the end station is typically consolidated during technical commissioning.
EVL-3 | Ready for scientific investigation. In the developers’ hands, the method already produces data reliable enough to answer scientific questions, moving from “is it feasible?” to “does it work for science?”. This stage marks the transition from the end of technical commissioning to the beginning of scientific commissioning of the methodology.
EVL-4 | External reproducibility. Independent researchers, familiar with synchrotron experiments, can reproduce results with comparable quality and reproducibility. This is the “gate” for opening the method to experienced users under controlled conditions.
EVL-5 | Robust for general users. Non-specialist users (or users with minimal training) can obtain reproducible results within agreed tolerances. Operating procedures, documentation, and training materials are complete.
EVL-6 | Protocolized and scalable. Workflows are standardized and prepared for high throughput, with metadata-rich acquisition, automated quality control, and data pipelines that integrate smoothly with AI and machine learning models when applicable.
A simple principle governs the framework: an experiment’s EVL is limited by the lowest TVL among its critical dependencies. This makes it explicit where to invest to unlock the next step: sometimes a detector is the pivot, but more often the true limiter lies in stability, metrology, calibration, controls integration, acquisition synchronization, or latency in processing and reconstruction.
TVL, in turn, is derived from how well each subsystem or capability meets its KPPs (for example: beam quality delivered to the sample, detector performance, mechanical precision, synchronization, reconstruction algorithm performance, environmental conditions, and sample preparation protocols). A subsystem/capability may already have a high TVL while the experiment remains at a lower EVL. This happens because EVL must be experimentally validated for the experiment to reach the maximum level enabled by the TVLs.
Within this framework, robust and productive experiments (typically EVL-5/6) coexist with higher-risk concepts with strong potential for advancement (EVL-1/2). EVL guides user assignment and the allocation of beam time: EVL-3 remains within internal scientific commissioning, EVL-4 opens to expert users under controlled conditions, and EVL-5/6 serves a broader user community, potentially operating with greater autonomy. In addition, EVL supports operational risk management by requiring denser technical support and tighter scope at lower levels, while enabling reduced supervision, remote access, and greater scalability for mature experiments.
Each EVL step forward, in addition to requiring compatible TVLs, is supported by evidence and documentation that evolve from a documented scientific justification and dose/safety analyses (EVL-1/2), to an end-to-end demonstration with KPPs met and a functional analysis pipeline (EVL-3), repeatability (EVL-4), reproduction by an independent team with complete guides and training (EVL-5), and finally scalable standardization and automation with robust integration to computational resources (EVL-6). As a result, each methodology is assigned a “lifecycle card” that consolidates the current and target levels, critical TVLs, risks, mitigations, and responsible owners, increasing transparency and reproducibility through citable documentation.
The illustrative graphical example shows an experiment that can reach EVL-4, but whose evolution is limited by synchrotron beam characteristics (TVL). In the example, parameters such as beam size and coherent fraction reach a performance level operable by the operations team (Level 4), yet still without the robustness required for routine operation by general users.