Data Treatment Software

Coordinator: Thomas Holm Rod (ESS)
Partners: ESS, ILL, STFC, PSI, Juelich, CEA, Universita' degli studi di Parma

Overview

Participants at Workshop I in WP10.

This WP has two main objectives with respect to users:

Straightforward generation of scientific results for non-expert and industry users that are either directly publishable or can be used for creating new innovative products.
Data treatment software ready for users at ESS in 2020 – users will be familiar with the software and able to produce high-impact results from day one.

Therefore, the partners will

converge towards supporting the same data treatment software and make it available across facilities with a prioritization of high-throughput and industrially relevant techniques,
make it possible for users to leave a facility with analysed data and publishable results rather than just reduced data,
facilitate automated data treatment workflows,
make the software extendable and open for contributions from third party developers, e.g. at universities, and
exploit and support modelling based on atomistic simulations in data analysis in order to nurture high impact science.

The partners are facilities with scientific computing groups and the objectives will be achieved by exploiting the strength of each group to transform the existing, most widely used data treatment software into a form which is interoperable and sharable with all facilities and extended to cater for new requirements in data treatment.

Tasks and Partners

Task 1. Dissemination and management

Task Leader: ESS

A kick-off meeting and annual workshops will be organized so that input and requirements can be collected from the user community and SINE2020 facilities. Results in the form of software will be disseminated enabling other facilities and third party developers to exploit the developed software and contribute to it in a structured way. Facilities, software developers in universities as well as non-expert and industry users will be invited to relevant meetings .

The high-throughput techniques; SANS, reflectometry, and powder diffraction constitute the Phase I software together with imaging. These four techniques are the most important ones for potential industry use and are therefore given higher priority than the Phase II software, which contains QENS, INS and modelling software for neutron scattering and muon spectroscopy. All software will be open source and when demonstrated at the workshops it will be possible to download it from publicly available repositories.

Task 2. Guidelines and standards

Task Leader: STFC
Partners: ESS, FZJ, PSI

Guidelines and standards will be established that enable different software components to be linked together in an interoperable (plug and play) manner and facilitate joint development, both within and beyond the WP. C++ and Python will be the standard programming languages, as these are already employed by all the partners and the software to be developed in this WP, and NeXus, a variation of HDF5 tailored for neutrons, x-rays, and muons, will be the standard data format. Many of the guidelines and standards will be based on work in the Mantid project, as well as in the FP7 projects, NMI3-II and PaNData.

UPDATE: The Guidelines and standards document is now complete and available to download.

D10.1 Guidelines and Standards for Download (526 kB)

WP10 SINE 2020 Workshop 2 D10.1 Guidelines and Standards for Download

Task 3. Converging on Mantid for Continuum Sources

Task Leader: ILL
Partners: FZJ, PSI

The software framework, Mantid, for data reduction is jointly developed by the pulsed sources SNS and ISIS. ESS has also decided to adapt Mantid for its data reduction needs. In this task Mantid will be further developed and deployed at continuum sources for the techniques: powder diffraction, SANS, and reflectometry, and QENS in that order and demonstrated at Workshop III .

Task 4: Interoperable data treatment software

Task Leader: ESS
Partners: ESS, FZJ, PSI, ILL, CEA, UNIPR

In this task, support for data analysis required for getting from reduced data (produced by Mantid) to publishable data will be established for the Phase I techniques imaging, reflectometry and SANS and the Phase II software for QENS, INS, and modelling of neutron and muon experiments. The guidelines from Task 10.2 will form the basis for this and proof-of-concept, automated, user-friendly workflows for data analysis, including interfacing to Mantid where relevant, will be developed, illustrated with use cases and demonstrated at Workshop III or IV , depending on the specific software and technique. As a proof-of-concept of how the interoperability can be exploited, a virtual experiment, which can be used for e-training (WP3) will be setup based on the data treatment software from this WP.