Improving the fitting in Mantid for neutron and muon data

A team of scientists supported by SINE2020 is working on improving the fitting of neutron and muon data in Mantid. However, when do you know fitting of neutron and muon data has improved? To answer this question they developed a framework for comparing fitting minimizers targeted to serve Mantid users, but whose results are transferrable to other neutron and muon data treatment software.

SINE2020, 05/12/2016
Contact:

Discussions over the poster presentation at NOBUGS Discussions over the poster presentation at NOBUGS Picture courtesy of Lamar Moore.

Discussions over the poster presentation at NOBUGS. Picture courtesy of Lamar Moore.

A team, part of the SINE2020 Data Treatment Software work-package, are working together with colleagues from the ISIS Neutron Source, the European Spallation Source, the STFC Scientific Computing Department and Tessella to push data treatment software forward by improving fitting in Mantid. Their work was well received at the recent NOBUGS conference in Copenhagen, and is now the foundation for a new European framework for comparing fitting minimizers.

Fitting data with Mantid

Fitting is a core functionality in neutron, muon and x-ray data reduction and analysis software packages. It is required in tasks as diverse as instrument calibration, refinement of structures, and various data analysis methods specific to different scientific techniques.

One such software is Mantid: an extensible framework that supports high-performance computing for data manipulation and visualisation of scientific data. It is used at several neutron and muon facilities worldwide. The Mantid fitting system offers the flexibility to add and combine different functions, minimizers, constraints, and cost functions as plug-ins. Users can apply different combinations of these elements either via scripting or graphical user interfaces.

Fitting minimizers

When fitting a function to experimental or simulated data, the function parameters are chosen so that the model fits the data as closely as possible. A measure of how close a fit is to the data is defined by a cost function, and the parameters are chosen to minimize this function. Several local minimization methods (minimizers) are provided in Mantid, but there is a lack of openly-available comparisons between them.

The team’s primary aim is to improve 'Fit', the most used algorithm in Mantid, and thereby improve fitting in Mantid. Crucially, this framework for objectively comparing minimizers can be used by anyone to compare a new candidate minimizer against those that have already been assessed.

The team compared local minimizers for fitting a variety of datasets using Mantid. They conducted tests using real neutron data, as well as mathematical test problems. Thanks to this work, they could incorporate a new, flexible Trust Region minimizer into Mantid, which has been developed by the Computational Mathematics Group of the STFC Scientific Computing Department. The new Trust Region minimizer is currently the leading candidate to replace the default minimizer in Mantid. At each iteration an improved estimate of the minimizer is sought by reducing a simple model of the fit function while restricting the distance between the current and improved estimates.

Comparison of accuracy and run time for neutron and other datasets

The team tested the algorithm with neutron data from different instruments at the ISIS facility, as well as the NIST and CUTEst datasets.

For each test problem, the best possible results were given a score of 1. The relative score of a minimizer is the ratio between its performance and the performance of the best. They compared accuracy (sum of squared fitting errors) and run time. For example, a ranking of 1.25 for a given problem means:

  • (for accuracy) The minimizer produced a solution with squared residuals 25% larger than the best solution in Mantid.
  • (for run time) The minimizer took 25% more time than the fastest minimizer.

Future

The results obtained were very promising and the feedback from potential users of the algorithm was good, so the team plans to extend the comparison with more test problems from neutron and muon data, considering different science areas. Furthermore, on the basis of their comparisons, they intend to further develop and characterize the new, flexible Trust region minimizer, RALFit, whose aim is to improve the reliability and broaden the functionality of the Mantid fitting system.

References

Back to previous page