News  |    |  December 19, 2019

This AI researcher is trying to ward off a reproducibility crisis

News article by Elizabeth Gibney.
Published in Nature.

Joelle Pineau doesn’t want science’s reproducibility crisis to come to artificial intelligence (AI).

Spurred by her frustration with difficulties recreating results from other research teams, Pineau, a machine-learning scientist at McGill University and Facebook in Montreal, Canada, is now spearheading a movement to get AI researchers to open up their methods and code to scrutiny.

Alongside Koustuv Sinha, a PhD student at McGill, Pineau holds one of two new roles dedicated to reproducibility on the organizing committee for the Conference on Neural Information Processing Systems (NeurIPS), a major meeting for AI that this year attracted some 13,000 researchers. Ahead of this year’s conference in Vancouver, Canada, from 8 to 14 December, the committee asked scientists to provide their code and fill in a checklist of methodological details for each paper submitted. They also ran a competition that challenged researchers to recreate each other’s work.

Pineau spoke to Nature about the measures and how they’ve gone down with the community. [ . . . ]