Reproducible research

To add an entry, please tweet @jjvie or do a pull request.

Platforms for hosting reproducible code

• CodaLab made by Percy Liang (Stanford University) and Microsoft Research

From Tum, a Stanford student:

To use GPUs or perform large amounts of computation, you can run your own worker or even set up your own CodaLab server.

Data challenges for academia

• CodeOcean, the Ruby platform driven by the Hasso Platner Institute, Germany
• University of Big Data, the Python platform driven by Kyoto University, Japan
• Of course, Kaggle exists… but… how to say……… Google acquired it.
• crowdAI from EPFL, used for NIPS

Tools for remote execution of code

So you can make data challenges yourself!

• Camisole, a Python package that abstracts the complex stuff into an API simple to use, made by Prologin (non-profit student-driven organization that promotes CS through a programming contest) and based on isolate (used by IOI)
• Taskgrader, a Python package that abstracts the complex stuff into a wrapper simple to use, made by France-ioi (take that, war of standards)

Those are Docker free, but require Linux: (isolate can be seen as a light version of Docker, based on chroot. (It will not work on a Mac OS X platform. You can use a VM though.)