Investigating the interplay between rigorous statistical techniques used in precision science, and inference performed with modern Machine Learning. _Can techniques developed in physics help open the black box of Machine Learning?_
We demonstrate that even the simplest neural networks have truly multimodal solutions and it is only by combining these candidate models that the true probabilistic description can emerge.