OpenAI Plays Hide and Seek…and Breaks The Game! 🤖

One of the problems we have to be conscious about with self supervised learning is that the already existing problem of transparency and understandability gets worsened by orders of magnitude. If current billion parameter models are opaque, the result of SSL is, or will be, the equivalent of a black hole. How do you understand what the model has learned? Why has it learned that way? What things could have possible gone completely, horribly wrong in its quest to optimize its loss function? This somewhat humorous but also worrying paper comes to mind: [this video] I think we really need to consider how we can mitigate the risk of raising completely ununderstandable yet scarily capable machines.