Marcello Vitali-Rosati will give a lecture @ DHSI

As part of the Digital Humanities Summer Institute (DHSI), Marcello Vitali-Rosati will give a lecture on Monday June 2 from 9:00 to 10:30 am. The lecture will take place at the Université de Montréal and will be available online afterwards.

A Formal Definition of Creativity: LLM, Softmax and Temperature

"Machines can’t be creative! They can’t create anything new!" This is an idea already mentioned by Turing in his 1950 article (Computer Machinery and intelligence, the "Ada Lovelace objection") and which recurs notably in the recent (2021) seminal article “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?”. But what does "creative" mean? What if the question wasn’t whether or not a machine can be creative, but rather to give one or more formal and unambiguous definitions of creativity? In my presentation, I will analyze this hypothesis by focusing on the notion of temperature as implemented in the activation functions (softmax) of LLMs.