Impressive realtime relight, and it works on a digital painting too.

@jookia I'm surprised by the creativity of all this things. The break-down of the process behind this is really interesting;

@jookia Oh, you made fan art of Carrot and it's the first time I see it 🙂 With this light setting he looks a little bit creepy. 🤣

@SrEstegosaurio The author share a thread on twitter about the tech behinds it. It's really interesting and behind the complexity it simplifies the concept a lot:

@davidrevoy Impressive! While there are mistakes like in the top right, this surely is a super useful tool to speed up the process and help newer artists / those who struggle with lighting.

@davidrevoy How does that work? How does the image know what may be illuminated and what not?
An image has no depth in this sense.

@Tealk It looks like it recovers depth information by basically inventing it based on similar images. @davidrevoy

@Tealk @davidrevoy Theres lots of things that can be done with convolutional neural networks. Thats a type of AI model that extracts image information based on sets of mathematical filters, which were trained on huge datasets.
You can detect objects, classify objects, estimate depth or normal data from an image (see the MIDAS model), segment an image into its parts (used for autonomous cars) and a lot more.

Inscrivez-vous pour prendre part à la conversation

Le réseau social de l'avenir : pas de publicité, pas de surveillance institutionnelle, conception éthique et décentralisation ! Gardez le contrôle de vos données avec Mastodon !