At its Snap Partner Summit, the company announced that it was working on improvements to its camera. Snap wants in particular a camera that is able to highlight both very dark and very light skin, often abused by algorithms for a rendering that is far from flattering.
After Google, Snapchat also wants its camera to be more inclusive. The first one announced during his Google I / O conference that he was going to work on his camera so that it allows a better rendering of all skin tones, especially non-white ones. The second was more direct by explaining the difficulties that his camera had to highlight dark skin, but also too light, like many smartphone cameras.
To be taken for a shadow …
Because the real problem for a camera, it is too often to consider too black or too white skin as a problem of brightness or shade on the targeted individual. To solve this problem and make its camera more inclusive, Snap even has a dedicated team. ” I found that the camera tended to lighten my skin when I took a selfie, or darken when I was in low light and I couldn’t see myself.“, Explains Bertrand Saint-Preux, who heads the team. ” I wanted to improve this experience for myself and for the community. “
« We’ve looked at the types of image processing algorithms, and we’ve trained ours to take a lot of images of people into account. To train our machine learning algorithm, we had to capture a very wide variety of people to have the largest sample so that upon arrival, our experiments would work for everyone.“, Explains Frandroid Qi Pan, director of engineering and computer vision at Snap.
To ensure that the camera can work with all Snapchatters skin tones, the company has therefore worked to ensure that its camera detects very black or very white skin first and foremost as … skin and not as a problem. shadow or light.
Used billions of times every day by users, the camera must then allow the Snapchatter to be captured as it sees itself, with or without a filter. The first step was to work on flash capture in low light, which is often complicated to manage, and exposure correction. What now to better take into account the nuances of skin in selfie.
Filters for everyone and even several
Snap will now make the tool available to developers and creators to improve their uses, because the Snapchat camera is not only used within the application, but with various partners. This is obviously a way to become essential for the greatest number at a time when the selfie is a generational asset to show off.
During a Snap Partner Summit conference focused heavily on camera improvements, Snapchat also unveiled new features for users, including many filters, Map experiences and even more. lConnected Lenses. These will bring an unprecedented multi-user dimension. LEGO is the first partner to stick to it by offering the possibility of interacting with friends, all together in the same room or at a distance, on the same experience in augmented reality. You can make it evolve, play by simply sharing the experience in real time.
« For the first time, we are really able to get several people to experience augmented reality together and without having to send a video recording. But in real time», Explains Qi Pan. “You are going to be able to design things that will then add up to gradually build something bigger and change the augmented reality world of someone else who is not with you.. The first stone of a new way of using the camera with others and in the same way. Connected Lenses are already available.
The 3D body mesh will also change the experience, provided you are able to properly detect bodies. Among the first experiences offered, you can don Cruella’s dress in the next eponymous Disney film or try fittings at Prada and Farfetch. But this will notably be used by designers to offer greater and more relevant shopping experiences, because the simulation of fabric and visual effects will be able to follow your movements while adapting to your appearance.