Jewelry Ring AR

The best in class jewelry try-on application

The challenge

Selling jewelry is a non-trivial task, as it’s important to show the prospects an exceptional quality and uniqueness of items, which are pieces of art in themselves. That’s why companies tend to stick to brick-and-mortar stores, easily dismissing the ways to represent luxury items online. It is believed that nothing can compare to the experience of trying on a diamond ring. And, to some extent, it might be true.

Yet, there’s another option for jewelry try-on that needs to be reckoned with nowadays — an application as powerful as Jewelry Ring AR might do so much better than any classic store can ever dream of, for the prospects that have no physical access to the store due to different circumstances. 

While enabling people interested in trying on the jewelry, to actually do so using augmented reality, the company literally opens up a blue ocean of possibilities, accessing news customers that weren’t ever able to visit the physical store. 

However, if that is so, why only a handful of companies have adopted new technology? The answer is as simple as it might be: it’s extremely difficult to provide a perfect AR experience for the customers, making it possible for them to actually perceive the priceless value of a jewelry piece. 

Most of the AR solutions out there are either poorly performing or rough around the edges. You can’t use them to make your customers’ experience with the jewelry more rich and vibrant. 

And that’s the very reason why the Postindustria team decided to come up with the Jewelry Ring AR application, that is both easy to use and is able to highlight the beauty of rhinestones in the way a jewelry company needs it.

The Solution

Key Results

The most important thing to mention is that Jewelry Ring AR app doesn’t have any competitors both in the way it displays the 3D models of jewelry rings relying on powerful physical-based rendering, as well as in the tryon options, that allow moving one’s hand sidewards to see how the ring actually looks on one’s hand. 

These are the main differences that make Jewelry Ring AR application unique:

• The diamonds look astonishingly realistic. The physical-based rendering allows one to reproduce the way a rhinestone looks in a certain environment, down to the detail. We take into account the ray tracing, compute refractions and reflections inside the multiple transparent facets. As the result, we got beautiful realistic dispersion — same as you perceive in a real gemstone. An exact mathematical model describes real world characteristics, thus we got results indistinguishable from reality. 
• There’s no need to use any additional sensors for now. Everyone owning a smartphone with a camera can try on a diamond ring and see how it looks on one’s hand. The only thing you should do is install an application and select a finger on which you want to try on a ring. However, the next versions might use additional sensors even for richer quality. 
• The ring is perfectly fitted to one’s finger. You can even turn your hand to see how it looks.

How we achieved it

The process of creating Jewelry Ring AR was not as easy as it might seem. There were no ready solutions in the industry, and the main reason for this is that it’s extremely difficult to showcase pieces of jewelry, especially with diamonds, making them look physically correct, so you won’t distinguish them from the real ones.

Another thing to mention is that there weren’t any ready frameworks and neural networks to use in order to get the desired result out of the box. That’s why Postindustria’s experts started out with the research of what’s already there on the market — and have found nothing, which meant that the solution had to be delivered from scratch. 

The next step we took featured development of an application that would represent the pieces of jewelry in their beauty while they are positioned on one’s hand which is static. 

This model was built with TensorFlow framework which allows the detection of a hand and specifies the positions of each finger. After that the width of the finger got calculated and it was possible to render a ring that is perfectly adjusted to the selected finger. 

However, this approach had its limitations — for one, it was impossible to see how the ring moves along with the finger when you turn your wrist, and this is the exact thing the customers do in real life to see how the jewelry imem looks on them. 

Our team at Postindustria was determined to combat this problem, so as a result we decided to use our own neural network instead of the one used before for flat ring rendering. 

And we succeeded! 

How it all works

To make the ring move along with the finger even when one turns his hand sidewards, it’s important to detect certain keypoints of the shape which approximate the form on which the ring will appear as a rigid body. 

Then we need to use the transformation matrix for the camera to provide the proper ring placement on one’s finger. 

Our team needed to create our own synthetic hand to generate all the possible finger positions in order to use them as a markup for learning all movement outcomes. We took the keypoints of a certain shape and then the neural network predicts the positioning of the ring on one’s finger based on them.

Technologies used

Working on Jewelry Ring AR application took us not only the bravery to develop our own neural network, but also the knowledge of frameworks and technologies to fit the purposes of cross-platform compatibility. 

Our team used the following technologies to deliver an application:

• Unity (for rendering);
• Convolutional networks (Tensorflow).