Editor’s Note
This article highlights the groundbreaking collaboration between GIA and IBM, which leverages advanced technology to transform diamond analysis and certification. By processing millions of diamonds annually, this partnership aims to enhance accuracy, efficiency, and trust across the global diamond industry.
The collaboration between GIA and IBM to revolutionize the diamond industry.
GIA analyzes 4 million diamonds annually.
It is expected that 70-80% of all diamond stones will be analyzed using AI-based solutions.
Pritesh Patel, Chief Operating Officer, Gemological Institute of America
Patel approached IBM® Research with this vision, believing that with the right skills and technology, they could harness the power of AI in the cloud to assess diamond clarity.
IBM Research agreed, and the two organizations began building a strategic partnership. GIA provided specialized images along with data extracted from tens of millions of diamonds examined by diamond experts, and IBM provided AI capabilities and computing power. As a result of this partnership, GIA can now grade diamonds using a cloud-based AI approach.
Assessing a diamond’s clarity is a complex process. Gemologists meticulously examine each diamond using a loupe, microscope, or images to see if it has inclusions—small characteristics trapped within the stone’s structure. Inclusions can be very small internal spots or cracks that penetrate from the surface into the stone.
Flawless diamonds, those with no inclusions at all, are extremely rare. Less than 0.5% of all diamonds graded by GIA worldwide fall into this category. The vast majority of diamonds contain one or more inclusions, making each stone unique.
After IBM Research successfully conducted a proof of concept (POC) demonstrating that AI could indeed help automate the diamond grading process, the IBM® Global Cloud Acceleration Team (GCAT) stepped in to lead the project to the next stage. The GCAT team collaborated with the GIA Engineering DevOps team to guide the POC’s solution into a production-ready environment with separate development, test, production, and disaster recovery clusters.
Currently, the solution is in the testing phase and has entered full-scale production. GIA laboratories upload specialized images of each diamond to an IBM® Cloudant database on the IBM® Cloud. The system’s middleware layer consists of an IBM® Cloud Kubernetes Services cluster.
GIA’s cluster consists of three NVIDIA K80 GPUs, each with one shared virtual node and one bare metal node in a serverless architecture. NVIDIA GPUs are well-suited to GIA’s requirements as they can process high-resolution images quickly, helping to speed up the overall process. They also reduce the time needed to validate AI algorithms.
The new solution uses two custom algorithm models to analyze each diamond. An automatic plot AI model visually represents diamond inclusions, and a grading model assesses the diamond’s overall grade. This information is then sent to GIA’s gemologists via an iPad application, where the gemologist can evaluate and, if necessary, modify the automatic plot. These changes can then be fed back into the system to re-evaluate the grade and used to retrain the AI models to improve accuracy.
GIA’s mission to protect consumers is the driving force behind ideas for innovation. However, GIA had another goal for this project: to achieve the highest level of data security to protect the integrity of diamond grading. As diamonds are high-value commodities, protecting the grading process was paramount.
IBM® Cloud App ID enables the uploading and processing of diamond images with advanced security features such as multi-factor authentication, single sign-on, and custom password policies.
Images are stored using IBM® Cloud Object Storage, a cost-effective storage solution that meets all of GIA’s requirements for scalability and accessibility. Additionally, IBM Cloud Object Storage has built-in encryption capabilities and offers policy-enabled WORM (Write Once Read Many) storage with locking features.
Throughout the engagement, the IBM® Strategic Embedded Partnership (ESA) team has worked to develop a mutually beneficial strategic partnership between the two companies. The ESA team not only provides continuity and support from a business perspective but also helped establish the legal and business structures that underpin this type of partnership.
GIA grades millions of diamonds every year. Patel expects the new solution to handle 70-80% of these cases, primarily focusing on the smaller diamonds submitted, thereby allowing humans to concentrate on determining the grades of more complex cases that require human evaluation.
However, achieving these goals takes time. Currently, GIA is using the solution in two of its 11 laboratories, running AI and human evaluations in parallel as they refine the algorithms. Ultimately, thanks to the ongoing support of the GCAT team, GIA plans to make the new solution available across all laboratories.
Although the solution is still in its early stages, Patel expects to see several key benefits moving forward. The first is efficiency, he says.
The solution can also improve both accuracy and repeatability. Human diamond graders undergo rigorous training but work within the limits of physical senses. If two graders assess the same diamond, their evaluations may differ very slightly. Using AI virtually eliminates these minor differences, helping to accurately value diamonds when they go to market.
GIA stakes its reputation on the integrity and accuracy of the new solution. Just as it transformed the industry in the 1930s by introducing the jeweler’s loupe to the diamond grading process, this project will introduce an entirely new level of precision to the process.