In industrial metal-to-metal welding operations, companies are struggling to automate inspections to efficiently detect weld defects. To prevent costly product recalls, excessive scrap, re-work and other costs associated with poor quality, companies look to automate inspections and identify weld defects early and consistently.

The unsung heroes

Welding is the fusion of two compounds with heat. It’s a process that happens billions of times every day, and one which we all depend on. The chair you’re sitting in while reading this likely has dozens of welds. Your car has hundreds to thousands of welds. The electricity generated from hydroelectric dams travel hundreds of miles through transmission towers with thousands of welds to power your home. Unless something goes wrong, nobody ever thinks about welding. We only enjoy the benefits it brings us.
It is the manufacturers’ job to make sure you’re sitting comfortably in your chair, your car is operating safely, and your gas is flowing when you need it. This requires close collaboration across design, process engineering, technicians, quality control, and a trusted ecosystem of suppliers and equipment providers.
Manufacturers are the unsung heroes who make sure we’re safe, day in and day out. They do not get famous if they do their job well. However, if something goes wrong—accidents, recalls, leaks or even deaths—then manufacturers are the first ones to be questioned. In addition to the reputational cost and risk, bad welds in the automotive industry alone cost up to 9.9 billion USD per year, according to McKinsey.

Challenges in welding inspection

Take a moment to inspect the weld joint below. At first glance, can you determine whether this weld is good or bad?

Most likely you cannot. That’s all right, because almost nobody can tell from visual inspection. Just like an iceberg floating in the water, where only the clear white tip is visible and the danger lies invisible beneath the surface, many weld quality indicators are invisible to the human eye.

Figure 1 below is a chart of the most common arc welding defects. The color of the star next to each defect shows how visible each is to experienced subject matter experts.

Manufacturing processes use a combination of destructive and non-destructive quality testing methods to determine whether there is a discontinuity or defect with a weld. Let’s dive into the differences between these two forms of testing.

• Destructive testing includes the mechanical disassembly of a weld (e.g. grinding) and chemical etching (e.g. ethanol plus citric acid) to measure fusion parameters. It is the most accurate method of quality evaluation, and only a small number of samples is needed. However, after a defect is discovered, remediating it requires discarding all the welds that have taken place from the time of the discovery to remediation. The process is very costly and time consuming.  

• Non-Destructive testing is largely done by human visual inspection. Occasionally, it is augmented by ultra-sound testing, which is also human-driven. Once a defect is discovered and remediated, each weld completed during that time must also be tested. These types of inspections are subjective, inconsistent, cover only a subset of defects, and are both expensive and time-consuming.

The game changer

We are not the only ones thinking about this problem. Equipment and sensor providers are trying to address it, and most manufacturers are attempting to leverage advanced analytics and AI with varying degrees of success. Equipment providers focus on the data their components produce, while sensor providers focus on the information their sensors generate. We see several challenges with these approaches, including:

• They cover only a small subset of failure modes.
• They provide short term accuracy but suffer from long-term model drift.
• They don’t adapt to operational change.
• They make use of only certain types of data.
• They require a large amount of such data.

What is IBM Smart Edge for Welding on AWS?

IBM Smart Edge for Welding on AWS utilizes audio and visual capturing technology developed in collaboration with IBM Research. Using visual and audio recordings taken at the time of the weld, state-of-the-art artificial intelligence and machine learning models analyze the quality of the weld. If the quality does not meet standards, alerts are sent, and remediation action can take place without delay.

The solution substantially reduces the time between detection and remediation of defects, as well as the number of defects on the manufacturing line. The result is overall cost reduction. 

IBM Smart Edge for Welding on AWS uniquely leverages multi-modality and IBM Research’s patented multi-modal AI to provide accurate insights through a combination of:

1. Visual Analytics

• IBM Maximo Visual Inspection (MVI), both edge and AWS models allow us to analyze in-process welding videos in real-time with computer vision.
• Xiris Weld Cameras, purpose built industrial optical camera that provides never before seen high resolution in-process videos of the weld pool, wire, workpiece etc.
• Xiris Thermal Camera, a purpose built industrial thermal camera that visualizes heating and cooling behavior of a weld as it is being produced.

2. Acoustic Analytics

• IBM Acoustic Analytics, a proprietary, patented, purpose built neural network to analyze weld sounds.
• Xiris WeldMic a purpose-built industrial microphone that listens to the arc sound in real-time, like your most experienced weld technicians would.

3. AWS Edge and Cloud

• Industrial Edge Computing allows us to integrate seamlessly into your manufacturing environment, to create real-time insights, save and secure without any sensitive information ever leaving the plant.
• Cloud Computing, available as public, private or dedicated cloud deployment, enables scalability across production lines, plants, and even geographies.

Seeing the defect is believing

While visual inspection is tedious and highly error prone, and often miss to identify welding defects such as surface irregularities and discontinuities, computer vision system is able to detect anomalies and welding error with high degree of accuracy. Here are examples of a few latest AI-based approaches we currently deploy in our clients production operations:

Optical Video

The optical video clip below visualizes several components of a weld:

• Size and shape of the weld pool and how it solidifies as it cools;
• Behavior of the wire as it deposits filling material;
• Spatter that is generated;
• Turbulence in the shielding gas; and
• Holes forming from burns.

Thermal Video

The infrared video clip below visualizes several additional components of a weld:

• Thermal zones through color coding;
• Uniformity of the trail;
• Heat signatures, and size and purity of the weld pool; and
• Annotations created by our AI models (in this case for porosity) in real-time.

Acoustic Insights

The image below is a translation of the welding sound into a sound wave and sound spectrum, and identifies:

• Patterns of normal and abnormal behavior; and
• Classification of abnormalities to specific failure modes.

The result

By leveraging a combination of optical, thermal, and acoustic insights during the weld inspection process, two key manufacturing personas can better determine whether a welding discontinuity may result in a defect that will cost time and money:

1. Weld technician: works on the shopfloor and needs insights on weld performance in real-time to add, change, or optimize the process as needed. The dashboard below is built with ease of use in mind. The solution can be integrated into any platform and device used on the shopfloor, such as HMI or mobile devices.

2. Process engineer: wants to understand patterns and behavior across shifts, weeks, months, weld programs and materials to improve the overall manufacturing process.

Solutions benefit

Our clientshave reported the following benefits from their implementations of the solution:

• Improved quality through inspection of 100% of welds.
• Reduction of time and optimization of setting up the weld program.
• Accelerated launch of new products or changes.
• Identification of trends as early warning signs of defects and other real-time insights.
• Reduction of time between identification and resolution of an issue.
• Cost reductions through reduction of physical labor and human testing, material needed, and scrap material resulting from destructive testing, bad weld batches, and preventative remediation.
• Unidentified weld defects increase warranty risks and recalls. With this solution the risk is reduced because each weld is inspected, and quality standards are met.

As a result, a single factory has demonstrated potential savings of 18 million USD* a year through these cost reduction benefits. Warranty costs and recalls—which cost the automotive industry alone an estimated 9.9 billion USD a year—can be avoided or significantly reduced when they are due to bad welds. Brand reputation is maintained when delivering high quality and safe welds.

Partnering with AWS

IBM partnered with AWS to develop a solution to address the industry-wide manufacturing challenge of quickly identifying weld defects to enable fast remediation. The solution architecture includes cloud and edge components.

AWS Cloud has over 200 services that can be leveraged to enhance, optimize, and further customize this solution. IBM’s AI models are trained in AWS cloud and deployed to the edge for inferencing. All weld data is stored in the cloud in a low-cost storage environment for analysis and future model training. Amazon QuickSight can be used for Process Engineer dashboards and reporting. It enables automated process of model deployment to edge endpoints.

The edge environment of this architecture runs on AWS IoT Greengrass. Data is ingested from the shopfloor sensors (ex. cameras and microphones). It is pre-processed to eliminate excess noise from the audio data and blurred images from the video data. Then model orchestration and inferencing is executed through a machine learned model utilizing IBM Maximo Visual Inspection and IBM Acoustic Analyzer, to identify the quality of the weld and determine if it meets the set standards. Post processing takes place from alert notification and reporting, to transferring data to the cloud for further analysis, model training, compliance archiving, and other beneficial purposes.

Reference architecture

To conclude

IBM Smart Edge for Welding on AWS provides clients with an end-to-end, production-ready solution that generates bottom-line impact through the optimization of manufacturers’ welding processes. IBM in collaboration with IBM Research offers the power of AI, from Computer Vision with IBM Maximo Visual Inspection (MVI) to IBM Acoustic Analytics.

The solution provides manufacturers with real-time weld defect insights for faster problem diagnosis and remediation through a weld quality single pane of glass. Welding technicians and process engineers can inspect up to 100% of welds to determine the cause of welding defects in the earliest stages of the production process. This results in less repetitive defects and rework, along with reduced material waste providing opportunity for companies to accelerate sustainable industrial processes. As a result, manufacturers could reduce re-work costs by up to 18 million USD* per 1,000 robots annually based on scrap, material and labor cost savings.

Special thanks to our contributors and collaborators, including Manoj Nair, Caio Padula, Wilson Xu, Ofir Shani, Nisha Sharma, Penny Chong, and Tadanobu Inoue.

*based on IBM’s existing engagements, but individual results may vary