独家优惠奖金 100% 高达 1 BTC + 180 免费旋转

Ground glass sheet defects classification

Manual inspection of a glass sheet may be a kind of a tedious work. Since the matting process result isn’t always satisfying, it will require some improvement, and an automatic defect detection and classification may significantly help, eliminating the time-consuming manual labour and propensity for mistakes.

In this article I’ll introduce some possible approaches for a ground glass defect detection and recognition.

Ground glass sheet is photographed by a line-scan camera, which yields a detailed image of a surface. The objective is to detect, segment and classify different defects that may appear on the surface due to manufacturing flaws or external damage.

Since the glass is being inspected before matting, absence of bubbles or inclusions is expected, but other point defects may appear — those could be undermatted or completely transparent small regions. There also may be so called “splotches” — wide, extended surface defects that appear due to uneven acid allocation, and some linear defects — scratches and fissures, which may be produced by some internal stress or external impact.

Some photos contain unwanted objects to detect, like dust particles or chalk marks. Ability to ignore those will be a big advantage of the method used.

The problem of a glass defects recognition has already got some attention from different researchers:

Zhao, Jie & Kong, Qing-Jie & Zhao, Xu & Liu, Jiapeng & Liu, Yuncai. (2011). A Method for Detection and Classification of Glass Defects in Low Resolution Images. 10.1109/ICIG.2011.187.

In this work the team analyses transparent glass photos using the Canny edge detector and the novel Binary Feature Histogram method with the AdaBoost model on top.

Bandyopadhyay, Yajnavalkya. (2015). Glass Defect Detection and Sorting Using Computational Image Processing. International Journal of Emerging Technologies and Innovative Research. 2. 73–75.

Statistic approach, based on the pixel intensity variation computing. The less defective a glass sheet is, the lower the overall variance is.

Hongxi Zhang, Zhenduo Guo, Zegang Qi and Jiuge Wang, “Research of glass defects detection based on DFT and optimal threshold method,” 2012 International Conference on Computer Science and Information Processing (CSIP), Xi’an, Shaanxi, 2012, pp. 1044–1047.
doi: 10.1109/CSIP.2012.6309035

Discrete Fourier Transform-based approach. The team describes preprocessing technique, spectral manipulations over the result of the image DFT and the iterative optimal thresholding technique.

Oh, Chang-Hwan & Joo, Hyonam & Rew, Keun-Ho. (2007). Detecting low-contrast defect regions on glasses using highly robust model-fitting estimator. ICCAS 2007 — International Conference on Control, Automation and Systems. 10.1109/ICCAS.2007.4406684.

Practical application of a modification of a robust MF-estimator by X. Zhuang et. al. for detection and segmentation of defects on the LCD-glass.

The first thing to do is to crop the image to remove not only empty areas around the sheet, but also the sheet edges to reduce the chance of false-positive detection, triggered by the glass border.

Most of the manufacturing recording systems doesn’t yield uniformly lighted photo, due to imperfect lamps arrangement or usage of polarizing filters. In most cases it’s not an issue, but some methods may require histogram equalization, which emphasizes lighting non-uniformity. So the next optional step after the cropping and histogram equalization is a lighting gradient removal. That’s not necessary while searching for point defects (they have a sufficient contrast with the background), but it may have a drastic impact on splotches, scratches and cracks detection.

The suggested way to do it is to get a gradient mask by strong blurring of the original image. After that, the mask is being subtracted from the original image. This will also shift the mean pixel value to number around zero — to visualise the image again, it’s useful to add a mean value of the original image to the result of a mask subtraction.

If the lighting gradient goes in parallel with the image borders, another solution is to divide the image into strips unidirectional to the gradient, and work with those strips separately.

The last but not least preprocessing technique is the noise removal. The common method is a median filtering with a small-sized kernel (3–5 px) — larger kernels may lead to the image degradation by creating a massive amount of false edges due to the noise brought by the matte. It may happen (in case of the histogram equalization), that median filter will degrade the image with any kernel size — in this case it may be replaced or applied after the linear blur or Gaussian filter. Linear blur filter offers more aggressive noise reduction, while Gaussian preserves edges more carefully.

It’s hard to say exactly, when noise reduction should be applied. That should be tested individually for every task. Noise reduction often takes place before and after detectors application or some global transforms, and before any kind of segmentation.

Detection task can be formulated as getting the preprocessed glass sheet image as input and producing the mask of defects, according to which they can be easily extracted during further work, excluding unwanted elements.

The first method is thresholding — creating a binary image, containing values:

where Mᵢ,ⱼ is a binary mask j,i-th pixel value, Iᵢ,ⱼ is the image j,i-th pixel value, and T is a threshold. The method works fine on contrast images, when the items to separate have different brightness relative to the background, but may fail in any other case. So, in most applications this method should be preceeded by the contrast increasing techniques.

Since the unwanted objects (dust particles, chalk marks) are brighter than the background or defects, no additional thresholding is required.

One example of the automated thresholding algorithms is the Otsu’s method. It’s based on the pixel division to the “background” and “foreground” classes, and minimising the inner-class dispersion:

where t is the threshold value, ω₁ and ω₂ are probabilities of pixels of each class, and σ₁ and σ₂ are variances of the two classes. The inner-class dispersion is computed for the every possible value of t, and the optimal one is used for thresholding.

Another method of the defect detection is edge detection filters. Edge detection filters are filters that activate on the local brightness leaps. The result of the filter application is the edge map. There are a lot of possible edge detection filters, but Canny and Gabor deserve most attention.

Canny edge detector uses vertically and horizontally oriented one-dimensional combinations of the Gaussian filter and it’s first derivative. After that, the gradient map is computed from those two maps and is used at the Non-Maximum Supression step, which sharpens and thins the gradients. The last step is a Hysteresis Thresholding, used to leave only strong edges and weaker edges directly connected to them.

Gabor filter is a Gaussian kernel function modulated by a sinusoidal plane wave. It’s suitable for highly detailed images with a quasi-periodical patterns, like fingerprints or nature photos.

Gabor’s frequency and orientation representations are similar to the human visual system, and it works better on natural images, while Canny detector is more common in medical and scientific computer vision applications.

The last introduced method of defects detection will be the Fourier Transform (FT). This method decomposes the input signal (e.g. an image) to a sum of harmonic functions. Some of operations over images are much easier to apply to the image’s FT rather than to the image itself, and considering the FT invertibility it is a very powerful instrument for a global image processing.

FT decomposition may be visualized as the magnitude and phase graphs. Phase component isn’t as useful as the magnitude is, so it’s often ignored. The visualisation result is often shifted to move the lower-frequency component to the center (on the raw image it will be shifted to the corners):

Lower frequency part of photo contains information about it’s general structure, while the information about edges and details is located at the higher frequency part. To emphasize edges, the lower frequencies must be suppressed. That can be done by applying a simple mask, representing the high-pass filter:

The main disadvantage of the Fourier Transform is that it’s computationally expensive, which may not let use it in the real-time systems.

As we can see, all of these three methods have their own pros and cons. Thresholding is very nice in splotch segmentation, while the High-passing detects point defects even in the complex circumstances, like the lighting gradient and noise brought by the matte surface, while Gabor detector combines sensitivity of the High-pass filter and ability to ignore unwanted objects like Thresholding does.

The classification task is a task of assigning a class (element of a finite set) to an input data point. The task prerequisite is a presence of a set of data points with correctly assigned classes — training set. A training set is used to build a classifier — a function realised by a mathematical model, assigning the classes to the data points.

The data point is a features vector. Feature is a measurable property of a phenomenon being observed. Since most of the models cannot work with the raw data (images, audio, etc) itself, before feeding the images to a model it’s needed to extract the feature vectors from them. In this article I will introduce one of possible ways to classify the glass defects — kNN model based on the shape descriptors.

Shape descriptors are numeric representations of shape properties [1]. Using different shape descriptors, we can numerically express how much the shape is “round” or “spiked”, the relation between it’s width and height, how evenly is the shape distributed from it’s geometrical center, etc.

Considering the shape descriptors variety, not all of them are necessary to be used as features for our model. It’s useful to reduce the number of features by combining or throwing away redundant ones. That may have a positive impact to the performance of the sensitive to the input data dimensionality model, that applies, for example, to the kNN model. The effect will be even more notable on smaller datasets, since the lesser-dimensional data points are closer to each other. So, the seven shape descriptors were selected for using in the future model:

All these shape descriptors are not dependent on the shape rotation, translation and proportional scaling. The eighth feature will be the defect area — to distinguish between similarly shaped splotches and point defects.

It’s supposed, that differend kinds of defects will have different values of the shape descriptors — points will be more round, splotches will have more irregular form, scratches will have very low values of Formfactor and Roundness.

The kNN model’s name means “k nearest neighbours”. According to the name, for a data point on the input the model looks for k closest data points in a training set. The most prevalent class between theese k is assigned to the input data point.

To define, which data points are closest, the model utilizes a metric on the data points space. The most common is Euclidean:

However, despite the prevalence of Euclidean metric, it’s likely to be needed to penalize dissimilarity by any feature more than L2 metric does. There may help other Minkowski Lp metrics with lesser p.

Since the metric value is influenced by the features relative scale, it’s useful to normalize every feature data to the range [0, 1]. Shape descriptors’ values are normalized by definition (the Aspect Ratio fraction is flipped relative to the one in The Image Processing Handbook to ensure this), so the only feature to apply normalization is the shape’s area.

So, the kNN model building pipeline:

2. Asssign a correct class to the every image

3. Segment the defect from every image to get a binary masks set

4. Compute shape descriptors and area of every mask

5. Normalize area values

6. Define a metric and define a k

After the model is built and tested, it will be used this way:

2. Extract the defect mask from it

3. Compute shape descriptors and area

4. Find the k closest data points to the input

5. The most prevailing class between those k will be the model output

At the actual moment I’m collecting data set made of image fragments containing defects, and working on tools to automate the process. After that, I can actually start the working on the classifier for practical use.

Work on the classifier will comprise metric choice, definition of the optimal k and optimizations to make the system work faster.

Another further step is to apply and compare some different algorithms, such as Spatial Pyramid Pooling CNN, to achieve more detailed classification. That will require data set extension, possibly using the data augmentation techniques, and some work on the neural network architecture designing.

[1] Russ, J.C., Brent Neal, F. (2016). The Image Processing Handbook. Seventh Edition. ISBN 978–1–4987–4026–5. p.624

Add a comment

What is your view?

Send

Related posts:

One Thousand Writing Steps

I am going to take 1,000 writing steps, one each day, for the next thousand days. I have no education or expertise in creating writing. I’ve only written one short story, but it got published! Thank…