doi:10.1016/j.sigpro.2010.02.012 | How to Cite or Link Using DOI Copyright © 2010 Elsevier B.V. All rights reserved. |
Cited By in Scopus (0) |
Permissions & Reprints |
Fast communication
Ming Zhai^{a}^{, }^{}^{, }^{}^{, }^{} and Shan Fu^{b}^{, }^{}
Abstract
Quality inspection of aluminum foil products plays an important role for aluminum foil manufactures, but it is an arduous work for human. In our work, the target maneuver onset detection algorithms are applied to defect detection in aluminum foil images and the results are inspiring. It is assumed that the intensity of aluminum foil images is Gaussian distributed and the distribution of defect intensity is different from the normal. Under these assumptions Kalman filters with constant velocity (CV) model are used to filter defect images. During the filter process the maneuver onset detection algorithms are used to detect defects. The three maneuver onset detection algorithms, the measurement based chi-square detector (MR), the input estimate based chi-square detector (IE) and the input estimate based Gaussian significance detector (IEG) are tried respectively and the performance of the three algorithms are compared.
Keywords: Kalman filter; Input estimation; Measurement residual; Defects detection
Article Outline
- 1. Introduction
- 2. Preliminary
- 2.1. The Kalman filter and constant velocity model
- 2.2. The measurement residual based Chi-square detector (MR)
- 2.2. The input estimation based Chi-square detector (IE)
- 2.3. The input estimate based Gaussian significance detector (IEG)
- 3. Implementation
- 4. Performance comparison
- 5. Conclusion
- References
1. Introduction
Optical devices have been incorporated into mechatronic systems and many machine-vision based techniques have been used for quality control of products equipped with intelligence. In product quality control of aluminum foil manufacturing, quality inspection has to resort to machine-vision-based techniques for rigorous manufacture conditions. There are many machine-vision based quality control techniques. In Ref. [1], edges in images are used to detect defects in ceramic tiles. In the textile industry, Fourier analysis technique [2], morphology-based technique [3] and [8], wavelet-based technique [4] and [9], co-occurrence matrices and garbo-filter based technique [5] and [10] can be used to detect defects in fabric surface. In leather industry, oriented texture analysis is used to detect defects in leather [6]. In [7], the infrared light reflected by the metal sheets is used to analyze the defects of metal sheet. However, each defect detecting technique is designed for a special target, so those techniques are not suit for defects detection in aluminum foil. The defects detection in aluminum foil is an arduous work for human because defects often are small and similar to normal areas. Moreover the output of product is huge, sampled quality inspection cannot well satisfy the quality control demands. The real-time and automatic defect detection methods are urgently needed.The technique suitable for defect detection in aluminum foil should be designed according to the property of the aluminum foil product line. In our case the foil goes through the product line at nearly five meters per second. It means high resolution planar array cameras cannot be used, because high resolution planar cameras often have low frame rate and the redundant information in the image sequence will affect image processing speed. Therefore, a line scan camera is used. There are many types of defects in aluminum foil, some of which are shown in Fig. 1. Some defects have no apparent edges and the intensity of some defects is similar to normal areas (e.g. Fig. 1(A, D, E). Moreover some are very small (e.g. Fig. 1(C, E, G). So the edge-based technique and threshold-based binarization methods may not perform well. The texture-based frequency domain analysis methods cannot be used because there is no apparent texture information in aluminum foil images. It is more important that the speed of detecting process should be fast enough to satisfy the real time application. Furthermore the intensity of normal areas may vary (e.g. Fig. 1(E, H, K, M) and the sizes of defects are different (e.g. Fig. 1(B, C, F, I, L), these will bring troubles. Moreover, it must be considered that different types of defects often appear in one image. All these make defect detection difficult.
We collect a large number of foil images from the product line, and from these images the histogram of intensity is obtained. The histogram shows that the intensity of these images approximately obeys Gaussian distribution. Then we collect images with labeled defects and compute the histogram only using the pixels labeled as defects. The two histograms show that the defect intensity is different from the normal. The property can be used to detect defects in aluminum foil. Our contribution is that we apply the target maneuver onset detection algorithms in maneuver target tracking (MTT) field to solve the defect detecting problem. Three maneuver detectors, the measurement based chi-square detector (MR), the input estimate based chi-square detector (IE) and the input estimate based Gaussian significance detector (IEG), are tried and we obtain inspiring results. Our technique is that we assume the intensity of aluminum foil images is Gaussian distributed; Kalman filters with constant velocity (CV) model are used to filter defect images; defects are determined by the maneuver onset detectors during the filter process. Experiments show that our technique is effective for most types of defects and the processing speed can satisfy the real time applications.
This work is organized as follows: Section 1 the introduction; some basic notions are reviewed in Section 2; Section 3 the implementation; Section 4 performance comparison; Section 5 the conclusions.
2. Preliminary
In this section, we review some basic notions from the theory of Kalman filter, the constant velocity (CV) model, as well as the three target maneuver onset detectors, the measurement based chi-square detector (MR), the input estimate based chi-square detector (IE) and the input estimate based Gaussian significance detector (IEG), which we need in the sequel.
2.1. The Kalman filter and constant velocity model
The Kalman filter is a linear estimator based on the minimum mean squared error (MMSE) [11] and [12], for the sake of less computation and better performance it has been widely used since it is introduced. Here is a brief review of a Kalman filter.
Given a system equation and measurement equation as follows:
where x and z are state and measurement vectors respectively; v and w are process and measurement noise and are assumed to be Gaussian distributed with zero mean and covariance Q and R; u is the known input vector; F, H, and G are state, measurement and input matrixes respectively. F, G, H, Q and R are assumed known and possibly time-varying. The two noise sequences and the initial state are assumed mutually independent. The previous constituted the linear Gaussian (LG) assumption.
The main equations of a Kalman filter are as follows:
where the expression denotes the estimate of A at time k+1 estimated at time k; x and z are the state and measurement vectors; P and S are the state covariance and the innovation covariance matrixes; W and μ are the filter gain and the innovation (measurement residual).
In
the Kalman filter some kinetic models are often used to describe the
kinetic property of the target, like the constant velocity (CV) model
and the constant acceleration (CA) model [12].
Consider a target moving with constant velocity, the CV model is
suitable to describe the kinetic property of the target. In the CV
model, the state vector is x=(X,X^{′})^{T} where X is displacement, X^{′} is the differential of X, i.e., velocity. If the measurement is displacement then the measurement matrix is . Otherwise if the measurement is velocity then the measurement matrix is . In the CV model the state matrix is
2.2. The measurement residual based Chi-square detector (MR)
The
measurement residual based chi-square detector is often used to detect
target maneuver in maneuver target tracking (MTT) applications [12], [13], [14], [15], [16] and [17].
With the linear-Gaussian assumption, the measurement residuals of a
Kalman Filter are zero mean, Gaussian distributed and white; i.e., μ(k)N(0,S(k)) and
is a chi-square distributed variable, that is where n_{z}=dim(μ) is the dimension of vector μ. If
then decide a maneuver occurs, where 1−α is the confidence level. The decision which is based on a single sampling time in Eq. (4), can be replaced by a moving average (or moving sum) of the normalized innovations squared over a sliding window of s sampling times
The above is chi-square distributed with sn_{z} degrees of freedom. Alternatively, a fading memory average (also called exponentially discounted average)
where 0<β<1 and with initial condition ξ^{β}(0)=0, can be used. The variable ξ^{β} is approximately distributed as with n_{β}=(1+β/1−β)n_{z}. Its effective window length s is 1/1−β, and e.g. for β=0.95 one has s=20.
2.2. The input estimation based Chi-square detector (IE)
The
input estimation based chi-square detector is an algorithm to detect
target maneuver in maneuver target tracking (MTT) applications [12], [13], [14], [15], [16] and [17].
The detector consists of input estimation and chi-square significance
test. In the input estimation process two models are used, one is the
nonmaneuvering model (Eq. (8)) the other is the maneuvering model (Eq. (9)) with input u.
From the nonmaneuvering filter the input u can be detected and estimated. Denote the present time by k and assume that the target started maneuvering at time k−s, that is, the maneuver onset time is k−s. The unknown inputs during the interval [k−s,…, k] are u(i), i=k−s,…, k−1. The state from the mismatched nonmaneuvering filter based on Eq. (3) will be denoted by an asterisk:
with the initial condition , where W is the filter gain and Φ(i)=F[I−W(i)H].
If the inputs are known, the hypothetical correct filter based on Eq. (9) will yield estimates according to the recursion
The innovations corresponding to the nonmaneuvering filter (Eq. (10)) is
Combining Eqs. (10), (11), (12) and (13) yielding
Assuming the input to be constant over the interval [k−s,…,k−1], that is, u(j)=u, j=k−s,…, k−1, s is the window length, yielding
where
Based on Eq. (15) the input could be estimated via LS from
where is the estimate of the input u and
and ε, whose components are innovations (Eq. (13)), is zero mean with block-diagonal covariance matrix
where the covariance matrix of is
The
chi-square significance test is used to test the significance of the
estimation of input. A maneuver detection is declared if and only if
Eq. (21) is statistically significant. The significance test for the input is
then decide a maneuver occurred, where 1−α is the confidence level.
2.3. The input estimate based Gaussian significance detector (IEG)
The input estimate based Gaussian significance detector also is a target maneuver onset detection algorithm [18], [19] and [20].
It is different from the input estimate based chi-square detector that
it tests the significance of the input estimate by another statistical
significance test, the Gaussian significance test. When the input u is estimated using the method mentioned in Section 2.2, a maneuver detection can be declared if and only if a component of is statistically significant, that is
3. Implementation
The experiment setup is shown in Fig. 2. The lighting system is made of LED. The camera works at about 12 kHz with resolution 1024 in gray mode. A rotary encoder is used to generate grabbing signals for the frame grabber in order to keep the line-to-line scanning spacing at about 0.42 mm. The foil goes through the camera at about 5 m/s and its width is no more than 1550 mm.
The line scan camera scans the foil and in our case we constitute a image with every 768 scan line. Denote the image as I and denote the width and height of image I as M and N. In our case M=1024 and N=768.
From our statistics the intensity of the pixel in the images
approximately obeys Gaussian distribution. The distribution of the
defect intensity is different from the normal, so the defect area can
be detected by these maneuver onset detectors. Unlike the target
tracking, in our case image intensity of every row (column) is
considered as the measurement time sequence z in the measurement equation (Eq. (1)).
The measurement is considered as velocity and to be Gaussian
distributed. Under these conditions, CV model is used with measurement
matrix , state matrix
where T is a time interval. Since every column (row) is considered as a time sequence of the measurement of velocity in CV model, the Kalman filter can be used to filter the column from up to down. There are the N columns in image I so N Kalman filters are used respectively. Each column relates to one filter. In the filtering process of each column (row), the position where a maneuver occurs is considered to be a defect position. The defect position can be detected by any one of the three target maneuver onset detectors. The detection result is stored in the defect mask Σ, Σ is an M×N matrix, if the (i,j)th point in image I is detected to be a defect point by some detector then label the (i,j)th element of Σ with 1 otherwise label it with zero. Some defect masks obtained by MR detector are shown in Fig. 3. Moreover, some false detection results are shown in Fig. 4. In fact the false detection can be correctly detected by altering the parameters of the proposed method. The false detection do not often occur and the parameters should be set properly to make the proposed method cover most common defects so we choose the parameters suitable for most defects instead of the parameters suitable for a few defects.
If using MR detector to detect defects, during the filter process of every column (row) the measurement residual μ is calculated and tested by chi-square significance test to detect defects. If using IE or IEG detector, the input u should be estimated during the filter process and then the input estimate is tested by chi-square significance test or Gaussian significance test to detect defects.
It is found that the
position of the defect areas detected by each of the detectors will
deviate from the correct position of defects due to the detection delay
of the detectors so it will lead to imprecise location of defects. To
solve the problem, we detect image I from the four directions: up to down; down to up; left to right; and right to left. So 2M+2N filters are used. Denote Σ_{up}, Σ_{down}, Σ_{left}, Σ_{right} as the defect masks come from the detecting results of the four directions. Computed the logical OR of the four defect masks,
In the filter process, Kalman filter is used with measurement matrix and state matrix
where R is the variance of measurement noise. R should be set a bit larger than the variance of the intensity of the image I to obtain better performance. In our case we set R=1.2R_{I} where R_{I} is the intensity variance of image I. We assume there is no process noise, that is, Q is zero and experimental results prove this assumption is proper. In the MR detector, G is assumed to be zero, that is, no input is considered and in the other two detectors we set
4. Performance comparison
4.1. Performance of the three detectors
Define the correct detection rate (CDR), the fake alarm rate (FAR) and the missed detection rate (MDR) as follows:
Firstly we detect the defects in the defect image database which contained 10 types of defects with the three detectors with the same initial parameters respectively. In the experiments the fading memory average window is used. The CDR, FAR and MDR of the detection result of the three detectors are presented in Table 1 respectively. And then we test the three detectors in the real images obtained from the foil product line respectively and compared the detecting results with the results obtained by the skilled quality inspectors. The CDR, FAR and MDR of the results are similar to Table 1. Table 1 presents that the three detectors are all effective for defects detection and the difference of their performance are small.
The CDR, FAR and MDR of the three detectors at different window length.
Detection method | Window length(s) |
|||
---|---|---|---|---|
s=1 | s=3 | s=5 | s=7 | |
MR | ||||
CDR | 0.932 | 0.929 | 0.927 | 0.919 |
FAR | 0.052 | 0.049 | 0.044 | 0.043 |
MDR | 0.068 | 0.071 | 0.073 | 0.081 |
IE | ||||
CDR | 0.937 | 0.933 | 0.930 | 0.923 |
FAR | 0.049 | 0.044 | 0.040 | 0.037 |
MDR | 0.063 | 0.067 | 0.070 | 0.077 |
IEG | ||||
CDR | 0.935 | 0.934 | 0.931 | 0.920 |
FAR | 0.051 | 0.045 | 0.042 | 0.036 |
MDR | 0.095 | 0.066 | 0.069 | 0.080 |
4.2. Computation complexity
Because no serious attempt is made to optimize each detector beyond the obvious (for example, caching matrix inversion results, vectorizing operations and caching recurring information from sample to sample), we carry out the three detectors to detect defects in the images with size of 1024×768 at different window length on a PC with Intel E4500 CPU (2.2 GHz) and calculated the average processing time as an indicator of relative computational complexity of the three detector. The results are shown in Fig. 5.
Due to the simplicity of MR, it is least computationally intensive. The IE and IEG detectors require almost the same processing time, it is because the two detectors need the same input estimate process and the significance test functions are similar. The computation complexity of the MR detector is independent of the window length while the computation complexity of IE and IEG detector will increase with the window length. Due to noise it will increase false alarm to detect defect from one sample so a window is needed to obtain a moving sum to against noise. Small window length has less ability against noise while large window length will increase miss detection. So we compare the performance of different window lengths and find that it is proper to set the window length at 3 or 5 in our system.
5. Conclusion
We try the three target maneuver onset detectors, the MR, IE and IEG detectors, to detect defects in aluminum foil and experiments show that all the three detectors are effective for defects detection in aluminum foil. We also compare the computation complexity of the three methods on a PC with Intel E4500 CPU. Of the three detectors, the MR detector is least computationally intensive. The IE and IEG detectors have almost the same computation complexity. In Section 3, we detect the image from the four directions, the four detecting processes of the four directions are mutually independent, so the four processes can be carried our in parallel. If in parallel model, the process time can reduce three quarters and therefore it makes our technique practicable for real time applications. In our experiment the parameters are set to be suitable for detecting the defect more than five pixels and the defect less than 5 pixels will be ignored. If the parameters are set to be suitable for the defect less than 5 pixels and then the false alarm will burst due to noise. Fortunately the defects more than 5 pixels take up a major proportion in our case so the proposed method is applicable. The system is designed to work at 5 m/s. When the speed of foil exceeds 7 m/s, for the computation limit of the hardware, the detecting system will halt. It is a problem we will face in the future because our product line will be updated soon and we plan to solve this problem by using more powerful computer or using parallel computing methods. The defect detection is only the first step of our quality inspection system. After the defects are detected, it is required to recognize the detected defects and it will be our future work.