Non-Uniformity Correction With Temperature Influence Compensation

Non-uniformity correction with temperature influence compensation in microbolometer detector Michał Krupiński *, Grzegorz Bieszczad, Sławomir Gogler, Henryk Madura Institute of optoelectronic, Military University of Technology, Gen. Sylwestra Kaliskiego 2a, 00-908 Warsaw, Poland ABSTRACT Because of a significant impact of the microbolometer array temperature on the infrared image quality, it is necessary to compensate the influence of the temperature on the NUC process. In the most common applications two approaches are used: the first is a stabilization of the microbolometer array temperature by a thermoelectric cooler, the second is updating correction coefficients obtained from reference source, for example a shutter [14]. Both of the most common approaches have theirs disadvantages. The first case needs a considerable amount of energy for temperature stabilisation. The second one needs a reference target and a mechanical procedure to place the target at the front of the detector. Additionally, during calibration the reference target is blocking radiation from the scene, thus interrupting measurements with the thermal camera. In the article a non-uniformity correction method is presented which allows to compensate for the influence of detector’s temperature drift. For this purpose, dependency between output signal value and the temperature of the detector array was investigated. Additionally the influence of the temperature on the Offset and Gain coefficients was measured. Presented method utilizes estimated dependency between output signal of detectors and their temperature. In the presented method, the dependency between output signal value and the temperature of the detector is estimated during time of starting detector. The coefficients are estimated for every pixel. In the article proposed method allows to compensate the influence of detectors temperature fluctuation and increase a time between shutter actuation process. Keywords: non-uniformity correction, tec-less micro bolometer, shutter-less correction, 1. INTRODUCTION Microbolometer belongs to the group of thermal detectors and consist of temperature sensitive resistor which is exposed to measured radiation flux. Bolometer array employs a pixel structure prepared in silicon micromachining technology (MEMS). The detecting area is defined by a size of thin membrane, usually made of amorphous silicon (aSi) or vanadium oxide (VOx). Detecting area is coated with material having high absorption coefficient. Resistive bolometer changes its electrical resistance with temperature due to absorption of electromagnetic radiation [1]. The trend of resistance change depends on material on the active area. If the resistance increases with increasing temperature (metal) the bolometer is said to have a positive temperature coefficient, otherwise it is said to have negative temperature coefficient (semiconductors). The resistance of a bolometer is measured by electronic readout circuit and converted to output voltage signal. * email: mkrupinski@wat.edu.pl, phone: +48226839383, fax: +48226668950. Image Sensing Technologies: Materials, Devices, Systems, and Applications II, edited by Nibir K. Dhar, Achyut K. Dutta, Proc. of SPIE Vol. 9481, 948113 © 2015 SPIE · CCC code: 0277-786X/15/$18 · doi: 10.1117/12.2184645 Proc. of SPIE Vol. 9481 948113-1 Downloaded From: http://proceedings.spiedigitallibrary.org/ on 05/19/2015 Terms of Use: http://spiedl.org/terms a) b) Thermal isulation IR Absorpsion 0.1 µm Interconnetion 2.5 µm Thermometer α Si:H Thermal isulation Readout circuit pad Reflector Readout circuit Substrate Signal Fig. 1. Single microbolometer (a) and principal of its operation (b) Fig. 1 presents structure of a single microbolometer detector and its principle of operation. FPAs are made of a multitude of detector elements (for example 640 × 480 ), where each individual detector has different responsivity and offset due to detector-to-detector spread in the FPA fabrication process [2], and additionally can change with sensor operating temperature, biasing voltage variation or temperature of the observed scene [11]. The difference in responsivity and offset among detectors (which is called non-uniformity) additionally with its high sensitivity, produces fixed pattern noise (FPN) on produced image. Fixed pattern noise degrades parameters of infrared cameras like sensitivity or NEDT. Additionally it degrades image quality, radiometric accuracy and temperature resolution. 2. PROCEDURE OF A NON-UNIFORMITY CORRECTION A non-uniformity correction (NUC) is a digital process of removing fixed pattern noise from the image using digital signal processing techniques on detectors' output signal. Fixed pattern noise removal process requires knowledge of correction coefficients for every detector in the array [3]. The calibration-based technique is the simplest, most accurate and most common NUC method used to correct for non-uniformities [10]. Calibration-based methods include single point correction (SPC), two point correction (TPC), multiple point correction (also known as piecewise linear correction) and the improved two point correction algorithm. In these methods, the gain and offset parameters are estimated by exposing the FPA to different uniform blackbody temperatures depending on the algorithm adopted [5]. In multiple point correction the detector is exposed to several black bodies at different temperatures to divide nonlinear characteristic to piece–wise linear sections. This approach requires a significant increase in the capability of the NUC electronics and a precise calibration source [5]. The next group of non-uniformity correction is a group of scene-based algorithms. The Scene-based algorithms are divided into two types [7]: - statistical methods, - registration-based methods. The constraints for the statistical methods are the temporal statistics are roughly constant for each pixel. The constant statistics method assumes that the temporal means and variances are identical for all pixels [6]. This constraint assumes that the detectors’ array is periodically moving in the real-world and that the average statistics of each pixels’ sees should be constant when averaged over a very long time. In pathological situations where humans or machines are forced to stare at a single static scene for a long time, this assumption is violated. Additionally this method requires a large number frames for the NUC coefficients calculation. The accuracy of the correction strongly depends on the number of analysed images. The second one is a registration base group. This type of algorithms do not use or require any statistical assumptions about the scene and could estimate the nonuniformity in a much smaller number of image frames. These methods consider that each detector should have identical response when observing the same Proc. of SPIE Vol. 9481 948113-2 Downloaded From: http://proceedings.spiedigitallibrary.org/ on 05/19/2015 Terms of Use: http://spiedl.org/terms scene point [8, 9]. Therefore, registration based methods require accurate estimation of the motion between frames. All scene-based nonuniformity correction algorithms require that the objects in the image do not remain stationary for too long. This can be accomplished by either periodically moving the camera or else requiring objects in the scene to move [7]. If an object in the image violates this assumption and remains stationary for a large number of iterations, the object will blend into the background. If this stationary object eventually moves from the field of view, it will leave a reverse ghost image in the scene. First group of scene-based algorithms requires a large number of frames. In the second one there is a high risk of ghosting effect. The benefit of using scene-based algorithms is no need for calibration during manufacture. It allows for correcting the gain and the offset errors without using special calibration images and procedures. In practise a shutter is not used in the camera. In comparison to scene-based algorithms the calibration-based technique is more dependable, simplest and most accurate. There is no ghost effect or another risk like algorithm faults. The one of its disadvantage is calibration with use of a reference target which is indispensable. Additionally those methods need adequate amount of memory space for storing correction coefficients table depending on the number of calibration points. For this reasons the most commonly used algorithm is a two point correction method. This method allows to correct for both responsivity variation and offset spread, and is described by equation: U ij* (Φ ) = GijU ij (Φ ) + Oij , (1) * where Uij(Φ) is response of i,j detector before correction, U ij(Φ) is response after correction, Gij i Oij are GAIN and OFFSET correction coefficients of detectors characteristic. Correction coefficients GAIN and OFFSET are described by: Gij = U ( Φ 2 ) − U ( Φ1 ) U ij (Φ 2 ) − U ij (Φ1 ) , Oij = U (Φ1 ) − GijU ij (Φ1 ) (2) , (3) where Uij(Φ1) i Uij(Φ2) are responses of a single detector i,j to radiation flux coming from a black body at a temperature of T1 and T2, U(Φ1) i U(Φ2) are mean value of detector response calculated by formula: U (Φ ) = 1 MN M N ∑∑U i =1 j =1 ij (Φ ) , (4) where Uij(Φ) represents response of detector i,j to radiation flux Φ from black body, M is a number of rows, and N is a number of columns in detector array. The idea of the non-uniformity correction is presented in Fig. 2. a) b) c) Fig. 2. Characteristics of detectors: before NUC correction a), after one point correction b), during two point correction. Fig.2a represents a spread of the microbolometers’ characteristics before non-uniformity correction. The second image illustrate characteristics after one-point correction. In this case only a shift of the characteristics are corrected. In practise this correction are executed inside the camera by use of mechanical part called Proc. of SPIE Vol. 9481 948113-3 Downloaded From: http://proceedings.spiedigitallibrary.org/ on 05/19/2015 Terms of Use: http://spiedl.org/terms shutter. The last image represents microbolometers’ characteristics during two point correction with use of two reference black bodies. In this case the shift and a slope of the characteristics are corrected. Most often the GAIN table coefficients are calculated at test bench in laboratory. After calculation, detectors coefficients are stored in non-volatile memory in a camera and process of correcting output signal is performed by the camera in real-time. 3. INFLUENCE OF MICROBOLOMETER ARRAY TEMPERATURE ON INFRARED IMAGE The temperature of a detector array has important impact on a value of output signal and its parameters like NETD [3]. Therefore even insignificant change of detector temperature has a crucial influence on quality of non-uniformity correction process. Fig. 3a represents an image after two point correction for temperature of array detector equal to 29°C. Nonuniformity of this image is 2.7°mV. Detector temperature change of 3°C causes deterioration of nonuniformity correction process and as a result four times larger FPN (Fig. 3b ). Because of significant impact of a microbolometer array temperature on infrared image quality, it's necessary to compensate for the influence of the temperature on non-uniformity correction process. In the most common applications two approaches are used. The first of them is microbolometer array temperature stabilization by thermoelectric cooler with a special controller. The second is updating offset coefficients by using a shutter. Both of them have disadvantages. The first case needs considerable amount of energy. The second one needs reference target and some mechanical procedure to place the target at the front of the detector. a) b) 8300 8200 8200 8100 8100 8000 8000 ]900 ]900 ]800 ]800 300 ]]00 00 200 200 400 200 00 00 00 00 300 200 00 400 Fig. 3. Detector array signal after NUC at detector temperature equal to Tfpa=29°C (a), and after temperature drift of detector by 3°C with the same NUC coefficients Additionally during calibration the reference target blocks radiation from the scene interrupting measurements with such thermal camera. a) b) 0662E1. Fig. 4. GAIN (a) and OFFSET (b) influence on images non-uniformity as a function of detector array temperature. The NUC coefficients are calculated for a specified temperature of a detector. Therefore it is very important to find out what is the relationship between the NUC coefficients and non-uniformity of a corrected image. Impact of the detector array temperature on correction quality were specified on the basis of registration Proc. of SPIE Vol. 9481 948113-4 Downloaded From: http://proceedings.spiedigitallibrary.org/ on 05/19/2015 Terms of Use: http://spiedl.org/terms prepared at a laboratory test bench with an infrared camera and a black body [14]. Temperature of the microbolometer detector was set and stabilized by a thermoelectric cooler. Registration was performed for three temperatures of the black body and four temperatures of the infrared detector. It allowed to estimate changes in Residual Non uniformity (RNU) (5) as a function of the detector array temperature. RNU = S k ⋅ 100% Y Sk = 1 MN , (5) ∑∑ [Y − X ] M N 2 ij i =1 j =1 , (6) where Sk – standard deviation of the image defined by equation (6), Y – mean value of the image, Xij - value of output signal for bolometer i,j. The research shows that differences between following value of the RNU are rather inconsistent, for a constant GAIN table (Fig. 4a). Additionally value of calibration points used to calculate GAIN coefficients has bigger impact on the RNU value than a temperature drift of detector. The same test was performed for the OFFSET table coefficients (Fig. 4b). It can be seen that changes in detector temperature have much bigger influence on the RNU when the OFFSET coefficient are constant. This result allow to conclude that temperature of detector has more significant impact on the OFFSET coefficients than a GAIN. 4. NON-UNIFORMITY CORRECTION WITH TEMPERATURE DRIFT COMPENSATION In the proposed method a compensation of influence of microbolometer array temperature on image nonuniformity, bases on calculation of additional coefficients which allows to compensate for signal error caused by temperature drift. In the proposed method one needs to determine three coefficients tables: GAIN, OFFSET and drift compensation coefficients (DCC) of the detector array. The GAIN coefficients are determined at a test stand and are stored in the camera memory. Only the OFFSET table and DCC are calculated in the camera. The OFFSET is calculated base on radiation flux coming from reference body, called shutter, installed in the camera, temporarily blocking radiation from the scene. The OFFSET tables are determined at three different temperatures of the detector during natural heating up of the detector after switching the camera on. Power consumption of the bolometer array and a readout circuit cause natural heat up by 3-4°C. In this case user doesn’t need to use thermoelectric cooler or heater to change temperature of FPA. Temperature change after start of the detector is presented at Fig.5a. OFFSET tables are used to calculate DCC. DCC is non-linear approximation of OFFSET signal as a function of temperature of the detector array. unwpcL a b Fig. 5 Increase of FPA temperature during start (a) and detectors signal dependency of detector array temperature (b) Proc. of SPIE Vol. 9481 948113-5 Downloaded From: http://proceedings.spiedigitallibrary.org/ on 05/19/2015 Terms of Use: http://spiedl.org/terms The characteristic of output signal as a function of detector temperature is presented at Fig. 5b. It is assumed that the characteristic of the signal change as a function of detector temperature can be approximated by nonlinear function in the range of -5°C +5°C around detector temperature measured after start of the camera. EIeq1.ou EXj61U9I Fig. 6. Model of infrared detector with lens. Compensation of influence of the temperature drift on output signal is described by formula: ~ U ij (Φ ) = U ij* (Φ ) + DCCij o TC , (8) ~ * where: U ij (Φ ) - output signal of bolometer i,j after compensation, U ij (Φ ) - value of output signal before ~ compensation, TC- actual detector array temperature, Gij - compensation coefficient of second order approximation for signal error as a function of detector temperature: DCCij ≅ I err \ Tub , (9) where: I errTub = U ij (Tub ) − U ij (Tub ) , (10) Uij(Tub), U ij (Tub ) – response of detector i,j on constant reference radiation for temperatures set of detector array at Tub and mean value for all detectors. The equations takes under consideration only temperature of the detector. But in fact the changes of environment temperature effects the temperature of the detector and temperature of the lens. In this case detector is exposed to radiation coming from observed scene (or shutter during calibration), and lens (Fig.6). The influence of the cases temperature on the output signal of the detector can be measured in climatic chamber by changing environment temperature with constant temperature of the detector and a black body. Table. 1 Detector responsivity Average responsivity Test condition Value [mV/ K] Black Body Temperature Integration Time=120µs Tbb1= 20 Tbb2= 35 Tub = 28 Tenv= 28 15 FPA Temperature Integration Time=120µs Tbb = 20 Tub1 = 22 Tub2 = 28 Tenv = 25 -49 Proc. of SPIE Vol. 9481 948113-6 Downloaded From: http://proceedings.spiedigitallibrary.org/ on 05/19/2015 Terms of Use: http://spiedl.org/terms Case Temperature Integration Time=120µs Tbb= 35 Tub = 30 Tenv1= 28 Tenv2= 32 27 The temperature of the FPA was stabilised by thermoelectric cooler. In this case the temperature of the lens was the one variable in the camera. In the Tab.1 we can compare the responsivity of the detector to three temperatures: black body, FPA and case. The influence of case temperature is presented at Fig. 7. At picture 7a the influence of the case temperature on corrected output signal value is presented. For a test all of the registrations were corrected by OFFSET calculated at the base of registration where the temperature of the case was equal to 32°C. The picture 7b shows RNU value according to the same conditions. Each and every detector is exposed to radiation flux coming from scene and case. But the amount of energy coming to detector depends on detectors' coordination (x, y) in FPA [13]. 0ll26f COLL6Cj iceas Colnwu jc926 unwpsi a b Fig. 7. Influence of lens temperature on output signal a- signal change, b - RNU change This effect may be reduced by calculating table coefficient which allows to determine signal error caused by temperature change of the lens (environment). Procedure of determination of "lens coefficient" LC consists of registration two images where each of them represent radiation of different lens' temperatures (Tcase1=24, Tcase2=32) . The black body temperature Tbb and micro bolometer temperatures Tub must be constant. The procedure of calculation is represented by formula (11) : LCij = ( ( ( ( U ij (Tcase 2 ) − U ij (Tcase1 ) Tcase 2 − Tcase1 ⋅ (Tub − TNUC ) (11) ,where U ij (Tcase1 ) , and U ij (Tcase 2 ) : ( U ij (Tcase 2 ) = U (Tcase 2 ) − U ij (Tcase 2 ) (12) ( U ij (Tcase1 ) = U (Tcase1 ) − U ij (Tcase1 ) (13) ( ( LCij - lens coefficient for each detector, U ij (Tcase 2 ) - signal error for temperature Tcase 2 , U ij (Tcase1 ) - signal error for temperature Tcase1 , U (Tcase ) - mean signal value, Tub -actual detector temperature, TNUC - detectors temperature during last NUC correction. The result of lens temperature influences compensation is presented at Fig. 8. The plot with black line with circle dots represents RNU after compensation, the blue one is a reference RNU before compensation. Proc. of SPIE Vol. 9481 948113-7 Downloaded From: http://proceedings.spiedigitallibrary.org/ on 05/19/2015 Terms of Use: http://spiedl.org/terms OFF26F 10926 coinwu a b Fig. 8. Influence of lens temperature on output signal after correction a- signal change, b - RNU change 5. MEASUREMENT RESULTS Presented method was tested in a laboratory environment with the use of an infrared camera, three black bodies, data acquisition card, climatic chamber and a computer ( Fig. 9). Fig. 9. Test stand for verifying proposed method where: 1- micro bolometer with electronic readout circuit, 2- lens, 3- climatic chamber, 4- TEC driver, 5- computer with frame grabber, and control software, 6-black body, Tub- detector temperature, Tenv- environment temperature, Tcase- case (lens) temperature, Tbb- black body temperature. Black bodies are used to determine GAIN and OFFSET coefficients and for the method evaluation purposes. Temperature of detector array was set by thermoelectric module placed inside the microbolometer array package. Tested detector allowed reading measurement data together with the temperature of detector array owing to temperature sensor inside. All images were captured through Orange-Tree evaluation board with special application designed by our team [4]. Quality of uniformity improvement by proposed algorithm was measured with use of RNU (11). Fig. 10 compares images RNU after NUC correction to RNU after proposed correction method. A temporal noise value is 3.3. For the test only detectors temperature was changed. Proc. of SPIE Vol. 9481 948113-8 Downloaded From: http://proceedings.spiedigitallibrary.org/ on 05/19/2015 Terms of Use: http://spiedl.org/terms RNU [%] DCC T 1 ub1 =22, T =23, T =24 [C] ub2 ub3 One point NUC DCC T 0.75 DCC T ub1 =22, T =24, T =26 [C] ub2 ub3 ub1 =22, T =24, T =28 [C] ub2 ub3 0.5 0.25 0 22 23 24 25 26 27 28 29 Tub Fig. 10. RNU value of image as a function of detector array temperature. It should be noted in fig. 10 that the wider span of coefficient points we get the wider temperature range compensation works. Additionally we can compare how RNU changes after involving environment temperature (Fig. 11a). Fig. 11a shows how RNU changes after environment and micro bolometer temperatures drift. Environment temperature deteriorates corrections result. Involving lens coefficients can slightly expand temperature bou mc1 a b Fig. 11. RNU value of image as a function of detector array and environment temperature. range without needs of NUC actuation with shutter. Figure 11 b shows correction results after involving temperature influence compensation (DCC), and lens coefficients (LC). We can see that proposed method allows to maintain non uniformity of the images at lower level in wider environment temperature. It allows for longer use of camera without NUC actuation. The compensation coefficient table was calculated for three ranges of the detector temperatures which allows to research its influence on compensation quality. The narrower the range of the detector’s temperature, the smaller the value of the non-uniformity in compensated image. The result of proposed correction is presented at Fig. 12. Proc. of SPIE Vol. 9481 948113-9 Downloaded From: http://proceedings.spiedigitallibrary.org/ on 05/19/2015 Terms of Use: http://spiedl.org/terms 50 100 150 200 250 50 100 150 a 200 250 300 350 b Fig. 12. Image after 2-point NUC (a) and DCC correction (b) 6. CONCLUSION In the article a non-uniformity correction method is presented which allows to compensate for the influence of detector’s temperature drift. For this purpose, dependency between output signal value and the temperature of the detector array was investigated. Additionally the influence of the temperature on the Offset and Gain coefficients was measured. The research allows to conclude that the detector’s temperature has more impact on OFFSET coefficients than on GAIN. Presented method utilizes estimated dependency between output signal of detectors and their temperature during start of the camera. In the presented method, the shutter is used for establishing signal reference. Prepared measurements indicates that natural heating during start of the detector maybe used to establish compensation coefficients and increase time between shutter actuation. Additional LC coefficients calculated at the test stand improves non uniformity caused by cases temperature change. This scientific work was financed by The National Centre for Research and Development with grant number LIDER/015/479/L-4/12/NCBR/2013 REFERENCES [1] Kruse, P. W., [Uncooled Thermal Imagining], The Society of Photo+Optical Instrumentation Engineers (2002). [2] Milton, A. F.; Barone, F. B.; Kruer, M. R. “Influence of nonuniformity on infrared focal plane array performance,” Optical Engineering, Vol. 24, No. 5, pp. 855-862, 1985 [3] Orżanowski, T., Madura, H., Kastek, M., Sosnowski, T., “Nonuniformity correction algorithm for microbolometer infrared focal plane array,” Advanced Infrared Technology and Applications AITA 9, Leon (8-12.10.2007) [4] Sosnowski, T., Bieszczad, G., Madura, H., Kastek, M., Firmanty, K., “The calibration stand for thermal , camera module with cooled infrared focal plane array,” Proceedings of SPIE, Vol. 7660, 2010, No. 76603Y [5] Mudau, A. E., Willers, C. J., Griffith, D., Roux, F. P. J. L., “Non-Uniformity Correction and Bad Pixel Replacement on LWIR and MWIR Images Defense,” Peace, Safety and Security Council for Scientific and Industrial Research, CSIR [6] Harris, J., Chiang, Y., “Nonuniformity correction of infrared image sequences using the constant statistics constraint,” IEEE TIP 8, 1148–1151 (August 1999) Proc. of SPIE Vol. 9481 948113-10 Downloaded From: http://proceedings.spiedigitallibrary.org/ on 05/19/2015 Terms of Use: http://spiedl.org/terms [7] Chao, Zuo., Qian, Chen., Guohua, Gu., Xiubao, Sui., Weixian, Qian., “Scene-based nonuniformity correction method using multiscale constant statistics,” SPIE Optical Engineering, Vol 50, Issue 08, August 2011 [8] Hardie, R.C., Hayat, M.M., Armstrong, E., Yasuda, B., “Scene-based non-uniformity correction with video sequences and registration,” Appl. Opt. 39 (2000) 1241– 1250. [9] Chao, Zuo, Qian, Chen, Guohua, Gu, Xiubao Sui, Jianle Ren., “Improved interframe registration based nonuniformity correction for focal plane arrays,” Infrared Physics & Technology 55 (2012) 263–269 [10] Hayat, M.M., Ratliff, B.M., Tyo, J.S., Agi, K., “Generalized Algebraic Algorithm for Scene-based Nonuniformity Correction,” proc. SPIE, 5556, 122-136, 2004. [11] Bieszczad, G., Kastek, M., “Measurement of thermal behavior of detector array surface with the use of microscopic thermal camera,” MMS Volume 18, Issue 4, 2011, Pages 679-690 [12] Bieszczad, G., Kastek, M., “Measurement of thermal behavior of detector array surface with the use of microscopic thermal camera,” MMS Volume 18, Issue 4, 2011, Pages 679-690 [13] Gogler, S., Bieszczad, G., Krupiński, M., Zarzycka, A., “A method of determination of voltage sensitivity of microbolometric focal plane array with optical system radiometric considerations,” Przegląd Elektrotechniczny, Volume 89, Issue 10, 2013, Pages 65-68 [14] Krupiński, M., Bieszczad, G., Sosnowski, T., Madura, H., Gogler,S., “Non-uniformity correction in microbolometer array with temperature influence compensation,” Metrology and Measurement Systems., Vol. XXI (2014), No. 4, pp. 709–718. Proc. of SPIE Vol. 9481 948113-11 Downloaded From: http://proceedings.spiedigitallibrary.org/ on 05/19/2015 Terms of Use: http://spiedl.org/terms