Asian Journal of Physics Vol 29, Nos 10 – 12 (2020) 853-862

Scaling up low resolution noisy images in a multi-aperture imaging system

Suhita Tawade1,Suresh Panchal1, Rajeev Kumar3 and Unnikrishnan Gopinathan1,2
1Department of Applied Physics, Defence Institute of Advanced Technology, Girinagar, Pune-411 025, India
2Instruments Research & Development Establishment, Raipur Road, Dehradun-248 008, India
3Defence Institute of Advanced Technology, Girinagar, Pune-411 025, India
This article is dedicated to Prof FTS Yu for his significant contributions to Optics and Optical information Processing

A multi-aperture imaging camera is a computational imaging camera that uses a micro-lens array to achieve a smaller system volume as compared to a conventional camera without compromising the resolution and field of view. A micro-lens array with each micro-lens shifted from a regular grid form multiple, low-resolution images. A robust signal processing algorithm to generate a high-resolution image from the multiple low-resolution images generated by the micro-lens array forms an essential component of such a computational camera. In this paper, a robust image super-resolution algorithm for a multi-aperture imaging system using ℓ1 regularization is developed. The image formation in multiple imaging channels is modelled mathematically by taking into account the various degradations in the image formation process. The proposed super-resolution algorithm has two steps. In the first step the degradations in each imaging channel is corrected by solving the inverse problem of reconstructing the high-resolution image from a set of low-resolution images which is an ill-posed problem. The inverse problem is formulated as an optimization problem with a ℓ1 regularization term which is solved as an unconstrained problem with FISTA algorithm and as a constrained optimization problem with SALSA algorithm. In the second step, the images from the multiple channels are combined to generate a high-resolution image after correcting for sub-pixel shifts between the images thereby effectively achieving a sampling on a high-resolution grid. After applying the proposed algorithm on USAF resolution chart, perceivable improvement in the contrast of resolution line pattern as compared to low resolution image is observed. © Anita Publications. All rights reserved.
Keywords: Multi frame super-resolution, Multi-aperture camera, ℓ1 regularization, Super-resolution.
doi. 10.54955/AJP.29.10-12.2020.853-862

Peer Review Information
Method: Single- anonymous; Screened for Plagiarism? Yes
Buy this Article in Print © Anita Publications. All rights reserve

References

  1. Tanida J, Kumagai T, Yamada K, Miyatake S, Ishida K, Morimoto T, Kondou N, Miyazaki D, Ichioka Y, Thin observation module by bound optics (TOMBO): concept and experimental verification, Appl Opt, 40(2001)1806–1813.
  2. Kitamura Y, Shogenji R, Yamada K, Miyatake S, Miyamoto M, Morimoto T, Masaki Y, Kondou N, Miyazaki D, Tanida J, and Ichioka Y, Reconstruction of a high-resolution imageon a compound-eye image-capturing system, Appl Opt, 43(2004)1719–1727.
  3. Nitta K, Shogenji R, Miyatake S, Tanida J, Image reconstruction for thin observation module by iterative back-projection, Appl Opt, 45(2006)2893–2900.
  4. Shankar M, Willett R, Pitsianis N P, Kolste R Te, Chen C, Gibbons R, Brady D J, Ultra-thin multiple-channel LWIR imaging system, Proc SPIE, 6294(2006)629411; doi.org/10.1117/12.681386.
  5. Kanaev A V, Scribner D A, Ackerman J R, Fleet E F, Analysis and application of multi-framesuper-resolution processing for conventional imaging systems and lens-let arrays, Appl Opt, 46(2007)4320-4328.
  6. Kanaev A V, Ackerman J R, Fleet E F, Scribner D A, TOMBO sensor with scene-independent super-resolution processing, Opt Lett, 32(2008)2855–2857.
  7. Shankar M, Willett R, Pitsianis N, Schulz T, Gibbons R, Kolste R Te, Carriere J, Chen C, Prather D, and Brady D, Thin infrared imaging systems through multichannel sampling, Appl Opt, 47(2008)B1–B10.
  8. Portnoy A D, Pitsianis N P, Sun X, Brady D J, Multichannel sampling schemes for optical imaging systems, Appl Opt, 47(2008)B76–B85.
  9. Mendelowitz S, Klapp I, Mendlovic D, Design of an image restoration algorithm for the TOMBO system, J Opt Soc Am A, 30(2013)1193–1204.
  10. Gao Y, Yang P, Tang G, Xu B, Ao M, Experimental resolution comparison between the TOMBO and single lens systems, Proc SPIE 9227(2014), 92270B.
  11. Peleg S, Keren D, Schweitzer L, Improving image resolution using subpixel motion, Pattern Recogn Lett, 5(1987) 223–226.
  12. Irani M, Peleg S, Improving resolution by image registration, CVGIP: Graphical Models and Image Processing, 53(1991)231–239.
  13. Ur H, Gross D, Improved resolution from subpixel shifted pictures, CVGIP: Graphical Models and Image Processing, 54(1992)181–186.
  14. Elad M, Feuer A, Restoration of a single super-resolution image from several blurred,noisy, and under-sampled measured images, IEEE Trans Image Process, 6(1997)1646–1658.
  15. Hardie R, Barnard K J, Armstrong E E, Joint MAP registration and high-resolution image estimation using a sequence of under-sampled images, IEEE Trans Image Process, 6(1997)1621–1633.
  16. Chiang M C, Boult T E, Efficient super-resolution via image warping, Image Vis, 18(2000)761–771.
  17. Elad M, Hel-Or Y, A fast super-resolution reconstruction algorithm for pure translational motion and common space-invariant blur, IEEE Trans Image Process, 10(2001)1187–1193.
  18. Nguyen N, Milanfar P, Golub G H, A computationally efficient image super-resolution algorithm, IEEE Trans Image Process, 10(2001)573–583.
  19. Ng M, Bose N, Mathematical analysis of super-resolution methodology, IEEE Signal Process Mag, 20(2003)62–74.
  20. Park S, Park M, Kang M G, Super-resolution image reconstruction: a technical overview, IEEE Signal Process Mag, 20(2003)21–36.
  21. Farsiu S, Robinson M D, Elad M, Milanfar P, Fast and robust multi-framesuper-resolution, IEEE Trans Image Process, 13(2004)1327–1344.
  22. Young S S, Driggers R G, Super-resolution image reconstruction from a sequence of aliased imagery, Appl Opt, 45(2006)5073-5085.
  23. Shankar P M, Neifeld M A, Multi-framesuper-resolution of binary images, Appl Opt, 46(2007)1211–1222.
  24. Ashok A, Neifeld M A, Pseudorandom phase masks for super-resolution imaging from subpixel shifting, Appl Opt, 46(2007)2256–2268.
  25. Robinson M D, Stork D G, Joint digitaloptical design of super-resolutionmulti-frameimaging systems, Appl Opt, 47(2008)B11–B20.
  26. Choi K, Schulz T J, Signal-processing approaches for image-resolution restoration for TOMBO imagery, Appl Opt, 47(2008)B104–B116.
  27. Bishop T E, Favaro P, The Light field camera: Extended depth of field, aliasing, and super-resolution, IEEE Trans Pattern Anal Mach Intel, 34(2012)972–986.
  28. Young S S, Driggers R G, Jacobs E L, Signal Processing and Performance Analysis for Imaging systems, (Artech House, Boston), 2008.
  29. Milanfar P (Ed), Super Resolution Imaging, (CRC Press, Boca Raton), 2011.
  30. He K, Zhang X, Ren S, Sun J, Deep residual learning for image recognition, Proc IEEE conf on Computer Vision and Pattern Recognition, (2016)770-778.
  31. Dong C, Loy C C, He K, Tang X, Image super-resolution using deep convolutional networks, IEEE Trans Pattern Anal Mach Intel, 38(2016)295–307.
  32. Kim J, Kwon J, Lee K M, Accurate image super-resolution using very deep convolutional networks, Proc IEEE conf on Computer Vision and Pattern Recognition (CVPR), (2016)1646–1654.
  33. Lim B, Son S, Kim H, Nah S, Lee K M, Enhanced deep residual networks for single image super-resolution, IEEE conf on Computer Vision and Pattern Recognition (CVPR) Workshops, (2017)136–144.
  34. Tai Y, Yang J, Liu X, Image Super-resolution via deep recursive residual network, IEEE conference on Computer Vision and Pattern Recognition (CVPR), (2017)3147–3155.
  35. Candes E J, Fernandez-Granda C, Towards a mathematical theory of Super-resolution, Commun Pure Appl Math, 67(2014)906-956.
  36. Bertero M, Boccacci P, Introduction to inverse problems in imaging, (CRC Press) 1998.
  37. Barrett H H, Myers K J, Foundations of Imaging Science, (Wiley, )2003.
  38. Jin K H, McCann M T, Froustey E, Unser M, Deep convolutional neural network for inverse problems in imaging, IEEE Trans Image Process, 26(2017)4509-4522.
  39. Beck A, Teboulle M, A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J Imaging Sciences, 2(2009)183–202.
  40. Afonso M V, Bioucas-Dias J M, Figueiredo M A, Fast image recovery using variable splitting and constrained optimization, IEEE Trans Image Process, 19(2010)2345–2356.
  41. Afonso M V, Bioucas-Dias J M, Figueiredo M A, An augmented Lagrangian approach to the constrained optimization formulation of imaging inverse problems, IEEE Trans Image Process, 20(2011)681–695.
  42. Hansen P C, Nagy J G, O’Leary D P, Deblurring Images: Matrices, Spectra, and Filtering, (Society for Industrial and Applied Mathematics, Philadelphia). 2006.
  43. Guizar-Sicairos M, Thurman S T, Fienup J R, Efficient subpixel image registration algorithms, Opt Lett, 32(2008)156-158.
  44. Goodman J W, Introduction to Fourier Optics, 3rd edn, (Roberts and Company Publ), 2016.
  45. De K, Masilamani V, Image sharpness measure for blurred images in frequency domain, Procedia Engineering, 64(2013)149–158.
Scaling up low resolution noisy images in a multi-aperture imaging systemDownload

Related Posts