CMOS Time-of-Flight Range Image Sensors Shoji KAWAHITO This article describes CMOS time-of-flight TOF range image sensors and their pixel devices, so-called lock-in pixels. The CMOS-TOF range image sensors are categorized into two types; direct and indirect TOF measurement types. The direct TOF measurement uses a direct measurement of photon arrival timing using a single photon avalanche diode SPAD and a precise time-to-digital converter. In the indirect TOF measurement, the TOF is estimated by a mechanism of time-dependent photo charge modulation. Sinusoidal or pulsed light modulations are used for phase or delay measurements by demodulating it in the lock-in pixels. Lock-in pixels are realized with CCD-like devices implemented based on CMOS technologies or device structures compatible with standard CMOS image sensor technology or a little modification of it. Recent trend shows that pinned photodiodes based lock-in pixels in standard CMOS technology are becoming popular. This article discusses the principle, operation and performance of these TOF range imaging methods and devices and their performance limitations. Key words: time of flight measurement, single photon avalanche diode, charge demodulation pixel, lock-in pixel, CMOS image sensor CMOS time of flight TOF CMOS complementary metal oxide semiconductor CMOS-TOF CMOS TOF TOF TOF TOF TOF SPAD single photon avalanche diode TDC time to digital converter TOF CMOS-TOF 1. TOF TOF 100 ps TOF 432 8011 3 5 1 E-mail: kawahito@idl.rie.shizuoka.ac.jp 262 2
1 SPAD single-photon avalanche diode a b c SPAD pn TOF SPAD single photon avaranche diodespad 1 1 SPAD V bd 1 SPAD V bd p MOSFET quenting CMOS 1 1 1 1 SPAD 1 41 5 2012 SPAD 1 c CMOS n-well p pn p p-well TOF TDC SPAD CMOS TDC 2 SPAD TDC 1 a S TDC 2 TDC RS SPAD RS TOF TOF 1 mm 263 3
3 2 TDC a b TOF 6.6 ps 150 GHz CMOS 2 TDC coarse 1 fine TDC TDC 1 10 ps 10 ps TOF 32 32 150 ps 1.8 mm 1 SPAD TOF 32 10 TDC 128 128 TOF 2 10 TDC 3 TDC DLL delay locked loop 2 SPAD TOF TDC 264 4 TDC SPAD TOF 2. TOF TOF TOF 1995 3 4 CCD 3 2 1, 2 2 1, 2 2 2 2 2 1 2 L t d c L 1/2 ct d 2
4 4 4 1 4 4 4 1 4 a 0, a 1, a 2, a 3 j ϕ arctan a a a a 0 2 1 3 1 t d j /2p f m f m 4 4 4 B 1 4 4 4 TOF 5 1 CMOS 5 CCD CCD- CMOS PGM PGL PGR PGL PGR IG 4 1 41 5 2012 5 1 3 a 0, a 1, a 2, a 3 64 25 65 mm 21 mm TOF 2.5 pw 20 MHz 2 cm 5 2-tap 0.6 mm 176 144 40 mm 40 mm TOF 6 2-tap 5 PGR IG OUTG 1.2 m 5 mm CCD CMOS TOF CMOS 6 CMOS CMOS-TOF 7 b 3 1 n FD1, FD2 ktc 6 a TXD 265 5
7 6 CMOS a b X-X 7 7 TX1 TX2 PG FD1, FD2 TXD CMOS 320 240 15 mm 15 mm CMOS-TOF 25 ns 30 frames/s 1 V 5 mm 10 3 frames/s 2 mm CMOS 2 PMD photo mixing device 2-tap 204 204 1 8 a 2 2-tap p- 8 CMOS 8 CMOS a photogate b pinned photodiode 1 http://www.youtube.com/user/pmdtec 266 6
9 pinned photodiode 9 10 8 b 2-tap TOF 4 1 2 L ct d /2 cj /4p f m 2 1 4 B A A B N s A 2N s L N b N r i 3 L ai a 0 L L 8 N s B B N N N 3 9 10 1000 1 5 m 5 mm 41 5 2012 i 2 B 3 A N b 2 s b r 5 CCD IG FD CDS correlated double sampling 6 8 CMOS N reset N reset 2 V V 4 V th 26 mv V s N well V s 1 V N well 200,000 N well 100 4 2 CMOS TOF CCD 9 3. CMOS-TOF LED 267 7 th s N well
TOF TOF TOF TOF 1 C. Niclass, A. Rochas, P.-A. Besse and E. Charbon: Design and characterization of a CMOS 3-D image sensor based on single photon Avalanche diodes, IEEE J. Solid-State Circ., 40 2005 1847 1854. 2 C. Niclass, C. Favi, T. Kluter, M. Gersbach and E. Charbon: A 128 128 single-photon image sensor with column-level 10-Bit time-to-digitalconverter array, IEEE J. Solid-State Circ., 43 2008 2977 2989. 3 T. Spirig, P. Seitz, O. Vietze and F. Heitger: The lock-in CCD Two dimensional synchronous detection of light, IEEE J. Quantum Elecron., 31 1995 1705 1708. 4 R. Miyagawa and T. Kanade: CCD-based range-finding sensor, IEEE Trans. ED, 44 1997 1648 1652. 5 R. Lange and P. Seitz: Solid-state time-of-flight range camera, IEEE J. Quantum Electron., 37 2001 390 397. 6 B. Buttgen, T. Oggier, M. Lehmann, R. Kaufmann, S. Neukom, M. Richter, M. Schweizer, D. Beyler, R. Cook, C. Gimkiewicz, C. Urban, F. Metzler, P. Seitz and F. Lustenberger: High-speed and high-sensitive demodulation pixel for 3D imaging, Proc. SPIE, 6056 2006 22 33. 7 S. Kawahito, I. A. Halin, T. Ushinaga, T. Sawada, M. Homma and Y. Maeda: A CMOS time-of-flight range image sensor with gates-on-field-oxide structure, IEEE Sens. J., 7 2007 1578 1586. 8 D. Stoppa, N. Massari, L. Pancheri, M. Malfatti, M. Perenzoni and L. Gonzo: A range image sensor based on 10-mm lock-in pixels in 0.18 mm CMOS imaging technology, IEEE J. Solid- State Circ., 46 2011 248 258. 9 H. J. Yoon, S. Itoh and S. Kawahito: A CMOS image sensor with in-pixel two-stage charge transfer for fluorescence lifetime imaging, IEEE Trans. Electron Devices, 56 2009 214 221. 10 S. J. Kim, S. W. Han, B. Kang, K. Lee, D. K. Kim and C. Y. Kim: A three-dimensional time-of-flight CMOS image sensor with pinned-photodiode pixel structure, IEEE Electron Device Lett., 31 2010 1272 1274. 2012 3 28 268 8