The Tezzblog

August 30, 2013

Tezzaron is very excited about a phenomenal “smart video” device built in our FaStack technology!

Johns Hopkins University (JHU) and Universidad Nacional del Sur (UNS) worked together to develop a sensor-plus-processor in a single 3D-IC. The 2×2.5mm chip captures images and can process 30,000 frames per second! A prototype was built in a 130nm process as part of Tezzaron’s “Martin MPW” project.

The JHU/UNS chip contains two layers of silicon.  On the “top floor” is a 48×32 array of back-illuminated photodiodes.  Each photodiode has an A/D converter and enough digital circuitry, distributed across both layers, to perform digital processing locally. Information is shared with the 4 neighboring cells.  In this way, the initial image processing is performed in a highly parallel manner, pixel by pixel. Finally, the image is assembled and finished on the “bottom floor,” which contains blocks of higher logic and an 8051 processor.  The 8051 microprocessor also facilitates sequences of image processing tasks in a higher level language.

The auxiliary processing blocks perform arithmetic and pattern matching operations and the on-board memory allows programming of the functionality. The video processor can execute a set of different processing tasks and concatenate operations to produce complex filtering. Thanks to the on-board intelligence, it can be used to obtain higher level information – for example, determining how many objects are in an image. The 3D configuration allows the chip to run at very high speed while consuming less power than a multi-chip implementation. Researchers expect this technology to have wide applications in medical imaging, industrial quality control, facial recognition, and the detection of high-speed objects.  Its small size and low power budget also make it attractive for cameras, cell phones, and gaming.

UNS released three videos of some interesting experiments, linked here:

• Oct 2012: First image captured by the chip

• Nov 2012: Capturing the UNS logo (Explanation below)

Top Row:
1) Original image captured by photodiodes
2) Erosion of the correlated double sampling (CDS) image
3) Opening of the CDS image
4) Edge recognition of filtered image

Center Row:
1) Fixed pattern noise removal of the CDS image
2) Dilation of the CDS image
3) Closing of the CDS image
4) Edge recognition of binarized image

Bottom Row:
1) Median filter of the CDS Image
2) Bounding box growing first steps
3) Bounding box
4) Binarization of the opening image

• Dec 2012: Capturing a face and hand (Explanation below)

Left Column:
Top: Original image captured by photodiodes
Center:  Fixed pattern noise removal (Correlated Double Sampling)
Bottom: Median Filter applied to the Center Left Image.

Center Column:
Top: Image closure
Center: Median filter of closed image
Bottom: Dilation of filtered image

Right Column:
Top: Opening and closing
Center: Edge recognition (using difference between eroded image and original)
Bottom: Edge recognition (using difference between dilated image and original)

Tezzaron congratulates our colleagues and collaborators on this groundbreaking work!

 

Press coverage in Argentina:

Portinos:“Desarrollo argentino de chip que captura en 3D”
“Investigadores argentinos desarrollan un chip con capacidad para procesamiento en 3D para cámara de video inteligente. Aunque todavía en fase de prototipo, podrá tener aplicaciones en seguridad, salud e industria.”

iProfesional: “Investigadores argentinos desarrollan un chip 3D de cámara de video inteligente”
“Investigadores argentinos diseñaron junto con científicos de la Universidad de Johns Hopkins de EEUU el primer chip tridimensional (3D) que implementa una cámara inteligente, completamente funcional, en un proceso comercial 3D.”

La Gaceta: “Investigadores argentinos desarrollaron un chip 3D que puede captar más 30.000 imágenes por segundo”
“Un grupo de investigadores argentinos desarrolló un chip 3D para cámaras de video inteligentes. Este chipcaptura y procesa más de 30.000 imágenes por segundo. Aunque aún se trata de un prototipo, sus creadores sostienen que puede llegar a tener aplicaciones en los campos de la seguridad, de la salud y de la industria. El desarrollo está liderado por investigadores del Conicet y por la Universidad Nacional del Sur. También participan expertos de la universidad John Hopkins, de EEUU.”

Ebiz Latam: “Investigadores argentinos desarrollan un chip 3D para cámaras de video inteligentes”
“Investigadores argentinos diseñaron junto con científicos de la Universidad de Johns Hopkins de EE.UU., el primer chip tridimensional (3D) que implementa una cámara inteligente, completamente funcional, en un proceso comercial 3D.”