Semiconductor & Storage Products Company

Home
 

Visconti™ Automotive Image Recognition Processor Series for Advanced Driver Assistance Systems (ADAS)

Visconti™ Series is an image recognition processor LSI for using camera-based vision systems.  Visconti™ Series detects and recognizes target objects such as pedestrians, faces, hands, vehicles and their motion/action by image processing image data from camera inputs frame by frame (including some previous frames in case of detection of motion/action), and then outputs recognition results by displaying results with marks on a camera image data to an LCD panel, alerting with voice message or beep, or reporting results to other units via communication I/F.

Visconti™ Series target products are systems using image recognition technology.  There are advanced driver assistance systems (ADAS) which are used to detect pedestrians, vehicles, traffic signs, etc. in the automotive market.  Surveillance cameras and security equipment which use pedestrian detection or identification with face recognition in the industrial market, and game machines which use hand gesture or motion capture in the consumer market.

Application examples of advanced driver assistance systems (ADAS) and pedestrian detection are shown below.  Visconti™ Series can realize various advanced driver assistance applications such as lane departure warning (LDW), forward/backward collision warning (FCW/BCW), forward/backward pedestrian detection (NV/BOP), traffic sign recognition (TSR) and so on by recognizing in real time traffic lines, vehicles, pedestrians, traffic signs etc. arround a vehicle using image data from 1 to 4 sets of cameras.

Examples of Image Recognition Applications for ADAS

Toshiba's high-performance Heterogenous multi-core architecture image recognition processor LSIs "Visconti™" Series will help to enable various applications in the ADAS that detects and recognizes in real-time traffic lines, vehicles, pedestrians, traffic signs, traffic signals and so on around a own car, and state of driver and passengers by using multiple cameras.

Lane Departure Warning (LDW)

A LDW system warns a driver on a highway if their car may depart from the current travel lane without a lane change signal by detecting traffic in other lanes from image data from a front or rear camera.  The Visconti™ Series will help to realize such system.

Forward Collision Warning (FCW) and Backward Collision Warning (BCW)

A FCW system detects a vehicle in front of the own car by using image data from a stereo camera or a monocular camera with milliwave rader at the front of the own car and warns a driver of a potential collision risk, based on the distance between the two vehicles and the own vehicle's speed.
A BCW system detects a vehicle in rear of the own car by using image data from a rear monocular camera.

Traffic Sign Recognition (TSR)

A TSR system recognizes traffic signs using image data from a front-mounted monocular camera and displays the recognized information of the recognized traffic sign on a dispaly panel. This system may be used to compare the recognized speed limit with the vehicle speed read via CAN and then it notifies a driver. TMPV7504XBG and TMPV7506XBG will help to realize such system.

Obstacle Detection and Lane Change Assistance (LCA)

An obstacle (vehicles) detection system detects obstacles such as vehicles around own car for lane change assistance and forward collision warning. It uses three monocular cameras (one at the front and one each at the rear-right and rear-left).
A LCA system detects rear vehicles to assist to change lane by using 2 image data from rear-left and rear-right cameras.

Backover Prevention (BOP)

A backover prevention system detects pedestrians with rear-mounted camera(s) and highlights the detected pedestrians on the display panel and alerts the driver when a vehicle goes back. TMPV7506XBG will help to realize such system.

Night Vision (NV) and Foward Pedestrian Detection (FPD)

These systems detect pedestrians with front-mounted camera (far- or near-infrared camera in cace of the NV system) and highlights detected pedestrians on the display panel and alert a driver. Toshiba image recognition processor Visconti™ Series realizes the NV system to detect pedestrians at night with using infrared camera(s). TMPV7506XBG will help to realize the NV and FPD systems to detect pedestrians at daytime and/or night time.

Parking Assistance with Bird's Eye View

This system captures images around a vehicle from four fish-eye or wide angle lens cameras and converts them to bird's-eye view point image and displays synthesized image from the above 4 images and a top view image of the own vehicle to assist when parking. TMPV7506XBG has four channels of color camera inputs, an image processing acclerator for converting image data and a video output interface with synthesis of image data. The TMPV7506XBG will enable easily to such system.

Driver State Monitoring (DSM)

A DSM system tracks the driver's face direction and notifies if the driver is not paying attention to the ahead.

Left/Right-turn Awareness

A left/right-turn awareness system detects pedestrians, bicycles and motorcycles using cameras mounted on either the left or right side of the own vehicle and warns the driver of the danger of hitting them. TMPV7506XBG will help to realize such system.

 

Recommended Products

Part Number Product Name Main Features Package
TMPV7506XBG
(Visconti™2)
Image recognition processor One Toshiba original 32-bit RISC CPU MeP:Media embedded Processor, Four Toshiba original 32-bit RISC media processing engines (MPEs), Video input I/Fs (Up to 4 inputs, Up to 1.3Mpixel size, Color/Grayscale cameras), Video output I/F (Up to WVGA size, RGB888), Six image processing accelerators, DDR2 SDRAM controller, NOR Flash/SRAM controller, External MCU interface, PCI express, etc. 516-pin PBGA
TMPV7504XBG
(Visconti™2)
One Toshiba original 32-bit RISC CPU MeP:Media embedded Processor, Four Toshiba original 32-bit RISC media processing engines (MPEs), Video input I/Fs (Up to 2 inputs, Up to 1.3Mpixel size, Color/Grayscale cameras), Video output I/F (Up to WVGA size, RGB888), Five image processing accelerators, DDR2 SDRAM controller, NOR Flash/SRAM controller, External MCU interface, etc.
T5BG3XBG
(Visconti™)
Three Toshiba original 32-bit RISC media processing engines (MPEs), Video input I/Fs (Up to 3 inputs, Up to VGA size, Grayscale cameras), Video output I/F (Up to VGA size, RGB888), Affine transformation accelerator, SDRAM controller, NOR Flash/SRAM controller, External MCU interface, etc. 456-pin PBGA
TX4964FG Automotive Display Controller Toshiba 64-bit RISC core (TX49/L4); graphics display controller with per-pixel alpha-blending; 2D graphic accelerator; communication modules (2-channel full-CAN controller, SIO/UART, I2C); system control (24-bit RTOS system timer, watchdog timer, 3-channel 16-bit timer) 176-pin LQFP
TX4966XBG 64-bit TX49/L4 Core, 2-channel graphics display controller (DGC) with per-pixel alpha-blending, 2D/3D graphic accelerator, 2-channel camera/video frame grabber, APIX, Communication modules (4-channel full CAN, ESEI, SIO/UART, I2C, I2S), System control (24-bit RTOS system timer, watchdog timer, PWM, 8-channel 16-bit timer) 456-pin PBGA

Visconti™ Basic System Organisation

This figure shows the block diagram of a Basic System Organization using the Visconti™ Image Recognition Processor Series

Visconti™ Features

  • Heterogenous multi-core achitecture suitable for embedded image recognition systems.
    Visconti™ Series has the following different type of engines which can run simultaneously.  Therefore Visconti™ Series enables various image recognition applications in real time under low power consumption.
    • i. Image Processing Accelerators
      Visconti™ Series has image processing accelerator(s) which use image processing many times in each step of the image recognition flow and can operate at very high speed under low power consumption. Visconti™ has an affine transform accelerator that corrects lens distortion for fish-eye or wide angle lems and converts view-point.  Visconti™2 Series has not only the affine transform accelerator but also filter accelerators that reduces noises, detects edges, converts color space and so on, a histogram accelerator that makes histogram and a matching accelerator that calculates parallax using stereo image data and detects motion.  TMPV7506XBG of Visconti™2 Series has also a HOG(Histogram of Oriented Gradients) accelerator suitable for human detection, it can detect human who stood still in real time.
    • ii. Media Processing Engine (MPE) which uses Multi-grain Parallelism Architecture
      MPE consists of Toshiba original low power 32-bit RISC core MeP(Media embedded Processor) and MeP coprocessor suitable for multimedia processing such as image processing.  The MPE is a media processing engine which uses VLIW(Very Long Instruction Word) technology executes at same time multiple instructions including SIMD(Single Instruction Stream-Multiple Data Stream) instructions that operates multiple data simultaneously by one instruction.  Multi-core of MPE ( Visconti™ has 3 sets of MPE.  Visconti™2 Series has 4 sets of MPE.) executes simultaneously multiple applications with helping the above image processing accelerators and provides software solutions for implementation of various image processing algrithms.
    • iii. Control CPU
      Visconti™2 Series has various on-chip peripherals such as PCI Express, serial interfaces (UART?SPI?I2C) and so on.  Therefore a 32-bit RISC core "MeP(Media embedded Processor)" is added into Visconti™2 Series as a control CPU, and releases the above MPEs with controlling these on-chip peripherals by this CPU.
  • Provides a Software Development Kit (SDK) which supports the development of applications using C language.
    SDK includes various image processing libraries using the above image processing accelerators, device drivers for on-chip peripherals and tools such as C compiler, simulator for key modules of Visconti™ Series, etc.
  • Video input interfaces which connect multiple cameras and Video output interface which can synthesize multiple layers.
    Visconti™ Series can connect with up to 3 sets of 8-bit grayscale cameras.  Thereby Visconti™ Series enables simultaneously lane change assistance (LCA) application using left/right side cameras and forward or backward collision warning (FCW or BCW) application monitoring with a forward or backward camera.  Visconti™2 Series can connect up to 4 sets (TMPV7506XBG)/2 sets (TMPV7504XBG) of 8- to 12-bit grayscale cameras or high resolution (up to 1.3Mpixel) color cameras.  Thereby Visconti™2 Series improves recognition rate and enables traffic signal or sign recognition (TSR) applications that need color detection.  The TMPV7506XBG of Visconti™2 Series is suitable for a bird-eye view parking assistance application with synthesis of 4 camera image data.

Visconti™ Block Diagram

This is the block diagram of the Visconti™2 (TMPV7506XBG)

Visconti™2 (TMPV7506XBG)

This is the block diagram of the Visconti™2 (TMPV7504XBG)

Visconti™2 (TMPV7504XBG)

This is the block diagram of the Visconti™ (T5BG3XBG)

Visconti™ (T5BG3XBG)

Visconti™ General Specifications

 

Group Visconti™ Visconti™2
Product Name T5BG3XBG TMPV7504XBG TMPV7506XBG
CPU - Toshiba original 32-bit RISC CPU “MeP (Media embedded Processor)”
Image Processing Processors Media Processing Engine Toshiba Original Media Processing Engine “MPE” 3 sets Toshiba Original Media Processing Engine “MPE” 4 sets
Image Processing Accelerators Affine Transform 1ch 1ch
Filter - 2ch
Histogram - 1ch
Histogram of Oriented Gradients(HOG) - - 1ch
Matching - 1ch
On-chip Memory ROM - Masked ROM: 64KBytes
RAM - SRAM: 2080KBytes
On-chip Peripherals Video Input Interface Up to 3ch (Grayscale) Up to 2ch (Color/Grayscale) Up to 4ch (Color/Grayscale)
Video Output Interface 1ch (VGA size, RGB888) 1ch (WVGA size, RGB888)
Main Memory Controllers SDRAM Controller DDR2 SDRAM Controller
NOR Flash/SRAM Controller
External MCU Interface 1ch (8-bit bidirectional parallel I/O)
PCI Express - - 1 lane
CAN Controller - 3ch
I2C Interface - 4ch
UART Interface 1ch 5ch
SPI Interface - 4ch
PCM Interface - 2ch
Timer/Counter 4ch 11ch
Operating Frequency 150MHz 266MHz max.
Package BGA 456pins, 27mm×27mm BGA 516pins, 27mm×27mm

 

   
Terms and Conditions |  Privacy Policy |  CSR |  Imprint |  Site Map |  Copyright © Toshiba Electronics Europe GmbH., All Rights Reserved.