Sony’s Intelligent Vision Sensor represents a significant advancement in image sensor technology by integrating AI processing capabilities directly into the sensor unit. This innovation is the world’s first image sensor to incorporate an AI engine within its logic circuit, enabling high-speed edge AI processing alongside traditional image processing functions.
Technical Features
AI-Integrated Logic Chip
The sensor features a back-illuminated pixel chip with approximately 12.3 effective megapixels, which captures a wide angle of view. The logic chip, stacked with the pixel chip using Sony’s proprietary stacking technology, includes a Digital Signal Processor (DSP) dedicated to AI signal processing and embedded memory for AI models. This setup eliminates the need for external high-performance processors or memory, making it ideal for edge AI systems[1][3][4].
High-Speed AI Processing
Conventional image sensors require data transmission for each frame to perform AI processing, which can hinder real-time performance. Sony’s Intelligent Vision Sensor, however, completes ISP and AI processing within a single video frame, achieving high-precision, real-time object tracking with processing speeds as fast as 3.1 milliseconds for models like MobileNet V1[1][4].
Metadata Output and Privacy
The sensor outputs processed data as metadata, significantly reducing the data volume and addressing privacy concerns by not transmitting raw image data. Users can select various data output formats, including ISP format images (YUV/RGB) and region-specific extracts, allowing for flexible applications[1][4].
Selectable AI Models
Users can customize the sensor by writing and updating AI models to its embedded memory. This flexibility allows the sensor to adapt to different tasks and environments, such as counting visitors, detecting stock shortages, or heat mapping in retail settings[1][4].
Applications and Benefits
The integration of AI into the image sensor brings numerous advantages:
– Reduced Data Transfer: Minimizes the need for transmitting large image files to the cloud, cutting down on power consumption and telecommunications costs.
– Enhanced Privacy: By processing data on the sensor and only transmitting essential metadata, privacy risks are mitigated.
– Versatility: The ability to update AI models enables the sensor to be used in various scenarios without the need for different hardware setups.
Sony’s Intelligent Vision Sensor is a groundbreaking technology that not only enhances image processing capabilities but also paves the way for more efficient and secure AI applications in various fields, from retail to smart city infrastructure[1][3][4].
Further Reading
1. Intelligent Vision Sensors (AI Sensor) | Edge AI ＆ Sensing | Technology | Sony Semiconductor Solutions Group
2. https://www.sony.com/en/SonyInfo/sony_ai/technology/picture.html
3. Realizing the world’s first image sensor with built-in AI engine! Discover the stories behind the drive to create the world’s first of its kind | Feat…
4. Intelligent Vision Sensors with AI Processing Functionality from Sony | Security Info Watch
5. Exploring the AI Technology in the Sony A7RIV Mirrorless Camera: Setting a New Standard – Cameraland Sandton