What is the next step in nexoptical edge image processing
Published on August 24, 2021 at 11:00:10

Qualcomm products mentioned in this post are provided by Qualcomm technologies and its subsidiaries.

The term "image processing" includes a number of different tasks, including basic knowledge such as computational photography, computer vision algorithms, and even image compression. One thing all these programs have in common is that as the quality of input data improves, performance and results will also improve.

But what if there is no high-quality image? Image and video capture does not always provide high-quality data for data processing in the real world. For example, image frames may be noisy due to lack of light or improper shutter speed. Important image information may be lost when the ISP (processor managing camera data) performs tasks such as converting HDR level or simply compressing data. This means that subsequent downstream image processing algorithms in the camera pipeline may be forced to process less ideal data.

Qualcomm Advantage Network ( QaN) in the organizationPlatform solution ecosystem(PSE). Nexoptic recognizes the need to improve image data where it is captured. And thisclassThese capture locations are on the edge of mobile devices and Internet of things devices. Therefore, nexoptic developedALIIS ™(all-optical intelligent imaging solution), which is a set of image enhancement algorithm based on machine learning. Aliis can enhance and correct the defects in the images provided by the equipment camera and ISP pixel by pixel, and provide high-quality image data for the downstream camera pipeline process.

sufferAlexNetInspired by u-net, nexoptic's algorithm is based onConvolutional neural network(CNN) and can run at the edge of the device in real time. The model of this kind of algorithm uses the image processing ability of convolutional neural network to reduce the noise in low light image, which is the main challenge of image and video capture. Especially important, nexoptical uses the ability of convolutional neural network to extract advanced image information such as edge, contour and object from noise signal, so as to reconstruct the image.

Artificial intelligence in artificial intelligence

Aliis is an effective software ISP between the camera and subsequent downstream elements of the camera pipeline on mobile devices and Internet of things devices. Depending on the platform, aliis can benefit from hardware acceleration by running on a dedicated processor. Figure 1 provides a schematic diagram of the location of aliis in a typical mobile device or IOT Device Architecture:

Figure 1 - schematic diagram describing the role of aliis on a typical mobile device or Internet of things device

Nexoptic said that installing aliis at the edge of the device collecting data can bring many benefits. Most importantly, uncompressed raw data from ISP provides aliis with as much image information as possible. Like other AI at the device edge solutions, saving data on the device instead of sending it to the cloud for processing can improve privacy, reduce latency, and provideData locality, the latter two items are very important for real-time image processing.

In practice, the results are impressive. In one use case, aliis helped to improve the image classification performed by the commercial image classifier by 400%.

Of course, seeing is believing. Figure 2 shows an example of the types of image cleaning that aliis can perform:

Figure 2 –   An image that shows how aliis can reduce noise and improve image quality

Nexoptic has effectively completed the hard work of designing and training its models to ensure that developers can benefit from it. Therefore, the company usually describes aliis as "artificial intelligence in artificial intelligence", because its machine learning based algorithm can be used to clean up the data of computer vision model, which can run downstream of the camera pipeline. The company has also continuously optimized and retrained its various models and trained specific versions for different categories of cameras.

For snapdragon ® Aliis for mobile platforms

As a member of QaN, nexoptic has been aroundSnapdragon mobile platformThe constructed device determines the implementation method of optimizing aliis, and hasSnapdragon 855 mobile platformThe ability to process 2K video at a rate of 30 frames / second.

Nexoptic utilizationQualcomm Spectra ™ ISPProvide camera data andQualcomm ® Hexagon ™ DSPRun aliis on to supplement. They use tensorflow to build the model, and then useQualcomm ® Artificial intelligence (AI) neural processing software development kitThe derived model is quantified and converted to deep learning container (DLC) format, which can be run on hexagon DSP after optimization. They use additional optimization methods, including architecture search method, model distillation, hybrid precision network, filter and weight based pruning.

Working with nexoptic

Nexoptic is solving a unique problem. By using machine learning, real-time image capture at the edge of the device can be enhanced, so the downstream camera process can use high-quality image data. The company also said that the technology can be applied to intelligent security, mobile devices, automobiles, augmented reality and virtual realitymedical imaging , industrial automation and other fields.

Nexoptic is solving a unique problem. By using machine learning, real-time image capture at the edge of the device can be enhanced, so the downstream camera process can use high-quality image data. The company also said that the technology can be applied to intelligent security, mobile devices, automobiles, augmented reality and virtual realitymedical imaging , industrial automation and other fields.

Developers can use the company's snapdragon hardware development kit (HDK) to start building devices driven by snapdragon and capable of running nexoptical technology. The choice of which method to use depends on the application (for example, video or picture), resolution requirements, and other performance factors. For example, nexoptic recommends using the high-end snapdragon mobile platform, such asSnapdragon 778g 5g mobile platformSnapdragon 865 mobile platformandSnapdragon 888 mobile platformTo process high-resolution video.

For more information and resources,Please contact nexoptic。 In addition, be sure to see what they have been doing recentlyYouTubeAt the webinar, the company's technology and enterprise were briefly described at the meeting.

CSDN official wechat
Scan the two-dimensional code, make complaints about CSDN.
Micro signal: csdnnews
Microblog attention
[Disclaimer: CSDN publishes information in this column to disseminate more information and enrich network culture. The manuscript only represents the author's personal views and has nothing to do with CSDN. Its originality and the text and content stated in the text have not been confirmed by this website. This website does not make any guarantee or commitment to the authenticity, integrity and timeliness of this article and all or part of its content and text. Please readers only for reference, and please verify the relevant content by yourself. If you have any doubts or doubts about the manuscript, please contact CSDN immediately, and we will respond to you and deal with it quickly.]