With smartphones becoming a centerpiece in our lives, their ability to do more and more with incredible power and staggering efficiency is only going to grow more important. Through computational photography, smartphone cameras are able to achieve impressive results, but there are drawbacks. This method of creating images demands serious performance, and it can take time to render the final images. This can mean image previews don’t line up with the final result, and it can create a large gap in image quality between still photography and video, which can’t get the benefit of stacking multiple frames together for a wider dynamic range and cleaner imagery.

The demands of AI image processing are steep, and even phone SoC’s with ISPs and Neural Processing Units (NPUs) on board simply lack the power to perform this AI processing. This lack of power ultimately means shortcomings in image quality that wouldn’t be necessary with a sufficiently powerful NPU. In order to better handle the sheer amount of data coming from cameras, OPPO has built a dedicated and powerful NPU that can go well beyond the capabilities of current smartphones.

To demonstrate the huge leap a powerful NPU can offer, OPPO is introducing its first imaging NPU — MariSilicon X — which will launch in the Find X Series in early  2022. The company has spent 10 years working on its imaging technology, from lenses and sensors to in-house algorithms and now its own NPU. With the vertical integration of these features, OPPO’s MariSilicon X is set to offer new levels of photographic performance for smartphones.

List of OPPO MariSilicon X features

With a cutting-edge Imaging NPU built from the ground up, the MariSilicon X is able to offer serious performance. The built-in NPU can hit 18 TOPS@INT8 of processing and achieves an efficiency of 11.6 TOPS/watt. Its processing speeds need memory that can keep up, so OPPO has included a Terabit memory subsystem and an extra 8.5GB/s of DDR memory bandwidth on MariSilicon X. The memory plays a role in this efficiency as do many AI operations being hardwired into the NPU’s design. With this high performance, the MariSilicon X can perform noise reduction on 4K video at 40fps, fast enough to provide this in real-time.

The architecture of OPPO MariSilicon X

Currently the smartphone image processing pipeline involves capturing an image and compressing that image down from RAW into RGB and YUV — a process that sees much of the original image data lost. Phones then run their computational photography work on these compressed images, which severely limits the potential quality of the final product. The MariSilicon X’s NPU steps right into the middle of this process, as it has the speeds necessary to handle an image processing pipeline in the RAW domain. In other words, it can run its computational photography work on the original image while all of its data is still intact.

The MariSilicon X enables video capture at 4K with Ultra HDR in 20-bit and a 120dB dynamic range by combining multiple HDR frames in the RAW domain while recording. The effect is individual video frames that are on a level footing with still photography. The power of this processing and the speed it can achieve even allows for 4K AI Night Video — a first-time capability for Android — using RAW data with a final output that’s as vivid as the live preview.

Images of a woman compared between MariSilicon X, iPhone 13 Pro Max, Find X3 Pro
MariSilicon X supports 20bit Ultra-HDR Video

When combined with specialized hardware, the MariSilicon X NPU can take things even further. The MariSilicon X can support OPPO’s RGBW Pro mode using an RGBW sensor, which then lets it process red, green, blue (RGB) and W signals separately and combine them into a single image with an 8.6dB improvement to dynamic range and 70% more texture.

With the limitations of the current imaging pipeline and the breakthrough potential of a dedicated NPU with enough power to process images in the RAW domain, the benefit of adding imaging NPUs to smartphones is evident. They’ll allow users to capture photos at a higher quality and turn night video into a reality.

Images of colorful vases compared between MariSilicon X, iPhone 13 Pro Max, Find X3 Pro
MariSilicon X brings 4K AI Night Video on Android smartphones for the first time  where light intensities lower than 5 lux

You can learn more about the MariSilicon X and all of OPPO’s forthcoming innovations and technological offerings at OPPO Inno Day 2021.

POST A COMMENT

17 Comments

View All Comments

  • Silver5urfer - Wednesday, December 15, 2021 - link

    Everything is only camera right ? Garbage AI and ISP technology damn every single SoC and Phone only Camera. What is this constant nonsense. I would like to see SD Express NVMe slot or UFS expandable storage or a damn good underscreen camera with 3.5mm jack powered by a DAC or maybe a better battery with higher density that lasts more than 3 years.

    BS camera stuff. Oppo can keep this junk. Their software killed OnePlus Oxygen OS with bloatware. Their master BBK did it but BBK loves Oppo more than OP because for the former it's for masses. OnePlus is for tinkerers.
    Reply
  • blanarahul - Thursday, December 16, 2021 - link

    Camera is the main draw of high end smartphones for most people. Reply
  • amnesia0287 - Thursday, December 16, 2021 - link

    I don’t quite get why they would go to such an extreme bit depth. As it stands now, even real 10bit displays are still uncommon and 12bit displays are almost non existent.

    Most SLR/Mirrorless cameras are at best 14bit, and high end cinema cameras are 16-bit which is already an obscene range of color. (Specifically 48bit footage (16+16+16) has 281,474,976,710,656 colors which means 60bit images/video would have 1,152,921,504,606,850,000 colors).

    281 trillion is already massive overkill, but pushing that all the way past quadrillion to MORE than a quintillion just doesn’t make sense to me. I can’t think of a single valid use case. It of course should be able to use lower bit depths, but then it seems like it would be wasting massive amounts of registers for no reason.
    Reply
  • Lindegren - Thursday, December 16, 2021 - link

    Grading my dear, you can push the colors a lot, but you would need the headroom allowed by a higher colordepth Reply
  • Sivar - Thursday, December 16, 2021 - link

    I am interested.
    Also, this is far less annoying than the usual Anandtech banner ads.
    Reply
  • Duncan Macdonald - Thursday, December 16, 2021 - link

    One possible use case - getting good color in a scene with a very bright light source (eg the sun) in a portion of the picture.

    (However even in this case, the dispersion in the lenses and internal reflections would reduce the RAW image range to 16-bit or less even in a high end SLR let alone a phone camera with a BoM under $100.)
    Reply
  • DeathArrow - Friday, December 17, 2021 - link

    What's up with Anandtech? It's the last outlet to review tech, if they ever review a particular device. People who wrote interesting articles left without being replaced.

    And now it's rid with sponsored posts.

    I had the habit to check Anandtech daily, hoping that I might came around something interesting. Now I think it is a loss of time.
    Reply

Log in

Don't have an account? Sign up now