With smartphones becoming a centerpiece in our lives, their ability to do more and more with incredible power and staggering efficiency is only going to grow more important. Through computational photography, smartphone cameras are able to achieve impressive results, but there are drawbacks. This method of creating images demands serious performance, and it can take time to render the final images. This can mean image previews don’t line up with the final result, and it can create a large gap in image quality between still photography and video, which can’t get the benefit of stacking multiple frames together for a wider dynamic range and cleaner imagery.

The demands of AI image processing are steep, and even phone SoC’s with ISPs and Neural Processing Units (NPUs) on board simply lack the power to perform this AI processing. This lack of power ultimately means shortcomings in image quality that wouldn’t be necessary with a sufficiently powerful NPU. In order to better handle the sheer amount of data coming from cameras, OPPO has built a dedicated and powerful NPU that can go well beyond the capabilities of current smartphones.

To demonstrate the huge leap a powerful NPU can offer, OPPO is introducing its first imaging NPU — MariSilicon X — which will launch in the Find X Series in early  2022. The company has spent 10 years working on its imaging technology, from lenses and sensors to in-house algorithms and now its own NPU. With the vertical integration of these features, OPPO’s MariSilicon X is set to offer new levels of photographic performance for smartphones.

List of OPPO MariSilicon X features

With a cutting-edge Imaging NPU built from the ground up, the MariSilicon X is able to offer serious performance. The built-in NPU can hit 18 TOPS@INT8 of processing and achieves an efficiency of 11.6 TOPS/watt. Its processing speeds need memory that can keep up, so OPPO has included a Terabit memory subsystem and an extra 8.5GB/s of DDR memory bandwidth on MariSilicon X. The memory plays a role in this efficiency as do many AI operations being hardwired into the NPU’s design. With this high performance, the MariSilicon X can perform noise reduction on 4K video at 40fps, fast enough to provide this in real-time.

The architecture of OPPO MariSilicon X

Currently the smartphone image processing pipeline involves capturing an image and compressing that image down from RAW into RGB and YUV — a process that sees much of the original image data lost. Phones then run their computational photography work on these compressed images, which severely limits the potential quality of the final product. The MariSilicon X’s NPU steps right into the middle of this process, as it has the speeds necessary to handle an image processing pipeline in the RAW domain. In other words, it can run its computational photography work on the original image while all of its data is still intact.

The MariSilicon X enables video capture at 4K with Ultra HDR in 20-bit and a 120dB dynamic range by combining multiple HDR frames in the RAW domain while recording. The effect is individual video frames that are on a level footing with still photography. The power of this processing and the speed it can achieve even allows for 4K AI Night Video — a first-time capability for Android — using RAW data with a final output that’s as vivid as the live preview.

Images of a woman compared between MariSilicon X, iPhone 13 Pro Max, Find X3 Pro
MariSilicon X supports 20bit Ultra-HDR Video

When combined with specialized hardware, the MariSilicon X NPU can take things even further. The MariSilicon X can support OPPO’s RGBW Pro mode using an RGBW sensor, which then lets it process red, green, blue (RGB) and W signals separately and combine them into a single image with an 8.6dB improvement to dynamic range and 70% more texture.

With the limitations of the current imaging pipeline and the breakthrough potential of a dedicated NPU with enough power to process images in the RAW domain, the benefit of adding imaging NPUs to smartphones is evident. They’ll allow users to capture photos at a higher quality and turn night video into a reality.

Images of colorful vases compared between MariSilicon X, iPhone 13 Pro Max, Find X3 Pro
MariSilicon X brings 4K AI Night Video on Android smartphones for the first time  where light intensities lower than 5 lux

You can learn more about the MariSilicon X and all of OPPO’s forthcoming innovations and technological offerings at OPPO Inno Day 2021.

POST A COMMENT

17 Comments

View All Comments

  • Duncan Macdonald - Wednesday, December 15, 2021 - link

    Interesting - however there is some critical missing data - what is the power consumption in its different modes ? Phones have a very limited total energy and a limited power consumption (due to heat) - how much would the MariSilicon X impact those limits ?

    There is also the inevitable question of how much a phone's price would be inflated to include this chip.
    Reply
  • dxu - Thursday, December 16, 2021 - link

    MariSilicon X would consume 797mW when performing noise reduction on 4K video at 40fps.

    Source: https://mp.weixin.qq.com/s/_wWZ8kYUTSy350itGQuStA (In Chinese, but has much more detail
    Reply
  • brucethemoose - Wednesday, December 15, 2021 - link

    Does anyone from OPPO read the comment section?

    This is a great idea for a sponsored post. TBH it's my favorite from Anandtech so far, and it actually increases my interest in buying an OPPO phone.

    While performance specs are nice, but ya'll should post more technical details, like the layout of the various silicon blocks, memory specs, die sizes, fab process and so on. Those nitty gritty details are big draw here at Anandtech, even if they aren't particularly relevant to what us end users do with a phone.
    Reply
  • brucethemoose - Wednesday, December 15, 2021 - link

    Also, Anandtech *still* needs an edit button, even if it only has a 3 minute timer. Reply
  • Duwelon - Wednesday, December 15, 2021 - link

    Edit burtons are so 1999, long live the unedible web! Reply
  • Slash3 - Thursday, December 16, 2021 - link

    Please do not eat the web. Reply
  • Fulljack - Thursday, December 16, 2021 - link

    now anandtech needs upvote button Reply
  • zodiacfml - Wednesday, December 15, 2021 - link

    This is a better post vs from other sites. Amazing if this does video better than the iphone. the iphone pretty much up there in terms of processing with crazy wide dynamic range that the look of the video is much like HDR look of photographers around 10 years ago.
    I also did not know smartphone computational photography works on compressed data until now
    Reply
  • Byte - Sunday, December 19, 2021 - link

    seriously, the iphone 13 pro vids look twice as vibrant as real life when im shooting them, and looks like twice the detail of what i can see. we are past futuramas real life HD. Reply
  • linuxgeex - Wednesday, December 22, 2021 - link

    It's a shame though that with all that dynamic range available, iPhone still tends to clip highlights horrendously. Reply

Log in

Don't have an account? Sign up now