What Apple ProRAW is, and what it means for professional photographers

With iOS 14.3, Apple has rolled out support for ProRaw photos on the iPhone. Professional photographer Austin Mann has shared his initial thoughts on the format.

Apple ProRAW is here and professional photographers are speaking up about the power behind the format. Apple took the power of the A14 processor and camera system of the iPhone 12 Pro and created a new professional image format called ProRAW.

Austin Mann, best known in the Apple media sector for his iPhone camera reviews, has written a post summing up his thoughts on ProRAW and what it means for iPhone photography.

First, lets discuss what RAW is and why it is important in the first place. Austin says it is a file format that offers photographers more control, but lets look at exactly what that means.

What is a RAW image?

A RAW image file is exactly what it sounds like--an image file lacking any algorithmic change or influence from computational photography. When a user takes a RAW photo, they expect to have exactly what the sensor sees plus whatever data the ISP is able to provide in that instant.

RAW is important because it gives all of the creative power to the photographer. Algorithms and processes are written by humans and taught with opinionated code and artistic leans, therefore an image captured with computational photography will create a beautiful, yet opinionated image. Users can see these opinions expressed when comparing iPhone vs Samsung cameras and how those specific ISPs create an image.

Apple ProRAW takes this a step further. Not only are you provided the DNG image file with all the available data, but you're given some of that computational magic as well. This means you get a beautifully rendered image using Apple's ISP alongside a mountain of data for editing.

Halide, a professional iPhone photography app, is testing using ProRAW in its app and showed off how much detail can be pulled from the ProRAW image when properly processed.

Shadow recovery is just one aspect of RAW photos, users can also control white balance, noise, and sharpening without having to rely on whatever bias the iPhone ISP has.

Austin Mann's testing

Austin Mann showed a specific example of why users may want to take advantage of ProRAW formats with night photography. In the first image he shows a standard processed HEIC format straight from the iPhone 12 Pro Max camera, the second image shows the image captured and edited in ProRAW.

The noise reduction algorithm thought many of the stars in the sky were just noise that needed to be corrected, so the processed HEIC image shows much fewer stars. The second image shows how much control Austin had over the image while editing in Lightroom, showing that he could bring out more detail present in the image without losing important aspects like the stars in the sky.

Austin covers more aspects of ProRAW in the post, and went on to discuss how ProRAW enables image capture that rivals even professional cameras. He points out that ProRAW has 14 stops of dynamic range, which can help when taking photos in complex situations like at sunrise or sunset.

Apple ProRAW also has 12-bit color, which is magnitudes more color than HEIC, which is limited to 8-bit color. This translates to more nuanced shades of color in a ProRAW image which can be seen on HDR displays. Apple uses the P3 color gamut on its displays, making photos with 12-bit color really pop.

Halide developer comments on ProRAW

Ben Sandofsky is one of the developers working on Halide, and he made a large blog post covering in-depth details of ProRAW and third-party API use. He provides more clarity on what ProRAW is and how developers can take advantage of the powerful format.

As mentioned before, Apple ProRAW isn't a file format per say, its a package of information that includes an advanced DNG file. Ben explains that Apple is doing some tricky things with this file that change how RAW is being handled on iOS.

To reduce the complexity of the file, Apple performs the demosaic step and saves the image after that is complete. This is a step detailed further in the blog, but it basically takes the RGB structure of the base image and runs an algorithm to decide what colors you're image will have. Apple avoids sloppy demosaic algorithms in third-party programs or apps by doing this step themselves, which makes Apple's DNG file already different from one you'd get out of a DSLR.

The Halide developer says this method also gives Apple greater control over what future sensors look like. If Apple chooses a less conventional mosaic pattern for future iPhone sensors, software won't need to account for it since the ProRAW file will have already processed that.

Apple's control over the ProRAW format also means more compatibility. When Halide developed its RAW format it was limited to the wide angle and telephoto lens--the ultra-wide lens and selfie camera were not compatible with RAW formats. Apple ProRAW changes this and now apps like Halide can use Apple's RAW format to shoot with all lenses on the iPhone Pro.

ProRAW DNG files also include the results of computational photography performed when the image is taken. This means that Smart HDR data can be used for editing the image, which provide striking images like the tweet shared above. ProRAW still uses data from Deep Fusion as well, which reduces noise in low-light scenes.

Apple includes data from semantic image mapping in the DNG as well. These maps are generated based on what the ISP "sees" to separate a person from the background, which enables portrait features to work on single-lens iPhones.

The API for ProRAW is apparently very complex and allows developers to pick and choose which parts of the algorithmic data to include. This gives developers like those at Halide to control the shooting experience. One example given was reducing the ProRAW bit depth from 12-bit to 10-bit so plenty of color data is still captured while reducing the file size.

Users running iOS 14.3 on an iPhone 12 Pro or iPhone 12 Pro Max can shoot in ProRAW. Navigate to the Settings app, open the Camera section, and select "Formats." From here you can toggle ProRAW on and an indicator for RAW will show up in the Camera app.

ProRAW will be disabled by default. When you tap on the RAW toggle in the Camera app it will disable Live Photos and begin capturing in the ProRAW format. Images captured in ProRAW are much larger than HEIC and can eat up storage quickly. We've observed file sizes up to four times larger than HEIC when using ProRAW.

Live Photo and Portrait modes are not available when shooting in Apple ProRAW. ProRAW files tend to be around 25MB or larger. Apps that can take advantage of Apple's advanced DNG format have begun to roll out with the release of iOS 14.3. The Halide ProRAW update is now available on the App Store.