Halide Does Deep Dive Into iPhone SE's Software-Based Portrait Mode

Though it is a budget device with a single-lens camera, the iPhone SE features support for Portrait Mode, enabled through the powerful A13 chip in the smartphone.

iphonesehandson
It is the first of Apple's smartphones to offer Portrait Mode photos created entirely with software techniques rather than hardware, which prompted the developers behind popular iOS camera app Halide to take a deep dive into how it works.

The ‌iPhone SE‌ is equipped the same camera sensor as the iPhone 8, based on a recent teardown done by iFixit, but its camera can do more because it's using "Single Image Monocular Depth Estimation," aka generating Portrait Mode effects using a 2D image.

As Halide developer Ben Sandofsky points out, the ‌iPhone‌ XR is also a single-lens camera with Portrait Mode support, but the ‌iPhone‌ XR gets depth information through hardware. That's not possible on the ‌iPhone SE‌ because the older camera sensor doesn't support the feature.

Halide has discovered that unlike other iPhones, the ‌iPhone SE‌ can take a picture of another picture to attempt to develop a depth map. The app was even able to take a photo of an old slide film, adding depth effects to a 50 year old photo.

halideoldphoto

A picture of a picture and the resulting depth map from the ‌iPhone SE‌

The ‌iPhone SE‌'s Portrait Mode is somewhat limited because it only works with people, which is due to the neural network that powers the feature. When a Portrait Mode image without a person is captured, it fails in various ways because it can't create an accurate estimated depth map.

The ‌iPhone‌ XR also limited Portrait Mode to people alone, and using Portrait Mode with other objects requires upgrading to one of Apple's more expensive phones.

According to Halide, depth maps on the ‌iPhone SE‌ (or any phone with Portrait Mode) can be viewed by using the Halide app and then shooting in Depth mode. Halide's full breakdown of the ‌iPhone SE‌'s Portrait Mode can be read over on the Halide website.

Related Roundup: iPhone SE
Tag: Halide
Buyer's Guide: iPhone SE (Don't Buy)
Related Forum: iPhone

Popular Stories

iPhone SE 4 Vertical Camera Feature

iPhone SE 4 Rumored to Use Same Rear Chassis as iPhone 16

Friday July 19, 2024 7:16 am PDT by
Apple will adopt the same rear chassis manufacturing process for the iPhone SE 4 that it is using for the upcoming standard iPhone 16, claims a new rumor coming out of China. According to the Weibo-based leaker "Fixed Focus Digital," the backplate manufacturing process for the iPhone SE 4 is "exactly the same" as the standard model in Apple's upcoming iPhone 16 lineup, which is expected to...
iPhone 16 Pro Sizes Feature

iPhone 16 Series Is Just Two Months Away: Everything We Know

Monday July 15, 2024 4:44 am PDT by
Apple typically releases its new iPhone series around mid-September, which means we are about two months out from the launch of the iPhone 16. Like the iPhone 15 series, this year's lineup is expected to stick with four models – iPhone 16, iPhone 16 Plus, iPhone 16 Pro, and iPhone 16 Pro Max – although there are plenty of design differences and new features to take into account. To bring ...
bsod

Crowdstrike Says Global IT Outage Impacting Windows PCs, But Mac and Linux Hosts Not Affected

Friday July 19, 2024 3:12 am PDT by
A widespread system failure is currently affecting numerous Windows devices globally, causing critical boot failures across various industries, including banks, rail networks, airlines, retailers, broadcasters, healthcare, and many more sectors. The issue, manifesting as a Blue Screen of Death (BSOD), is preventing computers from starting up properly and forcing them into continuous recovery...
iphone 14 lineup

Cellebrite Unable to Unlock iPhones on iOS 17.4 or Later, Leak Reveals

Thursday July 18, 2024 4:18 am PDT by
Israel-based mobile forensics company Cellebrite is unable to unlock iPhones running iOS 17.4 or later, according to leaked documents verified by 404 Media. The documents provide a rare glimpse into the capabilities of the company's mobile forensics tools and highlight the ongoing security improvements in Apple's latest devices. The leaked "Cellebrite iOS Support Matrix" obtained by 404 Media...
Apple Watch Series 9

2024 Apple Watch Lineup: Key Changes We're Expecting

Tuesday July 16, 2024 7:59 am PDT by
Apple is seemingly planning a rework of the Apple Watch lineup for 2024, according to a range of reports from over the past year. Here's everything we know so far. Apple is expected to continue to offer three different Apple Watch models in five casing sizes, but the various display sizes will allegedly grow by up to 12% and the casings will get taller. Based on all of the latest rumors,...
tinypod apple watch

TinyPod Turns Your Apple Watch Into an iPod

Wednesday July 17, 2024 3:18 pm PDT by
If you have an old Apple Watch and you're not sure what to do with it, a new product called TinyPod might be the answer. Priced at $79, the TinyPod is a silicone case with a built-in scroll wheel that houses the Apple Watch chassis. When an Apple Watch is placed inside the TinyPod, the click wheel on the case is able to be used to scroll through the Apple Watch interface. The feature works...

Top Rated Comments

luvbug Avatar
55 months ago

Could you have used a nicer pic? lol
She's probably the author's (Halide's) grandmother. Take a break, you'll be old too, sooner than you can imagine.
Score: 21 Votes (Like | Disagree)
AngerDanger Avatar
55 months ago

Could you have used a nicer pic? lol
As you wish. :)



Attachment Image
Score: 18 Votes (Like | Disagree)
NickName99 Avatar
55 months ago

This is wildly impressive.
Agreed - this is far more impressive than most people realize. The Pixel phones and the iPhone XR achieve this with a single lens by leveraging focus pixels to generate a depth map from available 3D information. The SE doesn’t have enough focus pixels for that, so it’s relying purely on machine learning, building a depth map from 2D information.

I’m surprised it was more cost effective to do it this way rather than just make the changes necessary to fit the XR rear camera module. That really speaks to the expense of retooling manufacturing.
Score: 10 Votes (Like | Disagree)
farewelwilliams Avatar
55 months ago
Theoretically, you should be able to extract frames from a video, apply neural network from Portrait Mode on to each frame, and stitch it back into a video.

Highly likely iOS 14 will have Portrait Mode for video. Would be only enhanced with Lidar too. This will take cinematography on a smartphone to a whole new level.
Score: 9 Votes (Like | Disagree)
NickName99 Avatar
55 months ago

I had assumed the criticism was because the image is low resolution and blurry, making it a poor test image.
I‘d say the worse the quality of the photo (resolution and sharpness), the more impressive that it works.
Score: 8 Votes (Like | Disagree)
JSRinUK Avatar
55 months ago

wonder if it works on pets... or if it will in the future
Hoping that no one minds a photo of me here, but I did a test photo when playing with my dog yesterday.

I'm not sure how the portrait mode works, but I was quite impressed that both myself and my dog are in focus.



Attachment Image
Score: 6 Votes (Like | Disagree)