Wonder How ToGadget HacksNext RealityNull Byte
how to
Home
Smartphones

Everything You Should Know About Rolling Shutter on Your Phone's Camera

Apr 29, 2019 09:02 PM
636917068273702441.jpg
636921223933916063.jpg

Smartphone videos get better and better each year. Seven or eight years ago, who would have thought iPhone and Android phones could support 4K video recording. Some phones can even shoot slow-motion at 960 fps. But no matter the resolution or frame rate, a phone's rolling shutter can make quick movements in front of the camera appear wobbly, distorted, or with artifacts.

Depending on what you're going for, a twisted look for fast-moving objects being recorded may be the effect you want. However, most of the time it's just an unintended effect caused by the CMOS camera sensor built into your smartphone. Rippled. Warped. Distorted. Wobbly. Twisted. Slanted. Bent. Skewed. Stretched. Bouncing. Deformed. Jello effect. Light flickering. These are all things that can happen to your footage with a smartphone camera.

Rolling shutter is the sworn enemy of smartphone videographers, even those who prefer shooting digital video on other products like DSLRs and video cameras. If the camera uses a CMOS sensor, rolling shutter will likely be an issue, to some degree, anytime fast moving objects are in front of the lens. Luckily, there are a few things you can do to minimize or hide rolling shutter artifacts in your videos.

Pixel 3 XL footage seems more stretched that the iPhone XS Max.

But What Is Rolling Shutter Exactly?

To explain what rolling shutter is, we need to first establish what video is. As you may already know, a video is made up of a series of pictures, called frames, that when placed in sequence, create the illusion of movement. Typical videos use 24, 30, or 60 frames per second (fps).

Each frame is taken, or scanned, when light hits the phone's sensor. However, on smartphones and other digital cameras with CMOS sensors, the light isn't scanned and converted to digital form all at once. Instead, the sensor scans top to bottom, recording line by line. This is in contrast to most cameras that use CCD sensors, which favor global shutter, where the entire image is scanned at the same time (which is how it works with traditional film-based cameras).

Representation of how a smartphone scans with rolling shutter.

However, that is not to say that all CMOS sensors use rolling shutter while all CCD sensors use global shutter. CCD imaging sensors can use rolling shutter. More importantly, some professional digital video cameras employ global shutter with CMOS sensors, but it's not something that's common in smartphones yet. The iPhone XS and XS Max models have a global shutter on its infrared camera, part of the TrueDepth camera system, but that doesn't help us out much with shooting videos.

CMOS sensors with global shutters are too expensive to manufacture when compared to rolling shutter equivalents, at least, when talking about larger sensors. In those cases, if the global shutter isn't developed correctly, it would likely result in more noise and less dynamic range than equivalent CMOS sensors with rolling shutter. Smartphone CMOS sensors are much smaller and therefore cheaper to produce so, hopefully, we'll start seeing them very soon. However, video shot using rolling shutter sensors have more of that "cinematic" feel than with global sensors, which is why a lot of OEMs stick with rolling.

The problem with rolling shutter for digital filmmaking or videomaking is that it doesn't scan the image fast enough to accurately capture fast-moving images in front of the lens. This results in focal plane distortion. Aircraft propellers, fan blades, and rotor wings are common fast-moving items affected by rolling shutter's slow scanning.

The warped, distorted blades of a fan recorded on an iPhone 5S.

Image distortion due to a phone's rolling shutter can also be seen when filming spinning coins, vehicles driving by, sports activities, and fidget spinners. Filming at static subjects when you're inside a moving vehicle, such as a train or bus, will also distort images. Lighting may also flicker in and out like it's pulsing. And just moving the camera from side to side quickly can create unsightly artifacts.

Some iPhone footage from inside a moving train.

While camera makers are still attached to CMOS imaging sensors for smartphones, some are trying to make the problems of rolling shutter, well, less of a problem. For example, Sony has developed a new CMOS sensor with faster data readout speeds, reportedly four times faster than "conventional" products, resulting in less focal plane distortion, which could eventually make its way into Sony's Xperia line of phones. Sony is also continually developing CMOS sensors with global shutter, which could also make it into phones soon.

How Can You Minimize Rolling Shutter Distortion?

There are a number of techniques that videographers and digital cinematographers can use to reduce or hide rolling shutter distortion on phone footage before a video hits the world wide web. Some are free, while others will require spending a little dough. Take a look at all of the options we highlight here to see which one works best for your smartphone budget.

Guitar strings showing the ripple or jello effect from an iPhone 4.

Option 1: Use a Shorter Focal Length

Smartphone camera lenses have been multiplying steadily over the years. At one point, two rear cameras seemed revolutionary. Now, you might find upwards of four and even five rear camera lenses in your phone of choice. Here's the thing — while multiple lenses expand your mobile photography possibilities, they can add to rolling shutter distortion.

If you've ever tried to zoom in on a subject while keeping your camera steady, you know how difficult it can be. That's why we recommend you stick the shortest focal length you can (the widest lens available). Often, it's the default camera lens your iPhone or Android phone opens with. Shooting with a shorter focal length will help prevent noticeable shaking or distortion, especially when filming handheld.

A longer focal length means you're more likely to contribute to rolling shutter distortion with unintended camera movements during fast action in the scene, and the two together could make the artifacts worse.

636917062670577315.jpg

Make sure you're using the wider (shorter) lens on your iPhone or Android device.

Wider lenses allow you to cover more of an image without moving as much. If you try to pan over a scene with a long focal length, you'll need to move quickly to cover the same amount of space in the same amount of time. This forces the sensor to miss a lot of the light information in your scene, possibly resulting in more rolling shutter problems.

Option 2: Use a Tripod or Stabilizer

If you want to tackle the problem head-on, consider investing in a tripod to keep your iPhone or Android phone motionless. Rolling shutter coupled with a shaky hand will only make the footage look worse. So while a tripod will not actually remove rolling shutter distortion, it could help minimize it in comparison to handheld video. There are plenty of pro-level tripods out there, but a good starting point for smartphone videographers would be Joby's solutions:

If you need to move the camera around, a stationary location made possible by a tripod won't be able to help you out with that, unless all you need to do is pan around. When you need to follow in the footsteps of the subject, that's where handheld stabilizers and gimbals come in.

Again, these won't get rid of rolling shutter distortion but will mask it a bit in comparison to unsteady video using just your hands, and it will also help get rid of the micro jitters you may see in the unsteady video. The problem with these, however, is that they work best with phones that don't have OIS (optical image stabilization) built in, which could interfere with the stabilizer or gimbal's abilities.

Option 3: Choose a Phone with OIS/EIS

Optical image stabilization is when a camera is built with a mechanical stabilizer. OIS usually results in remarkably smooth videos compared to cameras without OIS, especially when using zoom lenses. However, the downside to this is that most, if not all, OIS systems in smartphones cannot be disabled, so they may conflict with gimbals as mentioned above.

Electronic image stabilization (EIS) is a different beast and can be by itself or on top of OIS footage. As you could guess, it's software designed to compensate for shaky hands when filming. Every OEM has its own version of stabilization software, but many do it well on phones. The downside with this one is that you can usually only get EIS with footage under 4K resolution since it requires shooting at a slightly higher resolution so that it can trim the footage back down after processing the shakes, and most phones max out at 4K right now.

Either way you go, with OIS, EIS, or a phone with an OIS/EIS combo, you're better off than you would be without any of the features. But back to the point of this article: rolling shutter distortion. As you could probably guess, just like tripods and gimbals, OIS and EIS will not reduce rolling shutter problems when recording fast-moving subjects, but they may make it less noticeable.

Option 4: Slow Down Your Shutter Speed

Depending on your device, you might need to get yourself a video recording app for this trick, such as Filmic Pro. To help smooth over rolling shutter distortion, slow down your shutter speed as much as possible. Shutter speed controls how quickly the shutter opens and closes per frame. The faster the shutter speed, the less light gets into the sensor. The slower, the more light gets in. The motion blur that results from a reduced shutter speed can hide rolling shutter, at least, to some degree.

You'll need to experiment on your end to see what you think you can get away with. Typically, filmmakers set their shutter speed to the inverse of the frame rate (e.g., 24 fps video should have a shutter speed of 1/48). As you lower the shutter speed, the video will become more and more blurred. This might not be too noticeable at first, but the lower you go, the choppier things get, to the point where things might look more like stop motion than live action video.

Option 5: Watch Your Movements

If you find that rolling shutter distortion is really bad when panning your phone, try panning in the opposite direction. Because of the way your phone's sensor scans for light, one direction might result in worse rolling shutter than the other.

Experiment. Check if you can see a noticeable difference between a left-to-right pan and a right-to-left pan. If one works much better than the other, you might want to stick to one specific movement for your videos. In the same vein, you might need to watch out for fast action in your shots. If your phone can't handle left-to-right pans, it'll probably have a rough time with a subject moving quickly left-to-right. In that case, plan and film the object moving right-to-left instead.

If you must pan in the direction that is severely affected by rolling shutter artifacts, reduce your speed. As noted in Option 1, moving the camera slowly gives the sensor more time to process the light, which will reduce the amount of rolling shutter distortion in your shot.

Option 6: Use an Editing Program

When all else fails, and rolling shutter distortion persists, it might be time to pull out the big guns. Desktop video editing programs often come with a rolling shutter repair tool to digitally reduce the effect from the comfort of your computer.

For example, iMovie on macOS has a "Stabilization Effect." It's not a true rolling shutter repair tool, but Apple does note in the app that it can be used to reduce the unwanted effects of rolling shutter. We'll give it a pass, as Apple offers this program free of charge for all Mac users. (Note: the iMovie app for iOS devices does not have this feature.)

If you're willing to get professional, Final Cut Pro X and Adobe Premiere both have specific rolling shutter repair tools to tackle wobbly, unstable video, in addition to traditional stabilization effects. It doesn't seem that Avid Media Composer has a rolling shutter repair tool, but a community thread some years back details a few plugin options that might help get the job done.

Those programs, of course, are professional and will cost a professional amount of money. We wish there was a good option on mobile, but at this time, it doesn't appear that any of the capable editing programs on iOS or Android support this type of correction. If you're serious about producing video content, you might want to consider picking one up. However, try any of the other, cheaper options on this list first to see if they work for you.

This article was produced during Gadget Hacks' special coverage on smartphone-based video creator tips for filming and editing. Check out the whole Videography series.

Cover image by Jake Peterson/Gadget Hacks

Related Articles

638567153342457803.jpg

Instagram 101: Change Your Profile Name on Instagram to Increase Search Traffic to Your Account
636328552039893816.jpg

Instagram 101: Get Thousands of IG Followers Per Day with This One Simple Trick

Comments

No Comments Exist

Be the first, drop a comment!