I've held off on this article as I did not feel the time was right, but it has become too evident that 4K is the next big thing for TV's and digital video recording. Many of us are lucky enough to own camcorders or cameras that output 1080p quality video, but now that 4K is becoming increasingly popular, should we toss those devices aside and purchase a 4K camera? I would like to argue that (for now) 1080p can suffice for 4K videos, and I would like to show you how and why...
Back in the Day
Just before I was about to be a sophomore, I switched high schools. I was late to the party so I only had a few choices for my electives. I opted for a video class that I thought would be easy; boy, was I wrong!
The former teacher was known to be very lax and the video class had been easy. But the current teacher wanted his students to really learn how to create video projects at a high standard. Before you were allowed to even touch a NLE (Non-Linear Editor), you had to learn how to edit with VHS. While not as hard as editing 8mm with a film splicer, it was still an arduous task. Once you had done one project on a VHS editing bay, you could then move onto a NLE.
When I first sat down in front of a computer (loaded with Adobe Premiere), I didn't know where to start. I could open the program, but that was about it. So I pulled over a classmate and asked them one question: Which button cut the video. From then on I educated myself on all the skills and technical aspects of production and post-production.
I was fortunate enough to get into digital video just before 24p became a huge revolutionary way to capture and output video. For the younger crowd, it was much like the jump from SD to HD (1080p). For newer crowds it will be akin to 1080p to 4K. I wouldn't say that things were simpler back then, but because there was less to contend with, it did give a sturdier background for digital video and all the new features that would start to arise.
Originally I wanted to do something in computers, but since that first class in high school, I have continued my studies and experimentation in digital video. I even went on to get a degree in film and media. That's not to say I forgot about computers altogether, otherwise this blog would not exist.
1080p vs. 4K
1080p is a video resolution of 1920 x 1080. 4K is a video resolution of 4096 x 2160. There are other 4K resolutions, but I won't get into that here (check "14 Things People Get Wrong..."). 4K is roughly 4 times larger than 1080p. You can check this by multiplying the numbers above then dividing the total for 4K by the total for 1080p. In pixels, the comparison is about 8 million to 2 million. This should help you understand why 4K is such better quality than 1080p, even without having a TV that will allow you to test that theory.
1080p vs. 2K
2K is a video resolution of 2048 x 1080. Again, there are some other resolutions, but I will not be getting into them. 2K is just barely taller than 1080p. 2K is the resolution used in most cinemas (there are some cinemas that do show movies in 4K).This may help understand why the need for cameras, camcorders, and TV's to have 2K never really came to fruition.
Be Forewarned
The best way to get 4K video is to obviously have a 4K camera or camcorder. A 4K smartphone may be cool, but it is not a good way to get 4K footage (even with hacked bitrates).
The only camera I know that can be hacked to become a 4K camera is the Sony F5, but the camera alone still costs thousands upon thousands of dollars. It would be better to get something like a Panasonic GH4, especially now since there are rumors that Panasonic may discontinue the line...
But if you can't afford those options, then follow along below.
Preparation for the Transformation
There are a few things you can and should do in order to make the best quality out of 1080p video footage.
The first is a pre-production step. If you own a Canon EOS, a Panasonic GH2, a GH1, or any other camera that can be hacked for higher bitrate, you should enable those settings. You will need to spend some ample research on how these are performed, how they can react, and how to reverse the process if needed. For an applicable Canon, you can use Magic Lantern, which will give you a setting to increase the bitrate. The process is easily reversible by just pulling out your battery. On a GH1 or GH2, the process is a little more in-depth as you need to actually replace the firmware and find a respective mod that will increase your bitrate (the Driftwood Moon or Cake is a good one for GH2 users). Of course, there are going to be a lot of cameras without any sort of hack, so you can skip this step if it does not apply. (Or pick up a cheap Canon that can use CHDK; camera list available here.)
For my tests I ended up using a hacked GH2 that got me about 80-100Mb/s for my 1080p footage. It came out superb and was a perfect candidate to try for a 4K upscale.
The second step is something anyone should be able to accomplish. After creating your video, but before throwing it into a NLE, download and install the free version of GoPro Studio. You can then increase your video footage's bit-depth from 8-bit to 10-bit. I won't go into the exact steps as I use NeoScene (which was a product that became discontinued when GoPro bought Cineform). They have the same capabilities, except that the GoPro Studio is an editing suite, and NeoScene includes the ability to convert AVCHD files (great for GH2 users like me!). If you want NeoScene for AVCHD footage, you may be able to find some valid serials sold on eBay. I have a posting here with the actual downloads that GoPro's site does not make available for discontinued users.
If you're asking why do the second preparation task, because you know that bit-depth is about color; it's because that this will likely help produce a better 4K version of your 1080p footage since it will have more color information when producing a top-notch 4K video. 8-bit deals with 256 colors, while 10-bit uses 1024 colors. And when doing the upscale process below, the computer is trying to produce non-existent details from details that do exist. Giving it more to work with is better than less in these circumstances.
How to Make 1080p into 2K
The easiest way to go about this is just to enlarge your video until the height is 2048 pixels high. Sure, some quality loss will occur, but not enough to be noticeable to the naked eye. To reiterate, there isn't much of a reason to do this unless you are specifically asked to do so (i.e. a client needs his digital video in 2K).
If this isn't satisfactory, you can use similar steps that will be discussed for 4K.
How to Make 1080p into 4K
Now, you could just scale 1080p footage to the dimensions of 4K, but unlike 2K, the quality will be substandard. Because we are enlarging the footage more than 4 times its original dimensions - where 2K is a slight pulling of the height - the quality will immediately take a hit and be noticeable by enthusiasts and the general public alike.
A simple way, which I do not recommend, is first scaling the 1080p footage to 4K, then use some post filter effects. You can throw on some Gaussian blur to smooth over the banding and artifacts that will undoubtedly arise, and use a bit of sharpen to get back a bit of that grain for realism's sake. This can be useful for quick jobs, but it doesn't give it the video footage quality it deserves.
Instead, you should use Adobe After Effects CC. If you don't have it, try out a trial version. If you use up the trial, either buy it or reinstall your OS and do the trial again (a bit extreme unless you have a spare computer you don't actively use).
Not long ago Adobe introduced a new "detail-preserving upscale" feature for Adobe After Effects CC. It is an effect and is fairly easy to use. The results are quite phenomenal, even moreso when compared to just a standard upscaling of the footage.
Now that you have the required tool to perform this trick, follow these steps to transform your 1080p footage into 4K:
- Open After Effects
- Select "New Composition".
- Change the values to 4096 x 2160.
- Name the composition whatever you like.
- Click "OK."
- Import the 1080p footage into your "Project" panel. (Drag-and-drop your footage on the Project panel, or right-click the "Project" panel and go to Import > File... to browse for your footage. File > Import > File... also works.)
- A "comp" will be created along with the footage import.
- Drag-and-drop the footage into your comp timeline.
- In the "Effects & Presets" panel, type in "Detail". The "Detail-Preserving Upscale" effect should appear. (If it does not, you are not on the latest version of Adobe After Effects CC.)
- Drag-and-drop the effect onto the footage in your comp timeline and it should automatically apply to your footage, and open the "Effects" panel.
- Ensure your timeline is at the beginning of your footage by dragging the time cursor to the start of the footage.
- Click the "Scale" stopwatch provided underneath the "detail-preserving upscale" effect in the "Effects" panel.
- Use the "Scale" option to increase the size of your footage until you fill the entire frame.
- Click the "Detail" stopwatch provided underneath the "detail-preserving upscale" effect in the "Effects" panel.
- Increase the amount of the "Detail" to 100%.
- Click the "Alpha" stopwatch provided underneath the "detail-preserving upscale" effect in the "Effects" panel. (This step and 17 are optional, however, it does seem to look slightly better if done.)
- Change the value from "Bicubic" to "Detail-preserving".
- Go to Edit > Add to Render Queue
- Click it.
- The render queue window should appear in place of the comp timeline. Adjust your settings as preferred.
- Click "Render".
Note: While 4096 x 2160 is true 4K, many "4K" TV's only support up to 3840 x 2160. So, you may want to check which value you should use depending on your monitor or TV that will be used for viewing.
Note II: The stopwatch is usually not needed unless you plan on changing that selected item's actions during playback. However, I found that not clicking on it for this effect made it ignore the filter altogether when outputted from the render queue. For my tests, it seemed to work if I just did the stopwatch for scaling, but to be safe you may want to do the stopwatch for all the effect properties.
There is no perfect upscaling solution, such as that, this is not a perfect solution. But this is by far the best method I have come across, as well as the least time-consuming. There will never be a perfect solution for upscaling because your are taking something smaller and making it bigger, which will always cause banding and artifacts.
Downscaling
If you don't know much about video and are thinking you should use the above steps to downscale video, don't. It's really just a waste of time. Problems in detail and quality only happen when going from small-to-big, not big-to-small. Decreasing the size of something will retain its detail and not create any of the issues noted above. You can simply scale down the footage in any NLE and be fine.
However, if you go from big-to-small, then attempt to go back by using small-to-big, the problems above will occur. It would be smarter to keep a backup of the full size as it will be perfect in comparison to trying to match the original with the solution I have presented.
4K You, 4K Me, 4K Everybody!
I have found other software to perform upscaling, but all have mediocre results and cannot come close to the quality retention Adobe After Effects provides. I should say that the larger the gap in dimensions, the less useful any of this becomes. If I try to upscale 480p to 4K, the results should look horrible regardless of what program is used. On the flip side, If trying to upscale from 4K to 6K, this method should still be valid. Because 8K is so massively large, this technique may or may not be valid when coming from a 4K source (certainly not 1080p).
If you need to test your footage, you can always upload it to YouTube where it should give you the option to use 4K when viewing. And if you need to keep the great quality of your 4K video, but need a more manageable file size for sending or uploading, check out "VP9 + Opus = WebM...", which will also have the 4K example I made.
Another thought to remember is that your footage is not 4K, just your outputted video is. For most of us, this method will suit us fine. Others may rather purchase a 4K camera for the real deal. But that doesn't mean to trade-in your 1080p camera, as you may need a dual-setup one day, and this technique can help produce a beautiful 4K video from that second camera that only offers 1080p...
I
try to keep myself up-to-date with the latest video codecs (such as
ARRI's new 3.2K ProRes) and containers. And it is time again for some new codecs and containers to take the field. With HTML5 large and in charge, online video will quickly become a bottleneck issue...
H.264
The
H.264 codec has been, and still is, a huge staple for video
compression. Whether it be for video cameras or a Blu-ray disc, H.264
provides great compression with great quality. There have been licensing
issues that have sprung up, but moreorless, H.264 has become the
standard for video compression everywhere.
The New Standards
As
with DVD to Blu-ray, H.264 could only last so long as demands for
better compression that sustain high quality video will and are needed.
One of the most obvious reasons for this is online video. While web
hosting can be purchased at inexpensive rates, adding more storage can
be extremely costly. And if you are constantly loading up videos, even
H.264 won't be able to help that budget. YouTube is a great alternative,
but some may want to host their own video files.
HTML5
HTML5
is meant to have already been standardized this year [July], and it is
now widely accepted across browser platforms. Even sites like YouTube
have "experimental" HTML5 video interfaces. The one thing HTML5 needed
though was faster video to go along with it...
H.265 vs. VP9
The
new codecs that are coming to the aid of video compression are H.265
and VP9. Both are meant to drastically reduce compression size while
still maintaining high quality. They are still in their infancy stages,
but are already a forced to be reckoned with.
H.265
is said to be better - in quality - than VP9. It is the obvious
successor of H.265, and from what test stills I have seen, it looks
amazing! Compression rates can supposedly reach 1/4th the size of H.264.
The H.265 stills I previewed were nearly impossible to tell from the
H.264 stills. I could only recall one still that seemed slightly worse
than the H.264 still. That being said, many are worried that H.265 will
have the same licensing issues that plagued H.264, but there is talk of
it being free, however unlikely that may be.
VP9
is from Google and is free. I have done a few tests and it is just as
amazing as H.265, in my opinion. I have had varied results in comparing
compression size of VP9 videos over uncompressed videos I have outputted
(such as a short intro at .5% or a full-length video at 12.5%), but
they have always come out just as perfect as the uncompressed video
itself.
NOTE:
I am not mentioning Daala as I have do not have a way of testing this,
and I don't believe the codec is widely support yet.
Great! So, where can I put this?
If
you have your own website and hosting, you could use an HTML5 video
player for VP9, as many of the latest browsers are able to decode it.
The same should go for H.265. For those of you using free websites, or
even those with paid hosting but little storage is available, YouTube
has you covered if you go the VP9 route!
Can you contain me? Can you hear me?
VP9 is just a video codec, what about the container and audio codec? What will YouTube accept?
WebM
is a container that will happily accept VP9. It is supported for HTML5.
As far as audio, Opus seems to work with VP9 in my tests, and is of
excellent quality (if that is a worry). It too is supported by HTML5,
and both are supported by YouTube (I did not find Opus officially
mentioned on their site, however). There, the secret's half-way out!
Why Should I Use This?
I had a couple reasons to use this that may or may not apply to you. My first was to simply get used to dealing with VP9. It's the future, and I intend to be ready for it. Another reason was that I had an old green screen video that would always look horrendous when uploaded to YouTube. I am hopeful that this method will fix this.
And maybe most importantly, I currently reside in a part of the world where Internet is slow and still costly. I can spend less time by encoding to VP9 and then uploading to a cloud service or YouTube, rather than trying to upload an uncompressed or high-quality H.264 MP4 video.
A final reason, that I'll go into detail below, is that it seems that less compression is had when using VP9 in YouTube. Of course VP9 is a form of compression, but the quality is so incredibly high for such a low file size that it is impossible for even me to see the difference between an uncompressed and VP9 video - when done right.
How to Convert Your Video to WebM (VP9 + Opus) for FREE!
This technique will show you how to quickly and easily convert your video to WebM using the VP9 and Opus codecs. While you can change the settings, doing so may cause issues in playback, and could greatly increase your time for processing.
Here are the steps:
- Output your video in whatever format you desire, but uncompressed may be best if possible.
- Download Hybrid, which is free.
- Install Hybrid, you can leave the install settings as-is.
- Open Hybrid.
- From the Main tab, choose "VP8/P9" for "Video handling" (the H.265 tab should change to VP8/9).
- Ignore this step if you do not have audio. Choose "custom" for "Audio handling". (An Audio tab should now appear.)
- In the upper-right, click the man walking out the door with a right arrow icon.
- Select your video.
- In the middle-right, click the man walking in the door with a left arrow icon.
- Select your folder and put in a file name with the extension ".webm".
- Click the VP8/9 tab.
- Choose "VP9" for "Codec".
- Choose "good" for "Basic speed control".
- Set "CPU utilization modifier" to the number of cores your CPU has. For example, 6-cores equates to a 5, while a single core equates to 0.
- You can leave "Encoding mode" as-is, but I recommend changing the bitrate to at least 6,000kbps (10,000kbps optimal), especially if the video has any titles or credits.
- Click the Audio tab. (Skip to step #20 if you have a silent video.)
- Check off "Reencode".
- Choose "opus" for "Audio format". (You can mess with other audio quality settings, but it is unnecessary, and may not work.)
- There are two "+" icons, click the one on the far right with up and down arrow icons below it.
- Click the Config tab.
- Clikc the Internals tab from the second row of tabs.
- Check off "Always use Avisynth".
- Click the Filtering tab.
- Click the Avisynth tab from the second row of tabs.
- Click the Misc tab from the third row of tabs.
- Check off "LibavVideoSource instead of FFmpegsource".
- Click the Jobs tab (only necessary if you have done this process before, otherwise skip to step #23).
- Highlight all "jobs" already in the queue.
- Right-click.
- Select "remove".
- Click the Main tab.
- Click the man shoveling with a small box containing a "+" icon.
Note: If for some reason your video outputs upside-down, go to Config > Input and check off "Flip raw rgb" under Decoding. This should only occur if you have a video that was made in a raw RGB format.
Give It Time!
My 2:56 minutes music video took less than an hour with the above settings. This will obviously extend the longer your source video is.
Playback?
I found that many video players would not play the file correctly. I normally use VLC, and the current 2.1.5 version already supports VP9, but for some reason will not play these files. I have been told that the new nightly builds will. However, MPC-HC plays the files just fine.
But That's All the Drawbacks, Right?
Well, that really depends on your settings. If you follow the settings above, then yes. But if you begin changing them, you could incur other drawbacks.
If someone were to choose "Best" instead of "good" for "Basic speed control", a short 3-minute video would take a few days to process! Or if someone were to choose 0 for "CPU utilization modifier", then it could take a lot longer to process since only a single core would be in use. And if someone were to leave "LibavVideoSource instead of FFmpegsource" unchecked, their video could come out with a stuttering effect.
Premiere Pro Users
If you are a Premiere Pro user, there is an option available to you to compress your video to WebM as VP9 in-program. Here is the free plug-in for that. Here is a free plug-in, from the same person, for Theora;
if you have any use for it. Place these in the "Plug-ins" folder of
where your Adobe Premiere Pro folder is. If you still don't know what to
do, there are plenty of articles on the web explaining how to place
Premiere Pro plug-ins.
The
reason I am not going into detail on how to use Premiere Pro for VP9 is
that I successfully attempted this and was able to get it going easily.
But after 4 days, it seemed the progress bar got stuck, and soon after,
I had a power outage and lost all the progress. It seems the maker of
the plug-in admits there are still issues, and I did find at least one post about the same problem I encountered.
You are
able to use two cores if you follow the plug-in maker's instructions,
but this is for best quality and will slow down the process further!
Examples
I have made three examples from the music video I tested this with. All three are in 1080p and uploaded to YouTube:
Uncompressed (16GB+) - On My Mind
H.264 (3.5GB+) - On My Mind
VP9 (123MB) - On My Mind
YouTube MP4 Download Size
Uncompressed - 76MB+
H.264 - 75MB+
VP9 - ≈79MB
I
have a slow Internet speed, and am constantly using up my bandwidth, so
each upload was a long wait for me. But for those with speeds triple 3-4Mbps, it should be a breeze.
YouTube Compression Percentages
You'll
notice, if trying to download these videos as MP4, that the uncompressed,
H.264, and VP9 videos are all practically the same size after YouTube compression.
Some
simple math tells us that the VP9 video is about 3.5% of the entire
size of the H.264 video. One might incorrectly think that the YouTube compression
size should then be equally proportionate, but it isn't. The VP9 MP4
size is actually about 105% of the H.264 (and uncompressed) MP4 video. If we compare the MP4 sizes to their
original counterparts we get almost .5% for uncompressed, around 2.14%
for H.264, and 64.23% for VP9. This is a good thing as we want as little
compression done by YouTube as possible in order to maintain high quality.
Music Video
The footage was all shot on a Panasonic Lumix GH2 hacked to use MJPEG (.MOV @ 30fps) at 80-100Mbps, utilizing mostly Canon FD lenses. I used Premiere Pro to edit and Da Vinci Resolve Lite to color grade. I did add a few effects in Premiere Pro as well.
This
was the first day of using my GH2, and I was rushing to learn how to
use and hack it. I should've been able to achieve the same bitrates with
AVCHD, but something wasn't right. I couldn't experiment more as I was
losing sun. We shot our guerrilla-styled music video in about 2-3 hours.
Thumbnail Tidbit
One
last odd thing I noticed from YouTube was despite being the exact same
music video, the VP9 version had different thumbnail choices than the
other two. Why? I don't know...
Conclusion
VP9 quality is superb, and
the compression is as amazing as it is made out to be. Once VP9 has gone
through its trials and gains more support, I am sure it will be one of the best
codecs for uploading, sending, or displaying video.
NeoScene is a discontinued software product. If you were a digital filmmaker a few years back, you may have heard of it. If you were a color corrector a few years back, you may own it. Even if you have not heard of it, I am here to explain why digital filmmakers should be severely disappointed it no longer exists.
What WAS NeoScene?
NeoScene was a product of CineForm that converted certain camera files (official camera list) from an 8-bit colorspace to a 10-bit colorspace. While this had no affect on the shooting of a video, it would greatly improve color grading. The price was $129, and was available until late 2012/early 2013. When GoPro bought CineForm, NeoScene was discontinued.
8-bit vs. 10-bit
While 10-bit over 8-bit seems like a small difference, it actually makes a huge impact. 8-bit is normally shot by cameras that have a chroma subsampling of 4:2:0 or 4:1:1 (which are arguably the same in quality). Without going into great detail, the three numbers denote light and color. The best images are 4:4:4, but this is uncompressed footage and thus requires large amounts of storage. 4:2:2, used in 10-bit cameras, is said to be unnoticeable when compared to 4:4:4 with the human eye. In addition to looking good, 4:2:2 is compressed, which means it saves storage space (which can greatly lessen budget costs).
The above would seemingly be useful for when shooting video, not performing post-production. However, color is involved. When color bit-depth is increased the color palette increases as well. 8-bit colorspace is made up of 256 colors. 10-bit colorspace is made up of 1024 colors. That is a 300% increase! And for a color corrector, this extra amount of colors enables a much wider range of precision on overall color quality.
GoPro Studio Free vs NeoScene
GoPro restructured all of the products offered by CineForm, and NeoScene was the last to go. NeoScene became integrated into the GoPro Studio editions, including its free edition! This was one of the many upsides of the GoPro restructuring. The GoPro Studio Free boasts not only the free bit-depth conversion, but it also gives a free editing and color correcting suite, and it gave the CineForm MOV export codec to 3rd-party applications like Sony Vegas or Premiere Pro.
The GoPro Studio Premium ($299) and Professional ($999) versions state 10-bit and 12-bit (4096 colors) chroma formats on the GoPro website, which I assume means that it can convert 8-bit and 10-bit to larger bit-depths. Knowing all this would make anyone think that GoPro's restructuring was a welcome change...
The NeoScene Legacy
Despite the incredible amount of items offered with the GoPro Studio Free edition, it had dropped one major aspect that NeoScene had provided: AVCHD. Not even the paid GoPro Studio versions make mention of this capability.
This means that any camera that uses the popular AVCHD recording format is now out of luck. There are cameras that can switch between the AVCHD and MJPEG format; but it is widely argued that AVCHD provides better quality (if for nothing else) through less noise. And if someone uses a camera such as the Panasonic Lumix GH2, that can be hacked for extreme bitrates and quality, not having this feature is a huge letdown.
What Are Ya Gonna Do?
If a camera does not use AVCHD, then there is nothing to worry about. If a camera uses AVCHD, but has the MJPEG option, then MJPEG may be a better way to go. This can also be tested by shooting test clips in both formats, then seeing how the color correcting holds up for the 8-bit footage (AVCHD) and the converted 10-bit footage (MJPEG).
The last option is to use NeoScene. GoPro has since rid itself of any download links to NeoScene (which is bad form by any company with discontinued products). I went on a hunt to find any and came up dry. Fortunately, I had saved a few as I always worried something like this would happen:
The bundle includes 4 versions of NeoScene for Windows (1.6.0, 5.5.1, 5.5.4, 5.6.1), one Mac version (5.5.5), ac3filter 1.63 & 2.1 (2.1 has been reported to cause errors), and documentation from the website (from December 2012).
NeoScene Downloads
The NeoScene downloads should have a trial that I believe lasts 30-days. If an old customer still has the activation/serial information, use that and it should become the full version.
For those who were not lucky enough to purchase an activation and serial number while NeoScene lived, refer back to some of my older posts on how to get around trial limits, legally. (Hint: It involves Windows System Restore points and disconnecting the Internet.)
