H.264 Is Magic (2016)

dzdt

Article is dated (2016).

pwython

Ten years later, 2026 magic: AV1 https://engineering.fb.com/2026/06/22/video-engineering/adop... And soon AV2 https://av2.aomedia.org/

HelloUsername

Previous discussions: 4.nov.2016 https://news.ycombinator.com/item?id=12871403 219 comments 17.mar.2022 https://news.ycombinator.com/item?id=30710574 219 comments 24.may.2019 https://news.ycombinator.com/item?id=19997813 180 comments 14.jun.2024 https://news.ycombinator.com/item?id=40681306 205 comments

WhitneyLand

"The H.264 video, 300 frames long is 175KB. A single frame of that video in PNG is 1015KB. It looks like we're storing 300 times the amount of data in the video. But the file size is a fifth. So H.264 would seem to be 1500x as efficient as PNG. How is this even possible? All right, what's the trick?" Mostly the trick is they're comparing lossy to lossless compression which doesn't even make any sense.

tvelichkov

This reminds me that I still haven't watched `part 2` of a very nice video from 3Blue1Brown - "Compression is Intelligence". YouTube suggestions, definitely failed me with this one.

shevy-java

The only question I have: Should I use H.264 or H.265?

rishabhd

It is tradition to re-read this every year. Along with the m1 blackmagic one.

whall6

Socrates was certainly a pioneer of conveying information efficiently

MadameMinty

Worth pointing out that PNG is also just awful and should probably die like BMP died. WebP was already available at the time this article was written: PNG: 1039993 bytes - baseline optimized*: 541539 WebP: 398974 - lossless mode, of course. JPEG XL: 310778 - lossless, quick, and easy. *after monumental compute effort; still probably some room for improvement but the effort is ridiculous.

Hasz

I really like the idea that compression is intelligence, there are so many tricks an interesting ideas related to it. A bunch of them are described in the article, but you can imagine a long tail of many more inscrutable tricks chained together to get really incredible shrinkage in data. Hutter prize (I think mentioned in a 3b1b video somewhere else in this thread) is a great example. Given this was written before advent of LLM, I would love to see someone throw openAI/Anthropic levels of compute at a similar problem to see what the swarm can produce. https://arxiv.org/html/2404.09937v1

ksec

H.264 High Profile is close to Patent Free. [1] I often wonder how much more efficiency could we get if we were to increase block size outside of spec. [1] https://meta.wikimedia.org/wiki/Have_the_patents_for_H.264_M...

Dwedit

One year left before the "last major patent" expires.

Dwedit

One uncommonly used feature of H.264 is support for lossless video. It's a bit hard to use due to tooling (not easy to discover and select the feature), and is sometimes incompatible with web browsers, but you can get actual lossless video using a mainstream video codec. You can verify that it really is lossless, and doesn't have any conversion step where it's quantized to 8-bit YUV. "x264vfw" is one tool which supports this format, even inside of an AVI file. So older software that uses Video For Windows can create those files.

sacredcows

This reminds me of that old nursery rhyme. Billy was a chemist's son, but Billy is no more, what he thought was H2O was H.264

jruohonen

Ref.: https://news.ycombinator.com/item?id=38099633

amelius

I wonder if an AI can make a better codec, and if we would think of it as "magic".

tgtweak

Had the pleasure of working with Jason Garrett-Glaser (better known as DarkShikari) circa 2011 on a patch for x264 to better utilize more cores (slice-threaded lookahead). At the time he was working with Sony to optimize their PlayStation game streaming service, and this had the potential to lower encoding latency (critical to game streaming) even on same-core systems by spreading the frame over more cores to do more parallel work, and doing more intelligent scheduling and convergence after that tile-based work was reconverged into a frame. At the time we were using servers with 48 physical cores to do transcoding, a bit of a rarity in a single NUMA space back then and there were no instances to test this on so I flamegraph traced a few encodes at very high thread parallel counts and send back the instrumented profiles (from a custom compiled version) showing the core work being done and he used that to optimize the motion estimation code which worked on the first shot to perfectly level out cpu usage and increase encoding speed across ~24 cores (was previously topping out around 8 and running one core hot iirc). I got a look into how exotic and complex this project was, particularly in terms how low level and advanced things like motion estimation, quantizing and perceptual compression were... it was and still is sort of magic how many smart techniques are baked into something that we barely pay attention to. They were, even at that time, working on x265. His blog, Diary of an x264 developer, was a great read too for anyone wanting to get a vantage point on how crazy that codebase was. [1] https://github.com/mstorsjo/x264/blob/master/doc/threads.txt (edit: was Gaikai that was acquired by Sony in 2012 and is the basis for PlayStation Now)

dizzant

I'm surprised that 10 years later, the article still teaches an incorrect conceptual understanding of "quantization". The context heavily implies that quantization is "discarding information" generally (e.g. clipping, masking). The wikipedia article he links in the footnote disagrees, and so do I: > quantization is the process of mapping input values from a large set (often a continuous set) to output values in a (countable) smaller set, often with a finite number of elements. Quantization is a function over values, or the image itself. Clipping and masking, by contrast, are functions over the space/frequency dimensions, i.e. the pixel indices ("preimage"). A rule like "apply a circular mask" or "drop the first quadrant" does not depend on the values! The most extreme example of quantization is a threshold filter. Regardless of location, output "1" if the pixel is brighter than X, or "0" otherwise. This depends only on the value, not on the location. He also misses the opportunity to point out that chroma subsampling is another application of quantization, this time to chroma values (drop the low-order bits). Codecs like H.264 and JPEG use spatially-varying quantization rules to effectively discard more information at high frequencies, so his explanation of H.264 is spot on, but a non-expert encountering quantization for the first time would probably come away with the wrong idea. For an article explicitly targeted to non-experts that goes out of its way to introduce the term, it's a pretty important oversight.

tombert

I remember reading this article when it was new, and I left fully agreeing, but more broadly: modern video compression is magic. Like, it's baffling to me how much we can compress high-quality video nowadays. HEVC and AV1 and VP9 are all also very cool and they still sort of break my brain. I understand how they work but it doesn't feel like it should compress as well as it does.

Dan42

Well, that was disappointing. Everything in that article is generic to any video compression codec, even MPEG. I thought this would discuss why H.264 compresses so much more than predecessors, but not a word about that. Should have been titled "video compression is magic".