This article describes briefly what H.264 is and how to get H.264 encoding support for Avidemux. It also summarizes and explains the x264 options available in Avidemux. This can be considered a (simple) guide to the encoder.

H.264, which is also known as “MPEG-4 Part-10” or “MPEG-4 Advanced Video Coding” (AVC), is a digital video compression standard, which is noted for achieving very high data compression. While H.264 generally requires more CPU power for playback than video encoded with the older MPEG-4 Part-2 standard (as used by Xvid or DivX), the compression efficiency is much better! That means: With H.264/AVC you can get a significant better quality at the same file size -or- you can get the same quality at a significant smaller file size (compared to MPEG-4 ASP). While H.264 compresses much more efficient than MPEG-4 Part-2, the advantage over MPEG-2 is even greater.

More detailed information about H.264 can be found in the corresponding Wikipedia article. A comparison of various H.264 encoders against MPEG-4 Part-2, MPEG-2 and other video formats can be found here.

While Avidemux uses “built-in” libavcodec from FFmpeg for H.264 decoding, it needs an additional (external) library for H.264 encoding. Therefore Avidemux uses x264. x264 is a free library for encoding H.264/AVC video streams. The code is written from scratch by Laurent Aimar, Loren Merritt, Eric Petit (OS X), Min Chen (VfW/asm), Justin Clay (VfW), Måns Rullgård, Radek Czyz, Christian Heine (asm), Alex Izvorski (asm), and Alex Wright. It is released under the terms of the GPL license. So to clarify, the encoder library is called x264 while the compression standard it uses is called H.264 (or MPEG-4 AVC). In other words: The x264 encoder software creates H.264/AVC video. It should be noted that x264 while being “free” software can compete with commercial H.264 encoders in terms of quality and speed. Major companies in the video business, such as Youtube and Facebook, are known to use the x264 encoder.

If x264 is not available in your version of Avidemux, there is a guide on how to download and compile x264 by yourself. It is in the H.264 section.

After you compile x264, you will have to re-compile Avidemux to build in the x264 feature. There is also a guide on how to do this in the Install -> Compile Avidemux from SVN (Subversion) section.

Note that if you are using the pre-compiled Avidemux builds for Microsoft Windows, the required x264 library ships with the installer. Hence no additional software is required! Stuff like “Codec Packs”, “VFW Codecs” or “DirectShow Filters” will not work with Avidemux! Anyway, the latest builds of the x264 library for Avidemux can be found in this thread (make sure you navigate to the very last post!). These builds usually are newer - and less tested - than the ones that ships with Avidemux.

Avidemux contains most of the options available in the x264 library. For options not yet available, see the “Unavailable” section in this article.

x264.png

• Encoding Mode:
  • Single Pass - Constant Quantizer: This mode is also known as the “QP Mode”. It will encode your video to a constant quantizer, so you will choose the target quantizer, not the target bitrate. The quantizer is a measure for the amount of data loss: a higher quantizer means that more data will be lost, which results in a better compression (smaller file), but also delivers worse visual quality. In contrast a lower quantizer means that less data will be lost, which results in a better visual quality, but also compresses worse (larger file). H.264 uses a quantizer scale between 0 and 52. The default quantizer value is 26. If you are targeting for a certain level of quality and don't care much about the final file size, then you might consider using the QP mode. But if you are targeting for a certain file size (or a certain average bitrate), then keep away from QP Mode! That's because the final size (the average bitrate) is completely unpredictable in this mode.
    • Please note: The CRF mode should be preferred over the QP mode! Also the Adaptive Quantization (AQ) will be disabled in QP mode, while it's enabled (by default) in CRF mode.
  • Single Pass - Constant Rate Factor: This mode is also known as the “CRF Mode” or “Constant Quality” mode. It basically works similar to the QP Mode (see above), but it will encode with an average quantizer instead of a constant one. To be more precise, this mode encodes at a constant “rate factor”, which is derived from the specified quantizer. Internally CRF mode uses the same ratecontrol algorithm as x264's ABR mode, only without a target bitrate. The advantage of the CRF mode is that it suits the human perception much better than the QP mode. For example it will raise the quantizers in “fast” scenes where the loss won't be visible anyway and lower the quantizers in “slow” scenes. Therefore the CRF mode should give the same subjective quality as QP mode, but it usually achieves a significant higher compression. It's recommended to prefer CRF mode over QP mode, although CRF is a bit slower. When switching from QP to CRF mode, you may want to slightly lower the quantizer. This should give approximately the same file size as before, but better visual quality! Another important advantage of CRF mode is that it will benefit from adaptive quantization, something that QP mode can't do.
    • Please note: Even the CRF mode doesn't deliver “perfect” constant quality! A specific CRF value will only deliver similar quality for various sources, as long as you don't change any other settings. Using “slower” settings with the same CRF value will either produce a smaller file at same quality or a produce a file of the same size at a better quality. It's also possible that both, the size and the quality, will be increased. The “quality per size” ratio will be improved anyway.
  • Remarks: Choosing the proper quantizer setting for a CRF (or QP) encode is not trivial! That's because visual quality is highly subjective: What some people consider “good quality” other people will consider “horrible quality” - and vice versa. Furthermore the quantizer setting highly depends on contents of your video. Nevertheless a quantizer setting in the range between 16 and 32 should give satisfactory results in most cases. Using a quantizer lower than 16 usually is overkill, except for mastering purposes. Using a quantizer higher than 32 will result in almost unwatchable video. A quantizer of 22 seems to be reasonable for most purposes. Nevertheless material with few textures, like Anime and Cartoons, can cope with much higher quantizers. At the same time “real life” footage with a lot of textures might require much lower quantizers, especially in dark scenes. There also is a rule of thumb: Lowering the CRF value by 6 will double filesize, lowering CRF by 1 will raise filesize by ~12.5% (very roughly). Furthermore the common practice is as follows: Start with a low CRF value, such as 16. Then raise the CRF value in steps of one, until the quality becomes intolerable. This way you'll find the highest possible CRF value that still gives an accepted quality for your eyes. Once you found it, you can use that value for all your future encodes.
  • Single Pass - Bitrate: This mode will encode your video at an average bitrate with only one single pass. So this mode only requires half the time of a “Two Pass” encode. In contrast to CRF mode (and QP mode) the resulting average bitrate is known in advance. Therefore it's easy to predict the final file size. A higher bitrate will result in a better visual quality, but of course it will also result in a bigger file. A lower bitrate will results in a smaller file, but it will also result in a worse visual quality. Unfortunately the encoder doesn't “know” the content of the video in advance when encoding with only one pass. Hence the encoder's ability to adjust the bitrate with respect to the content of the video is extremely limited in this mode! Only “local” optimizations are possible. This results in a pretty bad video quality (compared to a “Two Pass” encode), especially at medium and lower bitrates! Therefore it's highly recommended to not use this mode, unless you really need to do it in one pass.
  • Two Pass - Average Bitrate: This mode will encode your video at an average bitrate and it will use two encoding passes. Consequently this mode requires twice the time of a “Single Pass” encode (roughly). In contrast to CRF mode (and QP mode) the resulting average bitrate is known in advance. Therefore it's easy to predict the final file size. A higher bitrate will result in a better visual quality, but of course it will also result in a bigger file. A lower bitrate will results in a smaller file, but it will also result in a worse visual quality. During the first pass the encoder will perform a detailed analysis of the video and create a so-called “stats” file. Then during the second pass the actual encoding takes place and the final file is created. The advantage of using two passes is that during the second pass the encoder can rely on the data collected during the first pass. This allows the encoder to distribute the available bits among the entire video. For example “high motion” scenes will get a significant higher bitrate than “static” scenes. This is done in order to keep the visual quality constant over the whole movie. Ugly “blocking” on fast/spontaneous movement (as seen in “Single Pass” encodes) is avoided. Therefore a “Two Pass” encode provides the maximum visual quality for the given target bitrate (file size). It's highly recommended to always use this mode, if you are targeting for a certain average bitrate!
  • Two Pass - File Size: This mode will actually use the “Two Pass - Average Bitrate” mode. The one and only difference is that Avidemux will automatically calculate the required bitrate for you. This way a specific target file size can be hit easily. Just enter your desired file size (for example “700 MB” for a CD-R media or “4700 MB” for a DVD-R media) and that's it! All the rest works exactly as described for the “Two Pass - Average Bitrate” mode.
    • Please note: x264 will not take the audio bitrate and the container overhead into account. Hence the target size specified in the x264 dialog only effects the video part of the file. If your file contains at least one audio track, then the actual file will come out bigger than the specified size. Also the container adds some additional overhead to the file. So please use Avidemux' “Calculator” tool to set up the target file size properly!
  • Remarks: Choosing the proper target bitrate for a bitrate-based encode (“Two Pass” or “Single Pass” mode) is not trivial at all! That's because the bitrate required for satisfactory results highly depends on the “compressibility” of your video and also on your personal preferences. For example “clean” sources can get away with a significant smaller bitrate than noisy/grainy sources. Also animated footage usually can get away with much smaller bitrates than “real life” footage. Anyway, in most cases an average bitrate in the range between 500 kbps and 2500 kbps should give acceptable results for most SD material, like Video-DVD backups. Average bitrates above 2500 kbps are considered “overkill” for SD material. Of course exceptions exits! Also be aware that when dealing with HD material (720p or 1080p) significant higher bitrates will be required. Bitrates of 10 mbps and above aren't unusual for HD encodes. Note that pre-processing, such as denoising, can reduce the bitrate requirement of a source.
    • Please note: Since it's pretty hard to decide on a specific bitrate, you are usually better off by using the CRF mode instead of one of the bitrate-based modes.
  • Lossless Mode: x264 also supports “true” lossless compression. Using this mode there will be absolutely no degradation to your video data. But lossless compression will take significant more bitrate than any lossy compression. So converting from a lossy format (e.g. MPEG-2 or MPEG-4 ASP) to lossless compression will produce a file much bigger than the source! Anyway, x264 in “lossless” mode will usually still need less bitrate than other lossless encoders, such as HuffYUV or FFV1. And it's much smaller than uncompressed video (e.g. raw YUV/RGB data), of course. In order to enforce lossless compression, you must choose the Constant Quantizer mode and you must set the quantizer to a value of 0. Note that playback of lossless H.264 requires a decoder capable of the “Predictive Lossless” profile. Decoders that support this include libavcodec/ffmpeg (ffdshow, MPlayer, etc.) and CoreAVC Decoder. Other decoders may display garbled output (or no output at all).

• The setting enables Macroblock-Tree Rate Control (aka “MB-Tree”). It tracks the propagation of information from future blocks to past blocks across motion vectors. It could be described as localizing qcomp (quantizer curve compression) to act on individual blocks instead of whole scenes. Thus instead of lowering quality in high-complexity scenes (like x264 does without MB-Tree), it'll only lower quality on the complex part of the scene, while for example a static background will remain high-quality. It also has many other more subtle effects, some potentially negative, most probably not. This helps at all bitrates, but can even help at phenomenally low bitrates where the video would otherwise fall apart completely. Note that MB-Tree now handles fades much better thanks to Weight-P. Since MB-Tree greatly improves the overall quality, it should always be enabled. See this thread for more information on how MB-Tree works.
  • Frametype Lookahead: This settings specifies the number of frames for frame-type lookahead, i.e. the distance (in frames) MB-Tree looks into the future. The more frames MB-Tree uses for lookahead, the more efficiently it works. Or in other words: A higher value will improve the result, while a lower value will hurt the result. Therefore you should use the highest value you can afford. Unfortunately a larger lookahead will slow down the encoding speed and increase memory usage at the same time. The default value is 40, which should be well-balanced for most purposes. If quality is more important than encoding speed, you should increase the value. However going higher than 60 is not recommended, because even higher values will only give a very minor additional improvement (if at all). If speed is more important the quality, you may lower the value. But going lower than 30 is not recommended.
  • Warning: If you encounter crashes with high Frametype Lookahead values, then this is probably because you ran out of memory! In that case you must lower the value in order to avoid the crash.

• This setting controls how many threads x264 will use for encoding. Thanks to its multi-threading implementation, x264 is able to fully utilize the processing power of modern multi-core processors. This is achieved by encoding several frames in parallel (see http://git.videolan.org/gitweb.cgi?p=x264.git;a=blob;f=doc/threads.txt for details). Tests have shown that x264 scales extremely well up to at least 16 cores. Anyway, to make optimal usage of your multi-core machine, the correct number of threads must be selected. The following options are available:
  • Disabled: Disables multi-threading. One single thread will be used. This makes no difference for single-core machines, but will significantly slow down x264 on multi-core machines.
  • Auto-Detect: Let x264 decide the optimal number of threads. The formula used is: threads = cpu_cores * 3/2. Tests have shown that this formula (usually) gives the optimal performance. It's highly recommended to keep this setting!
  • Custom: Manually overwrite the x264 auto detection. Only use this if you have a really good reason to mistrust x264's detection.
  • Remarks: Many people complain that the CPU load doesn't reach 100% in Taskmanager when encoding a video with x264, even with multi-threading enabled. This may have several reasons. Most likely something in the processing chain is bottlenecking x264. For example a single-threaded decoder and/or compute-intensive video filters may easily become the performance bottleneck. In that case x264 has to wait for the input and becomes idle. So in fact it's not a problem in x264 itself. Another “problem” is Intel's Hyperthreading technology, as used by the Pentium 4 and the Core i7 (Nehalem) processor. With Hyperthreading there are two virtual cores per physical core. Hence a CPU load of 50% indicates that all physical core are busy (which equals 100% load on a None-Hyperthreading CPU). Last but not least the efficiency of multi-threading shouldn't be measured with CPU load as displayed by Taskmanager. Instead the throughput (that is: the number of frames encoded per second) must be measured. So please keep in mind that high CPU load alone doesn't imply good performance!

• ME Method: Video compression works by discarding redundant information between consecutive frames. For example P-Frames will be predicted from the previous frame(s). Then only the difference between the predicted frame and the original source frame is saved in the bitstream. That is called the “residual”. The more accurate a frame is predicted, the less data needs to be stored. Because objects tend to move between neighboring frames, detecting and compensating motion is essential for an accurate prediction! The ME Method determines which algorithm is used to search for motion and to calculate the so-called “motion vectors”. Using a more accurate search method will result in a better visual quality, but will also take more time for encoding. Using a faster method will speedup the encoding process, but will also result in a worse visual quality. Since the ME Method has a huge impact on encoding speed and on the visual quality of your video, you should decide carefully! It's highly recommended to not go below the default setting. You should consider an even slower mode, if quality is more important than encoding time. The following methods are available:
  • Diamond Search (DIA): Four sided shape analysis - This is the fastest method, but it also provides the worst quality. Use this for method only if encoding speed is more important than quality.
    • Complexity: O(n) in worst case, faster in average case.
  • Hexagonal Search (HEX): Six sided shape analysis - The is the default method. It provides reasonable quality and still works pretty fast.
    • Complexity: O(n) in worst case, faster in average case.
  • Uneven Multi-Hexagon Search (UMH): More detailed version of Hexagonal search - This method provides high quality, but works slower than the simple “HEX” method. If you prefer quality over speed, then use this method.
    • Complexity: O(n).
  • Exhaustive Search (ESA): Complete and extensive analysis - This brute-force method works very slow, but the resulting quality usually is only slightly better compared to the “UMH” method (if at all).
    • Complexity: O(n²).
  • Hadamard Exhaustive Search (TESA): Improved version of “ESA” method using Hadamard transform - This method runs even slower than the “ESA” method. Use this method, if you have got a lot of time to waste.
    • Complexity: O(n²).
  • Remarks: Tests have shown that the “Exhaustive Search” is significant slower than “Uneven Multi-Hexagon Search”, but it doesn't necessarily produce a better perceived quality. Furthermore “Hadamard Exhaustive Search” will take at least twice as long as “Uneven Multi-Hexagon Search”. Therefore using a method slower than “Uneven Multi-Hexagon Search” generally isn't worth the extra encoding time.

• Subpixel Refinement: This setting (also known as “Sub ME”) controls the precision of the motion estimation process. The higher the precision, the better the results. Therefore you should always use the highest mode that you can afford. Of course higher precision takes more time for encoding. Note that regardless of the setting, QPel motion estimation is always used. RDO is equal to using the VHQ setting of the Xvid encoder. It's highly recommended to not go below the default value of 6, because Psy-RDO requires at least Sube ME 6. If you have the time, then you should consider using an even higher value. In case visual quality is more important than encoding time, you should even go with the maximum! Mode 9 (or even 10) gives the best quality. If you prefer speed over quality, use mode 2. You should never go below mode 2, even for “fast and dirty” encodes.The following Sub-ME modes are currently available:
  1. QPel SAD (Fastest, worst quality)
  2. QPel SATD
  3. HPel on MB, then QPel
  4. Always QPel
  5. QPel & bidirectional ME
  6. RD on I- and P-Frames (Default, lowest mode that supports Psy-RDO)
  7. RD on all frames
  8. RD refinement on I- and P-Frames
  9. RD refinement on all frames
  10. RD refinement on all frames + QPRD (Slowest, best quality)

• Range: This setting defines how many pixels are analyzed for motion estimation. Higher range values result in a more accurate analysis, but will also slow down the encoding speed significantly. Lower values will speed-up the encoding process, but will also result in a less accurate analysis. Note that high resolution material generally benefits more from higher range settings than low resolution martial. That's because objects tend to move farther (with respect to pixels) in HD video. Anyway, the default value of 16 is sufficient for most videos! The “Diamond Search” and “Hexagonal Search” methods are even limited to maximum range of 16. If quality is more important than encoding speed and if you are using the “Uneven Multi-Hexagon Search” method (or an even slower method), you may want to raise range to a value of 24 or even 32. Depending on the selected ME method, the range value may be rounded up to a multiple of two or four.

• Maximum Motion Vector Length: This setting can be used to limited the maximum length of each motion vector. By default x264 will limit the maximum motion vector length based on the detected level. You can use this option to overwrite x264's decision. It's highly recommended to not use this option, unless you have a very good reason to do so!

• Minimum Buffer Between Threads: x264 uses a frame-based multi-threading method. To allow encoding of multiple frames in parallel, x264 has to ensure that any given macroblock uses motion vectors only from pieces of the reference frames that have been encoded already. This is usually not noticeable, but can matter for very fast upward motion. By default x264 will decide the minimum space between threads based on the number of threads. You can use this option to overwrite x264's decision. It's highly recommended to not use this option, unless you have a very good reason to do so!

• B-Frame Direct Mode: This feature allows B-frames to use “predicted” motion vectors instead of actually coding each frame's motion. This should save some bitrate and will improves the compression. Therefore it's recommended to always keep this setting enabled. There are four different modes available:
  • None: Disabled. For testing only. Not recommended.
  • Auto: Let the encoder decide the optimal setting for each frame. Highly recommended for all RC modes, but more efficient in Two Pass mode.
  • Temporal: Enforce prediction from neighboring frames.
  • Spatial: Enforce prediction from neighboring blocks within the current frame (usually preferred over Temporal).

• Weighted Prediction for B-Frames: This feature allows the encoder to produce more accurate B-Frames by “weighting” the reference frames in a none-symmetric way. This comes at the cost of some encoding speed. Since weighted B-Frames will generally improve the visual quality, it's recommended to always keep this setting enabled, except encoding speed is more important than quality.

• Weighted Prediction for B-Frames: This feature allows the encoder to produce more accurate B-Frames by “weighting” the reference frames in a none-symmetric way. This comes at the cost of some encoding speed. Since weighted B-Frames will generally improve the visual quality, it's recommended to always keep this setting enabled, except encoding speed is more important than quality.

• Weighted Prediction for P-Frames: This feature allows the encoder to detect fades and weight the P-Frames accordingly. This greatly improves the quality in fades and thus should always be used!
  • Blind Offset: Uses a “blind” offset without performing an analysis. Provides only a small quality improvement in fades.
  • Smart Mode: Fade detection. Provides full quality improvement in fades. Especially useful with MB-Tree. This is the recommended mode!
  • Disabled: Don't use Weight-P at all. Not recommended.
  • Warning: Some H.264 decoders are known to be broken with respect to Weight-P. See this post for a list of broken decoders. With Weight-P enabled you will get distorted output, if you use one of the broken decoders! Most notably CoreAVC Decoder 1.9.x has a well-known Weight-P bug that won't be fixed. Either update to CoreAVC 2.0, use a different decoder (e.g. ffdshow or DivX H.264 decoder) or disable Weight-P. The last solution is the worst, of course.

• 8×8 Adaptive DCT Transform: This settings enabled the adaptive 8×8 DCT transform. This will significantly improve the visual quality at a minor speed cost. In fact this option is known to give the bests speed/quality trade-off of all options. Unfortunately it requires a “High Profile” capable H.264 decoder. It's highly recommended to keep this option enabled, if possible!
• 8×8, 8×16 and 16×8 P-Frame Search: This settings enables the 8×8 partitions on P-Frames and thus improves the visual quality of these frames. It's recommended to keep this option enabled!
• 8×8, 8×16 and 16×8 B-Frame Search: This settings enables the 8×8 partitions on B-Frames and thus improves the visual quality of these frames. It's recommended to keep this option enabled!
• 4×4, 4×8 and 8×4 P-Frame Search: This settings enables the 4×4 partitions on P-Frames, but usually the quality improvement will be negligible. Therefore this option is not worth the additional encoding time and thus can safely be turned off.
• 8×8 I-Frame Search: This settings enables the 8×8 partitions on I-Frames and thus improves the visual quality of these frames, but it requires 8×8 Adaptive DCT Transform. It's recommended to keep this option enabled, if possible!
• 4×4 I-Frame Search: This settings enables the 4×4 partitions on I-Frames and thus improves the visual quality of these frames. It's recommended to keep this option enabled!
• Remarks: During the encoding process, the encoder will break down the video into so-called “Macroblocks”. Then it will search for similar blocks in order to discard redundant data (see Motion Estimation). The macroblocks can be subdivided into 16×8, 8×16, 8×8, 4×8, 8×4, and 4×4 partitions. Analysing more of these partitions results in a more accurate prediction and thus improves the visual quality. Unfortunately this comes at the cost of additional encoding time. Generally it's recommended to keep all partition types enabled, except the for the “4×4 P-Frame” partitions. That's because the 4×4/4×8/8×4 partition search on P-Frames costs a significant amount of encoding time, but the gain in quality usually is negligible (only low resolution video may benefit). Note that some of the partition options depend on each other! Furthermore you have to take into account that 8×8 Adaptive DCT Transform (and consequently 8×8 I-Frame Search) are “High Profile” features and will require a suitable H.264 decoder, such as MPlayer, ffdshow or CoreAVC. Nevertheless 8×8 Adaptive DCT Transform and 8×8 I-Frame Search are extremely useful features.

• CABAC: This setting enables CABAC entropy encoding, one of x264' most impressive features. CABAC (Context Adaptive Binary Arithmetic Coding) works absolutely lossless, but gives an extra compression boost of ~15%. At higher quantizers CABAC can save even more bitrate - up to 50% and more is possible (see http://akuvian.org/src/x264/entropy.png). Consequently with CABAC enable you will either get a smaller file at same quality (CRF and QP mode) or better quality at the same file size (2-Pass mode). Therefore it's highly recommended to keep CABAC enabled in all cases! Nevertheless CABAC requires additional CPU time for both, encoding and decoding! The extra CPU time required for CABAC highly depends on the bitrate. Note that CABAC can easily become the most compute-intensive part of H.264 decoding! If you decide to disable CABAC (which you usually should not do), then the less efficient but faster CAVLC (Context Adaptive Variable Length Coding) will be used.
  • Remarks: Note that CABAC requires at least a “Main” Profile capable H.264 decoder. If you are targeting for the “Baseline” or “Extended” Profile, then CAVLC must be used!

• Pure Interlaced Mode: This settings enables interlaced encoding, so enable this setting only if you video is interlaced. In case your video is progressive (that is: non-interlaced) or if you don't know what “interlaced” means, keep away from this setting! Be aware that encoding an interlaced video as progressive will destroy the content! At the same time encoding a progressive as interlace is feasible, but significantly hurts encoding efficiency! Last but not least note that x264's implementation of interlaced encoding is not as efficient as it could be. Hence if you are dealing with an interlaced source, you are far better off using a deinterlace filter and encode the video as progressive.
  • Remarks: Now that CRT screens are dying out and LCD/Plasma screens are dominating the world, interlaced content needs to be deinterlaced at playback-time anyway. Unfortunately some screens use pretty poor deinterlaces, resulting in a jittery/unsharp image. Therefore the preferred way is to deinterlace before encoding the video, using a high quality deinterlacer/bobber, such as Yadif or TDeint.

• Loop Filter: This setting controls one of x264' most important features: the Inloop Deblocking filter. In contrast to MPEG-4 ASP (DivX, Xvid, etc.) the Inloop Deblocking is a mandatory feature of the H.264 standard. So the encoder, x264 in this case, can rely on the decoder to perform a proper deblocking. Furthermore all P- and B-Frames in H.264 streams refer to the deblocked frames instead of the unprocessed ones, which improves the compressibility. There is absolutely no reason the completely disable the Inloop Deblocking, so it's highly recommended to keep it enabled in all cases. There are two settings available to configure the Inloop Deblocking filter:
  • Strength: This setting is also called “Alpha Deblocking”. It controls how much the Deblocking filter will smooth the video, so it has an important effect on the overall sharpness of your video. The default value is 0 and should be enough to smooth out all the blocks from your video, especially in Quantizer Modes (QP or CRF). Negative values will give a more sharp video, but they will also increases the danger of visible block artifacts! In contrast positive values will result in a smoother video, but they will also remove more details.
  • Threshold: This setting is also called “Beta Deblocking” and it's more difficult to handle than Alpha Deblocking. It controls the threshold for block detection. The default value is 0 and should be enough to detect all blocks in your video. Negative values will “save” more details, but more blocks might slip through (especially in flat areas). In contrast positive values will remove more details and catch more blocks.
  • Remarks: Generally there is no need to change the default setting of 0:0 for Strength:Threshold, as it gives very good results for a wide range of videos. Nevertheless you can try out different settings to find the optimal settings for your eyes. If you like a more sharp video and don't mind a few blocks here and there, then you might be happy with -2:-1. This might also be worth a try for MPEG-4 ASP (DivX, Xvid, etc.) users! If you like a smooth and clean image or encode a lot of Anime stuff, then you can try something like 1:2. Nevertheless you should not leave the range between -3 and +2 for both settings!

• Max. Ref. frames: In contrast to MPEG-4 ASP, H.264 allows multiple reference frames. This setting controls how many frames can be referenced by P- and B-Frames. Higher values will usually result in a more efficient compression, which means better visual quality at same file size. Unfortunately more reference frames will require more time for encoding (and also a tiny bit more CPU power for playback). By default the number of reference frames is limited to 1. It's highly recommended to raise the number of references to at least 3. Nevertheless using more than 4 or 5 reference frames for “real life” footage should be avoided, as it won't improve the results any further! At the same time Anime and cartoons benefit a lot from additional reference frames. Sometimes even the maximum of 16 reference frames can be helpful for such material.
  • Remarks: While “software” players usually support any number of reference frames, “hardware” players are limited to a maximum number of reference frames! The maximum number of reference frames can be calculated from the “Max Decoded Picture Buffer Size” (MaxDPB) and the resolution of the video. The MaxDPB value is defined by the individual H.264 Profile supported by the player (for details see Annex A of the H.264 specs).

• Max Consecutive: This setting controls the maximum number of consecutive B-Frames. B-Frames refer to both, the previous and the following I-Frame (or P-Frame). This way B-Frames can compress even more efficient than P-Frames. B-Frames can significantly improve the visual quality of the video at the same file size. Therefore using B-Frames is highly recommended. Also note that allowing more B-Frames will never hurt the quality: You can even safely choose the maximum of 16 consecutive B-Frames. That's because you only specify the upper bound for the number of consecutive B-Frames. x264 will still decide how many consecutive B-Frames are actually used. So even if you allow up to 16 consecutive B-Frames, the encoder will rarely go that high. Nevertheless limiting the maximum number of B-Frames to less than 16 is reasonable, because most videos won't benefit from using more than ~4 consecutive B-Frames anyway! Raising the B-Frame limit higher than that would only slowdown the encoding process for no real benefit! If you set the B-Frame limit to 0 (the default), B-Frames will be disabled. Of course disabling B-Frames is not recommended!