Gentoo Wiki ArchivesGentoo Wiki

HOWTO_DVD_to_Matroska_and_H.264

Contents

Introduction

Wikipedia has an article on:
Matroska

Matroska is an open source, open standards multimedia container format, similar to AVI in concept, but far more advanced in practice. This guide will introduce you to some of the tools available to create, edit, and play Matroska files.

Wikipedia has an article on:
H.264/MPEG-4 AVC

Excellent documentation is, as always, provided in the MAN pages of the programs used, but this guide will kick-start you into making high quality rips using the original subtitle and audio streams, and transcoding the MPEG-2 video to H.264, the codec used in Blu-ray movies, for reducing size while preserving the video quality as well as possible.

Matroska is supported in VLC, MPlayer, and Totem.

All of the commands involved in the ripping process are given with bash variables, so that they can be easily copy-pasted into bash scripts, automating the ripping process for highest convenience.

Practical Uses

If you have ever encoded movies into an AVI, and been slightly frustrated with the format's limitations, then Matroska is probably what you are looking for. Matroska boasts advanced features over AVI and WMV such as being able to include detailed chapter information, sub-chapters, titles, multiple audio and video streams, menus, subtitles, attachments and custom metadata.

One practical application would be that you have a DVD you want to watch on your computer, but you'd like to keep as much as the original format as possible (title, subtitles, audio tracks, chapter selection) but with the bonus of having the video re-encoded to something else (MPEG4 or Theora, for examples) to save on space.

DVD to Matroska

There are a few tools that you can use to complete every step, from ripping to watching, but in our howto we will focus on mplayer/mencoder/transcode to rip the DVD, mkvmerge to build the Matroska file (part of mkvtoolnix), and VLC to watch the movie.

Rip VOB to Hard Drive

The first step is to actually get the video files from the DVD that you want to encode. It's generally a good idea to rip the movie to the harddrive first, because encoding will go much faster reading from the harddrive, versus reading directly from the DVD disc drive.

This HOWTO is based on a full-length feature movie. Ripping TV DVDs isn't any more difficult -- you just have to find the actual title number, whereas most ripping tools with movies will just pick the largest, which is almost always the movie.

For our example, we'll actually use MPlayer to rip a DVD, since it's one very simple step, and generally always calculates the starting and stop point of the movies correctly (instead of messing with various vob-ripping options). Other tools for getting the VOB file to the harddrive are vobcopy, dvdbackup and dvdcpy.

Once you've emerged MPlayer with the correct DVD support, with your DVD in the drive, first make sure you can play the movie:

mplayer dvd://${TRACK}

${TRACK} is the track number of the title we want to rip. Gathering your frequently used ripping commands in a simple bash script with variables like ${TRACK} will save a lot on typing and headaches in the long run. Once you have found the number of the title, you're ready to rip it straight to a .VOB.

mplayer dvd://${TRACK} -dumpstream -dumpfile ${RIPPATH}movie.vob

${RIPPATH} is another variable that you can set to the path where you want to store your intermediate files in the ripping process to automate the process through bash scripts. Extraction of the VOB file will probably take a few minutes to do, and likely will produce very little output, so be patient. Note that the ripped VOB file takes up a lot of space for a full-length movie. In some cases, it will take a while to get the CSS keys.

The VOB file you just ripped will still contain all the audio/video/subtitle tracks the original DVD had. We only need the actual DVD for a few more seconds.

Chapters and Subtitle Information

We need physical access to the the DVD disc to extract chapter information and color information for the subtitles.

Getting the chapter information is very simple, thanks to a tool called dvdxchap, which is part of the ogmtools package. So, just emerge media-sound/ogmtools, then run the program and dump the output to a text file. Assuming that /dev/dvd corresponds to your DVD device:

dvdxchap -t ${TRACK} /dev/dvd > ${RIPPATH}chapters.txt

If you look at chapters.txt you'll see the format is something like this:

CHAPTER01=00:00:00.000
CHAPTER01NAME=Chapter 01
CHAPTER02=00:02:30.000
CHAPTER02NAME=Chapter 02
CHAPTER03=00:02:42.300
CHAPTER03NAME=Chapter 03

The time index is in format of hours:minutes:seconds.milliseconds. The chapter name can be whatever you like, so feel free to edit it if you like. Or, you can always create a custom chapter list of your own.

When we later rip the subtitles, we need information concerning the placement and color palette of the subtitles. This information is retrieved from vts_01_0.ifo. We therefore copy this file to our ripping directory for later use.

cp /mnt/dvd/video_ts/vts_01_0.ifo ${RIPPATH}

Note that /mnt/dvd is just an example of the mountpoint of the DVD, and may differ on your system.

For the rest of the ripping process, the DVD disc is no longer needed. If the subtitles on the disc are correctly mastered, it can be more convenient to browse through the available subtitle and audio tracks (see below) using the MPlayer GUI than doing everything from command line, though.

Subtitles

In order to rip subtitles, we have to know the number of the subtitles tracks that we want to rip. If the disc is correctly mastered, you can switch between available subtitle tracks in the mplayer GUI (default key for changing subtitles is 'j'). You can get a hint as to which subtitle languages that are available by issuing

mplayer -v dvd://${TRACK} 

and examining the output. A typical output would be:

...
subtitle ( sid ): 0 language: da
subtitle ( sid ): 1 language: en
subtitle ( sid ): 2 language: no
subtitle ( sid ): 3 language: fi
subtitle ( sid ): 4 language: sv
subtitle ( sid ): 5 language: en
subtitle ( sid ): 6 language: en
...

Many times however, this information is wrong, which in the case of this example becomes apparent later in the mplayer output:

...
==> Found subtitle: 0
==> Found subtitle: 1
==> Found subtitle: 2
==> Found subtitle: 3
==> Found subtitle: 4
==> Found subtitle: 5
==> Found subtitle: 6
==> Found subtitle: 7
==> Found subtitle: 8
==> Found subtitle: 9
==> Found subtitle: 10
==> Found subtitle: 11
...

The VOB file actually contains 12 subtitles in this case, and to identify which language corresponds to which subtitle number, you can issue:

mplayer -sid ${SID} ${RIPPATH}movie.vob

where ${SID} is the number of the subtitle (sid = subtitle ID).

Once you know which subtitle tracks to rip, use the following commands to extract the subtitles (tccat and tcextract are part of the transcode package, and subtitle2vobsub can be found in the subtitleripper package):

tccat -i ${RIPPATH}movie.vob -L | tcextract -x ps1 -t vob -a ${HEXSID} > ${RIPPATH}subs-${SID}
subtitle2vobsub -o ${RIPPATH}vobsubs -i ${RIPPATH}vts_01_0.ifo -a ${SID}< ${RIPPATH}subs-${SID}

Note that we are using the copied vts_01_0.ifo from the previous step. Running these commands for several SIDs will gather all the subtitles in the vobsubs.vob and vobsubs.idx files, which makes it more convenient to automate bitrate calculation later. ${HEXSID} is the hexadecimal number 20 (decimal number 32) plus the SID. For SIDs 0 to 9, this translates into HEXSID 0x20 to 0x29. For higher SIDs, you have to work a little harder. SID 18 (hexadecimal 12) corresponds to HEXSID 0x32. Note: Some versions of tcextract (v1.0.3 for instance) seem use decimal numbers instead of hex as described here.

It is always a good idea to check that you extracted the right subtitles. You can do this by running:

mkdir ${RIPPATH}${SID}
subtitle2pgm -o ${RIPPATH}${SID}/${SID} -c 255,0,0,255 < ${RIPPATH}subs-${SID}

The reason why we create a subdirectory for this extraction is because there are a huge number of individual pgm files created in the process, and we want somewhere to put them where they don't litter too much. Once you have checked the pgm files, you can delete them.

Audio

Identify the audio tracks that you are interested in ripping. A look at the output of mplayer as with the subtitles gives you a hint about the audio track IDs (AIDs) of interest. In our example we observe this part in the output from mplayer:

audio stream: 0 format: ac3 (5.1) language: en aid: 128.
audio stream: 1 format: ac3 (stereo) language: en aid: 129.

As you can see, AIDs start at 128. Listen to each audio track you are interested in by running:

mplayer -aid ${AID} ${RIPPATH}movie.vob

For each audio track that you want to rip, issue

mplayer ${RIPPATH}movie.vob -aid ${AID} -dumpaudio -dumpfile ${RIPPATH}audio${AID}.ac3 

assuming that you are ripping AC3 tracks (tracks can be in other formats, e.g. DTS).

Using the mplayer GUI, you can play the resulting AC3 file. Write down the length of the track in hours, minutes and seconds. This will help when we later calculate the bitrate of the transcoded video stream.

Since the Matroska container accepts many audio formats, the audio stream can be transcoded to another format with lower bitrate. Keep in mind that it's best to keep conversion from one lossy format to another to a minimum, however.

Preparing to Transcode

Before transcoding, we want to determine how the video should be cropped, and calculate the bitrate of the transcoded video stream.

Cropping

To determine cropping parameters automatically, issue

mplayer ${RIPPATH}movie.vob -vf cropdetect -sb 50000000

The -sb 50000000 part is included to skip the first ~50MB of VOB data, since movies sometimes contain a title screen that has a larger frame than the movie itself. To minimize wasting space on encoding black borders, we want to crop as tightly as possible. Record the detected cropping parameters (for example 720:560:0:6).

Bitrate

An excellent bitrate calculator that suits our need can be found at Marc Rintsch's homepage (direct link). Extract bitrate.py from the downloaded tar file and run it:

./bitrate.py -o 0.5 -t ${TARGETSIZE} ${DURATION} ${RIPPATH}audio{AID}.ac3 ${RIPPATH}vobsubs.idx ${RIPPATH}vobsubs.sub

Include all files that should be merged with the video file into the resulting MKV container file in the command line ( including different audio tracks, for example).

${TARGETSIZE} sets the target size in MB of the resulting MKV file. A target size of 2240 will result in MKV files small enough to fit two movies on a single-layered DVD, with some room (about 30MB) for errors. According to the Doom9 Codec shoot-out 2005, encoding the video with the x264 codec at these bitrates (equvalent to 3CDs, as mentioned in the codec shoot-out) will result in quality that is very close to the original VOB file.

${DURATION} is the duration of the movie (e.g. 1:23:45 for 1 hour, 23 minutes and 45 seconds) that we found earlier using the mplayer GUI.

Write down the bitrate in kilobits per second (kbits/s)

Transcode to H.264

Matroska actually supports a lot of codecs that can be wrapped in an .mkv file. See mkvmerge --list-types for the full list.

mkvmerge --list-types

For this example, we will be transcoding the video to H.264. This is a more modern codec than XViD or DivX, resulting in higher quality when comparing video streams encoded at equivalent bitrates. Make sure that you have enabled the x264 flag when emerging MPlayer.

With ${CROP} specifying our cropping parameters and ${BITRATE} set the desired (integer/whole number) bitrate, threads=# for multi-core cpus, in addition to our previous variables,issue the following commands for 2-pass encoding (the first pass only analyzes the video, so we discard the output by directing it to /dev/null):

mencoder ${RIPPATH}movie.vob \
-vf pullup,softskip,crop=${CROP},harddup -nosound \
-ovc x264 \
-x264encopts bitrate=${BITRATE}:\
subq=5:bframes=3:\
b_pyramid:weight_b:turbo=1:threads=auto:pass=1 \
-of rawvideo \
-o /dev/null

mencoder ${RIPPATH}movie.vob \
-vf pullup,softskip,crop=${CROP},harddup \
-nosound \
-ovc x264 \
-x264encopts bitrate=${BITRATE}:\
subq=5:8x8dct:frameref=2:bframes=3:\
b_pyramid:weight_b:threads=auto:pass=2 \
-of rawvideo \
-o ${RIPPATH}movie.264

The encoding parameters used give a nice tradeoff of encoding speed vs. quality, but can be adjusted for greater encoding speed or higher video quality by following the advice on e.g. mplayerhq.hu (scroll to the bottom of the page if you are impatient).

Encoding takes awhile. An average movie will take 3-5 hours on the first pass and the second pass will take a little longer.

It is important to note that one of the advantages of the Matroska container is it supports software scaling of video files so there is no need to scale your video for the sole purpose of correcting the aspect when encoding your video, you can specify the display aspect when muxing the file. The mencoder commands given will however scale the image to 1:1 aspect ratio.

Put Raw Video in MP4 Container

The transcoding process outputs raw H.264 video, which has to be wrapped in an MP4 container prior to merging. Looking at other pages describing this step, it turns out that MP4Box is a common tool. All current versions of MP4Box are unable to handle files over 2GB in size, though, why we use mp4creator from the mpeg4ip package instead (emerge mpeg4ip).

mp4creator -c movie.264 -rate 25 movie.mp4

Note that -rate 25 specifies a framerate of 25 fps, which has to be adjusted for non-PAL movies, of course. NTSC for example uses 29.97.

Merge Multimedia Streams

To merge video, audio and subtitle streams, and the chapter information, we will be using mkvmerge, which is part of the mkvtoolnix set of tools. Be sure to enable the wxwindows flag when emerging mkvtoolnix, in order to build the Matroska merging GUI (mmg)

emerge media-video/mkvtoolnix

A basic merge would look something like this:

mkvmerge --title "${MOVIE_TITLE}" -o ${RIPPATH}movie.mkv --chapters ${RIPPATH}chapters.txt --default-duration 0:${VIDEO_FPS}fps ${RIPPATH}movie.264 ${RIPPATH}audio${AID}.ac3

While mkvmerge can be used directly from the command line, mmg makes the job a lot easier, and continuously shows the resulting command line, for the day that you want to go hardcore. Add the mp4, ac3 and idx files from earlier steps to the list of input files in mmg. There are a lot of options to set, and the more information you provide, the nicer the resulting Matroska file, of course. Be sure to specify track names, track languages, file/segment title, chapter file at least.

Watching Movies

There are a lot of media players out there, but when it comes to Matroska support, VLC (VideoLan Client) is heads above the rest. The main outstanding feature that this has above others is chapter support, meaning you can jump ahead and back from one chapters to another. If you're not interested in that, MPlayer and Xine can both play Matroska audio and video files as well. Just remember to add matroska to your USE flags when emerging the media players.

Please note that as of mplayer-1.0_rc1_p20070824, the matroska was removed (matroska support is built by default). As for xine, there seems to be support for matroska: there is no matroska USE flag, but matroska plays fine in kaffeine, using xine-lib-1.1.15-r1.

MPlayer Chapter Support

The MPlayer developers have added Matroska chapter support, and it's now available in an ebuild snapshot.

Just emerge media-video/mplayer-1.0_pre20060810. To skip chapters, use @ for the next chapter, and ! to go back a chapter.

Graphical interfaces

emerge -tva ogmrip
Toolkit: GTK+2

This HOWTO is perfect for the technically oriented users. However, OGMRip might suit your needs in case you are looking for a clean HIG-compliant GTK+2 GUI frontend to MPlayer that can encode video to H.264, audio to AAC and put everything into a Matroska container (this listing is far from exhaustive). Please note that shRip is OGMRip's CLI interface. Because of the inherent GUI weaknesses, you will of course fail to find all the options that MPlayer provides. Still, OGMRip is worth a try if you feel rusty about all the available encoding terminology and options.

Recently, as of v. 0.12.0, OGMRip has introduced "profiles", i.e. use or create predefined settings for specific tasks. The best part of the newish "profiles" is that there is a "User" Video Quality setting, which may be manually adjusted by User. It seems that even the lowest ("Normal") OGMRip default Quality settings are "Extreme" for my hardware, an Intel(R) Pentium(R) M processor 1.60GHz and 482MB Memory. In case you have "hardware issues", in OGMRip you can manually set most of the H.264 encoding options suggested in this HOWTO.

emerge -tva shrip
emerge -tva mkvtoolnix
Toolkit: wxGTK and QT

A tool that may help you dealing with .mkv files is the mkvtoolnix package. To use the desired toolkit, simply adjust your use flags appropriately in package.use before emerging the package.

See also

Retrieved from "http://www.gentoo-wiki.info/HOWTO_DVD_to_Matroska_and_H.264"

Last modified: Mon, 06 Oct 2008 21:19:00 +1100 Hits: 54,206

Created by NickStallman.net, Luxury Homes Australia
Real estate agents should list their apartments, townhouses and units in Australia.