I Don't Watch Commercials

I do not miss the advertisements

Previously, I blogged about how I got to be a MythTV user. Possibly the best thing about Myth is automatic commercial skipping! It is not perfect, but when it works, you barely know your're watching broadcast television. It works by marking suspected commercials and jumping over those marks during playback. There are some very clever developers that wrote the code to go back over a recording looking for station logos, special video frames, closed-captioning breaks, among other things. This operation takes (on my machines) about 20 minutes per hour of real video and can (currently) happen only after a show is finished recorded. Each flagging job queues up so depending on how many recordings just completed it could be hours before the show is really ready to watch. You can always watch a show immediately after it begins recording. I have become so accustomed to time-shifting all my television that the wait between live broadcast and prepared recording is unnoticable. Watching a well-flagged television show is quite cool and even if the flagger misses, it is an easy thing to rewind then skip ahead in 30 second jumps. We do miss the TiVo sounds just a little, so someone makes them when necessary; ba-doop ba-doop ba-doop.

In addition to the volume of bad adverts, we miss out on the occasional not-so-bad commercial. The benefits and detriments of a comsumer culture is left as an exercise for the reader, but it feels nice to be free of stupid, stupid commercials. Adrienne felt that she was missing out on too much new TV, so she went out and subscribed to a television magazine. I guess people are willing to pay for targeted advertisements.

Yet another MythTV Blog

TiVo was cool, but I wanted something more.

Enter Robert X Cringely, a technology columnist for PBS . He has 20-some years in the industry and writes compelling articles. In late 2004, I read this article which launched me into a homebrew PVR project. Cringely wrote about Andrew Greig, a man in Canada who works for Starnix, an international Open Source software and services consultancy. In addition to providing free VIOP and Wi-Fi to his neighbors Andrew runs a (very large) installation of MythTv. I wish I could see this guy's setup. Quoting the article:

Andrew's server runs Myth TV, an Open Source digital video recorder application, storing on disk in MPEG-4 format more than 30,000 TV episodes, movies and MP3 music files. "As each new user comes online, I add another TV card to the system so they can watch live TV," says Andrew, "but since there are only so many episodes of SpongeBob SquarePants, nearly everything that isn't news or sports is typically served from disk with full ability to jump forward or back at will. We've reached the point now where the PVR has so much in storage already that it is set to simply record anything that isn't already on disk."

Think about it. These folks up in Canada can not only watch everything we can watch on TV, on a whim they can watch every episode of the original Star Trek in the order they were broadcast ALL ON ONE WEEKEND.

Doing a little math on a typical 60-minute TV show (at 1.5-2 megabits-per-second) we get 673-900MB per show. If half of those 30,000 files are television shows, I estimate 10-15 TB of storage. That's a lot of juice to run out of your house. I am not sure how many users his amplified Wi-Fi reaches, but "most" of my suburban neighborhood is probably 50 - 100 homes. And I thought I had IO issues.

So I Set Out

I am a computer dork, so I had some extra hardware around with which to start my project. Taking the best of the leftovers, I put together an AMD 1GHz processor, 512 MB of memory and a 40 Gb disk. Still, I needed a way to get the television into the machine. At the time, I was living with my wife and 6 other roommates in an old Stanford party house. Talking with the roommates about the possibility of watching television from their computers, one of them offered up a capture card to get me started. Coupling the cap-card with a CreativeLabs sound card, I was ready to go.

MythTV is notorious for being difficult

I work with Solaris and live with FreeBSD, so I had a decent idea about the complexity of installing and configuring a unix-like OS such as Linux. I did a quick scan and chose Debian as my distro, primarily because MythTV is developed on that version. My first-ever linux install went fine, as did the MythTV install. Debian's package management is almost as cool as FreeBSD's ports. I did struggle for a few days compiling a kernel that had the correct modules and drivers for the sound and cap cards and the rest of my hardware.

The MythBox began recording

So now I could record one channel and store the show in rtjpeg format. While recording the server had encode the stream in software, loading my machine to about 70% utilization. Acceptable, but I could not grow my setup without more horsepower.

What makes MythTV so great

To me, a big draw towards MythTV is its server/client architecture. I ran a master back-end which did nothing but record programs. Because of MythTv I reinstalled my desktop with GenToo linux and the MythTV software. (I also switched the backend to GenToo.) On my desktop, I ran a slave backend (server) and the frontend (client) software. With the extra processing power, my desktop could flag commercials - a CPU intensive procedure - while the backend with the cap card could record and encode television. Any time I wanted to, I could pull up mythfrontend and watch recorded shows (and live tv if the recording schedule permitted).

My wife is a TV Junkie

She was drooling just as much as I was over my new project giving me a high WAF. Adrienne liked the idea for slightly different reasons, but the encouragment was strong. She had me dual-boot her desktop so she could watch recorded TV and surf. With her gentle pushing and the apathetic interest of the other suitemates, I got Samba and dsmyth working so Windows users could watch the recordings in wmp. This got me a second capture card on loan from another roommate. My desktop would now have to pull double-duty as a workhorse and encoding machine. Around this time, I added a 160GB drive for storage to the system.

Launch the Shuttle

For $500, I put together a Shuttle bare-bones system with a DVD RW. Once I got all the right software and drivers on this machine, we set it next to the telly for MythTV bliss. Add a streamzap remote for extra coolness. Now I also had an extra backend system for commercial flagging.

I though 6 tuners would be enough

Towards the goal of recording all chanels all the time, I set out to increase the tuner capacity of my system. Right about that time Hauppauge released the PVR-500, a dual tuner capture card that does MPEG encoding in hardware. This removes the recording load from my master backend leaving only IO management (and various backend duties, and recording scheduling, and mysql database) to the poor little server. I figured four tuners would be adequete and six more than enough. So I bought three PVR-500s. Two went into the master backend and one into the shuttle, mostly for live-tv watching. I doubt we will ever consume all six tuners all the time, but it only took a month before all six were active for a few hours! Overall, six is enough.

As it stands today

After 18 months of using MythTv (started with v0.16), my setup has not changed much.

The backend is still the same almost-overloaded server:

• 1.6 GHz AMD Processor
• 1 GB Memory
• 2 160 GB disks striped for write-performance
• 2 Hauppauge PVR-500s (4 tuners) doing hardware encoding

The frontend quietly keeps pace with everything:

• 1.2 GHz AMD Processor
• 512 MB Memory
• 40 GB disk for live tv buffering
• A single PVR-500 (2 tuners) for extra recording power and watching Live TV

Other machines in the MythTV network include my desktop doing commercial flagging and a dedicated mysql server.

Next time

• Jumpy, jittery, convoluted video and audio
• ivtv0 warning: ENC Stream 0 OVERFLOW #981: Stealing a Buffer, 512 currently allocated
• Video lag/freezing on the frontend
• Large databases' impact on the backend
• Old settings clutter the database