Psychoacoustic encoding: What Nielsen uses to track TV viewing

Nielsen Media Research developed psychoacoustic encoding as the successor to its "Mark II" TV measurement device.

That 20-year-old workhorse used taps and probes hard-wired into a TV or cable box that read channel numbers, or identified the semi-transparent network TV "buttons" or logos tucked into the corner of TV screens during shows. Analog modems in the back provided nightly phone calls to a data center, where the day's "viewing events" — every minute of TV watched, including channel changes — were deposited at 2 a.m.

The psychoacoustic encoding method relies on digital signals embedded in the audio of broadcast TV shows. These signals — which last for a fraction of a second — are slipped into the audio tracks of TV shows approximately every 2.5 seconds, except for periods of sustained silence.

"If you heard it, it would be a crrrkkkkk kind of sound," says Bob Luff, CTO at Nielsen. While the codes themselves can be heard by the human ear, they are inserted into audio at points where they are imperceptible. "But if I smack a pair of symbols together, then play that cracking sound, you wouldn't hear it. That's how the psychoacoustic encoding works."

These signals hidden in the active audio include an accurate time stamp — "down to the one-hundredth of a second," Luff says, identifying when the show was played, the network broadcast it, and the call letters of the local affiliate showing the program.

TV networks and broadcasters use equipment called a NAVE (Nielsen Audio Video Encoder) to "burp" these signature codes into the program audio, which are picked up by devices installed in the 40,000 Nielsen homes — the company's statistical sample base. Called A/P (Active/Passive) monitors, these cable-box-sized gadgets tie into the audio output of a TV or home theater system and actively decode and store the psychoacoustic signals.

Psychoacoustic signatures were first developed to solve the problem of time-shifted viewing, which has been around since VCRs could tape shows, but is now more widespread thanks to TiVO and digital video recorder (DVR) technology. The sound-based signatures in the audio allow Nielsen to figure out when a show is really viewed, vs. the time of its scheduled broadcast; the codes circumvent the problem of time-shifted viewing, because the audio burps show up on recorded programs when played from a DVR hard drive or VCR tape, Luff says.

The "passive" aspect of the monitoring is also audio-based, but takes it a step further. Passive signature codes are created by exposing snippets of network TV show audio to an algorithm "that converts the wiggly waves of sound into a thin stream of numbers," Luff says. "That stream is unique, and won't match with anything else in the world … so, for instance, Jack Bauer's voice on [an] episode of "24" — that's unique; an episode of "Lost," "American Idol," — those are all unique and you get a good match."

A/P devices listen for both the actively injected codes signatures, and the unique digital signatures of the audio itself, which provides a kind of checksum, making the data collection process more reliable, Luff says. "There's nothing the station can do to gum up the works with signatures, other than turn their transmitters off," Luff says. "This is the pervasive technology that gets us out of this cat and mouse game."