February 3, 2016 - Comments Off on Comedy Central Dumps its Obsolete Optical Media Archive
by Kevin Gepford
Comedy Central migrated its entire Brand Creative archive to a modern system, and tossed 18 years worth of optical media into the dumpster.
We had amassed 800+ CD’s and DVD’s in a comprehensive archive of all source files — for every single print project generated by the Brand Creative group since the team… well, pretty much since the team’s very beginnings.
These archives were already in decent shape when I arrived at Comedy Central nearly 12 years earlier. Projects had been laboriously burned onto sequentially numbered discs… in duplicate, to be safe. The discs were stashed in a nicely organized drawer. We continued to maintain and grow this grassroots system for the next decade and relied on trusty NeoFinder to catalog it all. Over time our archive grew to 1.75TB in size (double that, to 3.5TB and about 1,600 discs, if you include the dupes).
Maybe you’re not impressed; today’s servers and RAID arrays are huge, and fast. You can move 4GB over the internet in less time than it ever took to burn a disc. But cast your mind back to 2003, and try to remember what the world was like in those dark days. Hard drives were small. Servers were sluggish. Syquest and Zip drives had only recently been deprecated.
Optical media was the best we had, and it was fantastic. The marketing director could request printouts of an old campaign, and we’d drop what we were doing to research which disc the files were on. We’d load that disc, find the files, open the files, load the fonts, re-link all the assets, and make the printout. Actually, if you think about it that way, it really wasn’t so great. But it worked.
To solve that problem I implemented a DAM system based on Extensis Portfolio. One of its purposes was to hold PDF’s of all our finished work so anyone on the team could easily review or print out past projects. And wow! This slashed our interruptions by about 95 percent! If somebody genuinely needed source files, we’d lug open the Big Drawer of Archives and pull the disc.
This worked well enough for nearly a dozen years. Even though optical media was technologically stagnant, it was cheap and it got the job done. I suppose that’s why we stuck with discs for so long. Also, the whole group was still hooked on DVD’s, especially for our key raison d’être: video promos.
But in 2013 the world changed.
Four things happened.
New Mac Pro workstations began arriving sans built-in DVD drives. Next, our massive videotape library was converted to digital format, per Viacom corporate mandate. At the same time, our SAN got a 10x capacity boost… with fiber-optic connections and an awesome LTO data storage system to go with it — complete with an internal camera so anyone can watch that busy little robot moving tapes back and forth. The LTO was designed to archive all the digital footage and source files for our motion graphics retired projects. Get those files off the SAN and into cold storage!
These events represented a giant leap forward in Comedy Central’s maturity model. And it was exactly what I’d been waiting for — a robust and enduring system with the benefit of full institutional support. At long last, our archive would get a proper home!
But first came suffering: We had 802 discs to migrate.
We met that challenge with an incredible technology called “Intern”. During the process of transferring our archives to a holding pen on the SAN, we discovered scratched discs. Delaminated discs. Oxidized discs. Discs smeared with terrifying unknown substances. We encountered a significant failure rate with some of our oldest media. There were duplicate discs, yes. But some of those dupes also failed. Failure means this: If the copying process hits one error, the whole operation aborts. Our last resort was the Terminal app, using the “ditto” command which copies everything possible — it skips the damaged disc sectors and keeps moving, ensuring minimal loss.
Our final step was to migrate all these assets onto LTO. And, just because I’m obsessive, I requisitioned a modest RAID drive to maintain a local copy.
Then, I shed a tear, and said goodbye. Everything into the dumpster.
Optical media was great, but that era is over. Organizations are seriously investing in media management, finally giving your precious archives the institutional support they deserve.