A well-known weekly, weekend comedy show was experiencing these challenges firsthand.
Having outgrown the capabilities of their tape archive, they urgently needed a faster, more reliable, highly scalable archive solution to provide their post-production staff speedy access to 40+ years of content — over 800 episodes, millions of digital assets, and petabytes of data. And they required a less labor-intensive process that would eliminate tape handling and migrations.
The Challenge of Achieving Flexible and Searchable Archive
The show’s previous archive solution employed tape libraries along with an offsite warehouse for long-term storage. Archiving was controlled via a media asset manager which maintained the asset database.
To ensure integrity, it was periodically necessary to read and re-write the tapes to newer tape media. In addition to the headache of merely transferring the tape data from tape to tape, were the regular format transitions that occurred when tape readers progressed from one generation to another.
As the post production supervisor put it, “over the course of my career, I’ve seen every part of the chain get changed. Ensuring access through all of those transitions has itself been a full-time job.”
Beyond the interoperability challenges, tape was unreliable. The tape libraries, tape readers and tapes themselves could all fail at inopportune moments.
The manager recalled, “many times I’ve had my hand in the library trying to fix a jam while the robot is whizzing around in there. When we’re on a deadline, we do what it takes, but sometimes it’s perilous.”
Tapes also present logistical challenges. Tapes had to be moved among sites, and when it was necessary to retrieve them, even the traffic could be a factor in meeting a deadline.
Another limitation of tape is the ability to find assets. The search capabilities were only as good as the underlying media asset manager. When searching for specific clips, the producers were limited to the MAM capabilities and the indexing decisions that had been made years ago.
“It seemed crazy that in the era of Google we would be limited by primitive search of our most valuable resource, but we were. Finding media could take hours if not days,” the manager added.
The Answer: Object Storage
From these challenges, the studio’s engineering department compiled three top objectives for their new active archive. After well researched deliberation and testing, the program’s engineering department concluded that Cloudian’s object storage solution was the singular means to achieve these objectives. With
Cloudian, they can:
Break the chain of dependencies with a solution that ensures long term, risk free access to media.
- Freedom from drivers: There are no proprietary hardware or drivers – it’s all HTTP.
- Portability: Objects can be moved from one storage environment to another. Move between vendors or to the cloud. Cloudian even has functionality built in that can make this automatic, if you choose.
- Hardware independence: Object storage is built on industry-standard servers, so hardware can be refreshed at minimal expense.
- MAM independence: Object storage leverages tags that are stored with the media. Locate media via standard search tools, independent of the MAM database. A database can always be rebuilt, if needed, using those tags.
Achieve rapid search which is scalable and can evolve as search tools and requirements progress.
- Object storage integrates rich metadata to tag (or “label”) assets. Tag assets with complete descriptions and find media using a Google-like search, wherever the asset is.
- Plus, the scheme for organizing and finding assets can evolve over time.
“In previous environments, such as tape, we were stuck with whatever organizing scheme we started with. Now we can change things and even go back and re-tag media if needed.” recalled the manager.
Plan for the exponential growth that comes with increased format resolution. 4K and 8K will inevitably be followed by other new formats, so it’s essential to plan for the known and unknown.
- With simple scalability, object storage simplifies the ever-present task of adding capacity. The technology is built on “nodes” (like storage bricks) that can be added as needed, allowing you to start small to keep costs down, and add new capacity at any time, without even a service call.
A New Storage Environment
Currently, the show employs an all-flash SAN for primary storage and Cloudian object storage as the active archive. A second Cloudian cluster at an offsite location holds the disaster recovery copy. Replication is managed by Cloudian’s built-in data management features.
“This is where we’ve been trying to go since I ingested that first tape and watched storage space disappear,” the manager said. “My job is to make sure our assets are stored, safe, and accessible. We’re finally there with an answer that will hold up over time. We’ve had our last ever data migration, and that feels good.”
