The process of archiving in media has undergone major changes in the past couple of years. This shift also indicates that the media industry is in quite the transition nowadays, as workflows demand more from media archive storage, leading to emerging capacity and search challenges.
Previously, media owners have responded by migrating away from tape to newer disk solutions, but even those are becoming outdated in the face of today’s new rage: hybrid cloud solutions. At the same time, they’ve also opted to integrate advanced solutions to accelerate search within those massive archives.
We’ll dive into current media archive challenges and see what becomes possible with cloud-based archiving—if done well!
Archive vs. Backup – What’s the Difference?
Before exploring those trends and the benefits of archiving, let’s make sure we’re all on the same page about archiving.
For instance, what’s the difference between backup and archive in media?
You’re dealing with a backup of your production storage when your media files are replicated to secondary storage, thus providing greater protection if something happens to the primary source. File replication is cyclical, as the production storage is continuously checked for new and changed files. Files are also overwritten after the specified retention time is reached.
On the other hand, archiving is “simply” data migration; you move files that are no longer necessary for daily production to long-term storage, but they remain easily retrievable. This enables the collection of all completed productions in one place and safekeeping for reference, compliance, and re-use in the future.
If there’s one key takeaway here, both archive and backup are vital in the media industry!
Media Archive Trends
Generally, the most dominating trend or challenge is connected to capacity. But given how quickly media is growing in both file size and just sheer quantity, it’s not surprising at all. More and more media players are shooting all their newest productions in 4K and run out of space constantly.
That’s why more than half of the industry faces capacity challenges—which are being constantly met by ever-improving hybrid cloud solutions.
Capacity Management Challenge
As mentioned above, with how pervasive 4K media is now (with 8K already here), capacity management is the #1 challenge on most media owners’ minds.
It also doesn’t help with capacity issues that producers are out to capture, save, and store every bit of their shoot. Their cameras are running non-stop, and every bit of that footage will go to the archive.
Today, the average archive size of a hypothetically average media owner is between 1.5-2 petabytes. A few years ago, that number was “only” 1.2, and still, more than half of media players reported ongoing capacity challenges.
Media Archive Storage Difficulties
As a side effect, finding assets within these archives becomes a headache. In some cases, this issue is so serious that they re-buy stock footage rather than search for the footage they already have.
Unfortunately, search tools across the industry are still not as advanced as they should be. Imagine this; a few years back, quite a few media players still used a combination of Excel sheets and labels on media. This was the case for archives of all sizes, even those over 500 terabytes.
Furthermore, players employing MAMs or other databases also faced similar challenges, complaining about how time-consuming media asset management is.
Metadata, however, can greatly facilitate media archive storage search, as it is the key to the archive, which enables complex searches for technical and descriptive criteria.
Consequently, the vast majority of the industry today is looking for AI and machine learning tools to enrich metadata and improve search intelligence.
Hybrid Cloud – The Best of Both Worlds
With these challenges in mind, it is no surprise that media players are significantly interested in the hybrid cloud. Current trends are showing a steady migration towards hybrid cloud solutions. It looks like it will take over media archiving in this decade.
But what makes the hybrid cloud so good?
It combines on-premises storage with cloud storage within a single management environment. This way, you can utilize the quick-access and security options of an on-prem storage unit and the utility of cloud storage at the same time.
In such hybrid settings, the cloud can potentially be used for content distribution and even disaster recovery. It can also be paired with cloud-based applications for transcoding or metadata enrichment.
Benefits of a Cloud-Based Archive
Let’s see what a cloud-based archive can do for you and your media business:
- Free up expensive production storage
- Archive from a broad range of popular OS types
- Reuse, repurpose, reference, and monetize your files
- Customize metadata to your liking and speed up search
- Browse and search the archive catalog with thumbnails and proxies
- Reduce backup size and runtime by migrating files from production storage
- Protect assets from disaster and for future use (requires no further migration or upgrade)
Having Money with OPEX
Capital Expenditures (CAPEX) are a sum of the costs that organizations spend on purchasing hardware to be used for more than one year. In media, when dealing with CAPEX, think of investments to buy storage and servers.
On the other hand, Operating Expenses (OPEX) are the daily cost of a company’s business operations. For example, in the context of media archives, it’s the monthly fee of a cloud storage provider.
With the right cloud storage, you can reduce your CAPEX with monthly payments fitted to the exact amount of cloud storage your media files take up. This will also save on maintenance and data migration costs.
Additionally, migrating files to the cloud makes the expansion of high performance and costly production storage unnecessary. In the long run, this aspect alone will make investing in a cloud-based archive worth all the money.
Planning a Media Archive
Making a plan for archiving is crucial, especially when considering factors such as the number and size of the files to be archived, the actual size of the archive (storage space), and the number of people who’ll be using it.
Forward-thinking can make sure that the archive will deliver the value it is built for and the inclusion of people who will configure, operate and use the system to guarantee a high level of acceptance and avoid blind spots in your planning.
When mapping out the plan, go through these six questions below and answer them to the best of your knowledge:
- Defining users: who administers, who uses, and who archives?
- Deciding and selecting: what goes into the archive, and when?
- Figuring out what will be searched for: what kind of metadata is needed to describe the particular data you need?
- Security: What hardware, software, OS, infrastructure, network, and medium will the media be archived?
- Security requirements should be fulfilled: off-site storage, duplication, storage duration, test cycles of media, generation migration, etc.
- Retrieval: who searches with what criteria? Who is allowed to restore, on what storage, and for what use?
How to Name Media Files?
The file name is probably the most robust metadata element you’ll deal with, mostly because it is the only metadata type available all the time, independent of the database, application, MAM, catalog, or another system that can store and read metadata.
It can migrate through a range of different OS’s and file systems. Utilizing metadata can help make sense of a file that was misplaced, leftover, or transferred to another location.
This is why creating a solid and consistent naming convention for your media files is our last tip in this blog post. It is mandatory if you want to ensure the efficient operation of your archive. As metadata is the foundation for working, searching, and sharing files, smart and consistent file names are good metadata.
Our full-scope OTT solution, ENT.360, is an enterprise-ready, software-only AWS native architecture that answers these challenges. With it, we architected and developed a global, monetized content distribution platform for a market-leading streaming service provider that was available in 50+ countries across more than 15 devices. ENT.360 is Well-Architected Solution.
Would you like to see the full scope of our Cloud.Blue services? Head over here and see what we can deliver for you in cloud architecture and development, AWS Serverless, AI, media services, DevOps, digital transformation, and service monitoring solutions!