Miss the first two parts of the series? Check them out here!
Let's look at another option: Our platform software.
We put in an NDI capture volume. So we go into our software and we have the option of an NDI receive and an NDI send. With something like a logo file, you hit send and the video is out there for viewing.
In this case, we are showing an NDI file and, as you can see, the video shows our office. This is not a still photo, it's live footage. And here's the big reveal: There's a UPS truck driving around our building. Pretty exciting, isn't it? This is just video from a RED camera that is pointed right out our window. It's also a feed on the network. We capture that live feed and, with lots of encoding in there, it's already prepped and ready. If you want to put this footage immediately into Premiere Pro or Avid, there is no waiting to do that. There's no moving that data around. If nothing else, that's been saved to our RAID storage and it's now protected.
We also have this little chop button for the camera. If you want to start editing some of the footage even before the video is done, you hit chop. This stops the file on one particular frame, and the footage you want edited is picked instantly from the feed by an editor. Editing starts immediately. Simultaneously, the camera starts up a new video beginning with the very next frame. Nothing is lost in the transfer, as the camera keeps capturing video at the same time as the editor is making selections back at the studio.
This type of workflow would never be possible with SDI or SDI switches or splitters. But it can be integrated into a ProMAX Platform. We actually have an SDI switcher as an option on our Platform but it's like $10,000. That's nuts.
RAIDS, LTOs, and other Backup
There are three basic ways to backup: disk-to-disk, disk-to-tape, and disk-to-cloud backups. These are common options and mostly what people in our industry use. Each method has advantages and disadvantages.
I find one thing very elusive about backup. I love the idea of the cloud and backup in the cloud. As you know, the cloud is off-site in a different location, and that's a beautiful thing. But what's always elusive to me is how fast can I make that happen. How fast can I retrieve material from the cloud? To figure this out, I made some cloud and LTO charts to find out.
Retrieving a stored file can take minutes, hours, and days. Actually, some downloads can even go on for years. If you have a ten-meg connection but you want to back up 100 terabytes, it's going to take at least two years to get it downloaded. The cloud is awesome but you want to watch out for recurring costs and speed.
If you want to do the job cheaply, disc-to-disc backup is really the way to go. The costs for Synology RAIDs or QNAP RAIDs are ridiculously low—you can get one of those set up for $2,000. Not bad.
I have a customer who has three Thunderbolt RAIDs in their drives. They had three Thunderbolts that they got for $300 each but they didn't come with drives. So they loaded them up with 10 terabyte drives. The whole thing cost them like $1200 and they had 350 terabyte RAIDS. That's awesome. Get something like those, but use Carbon Copy Cloner. Of course, I'd be thrilled if you used ProMAX because we've got that all built in on everything. But you can build one of these without spending a lot of money. If nothing else, get four or five terabyte drives off of Amazon and use some kind of application to back these up.
Again, if you go back to that core question: Is your data critical? If the answer is yes, you absolutely shouldn't rely on having only one copy of a livestream. Don't do it!
What about LTO? It's got upsides and downsides. We do a lot with LTO. It's slower than this, faster than this. Kind of the middle option. I like it a lot. Depending on how much data you are backing up and depending on how you organize that, different options work best for different people.
No Room for Failure
There are also replication and failover environments that go beyond backups. Very few people do this. But we did put in a replication system for an NFL team. They have more money then they know what to do with, so they didn't mind paying. But this is the scary part: A lot of the time I feel organizations believe that they have a high level of availability in their storage but what they really have is a USB drive's worth of availability.
In the NFL, if they have 5 minutes of downtime for the video, people get fired. That's an important thing to think about. How much downtime can you afford? It's an uncomfortable question. I ask that of my customers now and then but I actually don't like asking it.
Sometimes you may face two days of downtime and you are mad at your hardware vendor. But you find a way to finish over the weekend, even if it means it will eat up parts your days off. This is a replication environment where automation is the answer. This is an environment where you need two servers mirroring all the incoming data in real time. So this feed is going through two separate switches on two separate paths. A switch can fail, a server can fail. There are all sorts of redundancies here; there is no single point of failure. This is a replication environment.
There again in your organization, the answer to the question may be, " We cannot have downtime." This is the kind of solution you have to look at. The good news is that it's a lot more accessible and a lot more realistic to do this now than it was even five years ago. Five years ago you would have spent $2 million to set this up. You could do this for a much more reasonable price these days.
Archive vs Backup
Many people don't understand the difference between archiving and backing up. The two get jumbled together all of the time. These are two separate things to think about because they are two separate parts of your business. We were just talking about mission-critical data, how do we protect that? That's all about backing up yourself. But when it comes to archive, you get down to the core question about whether you may need access to a show or some footage now or later. Because NDI sets it up so we can automatically grab those files whenever we want to, we can automatically tag those files and automatically archive those files. Everything is in a single ecosystem now. So we've got a lot of different formats for archiving. There are Sony ODA, LTO, the cloud.
There are two things you have to think about if you want to access a show later.
- What is the archive format you are using?
- How are you going to search and find this material later on?
Though you can have the best archive format in the world, if you're not planning on your method for searching and finding those files, then you are not going to find them.
This is the biggest mistake I see for finding files in an archive: A folder structure is NOT a search tool. People rely on folder structures and they think that that's how they will find files later. But if you or anybody else in your organization ever needs to locate files beyond six months from now, you and those you work with need a more formal way to search for and find files. I know from my own experience. I used to use a folder structure and always thought I knew exactly where a file was stored. I put there. But when I went back to find it six months later, I couldn't remember where I put it.
So what are your alternatives? You can either have a kind of a software-driven system or an archive-driven product. For the first type, I'm sure you've also have heard of the buzz term metadata. Metadata is information that you are associating with these files. And the asset management systems are an interface for you to be able to search and find these files. This is not information based on a folder structure or the name of a file. This is what you want to talk about when you want long-term use of this data. You have to be properly organizing it, properly tagging it, and doing things so you can access it years from now. You don't have to spend hundreds of thousands of dollars to do this. I know lots of people who do this using FileMaker Pro. It's cheap, it's easy, and it works.
Then you also have archive-driven products, like ours at ProMAX, where you're combining on some level the archive piece and the asset management piece. These are also addressed on such applications as those from Axle AI, CATV, and others.
The Final Search
I came in here earlier to set up some demos. One that I created was an asset management metadata field called Film, to which I added some movie titles. Later, I came in here and searched for Film/Back to the Future. The idea is that you can actually tag that information. The reason why people talk about metadata and why it's so absolutely critical is that naming conventions and folder structures have failed again and again and again over the years.
The one thing we've found that actually sticks—we being the whole industry—is metadata. Create logical metadata based on how people are going to search. Maybe you start with a project number, a really common starting point. Rather than remembering where something is or rather than having a naming convention that you are searching for, you come in with that project number and search for it and find it. Through numbers, you can actually find what you are looking for.
This approach to finding files will help more than your coworkers. You also want future employees at your company to be successful after you're no longer there. Using project numbers is a way to do that. As you search your archives, you can search for your online storage. This allows you to search through all your databases. So with these asset management systems, you are building up a database that connects your metadata to your files—no matter where they are stored.
More About the Author
Nathaniel Cooper, ProMAX COO
Cooper has been with ProMax six years. After five years in sales and as sales manager, he became the COO within the last year. A 2001 graduate of Conservatory of Recording Arts and Sciences in Mesa, Arizona, he began his career working for a St. Louis recording studio. After a couple of years, he grabbed a chance to work for a shared-server business venture. In 2018, ProMAX CEO Jess Hartmann named Nathaniel as the ProMAX Chief Operating Officer. His primary goal is continuing the company's growth within the Shared Server/Asset Management/Storage industry. Leadership at ProMAX is dedicated to developing technology solutions that enable end-to-end integrated workflows and improve the video content creation process.
A Breadth of Uses
During the presentation by ProMAX COO Nathaniel Cooper, he had a visit from Dr. Andrew Cross, President of NewTek. Here's how Dr. Cross describes the market reaction to the emergence of NDI.
"I am seeing NDI used in so many different ways by so many people that I am slightly taken aback by that. Last night I got an email from a guy who used it on the LED and laser signage on the side of a building. I had absolutely no idea NDI would ever be used for something like that. I've had personal friends who do things like using it with a graphics operator from their video production. They had somebody running graphics totally separate from the production. I had kind of hoped it would be good for anyone who wants to transfer local video on a network. I shouldn't be surprised but I find the breadth of uses has surprised me."