On-Set with DIT Rich Roddman – Talking High Performance Shared Storage

A Tao Colorist Newsletter Sponsor Case Study

DIT Rich Roddman

Rich Roddman is a professional DIT and Colorist, based in Tampa, Florida.

Rich Roddman, C.S.I. is a highly experienced and talented DIT and colorist working out of  his own SilverBox Studios in Tampa, Florida. He’s been a good friend of the Tao since I met him in Orlando in 2013. I’ve always been impressed with his professionalism and his dedication to making his DIT craft as frictionless as possible (a big deal for on-set professionals).

Earlier this year LumaForge reached out to me to be a sponsor on our weekly Tao Colorist Newsletter. As part of my due diligence, I helped Rich and LumaForge team up for a very data-intensive, quick-turnaround, multi-editor live event that Rich was about to do for Wrestlemania.

This story is about how Rich teamed up with LumaForge—and his experience using their Jellyfish shared storage solution. He ended up keeping the Jellyfish for an entire week, testing it on three very different jobs: a 4K Sony F55 multi-camera “Behind the Scenes” shoot; a Phantom Flex and Arri XT product shoot; and a RED 5K .r3d direct response shoot.

It’s an interesting look at the concerns and challenges of a professional DIT and how the Jellyfish may have found an unexpected niche.


How a Routine Dinner Led to a ‘Shared Storage’ Case Study

Just before NAB 2016, I was having dinner with Rich Roddman. His DIT work brings him to Orlando fairly regularly and we try to get together when he’s in town. At this particular dinner, Rich was a little anxious about an upcoming gig for a live Wrestlemania event.

Rich Roddman at stage level for Wrestlemania in April 2016

Rich needed to simultaneously offload, archive and transcode (4) Sony F55s and (2) Canon C300 Mark II’s for a Behind the Scenes camera crew… and finish before the lights were turned off on him.

 

Rich expressed concern about managing the data flow for a team of documentary cameras

This Wrestlemania gig was going to be very demanding. He needed to offload multiple cameras, simultaneously, for the Behind the Scenes team. At the same time, he needed to transcode the camera original footage to hand off to the editorial team and he needed to make multiple backup copies for archival and safety purposes.

Rich was about to operate outside his comfort zone and he was a little anxious about it.

A short time earlier, LumaForge introduced me to their portable Jellyfish high-performance shared storage system

Based on what Rich was telling me about the demands of this gig, I though the Jellyfish sounded like a great solution for him. And since I really wanted to get someone to verify that this new sponsor of my Newsletter was legit, I asked if he’d like me to hook the two of them together?

Maybe I could get Rich to find out if the Jellyfish was for real? Or if I should be worried about my new Sponsor?

Rich agreed. And after a few emails, LumaForge was onboard and sent Rich a demo unit to use. Interestingly, the LumaForge team hadn’t really considered DITs like Rich as a target market. They were so focused on solving post-production problems, they were as curious about Rich’s feedback as Rich was anxious about solving his multi-camera live event problem.

After the event, Rich and I spoke on the phone… and his demeanor was unmistakable

From 90 miles away I could hear his grin through his words, “Patrick, I wrapped before the grips! That NEVER happens… especially with as many cameras as I was offloading. Even the camera crew was asking me how the heck I was working so quickly!”

For a DIT that is to-the-moon praise, indeed.

He told me had two more jobs that week and LumaForge was letting him hold on to the gear to test out—one was a Phantom job and the other a 5K RED shoot. Shortly after Rich sent his demo unit back to LumaForge, they sent Rich a questionnaire about his experience.

What follows is Rich Roddman’s own words about his experience using Jellyfish in multiple different on-set jobs

And in case you’re wondering, Rich called me this week to let me know he’s getting ready to pull the trigger on buying one of these units. That, more than anything, makes me feel good about having LumaForge as a Tao Colorist Newsletter sponsor. Enjoy reading the details of Rich’s experience!



What made you feel like you needed shared storage on set?

As a DIT I am always looking to improve workflows for speed and data safety while on location. Normally you are one of the last to leave the set with everyone waiting for you to finish. Anything that can speed up the process while maintaining the integrity of the media is worth exploring; it makes me look good and I hate being the reason anyone has sit around and wait.

What were you worried about with the various shared storage systems you had seen?

When I first looked into working with shared storage systems the two things that immediately stood out were the general size of the systems and the cost. They were designed to be installed into post houses and not be moved. They weighed 80-100 pounds and used great amounts of power and still were quite fragile for working day after day on-set and moving around to different locations. Then I added the purchase price of the system into the equation and there was almost no ROI

Rich's DIT cart, ready to rock for Wrestlemania

Rich’s DIT cart, ready to rock for Wrestlemania. The Jellyfish is the silver unit on the top shelf behind the keyboard. (click to open full size in a new tab)

What was the reason you were always the last one to leave on set?

After 12, 14, sometimes 16 hours the only words you want to hear from the AD’s mouth is “That’s a wrap!” It has the same effect as “Start your engines” at a car race; it means the shoot is over and everyone sprints to pack up and go home.

Except for the DIT.

There is always footage that’s sitting in the camera and has to be backed up and processed. Depending on what the project specs require, you’re usually working away as all those around you pack up and say their goodbyes. Rarely do you beat out the grip & electric department—and then, only if they have a full truck to pack up and the director hasn’t been doing a continuous roll throughout the day (which nowadays, happens all the time).

One might think, “What’s the big deal? It’s just hard drives and data?” but someone still must copy and process it. And that someone is me, the DIT.

What was the workflow you had in mind when preparing for this Wrestlemania job?

To quote Tom Petty, “The waiting is the hardest part” and as a DIT you are always waiting on one of two things:

  • 10-20 minutes for files to transfer from the card
  • 30 minutes for transcoding for editorial

And always… you’re only doing one of those things at a time before moving on to the other task

I thought, there has to be some way of multitasking. But with bus powered drives (which clients always use for delivery) and a single computer almost any solution that multi-tasked for me, slowed down the process—or worse, corrupted the media… which is completely unacceptable.

So I wondered why there can’t be a Raid 5 type system that I can connect multiple computers into so I can offload media and at the same time transcode media with other computers, allowing me to remove the waiting time for starting the next card? I just hadn’t found a system that met my on-set needs.

Why was preventing you from executing that workflow in the real world?

The combination of single drive enclosures with computers running background tasks made using my existing gear impractical. And the robust shared storage systems I looked into were too expensive and fragile to justify bringing into the field. Trying to create my own system opened the door for media corruption.

When Patrick mentioned the Jellyfish to you, what were your concerns?

The LumaForge Jellyfish hard at work

The LumaForge Jellyfish hard at work at WrestleMania

The first concern was: Is this going to work as advertised?

Being able to copy and work from a single raid using multiple computers in the field seems like a simple concept but none of the major drive companies were doing anything close to that—not while also maintaining my data integrity, minimizing latency between computers and having the physical SAN being jostled about.

Then I was worried that I’d need a degree in IT to set it up. But once we had the Jellyfish up and running, that concern fell to the side quickly. A mere mortal can set this up. 

The LumaForge Jellyfish hard at work

Rich’s DIT cart is packed and ready for shipping. The Jellyfish is on the top shelf of the cart.

My next question was: Could the Jellyfish take the ground and pound of location shooting? Not only do I have to deal with airline weight limitations but what about setting up on a sea wall or in an airplane hanger? None of the locations are ideal for hard drives but the Jellyfish didn’t seem to care. It fired up every time and over 10TBs of media sitting on it’s drives continued to scream as 5K R3D files moved across its cables and connectors.

What was your setup across your week with Jellyfish (Wrestlemania, Phantom, RED)? 

The system we were using allowed us to have 3 computers running on the 10gb ethernet connections at the same time. This gave me a variety of hardware options. The basic setup was MacBook Pros ingesting the media and a New Mac Pro running DaVinci Resolve for processing and render.

The equipment we had with the Jellyfish was consistent for all of the jobs we tested it on.  Two Apple MacBook Pros 15” and a new Mac Pro 6-core with 64GB of Ram and the D700 Graphic cards for more GPU processing power with Resolve.

How much media was passed through/how much transcoding, etc.?

For WrestleMania our main concern was ingesting all the footage from the 6 cameras during the 5-6 hour event (from the time the gates opened). We were shooting 4K XAVC on four Sony F55s plus two Canon C300 Mark II’s and there was some transcoding involved but it was not a priority for the evening. Our main concern was getting all the footage backed up before the venue turned the lights out.

Within that 7-hour window, we created and copied over 4 TB of media. I was able to walk out on time with the rest of the crew!

That following Tuesday we were working with the Phantom Flex and Alexa XT offloading both .cine files and ARRI Raw files, then transcoding them to ProRes LT 1080p. The schedule had only one camera working at a time so I only needed to set up one laptop along with the Mac Pro to the Jellyfish.

We created a little over 2TB between the two cameras which sounds like a lot of data but in reality, with these codecs there was not a lot of footage recorded time wise. Again the Jellyfish had me waiting on new media instead of the camera department waiting on my transfers to finish. The camera department was surprised how fast I was returning cards and asking for more.

That weekend was the infomercial shoot using the RED and 5K footage of R3D Raw media. Once again we deployed two laptops and a new Mac Pro. One laptop to ingest the new 5K footage, a second to playback 5K footage that was shot 2 months prior (showing Before shots of the guests being interviewed).

And the Mac Pro was creating h.264 1080p files with timecode burn-ins for the post house, while the rest was going on.  We created just under 3TBs that day with another 4TBs loaded into the project for on-set playback.

What was the most real world bandwidth you were able to pull across the various computers at a given time?  How much were you taxing the system?

LumaForge Jellyfish simultaneous transfer speeds

In a quick moment of ‘downtime’, Rich did a quick test of Jellyfish’s bandwidth on two laptops. (click to open full size in a new tab)

To be honest, once everything was setup and running smoothly it was a real world production so testing our speed took a back burner. But there was one moment at WrestleMania where we had to offload 3 SxS pro cards at the same time. Each with a 110-115 GB of 4K Sony XAVC media on them.

With the two laptops and the one Mac Pro, 3 cards were offloaded and had a 64bit checksum run in under 9 minutes.

That’s over 330Gb in less that 9 minutes. It is truly a game changer.

What sort of “a-ha” moments did you have?  Did your workflow change at all from Job 1 to Job 3?

Each project presented its own version of an “a-ha” moment.

Working at WrestleMania
When wrap was finally called we had 6 cameras show up with 8 cards (2 they meant to send back earlier) yet within the hour (and before the camera team had even finished packing up their camera gear) we had offloaded all the media from the entire day onto the Jellyfish. Everyone was surprised, yet that was my plan and the Jellyfish made it a reality.

Shooting a beer commercial
The second job was working on a beer commercial for a Japanese company using a Phantom Flex shooting at 1600fps. Our plan was to connect the Jellyfish directly to the 10gb port on the camera which would allow us to bypass two steps of transferring the hundreds of gigs of media created by the camera before converting them to ProRes dailies as requested by the Post house.

Unfortunately, we were not able to test this connection. The schedule had the camera practicing the location and positions of shots continually before the hero talent showed up on set. But I used the Jellyfish to offload media from the Phantom and ARRI RAW (from an Alexa XT) while making dailies. The speed of the Raid on Jellyfish allowed me to have all the offloads and transcodes completed—again before other departments had finished breaking down after the wrap.

Playback and Capture for an Infomercial
The 3rd job that week using the Jellyfish was working with a RED Dragon camera. This was the second shoot in a Before and After production for a new workout device. It was a combination of recording off-speed footage of the people working out on the machine along with their interviews of the experience over the past two months with it, all shot in 5K.

The director decided, on set, that she needs the talent to see themselves as they were two months ago, while they were being interviewed. Lucky for me I had guessed I would need that older footage and had loaded it onto the Jellyfish the night before. So at various points throughout the day I was:

  • offloading RED R3D media from the RED mag on one laptop
  • playing back 5K RED media in Premiere, set at full quality, from another laptop
  • transcoding h.264 timecode burn-in dailies for the post house in Resolve on a new Mac Pro

At times, this happened simultaneously and without any issues or perceived slowdowns.  I don’t know how any of these projects could have possibly worked so smoothly without the Jellyfish.

How do you see the Jellyfish changing workflows for the DIT?  What sort of workflows could this help open up on future jobs for you?

As most DITs will tell you, each project is just like a snowflake. They may all look the same from a distance but up close the details make each one individual and unique. Having a system with the speed and the flexibility that the Jellyfish offers allows me to take on projects that at one time would have been impossible to fit into certain time or budget windows.

In other words, it will allow me to take on new jobs that I otherwise would have had to turn down or put restrictions on.

Was the Jellyfish difficult to set up?

The Jellyfish could not have been easier to use:

  • Turn it on
  • Connect the Cat 7 cable to the ports from the Thunderbolt-to-10gb adapter, then the Thunderbolt to the computer
  • Open up the ShareClient app from Lumaforge
  • Mount the drive
  • Start working

How was support from Lumaforge?

The support I received from the Lumaforge team was amazing. When the Jellyfish first arrived I was having difficulty getting two of the four 10gb ports to work. After a quick call running down the start up procedure, we decided to check the card. Turns out a bumpy flight from the West Coast was to blame. Once I re-seated the 10gb card everything worked perfectly for each project I tested it with.

Do you see yourself using it again on future projects?

Without a doubt. The two basic questions asked by almost everyone that I encountered across the three separate productions were. First: How did it take this long for a company to see the need for such an on-set device where time is, literally, money. The other question: Where do they get their hands on one? (Editor: Check out more details about the Jellyfish)

I would like to modify the Jellyfish a little bit and make the casing a little more road-friendly but as far as what the insides can do, well, so far, color me thrilled.


About Rich Roddman, C.S.I.

With almost 30 years in the TV production industry, Rich likes to tell people that he fought in the trenches of the digital revolution. This gives him a unique perspective as to what works or does not work in today’s full digital workflow. This continues as technology increases it’s ever changing pace, widening the gaps between production and post production abilities and expectations. Rich specializes in bridging those gaps for his clients.

Rich opened SilverBox Studios in 2009 providing services connecting on-set to the edit suite, with successful workflows for productions large and small. In 2015 SilverBox Studios opened the doors to The Crayon Factory which is Tampa Bay’s only independent color grading and editing suite; allowing producers access to services they used to fly to Atlanta or New York to obtain. Rich was just recently accepted as a member of the Colorist Society International; a guild representing colorists from around the world.

Connect with Rich Roddman at:

Tags: , , , ,

3 Comments

  1. Mark August 10, 2016 at 3:03 PM #

    Hi, thanks for the great article. I frequently get Data Wrangler jobs with some transcoding needed too (ie not a DIT). One of the biggest bottle neck is the bus powered disks given by production, as you mentioned yourself. How did you overcome it in the above cases? The first backup and transcoding was greatly sped up with this system but how did you deliver? I’m never allowed to leave set without splitting backups (at least 2). Thanks

    • Patrick Inhofer August 10, 2016 at 7:53 PM #

      Hi Mark – Thanks for your comment! I’ve just emailed Rich to alert him to your question. You should get a response soon’ish 🙂

    • Rich Roddman August 10, 2016 at 8:52 PM #

      Hi Mark,
      That’s a great question that more and more data people are asking. For the majority of my jobs I provide a third drive as a redundant backup to the master and clone drives the clients bring. I’ll hold on to their footage for about two weeks which is plenty of time for them to back their media to the system. This way if both client drives fail or get lost and yes, it has happened 3 times so far, all is not lost. The drive I bring is always a fast thunderbolt raid or an SSD, normally always faster than the client drive. Now to the root of your question. I use Pomfort’s Silverstack for the majority of my offloading. They have a feature called cascading copy which will write to the fastest drive first (my raid) create a checksum and then write from my raid to the slower drives using the camera checksum to confirm the perfect copy. This way the camera card can already be back in the rounds if it really needs to be.
      As for the transcodes, I render them again back to my drive and then use ShotPut Pro to copy them to the two or more client drives at the same time. Again using the checksum to be sure it’s a good copy.
      For the WrestleMania job the Jellyfish acted as my raid in the process I just listed. But it’s grand advantage lets me connect several computers to offload multiple cards.

      Hope that answered your question.
      Thanks,
      Rich-

Post Comment