As I sat here pondering, I thought about some of the issues that have frustrated me in the past and one that really stuck out in my mind is a bit of a math calculation that I have to do quite often; calculating the synchronization offset for multiple cameras in my video editor in order to synchronize the video clips. It is not that it is hard to do, however it is a bit tedious if you have to do it with just about every video editing project you take on.
Just a little bit of history...
I have used many video editing software packages over the past 25 to 30 years and for those which allow editing of multiple video streams on the timeline, this becomes a bit of an issue. There are many good video editing programs available now, some only focus on editing one video stream and allow you to create some very good video productions that you can then use to author a DVD or send up to your YouTube channel. However, some of the more capable packages allow you to edit more than one video stream on the timeline and when that is the case, you usually have to synchronize the video relative to one of the other video tracks.
There is a local Gospel music group here in the Central Texas area called The Steel Magnolias. They've been active here in the area for 6 or 7 years. About 4 years ago, they approached me to help them with some video one of their 'fans' shot for them at one of their appearances. They wanted to be able to put it onto a DVD to sell at their product table in order to raise money for their ministry.
I got the video from them and loaded it into my editing software I was using at the time, Pinnacle Studio 9. Up to this point I really had not needed to do any editing with video shot via multiple cameras but it was a real challenge with Pinnacle Studio, not because it was hard to calculate the offsets, but because in order to create the desired scenes, you had to actually cut video out of the top-most video stream in order for the rendering engine to display the video stream below it.
This resulted in a nightmare if you had to go back and try creating a different scene as the process of making the cuts in the top-most video stream was destructive to the content of the timeline. Once you delete it, you can't get it back. You have to start all over with your project.
I started looking at a lot of other video editing software with an eye toward how multiple video streams are handled. Every package I looked at, except one, used the very same destructive approach toward editing multiple video streams. The only software package that didn't do destructive timeline editing with respect to creating scenes from multiple cameras was a product called EditStudio Pro from Mediachance Software.
As soon as I downloaded the trial version of the software I knew it was a great video editor. At the time it was only $90, which was an absolute steal for what it provided in functionality. I had used several other Mediachance products for doing photo editing and creating high dynamic range photos and I always found their software to be very worthwhile. EditStudio Pro was certainly no exception.
The thing that makes it different in terms of editing multiple video streams from multiple cameras is you can add what is called a "Multi-Camera Layer" to the timeline along with your multiple video layers. When you add this layer, you then have the ability to assign each of your video tracks to a "Camera" button that is available for you to click on at the desired point in the timeline in order to post a key frame into the Multi-Camera layer to switch the display from one stream to another. The real beauty of this method is you do not do ANY cutting of video information out of the timeline to create your scenes.
The Multi-Camera layer is used by the rendering engine to automatically switch from one stream to the desired stream at the time the video is rendered. The Multi-Camera layer also enables the use of an editing display that will show up to 4 video streams on a monitor so you can identify which camera would be best to create the scene at any given point in the timeline.
Another very distinct advantage to the use of the Multi-Camera layer is if you (or your client) decides they don't like a particular sequence of scenes, you can simply add a new Multi-Camera layer and simply disable the old one, just in case you might want to come back to it later. You can then create your new scenes on the new Multi-Camera layer and get your client's approval, without the hassle of dealing with re-building your video timeline as you would have to do in virtually every other video editor.
I originally purchased EditStudio Pro in 2009 and it became my go-to video editing package almost immediately. It has extensive text titling, video effects, transitions, masking and matting, just about anything you want in a good editing package.
Recently, I saw on their web site they have reduced the price to $49 as they are no longer licensed to distribute the MPEG 2 rendering engine that was used to output to DVD video format. It's really not a big deal to me, even today (although my installation was installed prior to this turn of events) as you can simply save your video out as DV format and use your DVD authoring software to render the video into the MPEG 2 format. For $49 bucks, this is probably the best video editing deal on the planet.
The only other drawback that has been caused by the passage of time and technological capabilities and advances in video cameras is EditStudio does not import MP4 or HD video formats that some of the new cameras record into. It's still not a big deal, just import the video and convert it to DV video and then pull it into EditStudio.
Here's a video I created for The Steel Magnolias using EditStudio Pro 6.05 using multiple video camera streams.
So why did I write the calculator?
All of this being said, I still had the issue of doing all of the time offset calculations bugging me. I had to figure out an easy way to do the calculations and eliminate having to find a piece of scratch paper to do all the math when I needed to synchronize streams.
One aspect of EditStudio that is also different than most other video editing software packages is the format used to display the position of the timeline cursor is NOT a format that is part of the SMPTE (Society of Motion Picture and Television Engineers) timecode standard.
EditStudio uses the timecode format of:
0h 00m 00s 000
This format represents hours, minutes, seconds and milliseconds.
The SMPTE timecode standard used in many of the other video editors is:
00:00:00.00 or 00:00:00:00
This format represents hours, minutes, seconds and frames. The portion of the timecode that represents frames may be delimited by either a period or a colon. In some cases it can also be delimited by a semi-colon.
There are also several different video standards in use in the television and motion picture industry:
Most common here in the United States is NTSC (National Television System Committee), however there are also other standards in use around the world. PAL (Phase Alternating Line) and SECAM (Séquentiel couleur à mémoire, French for "Sequential Color with Memory")
Each of these video standards has their own frame rate or number of frames that are displayed in one second. For example, NTSC is 29.97 frames per second. PAL and SECAM are 25 frame per second and the film industry uses a couple of different standards: Film NTSC, which has a frame rate of 23.98 frames per second and Film, which has a frame rate of 24 frames per second. With the invention of Hi-Definition video, there are also NTSC 59.94 frames per second and PAL at 50 frames per second.
Naturally, you have to take the frame rate into consideration when calculating the time offsets on the video timeline. so the first thing you need to do in order be accurate is determine how many milliseconds each frame consumes in a second. For NTSC at 29.97 frames per second, the individual frames consume 33.36670 milliseconds. a PAL/SECAM frame consumes 40 milliseconds and a Film NTSC frame consumes 41.70838 milliseconds. (it all depends upon how accurate you need to be) while Film (24 fps) consumes 41.66667 milliseconds.
Once you have the frames broken down to milliseconds, you can then convert all of the other time elements to milliseconds to make the math much easier:
- 1 hour equals 3,600,000 milliseconds
- 1 minute equals 60,000 milliseconds
- 1 second equals 1000 milliseconds
So, a time value on the timeline of your favorite video editor (one that doesn't convert to milliseconds already) would be represented as follows: (assuming NTSC 29.97 fps)
01:33:47.13
hh:mm:ss.ff
3,600,000 + 1,980,000 + 47,000 + 433.7671
Naturally, doing all the decimal math would be a pain in the butt so we round UP the number of milliseconds represented by the frames to 434 milliseconds.
3,600,000 + 1,980,000 + 47,000 + 434 = 5627434 milliseconds of video. Now we have established a position on the video time line that can be very easily compared with any other position to determine whether we need to add or subtract time from the beginning of the video clip to synchronize the clips.
So let's say Camera 1 video begins at 5627434 on the timeline and the scene we want to synchronize to begins at 01:47:30.22 or 3,600,000 + 2,820,000 + 30,000 + 734.0674 or 6450734 milliseconds.
Camera 2 begins on the timeline at 01:32:00.00 ( 3,600,000 + 1,920,000 = 5520000 ) and the scene to synchronize is located at position 01:40:21.17 ( 3,600,000 + 2,400,000 + 21,000 + 567 = 6021567 )
Now to do the calculations, we have to first find the difference between the synchronization position for each camera. You can pick either stream to do your calculation, however if you pick one you may find you will need to ADD time to the beginning of the timeline in your video editor (some editors will allow this, and some will not). So for this example, we'll use Camera 1 sync - Camera 2 sync:
6450734 - 6021567 = 429167 or 00:07:09.05 so to synchronize our synchronization points we would have to move the Camera 2 video clip to the RIGHT on the timeline by 7 minutes, 9 seconds and 5 frames.
If we were to have chosen to use Camera 2 as our reference stream, the difference would be: -429,167 so we would have to move Camera 1 to the LEFT on the timeline by 7 minutes, 9 seconds and 5 frames to synchronize the desired scenes.
If at all possible you almost always want to move video to the right on the timeline, because you may find one of your video clips start times to be at 00:00:00.0 on the timeline which would mean you would have to insert time at the beginning of the timeline in order to move it the desired offset.
As you can see, you can go through several calculations only to find (depending upon the number of video streams you need to synchronize) that you need to use a different video stream as your primary or reference stream.
I decided I wanted a way to calculate this information quickly and present it in a way to easily determine the best stream to use (since I typically deal with 4 camera streams at a time) as the primary stream to minimize the impact on the timeline. The following image shows the results of using each of the video streams I used in my testing:
As you can see, using Camera 1 or Camera 2 in the above image would be much easier in terms of moving video streams on the timeline than using Camera 4 and especially Camera 3, which would require you to add time to the beginning of the timeline. The arrows show the direction the video stream should move in relation to the primary video stream.
Here is the same test data using the EditStudio Pro timecode format:
Using the program...
- Select the video standard you wish to use from the drop list. The program will then calculate the frame duration in milliseconds to use in the calculations.
- Select whether you want to use SMPTE time codes or if you are using the EditStudio Pro time codes format. The default is to use the EditStudio timecode format.
- Select the Camera you wish to use as your primary video stream, you can always select a different one later and re-calculate.
- Enter the Starting time on the timeline for the video clip for Camera 1.
- Enter the Ending time on the timeline for the video clip for Camera 1. The duration will automatically be calculated.
- Enter the Synchronization time from the timeline for the video clip for Camera 1.
- Repeat steps 4 through 6 for each of the video streams on the timeline as desired. You have to have at least 2 streams filled out, otherwise, what's the point of the calculation?
- Click on the Calculate button at the bottom of the screen. The Offset Time will automatically be calculated for each of the video streams and you will get a visual indicator as to the direction on the timeline the video clip will be moved. If you see an arrow pointing to the left, you should select a new Primary Camera and press the Calculate button again to recalculate relative to the new Primary Camera stream.
- You can press the Offset Time button (to the left of the Offset Time field) to copy the contents of the Offset field to the Windows Clipboard so you don't have to retype the time in your video editor to move the starting position of the video clip.
If you have any questions at all, you can certainly contact me and I'll be glad to answer them if I can.
Where can you download the program?
That's an easy question to answer. Just use the following link to download it from my Google Drive.Multiple Camera Synchronization Calculator
How do I get updates?
All you have to do is re-download the program from the link posted above. You'll be downloading the most current version.What is the program written in?
The Multiple Camera Video Synchronization Calculator is written in a software language known as Clarion for Windows 6.3 (build 9058) from http://www.softvelocity.comClarion has been around as a programming language since 1986 under the MS/PC-DOS operating system and I've been using it as my primary programming language since 1989. Clarion for Windows came about in October of 1994 and the product is still actively marketed.
