THE COMPLETE GUIDE TO HD FORMATS
- INTRODUCTION
- RIGHT TOOLS FOR THE JOB
- ACQUISITION FRAME RATES
- THERE ARE ONLY 2 BASIC TYPES OF HD EQUIPMENT
- BROADCAST HD FORMATS
- NON-BROADCAST FORMATS
- POST PRODUCTION
- INTERNATIONAL DELIVERABLES
- CONCLUSION
INTRODUCTION
The range of HD equipment available spans many different formats, frame sizes, frame rates including 24p, progressive and interlaced modes, compression types and other factors. There are also 3 main HD acquisition tape formats, plus new technologies which work on non-volatile memory sticks and another using blu-ray type disks. Some of the concepts are quite difficult to understand and information quite hard to disseminate, however, this essay seeks to make this easy and explain how to tell which is the right format and camera for your production.
I will begin with an introduction to the various formats and distinction between Broadcast and low-cost HD formats, then a discussion of each format's advantages and disadvantages. I will lead this into a simple discussion of HD post-production and HD grading before concluding on international distribution and making deliverables.
RIGHT TOOLS FOR THE JOB
The first consideration when choosing a format is to gauge the needs of the programme.
Are you hoping for an international sale or do you have international co-producers? This may dictate the delivery requirements of the programme, which may influence the post-production route and limit the choice of formats for acquisition and in turn the acquisition frame rate.
Clearly the DoP's choice of camera is paramount, but the basic choice of the format is critical, since shooting on an inappropriate format for your post-production route may burden you with unforeseen costs and this should be considered before you select your camera. My advice is, that in every HD production, seek advice and be clear as to how you plan to complete your post-production before you begin shooting and select your production format and equipment around this.
ACQUISITION FRAME RATES
As a guide, all European programmes are produced at 25 fps, so acquisition and post are produced at these formats and it is easy. Just because you shoot at one frame rate does not mean that you can't do your post-production at another frame rate - so although US and International programmes are frequently demanded at either 24 fps 30 fps, you can still shoot and post your programme at 25 fps as it is both simpler and cheaper, though you can obviously shoot and post at the required frame rate.
Production at 24 or 25 can easily be used at 25 or 24 just by changing the tape play speed, so these speeds are more commonly used. Production at 30 is relevant only for use in the "NTSC" countries, and then only when the material is interlaced. 30 fps is also very difficult to convert to 24 or 25 and takes a very good standards converter, so steer away from shooting at 30 unless production actually ask for it and understand the issues. If the material is shot progressive, they will want a film-look so they can easily convert from 24 or 25 to 60 (called a 2:3 pull-down). Discussion of the international frame rates is important but outside the scope of this essay, so I will leave it at that, so as not to scare you, though there is a final section on international deliverables for those interested.
The second consideration is whether you have a particular post-production route that you would prefer to follow? If you are planning to use an HD firewire-based edit suite, such as FCP, then this will also limit the choice of acquisition formats too since some but not all HD formats permit low cost firewire-based post production.
THERE ARE ONLY 2 BASIC TYPES OF HD EQUIPMENT
In my opinion there are 2 basic types of HD - Broadcast and non Broadcast quality. This artificial distinction helps to explain the various type of equipment and their practical uses..
The Broadcast HD formats are for 'proper' HD productions, dramas, documentaries and feature films, whilst consumer HD camcorders are basically high performance DV camcorders. The resolution of these cameras are higher than DV and some of the cameras have good, flexible menus and can perform surprisingly well. It may be a bit condescending to describe them as similar to DV, but they are closer to DV camcorders than to HDCAM or Varicam ones.
So if you have an International co-production - you will need to use Broadcast HD, and if you are producing for the local market or a low cost HD format, then the cheaper HD formats such as HDV may be considered. Some P2 and XDCAM EX fall in to a 'grey' category, which are neither one nor the other - more about those later!
.jpg)
If you are producing for the big screen either for HD projection or 35mm blow-up for feature release, then do not be dissuaded from experimenting from using new low-cost technology. Insist on a camera test from your rental house and shoot some scenes with lighting, textures and contrast that you plan to use in your production. Although the next bit will cost some money, I would recommend that you grade a couple of scenes and print them to 35mm and view the images in a proper viewing theatre. If you achieve your 'look' and the Director and DoP are happy, then the choice of production tools are appropriate for you and you will have won the confidence of your production team.
Even if you are not producing an epic for the large screen, I would always recommend a camera test for all HD productions, since the choice of camera format and camera head are not the only considerations - there are lenses, matte boxes, follow focus rigs, viewfinders, heads, legs and a wide range of other accessories to select in order to aid your DoP to achieve his desired look. A camera test will aid him in making these decisions and ensure that he is appropriately equipped to shoot your programme; it should also be a free service from your rental company and will engender confidence in the production tools, some of which may be being used by your crew for the first time on this production.
BROADCAST HD FORMATS
Broadcast HD formats allow digital cinematographic acquisition shooting in shuttered film mode (24p or 25p progressive modes) with film colour lookup table (cine gamma) to create a realistic 'film look'. The shooting is also very similar to shooting on film as the camera can be supplied with normal film-style accessories including appropriate matte boxes, follow focus, bridge plates, viewfinder extensions etc. They can also be supplied in a documentary (ENG) style and shoot in a video-look (50i interlaced mode) with video colour lookup table (video gamma) with a similar setup to shooting on Digi Beta. The wide product range covers equipment that shoots in all of the different frame rates depending on the application.
There are only now 4 high quality mainstream long-form camcorder formats that exist at present - HDCAM, DVPro HD (covers the Varicam and HDX-900), P2 (which is the same as DVCPro HD, just records onto memory sticks instead of tape, AVC Intra (a new development of DVCPro HD for high-end projects) and XDCAM HD. In this, I do not HDCAM SR using cameras such as the Sony F-23, Panavision Genesis or ARRI D-20 which are not very portable and also very expensive - effectively they are not true camcorders either, but a separate camera and recorder which can bolt together as a massive unit. So for the purposes of this essay, these will be excluded, other than a cursory mention of them.
The new range of “Digital Film” cameras, such as the Red one, Digital Imaging and others present difficulties of their own - more on that later.
All of the camcorders in this category with the notable exception of the small P2 camcorders use the identical 2/3” B4 lens mount. This means that they can all interchange their lenses with the best HD zoom and prime lenses in the market. It is also worthy of mention that SD lenses are fully compatible with all Broadcast HD camcorders but do not have the speed or performance of HD lenses. In particular, they will exhibit large amounts of focus breathing (zoom effect when focussing - very noticeable with large screen display), barrel distortion (curvy lines at the extremities of the screen) as well as chromatic aberration (colour smudge) and lens ramping (iris change during zoom). Just because a lens fits doesn't mean it's suitable. My conclusion is that HD lenses are worth the additional money, so don't be persuaded to sell yourself short! Some new low-cost HD lenses have just come on the market, which VMI supply as standard on HPX-500 P2 Camcorders. Whilst being suitable for low cost productions, these should not be confused with the Broadcast video and cine-style lenses used on the larger camcorders, due to them being built to fit a price-point.
All Broadcast HD formats are capable of being fitted with HD Prime lenses and many DoPs like using them for the shallow depth of field that they can produce.
A shallow depth of field is possible with pretty much all HD lenses, as long as you have a suitably good DoP and are flexible as to how far in front of the lens you shoot the subject but the additional quality and speed of HD primes (T1.5) makes this much easier.
35mm lenses can also be fitted with a P+S Pro35 adapter to instantly create the same DoF as 35mm - there are disadvantages to doing this such the additional physical size, losing 1.5 - 2 stops of sensitivity and are also being prone to flare in bright sunlight. These can deliver excellent results and the ubiquitous 'film look' with ease but you have to cope with the problems of using them as well.
HDCAM SR
It is fitting to begin with the best HD tape-based format to begin with: HDCAM SR. Created by Sony to rival Panasonic HD-D5, this has rapidly become the standard mastering format for all HD programmes today, mainly due to its low compression rate with near loss-less quality and also 12 audio channels - this enables Dolby 5.1, stereo and multi-language versions to be recorded onto one tape without all of the messing about of separate audio masters, so that one version can become a master for several language versions - a great advantage.
It is also used as an acquisition format for top top-end cameras such the Sony F-23, Panavision Genesis and ARRI D-20 (and can be used for Sony F900R etc as well). Whilst these are being used for top-end features and the occasional drama, they are very much in the minority compared with the other formats and are of an order more expensive.
The interesting debate is whether we actually need quality above HDCAM and DVCPro and this is an important issue to address.
When you shoot on film, every successive generation of film print is an inferior copy of the previous one, so it makes logical sense to shoot on the best format possible - 70mm is better than 35mm and 35mm is better than S-16. With HD, most post stages are loss-less, so as long as you take care to use the best archival digital format between post-production stages, your mastering quality ought to be indistinguishable from your original rushes recordings.
HDCAM and DVCPro HD have long been used for shooting productions which have been blown up to the big screen effectively, so although these better formats are available, not all programmes benefit greatly from their additional performance.
However one caveat is that if you have a lot of graphics compositing or blue-screen work, then recording less compressed images is a real advantage and in situations like these, the additional data recorded is a real advantage and the additional cost merited. However, for 95% of other stuff, it will simply cost you more money - you decide!
HDCAM
Of these mainstream formats, HDCAM is the most popular. It records 1920x1080 to tape (camera captures 1440x1080), is capable of extremely filmic image quality with 25p recording modes and cine-gamma and the cameras are made by Sony so controls are instantly recognisable to cameramen.
The format has a product range which can shoot at all of the international frame rates including the universally popular 25p and 24p progressive modes. The most popular unit in Europe has been the Sony HDW-750P which can shoot in 25p 'film look' and 50i 'interlaced TV look' and this has recently been superseded by the Sony HDW-790P which can shoot at the same frame rates, The aging HDW-F900H has been superseded by the lighter HDW-F900R as the standard unit for shooting in all frame rates for either Europe or US markets and including 24p.
Although there is a significant price difference between the HDW-790P and HDW-F900R, many claim that the performance of both camcorders at 25fps is very comparable (some say identical!)
Other cameras exist including HDW-730S which is not capable of 25p 'film look' and only shoots in TV-look interlaced modes, though this obsolete camcorder is rapidly falling out of favour (it only shoots in 50i and also 59.94i for the US). Also, in the US only, the HDW-750 and HDW-790 (note no P on the end - confusingly, these cameras only shoot in 59.94i interlaced mode only and should NOT be confused with the HDW-750P and HDW-790P available in Europe)
HDW-790P vs. HDW-F900R HDCAM Camcorder
It used to be that the HDW-750P and HDW-F900 had important differences and it was easy to distinguish these units apart.
The latest 3rd generation offerings from Sony are such that there is very little to choose between them for all projects shot at a European frame rate, i.e. 25p and 50i. However, if you need to shoot at either 24p or a US HD format (29.97p and 59.94i), then only the HDW-F900R is suitable. If your experience is only the old HDW-F900, then be prepared for a smaller (same size as the HDW-750P and HDW-790), lighter, more power-frugal unit, which really delivers cracking results!
The resolution of both units are identical and so is sampling (both are 3:1:1). Both also use the same Hyper HAD CCD sensors. Both been measured to resolve 12 stops of latitude using Zeiss prime lenses. Sensitivity of both units is very similar (around 400ASA with an appropriate setup card).
Even in colour depth are they both alike - Both use 12 bit colour quantisation, meaning that over 4 billion colours can be registered by both camcorders.
Both have the same digital HD and down-converted outputs etc etc.
If however, you have a limited budget and are shooting at 25p, then seriously consider shooting on the HDW-790 instead of the F900R and use the budget saving on better lenses - I can assure you that on this difference you WILL be able to notice the difference...
Better still, if you have a DoP who is insisting on one model over another - arrange a test shoot with both camcorders and see if anyone can tell the difference!
With the trusted Sony brand name, the interface is logical and robust and the cameras are amongst the most reliable in its class. As the cliché goes, “it does what it says on the tin”
DVCPro HD
Panasonic produce a competing format to HDCAM called DVCPro 100 or DVCPro HD. There are 2 x discrete formats which use the same DVCPro tapes: the first is Varicam (720p) and the second is a 1080p version which is used by the HDX-900 Camcorder. The tapes and VTRs are common but the similarity ends there.
Varicam - 720p DVCPro HD Camcorder
Like the Sony HDW-F900, the Varicam is a 'world camera' meaning that it shoots on all standard world HD frame rates including 23.98p and 25p. However, the Varicam system is an enigma of HD tape acquisition. It records smaller frame sizes (1280x720) than other HD formats but don't be put off by this. It can shoot at continuously variable frame rates between 4 and 60/second including 24p and 25p. The image processing and clever compression ensures that you can blow up for the very large screen with spectacular results (I have seen both Varicam and HDCAM on a 60ft screen and both looked equally stunning). The wide range of contrast latitude (measured at 13 stops by Alan Roberts) and cine gamma always ensures beautifully filmic results. Varicam also always records 60 frames per second irrespective of the frame rate mode and the clever frame flagging system permits multiple frame rates for acquisition and slow and fast frame speeds for special ramping shots (compared with HDCAM which shoots at fixed frame rates with a maximum of 30 fps).
Despite shooting in 720p mode, the VTRs will automatically uprez to 1080 so there is full compatibility with other HD formats in post-production and is also compatible with all 1080 delivery requirements for Discovery HD etc. It is also compatible with all AVIDs and Final Cut Pro for off-line and for on-line with 3 main considerations.
- The first is that the majority of your acquisition material must be at a standard frame rate (24p or 25p etc) for easy post.
- Secondly, should you shoot at any other frame rates including faster than real-time images (i.e. not 24P, 25P or any of the 7 prearranged production speeds, then you MUST record the 'off-speed' images onto a separate tape, as these images will need a special treatment. These images will need to undergo a post-production stage called frame-rate conversion in order to digitally slow down the images and uprez them to 1080. Without this stage, the footage is useless as it is incompatible with pretty much every post-production technology and the conversion can cost up to £400 per hour (VMI charge £150 per hour for this service), so ensure that you have budgeted adequately for this. This stage can be very effectively done using Apple Final Cut Pro - see the FCP/DVCPro section below.
- The third issue is that whilst Varicam has complete firewire support with Final Cut Pro, it is not for the feint hearted. It edits in the native 720P environment and additional HD-SDI cards are needed if mastering onto HDCAM or other 1080 line formats. To read a success story of a Varicam program being edited on Final Cut Pro, please read the testimonial on http://www.kenstone.net/fcp_homepage/dvcpro_hd_workflow_balis.html
Editing Varicam using the HD firewire port needs very specific settings, which are not intuitively obvious - in fact it only works if you work in 60 fps mode, irrespective of the project frame rate for example and the editor needs to be very clear about how Varicam and Final Cut Pro interact.
An alternative solution is to edit using edit suites with either an HD-SDI capture card or on dedicated uncompressed HD edit suites, in the same way as editing from HDCAM.
Irrespective of the post-production route, variable 'off-speed' footage will need to undergo a frame -rate conversion stage prior to off-line or on-line editing, whether this is done externally or within a PC or Mac environment using appropriate software plug-ins. A caveat here - be aware that various versions exist and ensure exactly what frame rates are compatible with your plug-in version prior to shooting - it used to be that only 59.95p was compatible with the FCP frame rate converter plug-in but I understand that this is no longer the case.
Panasonic HDX-900 1080p DVCPro HD Camcorder
Due to the difficulties of editing Varicam pictures, Panasonic brought out an equivalent camcorder to compete with the Sony HDW-790, which they called the HDX-900. This stunning camcorder also shoots onto DVCPro tapes but shoots 1920x1080 images in 25p or 50i, the same as the Sony HDW-790. The same highly regarded image processing as the Varicam has been introduced into this camcorder and it copes with a very large contrast latitude, same as the Varicam. It shoots extremely beautiful 'film look' images and in common with all of the camcorders already mentioned, is compatible with all HD zoom and prime lenses.
However, one major advantage of this camera and its format is that there is a total support for firewire post-production in 1080 mode. This means that for off-line and on-line, the same low-cost non-linear edit suites can be used using a single HD firewire cable alone. More on this later.
Compatible with 24p
Also in common with the HDW-F900 and Varicam, this is also a 'world camera' which means that it shoots in all world HD TV formats of interlaced and progressive in the European and US frame rates (25p, 29.97p, 50i & 50.94i) in 1920x1080 resolution. However, unlike the HDW-790, this camcorder can also shoot at the 23.98 and 24p frame rates in 720 mode for cinema release.
P2
The idea is that a common compression scheme does not have to always shoot onto tape - instead, a new workflow exists whereby a non-volatile memory card is used as a transitory medium, prior to archiving onto hard disks, where it will remain for life!
This is a relatively new idea, which is interesting since the cost of hard drive storage has plummeted so that hard drive storage is now cheaper than buying tapes!
P2 is no longer a new HD format - the workflow is now very proven and the native codecs widely adopted by post-production software applications. Invented by Panasonic, this workflow uses non-volatile memory sticks to record pictures on for easy post production. The idea is that pictures can be recorded onto DVCPro HD for the main production and that the memory cards provide a temporary transient storage medium, prior to being transferred to the main storage medium where it can be used and then later archived.
It is supported by small and large camcorders that record using the same compression scheme onto memory cards and tapes for a totally seamless HD post production.
This is how it works:
- You shoot onto memory cards on location - currently 16GB cards are the most common storage used (but 32 GB cards have just come out) which give 16mins of HD at 25p (or 38mins of HD in 25pn) or 64 mins of SD in 25p
- At lunchtime, with 4 x cards worth of material filled up, you use a box called a G10 P2 reader archive unit, to copy all media onto standard cheap removable firewire drives. This unit has a built-in screen, so that transferred media can be checked for peace of mind. The P2 cards are then wiped ready for reuse
- The firewire drive (1 TB can store more than 40 hours of HD material in 25Pn mode) can then be plugged directly into your post-production unit, like Final Cut Pro or AVID Media Composer Express, and then the images edited directly in the native HD format without the need for digitising or low quality off-line editing
- On completion of the programme, the drive complete with edited timeline, graphics etc, can be given to the post-production
(38 mins of HD when shooting 25pn - note 25pn only compatible with FCP)
The advantages of P2 over HDCAM are that all editing, including off-line are at the same HD quality, post-production machinery is cheap (all based around using HD firewire with Final Cut Pro and AVID Xpress) and the net result is that post-production ought to be cheaper than using conventional post production. These memory cards are currently quite expensive but the workflow is excellent - the success of the format will depend on how well producers adopt its novel post-production workflow but a strategic acceptance by the BBC that DVCPro HD using non-linear acquisition is its preferred choice should ensure the continued success of the format.
Also, if you are worried about hard drive failures, there are a range of new RAID 0 hard drives which record all data twice independently, so that in the event of drive failure, you can be confident that your data will survive.
Panasonic HVX-200
This was the first P2 based non-linear capture unit released in late 2006. It is a great unit with excellent optics which comes in either UK or US versions. It shoots HD format images of 720/50 or 1080 DVCPro HD modes onto P2 cards only and also records standard definition miniDV images onto mini-DV tape.
It is small, lightweight and a perfect accompaniment to a single unit package using either the HDX-900, Varicam or now HPX-500 units, as it uses the same compression scheme for easy post-production and the pictures closely match these camcorders too!
The UK version records in 1080 mode in 25p or 50i or in 720/50 mode in 25p or 50p. 50p would be used for slow motion. It is also capable of shooting 25Pn and 50Pn modes - see HPX-500 description below.
DVCPro HD 720/50 or 720/60 Modes
One irritation of the Panasonic 720 format is the existence of 2 x discrete versions for European and US production, namely 720/50 and 720/60. This is analogous to the PAL and NTSC formats and some camcorders are only capable of shooting on one of these versions. - Also the first Panasonic DVCpro HD VTR, the AJ-HD-1200 only supports the earlier 720/60 format, so it won't play back the Panasonic HVX-200 material if shot in 720/50 mode (either 25P or 50P). The later AJ-HD-1400 VTR overcomes this and is compatible with all DVCPro frame rates, both SD and HD.
For reference, the HDX-900 and HPX-500 are capable of 720/50 and 720/60 but the HVX-200 is only capable of shooting in 720/50 mode for 25p and 50p. All of the above can shoot in 25p and 50i in 1080 mode. Confusing huh?!
Panasonic HPX-500
This is a low-cost 2/3” 'World camera' which means that it works in all frame rates including 23.98p, 24p, 25p and the US frame rates too. It only records onto P2 memory cards and works in the 1080 DVCPro HD codec and also 720 mode as well.
On paper it does everything - 1080i and 1080p, slow motion ramping up to 60p using 720/50and 720/60 modes; firewire output; non-linear capture; great workflow; 2/3” lens mounting which means that a vast lens selection is available to hire for it. It delivers too but don't expect it to perform like an HDCAM HDW-790 or Panasonic Varicam because this is a low cost camcorder.
It also can switch between shooting in 720/50 or 720/60 modes for true US or European HD acquisition.
The really clever thing about this camcorder is that it can shoot in the Pn format DVCPro HD, which means that you can shoot slow motion images without recording replicated frames (like the varicam), so providing this format is compatible with your post-production suite (the latest AVIDs and FCP 7 can do this), then you can shoot slow-motion images up to 60 frames per second (60Pn) and edit them directly without the need for costly frame-rate conversion, necessary on Varicam.
In a nutshell, this is set to be the camcorder to replace the Sony DSR-570 DVCAM Camcorder. In my opinion it is designed to fit into the DVCAM acquisition and Digi Beta production budget but now deliver in true HD. Costs can be saved in post-production to help the budget and cameramen will be pleased with its standard size, weight and layout.
Panasonic Archive and transfer units
Since P2 cards are a transitory storage medium, it will always be necessary to copy the data off the P2 cards onto some hard drives. The Panasonic P2 Store offers a portable 60GB storage device to transfer up to 120 minutes of P2 storage onto a portable hard drive without the need of a portable PC or Mac and then transfer the footage onto hard drives back at base.
One of the drawbacks of the P2 store is that it does not have a display to have a visual check of the data that you are transferring - a green LED works when you have confidence in the technology but some of us need more than this, so we have the AG-HPG10 archive unit.
The AG-HPG10 is a portable P2 playback device, which has 2 x P2 slots and runs off batteries. What is remarkable about this device is that it can drive USB2 or firewire hard drives without using a portable Mac or PC, so you are not limited to using small drives.
Once you transfer your material onto hard drive, you can check playback of the transferred media off the hard drive to give you peace of mind.
This means that you no longer need to carry a PC or mac with you on a long trip and can make safety archive copies of all of your media with confidence. It also has analogue and digital outputs, so that you are able to play P2 material directly on larger monitors.
One further comment here about drives - it may seem that the best thing to so is to use massive drives on location so that you have limited numbers of drives to work with but this is not necessarily the case. Firstly, only drives 250GB or less can be powered from the G10 unit via USB2, so it makes sense to use these and also, the latest Mac OS only allows certain quantities of drive numbers, so increasing the quantity of P2 cards used uses up this capacity quite quickly - we have found that the combination of using 16GB P2 cards and 250GB ruggedised USB-2 hard drives are the best combination when using the G10 archive unit.
AVC Intra
Panasonic have created a new format for dramas and features to compete at the very highest quality and still be tape less. It is called AVC Intra and uses P2 memory cards, the same as the HPX-500 and HVX-200 but records in 4:2:2 sampling at true 1920x1080. In other words, this records greater resolution and greater colour information than the Sony HDW-F900R and it does so using removable P2 memory cards.
Clever compression means that it takes the same storage as DVCPro HD using P2 but you must use a post-production tool which works in the native AVC Intra codec.
The Panasonic HPX-3000 is the camera to go for as it records onto AVC Intra and also DVCpro HD in 1920x1080. It shoots in all of the frame rates but will not record 60p, as this uses the 1280x720 format that this camcorder does not support. However, it is expensive but what it competes with is much more expensive still - since it offers higher quality than even the top Sony HDW-F900R HDCAM Camcorder and competes with the Sony F-23, ARRI D-20 and Panavision Genesis! Whether it succeeds in the market depends very much on how well DoPs accept it. VMI are not expecting to take delivery of these cameras until Summer 08.
XDCAM HD
Think of HDCAM and you will have a tape format and of P2 and you have a solid-state memory format - but think about a format which records onto 'flat tapes' and you have XDCAM HD. These record onto a new format called XDCAM HD which are magneto-optical disks and promise to fill the middle ground for low cost professional TV production.
In HD, this middle ground is being contested by P2 with the Panasonic HPX-500 Camcorder and at the higher-end, with the Sony HDW-790P and now also with the Sony PDW-F355 XDCAM Camcorder.
In the second quarter of 2006, Sony released a new range of camcorders using a brand new format called XDCAM HD - before you groan about yet another format to learn about, understand that it is a professional range of camcorders with a similar feel and specification to the popular DSR 570 camcorders but in HD. They shoot in 1440x1080 format using true 25p and the range now begins with the second-generation PDW-F355 camcorder costing not much more than the DSR 570 did. The medium is also not tape based but records onto blu-ray type disks at 4 x types of compression schemes, 50, 40, 30 and 25Mb per second to vary quality and recording durations dependant on the programme. This technology is not actually new but a modification on a standard definition format produced by Sony called XDCAM which has not been at all popular.
The first XDCAM HD camera, the PDW-F350 Camera permitted variable speed recording between 4 and 60fps and only a data rate up to 40 Mb/s but it has not been popular and Sony have released the new range including the top of the range PDW-530 to compete with HDCAM, including a 2/3” lens mount and increased the date rate to 50Mb to improve the HD quality to compete with HDCAM and DVCPro HD, though both of these formats have a higher recording data rate to tape.
A major advantage when using XDCAM, since the recording medium is non-linear, is that it works very well with non-linear edit suites (NLEs), promising faster than real time digitise of proxy files etc etc. It remains to be seen whether these elegant new functions will find popularity outside the news environment but the initial specifications supplied by Sony will ensure that this format will be very popular.
However, if you are working in a traditional acquisition environment with traditional AVID/FCP workflow, then there is not really a lot of benefit to shooting on these camcorders that you cannot already have with HDCAM, although HDCAM records with a higher data rate. Sometimes simpler is better.
It remains to be seen whether the major HD Broadcasters will accept this format in mainstream production, since at the top end, the data rate at the higest quality is still only half that of DVCpro 100 and AVC Intra but it is clearly the aim of Sony that XDCAM should completely take the place of DVCAM and make HD acquisition possible at current DVCAM rates and at the same time provide a non-linear version of HDCAM as well.
Digital Film - The RED ONE and Others
There is a new breed of camcorders out there which we are beginning to see come out and be used. These shoot high quality digital images onto non-volatile memory storage and frequently have PL lens mounts for using film lenses designed for 35mm film cameras.
The main benefit to using these camcorders is that by using a 35mm sized imaging sensor, the same depth of field as using 35mm is assured, since film lenses can be fitted directly onto the camcorder without using a Pro-35 or other lens adapter - check though, as the PL mount for most 35mm lenses is a costly option on the Red camera and not everybody has selected this option when ordering the cameras.
They are very new and time will tell if they make as dramatic an introduction into the market as the manufacturers promise.
The main problems to using them seem to be that editing in 2K (2048 x 1080) or even 4K (4096 x 2160) is very difficult at present since the data files are huge and the performance of these cameras has not been widely tested. Time will tell how they are received and mainly used. The codec does not record uncompressed images but instead heavily compresses data in a proprietary data format which is neither standard nor well tested.
Also, there is an illusion here that a 'cheap' acquisition unit of $18,000 will bring cheap digital film production to all. The camera body may be cheap but once all of the constituent elements of viewfinder, HD monitors, matte box, follow focus etc etc are all added together, you will still have a £50K package which no longer looks so promising. The RED lenses are cheap, but my experience has shown that film makers are seldom content with 'cheap' lenses and tend to be much more selective about such critical elements as optics, the cost of which makes the camera part seem very cheap and insignificant indeed. Finally, there is the whole concept of shooting with a hard drive bolted to the back of the camera - this frankly is not for everyone.
However, time will tell if this is the new panacea of low cost digital film or another example of the Kings new clothes!
NON-BROADCAST FORMATS
In 1995 when DV was announced to the world and VMI bought these cameras by the truckload, all of the Broadcasters of the world snubbed these modest little cameras and announced that they would NEVER broadcast programmes from them. 10 years on and there are very few programmes produced that don't have even a little DV or now HDV in them somewhere.
The new low cost 'non-broadcast' HD formats are in the same category as DV was in 1995. They are effectively high-resolution DV camcorders which use clever technology to record high quality images using massive amounts of compression - as a comparison, HDCAM compresses 4.3.1 times (DVCPro HD is 6.7:1) and HDV around 30 times! However, the completely differing compression schemes make it quite meaningless to compare these figures with those for MPEG because they make no sense. However it is fair to say that HDV has a far lower bit rate and much lower quality images recorded to tape than either HDCAM or Varicam.
To my knowledge, without exception, all world Broadcasters have dismissed HDV as not being good enough HD content for HD broadcast, though this may change over time as budgetary demands dictate that HD programmes are made to SD budgets and lower.
Currently, there are 2 discrete HDV formats produced by JVC and a consortium which includes Sony which aims to totally replace current DV technology. Both XDCAM EX and Panasonic P2 with the HVX-200 Cameras will probably spell the death knell of HDV and DVCAM and end up taking its place, since people will be able to utilise cost-effective post production tools and low-cost non-linear acquisition to produce HD productions at current standard definition budgets.
HDV
Consumer HD formats are basically DV Camcorders with additional resolution and functionality - whilst they may work at HD resolution, HD Broadcasters do not currently consider the pictures to be of high enough quality to be suitable for HD Broadcast, though this may change over time.
With the exception of the JVC unit, HDV Camcorders do not have the same 24p/25p true progressive acquisition ability (film modes are usually cheats) or the same dynamic range and filmic qualities of the more expensive Broadcast HD camcorders. Also lenses, viewfinders, image processing and CCDs are all of a lesser quality to their 'Broadcast' HD counterparts, which explains the marked quality difference of images between the formats.
There are special HDV tapes which are purported to have better quality coatings to ensure faithful archival characteristics for years to come but you can use low cost DV tapes instead if you want to save money. In order to achieve the incredible resolution on a DV tape, the systems use up to a 30:1 MPEG2 compression ratio and store the images experienced between frames within a 12 frame segment. This works extremely well and drop out is rare, however, when dropout is experienced, you are likely to lose the entire image for up to half a second, as the system struggles to build a complete frame from the missing information. As such, it is a much less durable format than either conventional DV or any of the Broadcast HD formats and the best quality tapes are imperative for safeguarding against catastrophic dropout which disturbs the whole recording. Time will tell how much of a problem this will be for older recordings or those made in testing conditions.
Cameras are small and light and very similar to DV Camcorders. They use 1/3” CCDs which do not permit Broadcast SD or HD lenses to be mounted. P+S produce a 35mm lens adapter, the 'Mini-35' and there is the 'Movietube' which allow 35mm prime lenses to be mounted onto the front of HDV camcorders and whilst these work quite well, they are still prone to losing aperture (2 stops) and having flare, the same as the Pro 35 adapter for the larger HD camcorders.
With all of these non 2/3” CCD Camcorders, be very wary of some lens adapters promising to fit Broadcast and HD lenses onto them. The adapters contain no glass, so they yield softer images and don't give the same wide angle aspect as you would expect in a large camera (they always will have an artificial telephoto effect compared with when used in a large camera)
To confuse matters, in spite of there only being one logo for HDV, there are actually 2 discrete HDV formats produced by both JVC and Sony, they record different data rates and different image sizes and the VTRs are entirely incompatible - so if you are going to shoot with the JVC unit, ensure that you have a proven post-production route in place before you start as these VTRs are in limited supply and the format is incompatible with some firewire-based edit systems.
Sony Z1E
The Sony HDV format (1080i) shared by Canon and others records a very impressive 1440x1080 interlaced images (50i) which means that if you want to have a 'film look' 25p progressive look, then the camera performs a field doubling effect within the camera, effectively halving the resolution. Post-production is fairly easy now that various converter boxes have been released to permit HDV footage to be edited on conventional AVID-type edit suites. The firewire editing capability is very good and works well providing you have a suitable VTR, edit application and ONLY HDV footage - be wary if you have multiple formats, firewire editing works only with one format at a time - more on this later.
The Sony Z1E is a magnificent piece of engineering. It is a 3CCD Camcorder with a fairly good quality built-in lens, LCD viewfinder, steady-shot function and other refinements, and is not much larger than the ubiquitous PD150 DV Camcorder. It shoots in DV mode (DV or DVCAM) or alternatively in HDV mode in 1080i. There is a 'film effect' mode, which emulates 25p film-look by halving vertical resolution. It has different film colour lookup tables - (film gamma) to make images look more filmic and is priced at a disposable DV camcorder price of around £3,000. This is set to be superseded by the Sony HVR-Z7 during 2008.
There are a wide range of accessories available for this camera including 'cine-style' matte boxes, filters, underwater housings etc and mounts and the industry has immediately adopted this camera as the de-facto replacement to the popular PD150.
XDCAM EX
Just to confuse matter still further, the end of 2007 saw the introduction of yet another HD format - (just what the world needs!). This is Sony's answer to Panasonic's P2 system and it is called XDCAM EX.
This is deliberately confusing as it is completely different to XDCAM HD which shoots onto Magneto Optical disks (think of flat tapes…). XDCAM EX uses the same compression scheme as XDCAM but records instead onto low cost SxS ExpressCard media cards.
Like the P2 system, this is only a practical system if the post-production workflow is seamless integrated into the native compression scheme. Since this is a relatively new codec, it is not yet compatible with all current software packages in all frame rates, so be aware of this before you begin shooting and CHECK!
I have even heard stories that people have no been able to transfer media faster than real, which is one of the advantages of non-linear capture.
Sony PMW-EX1 XDCAM EX Camcorder
The PMW-EX1 is Sony's first camcorder, which unusually for a small camcorder shoots in all of the frame rates including 25p, 50i, 59.94i, 50i, and native 23.98P, as well being 1080i/720P switchable.
There is also a choice of a 35Mb/s High Quality mode or a 25Mb/s, HDV 1080i compatible mode. Once again, check that the data bit rate, frame rate and compression scheme is compatible with your software system as some people have been found sadly wonting! Otherwise you will be looking at real time data transfer which will really slow you down and defeats all of the advantages of non-linear capture!.
Its main advantage is that using two memory card slots with a pair of 16 GB SxS PRO memory cards, it can record up to 140 minutes of HD footage. This is much greater capacity than P2 but with more highly compressed images. It also is capable of under and over-cranking images in 720 mode of up to 60 fps with the benefit of immediate playback.
Like its big brother XDCAM HD, the benefit of using XDCAM EX is simply its way of manipulating images in 'off-line' using low quality 'proxy files' and manipulating images immediately using the express card data storage that is compatible with modern laptops and PCs.
This looks like it will be a great little camera but the workflow hasn't had all of the bugs ironed out yet and some of the Broadcasters can't really decide if the quality is HD Broadcast quality or not yet. However maximum bang for your buck - but only if it works for you!
JVC HDV
The JVC HDV format (720p) shoots true progressive 25p and 24p but uses a smaller image size of 1280x720 but compared to the Z1E, it actually maintains a better horizontal resolution and yields sharper pictures than the Z1E (. In spite of this, JVC have brought out a camcorder with interchangeable lenses called the GY-HD100. It shoots true progressive 25p mode with a realistic film gamma mode and several other nice features for little more than the Sony Z1E (about £5,000). Also, extraordinarily for the price, it also shoots in native 24p, something that currently no other camcorder in its class is capable of doing, however it is worth being aware of certain difficulties when using this camera and format.
Firstly, it records in the 19Mb HDV format, so that none of the other manufacturer HDV VTRs will play back its footage; the firewire output is not in the standard 25Mb HDV firewire protocol shared by other manufacturers, so you will either have to render all footage to convert it to 25 Mb (very time consuming), or use the Canopus edit software which to my knowledge, is the only software capable of editing the native JVC HDV footage (more may well be available now). It is also heavy on batteries, so be prepared to dump the manufacturers own battery system to the specifically designed IDX solution which works well but adds about £1,000 to the purchase price. Finally, the Fuji 16x lens which comes with the camcorder as standard it not very good, though of course, it is stunning for the price! Expect to upgrade to the Fuji 13x wide-angle lens which is much better quality, though it costs more than the camera and not many hire facilities stock it.
However, bearing all of this in mind, if you are considering shooting a true 24p or 25p HDV project and have overcome these difficulties, then it is a stunning solution for a bargain bucket price.
POST PRODUCTION
Intro
Firewire has become a commonly used data transfer format, which is capable of terrifically enhanced workflow benefits both for off-line and on-line editing.
All HD formats are capable of using low-cost firewire-based edit suites for off-line purposes but only DVCPro is capable of firewire post-production during the HD on-line phase.
Down conversion Copies
Whilst HD recordings are entirely incompatible with SD equipment, all HD VTRs provide standard definition outputs, which are compatible with PAL monitors, AVIDs and other equipment. As such, it is not necessary to make down-conversion DV copies of HD rushes in order to load them into an AVID or Final Cut Pro (FCP). It is actually more efficient to load each HD reel into an appropriate HD VTR and digitise each reel in Standard Definition as the VTRs down-convert in real time whilst you play back.
There is no argument for making down conversion copies because it cycles the masters less: In fact by digitising rushes a reel at a time, each reel is cycled only once - the same as making off-line copies, only it costs you less money!
VTRs for down-conversion playback
All HD VTRs of all formats permit standard definition output of HD material in real-time. This is of course frame-rate dependent, since if you are shooting at an unusual frame rate, such as 24 fps, there is no equivalent SD format at this speed. This is further justification for shooting at 25 fps - it saves money and hassle, since you never need to make down conversion copies but can digitise directly from the HD tapes into a SD off-line.
SD Off and On-line Edit
All HD VTRs with rare exception* are fully compatible with all standard definition off-line and on-line edit tools. They will plug directly into all edit systems as they use all standard cables and interfaces to emulate Digi Beta, Beta SP, DV or other common format. Connecting HD VTRs to AVIDs or other edit tools, relies on the frame rate being compatible with the edit workstation. As a rule, 25fps (25P & 50i) are always compatible (as are 29.97 and 59.94i - called NTSC) and you would be best to check with all other frame rates. 24 fps editing is frequently absent from AVID products unless they are called 'film composers'. FCP is generally compatible with all frame rates.
*The Sony HVR-M10 VTR has no 9 pin RS 422 interface making it incompatible with conventional AVID systems unless an external Miranda or Convergent box is used as well.
Firewire Edit
Firewire has long been used for SD DV production. The merits of using firewire are clear. HDV or equivalent camcorders compress footage by 30 times so that it takes only a small amount of space for storage. The HD firewire cable then transfers the actual data off the tape without expanding it, to keep the data throughput small. The edit system actually works with the same data as is recorded onto tape, so that you are actually editing cloned camera footage. When you record the images back to tape again, the same data is cloned once more.
Without the additional compression/expansion and other digital circuitry to colour and distort the images, the camera rushes are theoretically unchanged (digital clone) by the process and since the data rates used are quite low, computer hardware and storage can also be a fairly basic and low cost - this makes the whole process very cheap!
Until now, proper firewire editing has not been achievable with HD footage. However you can now use HD firewire and complete your entire digital HD post-production using either HDV, DVCPro 100 (using either Varicam or the HDX-900 DVCPro HD Camcorder) and of course P2, XDCAM EX and XDCAM HD - be wary though of compatibility issues with your software package and software version, since obselete software versions are rarely compatible with future hardware releases. The advice here is to check in advance what combination of software and hardware you will be using and test it prior to shooting your first scene!
HDCAM can only work with firewire for off-line purposes and it is fully compatible with both AVID Xpress pro and FCP as well as non-firewire workstations.
Varicam does have full firewire compatibility for on-line when using Final Cut Pro and the software includes a comprehensive frame-rate converter plug-in for rendering slow motion ramping effects from variable speed varicam footage. It is however imperative to ensure that you are aware of the careful settings when using this system. It is important to consider that Varicam is a native 1280x720P format - this means that the firewire post-production stage also edits in 1280x720P. A completed programme can only be transferred back to DVCPro HD in 1280x720P mode (60P) and the VTR will take care of the flagged frames.
If you choose to master onto a 1080 line format such as HD-D5 or HDCAM in 25P, then it will be necessary to uprez the images from 720P to 1080P. This is normally done by fitting an HD-SDI card into the Final Cut Pro system (such as the Black magic card) which will automatically perform the uprez in real-time.
Another caveat is that you should be very wary if you are planning to edit with a firewire-based edit application if you have multiple formats to edit from. The only solution available to you may be either to dub all of your multi-format rushes onto one common HDV format, which may compromise your quality or use an uncompressed HD-SDI workstation such as AVID Nitris to edit from. Both stages are likely to add unforeseen cost to the post process.
P2 Post production
If you shoot on P2 series camcorders in the field then you will be shooting onto media files, then this is how it works:
- You shoot onto memory cards on location - currently 16GB cards are the highest storage duration available which give 16mins of HD at 25p (or 38mins or HD in 25pn) or 64 mins of SD in 25p
- At lunchtime, with 4 x cards worth of material filled up, you use a box called a G10 P2 reader archive unit, to copy all media onto standard cheap removable firewire drives. This unit has a built-in screen, so that transferred media can be checked for peace of mind. The P2 cards are then wiped ready for reuse
- The firewire drive (1 TB can store more than 40 hours of HD material in 25Pn mode) can then be plugged directly into your post-production unit, like Final Cut Pro or AVID Media Composer Express, and then the images edited directly in the native HD format without the need for digitising or low quality off-line editing
- On completion of the programme, the drive complete with edited timeline, graphics etc, can be given to the post-production
38 mins of HD when shooting 25pn
The advantages of P2 over HDCAM are that all editing, including off-line are at the same HD quality, post-production machinery is cheap (all based around using HD firewire with Final Cut Pro and AVID Xpress) and the net result is that post-production ought to be cheaper than using conventional post production. These memory cards are currently quite expensive but the workflow is excellent - the success of the format will depend on how well producers adopt its novel post-production workflow but a strategic acceptance by the BBC that DVCPro HD using non-linear acquisition is its preferred choice should ensure the continued success of the format.
One further comment here about drives - it may seem that the best thing to so is to use massive drives on location so that you have limited numbers of drives to work with but this is not necessarily the case. Firstly, only drives 250GB or less can be powered from the G10 unit via USB2, so it makes sense to use these and also, the latest Mac OS only allows certain quantities of drive numbers, so increasing the quantity of P2 cards used uses up this capacity quite quickly - we have found that the combination of using 16GB P2 cards and 250GB ruggedised USB-2 hard drives are the best combination when using the G10 archive unit.
HD Post-production Workflow
HD On-line is effectively the same process as SD On-line, only the equipment is more expensive and the edit takes a bit longer. With HD images being 5 times the size of SD images, any computer graphics (CG) rendering takes a commensurately longer time to complete (be wary if you are rendering HD graphics). Other than that, all HD footage which enters an uncompressed HD post-production route is uprezzed to a common size of 1920x1080 at the same frame rate as it was acquired and here it can be edited with multiple formats with ease. This is not the case when using a firewire based edit solution, so be wary. Any SD footage is uprezzed to 1920x1080 and treated as simply another HD format, so that it can also be edited in HD without issue.
The grade
The grade is very important to an HD production. Whilst it is possible to do a basic grade using video editing tools such as AVID Nitris, dedicated grading hardware will always give better results which can be achieved in less time. Since the grade will give you the end 'gloss' of the production, it is essential that the programme is graded properly. If in doubt, perform a test grade after the test shoot to assess whether the edit/grading tools can do the job effectively.
INTERNATIONAL DELIVERABLES
Until now, I have steered away from using techni-speak, but here it is required, so prepare yourself!
Since HD is a worldwide format, it is inevitable that world distribution will require re-versioning in multiple formats and also multiple frame rates. As a rule, there are 3 popular acquisition frame rates worldwide. 24P for exclusively film production (not relevant for international distribution so won't be mentioned again), 25P for European production and international delivery and 23.98P for US production and international delivery. There is a 4% difference in speed between the 23.98 and 25P frame rates. As already mentioned, shooting at 29.97P or 59.94i makes the conversion to 25P very difficult, so avoid this acquisition rate if you can.
Both 23.98P and 25P acquired programmes can be reversioned into all world frame rates and standards for delivery - however, changing the frame rate may change the programme speed by a small amount which affects both pictures and audio depending on the changed acquisition and delivery speeds.
Since 4% is such a small amount to change the playback speed, that image movement is not perceptibly changed by changing the playback speed by 4% - after all, we do not noticeably detect that feature films are sped up 4% in their conversion from 24 fps to our native 25 fps for DVD distribution and PAL broadcast. The audio is however changed by roughly a semitone by speeding up or slowing down by 4%. This is correctable by laying-off the audio into a hard disk recorder and digitally pitch-correcting the audio and then laying back to tape. This inexpensive procedure is often not done since the pitch change is common throughout the entire programme and the effect is quite small anyway.
The good news is that many VTRs can automatically give reversioned outputs to all frame rates such as 23.98P, 29.97P, 59.94i etc automatically simply by selecting the desired frame rate in the menu. By changing the frame rate, the speed and duration of the programme may change by approx. 4% or not at all, depending on the acquisition speed and the delivery speed. These reversioning options and change in speed is possible for both 25P and 23.98P acquisitioned material of all formats.
As a guide, the US requires HD delivered in either 59.94i (29.97P) or 23.98P. In order to create programmes at these frame rates, there are two main options available:
- Shoot in 25P, post at 25P, audio post at 25P and mastering at 25P.
For international deliveries, the programme can be reversioned to either 29.97P or 23.98P and the speed reduce by around 4% entailing an increase in duration of 4%. Optional digital pitch correction is possible at this stage. - Shoot in 23.98P, post at 23.98P, audio post at 23.98P or 29.97P and mastering at 23.98P.
For international deliveries, the programme can be reversioned to either 29.97P or 23.98P and the speed and duration of the programme will not change at all. There is no audio pitch correction needed here as the audio pitch does not change in the conversion from 23.98P to 29.97P. However, for delivery to 25P for Europe and the UK, it will be necessary to speed up the programme by 4% which will reduce the duration by 4% and optional digital pitch correction is possible after this stage.
The conclusion to this is that for international delivery, you can choose to shoot in either 23.98P or 25P. However, 25P acquisition and post is easier and cheaper than 23.98P which requires special post-production tools and requires the making of off-line copies, not required for 25P production. Should you require to reversion your finished programme with frame rate changes that don't result in duration changes, these are very bad news. They are likely to be rejected by any discerning client because the motion will look awful although the audio pitch won't change. Better to endure some minor speed and duration change to your programme that does not involve introducing movement artefacts.
Conclusion
There are many routes to HD production and many things to consider such as deliverables, post-production, frame rate and workflow.
Begin with the deliverables requirement, as this will influence your post-production workflow. Having selected your workflow, this will lead you to a range of equipment options to consider for acquisition - and these will be heavily influenced by the 'look' of the programme and DoP choice. Always keep your production format and shooting speed constant and ensure that you are clear about your post-production route before you begin shooting, consulting experts if necessary.
If you follow these guidelines, it may help to avoid costly mistakes post-production and ensure that you don't suffer any expensive surprises!
Barry Bassett, Managing Director, VMI