VirtualDub Plugin SDK 1.1

The VirtualDub Plugin SDK

This SDK describes how to make plugins for VirtualDub 1.4.12 or higher. Features implemented through VirtualDub 1.9.1 are covered.

Copyright (C) 2007-2009 Avery Lee.


Legal information

VirtualDub Plugin SDK
Copyright (C) 2007-2008 Avery Lee

This SDK is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This SDK is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this SDK; if not, write to the Free Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.

Note: Some of the source code and header files in the VirtualDub Plugin SDK are licensed under a "zlib-style" license which is more liberal than the GPL. Check the copyright notice in the individual files for details.

            GNU GENERAL PUBLIC LICENSE
               Version 2, June 1991

 Copyright (C) 1989, 1991 Free Software Foundation, Inc.
                          675 Mass Ave, Cambridge, MA 02139, USA
 Everyone is permitted to copy and distribute verbatim copies
 of this license document, but changing it is not allowed.

                Preamble

  The licenses for most software are designed to take away your
freedom to share and change it.  By contrast, the GNU General Public
License is intended to guarantee your freedom to share and change free
software--to make sure the software is free for all its users.  This
General Public License applies to most of the Free Software
Foundation's software and to any other program whose authors commit to
using it.  (Some other Free Software Foundation software is covered by
the GNU Library General Public License instead.)  You can apply it to
your programs, too.

  When we speak of free software, we are referring to freedom, not
price.  Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
this service if you wish), that you receive source code or can get it
if you want it, that you can change the software or use pieces of it
in new free programs; and that you know you can do these things.

  To protect your rights, we need to make restrictions that forbid
anyone to deny you these rights or to ask you to surrender the rights.
These restrictions translate to certain responsibilities for you if you
distribute copies of the software, or if you modify it.

  For example, if you distribute copies of such a program, whether
gratis or for a fee, you must give the recipients all the rights that
you have.  You must make sure that they, too, receive or can get the
source code.  And you must show them these terms so they know their
rights.

  We protect your rights with two steps: (1) copyright the software, and
(2) offer you this license which gives you legal permission to copy,
distribute and/or modify the software.

  Also, for each author's protection and ours, we want to make certain
that everyone understands that there is no warranty for this free
software.  If the software is modified by someone else and passed on, we
want its recipients to know that what they have is not the original, so
that any problems introduced by others will not reflect on the original
authors' reputations.

  Finally, any free program is threatened constantly by software
patents.  We wish to avoid the danger that redistributors of a free
program will individually obtain patent licenses, in effect making the
program proprietary.  To prevent this, we have made it clear that any
patent must be licensed for everyone's free use or not licensed at all.

  The precise terms and conditions for copying, distribution and
modification follow.

            GNU GENERAL PUBLIC LICENSE
   TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION

  0. This License applies to any program or other work which contains
a notice placed by the copyright holder saying it may be distributed
under the terms of this General Public License.  The "Program", below,
refers to any such program or work, and a "work based on the Program"
means either the Program or any derivative work under copyright law:
that is to say, a work containing the Program or a portion of it,
either verbatim or with modifications and/or translated into another
language.  (Hereinafter, translation is included without limitation in
the term "modification".)  Each licensee is addressed as "you".

Activities other than copying, distribution and modification are not
covered by this License; they are outside its scope.  The act of
running the Program is not restricted, and the output from the Program
is covered only if its contents constitute a work based on the
Program (independent of having been made by running the Program).
Whether that is true depends on what the Program does.

  1. You may copy and distribute verbatim copies of the Program's
source code as you receive it, in any medium, provided that you
conspicuously and appropriately publish on each copy an appropriate
copyright notice and disclaimer of warranty; keep intact all the
notices that refer to this License and to the absence of any warranty;
and give any other recipients of the Program a copy of this License
along with the Program.

You may charge a fee for the physical act of transferring a copy, and
you may at your option offer warranty protection in exchange for a fee.

  2. You may modify your copy or copies of the Program or any portion
of it, thus forming a work based on the Program, and copy and
distribute such modifications or work under the terms of Section 1
above, provided that you also meet all of these conditions:

    a) You must cause the modified files to carry prominent notices
    stating that you changed the files and the date of any change.

    b) You must cause any work that you distribute or publish, that in
    whole or in part contains or is derived from the Program or any
    part thereof, to be licensed as a whole at no charge to all third
    parties under the terms of this License.

    c) If the modified program normally reads commands interactively
    when run, you must cause it, when started running for such
    interactive use in the most ordinary way, to print or display an
    announcement including an appropriate copyright notice and a
    notice that there is no warranty (or else, saying that you provide
    a warranty) and that users may redistribute the program under
    these conditions, and telling the user how to view a copy of this
    License.  (Exception: if the Program itself is interactive but
    does not normally print such an announcement, your work based on
    the Program is not required to print an announcement.)

These requirements apply to the modified work as a whole.  If
identifiable sections of that work are not derived from the Program,
and can be reasonably considered independent and separate works in
themselves, then this License, and its terms, do not apply to those
sections when you distribute them as separate works.  But when you
distribute the same sections as part of a whole which is a work based
on the Program, the distribution of the whole must be on the terms of
this License, whose permissions for other licensees extend to the
entire whole, and thus to each and every part regardless of who wrote it.

Thus, it is not the intent of this section to claim rights or contest
your rights to work written entirely by you; rather, the intent is to
exercise the right to control the distribution of derivative or
collective works based on the Program.

In addition, mere aggregation of another work not based on the Program
with the Program (or with a work based on the Program) on a volume of
a storage or distribution medium does not bring the other work under
the scope of this License.

  3. You may copy and distribute the Program (or a work based on it,
under Section 2) in object code or executable form under the terms of
Sections 1 and 2 above provided that you also do one of the following:

    a) Accompany it with the complete corresponding machine-readable
    source code, which must be distributed under the terms of Sections
    1 and 2 above on a medium customarily used for software interchange; or,

    b) Accompany it with a written offer, valid for at least three
    years, to give any third party, for a charge no more than your
    cost of physically performing source distribution, a complete
    machine-readable copy of the corresponding source code, to be
    distributed under the terms of Sections 1 and 2 above on a medium
    customarily used for software interchange; or,

    c) Accompany it with the information you received as to the offer
    to distribute corresponding source code.  (This alternative is
    allowed only for noncommercial distribution and only if you
    received the program in object code or executable form with such
    an offer, in accord with Subsection b above.)

The source code for a work means the preferred form of the work for
making modifications to it.  For an executable work, complete source
code means all the source code for all modules it contains, plus any
associated interface definition files, plus the scripts used to
control compilation and installation of the executable.  However, as a
special exception, the source code distributed need not include
anything that is normally distributed (in either source or binary
form) with the major components (compiler, kernel, and so on) of the
operating system on which the executable runs, unless that component
itself accompanies the executable.

If distribution of executable or object code is made by offering
access to copy from a designated place, then offering equivalent
access to copy the source code from the same place counts as
distribution of the source code, even though third parties are not
compelled to copy the source along with the object code.

  4. You may not copy, modify, sublicense, or distribute the Program
except as expressly provided under this License.  Any attempt
otherwise to copy, modify, sublicense or distribute the Program is
void, and will automatically terminate your rights under this License.
However, parties who have received copies, or rights, from you under
this License will not have their licenses terminated so long as such
parties remain in full compliance.

  5. You are not required to accept this License, since you have not
signed it.  However, nothing else grants you permission to modify or
distribute the Program or its derivative works.  These actions are
prohibited by law if you do not accept this License.  Therefore, by
modifying or distributing the Program (or any work based on the
Program), you indicate your acceptance of this License to do so, and
all its terms and conditions for copying, distributing or modifying
the Program or works based on it.

  6. Each time you redistribute the Program (or any work based on the
Program), the recipient automatically receives a license from the
original licensor to copy, distribute or modify the Program subject to
these terms and conditions.  You may not impose any further
restrictions on the recipients' exercise of the rights granted herein.
You are not responsible for enforcing compliance by third parties to
this License.

  7. If, as a consequence of a court judgment or allegation of patent
infringement or for any other reason (not limited to patent issues),
conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License.  If you cannot
distribute so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you
may not distribute the Program at all.  For example, if a patent
license would not permit royalty-free redistribution of the Program by
all those who receive copies directly or indirectly through you, then
the only way you could satisfy both it and this License would be to
refrain entirely from distribution of the Program.

If any portion of this section is held invalid or unenforceable under
any particular circumstance, the balance of the section is intended to
apply and the section as a whole is intended to apply in other
circumstances.

It is not the purpose of this section to induce you to infringe any
patents or other property right claims or to contest validity of any
such claims; this section has the sole purpose of protecting the
integrity of the free software distribution system, which is
implemented by public license practices.  Many people have made
generous contributions to the wide range of software distributed
through that system in reliance on consistent application of that
system; it is up to the author/donor to decide if he or she is willing
to distribute software through any other system and a licensee cannot
impose that choice.

This section is intended to make thoroughly clear what is believed to
be a consequence of the rest of this License.

  8. If the distribution and/or use of the Program is restricted in
certain countries either by patents or by copyrighted interfaces, the
original copyright holder who places the Program under this License
may add an explicit geographical distribution limitation excluding
those countries, so that distribution is permitted only in or among
countries not thus excluded.  In such case, this License incorporates
the limitation as if written in the body of this License.

  9. The Free Software Foundation may publish revised and/or new versions
of the General Public License from time to time.  Such new versions will
be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.

Each version is given a distinguishing version number.  If the Program
specifies a version number of this License which applies to it and "any
later version", you have the option of following the terms and conditions
either of that version or of any later version published by the Free
Software Foundation.  If the Program does not specify a version number of
this License, you may choose any version ever published by the Free Software
Foundation.

  10. If you wish to incorporate parts of the Program into other free
programs whose distribution conditions are different, write to the author
to ask for permission.  For software which is copyrighted by the Free
Software Foundation, write to the Free Software Foundation; we sometimes
make exceptions for this.  Our decision will be guided by the two goals
of preserving the free status of all derivatives of our free software and
of promoting the sharing and reuse of software generally.

                NO WARRANTY

  11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW.  EXCEPT WHEN
OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.  THE ENTIRE RISK AS
TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU.  SHOULD THE
PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,
REPAIR OR CORRECTION.

  12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED
TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY
YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
POSSIBILITY OF SUCH DAMAGES.

             END OF TERMS AND CONDITIONS

    Appendix: How to Apply These Terms to Your New Programs

  If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.

  To do so, attach the following notices to the program.  It is safest
to attach them to the start of each source file to most effectively
convey the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.

    <one line to give the program's name and a brief idea of what it does.>
    Copyright (C) 19yy  <name of author>

    This program is free software; you can redistribute it and/or modify
    it under the terms of the GNU General Public License as published by
    the Free Software Foundation; either version 2 of the License, or
    (at your option) any later version.

    This program is distributed in the hope that it will be useful,
    but WITHOUT ANY WARRANTY; without even the implied warranty of
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
    GNU General Public License for more details.

    You should have received a copy of the GNU General Public License
    along with this program; if not, write to the Free Software
    Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.

Also add information on how to contact you by electronic and paper mail.

If the program is interactive, make it output a short notice like this
when it starts in an interactive mode:

    Gnomovision version 69, Copyright (C) 19yy name of author
    Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
    This is free software, and you are welcome to redistribute it
    under certain conditions; type `show c' for details.

The hypothetical commands `show w' and `show c' should show the appropriate
parts of the General Public License.  Of course, the commands you use may
be called something other than `show w' and `show c'; they could even be
mouse-clicks or menu items--whatever suits your program.

You should also get your employer (if you work as a programmer) or your
school, if any, to sign a "copyright disclaimer" for the program, if
necessary.  Here is a sample; alter the names:

  Yoyodyne, Inc., hereby disclaims all copyright interest in the program
  `Gnomovision' (which makes passes at compilers) written by James Hacker.

  <signature of Ty Coon>, 1 April 1989
  Ty Coon, President of Vice

This General Public License does not permit incorporating your program into
proprietary programs.  If your program is a subroutine library, you may
consider it more useful to permit linking proprietary applications with the
library.  If this is what you want to do, use the GNU Library General
Public License instead of this License.


What's new in the Plugin SDK

Periodically new features are added to the VirtualDub filter API. Except where noted, these features are implemented such that their use is optional; old filters continue to work in hosts supporting newer versions, and filters can be made to work with older hosts through version detection.

Version 1.1

  • VDXFrame: Fixed bug in VDXVideoFilter that wrote broken script config lines for filters that didn't export any via GetScriptString().
  • Video filters: Now covers API V14 (VirtualDub 1.9.1).

Version 1.0

  • Initial release.


Setting up your development environment

Requirements

The VirtualDub Plugin SDK primarily supports the following compilers:

  • Microsoft Visual C++ 6.0 SP5 + Processor Pack
  • Microsoft Visual Studio 2005 SP1

Project files are not provided for Visual C++ .NET (2002), Visual C++ .NET (2003), or Visual Studio 2008, but the source code and include files should be compatible.

It is possible to create a plugin with other compilers, such as GCC, but doing so requires some modifications to the header files and isn't supported. In particular, the IVDXFilterPreview interfaces uses the Visual C++ __thiscall calling convention, which may require workarounds. Video filters have also been written in the past in other languages, such as Borland's Delphi, but doing so is definitely out of scope for this SDK.

Setting up the project

Create a standard Win32 dynamically linked library (DLL) project. In Visual Studio 2005, this is a suboption inside the Win32 Project template. Do not create a .NET (CLR) based project.

Add the include folder from this SDK to your project's preprocessor include paths. This configuration option exists under C/C++, General in Visual Studio 2005 and C/C++, Preprocess in Visual C++ 6.0.

If you are using Visual Studio .NET (2002) or later, under Code Generation, change the C Runtime Library (CRT) used by your DLL to Debug Multithreaded for a Debug build and Multithreaded for a Release build. This avoids a dependency on the CRT DLLs, which you would otherwise have to redistribute with your application. The CRT DLLs are especially annoying to redistribute in Visual Studio 2005.

If you are using the VDXFrame helper library, you should insert it into your workspace/solution and add it as a dependency to your project. The Plugin SDK does not come with .libs, so you will need to compile the library.

For input driver plugin DLLs, you will probably need to add a module definition (.def) file to your project to get the DLL entry point exported properly. See SampleInputDriver.def from the sample input driver; you can probably copy that file verbatim, and then select it under Linker, Input, Module Definition File in project settings. In Visual C++ 6.0, you will need to add the /def: flag manually to the linker command line.

Under link options, you should change the extension of the output DLL from .dll to .vdf for a video filter DLL or .vdplugin for an input driver DLL. You can also use .vdplugin for video filters if you are targeting recent versions of VirtualDub only.

Debugging your plugin

You can test and debug your plugin easily by having your debugger launch VirtualDub with the plugin automatically loaded. To do so, configure your debugger to use the following command:

VirtualDub /F path-to-filter

In Microsoft Visual Studio 2005, you can do this under project settings, Configuration Properties, Debugging. Set Command to point to VirtualDub.exe, and Command Arguments to /F "$(TargetPath)".

Once this has been set up, you should be able to set breakpoints and trace execution within your video filter under the debugger. You may also want to save a configuration file with your filter already inserted into the video filter chain so that you don't have to do so every time; you can do so by saving a configuration file in VirtualDub and executing it via the /s command line switch.

Note: Recent versions of VirtualDub automatically unload plugin DLLs when they are not in use, so you may have trouble with breakpoints initially not working in your debugger. You can lock your DLL in memory by adding filters to the filter chain or opening files with the input driver, both of which will keep the necessary plugin DLL loaded.


Naming conventions

By convention, most symbols in VirtualDub contain the prefix VD in order to avoid conflicts with other symbols. To distinguish between VirtualDub internal symbols and symbols meant for external usage, symbols in the plugin API and in the Plugin SDK headers are prefixed with VDX instead to denote them for external usage. In some cases structures are declared identically between VirtualDub itself and the Plugin SDK headers, but the SDK versions with the VDX prefix are the ones intended for use by plugins.

Interfaces, represented in C++ by classes with only pure virtual methods, use an additional I prefix, and therefore include the tag IVDX.

Binary API conventions

VirtualDub itself is compiled with the Microsoft Visual C++ Compiler, versions 12.00 (VC6) to 14.00 (VC8). The calling convention used by the plugin API are thus based on the binary ABI used by the VC++ compiler and the Microsoft Windows platform. However, the SDK interfaces are designed to be amenable to other compilers when possible.

By default, calls in the 32-bit plugin API use the stdcall calling convention whenever possible, except for calls taking variable arguments, which use cdecl. However, the video filter interface predates this Plugin SDK and therefore uses slightly different calling conventions. In particular, most function calls use cdecl and method calls on interfaces use thiscall. The thiscall convention in particular may cause some issues if you are attempting to use a compiler other than Visual C++, because it specifies a caller-pops convention with the this pointer in ECX. If this is the case, you may have luck modifying the headers to use fastcall instead with an extra dummy first int parameter.


Plugin initialization

All plugin DLLs must export the following single function:

const VDXPluginInfo *const *VDXAPIENTRY VDGetPluginInfo();

This must be exported without any decoration (name mangling) and must use the VDXAPIENTRY calling convention, which maps to __stdcall. As the Microsoft Visual C++ compiler is not generally disposed to naming the function appropriately, a module definition (.def) file should be used to export the function correctly. The Microsoft DUMPBIN utility can be used with the /exports flag to verify the exported names.

VDGetPluginInfo() returns a pointer array where each entry points to a VDXPluginInfo structure for each driver or filter in the module. Multiple drivers or filters can thus be included in a single DLL. The array is terminated with a NULL pointer.

Note: Plugin DLLs that only contain video filters do not have to use this mechanism, because the API for video filters predates this mechanism. See Creating a video filter" for details.

API versioning

Each plugin API has a version number associated with it, which is increased as new features are added or breaking changes are made. This version is exposed to the plugin and allows newer API features to be used without breaking compatibility with older hosts.

Plugins implementing or using those APIs also expose a pair of version numbers, the compatible version and the latest version. These are used by the host to check for plugin compatibility. The compatible version is the lowest version supported by the plugin, and causes the plugin to be rejected if it is newer than the host's highest supported version. The latest version indicates the largest feature set used by the plugin and can be used by the host to enable compatibility code or reject plugins that are too old. Generally, the compatibility version is hardcoded, whereas the latest version is set to the API version constant in the SDK headers and reflects the headers the plugin was built against.

The plugin does not have to check for host API versions that are too old or too new; this is the responsibility of the host. It only has to check API versions if it can use features that are only supported in some API versions within its [compatible, latest] version bracket. Plugins should always use a >= test instead of an == test when checking versions, as the reported API version may be newer than the one the plugin was built against.

Auto-loading on startup

The 32-bit version of VirtualDub searches the plugins32 directory under its program location for files matching the pattern *.vdplugin; the 64-bit version searches plugins64 instead. Any plugin DLLs found are automatically loaded on startup.

Platform compatibility

Due to limitations in Windows, the 32-bit version of VirtualDub can only load 32-bit plugins, and the 64-bit X64 version of VirtualDub can only load 64-bit X64 plugins. However, it is possible to build both from shared source code if a plugin is written portably.

Video filter API compatibility

Current versions of VirtualDub will still search the plugins directory and load DLLs using the old .vdf extension. It is recommended that plugin vendors migrate to the plugins32 and plugins64 directories for new plugins in order to avoid errors between the 32-bit and 64-bit versions of VirtualDub, which both try to load from the shared plugins directory.

A plugin DLL can expose both old filters and new filters/drivers by exporting both VDGetPluginInfo() and VirtualdubFilterModuleInit2 functions.


Dynamic loading

The plugins definitions in the SDK are designed to allow dynamic loading, where a host loads plugins only when needed. This reduces memory footprint and resource usage, and also sidesteps a problem with thread-local storage (TLS) handles on older operating systems — these are in short supply per thread, and can be exhausted without dynamic loading as one is consumed for every instance of the Microsoft C runtime library (CRT) loaded in memory.

Dynamic loading is mostly invisible to plugins, since the host automatically shadows important data from the plugin and loads the plugin only when necessary. However, this does mean that the plugin should load and unload quickly. Also, if the plugin needs to stay in memory longer, such as to wait for shutdown or for the user to close status UI, the plugin can keep itself in memory by incrementing its DLL reference count, such as by the Win32 LoadLibrary() or GetModuleHandleEx() functions; the plugin won't unload when the host disconnects, and when it reconnects, it will use the DLL that is still resident.


Reference counting

Objects in the VirtualDub plugin API are reference counted in order to support shared ownership. Instead of being explicitly destroyed, each object tracks the number of references on it and automatically destroys itself when no references are left.

For those of you familiar with Microsoft COM, the reference counting scheme used here is very similar to that of the COM IUnknown interface.

Creating and destroying references

References to an object are not stored, but are instead virtually counted via the AddRef() and Release() methods. AddRef() adds a reference, whereas Release() removes a reference. For every call to AddRef(), there must be a corresponding call to Release().

Factory methods that return a new object generally have the following prototype:

bool CreateObject(Type **ppObject);

These return true and place a pointer to the new object in *ppObject on success. The new object is returned with one implicit reference that must be removed using Release().

Implementing AddRef and Release

AddRef() and Release() must be implemented in a thread-safe manner, as they may be called on any thread at any time. In Windows, this can be done using the InterlockedIncrement() and InterlockedDecrement() functions, which atomically increment or decrement a 32-bit LONG variable.

Use of the vdxunknown<> template, which is included in the SDK, is highly recommended. When used as a base class, it automatically implements AddRef() and Release() correctly on a specified interface.

When implementing multiple interfaces, all interfaces must share the same reference count. In particular, do not inherit from two base classes that each implement a reference counter without ensuring that only one is used. Inheriting two reference counters is inadvisable for other reasons, but if you must do so, ensure that AddRef() and Release() are consistent across all interfaces supported by an object.

Internal reference counts

It is permissible for objects to internally obtain references on each other. For instance, if your video decoder model uses shared data, you may want to make the shared data reference counted and obtain a reference on it to prevent it from disappearing before the video decoder model. The only caveat to doing this is that a reference loop must never be created, because it will cause objects to leak. For instance, two objects A and B must not keep a permanent reference on each other, as both objects would then have an outstanding reference count of one with no external references.


Using CPU instruction set extensions

Vector instruction sets can greatly accelerate plugin operations, particularly video decoding, and it is worthwhile to use them whenever possible. Unfortunately, all vector instructions are part of instruction set extensions in the IA-32 instruction set architecture (ISA), and thus are not always available, depending on the CPU. Attempting to use an instruction set that is not available results in a crash and is very sloppy, so it is required that plugins check for CPU instruction set extensions before using them.

Checking for instruction set extensions

The 32-bit plugin API only guarantees an 80486 CPU, so the only optional instructions that are guaranteed are XADD and CMPXCHG; none of the vector instruction sets, such as MMX or SSE, are guaranteed. Therefore, before using any of those instruction sets, the plugin should call the GetCPUFeatureFlags() method first, and fall back gracefully or fail if instruction set support is missing.

On a 64-bit (X64) system, the minimum feature set supported by the ISA is considerably higher. Plugins can assume that the MMX, SSE, and SSE2 instruction set extensions are always available. Support for 3DNow! cannot be assumed, as it is not available on all X64-capable processors.

You don't have to use the CPU detection offered by the host — you can use your own detection instead. This can be handy if your compiler offers code generation that can take advantage of instruction set extensions. However, this does mean that users cannot easily override the detection through the host's user interface if something goes wrong with the CPU detection.

Using instruction set extensions

As some instruction set extensions conflict, care must be used when using such extensions. In particular, any plugins that use MMX must always ensure that the MMX state is cleared with an EMMS or FEMMS instruction prior to returning to the operating system or to the host in order to avoid corrupting floating point (x87) calculations. It is guaranteed that the host will do the same before calling into plugins. This is required by the cdecl and stdcall calling conventions.

While this is not required or guaranteed by the API, VirtualDub guards all control transfers in and out of plugins against MMX state mistakes. It ensures that bad MMX state never leaks in or out of a plugin, and will display warnings if it has detected a plugin leaking MMX state.

Overriding VirtualDub's CPU detection

Invoking VirtualDub with the /safecpu command line switch causes it to disable use of all enhanced instruction sets, and to report none available to all plugins. CPU feature detection can also be disabled or overridden in Options, Preferences, CPU.


About video filters

Video filter plugins allow custom image processing algorithms to be integrated into VirtualDub's video processing pipeline.

Processing model

The video filter pipeline consists of a linear chain of filter instances, where a video frame enters in the start of the chain and emerges out the back:

start deinterlace resize blur output

Each video filter instance receives a source frame and a destination buffer and is responsible for processing the frame into the destination buffer. The source and destination sizes can be different, allowing a video filter to shrink or enlarge video as it passes through.

Filter instances do not interact with each other; all interaction is through the host, which moves frames between instances as required.

Sequence transformations

Video filters can also add or remove video frames by controlling the timing of video frames. By default, the output video stream corresponds 1:1 with the input stream, such that frame 547 in the output is a processed version of frame 547 in the input. A video filter can choose to change this mapping, however, allowing frames to be reordered, duplicated, or dropped prior to reaching the filter. It is also possible to change the length and frame rate of the video stream.


What's new in the video filter API

Periodically new features are added to the VirtualDub filter API. Except where noted, these features are implemented such that their use is optional; old filters continue to work in hosts supporting newer versions, and filters can be made to work with older hosts through version detection.

API version 14 (VirtualDub 1.9.1)

  • All filter frames are now tagged with their frame number and starting/ending timestamps.
  • paramProc can now specify FILTERPARAM_ALIGN_SCANLINES to request 16-byte alignment for source and output buffer scanlines.
  • A filter can indicate when its output depends only on its input and configuration parameters, and not frame numbers, by means of the FILTERPARAM_PURE_TRANSFORM flag.
  • copyProc2 has been added to fix an annoyance with the activation structure pointer.
  • Filters may reference multiple source frames via the prefetchProc2 method.
  • Smart rendering is now supported directly on filters, by allowing the filter itself to note when a direct copy is possible through a direct-mode prefetch.
  • Pixel aspect ratio is now specified on filter buffers when available, and filters can alter the PAR as appropriate.
  • A method called eventProc has been added to notify the filter of out of band events, such as upstream parameter changes that require cache invalidation.

API version 13 (VirtualDub 1.8.2)

  • FilterStateInfo now includes the output frame as well as the destination frame.

API version 12 (VirtualDub 1.8.0)

  • Support for processing YCbCr encoded video.
  • Filters can modify the frame timing and length of a video stream.

API version 11 (VirtualDub 1.7.0)

  • The src bitmap structure is guaranteed to be set up prior to configProc and can be used for UI purposes when video is present.
  • Added IVDXFilterPreview2 interface to allow filters to determine if the preview window is still open or not.

API version 10 (VirtualDub 1.5.10)

  • The FilterStateInfo structure now has a flags field indicating whether the filter chain is being initialized for preview, processing, or real-time (capture) mode.

API version 9 (VirtualDub 1.4.12)

  • Added copyProc to support full C++ object semantics for filter data.

API version 8 (VirtualDub 1.4.11)

  • Added stringProc2 as a bounds-checked version of stringProc.


Programming model

A video filter plugin exposes one or more video filters, which appear in VirtualDub's filter list. Each time the filter is added to the list, a filter instance is created, which corresponds to a stage in the pipeline where the filter processes frames. Multiple instances of the same filter may be active within the same chain.

Filter code is called with pointers to VDXFilterActivation and VDXFilterFunctions structures. The VDXFilterActivation structure contains information about the incoming video stream, input and output bitmap structures, and data assocated with the filter instance. VDXFilterFunctions points to a series of callbacks the filter can use to issue requests to the host.

Instance data

Each instance also has a memory block associated with it, pointed to by the filter_data field in the VDXFilterActivation structure, which holds per-instance data. VirtualDub allocates this memory according to the inst_data_size field in the filter definition, but otherwise, the contents are completely up to the filter.

When a filter instance is created, the initProc routine is called, if present, to initialize the instance data associated with the filter instance. A corresponding deinitProc routine may be included to clean up instance data before the instance is destroyed, and copyProc can be specified to handle cloning of it. These can be implemented to call the C++ constructor, destructor, and copy constructor of a class in order to treat the instance data as a C++ object with methods.

Video filter operation

The paramProc function of a filter is called to determine the basic processing parameters of a filter, most notably the output it produces for a given input. This is done repeatedly as the filter chain is built; VirtualDub's video filter chain dialog, for instance, uses paramProc to determine frame sizes to display in the dialog. It's worth nothing that paramProc cannot fail — even if the configuration parameters are invalid or the input frame size cannot be supported, paramProc must return a reasonable default.

Once the basic operating parameters of all instances in the filter chain have been determined, the host can then prepare all filter instances for processing video frames by calling the startProc function of each instance in turn. This is where filters perform heavy initialization that isn't otherwise necessary, such as allocating working buffers and reading auxiliary data files. Unlike paramProc, startProc can and should fail with an exception if configuration parameters are invalid or the filter instance otherwise cannot start.

During main filter chain operation, runProc is called repeatedly on each filter instance to process each video frame in turn. Once all video frames have been processed, endProc is called to notify filter instances of the end of the operation and allow each to clean up resources that were allocated in startProc.

Threading

An instance of a video filter is generally active on only one thread at a time. However, different instances of the same video filter may run in parallel. The code for a video filter must therefore be re-entrant, and if any data is shared between instances, access to the shared data must be done in a thread-safe manner.


Handling bitmaps

Images are passed in and out of video filters through the VFBitmap structure. Each VFBitmap represents an image as follows:

The image is stored as a series of sequential scanlines, where each scanline consists of a series of pixels. Individual fields in VFBitmap describe a bitmap as follows:

data
Points to the beginning of the first scanline, which is at the bottom-left corner of the image.
palette
Not used for video filters.
depth
Indicates the bit depth of the image. For video filters, this is typically 32, indicating 24-bit RGB with an unused 8-bit alpha channel in the high byte (fourth byte in memory order). If this is zero, it means the bitmap uses an alternate format and the mpPixmap field must be used.
w
Width of the image, in pixels. Always positive.
h
Height of the image, in pixels. Always positive.
pitch
Distance from the beginning of one scanline to the beginning of the next, in bytes.
V12+ only: Bitmaps can be stored top-down as well as bottom-up. The pitch value value is positive if the image is stored bottom-up in memory and negative if the image is stored top-down. This is only permitted if the filter declares support for flexible formats by returning FILTERPARAM_SUPPORTS_ALTFORMATS from paramProc; otherwise, the host ensures that the filter receives a bottom-up orientation with a positive pitch, flipping the bitmap beforehand if necessary.
modulo
Distance from the end of one scanline to the beginning of the next, in bytes. This value is positive or zero if the image is stored bottom-up in memory and negative if the image is stored top-down. A value of zero indicates that the image is stored bottom-up with no padding between scanlines. For a 32-bit bitmap, modulo is equal to pitch - 4*w.
size
Size of the bitmap, in bytes. This is the amount of memory required to hold all of the scanlines.
offset
Offset from the start of the buffer to the beginning of the bitmap, in bytes.
V12+ only: For image formats that use multiple planes, this points to the base of plane 0.
mpPixmapLayout
V12+ only: More flexible description of bitmap layout, which is able to accommodate multi-plane and YCbCr formats. In paramProc, this field is always valid when provided by the host. When set by the filter in paramProc, this field is activated when the depth field of the bitmap is set to zero.
mpPixmap
V12+ only: More flexible description of bitmap, which is able to accommodate multi-plane and YCbCr formats. This field is always valid in runProc, even if mpPixmapLayout was not used in paramProc. For filters that require V12+ of the API, mpPixmap may be used exclusively in runProc in lieu of the regular data, pitch, etc. fields above.

Alignment and padding

Since the video filter system works with 32-bit bitmaps, scanlines are guaranteed to be aligned to 32-bit boundaries. No further alignment is guaranteed. In particular, filter code must not assume that scanlines are 16-byte aligned for SSE code.

It is important to note that there may be padding between scanlines. The pitch field indicates the true spacing in bytes, and should be used to step from one scanline to the next:

p = (uint32 *)((char *)p + pitch);

Padding must not be overwritten. In particular, this is the wrong way to copy from one bitmap to the next:

// DO NOT DO THIS!
memcpy(dst.data, src.data, src.pitch * src.h);

You should instead copy the scanlines individually:

char *dstp = (char *)dst.data;
const char *srcp = (const char *)src.data;
for(int y=0; y< ++y) {
    memcpy(dstp, srcp, w*4);
    dstp += dst.pitch;
    srcp += src.pitch;
}

16 byte alignment via FILTERPARAM_ALIGN_SCANLINES (V14+ only)

If the paramProc() method specifies the FILTERPARAM_ALIGN_SCANLINES flag, then additional constraints are placed on the host and some constraints on the filter are relaxed. Specifically:

  • All scanlines are guaranteed to be aligned to a multiple of 16. This means that all pitch and data fields are also guaranteed to be a multiple of 16.
  • Source scanlines can safely be read up to the next 16 byte boundary even if the last 16 byte word runs over the end of the scanline. The contents of the additional bytes are undefined.
  • Output scanlines can safely be written up to the next 16 byte boundary even if the last 16 byte word runs over the end of the scanline. The contents of the additional bytes are ignored.

The result is that filters which make extensive use of SSE/2/3/4 routines can run faster with fewer alignment issues.

VirtualDub specific: Use of the alignment flag may force a copy to realign the buffer if the current buffer is not aligned suitably.


Creating a video filter

The video filter API is designed to allow filters to be partially implemented, reducing the amount of boilerplate code that must accompany a filter.

Initializing the filter module

A filter DLL must export two entry points: VirtualdubFilterModuleInit2 and VirtualdubFilterModuleDeinit. These methods register and deregister the filters within a DLL, respectively. Here is an example of how these filters can be implemented:

extern FilterDefinition g_myFilterDef;
FilterDefinition *g_registeredFilterDef;

extern "C" __declspec(dllexport) int __cdecl VirtualdubFilterModuleInit2(struct VDXFilterModule *fm, const VDXFilterFunctions *ff, int& vdfd_ver, int& vdfd_compat) {
    g_registeredFilterDef = ff->addFilter(fm, &g_myFilterDef, sizeof(FilterDefinition));

    vdfd_ver        = VIRTUALDUB_FILTERDEF_VERSION;
    vdfd_compat     = 8;    // we need this version for copy constructor support

    return 0;
}

extern "C" __declspec(dllexport) void __cdecl VirtualdubFilterModuleDeinit(struct VDXFilterModule *fm, const VDXFilterFunctions *ff) {
    // Note: This must be the pointer returned from addFilter(), NOT your original
    // declaration.
    ff->removeFilter(g_registeredFilterDef);
}

Note that these functions must be exported from the DLL with no name mangling or decoration on the names. This is the reason that the functions are exported as extern "C". If you are using the Microsoft Visual C++ compiler, you can use the DUMPBIN tool with the /LINK flag to check the names. There should be no underscores or at-signs (@) present.

The vdfd_ver and vdfd_compat parameters are particularly important. On entry, these correspond to the highest and lowest plugin API versions supported by the host. These are set by the filter module init routine to the highest and lowest plugin API version supported by the filters in the module. The host can then choose to reject the filter module if it is not compatible, or adjust its interface logic as necessary to suit the API versions the filter expects. Similarly, as necessary, the filter should check the incoming vdfd_ver value and adjust its use of the API accordingly. On exit, vdfd_ver should always be set to the VIRTUALDUB_FILTERDEF_VERSION constant used to compile the filter; vdfd_compat should be set to the lowest API version that the filters can deal with.

Defining the video filter

Next, a FilterDefinition structure must then be created for every filter in the DLL. This structure consists of a few strings describing the filter and a series of callbacks indicating which routines the filter supports. Most of the callbacks can be set to NULL in order to invoke default behavior; in fact, the only callback that is required is runProc. Therefore, this is a valid definition:

static struct FilterDefinition myfilter_definition={
    0,0,NULL,                       // next, prev, and module (set to zero)
    "my filter",                    // name
    "Description of my filter.",    // description
    "Author of my filter.",         // author / maker
    NULL,                           // no private data
    0,                              // no instance data size
    NULL,                           // no initProc
    NULL,                           // no deinitProc
    myfilter_run,                   // runProc
    // allow the compiler to zero-initialize the rest of the fields
};

(Important note: The structure must not be declared const, as older versions of VirtualDub will write to the structure.)


Setting filter parameters

By implementing the paramProc method, a filter can inform the host of its operating parameters, including the relationship between the input and output frame sizes, frame delays, and buffer overlap behavior.

A basic paramProc implementation

The simplest paramProc just returns a code:

long paramProc(VDXFilterActivation *fa, const VDXFilterFunctions *ff) {
    return FILTERPARAM_SWAP_BUFFERS;
}

The return value is a bitfield indicating the behavior of the filter. paramProc can specify a lot of parameters, but all of them have defaults so that most filters only have to do minimal work here. In this case, the filter is specifying that its source and outputs have the same frame size, frame rate, length, aspect ratio, and format, and that it copies from one buffer to another. This configuration will work for most filters.

In-place vs. copy operation

The first option that filters can choose to alter is the FILTERPARAM_SWAP_BUFFERS bit on the return value. This indicates whether the filter works in-place on a video frame, or whether it copies from one frame to another. Whichever one is better depends on the image processing that your filter does in runProc. For some algorithms, it is most convenient to read pixels from a buffer, modify them, and write them back to the same buffer. In that case, the filter should operate in-place and return with the FILTERPARAM_SWAP_BUFFERS bit cleared. Other algorithms that read overlapping areas of the source image need to keep source and output frames separate and should set that bit instead.

A filter can choose between in-place and copy operation depending on configuration parameters, but it cannot change in the middle of a render operation.

This chart outlines some of the effects of choosing the various buffering modes:

Characteristic | In-place | Copy
Speed | Generally faster, as only one buffer is accessed. | Generally slower, as two buffers are accessed.
Memory usage | No additional memory. | Requires an additional frame buffer.
Source frame access | The source frame is modified as pixels are processed, so only the remainder of the source can be read. | The source frame stays unmodified and all of it is valid for the entire image operation.
Format conversion | Hard or impossible to output a different image format than the source. | Trivial to use different source and output formats.
Appropriate filter types | Rendering filters, filters with only temporal processing, color transforms. | Filters that do resampling or warping or use spatial filter kernels.

Altering image parameters

By default, the destination buffer defaults to having the same size as the source buffer. This can be changed in paramProc by altering the parameters of the dst field in the VDXFilterActivation structure. The src field contains information about the input to the filter and can be used to compute the output parameters:

long paramProc(VDXFilterActivation *fa, const VDXFilterFunctions *ff) {
    fa->dst.w = 320;
    fa->dst.h = fa->src.h;
    fa->dst.AlignTo4();

    return FILTERPARAM_SWAP_BUFFERS;
}

Changing the width and the height of the output image requires recomputing the pitch, modulo, and size fields as well; the inline AlignTo4() method does this for you.

It is also possible to change the parameters for the destination buffer in an in-place filter. In fact, at the very least, fa->src.offset should be copied to fa->dst.offset in order for cropping to work. It is also possible to alter the other parameters, allowing very fast cropping (by altering data, w, and h) and field dropping (by altering h and pitch).

Internal frame delays

Video filters normally operate in one-frame-in, one-frame-out mode. However, it is sometimes necessary for a filter to reference a window of adjacent frames. There are two ways to do this.

The first way is to set the FILTERPARAM_NEEDS_LAST flag in paramProc. This tells the host to preserve the previous source frame fed to the filter, which is then accessible via fa->last. The last source frame buffer uses the same format as the source buffer. This allows for a window of two frames.

The other way is to buffer the frames internally in the filter itself. While this allows unlimited window sizes, there is still the problem that the filter can only reference past frames in this manner, and not future frames. The FILTERPARAM_HAS_LAG(count) macro overcomes this by indicating to the host that the filter produces frames with a delay. For instance, returning a bitfield including FILTERPARAM_HAS_LAG(3) means that the filter outputs filtered frame 0 when it receives frame 3. This permits source frame windows that include future frames as well as past frames.

Note: Versions of VirtualDub prior to 1.9.1 do not support frame lags in filters when running in capture or frameserver mode.

Supporting alternate image formats (V12+ only)

In earlier versions of the filter API, the source and destination images always use a 32-bit RGB format. In V12 and up, there is a parallel enhanced set of image structures that support more advanced image configurations. The mpPixmapLayout field in the V12+ extended VFBitmap structure is used for this purpose:

long paramProc(VDXFilterActivation *fa, const VDXFilterFunctions *ff) {
    const VDXPixmapLayout& pxsrc = *fa->src.mpPixmapLayout;
          VDXPixmapLayout& pxdst = *fa->dst.mpPixmapLayout;

    // check for a source format that we support
    if (pxsrc.format != nsVDXPixmap::kPixFormat_YUV422_UYVY)
        return FILTERPARAM_NOT_SUPPORTED;

    // set old depth value to zero to indicate new pixmap layout should be used
    fa->dst.depth = 0;

    pxdst.w = 320;
    pxdst.h = pxsrc.h;
    pxdst.format = nsVDXPixmap::kPixFormat_YUV422_UYVY;
    pxdst.pitch = 0;

    return FILTERPARAM_SWAP_BUFFERS | FILTERPARAM_SUPPORTS_ALTFORMATS;
}

There are a few steps involved in enabling the new support:

  • The API version must be determined to be at least V12. Otherwise, accessing the mpPixmapLayout field may cause a crash or return invalid data. (If you declare at least V12 as the minimum API version for your filter, there is no need to check the version explicitly.)
  • The depth field of the fa->dst structure must be set to zero to indicate that mpPixmapLayout is active.
  • Setting the pitch field of the pixmap layout to zero tells the host to establish default parameters for most fields in the layout, most notably data and pitch. Only w, h, and format then need to be specified.
  • In order to allow arbitrary input formats to be received by paramProc, the function must return with the FILTERPARAM_SUPPORTS_ALTFORMATS bit set (this is included in FILTERPARAM_NOT_SUPPORTED). The host will then call paramProc multiple times to determine which source formats can be used. Otherwise, the filter is assumed to only support 32-bit RGB and will never receive any other format.

Note: It isn't necessary for a filter to support 32-bit RGB input or output; it's acceptable for a filter to accept only YCbCr formats or other RGB formats. This may restrict when your filter can be used with some hosts, however, and will prohibit your filter from being used with any hosts that don't support at least API V12. However, VirtualDub, when confronted with a filter doesn't support the format produced by the previous filter, will search for a format that the filter does support, and then perform an implicit image conversion on entry to the filter.

Altering frame timing (V12+ only)

The mFrameRateHi and mFrameRateLo fields of the VFBitmap structure indicate the frame rate of a video stream. The two fields form a fraction, such that mFrameRateHi / mFrameRateLo is the stream frame rate. paramProc can change the frame rate of a stream by altering the value of these fields in the destination format (fa->dst).

By default, when the frame rate of the video stream is changed, the host samples frames from the source according to timestamp, so that the video still plays at the same speed. This is done by choosing the source frame that corresponds to the time of the center of each output frame:

In most cases, the default behavior provides sensible and predictable behavior. Occasionally, though, it may be desirable to directly control the frame mapping in order to ensure consistency with the filter logic, particularly in edge cases (such as the 100ms case above). This is especially important for filters that rely on exact frame timing, such as field split filters. Override the prefetchProc entry point to take control:

sint64 prefetchProc(const VDXFilterActivation *fa, const VDXFilterFunctions *ff, sint64 frame) {
    // Map every pair of even and odd frames exactly to the same source frame.
    return frame >> 1;
}

Note: prefetchProc must be implemented in a thread-safe, reentrant manner and be able to execute independently of runProc. Prefetch implementations that rely on image processing results are particularly a no-no.

It is also possible to change the length of the video stream by altering the mFrameCount field. Because audio is not affected by changes in the video filter chain, this is typically only useful when making a video filter that is intended to correct a timing error or otherwise cooperate with an audio filter that is making a similar change.


Processing video frames (runProc)

The runProc method of a video filter is the main method that actually processes image data.

Source and output frame buffers

The source frame buffers are specified by the src field of the VDXFilterActivation structure, and the output frame buffer is specified by the dst field. The task of runProc is to process the source image and write the result into the output image. The host has already allocated both frame buffers and fetched the source frame by the time the filter is invoked, so the only thing that needs to happen is to process the image data.

Note that if paramProc specified that the filter is operating in in-place mode, the source and output frame buffers will point to the same memory.

Execution

runProc is always single-threaded — there is never a case where it is invoked twice on the same filter instance at the same time. However, it is possible that two different instances of the same filter may execute concurrently. Therefore, it is safe to use a buffer that is attached to the filter instance, but two instances should generally not share a buffer that both read and write to.

When possible, memory allocation should not occur within runProc, and it is better to preallocate buffers in startProc instead. There are two reasons for this: it's faster, and it means that allocation failures are caught before the render begins instead of during the render itself. This also goes for other external resources, such as reading images off disk. The exception is if there are enough of them that preloading them is impractical; secondary video files, for instance, would be better off read on the fly.

Sample implementation

This is an example runProc implementation which simply divides all pixel values by two, thus reducing the image to half brightness. Note that runProc in a production filter usually considerably more complex, both in functionality and in implementation.

int runProc(const VDXFilterActivation *fa, const VDXFilterFunctions *ff) {
    int           w        = fa->dst.w;
    int           h        = fa->dst.h;
    uint32       *dst      = (uint32 *)fa->dst.data;
    ptrdiff_t     dstpitch = fa->dst.pitch;
    const uint32 *src      = (const uint32 *)fa->src.data;
    ptrdiff_t     srcpitch = fa->dst.pitch;

    // loop over all rows
    for(int y=0; y<h; ++y) {
        // loop over all pixels in current row
        for(int x=0; x<w; ++x) {
            uint32 pixelValue = src[x];

            // divide all channels by 2
            dst[x] = (pixelValue & 0xfefefe) >> 1;
        }

        // advance to next row in destination and source pixmaps
        dst = (uint32 *)((char *)dst + dstpitch);
        src = (const uint32 *)((const char *)src + srcpitch);
    }

    return 0;
}


Managing filter data

Once you begin writing more complex video filters, you will want to allocate complex data structures and side buffers that must be deallocated at some point before program exit. The video filter API provides several hooks to do this.

Filter instance data

Because a filter can be used multiple times within the same filter chain, each entry in the filter chain associated with a filter is called a filter instance, and can have instance data associated with it. This instance data is pointed to by the fa->filter_data field. The instance data is allocated by the host, but is managed by the filter. The size of the instance data is set in the filter definition. Each instance of the filter can thus have its own parameters and operating data.

The easiest way to manage filter data is by storing it in a data structure:

struct MyData {
    int fillValue;
};

int runProc(const VDXFilterActivation *fa, const VDXFilterFunctions *ff) {
    MyData       *data     = (MyData *)fa->filter_data;
    int           w        = fa->dst.w;
    int           h        = fa->dst.h;
    uint32       *dst      = (uint32 *)fa->dst.data;
    ptrdiff_t     dstpitch = fa->dst.pitch;

    // loop over all rows
    for(int y=0; y<h; ++y) {
        memset(dst, data->fillValue, w * 4);

        // advance to next row in destination
        dst = (uint32 *)((char *)dst + dstpitch);
    }

    return 0;
}

static struct FilterDefinition myfilter={
    0,0,NULL,                       // next, prev, and module (set to zero)
    "my filter",                    // name
    "Description of my filter.",    // description
    "Author of my filter.",         // author / maker
    NULL,                           // no private data
    sizeof(MyData),                 // instance data size
    NULL,                           // initProc
    NULL,                           // deinitProc
    runProc,                        // runProc
};

startProc and endProc

While the instance data structure is useful, it would be rather inconvenient if everything in it had to be initialized on first use and there were no convenient place to clean it up.

The most useful entry points to implement are startProc and endProc, which are called at the beginning and end of each rendering operation, respectively. This is useful for allocating large image buffers that are only necessary when actually processing video — the buffers can be left unallocated to save memory, and allocated only when necessary and once the source image parameters are known. This also has the benefit of keeping paramProc simple.

One gotcha with endProc: it may be called more times than startProc and is required to be safe in this regard. Therefore, you should initialize any pointers to NULL on initialization and clear them when deallocating buffers:

int endProc(VDXFilterActivation *fa, const VDXFilterFunctions *ff) {
    MyData *p = (MyData *)fa->filter_data;

    if (p->buffer) {
        delete p->buffer;
        p->buffer = NULL;
    }
}

By default, the memory allocated and placed in fa->filter_data is zero-initialized, like global variables in C, so any pointers in that structure should already be NULL.

initProc and deinitProc

For data that persists across rendering operations, the initProc and deinitProc entry points should be used. These entry points handle initialization and shutdown for fa->filter_data, the data that persists for the life of the filter instance. initProc is called exactly once when the filter instance is initialized, and deinitProc exactly once before it is destroyed.

There is one nit in the above, and that's the copy operation. In certain cases, the host may need to temporarily copy the filter structure. As a result, by default, the filter data structure must be directly copyable. In other words, it must be OK to copy the structure with memcpy(). VirtualDub does this in its filter list UI. Here are a couple of the ugly cases:

  • The user makes a change to filter configuration and clicks OK in the dialog.
    • Filter data A is copied to temporary data B.
    • Data B is edited in the config dialog.
    • Data B is accepted, so data A is freed.
    • Data B is used for rendering.
    • deinitProc receives data B.
    • Data B is freed.
  • The user makes a change to filter configuration and clicks Cancel in the dialog.
    • Filter data A is copied to temporary data B.
    • Data B is edited in the config dialog.
    • Data B is rejected, so it is freed.
    • Data A is used for rendering.
    • deinitProc receives data A.
    • Data A is freed.

For the most part, the easiest way to deal with this is simply to embed data structures directly in the filter data structure as inline fields and arrays.

Flexible copying with copyProc (V9+ only)

Obviously, this is inconvenient for all but the simplest of filter data, so there is a better way: the copyProc entry point. This entry point allows the filter to control the copy process so that any cloned filter data structures are allocated properly. It also enables deinitProc to be called on any additional copies created. Therefore, both of the above scenarios are changed as follows:

  • The user makes a change to filter configuration and clicks OK in the dialog.
    • Filter data A is copied to temporary data B.
    • Data B is edited in the config dialog.
    • Data B is accepted.
    • deinitProc receives data A.
    • Data A is freed.
    • Data B is used for rendering.
    • deinitProc receives data B.
    • Data B is freed.
  • The user makes a change to filter configuration and clicks Cancel in the dialog.
    • Filter data A is copied to temporary data B.
    • Data B is edited in the config dialog.
    • Data B is rejected.
    • deinitProc receives data B.
    • Data B is freed.
    • Data A is used for rendering.
    • deinitProc receives data A.
    • Data A is freed.

With copyProc, it becomes possible to map the allocation and deallocation hooks onto traditional C++ object semantics, which makes filter data management much easier. In particular, the three hooks can be mapped as follows:

Filter API entry point | Equivalent C++ method
initProc | Constructor
deinitProc | Destructor
copyProc | Copy constructor

The tricky part is that, unlike the standard C++ mechanisms, the host does memory allocation instead of the filter. To solve this problem, placement versions of the above primitives are used. Explaining this aspect of C++ is out of scope for this SDK, so here's an example implementation:

// required to use placement new
#include <new>

struct MyData {
   // Normally, you couldn't include an object like this in the filter data, because
   // the host would copy it with memcpy() and cause havoc. With copyProc, however, the
   // filter invokes the standard copy constructor, and the std::string object is copied
   // correctly.
   std::string mFilename;
};

int initProc(VDXFilterActivation *fa, const VDXFilterFunctions *ff) {
    // Invoke placement new to construct the class object in the filter data memory
    // allocated by the host.
    new(fa->filter_data) MyData();
    return 0;
}

void deinitProc(VDXFilterActivation *fa, const VDXFilterFunctions *ff) {
    MyData *p = (MyData *)fa->filter_data;

    // Invoke the class destructor. This destroys any sub-object in the object without
    // invoking the memory deallocator, as the delete keyword would.
    p->~MyData();
}

void copyProc(VDXFilterActivation *fa, const VDXFilterFunctions *ff, void *dst) {
    MyData *p = (MyData *)fa->filter_data;

    // Invoke the copy constructor directly using placement new syntax, constructing the
    // new object instance directly into the supplied memory.
    new(dst) MyData(*p);
}

static struct FilterDefinition myfilter={
    0,0,NULL,                       // next, prev, and module (set to zero)
    "my filter",                    // name
    "Description of my filter.",    // description
    "Author of my filter.",         // author / maker
    NULL,                           // no private data
    sizeof(MyData),                 // instance data size
    initProc,                       // initProc
    deinitProc,                     // deinitProc
    runProc,                        // runProc
    NULL,                           // no paramProc
    NULL,                           // no configProc
    NULL,                           // no stringProc
    NULL,                           // no startProc
    NULL,                           // no endProc
    NULL,                           // no script object
    NULL,                           // no scriptStringProc
    NULL,                           // no stringProc2
    NULL,                           // no serializeProc
    NULL,                           // no deserializeProc
    copyProc                        // copyProc
};


Creating time-varying filters

Many video filters perform the same operation on a video frame regardless of where it lies in the video stream, but some filters change operation over time depending on the frame number, notably filters that overlay additional graphics or text on top of the video based on an external script.

Obtaining timing information on the video stream

While it is possible to count frame numbers in runProc in the filter alone, this is unreliable. A better way is to use the FilterStateInfo structure, pointed to by the fa->pfsi pointer, which indicates the timing for the current frame.

FilterStateInfo is divided into two sets of fields, those for the source video and those for the output video. The difference depends upon how the filter should behave with regard to edits on the timeline in the host. If the filter's behavior is tied to the source — that is, it needs to track the original video or audio content — then it should use the source frame numbers. If the filter needs to track post-edit frame numbers instead, such as applying a fade at a predetermined time, then it should use output frame numbers instead.

Note: The pfsi field may be NULL in some host operating modes or when no input video is present when certain filter API calls are issued. If a filter requires this field, it should check its value in startProc and runProc and report an error if it is absent instead of crashing.

V12+ only: The lMicrosecsPerFrame and lMicrosecsPerSrcFrame fields have limited precision. Use the mFrameRateLo and mFrameRateHi fields of fa->src instead for better precision.

Timing interaction between different filters and timeline edits

When multiple filters are involved, the interaction between different filters and the fields in the FilterStateInfo structure can be difficult to follow. Here is an example, where we have a blur filter followed by a bob doubler filter, and finally some edits on the timeline:

Here we have a blur filter that processes frames 1:1, a bob doubler which produces two output frames for each source frame, and edits in the timeline to extract frames 16-17 and 20-21.

The keys to remember are that:

  • Source and output frame numbers seen by a filter are always the same unless the filter changes frame ordering via prefetchProc.
  • The output frame numbers for a filter become the source frame numbers for the next filter.
  • Destination frame numbers always correspond to the final output and are the same for all filters.

Which frame number you use depends on what you intend to do with it. If you are displaying a running timestamp, you probably want to use the destination frame number. If you're pulling data from an external source like a script, you likely want to use the source frame number. Finally, if you're making a filter like the built-in bob doubler and produce two fields for each source frame, you should use the output frame number to tell which field to output for a given source frame.

Note: Although frames can be reordered starting at V12, making the source and output frame numbers different, the output frame number is only available in API V13.

Differentiating rendering modes (V10+ only)

For filters that have profiling and rendering passes or otherwise have complex behavior, it may be useful to determine whether the host is running in preview or rendering mode. This is done via the flags field of the FilterStateInfo structure, which is only present in API V10 or above. This flag field also indicates if real-time mode is active, where the filter may want to apply lower-quality approximations for speed.

VirtualDub specific: The kStateRealTime flag is set in capture mode.


Handling aspect ratio

V13 or earlier: Aspect ratio support is only available starting with API V14.

Starting with API V14, source and output streams are tagged with a fraction indicating the pixel aspect ratio of video frames. This indicates whenever the pixels are non-square.

In general, most video processing algorithms aren't affected by aspect ratios, but in some cases it can be useful. For instance, aspect ratio information can be used to ensure that text is not rendered squished or stretched.

Reading and controlling the pixel aspect ratio

The pixel aspect ratio (PAR) is indicated by the mAspectRatioHi and mAspectRatioLo fields of the VDXFBitmap structure. The ratio of these two is the PAR, with ratios above one indicating a pixel that is wider than it is tall. If the fraction is 0/0, it means that the pixel aspect ratio is unknown. In that case, your filter should assume square pixels (1:1).

To change the PAR, the filter should modify the aspect ratio fields on the fa->dst bitmap in its paramProc method. Note that changing the aspect ratio does not actually change any of the video data, so this should only be used to indicate a true change in aspect. It should not, for example, be used to cheaply stretch video horizontally.

Pixel aspect ratios for YCbCr video

In cases where the video images are composed of multiple size pixels, such as the luma and chroma planes of YCbCr-encoded video, the PAR is described for the luma plane. A conversion between formats with different color spaces or chroma subsampling ratios does not alter the PAR.

Frame aspect ratio vs. pixel aspect ratio

Sometimes aspect ratios are described in terms of the frame rather than each pixel. Widescreen, for example, is described as 16:9 regardless of the resolution of the image. The video filter API does not encode the frame aspect ratio directly and it is implicit based on the frame resolution and the PAR:

frame_aspect_ratio = pixel_aspect_ratio * width / height

For instance, a video stream with a PAR of 8/9 and a resolution of 720x480 has a frame aspect ratio of 4/3 (4:3).

How aspect ratio is used (VirtualDub specific)

In VirtualDub, the aspect ratio indicated at the end of the filter chain is used to correct the display width. The user can override this, but one of the options on the aspect ratio menu of the display panes is to use the reported aspect ratio.

Note that AVI does not generally support specifying aspect ratios. Some video decoders report aspect ratio based on information in the encoded video stream, but there is no general mechanism for doing so or for controlling this through the encoding API. Therefore, video that emerges with a pixel aspect ratio (PAR) other than 1:1 will have its aspect ratio reset when written to AVI unless another filter is used to correct the PAR by resampling the image and changing the resolution to compensate.


Prefetching multiple source frames

By default, VirtualDub video filters operate in 1:1 mode, where each source frame corresponds to exactly one output frame. The output frame stream has the same frame count, frame rate, and frame timings as the source stream.

V13 or earlier: Prefetching multiple source frames per output frame is only possible starting with API V14. If you are targeting an earlier API version, you need to use alternative ways of handling this, such as using internal buffering. See setting filter parameters on how to declare an internal lagging delay in your filter to accommodate this.

Requesting multiple source frames

In order to receive multiple source frames, a filter must implement the prefetchProc2 method. This method receives a prefetcher object which is invoked by the filter for each source frame required.

A filter does not need to check for source frame numbers lying outside of the source stream when prefetching. Any frame numbers that are outside of the stream are clamped to the beginning or end as appropriate. That means, for instance, that a filter that needs a five-frame window can simply request frames -2 and -1 at the start of the window and receive frame 0 twice in those slots. The frames will be identified as frame 0, however.

The prefetch pattern can be different for each output frame, and a filter is not required to prefetch the same number of source frames each time. The filter does have to prefetch any frame that it might need in the runProc() method, though, because by the time that runs it is too late to fetch any more frames.

VirtualDub specific: Starting with VirtualDub 1.9.1, the filter system caches frames within the filter pipeline. This minimizes the cost of redundantly fetching source frames.

Processing with multiple source frames

Normally, the runProc method receives input and output frames via the src and dst fields of the VDXFilterActivation structure. The V14 API adds the mpInputFrames and mpOutputFrames arrays, which contain pointers to source and output frames. The src and dst fields alias onto the first entry of those arrays, whereas any additional frames take up additional slots.

If your filter does not need more than one source frame, it can continue to use src and dst.

Identifying source frames

Source frames always arrive in the same order that they are prefetched. If you are making a filter that uses a window of source frames, you can request them in order and rely on that order being consistent. The host can choose to read frames out of order, but this is invisible to your filter.

In cases where position is not enough, such as when some output frames require more source frames than others, you can supply a cookie to the PrefetchFrame() method. The cookie is an arbitrary 64-bit integer that is copied into the mCookie field of the VDXFBitmap structure of the corresponding frame when runProc() is called. One way to use this is to store an index into the cookie and use that to sort the frames into an array in the run method.

Source frames are also tagged by their frame number in the mFrameNumber field. Note, however, that this is the true frame number, which is different from the requested frame number when the frame number has had to be clamped to be within the stream. In other words, a request for frame -2 will return a frame that is tagged as frame 0.

Fetching frames in direct mode

With the prefetcher object, you can also invoke the PrefetchFrameDirect() method to indicate that a frame should be copied directly from source to output. In addition to speeding up rendering for frames that your filter does not need to process, this also permits smart rendering logic to bypass the entire filter chain whenever possible and copy compressed frames directly from source. This runs much faster and preserves full quality in the source. If your filter is designed to only process a portion of its video, you should use PrefetchFrameDirect() whenever possible to enable direct mode operation.

Note that even if you mark a frame as direct, your filter's runProc method may still be called. One reason this may occur is if the frame cannot be copied in direct mode. When this happens, your filter should copy the frame as required.

Direct mode requires that source and output frame formats be compatible. If the formats or frame sizes do not match, it is a logic error to request a direct copy. You can, however, request a direct copy between streams that are of different frame rates or aspect ratios.

Prefetching symbolic frames

A tricky filter can use cached information to determine that some source frames may not be necessary. This is particularly possible if the filter has already processed an output frame before. For instance, a filter may have cached the location of a detected scene transition, and avoid prefetching source frames across that transition. Although caching of source frames helps, avoiding prefetches entirely reduces CPU usage in the filter system.

The problem with doing this is that it can obscure the relationship between frames in the filter pipeline and sometimes the host needs to know that frame 100 after the filter comes from frame 50 in the source, perhaps for audio purposes. The filter can do a symbolic prefetch via the PrefetchFrameSymbolic() to declare the equivalent source frame without actually doing a frame fetch. This then allows the host to cheaply follow frame dependencies.

Symbolic fetches are optional. If the filter does not do a symbolic fetch, the host assumes that the symbolic frame is the mean of the prefetched source frames, or non-existent if no source frames are requested. A direct frame automatically becomes the symbolic frame. If the filter both issues regular frame prefetches and a symbolic prefetch, the symbolic prefetch takes precedence. Therefore, it is safe simply to always do the symbolic prefetch.

Prefetch in capture mode (VirtualDub specific)

The prefetcher allows a filter to request source frames in entirely arbitrary order, but this can cause problems when the filter chain must be run in push mode. This happens if filters are enabled during live video capture. In this case, VirtualDub attempts to maintain a queue of source frames to satisfy prefetch requests. If a frame is prefetched that is too old and has already scrolled off the queue, the oldest available frame is supplied instead. Whenever possible, filters should attempt to be monotonic in their prefetches to support this. More specifically, it is best that the oldest source frame required never go backward when prefetching successive output frames.


Making a filter configurable

Once you have a video filter up and running, it's useful to replace hard-coded constants in the filter with configurable parameters. This makes it easier to tune those parameters and also exposes more control to end users.

The configProc entry point

configProc is the entry point used for interactive configuration of a filter. It is the only entry point in a video filter which is allowed to display UI. Typically, the main action performed by configProc is to display a modal dialog:

#include <windows.h>
#include "resource.h"

HINSTANCE g_hInst;

INT_PTR CALLBACK MyDlgProc(HWND hdlg, UINT msg, WPARAM wParam, LPARAM lParam) {
    MyData *data = (MyData *)GetWindowLongPtr(hdlg, DWLP_USER);
    switch(msg) {
        case WM_INITDIALOG:
            SetWindowLongPtr(hdlg, DWLP_USER, (LONG_PTR)lParam);
            data = (MyData *)lParam;

            CheckDlgButton(hdlg, IDC_ACTIVE, data->active ? BST_CHECKED : BST_UNCHECKED);
            return TRUE;

        case WM_COMMAND:
            switch(LOWORD(wParam)) {
                case IDOK:
                    data->active = (0 != IsDlgButtonChecked(hdlg, IDC_ACTIVE));
                    EndDialog(hdlg, true);
                    return TRUE;

                case IDCANCEL:
                    EndDialog(hdlg, false);
                    return TRUE;
            }
            break;
    }

    return FALSE;
}

int configProc(VDXFilterActivation *fa, const VDXFilterFunctions *ff, VDXHWND hwndParent) {
    MyData *data = (MyData *)fa->filter_data;

    return !DialogBoxParam(g_hInst, MAKEINTRESOURCE(IDD_MY_DIALOG), (HWND)hwndParent, MyDlgProc, (LPARAM)data);
}

The example above constructs a modal dialog by direct Win32 UI programming, which understandably can be somewhat cumbersome. Feel free to use whatever UI toolkit you desire, just as long as you keep the following in mind:

  • Configuration parameters should not be stored in global variables. There may be multiple instances of your filter in the same filter chain, each with different configuration values.
  • configProc should return a non-zero value when the user clicks Cancel (note that this is an error code convention, reversed from the usual true-for-success convention). This is used by the host to determine when a filter hasn't been configured. Current versions of VirtualDub display the configuration dialog for a filter instance automatically on add, and will auto-delete the instance if the dialog is cancelled. Failing to return the correct code will cause strange behavior here.
  • If the user clicks Cancel, the configuration parameters in the instance data should not be changed. This was accomplished above by delaying data exchange until the IDOK button command, but an alternate way is to copy off the data prior to invoking the dialog and restore it afterward on cancellation. This is useful for filter preview, which is described later.
  • VDXHWND is an alias for HWND used to reduce header dependencies on <windows.h>; it can be casted as such.
  • Some Win32 UI examples use NULL for the HINSTANCE parameter passed to UI functions. This will not work in a video filter, since filters are implemented in DLLs. Your filter should implement a DllMain() function instead and store the DLL's HINSTANCE in a global variable, or use another method of obtaining the DLL's instance handle.
  • If you use Win32 common controls, make sure your filter DLL calls InitCommonControls() or equivalent to ensure that comctl32 is pulled in. VirtualDub does this, but hosts are not required to do so.

V11+ only: fa->src is guaranteed to be set up prior to calling configProc. This can be used in the configuration dialog UI to base parameters on the input video, such as providing a relative size parameter.

Displaying a configuration blurb

The stringProc2 method allows the filter to expose a short, user-readable blurb that briefly describes the filter's configuration. The host can use this in filter chain lists to allow the user to scan the configuration of each filter at a glance. stringProc2 simply produces a short string:

static void stringProc2(const VDXFilterActivation *fa, const VDXFilterFunctions *ff, char *buf, int maxlen) {
    MyData *mfd = (MyData *)fa->filter_data;

    _snprintf(buf, maxlen, " (%s)", mfd->enabled ? "enabled" : "disabled");
}

Not every parameter needs to be exposed in the blurb, as it's for visual scanning only and is never interpreted by program code other than to display to the user.

V1-V7 only: Older versions of the filter API only support the stringProc entry point, which serves the same purpose but does not provide a buffer size. You should always support stringProc2 for reliability, but you can support older host versions as well by implementing stringProc in terms of stringProc2 with a small hardcoded buffer size of 80.

Interactive filter preview

By using the fa->ifp field in the VDXFilterActivation structure, it is possible for the user to interactively view results of changing filter parameters. This is accomplished by running the filter while its configuration dialog is displayed. The host automatically manages the UI for the preview window, easing implementation in the filter. The steps to get this working are as follows:

  • Check if fa->ifp is NULL. It may not be available if no video is loaded.
  • Copy fa->ifp to a location accessible by the dialog procedure code.
  • Call IVDXFilterPreview::InitButton() to associate filter preview with a button on your dialog. By doing this, the host will automatically toggle the button's caption text between "Show preview" and "Hide preview" automatically as the preview window is opened and closed.

    VirtualDub 1.7.0 and later: VirtualDub attempts to preserve any existing keyboard shortcuts prefixed by & in the existing button caption.
  • When the button is clicked, call IVDXFilterPreview::Toggle() with the window handle of your dialog. The window handle is used as the owner for the preview window, so that it always appears above the filter dialog.
  • Call IVDXFilterPreview::SetButtonCallback() to be notified when the preview window is opened or closed. This can be used to implement custom button change behavior.
  • IVDXFilterPreview::Display() can be used to explicitly control the display of the preview window.
  • IVDXFilterPreview::Close() closes the preview window and breaks any existing connection to a preview button.
  • When a configuration parameter is changed, call IVDXFilterPreview::RedoFrame() to cause the current frame to be refiltered and redisplayed. This means that you do need to change the filter parameters directly on the fa->filter_data structure. Make sure any unwanted changes are rolled back if the dialog is cancelled.
  • For changes that require the filter chain to be reconstructed, such as changes in frame size, call IVDXFilterPreview::UndoSystem() prior to effecting the change, and IVDXFilterPreview::RedoSystem() afterward.
  • V11+ only: fa->ifp2 provides the IVDXFilterPreview2 interface, which can be used to query if the preview window is currently displayed.

Sampling the input video

The sampling functionality in the filter API allows your configuration dialog to process frames in the input video and use it to augment the UI. For instance, a filter designed to modify image luminance levels can analyze a video frame and display a histogram to aid the user in setting appropriate ranges, or even automatically set the parameters based on image analysis.

To implement sampling:

  • Implement filter preview first. The preview UI is required for sampling to be used.
  • Add sampling buttons to your UI. Two buttons are needed, one for sampling the current frame, and the other for sampling the whole video. You may want to implement the button callback in order to enable those buttons only when the preview display is updated.
  • Implement a sampling callback and hook it up via IVDXFilterPreview::SetSamplingCallback().
  • Call IVDXFilterPreview::SampleCurrentFrame() or IVDXFilterPreview::SampleFrames() whenever the appropriate button is pressed.
  • The sampling callback is invoked for each frame processed during the sampling operation.

    VirtualDub specific: The SampleFrames() function displays UI allowing the user to sample only a subset of the frames in the video for speed. This occurs invisibly without the filter knowing.

Note that it is not possible for the filter to autonomously request frames — sampling must always be driven by the user through the filter preview UI.

V12+ only: The format of the video frames passed to the filter is the one negotiated by the filter's paramProc routine, since the filter chain is always running when sampling occurs. If your filter accepts non-RGB formats, your sampling routine must also accept them as well.


Scripting support

Video filters can expose a Config() method that allows scripts to control filter configuration values, without requiring user intervention. This is also required for your filter to work in batch mode (job control). It is highly recommended that any filter offering configurable parameters implement scripting/batch support.

Implementing the Config() method

To expose the script method, the script_obj field of the filter definition must be filled. This points to a script object definition, which describes the Config() method group exposed by the filter. Typically this just points to a function list:

VDXScriptObject script_obj = { NULL, script_functions, NULL };

The function list, in turn, describes all of the methods exposed in terms of a C/C++ handler function, name, and argument list:

VDXScriptFunctionDef script_functions[] = {
    { (VDXScriptFunctionPtr)configScriptFunc, "Config", "0i" },
    { NULL, NULL, NULL },
};

Normally, the function list only consists of one function entry and the NULL sentinel. The third argument describes the argument list to the script function, and consists of the first character, which describes the return type, and subsequent characters, which correspond to arguments. The return value should always be a zero (0) for void, while the argument characters can be i for integer or s for string.

V11+ only: Function parameters can also be of type 64-bit long (l) or double (d).

The function callback is called with a pointer to the script interpreter, filter activation context, arguments, and argument count:

static void configScriptFunc(IVDXScriptInterpreter *isi, void *lpVoid, VDXScriptValue *argv, int argc) {
    VDXFilterActivation *fa = (VDXFilterActivation *)lpVoid;

    MyData *mfd = (MyData *)fa->filter_data;

    mfd->width = argv[0].asDouble();
    mfd->height = atoi(*argv[1].asString());

    if (argc >= 3 && argv[2].isInt())
        mfd->mode = argv[2].asInt();
}

Note: Due to the cast when initializing the VDXScriptFunctionDef structure, it is possible to accidentally supply the wrong function prototype when declaring the function. Make sure that the prototype is correct, including the calling convention (__cdecl). The return value will vary depending on the return type specified in the function signature -- it should be void for a 0.

Creating the script line

Once the script method has been declared, an additional fssProc function must be supplied to emit the Config() script call describing the current configuration. This generally simply calls a printf() style function to generate the script line:

bool fssProc(VDXFilterActivation *fa, const VDXFilterFunctions *ff, char *buf, int bufsize) {
    MyData *mfd = (MyData *)fa->filter_data;

    if ((unsigned)_snprintf(buf, buflen, "Config(%g, \"%d\", %d)"
            , mfd->width
            , mfd->height
            , mfd->mode) >= (unsigned)(bufsize - 1))
        return false;

    return true;
}

Note: Bounded versions of sprintf(), such as the VC++-specific _snprintf() call above, should be used to avoid a buffer overflow in case the script string is too large to fit.

Raising errors

The script interpreter checks argument types against the method prototypes specified in the function list, so it is not normally required for the handler function to check types manually. If the handler function does detect an error, it can raise the error by using the EXT_SCRIPT_ERROR() macro, which in turns calls the IVDXScriptInterpreter::ScriptError() method. This method throws a C++ exception and thus does not return.

In general, however, it is better to store the invalid parameters and flag the error on the startProc method. This allows for more friendly reporting and consolidates error handling logic in the filter.

Function overloading

Once a filter has been released, it may be necessary to update the filter's script interface in a way that is backwards compatible. This is done by overloading the Config() method with multiple argument lists:

VDXScriptFunctionDef script_functions[] = {
    { (VDXScriptFunctionPtr)configScriptFunc, "Config", "0i" },
    { (VDXScriptFunctionPtr)configScriptFunc, NULL, "0iis" },
    { NULL, NULL, NULL },
};

The Config name should only be specified for the first entry, with subsequent entries using NULL for the name. The handler function can be the same or different for each entry; if it is the same, the function must determine which prototype was used by checking the argument count and parameter types.

Once overloads have been declared, the script engine chooses the best match based on the arguments provided and the argument lists of each overload. If multiple methods match and no match is clearly better than the rest, an ambiguity error is raised and script execution fails.

Tip: Avoid declaring overloads that differ only in argument types of int, double, or long. This can produce unexpected ambiguities when the config string is emitted with numbers that are valid int values, such as zero. It is safest to extend the method list by adding parameters or by swapping numbers and strings.


CPU dependent optimization

Depending on the CPU in the user's machine, it may be possible to use certain instruction set extensions to accelerate execution of the filter, such as MMX or SSE2. Good use of these extensions can result in significant speedups, as much as 2-4x. However, optional extensions must be checked for before they are used.

Checking for CPU features

The video filter API exports several entry points that allow the filter to query for optional CPU features. Although it is possible for the filter to query the CPU directly, using the host callbacks has the benefit that the filter tracks any CPU feature override UI that is in the host. The isFPUEnabled() call returns true if FPU (x87) optimizations should be used; isMMXEnabled() returns true if MMX should be used. For more advanced features, getCPUFlags() also reports support for integer SSE, SSE, SSE2, 3DNow!, and 3DNow! Professional.

Keep in mind when writing code that the video filter API offers no guarantees with regard to any CPU extensions — only the bare instruction set is supported. In particular, on x86, neither MMX nor P6 conditional moves (CMOVcc/FCMOVcc) should be used before checking feature flags. On x64, SSE2 is standard, but 3DNow! is not.

Note: The host can change the value of flags on the fly in response to a change the user preferences. A filter does not have to support all dynamic changes — it can cache the state of the flags when required.

Note: If you are using compiler options to generate code that uses instruction set extensions, such as /QxW on Intel C/C++ or /arch:SSE2 with Microsoft Visual C++, you must ensure that such code is not executed until support for the extensions is verified. This can be done either by instructing the compiler to check for the extensions (ex: /QaxW in Intel C/C++), or selectively compiling the startup code for the filter with CPU-specific optimizations disabled. A sledgehammer-like method would be to compile only the module initialization routine in this manner and have it add filter entries to the host only if the host reports that the necessary CPU extensions are available.

VirtualDub specific: VirtualDub relies on 80486 instructions to be supported, but does not guarantee support for instructions introduced in later CPUs.

Requesting aligned buffers (V14+ only)

When working with vector instruction sets it is frequently advantageous to have aligned data, or data that is placed at addresses in memory that are a multiple of some alignment size. For MMX instructions, this is 4 or 8 bytes, and for SSE and above, it is 16 bytes. By default, it is only guaranteed that filter image buffers are aligned by natural alignment, or the usual alignment for the pixel type. For 32-bit RGB frame buffers this is 4 bytes, and for most others, it is only byte alignment. This complicates vectorization as vector instruction sets often have awkward and slow handling of unaligned data and require fixup routines to handle odd pixel counts.

Starting with the V14 API, it is possible to request 16 byte alignment of all scanlines by having paramProc return the FILTERPARAM_ALIGN_SCANLINES flag. This flag modifies the allocation of frame buffers so that scanlines are always aligned to a 16 byte boundary and are a multiple of 16 bytes long. This simplifies vectorization since routines can read vectors directly from memory and not have to worry about unaligned loads or crashing due to reading beyond the end of a scanline.

The FILTERPARAM_ALIGN_SCANLINES flag also has another effect, which is that it also pads out scanlines in the output buffer so that the filter can write multiples of 16 bytes. This also reduces the complexity of the filter as it often makes it unnecessary to have fixup code for images that are an odd number of pixels in width. Padding applies to all planes, so the quarter-size chroma planes in a 4:1:0 YCbCr buffer will also be padded to a multiple of 16 bytes.

Note that there are a couple of gotchas to aligned scanline support. The first is that, although the padding at the end of output scanlines is ignored and can be written with any value, the extra padding on source scanlines is not guaranteed to have any particular value. This means that filters still need to be carefully written so that those undefined values don't affect the output, which might otherwise show up as noise on the right side of the image. The second and more minor gotcha is that in some cases using this flag can impose a small performance penalty as the host may have to realign buffers that aren't aligned appropriately from the source.


Getting started with the VDXFrame video filter wrappers

The video filter API is intended to be implemented directly by filters. However, doing so can be inconvenient, because it requires writing a series of entry points in function style. The VDXFrame static library contains wrappers to simplify the writing of video filters.

Including the VDXFrame static library

Since VDXFrame is a static library, you need to link it into your DLL. Project files are included for both Visual C++ 6.0 and Visual Studio 2005. To use VDXFrame, add the project to your workspace or solution, and then set your DLL project to depend on the VDXFrame project. The include path for your project must also include the include directory of the Plugin SDK where the vd2/VDXFrame directory is located; you should already have this included if you are using the main video filter plugin headers.

Note: Because VDXFrame is a static library, its code is linked into your DLL and thus isn't part of the video filter API itself. Regardless of whether you use VDXFrame or not, your filter exports the same entry points and looks the same to the host.

Wrapper operation

The VDXFrame video filter wrappers are designed to be used as base classes. Your filter derives from them, and the wrapper base handles the translation between function entry points in the API and virtual methods in the derived class. In particular, this avoids the need to manually shuttle instance data around in (void *) pointers and cast to and from structure pointers.

As an example, video filters can store instance data in the fa->filter_data pointer. Every entry point must be written as a straight function, which then must access and unpack the filter_data pointer to access instance data:

int runProc(const VDXFilterActivation *fa, const VDXFilterFunctions *ff) {
    MyFilterData *mfd = (MyFilterData *)fa->filter_data;

    int brightness = mfd->mBrightnessSetting;
    int contrast = mfd->mContrastSetting;

    // ...
}

In contrast, the VideoFilter wrapper allows data to be accessed directly through the implicit this pointer of a class method:

void MyFilter::Run() {
    int brightness = mBrightnessSetting;
    int contrast = mContrastSetting;

    // ...
}


Writing the module entry point

The first task when writing a video filter is to declare the DLL entry points that allow the host to find the filters within it. VDXFrame includes standard entry points that simplify this task.

Declaring the entry point

To declare the entry point, use the macros from VideoFilterEntry.h:

#include <vd2/VDXFrame/VideoFilterEntry.h>

VDX_DECLARE_VFMODULE()

By default, this declares both the VirtualdubFilterModuleInit2 and VirtualdubFilterModuleDeinit entry points, and declares a required video filter API version of API V9 (VIRTUALDUB_FILTERDEF_COMPATIBLE_COPYCTOR). If you need to declare a higher API version minimum, you can use the alternate macro:

VDX_DECLARE_VFMODULE_APIVER(14)

Declaring video filters

The entry points supplied by VDXFrame automatically register all filters in a list supplied by the filter DLL. The list is specified using a second set of macros:

VDX_DECLARE_VIDEOFILTERS_BEGIN()
    VDX_DECLARE_VIDEOFILTER(definitionSymbolName)
VDX_DECLARE_VIDEOFILTERS_END()

definitionSymbolName is the name of the symbol of type VDXFilterDefinition exported elsewhere in the DLL. For each video filter, a new VDX_DECLARE_VIDEOFILTER line should be added between the begin and end macros. This is designed to allow the video filters and their definitions to be


Creating a filter

Once you have the entry point in place, you can begin writing the video filters themselves.

Deriving from VDXVideoFilter

To create a video filter, first derive from the VDXVideoFilter base class to produce the derived class for your filter. There are two methods on the wrapper base class that must be overridden, GetParams() and Run(). These correspond to the paramProc and runProc entry points, respectively. Therefore, this is the minimal filter implementation:

#include <vd2/VDXFrame/VideoFilter.h>

class EmptyFilter : public VDXVideoFilter {
public:
    virtual uint32 GetParams();
    virtual void Run();
};

uint32 EmptyFilter::GetParams() {
    return FILTERPARAM_SWAP_BUFFERS;
}

void EmptyFilter::Run() {
}

The VDXVideoFilter class contains protected fields called fa and ff that correspond to the filter activation and functions structures traditionally passed to the filter entry points. This avoids having to declare them on every method and allows code written to use those variables to port directly over to the wrappers.

Other entry points in the video filter API are also exposed in the VideoFilter base class, and can be implemented as simply as overloading the appropriate base class method. The base class methods are sometimes simplified — for instance, both the stringProc and stringProc2 API methods can be implemented by overloading VDXVideoFilter::GetSettingsString().

Creating the video filter definition structure

Once you have the video filter class in place, the video filter definition can be created using the VDXVideoFilterDefinition template:

extern VDXFilterDefinition filterDef_emptyFilter =
    VDXVideoFilterDefinition<EmptyFilter>(
        "My Name",
        "Sample: Empty Filter",
        "Sample filter from the VirtualDub Plugin SDK: Does nothing");

The three arguments to the constructor are author, name, and description. All other fields are derived from the filter class.

That's it — you should now have a working video filter.


Adding configurability

Once the basic structure is in place, interesting filters need some sort of configurability. VDXFrame also has wrappers to make this easier. As usual, these are optional and you can use another UI library with the filter base class.

Splitting off the configuration class

One of the side effects of making it easier to add instance data to a filter is that the filter class itself is likely to contain fields that are not related to configuration parameters. To make writing the configuration UI easier, it's recommended that the configuration data be split off into a separate structure:

struct MyFilterConfig {
    int mBrightness;
    int mContrast;

    MyFilterConfig()
        : mBrightness(0)
        , mContrast(100)
    {
    }
};

Doing this makes it more likely that either the structure can be copied as-is, or that it is easier to write the copy constructor and assignment operators.

Writing the configuration dialog

Once the configuration structure is lifted out into a separate class, the dialog class can be created. This is done by deriving from VDXVideoFilterDialog:

class MyFilterDialog : public VDXVideoFilterDialog {
public:
    MyFilterDialog(MyFilterConfig& config, IVDXFilterPreview *ifp) : mConfig(config), mpPreview(ifp) {}

    bool Show(VDXHWND parent) {
        return 0 != VDXVideoFilterDialog::Show(NULL, MAKEINTRESOURCE(IDD_FILTER_CONFIG), (HWND)parent);
    }

    virtual INT_PTR DlgProc(UINT msg, WPARAM wParam, LPARAM lParam);

protected:
    MyFilterConfig& mConfig;
    IVDXFilterPreview *mpPreview;
};

INT_PTR MyFilterDialog::DlgProc(UINT msg, WPARAM wParam, LPARAM lParam) {
    return FALSE;
}

Sadly, this is as much as VDXVideoFilterDialog will for you, as you will still have to write Win32 dialog code. However, the base class takes care of wrapping and unwrapping the this pointer, making the rest easier. You can extend the base class with additional methods to add common message handlers.

Hooking up the dialog

Now that the dialog is created, the Configure() method can be added to the filter class:

bool MyFilter::Configure(VDXHWND hwnd) {
    MyFilterConfig oldConfig(mConfig);
    MyFilterDialog dlg(mConfig, fa->ifp);

    if (dlg.Show(hwnd))
        return true;

    mConfig = oldConfig;
    return false;
}

The copying of the configuration object in and out may seem unnecessary, but this allows the dialog and the filter to share the same configuration structure so that the dialog can update the filter for live previews. It also means that the dialog class doesn't have to worry about rolling back the changes to the configuration object.

An additional advantage of this split structure between the filter, dialog, and configuration objects is that it allows the platform-dependent dialog code to be isolated in a different module, if desired. The VDXHWND type makes this easier.

Note that there is a gotcha involved with the VDXFrame implementation of the Configure() method: the VDXVideoFilterDefinition template uses an overload trick to determine if the filter class overloads Configure(). Therefore, a filter cannot "unimplement" the Configure() method by calling the base VDXVideoFilter::Configure() method.

Summarizing settings in the filter list

The stringProc and stringProc2 methods of a filter generate a short blurb for filter list UI. VDXVideoFilter consolidates both of these into one method and provides a SafePrintf() helper to simplify its implementation:

void EmptyFilter::GetSettingString(char *buf, int maxlen) {
    SafePrintf(buf, maxlen, " (%d)", mConfig.mBrightness);
}


Adding script support

Adding script support to a VDXFrame-based filter is also simplified.

Declaring the script object and script methods

To declare the script object, use the VDXVF_DECLARE_SCRIPT_METHODS macro within the filter class definition:

public:
    virtual void Run();

    // ...

    VDXVF_DECLARE_SCRIPT_METHODS();

protected:

Note that this must be done publicly as the script object needs to be accessible to the VDXVideoFilterDefinition template.

Next, define the script method table and the methods themselves:

VDXVF_BEGIN_SCRIPT_METHODS(MyFilter)
VDXVF_DEFINE_SCRIPT_METHOD(MyFilter, ScriptConfig, "ii")
VDXVF_DEFINE_SCRIPT_METHOD2(MyFilter, ScriptConfig2, "iii")
VDXVF_END_SCRIPT_METHODS()

This creates overloads of the Config() script function. Use VDXVF_DEFINE_SCRIPT_METHOD for the first overload and VDXVF_DEFINE_SCRIPT_METHOD2 for the subsequent ones. The third argument is the signature string. Note that unlike the raw definition, the string used here does not include the return value, which is always declared as 0 (void).

Now the script methods themselves can be implemented:

void MyFilter::ScriptConfig(IVDXScriptInterpreter *isi, const VDXScriptValue *argv, int argc) {
    mConfig.mBrightness = argv[0].asInt();
    mConfig.mContrast = 100;
}

void MyFilter::ScriptConfig2(IVDXScriptInterpreter *isi, const VDXScriptValue *argv, int argc) {
    mConfig.mBrightness = argv[0].asInt();
    mConfig.mContrast = argv[1].asInt();
}

As usual, VDXFrame marshals the this pointer, so these are declared as non-static methods.

Writing the script string

Finally, implement GetScriptString() to generate the Config() call in script. This is done the same as GetSettingsString().

void MyFilter::GetSettingString(char *buf, int maxlen) {
    SafePrintf(buf, maxlen, "Config(%d,%d)"
            , mConfig.mBrightness
            , mConfig.mContrast
            );
}


Advice and gotchas

Here are some things to watch out for when writing your video filter:

In-place filter cropping

The fa->dst.offset field may inadvertently introduce a bug in your filter if you are writing it as in-place. By default, this field is initialized to zero, placing the destination bitmap to the start of the buffer. This is appropriate for two-buffer filters, but for an in-place filter it results in the destination bitmap being misplaced compared to the source bitmap. In most cases this isn't a problem, except when cropping is enabled. When cropping occurs on the source bitmap, this can cause the cropping to malfunction.

To avoid this problem, simply copy over the offset from the source bitmap to the output bitmap:

            fa->dst.offset = fa->src.offset;

Buffer overruns

When a scanline is 319 pixels (1276 bytes) in length, you must read or write no more than exactly 1276 bytes. You may not round this value up, write 1280 bytes, and hope there is additional unused memory at the end of the scanline. Doing so may cause a number of issues including corrupting adjacent buffers, destabilizing the application, or causing a crash. The same goes for alignment — you cannot assume that a scanline is aligned to 16 bytes just because it would be convenient. This may require some inconvenient fixup code for the odd bytes along the side.

There are a couple of exceptions. The first is that if you are running on a host that supports API V14 or higher, you can request aligned scanlines to simplify the situation (see CPU dependent optimization). The second is that you can bend the rules slightly if you are doing an advanced trick for optimizing unaligned accesses.

The trick is that you can safely read an unaligned word by reading the two aligned words that contains it. In particular, you may read all bytes in any 16 byte region aligned to a 16 byte boundary, or write all of those bytes as long as you do not actually change the values of bytes not belonging to the scanline. In other words, if a scanline spans 0136F004 to 0136F503, you can issue an aligned 16-byte read at 0136F000 without risking an access violation. Optimized algorithms that require aligned scanlines can often be used on unaligned scanlines simply by applying appropriate masks on the edges.

V14+ only: If FILTERPARAM_ALIGN_SCANLINES is set in paramProc(), you can safely write an integral number of 16 byte xmmwords even if this runs beyond the end of the scanline. For instance, if you are writing a 319 pixel wide, 32-bit scanline, you would normally only be able to write 319 * 4 = 1276 bytes, but instead you can write 80*16 = 1280 bytes. The contents of the bytes beyond the end of the scanline are ignored.

Pointer arithmetic (a.k.a. how not to crash on an upside-down image)

When treating the pitch for a bitmap/pixmap, you should always treat it as signed and use the standard C/C++ standard ptrdiff_t type. In particular, pixmaps can and often do have negative pitches, and using an unsigned type will cause crashes. In most cases you can get away with using a regular signed int, but it's simpler and actually faster just to use the correct type instead.

Alpha channel

The upper byte of each 32-bit pixel, the alpha channel, is unused. Its value is completely arbitrary on entry to the video filter and ignored on output. If you are porting image processing code from other sources, make sure it does not rely on the alpha byte to be set in any particular manner.

Runtime libraries

Make sure your filter doesn't depend on any runtime libraries that you don't ship with the filter binary. With Microsoft Visual C++, make sure you either statically link your filter to the C runtime library (CRT) or distribute the CRT DLL. This is particularly important for versions of VC++ beyond 6.0, which no longer use MSVCRT.DLL when the DLL version of the CRT is enabled. In VC++, the CRT linking mode is controlled in the Code Generation page of the compiler options in the project's settings. Statically linking filter DLLs to the CRT is the safest and most convenient model for distribution, and is recommended unless you are experienced with CRT distribution issues.

VirtualDub specific: Older versions of VirtualDub could run up against the operating system's limit for thread local storage (TLS) handles, of which one was required for each instance of the CRT loaded. These are consumed for each filter loaded when filters are statically linked to the CRT, and other DLLs that needed to load like video and audio codecs also consumed TLS slots. Because there are only 64 TLS slots available per process in Windows 95/98/NT4 and 80 in Windows 98, it was possible to have enough filters and codecs installed that DLLs would fail to load. This is mostly a non-issue now because the TLS slot limit was raised to 2088 in Windows 2000 and VirtualDub started dynamically loading and unloading filter DLLs starting with 1.5.0.

Don't open UI outside of configProc

You aren't guaranteed that the thread that startProc or runProc is called on is a UI thread. COM may not be initialized — particularly important for the shell APIs — and the thread may not even have a message pump. In fact, you aren't even guaranteed that two consecutive calls to runProc occur on the same thread, only that two threads won't run that function at the same time. Therefore, one thing that you should never do is attempt to create a window or dialog from those functions. Attempting to do so will likely destabilize the host process. UI should be opened only in configProc.

If you really must create UI in normally non-interactive entry points, the only safe way to do so is to launch a separate thread and perform all UI operations there. When doing so, you must make sure that this UI never attempts to block on a host thread, such as by calling SendMessage() on a host window, as that may cause a deadlock. If this UI can persist longer than filter instances, you must also hold a reference on the filter DLL so that it isn't unloaded by the OS when the host attempts to unload it.

Leave the floating point settings alone

The precision mode bits in the x87 floating point control word (FPUCW) belong to the application. So do the exception mask bits. And the SSE flush-to-zero (FTZ) and denormals-are-zero (DAZ) bits. Do you see a pattern here? Leave 'em alone and don't attempt to flip them in your filter unless you change them back before returning to the host. In fact, changing some of these bits and calling external code is also a violation of the Win32 calling interface. Don't do it, or at least, do it on a thread that you control.

The most common way that this rule is violated is via the compiler runtime. Microsoft Visual C++ shouldn't cause a problem here, even with /arch:SSE or /arch:SSE2, but older Borland C/C++ and Delphi runtimes attempted to change FPU exception and precision settings on initialization, and some versions of the Intel C/C++ runtime may also attempt to do so depending on compile settings. Make sure these mis-features are disabled in your filter's compilation settings.

The other way to violate is rule by accident is to attempt to initialize Direct3D, which by default changes the x87 FPU precision to 32-bit floating point for the current thread. You can avoid this by initializing Direct3D in a worker thread, which you will probably need to do anyway because of the threading issues noted above, even with D3DCREATE_MULTITHREADED. If you are initializing and shutting down Direct3D within a single call, another way to avoid this problem is to set D3DCREATE_FPU_PRESERVE. Typically the vertex processing load is so low in a video filter that this shouldn't introduce any noticeable performance penalty.

VirtualDub specific: VirtualDub aggressively checks for and corrects any detected violations in the above rules. In many cases, if your filter leaves FPU settings in an incorrect state, it will force them back to the correct values. For certain egregious violations, most notably leaving MMX active, it will also display a warning to the user that the filter is broken.


Deprecated features

The following features are supported in current versions of the video filter API, but may perform inefficiently in newer hosts or may not be supported in future versions. The functionality is described here for completeness and to provide migration advice.

VBitmap rendering methods (V1-V11 only)

The VFBitmap structure exposes a number of methods for processing video images, such as BitBlt(). This is the list of virtual methods exposed, in vtable order:

/// Initialize bitmap parameters for a 16-bit (555), 24-bit, or 32-bit bitmap with the given
/// size and base data pointer. Returns *this.
VBitmap& init(void *data, PixDim w, PixDim h, int depth);

/// Initialize bitmap parameters for a 16-bit (555), 24-bit, or 32-bit bitmap with the given
/// Win32 bitmap header. Only bottom-up bitmaps are supported. Returns *this.
VBitmap& init(void *data, BITMAPINFOHEADER *);

/// Compute a BITMAPINFOHEADER for the current bitmap.
void MakeBitmapHeader(BITMAPINFOHEADER *bih) const;

/// Compute pitch, modulo, and size for scanlines aligned to 4-byte boundaries.
void AlignTo4();

/// Compute pitch, modulo, and size for scanlines aligned to 8-byte boundaries.
void AlignTo8();

/// Blit a rectangle of size (dx, dy) from (x1, y1) in the source bitmap to (x2, y2) in the
/// destination (this) bitmap. Color conversion is allowed.
void BitBlt(PixCoord x2, PixCoord y2, const VBitmap *src,
            PixCoord x1, PixCoord y1, PixDim dx, PixDim dy) const;

/// Blit a rectangle of size (dx, dy) from (x1, y1) in the source bitmap to (x2, y2) in the
/// destination (this) bitmap. Color conversion is allowed. Conversion blits from 24-bit or 32-bit
/// to 16-bit results in a conversion with 4x4 ordered dithering. If to565 is true, a depth
/// value of 16 in the destination indicates 565 instead of 555 channels.
void BitBltDither(PixCoord x2, PixCoord y2, const VBitmap *src,
                  PixDim x1, PixDim y1, PixDim dx, PixDim dy, bool to565) const;

/// Same as BitBlt(), except that a depth value of 16 for the source bitmap is interpreted as
/// 565 instead of 555.
void BitBlt565(PixCoord x2, PixCoord y2, const VBitmap *src, PixDim x1, PixDim y1, PixDim dx, PixDim dy) const;

/// Translate a rectangle of size (dx, dy) from (x1, y1) in the source bitmap to (x2, y2) in
/// the destination (this) bitmap. Both bitmaps must be 32-bit. The red, green, and channels
/// are translated via a 256 byte lookup table pointed to by tbl.
bool BitBltXlat1(PixCoord x2, PixCoord y2, const VBitmap *src,
                 PixCoord x1, PixCoord y1, PixDim dx, PixDim dy, const Pixel8 *tbl) const;

/// Translate a rectangle of size (dx, dy) from (x1, y1) in the source bitmap to (x2, y2) in
/// the destination (this) bitmap. Both bitmaps must be 32-bit. The red, green, and channels
/// are translated via a 256 entry lookup table pointed to by tbl. The red, green, and blue
/// channels of the lookup table are used for the respective channel lookups.
bool BitBltXlat3(PixCoord x2, PixCoord y2, const VBitmap *src,
                 PixCoord x1, PixCoord y1, PixDim dx, PixDim dy, const Pixel32 *tbl) const;

/// Stretch a rectangle (x2, y2)-(x2+dx1, y2+dy1) in the source bitmap to (x1, y1)-(x1+dx, y1+dy)
/// in the destination (this) bitmap using point sampling. Both bitmaps must be 32-bit.
bool StretchBltNearestFast(PixCoord x1, PixCoord y1, PixDim dx, PixDim dy,
                           const VBitmap *src, double x2, double y2, double dx1, double dy1) const;

/// Stretch a rectangle (x2, y2)-(x2+dx1, y2+dy1) in the source bitmap to (x1, y1)-(x1+dx, y1+dy)
/// in the destination (this) bitmap using bilinear filtering. Both bitmaps must be 32-bit. When
/// stretching, pixels are replicated along the border (clamp rule).
bool StretchBltBilinearFast(PixCoord x1, PixCoord y1, PixDim dx, PixDim dy,
                            const VBitmap *src, double x2, double y2, double dx1, double dy1) const;

/// Fill a rectangle in the current bitmap with the given 32-bit color value. The bitmap must be
/// 32-bit.
bool RectFill(PixCoord x1, PixCoord y1, PixDim dx, PixDim dy, Pixel32 c) const;

/// Compute a histogram of the color values within the given rectangle. pHisto points to a 256 entry
/// table of pixel counts; iHistoType must be one of the following constants:
///
///     HISTO_LUMA  (0): Compute histogram of Y = 0.211R + 0.715G + 0.074B luminance values.
///     HISTO_GRAY  (1): Compute histogram of red, green, and blue values sequentially (3x counts).
///     HISTO_RED   (2): Compute histogram of red values.
///     HISTO_GREEN (3): Compute histogram of green values.
///     HISTO_BLUE  (4): Compute histogram of blue values.
///
/// 16-bit (555), 24-bit, and 32-bit bitmaps are supported.
///
bool Histogram(PixCoord x, PixCoord y, PixCoord dx, PixCoord dy, long *pHisto, int iHistoType) const;

Custom VBitmap objects can also be declared in the filter. The memory for the bitmap must be allocated by the filter, but the VBitmap object can be constructed directly. The InitVTables callback is used by the filter to retrieve the VBitmap virtual table (vtbl) pointer, which is then used by the VBitmap constructor to initialize the object.

V12+ only: It is invalid to use non-inline (virtual) methods within VBitmap in filters that declare a high API version of V12 or later. Their definitions have been removed from the standard video filter include file. The VBitmap object does not include the mpPixmap pointer from the enclosing VFBitmap structure, so it is not possible to use any of the methods on a bitmap with a non-RGB format, even in a filter that declares API V11 or lower and normally still has access to the functions.

Using Windows GDI with filter bitmaps (V1-V11 only)

By setting the NEEDS_HDC flag on the dwFlags field of the VFBitmap structure, it is possible to request a GDI display context (DC) for the bitmap, which becomes available in the hdc field during operation. This allows Win32 GDI drawing functions to be used directly on the destination bitmap and BitBlt() operations to be used from the source images.

Because of restrictions in what images can be exposed and the relatively slow operation of GDI drawing primitives compared to other available rendering techniques, it is highly recommended that older filters be migrated away from using GDI for video image processing and any new filters avoid its use entirely. Custom or alternate image processing routines can frequently be an order of magnitude faster than GDI.

V12+ only: It is invalid to request an HDC for any bitmap that has a format other than 32-bit RGB, mainly because GDI does not support non-RGB formats. Use of HDCs will thus prevent alternate format support in your filter.

VirtualDub specific: Starting with versions that support API V12, requesting HDCs in a filter imposes a performance penalty, as the pixmap images must be copied in and out of the GDI bitmap.

ESP omission buffer (V1-V11 only)

When runProc is active, the ArbitraryUserPointer field of the Win32 thread information block (TIB) is defined to point to a special data structure:

struct {
    char data[64];
    void *espsave;
};

This data structure is meant to be used by specially crafted assembly language code to temporarily reuse the stack pointer (ESP) as an eighth general purpose register. The FS selector points to the TIB, so ArbitraryUserPointer is at fs:dword ptr [14h]. Because each thread has its own copy of this structure, a function can be made to be re-entrant and still reuse ESP as follows:

mov     eax,dword ptr fs:[14h]

;[move stack values into thread-specific space]

mov     dword ptr [eax+40h],esp
mov     esp,new_value

...

mov     eax,dword ptr fs:[14h]
mov     esp,dword ptr [eax+40h]

This technique requires specially crafted assembly language code and must be done with extreme caution — if an exception occurs while ESP has been hijacked, the host program will immediately crash silently without warning, with no exception handlers firing.

For new filters, it is recommended that the structured exception handling (SEH) chain pointer be used instead. This avoids conflicting with any other uses the host may have for the ArbitraryUserPointer field in the TIB and works with both old and new hosts. To use the technique, allocate the desired block on the stack and place a dummy SEH chain link at the bottom. Here's a full example:

push    0                       ;set handler in SEH stub to NULL (force invalid)
push    fs:dword ptr [0]        ;set next-link in SEH stub to current SEH chain head
mov     fs:dword ptr [0], esp   ;link stub into SEH chain

...

mov     esp, fs:dword ptr [0]   ;recover stack pointer
mov     eax, [esp]              ;get previous SEH chain head
mov     fs:dword ptr [0], eax   ;pop SEH stub off of SEH chain

Host compatibility warning: Avisynth does not support the fs:[14h] pointer in its VirtualDub filter adapter.

X64 only: The 64-bit (X64) version of the video filter API does not support this feature.

V12+ only: This functionality is no longer supported in future versions of the API. New filters marked as V12+ should not use the fs:[14h] field and may crash if they attempt to do so.


Migrating existing filter source from the old Filter SDK

If you have filters compiled against the old Filter SDK, you need to make a couple of changes to projects to get them to compile with this SDK.

Note: The old headers in the filter SDK are still correct for the API versions they represent, so you don't have to migrate your old filters — you should only consider doing so if there is functionality in this SDK that you wish to take advantage of.

Switching headers

The new header that you want to include is <vd2/plugin/vdvideofilt.h>; put the base include directory in your C++ preprocessor include path. However, this header changes the names of many symbols compared to the old Filter SDK headers, so you likely won't be able to use it directly.

Your best bet for compatibility is to use the migration headers in include/vd2/OldFilterSDK. Add this directory to your include path as well, and chances are your filter will compile. This provides a set of headers which are similar to the layout of the old headers but redirect to symbols defined in the new vdvideofilt.h header. There are a few cases that can still break, most notably calling image processing routines in VBitmap and forward-declaring types.

If you need full backwards compatibility, the old Filter SDK headers have been provided in the extras/FilterSDK folder. It's recommended that you migrate to the newer headers when convenient to gain access to newer API features.

64-bit warning

Before porting your filter to x64, you should port to the new headers first. The old headers contain some declarations which are not correct when compiling for 64-bit and will cause fatal errors in your video filter.


About input drivers

Input driver plugins allow custom file formats to be exposed to VirtualDub. They are combinations of a file detector, file parser, and decoder, which allows great flexibility in supporting varieties of file formats.

General control flow

The first object in an input plugin is the input file driver, which implements the IVDXInputDriver interface and supplies very basic information about the supported file format, such as the human-readable name for the file open dialog and common file extensions. In addition, the driver also handles auto-detection of the file format, which allows the host to detect which plugin should handle a format by either the filename or specific bytes in the file. Each plugin votes based on its priority and its confidence in the match, and either the plugin with the best vote is used to open the file, or an error is reported to the user saying that the file is not supported.

Once identified, the file is passed off to an input file object. Input file objects implement IVDXInputFile interface. The job of an input file object is to parse the input file and expose streams to the host, such as audio and video streams. Although not required, the input file object also often holds the base lock on the file, which prevents other applications from modifying the file while it is in use.

An input file object, in turn, may create one or more stream objects, which implement IVDXStreamSource. Stream objects expose streams of samples, which may be either video samples or audio samples. Generic timing and read services are exposed on sample streams. A stream in the input file may be opened multiple times to create multiple stream objects, allowing multiple parallel reads into the same stream on disk.

Video streams also implement IVDXVideoSource, which exposes additional information about the video stream, such as frame size, frame rate, and frame order. The video samples may be decoded to raw images using IVDXVideoDecoder, which does the actual video decoding, and IVDXVideoDecoderModel, which tracks the required decoding order. Video decoder models can support streams with just key frames, key frames + delta frames (I/P), intra/predicted/bidirectional frames (I/P/B), or even more complex frame dependency patterns.

The net result of all of this is that an input plugin can expose an entirely new file format, along with an entirely new video compression algorithm.


What's new in the input driver API

API version 3 (VirtualDub 1.7.X - preliminary)

  • IVDXStreamSourceV3 has been added to expose additional stream information.
  • True VBR audio can now be exposed via the kFlagVariableSizeSamples flag on the VDXStreamSourceInfoV3 flags field.
  • The plugin can now defer to the host for a video decoder by returning true from CreateVideoDecoder().

API version 2 (VirtualDub 1.7.5)

Many bug fixes to VirtualDub's implementation. It is recommended that both hosts and filters alike target API V2 as the minimum version.

API version 1 (VirtualDub 1.7.4)

First version.


Autodetect

Every input driver exposes methods to support automatic format detection. This allows the host to automatically choose a viable input driver based on the data within a file and the filename. The most appropriate input driver is used to open the file.

Detection by signature

Each input driver has a signature associated with it that is used to perform signature matching against a file. This signature is described in the mpSignature and mSignatureLength fields of the input driver definition and consists of a pair of match/mask bytes. These pairs are applied starting at the beginning of the file. Each pair matches only if the bits in the file byte match those in the match byte, for all bits set in the mask byte. The highest priority signature that matches the file data wins.

The DetectBySignature() method is also used if the input driver definition has the CustomSignature bit set. This method receives the raw header and footer bytes as well as the file size and can do more complex matching. It can also flag a poor match, which wins only if there are no input drivers that find a solid match; this is useful for poorly detectable file formats. A downside to using a custom signature, though, is that it requires your input driver to be loaded in order to check the signature, which isn't required otherwise.

Detection by file name

For file formats that lack no distinctive signature or otherwise conflict with other file formats, a detection by filename is possible. The input driver definition can expose a pipe (|) delimited list of file name patterns to match against; a positive match makes the input driver eligible for handling the file.

A file name match is considered lower priority than a signature match.

Priority

Input drivers also have a priority associated with them. The default priority is zero; a higher priority gives an input driver a higher chance to claim a file than one with a lower priority. This is particularly important when writing an input driver that is meant to replace one of the host input drivers, such as VirtualDub's internal MPEG-1 driver. See the mPriority field of VDXInputDriverDefinition for more details.


Direct mode

Direct mode allows a host to retrieve the raw data and format of a stream and copy it into another stream without requiring decoding or recompression. While greatly restrictive in terms of processing, this is advantageous from performance and quality preservation standpoints when possible.

Exposing stream data directly

To support direct mode, a source must support the GetDirectFormat() and GetDirectFormatLen() methods. These expose the stream formats as either a Windows BITMAPINFOHEADER for video stream sources, or WAVEFORMATEX for audio stream sources. If NULL is returned, then direct mode is not possible for the given stream.


Video frames vs. samples

A video stream consists of video samples, which consist of raw data; these are then converted to frames by the video decoder. In most cases these are synonymous, but when complex frame relations are present — notably bidirectional prediction — the two may need to be different.

It is required that frames be mapped 1:1 to samples and that frame numbers be contiguous starting at zero. A video stream with 500 frames thus has frames numbered from 0 to 499, inclusive. It is also required that samples map 1:1 to frames and also be numbered starting at zero, so the samples will also be numbered 0 to 499, inclusive. However, the samples don't have to have the same order; the GetFrameNumberForSample() and GetSampleNumberForFrame() methods allow for mapping between the two. This allows the samples to be ordered in decoding order instead of presentation order for convenience.


Video decoder model

The video decoder model simulates the video decoder's conversion of samples to decoded frames. It is used by the host to determine ahead of time which samples will be required by the decoder without having to immediately decode the samples.

Operation

Video decoder models implement the IVDXVideoDecoderModel interface. The primary methods that driving the video decoder model operation are SetDesiredFrame(), which selects the frame to decode, and GetNextRequiredSample(), which returns all of the samples required to decode that frame. The video decoder model generally shares data structures with the parent video source in order to implement these methods.

For each sample that is required to decode the frame, the video decoder model updates the video decoder state in the model with the results of decoding that sample, returns the sample number to the caller. It is possible for the host to stop sending samples before the frame is fully decoded — the video decoder model must still reflect the changes to the video decoder up to that point, if it affects the decoding of further frames.

It is important to note that the video decoder model does not receive any sample data itself; it must predict the necessary samples solely by internal information. This is usually determined either by reading an index or doing a prescan on file open. The video decoder model thus allows the host to prefetch video samples before they are needed.

Key frame / delta frame decoding model (I/P frames only)

In a simple delta compression model, frames are composed of either key frames, which are self-contained, or delta frames, which are stored as differences from the previous frame. These are sometimes known as Intra (I) frames and Forward Predicted (P) frames.

If the decoder only has one frame buffer, then the decoding model is simple: decoding a sample updates the frame buffer with the results of that sample. The samples needed to decode a frame are the one for that frame and all samples prior to that, back to either the one after the last decoded frame or the last key frame, whichever is closer. In other words, the needed logic is as follows:

SetDesiredFrame(desired_sample) {
    target_sample = desired_sample;

    // find the nearest key frame
    next_sample = nearest_key(desired_sample);

    // use the last decoded frame if it is closer
    if (last_sample >= next_sample && last_sample <= desired_sample)
        next_sample = last_sample + 1;

    // check if we already have the frame
    if (next_sample > desired_sample)
        next_sample = -1;
}

GetNextRequiredSample() {
    if (next_sample < 0 || next_sample > target_sample)
        return -1;
    return next_sample++;
}

Note that if the desired frame has already been decoded — in that case no samples are required at all, and -1 is returned immediately. This is a special case where the client is required to send a dummy frame in order to inform the decoder to present one of its internally buffered frames.

If there are no delta frames, the decoder model simply requests the sample corresponding to each frame.

This decoder model is sufficiently common that it is provided automatically by the host when the mDecoderModel field of the VDXVideoSourceInfo structure is set to kDecoderModelDefaultIP.

Bug alert
The default I/P model does not work in VirtualDub 1.7.4 (API V1). It does work in 1.7.5 (API V2).

Sources with bidirectionally predicted frames (I/P/B frames)

If the video source contains bidirectional prediction, the decoder model becomes considerably more complicated, because the decoding order is not the same as the presentation order. This is required because each bidirectionally predicted (B) frame requires both a future and a past frame, and thus the future frame must be decoded out of order:

Frame type | I | B | B | P | B | B | P | B | B | P
Display (frame) order | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9
Decoding (sample) order | 0 | 3 | 1 | 2 | 6 | 4 | 5 | 9 | 7 | 8

Whereas a delta-frame decoder is able to use only one or two frame buffers, a B-frame decoder needs to use three buffers: one for the previous I/P frame (forward prediction), one for the next I/P frame (backward prediction), and one for a B-frame predicted between the first two.

The full logic for handling B-frame decoding is too complex to include here, but the basic gist is as follows:

  • If the desired frame is already present in an internal buffer, that internal buffer is presented.
  • For I and P frames, the decoding model requests only samples corresponding to I/P frames, and is the same as the I/P frame model except that it keeps the previous I/P frame as well as the one being decoded.
  • For B frames, the decoding model first requests samples until the previous I/P frame is present, then the next I/P frame, and finally the sample for the desired B frame.


Video decoder

The video decoder is responsible for converting samples into frames.

Interaction with the video decoder model

A video decoder cannot request frames by itself, so it is dependent upon the host providing the necessary samples for each desired frame. It does this by consulting a video decoding model, which simulates the decoder and allows the host to prefetch video samples ahead of time. This means that any internal buffer manipulation in the decoder must match that done by the decoder model, or the two will desynchronize and frame decoding will malfunction.

The video decoder must operate entirely independently of the video decoder model, as video frames can be decoded at different times and on different threads than the decoder model. It is also possible, although not generally required, to use multiple video decoder models with a video decoder or vice versa. Like the decoder model, however, the video decoder is tied to and can share data structures with its parent video source.

Video decoder structure

A video decoder consists of a number of internal buffers and a frame buffer which is exposed to the host. Each decoded sample changes one or more of the internal buffers, one of which is then converted into a decoded frame in the frame buffer. The host need not know the number or structure of the internal buffers, as these are abstracted by the video decoder model.

Frame buffer formats

Video decoders can expose a number of video frame formats to the host, which are exposed via the VDXPixmapFormat enumeration:

XRGB1555 | 16-bit RGB, 5 bits per sample, RGB bitfield ordering
RGB565 | 16-bit RGB, 5 bits per red/blue sample, 6 bits per green sample, RGB bitfield ordering
RGB888 | 24-bit RGB, 8 bits per sample, BGR byte ordering
XRGB8888 | 32-bit RGB, 8 bits per sample, BGR byte ordering followed by unused byte
Y8 | Luma-only (grayscale) encoding, with luma range according to BT.601
YUV422_UYVY | 4:2:2 YCbCr interleaved encoding (half horizontal chroma resolution), Cb/Y1/Cr/Y2 byte order
YUV422_YUYV | 4:2:2 YCbCr interleaved encoding (half horizontal chroma resolution), Y1/Cb/Y2/Cr byte order
YUV444_Planar | 4:4:4 YCbCr planar encoding (full chroma resolution)
YUV422_Planar | 4:2:2 YCbCr planar encoding (half horizontal chroma resolution); chroma samples are coaligned with left luma sample
YUV420_Planar | 4:2:0 YCbCr planar encoding (half horizontal/vertical chroma resolution); chroma samples are midway between left luma samples
YUV411_Planar | 4:1:1 YCbCr planar encoding (quarter horizontal chroma resolution); chroma samples are coaligned with leftmost luma sample
YUV410_Planar | 4:1:0 YCbCr planar encoding (quarter horizontal/vertical chroma resolution)

All YCbCr encodings use the BT.601 color space, with luma having a range of 16-235 and chroma 16-240. A video decoder may support only a small subset of these formats, but it must support at least one, and it is recommended that at least 24-bit RGB (RGB888) be supported. If the video format uses YCbCr internally, as most do, supporting at least one of the 4:2:2 interleaved YCbCr formats (YUV422_UYVY or YUV422_YUYV) is also recommended.

Frame buffer layout linearization

Video decoders expose their frame buffers using the VDXPixmap structure, which describes up to three planes of arbitrary addressing and pitch. For interleaved (chunky) formats, the planes must be aligned to at least the natural alignment of a pixel group (2 bytes for 16-bit RGB formats, 4 bytes for 32-bit RGB and 4:2:2 YCbCr). The pitch describes the memory offset between scanlines and may be greater than the valid data length within a row, in order to accommodate extra padding for alignment, and it may be negative to support bottom-up orientation.

The host can request frame buffer linearization when selecting a format. When this is set, the decoder is required to arrange the planes back-to-back at a known starting address (returned by GetFrameBufferBase()) and in a Windows DIB compatible layout. This means:

  • RGB pixmaps must be stored in a bottom-up orientation, with scanlines aligned and padded to DWORD (4 byte) boundaries.
  • YCbCr planes must be stored back-to-back in top-down orientation without padding between scanlines. The plane order is Y, Cr (V), Cb (U).

Linearization allows the host to pass the data directly to a video codec without requiring an additional memory copy.


The IVDXFilterPreview interface

IVDXFilterPreview is an interface provided by the host that enables display of a preview window that allows the user to instantly preview the results of changing filter parameters. It is accessed via the ifp field of the VDXFilterActivation data structure.


IVDXFilterPreview::Close

Closes the filter preview window.

void Close();

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Remarks

Close() can safely be called even if the preview window is already closed or preview is not enabled.


IVDXFilterPreview::InitButton

Specifies a button to automatically be updated when the filter preview window is opened or closed.

void InitButton(VDXHWND hwndButton);

Parameters

hwndButton Button to update when the filter preview window state changes.

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Remarks

InitButton() reduces the work a filter has to do to support filter preview by having the host automatically change the button label between Hide/Show Preview and to disable it if filter previewing is not enabled.

In cases where the automatic update performed by InitButton() is not suitable, a button callback can be used instead.

VirtualDub specific: Starting with 1.7.0, any keyboard accelerator on the button is preserved if possible.


IVDXFilterPreview::isPreviewEnabled

Queries whether previewing is enabled.

bool isPreviewEnabled();

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Return value

Returns true if a video source is available and previewing is enabled, false otherwise.

Remarks

Note that this function queries whether preview is possible, not whether the preview window is currently displayed.

Filters can be configured when no video source is present. In that case, no live preview is possible.


IVDXFilterPreview::RedoFrame

Refilters and redisplays the current frame.

void RedoFrame();

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Remarks

RedoFrame() is used whenever configuration parameters are changed on the filter and allows the preview to be updated with the new result. Unlike RedoSystem(), RedoFrame() only regenerates frames and should be preferred whenever possible for speed.

V14+ only: RedoFrame() also invalidates all frame caches for the filter being configured and all downstream filters. This prevents any cached frames generated with old filter settings from being displayed.


IVDXFilterPreview::RedoSystem

Reinitializes the filter system and then regenerates and redisplays the current frame.

void RedoSystem();

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Remarks

RedoFrame() is used whenever configuration parameters are changed on the filter and the filter system must be initialized to react to the new parameters. Unlike RedoFrame(), RedoSystem() will work whenever parameters that affect downstream filters are changed, such as frame size or format. Because reinitializing the filter system is expensive, this should only be done when necessary.

V14+ only: RedoSystem() also invalidates all frame caches for the filter being configured and all downstream filters. This prevents any cached frames generated with old filter settings from being displayed.


IVDXFilterPreview::SampleCurrentFrame

Requests the current frame for analysis.

void SampleCurrentFrame();

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Remarks

SampleCurrentFrame() is usually called in response to a user request to display information about the current frame, such as an image histogram. It causes the source frame to be sent to the sampling callback specified by SetSampleCallback().

V12+ only: The frame is sent to the callback in the same format as the filter would receive in its Run() method. This means that the sampling callback must be able to process any source image format that the filter supports.


IVDXFilterPreview::SampleFrames

Requests analysis of the entire source video stream.

void SampleFrames();

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Remarks

SampleFrames() is usually called in response to a user request to display information about the source video, such as an aggregate image histogram. It causes a series of source frames to be sent to the sampling callback specified by SetSampleCallback().

VirtualDub specific: This method causes UI to be displayed that allows the user to select from several modes for sampling the source video. The user can choose to only sample key frames or key frames at an interval to trade off speed for accuracy.


IVDXFilterPreview::SetButtonCallback

Sets the callback issued when the preview window is opened or closed.

void SetButtonCallback(VDXFilterPreviewButtonCallback cb, void *cbdata);

Parameters

cb | Callback function to call when preview window open state changes.
cbdata | Abstract data to be passed to callback function.

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).


IVDXFilterPreview::SetSampleCallback

Sets the callback function invoked when a frame is sampled for analysis.

void SetSampleCallback(VDXFilterPreviewSampleCallback cb, void *cbdata);

Parameters

cb | Callback function to call when a frame is sampled.
cbdata | Abstract data to be passed to callback function.

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).


IVDXFilterPreview::Toggle

Toggles the display of the preview window.

void Toggle(VDXHWND hwnd);

Parameters

hwnd Owner window to use for placement of the preview window.

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Remarks

If the preview window is displayed it is closed. If it is closed and preview is enabled, the preview window is opened and positioned next to the owning window.

Toggle() may be called even if preview is not enabled. In that case, the call is ignored.

The window handle for the filter's configuration dialog should be passed as the owning window. This allows the filter preview window to stay on top and to position itself appropriately.


IVDXFilterPreview::UndoSystem

Deinitializes the filter system in preparation for a complex update.

void UndoSystem();

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Remarks

UndoSystem() is used to temporarily take down the filter system when changes must be made that can't be done while the filter is initialized. The pattern for making such a change is as follows:

  • Call UndoSystem() to stop the filter system.
  • Change filter parameters.
  • Call RedoSystem() to restart the filter system.

In cases where the filter system doesn't need to be stopped before making the changes, it is OK to simply make the changes and call RedoSystem().

If the filter system is not currently running or preview is not enabled, calls to UndoSystem() are ignored.


The IVDXFilterPreview2 interface (V11+ only)

IVDXFilterPreview2 is a slight extension to the IVDXFilterPreview interface in order to add one additional method, IsPreviewDisplayed(). It is possible to emulate this on earlier API versions via the button callback, although it requires more work.


IVDXFilterPreview2::IsPreviewDisplayed

Queries whether the preview window is disabled.

bool IsPreviewDisplayed();

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Return value

Returns true if the preview window is displayed, and false otherwise.

Remarks

If preview is not enabled, IsPreviewDisplayed() returns false.


IVDXPluginCallbacks::SetError

Sets the current error to a formatted string.

void SetError(const char *format, ...);

Parameters

format printf-style format to apply to subsequent arguments to produce the error string.

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Remarks

This allows errors to be returned from certain methods that are able to return errors; not calling any of the set-error methods implies normal return value semantics. It cannot be called from some methods that are defined not to fail.


IVDXPluginCallbacks::SetErrorOutOfMemory

Sets the current error to an out-of-memory error.

void SetErrorOutOfMemory();

Parameters

format printf-style format to apply to subsequent arguments to produce the error string.

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Remarks

This allows an out-of-memory error to be returned from certain methods that are able to return errors; not calling any of the set-error methods implies normal return value semantics. It cannot be called from some methods that are defined not to fail.


IVDXPluginCallbacks::GetCPUFeatureFlags

Retrieves the set of currently enabled CPU instruction set extensions.

uint32 GetCPUFeatureFlags();

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Return value

A bitfield indicating which extensions are active:
kVDXCPUF_MMX | MMX is enabled.
kVDXCPUF_ISSE | The integer portion of Streaming SIMD Extensions (SSE) is enabled. This is the portion of SSE also supported by the original Athlon, including shufw, pinsrw, pextrw, pavgb, pavgw, pmulhuw, movntq, maskmovq, prefetcht0, prefetcht1, prefetcht2, prefetchnta, and sfence.
kVDXCPUF_SSE | Streaming SIMD Extensions (SSE) is available, including floating-point vector instructions.
kVDXCPUF_SSE2 | Streaming SIMD Extensions 2 (SSE2) is available, including 128-bit integer vector instructions.
kVDXCPUF_3DNOW | 3DNow! is available, including 64-bit floating-point instructions.
kVDXCPUF_3DNOW_EXT | 3DNow! Professional is available.
kVDXCPUF_SSE3 | Streaming SIMD Extensions 3 (SSE3) is available, including horizontal add/subtract instructions.
kVDXCPUF_SSSE3 | Supplemental Streaming SIMD Extensions 3 (SSSE3) is available, including absolute value and double-width shift instructions.

Remarks

This function returns the set of currently enabled CPU extensions, which is not always the same as the available extensions. Usually these will be the same, but it is possible for the user to override the enabled CPU extensions.

The enabled set can change between calls. If this is a problem, the value should be cached appropriately.


IVDXAudioSource::GetAudioSourceInfo

Queries the audio source for basic source information.

void GetAudioSourceInfo(VDXAudioSourceInfo& info);

Parameters

info [out] Basic information about this audio source.

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).


IVDXInputDriver::DetectBySignature

Attempts to detect whether a file can be handled by this input handler based on data from the file.

int DetectBySignature(const void *pHeader, sint32 nHeaderSize, const void *pFooter, sint32 nFooterSize, sint64 nFileSize);

Parameters

pHeader | Pointer to bytes from the start of the file available for signature detection.
nHeaderSize | Size of the buffer pointed to by pHeader.
pFooter | Pointer to bytes from the end of the file available for signature detection.
nFooterSize | Size of the buffer pointed to by pFooter.
nFileSize | Size of the file, in bytes.

Thread safety

This method is thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Return value

-1 if the file could not be recognized, 0 if the file match is sketchy, and 1 if the match is definitive.

Remarks

This function is optional and is only called if kFlagCustomSignature is set on the mFlags field of the input driver definition. It is better to use only the signature block in the definition, if possible, because enabling this method requires the driver to be loaded every time a file needs to be tested. If both the signature block and a custom signature are enabled, the test passes only if both tests are successful.


IVDXInputDriver::CreateInputFile

Creates an input file of the type managed by the input driver.

bool CreateInputFile(uint32 flags, IVDXInputFile **ppFile);

Parameters

flags | Flags to customize the input file object behavior.
ppFile | Pointer to variable to receive new input file object.

Thread safety

This method is thread-safe.

Errors

This method may fail with an error (see SetError()).

Return value

True if an input file object was created, or false if out of memory.

Remarks

The input file object should have an outstanding reference on it. VirtualDub will call IVDXUnknown::Release() for this reference when it is finished with the object.


IVDXInputFile::PromptForOptions

Displays a dialog asking the user to set options for interpreting a file and creates an object with the options.

bool PromptForOptions(VDXHWND hwnd, IVDXInputOptions **ppOptions);

Parameters

hwnd | Parent window handle to use for the interactive dialog.
ppOptions | Pointer to variable to receive options object.

Thread safety

This method is not thread-safe.

Errors

This method may fail with an error (see SetError()).

Return value

True if successful, false if out of memory.


IVDXInputFile::CreateOptions

Creates an options object based on previously serialized data from IVDXInputOptions::Write().

bool CreateOptions(const void *buf, uint32 len, IVDXInputOptions **ppOptions);

Parameters

buf | Pointer to buffer with serialized options data.
len | Length of serialized options data, in bytes.
ppOptions | Pointer to variable to receive options object.

Thread safety

This method is not thread-safe.

Errors

This method may fail with an error (see SetError()).

Return value

True if successful, false if out of memory.

Remarks

The serialized data must be from the Write() method of an input options object for this input file type; it is assumed that input options data from different input drivers is not mixed. However, implementations should tag serialized data appopriately to at least allow for detection and handling of serialized data from older and newer versions of the same input driver.


IVDXInputFile::Init

Opens a file with the given options.

void Init(const wchar_t *path, IVDXInputOptions *options);

Parameters

path | Path of file to be opened.
options | Options object with settings to customize the file open. This may be NULL.

Thread safety

This method is not thread-safe.

Errors

This method may fail with an error (see SetError()).

Remarks

This method can only be called once per input file object; to open another file, another input object must be created.

Although usually hosts will detect non-existent file paths beforehand, this method must still handle file-not-found, access denied, and other errors that may occur during the file open.


IVDXInputFile::Append

Appends the streams in the given file to the current streams, forming larger virtual streams.

bool Append(const wchar_t *path);

Parameters

path Path of file to be appended, or NULL to query if appending is supported.

Thread safety

This method is not thread-safe.

Errors

This method may fail with an error (see SetError()).

Return value

If path is non-NULL, return true if append was successful, and false if append was silently aborted (perhaps due to user intervention); if path is NULL, returns true if appending was successful.

Remarks

The appending is virtual; it must not alter the source files on disk.


IVDXInputFile::DisplayInfo

Displays a dialog showing additional information about the file. The contents of this dialog are up to the implementation.

void DisplayInfo(VDXHWND hwndParent);

Parameters

hwndParent Parent window handle to use when creating the dialog.

Thread safety

This method is not thread-safe.

Errors

This method may fail with an error (see SetError()).


IVDXInputFile::GetVideoSource

Opens a stream on an input file.

bool GetVideoSource(int index, IVDXVideoSource **ppVS);

Parameters

index | Index of stream to open, where zero is the first stream, if any. Stream numbers must be contiguous.
ppVS | Pointer to variable to receive video source for the stream.

Thread safety

This method is not thread-safe.

Errors

This method may fail with an error (see SetError()).

Return value

True if successful, false if the stream with the given index does not exist.

Remarks

The video stream object should have an outstanding reference on it. VirtualDub will call IVDXUnknown::Release() for this reference when it is finished with the object.

It is allowed to open a video stream more than once; the returned video source objects must be independent.


IVDXInputFile::GetAudioSource

Opens a stream on an input file.

bool GetAudioSource(int index, IVDXAudioSource **ppAS);

Parameters

index | Index of stream to open, where zero is the first stream, if any. Stream numbers must be contiguous.
ppAS | Pointer to variable to receive audio source for the stream.

Thread safety

This method is not thread-safe.

Errors

This method may fail with an error (see SetError()).

Return value

True if successful, false if the stream with the given index does not exist.

Remarks

The audio stream object should have an outstanding reference on it. VirtualDub will call IVDXUnknown::Release() for this reference when it is finished with the object.

It is allowed to open an audio stream more than once; the returned audio source objects must be independent.


IVDXInputOptions::Write

Writes the contents of the option structure to a block of memory.

uint32 Write(void *buf, uint32 buflen);

Parameters

buf | Buffer to accept serialized data. This can be NULL to determine the size of the required buffer.
buflen | Length of buffer pointed to by buf, in bytes. This must be zero if buf == NULL.

Thread safety

This method is not thread-safe.

Errors

This method may fail with an error (see SetError()).

Return value

If the size of the buffer is at least enough to hold the serialized data, the number of bytes written to buf is returned. Otherwise, the number of bytes required is returned.

Remarks

This function is typically called twice: once to retrieve the required buffer size, and again to actually write the data. The size of the data should always be consistent between calls.


IVDXStreamSource::GetStreamSourceInfo

Obtains basic information about the stream.

void GetStreamSourceInfo(VDXStreamSourceInfo& info);

Parameters

info Object to fill out with stream information.

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).


IVDXStreamSource::Read

Reads samples from the stream.

bool Read(sint64 start, uint32 count, void *buffer, uint32 bufferSize, uint32 *bytesRead, uint32 *samplesRead);

Parameters

start | Sample number to at which to start reading, from 0 to N where N is the number of samples in the stream. Note that the end of the stream is a valid starting point.
count | Maximum number of samples to read.
buffer | Buffer in which to place samples, or NULL to query the size of samples in the stream.
bufferSize | Maximum number of bytes to place into buffer. This parameter is ignored if buffer is NULL.
bytesRead | [out] Number of bytes actually read, or if buffer==NULL, would be read. This pointer cannot be NULL.
sampleRead | [out] Number of samples actually read, or if buffer==NULL, would be read. This pointer cannot be NULL.

Thread safety

This method is not thread-safe.

Errors

This method may fail with an error (see SetError()).

Return value

True if successful, or false if no samples could be read because the buffer is not large enough to hold a single sample.

Remarks

This method is loosely modeled after the AVIStreamRead() function. A common usage pattern is to attempt to read samples with an existing buffer, and to retry the read with a larger buffer if the existing buffer is too small.

The sample count may be reduced below the requested number if it is convenient, or if some samples can be returned immediately. In particular, a video stream implementation can choose to never return more than one sample. The sample count must not be reduced to zero unless start is at the end of the stream.

It is critical that this function never write more than bufferSize bytes into the given buffer. If the buffer is not large enough to hold the requested number of samples, it should automatically reduce the number of samples read and return the actual count in *samplesRead. If the buffer cannot hold even the first sample, then Read() should return false, set *samplesRead to the number of samples that would have been read, and *bytesRead to the number of bytes required for those samples. In all cases, if *samplesRead > 0, then *bytesRead is at least as large as the first sample.

If start is at the end of the stream, then this function should return zero bytes/samples and true as the return value.

Implementations must not rely on any particular address alignment for the supplied buffer.


IVDXStreamSource::GetDirectFormat

Obtains the direct format for the stream, if present.

const void *GetDirectFormat();

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Return value

A pointer to the direct format structure, or NULL if direct stream copy is not supported. The direct format is a VDXBITMAPINFOHEADER for a video stream and VDXWAVEFORMATEX for an audio stream.

Remarks

If an implementation wishes to support the direct stream copy feature, it must return a non-null direct format. This is the format that is used for the output stream. The pointer must remain valid for the life of the stream source.


IVDXStreamSource::GetDirectFormatLen

Obtains the size of the direct format structure for the stream, if present.

int GetDirectFormatLen();

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Return value

The length of the direct format structure, in bytes, or zero if one is not present.


IVDXStreamSource::GetDecodeErrorMode

Queries the stream source for the current error handling strategy.

ErrorMode GetDecodeErrorMode();

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Return value

The current error handling mode for this stream source.


IVDXStreamSource::SetDecodeErrorMode

Sets the error handling strategy for the stream source.

void SetDecodeErrorMode(ErrorMode mode);

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Remarks

The requested error mode must be one for which IsDecodeErrorModeSupported() returns true.


IVDXStreamSource::IsDecodeErrorModeSupported

Queries whether the stream source supports a particular error handling strategy.

bool IsDecodeErrorModeSupported(ErrorMode mode);

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Return value

Returns true if the given error handling mode is supported.

Remarks

At least one error handling mode must be supported.


IVDXStreamSource::IsVBR

Queries whether a stream has a variable relationship of samples to time.

bool IsVBR();

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Return value

True if the sample-to-time ratio is variable, or false if it is constant.

Remarks

When true, this implies a non-linear relationship between samples and time; the TimeToPositionVBR() and PositionToTimeVBR() methods give the correct translation.

This is used to support VBR in audio streams. It is ignored for video streams.


IVDXStreamSource::TimeToPositionVBR

Converts a timestamp to a sample number.

sint64 TimeToPositionVBR(sint64 us);

Parameters

us Timestamp in microseconds.

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Return value

The number of the nearest sample.

Remarks

This function is only used if IsVBR() returns true. Otherwise, the host automatically assumes a linear relationship between position and time and uses round(us / 1000000.0 * rate / scale) instead.


IVDXStreamSource::PositionToTimeVBR

Converts a sample number to a timestamp.

sint64 PositionToTimeVBR(sint64 sample);

Parameters

sample Sample number.

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Return value

The timestamp of the start of the sample, in microseconds.

Remarks

This function is only used if IsVBR() returns true. Otherwise, the host automatically assumes a linear relationship between position and time and uses round(sample / rate * scale * 1000000.0) instead.


IVDXStreamSourceV3::GetStreamSourceInfoV3

Obtains additional information about the stream.

void GetStreamSourceInfoV3(VDXStreamSourceInfo& info);

Parameters

info Object to fill out with stream information.

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

API version requirement

Requires input driver API V3 or later.

Remarks

GetStreamSourceInfoV3 reports additional information about the stream over what is reported in API V2, such as a suggested handler FOURCC to use when searching for codecs, and whether the stream uses variable size samples or not (mainly important for full VBR audio support).


IVDXVideoDecoder::DecodeFrame

Decodes a sample, possibly producing a frame in the current framebuffer.

const void *DecodeFrame(const void *inputBuffer, uint32 data_len, bool is_preroll, sint64 sampleNumber, sint64 targetSample);

Parameters

inputBuffer | Pointer to the sample data. This may be NULL if data_len is zero; otherwise, it must point to a DWORD-aligned buffer with at least data_len bytes, followed by the number of padding bytes indicated by GetDecodePadding().
data_len | The number of bytes in the sample data.
is_preroll | True if the sample should be decoded to internal buffers only; false if the image data should also be decoded to the framebuffer.
sampleNumber | The number of the sample provided for decoding, or -1 if no sample is provided.
targetSample | The sample number of the frame being targeted by the sequence of samples being decoded.

Thread safety

This method is not thread-safe.

Errors

This method may fail with an error (see SetError()).

Return value

If is_preroll is false, a pointer to the framebuffer base (see GetFrameBuffer() for details).

Remarks

In general, the host will call GetNextRequiredSample(), read the required samples, and feed the samples one at a time to this routine until the frame is decoded; is_preroll will be true except for the last sample. Note that it is possible for the host to stop sending samples and start decoding toward another frame, so the decoder must have well-defined state after each sample, although possibly non-optimal.

A call to DecodeFrame() with inputBuffer == NULL, data_len == 0, is_preroll == false, and sampleNumber == -1 indicates a request to decode a frame that is already resident in an internal buffer. The host will issue a call like this when the video decoder model reports no samples are required to decode a frame. This is mainly required for decoders that support B frames, where a frame may need to be re-converted from an internal framebuffer. It can also be used to support caching of expensive frames in I and I/P decoding models, however.

Decoders must not read beyond the end of the buffer. The GetDecodePadding() method allows the decoder to request additional padding at the end of the buffer to ease decoding; this extra space is not initialized to any particular value and is not included in data_len. Reading beyond data_len + GetDecodePadding() is an extremely bad idea and will result in random crashes.


IVDXVideoDecoder::GetDecodePadding

Returns the amount of padding at the end of samples required by the decoder.

uint32 GetDecodePadding();

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Return value

The number of bytes of padding to be added to the sample buffer beyond the data length. Generally, this should be a small number, around 16 bytes or less.

Remarks

Padding at the end of a sample makes it easier for some types of decoders, especially those that deal with bit encodings. For instance, it is common for decoders that read variable length codes (VLC) to read up to three bytes beyond the end of the coded stream. A small amount of padding can make such decoders more efficient by not requiring onerous bounds checking code. For instance, a 1024 byte sample would arrive at the bottom of a 1028 byte buffer if GetDecodePadding() returned 4. However, some checking must still be implemented to avoid crashing on bad data.

Every sample that is buffered in the host is enlarged by this amount, so it is best not to request large amounts of padding. Requesting 1K of padding for each sample, for instance, could result in a substantial amount of extra memory being allocated.

The amount of padding must always be the same for a particular decoder object.


IVDXVideoDecoder::Reset

Discards all cached frame data from the decoder.

void Reset();

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Remarks

This must produce the same effects to the decoder model as IVDXVideoDecoderModel::Reset().


IVDXVideoDecoder::IsFrameBufferValid

True if the frame buffer in the decoder holds a valid decoded frame.

bool IsFrameBufferValid();

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).


IVDXVideoDecoder::GetFrameBuffer

Returns a pointer to the raw frame buffer.

const void *GetFrameBuffer();

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Return value

A pointer to the base of the frame buffer.

Remarks

The returned pointer may be different from the data pointer returned by GetTargetFormat() if the image is stored in bottom-up orientation.


IVDXVideoDecoder::SetTargetFormat

Sets the desired image format for decoded frames.

bool SetTargetFormat(int format, bool useDIBAlignment);

Parameters

format | Desired format for decoded frame. This must be one of the values from the VDXPixmapFormat enumeration. If it is zero (unknown), the default format is selected.
useDIBAlignment | Set if the frame buffer should conform to Windows DIB layout. This is used to avoid extra memory copies when a DIB is required. For RGB formats, the bitmap should be stored bottom-up with scanlines aligned to doubleword boundaries, i.e. pitch = -4*ceil(width * bitdepth / 32). For YCbCr formats, the bitmap should be stored top-down with scanlines having byte alignment, i.e. pitch = ceil(width * bitdepth / 8).

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Return value

True if the format was successfully changed, or false if the format is not supported.

Remarks

After SetTargetFormat() succeeds, the decoder state is reset as if Reset() were called.

SetTargetFormat(0) allows the decoder to select the most appropriate format for decoding, usually the one that is closest to the native decoded format of the compressed stream. A SetTargetFormat(0) call should never fail, because that would mean no formats are supported for decoding (which would be useless).


IVDXVideoDecoder::SetDecompressedFormat

Sets the desired image format for decoded frames using a custom Windows DIB format.

bool SetDecompressedFormat(const VDXBITMAPINFOHEADER *pbih);

Parameters

pbih Desired format for decoded frame.

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Return value

True if the format was successfully changed, or false if the format is not supported.

Remarks

After SetDecompressedFormat() succeeds, the decoder state is reset as if Reset() were called.

This function is never called with any bitmap formats already covered by the VDXPixmapFormat enumeration. In particular, it is OK for a decoder to always return false from SetDecompressedFormat() if no special formats are supported for decoding.


IVDXVideoDecoder::IsDecodable

Queries whether a frame can be decoded with no more than one additional sample from the current decoder state.

bool IsDecodable(sint64 sample_num);

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Return value

True if the frame can be decoded with zero or one additional sample.


IVDXVideoDecoderModel::Reset

Resets the decoder model back to an empty decoder state.

void Reset();

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Remarks

Reset() empties any cached frame data from the modeled decoder, so any subsequent frame decode sequences start from scratch.


IVDXVideoDecoderModel::SetDesiredFrame

Selects the frame to decode. GetNextRequiredSample() is then called repeatedly to determine the required dependencies for this frame.

void SetDesiredFrame(sint64 sample_num);

Parameters

sample_num The sample number of the desired frame.

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Remarks

While this function selects a frame for decoding, it takes a sample number. This is usually easier to handle in the decoder model, as the decoder model typically needs to look up dependencies by decode order. Since the sample number usually matches decoding order, this saves a conversion from frame order to sample order.


IVDXVideoDecoderModel::GetNextRequiredSample

Returns the next source sample required to decode the current target frame, or -1 if no more samples are required.

sint64 GetNextRequiredSample(bool& is_preroll);

Parameters

is_preroll [Out] Set to true if the sample is only required for internal decoding, or false if the sample is decoded to the final frame.

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Remarks

SetDesiredFrame() must be called before GetNextRequiredSample(). If this method immediately returns -1 after the target frame is set, it means that the frame is already resident in an internal decoder buffer, in which case the client must call IVDXVideoDecoder::DecodeFrame() with a zero-byte frame to decode the frame.


IVDXVideoDecoderModel::GetRequiredCount

Returns the number of source samples required to decode the desired frame, based on the current decoder state.

int GetRequiredCount();

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Remarks

A return value of zero is possible if the frame is already resident in an internal frame buffer.

This is used in smart rendering mode by the encoding engine to determine if a frame can be copied in Direct mode or if the engine must drop back to full processing mode.


IVDXVideoSource::GetVideoSourceInfo

Obtains a basic description of the video source.

void GetVideoSourceInfo(VDXVideoSourceInfo& info);

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Remarks

The description returned by GetVideoSourceInfo() should always be constant for a particular IVDXVideoSource instance; the frame rate or resolution should not change between calls, for instance.


IVDXVideoSource::CreateVideoDecoderModel

Creates a new video decoder model object for this video source.

bool CreateVideoDecoderModel(IVDXVideoDecoderModel **ppDecoderModel);

Parameters

ppDecoderModel Pointer to location to receive new decoder model object pointer.

Thread safety

This method is not thread-safe.

Errors

This method may fail with an error (see SetError()).

Return value

True if creation was successful, false for failure.

Remarks

It is possible for multiple video decoder models to be created from the same video source; implementations must not return the same decoder model object when the host requests two simultaneous decoder models.

The decoder model should have one outstanding reference on it; the host will call IVDXUnknown::Release() when it is finished with the object.


IVDXVideoSource::CreateVideoDecoder

Creates a new video decoder object for this video source.

bool CreateVideoDecoder(IVDXVideoDecoder **ppDecoder);

Parameters

ppDecoderModel Pointer to location to receive new decoder object pointer.

Thread safety

This method is not thread-safe.

Errors

This method may fail with an error (see SetError()).

Return value

True if creation was successful, false for failure.

Remarks

It is possible for multiple video decoders to be created from the same video source; implementations must not return the same decoder object when the host requests two simultaneous decoders.

The decoder should have one outstanding reference on it; the host will call IVDXUnknown::Release() when it is finished with the object.

V3+ only: Returning false from this method is no longer always a fatal error. The host may elect to use the format returned by GetDirectFormat to supply a default decoder.


IVDXVideoSource::GetSampleInfo

Retrieves information about a specific sample.

void GetSampleInfo(sint64 sample_num, VDXVideoFrameInfo& frameInfo);

Parameters

sample_num | Number of sample for which information is being queried.
frameInfo | Object in which to place information about the sample.

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Remarks

The information for a sample must be consistent; it may not change between calls to GetSampleInfo() for the same sample on the same video source instance.


IVDXVideoSource::IsKey

Returns true if the sample corresponds to a key frame, false otherwise.

bool IsKey(sint64 sample_num);

Parameters

sample_num Number of sample being queried.

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Remarks

The result of IsKey() should be consistent between calls for the same sample, and with the information returned by GetSampleInfo().

For a video source that follows the traditional AVI predictive model, this function returns true for a key frame and false for a delta frame or null frame. The first frame in the source should always be a key frame.

Video sources that use bidirectional prediction, such as MPEG-1/2, should return true for an I frame and false for P and B frames.

When using the default I/P decoder model, IsKey() is called frequently.


IVDXVideoSource::GetFrameNumberForSample

Converts a frame number to the corresponding sample number.

sint64 GetFrameNumberForSample(sint64 sample_num);

Parameters

sample_num Sample number to convert to a frame number. This is always in the range of 0 to N-1, where N is the number of frames in the video source.

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Return value

The frame number of the frame decoded by that sample. This must be in the range of 0 to N-1, where N is the number of frames in the video source.

Remarks

This mapping must be reversible with GetSampleNumberForFrame() — that is, GetFrameNumberForSample(GetSampleNumberForFrame(frame)) == frame in all cases.

When using the default I/P decoder model, this function must return sample_num directly.


IVDXVideoSource::GetSampleNumberForFrame

Converts a sample number to the corresponding frame number.

sint64 GetSampleNumberForFrame(sint64 frame_num);

Parameters

sample_num Frame number to convert to a sample number. This is always in the range of 0 to N-1, where N is the number of frames in the video source.

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Return value

The sample number of the corresponding frame. This must be in the range of 0 to N-1, where N is the number of frames in the video source.

Remarks

This mapping must be reversible with GetFrameNumberForSample() — that is, GetFrameNumberForSample(GetSampleNumberForFrame(frame)) == frame in all cases.

When using the default I/P decoder model, this function must return sample_num directly.


IVDXVideoSource::GetRealFrame

Returns the actual frame being decoded for a given frame in the source.

sint64 GetRealFrame(sint64 frame_num);

Parameters

frame_num Number of frame being queried.

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Return value

The frame number of the actual frame that is decoded.

Remarks

This routine is used to determine when multiple frame entries in the video source are being supported by a single actual frame, such as due to covering a dropout or due to frame duplication. It is used in Direct Stream Copy mode to determine when a frame can be omitted. GetRealFrame(frame_num) != frame_num implies that a particular frame is supported by a dummy sample which can be omitted during copying. For instance, if GetRealFrame() returns 2 for frames 2-5, the underlying samples for frames 3-5 can be replaced with null frames or deleted without disturbing decoding.


IVDXVideoSource::GetSampleBytePosition

Returns the approximate starting byte position of the sample in the underlying storage medium.

sint64 GetSampleBytePosition(sint64 sample_num);

Parameters

sample_num Sample to query.

Thread safety

This method is not thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Return value

Starting byte position of the sample data, or -1 if the byte position is not available.

Remarks

This is used by the UI when the timeline display is customized to display the position of samples with the %B formatting specifier. It is OK if an implementation returns -1 for all samples, in which case N/A is displayed.


IVDXUnknown::AddRef

Increments the reference count on an object.

int AddRef();

Thread safety

This method is thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Return value

A non-zero positive value indicating the new reference count on the object.

Remarks

The IVDXUnknown::AddRef() method is very similar to the IUnknown::AddRef() method in the Microsoft Component Object Model (COM). It is used to obtain an independent strong reference on the object, such that the object is not destroyed until all strong references on the object are gone. Each call to AddRef() on an object must have a matching call to Release().

As AddRef() may be called more widely than the other methods on an object, initialization or other types of nontrivial processing must not be done in AddRef(). AddRef() must not call into any callbacks within the host, nor may it fail.

It is vitally important that the reference count be maintained in a thread-safe manner, as AddRef() and Release() may be called from multiple threads even on an object whose functional methods are not thread-safe. The simplest way to do this is through the InterlockedIncrement() and InterlockedDecrement() methods in the Win32 Platform SDK:

int MyClass::AddRef() {
    return (int)InterlockedIncrement(&mRefCount);
}

In Visual C++, the _InterlockedIncrement() and _InterlockedDecrement() intrinsics may be used.


IVDXUnknown::AsInterface

Obtains interfaces from an object.

void *AsInterface(uint32 iid);

Parameters

iid 32-bit interface ID of the desired interface.

Thread safety

This method is thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Return value

A pointer to the desired interface, or NULL if the interface is not supported.

Remarks

AsInterface() is used to obtain alternate interfaces on an object, where the interface ID for an interface is typically a kIID member. This method must obey the following rules:

  • IVDXUnknown must always be available.
  • All IVDXUnknown interfaces on an object are equivalent; AddRef() on one and Release() on another is valid.
  • All interfaces must be available from all other interfaces.
  • The set of interfaces available on an object is always constant; interfaces cannot be added or removed on the fly.

This method is similar to the Microsoft COM IUnknown::QueryInterface(), but unlike that method, AsInterface() does not add a reference to the object.

It is OK to expose interfaces for a higher API version than the one in use. For instance, a filter that supports a V3 interface does not have to hide it if the host only supports V2. However, any interfaces that are exposed must work as documented — if a host advertises a V2 API but exposes a V3 interface, the V3-specific features in the interface must work. This allows a host to selectively expose V3 features.


IVDXUnknown::Release

Decrements the reference count on an object and destroys it if the reference count is zero.

int Release();

Thread safety

This method is thread-safe.

Errors

Errors may not be returned from this function (see SetError()).

Return value

A non-negative value indicating the new reference count on the object. A return value of zero indicates that the object was destroyed.

Remarks

The IVDXUnknown::Release() method is very similar to the IUnknown::Release() method in the Microsoft Component Object Model (COM). It is used to drop an independent strong reference on the object; the object is automatically destroyed when all strong references are gone.

As Release() may be called more widely than the other methods on an object, initialization or other types of nontrivial processing must not be done in Release(). Release() must not call into any callbacks within the host, nor may it fail.

It is vitally important that the reference count be maintained in a thread-safe manner, as AddRef() and Release() may be called from multiple threads even on an object whose functional methods are not thread-safe. The simplest way to do this is through the InterlockedIncrement() and InterlockedDecrement() methods in the Win32 Platform SDK:

int MyClass::Release() {
    int newRefCount = (int)InterlockedDecrement(&mRefCount);
    if (!newRefCount)
        delete this;
    return newRefCount;
}

In Visual C++, the _InterlockedIncrement() and _InterlockedDecrement() intrinsics may be used.

An implementation of Release() must not read an internal reference count variable more than once, as this can lead to failures. The following implementation is incorrect:

int MyClass::Release() {
    int newRefCount = (int)InterlockedDecrement(&mRefCount);

    // This is incorrect, as two threads may see mRefCount go to zero, causing the
    // object to be destroyed twice.
    if (!mRefCount)
        delete this;

    // This is incorrect, as the reference count variable may be read after the enclosing
    // object is destroyed.
    return mRefCount;
}


configProc

Opens up a Windows GUI dialog box to allow the user to configure the filter's settings.

int configProc(VDXFilterActivation *fa, const VDXFilterFunctions *ff, HWND hWnd);

Parameters

fa | Pointer to filter activation structure.
ff | Pointer to callback function structure.
hwnd | Handle of parent window to use for dialog.

Thread safety

This method is not thread-safe.

Exceptions

This function must not throw exceptions (see Except()).

Return value

Zero if configuration is successful, non-zero on error or if the user canceled configuration.

Remarks

If you do not have a configuration dialog, omit configProc. VirtualDub attempts to open a filter's configuration dialog when a filter is first added, and if configProc returns non-zero, the filter is not added to the chain.

Generally, configProc is implemented as a call to DialogBoxParam() with the given parent window handle. Remember, VirtualDub starts the process, so you will need to use your filter's cached module handle to access dialog box resources.

Any settings exposed here should also be replicated in the filter's scripting interface, so that the filter works properly in batch mode and in Avisynth.

Please make sure your dialog fits in a 640x480 screen -- some video cards with integrated capture do not work unless the desktop resolution is lowered, and so your filter may still be used on a system with a 640x480 or 800x600 desktop.


copyProc

Creates a clone of the filter instance.

int copyProc(VDXFilterActivation *fa, const VDXFilterFunctions *ff, void *dst);

Parameters

fa | Pointer to filter activation structure.
ff | Pointer to callback function structure.
dst | Memory location for new instance data.

Thread safety

This method is not thread-safe.

Exceptions

Exceptions may be thrown from this function (see Except()).

Default behavior

If omitted, the instance data is copied verbatim.

Remarks

This function allows you to implement a copy constructor for your filter, which then allows you to store pointers to allocated and destructable objects in your filter structure. To do this, you must tie copyProc to your class's copy constructor, initProc to your regular constructor, and deinitProc to your destructor. See using a class object for your filter for more details.


copyProc2

Creates a clone of the filter instance.

void copyProc2(VDXFilterActivation *fa, const VDXFilterFunctions *ff, void *dst, VDXFilterActivation *fa2, const VDXFilterFunctions *ff2);

Parameters

fa | Pointer to filter activation structure for current instance.
ff | Pointer to callback function structure for current instance.
dst | Memory location for new instance data.
fa2 | Pointer to filter activation structure for new instance.
ff2 | Pointer to callback function structure for new instance.

Thread safety

This method is not thread-safe.

Exceptions

Exceptions may be thrown from this function (see Except()).

Default behavior

If omitted, a byte-by-byte memcpy() is done on the instance data.

Remarks

copyProc2 is similar to copyProc, but it includes the activation and function array pointers for the new instance. This fixes an annoyance with the copyProc function where the new activation structure is unknown, so it must be set on all calls that can come next. This is not necessary if copyProc2 is used.

Note that copyProc2 is only available on API V14, so if your filter works on an earlier version, you must still deal with copyProc and its restrictions.


deinitProc

Cleans up per-instance data for a filter.

void deinitProc(VDXFilterActivation *fa, const VDXFilterFunctions *ff);

Parameters

fa | Pointer to filter activation structure.
ff | Pointer to callback function structure.

Thread safety

This method is not thread-safe.

Exceptions

This function must not throw exceptions (see Except()).

Default behavior

If omitted, no default action is taken.

Remarks

This function is called whenever a filter instance is about to be destroyed, and gives the filter a chance to dispose of persistent filter data. Simple filters will not need this function.


endProc

Cleans up filter instance resources when an operation ends.

void endProc(VDXFilterActivation *fa, const VDXFilterFunctions *ff);

Parameters

fa | Pointer to filter activation structure.
ff | Pointer to callback function structure.

Thread safety

This method is not thread-safe.

Exceptions

This function must not throw exceptions (see Except()).

Default behavior

If omitted, no default action is taken.

Remarks

endProc is called for all filters, even if the filter initialization process is aborted. This means that a filter may have its endProc called but not startProc, so endProc should check if a resource has been allocated and mark it as unallocated after it is freed:

            if (mfd->resource) {
                free(mfd->resource);
                mfd->resource = NULL;
            }


eventProc

Invoked to notify the filter of out of band events during rendering.

bool eventProc(const VDXFilterActivation *fa, const VDXFilterFunctions *ff, uint32 event, void *eventData);

Parameters

fa | Pointer to filter activation structure.
ff | Pointer to callback function structure.
event | ID of event.
eventData | Optional data structure for event. The presence and type of this data is different for each event type.

Thread safety

This method is not thread-safe.

Exceptions

This function must not throw exceptions (see Except()).

Return value

True if the event was recognized and handled, or false otherwise. A host may elect not to send further events of a type if a filter indicates that it does not handle that event type.

API version requirement

Requires V14 or later.

Default behavior

If omitted, no default action is taken.

Remarks

Event IDs use kVDXVFEvent_ as their symbol prefix. The events that can be delivered are as follows:

InvalidateCaches A change has occurred in an upstream filter that requires caches to be flushed on downstream filters. This generally occurs when upstream filter parameters are tweaked through the UI. Invalidation events are used because they are faster than reinitializing the filter chain.


fssProc

Generates a script command to configure the filter to its current state.

void fssProc(VDXFilterActivation *fa, const VDXFilterFunctions *ff, char *buf, int buflen);

Parameters

fa | Pointer to filter activation structure.
ff | Pointer to callback function structure.
buf | Pointer to buffer to receive configuration command.
buflen | Maximum number of bytes to write to the buffer, including any null terminator.

Thread safety

This method is not thread-safe.

Exceptions

This function must not throw exceptions (see Except()).

Return value

True if the string should be included, false otherwise.

Default behavior

If omitted, no default action is taken. This means that the filters settings would not be restored in a script job.

Remarks

This function is the only method by which filters can store configuration parameters between the time a job entry is saved, and when that entry is executed.The string returned by fssProc is appended onto a configuration string of the form:

VirtualDub.video.filters.instance[0].

and thus the filter provides only the method call part of the string. Use _snprintf() or _vsnprintf() to generate the string, in case you might overflow the buffer. If you return false from the function, VirtualDub simply omits the script line entirely. Note that VirtualDub only has a 512 character buffer when executing script lines, so you can only count on 450 characters of space for your string even though there is usually 2K or so available to this function.


initProc

Initializes a filter's per-instance data.

int initProc(VDXFilterActivation *fa, const VDXFilterFunctions *ff);

Parameters

fa | Pointer to filter activation structure.
ff | Pointer to callback function structure.

Thread safety

This method is not thread-safe.

Exceptions

Exceptions may be thrown from this function (see Except()).

Return value

Zero on success, non-zero on failure.

Default behavior

If omitted, no default action is taken.

Remarks

This function is called whenever a filter instance is created, and is usually used to initialize variables in the private data structure. Regardless of whether it is provided, VirtualDub will initialize the data structure to zeroes, so most filters will not need this function.


paramProc

Initializes a filter's per-instance data.

long paramProc(VDXFilterActivation *fa, const VDXFilterFunctions *ff);

Parameters

fa | Pointer to filter activation structure.
ff | Pointer to callback function structure.

Thread safety

This method is not thread-safe.

Exceptions

This function must not throw exceptions (see Except()).

Return value

A bitfield indicating filter behavior:
FILTERPARAM_SWAP_BUFFERS
The input and output buffers must be distinct. If this flag is absent, they overlap (in-place mode).
FILTERPARAM_NEEDS_LAST
Enables a third buffer containing the previous input frame to the filter.
FILTERPARAM_HAS_LAG(count)
V7+ only: Indicates that the filter's output lags behind its input by the specified number of frames.
FILTERPARAM_SUPPORTS_ALTFORMATS
V12+ only: The filter supports formats other than 32-bit RGB.
FILTERPARAM_NOT_SUPPORTED
V12+ only: The filter does not support the given input format. Note that this flag should not be combined with any other values, as it sets all bits.
FILTERPARAM_ALIGN_SCANLINES
V14+ only: All source and output scanlines are guaranteed to be aligned to 16-byte boundaries and to be readable/writable to the next 16-byte boundary. This is useful for fast SSE optimized routines.
FILTERPARAM_PURE_TRANSFORM
V14+ only: The filter has no side effects and produces output that is solely dependent upon the source frame data and configuration parameters. In particular, running the filter on the same source frames will produce the same output, even if the filter is run multiple times to produce the same output frame, and even if the output frame numbers are different. This allows the host to deduce when two output frames would be the same and optimize the output appropriately.

Default behavior

If omitted, a return value of FILTERPARAM_SWAP_BUFFERS is assumed.

Remarks

paramProc can be called relatively frequently when the user is arranging filter order, so it should not take a long time to execute. The function should also be robust to improper frame sizes, such as 1x2 bitmaps -- frame size validation should be handled in startProc instead.

Be careful when setting fa->dst.offset. In a two-buffer filter, the destination buffer offset should be set to zero, unless the filter creates a larger image and returns a subrectangle of it. However, in an in-place filter, the destination offset must be at least as much as the source offset, and if the destination image is the same size, it must be exactly the same. Setting it to zero in an in-place filter can result in x1/y2 cropping not working correctly in VirtualDub's filter chain.

V12+ only: Setting the depth field of the destination bitmap to zero activates the mpPixmapLayout field and allows formats other than 32-bit RGB.

V12+ only: Setting the mpPixmapLayout->pitch field to zero tells the host to compute appropriate pitch and offset parameters. In this case, only the width, height, and format fields are used.


prefetchProc

Retrieves the input frame number required to produce a given output frame.

sint64 prefetchProc(const VDXFilterActivation *fa, const VDXFilterFunctions *ff, sint64 frame);

Parameters

fa | Pointer to filter activation structure.
ff | Pointer to callback function structure.

Thread safety

This method is thread-safe.

Exceptions

This function must not throw exceptions (see Except()).

Return value

The source frame required for the given output frame.

API version requirement

Requires V12 or later.

Default behavior

If omitted, the source frame is automatically chosen based on the output frame timestamp. This makes the frame numbers equal if the filter does not change the video frame rate.

Remarks

The purpose of prefetchProc is to allow the filter to direct the host in how to prefetch source frames for processing by a filter. For each output frame, prefetchProc specifies which source frame is needed to produce that output frame. This is similar to what would be passed to a GetFrame() function except that the actual frame fetching is asynchronous. By default, when prefetchProc is omitted, the host assumes that the source frame with the nearest timestamp to that of the output frame should be used.

prefetchProc is used to implement frame reordering, particularly in the case where the frame rate is modified by the filter. In most cases the default behavior suffices, but for some filters, particularly those that process fields, prefetchProc should be overridden to ensure exact frame timing.

There are some big gotchas with implementing this function:

  • Unlike most callbacks, prefetchProc must be implemented as thread-safe. Prefetching can occur at any time during execution and in parallel to frame processing, in particular runProc. Any mutable data that is accessed, like a cache, must be properly synchronized with a mutex or other mutual exclusion mechanism.
  • Prefetching can't depend on frame processing. For instance, a filter can't analyze previous frames in order to determine the next frame to fetch.
  • The prefetch pattern must be constant. Between startProc and endProc, a particular output frame must always map to the same source frame, no matter how many times prefetchProc is called and in what order.
  • Unused, extra prefetches can occur. A filter must not rely on a prefetch resulting in a call to runProc on the prefetched frame.

Audio is not reordered along with the frames, so this is not intended to allow for editing.


prefetchProc2

Lists the input frames required to produce a given output frame.

bool prefetchProc2(const VDXFilterActivation *fa, const VDXFilterFunctions *ff, sint64 frame, IVDXVideoPrefetcher *prefetcher);

Parameters

fa | Pointer to filter activation structure.
ff | Pointer to callback function structure.
frame | Output frame for which to determine inputs.

Thread safety

This method is thread-safe.

Exceptions

This function must not throw exceptions (see Except()).

Return value

True if handled, false otherwise.

API version requirement

Requires V14 or later.

Default behavior

If omitted or not handled, the source frame is automatically chosen based on the output frame timestamp. This makes the frame numbers equal if the filter does not change the video frame rate.

Remarks

prefetchProc2 is similar to prefetchProc, but with expanded capabilities via the prefetcher argument. These include:

  • Prefetching more than one source frame.
  • Associating an arbitrary 64-bit value (a cookie) with each source prefetch.
  • Specifying a direct association between a source frame and the filter's output, allowing the filter to be bypassed entirely for that frame.


runProc

Processes a video frame.

int runProc(const VDXFilterActivation *fa, const VDXFilterFunctions *ff);

Parameters

fa | Pointer to filter activation structure.
ff | Pointer to callback function structure.

Thread safety

This method is not thread-safe.

Exceptions

Exceptions may be thrown from this function (see Except()).

Return value

Zero if successful, non-zero on unknown failure.

Remarks

The input frame is found in fa->src, and the output frame is fa->dst. Neither structure should be modified by runProc. If your filter is an in-place filter, the source and destination structures are both aliased to the same buffer.

Of all the functions in a filter, this function needs to run as quickly as possible and should be the most heavily optimized of all functions. Frame-invariant computation, such as table generation, should be done in startProc or initProc instead.

Note: Do not modify your source buffer if your filter is not in-place. In some cases, VirtualDub will need to access your source buffer after your filter has already run.

Note: Your output buffer is not persistent -- if an in-place filter follows, it will work directly in the buffer. This means, unfortunately, that recursive filters will need an additional side buffer.


startProc

Prepares a filter instance for processing video frames.

int startProc(const VDXFilterActivation *fa, const VDXFilterFunctions *ff);

Parameters

fa | Pointer to filter activation structure.
ff | Pointer to callback function structure.

Thread safety

This method is not thread-safe.

Exceptions

Exceptions may be thrown from this function (see Except()).

Return value

Zero if successful, non-zero on unknown failure.

Default behavior

If omitted, no default action is taken.

Remarks

This function is the ideal place for memory allocation and table initialization to take place. Filter chain startup is comparatively rare, so this is the ideal place to put lengthy precomputation.

If you allocate a number of resources in this function, and one of the allocations fails, you do not have to release the resources that were allocated successfully. VirtualDub always calls endProc for all filters on failure, so you can free the resources there.


stringProc

Creates a user-readable string description of the filter instance's current configuration.

void stringProc(const VDXFilterActivation *fa, const VDXFilterFunctions *ff, char *buf);

Parameters

fa | Pointer to filter activation structure.
ff | Pointer to callback function structure.
buf | 128-byte buffer to receive string. This includes the terminating null, so only 127 characters may be placed.

Thread safety

This method is not thread-safe.

Exceptions

This function must not throw exceptions (see Except()).

Default behavior

If omitted, no default action is taken.

Remarks

Unfortunately, the length of the buffer is not passed to the filter. In VirtualDub 1.4d, this filter is a fixed-length 128-byte global buffer, but the beginning of the buffer is pre-filled with the input/output sizes and the name of the filters, so no more than 80 bytes of space is reliably available. Use the _snprintf() and _vsnprintf() functions to make sure you do not overrun the buffer. Like paramProc, this function is called relatively frequently in the user interface, so it shouldn't take too long.

Also, note that limited space is available in VirtualDub's filter list, and that the list does not scroll horizontally, so keep your descriptions short! The string is purely informative, so you do not need to show all of your filter's parameters.

New filters should supply both this function and stringProc2; the only difference is that stringProc2 receives a buffer length.


stringProc2

Creates a user-readable string description of the filter instance's current configuration.

void stringProc2(const VDXFilterActivation *fa, const VDXFilterFunctions *ff, char *buf, int maxlen);

Parameters

fa | Pointer to filter activation structure.
ff | Pointer to callback function structure.
buf | Buffer to receive string.
maxlen | Size of the buffer, in bytes. This includes the terminating null.

Thread safety

This method is not thread-safe.

Exceptions

This function must not throw exceptions (see Except()).

Default behavior

If omitted, no default action is taken.

Remarks

This function is the same as stringProc, except that this version supplies the length of the buffer, to avoid buffer overflows. New filters should supply both this function and stringProc; 1.4.10 and earlier will use the the older function, and 1.4.11 will use this version.


addFilter

Add a filter to the global filter list.

FilterDefinition *__cdecl addFilter(VDXFilterModule *fm, FilterDefinition *fd, int fd_len);

Parameters

fm | VDXFilterModule with which the filter is associated.
fd | FilterDefinition structure describing the filter.
fd_len | Size of the FilterDefinition structure, in bytes.

Thread safety

This method is not thread-safe.

Return value

Pointer to runtime FilterDefinition structure.

Remarks

This function is used in the DLL initialization process to register all of the filters contained in the DLL.


removeFilter

Remove a filter from the global filter list.

void __cdecl removeFilter(FilterDefinition *fd);

Parameters

fd Runtime FilterDefinition structure for the filter being removed. This must be the pointer returned from addFilter(), not the original FilterDefinition structure used to register the filter.

Thread safety

This method is not thread-safe.

Remarks

This function was intended to be used to remove a DLL's filters from the global list when the DLL is unloaded. However, older versions of VirtualDub do not unload filter DLLs, and newer versions do dynamic unloading and automatically remove associated filters when a module is unloaded. As such, it is not necessary to call this function.


isFPUEnabled

Queries whether optimizations using the floating-point unit (FPU) are enabled.

bool __cdecl isFPUEnabled();

Thread safety

This method is not thread-safe.

Return value

True if FPU optimizations should be enabled, false otherwise.

Remarks

This is equivalent to getCPUFlags() & CPUF_SUPPORTS_FPU.

Filters are free to ignore this function and use floating-point operations regardless; Windows will emulate them if an FPU is not present. This function simply provides a way for the filter to share the CPU detection and override logic in the host.


isMMXEnabled

Queries whether optimizations using the CPU's MMX unit are enabled.

bool __cdecl isMMXEnabled();

Thread safety

This method is not thread-safe.

Return value

True if MMX optimizations should be enabled, false otherwise.

Remarks

This is equivalent to getCPUFlags() & CPUF_SUPPORTS_MMX.

Filters are free to ignore this function and use their own detection logic for whether MMX is present; this function simply allows the CPU detection and override logic in the host to be reused. A filter should still check for MMX support for executing such instructions, however, because otherwise the result will be a crash.


InitVTables

Initializes virtual method table (vtable) pointers for remote object creation.

void __cdecl InitVTables(struct FilterVTbls *vtables);

Parameters

vtables Pointer to vtables structure to initialize.

Thread safety

This method is not thread-safe.

Remarks

This function is required to allow objects of type VBitmap to be initialized within the video filter. To enable this functionality, the symbol VDEXT_MAIN must be defined in exactly one module that includes the filter headers, and InitVTables(&g_vtbls) must be called prior to the first VBitmap creation.


ExceptOutOfMemory

Throws an out-of-memory exception.

void __cdecl ExceptOutOfMemory();

Thread safety

This method is not thread-safe.

Remarks

This may only be called in callback functions that expect exceptions. This function does not return.


Except

Throws an exception with an arbitrary message and printf-style formatting.

void __cdecl Except(const char *format, ...);

Thread safety

This method is not thread-safe.

Remarks

This may only be called in callback functions that expect exceptions. This function does not return.


getCPUFlags

Queries the host for enabled CPU-specific optimizations.

long __cdecl getCPUFlags();

Thread safety

This method is not thread-safe.

Return value

A bitfield indicating which CPU extensions are active:
CPUF_SUPPORTS_CPUID | The CPU supports the CPUID instruction.
CPUF_SUPPORTS_FPU | Optimizations using the x87 floating-point unit (FPU) are enabled.
CPUF_SUPPORTS_MMX | MMX is enabled.
CPUF_SUPPORTS_ISSE | The integer portion of Streaming SIMD Extensions (SSE) is enabled. This is the portion of SSE also supported by the original Athlon, including shufw, pinsrw, pextrw, pavgb, pavgw, pmulhuw, movntq, maskmovq, prefetcht0, prefetcht1, prefetcht2, prefetchnta, and sfence.
CPUF_SUPPORTS_SSE | Streaming SIMD Extensions (SSE) is available, including floating-point vector instructions.
CPUF_SUPPORTS_SSE2 | Streaming SIMD Extensions 2 (SSE2) is available, including 128-bit integer vector instructions.
CPUF_SUPPORTS_3DNOW | 3DNow! is available, including 64-bit floating-point instructions.
CPUF_SUPPORTS_3DNOW_EXT | 3DNow! Professional is available.

Remarks

This function returns the set of currently enabled CPU extensions, which is not always the same as the available extensions. Usually these will be the same, but it is possible for the user to override the enabled CPU extensions.

The enabled set can change between calls. If this is a problem, the value should be cached appropriately.


getHostVersionInfo

Returns information about the identity and version of the host.

long __cdecl getHostVersionInfo(char *buffer, int len);

Parameters

buffer | Pointer to buffer to receive version string.
len | Maximum number of characters to place in the buffer, including the null terminator.

Thread safety

This method is not thread-safe.

Return value

The build number of the host.

Remarks

This is mainly intended for informative purposes. Decisions regarding the host capabilities should be made using the API version instead, whenever possible.


VDXFilterStateInfo

Reflects the video pipeline state while a video filter is active.

Definition

struct VDXFilterStateInfo {
    sint32  lCurrentFrame;
    sint32  lMicrosecsPerFrame;
    sint32  lCurrentSourceFrame;
    sint32  lMicrosecsPerSrcFrame;
    sint32  lSourceFrameMS;
    sint32  lDestFrameMS;

    enum {
        kStateNone      = 0x00000000,
        kStatePreview   = 0x00000001,
        kStateRealTime  = 0x00000002,
        kStateMax       = 0xFFFFFFFF
    };

    uint32  flags;
    sint32  mOutputFrame;
};

Fields

lCurrentFrame
The frame number of the current frame being produced by the video filter, starting at zero.
lMicrosecsPerFrame
Approximate frame rate of the destination video stream, in microseconds per frame.
lCurrentSourceFrame
The frame number of the current source frame being processed by the video filter, starting at zero. This is useful for associating data that must be tracked with the source, such as field order and subtitling data. The frame number sequence need not be sequential; gaps in the frame sequence reflect where portions of the source may have been omitted from processing. It is also possible for source numbers to be duplicated when the source is resampled to a higher frame rate prior to the filter.
lMicrosecsPerSrcFrame
Approximate frame rate of the source video stream, in microseconds per frame.
lSourceFrameMS
Starting time of the source video frame, in milliseconds. Note that this is not simply the same as combining lCurrentSourceFrame and lMicrosecsPerSrcFrame, as the host can internally use a higher precision value in computing this field.
lDestFrameMS
Starting time of the destination video frame, in milliseconds. Note that this is not simply the same as combining lCurrentFrame and lMicrosecsPerFrame, as the host can internally use a higher precision value in computing this field.
flags
(V10+ only) A bitfield with the following flags:
kStatePreview
Set when the filter chain is being run for preview purposes only, such as when scrubbing through the timeline or previewing the output; cleared when the processed video is saved or an analysis pass is running. If your filter records data about the video or otherwise expects a full pass through it, you should suppress that action when this flag is set.
kStateRealTime
Set when the filter chain is being run in a situation with real-time constraints, such as preview or capture mode. A filter might switch to a different algorithm when this flag is present.
mOutputFrame
(V13+ only)

Frame number of the output, in the transformed sequence produced by the filter. The prefetchProc method maps each output frame to a source frame. When prefetchProc is omitted, therefore, lCurrentSourceFrame and mOutputFrame will be the same.

The main differences between lDestFrameMS and mOutputFrame have to do with editing and frame rate changing filters. lDestFrame counts up sequentially for each frame and is the same for all filters in the pipeline, regardless of whether there are frame rate doublers or editing involved. mOutputFrame, on the other hand, indicates frame numbers for frames just after they are produced by the filter, before they are reordered by editing or downstream prefetchProc routines.


VDXFraction

Declares a fraction of two 32-bit integers.

Definition

struct VDXFraction {
    uint32 mNumerator;
    uint32 mDenominator;
};

Fields

mNumerator
The numerator of the fraction. This normally may not be zero.
mDenominator
The denominator of the fraction. This normally may not be zero.

Remarks

The value of the fraction is, as expected, equal to the numerator divided by the denominator. A fraction object is used instead of a floating-point number to exactly represent common frame rates, most notably NTSC's field rate of 60000/1001 = ~59.94 fields/sec.

A fraction need not be reduced to lowest terms, either on the host or plugin side. 30/1 and 30000/1000 should always be treated exactly the same. If you have a frame rate stored as a period in microseconds in your plugin, feel free to return 1000000/period as the fraction.

Unless specified otherwise in a structure or API, the denominator should never be zero, as that indicates a division by zero. A numerator of zero is less troublesome but should also not be used unless allowed; a frame rate of zero in a video stream is also nonsensical.

When a fraction must be multiplied or divided, one approach is to attempt to multiply either the numerator or denominator as appropriate, and if an overflow would occur, divide the other term instead as an approximation. This should be done carefully, as the full range of either term is possible — in particular, expect FFFFFFFF to show up occasionally in the numerator.

If you want to get fancy, you can use Euclid's algorithm to compute the greatest common denominator (GCD) for reducing fractions and continued fractions to determine the best possible approximation for fractions that don't reduce into 32/32 bits. This may be overkill for most cases, though. To convert a floating-point value to a fraction, simply break the floating-point representation into mantissa and exponent and reduce.


VDXInputDriverDefinition

Defines an input driver plugin.

Definition

struct VDInputDriverDefinition {
    enum {
        kFlagNone               = 0x00000000,
        kFlagSupportsVideo      = 0x00000001,
        kFlagSupportsAudio      = 0x00000002,
        kFlagCustomSignature    = 0x00010000,
        kFlagAll                = 0xFFFFFFFF
    };
    uint32      mSize;              // size of this structure in bytes
    uint32      mFlags;
    sint32      mPriority;
    uint32      mSignatureLength;
    const void *mpSignature;
    const wchar_t *mpFilenameDetectPattern;
    const wchar_t *mpFilenamePattern;
    const wchar_t *mpDriverTagName;

    VDInputDriverCreateProc         mpCreate;
};

Fields

mSize
Set this field to sizeof(VDInputDriverDefinition). This is used for future extensibility.
mFlags
A bitfield indicating some miscellaneous properties of the input plugin:
kFlagSupportsVideo
Indicates that the file format supported by this plugin may support video, although not necessarily always. This indicates to the host that this plugin should be included in features that load files for video. If the file turns out not to have video, the host will display an error for the user or script.
Note
In VirtualDub, this causes a plugin to be included in the Open video file... dialog, and in the associated script function.
kFlagSupportsAudio
Indicates that the file format supported by this plugin may support audio, although not necessarily always. This indicates to the host that this plugin should be included in features that load files for video. If the file turns out not to have audio, the host will display an error for the user or script.
Note
As of VirtualDub 1.7.5 (API V2), this is not yet supported, but in a future version it will cause the plugin to become available in the WAV audio... feature.
kFlagCustomSignature
The input plugin supports the DetectBySignature() function. If set, the plugin is loaded and DetectBySignature() is called after the initial signature match via mpSignature (if non-null) to return a definitive result. Not setting this flag prevents DetectBySignature() from being called, which speeds up autodetection because the plugin does not need to be loaded into memory.
mPriority
A value indicating the relative priority of this plugin relative to other plugins during file autodetection, including ones internal to the host. Zero is the default; a plugin with a higher priority value wins over one with a lower priority value if the match result is the same. Note that a plugin that returns a better match result always wins regardless of priority. Plugins should exercise restraint here in using high priority values — in particular, plugins should never use the maximum value of 7FFFFFFF to "always win." Plugins that "handle" a format by returning a descriptive error message should always use a negative priority.
Note
VirtualDub internally uses zero for most of its internal input drivers. Notable exceptions are the fake ASF loader, which uses -1; the AVIFile-based AVI loader, which uses -4 to avoid conflicting with the normal AVI loader at 0; and the image loader, which uses -1 because TARGA signatures can conflict with other file types. If you want to override a format that VirtualDub itself supports, you should use at least 1 for priority.
mSignatureLength
The number of bytes in the signature; must be even. Generally this is set to sizeof(signature). Must be zero if mpSignature is NULL.
mpSignature

Pointer to the signature, if present. This is an array of pairs of bytes signifying a match pattern against the bytes starting at the beginning of the file, where the first byte in each pair is the match byte and the second byte is the mask byte. A byte pair in the signature matches the corresponding byte in the file only if the bits that are significant according to the mask are the same between the match byte and the file byte, i.e. (match & mask) == (data & mask). If the mask bytes are all FF, then the signature simply looks for a specific, contiguous string at the beginning of the file. Pairs of (0,0) allow bytes to be entirely skipped.

If a custom signature match routine is also present, then both this signature and the custom signature must pass for the plugin to be eligible for the file. This allows the signature in this structure to be used for a quick cull pass, and the plugin loaded for its custom signature match routine only if the first signature passes.

Bug alert
In versions of VirtualDub up to 1.7.5 (API V2), there is a bug where a plugin that only uses a signature string only returns a sketchy match (0) instead of a definitive match (1). The workaround is to use a custom signature match routine.
mpFilenameDetectPattern
The filename pattern used for autodetection; may be NULL if none is supported. This is applied to the incoming filename, including the extension if present, but not the drive letter or directory path. If this string matches, a poor match is automatically registered for this plugin during autodetection. The pattern string may include ? and * wildcards, and may include multiple patterns may be supplied separated by vertical bars.
Note
The filename matching algorithm applied is not guaranteed to exactly match the underlying OS mechanism. In particular, Windows has a number of odd special cases involving short (8.3) filenames and leading periods that VirtualDub's internal matching algorithm does not support. The implemented algorithm is instead equivalent to transforming the pattern string to a regular expression by replacing ? with . and * with .*. The filename detection pattern is mainly intended for patterns of the form *.ext that look for particular extensions, although it can be legitimately used to look for base filename patterns as well.
mpFilenamePattern
The filename pattern displayed and used in file selection UI. It should be of the form "readable-form|pattern|...," where the readable form is displayed to the user, and the pattern is a filename wildcard expression with semicolon separators. Multiple patterns may be supplied, separated by vertical bars. For instance, if your plugin supported text files and bitmaps, you could use: "Text files|*.txt;*.ini|Bitmaps|*.bmp". This expression is essentially the same as a Windows common file dialog's filter, except with vertical bars instead of nulls. Note that if multiple filters are included, the plugin has no way of knowing which was used, and that the plugin should not include an "All files (*.*)" escape as that is generally done by the host.
mpDriverTagName
A name that is used to identify this plugin in scripts and batch jobs. This name is not the primary name shown to users but should be human-readable. It should be stable to avoid breaking existing scripts, so version and build strings should not be embedded, but it is a good idea to make it fairly unique. Example: "Foobar's BLAH reader."
mpCreate
Pointer to function that creates an input driver object for this plugin.


VDXPixmap

Declares a pixel map (pixmap). This is sometimes also called a bitmap.

Definition

struct VDXPixmap {
    void            *data;
    const uint32    *palette;
    sint32          w;
    sint32          h;
    ptrdiff_t       pitch;
    sint32          format;

    // Auxiliary planes are always byte-per-pixel.
    void            *data2;     // Cb (U) for YCbCr
    ptrdiff_t       pitch2;
    void            *data3;     // Cr (V) for YCbCr
    ptrdiff_t       pitch3;
};

Fields

data
Pointer to the start of the first scanline in the primary plane. This must be aligned to the smallest natural alignment for the pixel type. For instance, two-byte pixels must be aligned to a 16-bit boundary, and four-byte pixels must be aligned to a 32-bit boundary. 24-bit pixels require no special alignment.
palette
For paletted formats, a pointer to 32-bit XRGB8888 palette entries. The alpha component is ignored.
w
Width of the primary plane, in pixels. This must be greater than zero.
h
Height of the primary plane, in pixels. This must be greater than zero.
pitch
The difference in bytes between the address of the start of one scanline and the next in the primary plane. This may be zero or negative. The pitch must maintain alignment across scanlines — a pixmap with 16-bit pixels may not use an odd pitch.
format
Format and color space of pixels in the pixmap.
data2
For formats with secondary planes, the start of the first scanline in plane 2.
pitch2
For formats with secondary planes, the difference in bytes between the address of the start of one scanline and the next in plane 2. This may be zero or negative.
data3
For formats with secondary planes, the start of the first scanline in plane 3.
pitch3
For formats with secondary planes, the difference in bytes between the address of the start of one scanline and the next in plane 3. This may be zero or negative.

Remarks

Most formats only have a primary plane and thus only use data and pitch. YCbCr formats with three planes use the secondary planes as well; Cb (U) is plane 2 and Cr (V) is plane 3. The secondary planes always have the same size, but may be of different size than the primary plane depending on the format. All three planes may have different pitches.

Scanlines should not overlap in pixmaps passed as destinations, but it is OK to do so for sources; in particular, a pitch of zero simply repeats one scanline down the entire image. Buffers should be declared with non-overlapping scanlines.

For formats that use subsampled secondary planes, such as 4:2:0 planar YCbCr, the pixmap must be sized such that the secondary planes consist of an integral number of samples. For instance, a 4:2:0 pixmap may have a size of 320x240, which leads to 160x120 secondary planes. It cannot have a size of 319x240, because this would lead to a secondary size of 159.5x120. The main reason for this restriction is due to a lack of consensus as to how this situation should be treated; some video drivers and codecs round up, some round down, some display garbage, and some just crash. For best compatibility, it is recommended that if this situation is encountered in the underlying file format that the subsampled formats be disabled or that the video be padded to the next non-problematic size.

In the original VirtualDub video filter API, the VBitmap structure's data field pointed to the bottom scanline, and a positive pitch meant bottom-up storage. VDXPixmap is reversed so that data points to the top scanline and a positive pitch indicates top-down storage. Pitch should therefore be negative whenever a Windows DIB compatible layout is required.


VDXStreamSourceInfo

Provides basic information about a stream in a file.

Definition

struct VDXStreamSourceInfo {
    VDXFraction     mSampleRate;
    sint64          mSampleCount;
    VDXFraction     mPixelAspectRatio;
};

Fields

mSampleRate
The average rate, in samples per second, at which the stream is played or was recorded.
mSampleCount
The number of samples in the stream. This should be non-negative.
mPixelAspectRatio
The ratio of width to height of a pixel in the decoded stream. This is ignored for non-video streams. Most videos played on a PC should be square, i.e. 1:1. Formats with pixels that are wide should have a ratio fraction greater than 1. This fraction may be set to an invalid value (0/0) if the pixel aspect ratio is not known.
Note
The pixel aspect ratio field is not currently used by VirtualDub.


VDXStreamSourceInfoV3

Provides additional information about a stream in a file.

Definition

struct VDXStreamSourceInfoV3 {
    VDXStreamSourceInfo mInfo;

    enum {
        kFlagVariableSizeSamples    = 0x00000001
    };

    uint32          mFlags;
    uint32          mfccHandler;
};

Fields

mInfo
Base API V1-V2 stream source information. See VDXStreamSourceInfo.
mFlags
A bitfield indicating some miscellaneous properties of the stream:
kFlagVariableSizeSamples
Indicates that the stream contains samples of different sizes. For an audio stream, this indicates full support for variable bit rate (VBR) audio; it is unnecessary for a video stream, where VBR is always assumed.
mfccHandler
Indicates the suggested handler for the decoder to use for the stream, or zero if there is no suggested handler. For a video stream, this is the FOURCC for a registered video codec. This is only a hint; a decoder may still be chosen other than the one indicated here.

API version requirement

Requires input driver API version 3 or later.


VDXVideoFrameInfo

Describes a video frame in a video stream.

Definition

struct VDXVideoFrameInfo {
    char    mTypeChar;
    uint8   mFrameType;
    sint64  mBytePosition;
};

Fields

mTypeChar
A readable character used to describe the video frame. For AVI-style streams, the characters 'K' and ' ' should be used for key and delta frames, respectively; for MPEG-style streams, 'I', 'P', and 'B' can be used. This is only for user interface purposes, so it should not be interpreted by hosts to detect the frame type.
mFrameType
A value indicating the frame's basic type, most notably its dependency on other frames. This must be one of values in the VDXVideoFrameInfo enumeration.
mBytePosition
The approximate starting byte position within the file. This can be used by the host both for informative purposes — such as indicating to the user the approximate byte position of corrupted data — or for optimization purposes, such as seek optimization. A value of -1 indicates that no byte position is available. It is OK to return -1 as the byte position all samples.