Handle API Callbacks Safely
See how to process API callbacks.
by Karl E. Peterson

August 2002 Issue  Download the original Classic VB code from this column.

Technology Toolbox: VB6, VB5

Q: Pass Interfaces Through API
After searching the newsgroups with Google, I thought I'd found a solution to processing API callbacks (specifically, EnumWindows) within a VB class module. As you know, AddressOf works only with standard BAS module routines. The technique I've been trying to implement involves "stealing" a reference to the class instance by copying a passed pointer into an uninstantiated instance of the class on each callback. This works for me for a couple callbacks, then VB crashes. What am I doing wrong?

A:
It's possible that the only thing you're "doing wrong" is interacting with your code as it's executing. That can cause problems, when the execution path results from a system-initiated callback. In cases like yours, a safer method exists that allows you to avoid all those "messy" calls to CopyMemory. Indeed, you can let VB do all the object pointer manipulation for you.

The key to this technique is understanding what VB actually passes to an API call when you declare a parameter using As Any for the type. Some shy away from this construct on the basis that it's inherently unsafe—and it is if you don't know what VB passes! The general rule is that if you don't specify ByVal, VB passes a pointer to the variable specified for an API parameter. With object variables, the use of ByVal tells VB to pass a pointer to the specified interface instance. Using this knowledge, you can effectively hand your callback routine a reference to the initiating class (see Listing 1).

If you've taken your declare for EnumWindows straight from the API Viewer, your first task is to change lParam—a parameter designed to pass custom data to the callback routine—from ByVal As Long to ByVal As Any. Place this declaration in your BAS module. Then create the kickoff routine, StartEnum, which your class modules call when they want to run an enumeration. When a class instance needs to fire up an enumeration, it calls StartEnum, passing a reference to itself:

Call MEnumWindows.StartEnum(Me)

StartEnum calls EnumWindows, passing the AddressOf its own dispatch routine (EnumWindowsProc) in lpEnumFunc, and the object reference as custom data in lParam. Windows then calls EnumWindowsProc once for each top-level window, passing back a handle to the window and your custom data. EnumWindowsProc declares the incoming lParam as CEnumWindows, so classic VB does its magic by creating a new reference to your existing class instance, and you're freed from having to use CopyMemory to cast the pointer to an uninstantiated object variable.

EnumWindowsProc has instant access to the proper class instance, and can call any of its public methods or properties. As EnumWindowsProc exits, the created class reference goes out of scope and VB decrements the object's reference count appropriately.

You can take this method one step further, and allow any of your project's classes to sink system callbacks. Create a new interface, IEnumWindowsSink, and implement it in any class (or form) that needs this functionality. Taking this extra step eliminates the need to specify a specific class name in your generic BAS module routines:

Option Explicit
' Generic IEnumWindowsSink interface
Public Function EnumWindowsProc( _
   ByVal hWnd As Long) As Boolean
   ' Stub procedure
End Function

Implement this interface in your callback classes by adding this line to their declarations section:

Implements IEnumWindowsSink

Then, select IEnumWindowsSink from the object dropdown in the class's code editor. The VB IDE inserts the first routine from the associated interface automatically into your class. Modify the BAS module routines used to fire off the enumeration so they expect a pointer to IEnumWindowsSink rather than CEnumWindows, and you're all set (see Listing 2).

Typically, you would use lParam to pass data used within the callback procedure. With the design I've outlined here, you can store this sort of data within the class instance, and there's no need to pass it at all. —K.E.P.


About the Author
Karl E. Peterson is a GIS analyst with a regional transportation-planning agency and serves as a member of the Visual Studio Magazine Technical Review and Editorial Advisory Boards. Online, he's a Microsoft MVP and a section leader on several VSM forums. Find more of Karl's VB samples at www.mvps.org/vb.