HOWTO: Replace a VB6 dll with a C# dll when methods have optional string parameters

My quest to replace my VB6 COM dll with a C# dll continues. In my last post I resolved the problems I was having where early binding to the DLL didn't work, but late binding did.

The problem I need to address this time is handling a VB6 method that has an optional String parameter. Something like:

Public Sub TestMethod(ByVal sArgIn As String, Optional sArgOut As String)

Looking in the TypeLibrary for the DLL (the *.IDL file we get from the OLEVIEW tool), we see this:

HRESULT TestMethod(
                [in] BSTR sArgIn, 
                [in, out, optional] BSTR* sArgOut);

Creating the TLB via the MIDL command works, but it gives the following warning:

.\MyDll13.IDL(30) : warning MIDL2400 : for oleautomation, optional parameters should be VARIANT or VARIANT * : [optional
] [ Parameter 'sArgOut' of Procedure 'TestMethod' ( Interface '_TestClass' ) ]

I'll now go ahead and create my .Net DLL. When I use .Net to implement the interface, it stubs the C# method as:

using System.Runtime.InteropServices;

namespace MyDllNet
{
    [ProgId("MyDll.TestClass")]
    [ComVisible(true)]
    [Guid("D860A2A8-5003-4714-AE59-918FE2B0FC42")]
    public class MyProxyClass : MyDll13ModifiedTA.TestClass
    {
        public string GetVersion()
        {...}

        public void TestMethod(string sArgIn, [OptionalAttribute]ref string sArgOut)
        {...}
    }
}

This looks good. After completing the code and compiling it, I run my original VB6 app which uses the TestMethod method from a VB6 app via late binding. It works fine when I pass two parameters, but if I only pass one parameter it falls over with "Run-time error '13' Type mismatch". Interestingly, the VB6 app which uses early binding works with both one or two parameters passed!

C# DLL implements IDL with optional BSTR* argument
VB6 Binding	Pass One Arg	Pass Two Args
Early Binding	OK	OK
Late Binding	OK	Error "Type mismatch"

Plan B. Plan B is to change the optional parameter in the IDL from BSTR* to VARIANT*. According to the MIDL error message, VARIANT* is the right type to use for an optional parameter in COM. Note that I'm not changing the original VB6 COM component, I'm just changing the definition in the Type Library.

Creating the TLB via the MIDL command no longer raises a warning; implementing this new IDL/TLB in Visual Studio yields the following method signature:

public void TestMethod(string sArgIn, [OptionalAttribute]ref object sArgOut) {}

But does it work? No, it's worse! Now it crashes the CLR!!

C# DLL implements IDL with optional VARIANT* argument
VB6 Binding	Pass One Arg	Pass Two Args
Early Binding	Error clr.dll APPCRASH	Error clr.dll APPCRASH
Late Binding	OK	OK

OK, this confirms that I don't know what I'm doing! But wait, if I remove all of the code from my "TestMethod" in the .Net DLL then everything works!?! The APPCRASH must be caused by the way I'm handling the object argument in my .Net code. Time to debug into the .Net dll component and see what's going on!

And it turns out that if I try to return anything at all (apart from null) in the optional parameter, I get a System.StackOverflowException in an Unknown Module. Almost time for me to give this one away. Just one last thing to try... VB.Net!

Plan C. VB.Net VB.Net offers more comprehensive COM support, presumably because it is supposed to be a migration path for people who have VB6. So I'll create a new VB.Net class project, and add a reference to the "optional BSTR*" assembly (create in Plan A above), and implement the interface in VB.Net. The VB.Net code looks something like:

Imports System.Runtime.InteropServices

<ProgId("MyDll.TestClass")>
<Guid("D860A2A8-5003-4714-AE59-918FE2B0FC42")>
Public Class MyProxyClass
    Implements MyDll13TldAssembly.TestClass

    Public Function GetVersion() As String Implements MyDll13TldAssembly._TestClass.GetVersion
       ...
    End Function

    Public Sub TestMethod(ByVal sArgIn As String, Optional ByRef sArgOut As String = Nothing) Implements MyDll13TldAssembly._TestClass.TestMethod
...
    End Sub
End Class

And guess what? It works! Late-binding, early-binding, the optional supplied, the optional not supplied. All combinations work!. Job done.

To Conclude: If you want to implement a VB6 COM interface, which has an optional String parameter, in .Net, then use VB.Net! (with some provisos)