4110
Delphi for the Microsoft .NET Framework Open Tools API
Marco Cantu
Wintech Italia Srl

Abstract: More than in the past, the Delphi IDE allows programmers to write their own wizards and plug-ins.

#4110: Diamondback Open Tools API

Marco Cantù, www.marcocantu.com

The Diamondback IDE allows programmers to write their own wizards and plug-ins even more than in the past.

This presentation covers the Open Tools API (OTA) of Borland Developer Studio (in particular, Diamondback for .NET). This API is basically an extension of the interfaced-based API available since Delphi 5 (and often technically called “IOTA interfaces”). The level of integration you can obtain with the Delphi IDE is very high: you can interact with designers, work directly on the editor, and obviously open up your own windows to let the user perform complex operations in a breeze. The presentation will highlight the core features of Delphi’s ToolsApi, and provide plenty of working examples as starting points for your own development. Examples will include database-related wizards, my OOP form wizard, class browsers, build process tools, and many others ported from past versions of Delphi (to show you the process and possible obstacles) and built from scratch to take advantage of new features of the Borland Developer Studio IDE.

Introduction: A Short History of Delphi's OTA

The Delphi IDE has had a very long history of openness and third party add-in have been flourishing for many years. Most of the Delphi developers have used GExperts, have heard of CodeRush, or have installed a plug in to have a multi-line component palette. Still, developers of these add-ins had often to move in the dark, with limited documentation coming at first in Ray Lischner's books (it was Delphi 2 time frame) and in some of my own volumes (particularly the Delphi Developers Handbook). These books and other references started to document the internal features of the IDE exposed to programmers. For example, the Experts interface (Experts was the original Delphi name for a Wizard) included a ToolServices object that was still around in Delphi 7 (and it still around now, but not really working!) and some interfaces to interact with editors and designers.

Since Delphi 3 (and up to Delphi 7), it is possible to write a package-based wizard, basically a design time package installed in the same way you do with component packages. The alternative was the use of DLL wizards, external DLLs installed by adding an entry in the registry. With the introduction of packages, the situations changed for better for the Delphi hackers, as a design time package is code that lives within the Delphi IDE itself. As a simple example, If you simply create a design time package with the following initialization code, as you install it Delphi own window shrinks:

initialization
  Application.MainForm.Width := 400;

This si actually possible also with a plug-in DLL that shares the core VCL packages with the IDE. Relying on the direct modification of Delphi's own components, however, is error-prone and subject to changes from version to version. For this reason, similar techniques should be used only as a last resort, to provide features not exposed by the public ToolsAPI. What's happened, in fact, is that starting with Delphi 4 and 5 the internal calls originally available for wizards got a formal name (Open Tools API) and were heavily re-arranged in a more modern fashion: although the internal API was based on interfaces since the very beginning (actually even before interfaces existed in the Delphi language), it was now designed around interfaces in a more flexible way. The OTA was extended over time up to Delphi 7, when it was first documented in a formal help file... only to be partially abandoned with the design of the new Borland Developer Studio (BDS) IDE for C# builder and Delphi 8.

Two Open Tools APIs at the Price of One

Actually it it correct to say that the Win32 OTA was only partially abandoned because Delphi 8 and Diamondback provide two different OTAs, a Managed OTA and an Unmanaged OTA. The reason is simple: the BDS IDE has been built with an “unmanaged” Win32 compiler, internally indicated as Delphi 7.1 for the C#Builder and Delphi 8 releases (nothing to do with the publicly available Delphi 7 update) and with Diamondback itself for the Diamondback release. If you want take advantage of most of the OTA features and to interact fully and directly with the IDE windows and controls you have to work with the Unmanaged OTA.

The Managed OTA, on the other hand, allows you to build .NET assemblies that interact with the IDE, like DLL wizards used to do in past versions of Delphi. The managed OTA is based on the Borland.Studio.ToolsApi assembly. This means that differently from the past you don't even have the source code of the interfaces. Not that this is a big issue, because it is possible to inspect everything and get the Pascal declarations (for example with Lutz Roeder's Reflector) and because Borland has published the internal comments of the units declaring these interfaces, which used to be the primary source of documentation.

Actually in Diamondback, Borland is expected to ship a XML file documenting the Borland.Studio.ToolsApi, so that you don't get only code completion but even help descriptions of the available interfaces, classes and methods of the Managed API. Notice that you can write wizards or other Delphi extensions based on the managed ToolsApi in any .NET language, not just Delphi for .NET. If you want to take advantage of .NET designers and .NET specific features, like the CodeDOM, you have to work with the Managed OTA.

Wizards

When you use both of the OTAs you often (but not always) define wizards. There are four possible type of wizards defined by four different interfaces:

• A Menu Wizard implements IOTAMenuWizard and will show up in a menu item listed under Help (the "classic").
• A Project Wizard implements IOTAProjectWizard and will show up in an entry added to the repository projects (in the File New dialog box).
• A Form Wizard implements IOTAFormWizard and is added to the repository forms.
• An Add-in Wizard implements the base interface IOTAWizard and allows you to do whatever you want: often use IOTAMainMenuService to create one ore more custom menu items. An add-in wizard can create its own menu item, but it doesn't need to. It can also install itself in other more "creative" ways, or install on or more "notifiers" so that it kicks in automatically when the user does certain operations.

But before I get ahead of myself let's see a couple of simple wizards.

A Very Simple Unmanaged Wizard

Let me start by showing you the code of a very simple packaged wizard based on the Unmanaged OTA. The reason I'm starting with this version is that this is something a large number of Delphi programmers have already seen and also to follow the actual versions over time (skipping the very early versions, of course). To create an unmanaged wizard, you need to include the ToolsApi unit (which is available with fully commented source code in the Source/ToolsApi folder) and define a class implementing the IOTAWizard interface (or one of the other wizard interfaces). The following class implements a IOTAMenuWizard:

type
  TMCFirstWizard = class(TNotifierObject, IOTAWizard, IOTAMenuWizard)
  private
    FCounter: Integer;
  protected
    // IOTAWizard
    function GetIDString: string;
    function GetName: string;
    function GetState: TWizardState;
    procedure Execute;
    // IOTAMenuWizard}
    function GetMenuText: string;
  public
    constructor Create;
  end;

The class inherits from TNotifierObject to have a ready-to-use do-nothing implementation of the core methods of the base IOTANotifier interface, including the IInterface support. The name and ID returned by each wizard must be unique, the state can be used to disable a wizard based on some conditions (maybe the status of the active designer, the selected text in the editor, and so on...), while the Execute function is where you write the actual code of the wizard (which in this simple demo updates a counter and shows a message box). As this is a menu wizard, we have one more method to implement to return the text of the menu item (notice that the text can change as this method is called every time the menu is displayed, as the example demonstrates):

function TMCFirstWizard.GetMenuText: string;
begin
  Result := Format ('First Wizard %d', [FCounter]);
end;

This counter is increased each time the wizard executes, displaying the new value along with an internal information extracted using the ToolsApi, the version of the IDE hosting the wizard:

procedure TMCFirstWizard.Execute;
var
  strVersion: string;
begin
  Inc(FCounter);
  strVersion := (BorlandIDEServices as IOTAServices).GetProductIdentifier;
  MessageBox(0,
    PChar(Format('Hello %d times from %s', [FCounter, strVersion])),
    'First Wizard', MB_OK);
end;

The code above introduces you to the use of the internal services, a technique required for each and every interaction between you and Delphi itself. In this case I need to access a method of the IOTAServices interface, a rather generic set of system configuration services. This is accomplished by querying the global object BorlandIDEServices for the interface you are interested in.

Finally, you have to add this unit to a design time package and register the wizard in the classic Register procedure, with some code like this that creates the wizard object and passes it to the system (you don't need to take care about destroying the wizard object, as it is reference counted and Delphi will do this for you):

procedure Register;
begin
  RegisterPackageWizard(TTolFirstWizard.Create);
end;

By installing the package Delphi adds it in the registry under HKCUSoftwareBorlandBDSx.0Known IDE Packages.

An Aside: The Unmanaged ToolsAPI for Delphi 8

Like Diamondback, also Delphi 8 made available an Unmanaged ToolsAPI, made of native packages built with the same compiler used to build the Delphi IDE. The problem, however, was that the IDE was built with an unreleased, internal version of the Delphi compiler. You cannot use Delphi 7 to build such a package. For this reason Borland (or more precisely, Allen Bauer) made available as-is an "IDE integration pack", you can download from Code Central. This includes a command line compiler and same base units and packages. In practice, Borland delivers you a set of files you have to copy (manually install) over the standard Delphi 7 files (in the bin and lib folders) and than edit the registry to turn your 7.0 version into a 7.1 version. If the installation is quite complex, by installing the proper IDE and debugger settings you'll be able to debug your packages with Delphi 7.1 while they are running within Delphi 8.

A Very Simple Managed Wizard

If this was a simple Unmanaged wizard, let's now look at a similar introductory examples on the Managed side of the ToolsApi, to get you a feeling of the differences before we start inspecting some more interesting APIs and examples. First, instead of using the ToolsApi unit you have to include the Borland.Studio.ToolsApi namespace. This defines the same set of four different wizard interfaces, but your class can simply inherit from TObject as we have no more reference counting and IInterface to handle. Notice also that the method names are slightly different more in line with .NET naming conventions. Here is the definition of the class:

type
  TIntroWizard = class(TObject, IOTAMenuWizard)
  private
    FCounter: Integer;
  public
    constructor Create;
    destructor Destroy; override;
    // IOTAWizard = interface
    procedure Destroyed;
    procedure Execute;
    function get_IDString: string;
    function get_Name: string;
    // IOTAMenuWizard = interface (IOTAWizard)
    function get_Checked: boolean;
    function get_Enabled: boolean;
    function get_MenuText: string;
    // specific code
    procedure ShowMessage (const text: string);
    class procedure IDERegister; static;
  end;

The rest of the code will look very similar to the other example, with the only exception of the registration code and of how you refer to the IDE services. This time in fact you use the global BorlandIDE object passing to its GetService method the type of the interface you are looking for as in the following registration/de-registration code. The other difference is that instead of using a Register method you define an IDERegister static class method:

var
  WizardId: Integer;

procedure UnRegister;
var
  WizService: IOTAWizardService;
begin
  WizService := BorlandIDE.GetService(typeof(IOTAWizardService));
  WizService.RemoveWizard (WizardId);
  // MessageBox.Show('UnRegister');
end;

class procedure TIntroWizard.IDERegister;
var
  WizService: IOTAWizardService;
begin
  WizService := BorlandIDE.GetService(typeof(IOTAWizardService));
  WizardId := WizService.AddWizard (TIntroWizard.Create);
  // MessageBox.Show('Register');
end;

By the way, the code used to access to a service in C# can be written like:

IOTAModuleServices Services = (IOTAModuleServices)              
  BorlandIDE.GetService(typeof(IOTAModuleServices));

The installation of this wizard (as any other Managed OTA Wizard) requires editing the registry. You need to find the key HKCUSoftwareBorlandBDS1.0Known IDE Assemblies and add a string entry for your new wizard, in the format "DllFileName=some string". The string key should refer to the DLL path (with a full path or using the $(BDS) shortcut if the DLL is installed under the BDS version program folder (as in "$(BDS)BinBorland.Studio.Together.dll"). The value of the registry key can be any string, including the classic "(untitled)" or you can provide a description. Only, don't leave the value empty, because this is how the IDE marks invalid entries (like assemblies that raise an exception during initialization).

Managed Recompilation Blues

The problem with this type of installation is that is is much less flexible than with that of a packaged Unmanaged wizard. In that case, in fact, you can simply recompile the package and the IDE will, uninstall it, recompile it, and reinstall it all automatically just by pressing the Ctrl+F9 keys. On the Managed side, instead, the situation is much less nice. In fact, to recompile the assembly you need first to remove it from the IDE, which is possible only if you close down Delphi itself. So the steps could be: Close Delphi, remove the wizard from the registry, restart, recompile, re-close, re-add the wizard in the registry, run Delphi again to test it.

Luckily, there are a couple of simpler approaches. One is to run a copy of Delphi for compiling (and eventually debug) the wizard running within a second instance. If the first instance has no dependencies from the assembly, it can recompile it at will. The second solution is to rename the wizard each time, so that the sequence becomes: Save the project with the other name, compile, close Delphi, edit the registry, restart Delphi.

The Structure of the OTA: Units and Assemblies

Now that we have build a simple example using the Unmanaged and the Managed OTA, let's have a look at what's available in the ToolsApi assembly.

On the Unmanaged OTA side in Diamondback we have the source code of several units, most of which are brand new (only the ToolsApi unit was in Delphi 7):

FileHistoryAPI.pas
PaletteAPI.pas         
PropInspAPI.pas        
StructureViewAPI.pas
ToolsAPI.pas

This is different from Delphi 7 that still had support for the older APIs with units like:

DesignIntf.pas           
EditIntf.pas             
ExptIntf.pas
FileIntf.pas             
IStreams.pas             
ToolIntf.pas             
ToolsAPI.pas             
TreeIntf.pas
VcsIntf.pas
VirtIntf.pas

You can still compile against some of these units in Diamondback, but most of them will issue an “obsolete” warning and their code might not work as you'd expect (with many methods raising exceptions as they are called).

On the Managed OTA side, the ToolsApi is made of a the Borland.Studio.ToolsAPI assembly. This assembly is not monolithic for the simple reason that a single IDE can be used to hosts multiple personalities (for example, a Delphi personality and a C# personality). This is visible in the structure of the assembly, which defines multiple namespaces:

Borland.SCI 
Borland.Studio 
Borland.Studio.Languages 
Borland.Studio.ToolsAPI 

In this talk I'll focus on the Borland.Studio.ToolsAPI namespace, which defines data types falling in different categories, shortly explored in the following sections.

The Structure of the OTA: Interfaces, Services and Notifiers

We have seen that to work with the Delphi IDE (both with the Managed or Unmanaged APIs) you need access information and interact with the environment using some of the available service interfaces. On the other hand you write the bulk of the code by implementing other interfaces, like the wizard interfaces introduced earlier. This is a list of available services and interfaces, arranged in a few groups for readability (the list is not complete, only a selection of the most relevant interfaces).

Note: historically, the APIs with a IOTA Prefix could be accessed by an external DLL only by hooking in the Borlndmm.dll via the sharemem unit, while the INTA Prefix (Native Tools API) required to plug in the VCL as well. When using packages, the distinction is far less relevant. Notice, anyway, that most of the new OTAs added to Diamondback use the IOTA prefix, even if some of them are strictly related to the VCL (like the IOTAProjectMenuCreatorNotifier that uses TMenuItem components).

The ToolsApi has a large number of interfaces, over 200, with different roles. For simplicity, interfaces can be divided in three broad categories:

• Services are the entry points for interacting with the system. The ToolsApi is made of a number of “services”, devoted to specific areas of the IDE. As we have already seen, you can access to services using BorlandIDE.GetService or extracting an interface from the BorlandIDEServices object.
• Notifiers are the call-back interfaces you implement in your code with a class that defines the proper methods of the interface. For example, wizards are technically notifiers. Other notifiers can be hooked to projects, files, and even toolbar changes or breakpoint events.
• Generic interfaces are used to access to or interact with specific data structures. For example, the editor service allows you to access to the editor, the editor views, the edit buffers, and so on. At times you have to implement a generic interface (such as the creator interfaces used to create projects or add new units to a project) but most of the time you simply use them. The difference between these interfaces and services is that services are the entry points for interacting with the system. The difference between these interfaces and notifiers is that notifiers are installed in the system (often at initialization time) while, for example, a creator interface is activated directly in your code.

As the most important entry points are the service interfaces, here I've provided a list of them divided in a few areas (this arrangement is totally mine). I've marked the services available in the past (Delphi 7) and in the Managed and Unmanaged ToolsApi of Diamondback. I skipped referring to the corresponding Delphi 8 APIs, which are an in-between, as not too many developers have used them. I might of course have missed a few, but I've tried to be as complete as possible. Symbols: X means available, X+ means with many new features, X- means almost useless, / means not really needed.

Global IDE Services

Add-In Configuration Services

Options, Project Configuration Services

Editors, Designers Services

Notifiers

Speaking of notifiers, I've already mentioned that the Unmanaged ToolsApi defines a ready to use TNotifierObject, which inherits from TInterfacedObject. This gives you a ready-to-use implementation of the IInterface/IUnknown interface (most often the older name is still is use) with reference counting support. It provides also a dummy implementation of the IOTANotifier interface, with empty methods (so you don't need to provide them in case you don't need to handle them). As we have seen, Wizards are technically notifiers (although they don't use the IOTANotifier methods), which is why I've used the TNotifierObject class as their base class. In any case, this is the base notifier interface:

type
  IOTANotifier = interface(IUnknown)
    ['{F17A7BCF-E07D-11D1-AB0B-00C04FB16FB3}']
    procedure AfterSave;
    procedure BeforeSave;
    procedure Destroyed;
    procedure Modified;
  end;

The Diamondback Unmanaged API provides also a ready-to-use TModuleNotifierObject, which inherits from TNotifierObject, providing a few more predefined methods supporting the IOTAModuleNotifier interface.

The number of notifier interfaces is quite large. I've already mentioned a few in the tables describing the services above. Here there are a few more I find particularly interesting. In general notice that the managed API has much fewer notifiers (only IOTAIdleNotifier). The reason is simple: many notifier interfaces have been changed to events, something possible thanks to the multicast events semantic.

Assorted Demos

After this extensive description of the available features, it is certainly worth looking at some examples. (Note: This is how the paper is arranged, the presentation will see the reference description and some examples intermixed! Also, I'm not listing much code here as the companion source code is a much better place to study it!)

After the limited initial demos, I'm presenting a few more, giving you broad references to their scope. I'm starting with a few “assorted” wizards and I'll later give you a list of features of the Cantools collection of Wizards I've ported from Delphi 7 to Diamondback. The initial set of demos includes mostly managed ToolsApi demos, including the following:

• A wizard that interacts with the VCL form designer to let you hide (and show again) non visual components.
• A wizard that converts a project source code to HTML
• A simple form wizard
• Global custom component and property editors, which take partial advantage of the ToolsApi.
• A very useful wizard to open ancestors of a form, in case of Visual Form Inheritance (helping the IDE to manage complex VFI situations).
• A sample demonstration of notifiers managing compilations and using the messages API.
• How to add add-in information to the about box and splash screen using the registry and not writing any internal code.
• A wizard showing internals of the IDE (menu structure, available forms, and the like).
• A wizard creating a dockable form in the IDE

Porting Cantools to Diamondback

To create a few interesting wizards for Diamondback I decided to port a set of tools I wrote a few years back, for Delphi 6 and Delphi 7. I first tried to port them to the Managed API, but soon realized this was a no go, as the approach to writing similar code is quite different. My wizards use many hacks to the IDE and a few low level techniques, and rely heavily on the VCL: This makes them best as candidates for the use of the Unmanaged API.

The set of tools and wizards includes a rebuild wizard to open and rebuild a large set of projects, class hierarchy navigation, unit generation and form generation and manipulation (in particular a wizard to turn VCL forms code into a more object-oriented style), a database form wizard that goes beyond the BDE, clipboard management tool, and a few more. Hacks used to include a multiline components palette (useless with the new BDS IDE), while it still has a way to (partially) modify the font of the Object Inspector.

The collection adds itself to the AboutBox and Splash Screen (in the new legal way, while it used a hack in Delphi 7), and uses a large number of services (particularly in the area of editing and compiling).

Conclusion

Delphi has always been an open IDE and the new Borland Developer's Studio keeps pushing in this direction with two different APIs: the traditional Win32 unmanaged ToolsAPI, which has had a face lift with many new additions) and a brand new managed API you can use from Delphi and other .NET languages. As Borland itself is facing the task of integrating in a single IDE a lot of tools build by various developers groups to support the entire ALM cycle, the need of an open and complete ToolsApi is not only for third-partyu integration (as it used to be) but is now vital even for the development of the product itself.

This is a large bonus to all the Delphi developers eager to customize the IDE in new ways, as they now have a chance to see more documentation and examples, but even more because they have way more power to tap “legally” into the IDE than in the past. On the other hand, some of the new APIs are quite complex to use and the IDE is more rich than in the past so that finding creating ways to enhance it might have become more difficult. But I'm pretty sure many enhancements will start to appear withing days the new product ships.

Author

Marco Cantù is the author of the bestselling Mastering Delphi series and an expert in Delphi and XML technologies. Marco has written books also on C++ and advanced Delphi programming, many articles, and spoken to developer's conferences around the world, including 10 Borland US Conferences. Marco lives in Italy, but you can easily reach him on www.marcocantu.com.

No Files recieved

Add or View comments on this article