VSTO and Contacts

I'm spending an amount of time looking at integrating our application into Outlook at the moment.

To give some background, I want to take the collection of contacts out of our database and put them into an Outlook Contact List - this is fine and easily done.
In fact there's a reasonable description of what is required here

What I then want to do is to make a note into our application when you send an email to one of these contacts; well that's pretty easy - or is it?

Attaching to the Application.ItemSend event is great, this tells us when we're about to send a message. From here, we can get the recipients of the message - that's fine.
We then need to get hold of the contact to see if it is one of our imported contacts.; well the recipient class has an AddressEntry property that has a GetContact method on it.

You'd think this would do the job for us now wouldn't you!
Well, it's like this...
If the contact is in a folder with the property set to be included in the address book, then yes, you can get at your imported contact. If the folder doesn't have this property set, then you don't!

I've spent a good few hours trying to track this down and the hint was on here as it says that it only returns the Contact in the address book.

Ho hum, it would be nice if this wasn't quite so obtuse, but there you go.