MSDTC Woes With NServiceBus And NHibernate

I’ve spent about 3 days trying to get something working that should’ve just worked. I basically wanted some .NET code to use a distributed transaction to update some data in a database, and then publish a message on the service bus. I want to do this in a distributed transaction because if something goes wrong, I want to roll back both transactions (the database change and the published message). Normally, this should just work if you have MS DTC configured correctly. On my machine, I enabled Network DTC Access, and allowed outbound transaction communication. On the database server, Network DTC Access was already enabled and both outbound and inbound communication was allowed.

Now the thing is, I'd either expect DTC to fail outright or to just work. But it shouldn’t fail in one situation, and work in another. On my machine, it failed in the following situation (which I'll further refer to as Situation A):

open a transaction scope
open an nhibernate session
hit the db
publish a message through nservicebus
close the nhibernate session
complete and close the transaction scope

Step 4 and 5 could be switched around but it didn’t make a difference. In Situation A, I always got a TransactionManagerCommunicationException with the following message:

Network access for Distributed Transaction Manager (MSDTC) has been disabled. Please enable DTC for network access in the security configuration for MSDTC using the Component Services Administrative tool.

Everyone who’s worked with MSDTC before probably knows that exception since it usually takes some fiddling with the settings to make things work. The thing is, I was pretty sure that my settings, as well as the ones on the database server were correct. Unfortunately, DTCPing didn’t confirm that since that too failed.

However, I also tried the following sequence of events (Situation B):

open a transaction scope
open an nhibernate session
publish a message
hit the db
close the nhibernate session
complete and close the transaction scope

And guess what. That actually worked. With full DTC transaction semantics. The DTC statistics on the server confirmed that it was indeed using a DTC transaction, and if I made the code fail with an exception both the database action and the published message were correctly rolled back.

So the question is: why on earth does it only work when I publish a message before I hit the db?

During my investigation I noticed that in Situation A, the internal transaction that the transaction scope was using was a SqlDelegatedTransaction. Which, if I'm not mistaken is an LTM transaction. When trying to send a message to a message queue, the transaction manager tries to promote the current transaction to an OletxCommittableTransaction since the OleTx transaction protocol is required when using MSMQ (it doesn’t support LTM transactions). For some reason, promoting the SqlDelegatedTransaction to a full DTC (OleTx) transaction fails on my machine.

In Situation B, the internal transaction is promoted to an OletxCommittableTransaction as soon as you try to send the message to a message queue. Once it’s time to hit the DB, NHibernate nicely works together with the OletxCommittableTransaction and everything just works.

Now, I have no idea on earth why promotion of a SqlDelegatedTransaction fails, but after a long number of attempts and experiments to get it working correctly, I sorta gave up and figured I'd have to resort to a hack. What I basically needed was for the transaction scope’s internal transaction to automatically be promoted to an OletxCommittableTransaction before I'd hit the database and without having to publish a dummy message at the beginning of the transaction.

I found one way of doing this which, while being a huge hack, is still relatively clean I think. I wrote the following class:

Then, right after opening the transaction scope and before doing anything else, I do this:

This basically tells the System.Transactions infrastructure that we’re adding our own Resource Manager to the current transaction. And because it’s a durable Resource Manager, it now automatically promotes the internal transaction to an OletxCommittableTransaction and everything just works. While our Resource Manager participates in the 2-phase-commit process, it doesn’t actually do anything. It’s sole purpose is to force the creation of an OletxCommittableTransaction.

Like I said, it’s a hack but it’s still relatively clean. I still have no idea why I needed to resort to this hack though… If anyone can shed some light on this, I'd highly appreciate it :)

Also, if you ever want to learn more about transactions in .NET or distributed transactions in particular, you really need to check out this article. Without it, I probably wouldn’t have figured out what to do.

Written by Davy Brion, published on 3/19/2010 11:52:16 AM
Categories: net-bugs , msdtc , nhibernate , nservicebus

Agatha Vs NServiceBus?

Ever since I open sourced Agatha, I've gotten questions from people who are wondering whether they should use Agatha or NServiceBus. I've also gotten questions about things that people wanted to do with Agatha but that aren’t really supported. And I've also noticed that people were coming to my site with search keywords like “agatha vs nservicebus”. The thing is, they are hardly comparable.

Agatha is a Request/Response Service Layer framework. It basically supports synchronous and asynchronous Request/Response style communication and tries to make it as easy as possible to write a service layer for that type of communication. Apart from being easy to use, it also encourages a clean implementation of your service layer and a way to minimize the repetitiveness of cross-cutting concerns. It also enables you to get better performance than with typical Remote Procedure Call or Remote Method Invocation style service layers because it will try to minimize roundtrips by batching requests and responses together. In the next release, it will also offer a caching layer so certain requests (depending on how you set it up) don’t need to be processed and cached responses can simply be returned. There’s also support for one-way requests (or fire-and-forget requests) but it has the same downsides as one-way requests with typical WCF services have. That’s pretty much all it does. That’s probably all it’ll ever do. In short, it’s just an upgrade over your typical WCF services.

NServiceBus on the other hand also does Request/Response, but that’s just one of the things it can do. Again, Agatha is essentially an RPC or RMI framework whereas NServiceBus is built on top of one-way messaging technology. This allows for much more possibilities when it comes to communicating between different applications or different parts or processes within the same application boundary. A great overview of those possibilities can be found here. Because it is based on messaging, there are a lot of benefits that you can get from NServiceBus that you’ll probably never get from Agatha. For one, the ‘Store & Forward’ messaging model from NServiceBus might seem similar to Agatha’s one-way requests on first sight, but there are some very important differences. If you send a fire-and-forget request with Agatha, and the service is currently down, then that request is essentially lost. With NServiceBus, the message is stored in a message queue and it will be processed when the target of the fire-and-forget message comes up again. From a reliability point of view, that’s obviously a tremendous improvement over what Agatha can offer you. Another area where NServiceBus truly shines (IMO) is in its Publish/Subscribe implementation. Some people have asked me if I'll ever provide something like that in Agatha and the answer has always been ‘no’. For one, it doesn’t fit into what Agatha tries to do and I don’t see the point in implementing it if a better implementation is available already in another project. There’s plenty more interesting things in NServiceBus to deal with more advanced scenarios, which you’ll have to find out about on your own ;)

So which one is more suitable for you or your applications? As with every technology choice, it depends. Agatha is great for most typical business applications. If you need to communicate between one or more different clients (with different I don’t mean multiple instances but different types of clients like a web app, a silverlight client and/or a WPF client) and one service (you can obviously use it with more than one service as well if you want to) on an application server which encapsulates your business layer then it is a very attractive solution, as long as you don’t need the superior reliability that NServiceBus can offer you. With superior reliability I particularly mean the ability to still process received messages once the service layer comes back up after having been unavailable. In my experience, most business applications don’t really need that, since most of those applications are entirely unusable if the service they depend on is down. In short, if all the possible downsides of using a WCF service layer are not an issue for your project, then Agatha will be a great fit.

If however, you need to maximize reliability, performance and general robustness of a critical enterprise application, then NServiceBus will definitely be a much better choice. If you’re dealing with many distributed parts, NServiceBus will also make things much easier for you than Agatha (or any other WCF service layer for that matter) will. And obviously, if you want to integrate multiple applications while reducing coupling between applications as much as possible, NServiceBus will also be a much better fit than Agatha. With NServiceBus, you’d only need to share an assembly containing the types of the messages. With Agatha, you either need to share an assembly with shared request/response types or use proxies for them and you would also have to know about the other applications since you’d need to access their services directly. This can get quite ugly pretty fast.

And in some cases, you can just use both of them at the same time. At work we have two projects that use Agatha for all of their internal communication within their own application boundary, yet they use NServiceBus to notify each other of certain events. Neither of the applications knows about the existence of the other… the only thing they share is one assembly with some shared message types. We’ve also started working on a new platform where each application in the platform will use Agatha for all of the communication within their own application boundary (since they’re all typical silverlight clients backed by a WCF service style applications) but they will use NServiceBus for every kind of communication that goes outside of the application boundary.

As with many things, it’s just a matter of choosing the right tool for the right job. Hopefully, this post will help some of you make that decision should you need to make it in the future :)

Written by Davy Brion, published on 1/26/2010 9:50:51 PM
Categories: agatha , nservicebus