In NServiceBus we have 2 different types of messages and that usually confuses people. So lets take a look at the different types, what they should be used for and who should use them.

The applicative message

This is the type of message that most users think of when we talk about messages. This is the message that you as a users of the framework will define in order to send data from a client to a server. In NServiceBus this is either a C# class or interface inheriting from the IMessage marker interface and in 3.0 possibly defined by a convention instead. This is the message that you own version and control. Let’s call it a “message” going forwards. To send this message over the wire we need to serialize it to any of the supported formats, json, xml, protobuf etc. The serialized data is wrapped in an envelope that can be given the messaging system for store and forward transmission to its final destination. We call this a TransportMessage. You can send multiple of your applicative messages in the same go making them part of the same TransportMessage using the Bus.Send(object[] messages) signature.

Transport messages, the workhorse of NServiceBus

This is the other of the message types and is owned, controlled and versioned by NServiceBus. If you have read the EAI patterns book this would roughly translate to a envelope wrapper. Other frameworks like Lokad.Cqrs just call them envelopes. The main purpose of this type of message is to carry the serialized payload across the wire but they also contains headers and infrastructure related information such as return address, time sent, id etc. TransportMessages is the physical unit that gets sent between a client and a server so If you look at messages in input queues, error or audit queues etc. it’s this kind of message you would see.

Transport messages forms the unit of work

It’s very important to understand that the unit of work when it comes to message processing is the Transport Message (TM). When NServiceBus picks up a TM from the input queue a transaction is created and all message handlers for the contained messages (remember it could be more than one) will execute in this transaction. If no exception is thrown the transaction will commit and the message will be removed. If any handler throws an error NServiceBus will retry a configurable number of time and then send the entire TM to the configured error queue. Most of the time messages will relate 1-1 to TM’s but you can use this feature if you want messages to be processed a whole inside a single transaction.

Can there be empty transport messages?

You as a user can’t send transport messages without contained messages but NServiceBus sends empty transport messages internally to control things like subscriptions, worker registration for the distributor etc. We call this special type if message a “ControlMessage” and they only contain headers.

Hopefully this post has shed some light when it comes to the different types of messages used in the NServiceBus lingo.

To tell NServiceBus to use Raven for persistence is a easy as calling Configure.RavenPersistence(). This is the default configuration and it uses the following conventions:

1. If no master node is configured it will assume that a Raven server is running at http://localhost:8080, the default url for RavenDB.
2. If a master node is configured the url used will be: http://{masternode}/:8080
3. If a connection string named “NServiceBus.Persistence” is found the value of the connectionString attribute will be used. This pretty much give you full control over which Raven server your endpoint will use

Overriding the defaults

There are a few situations where the default behavior might not be good enough for you.

• You want to use your own connection string – If you’re using Raven for your own data as well you might want to share connection string. You do this by using the Configure.RavenPersistence(string connectionString) signature. This tells NServiceBus to connection to the server specified in that string. The default connection string for RavenDB is “RavenDB”.
• You want to specify a explicit database name – If you want to control the database name in code instead of configuration you would use the  Configure.RavenPersistence(string connectionString, string databaseName) signature. This could be useful in multi tenant scenarios

What database will be used?

After connecting to a Raven server we need to decide which actual database to use. Unless we find a default database specified in the connection string NServiceBus will use the endpoint name as the database name. So if your endpoint is named “MyServer” the database name will be “MyServer”. Each endpoint will have a separate database unless you explicitly override using the connection string. RavenDB will automatically create the database if it doesn’t already exist.

A detailed explanation of the endpoint name concept can be found here.

Can I use the IDocumentStore used by NServiceBus for my own data?

No the Raven client is merged and internalized into the NServiceBus assemblies so if you want to use Raven for your own purposes you need to reference the Raven client and setup your own document store.

How do I look at the data?

Just open a web browser and type the url if the Raven server, this will open up the Raven DB management studio.

The endpoint name is the name of a individual endpoint and must be unique across your solution. The only exception to this rule is when you are scaling out using the master node concept where the same logical endpoint would reside on many physical servers, more on that in a upcoming post.

Conventions that use the endpoint name

The following conventions use the endpoint name to configure various parts of NServiceBus. Note that the {masternode} will be localhost if not specified.

• Local address – The local address (input queue) of your endpoint will be the {endpointname}
• Subscriptions – If you use the Msmq subscription storage the name of the queue used to store the subscribers will be {endpointname}.subscriptions
• Raven database name – The database name when storing data related to the endpoint will be {endpointname}
• Timeout manager address - The address of the timeout manager will be {endpointname}.timeouts@{masternode}
• Gateway input address - The address where the gateway will pickup outgoing messages will be {endpointname}.Gateway@{masternode}
• Gateway url - If you use the http/https channel the url that the gateway will listen to is defaulted to http(s)://localhost/{endpointname}
• Distributor input queue – The input queue of the distributor will be {endpointname}@{masternode}
• Distributor control queue - The control queue of the distributor will be {endpointname}.distributor.control@{masternode}
• Distributor storage queue - The local storage queue of the distributor will be {endpointname}.distributor.storage

How to specify the endpoint name

There are multiple ways that you can define the endpoint name. Lets look at them one by one.

• If not specified by the user the namespace of the class implementing IConfigureThisEndpoint will be used as the endpoint name when running the NServiceBus host. If your project is called MyServer and the config is in the root this will result in the endpoint name “MyServer”. This is the recommended way to name a endpoint.
• If not specified by the user when custom hosting NServiceBus the namespace of the class calling NServiceBus.Configure.With() will be used. For websites this will likely be the namespace of your global.asax.cs
• You can set the endpoint name using the [EndpointName] attribute on your endpoint configuration.
• You can implement the INameThisEndpoint interface on your endpoint config
• If you specify a explicit service name when installing the NServiceBus host this will be used as endpoint name: /serviceName:”"MyEndpoint”
• You can specify a endpoint name when running the NServiceBus host: /endpointName:”"MyEndpoint”

Upgrading from NServiceBus 2.6

Users upgrading from 2.6 need to name their endpoint to the same name as their current input queue since there is no way to explicitly set the input queue in NServiceBus 3.0

In closing

Naming your endpoint will be a very important thing going forward and the framework will try to push you in that direction by not making it to easy for you to override. This will hope fully help us further improve our “convention over configuration” support make it even easier for you to build, deploy and run NServiceBus in the future.

If you plan to use the default storage you need to make sure that you have a Raven server running that your endpoint can connect to. There is two ways to set this up:

1. Download and install RavenDB your self
2. Let NServiceBus do it for you

#2 Auto installation of RavenDB

RavenDB is included with the NServiceBus binaries (not ilmerged) and we have a infrastructure installer that will make sure that Raven is installed on the local machine when the installers are invoked.

The install is only performed if the following is true:

1. Raven persistence is configured for the endpoint
2. A custom connection string is not specified by the user
3. The current endpoint has no master node specified
4. Port 8080 is available
5. Raven is not already installed
6. The install is done using the explicit /installInfrastructure

The installers comes in two flavors, infrastructure installers and regular installers. Infrastructure installers are used for things that are not specific to a given endpoint eg. RavenDB, MSMQ. Regular installers are focused on setting up things that the current endpoint is depending on eg. queues, folders, databases etc. Infrastructure installers are always invoked before regular installers. Although the installers are mainly used internally you could use the for your own purposed as well. Perhaps you need to make sure folders are created, database scripts invoked etc etc.

To create your own installer is as easy as implementing the INeedToInstallInfrastructure<T> interface. The generic parameter gives you a way to restrict your installer to a specific platform. Currently this is either Windows or Azure. If you don’t care about the environment use the INeedToInstallInfrastructure interface instead. To create a regular installer you just implement the INeedToInstallSomething<T> interface again using the T to restrict it to a specific environment.

NServiceBus will scan the assemblies in the runtime directory for installers so you don’t need any code to register them.

When are they invoked?

When using the NServiceBus host installers are invoked as follows:

As you can see the installers are controlled by both the /install command line option to the host and the current profile used. You can of course implement your own profile if you have other requirements.

When self hosting NServiceBus you have to invoke the installers manually using:

Configure.Instance
   .ForInstallationOn<NServiceBus.Installation.Environments.Windows>()
   .Install()

A full example can be found here.

Finally there is one final way to run the infrastructure installers. If you invoke the host with the /installInfrastructure flag the host will run only the infrastructure installers for you without requiring you to configure anything. This can be useful when you setup a new server or a developer machine since that verifies and installs the required infrastructure for NServiceBus to run. The RunMeFirst.bat included in the download is doing just that.

Messages will always be processed in order

The reason why this is not true mainly comes from the fallacy that the network is reliable. This turns out, often painfully, to not be true and usually leads us to use message queuing systems like MSMQ, RabbitMQ, ActiveMQ, etc.

Queuing systems helps us mitigate the “network is reliable” fallacy by providing us with store and forward capabilities. This basically means that a sent message will be stored locally until the infrastructure can transfer the message safely to the recipient. To do all this in a performant and scalable way queuing systems usually won’t supports ordered delivery since that drives the complexity up making the former hard to achieve. You can think of this like a multi lane highway, some cars will travel faster than others, due to size, speed,number of stops,etc. This leads to messages overtaking each other and there for reach their destination in random order. Even if they do arrive in order there can be multiple concurrent threads processing them which leads to even more rearrangement.

But every thing seems to run fine on my machine?

This is what make the whole thing so tricky. Systems under light load will most of the time process messages in order. This means that you will only see this happening when your system is under heavy load making it very hard to debug.

But I use one thread and have a queuing system that guarantees order?

Yes running in this mode will solve give you ordering at the infrastructure level. But unfortunately this is not enough. Communication frameworks like NServiceBus will manage exceptions for you by doing a few retries and  then moving the message to a error queue. This means that a message can be retried from the error queue days after it arrived to its original destination. While this is happening you don’t want your entire system grind to a halt while that message is waiting for someone to issue a retry. This means that even though the infrastructure might support ordered delivery, messages can still be processed out of order at the application level.

So what should we do then?

There is unfortunately no silver bullet available that will solve all this for you. First of all you need to make sure that you never write code that assumes messages to arrive in a certain order, you can’t even assume that they will arrive at all .This leads you to write explicit code to make sure that your application behaves consistently in these situations. This is a good thing!

You can solve this in a few ways, some times the solution is to anchor the message in time so that the receiver can make correct decisions. Publishing price updates is a good example where having a ValidFrom – ValidTo will make processing order irrelevant. Another common way is to use sagas as a way to ensure that message ordering won’t matter. I won’t go into the details but if you’re interested I’ve done a few talks on this subject:

http://andreasohlund.net/2011/03/12/notes-from-my-e-van-talk-on-sagas/

http://skillsmatter.com/podcast/open-source-dot-net/reliable-integrations-nservicebus

In closing

Repeat after me: Don’t assume message ordering when building a message based system

This is not a big deal for commands because they are always used with in the boundary of a single service and it’s fair to require a service to use the same version of NServiceBus. But when it comes to events this becomes more of a problem since requiring all of your services to use the same version of NServiceBus and there by forcing them to upgrade NServiceBus all at once is not an ideal thing.

The solution

There are a couple of ways you can solve this. Jonathan mentions a few in his post. NServiceBus 3.0 has a few changes that will help you further.

1. Not bumping the assembly version for non-breaking changes. Following the semver.org semantics we only change our assembly version when we make changes that are not backwards compatible. This mean that 3.0.1 and 3.0.X will have this same assembly version, fileversion is of course changed for every release/build. This means that as long as you do a nuget update with the -safe flag your service contracts will stay compatible.
2. Support for running in what we call “Unobtrusive” mode. Running in unobtrusive mode removes the need for you to reference any NServiceBus assemblies from your own message assemblies. This obviously removes the problem altogether.

Unobtrusive mode

This new feature in NServiceBus 3.0 allows you to pass in your own conventions to determine which types are message definitions instead of using the IMessage interface.

Below is a snippet that shows how to define those conventions:

 Configure.With()
   .DefiningCommandsAs(t => t.Namespace != null && t.Namespace.EndsWith("Commands"))
   .DefiningEventsAs(t => t.Namespace != null && t.Namespace.EndsWith("Events"))
   .DefiningMessagesAs(t => t.Namespace != null && t.Namespace.EndsWith("Messages"))
   .DefiningEncryptedPropertiesAs(p => p.Name.StartsWith("Encrypted"));

The code above tells NServiceBus to treat all interfaces/classes with a namespace that ends with “Messages” as messages. As you see you can also specify conventions for the new ICommand and IEvent feature as well.

NServiceBus supports property level encryption by using a special WireEncryptedString property and the last line in the snippet is the unobtrusive way to accomplish that encryption.

That is pretty much all there is to it.

A working sample can be found here.

The session was recorded and can be found here:

http://skillsmatter.com/podcast/open-source-dot-net/nservicebus-3/wd-3134

Slides can be found here:

http://www.slideshare.net/andreasohlund/new-and-shiny-things-in-n-service-bus-30

Thanks to Skills Matter doing for recording and for being awesome in general.

Messages implementing ICommand

• Is not allowed to be published since all commands should have one logical owner and should be sent to the endpoint responsible for processing
• Can’t be subscribed and unsubscribed to
• Can’t implement IEvent

var bus = Configure.With()
    .DefaultBuilder()
    .XmlSerializer()
    .MsmqTransport()
    .UnicastBus()
    .SendOnly();

bus.Send(new TestMessage());

The only configuration needed when running in this mode is the destination of the messages you’re sending, you can do this either inline or through configuration.

A working sample can be found here.

That’s really all there is to it.