Wednesday, January 04, 2006

Temporal Sequencing In BizTalk

Temporal sequencing relates to the ability of a message broker to ensure that messages are sent out in the order in which they are received. In a simple broker this is a simple matter as each message is processed one by one in the order in which they are received. At a very basic level this can be insured by writing the messages to the data store as they are received and thenprocessing messages from the top of the table first.

Within BizTalk this is a larger issue since multiple treads of processing can occur at the same time, and there is no guarantee that each thread will execute at the same rate, or that the amount of processing time will be equal for each message. The processing time will be affected directly by the size and complexity of each message. This is further compounded by the ability of BizTalk to be clustered.

The result of this is that BizTalk does not guarantee the order of messages. It is important to realize this is not a design flaw rather an inherent result of the mechanisms used within BizTalk to ensure scalability and efficiency. Its also worth noting that for the large majority of cases this is not an issue and the speed of the product is far more important than the temporal sequencing of the messages.

In HealthCare however there are a number of valid circumstances where temporal sequencing is required. A good example of this would be admission. If a patient is admitted into a hospital typically an A01 would be raised to show that event. If a admitting clerk then realized that the name was incorrect they would make a change to demographics and this would be represented by an A08. If the A08 is received prior to the A01 by a target system then an error condition would be encountered. That situation is a problem but a worse situation would be where two A08 were sent with the second overriding the first. In that case the target system would end up with the wrong data, with no error raised.

One solution to the problem has been provided by Microsoft on GotDotNet ( Briefly (the documentation explains the problem and solution well) the provided solution utilizes an auxiliary database to track message order. When a message arrives into the receive pipeline an entry is made in a tracking database. When the message leaves the send pipeline the message is checked to ensure that it is the oldest. If it is not, the pipeline returns null and the message is not sent.An additional orchestration is used to insert a hold into the process so that other threads in action have a chance to complete, hopefully releasing the older messages.

The solution is quite simple and can be implemented by the code provided or a similar system can be easily coded.

One word of warning though; by including this mechanism in the broker performance will be impacted for a couple of basic reasons. Firstly the orchestration used to insert a hold will take processing power and time, as will the additional pipeline components used. Secondly each message is not free to process and release as fast as possible.

In the end the system is a trade off. If you require temporal sequencing then you will need to sacrifice some performance. If you dont then do not implement this solution.

(Thanks to Suren MachiRaju for posting this)