SOA - SOI - Integration - Biztalk

Why Contract-First? Part 1

2009-11-05T00:45:00.000-08:00

Contract first admits you taking control over your domain artefacts. It also loosely couples the external view from the internal implementation logic(black boxing). Just as a remainder, this doesn't mean you should build a big design upfront(BDUF).

When it comes to primitive SOA e.g. building services, the basic building blocks are schemas and service contracts. In technical terms it translates to xsd's and wsdl's.

Whats needed:
*Definition of the term endpoint -> ABC(adress, binding, contract). The adress specifies an url for the endpoint, the binding defines the concrete part(actual protocol) and the contract defines the abstract parts(messages and interface operations).
*Good understanding of wsdl and xsd

Why?
+Take control over your domain - think about your domain elements
+Better support for enterprise transformation - strategic thinking
+Let the clients have their contracts early delivered, work separately(loosely coupled, no implementation at this phase)
+Higher interoperability
+Artefacts usable in biztalk ports mapping system specific format to canonical format
+You can ignore horrific Microsoft-wizards
+Reusability for shared datatypes
+No vendor lock-in

-Learning curve
-Tooling support

How-to? step-by-step
1.Identify strategies for versioning and namespaces of schemas/endpoints, e.g. "http://YourCompany/Customer/Email/v1"
2.Create reusable datatypes

3.Create messages

4.Create operations
5.Compose endpoints

6.Governance(security, policy)

Code showing contract first!

How-to find the right granularity designing biztalk applications?

2009-10-14T04:27:00.000-07:00

Microsofts definition of a biztalk application:
“The BizTalk application is a feature of BizTalk Server that makes it quicker and easier to deploy, manage, and troubleshoot BizTalk Server business solutions. A BizTalk application is a logical grouping of the items, called "artifacts," used in a BizTalk Server business solution”

I don't think that this definition tells us about the fact what we really are realizing, that is, processes, services, systems functionality and further. We need to answer several questions:
- What function/purpose should the biztalk platform make in our enterprise?
- Should it serve as a Service Bus or an integration platform?

The answers gives us the initial requirements for establishing an initial design and architecture for the biztalk infrastructure.

Initial requirements for the picture below:
- ESB solution
- medium or big biztalk solution
- a lot artefacts to manage(orchestrations, schemas, maps, pipelines, ports)
- support for complex long-running processes
- support for messaging/CBR(content based router)
- different clients demanding similar functionality

Goals:
- reusable services/orchestrations
- easy to manage, deploy and redeploy artifacts
- few, or no dependencies between applications
- loosely coupled services/orchestrations using MsgBox direct binding
- versioning of assemblies supporting different versions, for example a long-running process
- organic growth of artefacts, create applications when needed

Other discussions on the same topic.

Supporting Dual MEP:s reusing same service

2009-10-06T01:34:00.000-07:00

The tenets of SOA...."loosely coupled" and "autonomous" services.
Here i'll show how you can reuse the same service, e.g the same service implementation, supporting different Message Exchange Patterns.
- synchronous two-way request-response pattern
- asynchronous one-way request pattern

* Reusing the same orchestration(service) for both synchronous and asynchronous interaction
* MsgBox direct-binding(e.g no logical/orchestration ports) -> more autonomous and self-contained
* To fulfil the above you have to copy all(*) BTS subscription-properties from inbound to outbound message. You also have to set the property (BTS.RouteDirectToTP = true).
* One way to determine if the request originates from a one-way or two-way conversation is to evaluate the property (BTS.EpmRRCorrelationToken exists "inboundmessage")

Consistent and unified error handling in Biztalk 2006

2009-09-01T04:37:00.001-07:00

To provide your solution an unified and consistent treatment of exception handling in biztalk 2006 these factors must be fulfilled or at least looked into.

• Standardize how application exceptions are detected and caught in the BizTalk envi-
ronment, i.e., messaging and orchestration subsystems.
• Provide common patterns that allow automated processes to react and manage appli-
cation exceptions.
• Provide a loosely coupled exception management pattern that facilitates reuse.
• Develop a common reporting paradigm of application exceptions and their available
message state that applies to any BizTalk subsystem.

The first important thing you have to do is enabling "Enable routing for failed messages" on receive port. The consequence is that instead of suspending the message the message will be published and a several properties will be promoted to the message context.

Now we have the ability to subscribe on all errormessages using System.Xml.XmlDocument for message type and evaluating the property ErrorType in namespace ErrorReport:

If orchestration Activate==true and Filter expression like:

(ErrorReport.ErrorType == "FailedMessage")

If filter on send port:
(ErrorReport.ErrorType == FailedMessage)

Asynchronous integration for service oriented integration using Biztalk Server 2006

2009-08-13T23:54:00.000-07:00

Three different strategies among asynchronous integration using biztalk server 2006.

Client-side asynchrony
Asynchronous integration with polling
Asynchronous integration with callback

Although option 1 may be the easiest to construct, it is essentially a synchronous SOAP interaction that just appears to be asynchronous to the client code because the connection between the client and the service remains active throughout the whole interaction. Option 1 is therefore not a good choice.

The difference between option 2 and 3 is more about where you want to hold the state. Option 2 holds the state on server-side, in this case Biztalk Server. Option 3 holds the state on the web server using the global cache. Both servers are multithreaded and quite efficient at keeping state for many concurrent requests. Both options are good choice and is more about where you have your skills and limitations and which area you're focusing on.

Option 2 requires a little extra work on the BizTalk Server to ensure proper correlation and to allow the concurrent servicing of multiple requests. Option 3 requires additional coding effort in the Web server to maintain and to clear the global cache.

I will highlight option 2 in more detail and discuss a few aspects you have to take care of.

First the client request with a customer id and server responds with a unique correlation id. An import detail here is to initialize the correlation set on the Send shape so the server knows which client should be correlated to the future poll request. Furthermore the process makes asynchronous requests to host systems. (The parallell branch is not true asynchronous, see "Parallel shape behaviour in BizTalk 2004 and >2006" of Yossi Dahan. A better choice for true asynchronues design is using self correlated ports and Start orchestration). The responses from the host systems aggregates into one message and is ready to be sent out. I'm using atomic scope to ensure transactional consistentency when adding the responses to the list.
The client polls the server with the actual correlation id. The request shape follows the initialized correlation set to correlate the correct instance. If the service has completed its work the server responds with the result otherwise it responds with an empty message. If not a correct correlation id is sent in the server responds with a soap exception. The left branch uses infinite looping to ensure listening for client polling requests. One important thing to mention is that you have to set the orchestration to long running and a corresponding timeout(in my case 60 seconds), otherwise you could increase the actual instances and bloating the server resources.