Understanding Content Based Routing in EAI Bridges

As I promised, this will be my second demo about Windows Azure Service Bus EAI/EDI Labs – April 2012 release

This second demo is a Customer Orders Routing scenario and intends to show some of the Enterprise Application Integration (EAI) capabilities in the cloud, especially how we can perform content-based routing in EAI Bridges. The goal is to send order messages to the cloud (EAI Bridge), apply transformation, enrich properties in the Bridge (like Property promotion), and Route messages; and treat them in different ways – different transformation, different endpoint, like queues or different lob targets:

• If it is a top-priority message, we need to put the message in the SQL table on-premise: CustomerOrders.
• If it is a normal message, we put it in an Azure Queue: CustomerOrderQueue (that we need to create in the AppFabric lab portal.
• And if it was an invalid message, we put the message in another SQL table on-premise: IncorrectOrders.

This demo will also show how we can apply Route Filters and Route Action.

I will not show all the steps needed to develop this project, like create schemas or maps. You can learn these steps in my previous post: Using EAI Bridges and LOB Target to connect to on-premises SQL Server. I will focus on the development/configuration of the EAI Bridges.

Note: The source code of the project that is available to download will contain all the scripts to create the database used in this demo. The database is composed of two simple but different tables:

• CustomerOrders (PONumber, CustomerName, ProductName, and Quantity)
• IncorrectOrders (PONumber, CustomerName, and OrderDate)

Create Service Bus EAI Project

The first thing, of course, is to create a new Visual Studio project:

• Open Visual Studio 2010, on the File menu, point to New, and then click Project.
• Under Installed Templates, select Visual C# > ServiceBus template.
• And then select the Enterprise Application Integration project type.
• Finally, give a name to your project: CustomersOrdersRoutingDemo.

By default, BridgeConfiguration.bcs is open after we create a new project. For now, we will close this file (window).

After we create our project, CustomersOrdersRoutingDemo, we need to add the following input schema:

• A best practice is always to rename the root node. In your sample, rename it to CustomerOrder.
• Under CustomerOrder, create the following structure:

The next step is to generate the SQL Schemas, again, this process is explained in detail in my last post, and the basic difference between this demo and the last is that we need to select to Insert operations so:

• In the Enterprise Application Integration project, from Server Explorer, expand ServiceBus Connect Servers.
• Then expand the server (URL) and the LOB Types.
• Right-click SQL type and select Add SQL Target.
• The Add a Target wizard starts. In the welcome page, click Next.
• On the Connection Parameters page, specify the details for the SQL Server to connect to (name of the server, the SQL instance, and the name of the database (Catalog)) and the credentials to use for the connection (leave the option Use Windows credentials checked). Click Next.
• On the Operations page, we will specify the operations that we want to provide. In this demo, I will choose only the insert operation:
  • From the left box, expand Tables, expand Tables > CustomOrders, select Insert, and then click the right arrow. The Insert operation must now be listed under the Selected operations section.
  • From the left box, expand Tables, expand Tables > IncorrectOrders, select Insert, and then click the right arrow. The Insert operation must now be listed under the Selected operations section.

• Click Next.
• Then you need to continue with the remaining steps of the Wizard until the end.

The next step is to import these schemas to our project:

• Select the Project name and in the LOB Target created previously, right-click and select the option Add Schemas to <project name>.
• This will open a new window where you can define:
  • A file name prefix, in my case, I put sqldemo_.
  • Folder Name, leave the default.
  • And set the Lob credentials, leave the default value.
• Click OK.

You will notice that a folder with the SQL Schemas was created in your project.

Create a map

The next step is to create two maps that will transform the input message CustomerOrder into the desired message type to send to the SQL.

The first map that we will call CustomerOrderToSQLOrderInsert.trfm will transform the input message “CustomerOrder” to the insert schema of the CustomerOrders table. This is a very basic mapping transformation.

The second map, CustomerOrderToSQLIncorrectOrderInsert.trfm, will transform the input message “CustomerOrder” to the insert schema of the IncorrectOrders table. Again, this is a very basic mapping transformation.

And the only thing we can mention and explain is this new Operation (aka functoid if we make an analogy with BizTalk): Generate Date Time.

If you are a BizTalk Developer, you will be accustomed to using the annoying Date and Time functoid. Annoying because if you need to customize the format of the date… basically you need to create your own scripting functoid because the output format of the Date and Time functoid is always CCYY-MM-DDThh:mm:ss. But now we have a brand new and amazing operation to generate date/time, and guess what… it supports the customization of the output format date!!!

And is not the only news in terms of Data / Time Operations! Again, if you are a BizTalk Developer know how annoying it is to do date conversions inside BizTalk Mapper, but now you will also have:

• DateTime Reformat operation: where we can set the input format of the date and define the correct output format

• And Adjust TimeZone operation: where for a specified input date, we set the input TimeZone and define the correct output TimeZone of the date

Please! When will these functoids be AVAILABLE in BizTalk maps?

Configuring and deploying an EAI Bridge

Finally, the most important part of this demo is the creation and configuration of our EAI Bridge (commonly referred to as pipelines).

As I mentioned in the beginning, this will be our goal: we will send messages to the cloud (EAI Bridge), apply transformation, enrich properties in the Bridge (like Property promotion in BizTalk), and Route messages; and treat them in different ways – different transformation, different endpoint like queues or different lob targets:

• If it is a top-priority message, we need to put the message in the SQL table on-premise: CustomerOrders.
• If it is a normal message, we put it in an Azure Queue: CustomerOrderQueue (that we need to create in the AppFabric lab portal.
• And if it was an invalid message, we put the message in another SQL table on-premise: IncorrectOrders.

To accomplish that, we need to:

• In the Solution Explorer window, open the bridge configuration surface file: BridgeConfiguration.bcs.
  • If I have to make a comparison between BizTalk, this file, for me, will be like the BizTalk Administration Console, where we create our receive port and receive location, define the pipeline associated with this location, where we are able to set a transformation, create filters to subscrive the messages, and create send ports – of cours,e without orchestrations.
• The first thing we need to do is to set our Service Namespace:
  • Right-click anywhere on the bridge configuration surface and click Properties, and then for the Service Namespace property, specify the registered service namespace.
• If we open the Toolbox windows and drag and drop an XML One-Way Bridge from the toolbox to the Bridge Configuration surface, and set this entity name to OrderBridgeRouting.

• STEP 1: Double-click the XML One-Way Bridge on the bridge configuration surface to configure the bridge.
  • In this Bridge, we will not apply any transformation; we will only specify the input schema and enrich the message (similar to property promotion in BizTalk) by extracting specified properties from the message. The extracted properties will then be used to route the message to different destinations.
  • On the XML One-Way Bridge design surface, within the Message Types box, click the plus icon (+) to open the Message Type Picker dialog box.
    • In the Message Type Picker dialog box, from the Available message types box, select the CustomerOrders message type, click the right arrow icon “right arrow” to associate the request schema with the Bridge, and then click OK.

• We can validate the message by setting the property Report Warnings as Errors to true in the Validate stage.
• Note: Because we need to route the message based on some of its properties, we need to enrich the message in the Enrich stage. Again, if we are a BizTalk Developer we are accustomed to promote the properties directly in our schemas, in ServiceBus EAI project this is slightly different and we need or can do this in two different stages: in the pre-transform Enrich stage and in the post-transform Enrich stage – this means that we can extract values from the message before and after we apply a transformation.
• Configure the pre-transform Enrich stage to extract the value of the Priority and Quantity elements from the source schema.
  • Within the Enrich stage, select the Enrich activity, and then from the Properties pane click the ellipsis button (…) against the Properties property to open the Property Definition dialog box.

• In the Property Definitions dialog box, click Add to open the Add Property dialog box. In the Add Property dialog box, do the following:
  • In the Source (Read From) section, set:
    • The Type field to Xpath from the drop-down list.
    • Specify the XPath query to extract the value of the Priority element from the request schema in the Identifier field (you can get the XPath query from the Instant XPath property in the schema).
    • In the Message Type field, you need to specify the schema that you are trying to extract the value from, so you need to select the CustomOrder schema.
  • In the Property (Write To) section, set:
    • In the Property Name field, you need to give the name to your property (promote property) that you are creating; in this case, put: Priority.
    • And in the Data Type field, you need to specify the data type for the property. Specify long.
• Click OK in the Add Property dialog box

• Repeat the same steps, this time to extract the Quantity element with the following values:
  • In the Source (Read From) section, set:
    • Type: XPath.
    • Identifier: Specify the XPath query to extract the value of the Quantity element from the request.
    • Message Type: select the CustomOrder schema.
  • In the Property (Write To) section, set:
    • Property Name: Quantity.
    • Data Type: long.

• Click OK in the Property Definition dialog box.
• Save the bridge configuration and go back to the Bridge Configuration designer surface.

Now we need to add to the Bridge Configuration designer surface the following entities:

• Two more XML One-Way Bridges and call them:
  • OrderTopPriorityBridge.
  • And InvalidOrderBridge.
• The SQL LOB Target that we created earlier, to accomplish that, we need to:
  • Drag and drop an SQL LOB Target (sandroazureeaiedibus/sql/customorder) from Server Explorer windows to the Bridge Configuration surface.
    • We can add the SQL LOB Target only one time or twice, as you will see in the pictures below.
• One Queue and set the entity name to CustomerOrderQueue.
• And connect them as the image shows

Or

In this demo, we will follow the first diagram!

Now we need to configure all the entities that we just added to our Bridge Configuration designer surface:

• STEP 2: Click the connection between XML One-Way Bridge OrderBridgeRouting and the XML One-Way Bridge OrderTopPriorityBridge.
  • In the Properties window, click the ellipsis (…) button for Filter Condition.
    • In the Route Filter Configuration dialog box, set the filter condition to Filter and add the following condition:
      • Priority = 100 AND Quantity > 0
      • Note 1: this editor is rudimentary and doesn´t have IntelliSense, so you need to write correctly the names of your properties
      • Note 2: The names of the properties are the ones we created earlier in the pre-transform Enrich stage in the OrderBridgeRouting bridge.

• Click OK.
• In the Properties window, leave the Route Action empty.
• STEP 3: Repeat the same steps this time for the connection between XML One-Way Bridge OrderBridgeRouting and the XML One-Way Bridge InvalidOrderBridge.
  • In the Route Filter Configuration dialog box, set the filter condition to Filter and add the following condition:
    • Quantity < 0

• Click OK.
• In the Properties window, leave the Route Action empty.
• STEP 4: Repeat the same steps this time for the connection between XML One-Way Bridge OrderBridgeRouting and the Queue CustomerOrderQueue.
  • In the Route Filter Configuration dialog box, set the filter condition to Filter and add the following condition:
    • Priority != 100 AND Quantity > 0

• Click OK.
• The XML bridges support both SOAP and REST message protocols. So, while sending a message from an XML One-Way Bridge to a Service Bus queue or a topic, you can specify whether the outgoing message will be a SOAP or a REST message:
  • In the Properties pane, for the Messaging Protocol property, from the drop-down list, select either SOAP or REST depending on how you want to send the message. I choose to select REST.
  • In the Properties pane, leave the Route Action empty.

Situation point:

• He had successfully configured the first XML One-Way Bridge (STEP 1) – which will serve as our input channel, i.e., we will send the messages to this runtime address.
• And he had successfully configured the connections between:
  • STEP 2: XML One-Way Bridge “OrderBridgeRouting” and the XML One-Way Bridge “OrderTopPriorityBridge”
  • STEP 3: XML One-Way Bridge “OrderBridgeRouting” and the XML One-Way Bridge “InvalidOrderBridge”
  • STEP 4: XML One-Way Bridge “OrderBridgeRouting” and the Queue “CustomerOrderQueue

That basically receives the message and routes it based on the context of the message to another XML Bridge or to a Queue.

Now we need to configure the destinations, the XML Bridges that are responsible for transforming messages in the format required by the destination, and the connections between them:

• STEP 5: Configure the XML One-Way Bridge OrderTopPriorityBridge.
• STEP 6: Configure the XML One-Way Bridge InvalidOrderBridge.
• STEP 7: Configure the connection between the XML One-Way Bridge OrderTopPriorityBridge to the SQL LOB Target entity.
• STEP 8: Configure the connection between the XML One-Way Bridge InvalidOrderBridge to the SQL LOB Target entity.
• STEP 9: And configure the Queue.

To configure the remaining steps, we need to:

• STEP 10: Configure the XML One-Way Bridge OrderTopPriorityBridge.
  • Double-click the XML One-Way Bridge OrderTopPriorityBridge on the bridge configuration surface to configure the bridge.
    • On the XML One-Way Bridge design surface, within the Message Types box, click the “plus” (+) icon to open the Message Type Picker dialog box.
    • In the Message Type Picker dialog box, from the Available message types box, select the CustomerOrders message type, click the right arrow icon to associate the request schema with the Bridge, and then click OK.
  • Within the Transform stage, select the Xml Transform activity, and then from the Properties window click the ellipsis button (…) against the Maps property to open the Map Selection dialog box.
    • From the list of maps displayed in the dialog box, select CustomerOrderToSQLOrderInsert.trfm and click OK.

• Save the bridge configuration and go back to the Bridge Configuration designer surface.
• STEP 11: Configure the XML One-Way Bridge InvalidOrderBridge.
  • Repeat the same process for the XML One-Way Bridge InvalidOrderBridge.
    • In the Message Type Picker dialog box, from the Available message types box, select the “CustomerOrders” message type, click the right arrow icon to associate the request schema with the Bridge, and then click OK.
    • Within the Transform stage, select the Xml Transform activity. From the list of maps displayed in the dialog box, select CustomerOrderToSQLIncorrectOrderInsert.trfm and click OK.
    • Save the bridge configuration and go back to the Bridge Configuration designer surface.
• STEP 12: Configure the connection between the XML One-Way Bridge OrderTopPriorityBridge to the SQL LOB Target entity
  • To set the filter condition, we need to:
    • Click the connection between the XML One-Way Bridge OrderTopPriorityBridge to the SQL LOB Target entity.
    • In the Properties window, click the ellipsis (…) button for Filter Condition.
    • In the Route Filter Configuration dialog box, set the filter condition to Match All.
    • Click OK.
  • To set the Route action so that the outgoing message to the LOB application has a SOAP action header.
    • Open Server Explorer and navigate to the SQL LOB Relay we created earlier. Right-click the relay, click Properties, and for the Operations property, copy the value of the first operation.
    • On the Bridge Configuration surface, click the connection between the XML One-Way Bridge OrderTopPriorityBridge and the SQL LOB Target entity.
    • In the Properties window, click the ellipsis (…) button for Route Action. In the Route Actions dialog box, click Add to open the Add Route Action dialog box. In the Add Route Action dialog box, do the following:
      • Under the Property (Read From) section, select Expression and then paste the value that you copied. Important: You must always specify the value for an expression within single quotes.
      • Under the Destination (Write To) section, set the Type to SOAP and the Identifier to Action.
      • Click OK in the Add Route Action dialog box to add the route action. Click OK in the Route Actions dialog box

• STEP 13: Configure the connection between the XML One-Way Bridge InvalidOrderBridge to the SQL LOB Target entity.
  • Repeat the same process for this connection.
    • Filter condition: Match All.
    • The Route action: Open Server Explorer and navigate to the SQL LOB Relay we created earlier. Right-click the relay, click Properties, and for the Operations property, copy the value of the second operation.

• STEP 14: And configure the Queue:
  • Clickthe Queue entity:
    • In the Properties window, in the Relative address Property, you need to define the name of the queue that you create in the AppFabric Lab Portal, in my case: CustomerOrderQueue.

Now we are ready to build our solution and deploy to the cloud!

What are Route Filters and Route Action?

Route Filters: The bridge enables you to route messages to the intended recipient based on filters. The filters are set to certain values that are passed as part of the message. For example, if the value in the element <Recipient> in the XML message is set to Finance, send the message to Service A. Otherwise, send the message to Service B.

Route Action: Route actions help in bridging protocol mismatch. For example, consider two applications, App A and App B. App A sends messages by using the REST protocol while App B receives only SOAP messages. If App A sends the message to the bridge instead, the bridge includes SOAP headers on the message as part of Route Action. The bridge then sends the message over to App B.