Neudesic Blogs

Passion for Innovation

BizTalk Server & Dynamics CRM Online: Integration with On-premises Lines of Business Systems Using BizTalk Server 2013

This is the eighth blog post in this series.  You can find the previous posts here:

·         Innovations in Integration on the Microsoft Platform

·         Lowering Barriers to Innovation: BizTalk IaaS and Walkthrough

·         BizTalk Sever 2013 Enhancements

·         Windows Azure BizTalk Services EDI Overview Including Portal Configuration

·         Windows Azure BizTalk Services - Utilizing EAI Bridge

·         Windows Azure BizTalk Services: Integrating with on-premises LOB systems using

the BizTalk Adapter Services

·         ReST (Representational State Transfer) in BizTalk 2013



Companies that leverage Dynamics CRM Online want to take full advantage of its rich set of capabilities by integrating with existing lines of business systems that are available on premise. It is common nowadays for organizations to have other lines of business systems that are hosted on premise, behind the firewall.


In this initial blog, I will show one solution that is available for integrating CRM Online with BizTalk 2013 hosted on premise. In the first scenario, I will demonstrate how data can be sent from the BizTalk application to CRM Online. 


BizTalk to CRM Online Integration

     Fig. 1: Overall Architecture of main components

CRM Online exposes the Organization Service (IOrganizationService). This is a built-in CRM WCF service that is used to interact with CRM data and metadata.


I will be simulating creating a new account via BizTalk and show how to send it to CRM Online. In order to run this scenario, you will need the following:

  • CRM Online account
  • BizTalk Server 2013


Summary of steps (additional details and screenshots below):


1.       Navigate to the CRM online organization instance to retrieve the OrganziationService URL.

2.       Use BizTalk WCF Service Consuming wizard to generate schema from the Organization Service web service.

3.       Create source schema (Account schema).

4.       Map source schema to the CRM online schema generated in step 2.

5.       Create orchestration (CreateAccount orchestration).

6.       Create custom binding and behavior; register them in BizTalk.

7.       Configure WCF custom send port.

8.       Configure BizTalk.

9.       Simulate sending message by dropping the account sample file in the predefined folder.

10.   Log into CRM Online and verify that the account has been created.




Using Organization Service to interact with CRM Online Walkthrough:


Step 1 - Log into CRM Online to retrieve Organization Service URL;

Go to Settings - > Customizations -> Developer Resources


Step 2 – Use BizTalk Publish Wizard to generate the schema and the binding file.

In BizTalk solution right-click schema project, select Add and then Add Generated Items; select Consume WCF Service Wizard; navigate through the wizard and enter the URL of your Dynamics Organization Service endpoint ( the url discovered in Step1); after clicking the Get button, click ‘Next’ and select Import.






Step 3 – Replace the generated Organization Service schema with the BizTalk schemas that are provided via CRM SDK (\SDK\Schemas\CRMBizTalkIntegration)


Below is the organizationservice_schemas_microsoft_com_xrm_2011_contracts_services schema. Note that the schema provides CRUD operations in addition to the other CRM-related operations. The nodes that are being used to create a new account are highlighted below. The schema provided by CRM is flexible; it exposes a key-value pair; the key maps to the CRM entity attribute (i.e. name, address1, city, or phone). LogicalName maps to the CRM entity name (i.e. account).


Step 4 – Create account schema.

Step 5 – Map account schema to the expected CRM schema format.




Step 6 – Create orchestration in BizTalk. The orchestration is receiving the message via rcvAccountPort; it then uses Construct Message shape to map the message received to the expected Organization Service schema and sends it to CRM. The response message is captured and written to the file system.  


Step 7 – Deploy BizTalk artifacts to BizTalk.


Step 8 – Configure OrganizationService send port.


Import the binding configuration (OrganizationService_Custom.BindingInfo) generated in Step-2. Rename the port from auto generated WcfSendPort_OrganizationService_CustomBinding_IOrganizationService_Custom to OrganizationService.


CRM Online is using Windows Live Id authentication. In order to configure the BizTalk send port to authenticate with CRM Online instance, a custom WCF behavior has to be built and registered in the Global Assembly Cache (GAC) and the configuration file (machine.config or in BizTalk WCF-Custom send handler).


Mikael Håkansson’s blog post provides a detailed explanation with the sample code for creating the custom WCF behavior.  I leveraged it for this demo. See below the detailed steps necessary to configure the OrganizationService send port.



OrganizationService send port.


OrganizationService Send Port – General tab view.

Note that no changes have been made to the custom binding generated in Step 2.



OrganizationService Send Port – Binding tab view.

Select customBinding as the Binding Type.


OganizationService Send Port – Behavior tab view

I am using liveIdAuthentication custom behavior which requires the following three properties:

  • crmuri  the organization service url   (
  • username - CRM username
  • password – CRM password


Step 9 – Start BizTalk application.

Drop the sample file that contains account-related information in the predefined incoming folder.


Sample file:

Verify in CRM Online that the account has been added.



Verify the response is received in the expected file system folder. 


The response contains the account id which was automatically generated when the new account entity was created in CRM.

Response Message:

The scenario that was presented above used the WCF SOAP Organization Service to integrate with the CRM Online application. Another option typically used for interacting with CRM Online is for BizTalk to leverage a proxy service which in turn uses CRM SDK API to authenticate and communicate with CRM; crmsvcutil tool is part of CRM SDK and can be leveraged to create early bound entity CRM classes that can be used in the proxy service. The proxy service approach is recommended for more complex integration scenarios.


Useful Resources:




Neuron ESB and RabbitMQ

This is the third post in our Neuron-ESB series. You can find the first two posts here:


Since 2005, most of my time as a developer / consultant has been spent working with Microsoft BizTalk. Microsoft introduced the ESB Guidance (now referred to as the ESB Toolkit) to BizTalk 2006 R2. This introduced capabilities which allowed BizTalk to function as an Enterprise Service Bus. However, I have sometimes found it to be very difficult to get that ESB to do what I want.

I have spent much of the last two years working with an Enterprise Service Bus called Neuron ESB. While no product is perfect, I find that getting it to do what I want generally takes much less work than I was used to, and I'm very happy about that. When I started with Neuron, it supported 4 basic internal transports - Peer, TCP, MSMQ, and Named Pipes. Starting with Neuron ESB 3.0, it now has an additional transport, RabbitMQ. This allows durable, guaranteed, and reliable messaging. The same capabilities are available through MSMQ, but I'm pleased to have an alternative, particularly since RabbitMQ is open source.

The most difficult part of my experience with RabbitMQ was getting it installed properly. Apparently the installer for Neuron ESB can handle that for you automatically (if I had read the manual beforehand I would have known that). In any case, here's what I did to install it.

I first tried to download RabbitMQ and install it. It runs on top of Erlang, so I saw a warning to install Erlang first.

I downloaded Erlang from
here. When I first tried to run the Erlang install, it complained about the installed version of the Visual C++ Redistributable. Once I uninstalled the latest versions of it (both 32 and 64 bit versions, although I'm not sure if both were necessary), the Erlang install ran without complaint. After that, the RabbitMQ install also ran through without complaint.

In Neuron, every messaging application you want to create should start with a topic. The topic defines the transport, QoS (Quality of Service), and auditing parameters. So I created a topic and chose RabbitMQ as the transport. To do that, I clicked on Messaging at the bottom left of the window, which navigates to the Messaging area of the application. Then I clicked Topics on the left hand side, and then New on the upper right frame:


To test the topic that uses RabbitMQ - I needed to create a publisher to send messages using the topic, and a subscriber to receive messages. The publisher is an abstract representation of a real world entity (SQL, CRM, C# app) which will be the message source, and the subscriber is an abstract representation of the receiver of the message. Here's what the publisher looks like in Neuron. To create the publisher, I clicked Publishers on the left and then clicked New on the upper right frame. Then I clicked the Edit Subscriptions button to add the ability to publish to the topic.

Creating a subscriber is equally trivial, click on Subscribers, create a new one, and then Edit Subscriptions to allow the subscriber to receive messages with the desired topic. Both publishers and subscribers are Parties in Neuron, and it is possible for a Party to be both a publisher and a subscriber.

After creating the publisher and the subscriber, two queues have now been created in RabbitMQ. To see them, click Deployment on the lower left, followed by RabbitMQ on the left navigation.

Actually testing this scenario highlights one place where Neuron really shines - an easy to use test client is provided which will allow sending messages through the system without needing to connect to any of the resources that will be used in the live environment. We'll need two instances of the test client, one to represent each role (publisher and subscriber). Click the Tools menu all the way at the top of the Neuron ESB Explorer window, then choose 2 Test Clients. In each client, use the drop down menu to choose the Party Id for each test client, and then click Connect. One client should be the publisher party, and the other the subscriber. If you use the Send tab on the publisher client to type and send a message, it will show up on the Receive tab of the subscriber client.

To get more of a sense of how all of how the queuing transport works, I recommend trying other scenarios. For example, try disconnecting the subscriber client temporarily, and then sending a message using the publisher. You will notice that the next time you connect the subscriber, it receives the queued message. Also, try connecting multiple clients as subscribers. When a message is sent, only one client will get the message. That's because all of the subscribers are pulling the message from the same queue, and once it's empty, no one else will see it.


Here are a few resources you can use:

White Paper: Implementing the Scatter Gather ESB Pattern with Neuron Pipelines



Posted: Jan 23 2014, 01:24 by Steve.Harclerode | Comments (0) RSS comment feed

Categories: Connected Systems | Neuron | Neuron ESB | Service Bus

Service Bus Brokered Messaging Overview

This is the third blog post in this series.  You can find the previous two posts here:

·        Windows Azure Service Bus Overview

·        Service Bus Relay

Service Bus Brokered Messaging


Service Bus brokered messaging provides enterprise class asynchronous messaging capabilities hosted in Windows Azure datacenters. The brokered messaging scheme can also be thought of as asynchronous or “temporally decoupled” messaging. The messaging infrastructure reliably stores messages until the consuming party is ready to receive them. This allows the components of distributed application to be disconnected.

The Service Bus provides Queues that can be used for point-to-point messaging and Topics & Subscriptions that can be used for publish-subscribe messaging. All data communicated through the Service Bus brokered messaging services are encapsulated in messages.

The Brokered Message represents the unit of communication between Service Bus clients and the serialized instances of the Brokered Message objects are transmitted through a wire when messaging clients communicate via queues and topics. The diagram below characterizes the Brokered Message structure.

Why Service Bus Brokered Messaging

The decoupled communication has many advantages by leveraging asynchronous, or decoupled messaging features that support publish-subscribe, temporal decoupling, and load balancing scenarios using the Service Bus messaging infrastructure. The Service Bus brokered messaging service provides a number of capabilities that make it an attractive choice for implementing enterprise class level messaging capabilities:

ü High Availability: The asynchronous and durable nature of messaging features allows developers to build applications that provide high availability for their customers and the hosting Azure data centers ensure the high availability of   Service Bus brokered messaging services.


ü Reliable Message Delivery: It is vital that messages sent from the source application be successfully delivered to a destination application, and the reliability of message delivery is often an important factor. BizTalk developers are very  familiar with the  concepts of durable messaging as all messages passing through a BizTalk message channel will be persisted in a SQL database. For web service operations WS-* standards such as WS-ReliableMessaging and WS-Reliability can be used to improve the reliability.

Reliable messaging systems will typically use durable storage to store messages that are in-transit. For example, if the middleware application receiving and processing messages from a queue suffers an outage, the front-end sending application can still place messages on the queues and topics. The request message will be stored in a durable store and can be received and processed when the receiving application comes back online.

The Azure Service Bus queues, topics and subscriptions leverage the high availability data storage services provided by the Windows Azure platform to ensure the messages are persisted reliably. The enqueueing and dequeuing of messages using transactions can help to ensure reliability in messaging applications.

The following semantics can be used to describe message reliability:

  • At-Most-Once Delivery
  • At Least Once Delivery
  • Exactly Once Delivery
  • Ordered Delivery


        Receive and Delete is fastest (At-most-once semantics). The Message will be lost if receiver crashes or transmission fails.


      Peek Lock (At-Least once semantics) - Message is locked when retrieved and reappears on broker when not deleted within lock timeout.

      Session and Peek Lock - Message is locked along with all subsequent messages with same session-id ensuring order.

ü     Low-Latency: Azure Service Bus brokered messaging provides durable messaging with low latency, however as the messaging infrastructure is hosted in Azure datacenters, you will probably find that the latency from Service Bus brokered messaging is proportional to the physical distance between the applications and the datacenter used for hosting.

ü     Scalability: Azure Service Bus brokered messaging services are hosted in Windows Azure datacenters and are able to scale automatically. Sudden increases in message load will be managed by the Azure hosting environment and additional resources allocated to handle the demand.

ü    Features: Structured message content, serializable message body, sophisticated message receipt semantics, message sessions and correlated messages expose send endpoint and listen endpoint.

ü    Programming Models: REST Interface, Direct Model (Managed .Net API) and WCF Model (NetMessagingBinding Class).


Queues offer First In, First Out (FIFO) message delivery to one or more competing consumers. That is, messages are typically received and processed by the receivers in the order in which they were added to the queue. When using queues, components of a distributed application do not communicate directly with each other, they instead exchange messages via a queue, which acts as an intermediary. Although the Service Bus brokered messaging infrastructure is hosted in Windows Azure data centers, the sending and receiving applications can be cloud-based or on-premise.


Here are examples of a few typical scenarios that can be implemented using Queues:

Load Leveling - Receiver receives and processes at its own pace. It can never be overloaded, can add receivers as queue length grows and reduce receiver if queue length is low or zero. Queues handle traffic spikes by never stressing out the backend. Please refer to the image below for an illustration:

Offline/Batch - Allows taking the receiver offline for servicing or other reasons. Requests are buffered up until the receiver is available again. Please refer to the image below for an illustration:


Load Balancing - Multiple receivers compete for messages on the same queue (or subscription). Provides automatic load balancing of work to receivers volunteering for jobs. Observing the queue length allows determination of whether more receivers are required.

Fan In - Information into a single queue from a range of data sources. Multi-Stage Aggregation or Rollup fan into a set of queues, perform aggregation or roll-up or reduction and fan further. Please refer to the image belowfor an illustration:


Below are few Queue fundamental and advanced capabilities:

Fundamental Capabilities

Advanced Capabilities


First-In-First-Out (FIFO)

Scheduled delivery



Exactly Once Delivery,
Ordered Delivery

Automatic Dead Lettering, Message Deferral and Poison Message Support


Transaction Support


In-Place Update


Receive Behavior

Blocking With or Without Timeout,

Server-Side Transaction Log


Receive Mode

Peek & Lock,
Receive & Delete

Storage Metrics


Exclusive Access Mode


Purge Queue Function


Lease/Lock Duration

60 Seconds (default)

Message Groups


Lease/Lock Granularity

Queue Level

Duplicate Detection


Batched Receive


WCF Integration


Batched Send


WF Integration


Service Bus Queues provide a number of advanced features such as sessions, transactions, duplicate detection, automatic dead-lettering, and durable publish/subscribe capabilities. They may be a preferred choice for building a hybrid application.

Topics and Subscriptions

In contrast to queues, in which each message is consumed by a single consumer, topics and subscriptions provide a one-to-many form of communication in a “publish/subscribe” pattern. Useful for scaling to very large numbers of recipients, each published message is made available to each subscription registered with the topic.




Topics are similar to the enqueueing end of a queue. Applications send messages to topics in exactly the same way that they send them to queues by using a MessageSender object. Topics provide no interface for dequeuing messages, but instead have a collection of zero or more subscriptions that will subscribe to the messages sent to the topic based on filter rules. Topics provide support for message expiration, deadlettering and duplicate detection. The techniques for sending messages to queues can be used to send messages to topics. Messages are sent to a topic and delivered to one or more associated subscriptions, depending on filter rules that can be set on a per-subscription basis.


Subscriptions are similar to the dequeuing end of a queue. Applications receive messages from subscriptions in exactly the same way they receive them from queues by using a QueueClient or MessageSession object. A topic can have zero or more subscriptions, but a subscription can only belong to one topic. Subscriptions provide no external interface for enqueueing messages. Internally, messages are enqueued on the subscription inside publish-subscribe channel based on routing logic in the subscription filters.

Subscriptions provide support for message expiration, dead lettering and message sessions. As topics provide support for duplicate message detection there is no need for this to be implemented on subscriptions as any duplicate messages should be detected by the topic before reaching the subscription. The subscriptions can use additional filters to restrict the messages that they want to receive. Messages are sent to a topic in the same way they are sent to a queue, but messages are not received from the topic directly. Instead, they are received from subscriptions.


Message Distribution (Taps and Fan-Out) - Each receiver gets its own copy of each message. Subscriptions are independent and allow for many independent 'taps' into a message stream. Subscribers can filter down the messages by interest.

Filter Expressions:

Filter expressions are used to determine which of the messages sent to the topic the subscription will subscribe to. There are currently four types of filter that can be added to a subscription:

ü  SqlFilter – Subscribes to messages based on a T-SQL like expression based using values in the message property dictionary

ü  CorrelationFilter – Subscribes to messages based on the value of the CorrelationId property of the message

ü  True Filter– Messages are always subscribed to

ü  False Filter – Messages are never subscribed to


Filtering allows up to 2000 rules per subscription and each matched rule yields a message copy.

Correlation in Service Bus is required to set up reply paths between sender and receiver. The available three correlation models in Service Bus are Message-correlation (Queues), Subscription-Correlation (Topics) and Session-Correlation.

Message Correlation (Queues):  Originator sets MessageId or CorrelationId, Receiver copies it to reply. Reply sent to Originator-owned Queue indicated by ReplyTo and Originator receives and dispatches on CorrelationId. High throughput needs and work usually completes in minimal time.

Subscription Correlation (Topics): Originator sets MessageId or CorrelationId, Receiver copies it to reply. Originator has Subscription on shared reply Topic with rule filtering on id and Originator receives and dispatches on CorrelationId.

Session Correlation: Originator sets ReplyToSessionId on outbound session, Receiver sets SessionId for reply session and Originator filters on known SessionId using session receiver. It’s a Reliable multiplexed duplex communication.

Brokered Messaging Capabilities In Action

The supported programming models for Queues and Topics in Service Bus are REST Interface, Direct Model (Managed .Net API) and WCF Model (NetMessagingBinding Class). Below are the fundamental classes we will use when working with Service Bus Queues.

NamespaceManager: Used in the management context and provides the ability to perform administrative functions such as creating, inspecting and deleting Queues on your namespace in the messaging fabric.

BrokeredMessage: Used in the runtime context and defines the message that you will send and consume over Service Bus Queues.

QueueDescription: Used in the admin context and describes a Service Bus Queue.

MessagingFactory: Used in the runtime context and as its name implies, acts as a factor for creating classes that allow you to send and receive messages on a Queue using new classes such as QueueClient, MessageSender and MessageReceiver.

QueueClient: Used in the runtime context and allows you to send and receive messages over a Queue.

MessageSender and MessageReceiver: Used in the runtime context and allow you to send and receive messages over a Queue, Topic or both.

Below are the steps to create a Queue sample application to send and receive the messages.

      ü   Create a Service Bus Namespace in the Windows Azure portal.        


     ü   After creating a service bus namespace, create a project in Visual Studio and install Service Bus NuGet package in your application. The NuGet Visual Studio extension makes it easy to install and update libraries and tools in Visual  Studio.


ü     Copy the namespace connection information from portal and set up a Service Bus Connection String in your application (You have to use different configuration mechanism to configure connection string when using with cloud services).


ü To explore or view the created Service Bus Queues and Topics, you can use Visual Studio 2013/2012 Server Explorer or Service Bus Explorer Tool (You can download Service Bus Explorer tool from this link

ü Connect to the Service Bus using Visual Studio or Explorer Tool, provide connection information to connect.


ü  In your application use the below code to create a Queue in Service Bus, you can use QueueExists method to check if a queue with a specified name already exists.


ü    To send messages to Queue, use the code below. Wwhile sending a message is very simple when using the QueueClient, another option is to use a MessageSender class. The MessageSender class is a lower level class that allows you to send messages without thinking about whether you are working with a Queue or a Topic.



ü  Run the sample. After sending the order message to Queue, check it in the Service Bus Explorer, and you will find a newly created Queue and message.



ü  To receive a message from the orders Queue, use the code below. You can also use the MessageReceiver class to receive messages.




We have discussed the NamespaceManager, BrokeredMessage, QueueDescription and QueueClient classes. Now, here are the new classes which we will use for Topics:


TopicDescription: Used in the management context and describes a Service Bus Topic.

TopicClient: Used in the runtime context and allows you to send messages to a Topic.

SubscriptionClient: Used in the runtime context and allows you to receive messages from a Subscription over a Topic.


Below are the steps to create a Topic sample application to send and receive the messages.



        ü  In your sample application, use the code below to create a OrdersTopic


ü  Use the code snippets below to create Subscriptions and filters for the OrdersTopic. The “AllOrders” filter is the default filter that is used if no filter is specified when a new subscription is created. When the AllOrders filter is used, all messages published to the topic are placed in the subscription's virtual queue.

ü  The most flexible type of filter supported by subscriptions is the SqlFilter, which implements a subset of SQL92. SQL filters operate on the properties of the messages that are published to the topic.

ü  The example below also creates subscriptions named "HighRangeOrders" and “LowRangeOrders” with a SqlFilter that only selects messages that have a custom OrderRangeId property greater than 100 and less than 100.


ü  Use the code snippet below to send Messages to OrdersTopic


ü  After running the sample, verify it in Service Bus Explorer. You will find the OrdersTopic and the subscriptions created.

ü  To receive messages from subscriptions: From the client application, I have submitted the message with OrderRangeId = 201. The message has been send to OrdersTopic and delivered to “AllOrders” and “HighRangeOrders” Subscriptions. You can see it in the screen shot below.




ü  From the client application, I have again submitted the message with OrderRangeId = 91. The message has been sent to OrdersTopic and delivered to the “AllOrders” and “LowRangeOrders” Subscriptions. You can see it in the screen shot below.


Azure Service Bus Brokered Messaging Resources


ü  Introducing Queues and Topics in Azure Service Bus

ü  Best Practices for Leveraging Windows Azure Service Bus Brokered Messaging API

ü  Service Bus Queues

ü  Service Bus Topics

ü  Windows Azure Service Bus

ü  Windows Azure Service Bus Brokered Messaging

ü  Windows Azure AppFabric Service Bus Brokered Messaging

ü  Capacity Planning for Service Bus Queues and Topics


If you’d like to learn about our Connected Systems practice at Neudesic, please visit this page:


Posted: Jan 16 2014, 04:38 by Mahesh.Pesani | Comments (0) RSS comment feed

Categories: Connected Systems | Service Bus

Neuron-ESB: Reliable Messaging

This is the second post in our Neuron-ESB series. You can find the first post here:

The result of almost a decade’s worth of expertise solving tough integration problems for our customers, Neuron-ESB is an Enterprise Service Bus middleware and integration platform built on Microsoft’s .NET framework and Windows Communication Foundation technology that is used to connect and coordinate the interaction of a significant number of diverse distributed applications.

Neuron-ESB was recently included in Gartner’s list of “Cool Vendors in Application and Integration Platforms” by Gartner Research.

To continue our blog series on Neuron-ESB, I am going to focus on one of the biggest challenges with Web-based protocols, reliable messaging. In this post, I will show how Neuron-ESB helps enterprises overcome messaging challenges by providing better:

  • Message Routing (content based, static)
  • Message Processing (transformation, enhancement/augmentation)
  • Message Reliability (guaranteed delivery)

A message is information that one party sends to, or receives from, the bus. A message contains both data as well as metadata. Respectively, these are referred to as the payload or header (context) properties of the message. Both are defined as parts of a Neuron-ESB message.

The payload of a Neuron-ESB message can be one of several formats:

  • Serializable .NET object – The ability to pass .NET objects provides flexibility for developers who prefer using objects over XML
  • Binary data – The ability to pass binary data provides flexibility for developers who have to share content that is neither serializable nor XML
  • Text data – Any type of string data
  • XML data – Any XML data. XSD schemas are not required to use XML as the payload

Message Routing

Neuron-ESB uses a “publish and subscribe” messaging infrastructure to abstract and dynamically route, process, and mediate messages between endpoints. Applications and/or services simply publish messages onto the bus, without regard to the type or number of consumers; similarly, they may subscribe to specific messages or groups of messages, without regard to the source of the messages. This frees the developers from spending time on messaging business logic and allows them to concentrate on the specific business logic associated with manipulating the message data.

A subscription is composed of a topic or sub topic, the permission in which a message can be sent to or from that topic (i.e. Send/Receive), and can be further optionally restricted using one or more conditions. A condition is either a pre-existing or ad-hoc filter expression (using predicates) that can include message header properties as well as message content. Sometimes this is referred to as “content-based routing”. Managing subscriptions for publishers and subscribers can be done through the Neuron-ESB Explorer. Users can quickly add topics, wildcards, reusable conditions (i.e. content-based routing) and ad-hoc conditions to define a subscription and activate it instantaneously.


Message Processing

Messages in Neuron-ESB are easily transformed within a process via XSLT or WCF service.

Transforming with XSLT

The Transform – XSLT process step can be used to apply an XSLT/XSL transform to the process message. Additionally, parameterized XSLT/XSL is supported. Parameters can be useful when the same value must be repeated many times within the document.


XSLT in the Transform – XSLT process step window.


The Transform – XSLT process step used in a VETO (Validate, Enrich, Transform, Operate) pattern.

Transforming via WCF

The Transform - Service Process calls a WCF service that implements the IESBTransformationService interface. The step modifies the message format or structure based on the implementation of the IESBTransformationService interface. The interface could be implemented to invoke a Microsoft BizTalk Server map, perform a code transform, call an existing transformation service, etc.

Message Reliability

One of the biggest challenges with Web-based protocols is reliable message delivery. There are many reasons that endpoints may not be available, and in every case the overall reliability of the application is only as good as the least reliable link. Because business applications often require high levels of delivery guarantee, Neuron-ESB provides a configurable policy object with a variety of options to help ensure message delivery.

Neuron-ESB provides a variety of methods to improve the reliability of messaging. By associating a policy with a service or adapter endpoint, the endpoint will execute the policy upon message failure, and can be configured to retry message transmission, log the message to the disk, or republish the message on a new topic. Policies can dramatically increase the robustness of message delivery and error handling.

An adapter policy configured to retry a failed event 5 times, then log to the Neuron audit database.

You Have the Power
Neuron-ESB provides the power and flexibility you expect from an integration platform in regards to message handling. Whether it be dynamically routing your messages based on the content, transforming your data to fit the needs of your client’s strict requirements, or ensuring your important data is delivered to its intended recipient, Neuron-ESB gives you the power to make it happen.

Neuron-ESB 3.0 Available for Download
The Neuron-ESB 3.0 trial edition can be downloaded by clicking on the link below:

Messaging with Neuron-ESB Resources:

If you’d like to learn about our Connected Systems practice at Neudesic, please visit this page:

Posted: Jan 09 2014, 02:47 by Jereme.Downs | Comments (0) RSS comment feed

Categories: Connected Systems | Neuron | Neuron ESB | Service Bus

Windows Azure BizTalk Services: EDI Capabilities

This is the seventh blog post in this series.  You can find the first six posts here:

Windows Azure BizTalk Services is truly an enterprise-ready integration platform that does not require platform  maintenance or management. BizTalk Services is offered by Microsoft on a “dedicated per tenant” basis, including support for SQL Server, Oracle, SAP and Siebel. While Windows Azure BizTalk Services is available for EAI integration and routine ESB-type transformation, this post focuses on its EDI capabilities.

BizTalk Services Setup

There is a 1-month free trial available for first-time users, but after that, Windows Azure BizTalk Services becomes a paid service with service level options and pay-as-you-go plans.

Payment plans such as Developer, Basic, Standard and Premium range from $48.36 to $2998.32 per month.

It is very easy to get started from the Windows Azure management portal. Click on the BizTalk Services Icon on the left menu and then click the “+” icon at bottom of the page.   From here, the process is wizard-driven.

You are allowed to provide a new SQL Server or use one you have already created. I used an existing one for this scenario.  If you create a new SQL Server, you will see some additional screenshots at this point, asking  additional questions like server name, credentials, etc.

You are allowed to use an existing storage or create a new one. You should attempt to keep all artifacts in the same Azure datacenter for performance considerations. Azure will definitely warn you if you did not comply. Should you need to do so, you are allowed to go cross-region. Click Finish. The provisioning process takes about 30 minutes to create the BizTalk Service instance.

 Two exciting new features recently implemented by Microsoft at the request of the developer community are an automatically-created Access Control Namespace and an improved Management Service process.

  • Access Control Namespace – Windows Azure BizTalk Services creates the ACS Namespace for you.  Previously, this process was not well-known or documented and developers had to know to create an ACS namespace that resides in the Active Directory section of the Azure portal.


  • Management Service Process – When you attempt to access your portal by clicking on it and hitting Manage, you will be required to register the new deployment, even though the name and secret were already created. You just have to copy it down and re-enter it. Though it is not as easy as it could be, it is still a  necessary step. Previously, however,  this process required a lot more handholding.


    At this point, go back into the Active Directory and locate the ACS Namespace that Windows Azure BizTalk Services created for you. Since mine was called “wabslive”,  I see “wabslive” as my BizTalk Service Name.  You could go there, or just click on the BizTalk Services portal, click on your new service and click on connections. You will get the information you need.

This will give you the password you need to complete the registration process. Enter this information and you will be in the portal.

Software Requirements

  1. Most browsers that I tested the process with seem to work well (IE, Chrome, Firefox)

  2. Visual Studio 2013

  3. Download and install the BizTalk Services SDK

  4. Depending on the Performance / System requirements you have you should choose the appropriate tier you provision depending on the:

    1. Number of Integration you need to host

    2. How much volume will you have

EDI Product Capabilities

  1. Typical BizTalk Administrator-like capabilities

    1. Partner configuration and Profiles

    2. Partner Agreement

    3. Create Success and Failed Routing

    4. Configure and use certificates

    5. Setup Transports and routes

      1. Success Route

      2. Failure Route

  2. Typical Developer capabilities

    1. Upload and Change Schemas

    2. Upload and Change Maps that need to be used

      The new mapping tool is radically different and has many improvements and is not just a migration of the BizTalk mapper tool. This also changes the eco system a little but for the most part, it seems to operate similarly. Advanced scripting functoid works like compiled code, so you don’t have to worry about scripts slowing down your process.

      You can use this tool to even migrate your existing BizTalk Maps to WAB maps using this tool.

    3. Upload and change certificates that are needed by Partner Profiles

    4. Upload and change custom assemblies



Bottom Line

Microsoft has finally released a platform that fits most businesses’ EDI needs without the usual high cost and need to have BizTalk infrastructure support staff. Capabilities are very light by design so that close to 80% of the requirements are met, and enhancements will be made on a regular basis. This ensures that the promise of an easy EDI platform is available and hosted by Microsoft. In addition, while it may require some custom development and administration, large enterprises will find capabilities like additional routing and orchestrating to be great new features. 



If you’d like to learn about our Connected Systems practice at Neudesic, please visit this page:

Posted: Dec 12 2013, 05:44 by Samar.Roy | Comments (0) RSS comment feed

Categories: Azure | Azure BizTalk Services | BizTalk | Connected Systems

Windows Azure BizTalk Services: Integrating with on-premises LOB systems using the BizTalk Adapter Services

Windows Azure BizTalk Services: Integrating with on-premises LOB systems using the BizTalk Adapter Services


This is the sixth blog post in this series.  You can find the first five posts here:

Hybrid Apps


Microsoft’s new Windows Azure BizTalk Services and BizTalk Adapter Services make it easy to adopt, develop and host an integration solution in the cloud and extend on-premises applications to the cloud, bringing your integration applications to market in a fraction of the time it takes to use an in-house integration system.

This blog post explains how to integrate your on-premises LOB systems, such as SQL, Oracle or SAP, to or through the cloud and how to validate, transform, enhance and enrich messages using Windows Azure BizTalk Services.


The following image illustrates the scenario that we are going to implement:

We are going to host a BizTalk Service within Windows Azure and link it to the BizTalk Adapter Services hosted within the organization’s on-premises environment. We are then going to send messages to the cloud endpoint hosted by the Windows Azure BizTalk Services Bridge, which will validate and transform the message, relay it to the on-premises BizTalk Adapter Service, and then to the on-premises LOB system, in this case, SQL Server.

How to provision Windows Azure BizTalk Services

First, you need to create a Windows Azure account at It will take only a few minutes to set it up.

Once you create the Windows Azure account, you will be presented with all available services. You need to choose “BizTalk Services” as illustrated in the images below:


Create a new BizTalk Service

I created a BizTalk service called “lobservicesdemo”. Once you create the BizTalk Service, you can download the SDK for Visual Studio and install it. Please note that you need Visual Studio 2012 to develop solutions for BizTalk Services.



Once you create the BizTalk Services, you will need to register them. Click the manage icon at the bottom of the BizTalk Service that you just created as shown in the picture below.

Register the BizTalk Service

You need to register the BizTalk Service that you just created.

Enter your BizTalk Service name, issuer name and the ACS shared secret.

For in depth detail on registering the BizTalk Service, please refer to this site.


Once you have successfully registered, the BizTalk Services manage portal will show up, as seen below.

Developing the BizTalk Service Application


Once you have download and installed the SDK, you are now ready to develop the BizTalk Services application.

Open Visual Studio and look for the “BizTalk Services” project template, select “BizTalk Service” and create the project. I created a project called “BizTalkService10” as shown in the picture below.


In the scenario we are implementing, we are using an order message with a specific schema (“CustomerOrdersSchema.xsd”) which is being received by the BizTalk Services Bridge, converted to a schema of an on-premises SQL Line of Business (SQL LOB) table, and then sent to be updated in the SQL Server table via the “BizTalk Adapter Services”.

You can create a simple custom customer order xml schema and add it to the project as shown in the picture below. This is a simple schema with few custom made elements.


Next, you need to generate the schema for your local SQL LOB table. First, you need to make sure that you have installed the BizTalk Adapter Service runtime components. You can also download these MSI’s from here:

There are the 2 MSI that you need to install:

·        WindowsAzureBizTalkServicesSDK.MSI

·        BizTalkAdapterService.MSI


Once you install the MSI’s, the following service will be installed in your local IIS:



Next, within the Visual Studio, you should add the BizTalk Adapter Services endpoints from the available LOB types as shown in the picture below. Open the Server Explorer in Visual Studio and start adding the adapter services for the LOB types. You can add SQL LOB connection to your local table. Find a simple table or create a simple table called “orders” to hold the orders records. The steps below walk you through how to do that.

Add SQL target.

Proceed to the Welcome screen.

Enter your local SQL Server name and the Database name. Make sure your logged in user id has access to the SQL tables.

I added my custom-made Orders tables. Insert operations as shown in the picture below.


You need to specify the run time security, the user id, and a password that has trusted access to the SQL server.

During the creation of the LOB target, you need to specify your Windows Azure Service Bus configuration. This will create a relay endpoint for your LOB target on the cloud. Here is the link explaining how to create the Azure Service Bus for your account. Please refer this article to setup the Azure Service Bus:

Enter the Azure Service Bus details as shown here.

Once you create the LOB Target, it creates an application in IIS and uses the runtime for a specific LOB target. As mentioned earlier, in this case I used SQL Server.

Your LOB target is now available to drag and drop to the BizTalk Services Bridge configuration. The cool thing about this is that it allows you to create the corresponding schemas and add them into your Visual Studio solutions for ease of use within your maps and in your bridges.

Here is how I added my LOB target schemas in to my projects:

Both the source schema (the custom-made customer schema) and the target schema your SQL LOB target schema) are available in the project. You can start using them in the maps and bridges. Please note that if you are familiar with BizTalk Server, bridges are similar to orchestrations and pipelines.

Next, you need to create the map that transforms the source message schema to the target message schema. Right click on the project and add a new item called Map as shown below.

Select and have the source schema for the map as the Customer Orders schema and the destination schema for the map as the Table Insert schema.

Now map the source elements to the destination elements as shown in the picture below. Save the map. You will later need to select this map in the one-way bridge within the Transform section.



Next, open the bridge canvas (.bcs) in the BizTalk Services project. In this scenario, I am using a one-way bridge. In the bridge configuration canvas, drag and drop an Xml One-Way Bridge from the toolbox as shown below.

Now you can drag and drop the LOB systems relay endpoint and connect it to the One-Way Bridge as shown below.

Next, using the Connector from the toolbox, you can link up the bridge and the LOB Services endpoint as shown in the diagram. Use the properties to set the filter condition. I set it to allow all messages to flow through without any restrictions (1=1). Set the route action accordingly. I set the route action as the SOAP Action to insert the record in to the orders table using the operation: “TableOp/Insert/dbo/orders” as can be seen below.

Now you have created the skeleton flow within the bridge canvas. Next, we need to configure the One-Way Bridge to map and transform the customer schema to the SQL LOB table insert schema which is sent as a message through the Service Bus Relay service to the on-premises BizTalk Adapter Services and from there to the LOB SQL Server.

Here is how the One-Way Bridge configuration looks:

In the Message Types section, add the available schemas. Select the CustomerOrdersSchema and the Insert operation for the TableOp/dbo/Orders#Insert schemas as shown below.


Next, in the Transform section, select the map that you created earlier.


Now the project is ready to be compiled and deployed to the cloud.

Right-click on the project and build the project. The image below shows the project was successfully built.


Once the project is successfully complied, it is ready to be deployed.

Right click on the project and click Deploy. Enter your BizTalk Services endpoint. Typically, this would be <yourbiztalkservicename> Then, enter your ACS namespace, issuer name (typically “owner”) and the shared secret password.

Please refer to this link on build and deploy:


Once the project is successfully deployed, it can be verified in the Azure BizTalk Services Portal. See below that under my lobservicedemo BizTalk Service namespace, you can see there are bridges deployed, and under the resources, there are schemas and maps deployed.

Schemas deployment verification


Maps deployment verification

Bridge deployment verification


Once the project is successfully deployed, it is ready to be tested.

I used a web application to post the customer order to the bridge endpoint. Below you can see the test web application which gets the order parameters as input and sends the customer order xml to the BizTalk Services Bridge endpoint.


This is the XML message send to the BizTalk Services Bridge.

This is the status of the transaction.

Sending data to end point: Tracking ID for the message sent is a64d05b9-5ab6-45c8-80e1-508c4d109e6b Message sent successfully.

It can also be tested using the messagesender utility come with the SDK or the samples.

MessageSender.exe <acsnamespace> owner <shared secret pass ******************> orders.xml "application/xml

Verify the local LOB SQL Systems

Now go to your SQL server, where the LOB database is, open up the table, and verify that the record has been inserted. As shown in the picture below, the record was successfully inserted in to the SQL Server table.



This blog post and demo demonstrates how to develop a brand new Windows Azure BizTalk Service and a BizTalk Adapter Service solution. We also successfully tested it with a Web App and verified the on-premises LOB SQL Server was updated. It’s very exciting to develop solutions with BizTalk Services and using the BizTalk Adapter Services. You should definitely try it out.






If you’d like to learn about our Connected Systems practice at Neudesic, please visit this page:

Service Bus Relay

This is the second blog post in this series.  You can find the first post here:


It was once very difficult to make services hosted behind an NAT or firewall visible to the outside world, but now, Windows Azure Service Bus Relay can expose WCF services in a secure manner without changes to your firewall or network infrastructure. WCF services can be hosted inside your network, but listen to incoming requests that are delegated through the Azure Service Bus.

Hybrid Solutions

Businesses are moving applications to the cloud for lower operational costs and easier scaling. In a scenario where a cloud application is required to communicate with an on-premise service, or any other application inside the corporate network behind an NAT or firewall, Service Bus Relay helps those cloud applications securely access the on-premise services without any network infrastructure changes, as illustrated in the image below:


Bindings separate the implementation of the service from how a service communicates with other services or clients. They specify the transport, encoding and protocol details on which the clients and services agree to communicate with each other. With this WCF feature, already existing WCF services can be exposed to the cloud with minimal change just by adding a Service Bus Relay endpoint.

Windows Azure Service Bus SDK provides WCF relay bindings that automate the integration between WCF services and clients. Relay bindings support most of the features of standard WCF bindings such as reliable messaging, WS-* security, web programming model, etc. By design few features are not supported which include atomic transaction flow and transport level authentication. Service bus relay supports one way, request-response and peer to peer messaging.

System Connectivity Mode

The Service Bus environment provides a ConnectivityMode setting that specifies how a service can connect to the Service Bus. AutoDetect is the default value.

Connection Mode



TCP connection to relay service through port 9351.


HTTP connection to relay service.


Automatically select either TCP or HTTP connection using auto-detection mechanism preferring TCP over HTTP whenever possible.

TCP Bindings Options

  • NetOnewayRelayBinding is a one way messaging binding that supports SOAP 1.2 over TCP with binary encoding by default. To use this binding all the operations in theservice contract have to be one-way (IsOneway=true) operations.
  • NetEventRelayBinding is similar to NetOnewayRelayBinding except that many WCF services can register to the same Service Bus address. So any message received at this address is sent to all the services currently subscribed to that address.
  • NetTcpRelayBinding is a two way messaging binding that supports SOAP 1.2 over TCP with binary encoding by default.

HTTP Bindings Options

All the TCP bindings require a WCF client, so in the case of interoperability with non WCF client there are several bindings that provide interoperability. Any client that agrees to standard protocol supported by each binding can call the service. Using these bindings, the WCF service establishes either a TCP or HTTP connection depending on the ConnectivityMode setting. When a message is received on the front node a control message is routed to the service to indicate how a rendezvous connection back to the front end node of the client can be made to establish a direct HTTP to the socket forwarder for relaying the messages.

  • BasicHttpRelayBinding supports SOAP protocol.
  • WebHttpRelayBinding supports REST protocol.
  • WS2007HttpRelayBinding supports SOAP protocol along with WS-* functionalities.


The Azure Access Control Service provides security for the Service Bus. First, the WCF service authenticates with the Access Control Service and provides the authorization token to the Service Bus. After the token is validated, the service establishes an outbound connection to the Service Bus to create a bidirectional socket.

Service authentication details are provided by adding transportClientEndpointBehavior behavior to the service. The following properties are available to set on the behavior:

  • CredentialType property specifies the authentication type used such as SharedSecret whose credentials are issued by the access control service.
  • ClientCredentials composite container specifies values required by the CredentialType property selected earlier.

Each relay binding has a security binding element that specifies client authentication, authorization and message encryption (example: ws2007HttpRelayBinding -> WSHttpRelaySecurityElement). The following properties are available to set on the binding:

  • Mode property defines the message security. This property can be set to None, Message, Transport and TransportWithMessageCredential. When Transport or Message property is selected follow the standard WCF security model to set property specific values.
  • RelayClientAuthenticationType property determines whether a client is required to authenticate with access control service.

In Transport security mode, all traffic sent to and from the relay service will be protected by SSL. However, any traffic passing through the relay service is in clear text. Ifcomplete end-to-end encryption is required, then Message security mode should be used.

Service Bus Management Portal

In order to use Service Bus Relay, we first need to create a Service Bus namespace. Let’s look at how to create a service bus and then obtain default management credentials for the namespace.

  • After logging in, click on Service Bus on the left side navigation.

  • Click Create on the bottom menu to open a dialog to create a Service Bus namespace.

  • After the Service Bus namespace is created, click Connection Information on the bottom menu which opens a dialog window to obtain default management credentials under the section Access Connection Information.

  • Service Bus Relays are displayed under the RELAYS tab, after a WCF Service is successfully hosted using relay binding.

Sample Code

In this sample, let’s create a service and client that communicate over Service Bus Relay. We will use SharedSecretTokenProvider to hold shared secret credentials to authenticate with Access Control Service to retrieve tokens for both service and client implementations.

WCF Service

First, let’s create a WCF Calculator service that will be hosted on premise.

  • Define the service contract with calculator operation such as Add, Subtract and Multiply.
public interface ICalculatorService
	int Add(int num1, int num2);

	int Substract(int num1, int num2);

	int Multiply(int num1, int num2);
  • Implement a service using the service contract to return computed results.
public class CalculatorService : ICalculatorService
	public int Add(int num1, int num2) { return (num1 + num2); }

	public int Substract(int num1, int num2) { return (num1 - num2); }

	public int Multiply(int num1, int num2) { return (num1 * num2); }
  • The following code programmatically hosts the service, creating a Service Bus endpoint using WS2007HttpRelayBinding. In this sample code replace:
    • 'serviceNamespace’ with your Service Bus namespace
    • ‘servicePath’ with value where you want host your service
    • ‘issuerName’ and ‘issuerKey’ with values from ACS connection information of the Service Bus name space.
// Http connection to the service bus
ServiceBusEnvironment.SystemConnectivity.Mode = ConnectivityMode.Http;

// Creates service bus relay endpoint for client to communicate with service. Example: https://[serviceNamespace][servicePath]
var uri = ServiceBusEnvironment.CreateServiceUri(Uri.UriSchemeHttps, serviceNamespace, servicePath);

var binding = new WS2007HttpRelayBinding();
binding.Security.Mode = EndToEndSecurityMode.Transport;
// Property setting to specify client to authenticate with ACS to gain access to the relay service.
binding.Security.RelayClientAuthenticationType = RelayClientAuthenticationType.RelayAccessToken;

var behavior = new TransportClientEndpointBehavior();
// Property setting to provide shared secret credential that service uses to authenticate with ACS
behavior.TokenProvider = TokenProvider.CreateSharedSecretTokenProvider(issuerName, issuerKey);

var serviceHost = new ServiceHost(typeof(CalculatorService));
var endpoint = serviceHost.AddServiceEndpoint(typeof(ICalculatorService), binding, uri);

Console.WriteLine(string.Format("Service Endpoint @ {0}", uri));
Console.WriteLine("Press [Enter] to exit");


Client Code

Let’s create a client that will consume the Calculator service via Service Bus Relay. In this sample, we will use ChannelFactory object to communicate with the service.

  • Client channel implementation
public interface ICalculatorChannel : ICalculatorService, IClientChannel { }
  • Client implementation
// Http connection to the service bus
ServiceBusEnvironment.SystemConnectivity.Mode = ConnectivityMode.Http;

// Service bus relay endpoint for the client to connect to communicate with service. Example: https://[serviceNamespace][servicePath]
var uri = ServiceBusEnvironment.CreateServiceUri(Uri.UriSchemeHttps, serviceNamespace, servicePath);

var binding = new WS2007HttpRelayBinding();
binding.Security.Mode = EndToEndSecurityMode.Transport;

var behavior = new TransportClientEndpointBehavior();
// Property setting to provide shared secret credential to authenticate with ACS to communicate with service
behavior.TokenProvider = TokenProvider.CreateSharedSecretTokenProvider(issuerName, issuerKey);

var channelFactory = new ChannelFactory(binding, new EndpointAddress(uri));

var channel = channelFactory.CreateChannel();

Console.WriteLine("ADD 10 & 20: {0}", channel.Add(10, 20));
Console.WriteLine("SUBSTRACT 10 & 20: {0}", channel.Substract(10, 20));
Console.WriteLine("MULTIPLE 10 & 20: {0}", channel.Multiply(10, 20));
Console.WriteLine("Press [Enter] to exit");



  • Run the Calculator Service as a console application on premise which creates a service bus relay endpoint for its clients to communicate.

  • Log into management portal to verify relay under RELAYS tab of the namespace.

  • Run the Calculator Client as a console application on a different machine or across networks where the service is impossible to access. The client still communicates with the service via service bus relay.


If you’d like to learn about our Connected Systems practice at Neudesic, please visit this page:

Posted: Nov 21 2013, 06:52 by Jay.Prathipati | Comments (11) RSS comment feed

Categories: AppFabric | Azure | Connected Systems | Service Bus | WCF

Desert Code Camp - Integration Patterns with Azure Service Bus

Thanks to all who attended my session on Integration Patterns with Azure Service Bus at Desert Code Camp!  Here are the bits as promised. 

Make sure you setup your own Service Bus namespace and create appropriate queues and topics. Once that is complete, update the config file section to include your namespace and key.  Also make sure you use sessions on your queues for the aggregator sample. (8.30 mb)

Posted: Nov 17 2013, 14:48 by Brendon.Birdoes | Comments (0) RSS comment feed

Categories: Azure | Connected Systems | Service Bus





Neudesic Social Media

Follow Neudesic on Twitter Follow Neudesic on Facebook Neudesic Neudesic