Neudesic Blogs

Passion for Innovation

Announcing the release of Neuron ESB 2.6!

I'm very happy to announce the release of Neuron ESB 2.6. If you are using an earlier release of Neuron ESB, you can download this latest release from here: Neuron ESB 2.6. This release significantly extends the Neuron ESB platform by introducing new capabilities that will allow businesses to more easily scale, develop, connect and operationally manage their solutions. Businesses turn toward Neuron ESB to solve their service and integration problems while reducing the total cost of ownership of their solutions.  Neuron ESB accomplishes this by continually refining and extending its core capabilities in ways that can be effectively leveraged in agile environments by Microsoft .NET developers.

Neuron ESB 2.6 introduces many new features and enhancements, some of which include:

* Multi Instance Runtime
This capability allows organizations to install and run
multiple instances of the Neuron ESB Runtime Service
on a single server. Each instance can be configured as
either a 32 or 64 bit process, capable of running side by
side. This allows organizations to easily partition
business solutions to run on a single server and support
multiple developers.

* Process Designer
Neuron ESB 2.6 now offers a flexible, easy to use,
Process Designer with a drag and drop interface that
ships with over 30 configurable Process Steps that do
everything from calling a service, updating a database or
queue to parsing an Excel file. Organizations can create
more complex business processes using newly
introduced Process Steps such as “While” and “For”
looping constructs as well as an “Excel to Xml” parsing
Process Step.

* Adapters and Connectivity
Several new adapters are introduced as well as
enhancements to assist organizations connect,
compose and expose new capabilities within their
environment. Also in this release is the addition of a fifth
transport which can be used to configure Topics. Some
of these adapters and enhancements are:

- Event Based SharePoint 2010 Publication
- Event Based Dynamics CRM 2011 Publication
- FTP/FTPS Adapter
- ODBC Enhancements to support ReturnValue and OutPut type parameters
- Topics configurable with Named Pipes

* Monitoring and Reporting
New administrative, monitoring and reporting capabilities
are introduced with the release of Neuron ESB 2.6.
However, one of the most sweeping changes affecting
custom reporting is the refactoring of the Neuron ESB
database structure. The Neuron Audit tables have been
refactored specifically enable custom reporting. For
example, custom properties that are added to the ESB
Message are stored in an XML data typed column
supporting XQuery. Some of the features are:

- WMI Performance Counters for Topics and Endpoints
- WMI Performance Counter for Request/Reply call Total Time
- WMI Failed Message Events
- Database Enhancements for Custom Reporting
- Filter Query User Interface for Reporting

* Deployment Management
Neuron ESB supports XCopy Deployment through the
use of its Environmental Variable feature set. Neuron ESB
2.6 augments this through the extension of Environmental
Variable support for the Neuron Audit Database. Some of
the features included to assist deployments are:

- Assignment of the Same Neuron Audit Database to more than one Deployment Group
- Environmental Variable support for Neuron Audit database
- Environmental Variable support for Service Endpoint ACL property
- Read Only ESB Solution Configuration (*.ESB) support within the Neuron ESB Explorer and Runtime


* Service Endpoints
Neuron ESB 2.6 plays a critical role as organizations
evolve to look toward Cloud offerings for service hosting
or business to business communication. As a web
service broker, Neuron ESB 2.6 now includes all of the
Microsoft Azure Service Bus Relay Bindings to facilitate
hybrid approaches to bridge On Premise integration with
Cloud based solutions. Neuron ESB 2.6 ensures that
communication between the Cloud and On Premise
systems can be handled securely and reliably, while
managing the integration requirements of the
organization. Other important features in this release are:

- Azure Service Bus Integration
- Delegation Support for REST based services
- Pluggable WCF Service Behaviors

 

The full details of "What's New in Neuron ESB 2.6" can be downloaded from our web site here: http://products.neudesic.com/media/pdf/whatsnew.pdf

This release includes all the accumulated fixes. The complete list of changes can be found in the Neuron Change Log located at the root of the Neuron ESB installation directory as well as on our web site.


Posted: Apr 02 2012, 05:19 by marty.wasznicky | Comments (0) RSS comment feed

Tags:
Categories: Azure | Connected Systems | Custom Application Development | Headlines | Neuron | Neuron ESB | WCF

Neuron ESB 2.5.13 Release Available Now!

I’m very pleased to announce the July Feature release update (version 2.5.13) for Neuron ESB. This can be downloaded here

The product team is very excited about this release as it’s contains many new features which our customers will immediately benefit from. This release moves us further in ensuring that developing and deploying Neuron solutions continues to become easier for our customers. This release contains accumulated fixes, documentation updates, as well as incremental, customer requested updates to Neuron ESB:

·         Custom Pipeline Steps – Users will be able to create a reusable custom Pipeline Steps, register them, and use them within the Neuron ESB Pipeline Designer by simple drag and drop.

·         MSMQ Pipeline Step – This will provide users the ability to send messages to a Queue within a pipeline.  It will also support correlated receives from a Queue as well.

·         Pipeline Execution Pipeline Step – Provides the ability to create composite pipelines by executing existing pipelines within a pipeline for easy reuse.

·         Global Pipeline Cache – This will support the creation and initialization of any variable type that can be used across all instances of a pipeline executed by a Neuron Party

·         Custom SOAP Header Support for the Service Pipeline Step

·         Dynamic Transform Pipeline Step – Allows users to determine XSLT to run at runtime, rather than design time

·         Dynamic Schema Pipeline Step – Allows users to determine Schema to validate against at runtime, rather than design time

·         Environmental Variable support for Service/Client Connectors – This was the last step to make deployment truly easy for all customers. This deprecates the use of the Addresses tab located on Deployment Groups.

·         Multi-threaded receive for MSMQ based topics – This provides performance enhancements.

·         MSMQ Time to Live – The ability to set this property at the Topic level

·         WSDL support for Client Connectors - Ability to associate an existing WSDL endpoint with Client connectors so developers can use “add service reference” or “add web reference” within Visual Studio by referencing the client connector configured urls

·         Resubmit – the ability to resubmit a message directly from either the Audit or Failed Message reports.  In the next feature release we’re going to try to introduce bulk resubmit as well as the ability to resubmit to a specific adapter or service endpoint.

·         Reorder Pipelines – The ability to re order pipelines for a party within the UI so that users do not have to remove and then re-add.

·         RFID Sync and Async enhancements – Refactored to increase performance by 10 fold, as well as remove need to modify configuration files and enhanced tracing

·         Scatter Gather Sample – Updated the scatter gather sample and documentation (see attached) to use new Dynamic Transform Pipeline Steps

·         Unique WMI Performance Counters– By default Neuron ESB generates performance counter names for Parties using the Party Name appended by a guid.  Now we provide an option to generate these names by only using the Party name.  This allows predictable use of 3rd party monitoring tools


Besides new features, this release includes all the accumulated bug fixes from the past few months, since our previous 2.5.10 release in April 2011. Some of these fixes include the following:

·         Performance enhancements for service endpoints

·         Exception propagation for service endpoints to calling clients

·         Several Neuron “Farm Mode” fixes/enhancements were introduced to ensure resilience and reliability. Neuron Farm mode is used to scale out Neuron servers when using TCP based topics in conjunction with  the remote Neuron ESB client API. 

·         Several “DirectConnect” fixes were introduced.  DirectConnect mode is used for routing to service endpoints within a Pipeline, circumventing the Neuron Pub/Sub engine

·         Under certain conditions a service connector’s service policy would fail to execute when the WCF Message could not be created from the Neuron ESB Message.

·         Removed duplicate event log registry entries which were causing event information to be written to the disk file in the Log directory rather than the event log.

·         Code pipeline step no longer uses temp files.

·         Exceptions that occur in a pipeline are now passed back to a requesting client.

·         Removed duplicate failure exceptions being logged in the failed message database

·         Corrected issue where Party name and direction were incorrectly logged or missing in failed message database

·         Corrected issue where the service url in the Service Pipeline Step could being used rather than the metadata url even though the value was empty.

 

As always, the complete list of changes can be found in the change log (Neuron Change Log) installed at the root of the Neuron ESB installation directory as well as on our support site.

As an addendum to our shipped documentation, I’ll be regularly posting information on how to use the new features, so continue to monitor our blog and forums!

 

Kind regards,

Marty Wasznicky
Neuron ESB

Neudesic, L.L.C.
Work: (949) 754-5223

Fax: (949) 754-6523

marty.wasznicky@neudesic.com
| www.neudesic.com

Posted: Jul 29 2011, 13:00 by marty.wasznicky | Comments (0) RSS comment feed

Tags:
Categories: AppFabric | Connected Systems | General | Headlines | Neuron | Neuron ESB | WCF

Exploring Azure AppFabric Service Bus V2 May CTP: Topics

As syndicated from www.rickgaribay.net

In my previous post, I discussed Azure AppFabric Service Bus Queues, a key new capability in the first CTP of the Azure AppFabric Service Bus V2 release that was announced on May 17th.

Queues are an important addition to Azure AppFabric Service Bus capabilities because they provide a solid foundation on which to build loosely coupled distributed messaging solutions. The natural decoupling of queues introduces a number of natural side effects that can further benefit non-functional quality attributes of your solution such as performance, scalability and availability.

The graphic on the right is taken from my recent whitepaper “Developing and Extending Apps for Windows Azure with Visual Studio”image and shows the perpetual mismatch of supply and demand of IT capacities. If we think of this mismatch as load on the Y axis being introduced over time, the result is either failure to deliver a service or spending too much on hardware. imageThe goal, then is to align the demand with capacity.

Queues allow us to get closer to the drawing on the left because capacity can be tuned to scale as needed and at its own pace. This is effective because with queues, a consumer/worker can be throttled to only consume what it can handle. If the consumer/worker is offline, items in the queue will queue up, providing classic “store and forward” capabilities. If the consumer/worker is very busy, it will only consume the messages it is able to reliably pull from the queue. If we add more consumers/workers, each consumer/worker will consume messages at its optimal rate (determined by processing capacity, tuning, etc.), resulting in a natural distribution of work. Of course, stronger, more capable consumers/workers may consume more messages, but as long as there are messages in the queue, there is work to be done and the capacity can be allocated accordingly.

As you can see, queues are a great pattern for building loosely coupled distributed solutions, and the ability to add consumers/workers to a process or message exchange in a manner that is transparent from the client/producer perspective makes queues even more useful.

This is the idea behind topics. Before I dive into topics though, let’s talk about the problem that topics are trying to solve.image

imageIn distributed systems, it is useful to design message exchanges in terms of publishers and subscribers. Publishers are clients that are either sending a request-response message and will wait for a response; or are sending a single, one way message and expect no response. Subscribers care about these messages for one reason or another and thus subscribe to these messages. This is the essence of the Publish-Subscribe, or more succinct “Pub-Sub” messaging pattern. A one-way pub-sub  message exchange pattern is modeled to my left, and again to my right to build on a concrete example. Purchases, be they on-line or at brick-and-mortar retail outlets typically involve a point-of-sale (POS) system.

One of the first things a smart, modern POS software does when a unit is sold is to update inventory on that product so that the company can make proactive, intelligent decisions about managing inventory levels. In most cases, this is an administrative function, that is (or should be) transparent to the customer. When an order/sale is placed, an event occurs which is of interest to an Inventory Service that is responsible for decrementing the inventory count on a shared store. Of course, this is just one of several things that likely need to happen when an order is placed. Credit card authorization as well as fulfillment (whatever that means in the context of the purchase) needs to take place as shown below on your left.

All of a sudden things are more complex than they were before. I want to decouple the POS Client from the downstream business of authorizing a credit card and shipping the product to the customer’s doorstep. Depending on the context, the credit authorization process may be request-response or one-way. For most high-volume online retailers, the financial outcome of imagethe transaction is transparent to the purchasing experience. Ever gotten an email from Amazon.com after you made your purchase letting you know that your order is in a pending state because you need to update your expiration date on file so it can authorize your credit card? This asynchronous approach is common to facilitate scale and performance, and it is also good business.

It is very important to note that when the credit card authorization is designed as one-way, there are a number of guarantees that must be made. First, the Credit Service must receive the message, no matter how busy the front-end or back-end services are. Second, but of equal importance, is that the Credit Service must receive the message once and only once. Failure to deliver on the first or second guarantee will lead to lost revenue, either due to lost transactions or very disgruntled customers.

Using queues is a first step towards achieving these desired outcomes. However, now we need to reason about who should have the responsibility of sending the message to each subscriber? It can’t be the POS Client, because we want to decouple it from this kind of intimate knowledge. Adding this responsibility to each subscriber is also just as bad, or arguably worse. image

What we need is an intermediary that forms both a logical and physical relationship between publishers and subscribers such that the minimum degree of coupling is accomplished between the two and no more. This is exactly what a topic provides.

Topics are nothing new. They have been the mainstay of JMS-based systems for years, and thus have proven their usefulness in the field of distributed computing as a great way to logically associate actions to events, thus achieving pub-sub in a minimally coupled manner.

In our scenario, when a sale occurs, the POS Client publishes a message to the “Orders” topic to signal a new order event. At this point, corresponding subscribers are notified by the topic. This logical relationship is modeled to your right. Each one of the subscribers might receive a copy of the same message, in which case we would define the message exchange pattern as multi-cast. It is also possible-and likely- that each service exposes a different contract and thus, transformation between a canonical message and the expected message must take place, but this is a subject for a later post.

How the subscribers receive the message, be it one-way or request-response, is a physical implementation decision. In the classic sense, the logical abstraction of a topic affords us some latitude in choosing a physical transport that will fuse the publisher and subscriber(s) together. 

Note: Theory aside, everything I am sharing here is based on my experience over the last few days jumping into these early bits. I am not an expert on the implementation of these capabilities or the features of the CTP and am merely sharing my learnings and thoughts as I explore these exciting new features. If you have questions, know of a different or better way to do something as it applies to the CTP, or have any any corrections, please use the comments below and I’ll gladly consider them and/or share them with the product team and post updates back here.

Exploring Azure AppFabric Service Bus V2 Topics

In Azure AppFabric Service Bus V2, the physical transport used in Topics is Azure AppFabric Service Bus Queues, which allows us to harness the physical advantages of queues and the logical abstraction that topics provide to design our distributed solutions at internet scale.

If you’ve played with the Azure AppFabric Service Bus Queues, or read my introduction to this series you might be wondering what makes Azure AppFabric Service Bus Topic so special. So far, you might be thinking that we could accomplish much of what I’ve discussed with Queues and you wouldn’t be alone. I struggled with this initially as well.

The way that Queues and Topics are implemented in the current technology preview, Topics don’t really seem all that useful until you want to refine when or under what conditions a Subscriber should receive a message beyond simply being subscribed to a Topic, and this can be pretty powerful.

In fact, we can code the scenario shown in the article on Queues to be functionally equivalent with Topics without immediately gaining much.

Creating Topics and Subscriptions

Start by creating a ServiceBusNamespaceClient and MessagingFactory just as before:

  1: ServiceBusNamespaceClient namespaceClient = new ServiceBusNamespaceClient( 
  2:         ServiceBusEnvironment.CreateServiceUri( 
  3:         "sb", serviceNamespace,  
  4:         string.Empty),  
  5:         sharedSecretCreds); 
  6:  
  7: MessagingFactory messagingFactory = MessagingFactory.Create( 
  8:     ServiceBusEnvironment.CreateServiceUri( 
  9:     "sb",  
 10:     serviceNamespace,  
 11:     string.Empty), 

Next, create a Topic called “Orders”:

  1: Topic ordersTopic = namespaceClient.CreateTopic("Orders");

The underlying queue infrastructure is created for you.

Now, create a subscription for the Inventory Service on the “Orders” topic:

  1: Subscription inventoryServiceSubscription = ordersTopic.AddSubscription("InventoryServiceSubscription");

At this point, the creation of the Topic and Subscription above would take place in a management context without regard to, or any knowledge of the actual publisher or subscriber(s).

I would expect tooling either from Microsoft or the community or both to start to crop up soon to provide a user experience for these types of management chores, including the ability to enumerate Queues, Topics, etc. For example, before I create a Topic, I need to ensure that the Topic doesn’t already exist or I will get an MessagingEntityAlreadyExistsException. I used the GetQueues method on the namespace client, and if the Topic or Queue entity exists, use the DeleteQueue or DeleteTopic method.

Publishing on a Topic

Now, the client/publisher creates a TopicClient and MessageSender and sends the message on the Orders Topic:

  1: TopicClient pOSClientPublisher = messagingFactory.CreateTopicClient("Orders"); 
  2:  
  3: MessageSender msgSender = pOSClientPublisher.CreateSender(); 
  4:  
  5: Order order = new Order(); 
  6: order.OrderId = 42; 
  7: order.Products.Add("Kinect", 70.50M); 
  8: order.Products.Add("SamsungFocus", 199.99M); 
  9: order.Total = order.Products["Kinect"] + order.Products["SamsungFocus"]; 
 10:  
 11: var msg = BrokeredMessage.CreateMessage(order); 
 12:             
 13: msgSender.Send(msg); 
 14: msgSender.Close();

Note that the client/publisher knows nothing about the subscriber. It is only bound to a logical Topic called “Orders” on line 1 above. It is running in some other process somewhere in the world (literally) that has an internet connection and can make an outbound connection on TCP port 9354***.

Subscribing to a Topic

On the receiving end, a SubscriptionClient is created along with a MessageReciever. The SubscriptionClient instance is created from the MessagingFactory instance which accepts the name of the topic and the subscription itself.

Note the RecieveMode is the same as before which will have the effect of ensuring that the Inventory Service Subscriber receives the message at most once (FWIW, I think PublisherClient and SubscriberClient make more sense that SubscriptionClient and TopicClient respectively, but the intent of the classes is pretty clear and again, these are early bits so expect changes as the team gets feedback and continues to bake the API):

  1: SubscriptionClient inventoryServiceSubscriber = messagingFactory.CreateSubscriptionClient("Orders", "InventoryServiceSubscription"); 
  2:              
  3: MessageReceiver msgReceiver = inventoryServiceSubscriber.CreateReceiver(ReceiveMode.PeekLock); 
  4:  
  5: var recdMsg = msgReceiver.Receive(); 
  6: msgReceiver.Close(); 
  7:  
  8: var recdOrder = recdMsg.GetBody<Order>(); 
  9:  
 10: Console.WriteLine("Received Order {0} on {1}.", recdOrder.OrderId, "Inventory Service Subscriber");

The code above would be wrapped into a polling algorithm that allows you to have fine control over the polling interval, which as Clemens Vasters pointed out in a side conversation recently is a key capability that allows you to throttle your subscribers. The samples in the SDK show a polling technique which works, but it would be nice to see an option for setting some config and letting the API do this for you.

Regardless of the approach you take to checking for messages, the only thing the subscriber knows about is the Topic name and the name of the subscription on line 1 above (having to provide the subscription name in addition to the topic name, seems a bit redundant to me and more coupling than is needed).

Multiple Subscribers

At this point, we’ve emulated the functionality of the Queuing example shown in my first post in the series.

While we’ve increased our level of abstraction, and are no longer thinking about (or care) about the fact that there’s a queue in between the publisher and subscriber, so far, topics haven’t bought us much just yet…

Think back to the scenario above. There are two other services that care about orders. We can create subscriptions for them just as we did for the Inventory Service at management/configuration time:

  1: Subscription fulfillmentServiceSubscription = ordersTopic.AddSubscription("FulfillmentServiceSubscription"); 
  2: Subscription creditServiceSubscription = ordersTopic.AddSubscription("CreditServiceSubscription");

Of course, new subscriptions can be added long after the Topic has been created, and this is one of the many powerful aspects of this logical abstraction from publishers and subscribers. This approach introduces agility into your solutions because you can add subscribers with minimal friction, and in a fully location transparent manner.

As with the publisher (TopicClient), subscribers (SubscriberClients) live in their own process anywhere in the world with an internet connection and can be fired up at will. If one is offline, or unavailable, the message will be queued (provided that previous subscribers have peeked the message (ReceiveMode.PeekLock) as opposed to popping it off the queue (ReceiveMode.ReceiveAndDelete). Below is the simple code for adding listeners/Subscribers for the Credit Service and Fulfillment Service:

  1: // Credit Service Subscriber 
  2: SubscriptionClient creditServiceSubscriber = messagingFactory.CreateSubscriptionClient("Orders", "CreditServiceSubscription"); 
  3:  
  4: msgReceiver = creditServiceSubscriber.CreateReceiver(ReceiveMode.PeekLock); 
  5: recdMsg = msgReceiver.Receive(); 
  6: msgReceiver.Close(); 
  7:  
  8: recdOrder = recdMsg.GetBody<Order>(); 
  9:  
 10: Console.WriteLine("Received Order {0} on {1}.", recdOrder.OrderId, "Credit Service Subscriber"); 
 11:  
 12: // Fulfillment Service Subscriber 
 13: SubscriptionClient fulfillmentServiceSubscriber = messagingFactory.CreateSubscriptionClient("Orders", "FulfillmentServiceSubscription"); 
 14:  
 15: msgReceiver = fulfillmentServiceSubscriber.CreateReceiver(ReceiveMode.PeekLock); 
 16: recdMsg = msgReceiver.Receive(); 
 17: msgReceiver.Close(); 
 18:  
 19: recdOrder = recdMsg.GetBody<Order>(); 
 20:  
 21: Console.WriteLine("Received Order {0} on {1}.", recdOrder.OrderId, “Fulfillment Service Subscriber");

Creatiimageng two additional SubscriptionClients for the Credit Service and Fulfillment Service results in all three subscribers getting the message as shown on the right. Again, in my examples, I am running each subscriber in the same process, but in the real world, these subscribers could be deployed anywhere in the world provided they can establish a connection to TCP 9354.

Rules/Actions, Sessions/Groups

Now, what if we wanted to partition the subscribers such that in addition to subscribing to a Topic, additional logic could be evaluated to determine if the subscribers are really interested in the message? Our online retailer probably (err, hopefully) has a centralized inventory management system and credit card processor, but may have different fulfillment centers across the world.

Based on the customer’s origin, the order should go to the closest fulfillment center to minimize cost and ship times (i.e. North America, South America, Africa, East, Europe, Asia, Australia).

Azure AppFabric Service Bus V2 supports this approach with Sessions, Rules and Actions. I group these into the idea of a message pipeline. In addition to the subscriptions, the Topic evaluates additional context or content of the published message configured at management time to introduce some additional filtering and very lightweight orchestration. The topic subscription is the right place for this to happen because again, it is a management-time task. Publishers and subscribers merely send/receive messages. It is the benefit of a logically centralized, yet physically distributed messaging model that affords us the ability to manage these details in a centralized way.

You can create a RuleDescription to evaluate some property or field in the message that indicates country of origin, and as an action, set a property on the message to identify the fulfillment center.

To illustrate this, first, I’ve added two properties to the BrokerMessage that I am publishing on the “Orders” Topic. I’ll use these properties when I configure my rule and action next:

  1: msg.Properties.Add("CountryOfOrigin", "USA"); 
  2: msg.Properties.Add("FulfillmentRegion", "");

Notice that in line 2 above, I’ve intentionally created the “FulfillmentRegion” property with an empty string, since we are going to apply some logic to determine the fulfillment region.

Now, I use a RuleDescription and SqlFilterExpression to determine if the CountryOfOrigin is the United States. If the SqlFilterExpression evaluates to true, then the SqlFilterAction fires and sets the FulfillmentRegion  to “North America”:

  1: RuleDescription fulfillmentRuleDescription = new RuleDescription(); 
  2: fulfillmentRuleDescription.FilterExpression = new SqlFilterExpression("CountryOfOrigin = 'USA'"); 
  3: fulfillmentRuleDescription.FilterAction = new SqlFilterAction("set fulfillmentRegion='North America'");

Of course, in the real world, there would be a more sophisticated process for identifying the country of origin, but simple, contrived examples make it so that articles get published. Winking smile

The evaluation and any corresponding actions must fire when the message is in-flight as any actions taken could influence the routing of the message, as with the example above which will meet a subscription rule we’ll configure on the Fulfillment Service Description next.

OK, so now we have some properties we can play with and we’ve defined a RuleDescription. The last thing we need to do is modify the FulfillmentServiceSubscription to include the RuleDescription I just created. This makes the FulfillmentSubscription conditional, based on the conditions we've defined in the instance of the RuleDescription called fulfillmentRuleDescription:

  1: Subscription fulfillmentServiceSubscription = ordersTopic.AddSubscription("FulfillmentServiceSubscription", fulfillmentRuleDescription);


Now, when I run my code, all three subscribers receive the order message just as before, however this time, we know that the only reason that the Fulfillment Service is getting the message is because it is acting as the North America fulfillment center. If I modify the CountryOfOrigin property in line 1 in the 3rd code sample up from here to anything but “North America” the Fulfillment Service will not receive the message at all.

As I continue to model out my subscribers, I could create a subscription for each fulfillment center that is capable of receiving real-time orders and then create RuleDescriptions accordingly. This would allow me to distribute fulfillment geographically (good for scale, reliability and happy customers) as well as ensuring that I am always only pulling back messages that I need. If, during peak times around holidays, the volume of orders increases, I can simply add additional fulfillment subscribers for that region to ensure that packages ship as quickly as possible and that no orders are lost.

 

Closing Thoughts

So far, I’m pretty impressed with the powerful messaging capabilities that Azure AppFabric Service Bus V2 Topics introduced in the May CTP, and I’m excited to see where things are going.

As Azure AppFabric Service Bus matures further, I would love to see additional transport channels supported by Azure AppFabric Service Bus Topics. Just as with the creation of the Topic and Subscriptions as a management function, the transport would also be defined and created/configured at management time. This is really where the power and elegance of Topics shines through in my opinion because the publisher and subscriber don’t know or care about the transport- they’re just connecting to a topic and sending and/or receiving messages.

By way of some nit picks, I think that PublisherClient makes more sense than TopicClient, and along with considering a modification to SubscriptionClient, having a PublisherClient and SubscriberClient that publish and subscribe to a Topic seems a lot cleaner and more intuitive to me.

I’m also trying to get used the fact that we need Clients and Senders/Receivers. Again, to me it would seem more intuitive to simply have a PublisherClient and SubscriberClient that own the sending and receiving. Perhaps we’re jus seeing visible seams in the API due to the early nature, or there’s a good reason for this that I haven’t thought of yet.

At PDC 10, the AppFabric team announced that they are investing in an “Integration Service” that will provide additional messaging capabilities by way of transformation and advanced mapping similar to how we leverage these capabilities in BizTalk today. I can see Topics getting much more robust when, in addition to modifying properties on the BrokerMessage, we can mediate and transform a message in-flight just before reaching a subscriber, and I can also think of some nice message enrichment patterns that would be very nice.

*** One important thing to note is that in the current May CTP, Queues and Topics do not provide the same NAT/Firewall traversal capabilities of their relay siblings. For the .NET Messaging API (which I’ve been using to share my learnings thus far) as well as the WCF ServiceBusMessagingBinding, outbound TCP 9354 is required. Also note that the channel type is Net.Tcp (think WCF). This means that in the current CTP, the only way to ensure full interoperability across publishers/subscribers and guarantee an outbound connection (assuming HTTP ports aren’t locked down) is to use the REST API, but I suspect we’ll see more parity of this important feature across client types for Queues and Topics.

What’s Next?

There’s still much to explore around sessions, filtering, the WCF ServiceBusMessagingBinding, the REST API and how we might bridge Azure AppFabric Service Bus with on-premise messaging capabilities. Exciting stuff- stay tuned!

Posted: May 30 2011, 08:50 by Rick.Garibay | Comments (0) RSS comment feed

Tags: , ,
Categories: AppFabric | Azure | Connected Systems | Headlines | WCF

Exploring Azure AppFabric Service Bus V2 May CTP: Queues

As syndicated from http://rickgaribay.net/

Today, the AppFabric team announced the first Community Technology Preview of the Azure AppFabric Service Bus V2.

The second release of Azure AppFabric Service bus is a significant milestone that the team has been hard at work on for several months. While I’ve had the privilege of attending a number of SDRs and watching this release move from ideation to actual bits, this is the first time I’ve been able to actually get my hands on the code, so I’ve spent the better part of this evening diving in.

First, if you are new to the Azure AppFabric Service Bus, there are many resources available to get you up to speed. I highly recommend this whitepaper by Aaron Skonnard: A Developer’s Guide to the Service Bus. If you are a visual learner, please consider checking out my webcast on AppFabric Service Bus here: http://rickgaribay.net/archive/2011/01/12/neudesic-appfabric-webcast-series.aspx.

In a nutshell, like most ESBs, Azure AppFabric Service Bus is the manifestation of a number of core messaging patterns that provide the ability to design messaging solutions that are loosely coupled. What makes the Azure AppFabric Service Bus unique is that it provides these capabilities at “internet scale”, meaning that it is designed to decouple clients and services regardless of whether they are running on premise or in the cloud. As a result, the AppFabric Service Bus is a key technology for enabling hybrid scenarios at the platform level (i.e. PaaS) and serves as a key differentiator in the market today for enabling organizations to adopt cloud computing in a pragmatic way.

Messaging patterns in general provide a common frame on which to think about and build composite applications. Cloud and hybrid computing necessitate many of the same messaging patterns found within the on-premise enterprise and introduce new complexities that are somewhat unique.  imageAzure AppFabric Service Bus V2 introduces tried and true messaging capabilities such as Queues, Topics, Pipes and Filters (Rules/Actions) as well as sequencing semantics and of course durability.

It is important to note that the Azure AppFabric Service Bus is not a replacement for on-premise publish-subscribe messaging. It enables new scenarios that allow you to integrate your current on-premise messaging and provide the ability to compose clouds, be they your own, your partners or those of private cloud providers such as Microsoft. The drawing on the right is from the Microsoft whitepaper I mentioned in the introduction. Notice that the Azure AppFabric Service Bus is providing the ability to integrate clients and services regardless of where they reside, and for non-trivial SOA, on-premise pub-sub for decoupling composed services and clients is essential.

Queues

Queues are typically used to provide temporal decoupling, which provides support for occasionally connected clients and/or services. With a queue, a client writes to a known endpoint (private/local or public/somewhere on the network) without regard to the state of the downstream service. If the service/consumer is running, it will read work items or messages from the queue. Otherwise, the queue will retain the message(s) until the service is available to retrieve the message or the message expires.

As promised, Azure AppFabric Service Bus V2 delivers on durable messaging capabilities beyond the current Message Buffer feature with the introduction of Queues.

Queues are supported in the .NET API (shown below), REST API and with a new WCF Binding called “ServiceBusMessagingBinding”. While my preferred approach will certainly be WCF, the .NET API helps to understand the new API. In addition, the process of creating a queue is required even with WCF since queue creation is outside of WCF’s immediate area of concern.

The first thing you need to do when working with Queues, Topics and Subscribers is create a Service Bus Namespace Client which is an anchor class for managing Service Bus entities:

ServiceBusNamespaceClient namespaceClient = new  ServiceBusNamespaceClient(ServiceBusEnvironment.CreateServiceUri("sb",  serviceNamespace, string.Empty),  sharedSecretCreds);

 

Once you have an instance of the ServiceBusNamespaceClient, you can create a queue by simply instantiating the Microsoft.ServiceBus.Messaging.Queue class (I’ve set the queueName field to “Orders”):

  Queue queue = namespaceClient.CreateQueue(queueName);

 
Next, create a Queue client. The Queue Client manages both send and receive operations:
 

1: MessagingFactory messagingFactory =

MessagingFactory.Create(ServiceBusEnvironment.CreateServiceUri("sb", serviceNamespace, string.Empty), sharedSecretCreds);

   2:  var queueClient = messagingFactory.CreateQueueClient(queue);
 

The code above uses the Microsoft.ServiceBus.Messaging.MessagingFactory which accepts your namespace and security credentials and returns an instance of the MessagingFactory.

Next, you create an instance of Microsoft.ServiceBus.Messaging.MessageSender using the CreateSender method:

   1:  var messageSender = queueClient.CreateSender();
   2:  messageSender.Send(msg);
   3:  messageSender.Close();
 
Line 7 below creates a BrokeredMessage, which is a new message class which represents a unit of communication between Service Bus clients. Note that this class has nothing to do with the System.ServiceModel.Channels Message class, however when using the ServiceBusMessagingBinding the classic WCF Message class is used. The BrokeredMessage class consists of a number of methods and some interesting properties including ContentType, CorrelationId, MessageId, Label and a property bag called Properties which we’ll explore as we progress through the new features.
 
In this example, I’m using an overload of CreateMessage that accepts a serializable object. The method uses the DataContractSerializer with a a binary XmlDictionaryWriter to create a BrokeredMessage.
 
   1:  Order order = new Order();
   2:  order.OrderId = 42;
   3:  order.Products.Add("Kinect",70.50M);
   4:  order.Products.Add("SamsungFocus", 199.99M);
   5:  order.Total = order.Products["Kinect"] + order.Products["SamsungFocus"];
   6:   
   7:  var msg = BrokeredMessage.CreateMessage(order);

 

Finally, we can send the message:

   1:  var messageSender = queueClient.CreateSender();
   2:  messageSender.Send(msg);
   3:  messageSender.Close();

 

With much of the infrastructure code out of the way, we can use the same queueClient instance to create a MessageReciever and request the BrokeredMessage message from the Orders queue:

   1:  var messageReceiver = queueClient.CreateReceiver(ReceiveMode.ReceiveAndDelete);
   2:  var recdMsg = messageReceiver.Receive();
   3:  messageReceiver.Close();
 

Note the ReceiveMode in line 1 above. This has the effect of enforcing an “Exactly-Once” “At Most Once” (thanks David Ingham) receive semantic since the first consumer to read the message will pop it off the queue. The opposite option is RecieveMode.PeekLock which provides “At Least Once” delivery semantics. As David Ingham, Program Manager on the AppFabric team kindly adds in his comments below: “If the consumer is able to record the MessageIds of messages that it has processed then you can achieve “ExactlyOnce” processing semantics with PeekLock mode too.” –Thanks David!

Once we have the BrokeredMessage, we can retrieve the body:
 
   1:  var recdOrder = recdMsg.GetBody<Order>();
   2:  Console.WriteLine("Received Order {0} with total of ${1}",recdOrder.OrderId,recdOrder.Total);
 

The cool thing about the CTP is that Microsoft is offering you a free labs environment in which to explore and play. In the CTP, you can create a maximum of 100 queues, with a maximum size of 100MB each and messages can not exceed a payload size of 256KB- pretty workable constraints for a first CTP.

To get started, and dive in for yourself, be sure to download the Azure AppFabric V2 SDK at: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=D89640FC-C552-446E-AEAD-B1E0D940F31B

In my next post, we’ll explore Topics and Subscriptions which allow for rich pub-sub including the ability to multicast to thousands of potential subscribers.

Hats off to the AppFabric Messaging team for all of their hard work with this release!

Posted: May 17 2011, 08:00 by Rick.Garibay | Comments (0) RSS comment feed

Tags: , ,
Categories: AppFabric | Azure | Connected Systems | General | Headlines | WCF

July 2010 App Fabric Release

There's an updated version of Azure App Fabric released yesterday - read more about it and get download details on Zane Adam's blog here. In addition to .NET 4.0 support, this release supports Silverlight development by enabling cross-domain calls (this also helps something called Flash, which seems to be a SL clone.) This was a feature initially rolled out to the community as part of the App Fabric Labs project - read more about App Fabric Labs here.

Posted: Jul 02 2010, 01:48 by mickey.williams | Comments (0) RSS comment feed

Tags:
Categories: Azure | Connected Systems | Custom Application Development | Headlines | Neudesic Main

Clustered Indexes Required for SQL Azure

There’s a great explanation about why we need clustered indexes for SQL Azure tables over at the SQL Azure team blog. Short answer? It’s because the platform is keeping three replicas of the data, and the index is required for the replication.

http://blogs.msdn.com/b/sqlazure/archive/2010/05/12/10011257.aspx

Posted: Jul 01 2010, 05:23 by mickey.williams | Comments (0) RSS comment feed

Tags:
Categories: Custom Application Development | Headlines | Neudesic Main | Azure

Tags

Categories

Archive

Blogroll

Neudesic Social Media

Follow Neudesic on Twitter Follow Neudesic on Facebook Neudesic Neudesic