Bridging Enterprise Chaos: Why Your Canonical Data Model Needs the Gang of Four

 If you’ve ever built a system that integrates with more than three other applications, you already know the sinking feeling.

System A speaks XML with camelCase fields. System B speaks JSON with snake_case fields. System C outputs a proprietary flat-file format from 2004 that relies on positional offsets. When you try to connect them all directly, you end up with an $N \times M$ spiderweb of point-to-point translators. Add a new field to System A, and suddenly three downstream pipelines break on a Tuesday night.

To solve this, we introduce a Canonical Data Model (CDM)—a single, neutral data standard that lives at the center of your architecture. Instead of translating every format into every other format, every app just translates to and from the CDM.

The catch? Writing that translation layer can easily turn into a massive ball of procedural mud if you aren't careful. That’s where classic object-oriented design—specifically the Gang of Four (GoF) patterns—comes to the rescue. Let's look at how to wire a CDM up cleanly so you don't regret it six months down the line.

The Core Problem: Keeping Translators Out of Your Domain Logic

The biggest mistake developers make when building an integration layer is scattering parsing and mapping code all over their core business logic. If your order-processing service has to know the difference between Salesforce’s customer schema and SAP’s customer schema, your architecture is leaking.

A good CDM acts as a protective boundary. But translating incoming payloads into your canonical objects requires structure. Otherwise, you get hundreds of lines of nested if/else statements checking field names.

Introducing GoF: The Adapter Pattern as Your First Line of Defense

When people talk about integration patterns, the Adapter Pattern is the undisputed workhorse.

Think about how an electrical travel adapter lets your US laptop plug into a European wall socket without rewriting the laptop's power supply. In code, an Adapter sits right at the edge of your service boundary. It takes whatever messy, vendor-specific payload just arrived and converts it into your clean, strongly-typed Canonical Data Model object.

[Legacy System Payload] ---> [Specific Adapter] ---> [Canonical Data Model Object] ---> [Core Business Logic]

By keeping this logic inside dedicated adapter classes, your core application code only ever deals with the CDM. If System A updates its API next week, you don’t rewrite your business rules—you just rewrite one adapter.

Dealing with Legacy Subsystems: The Facade Pattern

Sometimes, integrating with a legacy database or third-party service isn't just about changing field names; it's about untangling a messy sequence of operations. Maybe getting a single "Customer Profile" requires hitting three different database tables and merging the results.

This is where the Facade Pattern earns its keep.

Instead of letting your CDM mapper reach deep into the guts of a messy subsystem, wrap that subsystem in a Facade. The Facade presents a clean, simplified interface that your integration layer can easily read from and map into the canonical format. It keeps the noise isolated where it belongs.

Avoiding the "God Object" Trap

Writing about CDMs would be irresponsible without mentioning the elephant in the room: The God Object Anti-Pattern.

Because a Canonical Data Model is meant to represent everything in your domain (orders, customers, inventory, billing), it's temptingly easy to design a single, massive data structure that contains every possible field every system might ever need.

Before you know it, your CanonicalOrder class has 200 properties, and 180 of them are null depending on where the message came from.

To keep your CDM healthy:

1. Keep it lean: Only include fields that are core to the enterprise domain.

2. Use composition: Break large canonical objects down into logical sub-models (e.g., CanonicalAddress, CanonicalLineItem) rather than flattening everything into one giant class.

3. Iterate slowly: Don’t try to model the entire enterprise on day one. Start with the fields you actually need right now.

Wrapping Up

A Canonical Data Model saves you from the $N \times M$ integration headache, but design patterns from the Gang of Four provide the actual scaffolding that keeps the implementation maintainable. By leaning on Adapters to handle boundary translations and Facades to wrap messy legacy subsystems, you keep your codebase decoupled and ready for whatever weird third-party API comes next.

How are you handling data transformations in your current stack? Drop a comment below—I'd love to hear what works (or doesn't work) in your systems.

Threading and Perception

Today I thought to tell you something on one of the most complex part of .NET Framework I.e. Threading. I was discussing some of the issues with one of our team member on threading today;

from taking inspiration on that conversion; here are the facts of .NET Threading....

.NET Thread object is "ACTUALLY" not created by .NET; its Operating level thread we are just getting a decorated version of that thread. When we talk about Thread; we must remember the thread pool. The thread pool is just a wrapper class of OS Level thread pool which is optimized for faster allocating/deal locating the memory and thread data. While taking back used thread from user application the same class is also responsible to cleared off the data. This is the internal operation; as a user we don't need to care about it.

The second fact is: some of the blogs (NOT MSDN) mentioned that when you write the code something like:

Thread t = new Thread(DoSomething);

Its actually not a part of ThreadPool and its creating new thread; which may be right in a way. But this is not the fully truth. If we believe on the blog for an instance; we may not get duplicate thread ever. Now here i am no going in depth of Allocation thread memory, execution etc. but let me tell you one point; even if you write above statement in .NET; it serves the thread object from Pool only. I know you will not believe this fact.

I have created an example which works on above assumption have a look once.. its interesting to see. I am creating thread using above code and some times i am getting new thread (Displayed in White Font) and some times i get duplicate thread (Displayed in Yellow Font). It proves that internally .NET is keeping a pool of thread.

The second point; In example;I have also set some data on fresh thread each time. Which I am trying to recover when I get the duplicate thread object. And look at the result; The Thread Slot is still there; but .NET erased the data.

In summery you can say that:

1. Thread object never created/destroyed from .NET Managed Env. user always gets the thread object from pool only.

2. The data you associates with thread (In terms of Thread Slots and ThreadStatic variables) are not persisted on the thread when .NET collects the unused thread for pooling.

When Thread Slots are getting cleared off; there is no question to persist the value of ThreadStatic variables.

Now talking about WCF execution; like this example when you open the WCF Service Channel/Port; it takes one of the available thread from same pool and starts the execution of the code using the thread. You can never get the older data associated with associated with thread earlier. So its perfectly safe to use ThreadStatic variables.

This was about console application; lets take ASP.NET application in consideration. .NET takes thread from ThreadPool to execute the request. This thread is persisted in ThreadPool even though the values of ThreadStatic variables are not getting persisted.

Check-out my ASP.NET example and observe thread static behavior in PageLoad method.


In short; make the best use of ThreadStatic without any fear.

Google says Good Bye to China

Google has executed its first step towards closer of their China Operations (Google Search, Google News and Google Images mainly).

Users visiting Google.cn are now being redirected to Google.com.hk, where google offers uncensored search in simplified Chinese, specifically designed for users in mainland China and delivered via our servers in Hong Kong.

Google.cn is now Google.com.hk

--
Kaushal Patel

Playing with Generic List

Generic List a new feature provided by .NET 2.0 and above which can be declared as follow: List<string> lstString = new List<string>();
Creating Generic Object using Reflection
Now; there is nothing now in this. And we all know that the generics are type safe. That means you can not pass Child object's referece (Without type casting) where the base class is expected. In case if you need to create Generic List object dynamically depending upon your business scenarios you can use Reflection API to create generic object.

Lets see how to do this:

public static object CreateGenericObject(Type genericType, Type innerType, params object[] args)

{

System.Type speType = genericType.MakeGenericType(new System.Type[] { innerType });

return Activator.CreateInstance(speType, args);

}

And your client code would be as follows:

object geneticListObject = CreateGenericObject(typeof(List<>),typeof(int));

Now lets see one more feature of .NET 2.0 i.e. Inline functions (anonymous methods):

Lets assume you have a generic collection of objects (lets assume student); and you want to search on Student Name starting with "K". You dont need to make another call to the server for this. Here is the code for the same:

List<Student> localStudentList = mainStudentList.FindAll(delegate(Student student)
{
return student.StudentName.StartsWith("K");
});

the above code will return the list of students (in localStudentList object) whose name starts with K.To extends this you can write various AND and OR conditions. Isn't this good?

--
Kaushal Patel

Contact Me

BCL new features of the .NET Framework 4.0 Beta 2

Visual Studio 2010 and .NET Framework 4 Beta 2 are now available to download. .NET 4 Beta 2 contains various newly introduced BCL features and enhancements in addition to what was included in the earlier version of .NET Framework 4.0 Beta 1. Many of these enhancements were added in large part due to specific feedback and suggestions reported by customers through Microsoft Connect.

New BCL Features in .NET Framework Beta 2.0

1. Complex Number
2. Location
3. Stream Copy (Stream.CopyTo)
4. IObservable<T>
5. Enum.HasFlag
6. String.Connect & String.Join
7. String.IsNullOrWhiteSpace
8. TryParse (In Guid, Version and Enum)
9. Addition Environment.SpecialFolder, Environment.Is64BitProcess and Environment.Is64BitOperatingSystem
10. Path.Combine with params support
11. ObservableCollection<T> is now part of System.DLL
12. Int Pointer operations (IntPtr and UIntPtr Subtraction operators)
13. StringBuilder.Clear
14. ServiceInstaller.DelayedAutoStart
15. StopWatch.Restart

-- Kaushal Patel

Contact Me

Task Parallel Library Sample is out now!!!

Microsoft .NET Framework 4.0 TPL (Task Parallel Library) sample is out now. This sample has been created in .NET Framework 4.0 Beta 2

Click here to get the newest feature of .NET Framework 4.0

Microsoft Blue & Me

Blue&Me is a information & entertainment system that specially designed for Fiat Group cars as a part of Microsoft Auto initiatives and developed in a partnership which was started end of 2004 between Magenti (a Fiat group company) and MSFT. The system is based on modular structure which allows installation and use of uninformed services. The system offers various connectivity options like Bluetooth, USB, Mobile Phones, your personal media player along with hands-free operations allowing you to control all features by voice commands, using speech technology from Nuance Comm With the help of Magneti the system is developed to be compatible with most mobile phones and media players Blue&Me Nav is a next version of the same OS which adds GPS related features.

In india this os and its capabilities are not well known; at the Bologna Motor Show 2009, Fiat presented a new portable navigation unit addition to the Blue&Me system called Blue&Me Map. Exclusive for and to Fiat Professional's light commercial vehicles.

Visual Studio 2010 Beta 2

On the new years eve; microsoft launched its one of the major and popular development studio i.e Visual Studio 2010. Still its early to discuss the features and improvements that VS.NET 2010 provides. But keep watching this space for more on VS.NET 2010 in coming days.

To download VS.NET 2010 with .NET Framework Beta 4.0

Different ways to Implement the Singleton in .NET

Lots of blogs and sites are talking about the ways to implement the singleton in different ways; and surprisingly many of them some times miss the thread safety or the thread sync. According to the "Design Patterns: Elements of Reusable Object-Oriented Software" the implementation of Singleton should be as follow:







The above implementation may work well considering that singleton implementation can be breakable at the first call. In "GetInstance" method we are checking "instance == null" which returns true when 2 requests comes at the same time at the first call. Plus we are not taking the advantage of the .NET language features. Here is the thread safe implementation of Singleton in .NET C# language.






Here you need to watch the usage of volatile (One can consider the implementation of "volatile" exactly opposite of "ThreadStatic" keyword) and static keyword gives the grantee to be thread safe in .NET compilation. (According to Jeffrey Richter).

Microsoft Dublin & Velocity

Microsoft Windows Server AppFebric (Dublin and Velocity) is a bundle of integrated technologies which makes easier to develope, extend and manage the web and composite application that runs on IIS.

If you are searching for the Windows Azure platform AppFabric, which helps developers connect apps and services between Windows Azure and on-premises deployments, see the Windows Azure site.

What is Dublin?

Dublin enriches Windows Server to give enhanced hosting and management capabilities for WCF and WF services. Dublin also adds service management extensions to the hosting features of Internet Information Services (IIS) and Windows Activation Service (WAS), and the run-time components and services of the .NET Framework 4. Dublin addresses the challenges of hosting WCF- and WF-based applications by making it easier to deploy, configure, and manage applications.

Typically in Service Oriented Architecture people are building the service for the different operations and exposes these services to the consumer to consume. Since WCF and WF released; the deployment, monitoring and scaling these services never been an easy task. Some times to consuming these service it self a trouble making task. With the help of Dublin we can perform these tasks almost effortless. Dublin can be easily introduced through IIS management console using the Dublin module of Windows Power Shell.

What is Velocity?

Velocity is a highly scalable in-memory caching mechanism using which one can improve the performance of the application with all types of data. For any application one can leverage the Velocity rather then writing new caching schema. Velocity comes with Dublin; the same setup programme installs Velocity and Dublin.

For more details about the architecture details please Click Here!!


Keep watching this space for more technical updates.

Microsoft released Web Platform Installer 2.0

Commonly known as Web PI; a tool that makes user life easier to sync up with the latest updates for the MS products like Internet Information Services (IIS), Visual Web Developer, .NET Framework, SQL Server Express Edition etc. (List goes on and on and on......). The Web PI has Windows Web Application Gallery using which you can users can have pleasant experience whle blogging. It also provides the support for Content Management Activities.

 

 

For downloading Web PI 2.0 Click Here!!

 

For more technical updates and latest technology news keep watching this space,

 

All about WCF (Introduction)

Here in my first series of article I will be sharing top to bottom of the WCF (The Indigo).

WCF is the great way to approach large multi-system message based communication. It seamlessly support Transaction Management, Security Implementation, Exception and Fault handling. Concurrency management etc.

Usage of WCF comes with the great amount of flexibilities' to handle. WCF is a framework for services. The user can Create, Configure, Host and consume the services. All the services are mutually agreed on SOAP concept and communicating between WCF based clients.

WCF has mainly two categories of classes:

1. Service Model Class (Handles Service Configurations & Validations).

2. Channel Classes (Handles mode of communication and related functionalities).

Each WCF service can be associated with one ore more service endpoints. One service can be hosted in WCF with different mode of communication channel. Depending upon the usage and target client WCF Service can be configured (I will focus this more on this in my up coming articles).

Configuring endpoints require (A, B & C):

1. Address ('A') = Where?

2. Binder ('B') = How?

3. Contract ('C') = What?

Address: The address means the URL/URI where the service is located. Where is the service located?
Binder: The mode ofcommunication. How to communicate with the service?
Contract: The Service. What is my service?

With the XML Based configuration approach OR with the help of coding we can configure the endpoints.

Hosting WCF Service

WCF Service is can be segregated in to two parts normally:

1. Service Contract

2. Service Implementation

Service Contract:

Service Contract is just the collection of functionality that Service will be exposed. In a way you can assume that a Service Contract is just like the interfaces in .NET.

For example

In above example, I have created a sample service with just one operation i.e. MyOperation with one parameter. As you can see i have used Attributes. ServiceContract attributes comes under System.ServiceModel. Including this DLL as in reference list is must for WCF Service Host and WCF Consumer.

You can create multiple operations by using "OperationContract" attribute. By specifying OperationContract attribute means the operation will be available for WCF Service and the WCF client can consume that.

Service Implementation:

As named Service implementation is the implementation bin of WCF Service Contract. You can implement this WCF Service Interface in 'N' number of classes.

For example

In above example I have created "HelloIndigoService" which is implementing our "IHelloIndigoService" service interface.

WCF Service can be self-hosted service but to start with we will see simple .NET consol based hosting mechanism.

Attributes of Service Contract

Using declarative model you can apply different parameters (as per your requirements) on WCF Service Contract:

CallbackContract

User can specify the call-back service that represents opposite service contract in duplex message exchange.

Example

ConfigurationName

User can specify the pre-created configuration name which is/are defined in Application's configuration file.

Example

Name:

Name property is impacting on WSDL generation. With the help of this property user can specify XML Root Tag name of WSDL. Unlike the older messaging mechanisms, WCF provides great deal of control over WSDL generation. I will discuss more about forming the WSDL with OperationContract later in this article.

Example

Namespace:

Namespace is also related with WSDL formation. After you generated the proxy of WCF Service; the given namespace will be appear in WSDL Namespace.

Example

ProtectionLevel:

User needs to use ProtectionLevel according to their service requirements and the operations that service will be exposing. There are three different ProtectinLevels are available.

1. None:

2. Sign: The protected part is signed digitally to ensure the temper less submission of data

3. EncryptAndSign: This will encrypt data before sighed it digitally.

More details on ProtectionLevel : http://msdn.microsoft.com/en-us/library/aa347692.aspx

SessionMode:

Specifying the attribute with "Allowed", "NotAllowed" and "Required". As name says "NotAllowed" will not support Session. The main difference between "Allowed" and "Required" is: "Allowed" will support Session persistent if incoming binding supports. Where as "Required" will throw an error in case binding is not supporting.

ASP.NET Cookieless Session [Pros & Cons]

            

Enter Cookieless Sessions

you don't have to change anything in your ASP.NET application to enable cookieless sessions, just enter sessionState cookieless="true" in your web.config.

Internal Implementation of Cookieless Session in ASP.NET

The implementation of cookieless sessions includes couple of runtime modules:

1.   1. SessionStateModule [A standard session HTTP Module]
 2. Aspnet_filter.dll [Works as an executable]

Once the HTTP Request come at the server, a small piece of Win32 code works as an ISAPI filter. HTTP modules and ISAPI filters are in opinion of the same concept, except that HTTP modules are created by the managed code and require ASP.NET and CLR to trigger and work. Classic ISAPI filters like aspnet_filter.dll are invoked by IIS. Both capture IIS events fired during the processing of the request.

When the first request of a new browser session comes at server to process, the SessionStateModule reads about the cookie support in the web.config file [By default “machine.config” specifies cookie enabled session]. If the “cookieless” attribute of the  section is set to true, the module generates a new session ID, twists the URL by appending the session ID just before the requested page name, and redirects the browser to the newly created URL using the “HTTP 302 command”.

When each request reaches at the IIS boundary—far before it is handed over to ASP.NET—aspnet_filter.dll is given a chance to look at it. If the URL appends a session ID in parentheses, then the session ID is extracted and copied into a request header called AspFilterSessionId. The URL is then rewritten to exactly like the originally requested resource and let go. This time the ASP.NET session state module retrieves the session ID from the request header and proceeds with session-state binding.

The cookieless mechanism works great as long as the URL contains information that can be used to obtain the session ID. As you'll see in a moment, this poses some usage restrictions.

Let's review the pros and cons of cookieless sessions.

Advantages of Cookieless Session

In ASP.NET, session management and forms authentication are the only two system features that use cookies under the hood. With cookieless sessions, you can now deploy stateful applications that work regardless of the user's preferences about cookies. As of ASP.NET 1.x, though, cookies are still required to implement forms authentication. The good news is that in ASP.NET 2.0 forms authentication can optionally work in a cookieless fashion.

Another common reason advanced against cookies is security. This is a point that deserves a bit more attention.

Cookies are inert text files and as such can be replaced or poisoned by hackers, should they gain access to a machine. The real threat lies not much in what cookies can install on your client machine, but in what they can upload to the target site. Cookies are not programs and never run like programs; other software that gets installed on your machine, though, can use the built-in browser support for cookies to do bad things remotely.

Furthermore, cookies are at risk of theft. Once stolen, a cookie that contains valuable and personal information can disclose its contents to malicious hackers and favor other types of Web attacks.

Disadvantages of Cookieless Session

By looking the advantages of cookieless session we should not reach on any conclusion: Considering security aspect, your cookieless sessions are easier to hack in compare. Session Hijacking can act against this approach.

In brief, session hijacking occurs when an attacker gains access to the session state of a particular user. Basically, the attacker steals a valid session ID and uses that to get into the system and snoop into the data. One common way to get a valid session ID is stealing a valid session cookie. That said, if you think that cookieless sessions put your application on the safe side, you're deadly wrong. With cookieless sessions, in fact, the session ID shows up right in the address bar

With cookieless sessions, stealing session IDs is easier than ever.

Using cookieless sessions also raises issues with links. For example, you can't have absolute, fully qualified links in your ASP.NET pages. If you do this, each request that originates from that hyperlink will be considered as part of a new session. Cookieless sessions require that you always use relative URLs, like in ASP.NET postbacks. You can use a fully qualified URL only if you can embed the session ID in it. But how can you do that, since session IDs are generated at run time?

The following code breaks the session:

Click

To use absolute URLs, resort to a little trick that uses the ApplyAppPathModifier method on the HttpResponse class:

    href= >Click

The ApplyAppPathModifier method takes a string representing a URL and returns an absolute URL that embeds session information. For example, this trick is especially useful in situations in which you need to redirect from a HTTP page to an HTTPS page.