Recently, I was involved in coming up with a way to integrate Dynamics 2011 with other data sources that would be in XML format as well as data feeds into SQL tables. I was asked to limit the scope of integration to .NET and BizTalk tool-sets for a POC in the Microsoft stack of technologies. I started looking at BizTalk, but realized that with Microsoft not officially supporting the Dynamics adapter for the 2011 version, and that generating contracts and messages to a key/value pair service that Dynamics platform very cumbersome and difficult, I decided there should be a better way of doing this. (To see more information on this approach, read Richard Seroter’s article on this integration difficulty with BizTalk and Dynamics 2011).
So my next step was looking at the XRM SDK for Dynamics 2011. There was a nice little tool in there call crmsvcutil, that allowed me to generate the entity model and datacontext for any given Dynamics 2011 environment. Once I realized this, I knew this had potential.
The one challenge I had early on was, I wanted to decouple much of the details of data that was coming in, and the standard CRUD operations needed to save the data. So I implemented an abstract CRUD class that required an alternate key look-up against an entity. I did this by using generics, that way when implementing a concrete CRUD class, it would know what the return types would be and the abstract class required an implementation of getting a record by an alternate key, so the appropriate insert/update action could be taken.
I did the same type of things for a transaction against any given entity, having a base class that allowed for specific implementation, and allowed for overriding the save operation. The idea being that there could be a more complex multi-step transaction that needs to not use the base code, as it would commit incorrectly as an atomic unit of work. So having these base classes, I was able to implement them against any code generated by the crmsvcutil, and quickly create saves to entities, and override if I needed it to act differently. Cool thing was, most of the time I did not need to change much that was already implemented in the base.
Two other challenges I faced was, when the contract is generated from the data model, it does not include the values from the drop down lists. The second challenge I faced was when I mapped from an external contract to and entity, and then called into a transaction to save an entity, that entity was not a part of the data context, which was fine when there was no matches for the alternate key, but problematic when there was.
So, after doing a little reading on Microsoft site about the XRM framework, I realized there could be executed against the service the get a RetrieveEntityResponse which contains the metadata about an entity and all its attributes, by populating a RetrieveEntityRequest. The items in the request were, the logical name of the entity and filters you wished on the metadata. So I found this to be a very good candidate function to put in my entity base class that way every entity I implemented, would have this function to look up the metadata come along for the ride. Also, I noted this was a good candidate for implementing some type of caching, because these values I would consider to be slow to change and setting an appropriate expire interval on cache within a configuration file would be good. I also found this would be good for catching if an attribute was required by not part of the data passed in.
So for the second challenge I faced in mapping a new entity to an existing entity, I realized I should be able to use the list of attributes of the entity, and map the updates over to the one existing within the datacontext. Leveraging the metadata about an entity, I was able to create a generic mapper for any entity that can map all the attributes to the entity managed by the datacontext.
With this foundation of code written, I was able to expose a service contract and pass in data and map it over to the appropriate entity and call the appropriate transaction class for saving the data.
Form an architectural perspective a few things. There are tools out there such as Scribe for getting data into and even out of Dynamics CRM, so this approach is not the correct approach for all situations. The appropriate buy versus build analysis should be done. Secondly, I would challenge if the data being put into Dynamics CRM really will be mastered there. This is an important question as mapping data into a Dynamics CRM data structure through any integration is costly. It would be my suggestion if the data does not live in Dynamics or is not such a base object to most operations like customers and contacts, to have the system of record expose a service. If that is not possible and data is being shipped or mined, I would recommend looking at .NET Entity Framework ability to expose a service from data received against a data-store. This has the advantage that if in the future a service can get the data from the system of record, then changes to Dynamics are not needed. It also has the advantage of not needing populate custom entities within Dynamics, which can be costly on performance depending on the size of the data.
If you have any questions or different experiences with integration of Dynamics 2011, I would love to hear about them.