My previous post stated that semantic dissonance is the real challenge of content integration and I will try to illustrate this in this post by using a real world case.
Semantic dissonance between two content sources is a typical silo effect, something that happens when two or more information systems that have more or less overlapping content have been developed in isolation from each other, typically to support specific functions within an enterprise. Semantic (as well as structural) dissonance can be expected when information systems developed in two organizations that are to be integrated after a merger or an acquisition. But it is also a common scenario within a single organization.
A couple of years ago, I was working with preparing a new version of an e-commerce site for a global corporation. The site had been in a status quo condition for a couple of years and was now to be redesigned and equipped with a new user experience with more interactive features and richer content. The site was fed with product information such as product structures, descriptions, images, prices and stock quotes from a couple of back-end system, one of which provided the basic product information. The integration with the back-end systems had always been a problem child and not worked properly since it was first designed and implemented several years earlier. The back-end systems where custom developed and the e-commerce platform used for the web site was a standard product.
When we dug a bit deeper into the design and implementation of the current site and how it used the product catalog that was part of the e-commerce platform, it showed that there was semantic dissonance between the product catalog and the back-end system that provided master product information. To put it short, the definition of a product in the e-commerce platform did not match the definition of a product in the back-end systems. Despite the dissonance, the e-commerce site was fully functioning, but mostly due to a number of workarounds. What didn’t work was to administer the products and related content in the administration tool that came with the e-commerce platform. Nor was it possible to use the features that came out–of-the-box with the e-commerce platform – we would have to build them from scratch. Interestingly, the first version of the e-commerce site was designed and implemented by the same company that provided the e-commerce platform.
This insight put us on a long journey where we had to change not only the public e-commerce front-end and how we used the product catalog, but also the integration solution to the back-end systems. We had to convince the business people that this was absolutely necessary to be able to develop the e-commerce sites in the direction they wanted.
The tool that helped us succeed was an information model that helped us distance ourselves from the data models and identify and resolve the dissonance. We (re)defined the term "product" and identified the matching entities in each of the two systems. If looking only at the implementation, it now seemed as we had mapped two different entities with each other. But what differed were only the terms, not the concepts behind them. The information model clarified that the two different terms actually meant the same thing and thus guided the redesign of the e-commerce site and the integration solution.
The trick is how we created the information model. We did not bang the drums and announce to each and everyone that we would now need to redefine the term product and some related terms. That kind of approach would certainly have met resistance because everybody is naturally protecting “their” definitions. To suggest re-defining a term such as “product” in a successful business that was entirely build on their products would probably been political suicide. Instead, we started off with discussing the existing and somewhat inconsistent and confusing product definitions. We did not attack the flaws. We simply said that we needed to understand these terms and their definitions better. So, we modeled and discussed them together in a series of workshops. And behind the scenes, we created the information model piece by piece. It might seem cunning, and it was.