It is often the case that in an attempt to quickly set up business services and scale that people will turn right away to Off The Shelf (OTS) solutions, usually zeroing in what they have used before and just buying in and applying. The idea being to forego development costs of an in-house system in favour of an already existing system to get the business up to speed quickly.

Now, like all integration efforts with third party systems, there are a set of pro’s and con’s that come with this – but OTS solutions come with their own peculiar blend of problems, which will be discussed in this post.

Data Integration – the OTS spanner in the works

Now OTS is fine for an early stage business, but the true problem usually appears later on in the business life cycle (around 3 years), basically people want to know what is going on in the business in an integrated data sense and often there are more than one OTS systems in place. This creates what I call a ‘remote data owner’ problem, in that the OTS systems impose their own ideas of how data should be stored, displayed and manipulated – often at complete odds to each and likely at odds with what the business now requires.

This problem will show itself in several ways:

• No one true single way to ‘index’ a shared entity, like a product or invoice – naming schemes and transformations need to be defined as data moves between systems.
• Not clear where the Centre of Truth lies for entities integrated with OTS systems – who truly is responsible for the integrity of the data, is it the OTS system or something else? This can be a real nightmare when a system fails.
• Impossible to produce reports based on data that spans a mix of OTS and internal systems.

OTS Licensing Costs

The other problem with OTS solutions is that you have to pay for them and usually how much you use and depend upon them can be a direct factor in the cost; i.e. the more you make use of an OTS system and tie it into your business – the higher the cost.

This is especially true if you are using OTS technology within your own systems (say a framework that provides reliable scaling, or some top end database system). This is in effect a critical business dependency that comes with a premium price tag to match!

Putting the OTS Genie back in the bottle

Usually when we get involved a business will have a multitude of OTS systems in place, mostly around the following areas:

• Core accounting functions (Sales & Payroll);
• Inventory management (stock control);
• Account management (roles & permissions, subscriptions, etc)
• Content Management
• Customer Relationship Management

We might even find multiple different OTS systems doing the same function but in different locales or offices, this is not that unusual.

This reminds me of when they amalgamated the British Rail network into a handful of national operators from all the regional operators – they found they had well North of 300 different systems to integrate!

Now the first step in this is that we ask the following: What is the core business operational data flow?  i.e. how does the business enact its products and services in a work flow and a data flow sense?

Out of such a question we produce a diagram that then allows us to determine what it is the business needs to ‘own’ or ‘derisk’ to operate – i.e. if you were to remove an OTS (either by choice or the provider folds), could the business get back into a functional state quickly?  This should provides an immediate hit list of OTS systems that need closer examination as an immediate concern.

Wrap your critical OTS systems

Usually an OTS integration is quite a brutal affair and has more to do with coding sticky tape and tweaks to work flows than a clean ‘plug-in’ and go solution – a lot of rough edges have to be dealt with that weren’t evident in the beginning.

Now the trick is to do what I call a ‘OTS Wrap’ – which essentially is this:

1. Define all data inputs and outputs of the OTS system, in terms of formats and what other systems are involved;
2. Progressively introduce a well defined ‘translation layer’ – this is a specific system that sits around the OTS (hence the wrap) and effectively stops the direct coupling of the OTS with your other systems (and other OTS’s)

Now once this is done, we have effectively put the OTS in a well defined box – this means you can do the following:

• Monitor all data flowing in or out of the OTS at ease; so making it easy to populate a data mining service if you wish.
• Archive data interactions with the OTS – this could be used to replay in the case of failures or security incidents.
• Duplicate data flows to another system (may be something you have written yourself) to check for operational consistency.
• Easily swap out the OTS as needed, you don’t need change all the other system endpoints, just the internals of the box.

The other big win with this is that come renewal time, you now have the very real option of not renewing to either replace with your own system or another better suited OTS – you retain your ability to make well informed architectural & business decisions and not be at the mercy of the OTS.

If you have OTS’s that are tightly coupled into your core business, please get in touch, at Aykira we have specific experience rearchitecturing large online systems and saving businesses money by reducing their dependency on OTS solutions.