How to Create Business Value with a Microservice-Based Integration Platform

We live in a fast-moving digital world. IT systems are shooting up everywhere, and companies are opening new interfaces between internal systems and the rest of the world in the hope of financial savings and increased efficiency.

However, the dream may be short-lived, as hasty integration to the many new systems may create more problems than it solves. In fact, it often turns out that the systems that need to be integrated into the company's IT landscape are unable to communicate.

This new challenge can be difficult to tackle retrospectively. It is therefore crucial to define a strategy in advance on how to facilitate inter-system communication in your IT infrastructure.

When the IT stack presents challenges

If you are a company with a large - perhaps obsolete - IT stack, you will often experience a number of typical issues.

1. Legacy systems in the IT infrastructure become a burden as they no longer cover all needs.

2. There are hard dependencies between the systems in the infrastructure, making them difficult to maintain or replace.

3. Security holes occur because a system is not properly secured or access is granted to too many areas.

4. A very large number of systems makes it difficult to see how all the systems work together due to a lack of logging across systems.

5. The systems in the infrastructure may have specific definitions of business resources (a 'customer' in one system is probably not the same as a 'customer' in another)

6. It can be difficult to integrate new systems into the infrastructure if the interfaces are obsolete or poorly designed. These problems will accumulate as more and more systems are integrated. Therefore, preventive action should always be taken to avoid them.

Tackle the problems with an integration platform

A popular solution to integration problems is to extend your IT stack with an integration platform. An integration platform is a stand-alone application (or portfolio or applications) that serves as a link between the different components of the infrastructure and allows functionality to be added.

The integration platform can be used to develop solutions to all the above problems, independently of the other responsibilities of the infrastructure. Integration platforms can be as comprehensive as you wish - from a simple API Gateway to a comprehensive Enterprise Service Bus.

Depending on its scope, an integration platform can:

1. Add functionality on top of old systems, extending the life of the systems until they are retired.

2. Decouple IT systems from each other, so that a change in - or replacement of - one system does not require a change in all the dependent systems.

3. Add a layer of security around systems that are not themselves designed for security.

4. Introduce comprehensive and uniform monitoring of activities within the infrastructure in the form of logging and collection of metrics.

5. Assist in the translation of business resources between systems.

6. Provide a uniform interface that all systems can integrate to across the entire infrastructure, including by translating poor system interfaces.

Choose the right architecture for your integration platform

If you choose to implement an integration platform in your IT landscape, it is important to choose the right software architecture for the job:

1. It should be instantly scalable and available during maintenance, as it should not become a bottleneck in the infrastructure.

2. It should be possible to develop and deploy it quickly, as it often needs to be expanded at very short notice, perhaps even in several directions.

3. It should be possible to expand it securely without destroying existing functionality so that the entire infrastructure does not risk breaking when integrating into another system.

4. It must be convention-agnostic. For example, it is no use if it only integrates to SOAP APIs.

5. It must be robust, as it will have to deal with many errors and unexpected scenarios from the surrounding systems.

An architecture that covers all these needs is a containerized (or container-based) microservice architecture.

What are the benefits of a microservice-based integration platform?

Microservices are a modern (and de-facto standard) design principle for enterprise applications. The idea is simple: Instead of building one big application that is responsible for everything at the same time, you build a lot of small applications, each with its own narrow area of responsibility.

When microservices are 'containerised', it means that the small applications run in closed, virtual containers. These can be run independently of each other in a wide range of operating environments and independently of the underlying computing platform.

The benefits of a microservice architecture are particularly valuable when it comes to integration platforms:

1. Having a limited scope of responsibility for each microservice allows you to scale parts of the integration platform vertically - as needed. This scaling is made even easier by the fact that microservice instances can be abstracted away via containerisation and can therefore be managed by modern container management solutions such as Kubernetes.

2. Running multiple instances of the same microservice can prevent downtime during maintenance.

3. The loose coupling between microservices in the form of well-defined and narrow interfaces means that they can each be developed in parallel and much faster than traditional applications. It is therefore also easier to implement continuous delivery practices, further increasing the speed of development.

4. The loose coupling and separation of responsibilities also allow you to add a new service to your existing microservice architecture without having to worry about disrupting the other services.

5. At the same time, it offers ample opportunity to integrate robustness via circuit breakers, time-outs or similar in your infrastructure.

6. Since each microservice is an independent application in itself, it makes it possible to decide for each service which technologies and conventions it should be based on. So, the integration platform can support otherwise incompatible technologies and conventions at the same time.

Avoid the classic pitfalls

While a microservice-based integration platform can do wonders for a complex system portfolio, you should be careful when implementing it. There are several pitfalls in the process - pitfalls that can quickly make the integration adventure less adventurous.

A common problem is that the integration platform becomes too thick. This happens when business logic slips in or the platform generally takes over tasks it is not designed for.

Moreover, you should be careful not to use too many different technologies across your microservices in the integration platform. This can make the solution difficult to maintain and operate since the know-how required increases with each technology.

Finally, avoid too many internal connections between microservices in the integration platform, since this increases the complexity of the solution. The integration platform then becomes part of the problem it is trying to solve.

However, these problems can easily be avoided as long as you are aware of them.

An integration platform is a solution to a defined problem. A problem that many companies have experienced and many more will experience in the future.

With an integration platform, companies can secure their IT architecture in a flexible and robust way, bolstering it for decades to come. It will thus become an indispensable part of the infrastructure as the digital landscape expands in business and society.