Eram

Data Factory: implementing and managing Eram Group's multichannel flows

Context

Using Data as a driving force to support Eram Group's external growth

The Eram Group consists of 9 footwear and textile brands.
Historically based in Western France, the group's activities now span France and internationally.
The group's IT department has decided to centralize its resources in the cloud, replacing its historical data centers, in order to serve all these activities.

Challenges

A Data mediation platform to harmonize all data flows

The aim of the 'Mediation' project is the interconnection of all the components of the information system (IS) and the service of all the business locations that have to exchange data: workshops, warehouses, shops, head office, local information centers.
The challenge was to design a general-purpose flows management platform that could handle any flow type and scale as flows grew.

Our approach

Meet the challenges of harmonization and acceleration with a single workflow management platform

Create the "on cloud" platform based on open source components around a "micro service" architecture capable of adapting to the expected flow load, whether in continuous growth mode or in peak management mode.
Define open source building blocks for each of the platform components: kong (apim), Nifi (flows), Kafka (event queues), Air Flow (orchestration), ELK (logs and monitoring graphs).
Running Kubernetes containers
Google Cloud Platform Deployment
Transferring the know-how required to create and deploy new flows to the Eram teams

The results

Deployment and management of more than 200 flows on an omnichannel, elastic platform

The Mediation platform has now reached cruising speed, with over 200 flows deployed and new flows being added at an accelerated rate each month, following an initial POC roll-out based on a few prototype flows:

Resilient, highly available architecture (microservices)
Deploy to Google Cloud Platform and run with Kubernetes
Fully portable solution with no cloud provider dependency
Open source components that do not cause 'inflationary' effects when scaling up

The challenge for Eram was to design an architecture for a data mediation platform that takes into account the volume and type of data flows, the transformations required and the different protocols to be processed. Building block interoperability and version compatibility were among the issues we had to address. All the building blocks chosen are open source and containerised, so that they are not locked up in one specific cloud. The strengths of our platform are observability, scalability, performance optimization and resilience.

Jamel BEN AMAR

CTO, Smile