ARCHITECTURE OF MACHINE LEARNING PIPELINE

Writing a working machine learning code does not guarantee a production ready system. This blog is about designing a machine learning prototype through the engineering road map. What you need is a team of professional Software Engineers by your side to take your (disposable) proof of concept and turn it into a performant, reliable, loosely coupled and scalable system!

Given below is the roster of objectives that we need to build a production ready system:

• Reduces latency;
• Is integrated but loosely coupled with the other parts of the system, e.g. data stores, reporting, graphical user interface;
• Can scale both horizontally and vertically;
• Is message driven i.e. the system communicates via asynchronous, non-blocking message passing;
• Provides efficient computation with regards to workload management;
• Is fault-tolerant and self healing i.e. breakdown management;
• Supports batch and real-time processing.

I will start-off by writing some of the most commonly used design architectures:

1. Reactive Architecture: Properties of a reactive system:
   
   • Responsive: A responsive system provides rapid and consistent response times, establishing reliable upper bounds so they deliver reliable quality product.
   • Resilient: The system stays responsive in case of failure. Resilience is achieved by replication, containment, isolation and delegation. Failures are contained within each component, isolating components from each other and thereby ensuring that parts of the system can fail and recover without compromising the system as a whole. Recovery of each component is delegated to another (external) component and high-availability is ensured by replication where necessary. The client of a component is not burdened with handling its failures.
   • Elastic: The system stays responsive under varying workload.
   • Message Driven: Reactive Systems rely on asynchronous message-passing to establish a boundary between components that ensures loose coupling, isolation and location transparency. This boundary also provides the means to delegate failures as messages. Employing explicit message-passing enables load management, elasticity, and flow control by shaping and monitoring the message queues in the system and applying back-pressure when necessary. Location transparent messaging as a means of communication makes it possible for the management of failure to work with the same constructs and semantics across a cluster or within a single host. Non-blocking communication allows recipients to only consume resources while active, leading to less system overhead.

   



2. Service Oriented Architecture (SOA): SOA centres around the concept of decomposing business problems into services. The services share information via the network and they also share code (i.e. common components) to maintain consistency and reduce development effort.
   


3. Streaming Architecture:
   A streaming architecture comprises of the following components:
   • Producers: Applications that generate and send messages
   • Consumers: Applications that subscribe to and consume messages
   • Topics: Streams of records belonging to a particular category and stored as a sequence of ordered and immutable records partitioned and replicated across a distributed cluster
   • Stream Processors: Applications that process messages in a certain manner (e.g. data transformations, ML models, etc).


4. Lambda Architecture: The Lambda (λ) Architecture is designed to handle both real-time and historically aggregated batched data in an integrated fashion. It separates the duties of real-time and batch processing while query layers present a unified view of all of the data. The concept is simple: When data is generated, it is processed before stored, so analysis can include data generated in the last second, the last minute, or the last hour by only processing the incoming data — not all the data. This is by far the most commonly used architecture in machine learning where data is considered as the first class entity in the entire architecture. Shown below is the diagram reflective upon a superficial example of this setup.

   



5. Microservice Architecture: Microservices, is an architectural style that structures an application as a collection of small, autonomous, loosely coupled and collaborating services, modelled around a business domain. The services communicate using either synchronous protocols such as HTTP/REST or asynchronous protocols such as AMQP. They can be developed and deployed independently of one another. Each service has its own database in order to be decoupled from other services.


6. Representational State Transfer (REST) Architecture: REST is an architectural style for developing web services and it builds upon existing features of the internet’s HTTP. It allows transferring, accessing and manipulating textual data representations, in a stateless manner i.e. applications can communicate agnostically. A RESTful API service is exposed through a Uniform Resource Locator (URL), which provides the capability of data being created, requested, updated, or deleted (CRUD). It is best used to manage systems by decoupling the information that is produced and consumed from the technologies that produce and consume it!