A deep-dive into sizing your Runtime Fabric cluster

By -

Runtime Fabric has consolidated as the most flexible deployment option for the Mule runtimes. Anypoint Runtime Fabric can be installed in your on-premises data center or cloud-hosted environment (Azure, AWS). This type of deployment is similar to that of CloudHub in that it offers high availability and has zero-downtime for data centers.

To provide these capabilities, Anypoint Runtime Fabric must be installed following a set of minimum system requirements and a specific architecture. In this post, we’ll look at the server’s architecture, what type of servers we need, and how many.

Runtime Fabric supports configurations

But first things first, what type of role servers do we have in a Runtime Fabric architecture? There are two types of server/nodes:

  • Controller nodes: They are responsible for managing the cluster, running the orchestration services, and load balancing.
  • Worker nodes: These are the nodes responsible for running Mule applications.

Now that you know the type of nodes in our architecture, let’s learn how to combine them. Not every possible combination of nodes will be valid, we can’t just put any number of controllers and workers together. There’s a minimum of system requirements depending on the type of environment we’re creating. There are two types of supported configurations:

  • Development configuration: 
    • Minimum requirements:
      • 1 controller node
      • 2 worker nodes
  • Production configuration: 
    • Minimum requirements:
      • 3 Controller Nodes
      • 3 Worker Nodes

In terms of numbers, it’s also important to note that the maximum number of worker nodes is 16 and the maximum number of controller nodes is 5. The controllers, in addition, can be only in an odd number.

Let’s focus now on the controllers. As stated above, the minimum of controllers is one for dev and three for production. The maximum is five and it can only be an odd number. That leaves us with precise numbers:

  • For development, we can use one, three, or five controller nodes.
  • For production, we can only use three or five controller nodes.

Why are these numbers so specific? The answer resides in the underlying technology. 

Anypoint Runtime Fabric’s underlying technology

Runtime Fabric uses Kubernetes as the container orchestration technology and an instance of Etcd to maintain and persist the cluster state. This Etcd database is a distributed and reliable key-value pair store where Runtime Fabric saves the cluster’s defined configuration, its current state and the metadata associated. Etcd runs in all the controller nodes of the cluster and is responsible for ensuring there are no conflicts between the controllers and that every copy of the same data is consistent across the different controller nodes.

All the Runtime Fabric cluster information is stored in files, typically JSON or YAML. The goal of Etcd is to guarantee that this information is always the same, no matter from which node we’re accessing the cluster information.

The controllers will do one of two types of operations into this Etcd store: reads and writes. 

With reads it’s easy, since the same copy is available in all instances you can read from any node.

Writing is more complex. What if two write requests get to two different Etcd nodes? Which one goes through? In an Etcd cluster, not all the nodes process the write requests — only one of the nodes, known as the leader, is able to process a request to write. Internally, the nodes in an Etcd cluster elect a leader amongst them, one node becomes the leader and the rest become followers. The process of electing a leader in the cluster is out of the scope of this post, we’ll just mention that it’s based on the RAFT algorithm.

This way — when a write request is received by any controller node — the node does not process it. Instead, it forwards that request to the leader. It will be the leader writing to the database and afterwards makes sure that all the followers receive a copy of the data. The write is considered complete when the leader receives confirmation from the majority of the nodes in the cluster.

This concept of majority is the key in the functioning of Etcd and this is exactly what determines the number of controller nodes in a Runtime Fabric cluster. Majority is what we call quorum in a cluster terminology, it’s the minimum number of controller nodes that must be available for the cluster to function properly or make a successful write to its database. 

In a N nodes cluster, quorum = N/2 + 1. Let’s put some numbers to this formula and see our options in a Runtime Fabric architecture:

  • When we have one controller node, as in our minimum development configuration, if the controller goes down, the system goes down. 
  • If we’ve got 2 controllers, Quorum = 2/2 + 1 = 2. This means that if a controller goes down, the number of available controllers would be 1 and the Quorum 2. In this case, there would not be a quorum and the write requests would not be processed. So, even if you might think 2 controllers would give you better availability over 1 node, there’s no value in having 2 controller nodes.
  • That’s why the minimum number of controller nodes for high availability in our minimum production configuration is 3. With 3 controllers, quorum = 3/2 + 1 = 2, and now we can afford having one controller unavailable. In that scenario, the number of controllers available would be 2, as well as the quorum. Our cluster would keep on working with one failure.
  • Same logic applies for 4 controllers. With 4 controllers, quorum is 3, so if we lose one controller the cluster would keep on working. But if we lose another one, the number of available controllers would become 2 and Quorum 3. In other words, there’s no benefit in having 4 controllers over 3, we’ll have the same fault tolerance.
  • For that reason, if we want to provide more fault tolerance to our minimum configuration we will need a number of 5 controller nodes.
  • Five is the maximum number of recommended controllers for Runtime Fabric. More controllers are not really necessary to manage an environment of maximum 16 worker nodes.

Here below is a summary of the different values of controller nodes, their quorum value and the associated fault tolerance represented as the maximum number of failures.

CONTROLLER NODESQUORUMFAULT TOLERANCEE
110
220
321
431
532

And now your turn! How many servers are you going to include in your Runtime Fabric architecture. Check out our webinar to learn more about how to use Anypoint Runtime Fabric.

Popular posts from this blog

Benefici di Mule 4 - Parte 2

By -
Image
Come evidenziato nel primo post di questa serie , Mule 4 è un runtime con architettura diversa rispetto a quella della versione precedente Mule 3. I nuovi disegno ed architettura di questa versione portano dei miglioramenti sostanziali e nuove funzionalità. Nel post di oggi vedremo quali sono queste novità riguardo lo streaming e classloader isolation. Streaming Che cosa è lo streaming? In informatica, lo streaming è un meccanismo di trasmissione di dati che permette di fruire del contenuto desiderato senza la necessità di aspettare che questo ultimo sia completamente ricevuto. Nel contesto del runtime di Mule, lo streaming consiste  nella capacità di un event processor di incominciare a girare senza aspettare ad aver ricevuto tutti i dati dell processor precedente. Questo, come potete immaginare è incredibilmente utile quando si lavora con grandi quantità di dati, arrays o collections, come nel caso di un insieme di records che provengono dal database, da un file, oppure da uno stre
Read more

MuleSoft Metrics Accelerator

By -
Image
Metrics Accelerator es una herramienta para automatizar la extracción, agregación y la visualizacion de metricas, o mejor dicho, la carga de métricas en un sistema de visualización. Es una herramienta creada por nuestro equipo de Servicios Profesionales de MuleSoft, a partir del trabajo que han hecho para varios de nuestros clientes. Lo han estandarizado y lo han puesto a disposición de la comunidad como proyecto Open Source en GitHub. Por tanto, como proyecto open source se proporciona el código fuente y se deja abierto para que cualquiera pueda colaborar a evolucionarlo. Aquí tenéis el enlace al  proyecto en GitHub de Metrics Accelerator. En esta serie de posts vamos a ver en detalle la herramienta: desde cómo instalarlo hasta cómo crear dashboards en un sistema de visualización, viendo los detalles de funcionamiento y configuración. Aquí tenéis el detalle de los artículos dedicados a Metrics Accelerator: Metrics Accelerator (I) - Qué es y Cómo Instalarlo Metrics Accelerator (II) - T
Read more

The Mule Migration Assistant

By -
Image
La versione 4 del runtime di Mule è ormai da due anni con noi. Questo 2021 abbiamo due date importanti: una a Marzo, a partire dal 20 di Marzo non sarà più possibile pubblicare nuove applicazioni in Mule 3.9, e questa versione passerà a Extended Support fino il 2024. E la seconda data è a Novembre, a partire della quale finirà lo extended support alla versione 3.8 e di conseguenza tutte le applicazioni in questa versione non funzioneranno più. Potete trovare più informazione su questo link.   Version Release Date End of Standard Support End of Extended Support 4.3 April 30, 2020 April 30, 2022 April 30, 2024 4.2 May 2, 2019 May 2, 2021 May 2, 2023 4.1 March 20, 2018 November 2, 2020 November 2, 2022 3.9 long term supported October 9, 2017 March 20, 2021 March 20, 2024 3.8 long term supported May 16, 2016 November 16, 2018 November 16, 2021 3.7 July 9, 2015 November 16, 2017 January 25, 2020 3.6 January 15, 2015 January 15, 2017 N/A 3.5 long term supported May 20, 2014 July 15, 2016 *
Read more

Cómo configurar Postman para usar las Platform APIs de AnyPoint

By -
Image
Postman se ha convertido en los últimos años en la herramienta standard para cualquier desarrollador de APIs. Nos proporciona una interfaz intuitiva y sencilla para interactuar y probar APIs en general. Y, en particular, para nuestro trabajo con Mule hay dos escenarios donde Postman nos es muy útil. El primero, obviamente, como herramienta de testing. Una vez construidas nuestras Mule APIs podemos usar Postman para probar las distintas operaciones de nuestras APIs: GET, POST, PUT..., podemos construir nuestro JSON para las request e incluso podemos hacer nuestras pruebas con distintos métodos de autenticación en las APIs, por poner varios ejemplos. El otro caso de uso, para el que Postman nos es muy util como desarrolladores Mule, es para interactuar con las Platform APIs de Anypoint. Para los que no las conozcáis, las AnyPoint Platform APIs son el conjunto de APIs que MuleSoft pone a disposición para interactuar con todas las funcionalidades de AnyPoint Platform de forma programá
Read more

Benefici di Mule 4 - Parte I

By -
Image
La versione 4 del runtime di Mule è ormai da due anni con noi. Questo anno, come abbiamo visto in un altro post sul Mule Migration Assistant , è l'anno chiave per migrare da Mule 3. Mule 4 è la migliore versione del runtime al momento con tanti miglioramenti rispetto alla versione precedente. Per quelli che ancora non sono convinti, in questa serie di posts vedremo le principali novità di Mule 4. In questo primo post vedremo i cambiamenti fatti sulla struttura dei eventi Mule, la gestione dei threads e le caratteristiche non-blocking e self-tuning di Mule 4. Struttura dell’Evento Mule La prima cosa che si è semplificata nella versione 4 è la struttura del messaggio Mule, ovvero l’evento Mule. Un’evento Mule è l’unita di informazione, sulla quale lavora il runtime. Ogni messaggio inviato ad una applicazione Mule, sia un messaggio di tipo HTTP, JMS, MQ che in qualsiasi altro protocollo o formato è convertito a questo formato di evento, e poi ogni trasformazione o ulteriore elaborazi
Read more

MuleSoft Metrics Accelerator (I) - Qué es y Cómo Instalarlo

By -
Image
Metrics Accelerator es una herramienta para automatizar la extracción, agregación y la visualizacion de metricas, o mejor dicho, la carga de métricas en un sistema de visualización. Es una herramienta creada por nuestro equipo de Servicios Profesionales de MuleSoft, a partir del trabajo que han hecho para varios de nuestros clientes. Lo han estandarizado y lo han puesto a disposición de la comunidad como proyecto Open Source en GitHub. Por tanto, como proyecto open source se proporciona el código fuente y se deja abierto para que cualquiera pueda colaborar a evolucionarlo. Aquí tenéis el enlace al proyecto en GitHub de Metrics Accelerator. En esta serie de posts vamos a ver en detalle la herramienta: desde cómo instalarlo hasta cómo crear dashboards en un sistema de visualización, viendo los detalles de funcionamiento y configuración. En este primer post veremos cómo instalarlo y empezar a utilizarlo. Y, en los próximos veremos: Metrics Accelerator (II) - Tipos de Métricas Metrics Acce
Read more

MuleSoft Metrics Accelerator (IV) - Visualización en Splunk

By -
Image
En este nuevo post dedicado a Metrics Accelerator vamos a ver un ejemplo de cómo enviar nuestras métricas a un sistema de visualización, en concreto a Splunk. Este post pertenece a la serie de posts dedicada a Metrics Accelerator, si todavia no sabes qué es o cómo funciona te aconsejo empezar con los posts anteriores: Metrics Accelerator (I) - Qué es y Cómo Instalarlo Metrics Accelerator (II) - Tipos de Métricas Metrics Accelerator (III) - Modos de Ejecución Metrics Accelerator (IV) - Visualización en Splunk Metrics Accelerator permite el envio programado y automático de las métricas extraidas por la aplicación a un sistema externo de visualización. Como ejemplo, veamos como configurar Metrics Accelerator para enviar estos datos a Splunk en los siguientes sencillos pasos: 1. Configuración en Splunk Indices en Splunk. Desde Settings/Data seleccionamos la opcion Indexes y crearemos dos índices: metrics y platform _benefits , del tipo Events. Configurar o crear un nuevo HTTP Event Co
Read more

MuleSoft Metrics Accelerator (III) - Modos de Ejecución

By -
Image
En este tercer post de la serie dedicada a Metrics Accelerator  entraremos en sus detalles de configuración y veremos los ajustes necesarios para hacerlo funcionar en modo API o en modo Poller. Si todavía no conoces Metrics Accelerator o para qué sirve, te aconsejo que empieces por los primeros post y vuelvas a éste. Aquí abajo te dejo la lista de posts dedicada a Metrics Accelerator: Metrics Accelerator (I) - Qué es y Cómo Instalarlo Metrics Accelerator (II) - Tipos de Métricas Metrics Accelerator (III) - Modos de Ejecución Metrics Accelerator (IV) - Visualización en Splunk La aplicación de Metrics Accelerator viene preparada para trabajar en dos modos: API mode y Poller mode. Fundamentalmente para cubrir los dos grandes casos de uso que veremos a continuación: API Mode Como hemos visto durante el setup inicial en el primer post de esta serie , la aplicación nos proporciona una serie de endpoints que podemos consultar y obtener datos en cualquier momento. No hace falta configurar na
Read more

How to choose the CIDR block for your VPC

By -
Image
In one of my previous blog posts , we saw that creating a VPC is quite simple. We just need to provide values to four parameters and we can get our VPC up and running in minutes: Region CIDR Block Environments Business Groups From these four parameters, the CIDR block is usually the trickiest one to understand. And this parameter is critical — because once your Mule VPC is created, you can’t change your CIDR block. We’ll see, in the following lines, why this parameter is so important and how to properly calculate it. What is the CIDR block? To understand the CIDR block, we must first understand what an IP address is (don’t worry, this isn’t going to be a networking lesson). An IP address is the numerical representation of a location in a network. Similarly to how your phone number identifies your cell phone, your IP address identifies your device, your server, and your network interface. Computers only understand binary numbers, for that reason an IP address is just a sequence of
Read more

How many VPCs do I need?

By -
Image
In my previous blog post , we explained the different scenarios required to get a VPC in place. But there’s yet another question we need to answer before creating our first VPC. Is it enough with one VPC? How many VPCs do I really need? In this blog post, we’ll review some of the considerations to decide how many VPCs should be included into our network architecture. Backend location If we are going to be connecting our VPC(s) to our data center to provide your Mule apps with access to your back-end systems, the first question to ask is where the backend is. The Mule VPC will be created in an AWS Region, so create this VPC in the closest region to our backend to minimize the latency in the communications. However, many organizations don’t have a unique location for their data center. Sometimes data centers are geographically distributed in different on-prem locations or in different regions of a cloud provider. If that’s the case, you have two options: Create one VPC and connection
Read more