Upgrading the Mule Runtime is a maintenance operation of our Mule Infrastructure that needs to be carefully planned and documented. We’ll be upgrading the Mule Runtime every time we move between minor versions (from version 4.x to version 4.y) which, in case of the Edge Release Channel, will happen every three months. We don’t want to create disruption in our apps consumers that often. So, we need to understand well how to perform an upgrade. In this post, we will help you with that, we will review the three methods to upgrade the Mule Runtime on a Standalone instance:
- Upgrade using the Mule Upgrade Tool
- Upgrade via Runtime Manager
- Full manual upgrade
Upgrade using the Mule Upgrade Tool
The Mule Upgrade Tool is a command line utility provided by Mulesoft that helps us to upgrade an on premises instance of the Mule Runtime. The Mule Upgrade Tool also helps us to upgrade the Mule Agent, if that’s installed in the instance we’re running the tool in.The Mule Upgrade Tool does not upgrade the Anypoint Monitoring agent. For that, you need to uninstall it first and reinstall it once the Mule runtime has been upgraded.
The Mule Upgrade Tool helps us automate much of the upgrade process. The tool is very useful to identify configuration differences between the old and the new version and incompatibilities with connectors, libraries or other components. This helps us reduce the risk of missing essential configurations or dependencies and, therefore, reduces the chance of post-upgrade issues.
The Mule Upgrade Tool creates a backup before running the upgrade. This provides us with a roll back mechanism, allowing us to get back to the previous version very easily (and quickly) in case the upgrade does not go well.
The Mule Upgrade Tool is installed on the server where the mule runtime is running, on a specific folder of the Mule installation. The tool will perform an in-place upgrade. For that reason, to run the Mule Upgrade Tool, the Mule Runtime must be stopped. This means that, even though the tool automates much of the upgrade process, using the Mule Upgrade Tool requires a down time.
This has some pros and cons.
- The PROs are:
- It does not require additional servers for the upgrade.
- We don’t have to redeploy the apps, because they will remain in the same server. Depending the number of apps, redeploying all the apps can be a consirable amount of time.
- The main CONs are that it requires the Mule Runtime to stop during the upgrade. This means that, while the Mule Runtime is not available we will have to:
- Run the upgrade command
- Test and validate the apps are working as expected
- Restart the Mule Runtime
After the tool makes a backup of the current instance, it compares the configuration files of the current and the new runtime versions and if there’s any mismatch it will:
- Raise an error to let us know what files differ
- It will prepare a structure of folders where it will copy: the files that are ready, the files of the current instance that you need to review and the files in the new instance that mismatch the current version.
Since this is an in-place upgrade, the URLs of our apps will remain the same, so little impact on this aspect.
Lastly, the Mule Upgrade Tool can be used to upgrade Clusters. However, even if this is a cluster, all the nodes in the cluster need to be shut down to run the tool, as all the nodes in a cluster need to simultaneously run the same mule runtime version.
Have a look at our post on How to Upgrade the Mule Runtime with the Mule Upgrade Tool to follow a full step-by-step tutorial
Upgrade via Runtime Manager
Another way of upgrading the Mule Runtime is by setting up a new server in parallel to the existing one, with the newer mule runtime version and with a copy of the Mule Agent configuration from the current server.This way, when stopping the old server and starting the new one, Runtime Manager will pick up the new runtime version and redeploy all the apps automatically to the new server.
Have a look at our post on How to Upgrade an On Premises Mule Runtime with Runtime Manager to follow a full step-by-step tutorial
The customizations of the current server have to be manually set up on the new server, so we need to be careful not to miss any customization.
One of the benefits of this approach is that the new version of the Mule Runtime can be tested without installing the Mule agent. This means that, while the current Mule Runtime is running we can start the new mule runtime in this parallel server and do all the tests required before going live with the new version. We could even deploy an exact copy of our running applications and test how they run with the new runtime version while keeping them still not available.
However, the switch from the current runtime version and the new one requires to stop the runtime in the old server and start it in the new server (with the upgraded version). This means there will be a downtime with this method.
The good news, as mentioned, is that it is not required to redeploy the apps in the new server. Runtime Manager will upload a copy of the apps and start them automatically. It won’t be an immediate action though (so, this adds up to the down time) but it is automatic and should be quick. How quick that will depend on the number and the size of the total of apps in the server.
Lastly, the roll back is safe and quick. The current server is not modified during the whole upgrade process, so if needed we can roll back to the previous state just by stopping the new server and starting the old one.
Full Manual Upgrade
The third way of upgrading a Mule Runtime instance on premises is by doing a Full Manual Upgrade. In this case, we’ll be setting up a server in parallel to the current one, where we will install the new runtime version and redeploy the apps manually (or via our CI/CD).The process of a Full Manual Upgrade (at a very high level) would be:
- Set up a new server (or cluster) in parallel to the current mule server
- Install the newer version of the Mule Runtime on the new server
- Replicate all the configurations from the old server to the new one.
- Verify the new server can start and run the mule runtime with the new version + replicated customizations
- Register the new server in the Anypoint Runtime Manager
- Deploy an exact copy of the running apps in the new server
- Keep the new server only available for the Operations team
- Perform all the necessary tests to the apps in the new server to validate everything works in the new server (and if not, work on it till everything works)
- When the new server is validated switch traffic from the old server to the new one.
- Decommission the old server.
Out of the three methods, this is the method that requires more time and effort but, on the other hand, this is the only method we can use to perform a blue-green deployment strategy or provide Zero Downtime during the upgrade process.
It’s required to have a load balancer or similar mechanism in front of both servers to provide Zero Downtime and not to change the applications’ URLs.
When we use a parallel server for the upgrade, the direct URL of our apps will change. In a Mule Server, all the apps are published under the server hostname. So, if we change the server we will change the hostname and, therefore, the URL which our apps get published under would change, unless the Mule Server is behind a Load Balancer or an alias DNS. In that case, if our applications are fully stateless, it will be possible to run in parallel both mule runtime instances - the old version and the new one. This is how we could make the upgrade with zero downtime or use blue-green deployment
Which one is better? - My recommendation
As it happens in any conversation with an Architect, the answer is it depends. Here, there’s no black and white, right or wrong. Each method has its pros and cons, our job as Mule Architects is to find the method that better suits our environments and requirements. Nevertheless here’s some guidelines:- The quickest method is the Mule Upgrade Tool. It’s also the safest method, or in other words, the method that guarantees no errors due to configuration mismatches. It’s the most automated one. But it requires downtime.
- The full manual upgrade is the only method that gives us Zero Downtime and Blue-green deployments, although is the one that requires more effort and time
- The upgrade via Runtime Manager minimizes the redeploy of the apps and allows us to do all the necessary tests in parallel. But it also requires a downtime
- First, check how critical are the apps running in your mule server. Can we stop the mule runtime for a period of time? If so, for how long?
- If we require Zero Downtime because our apps need to keep on running at all time, then you’ll have to go for a full manual upgrade and provide the necessary infrastructure components for that (load balancers, extra servers, DNS, etc)
- If there’s a window of time in which we can run the upgrade then you need to evaluate how quick you can run your tests to validate all of your apps are running ok with the new mule runtime version.
- If you have a good and automated testing strategy that allows you to, once the runtime is upgraded, run your tests and validate them in quickly then the Mule Upgrade Tool is probably the best option for you, because it will minimize a lot the downtime.
- If you’ve set up a lot of customizations in your configuration files and you didn’t track those modifications probably the upgrade via Runtime Manager is the safest option for you. It’s true that the Mule Upgrade Tool identifies automatically mismatches in the configuration files but if you have to review all those changes, one by one and in some cases try to understand why you did those changes that would take too much time. Consider that resolving the conflicts and mismatches of the conf files is something that you would be doing while the mule runtime is stopped
- if you’re not sure about those modifications, probably it’s better to spin up a parallel server, with the newer version and go through those customizations with time, while the current version is still running and you’re preparing the new server.