How to Send Mule Logs to Logstash with a Log4j HTTP Appender

In previous posts, we’ve seen how to use a Log4j HTTP appender in our Mule Apps to send logs to Elasticsearch. However, in some cases, we need to do some extra processing to our logs before sending them to Elasticsearch. That’s when Logstash comes into play.

In this post, we will see how easy is to send logs from a Mule app to Logstash using an HTTP appender. We will follow the next steps:

Set up Logstash Pipeline

The way Logstash processes logs is by defining a pipeline. A Logstash pipeline is a series of processing stages that Logstash uses to ingest, transform, and route data from one or more input sources to one or more output destinations. Each pipeline consists of three main components: input, filter, and output. Together, these stages define how data flows through Logstash.

This is how we’ll define the Logstash pipeline in our example:

Input - Set up an HTTP Listener in Logstash

For the input of our Logstash Pipeline we will use the HTTP input plugin. To use it, before setting up our pipeline, from our Logstash server we need to verify if the plugin is already installed in our system or not. We can easily retrieve all the input plugins installed by running the command:

sudo /usr/share/logstash/bin/logstash-plugin list --group input

and we’ll look for logstash-input-http . If the plugin is not installed we can install it with the command:

sudo /usr/share/logstash/bin/logstash-plugin install logstash-input-http

Once we’ve got the plugin, we can set up the input of our conf file like this:

input {
    http {
        host => "0.0.0.0"
        port => 8080
    }
}

Where:

host - represents the interfaces of our Logstash machine on which we’ll be listening. (All of them with the value 0.0.0.0)
port - Will be the port on which Logstash will be listening for events. Put here your preferred port.

Filter

To keep it simple, we’ll not be including any filter. Here we just want to show how to set up mule and logstash to work together. However, if you’re interested in creating Filters for your mule apps, check out these posts where we’re using some interesting Grok filters:

Logstash Grok Filters for Mule Logs in PatternLayout - Part I
Logstash Grok Filters for Mule Logs in PatternLayout - Part II

Output - Send logs to a File

Here, we’ll define two outputs:

The terminal, so that we can debug in real time the events that are coming from our mule app
A file, where we’ll set Logstash to write our mule app logs.

For that, the output of our pipeline conf file will be:

output {
    stdout {
        codec => rubydebug
    }
    file {
        path => "/home/ubuntu/mule-app.log"
    }
}

Test the Pipeline

From a terminal in our Logstash server, run Logstash specifying the conf file

sudo /usr/share/logstash/bin/logstash -f [YOUR_CONF_FILE]

Verify in the terminal that Logstash starts and remains listening for events

From postman send a POST request with some text:

Verify in the terminal that logstash receives and processes the message:

Lastly, verify that Logstash has create a new file and written you postman requests:

Create Mule App

Head over to Anypoint Studio and create a new Mule Project. Then, create a new flow with an HTTP Listener for GET /hello and a Logger Processor with a hello message and a Set Payload with a simple message to confirm the endpoint works

Configure the Log4j HTTP Appender

We will now set up the HTTP Appender in our Log4j configuration. Open the log4j2.xml located at src/main/resources and set up a new appender by adding the following lines within the Appenders section:

<Configuration>
    <Appenders>
        ...
        <Http name="logstash" url="http://[YOUR_LOGSTASH_SERVER]:8080">
            <PatternLayout pattern="%-5p %d [%t] [processor: %X{processorPath}; event: %X{correlationId}] %c: %m%n" />
        </Http>
    </Appenders>
    ...
</Configuration>

As a Layout, we’ll just use the default PatternLayout, for simplicity. Check out these other posts if you want to customize your PatternLayout or use the JSONLayout.

After that, add the appender to the root logger by adding the following lines within the Loggers section:

<Configuration>
    ...
    <Loggers>
        ...
        <AsyncRoot level="INFO">
            <AppenderRef ref="file" />
            <AppenderRef ref="logstash" />
        </AsyncRoot>
    </Loggers>
</Configuration>

Run the Logstash Pipeline and the Mule App

Time to see if our example works. First, open a terminal in your Logstash server and run the Logstash Pipeline with the previous command:

sudo /usr/share/logstash/bin/logstash -f [YOUR_CONF_FILE]

Once you see in the terminal that Logstash is listening for incoming events, then head back to Anypoint Studio and run the app. In a few seconds, right after the app is deployed to Studio, you will see in the Terminal of the Logstash server that Logstash starts to process events.

Verify

Lastly, after the Mule app has been deployed and is running, send some requests to the test endpoint (with curl or postman) and check the logstash terminal.

First, verify you’re getting the log message you set up in your app

Next, open the output file and verify all the app logs are being written correctly: