When logging in Mule applications, the goal is to capture information that helps in understanding the application's behavior, diagnosing issues, and ensuring security and compliance.
Many times we think that including as much info as we can would help but it can be the opposite. Logging excessive information, like huge payloads, can obscure important information and be misleading for the person who would be analyzing our logs.
In our previous post, we discussed Why Logging? The importance of having a clear Logging Strategy. If you haven’t seen it, head over to that post and start from there. If we want to take logging seriously, the first step is to decide what we want to achieve with our logs. Once that is clear then we can define what information we should log to work towards those goals
Here is a good list of the different types of info that can be interesting to include in our logs.
General Contextual Information
- Timestamp: The exact date and time when the log entry was created, preferably in UTC.
- Log Level: The severity of the log (e.g., DEBUG, INFO, WARN, ERROR, FATAL).
- Flow Name: The name of the flow or subflow where the log is generated (
#[flow.name]).
- Application Name: The name of the Mule application, which is crucial in API-led connectivity architectures.
- Environment: The environment where the application is running (e.g., DEV, TEST, STAGING, PROD).
Execution Context
- Correlation ID: A unique identifier that traces the lifecycle of a request across different flows and services (
#[correlationId]).
- Event ID: The unique identifier for the specific Mule event (
#[message.id]).
- Transaction ID: If transactions are involved, log the transaction ID to trace operations within a transaction.
- Thread Name: The name of the thread processing the request, useful in concurrent environments.
Request and Response Details
- Inbound and Outbound Endpoints: Log details of the inbound (source) and outbound (target) endpoints, including protocol, host, port, and path.
- HTTP Method and URL: For HTTP-based flows, log the HTTP method (GET, POST, etc.) and the full URL of the request.
- Status Codes: Log the HTTP status code for both inbound and outbound requests to understand the success or failure of operations.
- Payload Details: Capture the payload ONLY at significant points in the flow. For large or sensitive payloads, log summaries, metadata, or truncated versions. Be careful with this, logging too many payloads can make it difficult to read through your logs.
- Headers and Query Parameters: Log relevant HTTP headers and query parameters, but avoid logging sensitive information (e.g., Authorization headers).
Performance and Metrics
- Execution Time: The time taken to process a request or execute a flow, particularly useful for performance monitoring.
- Throughput: Log the number of transactions or messages processed over a certain period, especially in batch processing flows.
- Resource Usage: CPU and memory usage at critical points, if relevant.
Business Context Information
- Business Process Milestones: Log key business events, such as when an order is placed, payment is processed, or a shipment is dispatched.
- Data Identifiers: Unique business identifiers like Order ID, Customer ID, Invoice Number, etc., to correlate technical logs with business processes.
- Threshold Violations: Log events when business thresholds are exceeded, such as unusually large transactions or failed payment attempts.
Security Information
- Authentication and Authorization Events: Log successful and failed login attempts, token validations, and access control checks.
- Sensitive Data Handling: Note where sensitive data is masked, encrypted, or sanitized in logs.
- Audit Logs: Record who did what and when, especially for actions that affect security, data integrity, or compliance. For the actions in the Anypoint platform, we can rely on the built-in Audit Logs of the platform.
System and Integration Details
- External System Responses: Log responses from external systems, including response codes, errors, and latency.
- Connector-Specific Information: For database, JMS, FTP, or other connectors, log connection details, query strings, or commands executed.
- Circuit Breaker States: If using circuit breakers, log when they are tripped or reset.
Custom Application-Specific Data
- Custom Variables: Log any custom variables or properties that are essential for understanding the flow's behaviour or decision-making process.
- Decision Points: Log outcomes of key decision points in the flow, such as routing or filtering logic so that it will be easier to track incoming requests.
Deployment, Versioning and Configuration Values
- Application Version: Log the version of the application or API that is currently running.
- Deployment Details: Log the deployment date, time, and any related deployment ID or change request reference.
- Configuration Details: During startup, log critical configuration values (like endpoint URLs, timeouts, or environment variables) to ensure they are correctly set. Be cautious not to log sensitive configurations like passwords or API keys.
In summary, as you can see, there are multiple types of info that we could/should include in our logs. Be smart with this list. The intention is not to include all of that info on each log message. As part of your logging strategy you need to define when to include these different pieces of information. To understand better this, have a look at this post on When to log in Mule