LMES Process Log Specification

Overview

This specification document aims to provide a standardized recording, storage, and management scheme for process logs in the LMES system. Through a standardized log system, core functions such as process traceability, problem diagnosis, and performance monitoring are realized. This specification covers log architecture design, classification system, storage strategy, format definition, interface usage, and best practices.

I. Log Architecture Design

1.1 Overall Architecture

The LMES process log system adopts a clear layered architecture design and implements a highly flexible log routing and storage mechanism through the NLog framework. This architecture forms a complete log processing link from the business initiation layer to the final storage layer, with clear responsibilities between layers, low coupling, and easy expansion and maintenance.

1.1.1 Layered Architecture Description

The architecture of the LMES process log system is divided into five core layers, forming a complete closed loop from business operations to final log storage:

Architecture Layer	Core Component	Main Responsibility	Implementation Method
Business Layer	Business components like BusinessActivity	Trigger log recording requirements, provide business context information	Call via Flow.Logger interface in business code
Log Service Layer	Flow.Logger Interface	Encapsulate log recording logic, unify log format	Provide standardized log recording methods (Info/Warn/Error, etc.)
Log Framework Layer	NLog Logger	Implement log routing, filtering, and formatting	Distribute logs to different targets based on configuration
Configuration Layer	NLog-Flow.config Configuration File	Define log output rules, paths, and formats	XML format configuration file, supports dynamic adjustment
Storage Layer	File System	Organize and store log files by process and station	Hierarchical directory structure, supports log file rotation

1.1.2 Log Flow Process

The complete flow process of logs from generation to storage is as follows:

1. During the execution of Business Layer Activity components, various business events and status information are recorded through the Flow.Logger interface.
2. The Log Service Layer receives these log requests and standardizes them according to the predefined format.
3. The processed logs are passed to the NLog Framework Layer for filtering and routing based on configured rules.
4. The NLog framework distributes logs to corresponding storage targets according to the configuration in NLog-Flow.config.
5. Finally, logs are stored in the file system as files, managed by process and station classification.

This layered design allows the LMES process log system to meet the flexibility of business needs while ensuring the standardization and traceability of log management.

1.2 Log Classification System

1.2.1 Category Naming Convention

CategoryName: The category name of the message generated by the logger.

Log Category	CategoryName Pattern	Description	Example
Process Log	CMS.Plugin.Processflow.ProcessName_StationName	Process execution log	CMS.Plugin.Processflow.PLA_Process1_Station1
Trace Log	CMS.Plugin.TraceLogger.ProcessName_StationName	Variable tracking related log	CMS.Plugin.TraceLogger.PLA_Process1_Station1

Default process log message category format: CMS.Plugin.Processflow.ProcessName_StationName. By default, the LMES host ProductionProcessProvider has been initialized according to the convention. If you want to customize it, you can modify it as shown below.

1.2.2 Log Level Definition

Level	Purpose	Usage Scenario
Trace	Detailed debug info	Variable tracking during development phase
Debug	Debug info	Process analysis during development phase
Info	General info	Normal process execution recording
Warn	Warning info	Abnormal situations but not affecting process
Error	Error info	Process execution failure
Fatal	Serious error	System-level failure

II. Log Storage Strategy

2.1 Storage Path Design

The LMES plugin provides the NLog-Flow.config file by default, used to store process logs in folders by {ProcessName}_{StationName}.

logs root directory/
├── flow_logs/
    ├── Process1_Station1/
    │   ├── flowlog_2024-01-01.log
    │   ├── tracelog_2024-01-01.log
    │   └── warnlog_2024-01-01.log
    ├── Process1_Station2/
    │   ├── flowlog_2024-01-01.log
    │   ├── tracelog_2024-01-01.log
    │   └── warnlog_2024-01-01.log
    ├── Process2_Station1/
    │   ├── flowlog_2024-01-01.log
    │   ├── tracelog_2024-01-01.log
    │   └── warnlog_2024-01-01.log
    └── ...

After enabling the above configuration, the process log storage path is as shown above. Note: Other business logs will not be stored in folders by {Process}_{Station}. If you want to store them in this path, please refer to the first part to configure the logger.

2.2 File Rotation Rules

File Type	Max File Size	Max Archive Count	File Naming Pattern
Process Log	20MB	100	flowlog_shortdate.log
Warning Log	20MB	10	warnlog_date:yyyy-MM-dd.log
Trace Log	20MB	50	tracelog_shortdate.log

2.3 NLog Configuration Integration

Subsequent host versions will load this configuration file as an NLog extension configuration file by default. If the current host version does not have this configuration, you can manually modify the $(CMSRootPath)/host/NLog.config file and add the following code:

<include file="NLog-Extend.config" ignoreErrors="true" />

2.4 NLog Configuration File Explanation

2.4.1 Core Configuration Elements

The NLog-Flow.config ($(CMSRootPath)/plugins/cms.plugin.messuite/Configs/NLog-Flow.config file) contains the following key configurations:

Variable Definition:

• basedir: Log root directory, default is ../logs
• logDirectory: Process log directory, value is flow_logs
• flowLogDirectory: Dynamic process station path, generated by replacing logger name {Process}_{Station}
• traceLogDirectory: Trace log dynamic path {Process}_{Station}

Target Configuration:

• flowLogfile: Process log file target (Info level)
• warnLogfile: Warning log file target (Warn level)
• traceLogfile: Trace log file target
• consoleLog: Console output target (used in development phase)

Rule Configuration:

• Trace Log: CMS.Plugin.TraceLogger.* → traceLogfile
• Process Warning: CMS.Plugin.Processflow.* (Warn level) → warnLogfile
• Process Log: CMS.Plugin.Processflow.* (Info level) → flowLogfile
• Console Output: CMS.Plugin.Processflow.* (Trace/Debug level) → consoleLog

2.4.2 Configuration File Inclusion Relationship

Environment	Configuration File Location	Inclusion Relationship
Host	$(CMSRootPath)/host/NLog.config	Includes NLog-Extend.config
Extension	NLog-Extend.config	Includes each plugin's process config
LMES Plugin	plugins/cms.plugin.messuite/Configs/NLog-Flow.config	Independent configuration
WCS Plugin	plugins/cms.plugin.wcs/Configs/NLog-Flow.config	Independent configuration

III. Log Format Specification

3.1 Standard Format Definition

3.1.1 NLog Log Format

[${longdate}] [${level}] [${threadid}] [${logger}] ${message} ${exception}

Field Description:

• ${longdate}: Timestamp (Format: yyyy-MM-dd HH:mm:ss.ffff)
• ${level}: Log level (Trace/Debug/Info/Warn/Error/Fatal)
• ${threadid}: Thread ID
• ${logger}: Log recorder name (CategoryName)
• ${message}: Log message content
• ${exception}: Exception information (if any)

3.1.2 Message Content Format

${ProcessNo},${StationName}_${ProcessName},【Instance=${ProcessInstanceId}】->【${StepName}】-> ${BusinessMessage}

Field Description:

• ProcessNo: Process definition number
• StationName: Current station name
• ProcessName: Process instance name
• ProcessInstanceId: Unique identifier of the process instance
• StepName: Current executing Activity name
• BusinessMessage: Detailed information related to business

3.2 Format Example Comparison

Inbound process log example

[2024-06-05 14:24:34.7023] [Info] [49] [CMS.Plugin.Processflow.PLA_Process1] 10001,PLA_GX1_Station1_GeneralProcess,【Instance=717568446925】->【Inbound Signal Monitor】->Enter

IV. Log Interface Usage

4.1 IFlowLogger Interface Specification

For process log output, LMES has encapsulated the corresponding log interface. Developers only need to get this interface in the corresponding business code and output business logs, without having to splice process-related fields.

4.1.1 Core Method Definition

Method	Purpose	Parameter Description
LogMessage(string message, string stepName)	Record general info	message: Business message, stepName: Executing step name
LogWarningMessage(string message, string stepName)	Record warning info	Same as above
LogErrorMessage(string message, string stepName)	Record error info	Same as above
LogExceptionMessage(Exception ex, string stepName)	Record exception info	ex: Exception object, stepName: Executing step name

4.1.2 Usage

In Activities inheriting from ProcessBusinessActivity, you can directly use Flow.Logger for log recording:

// Example 1: Record general info
Flow.Logger.LogMessage($"Recipe processing judgment, OrderNo={currentOrder?.Code}, ProductModel={currentProduct?.Model}", Name);

// Example 2: Record warning info
Flow.Logger.LogWarningMessage($"Material barcode: {materialCode} is empty", Name);

// Example 3: Record error info
Flow.Logger.LogErrorMessage($"Recipe processing judgment exception: {e.Message}", Name);

// Example 4: Record exception
Flow.Logger.LogExceptionMessage(ex, Name);

4.2 Activity Integration

4.2.1 Typical Usage Scenarios

1. Process Information Recording

Flow.Logger.LogMessage($"Recipe application info: ProductModel={formulaApplyModel.ProductModel}, FormulaCode={formulaApplyModel.FormulaCode}, FormulaVersion={formulaApplyModel.FormulaVersionName}", Name);

2. Exception Handling

try
{
    // Business logic
}
catch (Exception e)
{
    canEnterStation = false;
    errorMsg = $"Operation exception: {e.Message}";
    Flow.Logger.LogErrorMessage(errorMsg, Name);
}

3. Debug Information Output

Flow.Logger.LogMessage($"Collected parameters: {string.Join(",", parameterCollectVarNames)}, 【TraceId={traceId}】", Name);

V. Best Practices Guide

5.1 Log Recording Principles

1. Information Completeness: Ensure enough context information is included

   • Include key business data (Order No, Product Model, Material Code, etc.)
   • Record operation results and status
   • For exceptions, record detailed error information

2. Format Consistency: Follow standard message format

   • Use unified field separator (,)
   • Use unified key-value separator (=)
   • TraceId uses unified format: 【TraceId={traceId}】

3. Performance Consideration: Avoid overly frequent log recording

   • Do not record large amounts of logs in loops
   • Set log levels reasonably
   • Avoid recording Trace/Debug level logs in production environment

4. Security: Avoid recording sensitive information

   • Do not record passwords, keys, and other sensitive data
   • Pay attention to personal information protection

5.2 Development Guidelines

5.2.1 Recommended Log Recording Pattern

// Correct example: Include key info and TraceId
Flow.Logger.LogMessage($"Operation description, KeyParam={value}, 【TraceId={traceId}】", Name);

// Pattern to avoid: Missing key info
Flow.Logger.LogMessage("Operation completed", Name); // Not recommended

5.2.2 Exception Handling Pattern

try
{
    // Business logic
}
catch (Exception e)
{
    canEnterStation = false;
    errorMsg = $"Operation exception: {e.Message}";
    Flow.Logger.LogErrorMessage(errorMsg, Name);
    // Or use
    Flow.Logger.LogExceptionMessage(e, Name);
}

5.2.3 TraceId Usage Specification

For operations requiring correlation analysis, TraceId should be used:

var traceId = Flow.Instance.GetTraceId();
Flow.Logger.LogMessage($"Operation description, 【TraceId={traceId}】", Name);
// Pass traceId when executing business operation
await service.WriteValueAsync(this, parameters, cancellationToken, retryCount, retryDelay, traceId);

5.3 Environment Configuration Suggestions

5.3.1 Development Environment Configuration

• Enable console output (Trace/Debug level)
• Detailed exception stack information
• Shorter log rotation cycle

5.3.2 Production Environment Configuration

• Disable console output
• Only record Info and above levels
• Reasonable file size and archive limits

5.4 O&M Monitoring Suggestions

1. Log File Monitoring: Monitor log file size and rotation status
2. Error Log Alert: Set alerts for Error level logs
3. Performance Metric Correlation: Combine with LmesMetrics for performance analysis
4. Regular Cleanup: Formulate log file cleanup strategy

VI. Performance Monitoring Integration

6.1 Log and Metric Association

The LMES process log system integrates with the performance monitoring system (LmesMetrics) to achieve unified management of log recording and performance metric collection:

6.2 TraceId Tracking Mechanism

TraceId is a key mechanism for problem tracking and performance analysis:

Usage Scenarios:

• Recipe parameter distribution
• Variable value read/write operations
• Distributed operation tracking
• Performance bottleneck analysis

Example:

var traceId = Flow.Instance.GetTraceId();
Flow.Logger.LogMessage($"Collected parameters: {string.Join(",", parameterCollectVarNames)}, 【TraceId={traceId}】", Name);
await variableService.WriteValueAsync(this, parameters, cancellationToken, retryCount, retryDelay, traceId);

All related log records contain TraceId, facilitating problem tracking and performance analysis.

6.3 Performance Metric Types

Metric Type	Purpose	Example
Counter	Counter, used to count event occurrences	Inbound count, Outbound count
Histogram	Histogram, used to count value distribution	Process time consumption distribution
Gauge	Gauge, used to record current value	Current inbound product count

VII. Troubleshooting Guide

7.1 Common Problem Troubleshooting

7.1.1 Log files not separated by {Process}_{Station} folders

Root Cause Analysis:

• NLog-Extend.config not correctly included
• CategoryName setting incorrect, does not include station name
• NLog-Flow.config configuration error

Solution:

1. Check if the host NLog.config includes NLog-Extend.config
2. Verify CategoryName setting in ProductionProcessProvider, ensure format is {Process}_{Station}
3. Check logger name matching rules in NLog-Flow.config

7.1.2 Incorrect Log Format

Root Cause Analysis:

• Custom Logger does not follow IFlowLogger interface
• Message formatting logic error
• Did not use Flow.Logger interface

Solution:

1. Use Flow.Logger instead of custom Logger
2. Ensure correct stepName parameter is passed
3. Follow standard message format specification

7.1.3 Log File Not Generated

Root Cause Analysis:

• Log level setting too high
• File path permission issue
• NLog configuration loading failed

Solution:

1. Check minlevel setting in NLog-Flow.config
2. Verify read/write permissions of log directory

7.2 Log Analysis Tools

7.2.1 Text Search

Use tools like grep for quick positioning:

# Search error logs
grep "\[Error\]" flowlog_2024-01-01.log

# Search specific product code
grep "SN001" flowlog_2024-01-01.log

# Search TraceId
grep "TraceId=12345" flowlog_2024-01-01.log

7.2.2 TraceId Tracking

Correlate related logs via TraceId:

# Extract TraceId
grep -o "TraceId=[0-9]*" flowlog_2024-01-01.log | sort | uniq

# View all logs for a specific TraceId
grep "TraceId=12345" flowlog_2024-01-01.log

7.2.3 Timestamp Analysis

Analyze process execution time distribution:

# Extract timestamp
grep -o "\[20[0-9-: .]*\]" flowlog_2024-01-01.log

# Count log quantity per hour
grep -o "\[20[0-9-]* [0-9]*:" flowlog_2024-01-01.log | cut -d' ' -f2 | cut -d':' -f1 | sort | uniq -c

7.2.4 Error Statistics

Count frequency of various errors:

# Count error level logs
grep -c "\[Error\]" flowlog_2024-01-01.log
grep -c "\[Warn\]" flowlog_2024-01-01.log

# Count specific error types
grep "\[Error\]" flowlog_2024-01-01.log | grep -o "Exception.*" | sort | uniq -c | sort -rn

7.3 Fault Handling Process

7.3.1 Log Troubleshooting Steps

1. Determine Problem Scope

   • Confirm the process and station where the problem occurred
   • Determine problem occurrence time
   • Confirm impact scope

2. Find Related Logs

   • Locate log folder by {Process}_{Station}
   • Narrow down range by timestamp
   • Search keywords (Product Code, Order No, etc.)

3. Analyze Log Content

   • View error log details
   • Analyze exception stack information
   • Track TraceId related operations

4. Locate Problem Cause

   • Compare normal and abnormal logs
   • Check configuration and parameters
   • Verify business logic

5. Solve and Verify

   • Implement repair plan
   • Verify if problem is resolved
   • Record solution

7.4 Preventive Monitoring

1. Real-time Monitoring

   • Monitor error log growth trend
   • Set error rate threshold alerts
   • Monitor log file growth speed

2. Regular Analysis

   • Daily log summary analysis
   • Weekly trend analysis report
   • Monthly problem summary and optimization

3. Capacity Planning

   • Estimate log growth trend
   • Reasonably configure storage space
   • Formulate cleanup strategy

VIII. Summary

This specification document provides comprehensive guidance for LMES process logs, from architecture design to practical application, from development specifications to O&M monitoring. By following this specification, you can:

1. Improve Log Quality: Unified format and complete information facilitate analysis
2. Speed Up Problem Positioning: TraceId and structured logs improve troubleshooting efficiency
3. Ensure System Stability: Discover and solve problems in time
4. Optimize System Performance: Continuous optimization combined with performance metrics

It is recommended that developers strictly follow this specification in daily development, and O&M personnel regularly check log quality and system running status to continuously improve and perfect the log system.