LMES Flow Log Specification

Overview

This specification document aims to provide standardized recording, storage, and management solutions for the process logs of the LMES system. Through a standardized log system, core functions such as process traceability, problem diagnosis, and performance monitoring are realized. This specification covers log architecture design, classification system, storage strategy, format definition, interface usage, and best practices.

I. Log Architecture Design

1.1 Overall Architecture

The LMES process log system adopts a clear layered architecture design and achieves highly flexible log routing and storage mechanisms through the NLog framework. This architecture forms a complete log processing link from the business initiation layer to the final storage layer, with clear responsibilities between layers, low coupling, and easy extension and maintenance.

1.1.1 Layered Architecture Description

The architecture of the LMES process log system consists of five core layers, forming a complete closed loop from business operations to final log storage:

Architecture Layer	Core Component	Main Responsibility	Implementation Method
Business Layer	BusinessActivity and other business components	Trigger log recording requirements, provide business context information	Call via Flow.Logger interface in business code
Log Service Layer	Flow.Logger Interface	Encapsulate log recording logic, unify log format	Provide standardized log recording methods (Info/Warn/Error, etc.)
Log Framework Layer	NLog Logger	Implement log routing, filtering, and formatting	Distribute logs to different targets according to configuration
Configuration Layer	NLog-Flow.config configuration file	Define log output rules, paths, and formats	XML format configuration file, supports dynamic adjustment
Storage Layer	File System	Organize and store log files by process and station	Layered directory structure, supports log file rotation

1.1.2 Log Flow Process

The complete flow process of logs from generation to storage is as follows:

1. During the execution of Activity components in the business layer, various business events and status information are recorded through the Flow.Logger interface.
2. The log service layer receives these log requests and performs standardized processing according to the predefined format.
3. The processed logs are passed to the NLog framework layer for filtering and routing according to configured rules.
4. The NLog framework distributes logs to corresponding storage targets according to the configuration in NLog-Flow.config.
5. Finally, logs are stored as files in the file system, classified and managed by process and station.

This layered design allows the LMES process log system to meet the flexibility of business needs while ensuring the standardization and traceability of log management.

1.2 Log Classification System

1.2.1 Category Naming Convention

CategoryName: The category name of the message generated by the logger.

Log Category	CategoryName Pattern	Description	Example
Process Log	CMS.Plugin.Processflow.ProcessName_StationName	Process execution log	CMS.Plugin.Processflow.PLA_Process1_Station1
Trace Log	CMS.Plugin.TraceLogger.ProcessName_StationName	Variable tracking related log	CMS.Plugin.TraceLogger.PLA_Process1_Station1

Default process log message category format: CMS.Plugin.Processflow.ProcessName_StationName. The default LMES host ProductionProcessProvider has been initialized by convention. If you want to customize it, you can modify it as shown below.


1.2.2 Log Level Definition

Level	Purpose	Usage Scenario
Trace	Detailed debug information	Variable tracking during development phase
Debug	Debug information	Process analysis during development phase
Info	General information	Normal process execution record
Warn	Warning information	Abnormal situation but does not affect the process
Error	Error information	Process execution failure
Fatal	Severe error	System-level failure

II. Log Storage Strategy

2.1 Storage Path Design

The LMES plugin will provide the NLog-Flow.config file by default, used to store process logs in folders by {ProcessName}_{StationName}.

logs root directory/
├── flow_logs/
    ├── Process1_Station1/
    │   ├── flowlog_2024-01-01.log
    │   ├── tracelog_2024-01-01.log
    │   └── warnlog_2024-01-01.log
    ├── Process1_Station2/
    │   ├── flowlog_2024-01-01.log
    │   ├── tracelog_2024-01-01.log
    │   └── warnlog_2024-01-01.log
    ├── Process2_Station1/
    │   ├── flowlog_2024-01-01.log
    │   ├── tracelog_2024-01-01.log
    │   └── warnlog_2024-01-01.log
    └── ...

After enabling the above configuration, the process log storage path is as shown below. Note: Other business logs will not be stored in folders by {Process}_{Station}. If you want to store them in this path, please refer to the first part to configure the log recorder.


2.2 File Rotation Rules

File Type	Max File Size	Max Archive Quantity	File Naming Pattern
Process Log	20MB	100	flowlog_shortdate.log
Warning Log	20MB	10	warnlog_date:yyyy-MM-dd.log
Trace Log	20MB	50	tracelog_shortdate.log

2.3 NLog Configuration Integration

Subsequent versions of the host will load this configuration file as an NLog extension configuration file into the host by default. If the currently used host does not have this configuration yet, you can manually modify the $(CMSRootPath)/host/NLog.config file and add the following code:

<include file="NLog-Extend.config" ignoreErrors="true" />


2.4 NLog Configuration File Details

2.4.1 Core Configuration Elements

The NLog-Flow.config ($(CMSRootPath)/plugins/cms.plugin.messuite/Configs/NLog-Flow.config file) contains the following key configurations:


Variable Definition:

• basedir: Log root directory, defaults to ../logs
• logDirectory: Process log directory, value is flow_logs
• flowLogDirectory: Dynamic process station path, generated by replacing logger name {Process}_{Station}
• traceLogDirectory: Trace log dynamic path {Process}_{Station}

Target Configuration:

• flowLogfile: Process log file target (Info level)
• warnLogfile: Warning log file target (Warn level)
• traceLogfile: Trace log file target
• consoleLog: Console output target (Used during development phase)

Rule Configuration:

• Trace Log: CMS.Plugin.TraceLogger.* → traceLogfile
• Process Warning: CMS.Plugin.Processflow.* (Warn level) → warnLogfile
• Process Log: CMS.Plugin.Processflow.* (Info level) → flowLogfile
• Console Output: CMS.Plugin.Processflow.* (Trace/Debug level) → consoleLog

2.4.2 Configuration File Inclusion Relationship

Environment	Configuration File Location	Inclusion Relationship
Host	$(CMSRootPath)/host/NLog.config	Includes NLog-Extend.config
Extension	NLog-Extend.config	Includes each plugin process configuration
LMES Plugin	plugins/cms.plugin.messuite/Configs/NLog-Flow.config	Independent configuration
WCS Plugin	plugins/cms.plugin.wcs/Configs/NLog-Flow.config	Independent configuration

III. Log Format Specification

3.1 Standard Format Definition

3.1.1 NLog Log Format

[${longdate}] [${level}] [${threadid}] [${logger}] ${message} ${exception}

Field Description:

• ${longdate}: Timestamp (Format: yyyy-MM-dd HH:mm:ss.ffff)
• ${level}: Log level (Trace/Debug/Info/Warn/Error/Fatal)
• ${threadid}: Thread ID
• ${logger}: Log recorder name (CategoryName)
• ${message}: Log message content
• ${exception}: Exception information (if any)

3.1.2 Message Content Format

${ProcessCode},${StationName}_${ProcessName},【Instance=${ProcessInstanceId}】->【${ExecutionStepName}】-> ${ProcessBusinessMessage}

Field Description:

• ProcessCode: Process definition code
• StationName: Current station name
• ProcessName: Process instance name
• ProcessInstanceId: Unique identifier of process instance
• ExecutionStepName: Currently executing Activity name
• ProcessBusinessMessage: Detailed business-related information

3.2 Format Example Comparison Table

Inbound Process Log Example

[2024-06-05 14:24:34.7023] [Info] [49] [CMS.Plugin.Processflow.PLA_Process1] 10001,PLA_GX1_Station1_NormalProcessFlow,【Instance=717568446925】->【Inbound Signal Listener】->Enter

The effect is as follows: (Normal Process Inbound Process Log)


IV. Log Interface Usage

4.1 IFlowLogger Interface Specification

For the output of process logs, LMES has encapsulated the corresponding log interface. Developers only need to obtain the interface in the corresponding business code and output business logs, without needing to splice fields corresponding to the process.

4.1.1 Core Method Definition

Method	Purpose	Parameter Description
LogMessage(string message, string stepName)	Record general info	message: Business message, stepName: Execution step name
LogWarningMessage(string message, string stepName)	Record warning info	Same as above
LogErrorMessage(string message, string stepName)	Record error info	Same as above
LogExceptionMessage(Exception ex, string stepName)	Record exception info	ex: Exception object, stepName: Execution step name

4.1.2 Usage

In Activity inheriting from ProcessBusinessActivity, you can directly use Flow.Logger for log recording:

// Example 1: Record general info
Flow.Logger.LogMessage($"Formula processing judgment, Work Order Code={currentOrder?.Code}, Product Model={currentProduct?.Model}", Name);

// Example 2: Record warning info
Flow.Logger.LogWarningMessage($"Material Barcode: {materialCode} is empty", Name);

// Example 3: Record error info
Flow.Logger.LogErrorMessage($"Formula processing judgment exception: {e.Message}", Name);

// Example 4: Record exception
Flow.Logger.LogExceptionMessage(ex, Name);


4.2 Activity Integration

4.2.1 Typical Usage Scenarios

1. Process Information Recording

Flow.Logger.LogMessage($"Formula application info: Product Model={formulaApplyModel.ProductModel}, Formula Code={formulaApplyModel.FormulaCode}, Formula Version={formulaApplyModel.FormulaVersionName}", Name);

2. Exception Situation Handling

try
{
    // Business Logic
}
catch (Exception e)
{
    canEnterStation = false;
    errorMsg = $"Operation exception: {e.Message}";
    Flow.Logger.LogErrorMessage(errorMsg, Name);
}

3. Debug Information Output

Flow.Logger.LogMessage($"Collected parameters: {string.Join(",", parameterCollectVarNames)}, 【TraceId={traceId}】", Name);

Log output effect is as follows:


V. Best Practices Guide

5.1 Log Recording Principles

1. Information Completeness: Ensure sufficient context information is included

   • Include key business data (Work Order Code, Product Model, Material Code, etc.)
   • Record operation results and status
   • For abnormal situations, record detailed error information

2. Format Consistency: Follow standard message format

   • Use unified field separator (,)
   • Use unified key-value separator (=)
   • TraceId uses unified format: 【TraceId={traceId}】

3. Performance Consideration: Avoid overly frequent log recording

   • Do not record large amounts of logs in loops
   • Reasonably set log levels
   • Avoid recording Trace/Debug level logs in production environment

4. Security: Avoid recording sensitive information

   • Do not record passwords, keys, and other sensitive data
   • Pay attention to personal information protection

5.2 Development Guidance

6.2.1 Recommended Log Recording Pattern

// Correct Example: Include key information and TraceId
Flow.Logger.LogMessage($"Operation description, key parameter={value}, 【TraceId={traceId}】", Name);

// Avoided Pattern: Missing key information
Flow.Logger.LogMessage("Operation completed", Name); // Not recommended

5.2.2 Exception Handling Pattern

try
{
    // Business Logic
}
catch (Exception e)
{
    canEnterStation = false;
    errorMsg = $"Operation exception: {e.Message}";
    Flow.Logger.LogErrorMessage(errorMsg, Name);
    // Or use
    Flow.Logger.LogExceptionMessage(e, Name);
}

5.2.3 TraceId Usage Specification

For operations requiring correlation analysis, TraceId should be used:

var traceId = Flow.Instance.GetTraceId();
Flow.Logger.LogMessage($"Operation description, 【TraceId={traceId}】", Name);
// Pass traceId when executing business operations
await service.WriteValueAsync(this, parameters, cancellationToken, retryCount, retryDelay, traceId);

5.3 Environment Configuration Suggestions

5.3.1 Development Environment Configuration

• Enable console output (Trace/Debug level)
• Detailed exception stack information
• Shorter log rotation cycle

5.3.2 Production Environment Configuration

• Disable console output
• Only record Info and above levels
• Reasonable file size and archive limits

5.4 O&M Monitoring Suggestions

1. Log File Monitoring: Monitor log file size and rotation status
2. Error Log Alert: Set alerts for Error level logs
3. Performance Metric Correlation: Combine with LmesMetrics for performance analysis
4. Regular Cleanup: Formulate log file cleanup strategy

VI. Performance Monitoring Integration

6.1 Log and Metric Association

The LMES process log system integrates with the performance monitoring system (LmesMetrics) to achieve unified management of log recording and performance metric collection:

6.2 TraceId Tracking Mechanism

TraceId is a key mechanism for problem tracking and performance analysis:

Usage Scenarios:

• Formula parameter distribution
• Variable value read/write operations
• Distributed operation tracking
• Performance bottleneck analysis

Example:

var traceId = Flow.Instance.GetTraceId();
Flow.Logger.LogMessage($"Collected parameters: {string.Join(",", parameterCollectVarNames)}, 【TraceId={traceId}】", Name);
await variableService.WriteValueAsync(this, parameters, cancellationToken, retryCount, retryDelay, traceId);

All relevant log records contain TraceId, facilitating problem tracking and performance analysis.

6.3 Performance Metric Types

Metric Type	Purpose	Example
Counter	Counter, used to count event occurrences	Inbound count, Outbound count
Histogram	Histogram, used for statistical value distribution	Process time distribution
Gauge	Gauge, used to record current value	Current inbound product count

VII. Fault Diagnosis Guide

7.1 Common Troubleshooting

7.1.1 Log files not separated by {Process}_{Station} folder

Reason Analysis:

• NLog-Extend.config not correctly included
• CategoryName set incorrectly, does not contain station name
• NLog-Flow.config configuration error

Solution:

1. Check if base NLog.config includes NLog-Extend.config
2. Verify CategoryName setting in ProductionProcessProvider, ensure format is {Process}_{Station}
3. Check logger name matching rules in NLog-Flow.config

7.1.2 Incorrect Log Format

Reason Analysis:

• Custom Logger does not follow IFlowLogger interface
• Message formatting logic error
• Flow.Logger interface not used

Solution:

1. Use Flow.Logger instead of custom Logger
2. Ensure correct stepName parameter is passed
3. Follow standard message format specification

7.1.3 Log File Not Generated

Reason Analysis:

• Log level set too high
• File path permission issue
• NLog configuration loading failed

Solution:

1. Check minlevel setting in NLog-Flow.config
2. Verify read/write permissions of log directory

7.2 Log Analysis Tools

7.2.1 Text Search

Use tools like grep for quick location:

# Search error logs
grep "\[Error\]" flowlog_2024-01-01.log

# Search specific product code
grep "SN001" flowlog_2024-01-01.log

# Search TraceId
grep "TraceId=12345" flowlog_2024-01-01.log

7.2.2 TraceId Tracking

Correlate related logs through TraceId:

# Extract TraceId
grep -o "TraceId=[0-9]*" flowlog_2024-01-01.log | sort | uniq

# View all logs for specific TraceId
grep "TraceId=12345" flowlog_2024-01-01.log

7.2.3 Timestamp Analysis

Analyze process execution time distribution:

# Extract timestamp
grep -o "\[20[0-9-: .]*\]" flowlog_2024-01-01.log

# Count log quantity per hour
grep -o "\[20[0-9-]* [0-9]*:" flowlog_2024-01-01.log | cut -d' ' -f2 | cut -d':' -f1 | sort | uniq -c

7.2.4 Error Statistics

Count frequency of various errors:

# Count error level logs
grep -c "\[Error\]" flowlog_2024-01-01.log
grep -c "\[Warn\]" flowlog_2024-01-01.log

# Count specific error types
grep "\[Error\]" flowlog_2024-01-01.log | grep -o "Exception.*" | sort | uniq -c | sort -rn

7.3 Fault Handling Process

7.3.1 Log Troubleshooting Steps

1. Determine Problem Scope

   • Confirm the process and station where the problem occurred
   • Determine the time the problem occurred
   • Confirm impact scope

2. Find Relevant Logs

   • Locate log folder based on {Process}_{Station}
   • Narrow down scope based on timestamp
   • Search keywords (Product Code, Work Order Code, etc.)

3. Analyze Log Content

   • View error log details
   • Analyze exception stack information
   • Track TraceId related operations

4. Locate Problem Cause

   • Compare normal and abnormal logs
   • Check configuration and parameters
   • Verify business logic

5. Solve and Verify

   • Implement fix plan
   • Verify if problem is solved
   • Record solution

7.4 Preventive Monitoring

1. Real-time Monitoring

   • Monitor error log growth trend
   • Set error rate threshold alert
   • Monitor log file growth speed

2. Regular Analysis

   • Daily log summary analysis
   • Weekly trend analysis report
   • Monthly problem summary and optimization

3. Capacity Planning

   • Estimate log growth trend
   • Reasonably configure storage space
   • Formulate cleanup strategy

VIII. Summary

This specification document provides comprehensive guidance for LMES process logs, from architecture design to practical application, from development specifications to O&M monitoring. By following this specification, you can:

1. Improve Log Quality: Unified format and complete information facilitate analysis
2. Accelerate Problem Positioning: TraceId and structured logs improve troubleshooting efficiency
3. Ensure System Stability: Discover and solve problems in time
4. Optimize System Performance: Continuous optimization combined with performance metrics

It is recommended that developers strictly follow this specification in daily development, and O&M personnel regularly check log quality and system running status to continuously improve and perfect the log system.