Introduction
Computer programs record various events during their execution for further analysis. ASP Log Service provides the facility to record information about these events. Any application / ASP component in the cluster can use the Log Service to record the information. This article explores the key components, usage models, and API functionalities of Log Service, highlighting its role in structured and scalable event logging.
What is the Log Service?
Log Service persists the information recorded by its clients so that it is available for consumption at later points in time also. The consumer may be an offline consumer, consuming the information at some later point in time, or an online consumer, consuming the information as soon as it is generated. Log Service does not interpret the information recorded by its clients. It treats the information as an octet stream and does not apply any semantic meaning to it.
All information related to one event is stored as one unit. This unit is known as Log Record. Log Records are grouped together based on certain client-defined themes. This group is called Log Stream. These Log Streams can be shared by various components of an application or of different applications. The theme and the users of a Log Stream are defined by the application. These Log Streams flow into Handlers. One of the handlers is the File handler, which persists the Log Records into the Log File. Another one is Online Viewer, which is used to view the Log Records as they are generated. Log Service expects an archiving utility to siphon off Log Records from the Log File to other storage units.
Log Service provides interfaces for creating a Log Stream, opening an existing Log Stream, recording an event into an opened Log Stream, and closing a Log Stream. Log service supports three different kinds of logging. Those are Binary logging, TLV(Tag-Length-Value) logging, and ASCII logging. By changing the msgId field of clLogWriteAsync() API, the application can choose their logging type. Further, it provides interfaces to change the properties of Log Streams and for setting filters on the Log Stream to filter events at the generation side. It also provides interfaces for registering interest in receiving certain Log Streams and interfaces to receive these streams.
Usage Model
The usage model of the Log Service is the Producer-Consumer model. Loggers are the producers of the Log Records, whereas Log Stream Handlers are the consumers of these records. Multiple Loggers may log into the same Log Stream. Similarly, the same record may be consumed by many Log Stream Handlers simultaneously. The usage model can also be thought of as a publisher-subscriber model because Loggers and Log Stream Handlers are unaware of each other, and each Log Stream Handler receives all the logged records. One of the Log Stream Handlers is the Log File Handler that persists the Log Records in the Log File. Certain other applications, like the offline Log Viewer and Archive, work with the Log File Handler.
Functionality
The basic purpose of the Log Service is to record information provided by the Logger for future use. This information is provided in the form of a Log Record. Log Records flow through a Log Stream and are persisted by the Log Service in Log File.
How to use APIs of Log Service
The producer of Log Record is known as Logger, which uses clLogWriteAsync()/ clLogVWriteAsync() API to pour a Log Record into a Log Stream. To use the services of Log Service, the Logger has to initialize the Log Service by invoking clLogInitialize(). clLogInitialize() returns a ClLogHandleT that can be used in the subsequent operations on the Log Service to identify the initialization. When this handle is no longer required, the association can be closed by invoking clLogFinalize(). Subsequent to clLogInitialize(), the Logger gains access to a Log Stream by invoking clLogStreamOpen(). clLogStreamOpen() returns a ClLogStreamHandleT that can be used in clLogWriteAsync() API to identify the Log Stream. When a Log Stream is no longer required by a Logger, it can close the stream by invoking clLogStreamClose(). Any Log Client can change the filter of a Log Stream by invoking clLogFilterSet().
Logger can also do ASCII logging by using the following two macros. They are clAppLog(), clLog(). Both of them do formatted ASCII logging. ASP components use clLog() macro to log their messages to default sys stream. clLog() macro can log only into sys stream. But clAppLog() macro can log into any kind of streams. But both macros do only ASCII logging.
If the application uses the clLog() macro, they can directly use the macro just after calling the clLogInitialize() API. It holds good for using clAppLog() macro, but only for default streams such as appstream and sysstream which is defined in configuration file. If the user wants to log into their own stream by clAppLog() macro, then they can do logging only after opening the stream by clLogStreamOpen() call.
There are three types of consumers of the Log Records – one who consumes the Log Records on-line through Log Service, second who consume the Log Records at will through Log Service and others who consume the records by directly reading the file. The first kind of consumers is those who are consuming the records online. These consumers are known as Log Stream Handlers, and they get records in a push mode. They are continuously waiting for the records and process them as soon as they get them. Examples of such consumers are Log File Handler and on-line viewer. These handlers initialize the Log Service by invoking clLogInitialize(), and then they register their interest in a particular Log Stream by invoking clLogHandlerRegister(). These handlers can discover the active streams either by listening to the stream creation event or by getting the list of active streams by invoking clLogStreamListGet(). After registration, the handler starts getting the records through a callback ClLogRecordDeliveryCallbackT, which is registered during clLogInitialize(). Handler can acknowledge the receipt of the records by invoking clLogHandlerRecordAck(). When the handler is no longer interested in receiving the Log Records for a particular Log Stream, it can deregister itself by invoking clLogHandlerDeregister(). The second kind of consumer is the off-line consumer. These consumers operate in the pull mode, and they explicitly ask the Log Service for more Log Records, if available. Another important difference with the first kind of consumers is that these work on Log File instead of Log Stream. Examples of such consumers are Log Reader and Archiver. These consumers also initialize the Log Service by invoking clLogInitialize(), and then they open a Log File by invoking clLogFileOpen(). When these consumers need more records, they invoke clLogFileRecordsGet(). These types of consumers invoke clLogFileMetaDataGet() to get the metadata of the Log Streams persisted in the Log file. When the consumer is done with the file, it can close the file by invoking clLogFileClose(). The third kind of consumer is the off-line consumer, who may not be present in the cluster itself. This kind of consumer is out of the scope of Log Service. These consumers directly operate on the Log File. An offline viewer is one such consumer.
Conclusion
Log Service provides a robust and flexible framework for logging, enabling seamless integration with various applications and consumers. Supporting multiple logging formats and offering APIs for creating, managing, and filtering log streams ensures efficient storage and retrieval of log records. Whether used in a real-time streaming model or for offline analysis, Log Service enhances observability and system diagnostics. Its structured approach to logging makes it a valuable tool for developers and system administrators aiming for better application monitoring and debugging. If you want to integrate Log Service in both your application and in the OpenClovis SAFPlus Platform.
Other support, please send email to support@openclovis.org.