APPX Audit Log Feature 

This page describes the APPX Audit Log feature and provides instructions for enabling the APPX Audit Log feature.

Overview

The APPX Audit Log feature creates an xml log of APPX file I/O activity.  The feature can be enabled for individual files or for groups of files using the FMS group feature of APPX.  The level of activity detail logged can be configured to optionally include read, write, rewrite, delete, create, and restructure events.

To enable the APPX Audit Log feature, you must configure and start an instance of the APPX Audit Log Service, you must define a Log Profile in APPX System Administration, and you must assign the Log Profile to an FMS Group or to individual files. The APPX Audit Log Service receives logging requests from the various APPX sessions and writes the audit data to the log file.

Installing the APPX Audit Log Manager Command ( appxAuditMgr)

The APPX Audit Log Manager ( appxAuditMgr) command is installed automatically when you install APPX on your system. The installer sets the necessary owner and group permissions for the appxAuditMgr command. So, there is nothing additional that you need to do to install the appxAuditMgr command. However, after you install APPX, you will need to run the appxAuditMgr command to configure and start an instance of the APPX Audit Log Service to enable logging of file audit information for APPX sessions.

The appxAuditMgr command is installed into the "tools" subdirectory of the directory where you installed APPX. So, if you installed APPX in "/usr/local/appx", the full pathname of the appxAuditMgr command will be "/usr/local/appx/tools/appxAuditMgr".

The appxAuditMgr command must run with the permissions of the root user. Therefore, the owner of the appxAuditMgr command should be the root user and the SUID bit should be set so that the appxAuditMgr command can be run by users other than root but still be run with the permissions of the root user.

In the event that it is necessary to reset the permissions of the appxAuditMgr command, the following commands can be run by the root user to set the necessary owner and group permissions for the appxAuditMgr command.

cd /usr/local/appx/tools
chown root appxAuditMgr
chgrp appxgrp appxAuditMgr
chmod 4775 appxAuditMgr

You can check the permissions of the appxAuditMgr command by running the following command:

ls -l appxAuditMgr

The recommended permissions should be as follows:

-rwsrwxr-x 1 root appxgrp    636843 Jul 11 07:31 appxAuditMgr

Creating and Configuring an Audit Log Service

On Unix/Linux systems, an instance of the APPX Audit Log Service is initially created, configured, and started by running the appxAuditMgr command with the -install option. At least one appropriately configured instance of the APPX Audit Log Service must be created, configured, and started before running an APPX session for which file I/O activity is to be logged. You may create, configure, and start as many different instances of the APPX Audit Log Service as you desire. However, each concurrently running instance must be configured to receive file I/O audit data on a different TCP/IP port.   Each instance of the Audit Log Service will log file I/O activity to a separate xml file.  By creating more than one instance of the APPX Audit Log Service, you can separate the file I/O activity that is logged into separate files by assigning different Log Profiles to FMS groups or individual files.

Creating an Audit Log Service

Before file I/O activity can be logged, at least one instance of an APPX Audit Log Service must be configured and started.

The -install option of the appxAuditMgr command is used to initially create, configure, and start an instance of the APPX Audit Log Service. The following steps are performed:

1. A configuration file (ini) is created
2. An environment file (env) is created
3. A service is created, including required init files and links. ( On Red Hat, /etc/init.d/ )
4. The service is started

When creating an instance of the APPX Audit Log Service, you must provide the Service Name, Service Type, and TCP/IP Port Number.  For compete information on using the -install option of the appxAuditMgr command, please refer to the usage section of this page.

Service Name 

Each instance of an APPX Audit Log Service must have a unique name. When creating an instance of a service, the -name option may be used to specify the name that you want the service to have. If you do not specify a name, a name will be assigned for you for example, appxd-8060.

Service Type

When creating an instance of an APPX Audit Log Service, the -ServiceType option must be specified.  The value of this option must be "logmonitor".

TCP/IP Port Number

When creating an instance of an APPX Audit Log Service, the -SockPort option must be used to specify the TCP/IP port number on which the service is to listen for audit logging requests. Any available TCP/IP port number may be specified when installing an instance of the APPX Audit Log Service. However, as a matter of convention, most APPX administrators configure the APPX Audit Log Service to listen for connections on port 8070. If additional instances of the APPX Audit Log Service are configured, each instance is typically assigned the next available port number after 8070.

Changing the Configuration of an Audit Log Service

Two methods are available for modifying an existing instance of an APPX Audit Log Service.

Method 1 - The APPX Audit Log Manager Command (appxAuditMgr)

The -modify command and the -replace command of the appxAuditMgr tool can be used to modify or replace a previously configured instance of the APPX Audit Log Service. These options update the existing APPX Audit Log Service configuration files (ini and env) with the options specified. If you use this technique, the service will be automatically restarted for you, using the new settings. Note that when specifying options on the command line, you must prefix them with a dash.

Method 2 - Text Editor

A text editor can be used to directly edit the APPX Audit Log Service configuration files (ini and env). The configuration files include comments to help you make the desired changes. If you use this method to modify an existing configuration, you should exercise care to ensure that the syntax is correct. The preferred method for modifying an APPX Audit Log Service configuration is with Method 1 above.  When you edit the configuration files for an instance of the APPX Audit Log Service with a text editor, you must restart the service before the changes take effect.

Managing an Audit Log Service

Two methods are available for managing an existing instance of the APPX Audit Log Service.

Method 1 - appxAuditMgr command

The appxAuditMgr command can be used to manage an instance of the APPX Audit Log Service. The appxAuditMgr command can be used to start, stop, restart, or display the status of an instance of an APPX Audit Log Service. The following example shows how to use the appxLogMgr command to check on the status of an APPX Audit Log Service.

[root@500test tools]# appxLogMgr -status logmon-8436 
up and running (process 2390 servicing port 8436)

Method 2 - O/S Services 

Your operating system includes commands or programs that can be used to manage services. APPX Audit Log Services can be managed with these tools. The actual commands and programs vary depending on your operating system. Red Hat uses the service command. The following example shows how to use the Red hat service command to check on the status of an APPX Audit Log Service.

[root@500test tools]# service logmon-8436 status 
up and running (process 2390 servicing port 8436)

Usage (appxAuditMgr)

Synopsis - Service Configuration

The appxAuditMgr service configuration commands are used to create, configure, and remove an instance of an APPX Audit Log Service.

appxAuditMgr -install -ServiceType=logmonitor -SockPort=[TCP-Port] [options]... [VARIABLE=VALUE]...

appxLoginMgr -modify -serviceName=SERVICENAME [options]... [VARIABLE=VALUE]...

appxLoginMgr -replace -serviceName=SERVICENAME -ServiceType=logmonitor [options]... [VARIABLE=VALUE]...

appxLoginMgr -remove -serviceName=SERVICENAME

Configuration - Commands

-install -name=SERVICENAME -ServiceType=logmonitor [options]... [VARIABLE=VALUE]...

-install -port=PORT -ServiceType=logmonitor [options]... [VARIABLE=VALUE]...

The -install command is used to configure a new instance of an APPX Audit Log Service. Either form of the install command may be used.

The first form of the -install command requires only that a service name and service type be specified. All other options are optional including the TCP/IP port. Any option not specified will be configured with an appropriate default value.

The second form of the -install command requires only that a TCP/IP port and service type be specified. All other options are optional including the ServiceName. Any option not specified will be configured with an appropriate default value.

Both forms of the -install command allow additional configuration options to be specified. The configuration options specified are stored in the service configuration file (ini).

Both forms of the -install command optionally allow values to be specified for environment variables. If specified, the environment variables and their values are stored in the environment configuration file (env). There is currently no need to set any environment variables.

In addition to creating the service configuration file and the environment configuration file, the -install command also creates an operating system service that will be automatically started when the computer system is started.

After creating the configuration files and the operating system service, the -install command starts the service.

-modify -name=SERVICENAME [options]... [VARIABLE=VALUE]...

The -modify command is used to modify the configuration of an existing Audit Log Service. The specified options will be updated in the service configuration files. Any options not specified will not be changed. After updating the configuration files, the -modify command restarts the service.

Note that when specifying variables on the command line, you must prefix them with a dash if you are referring to settings such as DisplayName, or without a dash if you are referring to environment variables.

Note that the -modify command updates the service configuration file and the environment configuration file by removing the old files and creating new files with the updated options and environment variables.  Any comments that may have been manually added to these configuration files are not preserved.

-replace -name=SERVICENAME -ServiceType=logmonitor [options]... [VARIABLE=VALUE]...

The -replace command is used to replace an existing Audit Log Service with a new Audit Log Service with the same name. The -replace command is effectively the same as a -remove command followed by an -install command. After updating the configuration files, the -replace command restarts the service. Note that when specifying variables on the command line, you must prefix them with a dash if you are referring to settings such as DisplayName, or without a dash if you are referring to environment variables.

-remove -name=SERVICENAME

The -remove command is used to remove an existing Audit Log Service. The -remove command will remove the configuration files (ini and env) and the corresponding operating system service. If the service is running when the -remove command is executed, the -remove command will first stop the service and then remove the service.

Configuration - Options

Options - General

-name, -ServiceName=SERVICENAME

The ServiceName uniquely identifies an APPX Audit Log Service. When creating (installing) an Audit Log Service, the SERVICENAME value may be any string value that conforms to the rules for valid filenames on your server. If this option is omitted when an Audit Log Service is being created, the Audit Log Service will be created with a default ServiceName based on the following template: "appxd-" followed by the specified TCP/IP port number, e.g "appxd-8060".

-DisplayName=DISPLAYNAME

The DisplayName is a "user-friendly" descriptive name for an Audit Log Service. The DISPLAYNAME value will appear in your system's Services control panel and will be displayed by the ps command. If you don't specify a DISPLAYNAME when an Audit Log Service is being created, the Audit Log Service will be created with a DISPLAYNAME based on the SERVICENAME.

-LogDirectory={/tmp, LOGDIR}

When the service is started, two log files are created in the LOGDIR directory - an Audit Log Service log file (.log) and a status file (.stat). Both log files have the same name as the ServiceName but one has a file extension of .log and the other has a file extension of .stat. If the LogDirectory option is not specified, the log files are created in the /tmp directory.

-ServiceType=logmonitor

The only valid value when configuring an Audit Log Service is "logmonitor". Note: This option must be specified.

-ServiceDisable={true, false}

This option can be used to temporarily disable or "turn off" the Audit Log Service. If set to true, the Audit Log Service will still run but it will not accept requests to log data from APPX sessions.

-initScript={lsb, RedHat}

Used with -install option to specify the type of operating system that the service script is to be created for. If this option is not specified, appxAuditMgr will determine which type of service script to install.

Options - Audit Log

-LogNamePattern={/tmp/logmon%N.xml, AUDITLOGPATHNAME}

The LogNamePattern identifies the path and the file name for the audit log files that will be created by the Audit Log Service.  The file name can include a pattern to ensure that each file created by the Audit Log Service will have a unique name.

-LogRotationInterval={86400, MAXSECONDS}

The LogRotationInterval identifies the maximum time in seconds that an Audit Log file should be used before being closed and a new audit log file is created.  The default value of 86000 is the number of seconds in one day so, by default, the Audit Log Service will create a new audit log file each day

-LogRotationSize={1G, MAXSIZE}

The LogRotationSize is the maximum size that an Audit Log file is allowed to be.  When an audit log file reaches the specified MAXSIZE, it will be closed and a new audit log file will be created.

Options - TCP/IP

-port, -SockPort={8060, PORT}

Configure the service to listen for connection requests on the specified TCP/IP PORT number. This option is required with the -install option. You may choose any TCP/IP PORT number that is not reserved or already being used on your system.

Notes

For example, here is a command to create a log monitor:

./appxAuditMgr -install -serviceType=logmonitor -name=myLogMonitor -port=8064

Port number has to be different from the port number you are using for your users to login.

This port is not a listener, so you can not login directly to that port.

Once you have created this new Service Type, it will create .ini and .env files for you. In our example, appxAuditMgr will create myLogMonitor.ini and myLogMonitor.env files in the ./services directory. You can change the name of the log file (which defaults to /tmp/appxlog%N.xml) by setting the LogNamePattern in the myLogMonitor.ini file. You can also change the LogRotationInterval and LogRotationSize.

Since we rotate audit logs, you specify a LogNamePattern instead of just a filename. The pattern can include %N (which is translated to a monotonically increasing counter: 0, 1, 2, ..) or any of the date/time format specs. supported by the strftime() function (see 'man strftime' for a list of the patterns). For example, a LogNamePattern of '/tmp/appx_%F.xml' would generate names like:

/tmp/appx_2007-01-31.xml

/tmp/appx_2007-02-28.xml

The default pattern (as reflected in the serviceName. ini file) is:

LogNamePattern = /tmp/appxlog%N.xml

Each time the log monitor rotates to a new log file, it replaces %N with the next number in sequence (it was always starting at 0). You can use other specifiers in the LogNamePattern too, for example, "/tmp/appx-%D-%B-%Y" would result in file names like:

/tmp/appx-11-Jun-08

/tmp/appx-12-Jun-08

...

If you restart the log monitor on 11-Jun-08 (and your LogNamePattern specifies %D-%B-%Y), any existing log file with that name would be replaced. Of course, you can include time components in the LogNamePattern to avoid that problem (or add %N to the pattern to include a sequence number). Here's the complete list of valid specifiers (from the strftime man page):

%a The abbreviated weekday name according to the current locale.
%A The full weekday name according to the current locale.
%b The abbreviated month name according to the current locale.
%B The full month name according to the current locale.
%c The preferred date and time representation for the current locale.
%C The century number (year/100) as a 2-digit integer. (SU)
%d The day of the month as a decimal number (range 01 to 31).
%D Equivalent to %m/%d/%y. (Yecch -- for Americans only. Americans should note that in other coun-
tries %d/%m/%y is rather common. This means that in international context this format is ambiguous and should not be used.) (SU)
%e Like %d, the day of the month as a decimal number, but a leading zero is replaced by a space. (SU)
%E Modifier: use alternative format, see below. (SU)
%F Equivalent to %Y-%m-%d (the ISO 8601 date format). (C99)
%G The ISO 8601 year with century as a decimal number. The 4-digit year corresponding to the ISO week number (see %V). This has the same format and value as %y, except that if the ISO week number belongs to the previous or next year, that year is used instead. (TZ)
%g Like %G, but without century, that is, with a 2-digit year (00-99). (TZ)
%h Equivalent to %b. (SU)
%H The hour as a decimal number using a 24-hour clock (range 00 to 23).
%I The hour as a decimal number using a 12-hour clock (range 01 to 12).
%j The day of the year as a decimal number (range 001 to 366).
%k The hour (24-hour clock) as a decimal number (range 0 to 23); single digits are preceded by a blank. (See also %H.) (TZ)
%l The hour (12-hour clock) as a decimal number (range 1 to 12); single digits are preceded by a blank. (See also %I.) (TZ)
%m The month as a decimal number (range 01 to 12).
%M The minute as a decimal number (range 00 to 59).
%n A newline character. (SU)
%O Modifier: use alternative format, see below. (SU)
%p Either `AM' or `PM' according to the given time value, or the corresponding strings for the current locale. Noon is treated as `pm' and midnight as `am'.
%P Like %p but in lowercase: `am' or `pm' or a corresponding string for the current locale. (GNU)
%r The time in a.m. or p.m. notation. In the POSIX locale this is equivalent to `%I:%M:%S %p'. (SU)
%R The time in 24-hour notation (%H:%M). (SU) For a version including the seconds, see %T below.
%s The number of seconds since the Epoch, that is, since 1970-01-01 00:00:00 UTC. (TZ)
%S The second as a decimal number (range 00 to 60). (The range is up to 60 to allow for occasional leap seconds.)
%t A tab character. (SU) %T The time in 24-hour notation (%H:%M:%S). (SU)
%u The day of the week as a decimal, range 1 to 7, Monday being 1. See also %w. (SU)
%U The week number of the current year as a decimal number, range 00 to 53, starting with the first Sunday as the first day of week 01. See also %V and %W.
%V The ISO 8601:1988 week number of the current year as a decimal number, range 01 to 53, where week 1 is the first week that has at least 4 days in the current year, and with Monday as the first day of the week. See also %U and %W. (SU)
%w The day of the week as a decimal, range 0 to 6, Sunday being 0. See also %u.
%W The week number of the current year as a decimal number, range 00 to 53, starting with the first Monday as the first day of week 01.
%x The preferred date representation for the current locale without the time.
%X The preferred time representation for the current locale without the date.
%y The year as a decimal number without a century (range 00 to 99).
%Y The year as a decimal number including the century.
%z The time-zone as hour offset from GMT. Required to emit RFC 822-conformant dates (using "%a, %d %b %Y %H:%M:%S %z"). (GNU)
%Z The time zone or name or abbreviation.
%+ The date and time in date(1) format. (TZ) (Not supported in glibc2.)
%% A literal '%' character. 

After you have modified .ini file, you need to stop and re-start that service. To stop service, assuming you are in ./services directory of your Appx installation simply type:

./appxAuditMgr -stop -name=myLogMonitor

./appxAuditMgr -start -name=myLogMonitor

If you stop/start service as a root, make sure you give it a fully qualified path.

./appxAuditMgr -stop -name=/usr/local/appx/services/myLogMonitor

./appxAuditMgr -start -name=/usr/local/appx/services/myLogMonitor

Please note that -name parameter is required.

Before data can be written to the log file in XML format, you need to define a Log Profile for the monitor.

To define FMS group, go to System Administration, Configuration, Log Profile press F9 to add a new profile. You can name it anything you want. For server name you must give it your server name:port number that you created earlier with appxAuditMgr:

@0 1 0" _moz-userdefined="">@1" _moz-userdefined="">@2 1 2" _moz-userdefined="">@3 21600 pixelWidth" _moz-userdefined="">@3 21600 pixelHeight" _moz-userdefined="">@0 0 1" _moz-userdefined="">servername:8064

Then click on Log File Parameters and make sure you check parameters that you wish to log:

Now you are ready to define a new FMS group for the monitor.

To add a new FMS group you need to go to System Administration, Configuration, File System Groups and press F9 to add a new one for the monitor.

Give it an FMS group of 1 and click on 'FMS group attributes' button. On that screen enter the name of your Log Profile in the 'Log Profile' field. Note that if you already have an FMS group that is used by the file(s) you wish to monitor, you can simply add the Log Profile name to the existing FMS group. If the FMS group refers to a RDBMS (such as Oracle, SQL Server, etc), then only changes made by Appx will be logged.

You can now assign this FMS group to the file(s) that you wish to monitor.

To close existent log file and rotate the log, you need issue the following command:

kill -s SIGUSR1 <PID>

where PID is a process ID of the audit log listener. Existent log will be closed and rotated to the next one.

The log file is generated in XML format. Why XML, and not Appx/IO? The biggest reason is the size of the log files. On large, active systems the number of events can exceed the maximum file size. XML files can easily be imported into a RDBMS, which does not have the same file size limitation.

You can view the log file with a browser, with XML Notepad, or you can download SQL Express (free) and write queries against your XML file. You can also use the various xlst processing programs to create queries (such as xsltproc or xalan from Apache).

Each Audit Log document contains one primary enclosing <events> node which contains one or more <event> nodes. Each <event> node contains an eventID node which has a <type> element that identifies the type of event that is being logged. A variety of additional nodes (shown in the table below) are included in each <event> node depending on the value of the <type> element. The following values may occur for the <type> element:

type	eventID	sessionID	fileID	appxProcessID	eventRecordID	eventData	Structure
Read	Yes	Yes	Yes	Yes	Indexed Files	Yes	No
Update	Yes	Yes	Yes	Yes	Indexed Files	Yes	No
Insert	Yes	Yes	Yes	Yes	Indexed Files	Yes	No
Delete	Yes	Yes	Yes	Yes	Indexed Files	Yes	No
Scratch	Yes	Yes	Yes	Yes	No
FileCreate	Yes	Yes	Yes	Yes	No		Yes
Restructure	Yes	Yes	Yes	Yes	No

node	node/element	value
<eventID>
	<type/>	see above table
	<timeStamp/>	ccyymmddhhmmsstt
</eventID>
<sessionID>
	<processID/>	9(6)
	<userID/>	X(3)
</sessionID>
<fileID>
	<application/>
	<database/>
	<structureDate/>
	<filename/>
</fileID>
<appxProcessID>
	<type/>
	<name/>
	<application/>
	<version/>
	<database/>
	<lastChange/>
</appxProcessID>
<eventRecordID>
	<keySegment>	0-16 instances
</eventRecordID>
<keySegment>
	<fieldName>
	<fieldValue>
</keySegment>
<eventData>
	<field>	0-n instances
</eventData>
<field>		read, insert, delete
	<fieldName>
	<occurrence>
	<fieldValue>
</fieldData>
<fieldData>		update
	<fieldName>
	<oldValue>
	<newValue>
</fieldData
<Structure>
	<Field>	1-n instances
</Structure>
<Field>
	<fieldName>
	<fieldType>
	<occurrences>
	<rawLength>
	<offset>
</Field>
<eventData>
	<RecordSizeChange>
	<DeletedElement>
</eventData>
<RecordSizeChange>
	<old>
	<new>
</RecordSizeChange>
<DeletedElement/>
	<fieldName/>
</DeletedElement>

Sample scripts

Here are some sample xslt processing commands that you can use to do inquiries against the raw XML data.

The attached file 'structure.xslt' will extract any file create events from a log and produces an HTML table that shows the structure of each file that you create.

The attached file 'subrs.xslt' extracts all events where the process type is SUBR and produces an HTML summary table that shows the user id, date, time, event ID, application ID, version, and process name.

To use a stylesheet with xsltproc (the XSLT processor from libxml/xmlsoft):

$ xslt stylesheetFileName logFileName >output.html

for example:

$ xslt /stylesheets/structure.xslt /tmp/appxlog0.xml > /tmp/fileCreates.html

To use a stylesheet with xalan (the XSLT processor from Apache):

$ xalan -xsl stylesheetFileName <logFileName >output.html

for example:

$ xalan -xsl /stylesheets/structure.xslt < /tmp/appxlog0.xml > /tmp/fileCreates.html

Need examples of loading XML Data into an RDBMS

BUGS

No open bugs.

NOTES:

If log file is not closed/rotated properly, Internet Explorer will display your .xml file, but it will show an error at the end of the file saying "File not closed". Firefox, however, will throw an error and won't display file at all

In Linux, the way to close log and rotate it is to issue kill -s SIGUSR1 PID command.

How to close and rotate the log in Windows?

Comments:

Read what other users have said about this page or add your own comments.

---

-- SteveFrizzell - 20 Jun 2008