APPX Audit Log Feature 

This page describes the APPX Audit Log feature and provides instructions for enabling the APPX Audit Log feature.

Overview

The APPX Audit Log feature creates an xml log of APPX file I/O activity.  The feature can be enabled for individual files or for groups of files using the FMS group feature of APPX.  The level of activity detail logged can be configured to optionally include read, write, rewrite, delete, create, and restructure events.

To enable the APPX Audit Log feature, you must configure and start an instance of the APPX Audit Log Service, you must define a Log Profile in APPX System Administration, and you must assign the Log Profile to an FMS Group or to individual files. The APPX Audit Log Service receives logging requests from the various APPX sessions and writes the audit data to the log file.

Installing the APPX Audit Log Manager Command ( appxAuditMgr)

The APPX Audit Log Manager ( appxAuditMgr) command is installed automatically when you install APPX on your system. The installer sets the necessary owner and group permissions for the appxAuditMgr command. So, there is nothing additional that you need to do to install the appxAuditMgr command. However, after you install APPX, you will need to run the appxAuditMgr command to configure and start an instance of the APPX Audit Log Service to enable logging of file audit information for APPX sessions.

The appxAuditMgr command is installed into the "tools" subdirectory of the directory where you installed APPX. So, if you installed APPX in "/usr/local/appx", the full pathname of the appxAuditMgr command will be "/usr/local/appx/tools/appxAuditMgr".

The appxAuditMgr command must run with the permissions of the root user. Therefore, the owner of the appxAuditMgr command should be the root user and the SUID bit should be set so that the appxAuditMgr command can be run by users other than root but still be run with the permissions of the root user.

In the event that it is necessary to reset the permissions of the appxAuditMgr command, the following commands can be run by the root user to set the necessary owner and group permissions for the appxAuditMgr command.

cd /usr/local/appx/tools
chown root appxAuditMgr
chgrp appxgrp appxAuditMgr
chmod 4775 appxAuditMgr

You can check the permissions of the appxAuditMgr command by running the following command:

ls -l appxAuditMgr

The recommended permissions should be as follows:

-rwsrwxr-x 1 root appxgrp    636843 Jul 11 07:31 appxAuditMgr

Creating and Configuring the APPX Audit Log Service

On Unix/Linux systems, an instance of the APPX Audit Log Service is initially created, configured, and started by running the appxAuditMgr command with the -install option. At least one appropriately configured instance of the APPX Audit Log Service must be created, configured, and started before running an APPX session for which file I/O activity is to be logged. You may create, configure, and start as many different instances of the APPX Audit Log Service as you desire. However, each concurrently running instance must be configured to receive file I/O audit data on a different TCP/IP port.

Creating an Audit Log Service

Before file I/O activity can be logged, at least one instance of an APPX Connection Service must be configured and started.

The -install option of the appxAuditMgr command is used to initially create, configure, and start an instance of the APPX Audit Log Service. The following steps are performed:

1. A configuration file (ini) is created
2. An environment file (env) is created
3. A service is created, including required init files and links. ( On Red Hat, /etc/init.d/ )
4. The service is started

For compete information on using the -install option of the appxAuditMgr command, please refer to the usage section of this page.

Service Name 

Each instance of an APPX Audit Log Service must have a unique name. When creating an instance of a service, the -name option may be used to specify the name that you want the service to have. If you do not specify a name, a name will be assigned for you for example, appxd-8060.

Service Type

When creating an instance of an APPX Audit Log Service, the -ServiceType option must be specified.  The value of this option must be "logmonitor".

TCP/IP Port Number

When creating an instance of an APPX Audit Log Service, the -SockPort option must be used to specify the TCP/IP port number on which the service is to listen for audit logging requests. Any available TCP/IP port number may be specified when installing an instance of the APPX Audit Log Service. However, as a matter of convention, most APPX administrators configure the APPX Audit Log Service to listen for connections on port 8070. If additional instances of the APPX Audit Log Service are configured, each instance is typically assigned the next available port number after 8070.

Changing an Audit Log Service

Two methods are available for modifying an existing instance of an APPX Audit Log Service.

Method 1 - The APPX Audit Log Manager Command (appxAuditMgr)

The -modify command and the -replace command of the appxAuditMgr tool can be used to modify or replace a previously configured instance of the APPX Audit Log Service. These options update the existing APPX Audit Log Service configuration files (ini and env) with the options specified. If you use this technique, the service will be automatically restarted for you, using the new settings. Note that when specifying options on the command line, you must prefix them with a dash.

Method 2 - Text Editor

A text editor can be used to directly edit the APPX Audit Log Service configuration files (ini and env). The configuration files include comments to help you make the desired changes. If you use this method to modify an existing configuration, you should exercise care to ensure that the syntax is correct. The preferred method for modifying an APPX Audit Log Service configuration is with Method 1 above.

Managing an Audit Log Service

Two methods are available for managing an existing instance of the APPX Audit Log Service.

Method 1 - appxAuditMgr command

The appxAuditMgr command can be used to manage an instance of the APPX Audit Log Service. The appxAuditMgr command can be used to start, stop, restart, or display the status of an instance of an APPX Audit Log Service.

Method 2 - O/S Services

Your operating system includes commands or programs that can be used to manage services. APPX Audit Log Services can be managed with these tools. The actual commands and programs vary depending on your operating system. Red Hat uses the command line tool service

.

[root@tubes tools]# *service appxd-8060 status*
Warning - the engine that you named has the setuid bit enabled,
          you may not want that bit set for the authentication
          method that you have chosen (OS-User)
     To turn off the setuid bit, chmod u-s ../appx
up and running (process 13893 servicing port 8060)
[root@tubes tools]# 

Notes

For example, here is a command to create a log monitor:

./appxAuditMgr -install -serviceType=logmonitor -name=myLogMonitor -port=8064

Port number has to be different from the port number you are using for your users to login.

This port is not a listener, so you can not login directly to that port.

Once you have created this new Service Type, it will create .ini and .env files for you. In our example, appxAuditMgr will create myLogMonitor.ini and myLogMonitor.env files in the ./services directory. You can change the name of the log file (which defaults to /tmp/appxlog%N.xml) by setting the LogNamePattern in the myLogMonitor.ini file. You can also change the LogRotationInterval and LogRotationSize.

Since we rotate audit logs, you specify a LogNamePattern instead of just a filename. The pattern can include %N (which is translated to a monotonically increasing counter: 0, 1, 2, ..) or any of the date/time format specs. supported by the strftime() function (see 'man strftime' for a list of the patterns). For example, a LogNamePattern of '/tmp/appx_%F.xml' would generate names like:

/tmp/appx_2007-01-31.xml

/tmp/appx_2007-02-28.xml

The default pattern (as reflected in the serviceName. ini file) is:

LogNamePattern = /tmp/appxlog%N.xml

Each time the log monitor rotates to a new log file, it replaces %N with the next number in sequence (it was always starting at 0). You can use other specifiers in the LogNamePattern too, for example, "/tmp/appx-%D-%B-%Y" would result in file names like:

/tmp/appx-11-Jun-08

/tmp/appx-12-Jun-08

...

If you restart the log monitor on 11-Jun-08 (and your LogNamePattern specifies %D-%B-%Y), any existing log file with that name would be replaced. Of course, you can include time components in the LogNamePattern to avoid that problem (or add %N to the pattern to include a sequence number). Here's the complete list of valid specifiers (from the strftime man page):

%a The abbreviated weekday name according to the current locale.
%A The full weekday name according to the current locale.
%b The abbreviated month name according to the current locale.
%B The full month name according to the current locale.
%c The preferred date and time representation for the current locale.
%C The century number (year/100) as a 2-digit integer. (SU)
%d The day of the month as a decimal number (range 01 to 31).
%D Equivalent to %m/%d/%y. (Yecch -- for Americans only. Americans should note that in other coun-
tries %d/%m/%y is rather common. This means that in international context this format is ambiguous and should not be used.) (SU)
%e Like %d, the day of the month as a decimal number, but a leading zero is replaced by a space. (SU)
%E Modifier: use alternative format, see below. (SU)
%F Equivalent to %Y-%m-%d (the ISO 8601 date format). (C99)
%G The ISO 8601 year with century as a decimal number. The 4-digit year corresponding to the ISO week number (see %V). This has the same format and value as %y, except that if the ISO week number belongs to the previous or next year, that year is used instead. (TZ)
%g Like %G, but without century, that is, with a 2-digit year (00-99). (TZ)
%h Equivalent to %b. (SU)
%H The hour as a decimal number using a 24-hour clock (range 00 to 23).
%I The hour as a decimal number using a 12-hour clock (range 01 to 12).
%j The day of the year as a decimal number (range 001 to 366).
%k The hour (24-hour clock) as a decimal number (range 0 to 23); single digits are preceded by a blank. (See also %H.) (TZ)
%l The hour (12-hour clock) as a decimal number (range 1 to 12); single digits are preceded by a blank. (See also %I.) (TZ)
%m The month as a decimal number (range 01 to 12).
%M The minute as a decimal number (range 00 to 59).
%n A newline character. (SU)
%O Modifier: use alternative format, see below. (SU)
%p Either `AM' or `PM' according to the given time value, or the corresponding strings for the current locale. Noon is treated as `pm' and midnight as `am'.
%P Like %p but in lowercase: `am' or `pm' or a corresponding string for the current locale. (GNU)
%r The time in a.m. or p.m. notation. In the POSIX locale this is equivalent to `%I:%M:%S %p'. (SU)
%R The time in 24-hour notation (%H:%M). (SU) For a version including the seconds, see %T below.
%s The number of seconds since the Epoch, that is, since 1970-01-01 00:00:00 UTC. (TZ)
%S The second as a decimal number (range 00 to 60). (The range is up to 60 to allow for occasional leap seconds.)
%t A tab character. (SU) %T The time in 24-hour notation (%H:%M:%S). (SU)
%u The day of the week as a decimal, range 1 to 7, Monday being 1. See also %w. (SU)
%U The week number of the current year as a decimal number, range 00 to 53, starting with the first Sunday as the first day of week 01. See also %V and %W.
%V The ISO 8601:1988 week number of the current year as a decimal number, range 01 to 53, where week 1 is the first week that has at least 4 days in the current year, and with Monday as the first day of the week. See also %U and %W. (SU)
%w The day of the week as a decimal, range 0 to 6, Sunday being 0. See also %u.
%W The week number of the current year as a decimal number, range 00 to 53, starting with the first Monday as the first day of week 01.
%x The preferred date representation for the current locale without the time.
%X The preferred time representation for the current locale without the date.
%y The year as a decimal number without a century (range 00 to 99).
%Y The year as a decimal number including the century.
%z The time-zone as hour offset from GMT. Required to emit RFC 822-conformant dates (using "%a, %d %b %Y %H:%M:%S %z"). (GNU)
%Z The time zone or name or abbreviation.
%+ The date and time in date(1) format. (TZ) (Not supported in glibc2.)
%% A literal '%' character. 

After you have modified .ini file, you need to stop and re-start that service. To stop service, assuming you are in ./services directory of your Appx installation simply type:

./appxAuditMgr -stop -name=myLogMonitor

./appxAuditMgr -start -name=myLogMonitor

If you stop/start service as a root, make sure you give it a fully qualified path.

./appxAuditMgr -stop -name=/usr/local/appx/services/myLogMonitor

./appxAuditMgr -start -name=/usr/local/appx/services/myLogMonitor

Please note that -name parameter is required.

Before data can be written to the log file in XML format, you need to define a Log Profile for the monitor.

To define FMS group, go to System Administration, Configuration, Log Profile press F9 to add a new profile. You can name it anything you want. For server name you must give it your server name:port number that you created earlier with appxAuditMgr:

@0 1 0" _moz-userdefined="">@1" _moz-userdefined="">@2 1 2" _moz-userdefined="">@3 21600 pixelWidth" _moz-userdefined="">@3 21600 pixelHeight" _moz-userdefined="">@0 0 1" _moz-userdefined="">servername:8064

Then click on Log File Parameters and make sure you check parameters that you wish to log:

Now you are ready to define a new FMS group for the monitor.

To add a new FMS group you need to go to System Administration, Configuration, File System Groups and press F9 to add a new one for the monitor.

Give it an FMS group of 1 and click on 'FMS group attributes' button. On that screen enter the name of your Log Profile in the 'Log Profile' field. Note that if you already have an FMS group that is used by the file(s) you wish to monitor, you can simply add the Log Profile name to the existing FMS group. If the FMS group refers to a RDBMS (such as Oracle, SQL Server, etc), then only changes made by Appx will be logged.

You can now assign this FMS group to the file(s) that you wish to monitor.

To close existent log file and rotate the log, you need issue the following command:

kill -s SIGUSR1 <PID>

where PID is a process ID of the audit log listener. Existent log will be closed and rotated to the next one.

The log file is generated in XML format. Why XML, and not Appx/IO? The biggest reason is the size of the log files. On large, active systems the number of events can exceed the maximum file size. XML files can easily be imported into a RDBMS, which does not have the same file size limitation.

You can view the log file with a browser, with XML Notepad, or you can download SQL Express (free) and write queries against your XML file. You can also use the various xlst processing programs to create queries (such as xsltproc or xalan from Apache).

Each Audit Log document contains one primary enclosing <events> node which contains one or more <event> nodes. Each <event> node contains an eventID node which has a <type> element that identifies the type of event that is being logged. A variety of additional nodes (shown in the table below) are included in each <event> node depending on the value of the <type> element. The following values may occur for the <type> element:

Sample scripts

The attached file 'structure.xslt' will extract any file create events from a log and produces an HTML table that shows the structure of each file that you create.

The attached file 'subrs.xslt' extracts all events where the process type is SUBR and produces an HTML summary table that shows the user id, date, time, event ID, application ID, version, and process name.

To use a stylesheet with xsltproc (the XSLT processor from libxml/xmlsoft):

$ xslt stylesheetFileName logFileName >output.html

for example:

$ xslt /stylesheets/structure.xslt /tmp/appxlog0.xml > /tmp/fileCreates.html

To use a stylesheet with xalan (the XSLT processor from Apache):

$ xalan -xsl stylesheetFileName <logFileName >output.html

for example:

$ xalan -xsl /stylesheets/structure.xslt < /tmp/appxlog0.xml > /tmp/fileCreates.html

Need examples of loading XML Data into an RDBMS

BUGS

No open bugs.

NOTES:

If log file is not closed/rotated properly, Internet Explorer will display your .xml file, but it will show an error at the end of the file saying "File not closed". Firefox, however, will throw an error and won't display file at all

In Linux, the way to close log and rotate it is to issue kill -s SIGUSR1 PID command.

How to close and rotate the log in Windows?

Comments:

Read what other users have said about this page or add your own comments.

-- SteveFrizzell - 20 Jun 2008