APPX Monitor

Overview

Release 5.0.0 and higher of the APPX Utility includes a new utility in System Administration which can be used to monitor all APPX sessions.  This utility provides APPX System Administrators with visibility of which processes are being run by the various APPX users as well as status information for each session.

How to Enable the APPX Monitor Feature

To enable the APPX Monitor feature, each APPX session that is to be monitored must have two environment variables set at the time that the session is initiated.  Any APPX session does not have these two environment variables set will simply not be displayed on the APPX Monitor display. The APPX License Server session should also have these two environment variables set.

APPX_MONITOR_KEY=<key_number>

APPX_MONITOR_SLOTS=<session_count>

The easiest way be ensure that these environment variables are set for all APPX sessions including the APPX License Server session is to define them in the appx.env configuration file. Simply add the two environment variables to the appx.env file with appropriate values.  The example below assignes a key value of 1234 for the shared memory segment and specifies that the shared memory segment should be large enough to hold the status information for 100 APPX sessions.

#=============================================================================
# APPX Monitor configuration
#=============================================================================
APPX_MONITOR_KEY=1234
APPX_MONITOR_SLOTS=100

APPX_MONITOR_KEY Environment Variable

The APPX_MONITOR_KEY environment variable defines the IPC (interprocess communication) key of the shared memory segment in which the APPX session information is stored.  The <key_number> that you assign to the APPX_MONITOR_KEY environment variable must be a number and must not already be in use by any other shared memory segments that may exist on your system.  You should list the shared memory segments for your system and choose a key value that is not already in use.

Each APPX session updates the shared memory segment with information relating to which APPX process is being run and the status of the process. The APPX Monitor reads the shared memory segment to obtain and display the information for each APPX session. 

APPX_MONITOR_SLOTS Environment Variable

The APPX_MONITOR_SLOTS environment variable defines how many "slots" should be allocated in the shared memory segment for the APPX Monitor. The minimum number of slots should correspond to the maximum number of APPX sessions ever run on your APPX server.  For example, if you have 30 licensed users and each user runs no more than two sessions at a time, you should specify a value of at least 61. The following formula can be used to ensure that you allocate a sufficient number of slots:

<session_count> = <licensed_users> * 4 + 1

When the APPX Monitor's shared memory segment is created, it will always be created with the number of slots being a multiple of eight.  If you specify a <session_count> that is not a multiple of eight, the number of slots allocated will be increased to the next multiple of eight.

How to Run the APPX Monitor

The APPX Monitor status display process can be run from within APPX System Administration.

System Administration --> System Setup --> Monitor APPX Sessions

If the APPX Monitor feature has been properly configured, the APPX Monitor process should display a screen similar to the one below.

The Role of the APPX License Server

The license server cleans up orphaned entries in the APPX Monitor's shared memory segment.  When an APPX session terminates normally, it removes its entry from the APPX Monitor's shared memory segment.  However, if an APPX session crashes or is otherwise abnormally terminated, the session's entry will be left in the APPX Monitor's shared memory segment.  The APPX License Server session periodically identifies and removes entries for APPX sessions that are no longer running from the APPX Monitor shared memory segment. 

Managing Share Memory Segments

Under ideal conditions, once you have configured the APPX Monitor feature, there should be no need to be concerned with shared memory segments.  However, in the event that you do encounter a problem that is somehow related to the shared memory segment there a couple of things you can do.  First, you can reboot your server.  Rebooting the server removes all shared memory segments and will clear any problems that might be related to a shared memory segment.  However, if you don't want to reboot your server, there are two commands on Unix/Linux systems which you may find useful for managing shared memory segments.  One command will list the shared memory segments that exist on your server.  The other command can be used to remove a shared memory segment.

Command to List Shared Memory Segments

The following command can be used on a Unix/Linux system to list the shared memory segments that have been created on the server: 

ipcs -m

In the example below, the shared memory segment with an owner of "appx" is the shared memory segment for the APPX Monitor.  The key value can be used to identify the shared memory segment of the APPX Monitor.  In this case, the key value of 0x000004d2 is the hexadecimal equivalent of a decimal value of 1234 which is the value that we assigned to the APPX_MONITOR_KEY environment variable.

Command to Remove Shared Memory Segment

In the event that your APPX server has a problem related to the APPX Monitor's shared memory segment, you can remove the APPX Monitor's shared memory segment. The following command can be used on a Unix/Linux system to remove a shared memory segment.

ipcrm -m <shmid>

The appropriate value for <shmid> can be determined by listing the shared memory segments on your server to identify the APPX Monitor's shared memory segment.

Concerns

1. When we ask for an ID to a shared memory segment and a segment already exists with that key we don't do any checking to see that APPX created this memory and we aren't colliding with some other non-APPX process. I think our shared memory header record should include some known identifier at the beginning that we can inspect.
2. When we ask for an ID to a shared memory segment and a segment already exists with that key we don't use the number of slots out of the shared memory header. We use the number of slots defined in the APPX environment variable. Those values may not be the same. If the value from the environment variable is smaller we limit our access to the memory slots, if larger then our attach to the memory fails completely and silently. It's up the an administrator to manually locate and remove the shared memory segment, run APPX to create the new larger segment, then either wait for all sessions to log out and back in or force everyone to log out and back in to get attached to the new memory segment. Seems like we could put a stale indicator byte in our shared memory header block and when we need to extend the shared memory, set the stale bit, copy the contents of the old segment, delete it, allocate a larger segment, copy the data into it, and continue running. As processes go to update their slot if the header stale bit is set just un-attach/reattach to get to the current segment.
3. It looks like the SessionID used to find a memory slot is based on the row-id of the USAGE file record. Doesn't this create a problem if more than one APPX install path on a system share the same shared memory segment? Since this does not seem like a good thing, each copy can already be configured to use a different key to isolate each to it's own shared memory segment. But for safety I think we should have some information in our shared memory header that tells us the instance of APPX this segment is tied to. That way we would could catch attempts to cross over and use the wrong shared memory key.
4. Synchronized access to our shared memory segment(s) is controlled by locking either a byte in a dummy file on unix or a mutex on windows. In either case the name to the file or mutex is a fixed name. If we have several keyed memory segments in use for different installs of APPX they are all sharing the same synchronization lock. I think this locking should be based on the path to the install.
5. The license server has code that will clean up the shared memory slots left in use by orphan processes. However this will only work if the license server is running on the system that also contains the shared memory slot. This model does not work on Novell or other shared usage file installs. Not sure what to do about this.
6. I think the whole process is too manual. Coming up with key values and setting both Keys and Sizes as environment variable to activate the functionality seems wrong. Why don't we just have it on by default with a way to turn it off. Why don't we just determine a key based on the APPXPATH using the ftok() function, determine a nice initial size based on the license count and re-allocate as mentioned in item 2 above if we need more slots.
7. Problem: If you run a process like Vendors from the 1EX Main Menu and it blows up with a File Can't Be Accessed error, upon returning from the error screen to the menu the monitor still says it's running Vendors Input.
   

Test Results, Comments, Open Issues

The notes above also stated, "If APPX_MONITOR_SLOTS is too small, nothing tragic happens - new processes just won't update their status information in the monitor table." That sounds like it should be tested, so we killed all sessions, including the license server and appxd, edited the .env file to set the value to 2, and restarted APPX. We used the first login to run the APPX Monitor, and it properly showed that session plus the license server. Using Alt-F1, we started a second session, which also showed up on the Monitor display, a bit of a surprise there. In fact, subsequent sessions also showed up, until we had six sessions running. The seventh did not appear on the table. Closing each of those in the table worked normally, and closing the session(s) that were not shown did not cause any problems, although we did notice that the entry for the license server disappeared temporarily one time. It might be worth testing this with a slight variation - logging in as different users, to check to see if the user count is somehow a factor. But the fact that it allowed six sessions (not four) seems to make that unlikely. Follow-up note: Pete looked into the code to determine why we more than two sessions were being logged, and discovered that the actual number of slots is the next higher multiple of 8 from the value entered.

One facet that is not thoroughly tested is what the APPX Monitor will show for background tasks. It appears that an additional session will be displayed, but the background session we were testing with died too quickly to see what was really happening. This should be investigated further.

Limitations:

1. A single-user license has no license server so there is no way for the APPX Monitor to remove crashed sessions from the shared memory segment.  On the other hand, do you really need the APPX Monitor on a single-user system?

Suggested Enhancements:

1. The APPX Monitor status display needs an option to allow a System Administrator to kill an APPX session.
2. The APPX Monitor status display needs a drill down option to show more information for a specific session.
3. It would be nice if the APPX Monitor showed the process ID as well as the User ID.
4. It would be nice if, during query processes, it would show the name of the process itself, rather than QSLCT and QSORT.
5. The configuration process is too "manual".  The APPX Monitor should always be on and should automatically configure itself.

Comments:

-- AlKalter - 04 Apr 2008

• icps Command on Linux: