Dynamic Linking and the CALL Statement

Calling External Functions from APPX ILF Code

It is possible for system integrators to develop and compile functions(typically in a 3GL like C) which can be called from APPX ILF code, to provide custom functionality which is not available in the APPX engine itself.

Because linking effectively makes your functions "part" of the APPX engine execution environment, no "safety nets" can be put in place to prevent coding errors in your functions (for example, improper use of pointers) from affecting the APPX run-time environment.

Merely that you've tested the function and seen that it does produce the results you've expected does not guarantee that the function is bug-free, because unintended (and undetected) side effects may be occurring. Therefore, writing functions to be called from APPX should only be attempted by those who have a solid level of understanding of, and skill in, the language they are using to write those functions, and the platform (UNIX, Windows, VMS) on which they will run. APPX Software cannot be responsible for errors occurring as a result of improperly written, compiled, or CALLed external functions, so please program with care.

Also see Calling Windows API Functions for more examples of interfacing with external libraries.

Since the address space of your extension is shared with APPX, an error in your code can affect the APPX engine even after your extension has completed execution

Dynamic Linking

With APPX version 3.4 & up, a new interface to CALLing external functions was introduced: Dynamic linking. This document describes how to take advantage of this new capability in APPX. (Previously, dynamic linking was available only on VMS, AIX and Windows platforms.)

With "static" linking, a developer had to create a new APPX executable, from the appx.a library and a customized call.c module. The better way is "dynamic" linking, which is a way of combining functions in an external function library, with the APPX executable at run-time, WITHOUT end-user rebuilding of the APPX executable. This makes it much easier for C developers to extend the functionality of APPX, without needing detailed knowledge of how APPX loads and runs user-provided functions, and without the error-prone activity of maintaining the function pointer table.

Using dynamic linking, if a C developer wants to make a new CALLable function available to APPX designers, all he/she needs to do is compile it (using platform- specific instructions listed below) then tell the APPX designers the name of the function and the full path to the binary library file containing it.

An example of how an APPX designer would call a dynamically-linked function called "func", which the C programmer compiled into the library "/appx/callee", and which takes one integer parameter, would be:

SET  --- AI            99
PASS --- AI            FIELD     SHARE? Y
CALL     /appx/callee,func RESIDENT? N END? N FAIL 0

When APPX runs this section of ILF code, it knows how to go out and find the UNIX file /appx/callee, then locate the function "func" which is stored inside it. If you are calling a function in a Windows DLL, you would use a Windows-style pathname for the library, like "F:\appx\callee,func" instead of "/appx/callee,func".

In addition to the benefit of it being easier to add new functions to APPX via "dynamic" linking than via the old static linking method, there is another significant benefit: When a new release of APPX is introduced, no re-linking is required. The new engine will dynamically locate and run the code, just as the old one did. (Of course, there may be details specific to your code that would cause it not to run under a new version of APPX, but we expect that most user-written functions would be unaffected by an upgrade.)

For the UNIX C Programmer

Compiling External Functions For Dynamic Linking

Here is a list of the C compiler flags you need on each platform in order to be able to compile a dynamically called function. Please note that customers may have different compilers and/or different compiler versions and the following information may not be appropriate for all customers. The following configurations have been tested by APPX, and may or may not work at any specific customer site.

(You are not required to use the C language to create a shared function library. Any language which supports the creation of shared libraries whose functions can be called by ordinary C programs on that platform should be able to be used. Note that if you choose another language, you will be on your own to figure out how to structure your code to accept parameters passed by APPX, and how to compile the resulting functions into a library.)

For purposes of an example, we will show the commands used to create a binary file containing the 'func' function, from the following C code:

$ more callee.c

#include <stdio.h> 

#ifdef __cplusplus 
extern "C" 
{ 
#endif 

void func( int x ) 
{ 
printf( "inside of func - called with %d\n", x ); 

} 

#ifdef __cplusplus 
} 
#endif

Before continuing, note that this really is all that you need in your C program in order to allow APPX to access 'func'. No external declarations of APPX structures or APPX typedefs, etc. Not even a 'main'. The #ifdef's above take care of telling a C++ compiler not to mangle the symbol names in this file as it is compiled. APPX expects C-style names, not those that have been altered in the usual C++ way.

You can store one, or many, functions in a dynamically-linked module. Whatever makes sense for your application. The following are the cc and ld lines required to compile the above file, callee.c, into a 'callee' binary, whose 'func' function can be access from APPX as described in the ILF code listed above. You can substitute any name for "callee", as desired.

Solaris, SCO, Unixware, DG/UX, NCR:

$ cc -o callee -G callee.c

Linux:

$ cc -o callee -shared callee.c

HP/UX:

$ cc -Aa +z -c callee.c 
$ ld -b -o callee callee.o

AIX:

$ cat callee.exp 
#! 
func 

$ cc -c callee.c 
$ cc -g -o callee -bE:callee.exp callee.o

Special AIX instructions: Under AIX, you must create an 'exports' file that lists the functions you want to call. My file was called 'callee.exp' and lists one function. The first line contains "#!" and the succeeding lines contain "functionname"<RETURN>. An example of what this file would look like when cat'd is below, as is the command line you would use to set up the exported functions in callee.

After issuing the proper compile instructions for your platform, you will end up with a shared library called 'callee' (or whatever other name you used instead of 'callee'). Again, it can be named whatever you like (the '-o callee' argument determines the name of the library).

For the Windows Programmer

Compiling External Functions For Dynamic Linking

On the Windows platform, APPX can dynamically invoke any function which exists in a standard Windows DLL, which has been created using the standard C (rather than C++) linking conventions. (In particular, this is not the same thing as an ActiveX DLL.) APPX expects any DLL-based functions to have been compiled to use the _stdcall (another term for this is WINAPI), rather than _cdecl, calling conventions. Refer to your compiler documentation, for how to ensure that your functions meet this requirement.

The rather complex subject of writing a Win32 DLL is beyond the scope of this document. Please consult a Win32 programming book for details of how to develop and compile a DLL that can be accessed from a Win32 C program. Standard techniques for writing DLL's apply.

As with the UNIX shared library creation process, the DLL can be written in any language that supports creation of the proper type of DLL. This document includes descriptions of how to map APPX data types to standard C data types. If you choose another implementation language, you will be on your own to figure out how to structure your code to accept parameters passed by APPX, and how to compile the resulting functions into a library.

Guidelines for Designing Functions to be Called by APPX

Start simple!

Before you write a function that does "real work", write a really simple one and verify that you can call it from APPX without encountering errors. We suggest starting out with a function with NO parameters, that does something simple like open a text file in a directory writable by any user, and write a text string like "Function OK!" into it. You can then rectly interfacing between the code and APPX.
Be careful if you write functions that expect parameters to be passed as a RECORD containing multiple data fields, rather than individually.

The struct you use in your C function should contain fields which exactly match (in record offset and size) those in the techdoc for the file whose record layout you are using as a template to pass the group of fields. This may differ between platforms, due to varying alignment requirements, so please thoroughly test your external functions which use structs as parameters when moving to a new implementation platform (machine).

Given this potential downside, why would someone want to do it anyway? APPX will accept a maximum of 16 arguments for a CALLed function (that is, 16 PASS statements). If you want to PASS more data items than that, you must PASS groups of fields, instead of PASSing all of them individually. Also, some developers may simply prefer to group related fields in a RECORD structure, as a matter of style. Your first step when writing a function which expects to receive a RECORD or RECORD pointer as a parameter should be to verify that your struct lines up with the data (perhaps via a series of fprintf's of each individual structure member, into a file you can inspect).

Doing this before you write any code which assumes the struct matches the PASSed data may save you LOTS of debugging time down the road.
Follow the usual guidelines for robust function design, such as not requiring parameters to be passed by reference unless the function must modify them, free'ing all memory you allocate in the function before you exit the function, and being very careful when using pointers.
If you have to modify an ALPHA field passed from APPX, so you pass it by reference, be warned that although it's declared as a "char *" in your function, the field IS NOT null terminated.

In order to maximize function reliability, we recommend NOT hard-coding the length of this field in your function, but rather, having the APPX designer PASS the length of that field, as another parameter, to the called function. This way, your function won't fail (or behave very unpredictably, which is worse!) when the length of that ALPHA field is changed in the APPX data dictionary.
All called functions are run in the same OS process and thread context as the APPX engine.

This makes use of UNIX exec() style OS functions a very bad idea (it'll end APPX immediately without allowing for any cleanup). Using system() instead is the easiest alternative, although the UNIX fork()/exec() should work as well if you really, really need the return code from whatever external command you're executing.
If you expect the function to ever be called within an APPX environment that uses the Presentation Client, Desktop Client or HTML Client front end, avoid any sort of terminal I/O. Error messages reported on standard output or error, as most UNIX programmers tend to do it, will never be seen by users running either of the graphical clients.

A good alternative might be passing error messages back to the caller, via a shared ALPHA work field, which can be displayed if the function's return code indicates that an error occurred. Reading from standard input also is not available. Please obtain any needed user-provided parameters via an APPX INPUT process, and pass them to the external function.
If you are creating a function to be run under APPX for Windows, and expect that function to ever be called within an APPX environment that uses the Presentation Client, Desktop Client or HTML Client front end, DO NOT use Message Boxes, Dialogs, or ANY other Windows GUI features.

If you do use any of these types of GUI elements, IF they display at all (which is very unlikely), they will display on the server, and hang the end-user's client. Another symptom of this could be a USER32.DLL error indicated on the server, with the end-user's client being hung, although USER32.DLL errors can occur for unrelated reasons as well. Still another symptom could be a hung client with no error message at all. (Hopefully this is convincing you that it is a very bad idea to attempt to use GUI features in a server-side application.)

If your extension uses third-party functions, like fax library routines, be aware that not all function libraries which run under Windows have been written to be server-aware. Some still engage in the practice of putting up their own message boxes, particularly to inform the user of error conditions. This is not appropriate for a server application. Test your external functions thoroughly, and if users experience hangs once the code using these functions is deployed, be sure to check to see if USER32.DLL errors or any unexpected windows are appearing on the server.
APPX does not reset the effective user ID prior to calling a dynamically-linked function.

Since APPX for Windows always runs with the user ID of the person who invoked APPX (either locally or via one of the GUI clients), this is relevant only to the UNIX platform.

Under APPX for UNIX (and Presentation Client and Desktop/HTML Client configurations where the APPX server runs on UNIX), functions will be invoked with an effective user ID of the owner of the 'APPX' executable, which is usually the user 'APPX'. Thus, watch for issues involving ownerships and permissions on text or other files created by dynamically-linked functions. They might not be what you would first expect. (Prior to a UNIX command being invoked by RUN, the effective user ID is reset to the logon user ID.)

Why doesn't a CALL reset it as well? Because there are many things one would want to do in a CALLed function, for which the effective user ID is irrelevant, so it's wasteful to always incur the overhead of resetting it to the logon user, then setting it back to 'APPX' after each CALL. Besides, sometimes the function developer might WANT the effective user ID to be 'APPX', so that the function could, for example, write to a shared log file owned by 'APPX'.

If you ABSOLUTELY, POSITIVELY MUST execute code in your function under the UNIX user's logon user ID, you can use the following sequence of ILF code, to set the effective user ID to the UNIX users' ID before calling your function, and to set it back to 'APPX' afterward.
```
CALL ,RT_SET_UID RESIDENT? Y END? N FAIL 0
CALL /appx/callee,func RESIDENT? N END? N FAIL 0 
CALL ,RT_SET_APPX RESIDENT? Y END? N FAIL 0 
```
Not including the ",rt_set_appx" line may APPEAR to work, at least for a while, but it MUST be there, and MUST be executed whether or not the CALL of your external function completed successfully.

Failure to include it WILL result in difficult-to-trace errors being encountered by the APPX engine, due to security issues. DO NOT use ",rt_set_uid" and ",rt_set_appx" unless you are completely certain that you understand the concepts of effective and real user ID's under UNIX, and you are certain that your function requires their use.
Remember that modifications to your APPX Data Dictionary which change the size or type (or, in the case of fields being passed as part of a RECORD, the offset) of a data field PASSed to an external function, may (and almost certainly will) require that the external function be updated to work with the modified field definition. APPX has no way to detect such situations and warn the APPX designer about them.

For the APPX Programmer

To CALL a dynamically linked function, you specify the name of the library file containing the function, followed by a comma ',', followed by the function name.

Note: The function name may have a "." or "_" prepended by the compiler on some platforms, which you will have to include in the function name, if it's there. In UNIX, you can find out for sure what the compiler did to the function names with a command like: nm /appx/callee | grep func "nm" lists several columns of information about the symbols in the function library. If you see ".func" or "_func" instead of "func", in the first column of the output, then use that variation on the function name instead.

For example, to CALL the 'func' function in the /appx/callee file from APPX, use code like:

SET  --- AI            99 
PASS --- AI            FIELD     SHARE? N 
CALL     /appx/callee,func RESIDENT? N END? N FAIL 0

This would pass the integer value 99 to the called function.

The APPX designer should PASS one argument for each parameter required by the C function he/she is CALLing. The PASS statements should correspond to the order in which parameters are listed in the C function declaration. (See the function's developer for details.)

Be aware that if you change the data type, size (or if PASSing an entire RECORD structure, the offset) of a field being PASSed, via the APPX Data Dictionary, any external function in which that field is used should be reviewed, and any necessary changes should be made to it, in order to accommodate the change in the field definition.

Specific attention should be paid to the SHARE? flag values set in PASS statements, because whether they should be set to Y or N depends on how the C function was declared. (The original developer of the C function you are CALLing is the best source for information on this.)

In general, if you want to pass a copy of the original APPX field, because you do not need to modify that field in the C function, you should set "SHARE?" to "N". If you want to pass the original APPX field itself, and DO expect to modify that field in the C function, you should set "SHARE?" to "Y". In all cases, you must coordinate with the writer of the C function, to make sure both of you agree on which parameters are shared and which are not. If your SHARE? values don't match what the C programmer expected, the function will not work properly. More details on what "SHARE?" means, for various data types, follow.

If your CALLed function expects a parameter which is a pointer to an unsigned integer or long, set "SHARE?" to "Y" when you PASS it. This will perform "pass by reference", sending the address of the variable to your function, instead of sending its value. One of the ways this is useful is that a CALLed function can modify that value, perhaps by doing an elaborate mathematical calculation, so that a new value is reflected in that variable when you return to APPX.

If your CALLed function expects a parameter which is an actual unsigned integer, long or short, set "SHARE?" to "N" when you PASS it.

This will perform "pass by value", sending the actual variable value rather than its address. You can be confident that, unless the function you are calling has a bug that results in unpredictable memory over-writes, the value of the variable in APPX will not change, if you pass it "SHARE? N". (However, the function must be written to expect "SHARE? N" for that parameter. You cannot just arbitrarily change the value of the SHARE? flag and expect the function to work properly.) The field that you PASS should be a be a 4-byte binary field for "int" and "long" C data types, and a 2-byte binary field for the "short" data type.

If your CALLed function expects a parameter which is a "char *" (that is, a C string), and it DOES NOT want to modify the original APPX alpha or text field's value, set "SHARE?" to "N" when you PASS it. This will cause a pointer to a null-terminated copy of that APPX field to be passed to the CALLed function. "Null-terminated" means that a hex '00' character is appended to the end of the copy of the field. This is the C convention for working with character strings.

If your CALLed function expects a parameter which is a "char " (that is, a C string), and it DOES want to modify the original APPX alpha or text field's value, set "SHARE?" to "Y" when you PASS it. This will cause a pointer to the actual APPX field to be passed to the CALLed function. IT WILL NOT BE NULL-TERMINATED, so the writer of the C function must be careful when working with it.

If you PASS an APPX FILE, what the C program will receive is the full-path-name of the file, represented as a C character string. It does not receive any sort of open file pointer, record number, or record contents.

You can also PASS packed-decimal fields and date fields, but it is up to you to "decode" the APPX internal format. Needless to say, we recommend passing numeric values in binary form, and dates in (probably) character string form (convert the date to alpha before PASSing the alpha) rather than using the APPX internal formats for these fields.

It is also possible to PASS the actual data file record (so that you can modify it) or a copy of that record (just for reading). Again, be careful doing this, because when an APPX designer changes the record layout, the C function will malfunction in unpredictable ways. The record layout will match what is described in techdoc, in terms of offsets into the record and lengths for each field.

No conversions of data are performed when a record is PASSed, and there is no way to verify from within the C function that the size and layout of the PASSed record matches what the function was written to expect.

Finally, many C functions provide their callers with a "return code" value. If your CALLed function sets its return code to indicate various completion statuses, you can examine the 32-bit value returned by it (if any) by inspecting the APPX PDF "--- RETURN CODE".

Here is a list of how C types correspond to APPX data types and PASS share flags:

C function parameter	APPX data type	SHARE?
unsigned char	LOGIC (length 1)	N
unsigned char *	LOGIC (length 1)	Y
char *	ALPHA SUBSTRING TEXT	Y or N⁽¹⁾
unsigned integer	NUMERIC BINARY (length 4)	N⁽²⁾
unsigned integer *	NUMERIC BINARY (length 4)	Y⁽²⁾
unsigned long	NUMERIC BINARY (length 4)	N⁽²⁾
unsigned long *	NUMERIC BINARY (length 4)	Y⁽²⁾
unsigned short	NUMERIC BINARY (length 2)	N⁽²⁾
unsigned short *	NUMERIC BINARY (length 2)	Y⁽²⁾
struct	(PASS as RECORD)	N
struct *	(PASS as RECORD)	Y

(1) Pass with SHARE? N to send a pointer to a null-terminated read-only copy of the field to the C function. Pass with SHARE? Y to send a pointer to the actual APPX field, so that the C function can modify it.

(2) Use "unsigned" if the APPX numeric field has been declared as "Signed? N" at the data dictionary level, and omit it if the APPX field has been declared as "Signed? Y" at the data dictionary level.

How do I pass a parameter to the C routine and get a result back?

You can use the same example as above, with some very minor adjustments. (Changes are in red):

#include <stdio.h>

#ifdef __cplusplus
extern "C"
{
#endif

void func( int * x )
{
    printf( "inside of func - called with %d\n", *x );
    *x=55;
}

#ifdef __cplusplus
}
#endif

You must also change the "SHARE?" flag from 'N' to 'Y' on the PASS:

SET  --- AI            99
PASS --- AI            FIELD     SHARE? Y
CALL     /appx/callee,func RESIDENT? N END? N FAIL 0

The result of calling this function is that --- AI should be set to 55.

Comments:

Read what other users have said about this page or add your own comments.

Topic revision: r3 - 2016-02-19 - JeanNeron