Next: Proxy Objects Up: No Title Previous: Interface Type Checking

# Connection Management

This chapter describes how omniORB2 manages network connections.

# Background

In CORBA, the ORB is the ``middleware'' that allows a client to invoke an operation on an object without regard to its implementation or location. In order to invoke an operation on an object, a client needs to ``bind'' to the object by acquiring its object reference. Such a reference may be obtained as the result of an operation on another object (such as a naming service) or by conversion from a stringified representation previously generated by the same ORB. If the object is in a different address space, the binding process involves the ORB building a proxy object in the client's address space. The ORB arranges for invocations on the proxy object to be transparently mapped to equivalent invocations on the implementation object.

For the sake of interoperability, CORBA mandates that all ORBs should support IIOP as the means to communicate remote invocations over a TCP/IP connection. IIOP is asymmetric with respect to the roles of the parties at the two ends of a connection. At one end is the client which can only initiate remote invocations. At the other end is the server which can only receive remote invocations.

Notice that in CORBA, as in most distributed systems, remote bindings are established implicitly without application intervention. This provides the illusion that all objects are local, a property known as ``location transparency''. CORBA does not specify when such bindings should be established or how they should be multiplexed over the underlying network connections. Instead, ORBs are free to implement implicit binding by a variety of means.

The rest of this chapter describes how omniORB2 manages network connections and the programming interface to fine tune the management policy.

# The Model

OmniORB2 is designed from the ground up to be fully multi-threaded. The objective is to maximise the degree of concurrency and at the same time eliminate any unnecessary thread overhead. Another objective is to minimise the interference by the activities of other threads on the progress of a remote invocation. In other words, thread ``cross-talk'' should be minimised within the ORB. To achieve these objectives, the degree of multiplexing at every level is kept to a minimum.

On the client side of a connection, the thread that invokes on a proxy object drives the IIOP protocol directly and blocks on the connection to receive the reply. On the server side, a dedicated thread blocks on the connection. When it receives a request, it performs the up-call to the object and sends the reply when the upcall returns. There is no thread switching along the call chain.

With this design, there is at most one call in-flight at any time in a connection. If there is only one connection, concurrent invocations to the same remote address space would have to be serialised. To eliminate this limitation, omniORB2 implements a dynamic policy- multiple connections to the same remote address space are created on demand and cached when there are concurrent invocations in progress.

To be more precise, a network connection to another address space is only established when a remote invocation is about to be made. Therefore, there may be one or more object references in one address space that refers to objects in a different address space but unless the application invokes on these objects, no network connection is made. The maximum number of connections opened to another address space is 5 by default. Since 2.6.0, this parameter can be changed by setting the variable `omniORB::maxTcpConnectionPerServer` before calling `ORB_init`.

It is wasteful to leave a connection opened when it has been left unused for a considerable time. Too many idle connections could block out new connections to a server when it runs out of spare communication channels. For example, most unix platforms has a limit on the number of file handles a process can open. 64 is the usual default limit. The value can be increased to a maximum of a thousand or more by changing the ``ulimit'' in the shell.

# Idle Connection Shutdown

Inside the ORB, two separate threads are dedicated to scan for idle connections. One thread is responsible for outgoing connections and the other looks after incoming connections. The thread for incoming connections is only created when the BOA is initialised because only then will there be any incoming connections.

The threads scan all opened connections once every ``scan period''. If a connection is found to be idle for two consecutive periods, it will be closed. The threads use mark-and-swipe to detect if a connection is idle. When a connection is checked, a status flag attached to the connection is set. Every remote invocation using that connection would clear the flag. So if a connection's status flag is found to be set in two consecutive scans, the connection has been idled during the scan period.

The scan period for incoming and outgoing connections can be individually controlled by the following API:

    class omniORB {

    public:

      enum idleConnType { idleIncoming, idleOutgoing };

      static void idleConnectionScanPeriod(idleConnType direction,
                                           CORBA::ULong sec);

      static CORBA::ULong idleConnectionScanPeriod(idleConnType direction);

    };

The current value of the scan period (in seconds) is returned by the read-only `idleConnectionScanPeriod`. The scan period can be changed by the write-only `idleConnectionScanPeriod`. The scan period can be changed at any time but the write function is non-thread safe. Concurrent calls to this function could result in undefined behaviour. The scan periods may also be specified on the command-line by using the options `-ORBinConScanPeriod` and `-ORBoutConScanPeriod`. See section 4.1.

The default value for both incoming and outgoing connections is 30 seconds. The scan can be disabled completely by setting the scan period to 0.

# Interoperability Considerations

The IIOP specification allows both the client and the server to shutdown a connection unilaterally. When one end is about to shutdown a connection, it should send a closeConnection message to the other end. It should also make sure that the message will reach the other end before it proceeds to shutdown the connection.

The client should distinguish between an orderly and an abnormal connection shutdown. When a client receives a closeConnection message before the connection is closed, the condition is an orderly shutdown. If the message is not received, the condition is an abnormal shutdown. In an abnormal shutdown, the ORB should raise a `COMM_FAILURE` exception whereas in an orderly shutdown, the ORB should not raise an exception and should try to re-establish a new connection transparently.

OmniORB2 implements this semantics completely. However, it is known that some ORBs are not (yet) able to distinguish between an orderly and an abnormal shutdown. Usually this is manifested as the client in these ORBs seeing a `COMM_FAILURE` occasionally when connected to an omniORB2 server. The workaround is either to catch the exception in the application code and retries or to turn off the idle connection shutdown inside the omniORB2 server.

# Connection Acceptance

OmniORB2 provides the hook to implement a connection acceptance policy. Inside the ORB runtime, a thread is dedicated to receive new connections. When the thread is given the handle of a new connection by the operating system, it calls the policy module to decide if the connection can be accepted. If the answer is yes, the ORB will start serving requests coming in from that connection. Otherwise, the connection is shutdown immediately.

There can be a number of policy module implementations. The basic one is a dummy module which just accepts every connection.

In addition, a host-based access control module is available on unix platforms. The module uses the IP address of the client to decide if the connection can be accepted. The module is implemented using tcp_wrappers 7.6. The access control policy can be defined as rules in two access control files: `hosts.allow` and `hosts.deny`. The syntax of the rules is described in the manual page `hosts_access(5)` which can be found in appendix A. The syntax defines a simple access control language that is based on client (host name/address, user name), and server (process name, host name/address) patterns. When searching for a match on the server process name, the ORB uses the value of `omniORB::serverName`. `ORB_init` uses the argument `argv[0]` to set the default value of this variable. This can be overridden by the application by passing the option: `-ORBserverName <string>` to `ORB_init`.

The default location of the access control files is `/etc`. This can be overridden by the extra options in `omniORB.cfg`. For instance:

    # omniORB configuration file - extra options
    #

    GATEKEEPER_ALLOWFILE   /project/omni/var/hosts.allow

    GATEKEEPER_DENYFILE    /project/omni/var/hosts.deny

As each policy module is implemented as a separate library, the choice of policy module is determined at program linkage time.

For instance, if the host-based access control module is in use:

    % eg1 -ORBtraceLevel 2
    omniORB2 gateKeeper is tcpwrapGK 1.0 - based on tcp_wrappers_7.6
    I said,"Hello!". The Object said,"Hello!"

Whereas if the dummy module is in use:

    % eg1 -ORBtraceLevel 2
    omniORB2 gateKeeper is not installed. All incoming are accepted.
    I said,"Hello!". The Object said,"Hello!"

Next: Proxy Objects Up: No Title Previous: Interface Type Checking