| Cams Administrator's Guide |
This document provides more detailed information for implementing Cams policy server clustering, which provides fault-tolerant and scalable Cams security services for any number of Cams web agents. Additional information is contained in the following documents:
Reading these documents will provide additional context helpful for understanding the material in this document.
This section provides details on how to install and manage a cluster of Cams policy servers to ensure maximum fault tolerance, scalability, and simplicity.
The strategy for installing and managing a cluster of Cams policy servers is designed to meet the following requirements:
To meet these requirements, the following installation and configuration management strategy has been devised when deploying a Cams policy server cluster.
NOTE: If you want to maintain a single master configuration on a shared file system accessible to all Cams policy server hosts, you should use a RAID system in which files are striped across multiple disks and parity information is stored so that the contents of any failed disk can be reconstructed from the remaining disks.
For management simplicity, the goal is to create a single master copy of the Cams policy server configuration on a shared file system that can be copied to each Cams policy server host. When each Cams policy server is started on it's appropriate host, it will load a host-specific configuration file and global, per-cluster, and per-server files as needed. This provides a structured environment that enables every Cams policy server to use mostly shared configuration settings, while allowing for per-cluster and per-server customizations.
To achieve this configuration management goal, the following installation strategy is recommended. More details on each of these steps is provided in the Policy Server Clustering Quick Start.
The following section provides an example showing a Cams policy server cluster master configuration.
The following example creates a Cams policy server cluster corresponding to Figure 1. To summarize various components and relationships depicted in the figure:
In the instructions that follow, the token: ${cams.home} corresponds to the top directory in your master configuration directory tree.
Every Cams policy server in a cluster must be registered by creation of a file at:
${cams.home}/conf/cams-reg-IP_ADDRESS.conf
where IP_ADDRESS is the static Internet Protocol address assigned to each clustered host. For the example cluster, the following files must be created:
${cams.home}/conf/cams-reg-192.168.1.101.conf
${cams.home}/conf/cams-reg-192.168.1.102.conf
Examples 1 and 2 show the respective contents for these files. As you can see, the properties in these files declare the Cams policy server name and Cams cluster name. When a Cams policy server is started, it loads the registration file corresponding to its assigned IP address.
# #--- Cams Policy Server Registration for host: Orville # |
Example 1 - Cams policy server registration file for ${cams.home}/conf/cams-reg-192.168.1.101.conf
# #--- Cams Policy Server Registration for host: Wilbur # |
Example 2 - Cams Policy Server Registration File for ${cams.home}/conf/cams-reg-192.168.1.102.conf
Regardless of how may Cams policy servers (or clusters) you are running, registration files must be created in directory:
${cams.home}/conf
By default, Cams includes a file installed at:
${cams.home}/conf/cams-reg-default.conf
which contains the default Cams policy server name and cluster name. Example 3 shows the default contents. If an IP address-specific file is not found on a given host, the default registration is loaded.
# #--- Default Cams Policy Server Registration # |
Example 3 - The default Cams policy Server registration file: ${cams.home}/conf/cams-reg-default.conf
As you might guess, even a standalone Cams policy server has an associated Cams cluster name. In this case, the cluster only has one host. Figure 2 shows the default configuration if you had installed a Cams policy server on host Orville.
Figure 2 - The Default Cams Policy Server Configuration if Installed on Orville
For each Cams web agent, configure the Cams cluster name and per-server connection parameters. Example 4 shows the important cluster settings for a Cams web agent configuration file using the WrightBrothers cluster on the same local area network. All other agent configuration settings should remain the same as configured for a single Cams policy server environment. Important considerations when configuring Cams web agents include:
... # # Configure the Cams Cluster Name associated with this agent # cams.cluster.name=WrightBrothers # # Enable/disable Cams Cluster debugging # cams.cluster.debug=true # # Configure all Cams Policy Server URLs # cams.server.url.Orville=cams://192.168.1.101:9191 cams.server.url.Wilbur=cams://192.168.1.102:9191 ... |
Example 4 - A cams-webagent.conf file configured to use the WrightBrothers cluster
The master configuration hiearchy is designed to support management of any number of Cams policy server clusters by use of hierarchically organized directories and files, which may have one of the following scopes:
Each of these scopes is associated with a corresponding "home" directory:
These properties are set for every Cams policy server depending on its assigned cams.cluster.name and cams.server.name. Table 1 shows the key directories associated with the WrightBrothers policy server cluster when cams.home=cams
| Directory or File |
Comments
|
|---|---|
|
cams/
|
The global Cams directory (cams.home), which applies to all Cams clusters and servers. |
| cams/bin/ | The global Cams binaries and scripts directory for all Cams clusters and servers. |
|
cams/conf/
|
The global Cams configuration directory for all Cams clusters and servers. |
|
cams/classes/
|
Extracted Java classes global to all Cams clusters and servers. |
| cams/lib/ | Java and native libraries global to all Cams clusters and servers. |
|
cams/WrightBrothers/
|
cams.cluster.home for the WrightBrothers cluster |
|
cams/WrightBrothers/conf/
|
The cluster-wide configuration directory for the WrightBrothers cluster. |
|
cams/WrightBrothers/Orville/
|
cams.server.home for Cams policy server Orville in cluster WrightBrothers |
|
cams/WrightBrothers/Orville/conf/
|
The configuration directory for Orville within the WrightBrothers cluster |
|
cams/WrightBrothers/Wilbur/
|
cams.server.home for Cams policy server Wilbur in cluster WrightBrothers |
|
cams/WrightBrothers/Wilbur/conf/
|
The configuration directory for Wilbur within the WrightBrothers cluster |
Table 1 - Global, per-cluster, and per-server directories in the master Cams configuration
The configuration properties used by a Cams policy server are determined by the following rules:
This scheme enables use of global configuration properties for all Cams policy servers with the ability to override specific values at the cluster and server levels.
In some cases it may be desireable (or necessary) to configure each Cams policy server in a cluster to listen for Cams web agent connections on different TCP/IP ports. For example, if a Cams web agent must connect to Cams policy servers in a cluster across a firewall using Network Address Translation (NAT), but the firewall has only a single IP address available for NAT, then routing to different policy servers may be possible using different ports. Figure 3 shows an example network topology using port-based NAT through a firewall.
Figure 3 - Listening on different ports to support port-based NAT routing
In this example, the Cams web agent would use the connection parameters defined in Example 5.
... # # Configure the Cams Cluster Name associated with this agent # cams.cluster.name=WrightBrothers # Configure all Cams Policy Server URLs # cams.server.url.Orville=cams://64.151.190.3:9191 cams.server.url.Wilbur=cams://64.151.190.3:9192 ... |
Example 5 - Cams web agent connection configuration for port-based NAT
In the WrightBrothers cluster, suppose you'd like Cams policy server Orville to listen for Cams web agent connections on default port 9191 and Wilbur to listen on port 9192. Create the file: cams/WrightBrothers/Wilbur/cams.conf with the contents shown in Example 6. This property will override the value loaded from global configuration file: cams/conf/cams.conf, but only for Cams policy server Wilbur. All other properties will be inherited from the global and cluster-level configuration files. Orville will continue to use the default global property: cams.server.port=9191 set in the global configuration file.
# #--- The port used to listen for Cams agent connections # |
Example 6 - Listening for Cams web agent connections on TCP/IP port 9192 on Cams policy server Wilbur
To enable DEBUG-level messages for a specific Cams policy server, create a server-level cams.conf file and place it into the appropriate server directory. For example, set enable DEBUG-level messages for Orville, create or edit file: cams/conf/WrightBrothers/Orville/conf/cams.conf using the property shown in Example 7.
# #--- Enable or Disable Policy Server DEBUG-level messages # |
Example 7 - Enabling DEBUG-level messages for a specific Cams policy server
A Cams policy server license file is always named: cams-license-keys.xml. The first license file found in the following list is loaded:
Consequently, you must ensure that the license is valid for the Cams policy server that loads it. Cams policy servers are licensed by IP address and a given license file may reference one or more valid Cams policy server IP addresses.
Example 8 shows a sample cams-license-keys.xml file valid for multiple Cams policy servers. If this license applies to all Cams policy servers deployed in a cluster, but not to all clusters deployed at your site, install it at:
${cams.cluster.home}/conf/cams-license-keys.xml
<!-- A sample Cams policy server license for multiple policy servers --> <cams-license-keys> <cams-license-key name="cams-server" key="6b314b77492b7446794133544564445a294d507a667b6a4444365a3d" version="2.*" expiration="Never" contact="flyboy@wrightbrothers.com" ip-addresses="192.168.1.101,192.168.1.102" concurrent-sessions="unlimited" /> </cams-license-keys> |
Example 8 - A sample cams-license-keys.xml file for multiple Cams policy servers
Example 9 shows a sample cams-license-keys.xml file valid for a single Cams policy server. If using this license in a multiple policy server cluster, install it at:
${cams.server.home}/conf/cams-license-keys.xml
<!-- A sample Cams policy server license for one policy server --> <cams-license-keys> <cams-license-key name="cams-server" key="7a324b67492b74b6c941335a4564245a294d5a7a667bba4441365a3d" version="2.*" expiration="Never" contact="flyboy@wrightbrothers.com" ip-addresses="192.168.1.101" concurrent-sessions="unlimited" /> </cams-license-keys> |
Example 9 - A sample cams-license-keys.xml file for one Cams policy server
Example 10 shows a sample cams-license-keys.xml file valid for evaluation use. For simplicity, install licenses of this type at:
${cams.home}/conf/cams-license-keys.xml
<!-- A sample Cams policy server evaluation license --> <cams-license-keys> <cams-license-key name="cams-server" key="9b314a67391b74a6c931375a4564275a294d5c7a667fbf4141265b3e" version="2.*" expiration="4/1/2005" contact="flyboy@wrightbrothers.com" ip-addresses="any" concurrent-sessions="25" /> </cams-license-keys> |
Example 10 - A sample evalution cams-license-keys.xml file
Standard Cams security domain configuration files include:
For deployments of just one Cams policy server cluster, one set of security domain configuration files is needed. If your master configuration manages multiple Cams policy server clusters, each with different user communities and/or access control needs, then multiple sets of security domain configuration files will be needed.
By default Cams security domain configuration files are installed in a global directory tree as shown in Table 2.
| Security Domain Configuration Files |
|---|
|
${cams.home}/conf/domains/security-domain-registry.xml
|
| ${cams.home}/conf/domains/system/security-domain.xml |
|
${cams.home}/conf/domains/system/access-control-policy.xml
|
|
${cams.home}/conf/domains/system/login-config.xml
|
Table 2 - The default location for Cams security domain configuration files
Example 11 shows the property in configuration file ${cams.home}/conf/cams.conf that specifies the location of security-domain-registry.xml. As you can see, it is located at the global directory level.
...
#
#--- The factory class used to create the SecurityDomainRegistry
#--- and it's configuration parameter(s)
#
security.domain.registry.factory=\
com.cafesoft.security.engine.domain.XmlSecurityDomainRegistryFactory
security.domain.registry.factory.params=\
${cams.home}/conf/domains/security-domain-registry.xml
...
|
Example 11 - The default path to security-domain-registry.xml (used to register securty domains)
Likewise, the default security-domain-registry.xml contents is shown in Example 12. As you can see, the home directory for each security domain is located at the global configuration level. Also, the value of substitution variable: cams.logs.base.dir configures the log file base path to a global-level directory.
<security-domain-registry>
<!-- User-defined variables that apply across all security domains. -->
<var-list>
<var name="cams.logs.base.dir" value="${cams.home}/logs"/>
</var-list>
<!-- The system security domain (required) -->
<security-domain enabled="true">
<name>system</name>
<home>${cams.home}/conf/domains/system</home>
</security-domain>
<!-- The mydomain security domain (template) -->
<security-domain enabled="false">
<name>mydomain</name>
<home>${cams.home}/conf/domains/mydomain</home>
</security-domain>
</security-domain-registry>
|
Example 12 - The default path to security domain home directories in security-domain-registry.xml
If your Cams master configuration is being used to manage multiple Cams policy server clusters, consider managing each cluster-specific tree of Cams security domain configuration files using the structure shown in Table 3.
| Security Domain Configuration Files |
|---|
|
${cams.cluster.home}/conf/domains/security-domain-registry.xml
|
| ${cams.cluster.home}/conf/domains/system/security-domain.xml |
|
${cams.cluster.home}/conf/domains/system/access-control-policy.xml
|
|
${cams.cluster.home}/conf/domains/system/login-config.xml
|
Table 3 - Recommended location for managing cluster-specific Cams security domain files
To manage a set of security domain configuration files at the cluster-level, create a cluster-specific configuration file at: ${cams.cluster.home}/conf/cams.conf with the contents shown in Example 13. For the example WrightBrothers cluster, place this file at: cams/WrightBrothers/conf/cams.conf
...
#
#--- The cluster-specific path to the security domain registry
#
#
security.domain.registry.factory.params=\
${cams.cluster.home}/conf/domains/security-domain-registry.xml
...
|
Example 13 - Configuring a cluster-specific security-domain-registry.xml path
Next, copy the entire global directory hierarchy from: ${cams.home}/conf/domains to ${cams.cluster.home}/conf/domains. For the example WrightBrothers cluster, this would give you the directory tree shown in Table 4.
| Security Domain Configuration Files |
|---|
|
cams/WrightBrothers/conf/domains/security-domain-registry.xml
|
| cams/WrightBrothers/conf/domains/system/security-domain.xml |
|
cams/WrightBrothers/conf/domains/system/access-control-policy.xml
|
|
cams/WrightBrothers/conf/domains/system/login-config.xml
|
Table 4 - Cluster-wide Security Domain Configuration Files in the WrightBrothers Cluster
Finally, edit the contents of security-domain-registry.xml to reference security domain home directories at the cluster level as shown in Example 14. As you can see, all log files for the cluster will also be written to a cluster-level directory.
<security-domain-registry>
<!-- User-defined variables that apply across all security domains. -->
<var-list>
<var name="cams.logs.base.dir" value="${cams.cluster.home}/logs"/>
</var-list>
<!-- The system security domain (required) -->
<security-domain enabled="true">
<name>system</name>
<home>${cams.cluster.home}/conf/domains/system</home>
</security-domain>
<!-- The mydomain security domain (template) -->
<security-domain enabled="false">
<name>mydomain</name>
<home>${cams.cluster.home}/conf/domains/mydomain</home>
</security-domain>
</security-domain-registry>
|
Example 14 - A cluster-specific security-domain-registry.xml file for the WrightBrothers cluster
Cams policy servers have two log file types:
Security domain-specific log files include transaction log files and the security domain-specific trace log file. By default, each filename starts with the security domain name. Using the default Cams security-domain.xml file settings, the location to which security domain-specific log files are written for a given Cams policy server can be controlled by editing the value of variable cams.logs.base.dir in file security-domain-registry.xml. Example 15 shows the default value of this variable, which causes log files to be written to directory ${cams.home}/logs.
<security-domain-registry>
<!-- Variables that apply across all security domains. -->
<var-list>
<var name="cams.logs.base.dir" value="${cams.home}/logs"/>
</var-list>
...
</security-domain-registry>
|
Example 15 - Writing Cams Transaction Logs to a global directory
NOTE: Each Cams policy server should write log files to a local file system. This avoids contention between multiple Cams policy servers attempting to write to the same file and it eliminates a shared file system as a single point of failure for all Cams policy servers.
By substituting the value of ${cams.home} with ${cams.cluster.home} or ${cams.server.home}, log files can be written to a cluster-specific or a service-specific directory as desired.
In a single Cams policy server configuration only one Cams policy server trace log file created. By default, it is written to a global-level file at:
${cams.home}/logs/${cams.server.name}-trace.log
This is controlled by the configuration setting in a cams.conf file as shown in Example 16.
...
#
#--- Configure the Cams Policy Server Trace Logger
#
logger.class=com.cafesoft.cams.log.CamsTraceLogger
logger.file.path=${cams.home}/logs/${cams.server.name}-trace.log
logger.file.append=true
logger.file.bufferedIO=true
logger.file.bufferSize=8192
logger.file.maxSize=10MB
logger.file.maxBackupIndex=100
logger.enableConsole=false
logger.enableDebugFilter=false
logger.verbose=false
logger.debug=false
...
|
Example 16 - Configuring the Path where a Cams Policy Server Trace Log is Written
To override this global setting with a cluster-specific value, create a cams.conf file at:
${cams.cluster.home}/conf/cams.conf
and insert the property shown in Example 17. This will override the value in the global property file.
...
#
#--- Override the Cams Policy Server Trace Log Path
#
logger.file.path=${cams.cluster.home}/logs/${cams.server.name}-trace.log
...
|
Example 16 - Overriding the Cams policy server trace log path at the cluster level
Every Cams policy server is clustered, so there's no special startup instructions. Each Cams policy server must be started on the computer system for which it is licensed. The Cams startup scripts for your operating system are designed to work for every Cams policy server (e.g., there's no need to create separate startup scripts for each Cams policy server host). The context-sensitive values required for Cams policy servers are selected based on the hosts IP address, which results in unique assignment of a cams.cluster.name and cams.server.name.
On the other hand, stopping a clustered Cams policy server has more flexibility. By default Cams policy server shutdown scripts will shutdown Cams on the local computer system. However, you may also shutdown a Cams policy server on a remote computer system so long as you have access to Cams deployment directory on one of the Cams policy server hosts. The shutdown scripts enables you to specify the cams.cluster.name and cams.server.name of the policy server you'd like to shutdown.
To shutdown a Cams policy server on the local system when it has been installed as an operating system service, issue the following command:
Linux/UNIX:
/usr/sbin/service cams stop
Windows:
net stop cams
NOTE: You may also use a platform-specific management GUI to stop (and start) a Cams policy server in some cases. For example, under Windows NT/2000/2003, you may use the Control Panel/Services GUI.
To shutdown a Cams policy server running on the local computer system from a command line, issue the following command:
Linux/UNIX:
$CAMS_HOME/bin/shutdown.sh
Windows:
%CAMS_HOME%\bin\shutdown.bat
To shutdown a remote Cams policy server from the command line, you must be logged into a system on which a Cams policy server is installed. Simply specify the IP address or host name of the Cams policy server to be shutdown. The appropriate Cams policy server-specific properties and the shutdown client will attempt to connect with and initiate shutdown of the specified Cams policy server. For example:
Linux/UNIX:
$CAMS_HOME/bin/shutdown.sh orville
Windows:
%CAMS_HOME%\bin\shutdown.bat orville
If the computer system on which you've installed the Cams policy server is multi-homed (has multiple IP addresses assigned to its network intefaces), you may need to tell Cams the IP address to which it should bind.
If you need to use a different IP address on the local system, you must specify it as a command-line parameter when Cams is started. For example, if the local system has IP addresses: 192.168.1.101 and 192.168.1.102 assigned, then:
-Dcams.server.bindaddr="192.168.1.102"
will tell the policy server to bind to IP address 192.168.1.102. If the value of this parameter is not a valid IP address or is not an IP address assigned to the local system, then a fatal configuration error will be reported and the Cams policy server will fail on startup. If this parameter is not specified, the Cams policy server will bind to the first IP address returned from the list of IP addresses assigned to the local system.
To set the Cams policy server bind address:
Edit file: $CAMS_HOME/bin/runcams.sh and add the parameter as shown below:
P="-Dcams.home=${CAMS_HOME}"
P="${P} -DvalidateXML=\"true\""
P="${P} -Dcams.server.bindaddr=192.168.1.102"
if [ -n "${SECURITY_POLICY_FILE}" ] ; then
P="${P} -Djava.security.manager"
P="${P} -Djava.security.policy==${SECURITY_POLICY_FILE}"
fi
Edit file: %CAMS_HOME%\conf\runcams.conf and add the parameter as shown below:
# Java Additional Parameters wrapper.java.additional.1=-server wrapper.java.additional.2=-Dcams.home=.. wrapper.java.additional.3=-DvalidateXML=true wrapper.java.additional.4=-Dcams.server.bindaddr=192.168.1.102
If you need to run multiple Cams policy servers on a single multi-homed system, you'll need to copy the runcams.bat/runcams.conf or runcams.sh script to a new file and customize it for the second server.
To debug the cluster settings for a Cams policy server, logon to the computer system where the Cams policy server is intended to run and start the Cams policy server from a console. Some INFO-level messages displayed on the console will report important configuration settings as shown in Example 17.
NOTE: These messages are also written to the Cams policy server trace log file, but at a slightly later time. That's because the trace logger does not exist and cannot be configured until the configuration properties for the Cams policy server are loaded. Consequently, debugging cluster settings is more reliable from a console.
[INFO ] Initializing Cams Policy Server version: 2.0 |
Example 17 - Example INFO reported when Cams policy server Orville is successfully started
If a Cams policy server registration file cannot be found or a configuration setting is missing or invalid, one or more [ERROR] or [FATAL] messages will be reported.
To debug the cluster settings and operation of a Cams web agent, set the configuration property:
cams.cluster.debug=true
in the Cams web agent configuration file (e.g., cams-webagent.conf). This will cause the Cams web agent to report the Cams policy server to which a request is being delegated and will also report a failed or unavailable Cams policy server if the Cams web agent would otherwise attempt to use that server. To confirm proper Cams web agent detection of an unavailable Cams policy server, shutdown one of the Cams policy servers. Example 18 shows the INFO-level messages when Cams policy server Orville is shutdown.
... ... [INFO ] Attempting to reconnect to Orville at 192.168.1.101:9191 |
Example 18 - The Cams web agent connection monitor reports a failed policy server connection
Another way to confirm proper operation of Cams policy server clustering is to monitor server-side log files. By default, log files are written to directory: CAMS_HOME/logs on each Cams policy server host using the naming scheme:
${cams.server.name}-{cams.security-domain.name}-<Service Name>.log
For example: Orville-system-access-control.log
To confirm proper round-robin load balancing across Cams policy servers:
You should see an approximately equal number of logged access-control transactions in Wilbur-system-access-control.log and Orville-system-access-control.log.
Now shutdown one of the Cams policy servers (Orville) and attempt to access the same unconditionally granted or denied web resources. All access-control transactions should now be handled by the available Cams policy server (Wilbur).
Once you have confirmed proper operation of Cams policy server clustering, be sure to disable Cams cluster debug messages and enable Cams agent access control check caching (if desired):
cams.cluster.debug=false cams.access.check.cache=true
© Copyright 1996-2005 Cafésoft LLC. All rights reserved.