RHQ and AS7

This page it to discuss connections from RHQ to AS7. There are currently some discrepancies in the underlying model of AS 7 vs RHQ ,as well as additional requirements from the RHQ side.

Intro

RHQ uses a model where on a host (=platform) an agent is installed, which detects resources on the platform and also monitors them (along with platform-specific resources like disk space or free memory).

AS7 on the other side has an internal model as a tree, where nodes represent resources, that can have attributes and child nodes. The mapping to RHQ resources can not be done 1:1 in many cases, as the AS7 resources (in the model) are much more lightweight than their RHQ counterpart.

AS7 has two modes of operation:

Standalone, like previous AS servers
Domain mode, where a domain controller controls many managed servers

Communication

Both modes come with a management API, that runs on a separate port (and possibly interface) in the server. Two options are available

Port 9999 with the DMR (dynamic model representation) transport protocol
Port 9990/management with JSON over http

For the RHQ plugin I have chosen the JSON port, as this means that there are no dependencies of the plugin on any JBossAS classes. Through the detyped nature of the management interface, the dependencies are minimal anyway, but it is important not to run into classloading hell, as it was the case with AS5 especially. Most of the JSON stuff is encapsulated in org.rhq.plugins.jbossas7.json.* and org.rhq.plugins.jbossas7.AS*Connection - so if the need to switch over to DMR arises, the impact should be limited.

Nodes in the AS7 tree

Nodes in the AS7 tree are represented as list of type=name pairs like e.g.

server-group=main-server-group
deployment=test.war
server-group=main-server-group,deployment=test.war

Operations

The AS7 api lists for each resource a set of available operations. If you are using the AS7-CLI (bin/jboss-admin.sh) you can type a colon and then the tab-key to see the operations available. Otherwise you can issue the :read-operation-names operation to get a list of operations and the :read-operation-description(name=xxx) operation to get the parameters. To just list the attributes of a node use :read-resource.

Standalone mode

The RHQ agent talks through the AS7-Plugin with the server. The server gets auto-detected. If you want to support more than one server, you need to take all of them into inventory and then e.g. group them together, just like with AS4, 5 and 6.

Domain mode

h2. Scenario

The setup of resources within a AS 7 "domain" is a lot different then all we know within the classical AS releases before.
A domain can host and manage multiple servers on multiple platforms. All servers in a domain come from the same server family (meaning all being AS 7, but not e.g. AS 6 or AS 8).

In addition to the actual server processes, there is a host controller (HC) per platform that controls the servers on this platform. One platform hosts the domain controller (DC), which is the authoritative source for all configuration information in the whole domain. Clients (like a command line interface, the admin console or RHQ) talk to the domain controller (only) to obtain runtime state or to trigger configuration changes.

The domain controller then relays those requests to the appropriate host controllers which relay them to the individual server instances.

This indirection through the DomainController (DC) has the issue with current RHQ, that it hides the individual AS instances behind the DC and shows them

as child resources of the DC
on the same platform of the DC
which has the implication that the RHQ administrator does not see which platform hosts which AS and also can not easily get the logical connection between platform and server (e.g. for alerting).

This mismatch is also present in other scenarios where e.g. network devices (routers, switches) are monitored from one agent via SNMP or when the Nagios plugin is talking to a nagios server that is then talking to many resources and hosts that have no RHQ agent installed.

The decision here is that for a domain where several platforms are involved, RHQ requires that on each platform an agent is running in order to detect the local host controller and the hosts managed on that platform. Nevertheless, all commands that change configurations need to be issued on the domain controller.

The other issue here may be for monitoring where it is needed to quickly pull metric data from many metrics on many resources.
In this case the connection is directly made to the host controller on the host, where the managed server resides.

Server groups

All servers within a domain belong to one server-group. Server-groups form sets of servers with (mostly) identical configuration

In above graphic there are three server groups defined with 2,3,4 servers respectively.

Deployment of Content

Deployment of content into servers is a multi-step process; especially on domain level.
In the Domain, content ( jars, wars, ears, sars, ... ) are not put into a deploy directory,
but

Uploaded to the DC
Added to /deployments on the DC
added to server-group on the DC
- If enabled=true, this will make the content available on all the hosts of the server-group

It is possible to create complex deployment plans with order of deployments and rules for rollback and so on

For deployments on a Standalone server, there is still the possibility to drop them into a designated folder, but as the same api calls as described (except for adding to a server group) can be used, the same code path will in fact be used.