Using hardware control software
CSM hardware control software provides remote hardware control functions for cluster nodes and devices from a single point of control. CSM allows you to control cluster nodes remotely through access to the cluster management server. From the management server, an administrator runs cluster management commands using the command line, Web-based System Manager graphical user interface (GUI), System Management Interface Tool (SMIT) panels, or the DCEM graphical user interface.
CSM supports hardware control for non-node devices, and provides power control and where applicable, remote console access for a wide range of devices such as hardware control points, external console servers, and remote supervisor adapters. The predefined dynamic device group AllDevices includes all defined devices.
CSM hardware control functions depend on the specific hardware, software, network, and configuration requirements described in this book. The requirements for remote power are separate and distinct from the requirements for remote console. clusters without the hardware, software, network, or configuration required to use CSM hardware control can still have CSM installed on some or all cluster nodes. However, in such clusters the hardware control commands may be inoperable or provide only limited function.
CSM for AIX 5L supports remote hardware control for pSeries, xSeries, BladeCenter, SP Nodes, p660 nodes,
325 and 326, and
OpenPower 720 from an AIX management server. Hardware control commands can be run on the AIX management server to simultaneously control both AIX and Linux nodes in a mixed cluster. See the CSM for AIX 5L and Linux: Administration Guide for a complete description of CSM mixed clusters.
In CSM documentation “p660 nodes” refers to pSeries 660 nodes (which are not HMC-attached), and the M80, H80, 6H0/6H1, and 6M1 RS/6000 servers.
CSM for Linux supports remote hardware control for xSeries, pSeries, BladeCenter, and eServer 325 and 326 servers from a Linux management server.
The following list describes the CSM hardware control commands; see the man pages or the CSM for AIX 5L and Linux: Command and Technical Reference for detailed command usage information.
- chbmcconsusr
- Defines a remote console user name for node BMCs.
- chhwdev
- Changes a device definition in the CSM database.
- chrconsolecfg
- Removes, adds, or rewrites console entries in the Conserver configuration file.
- chsnmp
- Sets the SNMP agent configuration information for xSeries and BladeCenter servers.
- cspadm
- Administers the cspd daemons log file and debug flags.
- cspcfgframe
- Configures SP frame hardware control points for expansion I/O units.
- cspcfgio
- Displays the configuration information for expansion I/O units, IBM POWER3 SMP High Nodes (F/C 2054), and IBM 375 MHz POWER3 SMP High Nodes (F/C 2058).
- cspcmds
- Controls the state of SP Nodes, SP frames, and p660 nodes in the cluster.
- cspmon
- Monitors the state of SP Nodes, SP frames, p660 nodes, and expansion I/O units in the cluster.
- definehwdev
- Defines the devices in a CSM cluster.
- getadapters
- Collects information for LAN adapters.
- hwdevgrp
- Manages device group definitions in the CSM database.
- lsbmcconsusr
- Returns the remote console user names for node BMCs.
- lshwdev
- Lists the device definitions in the CSM database.
- lshwinfo
- Collects node information from one or more hardware control points.
- lshwstat
- Collects environmental and Vital Product Data (VPD) information from xSeries and BladeCenter servers. This command is not supported for CSM for Linux on pSeries.
- lssnmp
- Collects SNMP agent configuration information from xSeries and BladeCenter servers.
- netboot
- Initiates a network boot and install of an AIX node over the CSM cluster network.
- rconsole
- Opens a remote console for a node.
- rconsolerefresh
- Refreshes the Conserver daemon.
- reventlog
- Collects service processor log information for xSeries and BladeCenter servers.
- rmhwdev
- Removes device definitions from the CSM database.
- rpower
- Boots, resets, powers on and off, and queries nodes, devices and CECs.
- systemid
- Stores the user ID and password required for internal programs to access remote hardware.
Hardware control attributes
You must define the hardware-related attributes for nodes or non-node devices. In some cases default values are provided. If these defaults are acceptable, you do not need to provide the attribute values when you define the node. Hardware control attributes depend on the kind of hardware you plan to use.
For a list of the hardware control attributes that you define for a node, see Hardware control attributes.
For a list of the hardware control attributes that you define for a device, see Defining non-node devices for the cluster.
Hardware and network requirements
CSM for AIX 5L or Linux hardware control depends on the specific hardware and network requirements described in this book. The management server can be connected to cluster nodes and external networks using various configurations of IBM and non-IBM hardware and software that meet the CSM architecture requirements described in this book. For the specific cluster hardware models required to use CSM 1.4, see Planning for CSM for AIX nodes. See Hardware configuration for model-specific hardware control configuration requirements.
Note:
To perform hardware control on HMC-attached pSeries machines, Hardware Management Console (HMC) for pSeries release 3 version 1.0 or later is required. The HMC can be used to partition physical pSeries servers into multiple logical partitions (LPARs), or nodes, each containing its own operating system image.
Virtual LANs (VLANs)
A VLAN (virtual Local Area Network) is a division of a local area network by software rather than by physical arrangement of cables. Dividing a LAN into subgroups can simplify and speed up communications within a workgroup. Switching a user from one VLAN to another using software is also more efficient than rewiring the hardware.
IBM suggests creating one or more VLANs for the CSM management server, managed devices, and hardware control points, and one or more separate VLAN for the CSM management server and cluster nodes. Although cluster hardware control points and nodes can be on the same VLAN, limiting access to the management VLAN reduces security exposure for IP traffic on the management VLAN and access to hardware control points.
The VLANs refer to VLANs as defined by IEEE standards – see http://standards.ieee.org/ for details. Figure 1 Figure 3 shows a network partitioned into three virtual LANs; management, cluster, and public VLANs, which are defined as follows:
- management VLAN
- Hardware control commands such as rpower and rconsole are run on the management server and communicate to nodes through the management VLAN. The management VLAN connects the management server to the cluster hardware through an Ethernet connection. For optimal security, the management VLAN must be restricted to hardware control points, remote console servers, the management server, and root users. Routing between the management VLAN and cluster or public VLANs could compromise security on the management VLAN. The management VLAN is also referred to as the service VLAN in CSM documentation.
Note:
The management VLAN is subject to the RSA restriction of 10/100 Mb/s.
- cluster VLAN
- The cluster VLAN connects nodes to each other and to the management server through an Ethernet connection. Installation and CSM administration tasks such as running dsh are done on the cluster VLAN. Host names and attribute values for nodes on the cluster VLAN are stored in the CSM database.
- public VLAN
- The public VLAN connects the cluster nodes and management server to the site network. Applications are accessed and run on cluster nodes over the public VLAN. The public VLAN can be connected to nodes through a second Ethernet adapter in each node, or by routing to each node through the Ethernet switch.
Hardware control overview and sample configurations
CSM communicates with hardware control points to request node power status, reboot, and power on and off functions. A hardware control point is the specific piece of hardware through which the management server controls node hardware. Hardware control points should be on the management virtual LAN (VLAN) and connected to the hardware that ultimately controls the power functions.
The supported hardware control points are:
- HMC for HMC-managed pSeries nodes
- Frame supervisor for SP Nodes
- CSP serial port for p660 nodes
- RSA for xSeries
- Management module for BladeCenter
- BMC for the eServer 325, 326, xSeries 336, and xSeries 346
- APC MasterSwitch for non-node and other devices that do not have another hardware control point such as an RSA and require the APC MasterSwitch.
For SP Nodes and p660 nodes, the connection is from a dedicated tty port on the management server to the frame supervisor or CSP serial port through a serial RS-232 line.
For details on defining tty ports for SP Nodes and p660 nodes, see the “Prepare the control workstation” section in Chapter 2 of the PSSP: Installation and Migration Guide, located at http://www.ibm.com/servers/eserver/pseries/library/sp_books/pssp.html.
href=”/infocenter/clresctr/topic/com.ibm.cluster.pssp.doc/psspbooks.html”
Remote power software and configuration describes the remote power configurations for your cluster when you are using hardware control.
CSM communicates with console server hardware to open a console window for a node on the CSM management server. Console servers must be on the management VLAN, which connects the management server to the cluster hardware, and connected to node serial ports. (See Virtual LANs (VLANs).) This out of band network configuration allows a remote console to be opened from the management server even if the cluster VLAN is inaccessible. For example, if the cluster VLAN is offline, remote console can still access the target node to open a console window.
For HMC-attached pSeries, the HMC is the remote console server. For SP Nodes and p660 nodes, an independent device does not exist that can serve as a remote console server; console traffic is managed by the frame supervisor or p660 server firmware. xSeries servers can use any of the following console servers:
- MRV IR-8020, IR-8040, LX-4008S, LX-4016S, and LX-4032S
- Avocent CPS1600
- Cyclades AlterPath ACS48
Linux on pSeries clusters use the HMC for remote console; no additional console device is required or supported.
BladeCenter HS20-8678 blade servers that are part of an IBM 1350 Cluster require the 1350 Serial Port Module (SPM) option in order to support remote console. The SPM must be connected to an MRV IR-8020 or IR-8040 console server. HS20-8678 blade servers that are not part of a 1350 Cluster, or that do not have the SPM option, cannot support remote console. Consoles for these servers may be viewed, one at a time, by accessing the Management Modules web interface and selecting “Remote Control” from the “Blade Tasks” list.
BladeCenter HS20 (other than HS20-8678), HS40, and JS20 blade servers support remote console through the Ethernet Switch Module, using Serial Over LAN (SOL). Refer to your BladeCenter documentation for information on enabling and configuring SOL. To ensure maximum reliabilty, verify that the most up-to-date firmware is installed for the following components:
- BladeCenter HS20-8677 chassis: Management Module and Ethernet Switch Module
- HS20 (other than HS20-8678), HS40, and JS20 blade servers: Flash BIOS and Integrated Systems Management Processor (ISMP)
You can view the installed versions of these components in the Management Modules Web Interface by selecting “Firmware VPD” from the “Monitors” navigation panel. For information on the latest available versions, see the IBM Servers – Software and device drivers web page:
http://www.ibm.com/pc/support/site.wss/MIGR-4JTS2T.html
See your BladeCenter documentation for information on updating firmware levels.
xSeries 336 and 346 servers also support remote console via SOL. For these servers, remote consoles are opened through the on-board Baseboard Management Controller (BMC). See Remote console configuration. To ensure maximum reliability, ensure that the most up-to-date firmware is installed for the following components:
- Flash BIOS
- Baseboard Management Controller
For information on the latest available versions, see the IBM Servers software and device drivers web page: http://www.ibm.com/pc/support/site.wss/MIGR-4JTS2T.html.
Remote console software describes the remote console software for your cluster when you are using hardware control while Remote console configuration describes the different kinds of remote console configurations.
Linux on pSeries clusters use the HMC for remote console; no additional console device is required or supported.
Attention!
The diagrams discussed in this book are provided for conceptual explanation only. They are not intended to be literal depictions of how to configure a specific cluster. See the
Cluster 1350 for Linux or 1600 for AIX documentation resources listed in Related information for specific cluster hardware configuration details. For specific RSA and ISMP connectivity requirements, see the hardware documentation resources listed in Related information.
For a list of the hardware control attributes used in the diagrams, see Hardware control attributesHardware control attributes.
The examples and diagrams in this chapter are as follows:
Conceptual diagram: HMC-managed pSeries, SP, and p660 nodes
Figure 1 shows the hardware and networking configuration required for using CSM hardware control with IBM HMC-managed pSeries, SP, and p660 nodes. The HMC-attached pSeries server in the diagram is a single piece of hardware that has been partitioned by HMCs into 16 LPARs (nodes). The management server connects to the management and cluster VLANs through Ethernet adapters. Nodes on HMC-attached pSeries servers must be connected to the cluster VLAN and directly or indirectly to an HMC. Configuration for a public VLAN is flexible and can be defined by the system administrator. See Node hardware attributes for example node attribute definitions corresponding to Figure 1.
Each SP frame connects to the CSM management server through an RS-232 line cabled from the frame supervisor to a tty port on the management server. The frame supervisor requires a daisy-chain connection to the node supervisor for each server in the frame. Each p660 server connects to the CSM management server through its own RS-232 line cabled from its CSP serial port to a tty port on the management server. For multiple SP Nodes and p660 nodes, additional tty extender adapters must be connected to the management server.
Note:
For details on defining tty ports for SP Nodes and p660 nodes, see the “Prepare the control workstation” section in Chapter 2 of the
PSSP: Installation and Migration Guide, located at
http://www.ibm.com/servers/eserver/pseries/library/sp_books/pssp.html.
href=”/infocenter/clresctr/topic/com.ibm.cluster.pssp.doc/psspbooks.html”
CSM hardware control configuration for IBM HMC-attached pSeries, SP Nodes, and p660 nodes
Figure 1. CSM hardware control configuration for IBM HMC-attached pSeries, SP Nodes, and p660 nodes
AIX node attributes example
Figure 2 shows the relationship between CSM node attributes and the internal hardware names used in Figure 1. For remote power and remote console to work as expected for HMC-attached pSeries servers, this matching of node attribute names in the CSM database to the internal hardware values must be correct for all HMCs in the CSM cluster. Likewise, the tty port number on the management server and frame slot number must match the hardware control attribute values for all SP Nodes and p660 nodes in the CSM database.
CSM hardware control attribute values for AIX nodes
Figure 2. CSM hardware control attribute values for AIX nodes
Conceptual diagram: xSeries, BladeCenter, and eServer 325
Attention!
The diagrams discussed in this book are provided for conceptual explanation only. They are not intended to be literal depictions of how to configure a specific cluster. See the
Cluster 1350 resources listed in Related information for specific cluster hardware configuration details. For specific RSA and ISMP connectivity requirements, see the hardware documentation resources listed in Related information.
For a list of the hardware control attributes used in the diagrams, see Hardware control attributesHardware control attributes.
Figure 3 shows the hardware and networking configuration required for using CSM hardware control with xSeries, eServer 325, and BladeCenter servers running Linux.
The management server shown in the diagram connects to the management and cluster VLANs through Ethernet adapters. The console servers (mrv01, mrv02) connect to the management VLAN through Ethernet adapters, to the xSeries and eServer 325 servers, and to BladeCenter servers through the Cluster 1350 Serial Port Module option. BladeCenter HS20-8832 and JS20-8842 blade servers do not require an attached console server. They provide remote console independently using the Serial Over LAN feature.
The IBM Cluster 1350 Serial Port Module is an optional part that must be ordered separately to connect BladeCenter HS20-8678 blade servers to an MRV console server. This connection provides rconsole command function for HS20 BladeCenter servers. See the topic “Launching a remote console” in CSM for AIX 5L and Linux: Administration Guide for the part numbers and configuration required for this alternative to using a Web browser for HS20 BladeCenter remote console function.
JS20 BladeCenter servers support remote console function independently; they do not require an attached console server. JS20 servers use a serial over lan connection for remote console, which requires a connection to the eth1 Ethernet adapter. Thus, BladeCenter JS20 servers must connect to the management modules eth1 port. BladeCenter HS20 servers must connect to the management modules eth0 port
The management VLAN connects to the IBM Remote Supervisor Adapter (RSA) in select xSeries servers. The servers must be connected to the cluster VLAN through their first Ethernet adapters (eth0), and directly or indirectly to an RSA. An RSA connects to its node Internal Systems Management Processor (ISMP) port, and up to 24 node ISMP ports can be daisy-chained from the RSA ISMP port. See the RSA documentation listed in Related information for the number of nodes supported.
The management VLAN connects to non-node and other devices that do not have another hardware control point such as an RSA through the APC MasterSwitch (apc01) Ethernet port. Power cables connect the devices to the APC MasterSwitch. The management VLAN connects to blade servers through the BladeCenter chassis management module (mm01), and to eServer 325 through the baseboard management controller (BMC).
Applications usually run on the public VLAN, which connects to the servers through Ethernet ports. Configuration for a public VLAN is flexible and can be defined by the system administrator. See Node hardware attributes for example node attribute definitions corresponding to Figure 1.
CSM hardware control configuration for xSeries, BladeCenter, and eServer 325 Linux nodes
Figure 3. CSM hardware control configuration for xSeries, BladeCenter, and eServer 325 Linux nodes
Linux node attributes example
Figure 4 shows the relationship between the CSM node database attributes and the internal hardware names used in Figure 3. For remote power and remote console to work as expected, this matching of database attribute names to the internal hardware values must be correct for all management processors (ISMPs), remote supervisor adapters (RSAs), management modules, APCs, BMCs, and console servers in the CSM cluster.
CSM hardware control attribute values for Linux nodes
Figure 4. CSM hardware control attribute values for Linux nodes
Conceptual diagram: Linux on pSeries
Figure 5 shows the hardware and networking configuration required for using CSM hardware control with HMC-attached pSeries servers running Linux. The HMC-attached pSeries server in the diagram is a single piece of hardware that has been partitioned by HMCs into 16 LPARs (nodes). The management server connects to the management and cluster VLANs through Ethernet adapters. Nodes on HMC-attached pSeries servers must be connected to the cluster VLAN and directly or indirectly to an HMC. Configuration for a public VLAN is flexible and can be defined by the system administrator. See Node hardware attributes for example node attribute definitions.
CSM hardware control hardware configuration for Linux on pSeries
Figure 5. CSM hardware control hardware configuration for Linux on pSeries
Linux on pSeries node attributes example
Figure 6 is a detailed view of some nodes from Figure 5. The diagram shows the relationship between the CSM node database attributes and the internal hardware names used in Figure 3. For remote power and remote console to work as expected, this matching of database attribute names to the internal hardware values must be correct for all management processors (MPs), management processor adapters (MPAs), and console serial providers in the cluster.
CSM hardware control database attribute values for Linux nodes
Figure 6. CSM hardware control database attribute values for Linux nodes
Conceptual diagram: mixed cluster
Figure 7 shows the hardware and networking configuration required for using CSM hardware control in a mixed cluster with pSeries, SP Nodes, and p660 nodes running AIX; and xSeries, the eServer 325, and BladeCenter servers running Linux. The HMC-attached pSeries server has been partitioned by an HMC into 10 LPARs (nodes).
The management server shown in the diagram connects to the management and cluster VLANs through Ethernet adapters. The console servers (mrv01, mrv02) connect to the management VLAN through Ethernet adapters, to the xSeries servers through serial (COM) ports, and to BladeCenter HS20-8678 blade servers through the Cluster 1350 Serial Port Module option. BladeCenter HS20 (other than HS20-8678), blade servers do not require an attached console server. They provide remote console independently using the Serial Over LAN feature
The IBM Cluster 1350 Serial Port Module is an optional part that must be ordered separately to connect BladeCenter HS20-8678 blade servers to an MRV console server. This connection provides rconsole command function for BladeCenter servers. See CSM for AIX 5L and Linux: Administration GuideCSM for AIX 5L and Linux: Administration Guide for the configuration required for this alternative to using a Web browser for BladeCenter remote console function.
The management VLAN connects to the IBM Remote Supervisor Adapter (RSA) in select xSeries servers. The servers must be connected to the cluster VLAN through their first Ethernet adapters (eth0), and directly or indirectly to an RSA. An RSA connects to its node internal service management processor (ISMP) port, and up to nine node ISMP ports can be daisy-chained from the RSA ISMP port. See the RSA documentation listed in Related information for the number of nodes supported.
The management VLAN connects to non-node and other devices that do not have another hardware control point such as an RSA through the APC MasterSwitch (apc01) Ethernet port. Power outlets and cables connect the devices to the APC MasterSwitch. The management VLAN connects to the BladeCenter blade servers through the management module (mm01) on the BladeCenter chassis, and to the eServer 325 through the BMC.
Applications usually run on the cluster VLAN, which connects to the servers through Ethernet ports. Configuration for a public VLAN is flexible and can be defined by the system administrator. See Node hardware attributes for example node attribute definitions corresponding to Figure 1.
CSM hardware configuration for pSeries, SP Nodes, p660 nodes, xSeries, eServer 325, and BladeCenter
Figure 7. CSM hardware control configuration for pSeries, SP Nodes, p660 nodes, xSeries, eServer 325, and BladeCenter
Mixed cluster node attributes example
Figure 8 and Figure 9 show the relationship between the CSM node database attributes and the internal hardware names used in Figure 3. For remote power and remote console to work as expected, this matching of database attribute names to the internal hardware values must be correct for all HMCs, ISMPs, RSAs, APCs, BMCs, management modules, and console servers in the cluster.
CSM hardware control attribute values for AIX and Linux nodes
Figure 8. CSM hardware control attribute values for AIX and Linux nodes
CSM hardware control attribute values for AIX and Linux nodes (continued)
Figure 9. continued, CSM hardware control attribute values for AIX and Linux nodes
6 views