Softpanorama

May the source be with you, but remember the KISS principle ;-)
Home Switchboard Unix Administration Red Hat TCP/IP Networks Neoliberalism Toxic Managers
(slightly skeptical) Educational society promoting "Back to basics" movement against IT overcomplexity and  bastardization of classic Unix

HP iLO 4

News HP Servers Recommended Links Command line interface ILO TCP ports Firmware upgrade
Using ILO virtual CDROM  How do I reset the HP iLO IP address and password Connecting to HP iLO via SSH iLO 3 -- HP engineering fiasco Security Serial console
Dell DRAC Administration of Remote Servers  HP Smart Array P410 controller Sysadmin Horror Stories Humor Etc

Introduction

There are two major methods of Out-of-Band (OOB) Management for servers:

ILO is an old product which is updated by HP with each new generation of servers. 

HP upgrades the capabilities of its management processors with approximately each second generation of servers/blades.   “G8” and “G9” server and blades use iLO4. A quick list of features of iLO4 includes:

Licensing is complicated and really is IBM-style mess. See

You need advanced license (see HPE iLO Advanced Licensing )  in order to be able to install OS using ISO on the HTTP server. I hate HP for that.  60 day trial license is available.

Documentation looks like a translation  from some foreign language into English ;-)

Seasoned sysadmins do not expect from vendor documentation much. We are trained to work with manuals written by people who do not understand the product and do not care, but in this area HP still stands out.

Add to this the fact that dignistics is really bad.

For example HP ProLiant Integrated Lights-Out 3 v 1.20 User Guide (HP part number 616301-003) some paragraphs looks like translation from some other language to English, which is an innovative trend, I would say a real breakthrough,  for a large US company. As a result some quotes can well be submitted to the Onion (p. 85)

To use a physical CD/DVD-ROM drive in your client PC:
1. Select IRC within the Remote Console section. --[There is no IRC section in Remote control section]
2. Select the Virtual Drive tab --[There is no Virtual tab section]

and then select the drive letter of the desired physical CD/DVD-ROM --[There is no drop-menu with drive letters to select from] drive on your client PC from the drop-down menu.

The trick to understand this paragraph is that they are talking about a tab named "Virtual drive" on the remote console screen, not about Remote console/Remote console section of ILO3 main menu :-). Please note that there is no IRC tab on remote console screen either, only "Virtual drive" tab.

Another interesting tidbit is that ILO timeouts no matter what setting you put at Administration/Access/settings. So it takes your current setting for advisory and then does not follow up :-). If you installing OS on a remote server that might have an interesting effect on your mood as logout persistently resets virtual CD/DVD and virtual floppy drive.  I agree that it provide an excellent  security of this feature in a sense of making it impossible to use.  As we all agree that security is important, that well might be implicit HP design goal, although just dropping this feature would be an equally secure and less frustrating solution. .

Additional cost of Advance Licence which is needed for attaching ISO via HTTP server

HP servers are more expensive then servers of competitors, such as Dell. So it is naturally to expect from them more. And HP does include ILO in all of its servers by default; so it's cost is implicitly reflected in the price if, for example you compare the cost of HP server and equivalent Dell server.

But at some point HP brass became way too greedy and decided to "correct" this situation. The method they found is pretty interesting.

Unlike Dell DRAC, remote console capabilities are not free. HP demands additional license fee (around $300 for license with one year support) for the ability to use mount ISO from HTTP.  

In any case the price structure HP adopted is somewhat questionable. IMHO there should be no software license at all, as this is a specialized appliance, not a general purpose device.

For example Dell charges for hardware (Drac card) which is more logical: $99 for iDRAC6 express hardware, $349 hardware with enterprise version installed and $448 for version with 8GB SD card (this is 2011 prices without corporate discount).  As we can see the price of the device is suspiciously close to the price of the advanced license.  truth be told HP provides 60 evaluation license, but this is not enough to correct the situation.

Moreover, much chaeper DRAC (Drac cost in one time, HP advance license is a subsctiotion with per year costs) is much better product which provides some functionality that ILO currently does not have. for example ability to manipulate the *GB flash card, the size enough for putting DVD image into it and that allow you to avoid a slow connection problems when booting the server from the image. DRAC the ability to send SMTP alerts. And the ability to send email alerts directly to sysadmin has great value in a typical enterprise environment if we take into account the amount of red type necessary for accomplishing the same via monitoring system such as HP Operations Manager. 

ILO virtual DVD capabilities

They are really useful for booting an OS.  But ILO virtual DVD capabilities when it uses DVD on your PC are not very stable (and slow if you use VPN), They suffer from timeouts, which makes unattended installation problematic. For this pupose they are usable mostly for booting installer (after that you can get ISO image from a nearby server via FTP or HTTP).

Attaching ISO as virtual CDROM from a Web server

If you have advanced license and are attaching virtual CD/DVD drive make sure that you iether use IP address or configure DNS server on ILO. There is no diagnostics for "unvalid URL"

One time book should be set to virtual CDROM.

Red Hat Linux

On servers that have a locally attached IDE CD/DVD-ROM, the Virtual CD/DVD-ROM

device is accessible at /dev/cdrom1. However, on servers that do not have a locally

◦

attached CD/DVD-ROM, such as BL c-Class blade systems, the Virtual CD/DVD-ROM is

the first CD/DVD-ROM accessible at /dev/cdrom.

You can mount the Virtual CD/DVD-ROM as a normal CD/DVD-ROM device by using

the following command:

mount /mnt/cdrom1

Mounting a USB Virtual Media CD/DVD-ROM on Linux systems

1. Log in to iLO through the web interface.

2. Start the .NET IRC or Java IRC.

3. Select the Virtual Drives menu.

4. Select the CD/DVD-ROM to use.

5. Mount the drive by using the following commands:

For Red Hat Linux:

mount /dev/cdrom1 /mnt/cdrom1

For SLES:

mount /dev/scd0 /media/cdrom1

Over-engineered means unsecure

ILO is complex and as such is an extremely attractive target for state-supported hackers as in most corporation ILO are not well protected (generally it should be put on special "ILO-only" segment)  and this vector of attack is typically overlooked.

From the security standpoint ILO represents a perfect hidden backdoor to your server for state-supported hackers. Nothing more, nothing less.

Again, ILO should be secured by using a separate protected by firewall segment. For remote installations usage only VPN should be allowed.  Which of course means additional cost and complexity.

But please don't overlook this vector of attack. Due to overcomplexity of the codebase, state-supported hackers can breach built-in security as easy as  knife enters the butter.  If you need to protect corporate assets from this type of hackers in no way you can rely on built-in security features.

If you change password on Administrator account put a sticker on the server card, if access to the server is protected

On the first login create at least one other user account in addition to Administrator account.  This is an insurance that if you change Administrator password and forget it you still will be able to access ILO.  At least two additional accounts are recommended (for primary and secondary sysadmin of the server or blade enclosure).

If sysadmin leaves, and ILO credentials are not documented, you need a reboot  of the server to rest admin password. The passowrd supplied with the server or enclosure is printed on the card in the back nd as such self-documented.  If you change it put a sticker with the new password on the card if access to the server is protected (datacenter or server room with a lock)

Increase timeout to "Indefinite"

In Administration/Access setting screen set "Idle connection Timeout drop-down list to "Indefinite". Go to the Administration/Network tab and click apply. I think this setting is activated only on reboot of ILO, while new user setting does not require reboot. This is really Byzantine peace of equipment.
 

  1. Logout explicitly as you finished using ILO.

Initial setup

  1. Restart or power the server on (Important: only removing power from power supplies resets ILO).

  2. Press the F8 key when ILO prompt appears during POST.

  3. Select Network>DNS/DHCP, press the Enter key, and then select DHCP Enable. Press the spacebar to turn off DHCP. Be sure that DHCP Enable is set to Off, and save the changes.

  4. Select Network>NIC>TCP/IP, press the Enter key, and enter the appropriate information in the IP Address, Subnet Mask, and Gateway IP Address fields.

  5. Save the changes.

  6. Additional recommended settings:
    • Idle connection timeout (minutes) -- infinite
    • Require Login for iLO RBSU -- Disabled.
    • Serial Command Line Interface Status - Enabled - No Authentication

Applying firmware upgrade

You can use iLO Online updates for Windows and linux. Packages have different extensions (.exe for Windows and .scexe for Linux). Update can be applied in two ways:

Since web based firmware update supports only the firmware image file (.BIN file) it is important not to forget to extract the .BIN file from the firmware package first. The .BIN firmware update file is not available as a direct download option at HP.com

Steps

  1. Download iLO online firmware update package
  2. Extract the .BIN file from it. For Linux .scexe file can be extracted using command:
    sh *.scexe --unpack=directory
    For example 
    sh CP015458.scexe --unpack=directory.
  3. Load .bin file it into ILO using ILO browser

see Firmware upgrade for details

On Linux you can install it from the SCEXE file. Which actually opens an interesting backdoor for Troyanizing your servers :-).

To update firmware from the Linux operating system on target server:
Download the SCEXE file to the target server.

In older version this method worked on  servers but did not work on blades: installation used to freeze with the message "

Flashing is underway... 1 percent programmed.

but it does not destroy the flash ROM. You can still reboot ILO and do it via remote interface

Be careful with executing  CP016203.scexe in Linux on blades. It might fail and then what?

The most reliable way to upgrade firmware is to download the SCEXE file to a client running a Linux operating system. Execute: 

sh CP016203.scexe --unpack=directory.

This command will unpack the iLO3 bin into a user specified "directory". If the directory does not exist, the unpacker will attempt to create it. Then move .bin file to your windows client and do upgrade from ILO Web interface. See Uploading .bin file via ILO3 Web interface

The most reliable way is to download the SCEXE file to a client running a Linux operating system. Execute: 
sh CP016203.scexe --unpack=directory.

This command will unpack the iLO3 bin into a user specified "directory". If the directory does not exist, the unpacker will attempt to create it. Then move .bin file to your windows client and do upgrade from ILO Web interface. See Uploading .bin file via ILO3 Web interface

 

iLO Firmware update from USB drive

In case you screw things up you can use HP Smart Update Firmware DVD of the server to update the iLO firmware. To use HP Smart Update Manager on the Firmware Maintenance CD:

Rebooting the iLO

The blunders HP committed would be less biting if there were a possibility to reboot ILO separately from server using some hardware button. But there no such capability for stand-alone HP servers. Please note that there is such capability for blades.

HP  communicates this blunder to user in a very interesting way:

       You can reboot ILO

1) By removing all the power cables from the server.

2) By changing the system maintenance switch 1 on the system board (DL 580). That's convoluted operation:

Power the server OFF.
Disconnect the power cord from the server.
Remove the access panel.
Remove the controller and the riser board.
Push and hold the power button down for about a minute.
Remove and reseat all the memory DIMMS.
Disconnect and reconnect the VGA cable from the server.
Set the System maintenance switch 1 to the ON position.
Power the server back ON.
After the server has completed the Power-On Self Test, power the server OFF (If display is present).
Set the System maintenance switch 1 back to the OFF position.
Power the server back ON.

Top Visited
 Switchboard
Latest
Past week
Past month

NEWS CONTENTS

Old News ;-)

[Dec 12, 2013] Advisory HP Integrated Lights-Out 3 and HP Integrated Lights-Out 4 - False CPU Overheating Messages May Be Logged

Any HP ProLiant G7 and Gen8 series servers configured with Intel processors; iLO 3 Firmware Version 1.61 (or earlier) and iLO 4 Firmware Version 1.30 (or earlier).

RESOLUTION

To prevent the false messages from occurring, update the iLO firmware as follows:

Update the iLO 3 firmware to version 1.65 (or later):

Perform the following steps to obtain the latest iLO 3 firmware version:

  1. Click on the following link:

    http://www.hp.com/support/iLO3

  2. Select the appropriate operating system.
  3. Click on "Firmware - Lights-Out Management."
  4. Locate, download, and install iLO 3 Firmware Version 1.65 (or later).

Update the iLO 4 firmware to version 1.32 (or later):

Perform the following steps to obtain the latest iLO 4 firmware version:

  1. Click on the following link:

    http://www.hp.com/support/iLO4

  2. Select the appropriate operating system.
  3. Click on "Firmware - Lights-Out Management."
  4. Locate, download, and install iLO 4 Firmware Version 1.32 (or later).

Firmware update problem during remote update

HP Smart Update Manager (HP SUM) - HP Smart Update Manager (SUM) 5.x - Unable to Update HP ProLiant iLO3 Firmware from 1.2x to 1.5.

If using HP SUM bundled in the HP Support Pack for ProLiant (SPP) in order to update the HP ProLiant iLO3 firmware from 1.2x to 1.50 remotely, the update will fail.

The installation will report as failed, and the HP SUM error log will show the following:

====== HP Smart Update Manager Installation Log ====== Starting ILO firmware update. Initializing connection to ILO 192.168.10.1. Connection established to ILO. Checking the type of ILO. load -source http://192.168.10.20:63000/Rep1/cp016202/3.bin status=1 status_tag=COMMAND SPAWNED Mon Jan 14 11:33:27 2013 Attempting to load iLO3 firmware image. status=2 status_tag=COMMAND PROCESSING FAILED error_tag=COMMAND ERROR-UNSPECIFIED Mon Jan 14 11:33:27 2013 Unable to retrieve a valid iLO3 firmware image. Check file path and login credentials.

This issue is due to a bug in the iLO3 firmware.

HP SUM executes the load -source command from iLO3 SSH console, but when the iLO3 web pooler proceeds to download the firmware file, it will fail.

Therefore, it is an iLO3 issue and not with HP SUM.

NOTE: This issue does not happen if running HP SUM locally, only when trying remote updates.

Solution

This issue does not occur in 1.1x or 1.50 and later firmware. llpadding="0" width="100%">
NOTE: In order to update iLO3 to 1.50, the iLO3 firmware must be in the 1.2x versions.
As a workaround, use the following options:
  1. Run the iLO3 firmware update locally on the server. The firmware will be transferred via the iLO3 driver.
  2. If need to update several systems remotely, use the iLO RIBCL XML scripts. Use the script Update_Firmware.xml as a template.

HP Integrated Lights-Out 3 Firmware upgrade version 1.5

Type: Firmware - Lights-Out Management
Version: 1.50 (26 Oct 2012)
Operating System(s): Red Hat Enterprise Linux 5 Desktop (x86), Red Hat Enterprise Linux 5 Desktop (x86-64), Red Hat Enterprise Linux 5 Server (x86), Red Hat Enterprise Linux 5 Server (x86-64), Red Hat Enterprise Linux 6 Server (x86), Red Hat Enterprise Linux 6 Server (x86-64), SUSE Linux Enterprise Server 10 (AMD64/EM64T), SUSE Linux Enterprise Server 10 (x86), SUSE Linux Enterprise Server 11 (AMD64/EM64T), SUSE Linux Enterprise Server 11 (x86), VMware ESX/ESXi 4.0, VMware ESX/ESXi Server 3.5

The Command Line Interface (CLI) Command to Create a New User May Not Function in iLO 3 Firmware Version 1.26 or Version 1.28

SUPPORT COMMUNICATION - CUSTOMER ADVISORY
Document ID: c03573824

Version: 1

Advisory: HP Integrated Lights-Out 3 (iLO 3) - The Command Line Interface (CLI) Command to Create a New User May Not Function in iLO 3 Firmware Version 1.26 or Version 1.28
NOTICE: The information in this document, including products and software versions, is current as of the Release Date. This document is subject to change without notice.
Release Date: 2012-11-14

Last Updated: 2012-11-14

[ Dec 20, 2012 ] Advisory (Revision) HP Integrated Lights-Out 3 (iLO 3) - The iLO 3 ROM-Based Setup Utility May Incorrectly Display the Gateway

Solution is to upgrade to version 1.5 (October 2012)
12/10/2012

After upgrading HP Integrated Lights-Out 3 (iLO 3) Firmware to Version 1.28 and using the iLO 3 ROM-Based Setup Utility (RBSU) to configure a static IP address using the following steps, the next time that the iLO 3 RBSU Setup Utility is loaded, the iLO 3 Gateway IP address may be incorrectly displayed as 0.0.0.0. This is a "display only" issue in iLO 3 Firmware Version 1.28. No other iLO 3 Firmware versions are affected.

The condition occurs after performing the following steps:

Restart or power on the server. Press the F8 key when prompted during POST. The iLO 3 RBSU runs. Select Network > DNS/DHCP, press the ENTER key, and then select DHCP Enable . Press the spacebar to turn off DHCP. Be sure that DHCP Enable is set to Off, and save the changes. Select Network > NIC > TCP/IP, press the ENTER key, and enter the appropriate information in the IP Address, Subnet Mask, and Gateway IP Address fields. Save the changes. Exit iLO 3 RBSU. The change takes effect after exiting iLO 3 RBSU. SCOPE Any ProLiant server configuration using HP Integrated Lights-Out 3 (iLO 3) Firmware Version 1.28.

RESOLUTION This is a display only issue and can be safely ignored.

HP Integrated Lights-Out 3 (iLO 3) Firmware Version 1.50 (or later) corrects the Gateway IP Address display issue. To access the HP Integrated Lights-Out 3 (iLO 3) driver and software download page, click on the following URL:

http://www.hp.com/support/iLO3

OR

As a workaround, use the iLO 3 web GUI to verify that the Gateway IP address is displaying correctly on the network. The network settings can be checked by looking under Administrator -> Network.

[Jun 25, 2012] If an HP ProLiant Server Resets Unexpectedly, the Integrated Management Log (IML) Should Be Check

Document ID: c03370645

Version: 1

Notice: HP ProLiant Servers - If an HP ProLiant Server Resets Unexpectedly, the Integrated Management Log (IML) Should Be Checked to Determine if the Source of the Reset Is Indicated

NOTICE: The information in this document, including products and software versions, is current as of the Release Date. This document is subject to change without notice.

Release Date: 2012-06-12

Last Updated: 2012-06-12

DESCRIPTION

If an HP ProLiant server resets unexpectedly, the Integrated Management Log (IML) should be checked to determine if the source of the reset is indicated. This information can help determine if any action needs to be taken.

The Integrated Management Log (IML) records hardware events and stores them in a formatted table. It records the time of the event and categorizes events in severity levels such as:

DETAILS

There are numerous conditions that can result in a server unexpectedly resetting. Whenever such a reset occurs, the Integrated Management Log (IML) should be checked to determine if the cause of the reset is indicated. While the IML will not always indicate the cause of an unexpected reset, it will in most cases.

The IML can be viewed from the HP Integrated Lights-Out (iLO) web page or using other HP Management Tools. Some of the most commonly used methods are:

IML Viewer:

  1. Click Start->All Programs -> HP System Tools-> HP ProLiant Integrated Management Log Viewer.
  2. The IML Viewer will open and the IML logs will be displayed on the screen.

From HP Integrated Lights-Out (iLO):

  1. Open the iLO web interface.
  2. Enter the iLO Login name and Password.
  3. From the left menu, access IML Logs .

From the System Management Homepage (SMH):

  1. Click Start-> All Programs -> HP Management Agents -> HP System Management Homepage.
  2. Click on Logs . The log screen will be displayed.
  3. Click on Integrated Management Log .

[Jun 01, 2012] Systematic failure during installation of version 1.29 on HP blades

Only blades are affected. Standalone servers like , DL360, DL380, DL580 are OK. 
# ./CP016462_ILO3_1_28.scexe

FLASH_iLO3 v1.06 for Linux (Jan 17 2012)
(C) Copyright 2002-2011 Hewlett-Packard Development Company, L.P.
Firmware image: ilo3_128.bin
Current iLO 3 firmware version  1.26; Serial number ILOUSE0504TPN

Component XML file: CP016462.xml
CP016462.xml reports firmware version 1.28
This operation will update the firmware on the
iLO 3 in this server with version 1.28.
Continue (y/N)?y
Current firmware is 1.26 (Aug 26 2011 )
Firmware image is 0x800000(8388608) bytes
Committing to flash part...
******** DO NOT INTERRUPT! ********
/
Channel Interface transactions (Linux) returns 21!
Channel Interface call status: FIFO empty.
ERROR: received errcode 21
/
Channel Interface transactions (Linux) returns 21!
Channel Interface call status: FIFO empty.

Failed(5-21)!

ERROR: Unable to commit flash. [ilo3_128.bin]

[May 15, 2012] HP Integrated Lights-Out 3 (iLO 3) Version 1.25 (or Earlier) - iLO Virtual Media Service May Randomly Stop Responding When Attempting to Connect a Virtual Device in Remote Console

c03316654

Products: HP ProLiant DL Servers, HP ProLiant BL Server Blades, HP ProLiant Scalable Systems, Insight Control Software, HP ProLiant ML Servers

Description: iLO Virtual Media Service May Randomly Stop Responding When Attempting to Connect a Virtual Device in Remote Console

[May 15, 2012] Disabling an iLO 3 NIC May Cause the iLO 3 Link Status to be Incorrectly Displayed as Failed in HP System Management Homepage and HP Systems Insight Manager (HP SIM)

c03327392

Products: Software, ProLiant Servers, BladeSystem

Description: Advisory: HP Integrated Lights-Out 3 (iLO 3) - Disabling an iLO 3 NIC May Cause the iLO 3 Link Status to be Incorrectly Displayed as Failed in HP System Management Homepage and HP Systems Insight Manager (HP SIM)

[May 15, 2012] iLO 3 Firmware Flash Progress May Intermittently Stop at One Percent or Take Sev

HP

Document ID: c03318935

Version: 1

Advisory: HP Integrated Lights-Out 3 (iLO 3) - iLO 3 Firmware Flash Progress May Intermittently Stop at One Percent or Take Several Hours to Complete

NOTICE: The information in this document, including products and software versions, is current as of the Release Date. This document is subject to change without notice.

Release Date: 2012-05-07

Last Updated: 2012-05-07

DESCRIPTION

While upgrading the HP Integrated Lights-Out 3 (iLO 3) from any previous version prior to version 1.28 (or later), the firmware flash progress may intermittently stop at one percent or take several hours to complete. This occurs due to idle Secure Shell (SSH) connections that are incorrectly closed and then time-out, causing the iLO 3 CPU to become busy and stall or delay the firmware flash progress.

An error in the way SSH sessions are shut down causes iLO 3 to assume that the sessions are fully active and constantly transferring data, delaying the firmware flash progress. HP Systems Insight Manager (HP SIM) servers and other SSH management tools located on the same network as iLO 3 may open these SSH connections.

SCOPE

Any HP ProLiant server with Integrated Lights-Out 3 (iLO 3) Firmware Version 1.26 (or earlier).

RESOLUTION

This is resolved by upgrading Integrated Lights-Out 3 (iLO 3) to firmware version 1.28 (or later).

Note : The issue may occur while upgrading to version 1.28 (or later) if upgrading from a version of the firmware prior to version 1.28.

Until the firmware upgrade is installed, reset iLO 3 to clear the SSH connection state prior to flashing the firmware.

If the firmware flash progress has stopped, connect several simultaneous SSH sessions until no additional SSH connections are allowed. Then close them normally to reset the connection state and allow the flash progress to continue.

Perform the following steps to obtain the latest iLO 3 firmware version:

  1. Click on the following link:

    http://www.hp.com/support/iLO3

  2. Select the appropriate operating system.
  3. Click "Firmware - Lights-Out Management."
  4. Locate, download, and install iLO 3 Firmware Version 1.28 (or later).

[May 07, 2012] ILO3 FW 1.26-1.28 Drive Information Shows Only Physical Driver Stat

For ILO3 with firmware 1.26 and 1.28, web GUI and CLI does not show rebuilding status for Drive information .

Web GUI and CLI show only OK or Not installed .

[May 07, 2012] Integrated Lights-Out 3 (iLO 3) FIRMWARE UPGRADE REQUIRED to Prevent Unexpected Server

IMPORTANT : The Integrated Lights-Out 3 (iLO 3) firmware upgrade provided in the Resolution is required to prevent unexpected shutdowns, false CPU clock throttled messages, or incorrect values displayed via the PPIC command. HP recommends performing this upgrade at the customer's earliest possible convenience. Neglecting to perform the recommended iLO 3 firmware upgrade could result in the potential for subsequent errors to occur.

On an HP ProLiant SL390s G7 server, after upgrading the Integrated Lights-Out 3 (iLO 3) firmware to version 1.26, the server may experience unexpected shutdowns, false CPU clock throttled messages displayed in the operating system console, or incorrect values displayed via the ProLiant Power Interface Configuration (PPIC) command.

SCOPE
Any HP ProLiant SL390s G7 server (2U or 4U) after upgrading the Integrated Lights-Out 3 (iLO 3) firmware to version 1.26.

RESOLUTION
To prevent these issues from occurring, upgrade the Integrated Lights-Out 3 (iLO 3) to Firmware Version 1.28 (or later).

[May 07, 2012] HP Integrated Lights-Out 3 Version 1.25 (or Earlier) Browser Interface and Secure Shell (SSH) May Stop Responding When Attempting to Login to SSH Using a Private SSH-DSA Key Larger Than 2048 Bits

May 03 2012 | c03315526

Products: HP ProLiant BL Server Blades, Insight Control Software

Description: Advisory: HP BladeSystem ProLiant Server Blades - HP Integrated Lights-Out 3 Version 1.25 (or Earlier) Browser Interface and Secure Shell (SSH) May Stop Responding When Attempting to Login to SSH Using a Private SSH-DSA Key Larger Than 2048 Bits

HP Integrated Lights-Out 3 (iLO 3) - HP Proliant G7 Blade servers - iLO 3 Firmware version 1.25 Server Power On Issue Caused by Overprovisioning of Power

Resolution is to apply firmware version 1.26.

HP Proliant G7 Blade servers might exibit the issue where these servers are not powering On by default any longer when they have HP iLO firmare 1.25 applied.

The issue is that with iLO 3 firmware 1.25 some half height HP Proliant Blade servers get assigned 509 W of power and full height servers get 1800 W of power assigned. Sometimes, it is on an average of 60% more then what it should be.

See the screenshot below for visual indicator...

[May 07, 2012] Advisory HP ProLiant Servers - Integrated Lights-Out 3 and 4 (iLO 3-iLO 4) User Guides Incorrectly State That New Certificate Is Always Created When iLO Is Reset

2012-04-30

The HP Integrated Lights-Out 3 and 4 (iLO 3/iLO 4) User Guides incorrectly state that a new certificate is always created each time that iLO is reset (Diagnostics -> reset on iLO). For example, the iLO 3 Version 1.20 User Guide states the following on page 166:

If the iLO 3 self-signed certificate is installed permanently into some browsers and the iLO 3 is reset, you might not be able to log back in to iLO 3 because iLO 3 generates a new self-signed certificate every time it is reset.

HP ProLiant Integrated Lights-Out 3 Version 1.20 User Guide
http://h20000.www2.hp.com/bc/docs/support/SupportManual/c02774507/c02774507.pdf

Any HP ProLiant server with Integrated Lights-Out 3 or 4 (iLO 3/iLO 4)

[Apr 26, 2012] HP Lights-Out XML PERL Scripting Sample for Linux (ver 4.00.0)

Products: BladeSystem, Options and Accessories, ProLiant Servers, Software

OS: Other Legacy OS,Linux

Description: This is a set of sample XML scripts used to manage the configuration of Integrated Lights-Out (iLO) management processors and to control servers in which iLO devices are in use. Use LOCFG.PL (available in this bundle) to run the XML.

[Apr 16, 2012] Advisory: HP Integrated Lights-Out 3 (iLO 3) - The iLO 3 ROM-Based Setup Utility May Incorrectly Display the Gateway IP Address as 0.0.0.0 After Upgrading iLO 3 to Firmware Version 1.28 (c03283274)

Products: HP ProLiant DL Servers, HP ProLiant BL Server Blades, HP ProLiant ML Servers, HP ProLiant Scalable Systems, Insight Control Software

Description: Advisory: HP Integrated Lights-Out 3 (iLO 3) - The iLO 3 ROM-Based Setup Utility May Incorrectly Display the Gateway IP Address as 0.0.0.0 After Upgrading iLO 3 to Firmware Version 1.28

Advisory ProLiant Servers Integrated Lights-Out 3 - FIRMWARE UPGRADE RECOMMENDED to Avoid Java and Internet Explorer Unrespon

Sept 12, 2011

IMPORTANT : The Integrated Lights-Out 3 firmware upgrade provided in the Resolution is recommended to prevent Java and Internet Explorer from becoming unresponsive when attempting to acquire an already open Integrated Lights-Out 3 Remote Console session by opening the Java Remote Console applet. HP recommends performing this upgrade at the customer's earliest possible convenience. Neglecting to perform the recommended action and not performing the recommended resolution could result in the potential for subsequent errors to occur.

Attempting to acquire an already open Integrated Lights-Out 3 (iLO 3) Remote Console session by opening the Java Remote Console applet may cause Java and Internet Explorer to stop responding when running iLO 3 firmware version 1.20 (or earlier).

Any ProLiant server with Integrated Lights-Out 3 Version 1.20 (or earlier).

[Jan 02, 2012] Connecting to HP iLO using the command line by Emerson Takahashi

July 17, 2010 | setaOffice

Just ssh to the IP that you configured the iLO

emerson@shellcore:~ $ ssh [email protected]
[email protected]′s password:
User:Emerson Takahashi logged-in to ILO_TESTLABHP.(192.168.50.118)
iLO 2 Advanced Evaluation 1.81 at 11:05:47 Jan 15 2010
Server Name: proliant_g5
Server Power: On

hpiLO->

Since I will access through a text terminal, Linux is configured to use the serial port (configured through the file /boot/grub/grub.conf)

hpiLO-> help
status=0
status_tag=COMMAND COMPLETED

DMTF SMASH CLP Commands:

help : Used to get context sensitive help.
show : Used to show values of a property or contents of a collection target.
create : Used to create new user account in the name space of the MAP.
Example: create /map1/accounts1 username= password=
name=
group=

delete : Used to delete user account in the name space of the MAP.
Example: delete /map1/accounts1/

load : Used to move a binary image from an URL to the MAP. The URL is
limited to 80 characters
Example : load -source http://192.168.1.1/images/fw/iLO2_130.bin

reset : Used to cause a target to cycle from enabled to disabled and back to enabled.

set : Used to set a property or set of properties to a specific value.
start : Used to cause a target to change state to a higher run level.
stop : Used to cause a target to change state to a lower run level.
cd : Used to set the current default target.
Example: cd targetname

exit : Used to terminate the CLP session.
version : Used to query the version of the CLP implementation or other CLP
elements.

oemhp_ping : Used to determine if an IP address is reachable from this iLO 2.
Example : oemhp_ping 192.168.1.1 , where 192.168.1.1 is the IP address that you wish
to ping

oemhp_loadSSHKey : Used to authorize a SSH Key File from an URL The URL is
limited to 80 characters
Example : oemhp_loadSSHKey -source http://UserName:[email protected]/images/SSHkey1.ppk

HP CLI Commands:

POWER : Control server power.
UID : Control Unit-ID light.
NMI : Generate an NMI.
VM : Virtual media commands.
VSP : Invoke virtual serial port.

Type VSP and you're in. To login as root you need to include the serial port (in this case ttyS1) on your /etc/securetty file or you will be given the error message that your user or password is wrong.

hpiLO-> VSP

Starting virtual serial port.
Press 'ESC (' to return to the CLI Session.

hpiLO-> Virtual Serial Port active: IO=0x02F8 INT=3

login as:

[Nov 01, 2011] What TCP-IP ports does iLO 3 use

Aug 10, 2010 | HP Communities

You can look up what ports are used via the iLO 3 web interface. Expand the "Administration" menu on the left, then click on the "Access Settings" link. That screen will tell you the ports used by the various services.

Here are the defaults:
SSH 22
Web (non-SSL) 80
SSL 443
IPMI-over-LAN 623
Remote Console 17990
Virtual Media 17988

You might also need to enable other ports if you're using DHCP, DNS, SNTP, SNMP, and/or LDAP from iLO.

[Oct 31, 2011] c00257375 Best Practices for Integrated Lights-Out and Integrated Lights-Out 2, 3-rd edition

HP

For an iLO device to work properly when going across routers using port blocking and/or firewalls, ports 23, 80, 443, and 17988 must be open.

The directory services LDAP port (636) may be required. The Terminal Services RDP port (3389) may be required.

Port 23 is for the Telnet ports where the remote and graphical Remote Console is used, port 80 is for HTTP communications, port 443 is required for the HTTPS connection, and port 17988 is for Virtual Media.

LDAP traffic from a directory server uses random port numbers to enter the iLO device.

The inability to access the iLO management ports is often confused with incorrect proxy settings. When in doubt, disable proxy in Internet Explorer or Netscape.

[Oct 31, 2011] Default Port for the remote console for iLO 3

HP Communities

Hi Guys,

We have found that the remote console port defined for iLo3 has changed from being 3389 (standard RDP port) to 17990.

Can one of you please ask HP about the reasoning about this change and if it will be an issue if we change this to the standard 3389 port. The alternative is that we get NS to open the port 17990 on the firewall then we do not have to manually change every iLO 3 interface for servers in ecom.

***************************

David responded:

**************************

I think they're confusing 2 different things. Port #3389 is a standard RDP port and was valid for the "iLO Terminal Services Pass-through" but never was the port for accessing the iLO remote console. Since TS Pass-through is no longer available with iLO3, this doesn't apply.

[Oct 31, 2011] Opening firewall ports for iLO

Aug.20, 2009 | NachoTech

If you want to access an iLO behind a firewall, there are some TCP ports that need to be opened on the firewall to allow all iLO traffic to flow through. Here is a list of the default ports used by iLO, but these can be modified on iLO's Administration… Access… Services… tab.

ILO FUNCTION           SOCKET TYPE PORT NUMBER 
---------------------- ----------- -----------
Secure Shell (SSH)         TCP        22
Remote Console/Telnet      TCP        23
Web Server Non-SSL         TCP        80
Web Server SSL             TCP        443
Terminal Services          TCP        3389
Virtual Media              TCP        17988
Shared Remote Console      TCP        9300
Console Replay             TCP        17990
Raw Serial Data            TCP        3002

[Oct 12, 2011] HP Integrated Lights-Out 3 Version: 1.26 (29 Aug 2011)

Installation:

To update firmware from the Linux operating system on target server:
Download the SCEXE file to the target server. Execute: sh CP015458.scexe

To obtain firmware image for updating via iLO user interface, utilities, or scripting interface:
Download the SCEXE file to a client running a Linux operating system. Execute: sh CP015458.scexe --unpack=directory.

This command will unpack the iLO3 bin into a user specified "directory". If the directory does not exist, the unpacker will attempt to create it.

To use HP Smart Update Manager on the Firmware Maintenance CD:

[Oct 11, 2011] Reset ILO (Integrated Lights-Out 2) on HP Server

Stracca Blog

Recently I had the necessity to reset the ILO interface of an HP Proliant Server.
I found that you need to connect in ssh (or in telnet) to do it.
One connect give this commands:

cd /Map1
reset

Here an example:

User:admin logged-in to ILOGB87451B7E(10.1.1.15)
iLO 2 Advanced 1.81 at 11:05:47 Jan 15 2010
Server Name: myserver.mydomain.com
Server Power: On

hpiLO-> cd /Map1
status=0
status_tag=COMMAND COMPLETED

/Map1

hpiLO-> reset
status=0
status_tag=COMMAND COMPLETED
Resetting iLO.

CLI session stopped

[Mar 6, 2011] HP Integrated Lights-Out 3 (iLO 3) - Firmware CD Supplemental Update - Online ROM Flash Component for Linux

Type: Firmware - Lights-Out Management
Version: 1.28 (6 Mar 2012)

Products: HP ProLiant DL Servers, HP ProLiant BL Server Blades, HP ProLiant Scalable Systems, HP ProLiant ML Servers, Insight Control Software
OS: Windows,Linux,Other Legacy OS

Description: This component provides updated iLO firmware that can be installed directly on supported Linux Operating Systems. This component can also be used to obtain the firmware image for updating via iLO user interface, utilities, or through the scripting...

Upgrade Requirement:
Optional - Users should update to this version if their system is affected by one of the documented fixes or if there is a desire to utilize any of the enhanced functionality provided by this version.

Recommended Links

Google matched content

Softpanorama Recommended

Top articles

Sites

HP Integrated Lights-Out - Wikipedia, the free encyclopedia

ILO 4 Type Size Date
HPE iLO 4 User Guide PDF 6.7 MB Oct 2016
HPE iLO 4 Scripting and Command Line Guide PDF 3.0 MB Oct 2016
HPE iLO Federation User Guide PDF 1.0 MB Oct 2016
HPE iLO IPMI User Guide PDF 1.7 MB Oct 2016
HPE Integrated Lights-Out Security Technology Brief PDF 4.3 MB Oct 2016
HPE iLO Mobile iOS Application User Guide PDF 1.0 MB Aug 2016
HPE iLO Mobile Application for Android User Guide PDF 835 KB Aug 2016
HPE iLO Licensing Guide PDF 103.9 KB Jul 2017

HP Integrated Lights-Out 3 (iLO 3) - Manuals - HP Business Support Center

Proliant Watch Configuring HP Integrated Lights-Out (iLO) - HP Proliant Server

NachoTech Opening firewall ports for iLO

Setting up ssh tunnel to access a distant iLO - Tin's Journey

Random Findings

Topic: Configuring iLO2 and Debian with serial support

Starting a text-mode Debian install using the virtual serial port (VSP) via an ssh connection to iLO 2

no remcons in ilo2?

iLO 2

You will have to start an agetty process on the COM2 port in order to use the VSP. RHEL 5 example: 

S1:2345:respawn:/sbin/agetty 115200 ttyS1 vt100

In order to see the startup/shutdown messages on the VSP, add the following to the appropriate kernel line in /boot/grub/menu.lst: 

...
kernel /vmlinuz-2.6.18-8.el5 ro root=LABEL=/ console=tty0 console=ttyS1,115200
....

OpenSSH problems with iLO ssh server

The OpenSSH client on my Ubuntu 7.10 system is incompatible with the ssh server on an iLO system with 1.91 firmware. There may be problems with other OpenSSH/iLO versions. 

$ ssh -V
OpenSSH_4.6p1 Debian-5build1, OpenSSL 0.9.8e 23 Feb 2007

$ echo "QUIT" | nc iLO 22
SSH-2.0-mpSSH_0.0.1

$ ssh admin@iLO
admin@iLO's password: 
dispatch_protocol_error: type 100 seq 9
dispatch_protocol_error: type 100 seq 10
buffer_get_ret: trying to get more bytes 4 than in buffer 0
buffer_get_int: buffer error

As a workaround, use PuTTY.

Using Virtual Media with the Command-Line Interface

The iLO 2 Virtual Media Java applet does not work well over a low-speed WAN[1]. My Linux ISOs would routinely not boot when using this Java applet.

As an alternative, ISOs may be loaded from an HTTP server using Virtual Media with the iLO command-line interface.

Example: 

hutch@hutch:~$ ssh admin@iLO
admin@iLO's password:
User:admin logged-in to iLO(10.215.14.5)
iLO Advanced 1.42 at 08:37:01 Oct 03 2007
Server Name: CZC7124NST00
Server Power: On

</>hpiLO-> vm cdrom insert http://10.215.0.35/kickstart/boot_isos/5Server-i386_boot.iso

(Note: use IPs when specifying an HTTP server)

</>hpiLO-> vm cdrom get
VM Applet = Disconnected
Boot Option = NO_BOOT
Write Protect = Yes
Image Inserted = Connected
Image URL = http://10.215.0.35/kickstart/boot_isos/5Server-i386_boot.iso

(Note: the "NO_BOOT" means that the system will not boot off the "connected" image)

</>hpiLO-> vm cdrom set boot_once

(Note: The next boot will be from the connected image) 

</>hpiLO-> power reset

You will likely want to start a Remote Console via the iLO 2's HTTPS interface.

Virtual Power Options

From HP Integrated Lights-Out 2 User Guide for Firmware 1.35:

Retrieved from "http://brandonhutchinson.com/wiki/ILO_Notes"

Linux-BSD SysAdmin and Oracle DBA Guide How to login from serial port under Linux

How to login from serial port under Linux

Environment: Linux server A with at least one com port which is com1
Objective: enable login through com1 using null modem cable from another machine B which can be Windows or Linux.

Steps:

1. connect null modem cable between A and B at com1 port
2. on Linux server A, vi /etc/inittab and append the followin line:

s0:2345:respawn:/sbin/agetty 115200 ttyS0 vt100

3. If machine B is Linux, use minicom to configure the serial port speed as 115200, then connect
4. If machine B is Windows, use putty to directly connect to serial port, set speed as 115200, then login

note:

for HP Proliant DL serial servers, you can also use VSP (Virtual Serial Port) with ILO2, you can basically ssh into ILO IP address with Administrator login. Add the following lines to /etc/inittab

sx:2345:respawn:/sbin/agetty 115200 ttyS1 vt100

then run 'init q' to enable it, after that, you can use vsp command to connect to this serial port login.

FAQ:

1. how to change speed? - best practise.
vi /etc/inittab , for example, change 115200 to 9600, then comment out the line first, run 'init q' to re-read file, after that, uncomment it, issue 'init q' again.

2. how to enable root login through serail port?
Just add ttyS0 or ttyS1 into /etc/securetty.

3. How to use HP virtual serial port(VSP) to login for HP Proliant servers

use ssh connect to ILO ip address:(if there's firewall in between, enable port 22)
for ILO version 1, just run 'REMCONS'.
for ILO2 , type 'vsp' to connect to ttyS0 or ttyS1 to get console screen.

References:

1. HP Proliant server VSP documentation
http://h20000.www2.hp.com/bc/docs/support/SupportManual/c00263709/c00263709.pdf

2. Redhat knowledgebase

How do I set up a serial terminal and/or console in Red Hat Enterprise Linux?

http://kbase.redhat.com/faq/docs/DOC-7213

Useful Commands:
1. setserial -a /dev/ttyS0

Posted by Jephe Wu at 9/24/2009 01:57:00 PM

Labels: console, serial port

Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotes :  Somerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose Bierce :  Bernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOS :  Programming Languages History : PL/1 : Simula 67 : C : History of GCC development :  Scripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-Month :  How to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D

Copyright © 1996-2021 by Softpanorama Society. www.softpanorama.org was initially created as a service to the (now defunct) UN Sustainable Development Networking Programme (SDNP) without any remuneration. This document is an industrial compilation designed and created exclusively for educational use and is distributed under the Softpanorama Content License. Original materials copyright belong to respective owners. Quotes are made for educational purposes only in compliance with the fair use doctrine.

FAIR USE NOTICE This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available to advance understanding of computer science, IT technology, economic, scientific, and social issues. We believe this constitutes a 'fair use' of any such copyrighted material as provided by section 107 of the US Copyright Law according to which such material can be distributed without profit exclusively for research and educational purposes.

This is a Spartan WHYFF (We Help You For Free) site written by people for whom English is not a native language. Grammar and spelling errors should be expected. The site contain some broken links as it develops like a living tree...

 You can use PayPal to to buy a cup of coffee for authors of this site

Disclaimer:

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: March 12, 2019