Nagios XI: Automating Host Management

I was recently looking at how to automate adding and removing managed hosts and services in Nagios XI, which can be particularly useful in cloud computing and large environments where configuration management solutions have been implemented for provisioning. In these environments we typically use configuration files based on the attributes of a server role during the provisioning and configuration cycle.

Nagios XI contains a number of scripts in the directory /usr/local/nagiosxi/scripts that allow for automated host management, as below:

Script Description
reconfigure_nagios.sh Imports configuration files from the import directory, verifies configuration and restart Nagios if verification succeeds . If verification fails, configuration will be rolled back to the last working checkpoint. This is the command invoked from the web interface when selecting ‘Apply Configuration’.
nagiosql_delete_host.php Removes a host from the configuration database and removes the configuration file.
nagiosql_delete_service.php Removes services from the configuration database and removes the configuration file.

In order to automate adding managed hosts and services the method used was to create a single configuration file for a each host and each of its services to which service definitions are are only applied to a that host and not to a host list or host group and to name the configuration file according to the hostname. In the below example, I have created a single configuration file which defines the host and a managed service for CPU Usage and saved the configuration file as ‘server1.dean.local.cfg’.

define host {
 host_name server1.dean.local
 use xiwizard_windowsserver_host
 address server1.dean.local
 max_check_attempts 5
 check_interval 5
 retry_interval 1
 check_period xi_timeperiod_24x7
 notification_interval 60
 notification_period xi_timeperiod_24x7
 icon_image win_server.png
 statusmap_image win_server.png
 _xiwizard windowsserver
 register 1
 } 

define service {
 host_name server1.dean.local
 service_description CPU Usage
 use xiwizard_windowsserver_nsclient_service
 check_command check_xi_service_nsclient!!CPULOAD!-l 5,80,90
 max_check_attempts 5
 check_interval 5
 retry_interval 1
 check_period xi_timeperiod_24x7
 notification_interval 60
 notification_period xi_timeperiod_24x7
 _xiwizard windowsserver
 register 1
 }

Once the configuration file has been created we can place the file in the import directory located at ‘/usr/local/nagios/etc/import’ and invoke the script reconfigure_nagios.sh from the directory ‘/usr/local/nagiosxi/scripts’ to import the configuration file, verify the configuration and restart Nagios if successful. If the verification of the configuration fails, Nagios XI will restore the configuration files to the last working checkpoint but the imported configuration file will remain in the configuration database. In order to detect failures the following exit codes are returned where an exit code of ‘0’ to confirm that the configuration file has been successfully verified as a working configuration and Nagios has been restarted.

Exit Code Description
0 no problems detected
1 config verification failed
2 nagiosql login failed
3 nagiosql import failed
4 reset_config_perms failed
5 nagiosql_exportall.php failed (write configs failed)
6 /etc/init.d/nagios restart failed
7 db_connect failed

Now that we have added a managed host and services, how do we remove this the configuration database and delete the configuration file once the host is terminated? Providing the host has no dependent relationships we can firstly remove the services using the configuration name which matches the configuration file of the managed host (this is why it is important to name the configuration file according to hostname) and invoke the ‘nagiosql_delete_service.php’ from the directory ‘/usr/local/nagiosxi/scripts’ as the below example:

./nagiosql_delete_service.php --config=server1.dean.local

After the services have been successfully deleted we can remove the host by invoking the ‘nagiosql_delete_host.php’ script:

./nagiosql_delete_host.php --host=server1.dean.local

Once the host has been successfully removed, we can apply the new configuration as previosuly by invoking the ‘nagios_reconfigure_sh’ script. This method can also be applied to remove an imported configuration from the configuration database if verification of the configuration has failed during an import.

The above describes how to automate adding and removing hosts and services using Nagios XI and can be applied to your configuration management solutions during the provisioning and configuration cycle. In my scenario, I created a number of configuration files based on the attributes of server roles to which can be used as cookbook templates in Chef and using the ‘{node[‘fqdn’]}’ pattern to specify the host name in the definition file and the configuration file name.  I have also compiled PowerShell functions to perform the above which I will discuss in a later post.

Advertisements

Automating the creation of virtual machine hard disks and NTFS mounted volumes for a virtual machine.

As part of an automated delivery of a virtual machine running Microsoft SQL Server I had a requirement to create a number of virtual machine hard disks  and for each disk create a NTFS mounted folder.

In order to achieve this I was required to create two functions one to create the virtual machine hard disks and the second to create the NTFS mounted volume based on the SCSI Port and Target ID of the disk.

Firstly, once the virtual machine has been deployed from a base template, we want to add the virtual machine hard disks with the following requirements:

  • Create four virtual machine hard disks attached to a new VMware Paravirtual SCSI (PVSCSI) adapter.
  • Create two virtual machine hard disks attached to a new VMware Paravirtual SCSI (PVSCSI) adapter.
  • Create two virtual machine hard disks attached to a new VMware Paravirtual SCSI (PVSCSI) adapter.
  • Each group of virtual machine hard disks created are to be of the same capacity.
  • Each virtual machine hard disk will be provisioned with the thick provision eager zeroed format.
  • Each virtual machine hard disk will be created on the same datastore as  the virtual machine.

As discussed, I have created a function for the above named ‘Create-VirtualHardDisks’ to which I have configured the mandatory parameters ‘Number’ for the count of virtual machine hard disks to create and ‘CapacityGB’ for the size of the virtual machine hard disk.

Function Create-VirtualHardDisks {
Param ([Parameter(Mandatory=$true)][int] $Number, [Parameter(Mandatory=$true)][string] $VM, [Parameter(Mandatory=$true)][int] $CapacityGB)

For the number of virtual machine hard disks to be create we will invoke a ForEach statement to repeat the action a specific number of times, in this case the number of virtual machine hard disks that are required to be created.

ForEach ($HardDisk in (1..$Number))
    { 

Now we will invoke the New-HardDisk cmdlet to create the virtual machine hard disks specifying the virtual machine, CapacityGB and the storage format.

New-HardDisk -VM $VM -CapacityGB $CapacityGB -StorageFormat EagerZeroedThick

Once the virtual machine hard disks have been created we now create the VMware Paravirtual SCSI (PVSCSI) adapter and attach the virtual machine hard disks. In order to retrieve the virtual machine hard disks we will invoke the Get-HardDisk cmdlet and invoke the Select-Object cmdlet to retrieve the virtual machine hard disks created by specifying the last number of objects whihc is equal to the number of virtual machine hard disks created.

$HardDisks = Get-HardDisk -VM $VM | Select-Object -Last $Number

Once the virtual machine hard disks have been retrieved we will invoke the ‘New-ScsiController’ and create the VMware Paravirtual SCSI (PVSCSI) adapter and attach the virtual machine hard disks.

New-ScsiController -Type ParaVirtual -HardDisk $HardDisks    
}
}

Now that we have the virtual machine hard disks created, I am now required to create the NTFS mounted volumes based on the SCSI Port and SCSI Target ID of the disk, with the following requirements:

  • Create the empty folder on the parent volume.
  • Initialize the disk with the GPT partition style.
  • Create a new partition and use the maximum size of the disk.
  • Add a partition access path to the empty folder created.
  • Perform a full format of the  volume with the NTFS file system with a Unit Allocation Size of 64K and set the file system label to be that of the empty folder.

As previously, we will be creating a function to achieve this task, to which we will specify mandatory parameters for the virtual machine, NTFS mounted folder path and the SCSI Port and $SCSITargetID of the disk.

Function Create-NTFSMountedFolder { 
Param ([Parameter(Mandatory=$true)][String] $VM,[Parameter(Mandatory=$true)][String] $Folder,[Parameter(Mandatory=$true)][String] $SCSIPort,[Parameter(Mandatory=$true)][String] $SCSITargetID)

We will be invoking this action using the ‘Invoke-VMScript’ cmdlet so we create a script text string to perform the orchestration to create the NTFS mounted folder. Firstly we will invoke the ‘New-Item’ cmdlet to create the empty folder on the parent volume.

$ScriptText = "New-Item -ItemType Directory -Path $Folder;

Now we will be required to retrieve the disk based on the SCSI Port and TargetID. In order to retrieve the disk required we will invoke the ‘Get-Disk’ cmdlet where we will retrieve the disk number required by invoking the ‘Get-WMIObject’ cmdlet with the class Win32_Drive to filter the disks where the SCSI Port and TargetID are equal to the those specified in the mandatory parameter and return the Index value of the disk.

`$Disk = Get-Disk -Number (Get-WmiObject -Class Win32_DiskDrive | Where-Object {`$_.SCSIPort -eq $SCSIPort -and `$_.SCSITargetID -eq $SCSITargetID}).Index;

Once we have retrieved the disk, we will now initialize the disk with the GPT partition style by invoking the Initialize-Disk cmdlet where the disk number is retrieved from the above query by specifying the Number value.

Initialize-Disk -Number `$Disk.Number -PartitionStyle GPT -PassThru;

Next we will create a partition and use the maximum size of the disk by invoking the New-Partition cmdlet .

New-Partition -DiskNumber `$Disk.Number -UseMaximumSize;

For the empty folder created in the first step we will now create an access path by invoking the ‘Add-PartitionAccessPath’ cmdlet to create the access path on the disk and partition specified.

Add-PartitionAccessPath -DiskNumber `$Disk.Number -PartitionNumber 2 -AccessPath $Folder\;

Finally, we will retrieve the partition created using the ‘Get-Partition’ cmdlet and invoke the ‘Format-Volume’ cmdlet to perform a full format of the volume with Unit Allocation Size of 64K and set the file system label to be the name of the empty folder created.

Get-Partition -DiskNumber `$Disk.Number -PartitionNumber 2 | Format-Volume -FileSystem NTFS -AllocationUnitSize 65536  -Full -NewFileSystemLabel $Folder -Confirm:`$False"

Now we will invoke the ‘Invoke-VMScript’ cmdlet to run the above in the guest operating system to create the NTFS mounted folder.

Invoke-VMScript -VM $VM -ScriptText $ScriptText
}

So by invoking the above functions we be able to create the virtual machine hard disks and create a NTFS mounted folder. In the below example, I will create three virtual machine hard disks and then create three NTFS mounted folders with the names D:\Databases1, D:\Databases2 and D:\Databases3.

Create-VirtualHardDisks -VM vm1 -Number 3 -CapacityGB 20
Create-NTFSMountedFolder -VM vm1 -Folder D:\Databases1 -SCSIPort 4 -SCSITargetID 3
Create-NTFSMountedFolder -VM vm1 -Folder D:\Databases2 -SCSIPort 4 -SCSITargetID 4
Create-NTFSMountedFolder -VM vm1 -Folder D:\Databases3 -SCSIPort 4 -SCSITargetID 5

The above functions, can be downloaded from the below:

Create-VirtualHardDisks – https://github.com/dean1609/scripts/blob/master/Create-VirtualHardDisks.ps1

Create-NTFSMountedFolder – https://github.com/dean1609/scripts/blob/e48d13eeb13baefa71777371f265d60d55d5a4a6/Create-NTFSMountedFolder.ps1

VMware Automation using PowerCLI – Alan Renouf

For those interested in VMware Automation using PowerCLI, presentation by Alan Renouf ( ) as presented at the Northern California PSUG.

Preparing a virtual machine template with Microsoft SQL Server 2012

A recent requirement was to create a template within VMware to deploy a configured virtual machine with a named Database Engine instance of SQL Server 2012 installed. Previous versions of SQL Server did not allow for a clean sysprep image, in SQL Server 2012 this has changed with the introduction of the concept image preparation and image complete of stand-alone instances of SQL Server. So it looks like its time to through away those configuration scripts for previous automated deployments of SQL Server 2005 and 2008!

So how did I go about this…

I currently have a base template of Windows 2012 Server Datacenter Edition, which I will deploy a virtual machine to configure as my template for SQL Server installations, also prior to beginning this process, I have created an additional hard disks for my virtual machine to host the database files.

Now for the configuration steps to create the SQL Server template. Firstly we need to prepare a stand alone instance of SQL Server.

1) Mount the installation media in the CD/DVD drive (or copy the contents of the media to an installation folder) and begin the setup process.

2) From the SQL Server Installation Center, select Advanced.

3) Select Image preparation of a stand-alone instance of SQL Server.

4) Select product updates available which will be installed during the SQL Server features setup.

5) Select the features you want to prepare, currently only Database Engine and Reporting Services features can be selected.

6) Configure your Instance ID and Instance root directory.

7) The SQL Server features will now be ready to be included in the prepared image, this will create a configuration file at ‘C:\Program Files\Microsoft SQL Server\110\Setup Bootstrap\Log\YYYYMMDD_HHMMSS\ConfigurationFile.ini’.

8) Select Prepare to complete the Image process. If prompted restart the computer, to complete the installation before capturing your image.

Once the prepare image is complete and you have configured the virtual machine as per the requirements for your template, capture the Sysprep image and convert the virtual machine to a template.

Now, we are going to deploy the  virtual machine from the template captured in the above steps, in this instance I use a customization wizard to configure a number of settings.

Once the customization wizard has completed, and the deployed virtual machine is available, we now need to complete the configuration of a the prepared imaged instance of SQL Server.

1) Begin the setup process from either the mounted installation media or from the installation folder.

2) Select Advanced.

3) Select Image completion of a prepared stand-alone instance of SQL Server.

4) Enter the product key

5) Select the ID of the prepared instance to complete and review the features in the prepared instance.

6) Specify the instance name for the prepared instance.

7) Specify the service accounts and the collation configuration

8) Specify the authentication mode, administrators and data directories.

9) The SQL Server features will now be ready to be configured and completed, this will create a configuration file at ‘C:\Program Files\Microsoft SQL Server\110\Setup Bootstrap\Log\YYYYMMDD_HHMMSS\ConfigurationFile.ini’.

10) The image process will now be completed. If prompted restart the computer, to complete the installation before capturing your image.

Obviously, I do not want to perform this attended completion of a prepared stand-alone instance of SQL Server following each deployment of the template. This is where the configuration file created to complete the configuration file will allow for this to be run once following deployment.

I will run a script to run once on deployment, to complete the configuration of the prepared image instance of SQL Server as below:

Setup.exe /ACTION="CompleteImage" /QS /INSTANCENAME="SQL12" /INSTANCEID="SQL12" /AGTSVCACCOUNT="NT Service\SQLAgent$SQL12" /AGTSVCSTARTUPTYPE="Automatic" /SQLSVCSTARTUPTYPE="Automatic" /SQLCOLLATION="Latin1_General_CI_AS" /SQLSVCACCOUNT="NT Service\MSSQL$SQL12" /SQLSYSADMINACCOUNTS="Domain\Group" /SECURITYMODE="SQL" /SQLBACKUPDIR="D:\Backup" /SQLUSERDBDIR="D:\Databases" /SQLUSERDBLOGDIR="D:\Logs" /SQLTEMPDBDIR="D:\TempDB" /ADDCURRENTUSERASSQLADMIN="False" /TCPENABLED="1" /NPENABLED="0" /BROWSERSVCSTARTUPTYPE="Automatic"

Once the script has been run as above, a configurable instance of SQL Server will be available.

So based on the previous versions of SQL Server and the steps required post and pre deployment of a SQL Server template the above steps are simple to use and reduce the deployment time for new instances of SQL Server.