Chef – Creating encrypted data bags with Knife Solo

In order to use data bags with Chef Solo there is  a requirement to use the knife-solo_data_bag plugin maintained by Tommy Bishop which allows you to work withchef-solo and data bags. Once we have obtained the plugin, we will need to install the rubygem in our existing envrionment and configure the knife-solo plugin.

gem install knife-solo_data_bag

Once installed we can confirm we have a list of available knife solo command options available, as below:






In this example, the chef-solo repository is located on a Windows operating system, so we will create the knife-solo (~/.chef/knife.rb) configuration file and include the application (Notepad++) to be used as the knife editor when creating encrypted data bag items.

Finally, we will update our chef-solo configuration file with location of the data bag path to complete the configuration.

mkdir C:\chef-solo\.chef
mkdir C:\chef-solo\data_bags
echo "knife[:editor] = '"C:\Program Files (x86)\Notepad++\notepad++.exe" -nosession -multiInst'" > C:/chef-solo/.chef/knife.rb
echo data_bag_path   File.join(chefsolo, 'data_bags') >> C:\chef-solo\solo.rb

In order to create an encrypted data bag we need to generate a shared secret or a secret key, in this example I will generate a secret key using OpenSSL and output the content to a file (secret_key).

openssl rand -base64 512 | tr -d '\r\n' > secret_key

Now that we have our secret key we can now create an encrypted data bag, by invoking the following command to create the data bag ‘databag1’ with a item list of ‘databag1_passwords’.

knife solo data bag create databag1 databag1_passwords --secret-file secret_key -c C:\chef\.chef\knife.rb

The application to which you configured to be the knife editor will now generate and open a JSON file

 "id": "data_bag1_passwords"

We can now modify this file to include items we wish to store in the data bag, in my example a password value for both ‘service_account_1’ and ‘service_account_2’.

 "id": "data_bag1_passwords",
 "service_account_1": "ds879HBKJHBJH!*£",
 "service_account_2": "dasw87698KJBHB£*"

Once we have added the to list of items and saved the file, close your text editor to which the the knife-solo plugin should now return a success code similar to  ‘Created data_bag_item[data_bag1_passwords]’ to confirm the data bag item has been successful created.  Now lets check the content of the data bag item created in the chef-solo repository:

 "id": "data_bag1_passwords",
 "service_account_1": {
 "encrypted_data": "PCo7jiL8D4T+0EC5M1GvzqvNAIoVHtLF4i5M4VGtZjQYoV2KvFU6Qz3DgD1o\nvgug\n",
 "iv": "V8e9FY6EPlbMGSfBCkJgVQ==\n",
 "version": 1,
 "cipher": "aes-256-cbc"
 "service_account_2": {
 "encrypted_data": "Eh2NdqY/tPNbEBe+Du1LQmz8LTvGjj0zAv6aWYUp60RCOA7jkDb0NV7DgL84\nCSpY\n",
 "iv": "11PLeX4rvTyotGUgYQhYRA==\n",
 "version": 1,
 "cipher": "aes-256-cbc"

So once we have created the encrypted data bag how do we leverage this information, we can load the secret key and items using the Chef::EncryptedDataBagItem class and declare a variable for the item we require to use.

secret_key = Chef::EncryptedDataBagItem.load_secret("C:\chef-solo\_secret_key")
passwords = Chef::EncryptedDataBagItem.load('databag1', 'databag1_passwords', secret_key)
service_account_1_password = passwords['service_account_1']

All that now is required is to specify the variable in your attribute or recipie, as follows:




Chef and non-zero return codes

When installing a windows package using the windows_package resource the install may report a failure if the non zero return code is considered to be an installation failure.

In my example, the package would return the code 3010, which states a restart is required to complete the installation. However, we can use the success_code attribute parameter for the resource to set an array of possible successful return codes, which by default is 0,42 and 127.

So in order to prevent the package reporting a failure, the following example can be used by including the return code 3013 in the array. For Example:

windows_package 'package_name' do 
success_code [0,42,127,3013]

Windows File System Redirector with Chef Client/Ruby

I was recently using the windows_zipfile resource to unzip content to the path C:\Windows\System32\WindowsPowerShell\v1.0\Modules as part of a recipe in a Chef cookbook which would complete successfully, but on observation would unzip the contents to C:\Windows\SysWow64\WindowsPowerShell\v1.0\Modules and cause an error when further resources references this source.

The cause of this behaviour is due to the the chef-client and ruby versions are running as 32-bit and the file system redirector redirects access to C:\Windows\SysWow64 instead of C:\Windows\System32.

In order to workaround this issue we can substitute C:\Windows\Sysnative for C:\Windows\System32, by using this alias the file system redirection is bypassed.

In my example, I modified my original resource in the recipie to now include the alias.

windows_zipfile 'C:/Windows/System32/WindowsPowershell/v1.0/Modules/' do 
windows_zipfile 'C:/Windows/Sysnative/WindowsPowershell/v1.0/Modules/' do

Nagios XI: Automating Host Management

I was recently looking at how to automate adding and removing managed hosts and services in Nagios XI, which can be particularly useful in cloud computing and large environments where configuration management solutions have been implemented for provisioning. In these environments we typically use configuration files based on the attributes of a server role during the provisioning and configuration cycle.

Nagios XI contains a number of scripts in the directory /usr/local/nagiosxi/scripts that allow for automated host management, as below:

Script Description Imports configuration files from the import directory, verifies configuration and restart Nagios if verification succeeds . If verification fails, configuration will be rolled back to the last working checkpoint. This is the command invoked from the web interface when selecting ‘Apply Configuration’.
nagiosql_delete_host.php Removes a host from the configuration database and removes the configuration file.
nagiosql_delete_service.php Removes services from the configuration database and removes the configuration file.

In order to automate adding managed hosts and services the method used was to create a single configuration file for a each host and each of its services to which service definitions are are only applied to a that host and not to a host list or host group and to name the configuration file according to the hostname. In the below example, I have created a single configuration file which defines the host and a managed service for CPU Usage and saved the configuration file as ‘server1.dean.local.cfg’.

define host {
 host_name server1.dean.local
 use xiwizard_windowsserver_host
 address server1.dean.local
 max_check_attempts 5
 check_interval 5
 retry_interval 1
 check_period xi_timeperiod_24x7
 notification_interval 60
 notification_period xi_timeperiod_24x7
 icon_image win_server.png
 statusmap_image win_server.png
 _xiwizard windowsserver
 register 1

define service {
 host_name server1.dean.local
 service_description CPU Usage
 use xiwizard_windowsserver_nsclient_service
 check_command check_xi_service_nsclient!!CPULOAD!-l 5,80,90
 max_check_attempts 5
 check_interval 5
 retry_interval 1
 check_period xi_timeperiod_24x7
 notification_interval 60
 notification_period xi_timeperiod_24x7
 _xiwizard windowsserver
 register 1

Once the configuration file has been created we can place the file in the import directory located at ‘/usr/local/nagios/etc/import’ and invoke the script from the directory ‘/usr/local/nagiosxi/scripts’ to import the configuration file, verify the configuration and restart Nagios if successful. If the verification of the configuration fails, Nagios XI will restore the configuration files to the last working checkpoint but the imported configuration file will remain in the configuration database. In order to detect failures the following exit codes are returned where an exit code of ‘0’ to confirm that the configuration file has been successfully verified as a working configuration and Nagios has been restarted.

Exit Code Description
0 no problems detected
1 config verification failed
2 nagiosql login failed
3 nagiosql import failed
4 reset_config_perms failed
5 nagiosql_exportall.php failed (write configs failed)
6 /etc/init.d/nagios restart failed
7 db_connect failed

Now that we have added a managed host and services, how do we remove this the configuration database and delete the configuration file once the host is terminated? Providing the host has no dependent relationships we can firstly remove the services using the configuration name which matches the configuration file of the managed host (this is why it is important to name the configuration file according to hostname) and invoke the ‘nagiosql_delete_service.php’ from the directory ‘/usr/local/nagiosxi/scripts’ as the below example:

./nagiosql_delete_service.php --config=server1.dean.local

After the services have been successfully deleted we can remove the host by invoking the ‘nagiosql_delete_host.php’ script:

./nagiosql_delete_host.php --host=server1.dean.local

Once the host has been successfully removed, we can apply the new configuration as previosuly by invoking the ‘nagios_reconfigure_sh’ script. This method can also be applied to remove an imported configuration from the configuration database if verification of the configuration has failed during an import.

The above describes how to automate adding and removing hosts and services using Nagios XI and can be applied to your configuration management solutions during the provisioning and configuration cycle. In my scenario, I created a number of configuration files based on the attributes of server roles to which can be used as cookbook templates in Chef and using the ‘{node[‘fqdn’]}’ pattern to specify the host name in the definition file and the configuration file name.  I have also compiled PowerShell functions to perform the above which I will discuss in a later post.

Installing Chef Standalone Server

The following example will focus on standalone installation of Chef Server. Firstly, we will need to download the Chef Server package from to which I will be installing the package on an Ubuntu Linux 14.04 operating system.

Once the package is downloaded I uploaded the package to the standalone server and installed the package from the filesystem path.

dpkg -i chef-server-core_12.0.6-1_amd64.deb

Following successful  installation of the package we now need to start the services, this process may take a couple of minutes due to the number of services which are required to be started for a functioning Chef Server.

chef-server-ctl reconfigure

Now we will need to create a administrative user, organisation and to associate that administrative user to the organisation. This step will also create an RSA private key and save this to a specified path.

chef-server-ctl user-create <user_name> <first_name> <last_name> <email> <password> --filename <FILE_NAME>

For Example, the below will create an administrative user for myself and store the RSA private key in the file system path /home/deangrant/chef_deangrant.pem.

chef-server-ctl user-create deangrant Dean Grant dean.grant@dean.local *********** --filename home/deangrant/chef_deangrant.pem

To create the organisation and associate the user, there a number of rules to which the organisation name must adhere to as follows:

  • The organization name must begin with a lower-case letter or digit, may only contain lower-case letters, digits, hyphens, and underscores, and must be between 1 and 255 characters. For example: chef.
  • The full organization name must begin with a non-white space character and must be between 1 and 1023 characters. For example: Chef Software, Inc..

Once you have decided upon an organisation name, the below will create an organisation, associate a user and create an RSA private key for the organisation and store this to a specified path.

chef-server-ctl org-create <short_name> <full_organization_name> --association_user <user_name> --filename <FILE_NAME>

For Example, the below will create an organisation named ‘deanslab’, associate the user ‘deangrant’ and create the RSA private key.

chef-server-ctl org-create deanslab "Deans Lab" --association_user deangrant --filename /secure/org_deanslab_validator.pem

Now we have completed a standalone installation of Chef Server, created our first administrative user and assiocated that user with a newly created organisation. The above installation will only install the standalone components for Chef Server. In order to install additional features we can perform the following:

Chef Manage

chef-server-ctl install opscode-manage
opscode-manage-ctl reconfigure
chef-server-ctl reconfigure

Chef Push Jobs 

chef-server-ctl install opscode-push-jobs-server
opscode-push-jobs-server-ctl reconfigure
chef-server-ctl reconfigure

Chef Replication

chef-server-ctl install chef-sync
chef-sync-ctl reconfigure
chef-server-ctl reconfigure


chef-server-ctl install opscode-reporting
opscode-reporting-ctl reconfigure
chef-server-ctl reconfigure