Creating new address in a subnet using {php}IPAM REST API

In previous posts I have discussed phpIPAM which is an open-source IP address management application (IPAM). In this blog, I will discuss interacting with the REST API (version 2.0) to programatically create new IP addresses for a specific subnet.

For the purpose of this article, I will be using the Postman REST client to submit requests and interact with the API and the following variables reference values you will need to replace for your environment if you are replicating the process:

Variable Description Example
{appId} API application identifier dean
{subnet} Subnet in CIDR format 10.10.1.0/24
{subnetId} Id of subnet address 3

In order to use the API you will need to authenticate (required for none and SSL) with the username and password and use the HTTP authentication header.

If authentication is successful, you will receive the token to include in the token header for subsequent requests to the API. Also, you will receive the expiry of the token which is set to six hours by default. For each subsequent request to the API the expiration time for the token is reset.

POST api/{appId}/user/

By specifying the authorization type as ‘Basic Authentication’ and specifying a username and password this will generate the required Authorization header for the request by encoding the values to a base64 encoded string.

postman-phpipam-post-user

A status code of ‘200’ will be returned if the authentication attempt is successful, the required token value will also be returned in the response body with the expiration date.

{
  "code": 200,
  "success": true,
  "data": {
    "token": ".Cy9kljLjG=WIt9i.gmA%tUY",
    "expires": "2016-12-03 13:13:06"
  },
  "time": 5.036
}

Once the token has been received the cool stuff can now begin and we can start to interact with the API. In the next example, I am going to use a common use case for phpIPAM. I need to request the next available free IP address for a specified subnet and allocate this to a host.

So, lets look at the method we need to use to create a new address from a specified subnet from the addresses section of the documentation

POST  /api/{appId}/addresses/first_free/{subnetId}/

An alternative GET method exists, however whilst this will return the next available IP address it will not create the object for the new IP address in the database.

GET /api/{appId}/addresses/first_free/subnet

From the POST method we need to provide the Id of the subnet we require to create an IP address, therefore prior to invoking the request we need to return this value, by referencing the subnets section of the documentation, the following can be used by providing the subnet in CIDR format.

GET /api/{appId}/subnets/cidr/{subnet}/

In this example I am searching for the subnet containing the CIDR format 10.10.1.0/24. From the below response body we can retrieve the subnet Id from the data[0].id element and in this example, the subnet Id for the subnet 10.10.1.0/24 is ‘3’. As you can see from the headers section, I am now specifying the ‘token’ header previously returned from the successful authorization.

postman-phpipam-post-freeip

{
  "code": 200,
  "success": true,
  "data": [
    {
      "id": "3",
      "subnet": "10.10.1.0",
      "mask": "24",
      "sectionId": "1",
      "description": "Customer 1",
      "linked_subnet": null,
      "firewallAddressObject": null,
      "vrfId": "0",
      "masterSubnetId": "2",
      "allowRequests": "1",
      "vlanId": "0",
      "showName": "1",
      "device": "0",
      "permissions": "{\"3\":\"1\",\"2\":\"2\"}",
      "pingSubnet": "0",
      "discoverSubnet": "0",
      "DNSrecursive": "0",
      "DNSrecords": "0",
      "nameserverId": "0",
      "scanAgent": null,
      "isFolder": "0",
      "isFull": "0",
      "tag": "2",
      "threshold": "0",
      "location": null,
      "editDate": null,
      "links": [
        {
          "rel": "self",
          "href": "/api/dean/subnets/3/"
        }
      ]
    }
  ],
  "time": 0.002
}

Now, lets return to creating a new address in the specified subnet with the below request to create a new address in the subnet from the POST method. From the response body, the data element contains the IP address ‘10.10.1.26’ created for the subnet.

POST  /api/{appId}/addresses/first_free/3/

phpipam-post-first_free

{
  "code": 201,
  "success": true,
  "message": "Address created",
  "id": "34",
  "data": "10.10.1.26",
  "time": 0.015
}

We can also include a request body in the POST method to include information for creating the new IP address in the subnet. The supported parameters are listed for the address controller in the addresses section. In this example, I have included the hostname, description and owner values for the request.

phpipam-post-first_free-body

As you can see if you have read the provided API documentation, I am only touching the surface of what is possible by programmatically interacting with the API and how you can leverage this in your automation and orchestration practices.

Getting started with DevStack as OpenStack playground environment on Ubuntu

I was recently looking to deploy a minimal installation of OpenStack to use as a development environment and to minimise the complexity of installing a complete OpenStack environment where I can build and tear down the environment with minimal effort. This is where DevStack comes into play, as to quote:

DevStack is a series of extensible scripts used to quickly bring up a complete OpenStack environment based on the latest versions of everything from git master. It is used interactively as a development environment and as the basis for much of the OpenStack project’s functional testing.

For my minimal installation of DevStack I am using new installation of Ubuntu Server 16.10 (clean installation and isolated instance recommended due to significant changes to the system). In terms of hardware configuration, the following recommendations are provided. However, this will all be dependant on your use case for DevStack and what you require to achieve and DevStack may also be installed without satisfying the below criteria but you may experience performance/stability issues.

  • Processor – at least 2 cores
  • Memory – at least 8GB
  • Hard Drive – at least 60GB

First of all we will create a user account named ‘stack’ to install DevStack and grant the user sudo privileges with the no password parameter. It is important in this step that we do not complete the installation as the root user.

sudo adduser stack 
echo "stack ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers

Now log out as the current user and reconnect as the ‘stack’ user created in the previous step and clone the repository to the current directory, in this example i am changing the directory to ‘/var’ to create my local copy.

cd /var 
sudo git clone git://github.com/openstack-dev/devstack.git

We will now need to create a configuration file (/var/local.conf) to specify user configuration variables to use when the installation script (/var/devstack/stack.sh) is executed in a subsequent step. The installation will complete without

The below example, contains only password variables so that you are not required to input the values at installation. For a more detailed configuration file containing additional parameters, check out the sample from the respoistory.

cd /var/devstack
sudo vi local.conf 
[[local|localrc]]
ADMIN_PASSWORD=openstack
DATABASE_PASSWORD=$ADMIN_PASSWORD
RABBIT_PASSWORD=$ADMIN_PASSWORD
SERVICE_PASSWORD=$ADMIN_PASSWORD

Once the configuration file has been created, we can now execute the installation script (stack.sh). The installation process will take approximately 15-20 minutes to complete to which will receive console output. Once the installation has completed you will receive an installation summary, URLs, accounts and passwords.

./stack.sh

Once the installation has completed successfully you should be able to browse to the Horizon dashboard and authenticate with the admin credentials configured during installation, at http://{ip address}/dashboard.

horizon-login

If you need to remove the installation of DevStack there is a script included in the repository (./clean.sh) which will remove the installation of DevStack and dependancies and then cleanup the directories touched by the installation. For a detailed list of impacted files and directories during the installation refer to this link.

cd /var/devstack
./clean.sh
rm -rf /opt/stack
rm -rf /usr/local/bin

From my initial attempt at installation, I encountered a number of issues which appear to be permission related I believe this was due to not cloning the repository as the ‘stack’ user account used for the installation. In this case, to resolve you could run the below or alternatively clean-up your installation process and repeat the installation.

sudo chown -R stack:stack /var/devstack 
sudo chmod 770 /var/devstack

Regular Expression Cheat Sheet

Below is a quick reference guide for regular expressions that I frequently use and find useful.

Online RegEx debugger and library – https://regex101.com/

Windows PowerShell cmdlets to secure PSCredential Objects

I have previously discussed securing credentials using Windows Powershell atPowershell: Securing credentials using the PSCredential class. In this article, I will discuss a number of cmdlets I have created to secure credentials using a Advanced Encryption Standard (AES) encryption key to retrieve the content from a encrypted standard string.

As I am using an encryption key and storing the information in a content file, I will be using ACLs on the NTFS filesytem to control access. Alternative methods could be to store the encryption key in a database or use a certificate to control access to the item. Also, in practice I will store the encryption key on a remote file server.

Firstly, we need to create the encryption key using the ‘New-EncryptionKey’ cmdlet to which we use the RNGCryptoServiceProvider class to generate a random byte array for the encryption length. By default, the cmdlet using a 32-byte array to support the AES 256-bit encryption length. The cmdlet also supports using 128-bit and 192-bit encryption lengths which a 16-byte and 24-byte array. The random byte array for the specified encryption length is generated and sends the output to a file which will be the encryption key content.

Once the content has sent to the output file, the content of random byte array is then removed from the current session.

# Creates an AES 256-bit encryption key at the location D:\Output\Keys\mykey.key
New-EncryptionKey -Output D:\Output\Keys\mykey.key 

# Creates an AES 192-bit encryption key at the location D:\Output\Keys\mykey.key
New-EncryptionKey -Bytes 24 -Output D:\Output\Keys\mykey.key

We have now created an encryption key so that we may now convert the secure string for a credential object password using the specified encryption key and sends the output to a password file using the ‘New-EncryptedString’ cmdlet. The cmdlet will retrieve the content of the specified file containing the encryption key and from the stored credential objects convert the secure string of the password to an encrypted standard string and send the output to a file and clear the content of the stored encryption key from the current session.

New-EncryptedString -KeyFile D:\Output\Keys\mykey.key -PasswordFile D:\Output\Passwords\mypassword.txt 

Finally we want to retrieve the credential object to use as a variable to pass to a subsequent cmdlet which will require authentication. The content of the password file is retrieved and converted to a secure string using the content of the encryption key and stored as a password variable and passed to the PSCredential class to represent a set of security credentials and return the object. For subsequent cmdlets I can use the ‘$Password.GetNetworkCredential().Password’ property value for authentication from the PSCredential object.

$Password = Get-PSCredentialObject -Username administrator -KeyFile D:\Output\Keys\mykey.key -PasswordFile D:\Output\Passwords\mypassword.txt 

The cmdlets are available from the below:

New-EncryptionKey – https://github.com/dean1609/PowerShell/blob/master/Functions/New-EncryptionKey.ps1
New-EncryptedString – https://github.com/dean1609/PowerShell/blob/master/Functions/New-EncryptedString.ps1
Get-PSCredentialObject – https://github.com/dean1609/PowerShell/blob/master/Functions/Get-PSCredentialObject.ps1

Creating JetBrains YouTrack issues with Windows PowerShell

YouTrack is a proprietary, commercial browser-based bug tracker, issue tracking system and project management software developed by JetBrains. There is also a REST API provided which allows for various actions to be performed programmatically. In this article I will describe the cmdlets I have created to create issues for projects which leverage the Invoke-WebRequest cmdlet to interact with the REST API using Windows PowerShell.

The cmdlets require a minimum of Windows PowerShell 3.0 and uses the response object for HTML content without Document Object Model (DOM) parsing, this is required when Internet Explorer is not installed on the local instance invoking the cmdlets. An example of a use case for the cmdlets, is to be configured as an event handler from an infrastructure monitoring solution which will create issues for alarms raised.

The REST API is enabled by default and you can confirm connection and access permissions by browsing to ‘http://{baseURL}/rest/admin/project’ which should return an XML file with a list of all the projects. A more detailed description of the REST API can be found at https://confluence.jetbrains.com/display/YTD65/YouTrack+REST+API+Reference.

In this example, we are using cookie-based authorization. However this process can be adapted to also used Hub OAuth 2.0 authentication as well, which is described at https://www.jetbrains.com/hub/help/1.0/OAuth-2.0-Authorization.html.

Firstly, we need to establish a connection to the REST API using specified credentials and store the web request session object using the session variable. This will allow for cookie information to be re-used for use in subsequent web requests. The specified parameters of login and password are required to be used in the POST method as below:

POST /rest/user/login?{login}&{password}

The cmdlet Connect-YouTrack establishes a connection by specifying the YouTrack Uri, Username and Password parameters and returns the web request session object, which in this example I am storing in the Connection variable.

$Connection = Connect-YouTrack -YouTrackUri http://server1 -Username administrator -Password P@55Word! 

Once a connection has been established and the web session object has been returned, we can now create an issue in a project and specify a summary and description. The cmdlet ‘New-YouTrackItem’ will invoke a PUT request to the REST API with the specified parameters of project, summary and description as below.

PUT /rest/issue?{project}&{summary}&{description}

Once the item has been created, the URI of the new issue is retrieved from the response header and using a regular expression pattern match we retrieve the item number so that we can use a subsequent web request to update the item with additional information as a variable, in this example ‘NewIssue’. The Connection variable returned when establishing a connection to the REST API is passed to the session variable for authentication.

$NewIssue = New-YouTrackItem -YouTrackUri http://server1 -Project YT -Summary "Summary API Rest Method" -Description "Description API Rest Method" -SessionState $Connection

Finally, we want to apply a command to the issue created to update items with specified paramaters, as below.

POST /rest/issue/{issue}/execute?{command}

The cmdlet ‘Update-YouTrackItem’ specifies a mandatory parameter named ‘ExecuteCommand’ to specify all the items you wish to update in the issue. In this example, I will be invoking a command to set the priority as ‘High’, the type as ‘ Service Desk’ and category as ‘Support’ to update the issue previously created from the stored ‘NewIssue’ variable.

Update-YouTrackItem -YouTrackUri http://server1 -Item $NewIssue -ExecuteCommand "priority High type Service Desk category Support" -SessionState $Connection

The cmdlets are available from the below:

Connect-YouTrack – https://github.com/dean1609/PowerShell/blob/master/Functions/Connect-YouTrack.ps1
New-YouTrackItem – https://github.com/dean1609/PowerShell/blob/master/Functions/New-YouTrackItem.ps1
Update-YouTrackItem – https://github.com/dean1609/PowerShell/blob/master/Functions/Update-YouTrackItem.ps1

Implementing R functionality on Tableau Server

R (https://www.r-project.org/) is a free software environment for statistical computing and graphics. It compiles and runs on a wide variety of UNIX platforms, Windows and MacOS. In this example, I will be installing and configuring R on Ubuntu 14.04 and enabling R functionality with Tableau Server by installing the Rserve library.

​In order to install R ​​and R packages from the ‘Compreshensive R Archive Network’ (CRAN) we will use the Advanced Packaging Tool (APT) and therefre need ​​add the repository to ​​the list of sources ​as well as the ​public key ​to authenticate packages downloaded using APT, this will ensure we install the latest version of both R (r-base)and the CRAN package for Rserve.

sudo sh -c 'echo "deb http://cran.rstudio.com/bin/linux/ubuntu trusty/" >> /etc/apt/sources.list'​​
gpg --keyserver keyserver.ubuntu.com --recv-key E084DAB9
gpg -a --export E084DAB9 | sudo apt-key add -
sudo apt-get update 
sudo apt-get install ​ r-base​​

To verify the installation we can enter an interactive shell session, once loaded we shall quit the session

R 
q(save="no")

Now we will install the Rserve CRAN package by invoking the install.package() function in R. In order for the package to be available to all users this is installed as root (su).

sudo su - -c "R -e \"install.packages('Rserve', repos = 'http://cran.rstudio.com/')\""​

Again, we can verify the installation by entering the R interactive shell session, confirming the Rserve library is available and then quit the session.

R
​library(Rserve)
q(save="no")

By default Rserve only accepts local connections, in order to enable remote connections will we will need to modify the configuration file ‘/etc/Rserve.conf’. A detailed list of other Rserve connection properties that may be set see https://rforge.net/Rserve/doc.html#start.

remote enabled

In order to ensure the Rserve process is initialised at startyp as a daemon we will need to create the shell script ‘/etc/init.d/Rserve.sh’ as below. As no ownership of files are required for the invocation of the Rserve CRAN package, we will use the ‘nobody’ account to start the daemon.

#!/bin/bash 
sudo -u nobody R CMD Rserve --vanilla 

In order to execute the shell script we will require to set execute permissions to the shell script and add a link to initialise the shell script at startup. ​

sudo chmod 755 /etc/init.d/Rserve.sh
sudo update-rd.d /etc/init.d/Rserve.sh defaults

To confirm the Rserve process is initialised at startup using the shell script we can reboot the instance and confirm the process is running

ps aux | grep Rserve

The next step is optional, in this example I only want to permit inbound connections on the TCP service port 6311 (Rserve) from the Tableau Server. By default rules added to iptables are ephemeral and on restart will be removed. In order to save the configuration we will install the ‘iptables-persistent’ package.

sudo apt-get install iptables-persistent 

I will firstly insert a drop rule for all connections (IPv4) to the destination port 6311(tcp), then insert an accept rule for the Tableau Server (10.0.0.2) to the destination port 6311(tcp) and then save the updates to preserve the iptables configuration.

sudo iptables -I INPUT -p tcp -s 0.0.0.0/0 --dport 6311 -j DROP
sudo iptables -I INPUT -p tcp -s 10.0.0.2 --dport 6311 -j ACCEPT
sudo service iptables-persistent save​

The last step is to configure the Tableau Server VizSQL Server connection properties for a Rserve host (10.0.0.3) and port (6311) to enable R functionality within workbooks, optional configuration parameters are also available to use a username and password but in the example access is restricted by firewall rules.

tabadmin stop
​tabadmin set vizqlserver.rserve.host 10.0.0.3 
tabadmin set vizqlserver.rserve.port 6311
tabadmin configure
tabadmin start 

The configuration is now complete, to verify the VizSQL Server configuration, invoke ‘tabadmin configure -o ‘ to confirm the configuration parameter has been set by dumping the current configuration to a file.