How to Configure and Deploy a Bare Metal Controller using OpenStack Ironic - Part 3

Introduction

This is the third post in a series of blogs about how to configure and deploy a bare metal controller using Ironic. In the second post we described how to enable and configure IPMI. At this point, the controller server is ready to control remotely the secondary node via IPMI.

Host setup

Before installing the deployment tool, setup the controller server. The example below uses Ubuntu 16.04 as the operating system. Some commands in this section will refer to the “apt-get” package installation utility; use the appropriate package manager and packages for your Linux distribution.

The following commands will be executed as root user.

Configure proxy settings

If there is a proxy server, first setup the proxy configuration in the system. PyPI and Git have their own configuration files:

PROXY=<HTTP proxy address, ex: http://proxy.company.com>
PROXY_NAME=<proxy address, ex: proxy.company.com>
PROXY_PORT=<SOCKS4 proxy port>
MANAGEMENT_IP=<controller node first NIC IP address - management>

cat << EOF | sudo tee /etc/pip.conf
[global]
proxy = $PROXY
EOF

cat << EOF | sudo tee /etc/gitconfig
[http]
       proxy = $PROXY
EOF

cat << EOF | sudo tee /etc/proxy
export http_proxy=$PROXY
export ftp_proxy=$PROXY
export all_proxy=$PROXY
export ALL_PROXY=$PROXY
export FTP_PROXY=$PROXY
export HTTP_PROXY=$PROXY
export https_proxy=$PROXY
export HTTPS_PROXY=$PROXY
export no_proxy=localhost,127.0.0.1,127.0.1.1,$MANAGEMENT_IP
export NO_PROXY=$no_proxy
export GIT_PROXY_COMMAND=/usr/local/bin/git-proxy-wrapper
EOF

cat << EOF | sudo tee /usr/local/bin/git-proxy-wrapper
#!/bin/sh
_proxy=$PROXY_NAME
_proxyport=$PROXY_PORT
exec socat STDIO SOCKS4:$_proxy:$1:$2,socksport=$_proxyport
EOF

chmod +x /usr/local/bin/git-proxy-wrapper

if [ ! $(grep "source /etc/proxy" ~/.bashrc) ] ; then
cat << EOF | tee -a ~/.bashrc
source /etc/proxy
EOF
fi

source ~/.bashrc

Install the required libraries

Install all the required libraries and programs:

apt-get update
apt-get install -y git libvirt-bin python-pip curl ntp virt-manager libguestfs-tools apparmor-utils kpartx dmsetup xfsprogs genisoimage socat

pip install -U pip setuptools

apt-get install -y python-tox python-dev libffi-dev libssl-dev python3-dev ethtool ipmitool rand

apt-get upgrade -y

pip install -U pip python-openstackclient python-neutronclient python-ironicclient

# Working version of ansible with stable/mitaka kolla
sudo -E pip install -U 'ansible==1.9.5'
echo "The server will reboot now..."
reboot

Check host capabilities and enable IP forwarding

Now check the host virtual configuration and enable IPv4 forwarding:

if ! sudo virt-host-validate 1>/dev/null ; then
    echo "Failed, output:"
    virt-host-validate || true
    echo -e "\n\nyou may need to edit GRUB_CMDLINE_LINUX in /etc/default/grub and run grub-mkconfig --output=/boot/grub/grub.cfg"
else
    echo "OK"
fi

cat << EOF | tee /etc/sysctl.d/99-ip_forward.conf
net.ipv4.ip_forward=1
net.ipv4.conf.default.rp_filter=0
net.ipv4.conf.all.rp_filter=0
EOF

Configuring Docker

This section will configure Docker [1]. First, we are going to setup Docker as a service.

# Download Docker.
curl -sSL https://get.docker.io | bash
sudo systemctl stop docker

export DOCKER_CONF_DIR=/etc/systemd/system/docker.service.d
# Configure Docker service.
if [ ! -d $DOCKER_CONF_DIR ] ; then 
   mkdir $DOCKER_CONF_DIR
fi

Next, we’ll add the proxy configuration “if needed” and other configuration variables.

# Configure Docker proxy configuration.
cat << EOF | tee $DOCKER_CONF_DIR/proxy.conf
[Service]
Environment="HTTP_PROXY=$PROXY" "NO_PROXY=localhost,127.0.0.1,127.0.1.1,$MANAGEMENT_IP"
EOF

Once the configuration is complete, we will start the service, add a user and configure the shared mount point, which is a shared directory between the host and the containers.

# Set in Kolla configuration the shared directory.
cat << EOF | tee $DOCKER_CONF_DIR/kolla.conf
[Service]
MountFlags=shared
EOF

# Start the registry container.
cat << EOF | tee $DOCKER_CONF_DIR/exec.conf
[Service]
ExecStart=
ExecStart=/usr/bin/docker daemon -H fd:// --insecure-registry=127.0.0.1:4000
EOF

systemctl daemon-reload
systemctl show --property=Environment docker
systemctl start docker
systemctl enable docker

usermod -aG docker $USER

# Mount the shared point and start the Docker service.
/bin/mount --make-shared /run
sed -i "/exit 0/d" /etc/rc.local
echo "/bin/mount --make-shared /run || true" >> /etc/rc.local
echo "exit 0" >> /etc/rc.local
update-rc.d rc.local enable
service rc.local start

# Mount the shared point every time the server is restarted.
cat << EOF | tee /etc/rc.local_run_mount
#!/bin/bash
/bin/mount --make-shared /run
/usr/bin/docker restart neutron_metadata_agent neutron_l3_agent neutron_dhcp_agent
EOF

chmod +x /etc/rc.local_run_mount

cat << EOF | tee /etc/cron.d/rc.local_run_mount
@reboot root    /etc/rc.local_run_mount
EOF

Nova Compute service uses by default libvirt to spawn and control virtual machines. In this deployment we’ll configure Nova Compute to use Ironic virt driver instead of libvirt. To avoid any interference, disable libvirt:

systemctl stop libvirt-bin
systemctl disable libvirt-bin
rm -f /etc/apparmor.d/disable/docker
aa-disable /usr/sbin/libvirtd

Additional instructions on how to implement the next steps will be provided in follow-on blogs.

References

[1] What is Docker? (https://www.docker.com/what-docker)