Rasa X CI/CD

Rasa X Bot Deployment

Posted by Greg Stephens on October 09, 2019 · 7 mins read

This post is out of date. For a docker-compose based install look here and for a good example of a Rasa CI/CD pipeline look here.

I’ve been developing bots with the new Rasa X platform the past few months and am ready to automate the deployment of my bot. I’m going to do this with Jenkins but the process would be the same with any CI/CD platform. I’m focused here on a Docker with docker-compose setup.

In this post I’m going to review the process to deploy a new Rasa X bot instance including the training and related configuration data. At this point, I’m working with Rasa X 0.21.3. Future releases of Rasa X are expected to have an improved deployment workflow so the process should be much easier.

Note:
As of Rasa X 0.21.3, there are two install types. A local install type and a server install. The instructions here are based on the server install. There’s a forum post by hsm207 with an example Dockerfile for local installs.
Comments:
If you have comments regarding this post and suggestions for improvements, please post them at the Rasa X forum here.

Baseline

I’m assuming you have an existing Rasa bot or have exported all of the configuration of your bot from Rasa X and checked it into a Git or other repo. The custom content we need to get our bot up would typically include:

  • NLU Training Data
  • Core Training Data
  • domain.yml file with response templates and a list of intents, actions, entities, forms
  • config.yml with NLU & Core config
  • endpoints.yml
  • credential.yml
  • nginx.conf optional custom nginx.conf file
  • action agent

rasacli

I’m going to use the rasacli command line utility which I wrote to perform the import of the various configuration items. It will also allow you to export the configuration, train the model, list and set the active model among other things.

Install

Here are the steps I’m using to automate the install of a new instance:

Base Rasa X Install

First, we install the normal Rasa X server install.

Note:
Why can’t we simply pull docker images like we’ve done for Rasa. (It would be much simpler if Rasa distributed Rasa X in a manner that would allow us to use a docker-compose.yml).

The Rasa X install script does the following:

  • Installs python, pip and ansible on the host
  • Uses ansible to install a docker role
  • Downloads and runs an ansible playbook

Since I want to automate the install, I’m going to short circuit the prompts that are built into the ansible playbook. Here’s the script to do the install.

export RASA_HOME=/home/greg/rasax
curl -sSL -o install.sh https://storage.googleapis.com/rasa-x-releases/stable/install.sh
# Remove the wget of the playbook, we'll do that ourselves
sed -i 's/wget /#wget/' install.sh
# get the playbook before running the install so we can modify it
wget -qO rasa_x_playbook.yml https://storage.googleapis.com/rasa-x-releases/stable/rasa_x_playbook.yml
# suppress prompts
sed -i 's/terms_confirmed: False/terms_confirmed: True/' rasa_x_playbook.yml
mkdir -p $RASA_HOME/terms
echo ${USER} $(date) > ${RASA_HOME}/terms/agree.txt
awk '{sub(/dest: "\/.env"/, " METRICS_CONSENT=True\n      dest: \"/.env\"")}1' rasa_x_playbook.yml > temp.yml && mv temp.yml rasa_x_playbook.yml
# clean-up /tmp otherwise ansible can have apt-cache errors
sudo rm -rf /tmp/*
# do it
sudo -E bash ./install.sh

You now have a basic Rasa X install and you could cd to RASA_HOME if you want and start Rasa X with the sudo docker-compose up -d command.

Import Content

Now that Rasa X is installed, we need to import our content. I have all of my content in a git repo and it would be great if I could just checkout the repo directly into RASA_HOME but I haven’t been able get this to work.

Instead, I’m going to clone the repo under my RASA_HOME and move some of the files into place. Once this is done, I will use the rasacli utility to import my data.

Note:
I’m going to reference my github jokebot repo that is not currently available. I hope to post this project soon.

Initial Setup and Start

This first step will:

  • Clone my content repo
  • Put a docker-compose.override.yml file in place that defines containers for my (1) action agent; (2) user interface site; and (3) custom nginx
  • Put the action agent python file in place
  • Move the credentials.yml into place
  • Start Rasa X
  • Set the Rasa X password
cd $RASA_HOME
export DATA_DIR=botdata
git clone https://github.com/rgstephens/jokebot.git ${DATA_DIR}
# copy the docker-compose.override.yml file which contains my action agent, user interface and nginx containers
cp ${DATA_DIR}/docker-compose.override.yml .
# copy the action agent
cp -r ${DATA_DIR}/actions .
sudo cp ${DATA_DIR}/credentials.yml .
sudo docker-compose up -d
export RASA_PASS=mypassword
sudo python rasa_x_commands.py create --update admin me $RASA_PASS

Import Remaining Data

Now that Rasa X is running with all of our containers, I’ll use the rasacli to import my remaining configuration:

  • NLU training data
  • Core training data
  • domain.yml (response templates, intents, actions, entities, forms)
  • config.yml

The rasacli reads the RASA_PASS environment variable to get your password and RASA_PORT if you’re running Rasa X on an alternate port.

export RASA_PORT=5005
npm install -g rasacli
rasacli updtemplates ${DATA_DIR}/data/domain.yml
rasacli updconfig ${DATA_DIR}/data/config.yml
rasacli updtraining ${DATA_DIR}/data/training/*md
rasacli addstories ${DATA_DIR}/data/stories/*md

Train Model

Finally, let’s use the rasacli to train and activate the model:

MODEL=`rasacli modeltrain`
rasacli modelactivate -m $MODEL
← Previous Post Next Post →