Monday, March 31, 2014

Openshift Enterprise Installation Tips

In this post, I’ll document how to install Redhat OpenShift Enterprise.

For this blog, I’ll document how to install a single RHEL 6.5 host that services as both broker and node within the OpenShift architecture.
Lets get started!

Step 1 – Register with Redhat

You will need a valid subscription to Redhat OpenShift Enterprise, a 30 day evaluation can be obtained for free from the Redhat Customer Portal site located at https://access.redhat.com/home.

Step 2 – Download and Install RHEL 6.5

OpenShift Enterprise runs only on RHEL 6.5. You can download RHEL 6.5
from the Redhat Customer Portal.
Perform the RHEL 6.5 installation. I found that the RHEL instance needs to be configured to have a static IP address as DNS is used by Openshift to resolve all it’s endpoints. You then register the RHEL instance with the following command:
$ subscription-manager register
$ yum update
After the RHEL instance is ready, log into the Redhat Customer Portal and assign the OpenShift subscription to your RHEL instance at the portal link: Subscriptions > Subscription Management > Units
Select the host you want to assign the subscription to, then click on the Attach button to assign the Openshift subscription.

Step 3 – Set up ssh on the Node and Broker Hosts

Set up ssh passwords on all broker and node hosts, in my case I have a 2 hosts, one is a broker-node combination, and the other host is a pure node host.  You will need the node host to be able to ssh into the broker host with no password .  Create a key, install the key so that root can ssh into the broker host with no password.

Step 4 – Reboot the RHEL Instance

Step 5 – Setup yum

OpenShift is comprised of many different yum repositories. You will need to enable these on your RHEL instance to perform the OpenShift installation. Perform the following to list all the required repositories you will enable:
$ subscription-manager repos | grep ID
I found that the Oracle repositories could not be found and caused problems in the install, so I did not enable them. I also did not enable the debug repositories as they were not needed for the install.
You can enable the various repositories using this command syntax:
$ subscription-manager repoName --enable
Update the repository list as follows:
$ yum update
Begin the installation of Openshift as follows:
$ yum install openshift-enterprise-release
Validate the yum repository using the following command:
$ oo-admin-yum-validator
Make any corrections as suggested by the validator command.

Step 6 – Install OpenShift

We are ready to begin our install, in this blog, I’ve chosen to use the oo-install-ose script to guide us through the installation and configuration, this script is obtained from Redhat as follows:
https://access.redhat.com/site/documentation/en-US/OpenShift_Enterprise/2/html/Deployment_Guide/index.html
$ curl -o oo-install-ose.zip https://install.openshift.com/portable/oo-install-ose.zip
$ unzip oo-install-ose.zip
To limit the install from installing ALL available cartridges (think huge download!), you can restrict the installed cartridges by unzipping the oo-install-ose-20140219-1602.zip file, then editing the file inside named:  workflows/enterprise_deploy/openshift.sh, and adding the following configuration variable:
CONF_CARTRIDGES="python,ruby"
This setting will only download and install the python and ruby cartridges.  This is very use as I found many dependencies were not resolving for the JBoss set of cartridges.  To install those, you would take another approach which I’ll document in a later blog.
After you make that configuration change, zip the contents back up and replace the original zip file.  You are now ready to work with oo-nstall-ose.
Now you are ready to start the install:
$ ./oo-install-ose

Step 7 – Configuration

The oo-install-ose script will ask you for the hostname information you are performing the install on, I used the default example.com domain as the default domain name. I also selected the option of having the broker and node run on the same RHEL instance that we have configured. 

Conclusion

In an upcoming blog, we’ll create another node to add to the Openshift cluster, but for now, we’ll run both broker and node on the same host.

VoltDB - Creating an Openshift Cartridge - Part 2


In this blog post, I'll show you the steps required to fix a couple of things in our original cartridge.  

Determine the Openshift Internally Assigned IP Address


First, lets make our cartridge capable of being installed onto any Openshift web framework application, not just the DIY cartridge.

In Openshift, an internal IP address is assigned to your application when the web framework is used to create the initial application.  For example, with the DIY cartridge, the internally assigned IP address is found in the environment variable called OPENSHIFT_DIY_IP.  And if you were to select the Python web framework, the environment variable is called OPENSHIFT_PYTHON_IP, and for the PHP framework the variable is called OPENSHIFT_PHP_IP, and so on. 

The naming convention would tend to cause you to write scripts, like the ones for our VoltDB cartridge as follows:

if OPENSHIFT_DIY_IP then
 internalAddress = OPENSHIFT_DIY_IP
else if OPENSHIFT_PHP_IP thne
 internalAddress = OPENSHIFT_PHP_IP
else if OPENSHIFT_PYTHON_IP then
 internalAddress = OPENSHIFT_PYTHON_IP
else if.......

This is however not required given that within the OPENSHIFT_BASH_SDK script there exists two useful  bash functions named:

primary_cartridge_short_name()
primary_cartridge_private_ip_name()

These two functions can be used to determine the underlying web framework's IP address as follows:

function getinternalip() {
    cart_short_name=`primary_cartridge_short_name`

    cart_ip_name=`primary_cartridge_private_ip_name`

    ipaddressvar="OPENSHIFT_"$cart_short_name"_"$cart_ip_name

    echo $ipaddressvar

    echo ${!ipaddressvar}
}

Including this function into the VoltDB's bin/control script now allows us to start and stop VoltDB using a generic internal IP address as follows:

function start() {
...
                nohup voltdb create --internal 16700 --externalinterface $internaladdressip --zookeeper 16300 --httphost $internaladdressip -d $OPENSHIFT_VOLTDB_DIR/sample/deployment.xml  --zkhost $internaladdressip -H $internaladdressip --internalinterface $internaladdressip $OPENSHIFT_VOLTDB_DIR/sample/voltcache.jar > $OPENSHIFT_VOLTDB_DIR/logs/startup.log &
 ...
}   

function stop() {
        getinternalip

        client_result "stopping voltdb...."
        client_result "voltdb is running ...issuing stop ";
        export HOME=${HOME}/app-root/data
        voltadmin pause -H $internaladdressip;
        voltadmin shutdown -H $internaladdressip;

        return 1
}

You now see that our control script is agnostic as to what web framework we are installing VoltDB on.

Deploying Our Own Catalog


Next, we would like to alter the cartridge to allow us to run any VoltDB schema of our choosing.  This is accomplished simply by changing the scripts to use a generic application schema name and deployment file name in the start() function as follows:

nohup voltdb create --internal 16700 --externalinterface $internaladdressip --zookeeper 16300 --httphost $internaladdressip -d $OPENSHIFT_VOLTDB_DIR/sample/currentdeployment.xml  --zkhost $internaladdressip -H $internaladdressip --internalinterface $internaladdressip $OPENSHIFT_VOLTDB_DIR/sample/currentschema.jar > $OPENSHIFT_VOLTDB_DIR/logs/startup.log &


Instead of running the sample catalog for voltcache everytime, we will adjust our cartridge scripts to allow us to run a catalog of your choosing.

In the cartridge control script, we will modify the paths to the catalogs being executed as follows:

nohup voltdb create --internal 16700 --externalinterface $internaladdressip --zookeeper 16300 --httphost $internaladdressip -d $OPENSHIFT_DATA_DIR/currentdeployment.xml  --zkhost $internaladdressip -H $internaladdressip --internalinterface $internaladdressip $OPENSHIFT_DATA_DIR/currentschema.jar > $OPENSHIFT_VOLTDB_DIR/logs/startup.log &

Notice that we make use of the OPENSHIFT_DATA_DIR environment variable.  This location is writable by your account and is more generic than the sample directory which gets deployed with the cartridge itself. Now, the cartridge will look in this location for a catalog to start.

Initially when the cartridge is installed, the generic name will point to the voltcache example to provide a default example to install the cartridge with.  This is performed by means of the cartridge's setup script as follows:

cp ~/voltdb/sample/voltcache.jar $OPENSHIFT_DATA_DIR/currentschema.jar
cp ~/voltdb/sample/deployment.xml $OPENSHIFT_DATA_DIR/currentdeployment.xml


To supply your own schema and deployment files is simple if we make use of the rhc client utilities.  In this example, we'll upload the voltkv example that ships with VoltDB.  Make sure to build the voltkv example on your local workstation before running these commands!  In the following example, the application name we created is called voltdb.  From your client workstation (e.g. Fedora 20), you will enter the following commands to upload your own schema and deployment files:

$ cd ./voltdb-4.0.2.3/examples/voltkv

$ rhc cartridge stop jeffmccormick-voltdb-4.0.2.3 -a voltdb
Stopping jeffmccormick-voltdb-4.0.2.3 ... 
Failed to execute: 'control stop' for

/var/lib/openshift/53396dd1e0b8cdb04e000233/voltdb

$ rhc cartridge status jeffmccormick-voltdb-4.0.2.3 -a voltdb
RESULT:


voltdb is not running

$ rhc scp voltdb upload deployment.xml app-root/data/currentdeployment.xml
$ rhc scp voltdb upload voltkv.jar app-root/data/currentschema.jar

$ rhc cartridge start jeffmccormick-voltdb-4.0.2.3 -a voltdb
Starting jeffmccormick-voltdb-4.0.2.3 ... done

starting voltdb....
OPENSHIFT_PHP_IP 127.6.210.129

voltdb is not running...starting it now

Remember to add the http port setting of 16000 (port="16000")  in your deployment.xml files or VoltDB will not start on Openshift!

Now, on your client workstation, start port forwarding in one terminal window:

$ rhc port-forward -a voltdb

Then in another terminal window, run the voltkv client:

./run.sh client


jeffmc/edisk/tools/voltdb-4.0.2.3/examples/voltkv]: ./run.sh client
--------------------------------------------------------------------------------
 Command Line Configuration
--------------------------------------------------------------------------------

displayinterval = 5
duration = 30
entropy = 127
getputratio = 0.9
keysize = 32
latencyreport = false
maxvaluesize = 1024
minvaluesize = 1024
poolsize = 1000
preload = true
ratelimit = 2147483647
servers = localhost
statsfile = 
usecompression = false
warmup = 5

--------------------------------------------------------------------------------
 Setup & Initialization
--------------------------------------------------------------------------------

Connecting to VoltDB...
Connected to VoltDB node at: localhost.

Preloading data store...
Preloading complete.

--------------------------------------------------------------------------------
 Starting Benchmark
--------------------------------------------------------------------------------

Warming up...

Running benchmark...
00:00:05 Throughput 640/s, Aborts/Failures 0/0
00:00:10 Throughput 816/s, Aborts/Failures 0/0
00:00:15 Throughput 621/s, Aborts/Failures 0/0
00:00:20 Throughput 998/s, Aborts/Failures 0/0
00:00:25 Throughput 1155/s, Aborts/Failures 0/0
00:00:30 Throughput 1110/s, Aborts/Failures 0/0

--------------------------------------------------------------------------------
 KV Store Results
--------------------------------------------------------------------------------

A total of 29,717 operations were posted...
 - GETs:    24,020 Operations (0 Misses and 0 Failures)
                23 MB in compressed store data
                23 MB in uncompressed application data
         Network Throughput:  0.006 Gbps*
 - PUTs:     2,707 Operations (0 Failures)
                 2 MB in compressed store data
                 2 MB in uncompressed application data
         Network Throughput:  0.006 Gbps*
 - Total Network Throughput:  0.012 Gbps*

* Figure includes key & value traffic but not database protocol overhead.

--------------------------------------------------------------------------------
 Client Workload Statistics
--------------------------------------------------------------------------------

Average throughput:                  921 txns/sec


Conclusion

After this set of changes to the cartridge, you can now run VoltDB on any of the supported Openshift web frameworks, and also deploy your own VoltDB catalog.

Whats next?  Well, VoltDB has many features that relate to high availability and multi-node configuration, those are enterprise style features that are particularly challenging to implement and I'll address those in upcoming blogs.



Tuesday, March 25, 2014

VoltDB - Creating an Openshift Cartridge - Part 1

Introduction



This blog describes a new Openshift cartridge developed for the VoltDB database (www.voltdb.com).  I’ve been doing research on VoltDB and figured that others might benefit from having quick access to an Openshift hosted instance of the database.  VoltDB is truly impressive in both performance and in capabilities.


A ‘cartridge’ is a means of packaging and deploying an application, in this case the VoltDB database, onto the Openshift PaaS framework.    


The VoltDB cartridge in beta form is located at:


The combination of Openshift PaaS and VoltDB make for some interesting project ideas!

Performance Note


Openshift runs either on the public cloud hosted by Redhat at http://openshift.redhat.com, or you can install Openshift on your own hardware.  

Obviously for the highest performance, running VoltDB on your own private infrastructure will provide the highest performance.  But for simply testing out VoltDB and learning its various features, the public cloud infrastructure will work fine.

The cartridge we have developed for VoltDB will operate on either the public cloud Openshift or the private versions of Openshift, so the choice is yours.

Installation Scenario



Sample Deployment


Log into your account, define your domain name (e.g. voltdbtest):



snapshot1.png

upload your public key you want to use:

snapshot2.png

Click on the Add Application button to get started:





Create a DIY (do-it-yourself) Application, the DIY application is at the bottom of the Application.

snapshot3.png

Next, add the VoltDB cartridge to your application, click on the link that says Continue to the application overview page.:

snapshot5.png

Next, specify the following URL within the Install your own cartridge section of the web page:






At this point, if you click on the Add Cartridge button, it will install the VoltDB cartridge into your Openshift application:





At this point, VoltDB should be running on your application host and is ready to test!




Install Openshift Client Utilities


On your client host, in this example a Fedora 20 desktop, you will run the client for the VoltDB VoltCache example.  In order to do this, you will need to create networking tunnels to your Openshift application instance.  For this,  you will need to install the Redhat Openshift client tools called ‘rhc’.  

The Openshift client installation is documented at the following location:


Test the rhc client tools by displaying information (using the rhc app show command)  about your newly installed VoltDB application:




Also at this point, you can log into your new application by issuing the rhc ssh command as follows:



Start Port Forwarding


After the rhc installation, you will need to create the necessary port forwarding tunnels to your Openshift application instance as follows using the rhc port-forward command:




After the port forwarding has been started, you can now access from your local client (127.0.0.1) the various VoltDB ports that are running remotely on the Openshift cloud application you have installed.

Running the VoltCache Client


At this point, you will need to have downloaded the VoltDB community distribution to your client host.  The cartridge is based upon the VoltDB 4.0.2.3 Community Edition.  The VoltDB product can be downloaded from:


The connectivity (at least from my home office) to the public Openshift is really tested by this example and I found that I needed to reduce the speed at which the VoltDB client sends work to the cloud hosted VoltDB instance, so I adjusted the examples/voltcache/run.sh script used to execute the client as follows:

function benchmark() {
   srccompile
   java -classpath obj:$APPCLASSPATH:obj -Dlog4j.configuration=file://$LOG4J \
       voltcache.Benchmark \
       --threads=10 \
       --displayinterval=5 \
       --duration=20 \
       --servers=localhost \
       --poolsize=1000 \
       --preload=true \
       --getputratio=0.90 \
       --keysize=32 \
       --minvaluesize=1024 \
       --maxvaluesize=1024 \
       --usecompression=false


Here is an example of running the VoltDB VoltCache example client which performs a benchmark:

snapshot13.png

VoltDB Studio Web Application


A nice utility that is shipped with VoltDB is their Studio web application, found at the following URL:

Using Studio, you can issue a SQL query against the deployed VoltDB application.  Notice that the URL here is based on the rhc port-forward command being issued.  

Cartridge Development


Getting the cartridge to work in the Origin version of Openshift was a bit easier because you have the ability to bind to localhost (127.0.0.1).  In the public Openshift, binding to localhost (127.0.0.1) is not allowed.  

To get VoltDB to work in the public version of Openshift, there were a few steps required including:

adjusting ports


In the public Openshift, there are various port conflicts and protected ports, to avoid these problems, I’ve adjusted the VoltDB ports as follows:
  1. zookeeper port is 16300
  2. internal port is 16700
  3. http port is 16000

creating http host command line parameter


In this version of VoltDB, the internal HTTP server was binding to localhost by default and there was no way to specify a different host, so I modified the VoltDB code to allow for a new command line parameter called httphost to let me use the Openshift assigned internal IP address (e.g. OPENSHIFT_DIY_IP)

creating zookeeper host command line parameter


Also, in this version of VoltDB, the internal Zookeeper host was defined as localhost, so I created a new command line parameter called zkhost which allows me to override localhost with the Openshift internal IP address (e.g. OPENSHIFT_DIY_IP).

altering run.sh


To start VoltDB, I need to pass in various parameters to the voltdb command as follows:

voltdb create --internal 16700 --externalinterface $OPENSHIFT_DIY_IP --zookeeper 16300 --httphost $OPENSHIFT_DIY_IP -d $OPENSHIFT_VOLTDB_DIR/sample/deployment.xml  --zkhost $OPENSHIFT_DIY_IP -H $OPENSHIFT_DIY_IP --internalinterface $OPENSHIFT_DIY_IP

To stop the VoltDB, I need to pass in the new server host IP address to the voltadmin command as follows:

voltadmin pause -H $OPENSHIFT_DIY_IP;
voltadmin shutdown -H $OPENSHIFT_DIY_IP;

bundling and installing 64 bit JRE


VoltDB requires a 64 bit JRE to run.  Interestingly, the public Openshift does not include a 64 bit JRE but rather a 32 bit JRE even though it runs on 64 bit RHEL 6.5!  So, I needed to bundle the 64 bit JRE within the cartridge in order to have a proper JRE for VoltDB to run with.

Next Steps


This initial blog shows you some very basic ways to deploy a VoltDB database to the Openshift PaaS.  In upcoming blogs, I’ll show you how to modify the deployed VoltDB database schema / compiled catalog and also how to deploy it to Openshift web frameworks other than the DIY framework.

Here is a link to the Part 2 post where this topic continues:

jeffmc04.blogspot.com/2014/03/voltdb-creating-openshift-cartridge_31.html