bikle.com /Opinions/Tech Tips

Howto: AngularJS, Node.js App deployed to Heroku

Dan's AngularJS Learning Club (DALC)

Stock Market Predictions via MADlib Logistic Regression

Split Date Adjustments of Stock Market Data For Machine Learning

Stock Market Backtest with PostgreSQL 9.2 and LibSVM

Video Conferencing via Gmail

Join bikleTech!

Install Cygwin X-Windows and OpenSSH on Windows8

VirtualBox, Vagrant, Linux on Windows8

Linux 101

gem install pg: debugged

Ubuntu 12.04.3 Libraries needed by Ruby 2.0 and Rails 4.x

Selenium Can Test H2O Web UI

Understand Confusion Matrix via SQL

Call Java HelloWorld from JRuby

Deploy 0xdata H2O 10 Node Cluster on Hadoop on EC2

0xdata H2O on Linux Laptop

Stock Market/MADlib Linear Regression

Node.js For Rails on CentOS 6.4

Heroku Toolbelt

NOT IN to Left Outer Join

Install MADlib on Postgres 9.2 on CentOS 6.4

Install Postgres 9.2.4 on Ubuntu 12.04.2

Access Windows 8 BIOS on HP ENVY TS 15 Notebook PC
Page Top

Selenium Can Test H2O Web UI


Step 1 is start H2O on a testing server.

Here is my typical set of shell commands to do that on my laptop:

cen95 oracle ~
cen95 oracle ~
cen95 oracle ~ $ cd h2o_git/h2o
cen95 oracle ~/h2o_git/h2o $ grep git .git/config
	url = https://github.com/0xdata/h2o.git
cen95 oracle ~/h2o_git/h2o $ 
cen95 oracle ~/h2o_git/h2o $ git status
# On branch master
# Untracked files:
#   (use "git add <file>..." to include in what will be committed)
#
#	R/h2o-package/DESCRIPTION
#	R/h2o-package/h2o_package.pdf
#	R/h2o-package/man/h2o-package.Rd
#	hadoop/src/main/java/water/hadoop/h2odriver_config.java
nothing added to commit but untracked files present (use "git add" to track)
cen95 oracle ~/h2o_git/h2o $ 
cen95 oracle ~/h2o_git/h2o $ git pull origin master
From https://github.com/0xdata/h2o
 * branch            master     -> FETCH_HEAD
Already up-to-date.
cen95 oracle ~/h2o_git/h2o $ 
cen95 oracle ~/h2o_git/h2o $ 
cen95 oracle ~/h2o_git/h2o $ 



cen95 oracle ~/h2o_git/h2o $ 
cen95 oracle ~/h2o_git/h2o $ 
cen95 oracle ~/h2o_git/h2o $ make
PROJECT_VERSION is 1.7.0.99999
make clean PROJECT_VERSION=1.7.0.99999
make[1]: Entering directory `/home/oracle/h2o_git/h2o'
rm -fr target
snip...
snip...
snip...
InstallBuilder not found, skipping creation of windows and mac installer packages.
make[2]: Leaving directory `/home/oracle/h2o_git/h2o/installer'
make[1]: Leaving directory `/home/oracle/h2o_git/h2o'
cen95 oracle ~/h2o_git/h2o $ 
cen95 oracle ~/h2o_git/h2o $ 
cen95 oracle ~/h2o_git/h2o $ 

cen95 oracle ~/h2o_git/h2o $ 
cen95 oracle ~/h2o_git/h2o $ cd target
cen95 oracle ~/h2o_git/h2o/target $ 
cen95 oracle ~/h2o_git/h2o/target $ ll h2o.jar 
-rw-rw-r--. 1 oracle oracle 61291008 Sep  1 16:44 h2o.jar
cen95 oracle ~/h2o_git/h2o/target $ 
cen95 oracle ~/h2o_git/h2o/target $ 

cen95 oracle ~/h2o_git/h2o/target $ 
cen95 oracle ~/h2o_git/h2o/target $ java -Xmx50m -jar h2o.jar -name dan
04:47:27.403 main      INFO WATER: ----- H2O started -----
04:47:27.406 main      INFO WATER: Build git branch: master
04:47:27.410 main      INFO WATER: Build git hash: 0e27b407524e0d6b60af4b7fa3ab93c26591cd3f
04:47:27.415 main      INFO WATER: Build git describe: 0e27b40
04:47:27.415 main      INFO WATER: Build project version: 1.7.0.99999
04:47:27.416 main      INFO WATER: Built by: 'oracle'
04:47:27.417 main      INFO WATER: Built on: 'Sun Sep  1 16:44:04 PDT 2013'
04:47:27.419 main      INFO WATER: Java availableProcessors: 1
04:47:27.436 main      INFO WATER: Java heap totalMemory: 0.03 gb
04:47:27.436 main      INFO WATER: Java heap maxMemory: 0.05 gb
04:47:27.437 main      INFO WATER: ICE root: '/tmp'
04:47:27.573 main      INFO WATER: Internal communication uses port: 54322
+                                  Listening for HTTP and REST traffic on  http://192.168.1.95:54321/
04:47:27.812 main      INFO WATER: H2O cloud name: 'dan'
04:47:27.812 main      INFO WATER: (v1.7.0.99999) 'dan' on /192.168.1.95:54321, discovery address /225.1.131.145:57601
04:47:27.826 main      INFO WATER: Cloud of size 1 formed [/192.168.1.95:54321]
04:47:28.489 main      INFO WATER: Log dir: '/tmp/h2ologs'


Next, I start Firefox on the same host.

I browse localhost at the port I see displayed by the above Java command;
usually it is the URL displayed below:

http://localhost:54321

If I see the H2O homepage, I keep going.

Then, I check that I have the Selenium IDE Firefox plugin installed in Firefox.

I do this by clicking Tools, Add-Ons.

I type 'Selenium' into the search box.

Usually there, Selenium IDE is the first result unless I already have it installed.

Usually the name is Selenium Expert (Selenium IDE).

I install it and the installation bounces my instance of Firefox.

Then, I return to the localhost page, probably this:

http://localhost:54321

Now, Selenium IDE should be listed in the Tools menu.

If it's not there, you may encounter a painful process to fix it.

Essentially you need to match two pieces of software:

Firefox
Selenium IDE Firefox plugin

Plan A:
Usually I determine the version of Firefox I have from the help menu.
Then, I use Google to find a Selenium IDE Firefox plugin which works on that version.

Plan B:
Get a recent copy of Firefox.
Then, I use Google to find a Selenium IDE Firefox plugin which works on that version.

Recently I implemented Plan B on a Centos VM.

I found a copy of Firefox 24 at this URL:

http://download.cdn.mozilla.net/pub/mozilla.org/firefox/releases/24.0b7/linux-x86_64/en-US/firefox-24.0b7.tar.bz2

I installed it in my home-dir.

It would only run after I removed Firefox 17 from /usr/ using shell command:

yum erase firefox

Then using Google, I found a copy of Selenium IDE Firefox plugin which works on Firefox 24:

http://release.seleniumhq.org/selenium-ide/2.3.0/selenium-ide-2.3.0.xpi

It's important here to use Google rather than the plugin-searcher baked into Firefox.

Once Selenium IDE appears in the Tools menu I return to the localhost page, probably this:

http://localhost:54321

I click Selenium IDE in the Tools menu.

The IDE appears as a small form in a small window:

I type 

localhost:54321

into the field: Base URL

I click Actions.

I ensure that Record is checked.

I bring localhost:54321 page to focus.

I click one of the Blue-Try-it! buttons.

The IDE should come to life and record that click.

If so, keep clicking on various links.

Then return to the IDE and press the red-stop button.
The IDE should now be dormant.

I bring localhost:54321 page to focus.

I click the H2O home button so I'm back at localhost:54321

I return to the IDE.

I press the 2nd green play button (Play current test case).

Firefox should come to life and act as driven by robot.

Next I want to save this session I have cached in the IDE.

I click File,Save Test Case As, test100.html

It wants to place the HTML file in my home-dir.

Instead I place the file here:

~/test100/test100.html

Be aware that a useful feature of Selenium IDE is that the default format of test-cases is HTML.

I like this, they are easy to read with an editor or render in a browser.

Next, in the IDE, I export the test-case to file in Java format.

I create this: ~/test100/Test100.java

I look at it with my editor.

I see a package declaration:

package com.example.tests;

I move the file to here:

~/test100/com/example/tests/Test100.java

Next I ask Google a question:

How do I run WebDriver Selenium Tests in Java?

I find this page:

http://code.google.com/p/selenium/downloads/list

I wget this:

http://code.google.com/p/selenium/downloads/detail?name=selenium-server-standalone-2.35.0.jar

Next I download a copy of junit.jar from junit.org

I compile Test100.java

cen95 bikle ~/test100 $ 
cen95 bikle ~/test100 $ javac -cp .:com/example/tests:selenium-server-standalone-2.35.0.jar:junit-4.11.jar com/example/tests/Test100.java
cen95 bikle ~/test100 $ 

I run JUnit against Test100.java

cen95 bikle ~/test100 $ 
cen95 bikle ~/test100 $ java -cp .:com/example/tests:/usr/share/java/junit-4.11.jar:selenium-server-standalone-2.35.0.jar org.junit.runner.JUnitCore com.example.tests.Test100
JUnit version 4.11
.
Time: 8.297

OK (1 test)

cen95 bikle ~/test100 $ 
cen95 bikle ~/test100 $ 
cen95 bikle ~/test100 $ 

I notice that when JUnit starts running, a Firefox window pops up on my screen.

Then as JUnit runs, Firefox acts as run by robot.

The next probable step is to figure out how to run Firefox in a Xvfb:

http://en.wikipedia.org/wiki/Xvfb




Page Top

Understand Confusion Matrix via SQL

Predicted
class
Cat Dog Rabbit
Actual class
Cat 5 3 0
Dog 2 3 1
Rabbit 0 2 11



I copied the above table from wikipedia:
http://en.wikipedia.org/wiki/Confusion_matrix

The wikipedia article uses plain English to describe the Confusion Matrix.

It took me over a minute to understand the English.

I've been using SQL for a long time so I often use SQL
to understand data.

Thinking in SQL helped me understand the Confusion Matirx more.

I start by visualizing a table named results with two columns:
predicted_class and actual_class.

To count the number of predicted cats I'd type:

SELECT COUNT(*) FROM results WHERE predicted_class = 'Cat';

To count the number of actual rabbits I'd type:

SELECT COUNT(*) FROM results WHERE actual_class = 'Rabbit';

To count the number of results where the predicter predicted the
animal to be a dog and the animal actually was a dog, I'd type:

SELECT COUNT(*)
FROM results 
WHERE predicted_class = 'Dog'
AND   actual_class    = 'Dog';

According to the Confusion Matrix, the above query would return 3.

To count the number of results where the predicter predicted the
animal to be a rabbit and the animal actually was a cat, I'd type:

SELECT COUNT(*)
FROM results 
WHERE predicted_class = 'Rabbit'
AND   actual_class    = 'Cat';

According to the Confusion Matrix, the above query would return 0.

Both the article and the SQL tell me that I want the largest numbers to
reside on the diagonal of the matrix.

If I see a large number off the diagonal, I translate it to SQL to understand it better.

For example I see this:

SELECT COUNT(*)
FROM results 
WHERE predicted_class = 'Dog'
AND   actual_class    = 'Cat';

Which returns 3.

Too frequently the predictor sees actual cats which look like dogs.

Is it possible it also sees actual dogs which look like cats?

SELECT COUNT(*)
FROM results 
WHERE predicted_class = 'Cat'
AND   actual_class    = 'Dog';

The above query returns 2.

So yes, it sees actual dogs which look like cats.




Page Top

Call Java HelloWorld from JRuby



It is possible to access Java code from JRuby scripts.

A good page for some details about this is listed below:

https://github.com/jruby/jruby/wiki/CallingJavaFromJRuby

After I walked through some of the demos in the above page,
I asked, "How do I call the classic HelloWorld.java code from JRuby?"

I started this simple task with Google-search of HelloWorld.java:

https://www.google.com/search?q=HelloWorld.java

I found a copy of HelloWorldApp.java inside oracle.com

With my mouse I copied HelloWorldApp.java from

http://docs.oracle.com/javase/tutorial/getStarted/application/

to /tmp/HelloWorldApp.java

I then saw this:

z6 /tmp $ 
z6 /tmp $ 
z6 /tmp $ cat HelloWorldApp.java 
/**
 * The HelloWorldApp class implements an application that
 * simply prints "Hello World!" to standard output.
 */
class HelloWorldApp {
    public static void main(String[] args) {
        System.out.println("Hello World!"); // Display the string.
    }
}
z6 /tmp $ 
z6 /tmp $ 

Next, I compiled it:

z6 /tmp $ 
z6 /tmp $ which javac
~/jdk17/bin/javac
z6 /tmp $ 
z6 /tmp $ 

z6 /tmp $ 
z6 /tmp $ javac -version
javac 1.7.0_25
z6 /tmp $ 
z6 /tmp $ 

z6 /tmp $ 
z6 /tmp $ javac HelloWorldApp.java 
z6 /tmp $ 

z6 /tmp $ 
z6 /tmp $ ls -la HelloWorld*
-rw-r--r--. 1 oracle oracle 429 Sep  1 02:14 HelloWorldApp.class
-rw-rw-r--. 1 oracle oracle 261 Sep  1 02:12 HelloWorldApp.java
z6 /tmp $ 
z6 /tmp $ 

Then I tested it:

z6 /tmp $ 
z6 /tmp $ java HelloWorldApp
Hello World!
z6 /tmp $ 
z6 /tmp $ 

z6 /tmp $ 
z6 /tmp $ export CLASSPATH=/tmp
z6 /tmp $ cd /usr
z6 /usr $ java HelloWorldApp
Hello World!
z6 /usr $ 
z6 /usr $ 

Next, I returned to /tmp and wrote a simple JRuby script:


z6 /usr $ cd /tmp
z6 /tmp $ 


z6 /tmp $ 
z6 /tmp $ cat try1.rb 
# /tmp/try1.rb

# An attempt to run /tmp/HelloWorldApp.java

arg1 = ''
HelloWorldApp.main([arg1])
z6 /tmp $ 
z6 /tmp $ 

z6 /tmp $ 
z6 /tmp $ which jruby
~/jrubydir/bin/jruby
z6 /tmp $ 

z6 /tmp $ 
z6 /tmp $ jruby -v
jruby 1.7.4 (1.9.3p392) 2013-05-16 2390d3b on Java HotSpot(TM) 64-Bit Server VM 1.7.0_25-b15 [linux-amd64]
z6 /tmp $ 
z6 /tmp $ 


z6 /tmp $ 
z6 /tmp $ jruby try1.rb
NameError: uninitialized constant HelloWorldApp
  const_missing at org/jruby/RubyModule.java:2631
         (root) at try1.rb:6
z6 /tmp $ 
z6 /tmp $ 

Then, I searched:

https://www.google.com/search?q=Java+in+JRuby+gives+NameError%3A+uninitialized+constant

I added a line and tried again:

z6 /tmp $ 
z6 /tmp $ 
z6 /tmp $ cat try2.rb 
# /tmp/try2.rb

# An attempt to run /tmp/HelloWorldApp.java

java_import 'HelloWorldApp'

arg1 = ''
HelloWorldApp.main([arg1])

z6 /tmp $ 
z6 /tmp $ 
z6 /tmp $ 


z6 /tmp $ 
z6 /tmp $ 
z6 /tmp $ jruby try2.rb
Hello World!
z6 /tmp $ 
z6 /tmp $ 

So, to summarize this mini-lesson I say that:

To run /tmp/HelloWorldApp.java in JRuby I need to do 2 things:

- Put /tmp in the CLASSPATH env-variable
- java_import a string which tells JRuby the name of the Java class
- Imagine that HelloWorldApp acts as a Ruby class
- See that HelloWorldApp.main() is a class method
- Inspect HelloWorldApp.java; see that HelloWorldApp.main wants an Array of Strings

Actually that's 5 things.

Maybe these 5 things are in the JRuby documentation somewhere,
but I could not find them with Google-search.

Next, I worked on the task of placing HelloWorldApp.class in a JAR
file and calling that JAR file from JRuby.

From /tmp/ I unset CLASSPATH and then created the JAR file:

z6 /tmp $ 
z6 /tmp $ unset CLASSPATH
z6 /tmp $ 
z6 /tmp $ echo Main-Class: HelloWorldApp > manifest.txt
z6 /tmp $ jar cvfm HelloWorldApp.jar manifest.txt HelloWorldApp.class
added manifest
adding: HelloWorldApp.class(in = 432) (out= 290)(deflated 32%)
z6 /tmp $ 
z6 /tmp $ 


Next, I tested the JAR file by calling it from a far-away directory:

z6 /tmp $ 
z6 /tmp $ cd /usr
z6 /usr $ java -jar /tmp/HelloWorldApp.jar HelloWorldApp
Hello World!
z6 /usr $ 
z6 /usr $ 

Then, I returned to /tmp/, removed HelloWorldApp.class, and wrote try3.rb:

z6 /tmp $ 
z6 /tmp $ rm HelloWorldApp.class 
z6 /tmp $ 

z6 /tmp $ cat try3.rb 
# /tmp/try3.rb

# An attempt to run /tmp/HelloWorldApp.java

require '/tmp/HelloWorldApp.jar'
java_import  'HelloWorldApp'

arg1 = ''
HelloWorldApp.main([arg1])


Next, I tried it from the /usr/ directory:

z6 /tmp $ 
z6 /tmp $ cd /usr
cd /usr
z6 /usr $ 


z6 /usr $ 
z6 /usr $ jruby /tmp/try3.rb
jruby /tmp/try3.rb
Hello World!
z6 /usr $ 
z6 /usr $ 

Yay! it worked.

The main idea to remember here is that I can pull a JAR file into my script just
like a Ruby file by calling require() on the JAR file.

Question: 
I already know I can use CLASSPATH to tell JRuby about HelloWorldApp.class,
can I instead use require() to tell JRuby about /tmp/HelloWorldApp.class?

I ran an experiment.

First I recreated HelloWorldApp.class just as I had done earlier:

z6 /tmp $ 
z6 /tmp $ javac HelloWorldApp.java
z6 /tmp $ 
z6 /tmp $ 

Next, I made sure that JRuby could not find /tmp/HelloWorldApp.class from CLASSPATH:

z6 /tmp $ unset CLASSPATH
z6 /tmp $ 

z6 /tmp $ 
z6 /tmp $ ls -la HelloWorldApp.class
-rw-r--r--. 1 oracle oracle 432 Sep  1 03:20 HelloWorldApp.class
z6 /tmp $ 
z6 /tmp $ 

The final step of the experiment is to try to call HelloWorldApp.class via 
the Ruby require() call:

z6 /tmp $ 
z6 /tmp $ cat try4.rb 
# /tmp/try4.rb

# An attempt to run /tmp/HelloWorldApp.java

require '/tmp/HelloWorldApp.class'
java_import  'HelloWorldApp'

arg1 = ''
HelloWorldApp.main([arg1])

z6 /tmp $ 
z6 /tmp $ cd /usr
cd /usr
z6 /usr $ 

z6 /usr $ 
z6 /usr $ 
z6 /usr $ jruby /tmp/try4.rb
jruby /tmp/try4.rb
LoadError: use `java_import' to load normal Java classes: HelloWorldApp
  require at org/jruby/RubyKernel.java:1054
  require at /home/oracle/jruby-1.7.4/lib/ruby/shared/rubygems/custom_require.rb:36
   (root) at /tmp/try4.rb:5
z6 /usr $ 
z6 /usr $ 

So, the experiment gave me an error message (an unhelpful one at that).

In English this means to me the following statement:

To call a Java-class file from JRuby,
I need to put the file's directory in CLASSPATH,
OR I need to put the file in a JAR file and then
call require() on the JAR file.

It took me awhile to figure this out.
I wish this simple behavior was more clearly evident from reading JRuby-Java demos
I found on the web.


Page Top