JoeJag :: Tech

Nov

Checking JMX with JRuby

by Joe Wright

I’ve recently been playing around with JRuby and found the switch from MRI to be hassle free. One of the advantages of JRuby is it’s portability. You can use nearly all Ruby code straight away on the JVM without having to do any code changes, you just include the jruby jar in your classpath.

Another advantage is Java interoperability from JRuby (just require ‘java’). I’ve found this useful when using JMX to check certain conditions are in place while performing releases and for general monitoring.

The following script uses JRuby’s Java integration to walk a given ObjectName’s attributes within JMX.

require 'java'

# Return a JMX Connection
def connect_to_jmx service_url, credentials
	jmxUrl = javax.management.remote.JMXServiceURL.new(service_url)

	environment = java.util.HashMap.new
	environment.put(javax.management.remote.JMXConnector.CREDENTIALS,
                              credentials.to_java(:string))

	jmxCon = javax.management.remote.JMXConnectorFactory.connect(jmxUrl,
                                                                                              environment)
	return jmxCon.getMBeanServerConnection
end

# Walk the JMX tree and return the attributes.  Optionally limit to a single ObjectName
def walk_tree jmx_connection, limited_to=''
	names = jmx_connection.queryNames(
                                           javax.management.ObjectName.new(limited_to), nil)

	cached = Hash.new([])

	names.each do |name|
	   info = jmx_connection.getMBeanInfo(name)
	   info.getAttributes.each do | mbi|
		   attr = jmx_connection.getAttribute(name, mbi.getName)
		   cached.store(name.getCanonicalName,
                          cached[name.getCanonicalName] << { mbi.getName  => attr })
	   end
	end

	return cached
end

service_url = 'host:port'
credentials = ['username', 'password']
jmx_connection = connect_to_jmx(service_url, credentials)

results = walk_tree(jmx_connection, 'java.lang:type=OperatingSystem')

results.each do | key, attributes |
  puts ' * ' + key
  attributes.each do | attr |
	puts " ** #{attr.first[0]} :: #{attr.first[1]}"
  end
  puts
end

When invoked against a running JMX instance this will return something similar to:

> jruby jmx_walker.rb
 * java.lang:type=OperatingSystem
 ** FreePhysicalMemorySize :: 51955511296
 ** TotalPhysicalMemorySize :: 151873998848
 ** Name :: Linux
 ** Arch :: amd64
 ** Version :: 2.6.18-194.11.4.el5
 ** AvailableProcessors :: 24
 ** SystemLoadAverage :: 0.49

Hopefully this is of use to you, or inspires you to investigate JRuby a bit further. I’d highly recommend it for Java shops who want to take advantage of Ruby’s scripting ability.

Mar

A DSL for collections in Java

by Joe Wright

When writing Java code I often find it laborious to create collections using the java.util.* collection classes. To avoid this, I’ve been using a mini-DSL to reduce my collections code.

import static com.joejag.common.collections.Dsl.*;

// A list
List list = list("abc", "def");

// A set
Set set = set("Sleepy", "Sneezy", "Dozy");

// A Map
Map map = map(entry("Joe", 28), entry("Gerry", 39));

Here is the underlying code.

package com.joejag.common.collections;

import java.util.*;

public class Dsl {
    public static <T> List<T> list(T... args) {
        return Arrays.asList(args);
    }

    public static <T> Set<T> set(T... args) {
        Set<T> result = new HashSet<T>(args.length);
        result.addAll(Arrays.asList(args));
        return result;
    }

    public static <K, V> Map<K, V> map(Entry<? extends K, ? extends V>... entries) {
        Map<K, V> result = new HashMap<K, V>(entries.length);

        for (Entry<? extends K, ? extends V> entry : entries)
            if (entry.value != null)
                result.put(entry.key, entry.value);

        return result;
    }

    public static <K, V> Entry<K, V> entry(K key, V value) {
        return new Entry<K, V>(key, value);
    }

    public static class Entry<K, V> {
        K key;
        V value;

        public Entry(K key, V value) {
            this.key = key;
            this.value = value;
        }
    }

    public static void main(String args[]) {
        List<String> list = list("abc", "def");
        System.out.println(list);

        Set<String> set = set("Sleepy", "Sneezy", "Dozy");
        System.out.println(set);

        Map<String, Integer> map = map(entry("Joe", 28), entry("Gerry", 39));
        System.out.println(map);
    }
}

I’ve noticed that the Google Collections project has morphed into Guava which has great reusable code for collections and a lot of other common Java tasks.

Mar

Start a web server from your pwd using Ruby and Thin

by Joe Wright

Occasionally I’m interested in grabbing files from a directory on a remote host. This could be for another process to consume or to look at on my local work station.

The standard way of doing this is using scp, ftp or a file share. I prefer to start a short-lived web server that shares the file system.

To make this simple I use a ruby script to allow me to start a webserver from the directory I’m currently in.

['rubygems', 'thin', 'rack', 'socket'].each {|file| require file }
Thin::Server.start(IPSocket.getaddress(Socket.gethostname), 7777) do
  use Rack::CommonLogger
  run Rack::Directory.new(Dir.pwd)
end

As an alias for ~/.bashrc it looks like this:

alias rshare="ruby -rubygems -e \"['thin', 'rack', 'socket'].each {|file| require file };"\
" Thin::Server.start(IPSocket.getaddress(Socket.gethostname), 7777) {"\
" use Rack::CommonLogger; run Rack::Directory.new(Dir.pwd) }\""

This allows you to go to port 7777 on the host and retrieve the files you’re interested in.

@joejag /tmp $ rshare
>> Thin web server (v1.2.7 codename No Hup)
>> Maximum connections set to 1024
>> Listening on 10.0.0.2:7777, CTRL+C to stop
10.0.0.8 - - [01/Mar/2011 07:45:15] "GET / HTTP/1.1" 200 2153 0.0038

You will need to have the ‘thin’ and ‘rack’ gems installed to do this.

Feb

Creating a simple Java RESTful service using Jersey and Maven

by Joe Wright

There has been a lot of interest in Resource Orientated Architecture (ROA) online which has resulted in an explosion of excellent libraries that make it simple to use this architectural style.

The example in this article uses Java, Maven and a JAX-RS library called Jersey to create a very simple orders system.

Our order consists of a reference number and a customer name. The system has three entry points:

HTTP PUT to create an order
HTTP GET to see the details of an order
HTTP GET to see all the existing orders

Download the full source of this article here.

To make it simple to get it up and running I’ve added configuration for Jetty to run on port 9090. You can compile the source code and run a web server to use it by issuing this single Maven command:

mvn jetty:run

I recommend that you use curl to test your RESTful web services as it is easy to change the HTTP verb that you are using. You can manipulate the orders system using these curl commands.

# Add a new order for Bob with ID 1
curl -X PUT http://localhost:9090/orders-server/orders/1?customer_name=bob 

# Check the status of the order
curl -X GET http://localhost:9090/orders-server/orders/1 

# See all the orders in the system
curl -X GET http://localhost:9090/orders-server/orders/list

Note: HTTP GET is the default verb used by curl, I’ve explicitly listed GET in the examples to make it clear which verb is in use.

The code

To use Jersey you need to add a servlet to your web.xml and create a resource implementation class.

web.xml

The WEB-INF/web.xml fragment sets up the Jersey Servlet with a parameter listing the package to search for RESTful classes.

<servlet>
   <servlet-name>Jersey Web Application</servlet-name>
   <servlet-class>com.sun.jersey.spi.container.servlet.ServletContainer</servlet-class>
   <init-param>
      <param-name>com.sun.jersey.config.property.packages</param-name>
      <!-- Important bit -->
      <param-value>com.joejag.code.orders.restservices</param-value>
   </init-param>
   <load-on-startup>1</load-on-startup>
</servlet>

This reads any classes in the com.joejag.code.orders.restservices and looks for annotations declaring resources.

The Java class

On the implementation class you can see the @Path annotation on the class signature indicating the resource we are handling. This is used by Jersey to configure the URL used to interact with the resource.

Each method has a sub-path declared, an HTTP verb to respond from and the response type to produce. The method parameters are bound by using either part of the path (with @PathParam) or a separate query parameter (with @QueryParam).

package com.joejag.code.orders.restservices;

import java.util.Map;
import java.util.TreeMap;

import javax.ws.rs.GET;
import javax.ws.rs.PUT;
import javax.ws.rs.Path;
import javax.ws.rs.PathParam;
import javax.ws.rs.Produces;
import javax.ws.rs.QueryParam;
import javax.ws.rs.WebApplicationException;
import javax.ws.rs.core.Response;

@Path("orders")
public class OrdersService
{
   // Stores state simply in a static collection class.
   private static Map<String, String> orders = new TreeMap<String, String>();

   @Path("/{order}")
   @PUT
   @Produces("text/html")
   public String create(@PathParam("order") String order,
                                    @QueryParam("customer_name") String customerName)
   {
      orders.put(order, customerName);
      return "Added order #" + order;
   }

   @Path("/{order}")
   @GET
   @Produces("text/html")
   public String find(@PathParam("order") String order)
   {
      if (orders.containsKey(order))
         return "<h2>Details on Order #" + order +
                    "</h2><p>Customer name: " + orders.get(order);

      throw new WebApplicationException(Response.Status.NOT_FOUND);
   }

   @Path("/list")
   @GET
   @Produces("text/html")
   public String list()
   {
      String header = "<h2>All Orders</h2>\n";

      header += "<ul>";
      for (Map.Entry<String, String> order : orders.entrySet())
         header += "\n<li>#" + order.getKey() + " for " + order.getValue() + "</li>";

      header += "\n</ul>";

      return header;
   }
}

Download the full source of this article here.

Where to learn more

There are a number of great articles on how to develop JAX-RS REST applications. I recommend you start with the Jersey guide for Java applications:

If you are using Ruby then take a look at the wonderful Sinatra project.

Jan

Renaming MP3 tags with Ruby and the mp3info gem

by Joe Wright

Occasionally you get MP3s which have unconventional tags. I usually get this problem with compilation albums and it’s a bit dull to rename them in iTunes.

There’s a ruby gem which let’s you easily edit id3 tags called ‘mp3info‘.

Here is a script to get you started:

require "mp3info"

dir = '/opt/music/album_name/'
Dir.entries(dir).each do |file|
   next if file !~ /.mp3$/ # skip files not ending with .mp3
   Mp3Info.open(dir + file) do |mp3|
      puts mp3.tag.title
      puts mp3.tag.artist
      puts mp3.tag.album
      puts mp3.tag.tracknum

      # You would perform your text transform here, here's a simple change instead
      mp3.tag.title = mp3.tag.title + ' with my change'
      mp3.tag.artist = mp3.tag.artist + ' with my change'
   end
end

Jan

Installing CouchDB 1.0.1 on RHEL4 as the local user

by Joe Wright

On some of the servers I use I don’t have root access. This means occasionally it can be a hassle to install packages when the dependencies aren’t available.

Recently I had to install CouchDB and it’s dependencies from source. The tricky bit was having to edit the CouchDB configure script to specify the Curl libraries.

In this example I’ve assumed the user home directory is ‘/home/your_user’, with the Couch and it’s dependencies ending up in ‘/home/your_user/db/couch/’.

# Erlang 5.6.5
wget http://www.erlang.org/download/otp_src_R12B-5.tar.gz
tar -zxvf otp_src_R12B-5.tar.gz
cd otp_src_R12B-5
./configure --prefix=/home/your_user/db/couch/erlang
make && make install

# SpiderMonkey 1.7.0
wget http://ftp.mozilla.org/pub/mozilla.org/js/js-1.7.0.tar.gz
tar -zxvf js-1.7.0.tar.gz
cd js/src/
make -f Makefile.ref
make BUILD_OPT=1 JS_DIST=/home/your_user/db/couch/spidermonkey/ \
  -f Makefile.ref export

# ICU 4.6
wget http://download.icu-project.org/files/icu4c/4.6/icu4c-4_6-src.tgz
tar -zxvf icu4c-4_6-src.tgz
cd icu
./configure --prefix=/home/your_user/db/couch/icu
make && make install

# Curl 7.21.2
wget http://curl.haxx.se/download/curl-7.21.2.tar.gz
tar -zxvf curl-7.21.2.tar.gz
cd curl-7.21.2
./configure --prefix=/home/your_user/db/couch/curl
make && make install

# Add the programs to your $PATH
export PATH=$PATH:/home/your_user/db/couch/erlang/bin
export PATH=$PATH:/home/your_user/db/couch/spidermonkey/bin
export PATH=$PATH:/home/your_user/db/couch/icu/bin
export PATH=$PATH:/home/your_user/db/couch/curl/bin
export LD_LIBRARY_PATH=/home/your_user/db/couch/icu/lib

# CouchDB 1.0.1
wget http://mirrors.ukfast.co.uk/sites/ftp.apache.org//couchdb/1.0.1/apache-couchdb-1.0.1.tar.gz
tar -zxvf apache-couchdb-1.0.1.tar.gz
cd apache-couchdb-1.0.1
## edit configure.  Search for CURL_LDFLAGS=-lcurl.
## Replace with: CURL_LDFLAGS="-L/home/your_user/db/couch/curl/lib -lcurl"
./configure --prefix=/home/your_user/db/couch/couchdb \
  --with-js-lib=/home/your_user/db/couch/spidermonkey/lib \
  --with-js-include=/home/your_user/db/couch/spidermonkey/include \
  --with-erlang=/home/your_user/db/couch/erlang/lib/erlang/usr/include
make && make install

# Start up CouchDB
/home/your_user/db/couch/couchdb/bin/couchdb
# Time to Relax.
# From this command you should see: {"couchdb":"Welcome","version":"1.0.1"}
curl http://127.0.0.1:5984/

Jan

Adding colour to your unix prompt (PS1) and some handy ls aliases

by Joe Wright

This is my standard .bashrc that I use on all my boxes with a bash prompt. It gives you colourful listings when you use ‘ls’ and some shorthand versions of common ‘ls’ parameters.

The PS1 is set to just show the username, directory and the command in a brighter colour than the output from commands:

# ls aliases that use colour
alias la='ls -a --color=auto'
alias lla='ls -la --color=auto'
alias lt='ls -ltr --color=auto'
alias ll='ls -l --color=auto'
alias ls='ls --color=auto'
alias lsh='ls -lSh --color=auto'

# Unix prompt: /user /directory $
PS1='\[\e[0;32m\]\u\[\e[m\] \[\e[1;34m\]\w\[\e[m\] \[\e[1;32m\]\$\[\e[m\] \[\e[1;37m\]'

# start a web server in this directory
alias rshare="ruby -rubygems -e \"['thin', 'rack', 'socket'].each {|file| require file };"\
" Thin::Server.start(IPSocket.getaddress(Socket.gethostname), 7777) {"\
" use Rack::CommonLogger; run Rack::Directory.new(Dir.pwd) }\""

Example in /bin:
ps1

Nov

102

New language features in Java 7

by Joe Wright

I’m just back from the Devoxx conference in Antwerp. An update was given on the new language changes that will be in Java 7. The JDK currently has a release date of September 2010.

Here are 7 of the new features that have been completed:

~~Language support for collections~~ ( Postponed to Java 8 )
Automatic Resource Management
Improved Type Inference for Generic Instance Creation (diamond)
Underscores in numeric literals
Strings in switch
Binary literals
Simplified Varargs Method Invocation

There is a lot more to Java 7 then just these language changes. I’ll be exploring the rest of the release in future posts. One of the big debates is currently around Closures, which are a separate JSR.

Language support for collections

This has been postponed to Java 8! You could use my simple alternative until then.

Java will be getting first class language support for creating collections. The style change means that collections can be created like they are in Ruby, Perl etc.

Instead of:

List<String> list = new ArrayList<String>();
list.add("item");
String item = list.get(0);

Set<String> set = new HashSet<String>();
set.add("item");

Map<String, Integer> map = new HashMap<String, Integer>();
map.put("key", 1);
int value = map.get("key");

You will be able to use:

List<String> list = ["item"];
String item = list[0];

Set<String> set = {"item"};

Map<String, Integer> map = {"key" : 1};
int value = map["key"];

These collections are immutable.

Automatic Resource Management

Some resources in Java need to be closed manually like InputStream, Writers, Sockets, Sql classes. This language feature allows the try statement itself to declare one of more resources. These resources are scoped to the try block and are closed automatically.

This:

BufferedReader br = new BufferedReader(new FileReader(path));
try {
   return br.readLine();
} finally {
   br.close();
}

becomes:

try (BufferedReader br = new BufferedReader(new FileReader(path)) {
   return br.readLine();
}

You can declare more than one resource to close:

try (
   InputStream in = new FileInputStream(src);
   OutputStream out = new FileOutputStream(dest))
{
 // code
}

To support this behaviour all closable classes will be retro-fitted to implement a Closable interface.

Improved Type Inference for Generic Instance Creation (diamond)

This is a particular annoyance which is best served with an example:

Map<String, List<String>> anagrams = new HashMap<String, List<String>>();

becomes:

Map<String, List<String>> anagrams = new HashMap<>();

This is called the diamond operator: <> which infers the type from the reference declaration.

Underscores in numeric literals

Long numbers are hard to read. You can now split them up using an underscore in ints and longs:

int one_million = 1_000_000;

Strings in switch

Currently you can only use numbers or enums in switch statements. String has been added as a candidate:

String s = ...
switch(s) {
 case "quux":
    processQuux(s);
    // fall-through

  case "foo":
  case "bar":
    processFooOrBar(s);
    break;

  case "baz":
     processBaz(s);
    // fall-through

  default:
    processDefault(s);
    break;
}

Binary literals

Java code, due to its C heritage, has traditionally forced programmers to represent numbers in only decimal, octal, or hexadecimal.

As quite a few domains are bit orientated, this restriction can introduce errors. You can now create binary numbers using an 0b prefix.

int binary = 0b1001_1001;

Simplified Varargs Method Invocation

When a programmer tries to invoke a *varargs* (variable arity) method with a non-reifiable varargs type, the compiler currently generates an “unsafe operation” warning. JDK 7 moves the warning from the call site to the method declaration. This will enable API designers to use varargs due to the reduction of warnings reported.

This one is slightly more involved so you are better off looking at the proposal.

Sep

Coding Dojo: Example Katas

by Joe Wright

In my previous post I discussed the basics of what a Coding Dojo is and why you’d want to have one.

I host a Coding Dojo at my work every month, so I’ve decided to periodically update an index page of example Katas. They are usually based off existing Katas which are available on the main Coding Dojo website. I’d also recommend the prolific Sao Paulo Coding Dojo.

You should be able to reuse the same problems at your own Dojo. If you have any other recommendations then please email/twitter me, or comment on this post.

Current Problems

Minesweeper Kata

On a slightly related note, I just completed the common Minesweeper Kata.

Matt Wynne, who has an excellent blog, recently created a test harness using the Ruby BDD framework called Cucumber. It acts as a great way to introduce you to BDD and Github if you haven’t used them before.

Sep

The Coding Dojo

by Joe Wright

Joe Wright talks about Coding Dojos from TechMeetup on Vimeo.

A Coding Dojo is a programming session based around a simple coding challenge. Programmers of different skill levels are invited to engage in deliberate practice as equals. The goal is to learn, teach and improve with fellow software developers in a non-competitive setting.

I’ve been running a Coding Dojo at JPMorgan in Glasgow for over a year and I’ve found it to be a fantastic way of encouraging people to stretch their coding muscles and meet other fellow developers who they haven’t met before.

Background

Researchers at Florida State University have been trying to find out what makes certain people experts in a craft. They have studied people recognised as world class in their fields like Tiger Woods, Warren Buffet and Luciano Pavarotti to find out whether they are excelling as a consequence of possessing innate gifts or due to hard work.

Their research found that elite performers in many diverse domains have been found to practice on average the same amount every day, including weekends, over a period of at least 10 years. There is an intrinsic link between deliberate practice and high levels of skill.

The renowned pianist Vladimir Horowitz supposedly said, “If I don’t practice for a day, I know it. If I don’t practice for two days, my wife knows it. If I don’t practice for three days, the world knows it.”

Dave Thomas seized upon this idea in his book The Pragmatic Programmer. He suggested that programmers should be practising small problems outside of their job regularly, repeating the same problem in order to learn from previous mistakes. To encourage people to practice he posted a series of Code Katas on his website. The word Kata comes from martial arts and is a practice of repeating a small movement repeatedly in order to perfect it.

One of the limitations with the Code Kata is that when you complete the task on your own, you don’t receive any feedback. An important positive stimulus found in research around deliberate practice shows that having a master in the craft available to you while you learn is crucial.

Laurent Bossavit, a determined Parisian, decided that practising Katas at home was keeping this beneficial element away from him. He created the Coding Dojo concept, where he would present a rehearsed Kata from scratch in front of an audience. His weekly Paris Dojo started in 2005 and it has since inspired many other programmers to start their own.

Principles and Rules

The premise of a Coding Dojo is to promote continuous learning in a safe environment. It is both non-competitive and inclusive. People of different skill levels collaborate together, encouraging each other to talk openly about design and techniques.

The rules used at a Dojo depend on which format the Dojo uses; one rule that must be adhered is that you elect to use either Test Driven Development or Behaviour Driven Development. These practices allow for greater testing of a codebase and allow developers to express their needs from that codebase.

Everyone participating should be able to go home and recreate a solution to the same problem. If one person doesn’t understand, then the coding should stop until they are confident with what is taking place.

Formats

All Dojos take place in a room with at least one laptop over two hours. There is an overhead projector available where people can display and speak about their code. The Dojo itself can have differing formats which control the different way developers interact.

Prepared Kata (Parisian)

The Parisian style of Dojo requires the most effort up front by an individual. A single developer starts from scratch and codes a Kata in front of an audience. The developer will have tried the problem a few times before and shows their solution to the crowd over the projector.

While the presenter creates their code they are constantly speaking to the audience letting them know exactly what they are doing and why. Any member of the audience is allowed to interrupt the presenter and ask them why they are coding in a certain pattern.

Randori (Finnish)

The Randori style of Dojo was created by the Agile Finland group as a way of making a more interactive Dojo. It involves developers programming in rotating pairs to design a solution to a problem. The problem and proposed programming language are voted on before the event by the participants.

No one is required to have done any research into the problem before the Dojo begins. Pairs are given a 7 minute time frame to code in front of the audience; once their 7 minutes are up, another person from the audience replaces a member of the pair and takes over.

This format requires dedicated concentration from the participants in order to be successful. Because of this a person is normally made product owner to make sure that everyone is paying attention to the progress the current pair are making.

Individual Pairs (Glaswegian)

Due to the upfront effort required of the prepared Kata and the dedication needed for a Randori to succeed, I’ve been using a different approach at JPMorgan. The format I use is to introduce a problem, then split the group into pairs who choose their own programming language to craft a solution in. After two hours of coding the pairs perform a short presentation on their codebase, showing the progress and design they’ve made.

Start a Coding Dojo at Work

Hosting a Coding Dojo is a very cheap and effective way to train and improve with your colleagues. All you need is a room with a projector and some willing developers with laptops. It doesn’t require much money and it’s not complicated to set up. There are a lot of sample problems available online already and participants don’t have to attend every Dojo.

A Coding Dojo represents a different challenge to developers. It encourages them to speak to an audience about their code, which is a different skill to producing it. These skills are developed during a Dojo as it stimulates social and self organisational skills. It makes people become more rounded software engineers and discover potential areas of improvement.

So what’s stopping you?

Attending a Coding Dojo can help you improve enormously; it is a fun and challenging experience you share with other developers. It encourages deliberate practice by working together on a common problem during which you can learn, teach, accept criticism and defend your ideas. You will receive feedback not only on the code you produce, but the techniques you use. You will be exposed to new ways to use your favourite language and some different tricks you can perform with your code editor.

As Ron Jeffries points out “What changes people is what they do, not what they read. How many diet books have I read? Am I thinner?…”

Checking JMX with JRuby

A DSL for collections in Java

Start a web server from your pwd using Ruby and Thin

Creating a simple Java RESTful service using Jersey and Maven

The code

web.xml

The Java class

Where to learn more

Renaming MP3 tags with Ruby and the mp3info gem

Installing CouchDB 1.0.1 on RHEL4 as the local user

Adding colour to your unix prompt (PS1) and some handy ls aliases

New language features in Java 7

Language support for collections

Automatic Resource Management

Improved Type Inference for Generic Instance Creation (diamond)

Underscores in numeric literals

Strings in switch

Binary literals

Simplified Varargs Method Invocation

Coding Dojo: Example Katas

Current Problems

Minesweeper Kata

The Coding Dojo

Background

Principles and Rules

Formats

Start a Coding Dojo at Work

So what’s stopping you?

Further Reading

About

Twitter: @joe_jag

Categories