How to eyeball your clusters

A common task when dealing with multidimensional data is to cluster the data somehow. People have been known to spend a lot of time worrying about the clustering method, when really it's the distance measure (or implicit distance measure) that's more important. Garbage in, garbage out. So long as you don't use single or complete linkage without a good reason, you'll be fine.

I'm a great believer in looking at the data. That is, checking it out visually. This is mainly as a sanity check. How real are these clusters? Did I forget to scale the data so that everything is clustered on the basis of the variable with the biggest range?

The following R code clusters using the non-hierarchical method k-means clustering (so no nice dendrogram). Once all the points have been assigned to a particular cluster you can look at the data in 2D or 3D (using principal coordinate analysis, aka classical multidimensional scaling) and colour the points by cluster:

data <- read.table("Boston.txt")
data <- scale(data)

myclust <- kmeans(data, 10)
print(myclust)
summary(myclust)
myclust$size
myclust$cluster

# Represent the data in 2D and colour by cluster
distances <- dist(data, method="euclidean")
mycmdscale <- cmdscale(distances, 2)
plot(mycmdscale, cex=0)
points(mycmdscale, col=myclust$cluster)

# Let's try 3D (you need to install scatterplot3d first)
library(scatterplot3d)
mycmdscale <- cmdscale(distances, 3)
s3d <- scatterplot3d(mycmdscale, color=myclust$cluster)

# Let's try interactive 3D
library(rgl) # Need to install this package first
plot3d(mycmdscale, col=myclust$cluster, size=5)

Thanks to Rajarshi for pointing out how to generate the interactive 3D plot.

Using R from Python - the best of both worlds

R is a programming environment containing every known statistical method; however, it's ugly and difficult to program in. Python has only a small number of statistical methods (which are available through SciPy), but it's elegant and easy to program in. Both have graphing libraries which leave a lot to be desired in terms of ease of use, but the graphs from Python's matplotlib are a lot more polished. So, I'd like to use R from Python...

In order to do this, you will need to install the RPy library (as well as Python and R, of course). I'm working on Windows, but I've used this previously on Linux. As an example, here are three ways to access the same hclust object from Python. They each illustrate different aspects of the Python/R interface:

from rpy import *

# START OF METHOD 1
hclust = r("""
    a <- read.table("cpOfSimMatrix.txt")
    mydist <- dist(1-a)
    hclust(mydist)
""")
r("rm(a)") # Ends with an error if you leave anything in memory
# END OF METHOD 1

# START OF METHOD 2
set_default_mode(NO_CONVERSION)
a = r.read_table("cpOfSimMatrix.txt")
mydist = r.dist(r["-"](1,a)) # Note the trick for '1-a' here
set_default_mode(BASIC_CONVERSION)
hclust = r.hclust(mydist) # Converts R object to Python dict
# END OF METHOD 2

# START OF METHOD 3 (here's one I created earlier)
r.load(".RData")
hclust = r('myHclust')
# END OF METHOD 3

For further examples, check out Peter Cock's excellent pages on combining R and Python.