Operators that deserve to be better known: part IV

The "Materialize Data" operator can be very useful. It has the effect of making a fresh copy of the data which I take to mean the example set or sets. Using this operator can often cure some tricky problems.

As an example, here is a process that does a simple cross validation on the sample sonar data set (you may have to change the location for your setup). Inside the cross validation the intermediate training and test data sets are remembered. In a later loop, these saved example sets are recalled. This shows details of how cross validation works.

The "Remember" operator fails to save the correct example if the preceding "Materialize Data" is disabled. In addition, the collection that is recalled later does not display correctly (on my 64 bit Windows machine). Enabling the "Materialize Data" operator restores correct operation.

Reading and writing xlsx files in RapidMiner

The other day, I stumbled on a nifty way to import and export xlsx files into and out of RapidMiner.

There's an R package called "xlsx" that is able to read and write Excel files including Excel 2007 xlsx format. 

Here's a process that loads the iris data, writes this to an xlsx file at c:\temp\iris.xlsx and then reads it back again.

(Note: after the first run, comment out the line in the R script that installs the xlsx package to avoid downloading the R package again)

Edit: 12/7/12: The new version of RapidMiner supports this - hurrah!

Deleting attributes with a single valid value

Removing attributes where one or more of the examples are missing is easy using the "Select Attributes" operator with the option "no missing values".

If however, you want to additionally remove attributes where only a single example is valid and all the rest are missing, a neat way to do this is to use the "Remove Useless Attributes" operator. One of the parameters to this is "numerical min deviation" and this will remove any attribute with a deviation less than or equal to the value supplied which defaults to 0. Attributes with only a single valid value will have a deviation of 0 and will therefore be removed by this operator.