Search this blog

Showing posts with label Subprocess. Show all posts
Showing posts with label Subprocess. Show all posts

Sunday, 21 July 2013

Scaling attribute values using weights

Here's a process that multiplies each value of an attribute within one example set by a constant in another example set. The constants are specific for each attribute and the process uses weights derived from the example set. In effect, a matrix multiplication is happening.

At a high level, the process works as follows.

  1. The Iris data set is used with weights being produced using "Weight By Information Gain"
  2. These weights are transformed into an example set and stored for later use inside a Loop operator
  3. A subprocess is used to make sure everything works in the right order (this technique is also used inside the Loop).
  4. A "Loop Attributes" operator iterates over all attributes and generates a new attribute based on multiplying the existing value by a weight. The attribute name is required to be contained in the weights example set. 
  5. The weight for each example is calculated with a combination of filtering and macro extraction.

Thursday, 22 November 2012

Using the Subprocess operator to stop operator execution order from changing

When editing a large process containing a lot of branches where calculations are done for use later, the operator execution order can go wrong causing the process to fail.

For example, the following shows a relatively complex process with the operator execution order overlaid.


The operators in steps 23 to 25 are calculating a macro that is used in step 13. Clearly, the order is wrong and the process is broken and this can just happen as a result of editing the process. Luckily, this is easy to fix by using the "bring operator to front" operation but I use a trick with the "Subprocess" operator.

The following shows this.


The "Subprocess" operator at what is now step 11 fixes everything. The example set output from step 10 is not required but joining it to step 11 forces it to be executed before all the other steps that might use the macros it is calculating.

Inside the "Subprocess" operator the first input is simply copied directly to the output. The second input is not connected to anything. The following shows this.


As a bonus feature, it is also possible to use this operator to untangle and "neaten up" complex workflows as shown below in a hypothetical example.


The first input is connected to the fifth output, the second to the fourth and so on. There are no restrictions on the type of data flowing through the "Subprocess" operator so this is easy to implement with any type of connection.