Package

io.radanalytics.silex.rdd

multiplex

Permalink

package multiplex

Visibility
  1. Public
  2. All

Type Members

  1. class MuxRDDFunctions[T] extends Serializable

    Permalink

    Enhance RDDs with methods for generating multiplexed RDDs

    Enhance RDDs with methods for generating multiplexed RDDs

    T

    the element type of the RDD

    // enable multiplexing methods
    import io.radanalytics.silex.rdd.multiplex.implicits._
    // A boolean predicate on data elements
    val pred: Int => Boolean = ....
    // pos will contain data elements for which 'pred' was true.
    // neg will contain elements for which 'pred' was false.
    val (pos, neg) = data.flatMuxPartitions((data: Iterator[Int]) => {
      val pT = scala.collection.mutable.ArrayBuffer.empty[Int]
      val pF = scala.collection.mutable.ArrayBuffer.empty[Int]
      data.foreach { e => (if (pred(e)) pT else pF) += e }
      (pT, pF)
    })

Value Members

  1. object MuxRDDFunctions extends Serializable

    Permalink

    Definitions used by MuxRDDFunctions instances

  2. object implicits

    Permalink

    Implicit conversions to enhance RDDs with multiplexing methods

Ungrouped