multiplex

Type Members

1. class MuxRDDFunctions[T] extends Serializable

   

   Enhance RDDs with methods for generating multiplexed RDDs

   Enhance RDDs with methods for generating multiplexed RDDs

   T

   the element type of the RDD

   // enable multiplexing methods
   import io.radanalytics.silex.rdd.multiplex.implicits._
   // A boolean predicate on data elements
   val pred: Int => Boolean = ....
   // pos will contain data elements for which 'pred' was true.
   // neg will contain elements for which 'pred' was false.
   val (pos, neg) = data.flatMuxPartitions((data: Iterator[Int]) => {
     val pT = scala.collection.mutable.ArrayBuffer.empty[Int]
     val pF = scala.collection.mutable.ArrayBuffer.empty[Int]
     data.foreach { e => (if (pred(e)) pT else pF) += e }
     (pT, pF)
   })