SPARK SHUFFLE TUNING

WHEN DOES SHUFFLE OCCUR? A shuffle can occur when the resulting RDD from a transformation depends on other elements from the same or another RDD. But the above statement is not completely valid, for example, let us take a map operation: [crayon-5d38ef84745a3523890486/] The above map operation results in a PairedRDD with 1 attached to each […]
Source: Imaginea

Leave a Reply

Your email address will not be published.


*