Spark 2.4.4-foss-2019a-Hadoop-2.7-Java-11-Python-3.7.2

There is a newer version of Spark

Spark is Hadoop MapReduce done in memory

Accessing Spark 2.4.4-foss-2019a-Hadoop-2.7-Java-11-Python-3.7.2

To load the module for Spark 2.4.4-foss-2019a-Hadoop-2.7-Java-11-Python-3.7.2 please use this command on the BEAR systems (BlueBEAR, BEARCloud VMs, and CaStLeS VMs):

module load Spark/2.4.4-foss-2019a-Hadoop-2.7-Java-11-Python-3.7.2

BEAR Apps Version

2020a

Extensions

  • py4j 0.10.8.1
  • pyspark 2.4.4

More Information

For more information visit the Spark website.

Dependencies

This version of Spark has a direct dependency on: foss/2019a Java/11 Python/3.7.2-GCCcore-8.2.0

Other Versions

These versions of Spark are available on the BEAR systems (BlueBEAR, BEARCloud VMs, and CaStLeS VMs). These will be retained in accordance with our Applications Support and Retention Policy.

Last modified on 23rd September 2019