Spark 2.4.1-foss-2019b-Python-3.7.4-Java-1.8

There is a newer version of Spark

Spark is Hadoop MapReduce done in memory

Accessing Spark 2.4.1-foss-2019b-Python-3.7.4-Java-1.8

To load the module for Spark 2.4.1-foss-2019b-Python-3.7.4-Java-1.8 please use this command on the BEAR systems (BlueBEAR, BEAR Cloud VMs, and CaStLeS VMs):

module load Spark/2.4.1-foss-2019b-Python-3.7.4-Java-1.8

BEAR Apps Version

2020a

Extensions

  • py4j 0.10.7

More Information

For more information visit the Spark website.

Dependencies

This version of Spark has a direct dependency on: Arrow/0.16.0-foss-2019b-Python-3.7.4 foss/2019b Hadoop/2.10.0-GCCcore-8.3.0-native Java/1.8 Python/3.7.4-GCCcore-8.3.0

Required By

This version of Spark is a direct dependent of: Hail/0.2.61-foss-2019b-Python-3.7.4-Java-1.8

Other Versions

These versions of Spark are available on the BEAR systems (BlueBEAR, BEAR Cloud VMs, and CaStLeS VMs). These will be retained in accordance with our Applications Support and Retention Policy.

Last modified on 27th January 2021