Spark 3.1.1-fosscuda-2020b

Unsupported: Use of this version of Spark is not supported. More information on our Applications Support and Retention Policy.

Spark is Hadoop MapReduce done in memory

Accessing Spark 3.1.1-fosscuda-2020b

To load the module for Spark 3.1.1-fosscuda-2020b please use this command on the BEAR systems (BlueBEAR, BEARCloud VMs, and CaStLeS VMs):

module load bear-apps-unsupported/handbuilt/2019
module load Spark/3.1.1-fosscuda-2020b

There is a CPU version of this module: Spark 3.1.1-foss-2020b

BEAR Apps Version

2019h

Extensions

  • py4j 0.10.9.2

More Information

For more information visit the Spark website.

Dependencies

This version of Spark has a direct dependency on: Arrow/0.17.1-fosscuda-2020b fosscuda/2020b Java/11 Python/3.8.6-GCCcore-10.2.0

Required By

This version of Spark is a direct dependent of: hyperopt/0.2.5-fosscuda-2020b

Other Versions

These versions of Spark are available on the BEAR systems (BlueBEAR, BEARCloud VMs, and CaStLeS VMs). These will be retained in accordance with our Applications Support and Retention Policy.

Last modified on 28th July 2021