官方是这样说的:
Directory to use for "scratch" space in Spark, including map output files and RDDs that get stored on disk. This should be on a fast, local disk in your system. It can also be a comma-separated list of multiple directories on different disks. NOTE: In Spark 1.0 and later this will be overriden by SPARK_LOCAL_DIRS (Standalone, Mesos) or LOCAL_DIRS (YARN) environment variables set by the cluster manager.
大概就是MAP和RDD的临时数据可能存放在spark.local.dir(SPARK_LOCAL_DIRS)指定的目录中,默认是在/tmp中。
该目录的磁盘一定是快速的本地磁盘系统,我们可以考虑把这个目录来单独放成一个磁盘,或者以都好隔开放入多个磁盘,来提高i/o读写速度