http://spark.apache.org/docs/latest/rdd-programming-guide.html
The first thing a Spark program must do is to create a SparkContext object, which tells Spark how to access a cluster. To create a SparkContext
you first need to build a SparkConf object that contains information about your application.
Only one SparkContext may be active per JVM. You must stop()
the active SparkContext before creating a new one.
Configuration for a Spark application. Used to set various Spark parameters as key-value pairs.
All setter methods in this class support chaining. For example, you can write
new SparkConf().setMaster("local").setAppName("My app")
.
The appName
parameter is a name for your application to show on the cluster UI. master
is a Spark, Mesos or YARN cluster URL, or a special “local” string to run in local mode. In practice, when running on a cluster, you will not want to hardcode master
in the program, but rather launch the application with spark-submit
and receive it there. However, for local testing and unit tests, you can pass “local” to run Spark in-process.