How To Download Spark On Windows

Apache Spark - Installation

Spark is Hadoop's sub-project. Therefore, information technology is better to install Spark into a Linux based system. The following steps testify how to install Apache Spark.

Step 1: Verifying Java Installation

Java installation is ane of the mandatory things in installing Spark. Effort the post-obit control to verify the Coffee version.

$java -version

If Java is already, installed on your system, you get to see the following response −

java version "1.seven.0_71"  Java(TM) SE Runtime Environs (build 1.7.0_71-b13)  Java HotSpot(TM) Customer VM (build 25.0-b02, mixed mode)

In case y'all exercise not take Coffee installed on your organization, so Install Java earlier proceeding to next step.

Step 2: Verifying Scala installation

You should Scala language to implement Spark. So let us verify Scala installation using post-obit control.

$scala -version

If Scala is already installed on your system, you become to run across the post-obit response −

Scala lawmaking runner version 2.11.6 -- Copyright 2002-2013, LAMP/EPFL

In case you don't have Scala installed on your system, then go on to side by side step for Scala installation.

Stride 3: Downloading Scala

Download the latest version of Scala by visit the following link Download Scala. For this tutorial, we are using scala-2.xi.6 version. Afterwards downloading, y'all will find the Scala tar file in the download folder.

Step 4: Installing Scala

Follow the below given steps for installing Scala.

Excerpt the Scala tar file

Type the following command for extracting the Scala tar file.

$ tar xvf scala-2.eleven.6.tgz

Move Scala software files

Utilize the following commands for moving the Scala software files, to respective directory (/usr/local/scala).

$ su –  Password:  # cd /abode/Hadoop/Downloads/  # mv scala-2.11.6 /usr/local/scala  # exit

Set PATH for Scala

Utilize the following command for setting PATH for Scala.

$ consign PATH = $PATH:/usr/local/scala/bin

Verifying Scala Installation

After installation, it is better to verify it. Utilize the following control for verifying Scala installation.

$scala -version

If Scala is already installed on your system, you lot get to see the following response −

Scala code runner version ii.xi.six -- Copyright 2002-2013, LAMP/EPFL

Step 5: Downloading Apache Spark

Download the latest version of Spark past visiting the following link Download Spark. For this tutorial, we are using spark-1.3.1-bin-hadoop2.vi version. Later on downloading it, you volition find the Spark tar file in the download folder.

Step 6: Installing Spark

Follow the steps given below for installing Spark.

Extracting Spark tar

The following control for extracting the spark tar file.

$ tar xvf spark-1.3.1-bin-hadoop2.6.tgz

Moving Spark software files

The following commands for moving the Spark software files to respective directory (/usr/local/spark).

$ su –  Password:    # cd /dwelling house/Hadoop/Downloads/  # mv spark-1.3.1-bin-hadoop2.6 /usr/local/spark  # go out

Setting upward the environment for Spark

Add the following line to ~/.bashrc file. It ways calculation the location, where the spark software file are located to the PATH variable.

consign PATH=$PATH:/usr/local/spark/bin

Use the post-obit command for sourcing the ~/.bashrc file.

$ source ~/.bashrc

Step 7: Verifying the Spark Installation

Write the following command for opening Spark crush.

$spark-shell

If spark is installed successfully and so you will find the following output.

Spark assembly has been built with Hive, including Datanucleus jars on classpath  Using Spark's default log4j profile: org/apache/spark/log4j-defaults.backdrop  15/06/04 15:25:22 INFO SecurityManager: Irresolute view acls to: hadoop  fifteen/06/04 15:25:22 INFO SecurityManager: Changing modify acls to: hadoop 15/06/04 15:25:22 INFO SecurityManager: SecurityManager: authentication disabled;    ui acls disabled; users with view permissions: Fix(hadoop); users with modify permissions: Set(hadoop)  xv/06/04 15:25:22 INFO HttpServer: Starting HTTP Server  15/06/04 15:25:23 INFO Utils: Successfully started service 'HTTP grade server' on port 43292.  Welcome to        ____              __       / __/__  ___ _____/ /__      _\ \/ _ \/ _ `/ __/  '_/     /___/ .__/\_,_/_/ /_/\_\   version 1.4.0        /_/   		 Using Scala version ii.x.iv (Coffee HotSpot(TM) 64-Bit Server VM, Java 1.7.0_71)  Type in expressions to have them evaluated.  Spark context available every bit sc   scala>