Apache Spark

Pyspark

To use pyspark in Deepnote, you need to install a JDK first.

! sudo apt-get update
! sudo mkdir -p /usr/share/man/man1
! sudo apt-get install -y openjdk-11-jdk
! pip install pyspark

Now you can test it in a notebook cell with:

from pyspark import SparkContext
sc = SparkContext("local", "First App")

or in a terminal by opening an interactive shell:

pyspark