Connect to spark cluster

from pyspark.conf import SparkConf
from pyspark.sql import SparkSession, Catalog

spark_conf = SparkConf()
spark_conf.setAll([
    ('spark.master', 'spark://<master-ip>:7077'),
    ('spark.app.name', 'myApp'),
    ('spark.submit.deployMode', 'client'),
    ('spark.ui.showConsoleProgress', 'true'),
    ('spark.eventLog.enabled', 'false'),
    ('spark.logConf', 'false'),
    ('spark.driver.bindAddress', '0.0.0.0'),
    ('spark.driver.host', '<master-ip>')
])

spark = SparkSession.builder.config(conf=spark_conf).getOrCreate()

Similar codes

1 vote198 views1 answer
Get all keys and values from dictionary python
Python
5 votes270 views6 answers
Python 3 cheat sheet
Python
1 vote203 views1 answer
Why is python so popular
Python
0 vote200 views1 answer
Ordering column names sensibly in pandas
Python
-1 vote190 views1 answer
Pandas df multiindex
Python
22 votes203 views4 answers
Join two numpy arrays
Python
0 vote221 views1 answer
Python logging to syslog linux
Python
0 vote189 views1 answer
Pandas astype str still object
Python
7 votes236 views4 answers
Pip update django
Python Django
0 vote191 views1 answer
Amazon response 503 python
Python