Connect to spark cluster

from pyspark.conf import SparkConf
from pyspark.sql import SparkSession, Catalog

spark_conf = SparkConf()
spark_conf.setAll([
    ('spark.master', 'spark://<master-ip>:7077'),
    ('spark.app.name', 'myApp'),
    ('spark.submit.deployMode', 'client'),
    ('spark.ui.showConsoleProgress', 'true'),
    ('spark.eventLog.enabled', 'false'),
    ('spark.logConf', 'false'),
    ('spark.driver.bindAddress', '0.0.0.0'),
    ('spark.driver.host', '<master-ip>')
])

spark = SparkSession.builder.config(conf=spark_conf).getOrCreate()

Similar codes

0 vote175 views1 answer
Iterate over meta tag python
Python
1 vote176 views2 answers
Multiple values in python loop for x,y
Python
0 vote182 views1 answer
How can I keep the pyqt GUI responsive when performing a cpu intensive task?
Python
8 votes180 views2 answers
Min and max number in list python
Python
-1 vote179 views2 answers
How to clear combobox tkinter
Python
0 vote243 views2 answers
Flask api abort
Python Flask
40 votes203 views4 answers
Python datetime module
Python
-1 vote195 views1 answer
Store message sent by user in string discord py
Python
0 vote251 views9 answers
How to form a list from a file in python
Python
1 vote175 views1 answer
How to parse dicts in reqparse in flask
Python