Pyspark read from redshift

from pyspark.sql import SQLContext

sc = # existing SparkContext
sql_context = SQLContext(sc)

# Read data from a table
df = sql_context.read \
    .format("com.databricks.spark.redshift") \
    .option("url", "jdbc:redshift://redshifthost:5439/database?user=username&password=pass") \
    .option("dbtable", "my_table") \
    .option("tempdir", "s3n://path/for/temp/data") \
    .load()

Similar codes

9 votes99 views6 answers
Convert dataframe to float
Python
0 vote95 views1 answer
Check if path is a folder python
Python
0 vote116 views1 answer
Matplotlib dateformatter x axis
Python
6 votes125 views5 answers
Django logout
Python Django
11 votes166 views4 answers
Append dataframe to another dataframe
Python
1 vote110 views1 answer
Python fme logger
Python
1 vote100 views1 answer
Python print percent sign
Python
0 vote97 views1 answer
Create a dictionary from index and column pandas
Python
0 vote98 views1 answer
Queryset.raw() in django rest framework joining tables
Python
17 votes102 views8 answers
Jupyter notebook
Python