An error occurred while trying to connect to the Java server

Hi,

I’ve trying datalore to some data engineering exploration, I’ve tried PySpark, the package has been successfully imported, whereas when intializing sparkSession:

spark = SparkSession.builder.appName("try datalore").getOrCreate()
df = spark.read.json("xxx/data.json")
print(df.count())

I got the following error:

Traceback (most recent call last):
  at block 3, line 3
  at /opt/anaconda3/envs/datalore-user/lib/python3.7/site-packages/pyspark/sql/session.py, line 183, in getOrCreate
  at /opt/anaconda3/envs/datalore-user/lib/python3.7/site-packages/py4j/java_gateway.py, line 1255, in __call__
  at /opt/anaconda3/envs/datalore-user/lib/python3.7/site-packages/py4j/java_gateway.py, line 983, in send_command
  at /opt/anaconda3/envs/datalore-user/lib/python3.7/site-packages/py4j/java_gateway.py, line 931, in _get_connection
  at /opt/anaconda3/envs/datalore-user/lib/python3.7/site-packages/py4j/java_gateway.py, line 937, in _create_connection
  at /opt/anaconda3/envs/datalore-user/lib/python3.7/site-packages/py4j/java_gateway.py, line 1079, in start
Py4JNetworkError: An error occurred while trying to connect to the Java server (127.0.0.1:41171)

It turned out, Datalore Kernel doesn’t work well with pyspark – as a workaround you can switch to ipykernel mode.

Thanks for letting us know!