-
Notifications
You must be signed in to change notification settings - Fork 37
Open
Description
It seems that, when running eclairjs-node against the eclairjs/minimal-gateway docker container, the URL of the spark cluster that you want to talk to can be overridden by the Scala SparkContext that Toree creates by default. This results in jobs never reaching the targeted cluster and being executed locally instead.
// Launch Spark instance
var spark = require('eclairjs');
var sparkContext = new spark.SparkContext('spark://X.X.X.X:8443', "dataframe test");The above, when run with JUPYTER_PORT and JUPYTER_HOST pointing to the eclairjs/minimal-gateway docker container, runs locally (on the docker container) instead of on the specified cluster.
Metadata
Metadata
Assignees
Labels
No labels