Skip to content

Cluster Master URL setting overridden #103

@whereswaldon

Description

@whereswaldon

It seems that, when running eclairjs-node against the eclairjs/minimal-gateway docker container, the URL of the spark cluster that you want to talk to can be overridden by the Scala SparkContext that Toree creates by default. This results in jobs never reaching the targeted cluster and being executed locally instead.

// Launch Spark instance
var spark = require('eclairjs');
var sparkContext = new spark.SparkContext('spark://X.X.X.X:8443', "dataframe test");

The above, when run with JUPYTER_PORT and JUPYTER_HOST pointing to the eclairjs/minimal-gateway docker container, runs locally (on the docker container) instead of on the specified cluster.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions