Hello fellow spark users !
We are wondering, for a couple of month yet, if you would be interested in a spark tool ?
We think that managing spark cluster can be painful, especially for dev jobs. That is why we created a tool which spawn a spark cluster on the go when launching a `spark-submit` like command.
To sum up, to make it work we need :
* an OVH PCI project
* an OpenStack token
* install a binary called : `ovh-spark-submit`
Then to run a job (and spawn a cluster to run it) just type for example:
`ovh-spark-submit --OS_TOKEN {your token} --class com.ovh.example.SparkScalaApp --name SparkJob1 --executor-memory 2G --total-executor-cores 4 spark-word-count-assembly-0.1.jar`
No need to have spark client / libs installed, cluster size is computed according to `--executor-memory` and `--total-executor-cores` values.
Do you want us to add it in labs.ovh.com ?
We really would like to get your insight :)
Public Cloud OVHcloud - Spark on the go lab ?
Related questions
- Dimensionnement serveur MySQL
44609
07.11.2018 12:32
- [RESOLU] Connexion impossible en SSH
37252
05.06.2019 20:05
- Bonjour, Je n'est reçus aucun mot de passe root lors de mon achat!
32623
05.02.2018 20:47
- Gitlab private docker registry
32413
16.03.2018 13:05
- Ssh connection timed out port 22
31625
11.12.2019 08:21
- Configuration IP failover avec netplan (Ubuntu 17.10)
31185
12.01.2018 23:23
- Problème connexion ssh
30865
04.02.2018 09:46
- IP Failover sur Debian 9
30574
18.11.2016 20:40
- Instance Public Cloud en "error"
28061
15.12.2025 10:04
- Connexion OpenStack Swift Object Storage
24319
11.04.2019 10:09