[FEATURE REQUEST]: Show Scala version in microsoft-spark*.jar filename

[imback82]

Currently, scala version is not embedded in the microsoft-spark JAR name. This will be useful for the user when Spark 3.0 is released which uses 2.12 scala by default.

[imback82]

One suggestion is:
current: microsoft-spark-2.4.x-0.6.0.jar
new: microsoft-spark-2.4.x_2.11-0.6.0.jar

Any other ideas? @suhsteve @rapoth @Niharikadutta @elvaliuliuliu @AFFogarty

[elvaliuliuliu]

@imback82 May I ask why we need to add scala version in our jar filename? It seems like Spark 3.0 and most Spark 2.4.x uses 2.12.x scala (except Spark 2.4.0 uses 2.11.x scala), and Spark 2.3.x uses 2.11.x scala.

[imback82]

we want to have the consistent versioning with other scala packages in Maven (we plan to put these jars in Maven for livy integration).

[elvaliuliuliu]

Cool, I think microsoft-spark-2.4.x_2.11-0.6.0.jar is clear for me. Thanks.

[suhsteve]

I think we should put the scala version in front. I currently see jars in HDInsight named this way.

proposed format: microsoft-spark_<scala version>-<spark version>-<dotnet version>.jar
example: microsoft-spark_2.11-2.4.x-0.6.0.jar

Should we keep the .x ?

[imback82]

microsoft-spark-3.0_2.12-1.0.0.jar
microsoft-spark-2.4_2.11-1.0.0.jar

[imback82]

We will go with:

microsoft-spark-3-0_2.12-1.0.0.jar
microsoft-spark-2-4_2.11-1.0.0.jar

, simliar to https://github.com/apache/spark/blob/master/external/kafka-0-10/pom.xml