Friday, 17 June 2016

Spark: java.sql.SQLException: No suitable driver


Trying to retrieve data in using SQL context, displaying below error message


1. Invoke Spark-shell
2) Run below code in sql context

import org.apache.spark.sql.SQLContext
val sqlContext = new SQLContext(sc)
val url = "jdbc:mysql://localhost:3306/retail_db?user=root&password=root"
sqlContext.load("jdbc", Map("url" -> url,"dbtable" -> "departments")).collect().foreach(println)

3) Error shown as below at cmd prompt:
scala> sqlContext.load("jdbc", Map("url" -> url,"dbtable" -> "departments")).collect().foreach(println)
warning: there were 1 deprecation warning(s); re-run with -deprecation for details
java.sql.SQLException: No suitable driver
        at java.sql.DriverManager.getDriver(DriverManager.java:278)
        at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$2.apply(JdbcUtils.scala:50)
        at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$2.apply(JdbcUtils.scala:50)
        at scala.Option.getOrElse(Option.scala:120)
        at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.createConnectionFactory(JdbcUtils.scala:49)
        at org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD$.resolveTable(JDBCRDD.scala:120)
        at org.apache.spark.sql.execution.datasources.jdbc.JDBCRelation.<init>(JDBCRelation.scala:91)
        at org.apache.spark.sql.execution.datasources.jdbc.DefaultSource.createRelation(DefaultSource.scala:57)
        at org.apache.spark.sql.execution.datasources.ResolvedDataSource$.apply(ResolvedDataSource.scala:158)

Fix:
Invoke shell using below command 

spark-shell --driver-class-path /usr/local/hadoop/mysql-connector-java-5.1.39/mysql-connector-java-5.1.39-bin.jar

Execute above source code
scala> import org.apache.spark.sql.SQLContext
import org.apache.spark.sql.SQLContext

scala> val url = "jdbc:mysql://localhost:3306/retail_db?user=root&password=root"
url: String = jdbc:mysql://localhost:3306/retail_db?user=root&password=root

scala> sqlContext.load("jdbc", Map("url" -> url,"dbtable" -> "departments")).collect().foreach(println)
warning: there were 1 deprecation warning(s); re-run with -deprecation for details

Result: 
[2,Fitness]
[3,Footwear]
[4,Apparel]
[5,Golf]
[6,Outdoors]
[7,Fan Shop]


No comments:

Post a Comment