Hive Stats Publisher

Error:  Whenever a new hive table is created or data is inserted into it,  Hive stats publisher runs and throws error shown below.

15/03/30 11:50:54 ERROR jdbc.JDBCStatsPublisher: Error during JDBC initialization.
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Specified key was too long; max key length is 767 bytes
         at sun.reflect.GeneratedConstructorAccessor119.newInstance(Unknown Source)
         at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)


Reason: unique key length is longer than 767 bytes. In the hive_stats_db schema, Hive uses some  keys which are taking more than 767 bytes length. 
generally we use uft8 character encoding. It requires 3 bytes per character. 
If we dont really need that much of characters to store our unique keys, shall we reduce the length of the columns?
latin1 = 1 byte = 1 chararcter
uft8 = 3 byte = 1 chararcter


Drop and recreate the hive_stats_db with latin charset instead of utf8
Before that ensure if that schema doesn’t have any object and the charset is only utf8
In the mysql db:

drop database hive_stats_db

CREATE DATABASE `hive_stats_db` /*!40100 DEFAULT CHARACTER SET latin1 */;

Comments

Popular Posts