Hadoop Oozie学习笔记(七) E0903: Configuration does not have Jobtracker Kerberos异常解决
本博客属原创文章,转载请注明出处:http://guoyunsky.iyunv.com/blog/1396801欢迎加入Hadoop超级群: 180941958
最近开始监控Oozie所有workflow job,比如跟踪所有workflow job,有失败的job就邮件等通知.或者重启失败的job等.
这个异常就是发生在rerun的过程中.完整的异常如下:
Unknown exception in doAs
java.lang.reflect.UndeclaredThrowableException: Unknown exception in doAs
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
at org.apache.oozie.service.KerberosHadoopAccessorService.createFileSystem(KerberosHadoopAccessorService.java:194)
at org.apache.oozie.service.AuthorizationService.authorizeForApp(AuthorizationService.java:283)
at org.apache.oozie.servlet.BaseJobServlet.checkAuthorizationForApp(BaseJobServlet.java:171)
at org.apache.oozie.servlet.BaseJobServlet.doPut(BaseJobServlet.java:110)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:640)
at org.apache.oozie.servlet.JsonRestServlet.service(JsonRestServlet.java:281)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.oozie.servlet.AuthFilter$2.doFilter(AuthFilter.java:123)
at com.cloudera.alfredo.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:371)
at org.apache.oozie.servlet.AuthFilter.doFilter(AuthFilter.java:128)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.security.PrivilegedActionException: org.apache.oozie.service.HadoopAccessorException: E0903: Configuration does not have Jobtracker Kerberos name or rule
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
... 24 more
Caused by: org.apache.oozie.service.HadoopAccessorException: E0903: Configuration does not have Jobtracker Kerberos name or rule
at org.apache.oozie.service.KerberosHadoopAccessorService.validateConf(KerberosHadoopAccessorService.java:244)
at org.apache.oozie.service.KerberosHadoopAccessorService.access$000(KerberosHadoopAccessorService.java:49)
at org.apache.oozie.service.KerberosHadoopAccessorService$3.run(KerberosHadoopAccessorService.java:202)
at org.apache.oozie.service.KerberosHadoopAccessorService$3.run(KerberosHadoopAccessorService.java:194)
... 27 more
网上并没有找到解决方案,也不清楚 Jobtracker Kerberos到底是啥玩意.
于是跟踪下源码看,终于搞定.
1.解决方案:
只要在你的workflow/wf_job.properties,也就是你的workflow配置文件加上以下代码就行:
mapreduce.jobtracker.kerberos.principal=mapred/_HOST@LOCALHOST
dfs.namenode.kerberos.principal=hdfs/_HOST@LOCALHOST
2.原理:
跟踪源码发现,有如下判断之后才会触发该异常:
private void validateConf(Configuration conf) throws HadoopAccessorException {
if (conf.get(WorkflowAppService.HADOOP_JT_KERBEROS_NAME) == null) {
throw new HadoopAccessorException(ErrorCode.E0903);
}
if (conf.get(WorkflowAppService.HADOOP_NN_KERBEROS_NAME) == null) {
throw new HadoopAccessorException(ErrorCode.E0904);
}
}
从中可以发现只要Configuration中包含WorkflowAppService.HADOOP_JT_KERBEROS_NAME也就是
mapreduce.jobtracker.kerberos.principal和WorkflowAppService.HADOOP_NN_KERBEROS_NAME(dfs.namenode.kerberos.principal)即可.所以可以在配置文件,或者hadoop/conf/中配置.
更多文章、感悟、分享、勾搭，请用微信扫描:
页:
[1]