ct38 发表于 2018-10-29 13:41:36

Eclipse导入Hadoop 源码

  1.准备工作
  jdk:
  eclipse:
  Maven:
  libprotoc :https://developers.google.com/protocol-buffers/
  hadoop:http://www.apache.org/dyn/closer.cgi/hadoop/common/
  添加开源中国maven库:http://maven.oschina.net/home.html
  maven\conf\settings.xml
  path
  
  
  nexus-osc
  *
  Nexus osc
  http://maven.oschina.net/content/groups/public/
  
  
  2.导入
  将hadoop源码解压到一个目录,注意目录层次不要太深,否则可能无法解压。
  进入hadoop-maven-plugins文件夹,执行 mvn install
  返回源码根目录,执行 mvn eclipse:eclipse –DskipTests
  eclipse在任意目录创建新的WorkSpace
  eclipse设置Maven:window->preference->maven->{Installations...;user Settings:maven\conf\settings.xml}
  eclipse:File->inport->Existing Projects into WorkSpace->Hadoop源码根目录
  3.错误处理

[*]  maven下载pom失败->重新操作
[*]  hadoop-streaming中build path错误->Java Build Path->Source:

[*]  删除...hadoop-yarn-server-resourcemanager/conf
[*]  Link Source:源码根目录/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop- yarn-server-resourcemanager/conf,再随便起个名字;inclusion patterns:capacity-scheduler.xml;exclusion patters:**/*.java

[*]  org.apache.hadoop.io.serializer.avro.TestAvroSerialization
[*]  下载avro-tools-1.7.4.jar:http://archive.apache.org/dist/avro/avro-1.7.4/java/
[*]  进入目录:源码根目录\hadoop-common-project\hadoop-common\src\test\avro
[*]  java -jar path/to/avro-tools-1.7.4.jar compile schema avroRecord.avsc ..\java
[*]  eclipse 刷新
[*]  进入目录:源码根目录\hadoop-common-project\hadoop-common\src\test\proto
[*]  protoc --java_out=../java *.proto
[*]  eclipse 刷新
[*]  org.apache.hadoop.ipc.protobuf.TestProtos
[*]  project->clean..->clean all projects & Build the entire workspace

页: [1]
查看完整版本: Eclipse导入Hadoop 源码