compile hadoop-2.5.x on OS X(macbook)
same as compile hbase ,it 's necessary for u to compile your own dist of hadoop to get best performance specified to your hardwares.bellow are some steps needed by this theme:
1.install some tools
a.cmake 2.6+
download and set the path variable
this is used only if u want to make with native option,same like:
mvn package -Pdist,native -DskipTests=true-Dmaven.javadoc.skip=true
b.protobuf 2.5+
download and set variable
c.mavn 3.0+
same as above
2.comile hadoop by issuing
mvn package -Pdist,native -DskipTests=true-Dmaven.javadoc.skip=true -Dtar
the results will like below:
main:
$ tar cf hadoop-2.5.2.tar hadoop-2.5.2
$ gzip -f hadoop-2.5.2.tar
Hadoop dist tar available at: /home/xxx/build-hadoop/hadoop-2.5.2-src/hadoop-dist/target/hadoop-2.5.2.tar.gz
Executed tasks
--- maven-javadoc-plugin:2.8.1:jar (module-javadocs) @ hadoop-dist ---
Building jar: /home/xxx/build-hadoop/hadoop-2.5.2-src/hadoop-dist/target/hadoop-dist-2.5.2-javadoc.jar
------------------------------------------------------------------------
Reactor Summary:
Apache Hadoop Main ................................ SUCCESS
Apache Hadoop Project POM ......................... SUCCESS
Apache Hadoop Annotations ......................... SUCCESS
Apache Hadoop Assemblies .......................... SUCCESS
Apache Hadoop Project Dist POM .................... SUCCESS
Apache Hadoop Maven Plugins ....................... SUCCESS
Apache Hadoop MiniKDC ............................. SUCCESS
Apache Hadoop Auth ................................ SUCCESS
Apache Hadoop Auth Examples ....................... SUCCESS
Apache Hadoop Common .............................. SUCCESS
Apache Hadoop NFS ................................. SUCCESS
Apache Hadoop Common Project ...................... SUCCESS
Apache Hadoop HDFS ................................ SUCCESS
Apache Hadoop HttpFS .............................. SUCCESS
Apache Hadoop HDFS BookKeeper Journal ............. SUCCESS
Apache Hadoop HDFS-NFS ............................ SUCCESS
Apache Hadoop HDFS Project ........................ SUCCESS
hadoop-yarn ....................................... SUCCESS
hadoop-yarn-api ................................... SUCCESS
hadoop-yarn-common ................................ SUCCESS
hadoop-yarn-server ................................ SUCCESS
hadoop-yarn-server-common ......................... SUCCESS
hadoop-yarn-server-nodemanager .................... SUCCESS
hadoop-yarn-server-web-proxy ...................... SUCCESS
hadoop-yarn-server-applicationhistoryservice ...... SUCCESS
hadoop-yarn-server-resourcemanager ................ SUCCESS
hadoop-yarn-server-tests .......................... SUCCESS
hadoop-yarn-client ................................ SUCCESS
hadoop-yarn-applications .......................... SUCCESS
hadoop-yarn-applications-distributedshell ......... SUCCESS
hadoop-yarn-applications-unmanaged-am-launcher .... SUCCESS
hadoop-yarn-site .................................. SUCCESS
hadoop-yarn-project ............................... SUCCESS
hadoop-mapreduce-client ........................... SUCCESS
hadoop-mapreduce-client-core ...................... SUCCESS
hadoop-mapreduce-client-common .................... SUCCESS
hadoop-mapreduce-client-shuffle ................... SUCCESS
hadoop-mapreduce-client-app ....................... SUCCESS
hadoop-mapreduce-client-hs ........................ SUCCESS
hadoop-mapreduce-client-jobclient ................. SUCCESS
hadoop-mapreduce-client-hs-plugins ................ SUCCESS
Apache Hadoop MapReduce Examples .................. SUCCESS
hadoop-mapreduce .................................. SUCCESS
Apache Hadoop MapReduce Streaming ................. SUCCESS
Apache Hadoop Distributed Copy .................... SUCCESS
Apache Hadoop Archives ............................ SUCCESS
Apache Hadoop Rumen ............................... SUCCESS
Apache Hadoop Gridmix ............................. SUCCESS
Apache Hadoop Data Join ........................... SUCCESS
Apache Hadoop Extras .............................. SUCCESS
Apache Hadoop Pipes ............................... SUCCESS
Apache Hadoop OpenStack support ................... SUCCESS
Apache Hadoop Client .............................. SUCCESS
Apache Hadoop Mini-Cluster ........................ SUCCESS
Apache Hadoop Scheduler Load Simulator ............ SUCCESS
Apache Hadoop Tools Dist .......................... SUCCESS
Apache Hadoop Tools ............................... SUCCESS
Apache Hadoop Distribution ........................ SUCCESS
------------------------------------------------------------------------
BUILD SUCCESS
------------------------------------------------------------------------
Total time: 26:07.575s
Finished at: Fri Dec 12 16:12:56 CST 2014
Final Memory: 123M/940M
------------------------------------------------------------------------
3.problems
a.Exception in thread "main" java.lang.AssertionError: Missing tools.jar at: /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/Classes/classes.jar. Expression: file.exists()
this is as the compiler needs a file named tools.jar but localted at classes.jar under dir Classes but found nothing.so u can do workaround:
cd `java_home`/
mkdir Classes
cd Classes
ln -s ../lib/tools.jar classes.jar
but u should know,in macbook,the command 'ln' is ONLY effect when u locate in the target dir,otherwise u will see:
cd `java_home`
mkdir Classes
ln -s lib/tools.jar Classes/classes.jar
# or:ln -s lib/tools.jar /absolute/path/to/Classes/classes.jar
cp Classes/classes.jar /tmp
Error:no such file or dir found
2.failed to compile target:hadoop mavn doc
this is not important for this theme,so u can add a param to ignore it:
-Dmaven.javadoc.skip=true
Ref:
从零教你在Linux环境下（ubuntu 12.04）如何编译hadoop2.4
页:
[1]