hadoop standalone install

225025

　　只需要在hadoop-env.sh中修改java home，
不需要format,不需要copyFromLocal to hdfs
　　注意使用的是standalone状态下的hadoop
use 5s

hadoop@leibnitz-laptop:/cc/hadoop/standalone/hadoop-0.20.2$ ./bin/hadoop jar hadoop-0.20.2-examples.jar wordcount input/cluster output/wordcount
11/02/26 03:19:40 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
11/02/26 03:19:40 INFO input.FileInputFormat: Total input paths to process : 3
11/02/26 03:19:40 INFO mapred.JobClient: Running job: job_local_0001
11/02/26 03:19:40 INFO input.FileInputFormat: Total input paths to process : 3
11/02/26 03:19:40 INFO mapred.MapTask: io.sort.mb = 100
11/02/26 03:19:41 INFO mapred.MapTask: data buffer = 79691776/99614720
11/02/26 03:19:41 INFO mapred.MapTask: record buffer = 262144/327680
11/02/26 03:19:41 INFO mapred.JobClient:  map 0% reduce 0%
11/02/26 03:19:41 INFO mapred.MapTask: Starting flush of map output
11/02/26 03:19:42 INFO mapred.MapTask: Finished spill 0
11/02/26 03:19:42 INFO mapred.TaskRunner: Task:attempt_local_0001_m_000000_0 is done. And is in the process of commiting
11/02/26 03:19:42 INFO mapred.LocalJobRunner:
11/02/26 03:19:42 INFO mapred.TaskRunner: Task 'attempt_local_0001_m_000000_0' done.
11/02/26 03:19:42 INFO mapred.MapTask: io.sort.mb = 100
11/02/26 03:19:42 INFO mapred.MapTask: data buffer = 79691776/99614720
11/02/26 03:19:42 INFO mapred.MapTask: record buffer = 262144/327680
11/02/26 03:19:43 INFO mapred.MapTask: Spilling map output: record full = true
11/02/26 03:19:43 INFO mapred.MapTask: bufstart = 0; bufend = 2546041; bufvoid = 99614720
11/02/26 03:19:43 INFO mapred.MapTask: kvstart = 0; kvend = 262144; length = 327680
11/02/26 03:19:43 INFO mapred.MapTask: Starting flush of map output
11/02/26 03:19:43 INFO mapred.JobClient:  map 100% reduce 0%
11/02/26 03:19:44 INFO mapred.MapTask: Finished spill 0
11/02/26 03:19:44 INFO mapred.MapTask: Finished spill 1
11/02/26 03:19:44 INFO mapred.Merger: Merging 2 sorted segments
11/02/26 03:19:44 INFO mapred.Merger: Down to the last merge-pass, with 2 segments left of total size: 740450 bytes
11/02/26 03:19:44 INFO mapred.TaskRunner: Task:attempt_local_0001_m_000001_0 is done. And is in the process of commiting
11/02/26 03:19:44 INFO mapred.LocalJobRunner:
11/02/26 03:19:44 INFO mapred.TaskRunner: Task 'attempt_local_0001_m_000001_0' done.
11/02/26 03:19:44 INFO mapred.MapTask: io.sort.mb = 100
11/02/26 03:19:44 INFO mapred.MapTask: data buffer = 79691776/99614720
11/02/26 03:19:44 INFO mapred.MapTask: record buffer = 262144/327680
11/02/26 03:19:44 INFO mapred.MapTask: Starting flush of map output
11/02/26 03:19:45 INFO mapred.MapTask: Finished spill 0
11/02/26 03:19:45 INFO mapred.TaskRunner: Task:attempt_local_0001_m_000002_0 is done. And is in the process of commiting
11/02/26 03:19:45 INFO mapred.LocalJobRunner:
11/02/26 03:19:45 INFO mapred.TaskRunner: Task 'attempt_local_0001_m_000002_0' done.
11/02/26 03:19:45 INFO mapred.LocalJobRunner:
11/02/26 03:19:45 INFO mapred.Merger: Merging 3 sorted segments
11/02/26 03:19:45 INFO mapred.Merger: Down to the last merge-pass, with 3 segments left of total size: 1474267 bytes
11/02/26 03:19:45 INFO mapred.LocalJobRunner:
11/02/26 03:19:45 INFO mapred.TaskRunner: Task:attempt_local_0001_r_000000_0 is done. And is in the process of commiting
11/02/26 03:19:45 INFO mapred.LocalJobRunner:
11/02/26 03:19:45 INFO mapred.TaskRunner: Task attempt_local_0001_r_000000_0 is allowed to commit now
11/02/26 03:19:45 INFO output.FileOutputCommitter: Saved output of task 'attempt_local_0001_r_000000_0' to output/wordcount
11/02/26 03:19:45 INFO mapred.LocalJobRunner: reduce > reduce
11/02/26 03:19:45 INFO mapred.TaskRunner: Task 'attempt_local_0001_r_000000_0' done.
11/02/26 03:19:45 INFO mapred.JobClient:  map 100% reduce 100%
11/02/26 03:19:45 INFO mapred.JobClient: Job complete: job_local_0001
11/02/26 03:19:45 INFO mapred.JobClient: Counters: 12
11/02/26 03:19:45 INFO mapred.JobClient:   FileSystemCounters
11/02/26 03:19:45 INFO mapred.JobClient:     FILE_BYTES_READ=16082737
11/02/26 03:19:45 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=8416674
11/02/26 03:19:45 INFO mapred.JobClient:   Map-Reduce Framework
11/02/26 03:19:45 INFO mapred.JobClient:     Reduce input groups=82331
11/02/26 03:19:45 INFO mapred.JobClient:     Combine output records=102317
11/02/26 03:19:45 INFO mapred.JobClient:     Map input records=77931
11/02/26 03:19:45 INFO mapred.JobClient:     Reduce shuffle bytes=0
11/02/26 03:19:45 INFO mapred.JobClient:     Reduce output records=82331
11/02/26 03:19:45 INFO mapred.JobClient:     Spilled Records=255947
11/02/26 03:19:45 INFO mapred.JobClient:     Map output bytes=6076039
11/02/26 03:19:45 INFO mapred.JobClient:     Combine input records=629167
11/02/26 03:19:45 INFO mapred.JobClient:     Map output records=629167
11/02/26 03:19:45 INFO mapred.JobClient:     Reduce input records=102317