hadoop2.6.5+sqoop1.4.6环境部署与测试（二）
#在hdfs上建立/test_wordcount目录$ hdfs dfs -mkdir /test_wordcount
#将/etc/profile上传到/test_wordcount目录下
$ hdfs dfs -put /etc/profile /test_wordcount
$ hdfs dfs -ls /test_wordcount
Found 1 items
-rw-r--r-- 2 hadoop supergroup 2064 2017-08-06 21:28 /test_wordcount/profile
#使用wordcount程序进行测试
$ hadoop jar hadoop-mapreduce-examples-2.6.5.jar wordcount /test_wordcount/profile /test_wordcount_out
17/08/06 21:30:11 INFO client.RMProxy: Connecting to ResourceManager at hadoop-test-nn/10.0.1.100:8040
17/08/06 21:30:13 INFO input.FileInputFormat: Total input paths to process : 1
17/08/06 21:30:13 INFO mapreduce.JobSubmitter: number of splits:1
17/08/06 21:30:13 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1501950606475_0001
17/08/06 21:30:14 INFO impl.YarnClientImpl: Submitted application application_1501950606475_0001
17/08/06 21:30:14 INFO mapreduce.Job: The url to track the job: http://hadoop-test-nn:8088/proxy/application_1501950606475_0001/
17/08/06 21:30:14 INFO mapreduce.Job: Running job: job_1501950606475_0001
17/08/06 21:30:29 INFO mapreduce.Job: Job job_1501950606475_0001 running in uber mode : false
17/08/06 21:30:29 INFO mapreduce.Job:map 0% reduce 0%
17/08/06 21:30:39 INFO mapreduce.Job:map 100% reduce 0%
17/08/06 21:30:49 INFO mapreduce.Job:map 100% reduce 100%
17/08/06 21:30:50 INFO mapreduce.Job: Job job_1501950606475_0001 completed successfully
17/08/06 21:30:51 INFO mapreduce.Job: Counters: 49
File System Counters
FILE: Number of bytes read=2320
FILE: Number of bytes written=219547
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=2178
HDFS: Number of bytes written=1671
HDFS: Number of read operations=6
HDFS: Number of large read operations=0
HDFS: Number of write operations=2
Job Counters
Launched map tasks=1
Launched reduce tasks=1
Data-local map tasks=1
Total time spent by all maps in occupied slots (ms)=7536
Total time spent by all reduces in occupied slots (ms)=8136
Total time spent by all map tasks (ms)=7536
Total time spent by all reduce tasks (ms)=8136
Total vcore-milliseconds taken by all map tasks=7536
Total vcore-milliseconds taken by all reduce tasks=8136
Total megabyte-milliseconds taken by all map tasks=7716864
Total megabyte-milliseconds taken by all reduce tasks=8331264
Map-Reduce Framework
Map input records=84
Map output records=268
Map output bytes=2880
Map output materialized bytes=2320
Input split bytes=114
Combine input records=268
Combine output records=161
Reduce input groups=161
Reduce shuffle bytes=2320
Reduce input records=161
Reduce output records=161
Spilled Records=322
Shuffled Maps =1
Failed Shuffles=0
Merged Map outputs=1
GC time elapsed (ms)=186
CPU time spent (ms)=1850
Physical memory (bytes) snapshot=310579200
Virtual memory (bytes) snapshot=1682685952
Total committed heap usage (bytes)=164630528
Shuffle Errors
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
File Input Format Counters
Bytes Read=2064
File Output Format Counters
Bytes Written=1671
页:
[1]