单机测试NN时遇到的问题
在Hadoop的测试框架上写了一个小测试，用于比对NN使用不同策略分配数据块位置的速度：Configuration conf = new Configuration();
FileSystem.setDefaultUri(conf, "hdfs://localhost:" + 0);
conf.set("dfs.http.address", "0.0.0.0:0");
然后用该conf去NameNode.createNameNode()
->new Namenode()
->new FSNamesystem()
->this.replicator = BlockPlacementPolicy.getInstance(conf, this, clusterMap);//这里我用了Raid的Policy
->BlockPlacementPolicyRaid.initialize()
->为了得到this.xorPrefix调用RaidNode.xorDestinationPath(conf)
->FileSystem.get(p.toUri(), conf)
->CACHE.get(uri, conf)
->createFileSystem(uri, conf)
->FileSystem fs = (FileSystem)ReflectionUtils.newInstance(clazz, conf);得到DistributedFileSystem
->DistributedFileSystem.initialize()
->new DFSClient(namenode, conf, statistics)
->createRPCNamenode(nameNodeAddr, conf, ugi);
OK这时问题来了，在NN的建立过程中建立到NN的RPC，所以程序程序会停住不动。。。这时用jstack就可以查看得详细的调用过程
"main" prio=10 tid=0x0875ac00 nid=0xd70 in Object.wait()
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x506cdcc8> (a org.apache.hadoop.ipc.Client$Call)
at java.lang.Object.wait(Object.java:485)
at org.apache.hadoop.ipc.Client.call(Client.java:701)
- locked <0x506cdcc8> (a org.apache.hadoop.ipc.Client$Call)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:216)
at $Proxy4.getProtocolVersion(Unknown Source)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:348)
at org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:103)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:176)
at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:78)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1443)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:56)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1456)
- locked <0x50960460> (a org.apache.hadoop.fs.FileSystem$Cache)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:218)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:120)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:207)
at org.apache.hadoop.raid.RaidNode.xorDestinationPath(RaidNode.java:1072)
at org.apache.hadoop.hdfs.server.namenode.BlockPlacementPolicyRaid.initialize(BlockPlacementPolicyRaid.java:96)
at org.apache.hadoop.hdfs.server.namenode.BlockPlacementPolicy.getInstance(BlockPlacementPolicy.java:193)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.setConfigurationParameters(FSNamesystem.java:446)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:325)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:309)
at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:173)
at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:256)
at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:998)
at org.apache.hadoop.hdfs.NNThroughputBenchmark.<init>(NNThroughputBenchmark.java:112)
at org.apache.hadoop.hdfs.NNThroughputBenchmark.runBenchmark(NNThroughputBenchmark.java:1250)
at org.apache.hadoop.hdfs.NNThroughputBenchmark.main(NNThroughputBenchmark.java:1360)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:141)
at org.apache.hadoop.test.AllTestDriver.main(AllTestDriver.java:90)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:165)
at org.apache.hadoop.mapred.JobShell.run(JobShell.java:54)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.mapred.JobShell.main(JobShell.java:68)
解决：由于测试不需要用到xorPrefix,rsPrefix，所以直接把xorPrefix=/raid,rsPrefix=/raidrs。
页:
[1]