2013年8月14日星期三

WordCount Hadoop running on the issue! ! ! !

hadoop jar wordcount.jar org.myorg.WordCount. / wordcount / input /. / wordcount / ouput
run WordCount when the console does not have any information output, has been running.

I also closed hadoop safe mode, but after running or has been "stuck"

solved!
------ Solution ---------------------------------------- ----
configuration file error, leading to datanode process can not start
------ For reference only ---------------------- -----------------
to add a question: When I run hadoop fs-put input1.txt input: the following error

11/10/12 16:06:29 WARN hdfs.DFSClient: DataStreamer Exception: org.apache.hadoop.ipc.RemoteException: java.io.IOException: File / user/supertool/input/input1.txt could only be replicated to 0 nodes, instead of 1
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock (FSNamesystem.java: 1271)
at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock (NameNode.java: 422)
at sun.reflect.GeneratedMethodAccessor7.invoke (Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java: 25)
at java.lang.reflect.Method.invoke (Method.java: 597)
at org.apache.hadoop.ipc.RPC $ Server.call (RPC.java: 508)
at org.apache.hadoop.ipc.Server $ Handler $ 1.run (Server.java: 959)
at org.apache.hadoop.ipc.Server $ Handler $ 1.run (Server.java: 955)
at java.security.AccessController.doPrivileged (Native Method)
at javax.security.auth.Subject.doAs (Subject.java: 396)
at org.apache.hadoop.ipc.Server $ Handler.run (Server.java: 953)

at org.apache.hadoop.ipc.Client.call (Client.java: 740)
at org.apache.hadoop.ipc.RPC $ Invoker.invoke (RPC.java: 220)
at $ Proxy0.addBlock (Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java: 39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java: 25)
at java.lang.reflect.Method.invoke (Method.java: 597)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod (RetryInvocationHandler.java: 82)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke (RetryInvocationHandler.java: 59)
at $ Proxy0.addBlock (Unknown Source)
at org.apache.hadoop.hdfs.DFSClient $ DFSOutputStream.locateFollowingBlock (DFSClient.java: 2937)
at org.apache.hadoop.hdfs.DFSClient $ DFSOutputStream.nextBlockOutputStream (DFSClient.java: 2819)
at org.apache.hadoop.hdfs.DFSClient $ DFSOutputStream.access $ 2000 (DFSClient.java: 2102)
at org.apache.hadoop.hdfs.DFSClient $ DFSOutputStream $ DataStreamer.run (DFSClient.java: 2288)

11/10/12 16:06:29 WARN hdfs.DFSClient: Error Recovery for block null bad datanode [0] nodes == null
11/10/12 16:06:29 WARN hdfs.DFSClient: Could not get block locations. Source file "/ user/supertool/input/input1 . txt "- Aborting ...
put: java.io.IOException: File / user/supertool/input/input1.txt could only be replicated to 0 nodes, instead of 1
11/10/12 16:06:29 ERROR hdfs.DFSClient: Exception closing file / user/supertool/input/input1.txt: org.apache.hadoop. ipc.RemoteException: java.io.IOException: File / user/supertool/input/input1.txt could only be replicated to 0 nodes, instead of 1
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock (FSNamesystem.java: 1271)
at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock (NameNode.java: 422)
at sun.reflect.GeneratedMethodAccessor7.invoke (Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java: 25)
at java.lang.reflect.Method.invoke (Method.java: 597)
at org.apache.hadoop.ipc.RPC $ Server.call (RPC.java: 508)
at org.apache.hadoop.ipc.Server $ Handler $ 1.run (Server.java: 959)
at org.apache.hadoop.ipc.Server $ Handler $ 1.run (Server.java: 955)
at java.security.AccessController.doPrivileged (Native Method)
at javax.security.auth.Subject.doAs (Subject.java: 396)
at org.apache.hadoop.ipc.Server $ Handler.run (Server.java: 953)

org.apache.hadoop.ipc.RemoteException: java.io.IOException: File / user/supertool/input/input1.txt could only be replicated to 0 nodes , instead of 1
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock (FSNamesystem.java: 1271)
at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock (NameNode.java: 422)
at sun.reflect.GeneratedMethodAccessor7.invoke (Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java: 25)
at java.lang.reflect.Method.invoke (Method.java: 597)
at org.apache.hadoop.ipc.RPC $ Server.call (RPC.java: 508)
at org.apache.hadoop.ipc.Server $ Handler $ 1.run (Server.java: 959)
at org.apache.hadoop.ipc.Server $ Handler $ 1.run (Server.java: 955)
at java.security.AccessController.doPrivileged (Native Method)
at javax.security.auth.Subject.doAs (Subject.java: 396)
at org.apache.hadoop.ipc.Server $ Handler.run (Server.java: 953)

at org.apache.hadoop.ipc.Client.call (Client.java: 740)
at org.apache.hadoop.ipc.RPC $ Invoker.invoke (RPC.java: 220)
at $ Proxy0.addBlock (Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java: 39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java: 25)
at java.lang.reflect.Method.invoke (Method.java: 597)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod (RetryInvocationHandler.java: 82)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke (RetryInvocationHandler.java: 59)
at $ Proxy0.addBlock (Unknown Source)
at org.apache.hadoop.hdfs.DFSClient $ DFSOutputStream.locateFollowingBlock (DFSClient.java: 2937)
at org.apache.hadoop.hdfs.DFSClient $ DFSOutputStream.nextBlockOutputStream (DFSClient.java: 2819)
at org.apache.hadoop.hdfs.DFSClient $ DFSOutputStream.access $ 2000 (DFSClient.java: 2102)
at org.apache.hadoop.hdfs.DFSClient $ DFSOutputStream $ DataStreamer.run (DFSClient.java: 2288)

------ For reference only ---------------------------------- -----
can say how wrong configuration files under what I also met the same problem

没有评论:

发表评论