to do with the word frequency inverted index documents when submitting jobs when it did not reflect on the end. Shown
I can confirm that : 1 Hadoop has started, there is no problem ( because I tried previously run a successful program , you can run )
2: If my program in question , it should be thrown ah , do not throw ?
Here, I deliberately set to the output directory , output directory already exists , is no reflection.
Here is my setup code :
Configuration conf = new Configuration();
Job job = new Job(conf,"InvertedWork");
job.setJarByClass(InvertedDocument.class);
job.setMapperClass(InvertedMapper.class);
job.setReducerClass(InvertedReducer.class);
job.setCombinerClass(InvertedCombiner.class);
job.setPartitionerClass(InvertedPartioner.class);
job.setMapOutputKeyClass(Text.class);
job.setMapOutputValueClass(Text.class);
//这个不是默认的输入格式么,还要设?应该设置的是输入的key把
//job.setInputFormatClass(org.apache.hadoop.mapreduce.lib.input.TextInputFormat.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(Text.class);
FileInputFormat.addInputPath(job, new Path(args[0]));
FileOutputFormat.setOutputPath(job, new Path(args[1]));
System.exit(job.waitForCompletion(true)?0:1);
request advice twelve , in the end what is the reason ?
------ Solution ---------------------------------------- ----
1, with ECLIPSE debugging code in System.exit (job.waitForCompletion (true)? 0:1); break point before , can come here to see the program
2, to 50,030 page views InvertedWork JOB is pulled up
3, emptied JAVA CLASS files compiled source code , recompile
4, check the CLASSPATH HADOOP if there is redundant JAR package
5, view JDK_HOME / jdk / jre / lib / ext / if there is garbage JAR
If not, the test code and the source data to other environments COPY retested .
or to impress source and source data , so users can help you test in the end is the code problem or environmental issues.
------ Solution ---------------------------------------- ----
this really hard to say , you need to look at your original error source
------ For reference only ------------- --------------------------
alas, cheap hand , mistakenly deleted the source code , had their own rewrite it again, and then I discovery can run, but still thank you upstairs .
but still leaves the question : what causes this phenomenon, rewrite it , I did not write combiner
没有评论:
发表评论