AWS弹性马preduce似乎并没有被正确地转换分流到坛子里并没有、正确地、流到、弹性

由网友(相顾无言)分享简介:我有一个映射器和减速机的正常工作,当我在管道版本运行这些:I have a mapper and reducer that work fine when I run them in the piped version:cat data.csv | ./mapper.py | sort -k1,1 | ./reduc...

我有一个映射器和减速机的正常工作,当我在管道版本运行这些:

I have a mapper and reducer that work fine when I run them in the piped version:

cat data.csv | ./mapper.py | sort -k1,1 | ./reducer.py

我用弹性均线preducer向导,加载输入,输出,引导等的引导是成功的,但我仍然得到一个错误的执行。

I used the elastic mapreducer wizard, loaded inputs, outputs, bootstrap, etc. The bootstrap is successful, but I am still getting an error in execution.

这是我收到我的标准错误的步骤1中的错误...

This is the error I'm getting in my stderr for step 1...

+ /etc/init.d/hadoop-state-pusher-control stop
+ PID_FILE=/mnt/var/run/hadoop-state-pusher/hadoop-state-pusher.pid
+ LOG_FILE=/mnt/var/log/hadoop-state-pusher/hadoop-state-pusher.out
+ SVC_FILE=/mnt/var/lib/hadoop-state-pusher/run-hadoop-state-pusher
+ case $1 in
+ stop
+ echo 0
/etc/init.d/hadoop-state-pusher-control: line 35: /mnt/var/lib/hadoop-state-pusher/run-hadoop-state-pusher: No such file or directory
+ /etc/init.d/hadoop-state-pusher-control start
+ PID_FILE=/mnt/var/run/hadoop-state-pusher/hadoop-state-pusher.pid
+ LOG_FILE=/mnt/var/log/hadoop-state-pusher/hadoop-state-pusher.out
+ SVC_FILE=/mnt/var/lib/hadoop-state-pusher/run-hadoop-state-pusher
+ case $1 in
+ start
++ dirname /mnt/var/lib/hadoop-state-pusher/run-hadoop-state-pusher
+ sudo -u hadoop mkdir -p /mnt/var/lib/hadoop-state-pusher
+ echo 1
++ dirname /mnt/var/run/hadoop-state-pusher/hadoop-state-pusher.pid
+ sudo -u hadoop mkdir -p /mnt/var/run/hadoop-state-pusher
++ dirname /mnt/var/log/hadoop-state-pusher/hadoop-state-pusher.out
+ sudo -u hadoop mkdir -p /mnt/var/log/hadoop-state-pusher
+ disown %1
+ sleep 5
+ sudo -u hadoop /usr/bin/hadoop-state-pusher -server --pidfile /mnt/var/run/hadoop-state-pusher/hadoop-state-pusher.pid
+ exit 0
Command exiting with ret '0'

这是神秘的。这究竟是什么意思?

This is cryptic. What on earth does this mean?

这似乎也与安装的东西出了问题?其中其他日志文件可能会说一些内容丰富,我应该在哪里寻找?

It seems to have a problem with mounting something? Which of the other log files might say something informative, where I should be looking?

我想一个解决方案,我发现此处,在刚刚制作实例较大,但这并没有工作,同样的错误消息。

I tried a solution I found here, in just making the instance bigger, but this did not work, same error message.

推荐答案

我一直在寻找在错误日志文件中。有一个不同的(有像6?),其实给了我一些有用的蟒蛇调试信息。原来,我用了一个字符串interpolation.format(这种{}的,不是这种以数字{1}。瓦尔(A,B))这是不支持在Python< 2.7,这是什么默认安装在弹性马preduce使用的EC2映像上。

I was looking in the wrong log file. There is a different (there were like 6?) that actually gave me some useful python debugging information. It turned out I had used a string interpolation.format("of this kind {}, not this kind with a digit {1}".vars(a,b)) that was unsupported in python < 2.7, which was what was installed by default on the EC2 image used in elastic mapreduce.

阅读全文

相关推荐

最新文章