代码之家 › 专栏 › 技术社区 › Sarvagya Dubey

尝试执行Pig拉丁脚本时出现异常

apache-pig mapreduce hadoop

0

Sarvagya Dubey · 技术社区 · 7 年前

我自己在学习pig,在尝试探索一个数据集时遇到了一个异常。剧本有什么问题,为什么:

movies_data = LOAD '/movies_data' using PigStorage(',') as (id:chararray,title:chararray,year:int,rating:double,duration:double);
high   = FILTER movies_data by rating > 4.0;
high_rated = FOREACH high GENERATE movies_data.title,movies_data.year,movies_data.rating,movies_data.duration;
DUMP high_rated;

在map reduce执行的末尾,我得到了下面的错误。

2018-07-22 20:11:07,213 [main] ERROR org.apache.pig.tools.grunt.Grunt

ERROR 1066: Unable to open iterator for alias high_rated. 
Backend error : org.apache.pig.backend.executionengine.ExecException: 
ERROR 0: Scalar has more than one row in the output. 
1st : (1,The Nightmare Before Christmas,1993,3.9,4568.0), 
2nd :(2,The Mummy,1932,3.5,4388.0) 
(common cause: "JOIN" then "FOREACH ... GENERATE foo.bar" should be "foo::bar" )

2 回复 | 直到 7 年前

1

Rajeev Atmakuri 7 年前

首先,让我们看看如何解决你的问题。不需要使用别名访问字段。你的第三行可以是:

high_rated = FOREACH high GENERATE title, year, rating, duration;

如果出于某种原因要使用别名,则应使用引用运算符(::),如错误建议中所示。那么你的台词应该是:

high_rated = FOREACH high GENERATE movies_data::title, movies_data::year, movies_data::rating, movies_data::duration;

接下来,让我们试着理解错误消息背后的确切原因。尝试使用点运算符(.)访问字段时,pig将假定别名是标量(别名只有一行)因为你的化名有不止一行,它抱怨道你可以在这里阅读更多关于猪的鳞片: https://issues.apache.org/jira/browse/PIG-1434

在JIRA的release notes部分,您将在最后注意到,预期的错误消息与您得到的错误匹配:

If a relation contains more than single tuple, a runtime error is generated: 
"Scalar has more than one row in the output"

2

0

Jagadeeswara Reddy 7 年前

这对你很管用。

movies_data = LOAD '/movies_data' using PigStorage(',') as (id:chararray,title:chararray,year:int,rating:double,duration:double);
high   = FILTER movies_data by rating > 4.0;
 high_rated = FOREACH high GENERATE title,year,rating,duration;
DUMP high_rated;

允许满足筛选条件的所有列记录的筛选命令。