Replies: 7
Hi There,
I have a pig script which includes an “sh” command which executes sqoop. The pig script works fine when executed using “pig -f script.pig” but
fails when executed as part of an Oozie workflow.
Versions of each component:
Hadoop 1.2.0.1.3.0.0-107 (as part of HDP 1.2)
Apache Pig version 0.11.1.1.3.0.0-107 (rexported)
Sqoop 1.4.3.1.3.0.0-107
Oozie client build version: 3.3.2.1.3.0.0-107
We’ve been struglling with this issue and tried a few things (mostly with regards to classpaths of sqoop/pig/oozie) but none of them really helped.
We also tried to play with log4j properties with no luck.
The following exceptions are being logged to Oozie pig launcher map reduce job:
The following exception is thrown by sqoop (sensitive data is obfuscated of course):
Exception in thread “main” java.lang.NoClassDefFoundError: Could not initialize class org.apache.log4j.LogManager
at org.apache.log4j.Logger.getLogger(Logger.java:105)
at org.apache.sqoop.util.LoggingUtils.setDebugLevel(LoggingUtils.java:50)
at org.apache.sqoop.tool.BaseSqoopTool.applyCommonOptions(BaseSqoopTool.java:739)
at org.apache.sqoop.tool.ImportTool.applyOptions(ImportTool.java:722)
at org.apache.sqoop.tool.SqoopTool.parseArguments(SqoopTool.java:433)
at org.apache.sqoop.Sqoop.run(Sqoop.java:129)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
Thanks in advance.