使用 Ant 构建 Pig 的难度

Posted 2023-04-18

【中文标题】使用 Ant 构建 Pig 的难度

【英文标题】：Difficulty building Pig using Ant

【发布时间】：2014-06-17 13:42:22

【问题描述】：

我是一个猪新手，一直试图让它在我的 Windows 7 机器上运行。我的 hadoop (2.4.0) 在我的本地机器上运行良好，但是每当我尝试运行一个简单的 Pig 脚本时，每当我尝试转储文件的内容时它都会引发错误。

找到了here 的类似问题的解决方案，我尝试使用 Ant 重建我的 Pig (0.12.1)。 每当我跑步时

ant clean jar-withouthadoop -Dhadoopversion=23

我收到以下错误：

Buildfile: C:\pig-0.12.1-src\build.xml
......
[org.apache.ftpserver#ftpserver-core;1.0.0] in [hadoop20]
[ivy:resolve]   org.apache.mina#mina-core;2.0.0-M2 by [org.apache.mina#mina-core;2.0.0-M5] in [hadoop20]
---------------------------------------------------------------------
|                  |            modules            ||   artifacts   |
|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
|      master      |   0   |   0   |   0   |   0   ||   0   |   0   |
|      default     |  101  |   32  |   0   |   3   ||  101  |   0   |
|      runtime     |  101  |   32  |   0   |   3   ||  101  |   0   |
|      compile     |   89  |   29  |   0   |   2   ||   90  |   0   |
|       test       |  101  |   32  |   0   |   3   ||  101  |   0   |
|      javadoc     |  101  |   32  |   0   |   3   ||  101  |   0   |
|   releaseaudit   |   3   |   2   |   0   |   0   ||   3   |   0   |
|       jdiff      |   3   |   0   |   0   |   0   ||   3   |   0   |
|    checkstyle    |   10  |   3   |   0   |   0   ||   10  |   0   |
|     buildJar     |  101  |   32  |   0   |   3   ||  101  |   0   |
|     hadoop20     |   48  |   30  |   0   |   9   ||   39  |   0   |
|     hadoop23     |   40  |   13  |   0   |   0   ||   42  |   0   |
|      hbase94     |   1   |   0   |   0   |   0   ||   2   |   0   |
|      hbase95     |   7   |   0   |   0   |   0   ||   13  |   0   |
---------------------------------------------------------------------

ivy-compile:
[ivy:retrieve] :: retrieving :: org.apache.pig#pig
[ivy:retrieve]  confs: [compile]
[ivy:retrieve]  90 artifacts copied, 0 already retrieved (80170kB/407ms)
[ivy:cachepath] DEPRECATED: 'ivy.conf.file' is deprecated, use 'ivy.settings.file' instead
[ivy:cachepath] :: loading settings :: file = C:\pig-0.12.1-src\ivy\ivysettings.xml

init:
    [mkdir] Created dir: C:\pig-0.12.1-src\src-gen\org\apache\pig\impl\logicalLayer\parser
    [mkdir] Created dir: C:\pig-0.12.1-src\src-gen\org\apache\pig\tools\pigscript\parser
    [mkdir] Created dir: C:\pig-0.12.1-src\src-gen\org\apache\pig\tools\parameters
    [mkdir] Created dir: C:\pig-0.12.1-src\build\classes
    [mkdir] Created dir: C:\pig-0.12.1-src\build\test\classes
    [mkdir] Created dir: C:\pig-0.12.1-src\test\org\apache\pig\test\utils\dotGraph\parser
    [mkdir] Created dir: C:\pig-0.12.1-src\src-gen\org\apache\pig\data\parser
    [move] Moving 1 file to C:\pig-0.12.1-src\build\ivy\lib\Pig

cc-compile:
   [javacc] Java Compiler Compiler Version 4.2 (Parser Generator)
   [javacc] (type "javacc" with no arguments for help)
   [javacc] Reading from file C:\pig-0.12.1-
    ....
   [javacc] Parser generated successfully.

prepare:
    [mkdir] Created dir: C:\pig-0.12.1-src\src-gen\org\apache\pig\parser

genLexer:
     [java] error(1):  cannot write file : java.io.FileNotFoundException: C:\pig-0.12.1-src\src-gen\org\apache\pig\parser\C:\QueryLexer.java (The filename, directory name, or volume label syntax is incorrect)
     [java] java.io.FileOutputStream.open(Native Method)
     [java] java.io.FileOutputStream.<init>(FileOutputStream.java:221)
     [java] java.io.FileOutputStream.<init>(FileOutputStream.java:171)
     [java] java.io.FileWriter.<init>(FileWriter.java:90)
     [java] org.antlr.Tool.getOutputFile(Tool.java:871)
     [java] org.antlr.codegen.CodeGenerator.write(CodeGenerator.java:1262)
     [java] org.antlr.codegen.Target.genRecognizerFile(Target.java:94)
     [java] org.antlr.codegen.CodeGenerator.genRecognizer(CodeGenerator.java:460)
     [java] org.antlr.Tool.generateRecognizer(Tool.java:655)
     [java] org.antlr.Tool.process(Tool.java:468)
     [java] org.antlr.Tool.main(Tool.java:93)
BUILD FAILED
C:\pig-0.12.1-src\build.xml:495 Java returned: 1

我在这里做错了什么？我是缺少文件，还是 build.xml 中的配置问题？

编辑：我已经从 cmd 和 cygwin 终端运行了这个命令。它们都输出相同的错误。

EDIT2：我正在使用 Ant (1.9.4) 构建

最终编辑：原来我需要将 pig src 存档（pig-0.12.1-src.tar.gz）移动到 cygwin 主目录，解压缩，cd 到解压缩的存档并运行命令。

【问题讨论】：

您是否尝试使用来自cygwin 的相同命令？

我做了，产生了同样的错误。

【参考方案1】：

我遇到了同样的问题。我通过在我的 cygwin 家中通过 cygwin 编译 pig 来解决它。

我的常春藤设置是这样的：

<ivysettings>

 <!--
   Licensed to the Apache Software Foundation (ASF) under one or more
   contributor license agreements.  See the NOTICE file distributed with
   this work for additional information regarding copyright ownership.
   The ASF licenses this file to You under the Apache License, Version 2.0
   (the "License"); you may not use this file except in compliance with
   the License.  You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.
-->

 <!--
  see http://www.jayasoft.org/ivy/doc/configuration
  -->
  <!-- you can override this property to use mirrors
          http://repo1.maven.org/maven2/
          http://mirrors.dotsrc.org/maven2
          http://ftp.ggi-project.org/pub/packages/maven2
          http://mirrors.sunsite.dk/maven2
          http://public.planetmirror.com/pub/maven2
          http://ibiblio.lsu.edu/main/pub/packages/maven2
          http://www.ibiblio.net/pub/packages/maven2
  -->
  <property name="repo.maven.org" value="$mvnrepo" override="true"/>
  <property name="repo.jboss.org" value="http://repository.jboss.com/nexus/content/groups/public/" override="false"/>
  <property name="repo.apache.snapshots" value="http://repository.apache.org/content/groups/snapshots-group/" override="false"/>
  <property name="repo.dir" value="$user.home/.m2/repository" override="false"/>
  <property name="maven2.pattern" value="[organisation]/[module]/[revision]/[module]-[revision](-[classifier])"/>
  <property name="maven2.pattern.ext" value="$maven2.pattern.[ext]"/>
  <property name="snapshot.pattern" value="[organisation]/[module]/[revision]/[artifact]-[revision](-[classifier]).[ext]"/>
  <property name="resolvers" value="default" override="false"/>
  <property name="force-resolve" value="false" override="false"/>
  <!-- pull in the local repository -->
  <include url="$ivy.default.conf.dir/ivyconf-local.xml"/>
  <settings defaultResolver="$resolvers"/>
  <resolvers>
    <ibiblio name="maven2" root="$repo.maven.org" pattern="$maven2.pattern.ext" m2compatible="true"/>
    <ibiblio name="jboss-maven2" root="$repo.jboss.org" pattern="$maven2.pattern.ext" m2compatible="true"/>
    <ibiblio name="apache-snapshots" root="$repo.apache.snapshots" pattern="$snapshot.pattern"
         checkmodified="true" changingPattern=".*SNAPSHOT" m2compatible="true"/>
    <filesystem name="fs" m2compatible="true" checkconsistency="false" force="$force-resolve"
         checkmodified="true" changingPattern=".*SNAPSHOT">
      <artifact pattern="$repo.dir/$maven2.pattern.ext"/>
      <ivy pattern="$repo.dir/[organisation]/[module]/[revision]/[module]-[revision].pom"/>
    </filesystem>
    <chain name="internal" checkmodified="true">
      <resolver ref="fs"/>
    </chain>
    <chain name="external" dual="true">
      <resolver ref="maven2"/>
      <resolver ref="jboss-maven2"/>
      <resolver ref="apache-snapshots"/>
    </chain>
    <chain name="default" dual="true" checkmodified="true">
      <resolver ref="internal"/>
      <resolver ref="external"/>
    </chain>
  </resolvers>
  <modules>
    <module organisation="org.apache.pig" name=".*" resolver="internal"/>
  </modules>
</ivysettings>

【讨论】：

你能分享你的 ivy.settings 吗？