设为首页 收藏本站
查看: 673|回复: 0

[经验分享] com.hadoop.gplcompression#hadoop-lzo;0.4.19: not found

[复制链接]

尚未签到

发表于 2016-12-6 06:57:17 | 显示全部楼层 |阅读模式
$ git clone git://github.com/twitter/scalding.git
  Scalding set up:

$ cd scalding
$ export SCALDING_HOME=`pwd`
$ sbt update
$ sbt test
$ sbt assembly
  com.hadoop.gplcompression#hadoop-lzo;0.4.19: not found
  [warn]     ::::::::::::::::::::::::::::::::::::::::::::::
[warn]     ::          UNRESOLVED DEPENDENCIES         ::
[warn]     ::::::::::::::::::::::::::::::::::::::::::::::
[warn]     :: com.hadoop.gplcompression#hadoop-lzo;0.4.19: not found
[warn]     ::::::::::::::::::::::::::::::::::::::::::::::
[info] Updating {file:/home/abc/scala/scalding/}scalding-macros...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
sbt.ResolveException: unresolved dependency: com.hadoop.gplcompression#hadoop-lzo;0.4.19: not found
    at sbt.IvyActions$.sbt$IvyActions$$resolve(IvyActions.scala:217)
    at sbt.IvyActions$$anonfun$update$1.apply(IvyActions.scala:126)
    at sbt.IvyActions$$anonfun$update$1.apply(IvyActions.scala:125)
    at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:115)
    at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:115)
    at sbt.IvySbt$$anonfun$withIvy$1.apply(Ivy.scala:103)
    at sbt.IvySbt.sbt$IvySbt$$action$1(Ivy.scala:48)
    at sbt.IvySbt$$anon$3.call(Ivy.scala:57)
    at xsbt.boot.Locks$GlobalLock.withChannel$1(Locks.scala:98)
    at xsbt.boot.Locks$GlobalLock.xsbt$boot$Locks$GlobalLock$$withChannelRetries$1(Locks.scala:81)
    at xsbt.boot.Locks$GlobalLock$$anonfun$withFileLock$1.apply(Locks.scala:102)
    at xsbt.boot.Using$.withResource(Using.scala:11)
    at xsbt.boot.Using$.apply(Using.scala:10)
    at xsbt.boot.Locks$GlobalLock.ignoringDeadlockAvoided(Locks.scala:62)
    at xsbt.boot.Locks$GlobalLock.withLock(Locks.scala:52)
    at xsbt.boot.Locks$.apply0(Locks.scala:31)
    at xsbt.boot.Locks$.apply(Locks.scala:28)
    at sbt.IvySbt.withDefaultLogger(Ivy.scala:57)
    at sbt.IvySbt.withIvy(Ivy.scala:98)
    at sbt.IvySbt.withIvy(Ivy.scala:94)
    at sbt.IvySbt$Module.withModule(Ivy.scala:115)
    at sbt.IvyActions$.update(IvyActions.scala:125)
    at sbt.Classpaths$$anonfun$sbt$Classpaths$$work$1$1.apply(Defaults.scala:1223)
    at sbt.Classpaths$$anonfun$sbt$Classpaths$$work$1$1.apply(Defaults.scala:1221)
    at sbt.Classpaths$$anonfun$doWork$1$1$$anonfun$74.apply(Defaults.scala:1244)
    at sbt.Classpaths$$anonfun$doWork$1$1$$anonfun$74.apply(Defaults.scala:1242)
    at sbt.Tracked$$anonfun$lastOutput$1.apply(Tracked.scala:35)
    at sbt.Classpaths$$anonfun$doWork$1$1.apply(Defaults.scala:1246)
    at sbt.Classpaths$$anonfun$doWork$1$1.apply(Defaults.scala:1241)
    at sbt.Tracked$$anonfun$inputChanged$1.apply(Tracked.scala:45)
    at sbt.Classpaths$.cachedUpdate(Defaults.scala:1249)
    at sbt.Classpaths$$anonfun$updateTask$1.apply(Defaults.scala:1214)
    at sbt.Classpaths$$anonfun$updateTask$1.apply(Defaults.scala:1192)
    at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
    at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:42)
    at sbt.std.Transform$$anon$4.work(System.scala:64)
    at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:237)
    at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:237)
    at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:18)
    at sbt.Execute.work(Execute.scala:244)
    at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:237)
    at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:237)
    at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:160)
    at sbt.CompletionService$$anon$2.call(CompletionService.scala:30)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
[error] (scalding-commons/*:update) sbt.ResolveException: unresolved dependency: com.hadoop.gplcompression#hadoop-lzo;0.4.19: not found
  cd /scalding/project
  add this to the file
  "test" at "http://repo.typesafe.com/typesafe/releases",
  resolvers ++= Seq(
      "Local Maven Repository" at "file://" + Path.userHome.absolutePath + "/.m2/repository",
      "snapshots" at "https://oss.sonatype.org/content/repositories/snapshots",
      "releases" at "https://oss.sonatype.org/content/repositories/releases",
      "Concurrent Maven Repo" at "http://conjars.org/repo",
      "Clojars Repository" at "http://clojars.org/repo",
      "Twitter Maven" at "http://maven.twttr.com",
      "test" at "http://repo.typesafe.com/typesafe/releases",
      "Cloudera" at "https://repository.cloudera.com/artifactory/cloudera-repos/"
    ),
  [info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] downloading http://repo.typesafe.com/typesafe/releases/com/hadoop/gplcompression/hadoop-lzo/0.4.19/hadoop-lzo-0.4.19.jar ...
[info]     [SUCCESSFUL ] com.hadoop.gplcompression#hadoop-lzo;0.4.19!hadoop-lzo.jar (2311ms)
[info] Done updating.
[info] Updating {file:/home/abc/scala/scalding/}scalding-macros...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
[success] Total time: 160 s, completed Apr 21, 2015 7:09:36 PM
  问题记录:
  sbt test通不过,scalding-hadoop-test:test unsuccess
  注释scalding-hadoop-test文件夹下面所有extends WordSpec的语句,重新执行sbt test
  例子:
  // class WordCountTest extends WordSpec with Matchers {
//   "A WordCount job" should {
//     JobTest(new com.twitter.scalding.examples.WordCountJob(_))
//       .arg("input", "inputFile")
//       .arg("output", "outputFile")
//       .source(TextLine("inputFile"), List((0, "hack hack hack and hack")))
//       .sink[(String, Int)](TypedTsv[(String, Long)]("outputFile")){ outputBuffer =>
//         val outMap = outputBuffer.toMap
//         "count words correctly" in {
//           outMap("hack") shouldBe 4
//           outMap("and") shouldBe 1
//         }
//       }
//       .run
//       .finish
//   }
// }

运维网声明 1、欢迎大家加入本站运维交流群:群②:261659950 群⑤:202807635 群⑦870801961 群⑧679858003
2、本站所有主题由该帖子作者发表,该帖子作者与运维网享有帖子相关版权
3、所有作品的著作权均归原作者享有,请您和我们一样尊重他人的著作权等合法权益。如果您对作品感到满意,请购买正版
4、禁止制作、复制、发布和传播具有反动、淫秽、色情、暴力、凶杀等内容的信息,一经发现立即删除。若您因此触犯法律,一切后果自负,我们对此不承担任何责任
5、所有资源均系网友上传或者通过网络收集,我们仅提供一个展示、介绍、观摩学习的平台,我们不对其内容的准确性、可靠性、正当性、安全性、合法性等负责,亦不承担任何法律责任
6、所有作品仅供您个人学习、研究或欣赏,不得用于商业或者其他用途,否则,一切后果均由您自己承担,我们对此不承担任何法律责任
7、如涉及侵犯版权等问题,请您及时通知我们,我们将立即采取措施予以解决
8、联系人Email:admin@iyunv.com 网址:www.yunweiku.com

所有资源均系网友上传或者通过网络收集,我们仅提供一个展示、介绍、观摩学习的平台,我们不对其承担任何法律责任,如涉及侵犯版权等问题,请您及时通知我们,我们将立即处理,联系人Email:kefu@iyunv.com,QQ:1061981298 本贴地址:https://www.iyunv.com/thread-310115-1-1.html 上篇帖子: Hadoop实战-中高级部分 之 Hadoop 管理 下篇帖子: (zz)Debugging Hadoop applications using your Eclipse
您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

扫码加入运维网微信交流群X

扫码加入运维网微信交流群

扫描二维码加入运维网微信交流群,最新一手资源尽在官方微信交流群!快快加入我们吧...

扫描微信二维码查看详情

客服E-mail:kefu@iyunv.com 客服QQ:1061981298


QQ群⑦:运维网交流群⑦ QQ群⑧:运维网交流群⑧ k8s群:运维网kubernetes交流群


提醒:禁止发布任何违反国家法律、法规的言论与图片等内容;本站内容均来自个人观点与网络等信息,非本站认同之观点.


本站大部分资源是网友从网上搜集分享而来,其版权均归原作者及其网站所有,我们尊重他人的合法权益,如有内容侵犯您的合法权益,请及时与我们联系进行核实删除!



合作伙伴: 青云cloud

快速回复 返回顶部 返回列表