代码之家  ›  专栏  ›  技术社区  ›  Abhinav

运行NIFI需要什么Hadoop配置?

  •  1
  • Abhinav  · 技术社区  · 7 年前

    我们已经在现有的HDP上安装了NIFI(HDF服务)。安装成功,我可以运行NIFI而不出错。但我在推杆方面面临着一个问题

    2018-06-20 12:00:14246 warn[standardprocessscheduler thread-6]org.apache.hadoop.conf.configuration/tmp/core site.xml:an attempt to override final parameter:fs.defaultfs;ignoring.
    2018-06-20 12:00:14248错误[StandardProcessScheduler线程-6]o.apache.nifi.processors.hadoop.puthdfs puthdfs[id=11A40827-0164-1000-ffff-ffffffb07a04d9]hdfs配置错误-java.net.connectexception:连接被拒绝:
    java.net.connectException:连接被拒绝
    在sun.nio.ch.socketchannelimpl.checkconnect(本机方法)
    在Sun(717)
    在Orac.Apache .Hadoop.net .SoCKToWiTimeTimeOut.Connect(SokKyToItTimeOut.java:206)
    在Orac.Apache .Hadoop.net .NETULTL.Connect(NETULTL.java:531)
    在Orac.Apache .Hadoop.net .NETULTL.Connect(NETULTL.java:495)
    在Org.Apache .NIFI.处理器.Hadoop.ActudioAdOpPoad.CuffHDFurIurFortTimeUT(ActuthAdOpPortual.java:345)
    在Org.Apache .NIFI.处理器.Hadoop.ActuthAdOpPort. RESIDSFsRESULL(ActuthTooPoPortual.java:260)
    在Org.Apache .NIFI.处理器. Hadoop.ActudioToopPort.抽象调度(ActuthAdOpPortual.java:205)
    在sun.reflect.generatedMethodAccessor728.invoke处(未知源)
    在Sun中。
    在Java.Lang.Exrime.Frut.CuoKE(方法.java:498)
    在Org.Apache .NIFI .UTI.ErrorUntudio.NoWoKeMeDsOs注释(ErrorDuffiLS.java:137)
    在Org.Apache .NIFI .UTI.ErrorUntudio.NoWoKeMeDsOs注释(ErrorDuffiLS.java:125)
    在Org.Apache .NIFI .UTI.ErrorUntudio.NoWoKeMeDsOs注释(ErrorDuffiLS.java:70)
    在Org.Apache .NIFI.UTI.ErrorUntudio.NoCoMeMeDoSsOnNo注注(ErrorDuffiLS.java:47)
    在Org.Apache .NIFI.COntReal.StaldPracororNo.$ 1调用(StaldPracororNo.java:1334)
    在Org.Apache .NIFI.COntReal.StaldPracororNo.$ 1调用(StaldPracororNo.java:1330)
    在Java.UTI.Actudio.FuturTask.Run(FuturTestJava:Java:266)
    在180美元201。
    在Java.UTI.Orth.TealthTythPooExcel中,$StutoEdvutuestTask.Run(StuttTeCyoPoExcRetuoto.java:293)
    在Java.UTI.Orth.TycRePoLeExtUnor。RunWork(TythPoLeExcel)。Java:1142)
    在Java.UTI.Orth.TycRePoLeExtor中$WorkWork.Run(TythPoLeExcel).java:617)
    在Java.Lang.Trime.Run(线程.java:745)
    2018-06-20 12:00:14248错误[StandardProcessScheduler线程-6]org.apache.nifi.engine.flowEngine A流控制器任务执行异常停止
    java.util.concurrent.ExecutionException:java.lang.Reflect.InvocationTargetException
    在Java.UTI.Actudio.FuturTealWork.报表(FuturTestJava:Java 122)
    在Java.UTI.Actudio.FuturTrask.GET(FuturTestJava:Java 192)
    在Org.Apache NIFI.Engory.FuffyEng.EngEnter(流引擎.java:100)
    在Java.UTI.Orth.TycRePoLeExtUnor。RunWork(TythPoLeExcel)。Java:1150)
    在Java.UTI.Orth.TycRePoLeExtor中$WorkWork.Run(TythPoLeExcel).java:617)
    在Java.Lang.Trime.Run(线程.java:745)
    原因:java.lang.reflect.invocationTargetException:空
    

    我也附上了截图。

    .

    2018-06-20 12:00:14,246 WARN [StandardProcessScheduler Thread-6] org.apache.hadoop.conf.Configuration /tmp/core-site.xml:an attempt to override final parameter: fs.defaultFS;  Ignoring.
    2018-06-20 12:00:14,248 ERROR [StandardProcessScheduler Thread-6] o.apache.nifi.processors.hadoop.PutHDFS PutHDFS[id=11a40827-0164-1000-ffff-ffffb07a04d9] HDFS Configuration error - java.net.ConnectException: Connection refused: {}
    java.net.ConnectException: Connection refused
            at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
            at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
            at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
            at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531)
            at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:495)
            at org.apache.nifi.processors.hadoop.AbstractHadoopProcessor.checkHdfsUriForTimeout(AbstractHadoopProcessor.java:345)
            at org.apache.nifi.processors.hadoop.AbstractHadoopProcessor.resetHDFSResources(AbstractHadoopProcessor.java:260)
            at org.apache.nifi.processors.hadoop.AbstractHadoopProcessor.abstractOnScheduled(AbstractHadoopProcessor.java:205)
            at sun.reflect.GeneratedMethodAccessor728.invoke(Unknown Source)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            at java.lang.reflect.Method.invoke(Method.java:498)
            at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:137)
            at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:125)
            at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:70)
            at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:47)
            at org.apache.nifi.controller.StandardProcessorNode$1.call(StandardProcessorNode.java:1334)
            at org.apache.nifi.controller.StandardProcessorNode$1.call(StandardProcessorNode.java:1330)
            at java.util.concurrent.FutureTask.run(FutureTask.java:266)
            at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
            at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
            at java.lang.Thread.run(Thread.java:745)
    2018-06-20 12:00:14,248 ERROR [StandardProcessScheduler Thread-6] org.apache.nifi.engine.FlowEngine A flow controller task execution stopped abnormally
    java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException
            at java.util.concurrent.FutureTask.report(FutureTask.java:122)
            at java.util.concurrent.FutureTask.get(FutureTask.java:192)
            at org.apache.nifi.engine.FlowEngine.afterExecute(FlowEngine.java:100)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1150)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
            at java.lang.Thread.run(Thread.java:745)
    Caused by: java.lang.reflect.InvocationTargetException: null
    

    我也附上了截图。

    PutHDFS

    1 回复  |  直到 6 年前
        1
  •  1
  •   Óscar Andreu    6 年前

    正如@Bryanbede所说,您得到的是java.net.connectException:连接被拒绝,因此它似乎与Hadoop机器中的安全配置有关,而与NIFI本身无关,请尝试与Hadoop客户机连接。

    Hadoop cluster setup - java.net.ConnectException: Connection refused

    https://wiki.apache.org/hadoop/ConnectionRefused