未能启动 apache.spark.master

Posted 2023-04-15

【中文标题】未能启动 apache.spark.master

【英文标题】：failed to launch apache.spark.master

【发布时间】：2017-10-05 11:27:50

【问题描述】：

每当我在本地机器上运行 start-master.sh 命令时，我都会收到以下错误，请有人帮我解决这个问题

终端错误

我在终端中遇到的错误

starting org.apache.spark.deploy.master.Master, logging to /usr/local/spark-2.0.1-bin-hadoop2.6/logs/spark-andani-org.apache.spark.deploy.master.Master-1-andani.sakha.com.out
failed to launch org.apache.spark.deploy.master.Master:
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
at java.lang.Thread.run(Thread.java:748)

日志错误

如果我检查以下火花日志文件是错误

Exception in thread "main" java.net.BindException: Cannot assign requested address: Service 'sparkMaster' failed after 16 retries! Consider explicitly setting the appropriate port for the service 'sparkMaster' (for example spark.ui.port for SparkUI) to an available port or increasing spark.port.maxRetries.
   at sun.nio.ch.Net.bind0(Native Method)
   at sun.nio.ch.Net.bind(Net.java:433)
   at sun.nio.ch.Net.bind(Net.java:425)
   at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
   at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
   at io.netty.channel.socket.nio.NioserverSocketChannel.doBind(NioServerSocketChannel.java:125)
   at io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:485)
   at io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1089)
   at io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:430)
   at io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:415)
   at io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:903)
   at io.netty.channel.AbstractChannel.bind(AbstractChannel.java:198)
   at io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:348)
   at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:357)
   at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:357)
   at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
   at java.lang.Thread.run(Thread.java:748)

【参考方案1】：

错误是由于您的 sparkMaster 无法联系到您的内部 IP

检查您的 /etc/hosts 文件后，发现它指向正确的主机名，或者您之前的 IP 地址可能已更改。 重新配置并再次运行命令。