spark on yarn提交任務時報ClosedChannelException解決方案

来源:http://www.cnblogs.com/pojishou/archive/2017/01/31/6358588.html
-Advertisement-
Play Games

spark2.1出來了，想玩玩就搭了個原生的apache集群，但在standalone模式下沒有任何問題，基於apache hadoop 2.7.3使用spark on yarn一直報這個錯。 報錯日誌如下： 解決方案： 修改yarn-site.xml，添加下列property 分析： 按照上述配置 ...


spark2.1出來了，想玩玩就搭了個原生的apache集群，但在standalone模式下沒有任何問題，基於apache hadoop 2.7.3使用spark on yarn一直報這個錯。(Java 8)

報錯日誌如下：

Warning: Master yarn-client is deprecated since 2.0. Please use master "yarn" with specified deploy mode instead.
17/01/31 00:12:11 INFO spark.SparkContext: Running Spark version 2.1.0
17/01/31 00:12:11 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
17/01/31 00:12:11 INFO spark.SecurityManager: Changing view acls to: root
17/01/31 00:12:11 INFO spark.SecurityManager: Changing modify acls to: root
17/01/31 00:12:11 INFO spark.SecurityManager: Changing view acls groups to: 
17/01/31 00:12:11 INFO spark.SecurityManager: Changing modify acls groups to: 
17/01/31 00:12:11 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(root); groups with view permissions: Set(); users  with modify permissions: Set(root); groups with modify permissions: Set()
17/01/31 00:12:11 INFO util.Utils: Successfully started service 'sparkDriver' on port 59775.
17/01/31 00:12:11 INFO spark.SparkEnv: Registering MapOutputTracker
17/01/31 00:12:11 INFO spark.SparkEnv: Registering BlockManagerMaster
17/01/31 00:12:11 INFO storage.BlockManagerMasterEndpoint: Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information
17/01/31 00:12:11 INFO storage.BlockManagerMasterEndpoint: BlockManagerMasterEndpoint up
17/01/31 00:12:11 INFO storage.DiskBlockManager: Created local directory at /opt/program/spark-2.1.0-bin-hadoop2.7/blockmgr-b04fc6c2-501f-4df4-ae13-f6fb0aaa6470
17/01/31 00:12:11 INFO memory.MemoryStore: MemoryStore started with capacity 366.3 MB
17/01/31 00:12:11 INFO spark.SparkEnv: Registering OutputCommitCoordinator
17/01/31 00:12:12 INFO util.log: Logging initialized @2406ms
17/01/31 00:12:12 INFO server.Server: jetty-9.2.z-SNAPSHOT
17/01/31 00:12:12 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@64712be{/jobs,null,AVAILABLE}
17/01/31 00:12:12 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@53499d85{/jobs/json,null,AVAILABLE}
17/01/31 00:12:12 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@30ed9c6c{/jobs/job,null,AVAILABLE}
17/01/31 00:12:12 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@782a4fff{/jobs/job/json,null,AVAILABLE}
17/01/31 00:12:12 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@46c670a6{/stages,null,AVAILABLE}
17/01/31 00:12:12 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@59fc684e{/stages/json,null,AVAILABLE}
17/01/31 00:12:12 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5ae81e1{/stages/stage,null,AVAILABLE}
17/01/31 00:12:12 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@2fd1731c{/stages/stage/json,null,AVAILABLE}
17/01/31 00:12:12 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5ae76500{/stages/pool,null,AVAILABLE}
17/01/31 00:12:12 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@6063d80a{/stages/pool/json,null,AVAILABLE}
17/01/31 00:12:12 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1133ec6e{/storage,null,AVAILABLE}
17/01/31 00:12:12 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@355e34c7{/storage/json,null,AVAILABLE}
17/01/31 00:12:12 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@54709809{/storage/rdd,null,AVAILABLE}
17/01/31 00:12:12 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@2a2da905{/storage/rdd/json,null,AVAILABLE}
17/01/31 00:12:12 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@24f360b2{/environment,null,AVAILABLE}
17/01/31 00:12:12 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@60cf80e7{/environment/json,null,AVAILABLE}
17/01/31 00:12:12 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@302fec27{/executors,null,AVAILABLE}
17/01/31 00:12:12 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@770d0ea6{/executors/json,null,AVAILABLE}
17/01/31 00:12:12 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@48c40605{/executors/threadDump,null,AVAILABLE}
17/01/31 00:12:12 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@54107f42{/executors/threadDump/json,null,AVAILABLE}
17/01/31 00:12:12 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1b11ef33{/static,null,AVAILABLE}
17/01/31 00:12:12 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@476aac9{/,null,AVAILABLE}
17/01/31 00:12:12 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@6cea706c{/api,null,AVAILABLE}
17/01/31 00:12:12 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3bd7f8dc{/jobs/job/kill,null,AVAILABLE}
17/01/31 00:12:12 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@2f2bf0e2{/stages/stage/kill,null,AVAILABLE}
17/01/31 00:12:12 INFO server.ServerConnector: Started ServerConnector@780ec4a5{HTTP/1.1}{0.0.0.0:4040}
17/01/31 00:12:12 INFO server.Server: Started @2508ms
17/01/31 00:12:12 INFO util.Utils: Successfully started service 'SparkUI' on port 4040.
17/01/31 00:12:12 INFO ui.SparkUI: Bound SparkUI to 0.0.0.0, and started at http://192.168.56.101:4040
17/01/31 00:12:12 INFO spark.SparkContext: Added JAR file:/opt/spark/examples/jars/spark-examples_2.11-2.1.0.jar at spark://192.168.56.101:59775/jars/spark-examples_2.11-2.1.0.jar with timestamp 1485792732176
17/01/31 00:12:12 INFO client.RMProxy: Connecting to ResourceManager at node01/192.168.56.101:8032
17/01/31 00:12:13 INFO yarn.Client: Requesting a new application from cluster with 3 NodeManagers
17/01/31 00:12:13 INFO yarn.Client: Verifying our application has not requested more than the maximum memory capability of the cluster (8192 MB per container)
17/01/31 00:12:13 INFO yarn.Client: Will allocate AM container, with 896 MB memory including 384 MB overhead
17/01/31 00:12:13 INFO yarn.Client: Setting up container launch context for our AM
17/01/31 00:12:13 INFO yarn.Client: Setting up the launch environment for our AM container
17/01/31 00:12:13 INFO yarn.Client: Preparing resources for our AM container
17/01/31 00:12:13 WARN yarn.Client: Neither spark.yarn.jars nor spark.yarn.archive is set, falling back to uploading libraries under SPARK_HOME.
17/01/31 00:12:15 INFO yarn.Client: Uploading resource file:/opt/program/spark-2.1.0-bin-hadoop2.7/spark-6cafe3cb-9ed2-4f7f-b44f-a2bb447eaa30/__spark_libs__493282781411356296.zip -> hdfs://node01:9000/user/root/.sparkStaging/application_1485792095366_0003/__spark_libs__493282781411356296.zip
17/01/31 00:12:17 INFO yarn.Client: Uploading resource file:/opt/program/spark-2.1.0-bin-hadoop2.7/spark-6cafe3cb-9ed2-4f7f-b44f-a2bb447eaa30/__spark_conf__2188039824841197723.zip -> hdfs://node01:9000/user/root/.sparkStaging/application_1485792095366_0003/__spark_conf__.zip
17/01/31 00:12:17 INFO spark.SecurityManager: Changing view acls to: root
17/01/31 00:12:17 INFO spark.SecurityManager: Changing modify acls to: root
17/01/31 00:12:17 INFO spark.SecurityManager: Changing view acls groups to: 
17/01/31 00:12:17 INFO spark.SecurityManager: Changing modify acls groups to: 
17/01/31 00:12:17 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(root); groups with view permissions: Set(); users  with modify permissions: Set(root); groups with modify permissions: Set()
17/01/31 00:12:17 INFO yarn.Client: Submitting application application_1485792095366_0003 to ResourceManager
17/01/31 00:12:18 INFO impl.YarnClientImpl: Submitted application application_1485792095366_0003
17/01/31 00:12:18 INFO cluster.SchedulerExtensionServices: Starting Yarn extension services with app application_1485792095366_0003 and attemptId None
17/01/31 00:12:19 INFO yarn.Client: Application report for application_1485792095366_0003 (state: ACCEPTED)
17/01/31 00:12:19 INFO yarn.Client: 
         client token: N/A
         diagnostics: N/A
         ApplicationMaster host: N/A
         ApplicationMaster RPC port: -1
         queue: default
         start time: 1485792737991
         final status: UNDEFINED
         tracking URL: http://node01:8088/proxy/application_1485792095366_0003/
         user: root
17/01/31 00:12:20 INFO yarn.Client: Application report for application_1485792095366_0003 (state: ACCEPTED)
17/01/31 00:12:21 INFO yarn.Client: Application report for application_1485792095366_0003 (state: ACCEPTED)
17/01/31 00:12:21 INFO cluster.YarnSchedulerBackend$YarnSchedulerEndpoint: ApplicationMaster registered as NettyRpcEndpointRef(null)
17/01/31 00:12:21 INFO cluster.YarnClientSchedulerBackend: Add WebUI Filter. org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter, Map(PROXY_HOSTS -> node01, PROXY_URI_BASES -> http://node01:8088/proxy/application_1485792095366_0003), /proxy/application_1485792095366_0003
17/01/31 00:12:21 INFO ui.JettyUtils: Adding filter: org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter
17/01/31 00:12:22 INFO yarn.Client: Application report for application_1485792095366_0003 (state: ACCEPTED)
17/01/31 00:12:23 INFO yarn.Client: Application report for application_1485792095366_0003 (state: RUNNING)
17/01/31 00:12:23 INFO yarn.Client: 
         client token: N/A
         diagnostics: N/A
         ApplicationMaster host: 192.168.56.101
         ApplicationMaster RPC port: 0
         queue: default
         start time: 1485792737991
         final status: UNDEFINED
         tracking URL: http://node01:8088/proxy/application_1485792095366_0003/
         user: root
17/01/31 00:12:23 INFO cluster.YarnClientSchedulerBackend: Application application_1485792095366_0003 has started running.
17/01/31 00:12:23 INFO util.Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 38405.
17/01/31 00:12:23 INFO netty.NettyBlockTransferService: Server created on 192.168.56.101:38405
17/01/31 00:12:23 INFO storage.BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy
17/01/31 00:12:23 INFO storage.BlockManagerMaster: Registering BlockManager BlockManagerId(driver, 192.168.56.101, 38405, None)
17/01/31 00:12:23 INFO storage.BlockManagerMasterEndpoint: Registering block manager 192.168.56.101:38405 with 366.3 MB RAM, BlockManagerId(driver, 192.168.56.101, 38405, None)
17/01/31 00:12:23 INFO storage.BlockManagerMaster: Registered BlockManager BlockManagerId(driver, 192.168.56.101, 38405, None)
17/01/31 00:12:23 INFO storage.BlockManager: Initialized BlockManager: BlockManagerId(driver, 192.168.56.101, 38405, None)
17/01/31 00:12:23 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@643ba1ed{/metrics/json,null,AVAILABLE}
17/01/31 00:12:24 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(null) (192.168.56.101:43252) with ID 1
17/01/31 00:12:24 INFO storage.BlockManagerMasterEndpoint: Registering block manager node01:53420 with 366.3 MB RAM, BlockManagerId(1, node01, 53420, None)
17/01/31 00:12:25 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Disabling executor 1.
17/01/31 00:12:25 INFO scheduler.DAGScheduler: Executor lost: 1 (epoch 0)
17/01/31 00:12:25 ERROR client.TransportClient: Failed to send RPC 8305478367380188725 to /192.168.56.101:43246: java.nio.channels.ClosedChannelException
java.nio.channels.ClosedChannelException
        at io.netty.channel.AbstractChannel$AbstractUnsafe.write(...)(Unknown Source)
17/01/31 00:12:25 INFO storage.BlockManagerMasterEndpoint: Trying to remove executor 1 from BlockManagerMaster.
17/01/31 00:12:25 INFO storage.BlockManagerMasterEndpoint: Removing block manager BlockManagerId(1, node01, 53420, None)
17/01/31 00:12:25 INFO storage.BlockManagerMaster: Removed 1 successfully in removeExecutor
17/01/31 00:12:25 INFO scheduler.DAGScheduler: Shuffle files lost for executor: 1 (epoch 0)
17/01/31 00:12:25 WARN cluster.YarnSchedulerBackend$YarnSchedulerEndpoint: Attempted to get executor loss reason for executor id 1 at RPC address 192.168.56.101:43252, but got no response. Marking as slave lost.
java.io.IOException: Failed to send RPC 8305478367380188725 to /192.168.56.101:43246: java.nio.channels.ClosedChannelException
        at org.apache.spark.network.client.TransportClient$3.operationComplete(TransportClient.java:249)
        at org.apache.spark.network.client.TransportClient$3.operationComplete(TransportClient.java:233)
        at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:514)
        at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:488)
        at io.netty.util.concurrent.DefaultPromise.access$000(DefaultPromise.java:34)
        at io.netty.util.concurrent.DefaultPromise$1.run(DefaultPromise.java:438)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:408)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:455)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:140)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.nio.channels.ClosedChannelException
        at io.netty.channel.AbstractChannel$AbstractUnsafe.write(...)(Unknown Source)
17/01/31 00:12:25 ERROR cluster.YarnScheduler: Lost executor 1 on node01: Slave lost
17/01/31 00:12:26 INFO cluster.YarnSchedulerBackend$YarnSchedulerEndpoint: ApplicationMaster registered as NettyRpcEndpointRef(null)
17/01/31 00:12:26 INFO cluster.YarnClientSchedulerBackend: Add WebUI Filter. org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter, Map(PROXY_HOSTS -> node01, PROXY_URI_BASES -> http://node01:8088/proxy/application_1485792095366_0003), /proxy/application_1485792095366_0003
17/01/31 00:12:26 INFO ui.JettyUtils: Adding filter: org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter
17/01/31 00:12:28 ERROR cluster.YarnClientSchedulerBackend: Yarn application has already exited with state FINISHED!
17/01/31 00:12:28 INFO server.ServerConnector: Stopped ServerConnector@780ec4a5{HTTP/1.1}{0.0.0.0:4040}
17/01/31 00:12:28 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@2f2bf0e2{/stages/stage/kill,null,UNAVAILABLE}
17/01/31 00:12:28 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@3bd7f8dc{/jobs/job/kill,null,UNAVAILABLE}
17/01/31 00:12:28 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@6cea706c{/api,null,UNAVAILABLE}
17/01/31 00:12:28 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@476aac9{/,null,UNAVAILABLE}
17/01/31 00:12:28 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@1b11ef33{/static,null,UNAVAILABLE}
17/01/31 00:12:28 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@54107f42{/executors/threadDump/json,null,UNAVAILABLE}
17/01/31 00:12:28 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@48c40605{/executors/threadDump,null,UNAVAILABLE}
17/01/31 00:12:28 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@770d0ea6{/executors/json,null,UNAVAILABLE}
17/01/31 00:12:28 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@302fec27{/executors,null,UNAVAILABLE}
17/01/31 00:12:28 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@60cf80e7{/environment/json,null,UNAVAILABLE}
17/01/31 00:12:28 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@24f360b2{/environment,null,UNAVAILABLE}
17/01/31 00:12:28 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@2a2da905{/storage/rdd/json,null,UNAVAILABLE}
17/01/31 00:12:28 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@54709809{/storage/rdd,null,UNAVAILABLE}
17/01/31 00:12:28 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@355e34c7{/storage/json,null,UNAVAILABLE}
17/01/31 00:12:28 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@1133ec6e{/storage,null,UNAVAILABLE}
17/01/31 00:12:28 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@6063d80a{/stages/pool/json,null,UNAVAILABLE}
17/01/31 00:12:28 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@5ae76500{/stages/pool,null,UNAVAILABLE}
17/01/31 00:12:28 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@2fd1731c{/stages/stage/json,null,UNAVAILABLE}
17/01/31 00:12:28 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@5ae81e1{/stages/stage,null,UNAVAILABLE}
17/01/31 00:12:28 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@59fc684e{/stages/json,null,UNAVAILABLE}
17/01/31 00:12:28 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@46c670a6{/stages,null,UNAVAILABLE}
17/01/31 00:12:28 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@782a4fff{/jobs/job/json,null,UNAVAILABLE}
17/01/31 00:12:28 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@30ed9c6c{/jobs/job,null,UNAVAILABLE}
17/01/31 00:12:28 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@53499d85{/jobs/json,null,UNAVAILABLE}
17/01/31 00:12:28 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@64712be{/jobs,null,UNAVAILABLE}
17/01/31 00:12:28 INFO ui.SparkUI: Stopped Spark web UI at http://192.168.56.101:4040
17/01/31 00:12:28 ERROR client.TransportClient: Failed to send RPC 6600979308376699964 to /192.168.56.103:56283: java.nio.channels.ClosedChannelException
java.nio.channels.ClosedChannelException
        at io.netty.channel.AbstractChannel$AbstractUnsafe.write(...)(Unknown Source)
17/01/31 00:12:28 ERROR spark.SparkContext: Error initializing SparkContext.
java.lang.IllegalStateException: Spark context stopped while waiting for backend
        at org.apache.spark.scheduler.TaskSchedulerImpl.waitBackendReady(TaskSchedulerImpl.scala:614)
        at org.apache.spark.scheduler.TaskSchedulerImpl.postStartHook(TaskSchedulerImpl.scala:169)
        at org.apache.spark.SparkContext.<init>(SparkContext.scala:567)
        at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2313)
        at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:868)
        at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:860)
        at scala.Option.getOrElse(Option.scala:121)
        at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:860)
        at org.apache.spark.examples.SparkPi$.main(SparkPi.scala:31)
        at org.apache.spark.examples.SparkPi.main(SparkPi.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
        at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
        at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
17/01/31 00:12:28 ERROR cluster.YarnSchedulerBackend$YarnSchedulerEndpoint: Sending RequestExecutors(0,0,Map()) to AM was unsuccessful
java.io.IOException: Failed to send RPC 6600979308376699964 to /192.168.56.103:56283: java.nio.channels.ClosedChannelException
        at org.apache.spark.network.client.TransportClient$3.operationComplete(TransportClient.java:249)
        at org.apache.spark.network.client.TransportClient$3.operationComplete(TransportClient.java:233)
        at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:514)
        at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:488)
        at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:427)
        at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:129)
        at io.netty.channel.AbstractChannel$AbstractUnsafe.safeSetFailure(AbstractChannel.java:852)
        at io.netty.channel.AbstractChannel$AbstractUnsafe.write(AbstractChannel.java:738)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.write(DefaultChannelPipeline.java:1251)
        at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:743)
        at io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:735)
        at io.netty.channel.AbstractChannelHandlerContext.access$1900(AbstractChannelHandlerContext.java:36)
        at io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.
您的分享是我們最大的動力!

-Advertisement-
Play Games
更多相關文章
  • iOS開發入門 ☞ C語言系列二
    常量 C語言有豐富的數據類型，在開發中，一般用常量或者變數來表示這些數據 "量"表示數據。常量，則表示一些固定的數據，也就是不能改變的數據。 10.1;// 雙精度double 5.2f; //單精度float 'a'; // 字元型常量 'ab'; // 錯誤 '李'; // 錯誤寫法, 因為一個 ...
  • 關於NSLog列印調試
    有關NSLog調試問題 ...
  • iOS開發入門 ☞ C語言系列一
    前言 什麼是操作系統：操作系統是一種系統軟體，用來管理電腦的軟硬體資源。 主流操作系統： 移動攜帶型設備：Android、iOS、Windows Phone、Symbian（塞班）、Black Berry（黑莓） OS PC個人電腦：Windows、Mac OS、Unix、Linux 什麼是iOS ...
  • 製作一個功能豐富的Android天氣App
    簡易天氣是一個基於和風天氣數據採用MD設計的Android天氣App。目前的版本採用傳統的MVC模式構建。通過豐富多彩的頁面為用戶提供日常所需的天氣資訊。 項目說明 項目放在github上面 地址是： "easyweather開源地址" 項目利用的和風天氣 舊版本的API。 定位功能使用的是百度地圖 ...
  • Android仿QQ界面
    最近這幾天，一直跟著朋友們聚會什麼的，沒怎麼做項目，今天總算是有時間開電腦繼續做我的項目了。下麵我就把我做的效果展示一下。 這是模仿了qq的界面效果。因為代碼比較長就不粘貼代碼了。需要的小伙伴可以跟我私聊。 ...
  • iOS 圖片壓縮方法
    iOS 圖片壓縮方法 兩種圖片壓縮方法 兩種壓縮圖片的方法：壓縮圖片質量(Quality)，壓縮圖片尺寸(Size)。 壓縮圖片質量 通過 UIImage 和 NSData 的相互轉化，減小 JPEG 圖片的質量來壓縮圖片。UIImageJPEGRepresentation:: 第二個參數 comp ...
  • Oracle錯誤——ORA12514：監聽程式當前無法識別連接描述符中請求的服務
    在連接資料庫的時候，有時會遇到一個“ORA12514：監聽程式當前無法識別連接描述符中請求的服務”的錯誤，這個錯誤其實就是資料庫動態註冊（關於動態註冊會在稍後講解）不生效，導致監聽器無法識別客戶端連接符中提供的服務名，從而拒絕建立資料庫連接時報的錯誤信息，所以就需要對監聽器配置做修改。 在這裡，還需 ...
  • twemproxyRedis協議解析探索——剖析twemproxy代碼正編
    這篇文章會對twemproxyRedis協議解析代碼部分進行一番簡單的分析，同時給出twemproxy目前支持的所有Redis命令。在這篇文章開始前，我想大家去簡單地理解一下有限狀態機，當然不理解也是沒有問題的，有限狀態機僅僅能幫助我們更好地理解twemproxyRedis協議解析代碼部分。 red ...
一周排行
    -Advertisement-
    Play Games
  • 移動開發（一）：使用.NET MAUI開發第一個安卓APP
    移動開發（一）：使用.NET MAUI開發第一個安卓APP 對於工作多年的C#程式員來說，近來想嘗試開發一款安卓APP，考慮了很久最終選擇使用.NET MAUI這個微軟官方的框架來嘗試體驗開發安卓APP，畢竟是使用Visual Studio開發工具，使用起來也比較的順手，結合微軟官方的教程進行了安卓 ...
  • wpf ToggleButton選中效果和一個登錄界面
    前言 QuestPDF 是一個開源 .NET 庫，用於生成 PDF 文檔。使用了C# Fluent API方式可簡化開發、減少錯誤並提高工作效率。利用它可以輕鬆生成 PDF 報告、發票、導出文件等。 項目介紹 QuestPDF 是一個革命性的開源 .NET 庫，它徹底改變了我們生成 PDF 文檔的方 ...
  • 彈幕樹洞項目功能新增篇
    項目地址 項目後端地址： https://github.com/ZyPLJ/ZYTteeHole 項目前端頁面地址： ZyPLJ/TreeHoleVue (github.com) https://github.com/ZyPLJ/TreeHoleVue 目前項目測試訪問地址： http://tree ...
  • 第27篇 sqlserver2022詳細安裝步驟
    話不多說，直接開乾 一.下載 1.官方鏈接下載： https://www.microsoft.com/zh-cn/sql-server/sql-server-downloads 2.在下載目錄中找到下麵這個小的安裝包 SQL2022-SSEI-Dev.exe，運行開始下載SQL server； 二. ...
  • .NET 開源高性能 MQTT 類庫
    前言 隨著物聯網（IoT）技術的迅猛發展，MQTT（消息隊列遙測傳輸）協議憑藉其輕量級和高效性，已成為眾多物聯網應用的首選通信標準。 MQTTnet 作為一個高性能的 .NET 開源庫，為 .NET 平臺上的 MQTT 客戶端與伺服器開發提供了強大的支持。 本文將全面介紹 MQTTnet 的核心功能 ...
  • Serilog文檔翻譯系列（六） - 可用的接收器、增強器、格式化輸出
    Serilog支持多種接收器用於日誌存儲，增強器用於添加屬性，LogContext管理動態屬性，支持多種輸出格式包括純文本、JSON及ExpressionTemplate。還提供了自定義格式化選項，適用於不同需求。 ...
  • 警惕 Visual Studio 屬性求值副作用導致邏輯不符合預期
    目錄簡介獲取 HTML 文檔解析 HTML 文檔測試參考文章 簡介 動態內容網站使用 JavaScript 腳本動態檢索和渲染數據，爬取信息時需要模擬瀏覽器行為，否則獲取到的源碼基本是空的。 本文使用的爬取步驟如下： 使用 Selenium 獲取渲染後的 HTML 文檔 使用 HtmlAgility ...
  • [使用目前最新版]HybridCLR6.9.0+YooAsset2.2.4實現純C# Unity熱更新方案 (一)
    1.前言 什麼是熱更新 游戲或者軟體更新時，無需重新下載客戶端進行安裝，而是在應用程式啟動的情況下，在內部進行資源或者代碼更新 Unity目前常用熱更新解決方案 HybridCLR，Xlua，ILRuntime等 Unity目前常用資源管理解決方案 AssetBundles，Addressable， ...
  • 在 ASP.NET Core Web API 中使用操作篩選器統一處理通用操作
    本文章主要是在C# ASP.NET Core Web API框架實現向手機發送驗證碼簡訊功能。這裡我選擇是一個互億無線簡訊驗證碼平臺，其實像阿裡雲，騰訊雲上面也可以。 首先我們先去 互億無線 https://www.ihuyi.com/api/sms.html 去註冊一個賬號 註冊完成賬號後，它會送 ...
  • 第28篇 如何.net中實現高效可靠數據同步api
    通過以下方式可以高效，並保證數據同步的可靠性 1.API設計 使用RESTful設計，確保API端點明確，並使用適當的HTTP方法（如POST用於創建，PUT用於更新）。 設計清晰的請求和響應模型，以確保客戶端能夠理解預期格式。 2.數據驗證 在伺服器端進行嚴格的數據驗證，確保接收到的數據符合預期格 ...
所有分類