text:
[JVM-packages] XGBoostModel training failed with Rabit returns with exit code 1

I encountered the following problems when using xgboost4j-spark 1.6.1 , spark 3.2.1 and scala 2.12，
Can someone help me out？
spark parameters:
--conf spark.sql.shuffle.partitions=200
--conf spark.executor.instances=8
--conf spark.driver.memory=4g \
xgboost parameters:
"num_workers"->8,
"nthread"->1
error:
2022-06-17 06:10:10 [task-result-getter-0]  INFO [YarnClusterScheduler:57]: Removed TaskSet 2.0, whose tasks have all completed, from pool
2022-06-17 06:10:10 [dag-scheduler-event-loop]  INFO [DAGScheduler:57]: ResultStage 2 (collect at XGBoost.scala:431) finished in 2871.246 s
2022-06-17 06:10:10 [dag-scheduler-event-loop]  INFO [DAGScheduler:57]: Job 1 is finished. Cancelling potential speculative or zombie tasks for this job
2022-06-17 06:10:10 [dag-scheduler-event-loop]  INFO [YarnClusterScheduler:57]: Killing all running tasks in stage 2: Stage finished
2022-06-17 06:10:10 [Driver]  INFO [DAGScheduler:57]: Job 1 finished: collect at XGBoost.scala:431, took 3022.842636 s
2022-06-17 06:10:10 [Driver]  INFO [RabitTracker:213]: Tracker Process ends with exit code 1
2022-06-17 06:10:10 [Driver]  INFO [XGBoostSpark:433]: Rabit returns with exit code 1
2022-06-17 06:10:11 [Driver] ERROR [XGBoostSpark:455]: the job was aborted due to
ml.dmlc.xgboost4j.java.XGBoostError: XGBoostModel training failed.
at ml.dmlc.xgboost4j.scala.spark.XGBoost$.trainDistributed(XGBoost.scala:435)
at ml.dmlc.xgboost4j.scala.spark.XGBoostRegressor.train(XGBoostRegressor.scala:190)
at ml.dmlc.xgboost4j.scala.spark.XGBoostRegressor.train(XGBoostRegressor.scala:37)
at org.apache.spark.ml.Predictor.fit(Predictor.scala:151)
at SparkPi$.main(SparkPi.scala:52)
at SparkPi.main(SparkPi.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:737)
2022-06-17 06:10:11 [Driver] ERROR [ApplicationMaster:94]: User class threw exception: ml.dmlc.xgboost4j.java.XGBoostError: XGBoostModel training failed.
ml.dmlc.xgboost4j.java.XGBoostError: XGBoostModel training failed.
at ml.dmlc.xgboost4j.scala.spark.XGBoost$.trainDistributed(XGBoost.scala:435)
at ml.dmlc.xgboost4j.scala.spark.XGBoostRegressor.train(XGBoostRegressor.scala:190)
at ml.dmlc.xgboost4j.scala.spark.XGBoostRegressor.train(XGBoostRegressor.scala:37)
at org.apache.spark.ml.Predictor.fit(Predictor.scala:151)
at SparkPi$.main(SparkPi.scala:52)
at SparkPi.main(SparkPi.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:737)
2022-06-17 06:10:11 [Driver]  INFO [ApplicationMaster:57]: Final app status: FAILED, exitCode: 15, (reason: User class threw exception: ml.dmlc.xgboost4j.java.XGBoostError: XGBoostModel training failed.
at ml.dmlc.xgboost4j.scala.spark.XGBoost$.trainDistributed(XGBoost.scala:435)
at ml.dmlc.xgboost4j.scala.spark.XGBoostRegressor.train(XGBoostRegressor.scala:190)
at ml.dmlc.xgboost4j.scala.spark.XGBoostRegressor.train(XGBoostRegressor.scala:37)
at org.apache.spark.ml.Predictor.fit(Predictor.scala:151)
at SparkPi$.main(SparkPi.scala:52)
at SparkPi.main(SparkPi.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:737)

Hi, could you please post the log from executors?

executors seems like running successfully。。

Executors seems like running successfully。
log：
2022-06-17 14:04:32 [dispatcher-Executor]  INFO [YarnCoarseGrainedExecutorBackend:57]: Got assigned task 92
2022-06-17 14:04:32 [Executor task launch worker for task 60.0 in stage 1.0 (TID 92)]  INFO [Executor:57]: Running task 60.0 in stage 1.0 (TID 92)
2022-06-17 14:04:33 [Executor task launch worker for task 60.0 in stage 1.0 (TID 92)]  INFO [FileScanRDD:57]: Reading File path: afs://xingtian.afs.baidu.com:9902/user/feed_plat/job_data/development/zhengjianjia/spark/spark_train_c, range: 3976977000-4043259950, partition values: [empty row]
2022-06-17 14:04:53 [Executor task launch worker for task 60.0 in stage 1.0 (TID 92)]  INFO [Executor:57]: Finished task 60.0 in stage 1.0 (TID 92). 1759 bytes result sent to driver
2022-06-17 14:04:54 [dispatcher-Executor]  INFO [YarnCoarseGrainedExecutorBackend:57]: Got assigned task 102
2022-06-17 14:04:54 [Executor task launch worker for task 6.0 in stage 2.0 (TID 102)]  INFO [Executor:57]: Running task 6.0 in stage 2.0 (TID 102)
2022-06-17 14:04:54 [Executor task launch worker for task 6.0 in stage 2.0 (TID 102)]  INFO [MapOutputTrackerWorker:57]: Updating epoch to 1 and clearing cache
2022-06-17 14:04:54 [Executor task launch worker for task 6.0 in stage 2.0 (TID 102)]  INFO [TorrentBroadcast:57]: Started reading broadcast variable 4 with 1 pieces (estimated total size 4.0 MiB)
2022-06-17 14:04:54 [Executor task launch worker for task 6.0 in stage 2.0 (TID 102)]  INFO [TransportClientFactory:310]: Successfully created connection to yq01-hpc-lvliang-i1316.yq01.baidu.com/10.108.181.24:49763 after 1 ms (0 ms spent in bootstraps)
2022-06-17 14:04:54 [Executor task launch worker for task 6.0 in stage 2.0 (TID 102)]  INFO [MemoryStore:57]: Block broadcast_4_piece0 stored as bytes in memory (estimated size 5.2 KiB, free 2.8 GiB)
2022-06-17 14:04:54 [Executor task launch worker for task 6.0 in stage 2.0 (TID 102)]  INFO [TorrentBroadcast:57]: Reading broadcast variable 4 took 21 ms
2022-06-17 14:04:54 [Executor task launch worker for task 6.0 in stage 2.0 (TID 102)]  INFO [MemoryStore:57]: Block broadcast_4 stored as values in memory (estimated size 9.4 KiB, free 2.8 GiB)
2022-06-17 14:04:55 [Executor task launch worker for task 6.0 in stage 2.0 (TID 102)]  INFO [MemoryStore:57]: Block rdd_20_6 stored as values in memory (estimated size 4.8 MiB, free 2.8 GiB)
2022-06-17 14:04:56 [Executor task launch worker for task 6.0 in stage 2.0 (TID 102)]  INFO [MapOutputTrackerWorker:57]: Don't have map outputs for shuffle 0, fetching them
2022-06-17 14:04:56 [Executor task launch worker for task 6.0 in stage 2.0 (TID 102)]  INFO [MapOutputTrackerWorker:57]: Doing the fetch; tracker endpoint = NettyRpcEndpointRef(spark://MapOutputTracker@yq01-hpc-lvliang-i1316.yq01.baidu.com:57813)
2022-06-17 14:04:56 [Executor task launch worker for task 6.0 in stage 2.0 (TID 102)]  INFO [MapOutputTrackerWorker:57]: Got the map output locations
2022-06-17 14:04:56 [Executor task launch worker for task 6.0 in stage 2.0 (TID 102)]  INFO [ShuffleBlockFetcherIterator:57]: Getting 64 (255.7 MiB) non-empty blocks including 8 (32.0 MiB) local and 0 (0.0 B) host-local and 0 (0.0 B) push-merged-local and 56 (223.7 MiB) remote blocks
2022-06-17 14:04:56 [Executor task launch worker for task 6.0 in stage 2.0 (TID 102)]  INFO [TransportClientFactory:310]: Successfully created connection to yq01-hpc-lvliang-i1108.yq01.baidu.com/10.108.175.12:56296 after 11 ms (0 ms spent in bootstraps)
2022-06-17 14:04:56 [Executor task launch worker for task 6.0 in stage 2.0 (TID 102)]  INFO [ShuffleBlockFetcherIterator:57]: Started 5 remote fetches in 117 ms
2022-06-17 14:05:14 [Executor task launch worker for task 6.0 in stage 2.0 (TID 102)]  INFO [TransportClientFactory:310]: Successfully created connection to yq01-hpc-lvliang-i1111.yq01.baidu.com/10.108.175.15:55054 after 3 ms (0 ms spent in bootstraps)
2022-06-17 14:05:26 [Executor task launch worker for task 6.0 in stage 2.0 (TID 102)]  INFO [TransportClientFactory:310]: Successfully created connection to yq01-hpc-lvliang-i1106.yq01.baidu.com/10.108.174.44:52970 after 2 ms (0 ms spent in bootstraps)
2022-06-17 14:56:12 [Executor task launch worker for task 6.0 in stage 2.0 (TID 102)]  INFO [Executor:57]: 1 block locks were not released by task 6.0 in stage 2.0 (TID 102)
[rdd_20_6]
2022-06-17 14:56:12 [Executor task launch worker for task 6.0 in stage 2.0 (TID 102)]  INFO [Executor:57]: Finished task 6.0 in stage 2.0 (TID 102). 1177 bytes result sent to driver
2022-06-17 15:08:23 [dispatcher-Executor]  INFO [YarnCoarseGrainedExecutorBackend:57]: Driver commanded a shutdown
2022-06-17 15:08:24 [dispatcher-Executor]  INFO [YarnCoarseGrainedExecutorBackend:57]: Driver from yq01-hpc-lvliang-i1316.yq01.baidu.com:57813 disconnected during shutdown
2022-06-17 15:08:24 [dispatcher-Executor]  INFO [YarnCoarseGrainedExecutorBackend:57]: Driver from yq01-hpc-lvliang-i1316.yq01.baidu.com:57813 disconnected during shutdown
2022-06-17 15:08:24 [CoarseGrainedExecutorBackend-stop-executor]  INFO [MemoryStore:57]: MemoryStore cleared
2022-06-17 15:08:24 [CoarseGrainedExecutorBackend-stop-executor]  INFO [BlockManager:57]: BlockManager stopped
2022-06-17 15:08:24 [CoarseGrainedExecutorBackend-stop-executor]  INFO [TSDBReporter:222]: stop sending
2022-06-17 15:08:24 [Thread-2]  INFO [ShutdownHookManager:57]: Shutdown hook called
output：
[06:05:51] [690]	train-rmse:34.52551950994114094
[06:05:55] [691]	train-rmse:34.52534229661804943
[06:06:00] [692]	train-rmse:34.52519560304814661
[06:06:03] [693]	train-rmse:34.52512034711867273
[06:06:06] [694]	train-rmse:34.52495916896965156
[06:06:09] [695]	train-rmse:34.52486004181641022
[06:06:13] [696]	train-rmse:34.52474283998827076
[06:06:17] [697]	train-rmse:34.52446090707221771
[06:06:20] [698]	train-rmse:34.52429376354228197
[06:06:23] [699]	train-rmse:34.52418313626171908

train failed with local mode as well.
output log:
[15:50:46] [690]	train-rmse:0.00520688644065732
[15:50:46] [691]	train-rmse:0.00520692697203341
[15:50:46] [692]	train-rmse:0.00520696681770753
[15:50:46] [693]	train-rmse:0.00520700622236835
[15:50:46] [694]	train-rmse:0.00520704498998811
[15:50:46] [695]	train-rmse:0.00520708323702370
[15:50:47] [696]	train-rmse:0.00520712108582036
[15:50:47] [697]	train-rmse:0.00520715829172192
[15:50:47] [698]	train-rmse:0.00520719506255524
[15:50:47] [699]	train-rmse:0.00520723134969795
22/06/17 15:50:47 ERROR XGBoostSpark: the job was aborted due to
ml.dmlc.xgboost4j.java.XGBoostError: XGBoostModel training failed.
at ml.dmlc.xgboost4j.scala.spark.XGBoost$.trainDistributed(XGBoost.scala:435)
at ml.dmlc.xgboost4j.scala.spark.XGBoostRegressor.train(XGBoostRegressor.scala:190)
at ml.dmlc.xgboost4j.scala.spark.XGBoostRegressor.train(XGBoostRegressor.scala:37)
at org.apache.spark.ml.Predictor.fit(Predictor.scala:151)
at $line14.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw.(example:44)
at $line14.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw.(example:61)
at $line14.$read$$iw$$iw$$iw$$iw$$iw$$iw.(example:63)
at $line14.$read$$iw$$iw$$iw$$iw$$iw.(example:65)
at $line14.$read$$iw$$iw$$iw$$iw.(example:67)
at $line14.$read$$iw$$iw$$iw.(example:69)
at $line14.$read$$iw$$iw.(example:71)
at $line14.$read$$iw.(example:73)
at $line14.$read.(example:75)
at $line14.$read$.(example:79)
at $line14.$read$.(example)
at $line14.$eval$.$print$lzycompute(example:7)
at $line14.$eval$.$print(example:6)
at $line14.$eval.$print(example)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:747)
at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1020)
at scala.tools.nsc.interpreter.IMain.$anonfun$interpret$1(IMain.scala:568)
at scala.reflect.internal.util.ScalaClassLoader.asContext(ScalaClassLoader.scala:36)
at scala.reflect.internal.util.ScalaClassLoader.asContext$(ScalaClassLoader.scala:116)
at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:41)
at scala.tools.nsc.interpreter.IMain.loadAndRunReq$1(IMain.scala:567)
at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:594)
at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:564)
at scala.tools.nsc.interpreter.ILoop.$anonfun$pasteCommand$11(ILoop.scala:795)
at scala.tools.nsc.interpreter.IMain.withLabel(IMain.scala:111)
at scala.tools.nsc.interpreter.ILoop.interpretCode$1(ILoop.scala:795)
at scala.tools.nsc.interpreter.ILoop.pasteCommand(ILoop.scala:801)
at org.apache.spark.repl.SparkILoop.$anonfun$process$8(SparkILoop.scala:177)
at org.apache.spark.repl.SparkILoop.$anonfun$process$8$adapted(SparkILoop.scala:176)
at scala.collection.immutable.List.foreach(List.scala:431)
at org.apache.spark.repl.SparkILoop.loadInitFiles$1(SparkILoop.scala:176)
at org.apache.spark.repl.SparkILoop.$anonfun$process$4(SparkILoop.scala:166)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at scala.tools.nsc.interpreter.ILoop.$anonfun$mumly$1(ILoop.scala:166)
at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:206)
at scala.tools.nsc.interpreter.ILoop.mumly(ILoop.scala:163)
at org.apache.spark.repl.SparkILoop.loopPostInit$1(SparkILoop.scala:153)
at org.apache.spark.repl.SparkILoop.$anonfun$process$10(SparkILoop.scala:221)
at org.apache.spark.repl.SparkILoop.withSuppressedSettings$1(SparkILoop.scala:189)
at org.apache.spark.repl.SparkILoop.startup$1(SparkILoop.scala:201)
at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:236)
at org.apache.spark.repl.Main$.doMain(Main.scala:78)
at org.apache.spark.repl.Main$.main(Main.scala:58)
at org.apache.spark.repl.Main.main(Main.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:958)
at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1046)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1055)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
ml.dmlc.xgboost4j.java.XGBoostError: XGBoostModel training failed.
at ml.dmlc.xgboost4j.scala.spark.XGBoost$.trainDistributed(XGBoost.scala:435)
at ml.dmlc.xgboost4j.scala.spark.XGBoostRegressor.train(XGBoostRegressor.scala:190)
at ml.dmlc.xgboost4j.scala.spark.XGBoostRegressor.train(XGBoostRegressor.scala:37)
at org.apache.spark.ml.Predictor.fit(Predictor.scala:151)
... 60 elided

executors log:
2022-06-17 14:04:32 [dispatcher-Executor]  INFO [YarnCoarseGrainedExecutorBackend:57]: Got assigned task 92
2022-06-17 14:04:32 [Executor task launch worker for task 60.0 in stage 1.0 (TID 92)]  INFO [Executor:57]: Running task 60.0 in stage 1.0 (TID 92)
2022-06-17 14:04:33 [Executor task launch worker for task 60.0 in stage 1.0 (TID 92)]  INFO [FileScanRDD:57]: Reading File path: /train, range: 3976977000-4043259950, partition values: [empty row]
2022-06-17 14:04:53 [Executor task launch worker for task 60.0 in stage 1.0 (TID 92)]  INFO [Executor:57]: Finished task 60.0 in stage 1.0 (TID 92). 1759 bytes result sent to driver
2022-06-17 14:04:54 [dispatcher-Executor]  INFO [YarnCoarseGrainedExecutorBackend:57]: Got assigned task 102
2022-06-17 14:04:54 [Executor task launch worker for task 6.0 in stage 2.0 (TID 102)]  INFO [Executor:57]: Running task 6.0 in stage 2.0 (TID 102)
2022-06-17 14:04:54 [Executor task launch worker for task 6.0 in stage 2.0 (TID 102)]  INFO [MapOutputTrackerWorker:57]: Updating epoch to 1 and clearing cache
2022-06-17 14:04:54 [Executor task launch worker for task 6.0 in stage 2.0 (TID 102)]  INFO [TorrentBroadcast:57]: Started reading broadcast variable 4 with 1 pieces (estimated total size 4.0 MiB)
2022-06-17 14:04:54 [Executor task launch worker for task 6.0 in stage 2.0 (TID 102)]  INFO [TransportClientFactory:310]: Successfully created connection to after 1 ms (0 ms spent in bootstraps)
2022-06-17 14:04:54 [Executor task launch worker for task 6.0 in stage 2.0 (TID 102)]  INFO [MemoryStore:57]: Block broadcast_4_piece0 stored as bytes in memory (estimated size 5.2 KiB, free 2.8 GiB)
2022-06-17 14:04:54 [Executor task launch worker for task 6.0 in stage 2.0 (TID 102)]  INFO [TorrentBroadcast:57]: Reading broadcast variable 4 took 21 ms
2022-06-17 14:04:54 [Executor task launch worker for task 6.0 in stage 2.0 (TID 102)]  INFO [MemoryStore:57]: Block broadcast_4 stored as values in memory (estimated size 9.4 KiB, free 2.8 GiB)
2022-06-17 14:04:55 [Executor task launch worker for task 6.0 in stage 2.0 (TID 102)]  INFO [MemoryStore:57]: Block rdd_20_6 stored as values in memory (estimated size 4.8 MiB, free 2.8 GiB)
2022-06-17 14:04:56 [Executor task launch worker for task 6.0 in stage 2.0 (TID 102)]  INFO [MapOutputTrackerWorker:57]: Don't have map outputs for shuffle 0, fetching them
2022-06-17 14:04:56 [Executor task launch worker for task 6.0 in stage 2.0 (TID 102)]  INFO [MapOutputTrackerWorker:57]: Doing the fetch; tracker endpoint = NettyRpcEndpointRef(spark://MapOutputTracker@) 2022-06-17 14:04:56 [Executor task launch worker for task 6.0 in stage 2.0 (TID 102)]  INFO [MapOutputTrackerWorker:57]: Got the map output locations
2022-06-17 14:04:56 [Executor task launch worker for task 6.0 in stage 2.0 (TID 102)]  INFO [ShuffleBlockFetcherIterator:57]: Getting 64 (255.7 MiB) non-empty blocks including 8 (32.0 MiB) local and 0 (0.0 B) host-local and 0 (0.0 B) push-merged-local and 56 (223.7 MiB) remote blocks
2022-06-17 14:04:56 [Executor task launch worker for task 6.0 in stage 2.0 (TID 102)]  INFO [TransportClientFactory:310]: Successfully created connection to after 11 ms (0 ms spent in bootstraps)
2022-06-17 14:04:56 [Executor task launch worker for task 6.0 in stage 2.0 (TID 102)]  INFO [ShuffleBlockFetcherIterator:57]: Started 5 remote fetches in 117 ms
2022-06-17 14:05:14 [Executor task launch worker for task 6.0 in stage 2.0 (TID 102)]  INFO [TransportClientFactory:310]: Successfully created connection to after 3 ms (0 ms spent in bootstraps)
2022-06-17 14:05:26 [Executor task launch worker for task 6.0 in stage 2.0 (TID 102)]  INFO [TransportClientFactory:310]: Successfully created connection to after 2 ms (0 ms spent in bootstraps)
2022-06-17 14:56:12 [Executor task launch worker for task 6.0 in stage 2.0 (TID 102)]  INFO [Executor:57]: 1 block locks were not released by task 6.0 in stage 2.0 (TID 102)
[rdd_20_6]
2022-06-17 14:56:12 [Executor task launch worker for task 6.0 in stage 2.0 (TID 102)]  INFO [Executor:57]: Finished task 6.0 in stage 2.0 (TID 102). 1177 bytes result sent to driver
2022-06-17 15:08:23 [dispatcher-Executor]  INFO [YarnCoarseGrainedExecutorBackend:57]: Driver commanded a shutdown
2022-06-17 15:08:24 [dispatcher-Executor]  INFO [YarnCoarseGrainedExecutorBackend:57]: Driver from disconnected during shutdown
2022-06-17 15:08:24 [dispatcher-Executor]  INFO [YarnCoarseGrainedExecutorBackend:57]: Driver from isconnected during shutdown
2022-06-17 15:08:24 [CoarseGrainedExecutorBackend-stop-executor]  INFO [MemoryStore:57]: MemoryStore cleared
2022-06-17 15:08:24 [CoarseGrainedExecutorBackend-stop-executor]  INFO [BlockManager:57]: BlockManager stopped
2022-06-17 15:08:24 [CoarseGrainedExecutorBackend-stop-executor]  INFO [TSDBReporter:222]: stop sending
2022-06-17 15:08:24 [Thread-2]  INFO [ShutdownHookManager:57]: Shutdown hook called

Simplified code：
val xgbReg = new XGBoostRegressor(xgbParam).
setFeaturesCol("features").
setLabelCol("label")
        val xgbInput = spark.read.format("libsvm").load("adata")
        val xgbModel = xgbReg.fit(xgbInput)

        val results = xgbModel.transform(xgbInput)

        xgbModel.nativeBooster.saveModel(fs_out)

@trivialfis could you please help me out thist problem?

@wbo4958 Could you please take a look when you are available?

Sorry for late response, @Jasonzjj , Looks like driver sent the SHUTDOWN from the exeuctor log
 Driver commanded a shutdown
2022-06-17 15:08:24 [dispatcher-Executor] INFO [YarnCoarseGrainedExecutorBackend:57]: Driver from disconnected during shutdown

Could you please double-check?
if the executor log is correct one, please fill the executor log with exceptions.

last, did you solve then problem?

Got same error run on EMR, but it's ok run in local Mac

same error

same error

Closing as the tracker is rewritten https://github.com/dmlc/xgboost/pull/10112

authors: Jasonzjj Wanke15 diggzhang jbcsimple jxudata trivialfis wbo4958


text:
Switch missing values from 0 to NA in R package

For R users, I think that missing values are typically expressed as NAs, not 0s.

xgboost deal with missing values that are manually marked with a special number not in the data set, e.g. -99999. So set the default value to NA does not work.

We do support NA as missing now. I agree this is the right thing to do. And you can achieve this by explicitly setting missing=NA.
As this changes the API thus the compatiblity. I think we want to do the change on major revisions instead.
Thanks

@tqchen you are correct. I made a mistake in xgb.get.DMatrix. It is just fixed.
@kferris10 Can you please merge the change in your PR? Thank you!

This should be applied to python package also?

@sinhrks I think we can do that if most user think this is the right thing to do

I am fine with this one. Thanks for the contribution. @hetong007 Please take another review and see if you can merge it in

Thanks, merged.

source: gharchive/pull-request

repo: dmlc/xgboost

license_type: permissive

license_source: github-api


text:
[WIP] Get column matrix from GHistIndex.

Follow up on https://github.com/dmlc/xgboost/pull/7064 .

Fix training continuation
Close https://github.com/dmlc/xgboost/issues/6148 .

Fix corner case where sparse_threshold is changed during training.

Finish the DMatrix refactoring
Close https://github.com/dmlc/xgboost/issues/4354 .
Close https://github.com/dmlc/xgboost/issues/5143 .

todos

[ ] Add test for training continuation.
[ ] Benchmark.

Closing as not necessary.


text:
use RabitContext intead of init/finalize

Minor cleanup to make use of xgb.rabit.RabitContext.

@RAMitchell @trivialfis


text:
Adjust makeAction to return stable values for use as a react hook

Summary:
This PR addresses the issue of unstable return values from the makeAction function. The original implementation returned a new array with the same elements on each invocation, leading to unnecessary re-renders in React applications.
Changes Made:

Added Weakmap to cache actions arrays
Consume weakmap as cache when returning action arrays

This PR does not introduce any breaking changes and is backward compatible.

This PR didn't quite work as expected (WeakMaps take the object as the key), but I added a very similar approach in 0.20.0. Thanks for prompting this — it's smart since it makes things much smoother for frameworks like React that check equality.

authors: dmotz rogersanick

repo: dmotz/trystero

license: MIT


text:
chore(deps): update eslint to v8

CI - use node 16 as prerequisite for eslint update

:tada: This PR is included in version 1.8.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

authors: dmtrKovalenko piwysocki

repo: dmtrKovalenko/cypress-real-events


text:
Typo in the dbs-client documentation

Yuyi,
there seems to be a typo on the documentation for listFileSummaries. The key returned is num_file, not num_files as mentioned in the documentation here:
https://cms-http-group.web.cern.ch/cms-http-group/apidoc/dbs3-client/current/dbs.apis.html#dbs.apis.dbsClient.DbsApi.listFileSummaries

Thanks Alan, I will correct it in next release.
Yuyi

source: gharchive/issue

authors: amaltaro yuyiguo

repo: dmwm/DBS


text:
WMAgent fails jobs that stay idle too long, causes problems on some HPC

Impact of the bug
Jobs in HPC workflows are never run and (eventually permanently) failed by the agent because they stay idle for too long.
Describe the bug
Not sure it's really a bug, more like a feature that has undesired consequences on some HPC. Basically, the agent kills jobs that stay pending for too long and resubmits them. This seems to count again the retry limit. If it happens too often, the job is permanently failed. Some HPC resources (T3_US_ANL for instance) have very slow resource provisioning, which means the time between job submission and job start can be a few weeks.
How to reproduce it
Assign a workflow to a single site that has very slow resource provisioning.
Expected behavior
Since there is nothing we can do about the speed of the resource provisioning, we need a way to account for that in the agent somehow and not just have it assume that things are broken and permanently fail the jobs.

@hufnagel Dirk, this automatic job removal is configurable at JobStatusLite level. However, note that it applies globally to the agent, not specific to any site and/or workflow.
The configuration change is:
config.JobStatusLite.stateTimeouts = {'Running': 169200, 'Pending': 432000, 'Error': 300}

where Pending maps to jobs idle in the condor queue. That value is in seconds and the current value defines that jobs pending for 5 days will be removed by JobStatusLite.
So, we can increase that value in the HEPCloud agent and restart JobStatusLite. Please let us know what the value should be and we can do that.

Can you set it to 30 days? You know which agent to change?

To try to capture some of the Slack discussion: At the moment, this is seen as something requiring a quick fix (i.e. change an agent to have a really long time out and then use that agent just to submit to the relevant HPC site).  However, longer term, we will have a collection of resources that might have different latencies, and we'll need a way to tune the policy (e.g. how long to wait before giving up) on a site-by-site basis.  If that's not a fair summary, then please @hufnagel or @amaltaro, add additional comments to correct or clarify!
Let me point out one additional wrinkle: I don't really think that this is something that we want to configure at either the agent or the site level.  It's also a property of the workflow.  What I mean by this is that for one workflow, it might make sense to wait patiently in a queue for one month or more while for another workflow it might make more sense to give up after a few days and retry somewhere else.  So, for me, it seems like this is more of a "matchmaking" problem where workflows need to express their tolerance for long in queue wait times and resources need to advertise their typical (90% confidence level historical high and low values) of queue wait time and then the SI layer needs to try to match jobs from these workflows to the apporpriate resources.  I'm worried that any other approach (e.g. WMAgent setting time out values based on some database of site properties) would become operationally unsustainable.  In fact, the more I think about it, there might be other parameters we want to consider, like not submitting jobs from workflows that are more than some fraction done (e.g. 90%) to resources where the jobs could wait for one month to run.
I wonder what would be the right layer(s) to implement this?  E.g. would it be enough for the WM system just to populate a set of job attributes and then let the SI layer handle the matching and when appropriate restarting of the jobs as necessary?  How is it currently implemented?  Does the agent actively kill the job?  Or is it that a time-out value is passed on to SI?

I think the agent actively kills the job. The SI/HTCondor layer is happy with jobs that are pending forever if they don't find any resource they can match.

authors: amaltaro hufnagel klannon

repo: dmwm/WMCore


text:
quantize RequestDisk

I have been fighting all day with 126 to try and reduce the number of auto-cluster thinking that there was something really bad with tuning and resizing (thanks @juztas ).
Turned out that making RequestDisk being 10G or 20G made a huge difference (reduce the number of autocluster from 2800 to 1400).
It's quantized by 1G. I suggest 5G, at least

Instead of going bit by bit, how about we generalize this issue and quantize all ads that affect auto-clustering? That would require checking with the SI team to see what ads they round/quantize (I know about RequestMemory, which is far from optimal BTW) and what would be an ideal (or reasonable) quanta for each of them.

Yes, I agree with Alan, What is auto clustering rules? Could you explain how that works? If we know the algorithm for auto clustering, we can change all the parameters affect that as Alan said.

+1 on doing them all at once. +1 on starting with those that will have an immediate effect

@amaltaro this was done recently, wasn't it ?

Yes, we're still missing RequestMemory though, but we need to be more thoughtful about that one. I think a new issue can be opened.

authors: amaltaro ticoann vlimant


text:
[py2py3] src/python/A/B/C slice 22 - issue 10146 - src

Fixes #10146
Status
In development.
This is just a test to check if starting the migration from the last slice breaks something, see #10189 for the details of why we are testing this.
Related PRs

Previous migration PR: (if open it's just a test)
Next migration PR: (if open it's just a test)

Description
Run futurize and some manual changes on the first batch of src/python/A/B/C.
(any notable change will be reported here)
Is it backward compatible (if not, which system it affects?)
It should be. (Any possible cause for errors will we reported here)
External dependencies / deployment changes
Requires python-future in both py2 and py3 environments.

This will be superseded soon, I will create a new PR


text:
Update snapshot of Unified campaigns; update a couple json templates

Fixes #10241
Status
not-tested
Description
This PR brings in the following changes:

update the test campaigns to be injected into CouchDB (via parseUnifiedCampaigns.py)
update our dump of the Unified campaigns
update a few json templates to avoid transferring 90TB of data for a pileup that is no longer available on the Disk RSEs

Is it backward compatible (if not, which system it affects?)
yes
Related PRs
none
External dependencies / deployment changes
none

Templates with the RelVal pileup fail to complete, all the jobs fail in Task3/Step3 with:
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 1 lumi: 3 event: 208 stream: 5
   [1] Running path 'RAWSIMoutput_step'
   [2] Prefetching for module PoolOutputModule/'RAWSIMoutput'
   [3] Prefetching for module CSCDigiProducer/'simMuonCSCDigis'
   [4] Calling method for module MixingModule/'mix'
Exception Message:
Principal::getByLabel: Found zero products matching all criteria
Looking for type: std::vector<PCaloHit>
Looking for module label: g4SimHits
Looking for productInstanceName: CastorFI

   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

Will have to find another small pileup that works and that can be transferred... lovely!

Now these templates can trigger input data placement (very small data) and they all work fine.
FYI @todor-ivanov

Thank you @amaltaro

authors: amaltaro todor-ivanov


text:
Update requirements.txt and PyPI build scripts for py3 stack

Fixes #10955
Status
In development
Description
This PR updates requirements.txt to include only requirements needed for py3 WMCore services and the new wmagent-devtools package. It also updates build_pypi_packages.sh to build the packages.

Updated operators from == to ~=.
Removed dbs3-pycurl since it is a dependency of dbs3-client.
Removed dbs3-client as a dependency for wmagent-devtools since it was not in the spec.
Added gfal2-python for reqmgr2ms since it was in the spec
Added mysqlclient for wmagent since it was in the spec

Is it backward compatible (if not, which system it affects?)
NO, cannot build py2 PyPI packages for any WMCore service after merge.
Related PRs
None
External dependencies / deployment changes
None

To avoid potentials questions in a future, and to remind ourselves I suggest to add proper explanation into requirements.txt about all possible combinations ==, >= and ~=. I also advise to add proper references to python documentation about each rule. For instance:
# we use different version specifiers to avoid potential python conficts
# == when exact version of the package is required
# >= when specific or new version of the package is required
# ~= when we need compatibility of the version, e.g. stay within major version of the package
# for more details please refer to official Python documentation, see
# https://www.python.org/dev/peps/pep-0440/#version-specifiers

With such proper comment on top of requirements.txt we will keep a reminder to ourself how to move forward with package versions.

@amaltaro my view on this is the following:

venv captures and provides everything we need from Python point of view to run things as production or not, therefore I suggest to make venv build/deploy script be flexible to support all environments. For instance, deploy.sh production "wmagent reqmgr2" command will install wmagent, reqmgr2 in production environment, while deploy.sh development all will install all packages as development environment. Here I only refer to deploy.sh as deployment script, but its actual name is irrelevant.
we should separate venv from non-Python dependencies, i.e. if MariaDB or CouchDB is required it can and should be installed separately since they are independent from WMCore codebase. Such separation will allow to accomplish first item easily, and in fact it the "environment" may be boundaries can be vanishing. For example, I do all my development on my mac laptop. As such, in a past I installed independently from CMS the following services: MySQL, ORACLE, CouchDB, MongoDB, all monitoring tools. Since I can install them using external binaries, I don't need anything from WMCore to take care of them, and I only can clone WMCore and run any service as pure python application. Therefore, the only requirements in my view is presence of Python itself and Python libraries required by WMCore, e.g. CherryPy. Everything else can be run separately. Said that, what is required is set of scripts how to setup and inject schema files, and data which are required for WMCore services
we also should separate authentication from venv, i.e. to develop things I don't need CMSWEB environment and I can access anything through http://localhost:port. As such, authentication layer is only required when you put things into production and it should be independent from WMCore deployment. Said that, the auth layer should also be easy to install in local env. therefore I push hard to switch from apache FEs to auth-proxy-server (which is a static executable which can work in VM, in your local laptop and across OSes). We should not be bound to requirement to have full featured CMSWEB VM with full stack in order to develop and test our services.

@goughes I just noticed there is a conflict with  requirements_py3.txt. You probably need to rebase this PR. In case you need, here is a very good reference for that: https://steveklabnik.com/writing/how-to-squash-commits-in-a-github-pull-request

Fixed the conflict and bumped dbs3-client to 4.0.7. I think this is gtg now @amaltaro.

Thanks Erik! This was the only leftover that Valentin requested as well, so it's good to go now! Thanks!

authors: amaltaro goughes vkuznet


text:
Allow data-manager role to create requests in reqmgr2; check group me…

First issue found by Antanas, their DN was not able to make a POST call to reqmgr2/data/request (create a new request), I had to temporarily give him admin privileges:
https://cmsweb.cern.ch/sitedb/prod/people/pdmvserv%40vocms081.cern.ch
This PR changes basically two things:
a) we always check the group membership of the DN
b) we allow mainly "data-manager" and "production-operator" to also make requests.
@ticoann please review. I still need to test it in my VM.

BTW, there is also this file:
https://github.com/dmwm/WMCore/blob/6f3570a741779d209f0f720647642d51b64845da/src/python/WMCore/WMStats/Auth.py
with the same code. Git grep does not find any one using it. Shall I remove that code?

Thanks Alan, this give all the permission for every group, Is McM allowed to only creating and approving requests? Anyway, I will make this configurable so we can control this run time.  But lets push this for the testing. Do you want me to make new tag and make the pull request for testbed.

Thanks! I suggest we cut a new tag later this evening FNAL timezone :-)
I'll try to improve the authz in the coming weeks.

authors: amaltaro ticoann


text:
Javascript library CDN without a protocol does not work when composite files is enabled

With;

DNN 7.4.2
Host Settings | CDN Settings | Use javascript libraries CDN? = TRUE
Host Settings | Client Resource Management | Enable Composite Files = TRUE
A Javascript library (i.e. jQuery) CDN URL without a protocol (i.e. http/https)

Instead of the JS library being loaded from the CDN a virtually empty composite file is produced (example attached).
Including the protocol in a custom CDN for the JS library is a simple workaround for this issue.
I believe it's correct that CDN URL should not need to include a protocol, if missing it should inherit from the current page request.
I have 2 live DNN 7.4.2 installs (upgraded many times) and 1 clean(ish) dev DNN 7.4.2 install, all of which exhibit the same behaviour.
Regards,
Rhys
38dd4a269cef21535d1dce60fb203090.29.js.txt

This old issue https://dnntracker.atlassian.net/browse/DNN-3299 (closed - cannot reproduce) is the same thing, or very similar, for slightly earlier versions.

Is this still a valid issue?

I wanted to update the CDNs due to the recent jQuery library updates - is it even implemented into PersonaBar yet? Couldn't find anything related to CDNs in the UI and had to change it in the DB @HostSettings

I believe you are correct, the host settings for CDN aren't exposed.  The CDN settings for a JS library (which would only take effect after setting that host setting) can be adjusted by editing that library in the Extensions page of the Persona Bar.

authors: Navilois bdukes david-poindexter rmjcsltd

license: mit

license_source: bigquery


text:
Search results count varies on different pages

Description of bug
Search results count varies on different pages
Steps to reproduce

As a Registered User, do a search that will return multi-paged results. (i.e. Total: 50 results)
Then navigate to page 2 of the search results.
Pick anyone of the results in page 2 (for step 5)
Logout.
Log in as an Admin/Content Owner and make the item in #3 inaccessible to registered users.
Logout.
Log back in as a Registered User. Try to search again for the same keyword.

Current result
At page 1, Total Results will be 50. On page 2, Total results will instead be 49. On page 3, Total Results will be 50 again.
Expected result
Search results count should not vary.
Additional context
Related to: https://dnntracker.atlassian.net/browse/DNN-9442
Search results count should be calculated at the beginning of the search, considering the security restrictions for the user in order to avoid changes in the total result count.
Affected version

[x] 9.2.2
[x] 9.2.1
[x] 9.2
[x] 9.1.1
[ ] 9.1
[ ] 9.0

Closing since PR got merged

authors: daguiler valadas


text:
CRAS cant be compiled

Compiling CRAS (i386)...
tests/cras_test_client.c: In function 'audio_debug_info':
tests/cras_test_client.c:585:10: error: format '%lu' expects argument of type 'long unsigned int', but argument 2 has type 'long long unsigned int' [-Werror=format=]
   printf("stream: %lu dev: %u\n",
          ^
tests/cras_test_client.c: In function 'run_aecdump':
tests/cras_test_client.c:677:10: error: format '%lu' expects argument of type 'long unsigned int', but argument 3 has type 'uint64_t {aka long long unsigned int}' [-Werror=format=]
   printf("Dumping AEC info to %s, stream %lu, fd %d\n",
          ^
tests/cras_test_client.c: In function 'main':
tests/cras_test_client.c:1545:3: error: 'stream_id' may be used uninitialized in this function [-Werror=maybe-uninitialized]
   run_aecdump(client, stream_id, 1);
   ^
tests/cras_test_client.c: At top level:
cc1: error: unrecognized command line option '-Wno-int-in-bool-context' [-Werror]
cc1: all warnings being treated as errors
/usr/lib/gcc/x86_64-linux-gnu/5/../../../i386-linux-gnu/crt1.o: In function `_start':
(.text+0x18): undefined reference to `main'
collect2: error: ld returned 1 exit status
Failed to complete chroot setup.
Unmounting /mnt/stateful_partition/crouton/chroots/xenial...
chronos@localhost / $ god damnit...

whenever i try to install a chroot it always ends with this, ive deleted and reinstalled crouton (after assuming it was crouton) but it seems like more, anyone else ever had this issue?
and how did i cause this? i just used crouton to install xenial the normal way.
the command i used:
sudo sh -e ~/Downloads/crouton -t xorg,gtk-extra,cli-extra,core

day 2, still having the same issue, anyone gonna help?
btw @mareedsmdphd are you still having the same issue?

I just ran the following command and had no problems, the installation finished successfully:

sudo sh ~/Downloads/crouton -n CRAS -t xorg,gtk-extra,cli-extra,core

Here are the two snippets I found pertaining to CRAS in the output:

Compiling CRAS (i386)...
Installing CRAS...

Compiling CRAS (native)...
Installing CRAS...

I ran this on my Acer C710 running FlintOS because that's what I was using at the time.
Here's the croutonversion output:
chronos@localhost ~/Downloads $ sudo edit-chroot -l CRAS
name: CRAS
encrypted: no
Entering /mnt/stateful_partition/crouton/chroots/CRAS...
crouton: version 1-20180611074759~master:3697b784
release: xenial
architecture: amd64
xmethod: xorg
targets: xorg,gtk-extra,cli-extra,core
host: version unknown
kernel: Linux localhost 4.4.64-09104-gffd9a4db8e11 #7 SMP Tue Dec 19 21:19:20 CST 2017 x86_64 x86_64 x86_64 GNU/Linux
freon: yes
Unmounting /mnt/stateful_partition/crouton/chroots/CRAS...

When I get a chance I'll try installing it on one or two of my other Chromebooks and followup here.
Hope this helps,
-DennisLfromGA

Thanks, DennisLfromGA. Things here in Vegas were so very close.
chronos@localhost / $ sudo sh ~/Downloads/crouton -p '/media/removable/USB30a' -r buster -t kde_desktop,xiwi,chrome-dev -n buster3
Password:
Invalid target "kde_desktop".
chronos@localhost / $ sudo sh ~/Downloads/crouton -p '/media/removable/USB30a' -r buster -t kde,xiwi,chrome-dev -n buster3
WARNING: USB booting is enabled; consider disabling it.
WARNING: Signed boot verification is disabled; consider enabling it.
You can use the following command: sudo crossystem dev_boot_usb=0 dev_boot_signed_only=1
Installing buster-amd64 chroot to /media/removable/USB30a/chroots/buster3
Downloading debootstrap 1.0.101...
################################################################################################################################################################# 100.0%################################################################################################################################################################# 100.0%
Patching debootstrap...
Downloading bootstrap files...
W: Cannot check Release signature; keyring file not available /usr/share/keyrings/debian-archive-keyring.gpg
I: Retrieving InRelease
I: Retrieving Packages
I: Validating Packages
I: Resolving dependencies of required packages...
I: Resolving dependencies of base packages...
I: Checking component main on http://httpredir.debian.org/debian...
I: Retrieving libacl1 2.2.52-3+b1
I: Validating libacl1 2.2.52-3+b1
I: Retrieving adduser 3.117
and then:
he following packages were automatically installed and are no longer required:
binutils binutils-common binutils-x86-64-linux-gnu gcc gcc-7 gcc-7-multilib gcc-multilib lib32asan4 lib32atomic1 lib32cilkrts5 lib32gcc-7-dev lib32gcc1 lib32gomp1
lib32itm1 lib32mpx2 lib32quadmath0 lib32stdc++6 lib32ubsan0 libasan4 libasound2-dev:i386 libatomic1 libbinutils libc-dev-bin libc6-dev libc6-dev:i386 libc6-dev-i386
libc6-dev-x32 libc6-i386 libc6-x32 libcc1-0 libcilkrts5 libdpkg-perl libgcc-7-dev libglib2.0-0 libitm1 liblsan0 libmpx2 libquadmath0 libspeex-dev:i386 libspeex1:i386
libspeexdsp-dev:i386 libtsan0 libubsan0 libx32asan4 libx32atomic1 libx32cilkrts5 libx32gcc-7-dev libx32gcc1 libx32gomp1 libx32itm1 libx32quadmath0 libx32stdc++6
libx32ubsan0 linux-libc-dev linux-libc-dev:i386 pkg-config
Use 'apt autoremove' to remove them.
Suggested packages:
ed diffutils-doc
The following NEW packages will be installed:
patch
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 123 kB of archives.
After this operation, 241 kB of additional disk space will be used.
Get:1 http://cdn-fastly.deb.debian.org/debian buster/main amd64 patch amd64 2.7.6-2 [123 kB]
Fetched 123 kB in 2s (65.8 kB/s)
Selecting previously unselected package patch.
(Reading database ... 15853 files and directories currently installed.)
Preparing to unpack .../patch_2.7.6-2_amd64.deb ...
Unpacking patch (2.7.6-2) ...
Setting up patch (2.7.6-2) ...
patch set to automatically installed.
Compiling CRAS (i386)...
tests/cras_test_client.c: In function 'audio_debug_info':
tests/cras_test_client.c:585:21: error: format '%lu' expects argument of type 'long unsigned int', but argument 2 has type 'long long unsigned int' [-Werror=format=]
printf("stream: %lu dev: %u\n",
^
%llu
(unsigned long long)info->streams[i].stream_id,
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
tests/cras_test_client.c: In function 'run_aecdump':
tests/cras_test_client.c:677:44: error: format '%lu' expects argument of type 'long unsigned int', but argument 3 has type 'uint64_t {aka long long unsigned int}' [-Werror=format=]
printf("Dumping AEC info to %s, stream %lu, fd %d\n",
^
%llu
tests/cras_test_client.c: In function 'main':
tests/cras_test_client.c:1545:3: error: 'stream_id' may be used uninitialized in this function [-Werror=maybe-uninitialized]
run_aecdump(client, stream_id, 1);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors
/usr/lib/gcc/x86_64-linux-gnu/7/../../../i386-linux-gnu/Scrt1.o: In function _start': (.text+0x28): undefined reference to main'
collect2: error: ld returned 1 exit status
Failed to complete chroot setup.
Unmounting /media/removable/USB30a/chroots/buster3...
chronos@localhost / $
But you've just given me an idea--I don't have to put 6 lb of meat in one giant ball.
Thanks, guys.
Mike.

I just ran the following command on my Pixelbook and had no problems, the installation finished successfully:

sudo sh ~/Downloads/crouton -n cras -p /var/crouton -t xorg,gtk-extra,cli-extra,core

Here are the snippets I found pertaining to CRAS in the output:
chronos@localhost ~/Downloads $ grep CRAS CRAS.script
Fetching CRAS (branch 3a33a280aac17ceaac70220360dcc4688005dcee)...
Compiling CRAS (i386)...
Installing CRAS...
Compiling CRAS (native)...
Installing CRAS...
Audio from the chroot will now be forwarded to CRAS (Chromium OS audio server),
of CRAS. Should this happen, simply update your chroot.

This is the screenfetch output for reference:
chronos@localhost ~/Downloads $ screenfetch -E
             .,:loool:,.               chronos@localhost
         .,coooooooooooooc,.           OS: Chrome OS
      .,lllllllllllllllllllll,.        Kernel: x86_64 Linux 4.4.138-14447-g7e90a0e25e5e
     ;ccccccccccccccccccccccccc;       Uptime: 3d 7h 42m
   'ccccccccccccccccccccccccccccc.     Packages: 0
  ,ooc::::::::okO0000OOkkkkkkkkkkk:    Shell: bash 4.3.48
 .ooool;;;;:xK0kxxxxxk0XK0000000000.   CPU: Intel Core i5-7Y57 @ 4x 3.3GHz [48.0°C]
 :oooool;,;OKdddddddddddKX000000000d   GPU:
 lllllool;lNdllllllllllldNK000000000   RAM: 4257MiB / 7870MiB
 llllllllloMdcccccccccccoWK000000000
 ;cllllllllXXc:::::::::c0X000000000d
 .ccccllllllONkc;,,,;cxKK0000000000.
  .cccccclllllxOOOOOOkxO0000000000;
   .:cccccccclllllllloO0000000OOO,
     ,:ccccccccclllcd0000OOOOOOl.
       '::cccccccccdOOOOOOOkx:.
         ..,::ccccxOOOkkko;.
             ..,:dOkxl:.

And here's the croutonversion output:
chronos@localhost ~/Downloads $ sudo edit-chroot -c /var/crouton/chroots -l cras
name: cras
encrypted: no
Entering /var/crouton/chroots/cras...
crouton: version 1-20180611074759~master:3697b784
release: xenial
architecture: amd64
xmethod: xorg
targets: xorg,gtk-extra,cli-extra,core
host: version 10820.0.0 (Official Build) dev-channel eve
kernel: Linux localhost 4.4.138-14447-g7e90a0e25e5e #1 SMP PREEMPT Tue Jun 26 12:42:21 PDT 2018 x86_64 x86_64 x86_64 GNU/Linux
freon: yes
Unmounting /var/crouton/chroots/cras...

Since we can't get a croutonversion output could you please show us the first five lines displayed when you navigate to: chrome://version
With that have a few more details to see what we're dealing with.
Hope this helps,
-DennisLfromGA

here @DennisLfromGA:
screenfetch:
chronos@localhost ~/Downloads $ sudo bash screenFetch-master/screenfetch-dev
ls: cannot access '/var/db/pkg/*/*': No such file or directory
             .,:loool:,.               root@localhost
         .,coooooooooooooc,.           OS: Chrome OS
      .,lllllllllllllllllllll,.        Kernel: x86_64 Linux 4.4.138-14463-ga3af6d2337eb
     ;ccccccccccccccccccccccccc;       Uptime: 1d 10h 12m
   'ccccccccccccccccccccccccccccc.     Packages: 0
  ,ooc::::::::okO0000OOkkkkkkkkkkk:    Shell: sudo
 .ooool;;;;:xK0kxxxxxk0XK0000000000.   CPU: Intel Celeron N2840 @ 2x 2.5823GHz
 :oooool;,;OKdddddddddddKX000000000d   GPU:
 lllllool;lNdllllllllllldNK000000000   RAM: 3375MiB / 3865MiB
 llllllllloMdcccccccccccoWK000000000
 ;cllllllllXXc:::::::::c0X000000000d
 .ccccllllllONkc;,,,;cxKK0000000000.
  .cccccclllllxOOOOOOkxO0000000000;
   .:cccccccclllllllloO0000000OOO,
     ,:ccccccccclllcd0000OOOOOOl.
       '::cccccccccdOOOOOOOkx:.
         ..,::ccccxOOOkkko;.
             ..,:dOkxl:.
chronos@localhost ~/Downloads $

croutonversion:
chronos@localhost ~/Downloads $ sudo sh -e ~/Downloads/crouton -V
crouton: version 1-20180611074759~master:3697b784
chronos@localhost ~/Downloads $

@KuteKetX,

Since we can't get a croutonversion output could you please show us the first five lines displayed when you navigate to: chrome://version

Hope this helps,
-DennisLfromGA

So, I'm covering for a friend and had to drop by two different
hospitals--three patients and 6 residents. Headed home now, and plan to see
what's the issue.
(BTW, July 1 is when interns and residents start, so it's nice to avoid
teaching hospitals on this day.)
Anyhow, I'm not doing anything novel here; sure I've compiled CRAS without
issue many times. I'm on the Canary channel, so I don't find it remarkable.
My guess: the logic of timing and dependency enforcement may go south. New
compiler,? A subtle change might cause a cast to get skipped. I'm pretty
sure the folks at Google write more sophisticated code than I do. (Joe
Cello, Chris Date, Ted Codd, and I do know a bit about Relational Theory,
to hear one of us tell it.)
Suspect order of adding pkgs may be an issue. Real new stuff. I've only
managed a few non-trivial Sid installs.
I'll look in an hour. BTW, the screenfetch is just swell, I was going to
ask.
Mike
M A Reeds MD PhD DPhil (Oxon)
725-261-2802
mareeds@mareedsmdphd.net
On Jul 1, 2018 8:23 PM, "DennisL" notifications@github.com wrote:
@KuteKetX https://github.com/KuteKetX,
Since we can't get a croutonversion output could you please show us the
first five lines displayed when you navigate to: chrome://version
Hope this helps,
-DennisLfromGA
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dnschneid/crouton/issues/3811#issuecomment-401661269,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AF8HeX-aY6mNXgCWubxmPSK_mzTDQnDFks5uCZIogaJpZM4U-MS_
.

@DennisLfromGA i legit just put the output of crouton version at the bottom.

@KuteKetX,
Nope, that's different.
You gave us the version of the crouton installer not the version of the chroot which is derived from the croutonversion script in the chroot.  Since you can't successfully install a chroot I was trying to ascertain some of your Chromebook details by other means, namely: chrome://version
So, long story short:

please show us the first five lines displayed when you navigate to: chrome://version

-DennisLfromGA

Google Chrome | 69.0.3476.0 (Official Build) canary (64-bit)
-- | --
Revision | 86d261c5d8d1e6cd27cd72f8887c8de670f11b48-refs/branch-heads/3476@{#1}
Platform | 10828.0.0 (Official Build) canary-channel gnawty
Firmware Version | Google_Gnawty.5216.239.156
Customization ID | ACER-OLAY
JavaScript | V8 6.9.237.1
Flash | 30.0.0.134 /opt/google/chrome/pepper/libpepflashplayer.so
User Agent | Mozilla/5.0 (X11; CrOS x86_64 10828.0.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3476.0 Safari/537.36

@KuteKetX,
I suspect this may be a symptom of the 'canary' channel since @mareedsmdphd has reported it happening to him too, although I don't know if he has an Acer Chromebook 11 (C730 / C730E / C735) device also.  Maybe if some others could repo it we might be able to gather enough info. to determine the cause.  I couldn't repo it on my Pixelbook using the dev channel.
Hope this helps,
-DennisLfromGA

I filed this issue https://github.com/dnschneid/crouton/issues/3770 a while ago and it seems to be the same thing. No response, though.
Eventually, the updates from Canary will reach Dev, Beta, and Stable channels, so not sure what the next course of action will be.
My Model: (Dell Chromebook 11 CB1C13)
-CamdynS-

it used to work fine and i dont remember my chromebook ever updating.

So far, this only seems to be occurring on the 'canary' channel, if anyone is experiencing this issue on the beta or stable channel please let us know and show us the output of: sudo edit-chroot -all
Hope this helps,
-DennisLfromGA

-DennisLfromGA, several  days ago, I reverted to the stable channel, and was able to update 4 chroot.
When I went to the dev channel, the issue reappeared. Going beta, no issues.
I've saved output, so I'll send it along in an hour or so.
Mike

@mareedsmdphd,
I have two Chromebooks on the dev channel and have had no problems updating CrOS or my chroots in the past.  I haven't updated them to incorporate the latest merges but I'll do that hopefully tomorrow and report my results back here.
Hope this helps,
-DennisLfromGA

Hi, @coderofsalvation , here's what I did so as not to lose things.
I copied the contents of my Downloads to a removable medium (USB 3.0 formatted to EXT 4)
I downloaded and ran the Chromebook Recovery Utility:
Check out "Chromebook Recovery Utility": https://chrome.google.com/webstore/detail/chromebook-recovery-utili/jndclpdbaamdhonoechobihbbiimdgai?utm_source=gmail
Oh, naturally, I did a sync. Little difficult because I was unable to do a wifi connect: most network interfaces were corrupted and irreparable. Used tethering with my Android.
Anyhow, I recovered to stable, updated and then went to beta.
No data loss, but not a trivial amount of time.
Let us know how that works.
Mike

Yup, happenin' on the dev channel now too but the devs are on it. 👍

@DennisLfromGA , @drinkcat , @dnschneid 👍
Glad to see this closed. Was looking at Error Updating Chroot #3770, Error Updating Chroot #3770, and then https://bugs.chromium.org/p/chromium/issues/detail?id=864815, so I hope this went where I think it went.
crhk201807171845.txt

Google Chrome
68.0.3440.59 (Official Build) beta (64-bit)

Revision
d2bc7dee8086bc46fe6153b4b8c45da429245503-refs/branch-heads/3440@{#644}

Platform
10718.50.0 (Official Build) beta-channel banon

Firmware Version
Google_Banon.7287.373.0

ARC
4883785

JavaScript
V8 6.8.275.21

Flash
30.0.0.134 /opt/google/chrome/pepper/libpepflashplayer.so

User Agent
Mozilla/5.0 (X11; CrOS x86_64 10718.50.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.59 Safari/537.36

Command Line
/opt/google/chrome/chrome --ppapi-flash-path=/opt/google/chrome/pepper/libpepflashplayer.so --ppapi-flash-version=30.0.0.134 --ui-prioritize-in-gpu-process --use-gl=egl --enable-native-gpu-memory-buffers --gpu-sandbox-failures-fatal=yes --enable-logging --log-level=1 --use-cras --enable-wayland-server --user-data-dir=/home/chronos --system-developer-mode --login-profile=user --has-chromeos-keyboard --guest-wallpaper-large=/usr/share/chromeos-assets/wallpaper/guest_large.jpg --guest-wallpaper-small=/usr/share/chromeos-assets/wallpaper/guest_small.jpg --child-wallpaper-large=/usr/share/chromeos-assets/wallpaper/child_large.jpg --child-wallpaper-small=/usr/share/chromeos-assets/wallpaper/child_small.jpg --default-wallpaper-large=/usr/share/chromeos-assets/wallpaper/oem_large.jpg --default-wallpaper-small=/usr/share/chromeos-assets/wallpaper/oem_small.jpg --default-wallpaper-is-oem --enable-consumer-kiosk --arc-availability=officially-supported --enterprise-enrollment-initial-modulus=15 --enterprise-enrollment-modulus-limit=19 --login-manager --vmodule=arc/=1,chromeos/login/=1,auto_enrollment_controller=1,/ui/ozone/=1,/ui/display/manager/chromeos/=1,night_light=1,power_button_observer=2,webui_login_view=2,lock_state_controller=2,webui_screen_locker=2,screen_locker=2

Executable Path
/opt/google/chrome/chrome

Profile Path
/home/chronos/u-4a3c10609bf6f592dc377951c063b4cb460384eb

Variations
6a89113b-ccb9a6a5d01ab0d3-f23d1dea59aeb88e-3f4a17df34a6bf44-ca7d8d8064da5c1e-f23d1deab1681d28-1410f109041608a-3f4a17df38eb801c-3f4a17df7c1bc906-f55a797447e5d3db-3d47f4f4125b7f68-39db89b5d442dfb7-eeca42f71149accc-f23d1dea4dc30737-b8a5ea08a582a1b8-ad75ce173042ad4b-ca54bb4798be3390-d93a0620116c6887-d7bc6f4544827ee5-f23d1deaedbcf7c5-51ca58f35485fc4d-ca7d8d809773d3bd-f23d1dea93731dca-cdcbcaf49e5c75f1-ee212b16f79cb77b-3f4a17df4ea303a6-f5d1e975bcc34a89-3f4a17df6e6e0c7e-70658ae4d92562a9-cfe3c2ea4da5ae82-3f4a17df2c1d398c-3f4a17df6973a1cf-3f4a17df4932440-d21eb72dff29b1bd-54dd8886da460ac8-3f4a17df4bc337ce-69465896d1466cda-3f4a17df9a2f4e5b-ca7d8d8017507c76-3d47f4f4494d8760-52325d433ac60855-486e2a9cf296190c-652559964442aae2-7158671eed1d377-e1cc0f1412e17bc5-e1cc0f1475f0f0a0-6bdfffe7e2b18481-4ad60575e7e71889-4ad60575469cffe2-21549ebe3a4029d-f23d1dea8834fcca-cf4f6ead81c6897f-3f4a17df

So, I'll prepare a recovery image, slog some data, and rejoin the dev channel.
Guys are great, and really smart.
As all of you knew when you woke up this morning.
@mareedsmdphd

Downloaded the code with latest commits.  Still experiencing this error:
Compiling CRAS (native)...
tests/cras_test_client.c: In function 'run_aecdump':
tests/cras_test_client.c:678:10: error: format '%lu' expects argument of type 'long unsigned int', but argument 3 has type 'uint64_t' [-Werror=format=]
aecdump_file, stream_id, aecdump_fd);
^
cc1: all warnings being treated as errors
/usr/lib/gcc/arm-linux-gnueabihf/4.8/../../../arm-linux-gnueabihf/crt1.o: In function _start': (.text+0x28): undefined reference to main'
collect2: error: ld returned 1 exit status
Failed to complete chroot setup.
Output of chrome://version:

Google Chrome
69.0.3494.0 (Official Build) dev (32-bit)

Revision
e91414c45bcdc6397f1a38faa0c826fbc47cd772-refs/branch-heads/3494@{#1}

Platform
10888.0.0 (Official Build) dev-channel veyron_minnie

Firmware Version
Google_Veyron_Minnie.6588.237.0

ARC
4898482

Agreed, still having a bit of an issue here.

Google Chrome
70.0.3502.0 (Official Build) canary (64-bit)

Revision
bd32163382ec789e29eeceaf695587d76d77eb36-refs/branch-heads/3502@{#1}

Platform
10918.0.0 (Official Build) canary-channel wolf

Firmware Version
Google_Wolf.4389.24.62

Compiling CRAS (i386)...
/usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-linux-gnu/4.9/libgcc.a when searching for -lgcc
/usr/bin/ld: cannot find -lgcc
/usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-linux-gnu/4.9/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: cannot find -lgcc_s
collect2: error: ld returned 1 exit status
Failed to complete chroot setup.
Unmounting /media/removable/USB Drive/chroots/xenial...
Sending SIGTERM to processes under /media/removable/USB Drive/chroots/xenial...
-CamdynS-

Yes, actually I have experienced this on all 4 channels, using both my Samsung Chromebook 4 and my Acer Chromebook 315. Most recently, Ive been experiencing this ongoing issue using my Acer. My preference is usually the dev channel. However, Ive now attempted installing crouton  via:
sudo crouton -r <xenial> -t x11,core,lxde,audio,keyboard,xiwi,cli-extra  -n xenial
while on all channels (canary, dev, beta, and now stable) All giving me the same output. It's rather frustrating honestly, I don't understand what the problem is.
But, just to provide a bit more detail:
I download crouton. like anyone normally would from https://goo.gl/fd3zc
Then I Mount my sd card containing an ext4 fs to /media/SD
mkdir /media/SD
sudo mount /dev/sdb /media/SD
Then I create the "chroots" directories in /media/SD/chroots and I syslink it to /usr/local/chroots:
mkdir /media/SD/chroots
sudo ln -s /media/SD/chroots /use/local/chroots
Then I install crouton to /use/local/bin using:
sudo install -Dt /usr/local/bin -m 755 ~/Downloads/crouton
And then I attempt to sudo crouton  all day long... and I receive this as output ... all day long:
Compiling CRAS (i386)...
libcras/cras_client.c: In function 'request_floop_ready':
libcras/cras_client.c:1827:14: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
if (req == (struct floop_request *)tag)
^
libcras/cras_client.c: In function 'request_floop':
libcras/cras_client.c:4209:40: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
cras_fill_request_floop(&msg, params, (uint64_t)&req);
^
libcras/cras_client.c: At top level:
cc1: warning: unrecognized command line option '-Wno-int-in-bool-context'
/usr/bin/ld: cannot find -lcras
collect2: error: ld returned 1 exit status
Failed to complete chroot setup.
The chroot setup script may be broken. Your chroot is not fully configured.
Removing the chroot setup script. You may want to update your chroot again.
UID 1000 not found in xenial
Unmounting /media/SD/chroots/xenial...
Any extra help would be much appreciated. I've also tried this with other Ubuntu versions (Jessie, trusty), and other desktop environments (such as e17, and xfce)
Thanks

See #4804

authors: CamdynS-CBC DP44 DennisLfromGA Gorvetco KuteKetX dnschneid mareedsmdphd rudedog830

repo: dnschneid/crouton


text:
Device "ip_tables" does not exist on Alpine Linux

I'm trying to run a docker-in-docker (dind) (more specifically, docker:22.06.0-beta.0-dind-rootless) image on a Alpine Linux host, that is running inside VirtualBox, whenever the container starts up, it fails to launch the Docker inside the container, with the following error message:
Signature ok
subject=CN = docker:dind client
Getting CA Private Key
Device "ip_tables" does not exist.
modprobe: can't change directory to '/lib/modules': No such file or directory
open: No such file or directory
[rootlesskit:child ] error: executing [[ip tuntap add name tap0 mode tap] [ip link set tap0 address ...]]: exit status 1

I've figured out that the problem is not on VirtualBox part, as this error didn't happened on an Arch Linux host, so it's specific to Alpine
As Docker uses the host kernel, i've tried enabling it through modprobe ip_tables, but the same error persisted, is there any package that enables these kernel modules on the system, or it's something else that i'm missing?

Similar to https://github.com/docker-library/docker/issues/350#issuecomment-1017021899
https://github.com/rootless-containers/rootlesskit/issues/69#issuecomment-506813393 seems relevant

iproute2 installed as the busybox ip doesn't support tuntap

Sorry for the delay, even after installing iproute2 and ensuring that the ip binary being executed belongs to it, the same error occurs
I think it's also worth mentioning that docker:22.06.0-beta.0-dind-rootless already installs iproute2, so it shouldn't be necessary to manually install it when using this image as a base

By the way, do you know if the modprobe error messages above have anything to do with the container failing to execute, or this problem is solely due to the ip command?

I'm not able to reproduce a failure with that image (running on Debian Bullseye) -- my best guess is that maybe your host kernel is missing tun/tap support or something?

You're right, i've checked if Alpine kernel config includes support for tun/tap, and CONFIG_TUN is included as a module, so running modprobe tun solved the problem
Thanks a lot! @tianon @wglambert

I just hit the same issue. Aside from installing iproute2, I needed to load another module: modprobe iptable_nat.

authors: WhyNotHugo henriquehbr tianon wglambert

repo: docker-library/docker


text:
Kibana not connecting to Elasticsearch with username and password

We tried running the container as
docker run -e ELASTICSEARCH_URL='http://a.b.c.d:9200'
                   -e ELASTICSEARCH_USERNAME='elastic'
                   -e ELASTICSEARCH_PASSWORD='secret!'
                   -p 127.0.0.1:5601:5601 -d kibana:5.2.0

When we browse Kibana UI we see plugin:elasticsearch@5.2.0     Authentication Exception
we can reach ES by curl directly just fine
$ curl -u elastic:secret! http://a.b.c.d:9200/_cat/indices?v

Inside container variables are set just fine.
$ docker exec <cont name> bash -c 'echo "$ELASTICSEARCH_URL,$ELASTICSEARCH_USERNAME,$ELASTICSEARCH_PASSWORD"'

http://a.b.c.d:9200,elastic,secret!
But if we look at
$ docker exec <cont name> bash -c 'cat /etc/kibana/kibana.yml'

elasticsearch.url: 'http://a.b.c.d:9200' => this is fine
#elasticsearch.username: "user" . => This is not set
#elasticsearch.password: "pass" . => This is not set

This image is officially deprecated in favor of upstream's images (see https://www.elastic.co/guide/en/kibana/current/_pulling_the_image.html).
For more information, please see docker-library/elasticsearch#160, docker-library/docs#842, and docker-library/docs#945.

This image has been deprecated in favor of the official kibana image provided and maintained by elastic.co. The upstream images are available to pull via docker.elastic.co/kibana/kibana:[version] like 5.4.2. The images found here will receive no further updates once the 5.6.0 release is available upstream. Please adjust your usage accordingly.
Elastic provides open-source support for Kibana via the elastic/kibana GitHub repository and the Docker image via the elastic/kibana-docker GitHub repository, as well as community support via its forums.

Thanks!

authors: cspnanda tianon

repo: docker-library/kibana


text:
Remove store signature

libtrust schema1 signature are going away.
We should make the registry simpler on this.

fixed by 0c15ab69528059d21395c166b3ec3c72ec297c69

repo: docker/distribution


text:
document commit signing in the contribution guidelines

The CONTRIBUTING.md file for docker/distribution differs from docker/docker.  Should the differences be merged in (as appropriate)?  Specifically regarding signing commits.

To clarify, we need to add the Sign your work section to CONTRIBUTING.md.

authors: nubs stevvooe


text:
notifications: digest url should be sent in event target

Previously, the most accurate reference for a manifest was the tag url. After
adding pull by digest, all event notifications should refer directly to the
digest url. This ensures that event uniquely identifies the target of the
notification. Testing has been added for manifest pull events to check that
this doesn't change.
In addition, the listener interface has been refactored to only use the
repository name, rather than the full repository object.
Signed-off-by: Stephen J Day stephen.day@docker.com

LGTM

authors: RichardScothern stevvooe


text:
404 at /go/compose-native-build/

I found a broken link : https://docs.docker.com/go/compose-native-build/

Thanks; this will be fixed by https://github.com/docker/docker.github.io/pull/11858

Thanks; this will be fixed by https://github.com/docker/docker.github.io/pull/11858

authors: thaJeztah whalehub


text:
Feedback for: compose/compose-file.md

File: compose/compose-file.md
The update_config subsection under deploy still has two fields with description "TODO"

@caervs thanks for the feedback! I've added these descriptions to the file and republished it, so you can check it out here: https://docs.docker.com/compose/compose-file/#/deploy
Much appreciate it, closing the issue. Please ping us again if you find more problems.

Looks good. Thanks for the quick response, @londoncalling!

authors: caervs londoncalling


text:
compose/swarm.md constraints should under environment, rather than labels

URL: https://docs.docker.com/compose/swarm.md
There are 2 examples
version: "2"
services:
  foo:
    image: foo
    volumes_from: ["bar"]
    network_mode: "service:baz"
    labels:
      - "constraint:node==node-1"
  bar:
    image: bar
    labels:
      - "constraint:node==node-1"
  baz:
    image: baz
    labels:
      - "constraint:node==node-1"

and
# Schedule containers on a specific node
labels:
  - "constraint:node==node-1"

# Schedule containers on a node that has the 'storage' label set to 'ssd'
labels:
  - "constraint:storage==ssd"

# Schedule containers where the 'redis' image is already pulled
labels:
  - "affinity:image==redis"

I think both of them are wrong, the constraint should under environment. See the following example from https://docs.docker.com/swarm/scheduler/filter/#/example-node-constraints:
$ docker tcp://<manager_ip:manager_port>  run -d -P -e constraint:storage==ssd --name db mysql

The problem tried to be fixed at PR https://github.com/docker/docker.github.io/pull/348 , however, only part of it fixed. And this issue is still open... :(

The problem tried to be fixed at PR https://github.com/docker/docker.github.io/pull/348 , however, only part of it fixed. And this issue is still open... :(

@johndmulhausen this doesn't seem to be an issue anymore on master. Can probably be closed.


text:
Disable automatic language detection in *.md

On https://docs.docker.com/engine/tutorials/usingdocker/, the output of docker version has some words written normally, some are bold and numbers are colored. This is confusing, since this should be just plain text.
Therefore, each code block in the documentation should have its proper programming language declared, to allow for accurate syntax highlighting.

The best way is to have it default to none, I think.

Page has been taken down.

authors: johndmulhausen mstanleyjones rillig


text:
Update rhel.md

Under "Install using the repository" section
Step 2. echo "<DOCKER-EE-URL>" SHOULD BE echo "<DOCKER-EE-URL>/rhel"
Step 4. <DOCKER-EE-URL>/docker-ee.repo SHOULD BE <DOCKER-EE-URL>/rhel/docker-ee.repo

Proposed changes
Under "Install using the repository" section
Step 2. echo "<DOCKER-EE-URL>" SHOULD BE echo "<DOCKER-EE-URL>/rhel"
Step 4. <DOCKER-EE-URL>/docker-ee.repo SHOULD BE <DOCKER-EE-URL>/rhel/docker-ee.repo

Unreleased project version (optional)

Related issues (optional)

Sorry, this is an incorrect change. The URL schema you are using only applies to Docker EE for Linux, which gives you access to all of the EE bits for all suppoted Linux distributions. These instructions are for the RHEL-only subscription. Please refer to the Docker EE for Linux setup instructions on Docker Store, which tell you how to modify your URL in order to follow these instructions.
cc/ @friism

authors: jgeyser mstanleyjones


text:
Update index.md

Updated ARGS section to point out that args defined befor FROM in Dockerfiles do not get regonized after FROM without explicit redeclartion

Proposed changes

Unreleased project version (optional)

Related issues (optional)

Deploy preview for docsdocker ready!
Built with commit f314f2cd5162d23759a3737cbf50b33af26af98b
https://deploy-preview-8246--docsdocker.netlify.com

https://deploy-preview-8246--docsdocker.netlify.com/compose/compose-file/compose-file-v2/#args
shows the changes
but
https://docs.docker.com/compose/compose-file/#args
does not ?

@darwin-traver PTAL, thanks!

@L-Hudson Are we still waiting for a tech review on the updated content?

@usha-mandya yes we are still waiting for that i am amazed in the early days we got such stuff merged in Minuts something is wrong in this Organisation at present.

@tianon @crosbymichael can some one from you please review that we worked more well some years ago i don't know what happend to this repo

@usha-mandya yes we are still waiting for that i am amazed in the early days we got such stuff merged in Minuts something is wrong in this Organisation at present.

Hey @frank-dspeed ... honestly, it's that we're crazy understaffed at the moment. The folks in management are doing their absolute best to ramp up, but it takes both time and training to get everyone moving forward with speed, y'know? we're a team of four, when the volume of tickets in any given day should require a staff of around ten. And that doesn't even include the other content creation tasks. So. Long story short... we're doing our best. Sorry for the wait.

Not sure why we need this. This is describing a Dockerfile command behavior but the docs are for compose. And it is already documented in Dockerfile docs https://docs.docker.com/engine/reference/builder/#understand-how-arg-and-from-interact
Passing argument values (that is done in compose side) has no distinction between global and local args.

@andrewhsu @tonistiigi Thank you both for the quick responses and input! @frank-dspeed Please review the feedback provided by @tonistiigi and let us know if you have additional concerns. Apologies again for the delayed response - we appreciate your time and effort here.

@tonistiigi it is needed because it took me hours to find that i was building my first time a compose file to build a whole stack of images and that was the first case where i used ARGS then i was wondering why it did not work as expected.
@andrewhsu LGTM

@frank-dspeed - Hi Frank - I made some minor wording/syntax updates. @bermudezmt - Maria, can you give a quick look and merge if everything looks good? Thank you!

authors: GordonTheTurtle L-Hudson frank-dspeed paigehargrave tonistiigi usha-mandya


text:
readme.md in bitbucket repo not showing in hub.docker

Hi,
I have linked up my bitbucket repo with my hub.docker repo (magnetmessaging/mmx-standalone-dist-docker) for autobuild. The autobuild works great; however, the readme (/readme.md) is not showing. I suppose it should be shown automatically in the "Information" tab. Would you please let me know why?
Thank you.
skwok

Hi,
The Github issues are for problems with docker itself, not hub. For hub
problems please email support@docker.com
On Mon, Feb 16, 2015 at 2:29 PM, skwokmag notifications@github.com wrote:

Hi,
I have linked up my bitbucket repo with my hub.docker repo
(magnetmessaging/mmx-standalone-dist-docker) for autobuild. The autobuild
works great; however, the readme is not showing. I suppose it should be
shown automatically in the "Information" tab.
—
Reply to this email directly or view it on GitHub
https://github.com/docker/docker/issues/10840.

Thanks.

authors: jfrazelle skwokmag

repo: docker/docker


text:
cant install on mint LMDE (debian)

the install script is not able to detect the OS Linux Mint LMDE. which is the rolling version of mint based on Debian, Jessie.
I will be hacking this just needed to make a place holder

If you are reporting a new issue, make sure that we do not have any duplicates already open. You can ensure this by searching the issue list for this repository. If there is a duplicate, please close your issue and add a comment to the existing issue instead.
If you suspect your issue is a bug, please edit your issue description to include the BUG REPORT INFORMATION shown below. If you fail to provide this information within 7 days, we cannot debug your issue and will close it. We will, however, reopen it if you later provide the information.
For more information about reporting issues, see CONTRIBUTING.md.
You don't have to include this information if this is a feature request
(This is an automated, informational response)

BUG REPORT INFORMATION
Use the commands below to provide key information from your environment:
docker version:
docker info:
Provide additional environment details (AWS, VirtualBox, physical, etc.):
List the steps to reproduce the issue:
1.
2.
3.
Describe the results you received:
Describe the results you expected:
Provide additional info you think is important:
----------END REPORT ---------
#ENEEDMOREINFO

For what ever reason when I cloned the repo and rand the install script it just worked.
I dont have time to fix this right now so leaving open

@sanguis have you compared the online script with the one in the repository? Is it possible the script hosted at get.docker.com was still on an older version?

Closing as it's possibly was transient error.

authors: GordonTheTurtle LK4D4 sanguis thaJeztah


text:
Docker daemon panic when external registry auth service returns 401

Output of docker version:
Client:
 Version:      1.10.2
 API version:  1.22
 Go version:   go1.5.3
 Git commit:   c3959b1
 Built:        Mon Feb 22 21:37:01 2016
 OS/Arch:      linux/amd64

Server:
 Version:      1.10.2
 API version:  1.22
 Go version:   go1.5.3
 Git commit:   c3959b1
 Built:        Mon Feb 22 21:37:01 2016
 OS/Arch:      linux/amd64

Output of docker info:
Containers: 66
 Running: 0
 Paused: 0
 Stopped: 66
Images: 63
Server Version: 1.10.2
Storage Driver: btrfs
Execution Driver: native-0.2
Logging Driver: json-file
Plugins:
 Volume: local
 Network: bridge null host
Kernel Version: 3.16.0-36-generic
Operating System: Ubuntu 14.04.2 LTS
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 1.955 GiB
Name: <edited>
ID: C7NK:RNIQ:BYLM:I52O:EZAK:O7JP:XFZ2:7EPS:SZ4S:3GVO:E6YK:DZWO

Provide additional environment details (AWS, VirtualBox, physical, etc.):
VirtualBox VM
List the steps to reproduce the issue:

Setup distribution/docker version 2.3 with token authentication
Start a push/pull
When the docker daemon tries to get the authentication token, if the service returns the following response:

HTTP 401
ContentType: application/json
{"error": "unauthorized"}

Describe the results you received:
Docker daemon crashes
Describe the results you expected:
It should log an error saying what is wrong with th response
Provide additional info you think is important:
Panic stack:
panic: runtime error: index out of range

goroutine 176 [running]:
github.com/docker/docker/distribution.retryOnError(0x7f44f1ad06b8, 0xc820d65b40, 0x0, 0x0)
        /usr/src/docker/.gopath/src/github.com/docker/docker/distribution/registry.go:143 +0x138
github.com/docker/docker/distribution.retryOnError(0x7f44f1ad06e0, 0xc820d41bc0, 0x0, 0x0)
        /usr/src/docker/.gopath/src/github.com/docker/docker/distribution/registry.go:150 +0x530
github.com/docker/docker/distribution.(*v2PushDescriptor).Upload(0xc820904f00, 0x7f44f1266420, 0xc820905280, 0x7f44f1266460, 0xc820a381e0, 0x0, 0x0)
        /usr/src/docker/.gopath/src/github.com/docker/docker/distribution/push_v2.go:335 +0x12fb
github.com/docker/docker/distribution/xfer.(*LayerUploadManager).makeUploadFunc.func1.1(0xc820a381e0, 0xc820a38060, 0x7f44f1266810, 0xc820904f00, 0xc820a9f7a0)
        /usr/src/docker/.gopath/src/github.com/docker/docker/distribution/xfer/upload.go:108 +0x1a6
created by github.com/docker/docker/distribution/xfer.(*LayerUploadManager).makeUploadFunc.func1
        /usr/src/docker/.gopath/src/github.com/docker/docker/distribution/xfer/upload.go:150 +0x13e

Looking for related issues, I found https://github.com/docker/docker/issues/18569, which isn't panicking, but I spotted that the response that's  returned in that case is different;
{"message":"Unauthorized","details":null}

Thanks @thaJeztah for your answer, actually Docker is still panicking when changing the response to that

@Soulou it's being looked into while we speak; see the conversation on IRC here; https://botbot.me/freenode/docker-maintainers/2016-03-08/?msg=61733440&page=1

Ok got it, I'm in the channel, thanks for notifying me

Tell me if I can do anything to help/give more info

If an attempt to authenticate to the token server fails, the token server should return a 401 > > Unauthorized response indicating that the provided credentials are invalid.

Quoting from https://docs.docker.com/registry/spec/auth/token/
@Soulou this clearly seems your case I suppose. I don't see anywhere any reference about how those errors should be returned to the docker daemon but I believe the format should be:
{
	"errors": [{
		"code": "UNAUTHORIZED",
		"message": "a message",
		"detail": []
	}]
}

I don't know which token service you're using (I suppose it's a custom one), so could you provide with format I gave above?

I got that format from reading the code, I believe the code needs a proper check instead of panicing though

Perfect:
354ec3d6a342: Preparing
52d7ef0f0957: Waiting
17191cd7bd3d: Waiting
f4288da73a4c: Waiting
62429a5c28dd: Waiting
5f70bf18a086: Waiting
e8061ac24ae3: Waiting
3c84ae1bbde2: Waiting
b5f05191d11d: Waiting
unauthorized: no authorization provided

great, let's see if this is reproducible in 1.9 also, I believe now that it's up to the token service to comply with the format docker/distribution expects but I see docs lack wrt this kind of documentation. I'm not sure it's up to docker/docker to handle this - but I believe we could handle this better

distribution config.yml
version: 0.1
storage:
  filesystem:
    rootdirectory: /var/lib/registry
auth:
  token:
    realm: "http://localhost:5001"
    service: "registry"
    issuer: "auth-registry"
    rootcertbundle: /cert.pem
http:
  addr: 0.0.0.0:5000
  secret: thisisasecret

self signed Cert generation
sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout server.key -out server.pem

https://github.com/docker/docker/issues/21027#issuecomment-193788160 /cc @aaronlehmann does it needs to be documented or we could just check if v[0] in retry error is actually filled?

(or both) ;-)

@Soulou I cannot reproduce the panic when pulling could you double check it's an issue at pull time? I reproduced and I'm about to make a PR

I've to try with the pull operation, only tried when pushing so far. (I needed to get a first image in before pulling it ;-) )

@Soulou could you send me an e-mail at sebastiaan@docker.com? Wondering if you'd be interested to be added to our "testers" mailinglist

I've to try with the pull operation, only tried when pushing so far. (I needed to get a first image in before pulling it ;-) )

talked on IRC, the panic is only in push (not also pull)
pulls die with:
Error response from daemon: Get http://localhost:5005/v2/busybox/manifests/latest: <nil>

w/o panicing

This is resolved by https://github.com/docker/docker/pull/21030, but will have to be merged back to master after 1.10.3 has been released

@thaJeztah has this been merged back into master? does it need a label?

@runcom oh, good one; I think this was the only PR directly against the bump branch; can you open a PR to cherry-pick it back to master?

sure!

reopening for master

authors: Soulou runcom thaJeztah


text:
Always remove testexp.tar test file

File was not removed in case of failure
File was removed from /tmp but really created in integration-cli/

LGTM

Thanks @duglin, updated!

assuming janky is happy, still LGTM

@icecrime minor errors

@icecrime no compile :( your defer that has cleanedContainerID has to be after the assignment, right?

Oh god, I'm sorry :-/ Wanted to do this fast, and ended up doing :hankey:

squash?

LGTM

@icecrime can you squash this then LGTM

Done!

CI may fail as #12467

janky errors are not related to this so ignoring....

authors: HuKeping crosbymichael duglin estesp icecrime


text:
12553 fix typo

Started in response to https://github.com/docker/docker/issues/12553 but that looks like it has already been fixed. Grepped the codebase and found one more.

Can you please sign your commits following these rules:
https://github.com/docker/docker/blob/master/CONTRIBUTING.md#sign-your-work
The easiest way to do this is to amend the last commit:
$ git clone -b "12553-cli-doc-correction" git@github.com:buddhamagnet/docker.git somewhere
$ cd somewhere
$ git commit --amend -s --no-edit
$ git push -f

This will update the existing PR, so you do not need to open a new one.

LGTM

LGTM

@buddhamagnet nice! ty

authors: GordonTheTurtle buddhamagnet duglin estesp moxiegirl


text:
client/load: tiny typo fix

Signed-off-by: Ma Shimiao mashimiao.fnst@cn.fujitsu.com

Ah, great! Thank you.
LGTM

LGTM

authors: Mashimiao runcom thaJeztah


text:
Make official install instructions use curl instead of wget

YOU GO CURL
Fixes #14650

Please sign your commits following these rules:
https://github.com/docker/docker/blob/master/CONTRIBUTING.md#sign-your-work
The easiest way to do this is to amend the last commit:
$ git clone -b "wgetcurl" git@github.com:ankushagarwal/docker.git somewhere
$ cd somewhere
$ git commit --amend -s --no-edit
$ git push -f

Ammending updates the existing PR. You DO NOT need to open a new one.

Reviewers :
Docs : @moxiegirl @thaJeztah
Infra maintainers : @jfrazelle @crosbymichael

LGTM I like curl because I can remember the -sSL :-)

Me too :)

LGTM hahaha love the title

Shouldn't we add a little "If you don't have curl and/or have/like wget, the equivalent is wget -qO-" ?
But other than that, yeah curl -sSL is easier to remember :blush: so :+1:

but wget is cooler!

Right-click the URL in your webbrowser and choose "save linked file as..."  😜

LGTM thanks @ankushagarwal

@calavera cherry pick?

cherry picked into #15091

Next person to crack at this ought to add -f to all those curl invocations. :innocent:

authors: GordonTheTurtle LK4D4 ankushagarwal calavera jfrazelle moxiegirl thaJeztah tianon vdemeester


text:
Add shorten function for Go templates

This fix is part of the discussion in #28199 about using shorten to replace --no-trunc.
As part of the fix, a new function shorten has been added for Go templates so that it is possible to use
docker stack services --format "{{.ID | shorten}}: {{.Mode}} {{.Replicas}}"

to show short ID.
A unit test has been added.
This fix is related to #28199.
Signed-off-by: Yong Tang yong.tang.github@outlook.com

LGTM

Thanks @dnephin the PR has been updated with unit tests using assertion helpers. Other tests in pkg/templates/templates_test.go has also been updated to use it. (not necessarily related to shorten but the change is small enough).

Thanks @cpuguy83 @vdemeester for the comment. Now shorten takes the int for the max length of the truncation. Please take a look.

The correct word your looking for is truncate.

I agree that truncate would be better; it's also consistent with the current --no-trunc flag, so easier to explain that they relate.
Moving it back to design for a bit

We currently have
	"json":
	"split"
	"join"
	"title"
	"lower"
	"upper"
	"pad"

I am thinking maybe we could consider trunc?
It seems truncate is a litter long compared with other functions. Though either is fine for me.

👍 for truncate.

Thanks all for the review. The PR has been updated with truncate. The title and description of the PR has also been updated. Please take a look.

@cpuguy83 @dnephin @thaJeztah @vdemeester feel free to merge.

authors: LK4D4 dnephin stevvooe thaJeztah tiborvass yongtang


text:
Add support for different Dockerfile (Docker 1.5.0)

Since Docker allows multiple Dockerfile in a single directory, it would be great if docker-compose allows a Dockerfile as param for "build" instead of a directory.

That would be really great.
For example let's say I have a Django application, which uses Grunt for asset compilation. Today I have to put one of the container in a separate directory, which doesn't have access to my code at build time. This means that I have to run npm install and grunt install after the first build.
With the ability to use several Dockerfile in the same directory, this isn't an issue anymore.
tl;dr: several containers might need the code at build-time, hence it is useful.

Duplicate of #834

authors: birdy- dnephin k4nar

repo: docker/fig


text:
Unable to pull image

Error while pulling image: Get https://index.docker.io/v1/repositories/library/ghost/images: dial tcp: lookup index.docker.io on [::1]:53: read udp [::1]:44472->[::1]:53: read: connection refused

I would try to restart the VM to get the network layer reset: docker-machine restart default

repo: docker/kitematic


text:
Starting saved machine not working in Virtualbox

I have a machine that is in the saved state and I cannot restart it. When I do docker-machine -D start mymachine, I get error I pasted at bottom of this:
https://github.com/boot2docker/boot2docker-cli/issues/292

For me, it never works. I always re-create a new machine if it is in saved state.
Mac, machine version 0.3.0

@chouclee I was able to use the virtualbox dashboard to power off the machine and then use docker-machine to start it

@erichonkanen I never tried to start it via GUI. I can try your solution next time. Thanks.
"it never works" I mean by command line docker-machine start --.

I cannot start the machine after it is in the saved state. This is the error I get:
STDERR:
executing: /usr/bin/VBoxManage modifyvm dev --nic2 hostonly --nictype2 82540EM --hostonlyadapter2 vboxnet2 --cableconnected2 on
STDOUT:
STDERR: VBoxManage: error: The machine is not mutable (state is Saved)
VBoxManage: error: Details: code VBOX_E_INVALID_VM_STATE (0x80bb0002), component SessionMachine, interface IMachine, callee nsISupports
VBoxManage: error: Context: "COMSETTER(Enabled)(TRUE)" at line 1448 of file VBoxManageModifyVM.cpp
VBoxManage: error: The machine is not mutable (state is Saved)
VBoxManage: error: Details: code VBOX_E_INVALID_VM_STATE (0x80bb0002), component SessionMachine, interface IMachine, callee nsISupports
VBoxManage: error: Context: "COMSETTER(AttachmentType)(NetworkAttachmentType_HostOnly)" at line 1449 of file VBoxManageModifyVM.cpp

@saurabh-g so far the workaround is use the VB gui to power off the machine then start it again

@erichonkanen  Yeah, we have a start up script that sets up docker machine as a dev environment. I use vboxmanage controlvm dev poweroff as a part of the script. That does the trick too.

There seems to be a fix for this already in master with commit https://github.com/docker/machine/commit/9053ee24c79b38e757652484197678ee76117de7

Closed by 9053ee24c79b38e757652484197678ee76117de7

authors: chouclee dgageot erichonkanen saurabh-g sergiobuj

repo: docker/machine


text:
AWS driver with --engine-install-url https://test.docker.com/

Using the switch --engine-install-url https://test.docker.com/  together with the AWS driver seems to only install the 'client' and not the 'daemon'
docker-machine create -d amazonec2 .....
Error creating machine: Error running provisioning: ssh command error:
command : sudo systemctl -f start docker
err     : exit status 1
output  : Job for docker.service failed because the control process exited with error code. See "systemctl status docker.service" and "journalctl -xe" for details.

docker-machine ssh  "sudo docker version"
Client:
 Version:      17.06.0-ce-rc5
 API version:  1.30
 Go version:   go1.8.3
 Git commit:   b7e4173
 Built:        Tue Jun 20 07:13:24 2017
 OS/Arch:      linux/amd64
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

● docker.service - Docker Application Container Engine
   Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
  Drop-In: /etc/systemd/system/docker.service.d
           └─10-machine.conf
   Active: inactive (dead) (Result: exit-code) since Thu 2017-06-22 12:51:09 UTC; 25s ago
     Docs: https://docs.docker.com
  Process: 5444 ExecStart=/usr/bin/docker daemon -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --storage-driver aufs --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label env=test --label provider=amazonec2 --experimental=true (code=exited, status=1/FAILURE)
 Main PID: 5444 (code=exited, status=1/FAILURE)

Might be this: https://github.com/docker/machine/issues/4156

authors: axelssonHakan robvelor


text:
Add a "release-candidate" flag to machine creation

This flag will allow a machine created with this flag to be upgradeed to the latest RC

Add filtering logic to b2d check latest release
Only use b2d RC's if --release-candidate is set
Set EngineOpts.InstallURL to test.docker.com with --release-candidate

This ensures that most provisioners will work with this option too

Updates #2098
Signed-off-by: Dave Tucker dt@docker.com

Sorry to sit on this for so long @dave-tucker.  I think my preference would be to go for a --allow-rc flag in docker-machine upgrade instead.  Also, we can't make plugin API changes like are in included this PR.  cc @docker/machine-maintainers for comment and possible close.

Oops, didn't mean to close right away.  Wanted to get the other maintainers' opinions first.

+1 for a flag to update

What do you think about the flag for upgrade @dave-tucker ?  That way it would be docker-machine create -d virtualbox rc; docker-machine upgrade --release-candidate rc to create a VM with the RC and more flexibly update ones which are already created.

@nathanleclaire that was what I had hoped to do when starting out with this PR but IIRC, there didn't appear to be an easy way of changing the InstallURL once the machine was created...
The flag in this PR means "when upgraded, this machine will always have the latest release, including pre-releases" - it's stored in the machine's config
Correct me if I'm wrong, but I think the flag you describe means "when checking the latest release in this upgrade operation, include pre-releases" - it's not stored in the machine's config
That has a very weird behaviour in certain cases which was what eventually convinced me that this made sense to be a creation time flag (changing the machine config to allow RC's after it's created could be solved at a later date)
Let's assume for a moment that 1.10-rc1 is out:
docker-machine create -d virtualbox rc                     # Installs Docker 1.9
docker-machine upgrade --release-candidate rc      # Upgrades to Docker 1.10-rc1

Weird behaviour #1 - downgrading
When rc2 is released:
docker-machine upgrade rc                                     # Downgrades to Docker 1.9
docker-machine upgrade --release-candidate rc     # Upgrades to Docker 1.10-rc2

The "upgrade" flag now performs a downgrade from the RC to the latest GA release if I forget to add the correct flag
Weird behaviour #2 - rc flag has no effect
Let's assume now that I've successfully upgrade my machine to the final RC, and Docker 1.10 is released
Using the --release-candidate flag I get the same result as without it!
docker-machine upgrade rc                                      # Upgrades to Docker 1.10
docker-machine upgrade --release-candidate rc      # Upgrades to Docker 1.10

Re: this PR, I see two options

Close it in favour of a flag on upgrade
Fix it

Re 2), I can do that.
I'll need some help from someone with some familiarity with the code...
We might be able to remove the Driver API changes... The only thing we need is to know if we are trying to provision a "RC" or not, if we can get that bool without having to call the driver that would be most awesome!

authors: dave-tucker dgageot nathanleclaire


text:
use env in compose test

Signed-off-by: Victor Vieux vieux@docker.com

thanks @dnephin works great in 1.6.2

LGTM
ping @dongluochen @nishanttotla

LGTM

authors: abronan dongluochen vieux

repo: docker/swarm


text:
Simplify some redundant code

Signed-off-by: kevinetc123 kaiwentan@harmonycloud.cn

Codecov Report

Merging #1942 into master will decrease coverage by -0.34%.

@@            Coverage Diff             @@
##           master    #1942      +/-   ##
==========================================
- Coverage   54.51%   54.18%   -0.34%
==========================================
  Files         108      108
  Lines       18548    18547       -1
==========================================
- Hits        10112    10049      -63
- Misses       7216     7267      +51
- Partials     1220     1231      +11

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ce8e78a...f47e0c7. Read the comment docs.

LGTM

repo: docker/swarmkit


text:
CloseButtonVisible 不启作用

对于固定的容器，不建议用户关闭，设置  CloseButtonVisible 为false时，不起作用。  期望修复。

Dupliate to #356.

repo: dockpanelsuite/dockpanelsuite


text:
[composer] Undefined index: extra

Image service-cli 2.11 is build with global composer package zaporylie/composer-drupal-optimizations using this line in Dockerfile for example:
composer global require zaporylie/composer-drupal-optimizations:^1.1 >/dev/null; \
The version of this package shipped in those images are the 1.1.1 which is broken (Issue)
Images impacted (as far as i know) :

2.11-*
2.10-*

Workaround : fin composer global require zaporylie/composer-drupal-optimizations:1.1.2
Proposition : Rebuild new version of image (2.12) to shipped the fixed version of this package (1.1.2)

2.9-* seems to be impacted too...

@Klee-scoisne @vaza18 the images have been rebuilt, this should be all set. Run fin update to get the latest images, or fin update --project-images to pull images based on project usage.

And then run fin reset cli for each project.

Update done flawlessly on my side. (fin update + fin reset cli)
Thank for the bump

To be clear, we rolled out a 2.11.1 hotfix release.
Anyone using the default Docksal stack with Docksal v1.14.0+ (fin v1.97.0+) can get it by running fin update (one time, global) and fin reset cli (in each project) as was mentioned above.
For projects using customized stacks with older docksal/cli versions the solution is to bump the image version to 2.11.
If this becomes a problem for anyone, ping me. cc @vaza18

authors: Klee-scoisne lmakarov lpeabody shelane vaza18

repo: docksal/service-cli