text:
NullPointerException at fr.cenotelie.hime.redist.parsers.GSS.hasEdge after changing positions in grammar

Hi,
I have a simple grammar to parse a string of alternating PING and PONG terminals.
The grammar looks like this:
grammar PingPong
{
  options
  {
    Axiom = "s"; // the top variable for this grammar
    Separator = "SEPARATOR"; // the terminal that represent white space
    ParserType = "RNGLR";
    Runtime = "Java";
    AccessModifier = "Public";
    Namespace = "pingpong";
  }
  terminals
  {
    WHITE_SPACE -> U+0020 | U+0009 | U+000B | U+000C ;
    SEPARATOR   -> WHITE_SPACE+;
    PING       -> 'PING';
    PONG       -> 'PONG';
  }
  rules
  {
    s           -> ping
                | pong;
    ping        -> pong PING @OnPongPing
                | PING @OnPing;
    pong        -> ping PONG @OnPingPong
                | PONG @OnPong;
  }
}

I have generated the lexer and parser and created a simple evaluator to count the number of pings and pongs for the given input.
public class PingPongEvaluator extends PingPongParser.Actions implements Evaluator {
    public static final String KEY_PINGS = "pings";
    public static final String KEY_PONGS = "pongs";
    private double pings = 0;
    private double pongs = 0;

    @Override
    public void onPongPing(Symbol head, SemanticBody body) {
        pings++;
    }

    @Override
    public void onPing(Symbol head, SemanticBody body) {
        pings++;
    }

    @Override
    public void onPingPong(Symbol head, SemanticBody body) {
        pongs++;
    }

    @Override
    public void onPong(Symbol head, SemanticBody body) {
        pongs++;
    }

    @Override
    public Map<String, Double> getMetrics() {
        return Map.ofEntries(
                Map.entry(KEY_PINGS, pings),
                Map.entry(KEY_PONGS, pongs)
        );
    }
}

To test the lexer, parser and evaluator, I have created a simple unit test which works fine:
    @Test
    void pingPongPingPongPing() throws InitializationException {
        PingPongLexer lexer = new PingPongLexer("PINGPONGPINGPONGPING");
        PingPongEvaluator evaluator = new PingPongEvaluator();
        PingPongParser parser = new PingPongParser(lexer, evaluator);
        parser.parse();
        Map<String, Double> metrics = evaluator.getMetrics();
        assertEquals(3.0, metrics.get(PingPongEvaluator.KEY_PINGS).doubleValue());
        assertEquals(2.0, metrics.get(PingPongEvaluator.KEY_PONGS).doubleValue());
    }

When I swap the positions of the terminal and non-terminal in the grammar (PING pong instead of pong PING):
...
  rules
  {
    s           -> ping
                | pong;
    ping        -> PING pong @OnPongPing
                | PING @OnPing;
    pong        -> PONG ping @OnPingPong
                | PONG @OnPong;
  }
...

I get an exception:
java.lang.NullPointerException
	at fr.cenotelie.hime.redist.parsers.GSS.hasEdge(GSS.java:157)
	at fr.cenotelie.hime.redist.parsers.RNGLRParser.executeReduction(RNGLRParser.java:647)
	at fr.cenotelie.hime.redist.parsers.RNGLRParser.executeReduction(RNGLRParser.java:621)
	at fr.cenotelie.hime.redist.parsers.RNGLRParser.reducer(RNGLRParser.java:601)
	at fr.cenotelie.hime.redist.parsers.RNGLRParser.parse(RNGLRParser.java:566)
	at pingpong.calculator.PingPongMetricCalculatorTest.pingPongPingPongPing(PingPongMetricCalculatorTest.java:86)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:532)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:115)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:171)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:72)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:167)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:114)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:59)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$4(NodeTestTask.java:108)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:72)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:98)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:74)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$4(NodeTestTask.java:112)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:72)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:98)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:74)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$4(NodeTestTask.java:112)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:72)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:98)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:74)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:220)
	at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$6(DefaultLauncher.java:188)
	at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:202)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:181)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:128)
	at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:71)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38)
	at com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35)
	at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:235)
	at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:54)

Process finished with exit code 255

This only happens when the input is four or more terminals long.
I have tried multiple versions of Hime including the latest (3.5.1). Do you have any idea what could be wrong?
Thanks!

Thank you for reporting this. Indeed, there seems to be a bug in the Java runtime impacting all versions.

Technical note: The GSS.hasEdge method mistakenly uses the GSS generation data for nodes instead of edges, as is the case in the .Net and Rust runtime.

This issue is fixed on master. The fix will be backported to publish a version 3.5.2 of the Java runtime.

Thanks for the quick response!
I have just tested it and it's working 👍

Version 3.5.2 of the Java runtime that includes this fix has been published. It will be replicated into maven central in a few minutes.

I didn't know where to find the build status of the maven package. The link in the readme leads to a 404 (https://dev.azure.com/lwouters/cenotelie/_build/latest?definitionId=6&branchName=master), so I used jitpack.io in my pom to run the latest version directly from Github and verified it that way.

authors: spipnl woutersl

repo: cenotelie/hime


text:
Set status based on HTTP status code

Setting status code only in cases we are confident enough
to map the HTTP status code to error codes.
Fixes #457.

All addressed, PTAL.

@rakyll Sorry I'm late (been traveling); is it possible to export the status(..) function so that we can get rid of our own status(..) impl?

@jadekler, we need to discuss whether we want to export this API from the ochttp package. Can you copy paste the mapping section for now if you want to reuse?

@rakyll Certainly happy to - I'll make a note to check in with y'all when I'm back! 👍

source: gharchive/pull-request

authors: jadekler rakyll

repo: census-instrumentation/opencensus-go

license_type: permissive

license_source: bigquery


text:
add ability to attach custom headers to traces

This change adds the ability to attach custom gRPC metadata headers to trace requests.
The Go library has a WithHeaders() option with this functionality: https://pkg.go.dev/contrib.go.opencensus.io/exporter/ocagent#WithHeaders

/cc @odeke-em

Thank you for the PR @kirbyquerby!
Kindly pinging @punya @mackenziestarr @jsuereth to help with a review, merge and later release. Thank you all!

authors: kirbyquerby odeke-em

repo: census-instrumentation/opencensus-java


text:
Create README.md

For v12.1 mappings

Description of what has changed

Issues addressed by pull request

Please add, as the second paragraph:
This mapping update from ATT&CK v10.1 to v12.1 was accomplished by applying the Center’s ATT&CK Sync methodology and tools. This mappings update served as a case study for ATT&CK Sync and a 75% reduction in labor hours was achieved as compared to previous update efforts, due to the ability to quickly focus attention on only those mappings which were materially impacted by changes to ATT&CK. ATT&CK Sync is applicable to any project that relies on ATT&CK and is a freely available, open-source project, as are all Center resources. For more specifics on this effort, please visit the Case Study: NIST 800 53 Mappings.

license_source: github-api


text:
Add CDNJS version badge in readme

This will add the badge to show its version on CDNJS and also link to its page on CDNJS!

i've done something i don't normally do, i blindly accepted this PR!!
i'm trusting you know what you are doing :)

authors: ceolter kennynaoh

repo: ceolter/ag-grid

license: mit


text:
corporate security does not like 0755 mode on directories

By default multiple directories (/etc/ceph, /var/lib/ceph) get 0755 mode
Corporate security departments do not like to see 0755 mode on any directories without valid reasoning.
This always need evaluation when creating security papers.
As first step creating directories should have configurable mode
# from ceph-ansible/roles/ceph-defaults/tasks/create_ceph_initial_dirs.yml
---
- name: set_fact ceph_directories
  set_fact:
    ceph_directories:
      - /etc/ceph
      - /var/lib/ceph/
      - /var/lib/ceph/mon
      - /var/lib/ceph/osd
      - /var/lib/ceph/mds
      - /var/lib/ceph/tmp
      - /var/lib/ceph/radosgw
      - /var/lib/ceph/bootstrap-rgw
      - /var/lib/ceph/bootstrap-mds
      - /var/lib/ceph/bootstrap-osd
      - /var/lib/ceph/bootstrap-rbd
      - /var/run/ceph

- name: create ceph initial directories
  file:
    path: "{{ item }}"
    state: directory
    owner: "{{ ceph_uid }}"
    group: "{{ ceph_uid }}"
    mode: 0755
  with_items: "{{ ceph_directories }}"

Can be converted to following:
# from ceph-ansible/roles/ceph-defaults/tasks/create_ceph_initial_dirs.yml
---
- name: set_fact ceph_directories
  set_fact:
    ceph_directories:
      - /etc/ceph
      - /var/lib/ceph/
      - /var/lib/ceph/mon
      - /var/lib/ceph/osd
      - /var/lib/ceph/mds
      - /var/lib/ceph/tmp
      - /var/lib/ceph/radosgw
      - /var/lib/ceph/bootstrap-rgw
      - /var/lib/ceph/bootstrap-mds
      - /var/lib/ceph/bootstrap-osd
      - /var/lib/ceph/bootstrap-rbd
      - /var/run/ceph

- name: create ceph initial directories
  file:
    path: "{{ item }}"
    state: directory
    owner: "{{ ceph_uid }}"
    group: "{{ ceph_uid }}"
    mode: "{{ ceph_directories_mode | default('0755') }}"
  with_items: "{{ ceph_directories }}"

After parametrizing mode for directories corporate user can choose access rights
After this change anyone for whom it matters can easily overwrite access mode via group_vars/all.yml
This change adds functionality without braking anything
Simple to do ;)

Thanks @artur-fijalkowski for the detailed explanation would you mind sending a PR for this?
Thanks in advance.

@leseb https://github.com/ceph/ceph-ansible/pull/2962 brings this into life. I also added some additional places where mode is changed.

source: gharchive/issue

authors: artur-fijalkowski leseb

repo: ceph/ceph-ansible


text:
Error :  'ansible_lsb' is undefined

I git this error everytime I run ansible first time for a host. It disappears in the subsequent run though.
TASK: [ceph-common | add ceph stable repository] ******************************
fatal: [10.33.85.62] => One or more undefined variables: 'ansible_lsb' is undefined
FATAL: all hosts have already failed -- aborting

Hum I've never seen this issue before.
Which version of Ansible do you use?

@leseb . I am using ansible 1.7.2.

@Abhishekvrshny by any chance can you try to upgrade to something like 1.9?

Any updates on this?

@leseb . I am working on debian wheezy and it's ansible v1.7.2 that comes with the official repositories of debian wheezy (in wheezy-backports) and even with the later debian jessie. I was thinking of building ansible v1.9.2 for wheezy and then testing it out. Haven't been able to find time for that. IIRC, I did try ceph-ansible with ansible v1.9.2 once and it was fine.

@Abhishekvrshny building from source is easy take a look at https://github.com/ceph/ceph-ansible/blob/master/install-ansible.sh#L15-L22

Closing this as it appears to work on 1.9.x

authors: Abhishekvrshny leseb


text:
Unable to start Rados Gateway

I'm trying to deploy rgw on VMs using Centos 7.1, but even thou Vagrant and Ansible runs without any issues, the rados gateway is not running automatically.
I've checked port 80 of rgw0 and there's nothing listening on it. Also trying to run: "sudo /etc/init.d/ceph-radosgw start" results in the following:
Sep 18 15:36:56 ceph-rgw0 sudo[5700]: vagrant : TTY=pts/0 ; PWD=/home/vagrant ; USER=root ; COMMAND=/sbin/service ceph-radosgw start
Sep 18 15:36:56 ceph-rgw0 systemd[1]: Starting LSB: radosgw RESTful rados gateway...
Sep 18 15:36:56 ceph-rgw0 ceph-radosgw[5717]: Starting radosgw instance(s)...
Sep 18 15:36:57 ceph-rgw0 ceph-radosgw[5717]: /bin/radosgw is not running.
Sep 18 15:36:57 ceph-rgw0 systemd[1]: ceph-radosgw.service: control process exited, code=exited status=1
Sep 18 15:36:57 ceph-rgw0 systemd[1]: Failed to start LSB: radosgw RESTful rados gateway.
Sep 18 15:36:57 ceph-rgw0 systemd[1]: Unit ceph-radosgw.service entered failed state.
Any suggestions on how to get rgw running?

Can you at least run rgw manually? Do you have anything in rgw's logs?

Hello @leseb
I tried this command as suggested by @yehudasa:
sudo radosgw -c /etc/ceph/ceph.conf -f --log-to -stderr --debug-rgw=20 --debug-ms=1
Running the command returned an error about missing the keyring.
Here's the output:
[vagrant@ceph-rgw0 ~]$ sudo radosgw -c /etc/ceph/ceph.conf -f --log-to -stderr --debug-rgw=20 --debug-ms=1
2015-09-22 17:17:07.521508 7f51118a3880  0 ceph version 0.94.3
(95cefea9fd9ab740263bf8bb4796fd864d9afe2b), process radosgw, pid 8563
2015-09-22 17:17:07.558672 7f51118a3880  0 ext_mime_map_init(): failed
to open file=/etc/mime.types ret=-2
2015-09-22 17:17:07.559001 7f51118a3880  1 -- :/0 messenger.start
2015-09-22 17:17:07.559071 7f51118a3880 -1 monclient(hunting): ERROR:
missing keyring, cannot use cephx for authentication
2015-09-22 17:17:07.559079 7f51118a3880  0 librados: client.admin
initialization error (2) No such file or directory
2015-09-22 17:17:07.559447 7f51118a3880  1 -- :/1008563 mark_down_all
2015-09-22 17:17:07.559676 7f51118a3880  1 -- :/1008563 shutdown
complete.
2015-09-22 17:17:07.560027 7f51118a3880 -1 Couldn't init storage
provider (RADOS)
[vagrant@ceph-rgw0 ~]$

I added the client.admin keyring to ceph.conf:
[vagrant@ceph-rgw0 ~]$ cat /etc/ceph/ceph.conf | grep keyring
  keyring = /etc/ceph/ceph.client.admin.keyring
  keyring = /var/lib/ceph/radosgw/ceph-rgw.ceph-rgw0/keyring
  keyring = /var/lib/ceph/restapi/ceph-restapi/keyring

Now, when running again the command below:
sudo radosgw -c /etc/ceph/ceph.conf -f --log-to-stderr --debug-rgw=20 --debug-ms=1
I see a lot of logging that looks like rgw is running correctly, at
least I don't see any error or failure messages. They look more like
typical heartbeat type of messages:
2015-09-22 18:14:24.391473 7fd1cb7fe700  1 -- 192.168.42.40:0/1008928 -
-> 192.168.42.102:6804/5909 -- ping magic: 0 v1 -- ?+0 0x7fd1bc257420
con 0x2d5c6f0
2015-09-22 18:14:24.391484 7fd1cb7fe700  1 -- 192.168.42.40:0/1008928 -
-> 192.168.42.101:6804/5900 -- ping magic: 0 v1 -- ?+0 0x7fd1bc2576b0
con 0x2d6b890
2015-09-22 18:14:24.391492 7fd1cb7fe700  1 -- 192.168.42.40:0/1008928 -
-> 192.168.42.102:6800/5362 -- ping magic: 0 v1 -- ?+0 0x7fd1bc2579c0
con 0x2d72420
2015-09-22 18:14:24.391500 7fd1cb7fe700  1 -- 192.168.42.40:0/1008928 -
-> 192.168.42.101:6800/5358 -- ping magic: 0 v1 -- ?+0 0x7fd1bc257cd0
con 0x2d7a360
2015-09-22 18:14:24.395421 7fd1c82f3700  1 -- 192.168.42.40:0/1008928
<== osd.0 192.168.42.101:6800/5358 1089 ==== osd_op_reply(3082 notify.4
[watch ping cookie 47691488] v0'0 uv24 ondisk = 0) v6 ==== 175+0+0
(1255968610 0 0) 0x7fd1bc2500f0 con 0x2d7a360
2015-09-22 18:14:24.395533 7fd1c82f3700  1 -- 192.168.42.40:0/1008928
<== osd.0 192.168.42.101:6800/5358 1090 ==== osd_op_reply(3083 notify.6
[watch ping cookie 47695488] v0'0 uv24 ondisk = 0) v6 ==== 175+0+0
(1240792457 0 0) 0x7fd1bc2553c0 con 0x2d7a360
2015-09-22 18:14:24.395608 7fd1c82f3700  1 -- 192.168.42.40:0/1008928
<== osd.0 192.168.42.101:6800/5358 1091 ==== osd_op_reply(3084 notify.7
[watch ping cookie 47697776] v0'0 uv24 ondisk = 0) v6 ==== 175+0+0
(2040385786 0 0) 0x7fd1bc256710 con 0x2d7a360
2015-09-22 18:14:24.395715 7fd1c84f5700  1 -- 192.168.42.40:0/1008928
<== osd.2 192.168.42.102:6800/5362 1113 ==== osd_op_reply(3085 notify.1
[watch ping cookie 47658752] v0'0 uv24 ondisk = 0) v6 ==== 175+0+0
(4057252612 0 0) 0x7fd1b408bd60 con 0x2d72420
2015-09-22 18:14:24.395792 7fd1c84f5700  1 -- 192.168.42.40:0/1008928
<== osd.2 192.168.42.102:6800/5362 1114 ==== osd_op_reply(3086 notify.3
[watch ping cookie 47662672] v0'0 uv24 ondisk = 0) v6 ==== 175+0+0
(2102748158 0 0) 0x7fd1b408bd60 con 0x2d72420

The problem is that when trying to run either S3 or Swift tests
detailed here: http://docs.ceph.com/docs/master/radosgw/config/
I get this error: socket.error: [Errno 111] Connection refused
Running netstat doesn't seem to show anything listening on port 80
[vagrant@ceph-rgw0 ~]$ netstat -tpln
(No info could be read for "-p": geteuid()=1000 but you should be
root.)
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address
 State       PID/Program name
tcp        0      0 0.0.0.0:22              0.0.0.0:*
 LISTEN      -
tcp        0      0 0.0.0.0:7480            0.0.0.0:*
 LISTEN      -
tcp        0      0 127.0.0.1:25            0.0.0.0:*
 LISTEN      -
tcp6       0      0 :::22                   :::*
 LISTEN      -
tcp6       0      0 ::1:25                  :::*
 LISTEN      -

I'm running CentOS-7.1, here's how I modified vagrant_variables.yml
vagrant_box: bento/centos-7.1
vagrant_storagectl: 'SATA Controller'

Note the extra space of vagrant_storagectl
Thanks for the help :)

Hi Thiago,
On Tue, 22 Sep 2015, Thiago da Silva wrote:

Hello @leseb
I tried this command as suggested by @yehudasa:
sudo radosgw -c /etc/ceph/ceph.conf -f --log-to -stderr --debug-rgw=20 --debug-ms=1
Running the command returned an error about missing the keyring.
Here's the output:
[vagrant@ceph-rgw0 ~]$ sudo radosgw -c /etc/ceph/ceph.conf -f --log-to -stderr --debug-rgw=20 --debug-ms=1
2015-09-22 17:17:07.521508 7f51118a3880  0 ceph version 0.94.3
(95cefea9fd9ab740263bf8bb4796fd864d9afe2b), process radosgw, pid 8563
2015-09-22 17:17:07.558672 7f51118a3880  0 ext_mime_map_init(): failed
to open file=/etc/mime.types ret=-2
2015-09-22 17:17:07.559001 7f51118a3880  1 -- :/0 messenger.start
2015-09-22 17:17:07.559071 7f51118a3880 -1 monclient(hunting): ERROR:
missing keyring, cannot use cephx for authentication
2015-09-22 17:17:07.559079 7f51118a3880  0 librados: client.admin
initialization error (2) No such file or directory
2015-09-22 17:17:07.559447 7f51118a3880  1 -- :/1008563 mark_down_all
2015-09-22 17:17:07.559676 7f51118a3880  1 -- :/1008563 shutdown
complete.
2015-09-22 17:17:07.560027 7f51118a3880 -1 Couldn't init storage
provider (RADOS)
[vagrant@ceph-rgw0 ~]$
I added the client.admin keyring to ceph.conf:
[vagrant@ceph-rgw0 ~]$ cat /etc/ceph/ceph.conf | grep keyring
keyring = /etc/ceph/ceph.client.admin.keyring
keyring = /var/lib/ceph/radosgw/ceph-rgw.ceph-rgw0/keyring
keyring = /var/lib/ceph/restapi/ceph-restapi/keyring
Now, when running again the command below:
sudo radosgw -c /etc/ceph/ceph.conf -f --log-to-stderr --debug-rgw=20 --debug-ms=1
I see a lot of logging that looks like rgw is running correctly, at
least I don't see any error or failure messages. They look more like
typical heartbeat type of messages:
2015-09-22 18:14:24.391473 7fd1cb7fe700  1 -- 192.168.42.40:0/1008928 -
-> 192.168.42.102:6804/5909 -- ping magic: 0 v1 -- ?+0 0x7fd1bc257420
con 0x2d5c6f0
2015-09-22 18:14:24.391484 7fd1cb7fe700  1 -- 192.168.42.40:0/1008928 -
-> 192.168.42.101:6804/5900 -- ping magic: 0 v1 -- ?+0 0x7fd1bc2576b0
con 0x2d6b890
2015-09-22 18:14:24.391492 7fd1cb7fe700  1 -- 192.168.42.40:0/1008928 -
-> 192.168.42.102:6800/5362 -- ping magic: 0 v1 -- ?+0 0x7fd1bc2579c0
con 0x2d72420
2015-09-22 18:14:24.391500 7fd1cb7fe700  1 -- 192.168.42.40:0/1008928 -
-> 192.168.42.101:6800/5358 -- ping magic: 0 v1 -- ?+0 0x7fd1bc257cd0
con 0x2d7a360
2015-09-22 18:14:24.395421 7fd1c82f3700  1 -- 192.168.42.40:0/1008928
<== osd.0 192.168.42.101:6800/5358 1089 ==== osd_op_reply(3082 notify.4
[watch ping cookie 47691488] v0'0 uv24 ondisk = 0) v6 ==== 175+0+0
(1255968610 0 0) 0x7fd1bc2500f0 con 0x2d7a360
2015-09-22 18:14:24.395533 7fd1c82f3700  1 -- 192.168.42.40:0/1008928
<== osd.0 192.168.42.101:6800/5358 1090 ==== osd_op_reply(3083 notify.6
[watch ping cookie 47695488] v0'0 uv24 ondisk = 0) v6 ==== 175+0+0
(1240792457 0 0) 0x7fd1bc2553c0 con 0x2d7a360
2015-09-22 18:14:24.395608 7fd1c82f3700  1 -- 192.168.42.40:0/1008928
<== osd.0 192.168.42.101:6800/5358 1091 ==== osd_op_reply(3084 notify.7
[watch ping cookie 47697776] v0'0 uv24 ondisk = 0) v6 ==== 175+0+0
(2040385786 0 0) 0x7fd1bc256710 con 0x2d7a360
2015-09-22 18:14:24.395715 7fd1c84f5700  1 -- 192.168.42.40:0/1008928
<== osd.2 192.168.42.102:6800/5362 1113 ==== osd_op_reply(3085 notify.1
[watch ping cookie 47658752] v0'0 uv24 ondisk = 0) v6 ==== 175+0+0
(4057252612 0 0) 0x7fd1b408bd60 con 0x2d72420
2015-09-22 18:14:24.395792 7fd1c84f5700  1 -- 192.168.42.40:0/1008928
<== osd.2 192.168.42.102:6800/5362 1114 ==== osd_op_reply(3086 notify.3
[watch ping cookie 47662672] v0'0 uv24 ondisk = 0) v6 ==== 175+0+0
(2102748158 0 0) 0x7fd1b408bd60 con 0x2d72420
The problem is that when trying to run either S3 or Swift tests
detailed here: http://docs.ceph.com/docs/master/radosgw/config/
I get this error: socket.error: [Errno 111] Connection refused
Running netstat doesn't seem to show anything listening on port 80
[vagrant@ceph-rgw0 ~]$ netstat -tpln
(No info could be read for "-p": geteuid()=1000 but you should be
root.)
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address
State       PID/Program name
tcp        0      0 0.0.0.0:22              0.0.0.0:*
LISTEN      -
tcp        0      0 0.0.0.0:7480            0.0.0.0:*
LISTEN      -

This is radosgw, on port 7480 by default.  You can put it on port 80 with
rgw frontends = "civetweb port=80"
in ceph.conf.
sage

@liewegas,
ceph.conf was already configured with port 80
[vagrant@ceph-rgw0 ~]$ cat /etc/ceph/ceph.conf | grep civet
  rgw frontends = civetweb port=80

I changed the swift test to point to port 7480 and it seems to be working fine:
[vagrant@ceph-rgw0 ~]$ swift -A http://ceph-rgw0:7480/auth/1.0 -U testuser:swift -K KCwWMLOLqqyfvVsCVtECenMPnEB8fgqCRSW\/Gu+0 upload container1 s3test.py
s3test.py
[vagrant@ceph-rgw0 ~]$
[vagrant@ceph-rgw0 ~]$ swift -A http://ceph-rgw0:7480/auth/1.0 -U testuser:swift -K KCwWMLOLqqyfvVsCVtECenMPnEB8fgqCRSW\/Gu+0 list
container1
[vagrant@ceph-rgw0 ~]$
[vagrant@ceph-rgw0 ~]$ swift -A http://ceph-rgw0:7480/auth/1.0 -U testuser:swift -K KCwWMLOLqqyfvVsCVtECenMPnEB8fgqCRSW\/Gu+0 list container1
s3test.py

Now, I just need to figure out why port 80 is not working and why running sudo service ceph-radosgw start also fails...
Thanks for all the help!

@thiagodasilva make sure that the frontends line is in the correct ceph.conf section (you're running it under user client.admin by default, so you should have a [client.admin] section, and that's where the configurable is supposed to be if you're running it manually).

@yehudasa Here's the link for my ceph.conf: http://ur1.ca/nubaj
I guess the part that I'm confused about is that in the [client.rgw.ceph-rgw0] section there's a keyring option already set (not the client.admin keyring), but it was still necessary for me to add the keyring option in the [global] section.
I realize this is probably very specific to ceph-ansible and how @leseb configured the cluster, so I'm just trying to be helpful and figure out if there is indeed anything that needs to be fixed in ceph-ansible.

@thiagodasilva I'm not familiar with the ansible stuff, but I assume that when running rgw through it, it uses a different ceph user. That's why everything there is under the [client.rgw.ceph-rgw0] section. When you run it manually, it runs under the client.admin user, so different config versions are being applied. If you specify -n client.rgw.ceph-rgw0 on the command line, then it will use the other user that was configured.

@thiagodasilva you shouldn't be using the admin keyring to run rgw.
It might be an issue with the init script.
As @yehudasa said it uses a different user client.rgw.ceph-rgw0.
So you should try to run it manually again with the proper user.
This will give you:
radosgw -c /etc/ceph/ceph.conf -f -n client.rgw.ceph-rgw0

@leseb, @yehudasa,
Thanks, that fixed the problem. I'm now able to (manually) run with the proper user and rgw is listening on port 80.

@thiagodasilva I think now you need to have a look at the init script and check what goes wrong and why. :)

Closing this for now, feel free to reopen.

@leseb
Following Steps I did:

RGW=1 ../src/vstart.sh -d -n -x (worked fine)
swift -A http://localhost:8000/auth -U test:tester -K testing list(worked fine)
3.swift -A http://localhost:8000/auth -U test:tester -K testing upload mycontainer ceph (worked fine)

Now I implement jaeger tracing in /src/librados/IoCtxImpl.cc,
Now everything is working fine except for step 3: above
Error message :
Warning: failed to create container 'mycontainer': HTTPConnectionPool(host='localhost', port=8000): Max retries exceeded with url: /swift/v1/mycontainer (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3409f45150>: Failed to establish a new connection: [Errno 111] Connection refused',))
HTTPConnectionPool(host='localhost', port=8000): Max retries exceeded with url: /auth (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3409f28e50>: Failed to establish a new connection: [Errno 111] Connection refused',))

JaegerTracing implemented part :
oid librados::IoCtxImpl::queue_aio_write(AioCompletionImpl *c)
{
  std::cout<<"yes"<<std::endl;
    JTracer tracer;
    tracer.initTracer("Writing Started","/home/abhinav/Desktop/GSOC/deepika/ceph/src/librados/tracerConfig.yaml");
    Span span=tracer.newSpan("writing started");
    span->Finish();
    // try{
    //   auto yaml = YAML::LoadFile("/home/abhinav/Desktop/GSOC/deepika/ceph/src/librados/tracerConfig.yaml");
    // }catch(const YAML::ParserException& pe){
    //   // ldout<<pe.what()<<dendl;
    //   std::cout<<pe.what()<<std::endl;
    //   ofstream f;
    //   f.open("/home/abhinav/Desktop/err.txt");
    //   f<<pe.what();
    //   f.close();
    // }
      // auto config = jaegertracing::Config::parse(yaml);
    // auto tracer=jaegertracing::Tracer::make(
    //     "Writing",
    //     config,
    //     jaegertracing::logging::consoleLogger()
    // );
    // opentracing::Tracer::InitGlobal(
    //     static_pointer_cast<opentracing::Tracer>(tracer)
    // );
    // auto span = opentracing::Tracer::Global()->StartSpan("Span1");
    get();
    ofstream file;
    file.open("/home/abhinav/Desktop/write.txt",std::ios::out | std::ios::app);
    file<<"Writing /src/librados/IoCtxImpl.cc 310.\n";
    file.close();
    std::scoped_lock l{aio_write_list_lock};
    ceph_assert(c->io == this);
    c->aio_write_seq = ++aio_write_seq;
    ldout(client->cct, 20) << "queue_aio_write " << this << " completion " << c
        << " write_seq " << aio_write_seq << dendl;
    aio_write_list.push_back(&c->aio_write_list_item);
    // opentracing::Tracer::Global()->Close();
}

authors: leseb liewegas suab321321 thiagodasilva yehudasa


text:
why use both --mbrtogpt and -g option for sgdisk?

I found that both --mbrtogpt and -g option for sgdisk, but as I can tell from the sgdisk help info, they are just the same.
  -g, --mbrtogpt                                                       convert MBR to GPT

So should I submit a PR to remove -g?

Correct, yes please submit a PR.


text:
deploy: Fix snapshot controller deployment

Replace image from canary to the version mentioned
in build.env
Signed-off-by: Yug yuggupta27@gmail.com
Fixes: #1791

Show available bot commands
These commands are normally not required, but in case of issues, leave any of
the following bot commands in an otherwise empty comment in this PR:

/retest ci/centos/<job-name>: retest the <job-name> after unrelated
failure (please report the failure too!)
/retest all: run this in case the CentOS CI failed to start/report any test
progress or results

/retest /ci/centos/mini-e2e/k8s-1.20

/retest ci/centos/mini-e2e/k8s-1.20

@mergifyio backport release-3.2

@mergifyio backport release-3.2

@Mergifyio backport release-v3.2

@Mergifyio backport release-v3.2

repo: ceph/ceph-csi


text:
deploy: allow mkfsOptions

Describe what this PR does

Allows configuration of mkfsOptions parameter in the ceph-csi-rbd chart.

Is there anything that requires special attention
Do you have any questions? False
Is the change backward compatible? True
Are there concerns around backward compatibility? False
Provide any external context for the change, if any.
For example:

It was present in the examples, but was missing from the chart.

Related issues
N/A
Future concerns
List items that are not part of the PR and do not impact it's
functionality, but are work items that can be taken up subsequently.
Checklist:

[x] Commit Message Formatting: Commit titles and messages follow
guidelines in the developer
guide.
[x] Reviewed the developer guide on Submitting a Pull
Request
[ ] Pending release
notes
updated with breaking and/or notable changes for the next major release.
[x] Documentation has been updated, if necessary.
[ ] Unit tests have been added, if necessary.
[ ] Integration tests have been added, if necessary.

Show available bot commands
These commands are normally not required, but in case of issues, leave any of
the following bot commands in an otherwise empty comment in this PR:

/retest ci/centos/<job-name>: retest the <job-name> after unrelated
failure (please report the failure too!)

@Mergifyio queue

@mergifyio rebase

@mergifyio queue

/retest ci/centos/mini-e2e-helm/k8s-1.27

/retest ci/centos/mini-e2e-helm/k8s-1.27

Unrelated(?) failure:
  Nov  9 12:43:03.487: INFO: Waiting for PV pvc-df79f100-e412-408e-af7a-b8e3c8d443cf to bind to PVC rbd-pvc
  Nov  9 12:43:03.487: INFO: Waiting up to timeout=10m0s for PersistentVolumeClaims [rbd-pvc] to have phase Bound
  Nov  9 12:43:03.489: INFO: PersistentVolumeClaim rbd-pvc found and phase=Bound (1.875492ms)
  Nov  9 12:43:03.489: INFO: Waiting up to 10m0s for PersistentVolume pvc-df79f100-e412-408e-af7a-b8e3c8d443cf to have phase Bound
  Nov  9 12:43:03.491: INFO: PersistentVolume pvc-df79f100-e412-408e-af7a-b8e3c8d443cf found and phase=Bound (1.794611ms)
  W1109 12:43:03.499504   91009 warnings.go:70] would violate PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container "web-server" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "web-server" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "web-server" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container "web-server" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
  Nov  9 12:43:03.499: INFO: Waiting up to csi-rbd-demo-pod to be in Running state
  Nov  9 12:53:03.500: INFO: failed to write data in the pod: context deadline exceeded
  Nov  9 12:53:03.500: INFO: >>> kubeConfig: /root/.kube/config
  [FAILED] in [It] - /go/src/github.com/ceph/ceph-csi/e2e/utils.go:1125 @ 11/09/23 12:53:03.507

logs

@mergifyio requeue

authors: Sea-you nixpanic riya-singhal31


text:
Runit exits with status 127

With the latest master, using fedora 24 images, not able to start osd and it exits with
*** Booting runit daemon...
*** Runit started as PID 20
*** Runit exited with status 127
sh: /usr/bin/sv: No such file or directory
sh: /usr/bin/sv: No such file or directory
*** Killing all processes...

I see fixes #334 and #336, but seems like fedora 24 base image does not seem to have runit included in it.

Indeed, f24 doesn't support directory mode.
If you don't have any strong requirements in terms of distro (some of us do due to their companies affiliations) I'd recommend to go with an Ubuntu image :).
Thanks!

Definitely no strong requirements, but more of the OSD choosing to fix it given that we are supporting that container base :)

Sure ;)

authors: ganeshmaharaj leseb

repo: ceph/ceph-docker


text:
Added task level log files and job tasks html report file

Added task level log files.
job tasks html report file in archive directory.

Signed-off-by: sunilkumarn417 sunnagar@redhat.com

@susebot run deploy

@ryg- this is for rh branch, we will plan to submit when the full testing is completed.

http://pulpito.ceph.redhat.com/sunnagar-2020-09-28_08:07:08-smoke:ceph-volume-lvm-master-distro-basic-clara/.
failed test are due to connection issue, please ignore. but however both passed and failed results are captured.
Tests
Failed - http://magna002.ceph.redhat.com/sunnagar-2020-09-28_08:07:08-smoke:ceph-volume-lvm-master-distro-basic-clara/388925/
Passed - http://magna002.ceph.redhat.com/sunnagar-2020-09-28_08:07:08-smoke:ceph-volume-lvm-master-distro-basic-clara/388926/

authors: rakeshgm ryg- sunilkumarn417

repo: ceph/teuthology

license: MIT


text:
misc: CentOSStream => rpm

lsb_release -is identifies CentOS 8 Stream as CentOSStream.
Teuthology should recognize this as 'rpm'.
Signed-off-by: Marcus Watts mwatts@redhat.com

@susebot run deploy

authors: mdw-at-linuxbox tchaikov


text:
Fix issue with primary paginator being used for included resources

Fixes #1312
All Submissions:

[x] I've checked to ensure there aren't other open Pull Requests for the same update/change.
[ ] I've submitted a ticket for my issue if one did not already exist.
[ ] My submission passes all tests. (Please run the full test suite locally to cut down on noise from travis failures.)
[ ] I've used Github auto-closing keywords in the commit message or the description.
[X] I've added/updated tests for this change.

New Feature Submissions:

[ ] I've submitted an issue that describes this feature, and received the go ahead from the maintainers.
[ ] My submission includes new tests.
[ ] My submission maintains compliance with JSON:API.

Bug fixes and Changes to Core Features:

[ ] I've included an explanation of what the changes do and why I'd like you to include them.
[ ] I've provided test(s) that fails without the change.

Test Plan:
Reviewer Checklist:

[ ] Maintains compliance with JSON:API
[ ] Adequate test coverage exists to prevent regressions

Please approve this request! This bug is super frustrating!

authors: AELSchauer lgebhardt

repo: cerebris/jsonapi-resources


text:
Nightwatch changes for Toolkit 2.0

Summary
Update for the newest changes in terra-toolkit v2.0 release.
Additionally, changes were made for terra-base v2.0 release.
@cerner/terra

Resolves https://github.com/cerner/generator-terra-module/issues/68

repo: cerner/generator-terra-module


text:
CA cert in Secret not updated when self-signed CA itself gets renewed.

Describe the bug:
When using a self-signed issuer to managed an internal CA, when the CA itself gets renewed, none of the Certificates issued via the CA receive the updated CA cert, so once it expires all of the services using it fail to connect due to certificate expiration.
Expected behaviour:
When a CA managed by Cert-Manager is renewed, all Certificates issued by that CA should have their Secret updated with the new CA cert.
Steps to reproduce the bug:
Apply the following to a cluster
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: selfsigned
spec:
  selfSigned: {}
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: selfsigned-ca
spec:
  isCA: true
  commonName: selfsigned-ca
  secretName: selfsigned-ca
  # Shortest time allowed
  duration: "1h"
  privateKey:
    algorithm: ECDSA
    size: 256
  issuerRef:
    name: selfsigned
    kind: Issuer
    group: cert-manager.io
---
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: selfsigned-ca
spec:
  ca:
    secretName: selfsigned-ca
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: selfsigned-client
spec:
  secretName: selfsigned-client
  commonName: selfsigned-client
  issuerRef:
    name: selfsigned-ca
    kind: Issuer
    group: cert-manager.io

Note the CA fingerprint matches from the CA and client cert:
$ kubectl get secret selfsigned-ca -o jsonpath='{.data.ca\.crt}' | base64 -d | openssl x509 -fingerprint -noout
SHA1 Fingerprint=7C:53:15:A9:EE:75:20:43:88:5A:5F:8C:AE:53:C0:D1:2A:77:77:A3

$ kubectl get secret selfsigned-client -o jsonpath='{.data.ca\.crt}' | base64 -d | openssl x509 -fingerprint -noout
SHA1 Fingerprint=7C:53:15:A9:EE:75:20:43:88:5A:5F:8C:AE:53:C0:D1:2A:77:77:A3

Wait an hour for the CA certificate to be renewed...
Then check again:
$ kubectl get secret selfsigned-ca -o jsonpath='{.data.ca\.crt}' | base64 -d | openssl x509 -fingerprint -noout
SHA1 Fingerprint=B2:48:49:D4:CC:45:F5:46:BF:B9:7D:AB:71:2C:2E:31:7E:7A:FD:59

$ kubectl get secret selfsigned-client -o jsonpath='{.data.ca\.crt}' | base64 -d | openssl x509 -fingerprint -noout
SHA1 Fingerprint=7C:53:15:A9:EE:75:20:43:88:5A:5F:8C:AE:53:C0:D1:2A:77:77:A3

They no longer match, so the certificate bundle in the client Secret is wrong and workloads attempting to use it will see some sort of "certificate expiration" error.
Environment details::

Kubernetes version: v1.25.5
Cloud-provider/provisioner: N/A (bare-metal)
cert-manager version: v1.11.0
Install method: Helm via ArgoCD i.e. helm template ... | kubectl apply -f -

/kind bug

cert-manager cannot safely update already-issued certificates with the "new" CA after the CA is rotated - the certificates would have to be re-issued by the new CA.

Not sure I follow this, but then of course I'm not familiar with the exact workflow of the controller(s).
When the CA Certificate is renewed, the backing Secret would also change I assume. The Issuer controller could/should be watching that Secret for changes and requeue all the Certificates that match its issuerRef for renewal also.
It's not so far from the current auto-reissue of Certificates and I'm not sure where the danger is in this process, but then I'm not a PKI expert so happy to be taught :smiley:
As a temporary workaround (in case others land on this issue), is just to increase the duration of the CA Certificate to something unreasonably long (8760h ~ 1 year), but I think something more robust is needed. At the very least an alterable metric for each Certificate when it's CA bundle is close to expiry.

Not sure I follow this, but then of course I'm not familiar with the exact workflow of the controller(s).

There's a whole lot of "distributed systems" stuff which can go wrong. We can look for all certs which refer to the changed issuer when the issuer is changed, but that process can fail leaving us in a halfway state where some certs won't be updated. Plus, there could be a lot of certificates, and reissuing them all could in the worst case involve generating a lot of large RSA keys which isn't cheap.
Even if we update all the certs successfully, they might not be picked up and used immediately by the pods which are mounting the secrets, adding more complexity.
Obviously I'm talking mainly about edge cases there and we it's reasonable to say that the happy path could still be improved even if all the edge cases aren't solved - but it's an illustration of the difficulties we have.
We also simply don't have that much time for making changes like this ATM. I'd love for cert-manager to be better at this stuff (it's super annoying to me that we don't have support for pathLen yet) but we just don't have that much time. :frowning:

just to increase the duration of the CA Certificate to something unreasonably long (8760h ~ 1 year)

I think this is likely to be a good approach - and scheduling in a calendar event to renew manually in say 9 months. It sounds very old-fashioned and not-cloud-native, but for non-trivial PKI stuff it often ends up with some manual steps (albeit usually because a PKI will have an offline root certificate meaning that rotation of intermediates is necessarily manual)

Thanks for the detailed response. I see a lot of the issues you're listing that to me would be more operational than functional i.e. "If you have a lot of certs managed by cert-manager, that's going to mean a lot of CPU overhead at renewal time" imho is a given.

We can look for all certs which refer to the changed issuer when the issuer is changed, but that process can fail leaving us in a halfway state where some certs won't be updated.

I see this as part of the normal reconciliation loop: every time the Certificate reconciliation is queued, check the issuerRef matches the ca.crt and if not, start a regular renewal with a CertificateRequest. Nothing should be lost half way for long, even with controller restarts.

Plus, there could be a lot of certificates, and reissuing them all could in the worst case involve generating a lot of large RSA keys which isn't cheap.

Understandable, but more an issue for operators. If lots of certs are currently managed by cert-manager the same more-or-less applies, and it's why we have rate-limited queues and CPU scheduling.

Even if we update all the certs successfully, they might not be picked up and used immediately by the pods which are mounting the secrets, adding more complexity.

Totally true and an existing issue solved by the current advice of using a tools such as "wave" if the pod process is incapable of detecting & reloading changed certs.

We also simply don't have that much time for making changes like this ATM.

Would you be open to a collaborative PR? As I mentioned in Slack, I work in Open Source too and luckily my company is very much open to upstream contributions. If the changes as I understand them are possible and could be accepted I could see it being a valuable investment of my time :smile:
I don't mean to sound argumentative, just giving an outsiders perspective and making sure I fully understand the issues at hand :smiley:

I'm very open to collaboration and I'd gladly review a PR, but I'm also about to go on holiday for a while and realistically I won't personally be able to get back to doing anything until at least 2023-03-22 :frowning:  After that, though, I'll be working a bunch on my Kubecon talk which is related to this use case so I should be able to help a bit then!
@maelvls is currently working on open source stuff and @irbekrm is too (although Irbe is currently on holiday), so they might be able to point you in the right direction if you have questions.
I'd suggest maybe starting with a design doc PR so we can agree on what changes we might want to make. Maybe also start a discussion in #cert-manager-dev on Kubernetes slack.
I hope that's helpful! I'll check back in when I'm back - and thank you for showing an interest, I love this part of open source :grin:

Awesome, thanks for the info.

One implementation could be adding another check to the list of policies that the trigger controller checks before determining whether a cert needs to be re-issued. This check could just check that if the Certificate is for CA issuer (assuming then) check the issuer's CA secret and verify the the issued certificate matches the CA. We should then also update this event handler predicate as at the moment the trigger handler only runs on Certificate Secret events- the event handler predicate would be crucial to ensure that both the controller does get triggered when the CA Secret changes and does not get triggered (and run all the policy checks, some of which are expensive) when any random cluster Secret changes.
The above approach has a problem in that it would introduce more coupling between certificates controllers and the CA issuer, which may make any future refactoring difficult and does not fit the model well.
Alternative approach could be to add a new status field (i.e 'revision') that any issuer could implement that would signal to trigger controller that certificates need to be re-issued together with a new annotation for Certificates to signal for which 'revision' the Certificate was last successfully issued. With this, CA issuer could store something like CA cert fingerprint on its status and when it detects that a new CA cert has been stored in the Secret, it could bump the revision. We would add a new check to the trigger controller to trigger new issuance if the revision annotation does not match the revision field on the Certificate. That would allow us to avoid adding any issuer-specific logic to certificates controllers and any issuer, including external issuers, could implement their own logic for what would be considered a new 'revision' for them.
I've not thought about this in detail, but perhaps this approach would work. The question of whether something like this can be safely switched on in GA for all CA issuers still stands (i.e whether everyone wants this feature). Perhaps the CA issuer could have some new fields to allow users to specify if the revision needs bumping or not (perhaps too hacky). In theory, the issuer could have all kinds of logic, i.e to allow users to specify how long before expiry of CA cert all certs need to be renewed, to add some skew to avoid thundering herd of renewals etc, but that would be lots of extra complexity.
I'd be interested to see someone explore this approach in a design doc.
The third alternative would be to have some external component that applies Issuing condition when it detects that the CA certificate has been updated. That would need to either somehow store the fingerprint of the old certificate or have other means to distinguish between Secret updates because the CA certificate changed from other updates (i.e user added a new label).
The benefit of this is that the approach could be tested out and refined before adding the code in-tree.

I feel like the trust manager is the solution to this problem - updating just the ca.crt in previously issued client secrets would mean the chain is broken in that secret. Reissuing all the certs could take a long time and cause a thundering herd style scenario. You really need a way to distribute both the old CA cert and the new CA cert until such time as all certs have been refreshed with the new CA.

/remove-lifecycle stale

Just linking similar or relevant issues:

https://github.com/cert-manager/trust-manager/issues/144
https://github.com/cert-manager/cert-manager/issues/6224
https://github.com/cert-manager/trust-manager/issues/10

The advantage of not using trust-manager is that one can simply use csi driver without messing with additional mounts.

The advantage of not using trust-manager is that one can simply use csi driver without messing with additional mounts.

I understand that the UX of that is better, but it's really not safe to use ca.crt for trust purposes.
When a root X is rotated, you need tp trust both the old and new roots for a period, until everything has been updated to use the new chain. It's not possible to do that with ca.crt.

@SgtCoDFish

It's not possible to do that with ca.crt.

Why it is not possible? (What would need to change in cert-manager?)

Why it is not possible? (What would need to change in cert-manager?)

cert-manager would need to track every existing cert that's still valid and that could've been trusted, and then put all those certs in ca.crt - like an inventory of past valid CA certificates.
But it can't know which of those past certs should actually be trusted - you might have manually rotated one because it was exposed for example - so we can't really just track everything and then throw it in ca.crt!
trust-manager is a little bit more manual, because you'd have to track those roots explicitly and add them as sources in trust-manager - but it's safe in part because you have complete control. cert-manager can only guess!

@SgtCoDFish

But it can't know which of those past certs should actually be trusted - you might have manually rotated one because it was exposed for example

All valid certificates for a private key that signed the current tls.crt should be trusted. If a private key of the CA was exposed, so you manually rotated the certificate, the old private key will not match the new signed tls.crt, it will only match the new private key of the new CA.

Or am I wrong? (I do not understand why "cert-manager can only guess"?)

I think of it like this:
Imagine we have two different services, which we'll call X and Y. Assume X and Y need to talk to each other using TLS and both use ca.crt for trust purposes.
They both get their certs from an issuer we'll call A. Since they both are issued from A, they both trust A since that's what's in ca.crt.
Imagine that A is about to expire so we rotate it to B. We rotate X's certificate to be issued from B, but that means it no longer trusts Y's certificate which is still issued from A. Also, Y doesn't trust B until it has been rotated. This is the problem we have today - there's always downtime since we can't guarantee that both X and Y have their certs reissued immediately.
We could change it so that the issuer keeps track of all certs it uses, and puts all of them in ca.crt. Now we rotate X's certificate and it trusts Y's cert, but we have a problem because Y still doesn't know about B until its cert is rotated at least once.
Instead, we could change it so that we "pre-issue" B, so it appears in ca.crt but doesn't get used until we choose to use it. So we do that, then issue a new cert for X and Y using A, so their ca.crt now contains both A and B. Then we re-issue X's cert using B, and Y will trust it. No downtime, great. At this point though I think we've gone to more effort than it would've been to just use trust-manager, not to mention the many other benefits we'd get from trust-manager in this situation.
Going further, if we then realise we accidentally uploaded the private key for B on github somehow, we need to distrust B and rotate again to C.
So we need to add some way for cert-manager to know that ca.crt should contain A and C but not B... at this point, we've reinvented trust-manager but worse and harder to use, and this ca.crt logic needs to be re-implemented in an issuer-specific way for every issuer type.
Does that make sense? Sorry it's so long!

@SgtCoDFish First, thank you for your time you spent discussing the solutions.

... we've reinvented trust-manager but worse and harder to use ...

I think trust-manager does not need to be reinvented, but rather optionally integrated with cert-manager so that its bundle can also be used via CSI driver.

... and this ca.crt logic needs to be re-implemented in an issuer-specific way for every issuer type.

What would have to be done to make it possible to override ca.crt globally?

I think trust-manager does not need to be reinvented, but rather optionally integrated with cert-manager so that its bundle can also be used via CSI driver

I'm not sure I follow; could you give me an example? If a user is able to mount via CSI driver, why would that user not be able to mount a trust-manager-generated ConfigMap?

What would have to be done to make it possible to override ca.crt globally?

To use ca.crt safely, every issuer has to provide the CA which should be trusted for its chain (which isn't necc. simple), plus a history of past CAs which were used and which are still valid. We'd also at minimum need the ability to filter certain CAs out of whatever is returned.

As someone who just encountered this problem myself, I am still unable to grasp how (or where) using trust-manager would solve this issue. I am trying to sum up the discussion so far, to try to make sense of what has been written so far (also - rubber duck mode).
There are two problems at hand (note I use the names from the initial post for clarity):

Given that a certificate (selfsigned-client) can outlive its CA (selfsigned-ca), we end up with a ca.crt in the certificate (selfsigned-client), which is expired, and thus can't be used at all.
Directly using the CA certificate via selfsigned-ca is not possible either, since when the CA key rotates, the CA certificate gets updated, but that new certificate can't be used to verify the existing client certificate (selfsigned-client).

A tempting solution is to use both the CA certificate directly via selfsigned-ca, and ca.crt from selfsigned-client. This falls flat however, since we still have problem 1, as the certificate selfsigned-client is not re-generated when the CA expires (thus we end up with an expired ca.crt and a CA certificate that can't verify the client certificate - double fail!).
So from my point of view, the core issue resolves around the question "when does cert-manager need to re-generate selfsigned-client"? I would argue "somewhen before the old CA expires, and after the new one has been generated (rotation).

/remove-lifecycle stale

I feel like the trust manager is the solution to this problem - updating just the ca.crt in previously issued client secrets would mean the chain is broken in that secret. Reissuing all the certs could take a long time and cause a thundering herd style scenario. You really need a way to distribute both the old CA cert and the new CA cert until such time as all certs have been refreshed with the new CA.

Just gave this a try and it seems like trust-manager does help with this.
Here's the setup I tried
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: selfsigned-issuer
spec:
  selfSigned: {}
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: selfsigned-ca-short
  namespace: my-trust-namespace
spec:
  isCA: true
  commonName: selfsigned-ca-short
  secretName: ca-short
  duration: 1h
  privateKey:
    algorithm: ECDSA
    size: 256
  issuerRef:
    name: selfsigned-issuer
    kind: ClusterIssuer
    group: cert-manager.io
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: ca-issuer-short
spec:
  ca:
    secretName: ca-short
---
apiVersion: trust.cert-manager.io/v1alpha1
kind: Bundle
metadata:
  name: bundle
spec:
  sources:
  - secret:
      name: ca-short
      key: tls.crt
  target:
    configMap:
      key: ca.crt

I first wanted to test the "month 8 case" as mentioned in this comment from a related issue: https://github.com/cert-manager/cert-manager/issues/2478#issuecomment-1822608480
For my test case since the CA expires each hour, the valid window is from the 40th to 60th minute. I also had a MQTT broker service running on an issued leaf cert from this CA with expiry of 30 days.
Around the 40th minute, my observations:

cert-manager successfully renews the CA and trust-manager picks it up and distributes the new trust bundle correctly.

The leaf certificate isn't renewed, with Kubernetes reporting that its last transition time was when it was issued for the first time. Given the OP of the issue, this is expected.

I create an MQTT client pod with mosquitto client and mounted the renewed trust bundle as a file then passed it in via --ca-file. I try sending a message to the MQTT broker, and it works!
$ mosquitto_pub -h my-broker -p 8884 -m "hello" -t "source" -d --cafile /var/run/certs/ca.crt -D CONNECT authentication-method 'K8S-SAT' -D CONNECT authentication-data $(cat /var/run/secrets/tokens/mq-sat)
Client null sending CONNECT
Client $server-generated/393a34cb-f8a3-4b69-9a9f-4bc666176da9 received CONNACK (0)
Client $server-generated/393a34cb-f8a3-4b69-9a9f-4bc666176da9 sending PUBLISH (d0, q0, r0, m1, 'source', ... (5 bytes))
Client $server-generated/393a34cb-f8a3-4b69-9a9f-4bc666176da9 sending DISCONNECT

I use openssl and can verify the leaf cert chain validity with both the old CA and new CA
$ openssl verify -verbose -CAfile new-ca.crt broker-leaf.crt
broker-leaf.crt: OK
$ openssl verify -verbose -CAfile old-ca.crt broker-leaf.crt
broker-leaf.crt: OK

I think this works because, by default, cert-manager doesn't rotate the private key when renewing a cert, CA or not. Combine it with what @Jamstah said in this comment https://github.com/cert-manager/cert-manager/issues/5864#issuecomment-1553089806

...leaf certs are signed by the CA key and not by the CA cert. The same CA key can be used to create a new CA cert with new validity dates that would be able to be used in the chain with the leaf cert and validate it successfully, even though the original CA cert has expired. There's a good description here: https://serverfault.com/questions/878919/what-happens-to-code-sign-certificates-when-when-root-ca-expires

So in my case since the private key stayed the same, the leaf cert issued from the old CA can be cryptographically chained to the new CA even though the public key is different. I think this also means that as long as the client gets the newest trust bundle, the leaf cert can be used even though it way outlives the original CA (30 days vs 1 hour).
Lastly, after 60 minutes, the original CA expired, but I can still send messages and pass openssl verification as long as I use the renewed trust bundle. If I use the old trust bundle, I get an error from mosquitto saying OpenSSL Error[0]: error:0A000086:SSL routines::certificate verify failed and openssl verify fails with error 10 at 1 depth lookup: certificate has expired / error broker-leaf.crt: verification failed. Shows how important trust manager is.

/reopen

I encountered the same issue and resolved it by extending the duration of the self-signed CA, ensuring that the renewBefore period is longer than the duration of the issued certificate. This guarantees that the signed certificate always has a valid (non-expired) CA certificate.
For example:

Self-signed CA: duration of 6 months, with renewBefore set to 3 months
Issued Certificate: duration of 3 months, with renewBefore set to 1 day

authors: Jamstah NoodlesWang2024 SgtCoDFish TimJones ebuildy irbekrm jlian kernle32dll n9

repo: cert-manager/cert-manager


text:
Consider exposing previous certificates/keys in the kubernetes secret so that workloads can implement a grace period when a certificate rotates

Hi All,
Love your work on cert-manager. I have a certificate rotation scenario that I don't believe is currently supported.
Is your feature request related to a problem? Please describe.
We use cert-manager certificates to sign JWT access tokens. Currently we must set the private key rotation policy to Never, because there is no way to retain previous certificates/keys (that may still be valid).
Ideally we would like to be able to rotate certificates and keys regularly, without imposing a hard cut-over on the holders of existing access tokens. Implementing this would require us to pass multiple versions of a certificate/key when writing the kubernetes secret.
Describe the solution you'd like
One solution that could work:

In the spec of Certificate, add an additional optional timespan setting "retainRotated", that could default to "0d". This would define a time window between now and "retainRotated ago".
cert-manager would update the destination secret to provide all versions of the certificate/key that are valid during the time window
For backwards compatibility, cert-manager could continue to set tls.key, tls.crt, ca.crt, tls-combined.pem, key.der. for the most recent certificate.
In addition, it could set prior_tls_{n}.key, prior_tls_{n}.crt, prior_ca_{n}.crt, prior_tls-combined_{n}.pem, prior_key_{n}.der (or a similar naming scheme) for the n previous certificate versions that are valid during the window.

/kind feature

@nickwb could you provide some information explaining what your PKI looks like? Why do you need the certificate history when all your certificates can share a common parent certificate?

Hi @inteon,
Thanks for your interest.
As mentioned previously, we use these certificates for signing access tokens - specifically, JWT access tokens.
We do run cert-manager with a CA issuer, and our signing certificates are leaves of that CA. But our JWTs do not contain the entire signing certificate - this would make them unacceptably long, so there is no way to extract the public key of the signing certificate (even if we could hypothetically validate its trust chain).
Instead, the access token contains a kid parameter (as per https://www.rfc-editor.org/rfc/rfc7515#section-4.1.4) that contains an identifier for the certificate (I think it's either the X509 serial number or the X509 subject key identifier). If we had access to the historical certificates, we could then select the correct signing certificate in order to perform the verification procedure.
Certainly, I am open to suggestions if you think there is a mistake in how we have chosen to configure our PKI.

I'd love this functionality for the exact same reason.  @nickwb did you ever find another service with this feature or a way to support this workflow?

/remove-lifecycle rotten

Hi @stusmall,
(Sorry, I seem to have missed the notification for this).
Sadly no, our certificates are still set to Never rotate. We haven't attempted to evaluate any further options at this stage.
Let me know if you discover anything.
Cheers,
Nick

authors: inteon nickwb stusmall


text:
Revert in-tree ParseDN function now that upstream ParseDN function has been fixed

Branched from https://github.com/cert-manager/cert-manager/pull/6775
In https://github.com/cert-manager/cert-manager/pull/6761 we added our own literalSubject parser which follows RFC 4514 and RFC 2253.
Reverts https://github.com/cert-manager/cert-manager/pull/6761.
Since then, the ParseDN implementation in https://github.com/go-ldap/ldap and the https://github.com/go-asn1-ber/asn1-ber library were improved (see https://github.com/go-ldap/ldap/pull/497, https://github.com/go-ldap/ldap/pull/505, https://github.com/go-ldap/ldap/pull/504 and https://github.com/go-asn1-ber/asn1-ber/pull/42).
This PR removes our own ParseDN function in favor of this updated upstream parser.
Kind
/kind cleanup
Release Note
NONE

Branched from
// https://github.com/cert-manager/cert-manager/pull/6775
In https://github.com/cert-manager/cert-manager/pull/6761 we added our own literalSubject parser which follows RFC 4514 and RFC 2253.
Reverts https://github.com/cert-manager/cert-manager/pull/6761.
Since then, the ParseDN implementation in https://github.com/go-ldap/ldap and the https://github.com/go-asn1-ber/asn1-ber library were improved (see https://github.com/go-ldap/ldap/pull/497, https://github.com/go-ldap/ldap/pull/505, https://github.com/go-ldap/ldap/pull/504 and https://github.com/go-asn1-ber/asn1-ber/pull/42).
This PR removes our own ParseDN function in favor of this updated upstream parser.
Kind
/kind cleanup
Release Note
NONE

/retest

/retest

authors: Osamaabdalsabour inteon


text:
Add timing test for Check and Sign

This PR adds a test that makes sure Check and Sign are called after an expected backoff duration in case of an error.
By testing this behavior, we gain confidence in the error-retry capabilities of this library:

we want to make sure that failures are being retried
we want to make sure that there are no tight-loops that cause the functions to be called at a super high frequency

Hey. Does this PR still need a review?

/approve

/retest

/retest

authors: inteon maelvls

repo: cert-manager/issuer-lib


text:
Update the 1.4 release docs

Merge master into the release-1.4 branch so that the 1.4 docs are up to date after we release 1.5

/override dco

/lgtm

/approve

authors: irbekrm wallrj

repo: cert-manager/website


text:
Refactoring docker

Hey,
New PR canceling the use of the ubuntu image #45 , we stay on alpine.

I optimized the docker, I added the concept of context to easily switch to the version (7.2, ...)
I added a makefile file to easily handle the container
I updated the documentation

Regards

I have have an issue when building the docker image:

Nice catch, thanks @mickaelandrieu

It works on windows, on ubuntu 16.04 and probably on Mac OSX (don't have this shitty OS at home :p)
Thank you sir!

repo: certificationy/certificationy-cli


text:
Changed class type to quoted string string

The answer  "$builder->add('extra', hidden, ['mapped' => false])" makes a confusion, it should be quoted to look as a correct answer.

:+1:  you're right
Thank you for your contribution @a-ast
Mickaël

authors: a-ast mickaelandrieu

repo: certificationy/symfony-pack


text:
[BUG] cs_wallet locks not held when calling 'masternode outputs'

When giving the cli command masternode outputs, the client crashes due to locks on cs_wallet not being held.
To Reproduce

Compile a wallet with --enable-debug
Ensure you should have 20k masternode blocks in the wallet
With the cli, call masternode outputs
Result:

$ ion-cli -testnet=1 masternode outputs
Assertion failed: lock cs_wallet not held in wallet.cpp:3943; locks held:
error: couldn't connect to server

Desktop (please complete the following information):

OS: Ubuntu 18.04
Compiled from cevap/ion master commit a5d4c61ee7003f3b2e7338ca829d6167b86ee6d1
Version [e.g. 22]

Additional context
The following locks are asserted but not set in SelectCoinsMasternode():
void CWallet::LockCoin(const COutPoint& output)
{
    AssertLockHeld(cs_wallet); // setLockedCoins
    setLockedCoins.insert(output);
}

void CWallet::UnlockCoin(const COutPoint& output)
{
    AssertLockHeld(cs_wallet); // setLockedCoins
    setLockedCoins.erase(output);
}

I cant reproduce this bug, ubuntu bionic 18.04. Masternode outputs work as it is supposed to be used. I made the test only on main as testnet is unlaunched/noCoins.
Please test release binaries, I will compile in the evening master with "--enable-debug", to see if I can reproduce this bug

2 downloads, one of them my, the other from @ckti , please test it https://github.com/cevap/ion/issues/125 and other binaries too, like I said, I cant reproduce it for now with release binaries.

I tried with those gitian compiled binaries and that does not produce that error - which makes sense, because they are not compiled in debug mode. The assertion is only made in debug mode.
However, the issue remains: either the assertion isn't necessary, or the lock needs to be set before the assertion is made.

I close this issue,  #127  , open please if not resolved

authors: FornaxA cevap

repo: cevap/ion


text:
if/else each on one line not supported

if (true) { print(1); }
else { print(2); }

The formatter turns this into
if (true) { print(1); } else { print(2); }

because an else normally goes on the same line as the preceding }, under the assumption that this } is already on its own line. Single-line ifs violate that assumption.
Which solution would be preferred here?

Don’t allow single-line ifs with an else: turn this intoif (true) {
    print(1);
} else {
    print(2);
}

Tricky because we need to propagate this information into visitBlock.
Detect the “single-line if” case at the else, and preserve the code unchanged.
Also tricky because we need to propagate this information from visitIfClause to visitElseClause.

Thoughts?

If we decide to not allow single-line ifs with an else (first solution), then the implementation would probably be a Category of Blocks that are not allowed to be single-line (FormattingVisitor member), which visitIf adds blocks to and visitBlock consults. And in that case, I think I’d also like to add method blocks (and, if necessary, class/interface bodies) to that category as well.

Personally I'm a proponent of the first solution. I never write single line ifs in Java for example, I always use blocks even for the tiniest of statements, I find the visual regularity more readable myself and prefer it over making the code shorter.
But I admit it's completely personal, I can perfectly understand people seeing things differently.

Well, those people still have to live with the fact that we didn’t make braces optional in Ceylon ;)
I think I prefer the first solution as well. I wrote the original code in the issue, but still, it looks weird to me now.

@lucaswerkmeister I think whatever you do, it should be consistent with how you handle switch statements and try statements with catch clauses.

Well, currently, this is allowed:
try { print("hi"); } catch (Exception e) { e.printStackTrace(); } finally { print("done"); }

So I’d say those are some more candidates for blocks that aren’t allowed to have a single-line form.
It might actually make more sense to turn this around and say that only these kinds of blocks are allowed to be single-line (if they contain at most one statement):

if with no else
case
dynamic, probably
setters? not sure

Anything else? I looked for Block in the spec, and these are the only productions where single-line blocks make sense IMO. (Some constructs, like functions and getters, should instead use => in these cases; others should never me single-line IMO.)

authors: gavinking lucaswerkmeister quintesse


text:
Ceylon CI

Tracking issue for CI (https://ci-ceylon.rhcloud.com) improvements.

[ ] execute SDK tests
[x] build SDK doc
[ ] build IDEs
[x] badges in readme files
[ ] automatic checks of pull requests

[ ] execute distribution tests
[ ] build ceylon.formatter (required for Eclipse IDE, but better as separate step)
[ ] build ceylon.ast (please; required once/if @jvasileff’s Dart backend becomes official and/or gets CI)
[ ] build nightly distribution (zip, rpm, deb; try-beta too?)

@lucaswerkmeister that last point is already being done, right? https://ci-ceylon.rhcloud.com/job/ceylon-distribution/
The RPM and DEB would be a nice extra but I'm not sure how we can properly build both of them on the same machine.
Try-beta, would not be a good idea, we could create a special new site, try-nightly or something like that. But to me those are all separate items with different priorities.

Hm, good point, DEB and RPM are tricky to produce. Very low priority. But, like try-nightly, we can still track it here, right?

Well, I'd prefer to track all this in separate issues to be honest. @thradec was just being lazy ;)

Update: “build IDEs” seems to be done (@davidfestal @bjansen is that correct?), “automatic checks of pull requests” too.

authors: lucaswerkmeister quintesse thradec

repo: ceylon/ceylon


text:
Help installing Cezerin on Heroku

I was able to run the application on my local machine but wanted to push it to heroku but it isn't working at all.
Does anyone know how to install Cezerin on Heroku???
Thanks Much

Wow thanks for your help, I’ll let try it. Does the file upload work on Heroku?

The product images

This is another issue am facing , If I upload a product image on the heroku app the file is not being uploaded/created under path \public\content\images\products*
However if I already uploaded the images in my localhost app and then deployed the code base to heroku, then the images are loaded.

Hi, Thanks, I checked with Heroku documentation https://devcenter.heroku.com/articles/dynos#ephemeral-filesystem, and it seems like generally file not persists in a heroku instance, so I probably need to rely on some cloud storage or store the uploaded image(base64) on the mongoDB itself.

May be it makes sense to give Cloudinary url as a product image?

1. app.json file as per Heroku doc.

Hi, due to cezerin's architecture I could not able to find a way to host in same instance, so I hosted the app in 2 separate Heroku instances. If you already able to run cezerin in localhost then there's nothing much todo to run the same on Heroku except below few steps:
e.g.
1. created 2 instances like

* http://app.herokuapp.com for api

* http://app-client.herokuapp.com for store

1. add app.json file as per Heroku doc.

2. find and replace all

* localhost:3000 to app-client.herokuapp.com for store

* localhost:3001 to app.herokuapp.com for api

1. modify config/server.js as like below:

* apiListenPort: process.env.PORT || 3001,

* storeListenPort: process.env.PORT || 3000,

1. deploy the same cezerin code base to both the instances with below respective start/main script in package.json

* "start": "node dist/store/server/index.js", for store's instance

* "start": "node src/api/server/index.js", for api's instance

1. make sure dist/ and theme/* theme/dist/* are also uploaded properly (i.e.) .gitignore to ignore only node_modules dir

Hope this answer helps

Thanks
Please provide the example code for both API and client

Thank you everything is going well

authors: dr-montasir sivarajng tennysonp tigrankhachikyan

repo: cezerin/cezerin


text:
Modifying experimentalObjectRestSpread because it's reached TC39 stage 4

Modifying experimentalObjectRestSpread because it's reached TC39 stage 4
Changes

Modifying experimentalObjectRestSpread because it's reached TC39 stage 4 and thus is no longer experimental. https://github.com/tc39/proposal-object-rest-spread

Checklist

[ ] PR has an informative and human-readable title
[ ] Changes are limited to a single goal (no scope creep)
[ ] Code can be automatically merged (no conflicts)
[ ] Code follows the standards laid out in the development playbook
[ ] Passes all existing automated tests
[ ] Any change in functionality is tested
[ ] New functions are documented (with a description, list of inputs, and expected output)
[ ] Placeholder code is flagged / future todos are captured in comments
[ ] Visually tested in supported browsers and devices (see checklist below :point_down:)
[ ] Project documentation has been updated (including the "Unreleased" section of the CHANGELOG)
[ ] Reviewers requested with the Reviewers tool :arrow_right:

Testing checklist
Browsers

[ ] Chrome
[ ] Firefox
[ ] Safari
[ ] Internet Explorer 8, 9, 10, and 11
[ ] Edge
[ ] iOS Safari
[ ] Chrome for Android

Accessibility

[ ] Keyboard friendly
[ ] Screen reader friendly

Other

[ ] Is useable without CSS
[ ] Is useable without JS
[ ] Flexible from small to large screens
[ ] No linting errors or warnings
[ ] JavaScript tests are passing

Associated PRs https://github.com/cfpb/development/pull/170 https://github.com/cfpb/cfgov-refresh/pull/3970

authors: anselmbradford sebworks

repo: cfpb/capital-framework


text:
Add email signup to full width text module

On the Buying a House project, we needed a way to make email signups more visible on the journey browse pages (currently the only option is to add them in the sidefoot). This adds the ability to create email signups as insets in the FullWidthText module. (See Hubcap issue 84 for more details)
Additions

Email signup in full width text module
New H3 header & inline disclaimer options in email signup module

Notes

We decided to make the new header/disclaimer options available in the existing email signup module instead of creating a new module for a couple of reasons -- there might be future changes to email signup headers/disclaimers across the board, and it also might be useful in the short term to have the option for inline disclaimers in sidebar signups.
The email signup JS isn't currently added to the page when a signup is created in the full width text module. There's a PR open in wagtail-inventory that should fix this

Testing

Check out branch
python cfgov/manage.py migrate and gulp styles
In admin, create a new browse page & add a full width text module with an email signup followed by a content block
Verify that the signup displays as expected on page and that disclaimer/header option toggles work as expected
Verify that existing sidebar signup modules get the slug header/stacked disclaimer (the defaults for these options should mean that current signups are unchanged visually)

Screenshots
Changes to email signup module in admin

Signup in full width text module with H3 & inline disclaimer

Checklist

[ ] PR has an informative and human-readable title
[ ] Changes are limited to a single goal (no scope creep)
[ ] Code can be automatically merged (no conflicts)
[ ] Code follows the standards laid out in the development playbook
[ ] Passes all existing automated tests
[ ] Any change in functionality is tested
[ ] New functions are documented (with a description, list of inputs, and expected output)
[ ] Placeholder code is flagged / future todos are captured in comments
[ ] Visually tested in supported browsers and devices (see checklist below :point_down:)
[ ] Project documentation has been updated
[ ] Reviewers requested with the Reviewers tool :arrow_right:

Testing checklist
Browsers

[ ] Chrome
[ ] Firefox
[ ] Safari
[ ] Internet Explorer 8, 9, 10, and 11
[ ] Edge
[ ] iOS Safari
[ ] Chrome for Android

Accessibility

[ ] Keyboard friendly
[ ] Screen reader friendly

Other

[ ] Is useable without CSS
[ ] Is useable without JS
[ ] Flexible from small to large screens
[ ] No linting errors or warnings
[ ] JavaScript tests are passing

👍, looks good to me other than the breakpoint issue.

authors: sebworks virginiacc

repo: cfpb/cfgov-refresh


text:
Add block margin to legacy blog/newsroom pages

Content on LegacyBlogPage and LegacyNewsroomPages doesn't currently get rendered with our standard block spacing. For example, observe the lack of spacing between the bottom of the text and the tags on this page:
https://www.consumerfinance.gov/about-us/blog/you-have-the-right-to-someone-in-your-financial-corner/
Compare this to standard BlogPages and NewsroomPages, for example:
https://www.consumerfinance.gov/about-us/blog/long-hold-times-or-excellent-care-tell-us-about-the-customer-service-at-your-bank-or-credit-union/
This commit alters the article.html template (used for all of these page types) so that when the raw HTML content from legacy pages is rendered, it gets rendered with standard block spacing.
How to test this PR
A good production page to observe this on is:
https://www.consumerfinance.gov/about-us/blog/save-the-date-join-us-for-a-community-bank-advisory-council-meeting-in-washington-d-c-20160421/
With this PR, test locally at:
http://localhost:8000/about-us/blog/save-the-date-join-us-for-a-community-bank-advisory-council-meeting-in-washington-d-c-20160421/
Note the improved spacing below the text and observe that the raw HTML is still rendered properly on the page.
Screenshots

Before
After

Checklist

[x] PR has an informative and human-readable title
[x] Changes are limited to a single goal (no scope creep)
[x] Code follows the standards laid out in the CFPB development guidelines

Looks good to me. Thanks, @chosak! 👏

authors: chosak jenn-franklin


text:
Uncaught exception inside IOHttpClientAdapter

Package
dio
Version
5.3.3
Operating-System
Android
Output of flutter doctor -v
[✓] Flutter (Channel stable, 3.13.5, on KDE neon 5.27 6.2.0-33-generic, locale ro_RO.UTF-8)
    • Flutter version 3.13.5 on channel stable at /home/adrian/fvm/versions/stable
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 12fccda598 (3 days ago), 2023-09-19 13:56:11 -0700
    • Engine revision bd986c5ed2
    • Dart version 3.1.2
    • DevTools version 2.25.0

[✓] Android toolchain - develop for Android devices (Android SDK version 33.0.2)
    • Android SDK at /home/adrian/Android/Sdk
    • Platform android-33, build-tools 33.0.2
    • ANDROID_SDK_ROOT = /home/adrian/Android/Sdk
    • Java binary at: /home/adrian/Programe/android-studio/jbr/bin/java
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b829.9-10027231)
    • All Android licenses accepted.

[✓] Chrome - develop for the web
    • Chrome at google-chrome

[✓] Linux toolchain - develop for Linux desktop
    • Ubuntu clang version 14.0.0-1ubuntu1.1
    • cmake version 3.27.4
    • ninja version 1.10.1
    • pkg-config version 0.29.2

[✓] Android Studio (version 2022.3)
    • Android Studio at /home/adrian/Programe/android-studio
    • Flutter plugin version 75.1.2
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b829.9-10027231)

[✓] VS Code (version unknown)
    • VS Code at /snap/code/current
    • Flutter extension version 3.72.0
    ✗ Unable to determine VS Code version.

[✓] Connected device (4 available)
    • Redmi Note 8 Pro (mobile)  • jrhavkvg6xgutcxc • android-arm64  • Android 11 (API 30)
    • sdk gphone x86 64 (mobile) • emulator-5554    • android-x64    • Android 13 (API 33) (emulator)
    • Linux (desktop)            • linux            • linux-x64      • KDE neon 5.27 6.2.0-33-generic
    • Chrome (web)               • chrome           • web-javascript • Google Chrome 117.0.5938.88

[✓] Network resources
    • All expected network resources are available.

Dart Version
3.1.2
Steps to Reproduce

Stop internet access of device.
Make a Dio.get request.

Expected Result

Should not throw a DioException which cannot be catched.

Actual Result
It throws an uncaught exception inside IOHttpClientAdapter.

Don't know why, but it suddenly stopped after posting this issue...

Duplicate of #1869

repo: cfug/dio


text:
I can't use MediaType when I create MultipartFile

Package
dio
Version
5.4.3+1
Operating-System
Android, iOS, Web, MacOS, Linux, Windows
Adapter
Default Dio
Output of flutter doctor -v
!] Flutter (Channel stable, 3.19.0, on macOS 14.4.1 23E224 darwin-arm64, locale pt-BR)
    • Flutter version 3.19.0 on channel stable at /Users/usuario/fvm/versions/3.19.0
    ! Warning: `dart` on your path resolves to /opt/homebrew/Cellar/dart/3.2.0/libexec/bin/dart, which is not inside your current Flutter SDK checkout at /Users/usuario/fvm/versions/3.19.0. Consider adding /Users/usuario/fvm/versions/3.19.0/bin to the front of your path.
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision bae5e49bc2 (3 months ago), 2024-02-13 17:46:18 -0800
    • Engine revision 04817c99c9
    • Dart version 3.3.0
    • DevTools version 2.31.1
    • If those were intentional, you can disregard the above warnings; however it is recommended to use "git" directly to perform update checks and upgrades.

[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
    • Android SDK at /Users/usuario/Library/Android/sdk
    • Platform android-34, build-tools 34.0.0
    • ANDROID_HOME = /Users/usuario/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b829.9-10027231)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 15.3)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 15E204a
    • CocoaPods version 1.15.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2022.3)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b829.9-10027231)

[✓] VS Code (version 1.87.2)
    • VS Code at /Users/usuario/apps/Visual Studio Code.app/Contents
    • Flutter extension version 3.89.20240501

Dart Version
No response
Steps to Reproduce
When using MultipartFile.fromFile does it ask for a contentType parameter that is of type MediaType? but I can't pass the parameter because it is an http_parser parameter but dio does not export the http_parser
Expected Result
Future<FormData?> _convertForFormDataDio(
GenericFormData genericFormData) async {
final FormData formData = FormData();
for (String mediaPath in genericFormData.filePaths) {
  formData.files.add(
    MapEntry<String, MultipartFile>(
      'files',
      await MultipartFile.fromFile(
        mediaPath,
        filename: path.basename(mediaPath),
        contentType: MediaType.parse('.png') ,
        headers: <String, List<String>>{
          'Content-Type': <String>['multipart/form-data'],
        },
      ),
    ),
  );
}

for (String key in genericFormData.fields.keys) {
  formData.fields.add(MapEntry<String, String>(
      key, genericFormData.fields[key].toString()));
}

return formData;

}
Actual Result
Future<FormData?> _convertForFormDataDio(
GenericFormData genericFormData) async {
final FormData formData = FormData();
for (String mediaPath in genericFormData.filePaths) {
  formData.files.add(
    MapEntry<String, MultipartFile>(
      'files',
      await MultipartFile.fromFile(
        mediaPath,
        filename: path.basename(mediaPath),
        contentType: MediaType.parse('.png') ,//Undefined name 'MediaType'. Try correcting the name to one that is defined, or defining the name.
        headers: <String, List<String>>{
          'Content-Type': <String>['multipart/form-data'],
        },
      ),
    ),
  );
}

for (String key in genericFormData.fields.keys) {
  formData.fields.add(MapEntry<String, String>(
      key, genericFormData.fields[key].toString()));
}

return formData;

}

See https://github.com/cfug/dio/pull/2205 for previous discussion.
@cfug/devs Do we want to revisit this decision?

authors: MateusLucasDaSilva kuhnroyal


text:
Fixes wrong alignment of shader binding table entries

// How the hell did this work before?

Will open another pull request with further changes.

repo: cg-tuwien/Auto-Vk


text:
Exception when removing waypoint from LiveMap

If you open a waypoint from LiveMap (NB2102) and then press remove, then you
get the following exeption:
java.lang.IllegalArgumentException: geocode must not be empty
at cgeo.geocaching.CacheCache.removeCacheFromCache(CacheCache.java:42)
at cgeo.geocaching.cgData.removeCaches(cgData.java:3027)
at cgeo.geocaching.cgData.removeCache(cgData.java:3009)
at cgeo.geocaching.cgeoapplication.removeCache(cgeoapplication.java:536)
at cgeo.geocaching.cgeowaypoint$deleteWaypointListener.onClick(cgeowaypoint.java:316)

Fixed with mentioned commit. Remaining issue (no map refresh) tracked in #1156

authors: YraFyra rsudev

repo: cgeo/c-geo-opensource


text:
Road to 1.0.0

Goals

Remove need for Swift index JSON file.
Move Gazelle plugin to its own repository.
Create release 1.0.0.

Tasks

[x] Process SPM Package.swift and Package.resolved in the repository rule.

[x] Stop reading the index file in the repository rules.
[x] Be sure to return the correct repository info so that mod tidy will update the MODULE.bazel file with the correct use_repos declaration.
[x] Remove all of the index files in the examples.
[x] Remove all of the swift_deps START and swift_deps END comments.
[x] Update bzlmod/workspace to use from_package().

[x] Update the Gazelle plugin to not use the Swift index JSON file.
[x] Update //:swift_update_pkgs to just run SPM commands without using Go. Be sure to use the Swift toolchain, not the Swift installed on the system. (Possibly, create a new macro that creates the commands for running Swift package commands.) For now, I removed swift_update_packages. If people would like a way to update Swift packages using Bazel, we will do it as a separate feature.
[x] Fix examples

[x] grpc_package_example
[x] symlink_example
[x] ios_sim

[ ] Move Gazelle plugin to its own repository.

[ ] Create a repository for the Gazelle plugin.
[ ] Publish to BCR.
[ ] Remove Gazelle plugin from rules_swift_package_manager.

[ ] #1079
[ ] Create release 1.0.0.

Notes regarding Gazelle plugin

No more update_repos.
rules_swift_package_manager will define a repository called swift_deps_info. It will contain the list of direct dependencies for the workspace (i.e., those defined in the Package.swift).
The plugin

It will use repo.FindExternalRepo to find the swift_deps_info and read the information written there. (NOTE: We could just read the list of repo names by parsing the MODULE.bazel. However, I think that we want to support non-bzlmod a bit longer.)
It will then read the pkg_info.json file that lives at the root of every Swift package repository.
This will populate the DependencyIndex.

What do we need DependencyIndex for anymore? Aren't product names deterministic now?

What do we need DependencyIndex for anymore? Aren't product names deterministic now?
The Gazelle plugin needs to resolve module names to Bazel labels.

Ahh, so that won't be in this repo in the end? Got it.

For anyone following along at home, this branch has the examples working if you run bazel test //..., no index file required. 🙂 I started updating the Gazelle plugin to work properly in this new world. Here is the basic plan:

Provide a swift_deps_info repository rule that generates a JSON file with the module/product/package information that is needed to resolve modules to Bazel targets.
Update the Gazelle plugin to read this file and load it as the DependencyIndex.
I anticipate keeping the existing API for the DependencyIndex. So, I hope that there won't be any other major changes to the resolve and generate portions of the Gazelle plugin.
Remove the update repos portion of the Gazelle plugin.

Can we add https://github.com/cgrindel/rules_swift_package_manager/issues/1079 to this?

Done.

Just a quick update regarding the v1 release. I have been working on #1079. I have a draft PR (#1217) with everything working except the rules_xcodeproj no-sandbox build. Unfortunately, that fails with duplicate definitions. I see and mostly understand why this is happening. Unfortunately, I am struggling to prevent the duplicate module map files causing the errors.
I put together #1266 which attempts to prevent objc_library from creating its own module map file by specifying the one that rules_swift_package_manager creates. Unfortunately, this causes an error:
In file included from external/rules_swift_package_manager~~swift_deps~swiftpkg_firebase_ios_sdk/FirebaseInstallations/Source/Library/FIRInstallations.m:25:
external/rules_swift_package_manager~~swift_deps~swiftpkg_firebase_ios_sdk/FirebaseCore/Extension/FirebaseCoreInternal.h:15:1: error: use of '@import' when modules are disabled
@import FirebaseCore;
^
1 error generated.

Looking through the actual command-line, I do not see the -fmodules flag. I thought that this was added by setting enable_modules = True.
If anyone has any thoughts or suggestions, please send them my way.

authors: brentleyjones cgrindel


text:
🛑 Personal is down

In e4d7e2b, Personal (https://chadburrus.com) was down:

HTTP code: 521
Response time: 159 ms

Resolved: Personal is back up in 6570f00 after 14 minutes.

repo: chadburrus/upptime


text:
🛑 Personal is down

In 58c69dd, Personal (https://chadburrus.com) was down:

HTTP code: 521
Response time: 170 ms

Resolved: Personal is back up in 2f0df7f after 14 minutes.


text:
🛑 Enchanted Vacations is down

In af4c48e, Enchanted Vacations (https://yourenchantingvacation.com) was down:

HTTP code: 521
Response time: 154 ms

Resolved: Enchanted Vacations is back up in 0efe338 after 11 minutes.


text:
Adam optimizer report: ZeroDivisionError: float division by zero

version: chainer 2.0.2
I use Adam optimizer ,then report error, I found it was caused by this code(fix1==0?):
in adam.py:
@property
    def lr(self):
        fix1 = 1. - math.pow(self.hyperparam.beta1, self.t)
        fix2 = 1. - math.pow(self.hyperparam.beta2, self.t)
        return self.hyperparam.alpha * math.sqrt(fix2) / fix1

error log:
Traceback (most recent call last):
  File "AU_rcnn/train.py", line 237, in <module>
    main()
  File "AU_rcnn/train.py", line 233, in main
    trainer.run()
  File "/root/anaconda3/lib/python3.6/site-packages/chainer/training/trainer.py", line 285, in run
    initializer(self)
  File "/root/anaconda3/lib/python3.6/site-packages/chainer/training/extensions/exponential_shift.py", line 48, in initialize
    self._init = getattr(optimizer, self._attr)
  File "/root/anaconda3/lib/python3.6/site-packages/chainer/optimizers/adam.py", line 121, in lr
    return self.hyperparam.alpha * math.sqrt(fix2) / fix1
ZeroDivisionError: float division by zero

Answer?: https://stackoverflow.com/questions/46035824/adam-optimizer-report-error-in-chainer

authors: sharpstill t-sakai-kure

repo: chainer/chainer


text:
Release Tasks for v8.0.0b4 / v7.6.0

Previous release (v7.5.0): https://github.com/chainer/chainer/issues/8560
Chainer v7.6.0 (no v8.x release)

[x]

CuPy v8.0.0b4 / v7.6.0

[ ] CUB source tree integration https://github.com/cupy/cupy/pull/2584
[x] CUB Test https://github.com/cupy/cupy/pull/2598
[x] Optuna documentation https://github.com/cupy/cupy/issues/3383 PR: https://github.com/cupy/cupy/pull/3397
[x] Optuna test https://github.com/cupy/cupy/issues/3372

Release Plans (subject to change)

v8.0.0b4 / v7.6.0: June 25th
v8.0.0rc1 / v7.7.0: July 30th

Feature freeze

v8.0.0: August 27th

Generally, the code will be freezed 2 days before the release date for QA. All dates are in JST.

Chainer v7.6.0 will be skipped as changes since v7.4.0 are too trivial for the new release.


text:
Add chainerx.testing.integral_dtypes

Used in #6438.

Jenkins, test this please.

Jenkins CI test (for commit 9feb5943028c1f6a3ecf8403e032013e439a6e7a, target branch master) failed with status FAILURE.
(For contributors, please wait until the reviewer confirms the details of the error.)

LGTM.

authors: asi1024 chainer-ci niboshi


text:
Add '--db-echo' option

depends on #190
remove CHAINERUI_ENV=development and add --db-echo option.

merge master branch, please check again 🙇

repo: chainer/chainerui


text:
Consider already selected packages during solve

We ran into a very very slow solve. I can't tell if it was actually slow
or infinite, but it didn't seem to be convering very quickly. I realized
that we essentially recurse the entire graph for every constraint, even
for things we've already solved. This was surprising, because I thought
we handled that already, but it seems like we didn't.

I've added an example "abseil-regression.yaml" that took over a second
to solve (as of this writing) before this change and takes about 1ms
after this change. There's an open PR in wolfi that timed out because
the abseil dep graph got even more gnarly and took several hours, so
there was something (at least) quadratic going on here, oops.

I've deleted a test that was exercising a scenario where packages were
disallowed from providing themselves because there is no code path where
that was actually possible (all invocations had "true" hard-coded).

I also ran this against a several thousand image configs and they all
produced the same results before and after this change, so I am fairly
confident in it.

e2e tests are failing because a lot of our examples use alpine repos, and currently musl on x86_64 is at 1.2.5-r6 whereas musl on aarch64 is at 1.2.5-r5

I've tested this branch and it fixes the issue for me. Thanks! Do you mind referencing https://github.com/chainguard-dev/melange/issues/1651 in the commit log?

authors: dannf jonjohnsonjr

repo: chainguard-dev/apko


text:
diff: rename detector misbehaves with NodeJS chunks

See https://github.com/wolfi-dev/os/pull/16881
I'm not sure what the right way to detect and track renames is for this case. I assume there is some JSON file we can read that provides a clue, but in the worst case, we look at content similarity.
https://github.com/chainguard-dev/bincapz/blob/6bcc45f6950efff82a6122105c7594f1664fe710/pkg/action/diff.go#L138 is where the modified code would need to be inserted.

possibly related: #92

repo: chainguard-dev/bincapz


text:
Provide easier install instructions for all OSes

Our current documentation is mostly aimed at software developers/sysadmins that need to install many dependencies locally or have some knowledge of docker to edit a docker file and docker-compose.
For users who just want to try Transition, it should be possible to provide cookbook instructions, ideally with minimal dependencies installs, to get started with Transition, in Linux, mac and Windows.
For this, docker is probably the best bet, we just need to link to docker installation instructions for each OS, then explain how to run, maybe from the main build in dockerhub? Users ideally should not even need to clone this repo.
We then need to explain how to setup the project-specific volume.

@greenscientist does that make sense?

repo: chairemobilite/transition


text:
Finalize avalanche source

Missing features:

The bounce-averaged operator currently has a simplified pitch-angle distribution, which should be fixed at some point.
Fluid source term which counts the external particles created (those with p>pmax).
Add to equationsystem and provide interface support to de/activate.

I implemented the sign of the REs in the kinetic avalanche source. Now, the pitch of the RP source will have the same sign as j_hot. The operator has also been implemented in the f_re equation.
Closing this.

repo: chalmersplasmatheory/DREAM


text:
migrate to GitHub Actions

Let's plan to migrate to the wonderful GitHub Actions
🙌🏼

I believe this has been done in https://github.com/changesets/changesets/pull/1085

authors: bluwy maddhruv

repo: changesets/changesets