text:
Support operating on sparse-checkout of cdnjs

Cloning the cdnjs repository is a very expensive operation, as the repository is very big (11GB (compressed) at the time of writing, 100GB+ checked out) and contains a lot of files. Often we only need the package.json files and the directory tree, but not the actual file content of all the stored versions. If we ever want to switch to ex: GitHub Actions (https://github.com/cdnjs/bot-ansible/issues/11) we need this.
Combing partial clone and sparse checkout is much faster:
$ time git clone --filter=blob:none --sparse git@github.com:cdnjs/cdnjs.git
[...]
real	1m18,731s
$ cd cdnjs
$ time git sparse-checkout add '**/package.json'
[..]
real	0m24,713s
$ find | head -n 10
.
./ajax
./ajax/libs
./ajax/libs/zxcvbn
./ajax/libs/zxcvbn/package.json
./ajax/libs/zurb-ink
./ajax/libs/zurb-ink/package.json
./ajax/libs/zumper-angular-payments
./ajax/libs/zumper-angular-payments/package.json
$ du -hs
1,1G

Looking at the code, we need to change this code, so it also checks the tree (git ls-tree) (something similar to #67):
https://github.com/cdnjs/tools/blob/1928a3cca55a4f64c34fbbefabde74e90c57c52d/cmd/autoupdate/git.go#L113-L116
This code need to run git sparse-checkout add <PATH> first (if core.sparseCheckout is true):
https://github.com/cdnjs/tools/blob/1928a3cca55a4f64c34fbbefabde74e90c57c52d/packages/packages.go#L96-L98
This code need to be changed, so it doesn't get the existing versions from the filesystem but from the tree (git ls-tree):
https://github.com/cdnjs/tools/blob/1928a3cca55a4f64c34fbbefabde74e90c57c52d/packages/git.go#L15-L18
^ @xtuc was it implemented this way on purpose? (https://github.com/cdnjs/tools/commit/613817277c9dcbd184c135050d73fc88e0973213#diff-2538e4c3ee0b09db855e7569a1865cc2R16) Why do it stat the filesystem, wouldn't it be easier just to rely on git ls-tree?
/refs https://github.com/cdnjs/bot-ansible/issues/11

The bot wasn't designed by me/us, I rewrote it.
To be clear, we are not planning to optimize git because we are working to move away from it eventually.

To be clear, we are not planning to optimize git because we are working to move away from it eventually.

Fair enough :)

authors: klausenbusk xtuc

repo: cdnjs/tools


text:
Chrome App Window does not work

as this states, I am not too sure of this is my fork, but something has caused it to stop opening a chrome app window.
It used to do it just fine in the current repo state, but now it's not, I believe it may be possible that chrome has changed its flags

This does not appear to be the case, I ran chrome by hand with the flags needed and it opened just fine as it should,

This is only related to your fork so I'm closing this issue. Please track it in your PR comments or on your fork.

source: gharchive/issue

authors: Merith-TK deansheather

repo: cdr/sshcode

license: MIT

license_type: permissive

license_source: github-api


text:
Add ability to upload local binary

Adds --code-server-path flag that accepts a path to a file on the local machine. It will be uploaded to the code-server path on the remote server instead of the download script being executed.
There is no verification of the local binary, so a user could upload something that isn't a code-server binary or a custom binary or something else along those lines.
This allows remote servers with no internet access to have a code-server binary uploaded to them, which is useful for extremely secure environments.
Closes #73.

Just tested this with code-server v2 preview 9, and it just works.

How about --upload-code-server

The flag is now --upload-code-server.

source: gharchive/pull-request

authors: ammario deansheather


text:
🛑 tanecni-divadlo.cz is down

In 9a78b0b, tanecni-divadlo.cz (https://www.tanecni-divadlo.cz) was down:

HTTP code: 0
Response time: 0 ms

Resolved: tanecni-divadlo.cz is back up in ded2673.

repo: cebreus/upptime


text:
🛑 tanecni-divadlo.cz is down

In ccc0a7f, tanecni-divadlo.cz (https://www.tanecni-divadlo.cz) was down:

HTTP code: 0
Response time: 0 ms

Resolved: tanecni-divadlo.cz is back up in 9a3718c.


text:
🛑 Sébastien Thomas Bagnoud is down

In 5af4249, Sébastien Thomas Bagnoud (https://www.sebastienbagnoud.eu/) was down:

HTTP code: 403
Response time: 156 ms

Resolved: Sébastien Thomas Bagnoud is back up in dc577c3.


text:
🛑 tanecni-divadlo.cz is down

In 6a9e958, tanecni-divadlo.cz (https://www.tanecni-divadlo.cz) was down:

HTTP code: 0
Response time: 0 ms

Resolved: tanecni-divadlo.cz is back up in 5fdb288.


text:
🛑 tanecni-divadlo.cz is down

In 550eb29, tanecni-divadlo.cz (https://www.tanecni-divadlo.cz) was down:

HTTP code: 0
Response time: 0 ms

Resolved: tanecni-divadlo.cz is back up in 2daead5.


text:
🛑 CDN is down

In eaa8eef, CDN (https://cdn.ceccun.com/cdynamic/captive) was down:

HTTP code: 522
Response time: 15669 ms

Resolved: CDN is back up in ba48f73.

repo: ceccun/status


text:
🛑 CDN is down

In 5bb2e16, CDN (https://cdn.ceccun.com/cdynamic/captive) was down:

HTTP code: 522
Response time: 15265 ms

Resolved: CDN is back up in e17df2f.


text:
Create uk.arb.txt

Added Ukranian translate.
May be there are need to use special sign for:
"ae_": "Об'/єднані Арабські Емірати",
"ci_": "Кот-д'/івуар",
"mm_": "М'/янма",
"pm_": "Сен-П'/єр і Мікелон",
"tz_": "Танзанія, Об'/єднана Республіка Танзанія",
"vn_": "В'/єтнам",

why is this txt ?

updated

authors: Polyariz cedvdb


text:
Upgrade to Bootstrap 5.2.0

No real change on the JavaScript side... but it is an opportunity to provide an extern for the specific events of the carousel component.

Fixed in version 2.0.0.


text:
study openAPI spec and write specs for Discord API endpoints

We need to generate C functions for all Discord API endpoints by specs.  This work can be ported to support other rest APIs.

This task is to convert only endpoints to Open API specs.  We keep use our specs for data types.

I'll start working on it.

I propose using openapi-generator, we can wrap Orca to our own template written with Mustache and openapi-generator will do the rest for us. @mlite

We have no need for this in Discord API, as most of the endpoints have been covered already. This might make sense for other APIs that provide a OpenAPI specs such as Slack or GitHub

authors: LucasMull mlite

repo: cee-studio/orca


text:
testing(api): Adding API equivalency test for celestia-openrpc

Adds a test to ensure API equivalence to celestia-openrpc. It will not be used or added to CI in this repo, hence the build tag. celestia-openrpc will pull this repo, build with the build tag, and run the test itself.
The reason it shouldnt be added to the CI in celestia-node is because it would fail every time we change the API. It shouldn't fail here, it should fail in the other repo.

I like the name conformance a lot more, I will change the build tag - but I would maybe hold off on moving the test until we have more conformance test cases? WDYT

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 44.75%. Comparing base (2469e7a) to head (3876c1b).
Report is 72 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3405      +/-   ##
==========================================
- Coverage   44.83%   44.75%   -0.08%
==========================================
  Files         265      272       +7
  Lines       14620    15278     +658
==========================================
+ Hits         6555     6838     +283
- Misses       7313     7652     +339
- Partials      752      788      +36

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

I like the idea to have this test. But why test for external client implementation is in node repo?  It seems wrong that node will depend on that client after this PR

yeah actually i agree with @walldiss, this should live in the celestia-openrpc repo and run as a conformance test there

It can't, that's why its here :( I agree ofc but openrpc can't import node because thats the whole reason it exists

@distractedm1nd make a conformance test repo and put it there.

authors: codecov-commenter distractedm1nd ramin renaynay walldiss

repo: celestiaorg/celestia-node


text:
refactor!: Subcriber: AddValidator -> SetVerifier

Verifier is the more correct term to use for stateful verification
Set because in fact we currently allow only one verification func
Remove dependency pubsub to support returning header.VerifyError

Codecov Report

Merging #88 (8d94d8b) into main (6cf3094) will decrease coverage by 0.28%.
The diff coverage is 62.50%.

@@            Coverage Diff             @@
##             main      #88      +/-   ##
==========================================
- Coverage   67.36%   67.08%   -0.28%
==========================================
  Files          35       35
  Lines        2828     2844      +16
==========================================
+ Hits         1905     1908       +3
- Misses        771      783      +12
- Partials      152      153       +1

Files Changed
Coverage Δ

errors.go
0.00% <0.00%> (ø)

interface.go
0.00% <ø> (ø)

p2p/subscriber.go
47.16% <50.00%> (-1.62%)
:arrow_down:

sync/sync_head.go
63.28% <78.57%> (-0.22%)
:arrow_down:

headertest/subscriber.go
18.18% <100.00%> (ø)

sync/sync.go
70.28% <100.00%> (ø)

... and 1 file with indirect coverage changes

authors: Wondertan codecov-commenter

repo: celestiaorg/go-header


text:
Add MintCeloScheduleWrapper

Description
This PR adds MintCeloScheduleWrapper
Other changes
There are some changes mostly related to the test setup:

time-sensitive tests should be now deterministic (due to a fixed timestamp)
in contractkit there's a new utility method to activate MintCeloSchedule
due to the devchain update some data changed and snapshots had to be updated as well

Apart from this GoldTokenWrapper ABI is now taken from @celo/abis-12 because it's needed for the MintCeloSchedule setup.
Tested
Locally and on CI.
Backwards compatibility
It is backwards compatible.

PR-Codex overview
This PR introduces support for the MintGoldSchedule contract in the Celo network.
Detailed summary

Updated blockNumber in info-l2.test.ts
Added MintCeloScheduleWrapper in contractkit
Added MintGoldSchedule to kit.ts
Updated dependencies in package.json
Updated GoldTokenWrapper import in GoldTokenWrapper.ts
Added MintCeloScheduleWrapper to contract-cache.ts
Added MintCeloSchedule functions in test-utils/utils.ts

The following files were skipped due to too many changes: packages/sdk/contractkit/src/wrappers/MintCeloScheduleWrapper.test.ts, yarn.lock, packages/sdk/contractkit/src/wrappers/FeeCurrencyDirectoryWrapper.test.ts, packages/sdk/contractkit/src/wrappers/MintCeloScheduleWrapper.ts, docs/sdk/contractkit/classes/wrappers_GoldTokenWrapper.GoldTokenWrapper.md, docs/sdk/contractkit/classes/web3_contract_cache.Web3ContractCache.md, packages/cli/src/commands/network/parameters-l2.test.ts, docs/sdk/contractkit/modules/base.md, packages/sdk/contractkit/src/web3-contract-cache.ts, docs/sdk/contractkit/classes/wrappers_EpochRewards.EpochRewardsWrapper.md, packages/sdk/contractkit/src/test-utils/utils.ts, docs/sdk/contractkit/enums/base.CeloContract.md, docs/sdk/contractkit/classes/kit.ContractKit.md, packages/cli/src/commands/network/__snapshots__/contracts-l2.test.ts.snap, docs/sdk/contractkit/classes/wrappers_MintCeloScheduleWrapper.MintCeloScheduleWrapper.md, docs/sdk/contractkit/classes/contract_cache.WrapperCache.md

✨ Ask PR-Codex anything about this PR by commenting with /codex {your question}

thinking its best to wait for https://github.com/celo-org/celo-monorepo/pull/11040

authors: aaronmgdr shazarre

repo: celo-org/developer-tooling


text:
Trim the last element of an array

Trimming is applied to all the elements but the last one.

Thanks!

authors: asmolero cemdervis

repo: cemdervis/SharpConfig

license: mit

license_source: bigquery


text:
Only seeing Sent.com.google.api.client.http.HttpRequest.execute as traces on Stackdriver

What version of OpenCensus are you using?
0.25.0
What JVM are you using (java -version)?
openjdk version "1.8.0_232"
OpenJDK Runtime Environment (build 1.8.0_232-b09)
OpenJDK 64-Bit Server VM (build 25.232-b09, mixed mode)
What did you do?
I have a number of GRPC services written in Kotlin which are running in a Kubernetes cluster, on Google's GKE.
I've followed various pieces of documentation relevant for setting up Stackdriver trace using OpenCensus (notably https://cloud.google.com/trace/docs/setup/java?hl=en_GB).
Based on that, I'm building a lib which is a common dependency of my services. The lib depends on
api
  io.opencensus:opencensus-api:0.25.0,
  io.opencensus:opencensus-exporter-trace-stackdriver:0.25.0

runtimeOnly
  io.opencensus:opencensus-impl:0.25.0

Then the services that depend on that lib also feature a bit of initialization that makes use of given service account, along the lines of
fun initialize(base64SAKeyJson: String) {
    with(Tracing.getTraceConfig()) {
      val traceParams = activeTraceParams.toBuilder()
        .setSampler(Samplers.alwaysSample())
        .build()

      updateActiveTraceParams(traceParams)
    }

    val credentials = getGoogleCredentialsFromBase64SAKeyJson(base64SAKeyJson)
    val configuration = StackdriverTraceConfiguration.builder()
      .setCredentials(credentials)
      .build()

    StackdriverTraceExporter.createAndRegister(configuration)
  }

(The relevant service account is correctly configured with the cloudtrace.agent role and my service logs indicate no error relevant to tracing)
The problem I'm facing is, every single trace shows up as Sent.com.google.api.client.http.HttpRequest.execute which is obviously not as expected. (The traces themselves don't even seem to line up with calls that are made to the cluster.)
Here's a screenshot of Stackdriver's UI which hopefully better demonstrates the issue:

What did you expect to see?
Traces that correspond to calls going into the cluster.
What did you see instead?
Just instances of the Sent.com.google.api.client.http.HttpRequest.execute 'trace' which does not correspond to calls going into the cluster.

Thanks for reporting this!

What version of gRPC are you using?
Was this working for you before OpenCensus v0.25?

Just so you know: since gRPC v1.27.0, OpenCensus dependency is removed from grpc-core. A grpc-census artifact now exists to provide interceptor/stream tracer factory that applies OpenCensus stats/tracing features. Users who want census integration need to add grpc-census artifact to runtime classpath.

Hi @mayurkale22. Thanks a lot for your prompt reply.
Our version of GRPC is 1.27.2. Which judging by what you've mentioned is probably the cause of the issue. (I'm relatively confident our current setup has worked in the past with OpenCensus 0.24.0 alongside GRPC 1.22.2.)
So I'll give io.grpc:grpc-census:1.27.2 a go and report back here. 👌

Okay, so this seems to be working fine now. Thanks very much for your advice.
I still get the odd Sent.com.google.api.client.http.HttpRequest.execute which I haven't been able to figure out but it's rare enough that I can can live with.
Feel free to close 👍 .

Thanks for the confirmation. Closing this issue now.

I still get the odd Sent.com.google.api.client.http.HttpRequest.execute which I haven't been able to figure out but it's rare enough that I can can live with.

Feel free to open a separate issue to address this.

repo: census-instrumentation/opencensus-java


text:
creating keys for client is unnecessarily tied to copy_admin_key

In roles/ceph-client/tasks/create_users_keys.yml,  import_key parameter for ceph_key is determined by copy_admin_key. So, if copy_admin_key is false, the generated keys will not actually be registered to the ceph cluster, and vice versa. But that is odd since there is no relation between copying an admin key to hosts in the cluster and to registering client keys to the cluster. import_key should be always true, or a new variable should be introduced for import_key.
- name: create cephx key(s)
  ceph_key:
    state: present
    name: "{{ item.name }}"
    caps: "{{ item.caps }}"
    secret: "{{ item.key | default('') }}"
    containerized: "{{ docker_exec_cmd | default('') }}"
    cluster: "{{ cluster }}"
    dest: "{{ ceph_conf_key_directory }}"
    import_key: "{{ copy_admin_key }}"    # <== THIS
    mode: "{{ item.mode|default(omit) }}"
  with_items: "{{ keys }}"
  delegate_to: "{{ delegated_node }}"
  when:
    - cephx
    - keys | length > 0
    - inventory_hostname == groups.get(client_group_name) | first

@no2a since you need the admin key on the node from which you are supposed to import your created keys, linking import_key to copy_admin_key is a way to ensure you will be able to do it.
(delegated_node is not always a mon node.)

In my case, delegated_node is a mon node and I don't want to expose an admin key to client nodes (I believe it is a not-so-uncommon situation). But ceph-ansible cannot import client keys without unnecessarily storing the admin key on client hosts.
Although I can set copy_admin_key to false and import the generated keys by hand of cource, it would be nice if I could do it by just running ceph-ansible.

So, is there something wrong with setting import_keys to true always? If a deployer sets copy_admin_key to false and delegated_node has no admin key, then the task fails. It is a natural behavior. The deployer can recognize that he should have copied the admin key in his deployment.

@no2a I get what you are reporting here.
I don't see why it should be wrong with always setting import_keys to true since it's already its default value.
In any case I think it's worth to have a dedicated variable for this, I'll send a patch.
Thanks

@no2a Actually I think there is already a variable for this, it looks more like something I've missed when I've written this part of the code:
https://github.com/ceph/ceph-ansible/blob/master/roles/ceph-client/tasks/create_users_keys.yml#L36
if there is at least 1 mon we set this fact to True, otherwise, we default to copy_admin_key

repo: ceph/ceph-ansible


text:
Need a workaround when the ceph-csi rbdplugin pod failed "fsck" on the disk.

Describe the bug
A clear and concise description of what the bug is.
Environment details

Image/version of Ceph CSI driver : 1.2.1
Helm chart version :
Kernel version :
Mounter used for mounting PVC (for cephfs its fuse or kernel. for rbd its
krbd or rbd-nbd) : krbd
Kubernetes cluster version : v1.15.4
Ceph cluster version : v14.2.4

Steps to reproduce
Steps to reproduce the behavior:

Setup details: '...'
rook-ceph cluster is deployed with application using csi volume.
Deployment to trigger the issue '....'
docker daemon restarted and the application pod was reschedued. The rescheduled application pod failed to mount the volume.
See error
rbdplugin pod logs shows below error messages repeatedly. Complete log is attached.

I0727 17:06:41.056271    9037 utils.go:125] ID: 501047 GRPC response: {"usage":[{"available":3919605760,"total":5150212096,"unit":1,"used":1213829120},{"available":327195,"total":327680,"unit":2,"used":485}]}
I0727 17:06:53.137613    9037 utils.go:119] ID: 501048 GRPC call: /csi.v1.Node/NodeGetCapabilities
I0727 17:06:53.137636    9037 utils.go:120] ID: 501048 GRPC request: {}
I0727 17:06:53.138088    9037 utils.go:125] ID: 501048 GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":2}}}]}
I0727 17:06:53.276368    9037 utils.go:119] ID: 501049 GRPC call: /csi.v1.Node/NodeStageVolume
I0727 17:06:53.276389    9037 utils.go:120] ID: 501049 GRPC request: {"secrets":"***stripped***","staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-3c092a35-b824-46b5-a18f-c1e5db034cfd/globalmount","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":1}},"volume_context":{"clusterID":"rook-ceph","imageFeatures":"layering","imageFormat":"2","pool":"csireplpool","storage.kubernetes.io/csiProvisionerIdentity":"1591172062273-8081-rook-ceph.rbd.csi.ceph.com"},"volume_id":"0001-0009-rook-ceph-0000000000000001-b9a43413-a652-11ea-9a78-7ef490e8cee5"}
I0727 17:06:53.278296    9037 rbd_util.go:477] ID: 501049 setting disableInUseChecks on rbd volume to: false
I0727 17:06:53.325286    9037 rbd_util.go:140] ID: 501049 rbd: status csi-vol-b9a43413-a652-11ea-9a78-7ef490e8cee5 using mon 10.254.239.237:6789,10.254.1.104:6789,10.254.229.126:6789, pool csireplpool
W0727 17:06:53.387644    9037 rbd_util.go:162] ID: 501049 rbd: no watchers on csi-vol-b9a43413-a652-11ea-9a78-7ef490e8cee5
I0727 17:06:53.387673    9037 rbd_attach.go:202] ID: 501049 rbd: map mon 10.254.239.237:6789,10.254.1.104:6789,10.254.229.126:6789
I0727 17:06:53.452749    9037 nodeserver.go:147] ID: 501049 rbd image: 0001-0009-rook-ceph-0000000000000001-b9a43413-a652-11ea-9a78-7ef490e8cee5/csireplpool was successfully mapped at /dev/rbd9
I0727 17:06:53.452874    9037 mount_linux.go:515] Attempting to determine if disk "/dev/rbd9" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/rbd9])
I0727 17:06:53.462499    9037 mount_linux.go:518] Output: "DEVNAME=/dev/rbd9\nTYPE=ext4\n", err: <nil>
I0727 17:06:53.462524    9037 mount_linux.go:441] Checking for issues with fsck on disk: /dev/rbd9
E0727 17:06:53.487198    9037 nodeserver.go:345] ID: 501049 failed to mount device path (/dev/rbd9) to staging path (/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-3c092a35-b824-46b5-a18f-c1e5db034cfd/globalmount/0001-0009-rook-ceph-0000000000000001-b9a43413-a652-11ea-9a78-7ef490e8cee5) for volume (0001-0009-rook-ceph-0000000000000001-b9a43413-a652-11ea-9a78-7ef490e8cee5) error 'fsck' found errors on device /dev/rbd9 but could not correct them: fsck from util-linux 2.23.2
/dev/rbd9: Superblock needs_recovery flag is clear, but journal has data.
/dev/rbd9: Run journal anyway

/dev/rbd9: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.
        (i.e., without -a or -p options)
.
E0727 17:06:53.565721    9037 utils.go:123] ID: 501049 GRPC error: rpc error: code = Internal desc = 'fsck' found errors on device /dev/rbd9 but could not correct them: fsck from util-linux 2.23.2
/dev/rbd9: Superblock needs_recovery flag is clear, but journal has data.
/dev/rbd9: Run journal anyway

/dev/rbd9: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.
        (i.e., without -a or -p options)
.

Actual results
The rescheduled app pod is in "ContainerCreating" state failing to mount the volume.
Describe what happened
Expected behavior

Should be in "Running" state with PVC attached.
The PVC is not mounted or mapped. Can't run "fsck" manually.

A clear and concise description of what you expected to happen.
Logs
If the issue is in PVC creation, deletion, cloning please attach complete logs
of below containers.

csi-provisioner and csi-rbdplugin/csi-cephfsplugin container logs from the
provisioner pod.

If the issue is in PVC resize please attach complete logs of below containers.

csi-resizer and csi-rbdplugin/csi-cephfsplugin container logs from the
provisioner pod.

If the issue is in snapshot creation and deletion please attach complete logs
of below containers.

csi-snapshotter and csi-rbdplugin/csi-cephfsplugin container logs from the
provisioner pod.

If the issue is in PVC mounting please attach complete logs of below containers.

csi-rbdplugin/csi-cephfsplugin and driver-registrar container logs from
plugin pod from the node where the mount is failing.
rook-logs07-27-18-45-rbdplugin.txt

if required attach dmesg logs.

Note:- If its a rbd issue please provide only rbd related logs, if its a
cephfs issue please provide cephfs logs.
Additional context
Add any other context about the problem here.
For example:
Any existing bug report which describe about the similar issue/behavior

can you please try with the latest ceph-csi version and see the issue exists?

@Madhu-1 Thanks for your quick response. This is in the field. Could you help to see if there's any workaround so the manual fsck can run to recover the pod?

@Madhu-1 Thanks for your quick response. This is in the field. Could you help to see if there's any workaround so the manual fsck can run to recover the pod?

we do remove the map if the mounting fails, one thing you can try is, map this  PVC on another node manually and fsck command.

@Madhu-1 Could you point the source code so we can find the exact command for mapping the PVC?

here it is https://github.com/ceph/ceph-csi/blob/v1.2.1/pkg/rbd/rbd_attach.go#L199-L220

Could you provide an example mapping command?

rbd map <rbd_pool>/<rbd_image_name> -m monip:port --user  --keyfile

The code reads "rbd --id cr.id -m mon:port --keyfile cr.Keyfile map pool/rbd_image".
How to retrieve cr.id, cr.keyfile?

The key file is the "ca.crt"? What's the --user or --id ?
# knc get secret rook-csi-rbd-plugin-sa-token-bqr6l -o yaml
apiVersion: v1
data:
  ca.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURjakNDQWxxZ0F3SUJBZ0lJUmhGTlg5T01LMVV3RFFZSktvWklodmNOQVFFTEJRQXdVREVMTUFrR0ExVUUKQmhNQ1FVRXhDekFKQmdOVkJBZ01Ba0ZCTVFzd0NRWURWUVFIREFKQlFURUxNQWtHQTFVRUNnd0NRVUV4Q3pBSgpCZ05WQkFzTUFrRkJNUTB3Q3dZRFZRUUREQVJDUTAxVU1CNFhEVEl3TURReU16QXlNVGt4TVZvWERUSXlNRGN5Ck5UQXlNVGt4TVZvd1VERUxNQWtHQTFVRUJoTUNRVUV4Q3pBSkJnTlZCQWdNQWtGQk1Rc3dDUVlEVlFRSERBSkIKUVRFTE1Ba0dBMVVFQ2d3Q1FVRXhDekFKQmdOVkJBc01Ba0ZCTVEwd0N3WURWUVFEREFSQ1EwMVVNSUlCSWpBTgpCZ2txaGtpRzl3MEJBUUVGQUFPQ0FROEFNSUlCQ2dLQ0FRRUFyRFg1cDFhNS9RV0cwSlBxNHdUWnBHbURab1EzCjBrQzlCcDNheW9YcTkzRjdzY0ttK2dqZXNvUlRHU1lOZHo3THliVDlUM0FHdEI2eGdoN3NNVWppcGRCU3JaQVgKOXdwL1NiK2lSL0RLcjlSbWw3Y2Rua0ZqNFhyNzJMMTBPbmR1U05zSWFWckYwVE5MSlM0VHV5b3Vma0tkLzhxQwpqK3JDSExJUEwwWDBmcWZCeHE3WnUxZkJIQjRKOXB6V3J4RUsvQnJ3bGY3bGdERE93S3kxZlE4cWk2OVpDSHRmCmRNWFVsUDZnb2oxOVg3VlZLWGMzWk9LSVJ1NFN2ZkxURmZTVE5Uamt6UWxBc0ZHcjl0RVIzVnpLdjNxQlRoN2wKK1p2ZEpvbHlFdlI1WXMzUEludzRqVnZuSk1BTUpDS3JZOTM1YnZTY2hEVDVDUkNzYWRPREZZZmcxUUlEQVFBQgpvMUF3VGpBTUJnTlZIUk1FQlRBREFRSC9NQjBHQTFVZERnUVdCQlFzUmFRTzRUa243NThzMTlET01BbTlwaWo0CmpqQWZCZ05WSFNNRUdEQVdnQlFzUmFRTzRUa243NThzMTlET01BbTlwaWo0ampBTkJna3Foa2lHOXcwQkFRc0YKQUFPQ0FRRUFjbTBPVWtLNWZIaEczNzFrdkxSM1U3RlNnUmd5OHV5SkxqL0JYZ3Uyc3RvandwU0hDWXdwYUVHTgo2TGc5VTUveXRCVk9pS1IxeXc4d2xJNFIvN0xJVGQ1cUQzNk96TVZWZmlUbzZSdmZORWpyUVpNR3J5dnZCNjk2Cm9TVmwybmttUlgzbnhlOStlWDl5MjVXb2UweUpmUXIySGROc3ZvTG10UHEyYkw1Mlg1OFpnb0ZGU2Q4Y0o1U2EKUG03TmlSbUpmMG9tMFdqdmlPRVNoMUJzalBZNEhNQ2tiRUpvZElMalczWWp5Y3phU3VHUXdwRFFMQUJQc1VCUgpJWXFBUUNjRzE5TzlucFVyUDZPbjR5MlRERTlIb25hQ3I3T1VsOEhSQlZxRXFLYUhjdmpraStxQWpsalVWSnJ3ClJYUVBBaGxWSlZlVjE4U21xT0hwRTcwME1DdXZlQT09Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
  namespace: cm9vay1jZXBo
  token: ZXlKaGJHY2lPaUpTVXpJMU5pSXNJbXRwWkNJNklqYzBiMVV4TTFsdE1XNUdiRnBHWVRKMWVEWmxTM1pNVkRScmFtVnBWMGczZGpWWE9GUlRWVEV0TXpBaWZRLmV5SnBjM01pT2lKcmRXSmxjbTVsZEdWekwzTmxjblpwWTJWaFkyTnZkVzUwSWl3aWEzVmlaWEp1WlhSbGN5NXBieTl6WlhKMmFXTmxZV05qYjNWdWRDOXVZVzFsYzNCaFkyVWlPaUp5YjI5ckxXTmxjR2dpTENKcmRXSmxjbTVsZEdWekxtbHZMM05sY25acFkyVmhZMk52ZFc1MEwzTmxZM0psZEM1dVlXMWxJam9pY205dmF5MWpjMmt0Y21Ka0xYQnNkV2RwYmkxellTMTBiMnRsYmkxaWNYSTJiQ0lzSW10MVltVnlibVYwWlhNdWFXOHZjMlZ5ZG1salpXRmpZMjkxYm5RdmMyVnlkbWxqWlMxaFkyTnZkVzUwTG01aGJXVWlPaUp5YjI5ckxXTnphUzF5WW1RdGNHeDFaMmx1TFhOaElpd2lhM1ZpWlhKdVpYUmxjeTVwYnk5elpYSjJhV05sWVdOamIzVnVkQzl6WlhKMmFXTmxMV0ZqWTI5MWJuUXVkV2xrSWpvaU9USTFZakk1WmpjdFlUaG1NeTAwTUdJNUxXSXlOR0l0Tm1KalpUWTVZV001TWpNNElpd2ljM1ZpSWpvaWMzbHpkR1Z0T25ObGNuWnBZMlZoWTJOdmRXNTBPbkp2YjJzdFkyVndhRHB5YjI5ckxXTnphUzF5WW1RdGNHeDFaMmx1TFhOaEluMC5LMDc1VjVhbmhyVTYwb2VtaHlCakRybDZjbzNYV3RCUlFVd0I4YjN3bUEyVExPVE4xdWMzVllsWUxPa05OLWhqSUN0RkRyd0pMaFo0NkttdjNJYjNybHJIdWNBR25VekFBZDNTTlJMLUNZRTIySnRRLXpHREljTTltQUVMS1FyR29GeFNORDJUUHF5UWFtbEpxaUd0M2lCRm5XckpWdGE5dXdvNGx5MXBTZE5hQUcxcHZzM3NFd1FCRm55ME9rSUdTMlNhWFp6MGdSWWFXUkdfQ2Z2WDdWTTZ1WFFCcVNydUJVVGJSV0ZiWHBJU1hqcW94dEotNnM3dXlYREFJRF9PaXVxbUFlOXFtbVgwS2hGVl91NkV2RDRuX2FmcXRRN2x1SmpYdlBBVGxPejY3ck1CUlBMYXQwcnJxQS1wLXBCa1pCcXRWMW1MQmY0dFBfaUdLdTYzY0E=
kind: Secret
metadata:
  annotations:
    kubernetes.io/service-account.name: rook-csi-rbd-plugin-sa
    kubernetes.io/service-account.uid: 925b29f7-a8f3-40b9-b24b-6bce69ac9238
  creationTimestamp: "2020-07-27T22:26:13Z"
  name: rook-csi-rbd-plugin-sa-token-bqr6l
  namespace: rook-ceph
  resourceVersion: "56411039"
  selfLink: /api/v1/namespaces/rook-ceph/secrets/rook-csi-rbd-plugin-sa-token-bqr6l
  uid: 1845ada1-226c-43c0-a322-8e8b1db32dac
type: kubernetes.io/service-account-token

Tried with un-decoded ca.crt or decoded ca.crt. neither works.
#  rbd map csireplpool/0001-0009-rook-ceph-0000000000000001-724ee4bc-d06b-11ea-860b-3654d631dc71 -m 0.254.209.61:6789,10.254.243.208:6789,10.254.63.3:6789 --user rook-csi-rbd-plugin-sa  --keyfile ~/ca.crt
rbd: failed to get secret
2020-07-28 03:28:11.241 7fb6b7daab00 -1 auth: failed to decode key 'LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURjakNDQWxxZ0F3SUJBZ0lJUmhGTlg5T01LMVV3RFFZSktvWklodmNOQVFFTEJRQXdVREVMTUFrR0ExVUUKQmhNQ1FVRXhDekFKQmdOVkJBZ01Ba0ZCTVFzd0NRWURWUVFIREFKQlFURUxNQWtHQTFVRUNnd0NRVUV4Q3pBSgpCZ05WQkFzTUFrRkJNUTB3Q3dZRFZRUUREQVJDUTAxVU1CNFhEVEl3TURReU16QXlNVGt4TVZvWERUSXlNRGN5Ck5UQXlNVGt4TVZvd1VERUxNQWtHQTFVRUJoTUNRVUV4Q3pBSkJnTlZCQWdNQWtGQk1Rc3dDUVlEVlFRSERBSkIKUVRFTE1Ba0dBMVVFQ2d3Q1FVRXhDekFKQmdOVkJBc01Ba0ZCTVEwd0N3WURWUVFEREFSQ1EwMVVNSUlCSWpBTgpCZ2txaGtpRzl3MEJBUUVGQUFPQ0FROEFNSUlCQ2dLQ0FRRUFyRFg1cDFhNS9RV0cwSlBxNHdUWnBHbURab1EzCjBrQzlCcDNheW9YcTkzRjdzY0ttK2dqZXNvUlRHU1lOZHo3THliVDlUM0FHdEI2eGdoN3NNVWppcGRCU3JaQVgKOXdwL1NiK2lSL0RLcjlSbWw3Y2Rua0ZqNFhyNzJMMTBPbmR1U05zSWFWckYwVE5MSlM0VHV5b3Vma0tkLzhxQwpqK3JDSExJUEwwWDBmcWZCeHE3WnUxZkJIQjRKOXB6V3J4RUsvQnJ3bGY3bGdERE93S3kxZlE4cWk2OVpDSHRmCmRNWFVsUDZnb2oxOVg3VlZLWGMzWk9LSVJ1NFN2ZkxURmZTVE5Uamt6UWxBc0ZHcjl0RVIzVnpLdjNxQlRoN2wKK1p2ZEpvbHlFdlI1WXMzUEludzRqVnZuSk1BTUpDS3JZOTM1YnZTY2hEVDVDUkNzYWRPREZZZmcxUUlEQVFBQgpvMUF3VGpBTUJnTlZIUk1FQlRBREFRSC9NQjBHQTFVZERnUVdCQlFzUmFRTzRUa243NThzMTlET01BbTlwaWo0CmpqQWZCZ05WSFNNRUdEQVdnQlFzUmFRTzRUa243NThzMTlET01BbTlwaWo0ampBTkJna3Foa2lHOXcwQkFRc0YKQUFPQ0FRRUFjbTBPVWtLNWZIaEczNzFrdkxSM1U3RlNnUmd5OHV5SkxqL0JYZ3Uyc3RvandwU0hDWXdwYUVHTgo2TGc5VTUveXRCVk9pS1IxeXc4d2xJNFIvN0xJVGQ1cUQzNk96TVZWZmlUbzZSdmZORWpyUVpNR3J5dnZCNjk2Cm9TVmwybmttUlgzbnhlOStlWDl5MjVXb2UweUpmUXIySGROc3ZvTG10UHEyYkw1Mlg1OFpnb0ZGU2Q4Y0o1U2EKUG03TmlSbUpmMG9tMFdqdmlPRVNoMUJzalBZNEhNQ2tiRUpvZElMalczWWp5Y3phU3VHUXdwRFFMQUJQc1VCUgpJWXFBUUNjRzE5TzlucFVyUDZPbjR5MlRERTlIb25hQ3I3T1VsOEhSQlZxRXFLYUhjdmpraStxQWpsalVWSnJ3ClJYUVBBaGxWSlZlVjE4U21xT0hwRTcwME1DdXZlQT09Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K

try to use id "admin" and get the key file in manager pod,
/etc/ceph/ceph.client.admin.keyring

guess the volume is attached to the node already, could go to the node and repair the device?

@danielzhanghl The volume is not mapped, fsck failed then the volume is unmapped. So there's no device exists for manual repair.
@Madhu-1 Really appreciate if you can put a little more details on how to get the id and key file and use it in the command.
1). Is it "--id" or "--user".
2). The keyflie format? Should it be decoded or keep as it is as secret.

already provided that info on slack channel https://rook-io.slack.com/archives/CG3HUV94J/p1595907147301200?thread_ts=1595878052.299900&cid=CG3HUV94J or run ceph auth ls in the toolbox pod and use admin creds from the ceph  cluster

kubectl get secrets rook-csi-rbd-node -oyaml -nrook-ceph
apiVersion: v1
data:
  userID: Y3NpLXJiZC1ub2Rl
  userKey: QVFDdWxCOWZjUzJFRVJBQUs1UHNYcDN3M1JFbUhrcnNGbDYyMXc9PQ==
kind: Secret
metadata:
  creationTimestamp: "2020-07-28T02:59:58Z"
  managedFields:
  - apiVersion: v1
    fieldsType: FieldsV1
    fieldsV1:
      f:data:
        .: {}
        f:userID: {}
        f:userKey: {}
      f:metadata:
        f:ownerReferences:
          .: {}
          k:{"uid":"4079b742-de47-4ce2-b091-9300c8e997ff"}:
            .: {}
            f:apiVersion: {}
            f:blockOwnerDeletion: {}
            f:controller: {}
            f:kind: {}
            f:name: {}
            f:uid: {}
      f:type: {}
    manager: rook
    operation: Update
    time: "2020-07-28T02:59:58Z"
  name: rook-csi-rbd-node
  namespace: rook-ceph
  ownerReferences:
  - apiVersion: ceph.rook.io/v1
    blockOwnerDeletion: true
    controller: true
    kind: CephCluster
    name: rook-ceph
    uid: 4079b742-de47-4ce2-b091-9300c8e997ff
  resourceVersion: "281811"
  selfLink: /api/v1/namespaces/rook-ceph/secrets/rook-csi-rbd-node
  uid: 3380f875-bb42-41a7-b51e-ccff7e59b686
type: kubernetes.io/rook
[🎩]mrajanna@localhost rbd $]echo Y3NpLXJiZC1ub2Rl|base64 -d
csi-rbd-node
[🎩]mrajanna@localhost rbd $]echo QVFDdWxCOWZjUzJFRVJBQUs1UHNYcDN3M1JFbUhrcnNGbDYyMXc9PQ==|base64 -d
AQCulB9fcS2EERAAK5PsXp3w3REmHkrsFl621w==

use the  decoded value for --user and --keyring

Many thanks for your support. :+1:We were able to recover the pod.
Could you shed some light on the how the IP address was selected for the "watcher"?
We've a k8s cluster using Calico CNI via IPIP encapulate mode. There're two subnets among all hosts. And the watcher IP seems randomly allocated between the two subnets. Is the watcher IP used in any way? Do you see any issues with this type of IP config?

Many thanks for your support. 👍We were able to recover the pod.
Could you shed some light on the how the IP address was selected for the "watcher"?
We've a k8s cluster using Calico CNI via IPIP encapulate mode. There're two subnets among all hosts. And the watcher IP seems randomly allocated between the two subnets. Is the watcher IP used in any way? Do you see any issues with this type of IP config?

This is not something cephcsi can handle, it's better to check with ceph or rook team. Closing this issue as its fixed.

@Madhu-1 Could you please shed some light on how this can happen frequently in our site?

@yanchicago have you tried with the latest ceph-csi?
@nixpanic @humblec any idea?

@Madhu-1 Unfortunately, this is in the field, can't upgrade at will.

@Madhu-1 Our version is 3.2.1, this problem still exists. This problem must be fixed manually.

@Madhu-1 Is this problem fixed?

@cl51287 I haven't come across this problem @nixpanic @humblec any idea?. Do you have a set of steps to reproduce this one?

@cl51287 can you give more details about the issue please. Are you also using calico in your setup ? and whats the issue faced at your end ? do you come across the fsck errors ? if yes, when exactly it happens ? only when rbdplugin restarts in between? The ceph client connections are established on an IP and it has been tracked for client request and completion ...etc. If the client IP get changed frequently or while in use , I could expect some problems. But, I would like  to confirm whether hostnetworking is enabled for CSI pods in this setup or not? also in this calico setup, any changes to host IPs happen ?

Our error is the same as yanchicago, as follows:
May 14 10:27:00 k8s-test-0-114 kubelet: E0514 10:27:00.723771 3657 csi_attacher.go:320] kubernetes.io/csi: attacher.MountDevice failed: rpc error: code = Internal desc = 'fsck' fou nd errors on device /dev/rbd3 but could not correct them: fsck from util-linux 2.32.1 May 14 10:27:00 k8s-test-0-114 kubelet: /dev/rbd3 contains a file system with errors, check forced. May 14 10:27:00 k8s-test-0-114 kubelet: /dev/rbd3: Inode 23 has an invalid extent node (blk 63775, lblk 5509) May 14 10:27:00 k8s-test-0-114 kubelet: /dev/rbd3: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.
The above error occurred after the physical machine was down. At the same time, a hard disk on this physical machine also appeared in this situation, which needs fsck to repair.
We use the kernel mode to mount, rbdplugin restart will not appear this situation (at least we have not encountered until now) this situation so far we have only encountered it once, and we have not reproduced it in the follow-up.
We did not use calico. Csi pod hostNetwork is true.

Our error is the same as yanchicago, as follows:
`May 14 10:27:00 k8s-test-0-114 kubelet: E0514 10:27:00.723771 3657 csi_attacher.go:320] kubernetes.io/csi: attacher.MountDevice failed: rpc error: code = Internal desc = 'fsck' found errors on device /dev/rbd3 but could not correct them: fsck from util-linux 2.32.1
May 14 10:27:00 k8s-test-0-114 kubelet: /dev/rbd3 contains a file system with errors, check forced.
May 14 10:27:00 k8s-test-0-114 kubelet: /dev/rbd3: Inode 23 has an invalid extent node (blk 63775, lblk 5509)
May 14 10:27:00 k8s-test-0-114 kubelet: /dev/rbd3: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.`
The above error occurred after the physical machine was down. At the same time, a hard disk on this physical machine also appeared in this situation, which needs fsck to repair.

This is expected! Any journalling filesystem when it goes through incomplete transactions like this at situation like node shutdown, its expected or bound to happen.

We use the kernel mode to mount, rbdplugin restart will not appear this situation (at least we have not encountered until now) this situation so far we have only encountered it once, and we have not reproduced it in the follow-up.
We did not use calico. Csi pod hostNetwork is true.

This case is different or purely an illustration of node down scenario. I dont think any thing we can do from CSI side to cover this. More or less, this is working from filesystem point of view as expected/designed. !!

Yes, if the system goes down, it is expected that fsck needs to be repaired. But whether this can be automatically repaired in the csi, I see that the csi has been repaired in the log, but the repair failed.
Because if this happens in the production environment, business programs will not be automatically restored, and all programs on this machine need to be repaired manually.

repo: ceph/ceph-csi


text:
cephfs: use new 'ceph fs resize' command when available

Use ceph fs resize command when it's available in the cluster.if its not available fallback to the old style of resizing the subvolume
ceph change log:https://docs.ceph.com/docs/master/releases/nautilus/#v14-2-8-nautilus
Fixes #1002
Signed-off-by: Madhu Rajanna madhupr007@gmail.com
(cherry picked from commit b065726f19295f117c63eb9825226e6d0d6e9861)
(cherry picked from commit 9022d899eb6fd464f0be33701d8160ecd1317467)

There are two cherry-picked from lines in the commit message. Why is that?

Fixed now, earlier I cherry-picked an already cherry-picked commit

Merging this one as is for release-v2.1 branch and CI passed


text:
ci: update golangci-lint to latest release

This PR updates the  CI tool to the latest release i.e 1.43.0 and fixes all failures which are found in older tools. New tools checks are marked for TODO.
updates #2595
Signed-off-by: Madhu Rajanna madhupr007@gmail.com
@ceph/ceph-csi-maintainers @ceph/contributors requesting a review on priority as rebasing this kind of PR becomes difficult.

/retest ci/centos/mini-e2e/k8s-1.22

/retest ci/centos/mini-e2e/k8s-1.22

Failed to deploy snapshot pieces:
Error from server: error when creating "https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/v4.0.0/client/config/crd/snapshot.storage.k8s.io_volumesnapshotcontents.yaml": etcdserver: request timed out

logs

@mergifyio rebase

@mergifyio refresh


text:
s3tests: modify test_get_object_ifmodifiedsince_failed

Use key's last_modified instead of local time
Signed-off-by: Yehuda Sadeh yehuda@redhat.com

@andrewgaul here's the modified test, how does this one look?

:+1:

authors: andrewgaul yehudasa


text:
Validations are run twice when a resource is created

Checking out the code I've noticed that when a resource is created, validations are run twice. This is usually not a big deal (although they may bring some delay), but in my current case I have a before_validation in the model which builds other records related to the resource that it's being created. This cause to run validations twice and so building 2 times those related records, which is obviously not good.
I can workaround this and add a check if there are already those resources, but what do you think to call @model.save(validate: false) instead of @model.save?

That sounds like a good idea to me. Should be a simple change that shouldn't break existing code (unless people have models that override save and don't accept options).

authors: lgebhardt xdmx

repo: cerebris/jsonapi-resources


text:
Added initial TikZ list-and-make rule.

This is a basic rule that allows the use of the TikZ external/mode=list and make setting. TikZ creates a makefile called \jobname.makefile, and this rule will ensure that make gets called on the correct file. I haven't tested it with anything other than a Linux system, and there are probably improvements to make, but it does the job.

Hi!
Thanks for the contribution, the rule looks good. I just don't deserve being listed as author, it's your contribution. :wink:
Added to the main repository, than you!
Paulo

Oopsie, I forgot to merge first... :rofl:

@cereda  @zoqaeski Sorry for hijacking this PR, but I hoped asking here is maybe better than opening an issue: How is it possible to enforce compilation in parallel ala make -j4 ... ?

authors: bonanza123 cereda zoqaeski

repo: cereda/arara


text:
Remove date-time-picker from terra-core

Summary
Part of https://github.com/cerner/terra-framework/issues/129
Sibling PR that adds date-time-picker to terra-framework:  https://github.com/cerner/terra-framework/pull/403

closing this in favor of opening a new pr once theme variables are updated.

repo: cerner/terra-core


text:
react-router-url

Summary
This fixes the react-router URLs. Currently, if you go to https://terra-ui.herokuapp.com/components/core/badge, the route does not exist. This is because this route is generated by react-router. This pr fixes that issue.
This fix is the same thing react_on_rails suggests.
https://github.com/shakacode/react-webpack-rails-tutorial/blob/master/config/routes.rb#L13
Thanks for contributing to Terra.
@cerner/terra

I found a weird thing on the header id jump. For example:
If I want to hit http://localhost:3000/components/core/modal#isOpened directly, it will still jump to the top of http://localhost:3000/components/core/modal page. The header id doesn't work in here.
But if I hit http://localhost:3000/components/core/modal at first, and try to hit http://localhost:3000/components/core/modal#isOpened. It jumps to the header with isOpened id, works as expected.

I found a weird thing on the header id jump. For example:
If I want to hit http://localhost:3000/components/core/modal#isOpened directly, it will still jump to the top of http://localhost:3000/components/core/modal page. The header id doesn't work in here.

But if I hit http://localhost:3000/components/core/modal at first, and try to hit http://localhost:3000/components/core/modal#isOpened. It jumps to the header with isOpened id, works as expected.
It looks like this has been resolve in react-router v4.
https://github.com/ReactTraining/react-router/issues/394#issuecomment-220221604
What do you think about logging this as a seperate issue and fix it when we upgrade to react-router v4?

I'm fine with logging an issue for when we update RR to v4. Before that can we test the sample code from that issue link for RR v3 for creating a hashLinkScroll function and passing that to the Route onUpdate prop .
import React from 'react';
import { render } from 'react-dom';
import { Router, Route, browserHistory } from 'react-router';

const routes = (
  // your routes
);

function hashLinkScroll() {
  const { hash } = window.location;
  if (hash !== '') {
    // Push onto callback queue so it runs after the DOM is updated,
    // this is required when navigating from a different page so that
    // the element is rendered on the page before trying to getElementById.
    setTimeout(() => {
      const id = hash.replace('#', '');
      const element = document.getElementById(id);
      if (element) element.scrollIntoView();
    }, 0);
  }
}

render(
  <Router
    history={browserHistory}
    routes={routes}
    onUpdate={hashLinkScroll}
  />,
  document.getElementById('root')
)

+1 to add hashLinkScroll() functions. # scroll should be a quite common use case, and it will we better if we can support that before upgrading react-router to v4. I use it a lot.

The function they provided here did not fix the issue.
I logged this issue to fix the hash links: https://github.com/cerner/terra-ui.com/issues/29

authors: Matt-Butler bjankord windse7en


text:
Hi getting error Not ready: Internal error occurred: failed calling webhook "webhook.cert-manager.io": Post "https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=10s": Address is not allowed

Hi getting error Not ready: Internal error occurred: failed calling webhook "webhook.cert-manager.io": Post "https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=10s": Address is not allowed
I installed it on private EKS cluster.
Please help me

See the docs section for AWS EKS https://cert-manager.io/docs/concepts/webhook/#webhook-connection-problems-on-aws-eks

repo: cert-manager/cert-manager


text:
Unable to mount and read only file error

Warning  FailedMount  11m (x329 over 16h)  kubelet  Unable to attach or mount volumes: unmounted volumes=[tls], unattached volumes=[tls kube-api-access-2v6n8]: timed out waiting for the condition
Warning  FailedMount  88s (x482 over 16h)  kubelet  MountVolume.SetUp failed for volume "tls" : rpc error: code = Unknown desc = chmod /var/lib/kubelet/pods/0fd27403-622b-457c-b43f-606472572c59/volumes/kubernetes.io~csi/tls/mount: read-only file system

I'm also experiencing this issue trying to mount to a container. Some more details, if it helps:

Controller image: quay.io/jetstack/cert-manager-controller:v1.7.2
CSI driver image: quay.io/jetstack/cert-manager-csi-driver:v0.2.0
I'm running on Rancher K3s. I have a few services using cert-manager certificates that are working fine.

Certificate definition:
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: registry-tls
spec:
  secretName: registry-tls-secret
  dnsNames:
    - 'registry.redacted.net'
  issuerRef:
    name: letsencrypt-ca
    kind: ClusterIssuer

Pod definition:
apiVersion: v1
kind: Pod
metadata:
  name: docker-registry-pod
  labels:
    app: registry
spec:
  containers:
    - name: registry
      image: registry:2
      volumeMounts:
        - name: repo-vol
          mountPath: '/var/lib/registry'
        - name: tls
          mountPath: '/tls'
  volumes:
    - name: repo-vol
      persistentVolumeClaim:
        claimName: docker-registry-vol-claim
    - name: tls
      csi:
        readOnly: yes
        driver: csi.cert-manager.io
        volumeAttributes:
          csi.cert-manager.io/issuer-name: letsencrypt-ca
          csi.cert-manager.io/dns-names: registry.redacted.net

Logs from csi driver are a bunch of the following:
E0506 00:00:52.022851       1 server.go:109] driver "msg"="failed processing request" "error"="timed out waiting for the condition" "request"={} "rpc_method"="/csi.v1.Node/NodePublishVolume"
I0506 00:01:24.122823       1 nodeserver.go:74] driver "msg"="Registered new volume with storage backend" "pod_name"="docker-registry-pod"
I0506 00:01:24.122885       1 nodeserver.go:84] driver "msg"="Volume registered for management" "pod_name"="docker-registry-pod"
I0506 00:01:25.124239       1 manager.go:504] manager "msg"="Triggering new issuance" "volume_id"="csi-d00aa30048a62abe239705e4427130b3f8b9bec2b54487c35153d9a5ca3db6c1"
I0506 00:01:25.124309       1 manager.go:243] manager "msg"="Processing issuance" "volume_id"="csi-d00aa30048a62abe239705e4427130b3f8b9bec2b54487c35153d9a5ca3db6c1"
I0506 00:01:25.137035       1 manager.go:369] manager "msg"="Deleted CertificateRequest resource" "volume_id"="csi-d00aa30048a62abe239705e4427130b3f8b9bec2b54487c35153d9a5ca3db6c1" "name"="eedc4e94-592d-4857-aef2-4905ecdf75d9" "namespace"="default"
E0506 00:01:25.482172       1 manager.go:506] manager "msg"="Failed to issue certificate, retrying after applying exponential backoff" "error"="waiting for request: certificaterequest.cert-manager.io \"eedc4e94-592d-4857-aef2-4905ecdf75d9\" not found" "volume_id"="csi-d00aa30048a62abe239705e4427130b3f8b9bec2b54487c35153d9a5ca3db6c1"

Logs from cert-manager:
I0506 00:00:23.433363       1 conditions.go:261] Setting lastTransitionTime for CertificateRequest "eedc4e94-592d-4857-aef2-4905ecdf75d9" condition "Approved" to 2022-05-06 00:00:23.433346151 +0000 UTC m=+1446937.641072660
I0506 00:00:23.456865       1 conditions.go:261] Setting lastTransitionTime for CertificateRequest "eedc4e94-592d-4857-aef2-4905ecdf75d9" condition "Ready" to 2022-05-06 00:00:23.456854023 +0000 UTC m=+1446937.664580514
I0506 00:00:23.456913       1 conditions.go:261] Setting lastTransitionTime for CertificateRequest "eedc4e94-592d-4857-aef2-4905ecdf75d9" condition "Ready" to 2022-05-06 00:00:23.456904982 +0000 UTC m=+1446937.664631475
I0506 00:00:23.457423       1 conditions.go:261] Setting lastTransitionTime for CertificateRequest "eedc4e94-592d-4857-aef2-4905ecdf75d9" condition "Ready" to 2022-05-06 00:00:23.457415688 +0000 UTC m=+1446937.665142182
I0506 00:00:23.457598       1 conditions.go:261] Setting lastTransitionTime for CertificateRequest "eedc4e94-592d-4857-aef2-4905ecdf75d9" condition "Ready" to 2022-05-06 00:00:23.457589161 +0000 UTC m=+1446937.665315652
I0506 00:00:23.457627       1 conditions.go:261] Setting lastTransitionTime for CertificateRequest "eedc4e94-592d-4857-aef2-4905ecdf75d9" condition "Ready" to 2022-05-06 00:00:23.457613946 +0000 UTC m=+1446937.665340437
I0506 00:00:23.492341       1 controller.go:161] cert-manager/certificaterequests-issuer-ca "msg"="re-queuing item due to optimistic locking on resource" "key"="default/eedc4e94-592d-4857-aef2-4905ecdf75d9" "error"="Operation cannot be fulfilled on certificaterequests.cert-manager.io \"eedc4e94-592d-4857-aef2-4905ecdf75d9\": the object has been modified; please apply your changes to the latest version and try again"
I0506 00:00:23.499850       1 controller.go:161] cert-manager/certificaterequests-issuer-acme "msg"="re-queuing item due to optimistic locking on resource" "key"="default/eedc4e94-592d-4857-aef2-4905ecdf75d9" "error"="Operation cannot be fulfilled on certificaterequests.cert-manager.io \"eedc4e94-592d-4857-aef2-4905ecdf75d9\": the object has been modified; please apply your changes to the latest version and try again"
I0506 00:00:23.500874       1 controller.go:161] cert-manager/certificaterequests-issuer-vault "msg"="re-queuing item due to optimistic locking on resource" "key"="default/eedc4e94-592d-4857-aef2-4905ecdf75d9" "error"="Operation cannot be fulfilled on certificaterequests.cert-manager.io \"eedc4e94-592d-4857-aef2-4905ecdf75d9\": the object has been modified; please apply your changes to the latest version and try again"
I0506 00:00:23.501078       1 controller.go:161] cert-manager/certificaterequests-issuer-selfsigned "msg"="re-queuing item due to optimistic locking on resource" "key"="default/eedc4e94-592d-4857-aef2-4905ecdf75d9" "error"="Operation cannot be fulfilled on certificaterequests.cert-manager.io \"eedc4e94-592d-4857-aef2-4905ecdf75d9\": the object has been modified; please apply your changes to the latest version and try again"

I can't really tell what's happening here, tbh. It looks like the certificate -is- being issued, but not in time? So the CSI driver deletes the certificate request and tries again which again times out?
I'm using the Cloudflare DNS challenge solver for cert-manager, so it's possible that the propagation delay is to blame here. I've also tried it with the disable-auto-renew attribute.

ETA: after messing around with this for a bit, I eventually gave up and mounted the TLS secret as a volume on the container.
@glmdev I have the same issue, what did you do to fix it?

For anyone running into the above, can they please share what the status is of the CertificateRequest which is causing them problems?
kubectl describe cr eedc4e94-592d-4857-aef2-4905ecdf75d9

It sounds like everyone here is using Let's Encrypt with the CSI driver - is that correct?
I am not sure if the CSI driver is the best tool for this job if so, as the CSI driver will perform a whole new Order on every pod creation for a new certificate, which is likely going to eat up all your quota with their API.
Also to clarify, you don't need to use the CSI driver at all if you are creating a Certificate resource - you'll just want to mount the generated Secret into the Pod as a volume like you've resorted to above 😊

For now I fell back to manual certificate management, I'll check that when I get the time.
No, I have a local CA generated with openssl.

I am trying to use csi-driver for pod certs (mTLS). I am also getting similar error:
Events:
  Type     Reason       Age                   From               Message
  ----     ------       ----                  ----               -------
  Normal   Scheduled    46m                   default-scheduler  Successfully assigned mtls-demo/nginx-csi-app to worker-1
  Warning  FailedMount  10m (x16 over 44m)    kubelet            Unable to attach or mount volumes: unmounted volumes=[tls], unattached volumes=[tls kube-api-access-l5gxv]: timed out waiting for the condition
  Warning  FailedMount  4m10s (x23 over 45m)  kubelet            MountVolume.SetUp failed for volume "tls" : rpc error: code = Unknown desc = timed out waiting for the condition

cert-manager log:
I0819 12:46:49.655107       1 conditions.go:261] Setting lastTransitionTime for CertificateRequest "cc574045-998f-400a-87f6-c3641ada4bc3" condition "Approved" to 2022-08-19 12:46:49.655085489 +0000 UTC m=+20753.641048167
I0819 12:46:49.689897       1 conditions.go:261] Setting lastTransitionTime for CertificateRequest "cc574045-998f-400a-87f6-c3641ada4bc3" condition "Ready" to 2022-08-19 12:46:49.689824879 +0000 UTC m=+20753.675787511
I0819 12:46:49.690161       1 conditions.go:261] Setting lastTransitionTime for CertificateRequest "cc574045-998f-400a-87f6-c3641ada4bc3" condition "Ready" to 2022-08-19 12:46:49.690136449 +0000 UTC m=+20753.676099083
I0819 12:46:49.690568       1 conditions.go:261] Setting lastTransitionTime for CertificateRequest "cc574045-998f-400a-87f6-c3641ada4bc3" condition "Ready" to 2022-08-19 12:46:49.690537081 +0000 UTC m=+20753.676499702
I0819 12:46:49.690728       1 conditions.go:261] Setting lastTransitionTime for CertificateRequest "cc574045-998f-400a-87f6-c3641ada4bc3" condition "Ready" to 2022-08-19 12:46:49.690684788 +0000 UTC m=+20753.676647418
I0819 12:46:49.690923       1 conditions.go:261] Setting lastTransitionTime for CertificateRequest "cc574045-998f-400a-87f6-c3641ada4bc3" condition "Ready" to 2022-08-19 12:46:49.690901191 +0000 UTC m=+20753.676863811
I0819 12:46:49.775161       1 controller.go:161] cert-manager/certificaterequests-issuer-venafi "msg"="re-queuing item due to optimistic locking on resource" "error"="Operation cannot be fulfilled on certificaterequests.cert-manager.io \"cc574045-998f-400a-87f6-c3641ada4bc3\": the object has been modified; please apply your changes to the latest version and try again" "key"="mtls-demo/cc574045-998f-400a-87f6-c3641ada4bc3"
I0819 12:46:49.783453       1 controller.go:161] cert-manager/certificaterequests-issuer-acme "msg"="re-queuing item due to optimistic locking on resource" "error"="Operation cannot be fulfilled on certificaterequests.cert-manager.io \"cc574045-998f-400a-87f6-c3641ada4bc3\": the object has been modified; please apply your changes to the latest version and try again" "key"="mtls-demo/cc574045-998f-400a-87f6-c3641ada4bc3"
I0819 12:46:49.802477       1 controller.go:161] cert-manager/certificaterequests-issuer-vault "msg"="re-queuing item due to optimistic locking on resource" "error"="Operation cannot be fulfilled on certificaterequests.cert-manager.io \"cc574045-998f-400a-87f6-c3641ada4bc3\": the object has been modified; please apply your changes to the latest version and try again" "key"="mtls-demo/cc574045-998f-400a-87f6-c3641ada4bc3"
I0819 12:46:49.822315       1 controller.go:161] cert-manager/certificaterequests-issuer-selfsigned "msg"="re-queuing item due to optimistic locking on resource" "error"="Operation cannot be fulfilled on certificaterequests.cert-manager.io \"cc574045-998f-400a-87f6-c3641ada4bc3\": the object has been modified; please apply your changes to the latest version and try again" "key"="mtls-demo/cc574045-998f-400a-87f6-c3641ada4bc3"

$ kubectl describe cr cc574045-998f-400a-87f6-c3641ada4bc3 -n mtls-demo
Events:
  Type    Reason           Age   From                                                Message
  ----    ------           ----  ----                                                -------
  Normal  IssuerNotFound   85s   cert-manager-certificaterequests-issuer-ca          Referenced "ClusterIssuer" not found: clusterissuer.cert-manager.io "ca-issuer" not found
  Normal  cert-manager.io  85s   cert-manager-certificaterequests-approver           Certificate request has been approved by cert-manager.io
  Normal  IssuerNotFound   85s   cert-manager-certificaterequests-issuer-acme        Referenced "ClusterIssuer" not found: clusterissuer.cert-manager.io "ca-issuer" not found
  Normal  IssuerNotFound   85s   cert-manager-certificaterequests-issuer-selfsigned  Referenced "ClusterIssuer" not found: clusterissuer.cert-manager.io "ca-issuer" not found
  Normal  IssuerNotFound   85s   cert-manager-certificaterequests-issuer-vault       Referenced "ClusterIssuer" not found: clusterissuer.cert-manager.io "ca-issuer" not found
  Normal  IssuerNotFound   85s   cert-manager-certificaterequests-issuer-venafi      Referenced "ClusterIssuer" not found: clusterissuer.cert-manager.io "ca-issuer" not found

From what I understood from the doc, one doesn't need to create any Certificate resource (in which we define issuerRef kind as ClusterIssuer) explicitly. I still tried creating certificate (skipped specifying dnsNames) assuming that DNS name is already specified in the pod spec (csi.cert-manager.io/dns-names: ${POD_NAME}.${POD_NAMESPACE}.svc.cluster.local) but then my certificate failed with error which was actually expected:
admission webhook "webhook.cert-manager.io" denied the request: spec: Invalid value: "": at least one of commonName, dnsNames, uris ipAddresses, or emailAddresses must be set
Certificate:
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: csi-example-com
  namespace: mtls-demo
spec:
  secretName: csi-example-com-tls
  isCA: false
  issuerRef:
    name: ca-issuer
    name: letsencrypt-staging
    kind: ClusterIssuer

I will really appreciate any help.

Hi @GarbageYard, looking at the events of the CertificateRequest, the ca-issuer doesn't exist yet. Either check that the issuer you are trying to use have the correct name and type, or head over to the docs for more information on how to setup an issuer.
The Certificate resource you are creating there has nothing to do with the csi-driver. The webhook error is still correct, and enforces that you must define one of those SANs on the Certificate resource for it to be accepted.

Thanks, actually the name for the ClusterIssuer was incorrectly set. Now I am getting this error:
E0819 14:52:36.083386       1 sync.go:273] cert-manager/orders "msg"="failed to create Order resource due to bad request, marking Order as failed" "error"="400 urn:ietf:params:acme:error:rejectedIdentifier: Error creating new order :: Cannot issue for \"nginx-csi-app.mtls-demo.svc.cluster.local\": Domain name does not end with a valid public suffix (TLD)" "resource_kind"="Order" "resource_name"="9e3438d9-f5ff-4eb5-937a-a64f49f41d5a-1859061661" "resource_namespace"="mtls-demo" "resource_version"="v1"
Since .local is not reachable, I am wondering how the csi-driver is expected to work for pods then? I did check the docs for csi-driver and read that it doesn't support self-signed certs but for letsencrypt, it doesn't say such thing. I did read @munnerz comment above that csi-driver with letsencrypt is not really how it's meant to be used due to quota limit but I at least wanted to see this working with .local so I could then later use the same with real CA, venafi etc.

Even when I can get it to issue successfully, the mounted volumes are empty.
It doesn't help that the chosen logging tool is truncating messages..
nodeserver.go:100] driver "msg"="Volume registered for management" "pod_name"="foobar-0"
nodeserver.go:113] driver "msg"="Ensuring data directory for volume is mounted into pod..." "pod_name"="foobar-0"
nodeserver.go:132] driver "msg"="Bind mounting data directory to the pod's mount namespace" "pod_name"="foobar-0"
nodeserver.go:138] driver "msg"="Volume successfully provisioned and mounted" "pod_name"="foobar-0"

Using csi-driver with Let's Encrypt is not advised and off-topic.
Let's continue only discussing the original issue here (https://github.com/cert-manager/csi-driver/issues/45#issue-961653512).
For more questions about csi-driver + Let's Encrypt, please create a seperate issue.

I'm on a new project now, we use the CSI driver with no issue.

authors: GarbageYard JoshVanL Yamakaky cornfeedhobo glmdev inteon munnerz rajeshbhoye

repo: cert-manager/csi-driver


text:
Adjust cron-jobs run period

Cron-jobs should only run between 9:00 -  18:00

Closed due to #404.

authors: joaodiaslobo ruilopesm


text:
Statická patička na blogu pro tag "newsletter"

Terka Gagnon řešila partnerská loga i v newsletteru na blogu. Což nebude problém. Řeším formu:
Buď to tam budu každý měsíc cpát manuálně a nebo mám otázku, jestli je realizovatelné mít předpřipravenou patičku, která se k tomu hodí automaticky, pokud to bude mít tag "newsletter".
Nechám si poradit možnosti. Jak bys to viděl, @zoul? Díky!

Nachystat nějakou standardizovanou patičku není problém.
Dejme tomu, že bys jen někam vložil předchystaný tag:
{% partner_footer /%}

…a on by měl vygenerovat ta loga, což dává nejlepší smysl z pohledu UX. Která loga kdy vybere? Protože ti partneři se v čase mění. Je OK mít všude jen ty aktuální, třeba i u historických textů? Anebo bychom chtěli mít u každého textu poděkování právě těm partnerům, které jsme měli v době jeho vydání?
A konkrétní dotaz zní, jak to budeš řešit v Ecomailu. My jsme chtěli ten newsletter co nejvíc generovat automaticky (viz ten editor na https://newsletter.cesko.dev), aby šly právě takovéhle věci jako patička vygenerovat strojem a nestrávils u toho mládí ruční prací. Nepřeváděli jste vy ale ten newsletter zpátky na ruční sazbu v Ecomailu?

Super, díky za reakci!
Co se týče přístupu k relevantním logům partnerů - to je otázka na @MartinaHabova. Asi by mi dávalo smysl mít tam loga, která byla aktuální v době vydání toho newsletteru, tím by jsme se ale připravili o tu možnost používat ten předchystaný tag, pokud by to nešlo pošéfit třeba nějakým "verzováním" - {% partner22_footer /%} /  {% partner23_footer /%}, atd.
Ecomail se přesunul zpátky z jednoduchého generování přes editor na "mravenčí" práci v šabloně. Za účelem "zbeautifulizování" po vizuální stránce jsme se holt rozhodli, že bude šablona, kam já hodím texty. Takže od ledna už blogová verze ≠ mailová verze. Tudíž není potřeba řešit :)

Asi to bude ještě na pár komentářů tam a zpátky, než si ujasníme zadání, tak jsem přehodil debatu do Slacku a sem se vrátíme, až bude jasnější zadání.

Pokračujeme tady: https://github.com/cesko-digital/blog/issues/467.

authors: lukasnavesnik zoul

repo: cesko-digital/web


text:
one-shot Iterables

The "one-shot Iterable" vs. "reiterable Iterable", which already caused some discussion in the XML parser thread on the users mailing list and also for example in https://github.com/ceylon/ceylon-sdk/issues/593#issuecomment-236379752, can perhaps be accomodated by introducing interfaces
"iterable structure that can be iterated at least once"
shared interface OneShotIterable<out Element> {
    "returns always the one and same iterator"
    shared Iterator<Element> sameIterator();
}

"iterable structure that can be iterated multiple times"
shared interface ReIterable<out Element>
        satisfies OneShotIterable<Element> {
    "the first call returns sameIterator(), the following calls return different iterators."
    shared Iterator<Element> newIterator();
}

The name of the second interface and its method can be adapted to match the existing Iterable.
I admit this is a rather big intrusion into the existing ceylon.language package, but maybe it can be developed into a good solution. Right now, the type system cannot distinguish between single shot and reiterable Iterables.

As I've stated in the mailing list thread, I don't see any value in this, and I don't think we should add it to Ceylon. The interface OneShotIterable is isomorphic to <T> => T().
I would like to close this issue.

The conflict as I see it is that the ostensible purpose of Iterable (to be a "stream" type) can't be implemented in an immutable fashion. So, basically all of the language module and SDK collection module implementations of it are as a reiterable Iterable. However, the type allows developers implement it as a stream, and thereby break code that assumes all Iterable are reiterable.
This has bothered me for a while, and I would like to see a resolution.

However, the type allows developers implement it as a stream, and thereby break code that assumes all Iterable are reiterable.

Well, that's not true. It quite explicitly documents that such an implementation would be incorrect, and contrary to its semantics.

I thought you meant Iterable is "isomorphic" to T(), but this is about OneShotIterable.iterator() returning the same iterator, continuing to read the same stream.
From a Type perspective, it might make no difference if I get always the same iterator that continues to read from a stream or if I get different interators that each return different unrelated sequences of elements (like a randomized stream), but what is really lacking IMO is a type that corresponds to a concept like Java's InputStreams.
When the purpose of a library is to consume something like that (or when it wants to be able to consume such a stream), then it needs to know which methods it calls are compatible with that.
Or, perhaps at least no method consuming an Iterable may call Iterable's methods in a way that might cause iterator() to be called more than once during the whole library call. That would be easier if we had said method sameIterator().
Are you saying that every method accepting an Iterable that violates the previous paragraph is buggy @gavinking?

Or, perhaps at least no method consuming an Iterable may call Iterable's methods in a way that might cause iterator() to be called more than once during the whole library call.

IMO, that's absolutely the right way to go. Just because something can't be re-iterated doesn't mean that we might not want to filter it, turn it into a sequence, parse it as XML, etc.
With a bit of effort, any reasonable function can be written to perform its task using a single iterator,  and by doing so, the function becomes useful to a wide array of input sources.
Of course, for less effort, or for streams that are expected to be small, it may be more convenient to just take a List<Element>, or continue to accept {Element*} but actually process the result of input.sequence().

what is really lacking IMO is a type that corresponds to a concept like Java's InputStreams.

There already is such a type: Byte().

Byte() needs then to be wrapped as Iterable to have filtering and other Iterable features.
But, if the above is the right way to go, why has Iterable a method iterator() at all? Couldn't all the Iterable features, including for example the for loop, be based on the Iterable itself instead of its Iterator? If Iterable had a next() method (thus implementing Iterator, so to say)?

Byte() needs then to be wrapped as Iterable to have filtering and other Iterable features.

I'm yet to see the compelling usecases for that.

But, if the above is the right way to go, why has Iterable a method iterator() at all?

Because an Iterable stream can be reiterated, multiple times, from the "start". That's the basic nature of this construct.

I'm yet to see the compelling usecases for that.

You're saying it's only useful to filter, map, create a sequence or array, etc. for data that can be read multiple times? How does the re-readability of data indicate what you may want to do with it?
I'm not disagreeing we should use functions. But Iterables are very useful.

You're saying it's only useful to filter, map, create a sequence or array, etc. data that can be read multiple times?

I'm not saying it's conceptually incorrect. I'm just saying that I don't know a whole huge lot of real practical usecases for it.

I mean the problem is that there are deep conceptual problems with a version of Iterable that declares its own next() method. Consider:
value stream = someOnceOnlyStream();
value filteredStream = stream.filter((x) => isUseful(x));
value mappedStream = stream.map((x) => Awesome(x));
for (x in filteredStream) {
    //do stuff
}
for (x in mappedStream) {
   //oopps, mappedStream is empty since stream is exhausted!
}

You really want to start exposing users to such a fragile construct?! I don't.

Ok, so you say Iterables should always be reiterable.
Then, when writing a method that transforms some input, one should try to avoid taking an Iterable and use function as input to allow also once-readable input. That includes charset conversions and many other things in the SDK.

I agree with @quintesse's assertion that for (x in mappedStream) {} should throw in this case.
It's also been established that (without additional knowledge of the stream), whatever results are obtained in independent iterations of a stream should be considered independent and incompatible (i.e. don't trust size).
Given the independence of the results, it is not generally acceptable for a function to perform multiple iterations, because 99.9% of the time, the intention would be to combine the results under the assumption that they were obtained with the same inputs.
I know it's not ideal that we can't always re-use or re-read data sources, but I see no reasonable alternative but to accept the approach in https://github.com/ceylon/ceylon/issues/6406#issuecomment-236440529 for public APIs. The application code that creates the initial stream must deal with the non-re-iterability of the final result (if the input is non-reiterable.)
I do not believe APIs such as XML parsers should assume or require re-iterability, because you may want to process input from a database or URL without reading the entire source into memory first.

Ok, so you say Iterables should always be reiterable.

It's not what I "say". The actual API of Iterable essentially guarantees it! There is no reasonable way to implement Iterable without supporting reiteration unless your prepared to throw an exception from iterator(), which, I think we should all be able to agree, is an obviously not-very-nice thing to do in a language that is supposed to be highly typesafe.
So to actually obtain a non-reiterable Iterable, one would have to do something obviously nasty.

I agree with @quintesse's assertion that for (x in mappedStream) {} should throw in this case.

That's .... terrible.

Ok, so you say Iterables should always be reiterable.
Then, when writing a method that transforms some input, one should try to avoid taking an Iterable and use function as input to allow also once-readable input. That includes charset conversions and many other things in the SDK.

That doesn't follow at all. If your function only reads the stream once, you can accept any input. The most flexible, but probably over-the-top, would be <T|Finished>() | {T*}.
The real question is what a return type should be for {Character*} readUrl(Url url);.
Now, isn't it obvious that the result is technically re-iterable (a new network connection for each iterator() call), but that you would never really want to re-iterate it, to say, parse XML?
So, this entire discussion disappears. Just don't read Iterables multiple times and expect the results from the multiple iterations to be consistent with each other!

Alright, so, I think we've established that there is no way to have typesafe "one shot" streams. Agreed? Can I close the issue now, please?

I agree with @quintesse's assertion that for (x in mappedStream) {} should throw in this case.

That's .... terrible.

Sure, but if it's the best of the alternatives. After all, you can't even create a Sequence without an Iterable.

Now, isn't it obvious that the result is technically re-iterable (a new network connection for each iterator() call), but that you would never really want to re-iterate it, to say, parse XML?

I think that this is the real source of the disconnect here.
One can have documentation that advises that (1) a stream is not stable and (2) reiteration is expensive, without needing to say that the stream is not reiterable at all.

Sure, but if it's the best of the alternatives.

Not true. The best alternative is to just not have this nasty broken construct, close this issue, and move on.

It's that 1) sometimes you can only read data once, and 2) Iterables are very useful, and even essential to do things like create Sequences.

One can have documentation that advises that (1) a stream is not stable and (2) reiteration is expensive, without needing to say that the stream is not reiterable at all.

Agreed. "Good" functions won't attempt to re-read an Iterable, making the issue moot.

Alright, so, I think we've established that there is no way to have typesafe "one shot" streams. Agreed? Can I close the issue now, please?

Agreed!

That doesn't follow at all. If your function only reads the stream once, you can accept any input.

That's the other way around: the problematic issue is if a method accepts Iterable and iterates more than once. But we agree here.
I'm not quite fair-sighted to be fully convinced, but I'll see how I can connect my XML parser to different sorts of input (and yell for help if I get stuck, if you don't mind -- there might be some similar situation on output). Let's close this.

OK, great.

BTW, @gavinking, the specific problem I see with an XML parser having a parameter type of <Byte|Finished>() instead of {Byte*} is that, by your (good) logic that Iterable.iterator() shouldn't throw, the XML parser would be prohibited from returning a lazy {XMLElement*}.
Therefore, either 1) the entire XML document would have to be parsed and materialized before the return, or 2) the return type would be XMLElement|Finished(), taking us out of the domain of Iterables, leaving the caller to deal with the less useful type.
OTOH, if the XML parser accepts {Byte*}, doesn't attempt to re-read it, and returns a lazy {XMLElement*}, the function becomes much more convenient for callers. The XML parser stays agnostic to this entire discussion, "passing the buck" on stability and re-iterability to the caller.

OTOH, if the XML parser accepts {Byte*}

But that's the whole point of this discussion, if you just have an Iterable you're basically dealing with the lowest level API possible and you can't make use of any of the nice methods in Iterable. So you can't filter, you can't map, you can't do anything except get the next value. You might as well start programming assembly.

So you can't filter, you can't map, you can't do anything except get the next value

@quintesse I have no idea what you're taking about :smile: I'd chain all of the nice lazy methods of Iterable together, or use my memoizing Iterable wrapper, or use some other buffer for backtracking, or be lazy and call sequence(), or...
Bottom line, {Byte*} provides much, much more than <Byte|Finished>() and allows me to return an Iterable without wasting unacceptable amounts of memory. And then the caller can do all of this stuff with the result too!

authors: ASzc dlkw gavinking jvasileff quintesse

repo: ceylon/ceylon


text:
Updated Error Messages for #1

Made the error messages more direct. Per issue #1
Hope this helps!
Happy coding!

@haydenty you should have commented on the issue first, as two other people were already working on this.

Hi @haydenty , thank you so much! Let me take a look on messages, and I will merge if ok.
Thanks! ;)

@haydenty can you update README too?

@mbad0la sorry for not commenting, I'll try to do that next time. I thought for this type of issue multiple people could submit PR and @cezarlz could decide which one was the best.
@mbad0la  I fixed the alpha.
@cezarlz I can take a peek at the README as well. What are you all thinking in regards to updating it, just check grammer and content?

@haydenty the the 'alpha' message is better now IMO. By the way, what you do you think of the commas after 'Please' in the messages.
Do you think it is an improper punctuation use?

@mbad0la I was thinking about the comma as well and wasn't quite sure. It seems like it could go both ways.

Guys, please, just review the messages, some messages has a dot on the end and some doesn't have. What do you think? All messages must contains a dot on the end?
Good work guys! I'm really happy :D

Great work @haydenty
I agree with you!

@haydenty I think it would be better to remove the commas as I can see from this discussion at stackexchange.
@cezarlz merge this after @haydenty confirms whether he's fine with the current messages or he'd add another commit to remove the commas after "Please". Everything else looks fine to me. Good work :+1:

Nice discussion!
@haydenty, take a look and tell what do you think, please.

@mbad0la  nice find! I changed them so this should be good to pull request @cezarlz

authors: cezarlz haydenty mbad0la

repo: cezarlz/boss-validator


text:
Cancled OCR, Image Not Found (Image Occlusion Card)

OCR failed on my image occlusion cards. I'm running Anki Version 2.1.39 (576f0043). See error message below and attached screenshot:
Cancelled OCR processing with message :
['Traceback (most recent call last):\n', '  File "/Users/justinbrandt/Library/Application Support/Anki2/addons21/450181164/gui.py", line 58, in on_run_ocr\n    ocr.run_ocr_on_notes(note_ids=selected_nids)\n', '  File "/Users/justinbrandt/Library/Application Support/Anki2/addons21/450181164/ocr.py", line 308, in run_ocr_on_notes\n    notes_query = self.run_ocr_on_query(note_ids=note_ids)\n', '  File "/Users/justinbrandt/Library/Application Support/Anki2/addons21/450181164/ocr.py", line 278, in run_ocr_on_query\n    raw_results = self._ocr_batch_process(batched_txts=batched_txts)\n', '  File "/Users/justinbrandt/Library/Application Support/Anki2/addons21/450181164/ocr.py", line 96, in _ocr_batch_process\n    raw_results[batched_img_txt] = future.result()\n', '  File "concurrent/futures/_base.py", line 432, in result\n', '  File "concurrent/futures/_base.py", line 388, in __get_result\n', '  File "concurrent/futures/thread.py", line 57, in run\n', '  File "/Users/justinbrandt/Library/Application Support/Anki2/addons21/450181164/ocr.py", line 264, in _ocr_img\n    return pytesseract.image_to_string(str(img_pth), lang="+".join(languages or ["eng"]),\n', '  File "/Users/justinbrandt/Library/Application Support/Anki2/addons21/450181164/_vendor/pytesseract/pytesseract.py", line 368, in image_to_string\n    return {\n', '  File "/Users/justinbrandt/Library/Application Support/Anki2/addons21/450181164/_vendor/pytesseract/pytesseract.py", line 371, in \n    Output.STRING: lambda: run_and_get_output(*args),\n', '  File "/Users/justinbrandt/Library/Application Support/Anki2/addons21/450181164/_vendor/pytesseract/pytesseract.py", line 280, in run_and_get_output\n    run_tesseract(**kwargs)\n', '  File "/Users/justinbrandt/Library/Application Support/Anki2/addons21/450181164/_vendor/pytesseract/pytesseract.py", line 257, in run_tesseract\n    raise TesseractError(proc.returncode, get_errors(error_string))\n', "450181164._vendor.pytesseract.pytesseract.TesseractError: (-6, 'dyld: Library not loaded: ../../../leptonica/1.80.0/lib/liblept.5.dylib   Referenced from: /Users/justinbrandt/Library/Application Support/Anki2/addons21/450181164/deps/mac/tesseract/4.1.1/bin/tesseract   Reason: image not found')\n"]

Hi, are you able to display the filename of the image (by pressing Control-Shift-X (Command-Shift-X) while the cursor is in the image field? This is seems to be the same issue as #27 , and I can't figure out why!

Attempted a fix - https://github.com/cfculhane/AnkiOCR/releases/tag/V0.6.1 , please restart anki and check for updates of the plugin and try again :)

Fixed in https://github.com/cfculhane/AnkiOCR/releases/tag/V0.7.0 , please reopen if you have any further issues with this

repo: cfculhane/AnkiOCR


text:
Added dark mode favicon

Ticket: ENT-7951

repo: cfengine/build-website


text:
CancelToken.cancel should NOT return an exception when there's no current request.

Package
dio
Version
5.1.0
Output of flutter doctor -v
[√] Flutter (Channel stable, 3.7.8, on Microsoft Windows [version 10.0.22621.1413], locale fr-FR)
    • Flutter version 3.7.8 on channel stable at C:\Users\remy\Documents\dev\flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 90c64ed42b (6 days ago), 2023-03-21 11:27:08 -0500
    • Engine revision 9aa7816315
    • Dart version 2.19.5
    • DevTools version 2.20.1

[X] Windows Version (Unable to confirm if installed Windows version is 10 or greater)

[√] Android toolchain - develop for Android devices (Android SDK version 33.0.1)
    • Android SDK at C:\Users\remy\AppData\Local\Android\sdk
    • Platform android-33, build-tools 33.0.1
    • Java binary at: C:\Program Files\Android\Android Studio\jbr\bin\java
    • Java version OpenJDK Runtime Environment (build 11.0.15+0-b2043.56-9505619)
    • All Android licenses accepted.

[√] Chrome - develop for the web
    • CHROME_EXECUTABLE = C:\Program Files (x86)\Microsoft\Edge\Application\msedge.exe

[√] Visual Studio - develop for Windows (Visual Studio Community 2022 17.5.1)
    • Visual Studio at C:\Program Files\Microsoft Visual Studio\2022\Community
    • Visual Studio Community 2022 version 17.5.33424.131
    • Windows 10 SDK version 10.0.19041.0

[√] Android Studio (version 2022.1)
    • Android Studio at C:\Program Files\Android\Android Studio
    • Flutter plugin can be installed from:
       https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
       https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.15+0-b2043.56-9505619)

[√] IntelliJ IDEA Community Edition (version 2022.3)
    • IntelliJ at C:\Program Files\JetBrains\IntelliJ IDEA Community Edition
    • Flutter plugin can be installed from:
       https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
       https://plugins.jetbrains.com/plugin/6351-dart

[√] VS Code (version 1.76.2)
    • VS Code at C:\Users\remy\AppData\Local\Programs\Microsoft VS Code
    • Flutter extension version 3.60.0

[√] Connected device (3 available)
    • Windows (desktop) • windows • windows-x64    • Microsoft Windows [version 10.0.22621.1413]
    • Chrome (web)      • chrome  • web-javascript • unknown
    • Edge (web)        • edge    • web-javascript • Microsoft Edge 110.0.1587.63

[√] HTTP Host Availability
    • All required HTTP hosts are available

! Doctor found issues in 1 category.

Dart Version
2.19.5
Steps to Reproduce

Call CancelToken.cancel alone.
StateError is thrown

Expected Result
Since CancelToken is dissociated from the request, if there's no current request when calling cancel, this method should return without providing error.
We still could check for requestOptions before but what's the point?
To illustrate the issue, when closing a bloc I often call cancel to be sure that there is no pending request.
Actual Result
StateError is thrown when there's nothing to cancel.

I also facing same issue. Tests begun failing, because actual request is mocked so cancelToken. requestOptions always null and when disposing riverpod provider I can not cancel token anymore.
final _measurementProvider =
    FutureProvider.autoDispose.family<LastMeasurementDto?, String>(
  (ref, serial) {
    final cancelToken = CancelToken();
    ref.onDispose(cancelToken.cancel);
    return ref
        .read(WebApi.provider)
        .getSensorLastMeasurement(serial, cancelToken);
  },
);

authors: llfbandit sidlatau

repo: cfug/dio


text:
Prevent timeout events from being emitted when item is already fetched

Fixes #136. It was possible for timeout events to be emitted after a page had already emitted a 404 event. The reason for this is likely the fact that when 404's are handled, the HTTP response socket is closed, but the request socket isn't. I'm not too familiar with the node http api, but I'm guessing this is why the timeout event for that request can still be emitted.
Just like @fantapop thought, this was likely introduced in 40a71c1c5abffc2729316f4a90a0cb70b056612f, and this change makes simplecrawler behave the way it did before that commit.
Perhaps what's really needed here is for the request socket to be closed together with the response socket (once again, I haven't worked much with the node http module directly), but this fixes the problem until further notice anyway.

I'm wondering if there's room for some more generic logic that can be applied to all events, to verify that they should be emitted at certain stages. In the meantime this looks good.
Is it possible to reproduce it in a test — or wasn't that doable?

Sure, let me try and add a test to reproduce it!

This was a difficult one to reproduce in a test actually. Using a local URL, I'm having a hard time getting it to fail properly.
The example below uses a  URL that triggers the problem I see if I test this in production, though. Can you make any sense of why this URL would trigger this behavior, but not a local /410?
I've tried setting a Connection: close header on local responses, but that didn't help. The server in question responds in HTTP 1.0, but I don't know if that makes a difference or not.
@cgiffard let me know if you can reproduce the problem using the provided test case.
it("should not emit both 404 and timeout events", function(done) {

    this.timeout("7s");

    var localCrawler = new Crawler("www.jackan.com", "/sida/0%29;");
    var didTimeout = false;
    localCrawler.timeout = 1000;

    localCrawler.on("fetchtimeout", function(queueItem) {
        console.log("fetchtimeout", queueItem.url);
        didTimeout = true;
    });

    localCrawler.on("fetch404", function() {
        setTimeout(function() {
            didTimeout.should.equal(false);
            done();
        }, localCrawler.timeout * 4);
    });

    localCrawler.start();
});

@cgiffard you think we can merge this and release a new version as a solution until we find the root of the problem?

@cgiffard thanks for taking the time to push the new release :star2: :+1:

Hello, after this commit i found another problem here
clientRequest.setTimeout(crawler.timeout, function() {
if (queueItem.fetched) {
return;
}
        if (crawler.running && !queueItem.fetched) {
            crawler._openRequests--;
        }

        queueItem.fetched = true;
        queueItem.status = "timeout";
        crawler.emit("fetchtimeout", queueItem, crawler.timeout);
        clientRequest._crawlerHandled = true;
        clientRequest.abort();
    });

In my test case when some redirects happen one by one socket already die, but clientRequest is still alive and when timeout event fired queueItem.fetched = true and we can't proceed to clientRequest.abort(); and crawl stoppes forever because _openRequests == maxConcurrency.
I've modified code like this and everything works fine on several projects:
clientRequest.setTimeout(crawler.timeout, function() {
if (clientRequest._crawlerHandled) {
return;
}
        if (crawler.running && !queueItem.fetched) {
            crawler._openRequests--;
        }

        if (!queueItem.fetched) {
            queueItem.fetched = true;
            queueItem.status = "timeout";
            crawler.emit("fetchtimeout", queueItem, crawler.timeout);
        }

        clientRequest._crawlerHandled = true;
        clientRequest.abort();
    });

Please tell me if i can provide more usefull information.

Hmm I see, do you know anything more about the conditions that cause this problem? I had a lot of trouble creating a reproducable scenario for the first issue here.
Anyway, if you open a PR we could still merge it - these changes look good to me, and make sense (although @cgiffard would know a lot more about how the code behaves)

I've run project http://www.swannetoscar.com with maxConcurrency = 5
After 5 redirects _openrequest become 5 too and stoppes forevever
Fetch redirect: http://www.swannetoscar.com/factory/shirt/index/collar_product_id/918/ to http://www.swannetoscar.com/factory/shirt/#collar
Fetch redirect: http://www.swannetoscar.com/factory/shirt/index/collar_product_id/917/ to http://www.swannetoscar.com/factory/shirt/#collar
Fetch redirect: http://www.swannetoscar.com/factory/shirt/index/collar_product_id/919/ to http://www.swannetoscar.com/factory/shirt/#collar
Fetch redirect: http://www.swannetoscar.com/factory/shirt/index/collar_product_id/915/ to http://www.swannetoscar.com/factory/shirt/#collar
Fetch redirect: http://www.swannetoscar.com/factory/shirt/index/collar_product_id/916/ to http://www.swannetoscar.com/factory/shirt/#collar
I will open a PR later, thanks

authors: cgiffard fredrikekelund gorbatenkov

repo: cgiffard/node-simplecrawler


text:
Where is chainermn.iterators.create_synchronized_iterator

https://docs.chainer.org/en/stable/chainermn/reference/index.html,
In the chainermn.iterators.create_multi_node_iterator section, it says

Since this iterator shares batches through network in each iteration, communication might be large. If you train your model-parallel network on extremely large dataset, you can also consider to use chainermn.iterators.create_synchronized_iterator.

But I cannot find chainermn.iterators.create_synchronized_iterator either in the document or the repo.

By the way, the links should be `chainer.foo` (not ``chainer.foo``).

That is implemented in this PR.
https://github.com/chainer/chainer/pull/5396
It is a document mistake.

The fix is planned. I'm removing the stale label.

Fixed by #6345

authors: Aixile keisukefukuda kuenishi shu65 toslunar

repo: chainer/chainer


text:
Make link.to_*pu return self

Fix #5321

jenkins, test this please.

Jenkins CI test (for commit 687c0dddf6b0afb45c2bdd0d4b52f43ce0c51dec, target branch master) succeeded!

LGTM!

authors: chainer-ci okuta toslunar


text:
multiarch support

With the proot support landed, we can do multiarch builds, since --no-scripts is now a requirement.
From the apk side of things, this can be done by simply passing --arch [arch] when setting up the apkdb, e.g. in the InitApkDb step.
I'm told that we can possibly recruit @imjasonh to work on this.  Ideally we would probably build images for all of the archs at once, I suppose.

I'm told that we can possibly recruit @imjasonh to work on this.

Indeed! 🎉
How do we want this to look to users? Is this something they should specify in the YAML (archs: ['amd64', 'arm64', 'ppc64le']) or in a flag (apko build my.yaml --archs=amd64,arm64,ppc64le) -- or both?
Is --platforms=all reasonable to support? If this is quick/reliable enough I kind of think we could just make it the default. It's not the default for ko because it does a ton of needless builds, but if that's not the case for apko...

We should probably want to use the Alpine architecture names here, and map them to the OCI ones, but I'm open to doing it the other way around.

I don't have a strong opinion either way. We could even accept both and just warn/fail if both flavors are specified. --platform=amd64,x86_64 is technically a doable (though misguided) thing to request.

I don't either, do what you want to do with abandon of course :)

One thought I had is that OpenWrt and Adelie have different APK architecture names than Alpine does, so having the ability to override this in the YAML probably makes some sense.

I would suggest doing what Docker does, which is accept different forms of arches and internally use whatever you like:

Less headache for users

To be clear, we're talking about mapping OCI architecture names to the architecture names used by APK-ecosystem distributions, not what is actually published in registries, I would assume the registries do those mappings themselves.

I've got a WIP change here: https://github.com/chainguard-dev/apko/compare/main...imjasonh:feature/multiarch
As of right now:

apko build and build-minirootfs both only build tarballs for the current platform, ignoring whatever's in the YAML (with a warning)
apko publish defaults to all platforms*, unless the YAML specifies arch: [amd64,s390x,...], with an --arch=s390x,ppc64le,... flag to override that

Some open questions:

*what are "all platforms" in this case? I'm hard-coding to amd64,arm64,ppc64le,s390x currently, but that can be broader, and maybe dynamically fetched from...somewhere?
I'm only able to successfully build for non-amd64 if I build for amd64 first, either by defaulting to all archs, or --arch=amd64,s390x; --arch=s390x and --arch=s390x,amd64 don't work:

...
2022/03/04 18:54:40 running: /sbin/apk fix --root /tmp/apko-3048633314 --no-cache --update-cache --arch s390x
2022/03/04 18:54:40 [apk] fetch https://dl-cdn.alpinelinux.org/alpine/edge/main/s390x/APKINDEX.tar.gz
WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/edge/main: UNTRUSTED signature
ERROR: unable to select packages:
  alpine-base (no such package):
    required by: world[alpine-base]
Error: failed to build layer image: failed to fixate apk world: failed to run /sbin/apk fix --root /tmp/apko-3048633314 --no-cache --update-cache --arch s390x: exit status 1

I'm also hitting some auth bug that's definitely my own fault, but this feels like a good place to stop today. Let me know if that change looks off-base, I'll try to hack on it a bit more and get back to you.

I'm only able to successfully build for non-amd64 if I build for amd64 first

I presume that's because examples/alpine-base.yml doesn't have s390x public keys in keyring (WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/edge/main: UNTRUSTED signature)

I'm only able to successfully build for non-amd64 if I build for amd64 first

I presume that's because examples/alpine-base.yml doesn't have s390x public keys in keyring (WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/edge/main: UNTRUSTED signature)

Oh interesting. That makes sense I guess. In that case, I might need to think more about how to move keys around.
I can't just add s390x's keys when running the amd64 apko image, since they don't appear to be there:
Error: failed to build layer image: failed to initialize apk keyring: failed to read apk key: open /etc/apk/keys/alpine-devel@lists.alpinelinux.org-58e4f17d.rsa.pub: no such file or directory

I'm finding keys in the s390x image using crane export --platform=linux/s390x alpine - | tar -tvf - | grep etc/apk/keys -- these seem to be the same for ppc64le, but not arm64.
Is the answer remotely fetching keys (using #38), or I guess I could mount keys in?

I can refactor the alpine-keys package to be a virtual and allow installation of the appropriate keyrings that way.

Actually, there is /usr/share/apk/keys which already has the full alpine keyring...

afaik, subpackages only install to /etc/apk/keys per arch and full to /usr/share/apk/keys

We can probably adjust apko to use /usr/share/apk/keys/$ARCH as system keyring.

(and submit this as a best practice for apk distributions to place their entire keyring there)

We can probably adjust apko to use /usr/share/apk/keys/$ARCH as system keyring.

Not quite sure if this is what you had in mind, but in https://github.com/chainguard-dev/apko/commit/51fd8a573ebd69dbf631d8b865ae9045730503f1 I changed the default system keyring to b /usr/share/apk/keys/$ARCH, and put those files there. It still fails for me:
2022/03/04 20:14:40 installing key /usr/share/apk/keys/s390x/alpine-devel@lists.alpinelinux.org-616ac3bc.rsa.pub
2022/03/04 20:14:40 installing key /usr/share/apk/keys/s390x/alpine-devel@lists.alpinelinux.org-58e4f17d.rsa.pub
2022/03/04 20:14:40 synchronizing with desired apk world
2022/03/04 20:14:40 running: /sbin/apk fix --root /tmp/apko-2415625468 --no-cache --update-cache --arch s390x
2022/03/04 20:14:40 [apk] fetch https://dl-cdn.alpinelinux.org/alpine/edge/main/s390x/APKINDEX.tar.gz
WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/edge/main: UNTRUSTED signature
ERROR: unable to select packages:
  alpine-base (no such package):
    required by: world[alpine-base]
Error: failed to build layer image: failed to fixate apk world: failed to run /sbin/apk fix --root /tmp/apko-2415625468 --no-cache --update-cache --arch s390x: exit status 1

I'm sure I'm just doing something wrong.

It sounds like this approach isn't laughably off-base, so I'm just gonna open a draft PR so we can talk about specifics there.

apk looks for keys in /etc/apk/keys, so they have to be there or you have to use --keys-dir to specify location of keys
https://github.com/chainguard-dev/apko/commit/51fd8a573ebd69dbf631d8b865ae9045730503f1 only changed where keys directory is created

We should copy the /usr/share/apk/keys/$ARCH keys to /etc/apk/keys in the target.

Closing in favor of #81.

authors: imjasonh kaniini panekj

repo: chainguard-dev/apko


text:
Add more image support for flux

the current policy didn't allow for any other images through, such as

helm controller
kustomize controller
notification controller

Hey @BobyMCbobs thanks for the PR!
Can you add tests for some of these images you are adding coverage for?  It should just be a few blocks like this for source-controller:
https://github.com/chainguard-dev/policy-catalog/blob/f35e54b4bc38e41b304f8124df519a2d87db221a/policies/sigstore-landscape/policies_test.go#L138-L142
cc @hectorj2f @vaikas FYI

Hey @BobyMCbobs thanks for the PR!
Can you add tests for some of these images you are adding coverage for? It should just be a few blocks like this for source-controller:
https://github.com/chainguard-dev/policy-catalog/blob/f35e54b4bc38e41b304f8124df519a2d87db221a/policies/sigstore-landscape/policies_test.go#L138-L142
cc @hectorj2f @vaikas FYI

@mattmoor, thank you for your response.
Absolutely. Added further tests here https://github.com/chainguard-dev/policy-catalog/commit/acc4fb491a68c5e1ef084b62ba900ff530bb91ea

Thank you @mattmoor!

authors: BobyMCbobs mattmoor

repo: chainguard-dev/policy-catalog


text:
Add Getting Started document for javascript client

You can read the rendered document here

時間空いてしまって申し訳ないですが、ご指摘いただいていた点についてひと通り修正しました。

repo: chaintope/tapyrus-api-client-examples