text:
Feature/add mongo user role resources

Description of your changes
Added resources
azurerm_cosmosdb_mongo_role_definition
azurerm_cosmosdb_mongo_user_definition
Fixes #858
I have:

[ X ] Read and followed Crossplane's [contribution process].
[ X ] Run make reviewable to ensure this PR is ready for review.
[ ] Added backport release-x.y labels to auto-backport this PR if necessary.

How has this code been tested
Test with kind as local cluster provisioning in Azure.

/test-examples="examples/cosmosdb/v1beta1/mongoroledefinition.yaml"

/test-examples="examples/cosmosdb/v1beta1/mongoroledefinition.yaml"

/test-examples="examples/cosmosdb/v1beta1/mongouserdefinition.yaml"

@turkenf @ulucinar @sergenyalcin @erhancagirici
New resource ready for code-review. Note the new func mongoDatabaseBasedId in externalname.go which was needed for extracting the actual ID for these two resources.

Hi @knutejoh, thank you for the contribution, could you please rebase your PR to the main and resolve the conflicts to move forward?

/test-examples="examples/cosmosdb/v1beta1/mongouserdefinition.yaml"

@turkenf I've rebased the pullrequest and reran the tests.

authors: knutejoh turkenf

repo: crossplane-contrib/provider-upjet-azure


text:
Extremely slow reconcilliation of Managed Resource

What happened?
We're seeing very long reconcilliation times with an MR that depends on other MRs in the same composition.
In this case the composition is built from an IAMRole, IAMPolicy and IAMRolePolicyAttachment. The IAMRole and Policy MRs reconcile in a matter of seconds as expected, but the IAMRolePolicyAttachment can take 45 minutes to reconcile.
The IAMRolePolicyAttachment uses the attributes of the IAMRole and IAMPolicy for it's configuration:
        forProvider:
          policyArnSelector:
            matchControllerRef: true
          roleNameSelector:
            matchControllerRef: true

After making a claim, we notice that it doesn't reach a Ready state after several minutes and so we check the events on the IAMRolePolicyAttachment and we see this:
Warning  CannotResolveResourceReferences  6m2s (x2 over 6m2s)  managed/iamrolepolicyattachment.identity.aws.crossplane.io  cannot resolve references: spec.forProvider.policyArn: referenced field was empty (referenced resource may not yet be ready)

That seems to imply that no further attempts to reconcille have taken place for over six minutes!
We then wait, and wait, and eventually it reconciles and we see the following events:
  Warning  CannotResolveResourceReferences  38m (x2 over 38m)  managed/iamrolepolicyattachment.identity.aws.crossplane.io  cannot resolve references: spec.forProvider.policyArn: referenced field was empty (referenced resource may not yet be ready)
  Normal   CreatedExternalResource          1m36s              managed/iamrolepolicyattachment.identity.aws.crossplane.io  Successfully requested creation of external resource

These event messages imply that Crossplane wasn't even attempting to reconcille for some 38 minutes!? Not good.
I enabled debug mode via the ControllerConfig, but this just corroborates the events above. I see a log message starting:
Cannot resolve managed resource references
Then there is nothing for several minutes regarding that iamrolepolicyattachment until eventually this:
Reconciling {"controller": "managed/iamrolepolicyattachment.identity.aws.crossplane.io"
immediately followed by
Successfully requested creation of external resource {"controller": managed/iamrolepolicyattachment.identity.aws.crossplane.io"
Further, the issue seems to go away when we don't set the crossplane.io/external-name annotation on the XRC. Then the XRC and all the MRs are reconcilled successfully in a matter of seconds.
Here's the config we're using:

xrc
apiVersion: crossplane.example/v1alpha1
kind: IRSA
metadata:
  name: datasync
  labels:
    foo: bar
  annotations:
    crossplane.io/external-name: datasync
spec:
  parameters:
    description: "Data Sandbox S3 Sync"
    policyname: datasync
    policydoc: |
      {
        "Version": "2012-10-17",
        "Statement": [
          {
            "Effect": "Allow",
            "Action": [
              "s3:GetBucket"
            ],
            "Resource": "*"
          }
        ]
      }
  writeConnectionSecretToRef:
    name: datasync-irsa

xrd
apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
  name: irsacomposites.crossplane.example
spec:
  claimNames:
    kind: IRSA
    plural: irsas
  connectionSecretKeys:
  - roleArn
  defaultCompositionRef:
    name: irsacomposite
  group: crossplane.example
  names:
    kind: IRSAComposite
    plural: irsacomposites
  versions:
  - name: v1alpha1
    referenceable: true
    schema:
      openAPIV3Schema:
        properties:
          spec:
            properties:
              parameters:
                properties:
                  description:
                    description: Description to attach to IAM Resources
                    type: string
                  policydoc:
                    description: Policy Document
                    type: string
                  policyname:
                    description: IAM Policy Name
                    type: string
                  tags:
                    description: AWS resource tags
                    items:
                      properties:
                        key:
                          type: string
                        value:
                          type: string
                      type: object
                    type: array
                required:
                - policydoc
                - policyname
                - description
                type: object
            required:
            - parameters
            type: object
        type: object
    served: true

composition
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: irsacomposite
spec:
  compositeTypeRef:
    apiVersion: crossplane.example/v1alpha1
    kind: IRSAComposite
  patchSets:
  - name: Metadata
    patches:
    - fromFieldPath: metadata.labels
      type: FromCompositeFieldPath
    - fromFieldPath: metadata.annotations
      type: FromCompositeFieldPath
  resources:
  - base:
      apiVersion: identity.aws.crossplane.io/v1beta1
      kind: IAMRole
      spec:
        deletionPolicy: Delete
        forProvider:
          tags:
          - key: managed_by
            value: crossplane
        providerConfigRef:
          name: crossplane-provider-aws
        writeConnectionSecretToRef:
          namespace: crossplane
    connectionDetails:
    - fromFieldPath: status.atProvider.arn
      name: roleArn
      type: FromFieldPath
    patches:
    - patchSetName: Metadata
      type: PatchSet
    - fromFieldPath: spec.parameters.description
      toFieldPath: spec.forProvider.description
      type: FromCompositeFieldPath
    - fromFieldPath: spec.parameters.tags
      toFieldPath: spec.forProvider.tags
      type: FromCompositeFieldPath
    - fromFieldPath: metadata.uid
      toFieldPath: spec.writeConnectionSecretToRef.name
      type: FromCompositeFieldPath
    - fromFieldPath: spec.claimRef.namespace
      policy:
        fromFieldPath: Required
      toFieldPath: spec.forProvider.assumeRolePolicyDocument
      transforms:
      - string:
          fmt: |
            {
              "Version": "2012-10-17",
              "Statement": [
                {
                  "Sid": "",
                  "Effect": "Allow",
                  "Principal": {
                    "Federated": "arn:aws:iam::123456789:oidc-provider/oidc.eks.eu-west-1.amazonaws.com/id/XXX"
                  },
                  "Action": "sts:AssumeRoleWithWebIdentity",
                  "Condition": {
                  "StringLike": {
                    "oidc.eks.eu-west-1.amazonaws.com/id/XXX:sub": "system:serviceaccount:%s:*"
                  }
                  }
                }
              ]
            }
        type: string
      type: FromCompositeFieldPath
  - base:
      apiVersion: identity.aws.crossplane.io/v1alpha1
      kind: IAMPolicy
      spec:
        deletionPolicy: Delete
        forProvider: {}
        providerConfigRef:
          name: crossplane-provider-aws
        writeConnectionSecretToRef:
          namespace: crossplane
    patches:
    - fromFieldPath: spec.parameters.description
      toFieldPath: spec.forProvider.description
      type: FromCompositeFieldPath
    - fromFieldPath: spec.parameters.policydoc
      toFieldPath: spec.forProvider.document
      type: FromCompositeFieldPath
    - fromFieldPath: spec.parameters.policyname
      toFieldPath: spec.forProvider.name
      type: FromCompositeFieldPath
    - fromFieldPath: metadata.uid
      toFieldPath: spec.writeConnectionSecretToRef.name
      transforms:
      - string:
          fmt: '%s-iampolicy'
        type: string
      type: FromCompositeFieldPath
  - base:
      apiVersion: identity.aws.crossplane.io/v1beta1
      kind: IAMRolePolicyAttachment
      spec:
        deletionPolicy: Delete
        forProvider:
          policyArnSelector:
            matchControllerRef: true
          roleNameSelector:
            matchControllerRef: true
        providerConfigRef:
          name: crossplane-provider-aws
        writeConnectionSecretToRef:
          namespace: crossplane
    patches:
    - patchSetName: Metadata
      type: PatchSet
    - fromFieldPath: metadata.uid
      toFieldPath: spec.writeConnectionSecretToRef.name
      transforms:
      - string:
          fmt: '%s-iamattach'
        type: string
      type: FromCompositeFieldPath
  writeConnectionSecretsToNamespace: crossplane

Crossplane: 1.1.0
AWS Provider: 0.18
Kubernetes: 1.19 (EKS)

Related, I already opened this issue against crossplane core, but it does appear to be a provider issue so have raised here instead.
https://github.com/crossplane/crossplane/issues/2303

we hit the same issue (and I believe @benagricola as well)

I'm looking into this - it might be the assume role policy comparison going wrong, I have a local fix for this and with crossplane 1.3 rc  + provider-aws 0.18 w/ this patch I don't have any issue. Related to #704 - if you have comments on package structure there it would be awesome :)

We had a very similar issue. We use the simple resources without the Compositing mediation.
We have a IAM user, a IAM policy and a IAM policy attachment: IAM user and IAM policy take about 4 seconds to be ready and synced, the IAM policy attachment resource tries 2/3 times to create itself on AWS in the first second and after this the process hangs for about 20 minutes. Of course this amount of time is too much and using helm release standard wait time causes a failed deployment.
This behaviour is not always reproducible, and sometimes it happens for one release and not for the immediately subsequent one.

that's blocking me as well. one can test with:
apiVersion: identity.aws.crossplane.io/v1beta1
kind: IAMRole
metadata:
  name: cnp-example-4746m
spec:
  deletionPolicy: Delete
  forProvider:
    assumeRolePolicyDocument: |
      {
        "Version": "2012-10-17",
        "Statement": [
          {
            "Effect": "Allow",
            "Principal": {
              "Federated": "arn:aws:iam::1234567890:oidc-provider/oidc.eks.eu-central-1.amazonaws.com/id/F62ABAD7D90D310485955C799BD64CE4"
            },
            "Action": "sts:AssumeRoleWithWebIdentity",
            "Condition": {
              "StringEqualsIfExists": {
                "oidc.eks.eu-central-1.amazonaws.com/id/F62ABAD7D90D310485955C799BD64CE4:sub": "system:serviceaccount:cnp-example:cnp-example"
              }
            }
          }
        ]
      }
    description: '[CROSSPLANE] [DYNAMODB] cnp-example-4746m'
    maxSessionDuration: 3600
    path: /cnp/cnp-example/
    tags:
    - key: CostReference
      value: F-330951-51-05
  providerConfigRef:
    name: iat-1234567890
---
apiVersion: identity.aws.crossplane.io/v1alpha1
kind: IAMPolicy
metadata:
  name: cnp-example-4746m
spec:
  deletionPolicy: Delete
  forProvider:
    description: '[CROSSPLANE] [DYNAMODB] cnp-example-4746m'
    document: |
      {
          "Version": "2012-10-17",
          "Statement": [
              {
                  "Sid": "ListAndDescribe",
                  "Effect": "Allow",
                  "Action": [
                      "dynamodb:List*",
                      "dynamodb:DescribeReservedCapacity*",
                      "dynamodb:DescribeLimits",
                      "dynamodb:DescribeTimeToLive"
                  ],
                  "Resource": "*"
              },
              {
                  "Sid": "SpecificTable",
                  "Effect": "Allow",
                  "Action": [
                      "dynamodb:BatchGet*",
                      "dynamodb:DescribeStream",
                      "dynamodb:DescribeTable",
                      "dynamodb:Get*",
                      "dynamodb:Query",
                      "dynamodb:Scan",
                      "dynamodb:BatchWrite*",
                      "dynamodb:CreateTable",
                      "dynamodb:Delete*",
                      "dynamodb:Update*",
                      "dynamodb:PutItem"
                  ],
                  "Resource": "arn:aws:dynamodb:eu-central-1:1234567890:table/cnp-example-4746m"
              }
          ]
      }
    name: cnp-example-4746m
  providerConfigRef:
    name: iat-1234567890
---
apiVersion: identity.aws.crossplane.io/v1beta1
kind: IAMRolePolicyAttachment
metadata:
  name: cnp-example-4746m
spec:
  deletionPolicy: Delete
  forProvider:
    policyArnRef:
      name: cnp-example-4746m
    roleNameRef:
      name: cnp-example-4746m
  providerConfigRef:
    name: iat-1234567890

last 3 tests took in my case: 46min, 30min, 41min
workaround: combining the policyARN from other strings

we are facing the same issue when we try to create mounttargets for a filesystem. had to wait almost 50 mins before the mounttargets were provisioned. is there a chance that this issue will be fixed soon?

Hey @mf-lit, I tried to reproduce the issue with a slightly modified version of your files. But all three resources became ready in less than a minute.
NAME                                              READY   SYNCED   AGE
role.iam.aws.crossplane.io/datasync-tg8zp-tv8hb   True    True     34s

NAME                                                              READY   SYNCED   ROLENAME   POLICYARN                                   AGE
rolepolicyattachment.iam.aws.crossplane.io/datasync-tg8zp-k7f7g   True    True     datasync   arn:aws:iam::609897127049:policy/datasync   34s

NAME                                                ARN                                         READY   SYNCED   AGE
policy.iam.aws.crossplane.io/datasync-tg8zp-m8jnx   arn:aws:iam::609897127049:policy/datasync   True    True     34s

I needed to update apiVersion and kind fields to match the latest CRDs. Here is the composition.yaml file I used:
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: irsacomposite
spec:
  compositeTypeRef:
    apiVersion: crossplane.example/v1alpha1
    kind: IRSAComposite
  patchSets:
  - name: Metadata
    patches:
    - fromFieldPath: metadata.labels
      type: FromCompositeFieldPath
    - fromFieldPath: metadata.annotations
      type: FromCompositeFieldPath
  resources:
  - base:
      apiVersion: iam.aws.crossplane.io/v1beta1
      kind: Role
      spec:
        deletionPolicy: Delete
        forProvider:
          tags:
          - key: managed_by
            value: crossplane
        writeConnectionSecretToRef:
          namespace: crossplane
    connectionDetails:
    - fromFieldPath: status.atProvider.arn
      name: roleArn
      type: FromFieldPath
    patches:
    - patchSetName: Metadata
      type: PatchSet
    - fromFieldPath: spec.parameters.description
      toFieldPath: spec.forProvider.description
      type: FromCompositeFieldPath
    - fromFieldPath: spec.parameters.tags
      toFieldPath: spec.forProvider.tags
      type: FromCompositeFieldPath
    - fromFieldPath: metadata.uid
      toFieldPath: spec.writeConnectionSecretToRef.name
      transforms:
      - string:
          fmt: '%s-iamrole'
        type: string
      type: FromCompositeFieldPath
    - fromFieldPath: spec.claimRef.namespace
      policy:
        fromFieldPath: Required
      toFieldPath: spec.forProvider.assumeRolePolicyDocument
      transforms:
      - string:
          fmt: |
            {
              "Version": "2012-10-17",
              "Statement": [
                {
                  "Sid": "",
                  "Effect": "Allow",
                  "Principal": {
                    "Federated": "arn:aws:iam::123456789:oidc-provider/oidc.eks.eu-west-1.amazonaws.com/id/XXX"
                  },
                  "Action": "sts:AssumeRoleWithWebIdentity",
                  "Condition": {
                  "StringLike": {
                    "oidc.eks.eu-west-1.amazonaws.com/id/XXX:sub": "system:serviceaccount:%s:*"
                  }
                  }
                }
              ]
            }
        type: string
      type: FromCompositeFieldPath
  - base:
      apiVersion: iam.aws.crossplane.io/v1beta1
      kind: Policy
      spec:
        deletionPolicy: Delete
        forProvider: {}
        writeConnectionSecretToRef:
          namespace: crossplane
    patches:
    - fromFieldPath: spec.parameters.description
      toFieldPath: spec.forProvider.description
      type: FromCompositeFieldPath
    - fromFieldPath: spec.parameters.policydoc
      toFieldPath: spec.forProvider.document
      type: FromCompositeFieldPath
    - fromFieldPath: spec.parameters.policyname
      toFieldPath: spec.forProvider.name
      type: FromCompositeFieldPath
    - fromFieldPath: metadata.uid
      toFieldPath: spec.writeConnectionSecretToRef.name
      transforms:
      - string:
          fmt: '%s-iampolicy'
        type: string
      type: FromCompositeFieldPath
  - base:
      apiVersion: iam.aws.crossplane.io/v1beta1
      kind: RolePolicyAttachment
      spec:
        deletionPolicy: Delete
        forProvider:
          policyArnSelector:
            matchControllerRef: true
          roleNameSelector:
            matchControllerRef: true
        writeConnectionSecretToRef:
          namespace: crossplane
    patches:
    - patchSetName: Metadata
      type: PatchSet
    - fromFieldPath: metadata.uid
      toFieldPath: spec.writeConnectionSecretToRef.name
      transforms:
      - string:
          fmt: '%s-iamattach'
        type: string
      type: FromCompositeFieldPath
  writeConnectionSecretsToNamespace: crossplane

If anyone else has a recent example, I'm more than happy to try :)

Thanks for trying to reproduce this @ezgidemirel and sharing your results!
@mithie, if you have specific examples/yamls that you can share to help us reproduce this issue, that would be appreciated as well! :)

source: gharchive/issue

authors: Monska85 RafalMaleska chlunde ezgidemirel janwillies jbw976 mf-lit mithie

repo: crossplane/provider-aws

license_type: permissive

license_source: github-api


text:
[EmbeddingApi] Add autocase for API computeHorizontalScrollOffset & c…

…omputeVerticalScrollRange & onMeasure & setOverScrollMode
-Add autocase for API computeHorizontalScrollOffset & computeVerticalScrollRange & onMeasure & setOverScrollMode
-Cover the XWalkActivity and XWalkInitializer two ways
Impacted tests(approved): new 8, update 0, delete 0
Unit test platform: [Android]
Unit test result summary: pass 8, fail 0, block 0

@jennycao please help to review it.

LGTM

source: gharchive/pull-request

authors: jacky-young jennycao

repo: crosswalk-project/crosswalk-test-suite


text:
Video playback

@rvaish pointed out following

small improvement may be required, the moment  I run code - video resumes - most of the times, people may want to understand the result - then try something else with it — so control may be given to the user to resume the video, rather than auto start

In addition to this, we should also disable the playback until the user completes the task/question.

We're left with "disable the playback until the user completes the task/question". I think this can be put aside for now since it isn't a priority (I guess..).

authors: Ashukat imjching

repo: crowd-course/interactivelearningsystem

license: mit

license_source: bigquery


text:
updated PR milestone 1 (added myself to contributors)

added myself to contributors

fix your conflicts first

Fixed conflicts in new PR

authors: MisterCruz dmorina

repo: crowdresearch/crowdsource-platform


text:
Authentication fix

@nistala we are rolling back to Session Auth until we have the middleware to encrypt oauth cookies, oauth2 is supported as well
@aginzberg can you test?

@dmorina : just to confirm,just before this PR, request.user was always Anonymous . Right ??

Hey, only if you weren't logged in, if yes then it wasn't Anonymous

I'm out of date. Was this addressed with the recent logged-in check PR?
On Wed, Sep 2, 2015 at 8:12 AM, Durim Morina notifications@github.com
wrote:

Hey, only if you weren't logged in, if yes then it wasn't Anonymous
—
Reply to this email directly or view it on GitHub
https://github.com/crowdresearch/crowdsource-platform/pull/390#issuecomment-137120421
.

@mbernst no, this is a different question, when unauthenticated the backend will return Anonymous when trying to access request.user, this is a non-issue

authors: Macbull dmorina mbernst


text:
iptables firewall bouncer starting before the docker daemon on boot?

Based on my logs, it looks like by default the iptables firewall bouncer tries to initialize the iptables rules before the docker daemon can create its chain when the server is starting. Here is the error I got on boot today:
time="12-11-2022 06:43:29" level=info msg="iptables set-up : /usr/sbin/iptables -I DOCKER-USER -m set --match-set crowdsec-blacklists src -j DROP"
time="12-11-2022 06:43:29" level=warning msg="Error inserting set in iptables (/usr/sbin/iptables -I DOCKER-USER -m set --match-set crowdsec-blacklists src -j DROP): exit status 1 : iptables: No chain/target/match by that name.\n"
time="12-11-2022 06:43:29" level=fatal msg="iptables init failed: while inserting set in iptables: exit status 1"

Here is my bouncer config:
mode: iptables
pid_dir: /var/run/
update_frequency: 10s
daemonize: true
log_mode: file
log_dir: /var/log/
log_level: info
log_compression: true
log_max_size: 100
log_max_backups: 3
log_max_age: 30
api_url: http://127.0.0.1:8080/
api_key:
insecure_skip_verify: false
disable_ipv6: true
deny_action: DROP
deny_log: false
supported_decisions_types:
  - ban
#to change log prefix
#deny_log_prefix: "crowdsec: "
#to change the blacklists name
blacklists_ipv4: crowdsec-blacklists
blacklists_ipv6: crowdsec6-blacklists
#if present, insert rule in those chains
iptables_chains:
#  - INPUT
#  - FORWARD
  - DOCKER-USER

## nftables
nftables:
  ipv4:
    enabled: true
    set-only: false
    table: crowdsec
    chain: crowdsec-chain
  ipv6:
    enabled: true
    set-only: false
    table: crowdsec6
    chain: crowdsec6-chain
# packet filter
pf:
  # an empty string disables the anchor
  anchor_name: ""

I haven't made any changes to either the docker systemd service or the bouncer service, but just for giggles here is what my service file looks like:
[Unit]
Description=The firewall bouncer for CrowdSec
After=syslog.target network.target remote-fs.target nss-lookup.target crowdsec.service
Before=netfilter-persistent.service

[Service]
Type=notify
ExecStart=/usr/sbin/crowdsec-firewall-bouncer -c /etc/crowdsec/bouncers/crowdsec-firewall-bouncer.yaml
ExecStartPre=/usr/sbin/crowdsec-firewall-bouncer -c /etc/crowdsec/bouncers/crowdsec-firewall-bouncer.yaml -t
ExecStartPost=/bin/sleep 0.1

[Install]
WantedBy=multi-user.target

I have not tried adding docker.service to the After section of the unit file yet, as this is a production system. This is Ubuntu 20.04.5 if that matters either.
Any advice? Let me know if I may just be missing something

rather odd, is docker listed to enable on startup via systemd?
If so could you add to the after line
After=docker.service

Not working in my case.
I added the After=docker.service at the end of the After line
After=syslog.target network.target remote-fs.target nss-lookup.target crowdsec.service docker.service
and ReloadPropagatedFrom=docker.service to /etc/systemd/system/crowdsec-firewall-bouncer.service
But it isn't working
Even worse the /var/log/crowdsec-firewall-bouncer.log and systemctl status crowdsec-firewall-bouncer disagrees. The logs are saying the process failed, while the service status is fine, despite the Main PID has exited.
Otherwise, if I restart the service manually after some time, it works
Here are the "conflicting" results
cat /var/log/crowdsec-firewall-bouncer.log
time="29-11-2022 23:51:32" level=info msg="backend type : nftables"
time="29-11-2022 23:51:32" level=info msg="nftables initiated"
time="29-11-2022 23:51:32" level=info msg="Using API key auth"
time="29-11-2022 23:51:32" level=info msg="Processing new and deleted decisions . . ."
time="29-11-2022 23:52:02" level=error msg="auth-api: auth with api key failed return nil response, error: dial tcp 172.23.0.3:8080: i/o timeout"
time="29-11-2022 23:52:02" level=error msg="Get "http://172.23.0.3:8080/v1/decisions/stream?startup=true\": dial tcp 172.23.0.3:8080: i/o timeout"
time="29-11-2022 23:52:02" level=error msg="terminating bouncer process"
time="29-11-2022 23:52:02" level=error msg="process return with error: stream api init failed"
time="29-11-2022 23:52:02" level=info msg="removing 'crowdsec' table"
time="29-11-2022 23:52:02" level=info msg="removing 'crowdsec6' table"
root@HVDebian:~# systemctl status crowdsec-firewall-bouncer
● crowdsec-firewall-bouncer.service - The firewall bouncer for CrowdSec
Loaded: loaded (/etc/systemd/system/crowdsec-firewall-bouncer.service; enabled; vendor preset: enabled)
Active: inactive (dead) since Tue 2022-11-29 23:52:02 CET; 3min 8s ago
Process: 2016 ExecStartPre=/usr/sbin/crowdsec-firewall-bouncer -c /etc/crowdsec/bouncers/crowdsec-firewall-bouncer.yaml -t (code=e>
Process: 2043 ExecStart=/usr/sbin/crowdsec-firewall-bouncer -c /etc/crowdsec/bouncers/crowdsec-firewall-bouncer.yaml (code=exited,>
Process: 2058 ExecStartPost=/bin/sleep 0.1 (code=exited, status=0/SUCCESS)
Main PID: 2043 (code=exited, status=0/SUCCESS)
CPU: 29ms
nov. 29 23:51:32 HVDebian systemd[1]: Starting The firewall bouncer for CrowdSec...
nov. 29 23:51:32 HVDebian crowdsec-firewall-bouncer[2016]: time="2022-11-29T23:51:32+01:00" level=info msg="crowdsec-firewall-bouncer >
nov. 29 23:51:32 HVDebian crowdsec-firewall-bouncer[2016]: time="2022-11-29T23:51:32+01:00" level=info msg="config is valid"
nov. 29 23:51:32 HVDebian crowdsec-firewall-bouncer[2043]: time="2022-11-29T23:51:32+01:00" level=info msg="crowdsec-firewall-bouncer >
nov. 29 23:51:32 HVDebian systemd[1]: Started The firewall bouncer for CrowdSec.
nov. 29 23:52:02 HVDebian systemd[1]: crowdsec-firewall-bouncer.service: Succeeded.

Seems your error is happening because crowdsec is inside a container? (presumed from 172 address?) and on startup if the bouncer fails to reconnect it dies....
time="29-11-2022 23:52:02" level=error msg="auth-api: auth with api key failed return nil response, error: dial tcp 172.23.0.3:8080: i/o timeout"
time="29-11-2022 23:52:02" level=error msg="Get "[http://172.23.0.3:8080/v1/decisions/stream?startup=true\](http://172.23.0.3:8080/v1/decisions/stream?startup=true%5C)": dial tcp 172.23.0.3:8080: i/o timeout"

You are right, my crowdsec is in a docker container, and when the firewall-bouncer starts it cannot connect to crowdsec-agent (for some reason (probably delay) and the bouncer shutdowns, leaving my host with no protection :-(
I applied the suggested workaround, without success, so I commented this issue.
As far I understand, the FW starting before the docker daemon on boot is the case of this issue. I just added that the bouncer should not start before the crowdsec container inside docker. Does it deserve another issue ?
And another issue should probably be opended, for the misleading systemctl status of the firewall-bouncer service. Others seem to comment in this way https://github.com/crowdsecurity/cs-firewall-bouncer/issues/217 "But I'm wanted to underline that the bouncer does not resist to a network failure at boot time (whatever the reason).
I my mind, the bouncer should be designed to resist to any kind of storm."

I am facing the same issue :)
I did a workaround by restarting the bouncer after 2minutes via cron: @reboot sleep 120 && systemctl start crowdsec-firewall-bouncer
This is not optimal. Are there other options?

@floari
I've done a restart too, but only in the service description file. Moreover, it garantees a coherent status with
systemctl status crowdsec-firewall-bouncer
I added options, so the service cannot be active with an exited process, and that it must restart if inactive/exited.
RemainAfterExit=no
Restart=always
RestartSec=20
You can find the location of the service file with the command systemctl status crowdsec-firewall-bouncer
Here is my complete /etc/systemd/system/crowdsec-firewall-bouncer.service
[Unit]
Description=The firewall bouncer for CrowdSec
After=syslog.target network.target remote-fs.target nss-lookup.target crowdsec.service docker.service
Before=netfilter-persistent.service
ReloadPropagatedFrom=docker.service
StartLimitIntervalSec=20

[Service]
Type=notify
RemainAfterExit=no
Restart=always
RestartSec=20
ExecStart=/usr/sbin/crowdsec-firewall-bouncer -c /etc/crowdsec/bouncers/crowdsec-firewall-bouncer.yaml
ExecStartPre=/usr/sbin/crowdsec-firewall-bouncer -c /etc/crowdsec/bouncers/crowdsec-firewall-bouncer.yaml -t
ExecStartPost=/bin/sleep 0.1

[Install]
WantedBy=multi-user.target

@StSixtus
Thanks - thats working perfectly.
Maybe something to add in the documentation!

authors: LaurenceJJones StSixtus dakota-marshall floari

repo: crowdsecurity/cs-firewall-bouncer

license: MIT


text:
[REQUIRE CROWDSEC CHANGE] Add support for Caddy 2.5 log format

Fixes https://github.com/crowdsecurity/hub/issues/408

I think the time format is not supported by dateparse-enrich so its just getting the current time of the system in stead of the logs

Any word on getting Crowsdec updated so this can be merged? I'm held off on updating my Caddy server to give enough time for Crowdsec to get released, but Caddy is already on 2.5.1 and over a month since the 2.5.0 release.

With the latest release and https://github.com/crowdsecurity/crowdsec/pull/1576 it should be possible to implement an optimized approach for parsing the newer caddy logs.

@LePresidente could you please rebase your PR with latest master branch ?

Note: I would love to see this fixed. I was looking at adding either corazo WAF to Caddy or Crowsec. I choose for Crowdsec because there is a working Caddy module for download. Being bitten by this made me reconsider my decision. But I am happy to wait a couple of days for this to be solved ;). Thanks in advance.

still failing due to
results["s02-enrich"]["crowdsecurity/dateparse-enrich"][1].Evt.Enriched["MarshaledTime"]

being the current time instead of the log time.

@LePresidente  I've opened PR in your fork for fix.  Basically the date is not being transformed into a format crowdsec can recognize

Ah thanks for that didn't know you could just reorder the expression to match something already supported. I'll take note of that

authors: LePresidente Lednerb adworacz rgevaert sbs2001

repo: crowdsecurity/hub


text:
Issue-291: Add commit variable to cookiecutter context

Allows users to render the commit SHA in their downstream projects by adding the commit hash within the cookiecutter context dictionary.
This allows users to use the {{ cookiecutter._commit }} Jinja variable in their projects. Truncating to a short hash should be possible with {{ cookiecutter._commit|truncate(7, True, '', 0) }}.
Solves #291, and potentially cookiecutter issue 1212.

Failing CI check seems to be due to a GitHub connection error. Re-running the check may pass it. Also noting that some of these cookiecutter-test repos stated in test_api.py, and test_cli.py refer to forks or commits in forks, and not the cruft/cookiecutter-test repo.

This is exactly what I was looking for. I want to pass the cookiecutter commit hash into the template so that I can add it into my documents for traceability.
Looking forward to this getting merged.

Hi there, I know this is probably a question more for the maintainers but any movement on this? I'm just wondering when we can see this merged because I'm quite interested in using the functionality.

authors: ESKYoung holmboe szamfirov

repo: cruft/cruft


text:
Reducing deep result transfer size by order of magnitude

Creating test branch

@seambr great work man! Works nicely right now, not sure if we need to switch to a double loop. Will check some more once it is merged.
Appreciate this contribution!

repo: cryck/sticker-composer