text:
[Feature Request] Expiration options for eligibility/approver policies

Describe the bug
[Feature Request] Expiration options for eligibility/approver policies.
One workflow we've found to be common is that a team says "we need a workflow set up on x accounts for the duration of this sprint". We are still in a pilot phase with this tool so this is easily maintained manually (for now). I can foresee a scenario when we start adding hundreds of policies in the future, and without expiration, the list will grow to a daunting size. Having the ability to set auto-expiration on these policies will help keep the list maintainable, in theory.
I should note, we also plain to maintain the approver / eligibility policies as code at some point, which may render this point moot or add complexity if the policies remove themselves. I haven't given the underlying DB a deep look yet, but I could imagine that having an expiration date as a field and leaving the records in place would solve this issue.
Curious what the "team" team thinks about this scenario-- maybe there is an easier way to handle this?

Acknowledged. This is indeed a good use case and would add to the pipeline.

authors: robbycuenot tawoyinfa

repo: aws-samples/iam-identity-center-team


text:
When deleting VPC tags, the STNO does not change association/propagation

When we remove the tags Associated-with and Propagated-to tags and values , STNO does not reflect the change. I would expect that at least the STNO removes the Association and propagation from the TGW RT

Are you sure about this issue?
I tested a simple deployment and found that when I deleted the Associated-with and Propagated-to tags from the VPC, STNO updated the TGW Route Table propagation. It left tags on the VPC:

STNOStatus-VPCPropagation: 2022-09-28T14:15:46Z: VPC RT propagation has been disabled from the Transit Gateway Routing Table/Domain
STNOStatus-VPCAssociation: 2022-09-28T14:15:46Z: VPC has been dissociated with the Transit Gateway Routing Table/Domain

You would need to remove the Attach-to-tgw tag from the subnet(s) in order for the TGW Attachment(s) to be removed. For me, STNO updated tagging on the subnet:

STNOStatus-Subnet: 2022-09-28T14:14:29Z: Subnet removed from the TGW attachment.

I can confirm what paul-e-allen already wrote, removing the Attach-to-tgw tag from the subnets and then removing Associate-with and Propagate-to works as expected.
I'm closing the issue. If you're still experiencing problems, please reopen with information about your STNO version number and steps that help us to reproduce the problem.
Best regards,
Thiemo

source: gharchive/issue

authors: costi1010 paul-e-allen tbelmega

license_type: permissive

license_source: github-api


text:
Exporting schema to alexa with slots

Exporting schema to alexa dosent provide extra slots
To Reproduce
Create connection with alexa and then Import Intent slot matching example to your questions - and try to copy the schema from alexa.
{
  "interactionModel": {
    "languageModel": {
      "invocationName": "q and a",
      "types": [
        {
          "name": "EXAMPLE_QUESTIONS",
          "values": [
            {
              "name": {
                "value": "book a car"
              }
            },
            {
              "name": {
                "value": "reserver a car"
              }
            },
            {
              "name": {
                "value": "make a car reservation"
              }
            },
            {
              "name": {
                "value": "book a car for {PickUpDate}"
              }
            },
            {
              "name": {
                "value": "dummy utterance"
              }
            }
          ]
        }
      ],
      "intents": [
        {
          "slots": [
            {
              "name": "QnA_slot",
              "type": "EXAMPLE_QUESTIONS"
            }
          ],
          "name": "Qna_intent",
          "samples": [
            "{QnA_slot}"
          ]
        },
        {
          "name": "AMAZON.StopIntent"
        },
        {
          "name": "AMAZON.RepeatIntent"
        },
        {
          "name": "AMAZON.FallbackIntent"
        },
        {
          "name": "AMAZON.CancelIntent"
        }
      ]
    }
  }
}

and now when We try to ask alexa about that - it causes an error - it finds the question but it dont know what to do next

Expected behavior
Alexa should go with follow-up question - same as qnabot client
Version
QnaBot 5.2.0

Thanks for raising this issue and sorry for the delayed response. I've been able to reproduce this issue on the current release (v5.3.4). I'll let you know once we have a fix.

Currently, the Alexa export feature only exports utterances and not slots; this functionality was never implemented. We are adding this to our backlog for next year.

authors: marcburnie piotr-dziubek

repo: aws-solutions/qnabot-on-aws


text:
"IAM Authentication token has expired" while using driver with IAM authentication

Driver version
2.1.0.9
Redshift version
1.0.54052
Client Operating System
Java 11 lambda runtime
JAVA/JVM version
Java 11 lambda runtime
Problem description
Sometimes when our app is trying to create new connection it fails with "FATAL: IAM Authentication token has expired"

Expected behaviour: Able to create jdbc connection
Actual behaviour: JDBC connection sometimes fails with IAM Authentication token has expired.
Error message/stack trace:

Caused by: java.sql.SQLException: FATAL: IAM Authentication token has expired
at com.amazon.redshift.util.RedshiftException.getSQLException(RedshiftException.java:56)
at com.amazon.redshift.Driver.connect(Driver.java:339)
at java.sql/java.sql.DriverManager.getConnection(Unknown Source)
at java.sql/java.sql.DriverManager.getConnection(Unknown Source)
at com.tealium.redshift.automation.services.redshift.JDBCConnectionFactory.createConnection(JDBCConnectionFactory.java:20)
at com.tealium.redshift.automation.services.redshift.JDBCConnectionFactory_ClientProxy.createConnection(Unknown Source)

Any other details that can be helpful:

In class 'IamHelper' it caches credentials in HashMap:private static Map<String, GetClusterCredentialsResult> credentialsCache = new HashMap<String, GetClusterCredentialsResult>();and in the row 636 and similar places it has the following check:
if (credentials == null || (providerType == CredentialProviderType.PLUGIN && idpCredentialsRefresh)
  || credentials.getExpiration().before(new Date(System.currentTimeMillis() - 60 * 1000 * 5))) {

So at client side it considers that credentials from cache are usable additional 5 mins from expiration time.
Reproduction code

Below is a code snippet that uses the logic from jdbc driver and demonstrates that expired token is not considered expired using this code:
import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.util.Date;

class Scratch {
    public static void main(String[] args) throws ParseException {
        SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss.SSS");

        Date currentDate = dateFormat.parse("2023-09-19 12:00:00.000");
        Date expirationDate = dateFormat.parse("2023-09-19 11:56:00.000");

        System.out.println(expirationDate.before(new Date(currentDate.getTime() - 60 * 1000 * 5))); //gives false
        //potential fix
        System.out.println(expirationDate.before(new Date(currentDate.getTime() + 60 * 1000 * 5))); // gives true
    }
}

Hi @vitgorbunov , thanks for opening this issue and bringing this to our attention! I will take a look and get abck to you when I have an update.

@vitgorbunov I don't see anything wrong with current logic - we check if the token expired more than 5 mins ago, and if so you get an expired token error.
Were you intending to convey something else?

@bhvkshah If token has expired 1-4 minutes ago - cache will still consider it valid, and as a result the connection to be created will have invalid token.
We used to simply make the following call to get connection.
DriverManager.getConnection(String.format("jdbc:redshift:iam://%s:%s/%s", clusterId, region, db), user, null);
As you see I don't specify any password. It's generated by driver under the hood and cached, thus it's very confusing to get "IAM Authentication token has expired".  And I can't really retry the call until 5 minutes since token expired, because expired token is not removed from cache.
Now we have to maintain our own cache of temporary credentials and pass them to getConnection method.

I see your concern. We would have to test this fix for the use case of expiring credentials. Would you prefer to raise a PR for this yourself?

@vitgorbunov The fix for this was released in driver version 2.1.0.21. Thank you for your contribution to the Redshift JDBC Driver!

Thank you @bhvkshah !

authors: bhvkshah vitgorbunov

repo: aws/amazon-redshift-jdbc-driver

license_source: bigquery


text:
Occasional segfaults in event_loop teardown due to dead logger

Describe the bug
I am using distributed and pyarrow, which links to this SDK for s3 file access.
Using version 1.11.156, I see (intermittently, so a race condition of some kind) a segfault in the "EvntLoopCleanup" thread that AWS-SDK sets up.
See backtrace below.
Expected Behavior
No segfault
Current Behavior
Backtrace:
(gdb) info thread
  Id   Target Id                                           Frame
  1    Thread 0x7ffff7fe8740 (LWP 54918) "python"          _dl_close_worker (map=map@entry=0x5555c05d20a0,
    force=force@entry=false) at dl-close.c:794
  3    Thread 0x7fff809ff700 (LWP 54931) "jemalloc_bg_thd" 0x00007ffff7bc2065 in futex_abstimed_wait_cancelable (
    private=<optimized out>, abstime=0x7fff809fed70, expected=0, futex_word=0x7fff81016634)
    at ../sysdeps/unix/sysv/linux/futex-internal.h:205
  5    Thread 0x7fff15d9e700 (LWP 54934) "cuda-EvtHandlr"  0x00007ffff7132bb9 in __GI___poll (fds=0x55555b07c900, nfds=2,
    timeout=-1) at ../sysdeps/unix/sysv/linux/poll.c:29
  10   Thread 0x7fff1559d700 (LWP 54941) "AwsEventLoop 1"  0x00007ffff713f947 in epoll_wait (epfd=37, events=0x7fff1559c950,
    maxevents=100, timeout=100000) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
  11   Thread 0x7ffed9cd2700 (LWP 54942) "cuda-EvtHandlr"  0x00007ffff7132bb9 in __GI___poll (fds=0x7ffed4000bd0, nfds=10,
    timeout=100) at ../sysdeps/unix/sysv/linux/poll.c:29
  12   Thread 0x7ffed9270700 (LWP 54943) "python"          0x00007ffff7bc2065 in futex_abstimed_wait_cancelable (
    private=<optimized out>, abstime=0x7ffed926fe80, expected=0, futex_word=0x55555b156768)
    at ../sysdeps/unix/sysv/linux/futex-internal.h:205
  55   Thread 0x7ffe91dda700 (LWP 54988) "jemalloc_bg_thd" 0x00007ffff7bc1ad3 in futex_wait_cancelable (
    private=<optimized out>, expected=0, futex_word=0x7fff81016704) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
  57   Thread 0x7ffe8f728700 (LWP 54991) "jemalloc_bg_thd" 0x00007ffff7bc1ad3 in futex_wait_cancelable (
    private=<optimized out>, expected=0, futex_word=0x7fff810167d4) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
* 84   Thread 0x7fff8eb84700 (LWP 55056) "EvntLoopCleanup" 0x00007fff82829b90 in Aws::Utils::Logging::s_aws_logger_redirect_get_log_level (logger=0x7fff8290b2f0 <Aws::Utils::Logging::s_sdkCrtLogger>, subject=1025)
    at /usr/local/src/conda/aws-sdk-cpp-1.11.156/src/aws-cpp-sdk-core/source/utils/logging/CRTLogging.cpp:73

(gdb) bt
#0  0x00007fff82829b90 in Aws::Utils::Logging::s_aws_logger_redirect_get_log_level (
    logger=0x7fff8290b2f0 <Aws::Utils::Logging::s_sdkCrtLogger>, subject=1025)
    at /usr/local/src/conda/aws-sdk-cpp-1.11.156/src/aws-cpp-sdk-core/source/utils/logging/CRTLogging.cpp:73
#1  0x00007fff81e2e7d3 in s_destroy ()
   from /opt/conda/envs/dask/lib/python3.9/site-packages/pyarrow/../../../././libaws-c-io.so.1.0.0
#2  0x00007fff81e28786 in s_aws_event_loop_group_shutdown_sync ()
   from /opt/conda/envs/dask/lib/python3.9/site-packages/pyarrow/../../../././libaws-c-io.so.1.0.0
#3  0x00007fff81e287d9 in s_event_loop_destroy_async_thread_fn ()
   from /opt/conda/envs/dask/lib/python3.9/site-packages/pyarrow/../../../././libaws-c-io.so.1.0.0
#4  0x00007fff81fa9359 in thread_fn ()
   from /opt/conda/envs/dask/lib/python3.9/site-packages/pyarrow/../../../././libaws-c-common.so.1
#5  0x00007ffff7bbb6db in start_thread (arg=0x7fff8eb84700) at pthread_create.c:463
#6  0x00007ffff713f61f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
(gdb) p *CRTLogSystem._M_ptr
$8 = {_vptr.CRTLogSystemInterface = 0x0}

So it looks like in attempting to log something, the (static?) CRTLogSystem has already been torn down.
(I can attempt to build libaws-c-io with symbols as well to get more information).
Reproduction Steps
This is not that easy (I have not yet managed to minimise at all sorry!). Needs an NVIDIA gpu and a cuda toolkit in the host environment + nvidia container toolkit.
$ git clone https://github.com/dask/distributed
$ docker run --rm -it --gpus device=1 --network=host --volume $PWD:/workspace --workdir /workspace --entrypoint /bin/bash docker.io/gpuci/distributed:23.10-cuda11.5-devel-ubuntu18.04-py3.9
# in container
$ export PATH=/opt/conda/bin:/usr/local/cuda/bin:$PATH
$ export PARALLEL_LEVEL=${PARALLEL_LEVEL:-4}
$ cd /workspace
$ export DASK_DISTRIBUTED__ADMIN__SYSTEM_MONITOR__GIL__ENABLED=False
$ . /opt/conda/etc/profile.d/conda.sh
$ conda activate dask
$ python -m pip install -e .
$ python -m pip install git+https://github.com/dask/dask
# this might need to be run a few times
$ pytest distributed -v -m gpu --runslow -k 'ping_pong or transpose'

Possible Solution
No response
Additional Information/Context
SDK version 1.10.x doesn't exhibit this issue.
AWS CPP SDK version used
1.11.156
Compiler and Version used
g++ (conda-forge gcc 12.3.0-2) 12.3.0
Operating System and version
Ubuntu 18.04

Can you reproduce this on something other than a NIVDIA gpu? If so can you provide the detailed reproduction steps for that?

Can you reproduce this on something other than a NIVDIA gpu? If so can you provide the detailed reproduction steps for that?

I will give it a go and get back to you (it may take a little while), thanks!

@wence- has been busy with other tasks, but I finally succeeded in coming up with a reproducer that doesn't require GPUs, I'm not entirely sure the bug is the same as the stack I see is somewhat different from that in the original description:
(gdb) bt
#0  aws_mem_release (allocator=0x55c315d103b9, ptr=0x55c649b598d0) at /home/mambauser/aws-sdk-cpp/crt/aws-crt-cpp/crt/aws-c-common/source/allocator.c:210
#1  0x00007fbbf0accc55 in Aws::Crt::StlAllocator<char>::deallocate (this=0x55c649b598a0, this=0x55c649b598a0, p=<optimized out>)
    at /home/mambauser/aws-sdk-cpp/crt/aws-crt-cpp/include/aws/crt/StlAllocator.h:56
#2  std::allocator_traits<Aws::Crt::StlAllocator<char> >::deallocate (__n=<optimized out>, __p=<optimized out>, __a=...)
    at /opt/conda/envs/aws-segfault/x86_64-conda-linux-gnu/include/c++/11.4.0/bits/alloc_traits.h:345
#3  std::__cxx11::basic_string<char, std::char_traits<char>, Aws::Crt::StlAllocator<char> >::_M_destroy (__size=<optimized out>, this=0x55c649b598a0)
    at /opt/conda/envs/aws-segfault/x86_64-conda-linux-gnu/include/c++/11.4.0/bits/basic_string.h:245
#4  std::__cxx11::basic_string<char, std::char_traits<char>, Aws::Crt::StlAllocator<char> >::_M_dispose (this=0x55c649b598a0)
    at /opt/conda/envs/aws-segfault/x86_64-conda-linux-gnu/include/c++/11.4.0/bits/basic_string.h:240
#5  std::__cxx11::basic_string<char, std::char_traits<char>, Aws::Crt::StlAllocator<char> >::_M_dispose (this=0x55c649b598a0)
    at /opt/conda/envs/aws-segfault/x86_64-conda-linux-gnu/include/c++/11.4.0/bits/basic_string.h:237
#6  std::__cxx11::basic_string<char, std::char_traits<char>, Aws::Crt::StlAllocator<char> >::~basic_string (this=0x55c649b598a0, __in_chrg=<optimized out>)
    at /opt/conda/envs/aws-segfault/x86_64-conda-linux-gnu/include/c++/11.4.0/bits/basic_string.h:672
#7  std::default_delete<std::__cxx11::basic_string<char, std::char_traits<char>, Aws::Crt::StlAllocator<char> > >::operator() (this=<optimized out>,
    __ptr=0x55c649b598a0) at /opt/conda/envs/aws-segfault/x86_64-conda-linux-gnu/include/c++/11.4.0/bits/unique_ptr.h:85
#8  std::unique_ptr<std::__cxx11::basic_string<char, std::char_traits<char>, Aws::Crt::StlAllocator<char> >, std::default_delete<std::__cxx11::basic_string<char, std::char_traits<char>, Aws::Crt::StlAllocator<char> > > >::~unique_ptr (this=<optimized out>, __in_chrg=<optimized out>)
    at /opt/conda/envs/aws-segfault/x86_64-conda-linux-gnu/include/c++/11.4.0/bits/unique_ptr.h:361
#9  0x00007fbe99af3495 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#10 0x00007fbe99af3610 in exit () from /lib/x86_64-linux-gnu/libc.so.6
#11 0x00007fbe99ad7d97 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#12 0x00007fbe99ad7e40 in __libc_start_main () from /lib/x86_64-linux-gnu/libc.so.6
#13 0x000055c646edb421 in _start ()

To run the reproducer a system with Docker available is required, first save the snippet below named aws-segfault.dockerfile:
aws-segfault.dockerfile
FROM mambaorg/micromamba:jammy

# Install runtime, build and debug dependencies
RUN micromamba create -c conda-forge -n aws-segfault \
        python=3.10 \
        cuda-version=11.8 \
        pyarrow=12.0.1 \
        aws-sdk-cpp=1.11.156 \
        pytest \
        pytest-cov \
        pytest-timeout \
        cmake \
        c-compiler \
        cxx-compiler \
        gcc_linux-64=11.* \
        make \
        gdb \
        git

SHELL ["micromamba", "run", "-n", "aws-segfault", "/bin/bash", "-c"]

# Install distributed from source in developer mode
RUN cd ${HOME} \
    && git clone https://github.com/dask/distributed \
    && cd distributed \
    && git checkout 2023.9.2 \
    && pip install -e .

# Build and install aws-sdk-cpp from source with debug config
# Note: we must unset SCCACHE_BUCKET that is predefined by the base image
RUN unset SCCACHE_BUCKET \
    && cd ${HOME} \
    && git clone --recurse-submodules https://github.com/aws/aws-sdk-cpp \
    && cd aws-sdk-cpp \
    && git checkout 1.11.181 \
    && mkdir build \
    && cd build \
    && cmake .. \
        -DCMAKE_BUILD_TYPE=Debug \
        -DCMAKE_PREFIX_PATH=$CONDA_PREFIX \
        -DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX \
        -DBUILD_ONLY="s3" \
        -DAUTORUN_UNIT_TESTS=OFF \
        -DENABLE_TESTING=OFF \
    && cmake --build . --config=Debug \
    && cmake --install . --config=Debug

COPY test_aws_segfault.py /home/mambauser/distributed/test_aws_segfault.py
WORKDIR /home/mambauser/distributed

And save the following named test_aws_segfault.py to the same local directory:
test_aws_segfault.py
import pytest

pytestmark = pytest.mark.gpu

from distributed import Client
from distributed.deploy.local import LocalCluster
from distributed.utils_test import gen_test

@gen_test()
async def test_aws_segfault():
    from pyarrow.fs import FSSpecHandler, PyFileSystem
    da = pytest.importorskip("dask.array")

    async with LocalCluster(
        protocol="tcp", n_workers=2, threads_per_worker=2, dashboard_address=":0", asynchronous=True
    ) as cluster:
        async with Client(cluster, asynchronous=True):
            assert cluster.scheduler_address.startswith("tcp://")
            x = da.ones((10000, 10000), chunks=(1000, 1000)).persist()
            await x
            y = (x + x.T).sum()
            await y

With both files present in the local directory, build the docker image:
docker build -t aws-segfault:latest -f aws-segfault.dockerfile .

Then start the container:
docker run --rm -ti aws-segfault:latest /bin/bash

Inside the container run:
micromamba activate aws-segfault
python -m pytest -vs test_aws_segfault.py
# Or run with gdb
# gdb --args python -m pytest -vs test_aws_segfault.py
# run

With the instructions above I consistently got a segfault and didn't need to run multiple times to reproduce.

This should be fixed with these two PR's:

https://github.com/aws/aws-sdk-cpp/pull/2859
https://github.com/aws/aws-sdk-cpp/pull/2885
Please let us know if you're still running into this error.

Thanks, we'll keep an eye out.

authors: jmklix pentschev wence-

repo: aws/aws-sdk-cpp


text:
Failed To Write Unit Tests: CognitoUserPool Constructor Receives AmazonCognitoIdentityProviderClient Instance

Hi,
I'm writing a unit test, and in my code I'd like to use dependency injection for my Cognito Identity Provider, so I can mock it in my unit tests. But the CognitoUserPool received the AmazonCognitoIdentityProviderClient as a constructor argument.
Also, I want my Identity Provider registered as a singleton, because my application will use only one Identity Provider for all my users pool.
Is there a workaround or recommendation how to make this happen?
Adding a code sample below.
Thanks
internal class LoginHandler
{
  private readonly IAmazonCognitoIdentityProvider _identityProvider;
  public LoginHandler(IAmazonCognitoIdentityProvider identityProvider)
  {
     _identityProvider = identityProvider;
  }

  public override async Task<AuthResponse> LoginAsync(string clientId, string clientSecret, string username)
  {
    var userPool = new CognitoUserPool("app-user-pool-id", clientId, _identityProvider, clientSecret);
    throw new NotImplementedException();
  }
}

Hi,
I am not understanding the issue. What is blocking you from mocking IAmazonCognitoIdentityProvider and passing the mock as a parameter to the CognitoUserPool ?
You can do that with Moq.

Closing due to no activity.

authors: Laparka assyadh klaytaybai

repo: aws/aws-sdk-net-extensions-cognito


text:
LambdaStep using one of two possible options to call Lambda

Step Function can call Lamba in two ways - with lambda name in Parameters -> FunctionName or with ARN in Resource. Second one does simplify data moves between functions and allows links from the console to CloudWatch logs etc.
Both ways are described in https://docs.aws.amazon.com/step-functions/latest/dg/connect-lambda.html

Hi @tvaroska, there is a simple way of doing that.
from stepfunctions.steps.states import Task

lambda_task = Task(resource='your_lambda_arn')

agree, it is possible.
what I would like to improve is clarity of API. using Task for prefered way to call Lambda and LambdaTask for the secondary is not exactly clear

It is a design decision we had made to promote the usage of the new Lambda Invoke, this pattern is more consistent with other service integrations. The old way of using Lambda by putting the LambdaArn into the Resource field has its own limitation, it does not allow the customer to specify the Parameters field.

As mentioned in https://github.com/aws/aws-step-functions-data-science-sdk-python/issues/50, we would not want to update the LambdaStep for this change, I will close the PR.

authors: shunjd tvaroska


text:
Updated EKS-D to latest for all versions

Issue #, if available:
closes https://github.com/aws/eks-distro/issues/1464
Description of changes:

Updated to EKS-D 1.21-21, 1.22-13, 1.23-8, 1.24-3

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/hold
release is out and this can be unhold/merged whenever you want it

/cherry-pick release-0.12

/approve

Leave hold please, I'll merge when ready. Testing some template importing

/retest

source: gharchive/pull-request

authors: jaxesn kschumy

repo: aws/eks-anywhere-build-tooling


text:
upgrade cilium to 1.13.9

Issue #, if available:
5030
Description of changes:
upgrade cilium to 1.13.9
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/approve

/approve


text:
upload tink artifacts to the s3 bucket

Issue #, if available:
Description of changes:
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/lgtm
/approve


text:
Update docs with cgroups note

Description of changes: The requirement to downgrade cgroups from v2 to v1 in some versions of Ubuntu was removed in EKS Anywhere v0.16. However, those running v0.15 or earlier still need to know about this issue. So this PR adds the notes and troubleshooting tip back in, noting that the issue only applies to v0.15 and earlier.
This replaces PR #6042.

/lgtm

/lgtm

/cherry-pick release-0.17

/cherry-pick release-0.17

repo: aws/eks-anywhere


text:
Update EKS-A version in brew to v0.17.4

Issue #, if available:
Description of changes:
Testing (if applicable):
Documentation added/planned (if applicable):
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/approved

/approve
/cherry-pick release-0.17


text:
[PR BOT] Update base image tag file(s)

Issue #, if available:
Description of changes:
One last issue with the pr opening code, but the builds all succeeded and this is the correct tag file update.  Will fix the pr code tomorrow.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/approve

repo: aws/eks-distro-build-tooling


text:
add docker tars to server.tgz for k/k

Issue #, if available:
Description of changes:
While looking at using the new kind build-image support Dims added: https://github.com/kubernetes-sigs/kind/commit/e29f332cbb2412d683e357b4cff5cdbe9b906b16 I noticed there are a couple differences from the eks-d server.tgz and the upstream:

the 4 server component images are included next to the bins themselves
these image tars are in docker format instead of oci
kubectl is included

source.tgz is also included, but that is not needed for the kind flow i was testing so i have not added that to the server.tgz.
Per the buildkit docs the docker format is still compat with oci so in the cases where the existing image was being used, for example in the eks-a image build process, they should still import just fine. I verified this using the kind build, which imports it the same way using ctr.
The final server.tgz will be bigger, but I think this is a reasonable change to match upstream and make downstream usage of kind in eks-a simpler and more standard.
Testing:
I have built the kube components and tgz locally and created a kind image from it. I can also test once this is built in prow against the postsubmit artifacts.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/lgtm

/approve

authors: bhavi-koduru jaxesn

repo: aws/eks-distro


text:
Updated docs and created release annoucement for 1.18-4

⚠ This PR should be merged after the prod changes for 1.18-4 ⚠

Purpose

Update existing docs and create new docs for 1.18-4 release
Trigger release announcement for 1.18-4

Description of changes:

README.md

Updated all instances of kubernetes-1-18-eks-3 to kubernetes-1-18-eks-4

** docs/contents/index.md**

Refactored Release Version Dependencies list to make the different versions clearer
Added v1-18-eks-4 to the list

Created docs directory for 1.18-4 (docs/contents/releases/1-18/4/) and the following docs:

CHANGELOG-v1-18-eks-4.md

Summary of changes (new Kubernetes version, upgraded AL2, and patches changes)

index.md, which was copied-and-pasted from 1.18-3 with the following changes:

Updated text in non-CRD portion to be for 1.18-4
In CRD potion, made the following changes to the uri fields:

3 -> 4 in distro.eks.amazonaws.com/kubernetes-1-18/releases/4/[...]
1.18.9 -> 1.18.16 in distro.eks.amazonaws.com/[...]/kubernetes/v1.18.16
eks-1-18-3 -> ks-1-18-4 in public.ecr.aws/eks-distro/[...]-eks-1-18-4

release-announcement.txt

Very brief summary of release and the usual message of where to get it

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/hold
until prod changes

/unhold


text:
Export alias neither is suppported nor throws error when building

:bug: Bug Report
General Information

JSII Version: any
Platform:     any

What is the problem?
export * as foo from './foo';

In subdir foo are some exported classes.
This compiles w/o errors, though the generated packages do not contain any contents from foo.
My assumption here is that an export alias (or whatever that as foo part is called) simply is not supported.

This ought to be fully supported and is considered a submodule in the jsii type model. We even test this:
https://github.com/aws/jsii/blob/5701e4e4116e10ffacafa4930c0db23972f7e0e4/packages/jsii-calc/lib/index.ts#L10
At this stage I'ml going to be venturing there is something else causing problems that the compiler fails to check for... May I ask to see the code you have which causes this, or a minimal reproduction?

Sorry for the very late reply. I didn't work on this piece until now.
The behavior seems to have changed in the meantime. It no longer passes without errors. Instead it throws this error:

Exported APIs cannot use un-exported type "something.Something"

So I think this is kind of the same problem as #1818
In the submodule I have
export { Something } from './something.ts'

When I change this to
export * from './something.ts'

it works. Though the only exported item in something.ts is Something...
I tried to produce a minimal example repo, but I failed. So I think you're also correct in the assumption that this is somehow caused by or in combination with something else.
I'll point you to the most complex example possible instead. 😈 This is about line 2 in this file. I want to change it to:
export * as statement from './generated';

In the file ./generated/index.ts you see I have all the exports defined as
export { A4b } from "./alexaforbusiness";
export { ExecuteApi } from "./amazonapigateway";
export { Appflow } from "./amazonappflow";
...

jsii stops complainig when I change them all to:
export * from "./alexaforbusiness";
export * from "./amazonapigateway";
export * from "./amazonappflow";
...

Pinging @RomainMuller / @SomayaB Think you won't see this, because it's closed.

authors: RomainMuller udondan

repo: aws/jsii


text:
remove request size limit correctly

Description of changes:
remove request size limit correctly
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Codecov Report

Merging #120 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #120   +/-   ##
=======================================
  Coverage   96.32%   96.32%
=======================================
  Files          18       18
  Lines         708      708
  Branches       53       53
=======================================
  Hits          682      682
  Misses         17       17
  Partials        9        9

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b2bd7c7...42389a1. Read the comment docs.

authors: codecov-io jesterhazy

repo: aws/sagemaker-containers


text:
2024-11-07_01-34-50_master-next_aws-cli

Automatically created.

Backport failed for kirkstone-next, because it was unable to cherry-pick the commit(s).
Please cherry-pick the changes locally and resolve any conflicts.
git fetch origin kirkstone-next
git worktree add -d .worktree/backport-10277-to-kirkstone-next origin/kirkstone-next
cd .worktree/backport-10277-to-kirkstone-next
git switch --create backport-10277-to-kirkstone-next
git cherry-pick -x 5affad1c7fa736650be0c9248eb4b907266753e7

Backport failed for scarthgap-next, because it was unable to cherry-pick the commit(s).
Please cherry-pick the changes locally and resolve any conflicts.
git fetch origin scarthgap-next
git worktree add -d .worktree/backport-10277-to-scarthgap-next origin/scarthgap-next
cd .worktree/backport-10277-to-scarthgap-next
git switch --create backport-10277-to-scarthgap-next
git cherry-pick -x 5affad1c7fa736650be0c9248eb4b907266753e7

Backport failed for styhead-next, because it was unable to cherry-pick the commit(s).
Please cherry-pick the changes locally and resolve any conflicts.
git fetch origin styhead-next
git worktree add -d .worktree/backport-10277-to-styhead-next origin/styhead-next
cd .worktree/backport-10277-to-styhead-next
git switch --create backport-10277-to-styhead-next
git cherry-pick -x 5affad1c7fa736650be0c9248eb4b907266753e7

license: MIT


text:
aws-cloudfront needs love

We should redesign the API for the aws-cloudfront construct library. There's a lot we can do in order to make it friendlier, less error-prone and less flat and declarative. At the moment, it's basically "leaking" the low level API provided by CloudFormation.
Furthermore, it would be awesome if one could use Assets to point to a local directory that includes the web distribution content and have the toolkit do the work of uploading it to S3 and all that.

At a minimum I would recommend making the Cloudfront construct aware of other constructs such as:

elb.LoadBalancer
elbv2.ApplicationLoadBalancer
elbv2.NetworkLoadBalancer
apiGateway.LambdaRestApi
ecs.LoadBalancedFargateService (add the LB created by this construct)
ecs.LoadBalancedEc2Service (add the LB created by this construct)
ecs.FargateService and ecs.Ec2Service (lookup the attached LB)
s3.Bucket

CloudFront construct should be aware of how to get the domain name / address and potentially even the port from each construct and auto configure the origin details
I imagine an interface like:
const dist = new cloudfront.CloudFrontWebDistribution(this, 'WebsiteDistribution');

const ecsOrigin = dist.addOrigin(this, {
   target: myEcsServiceConstruct,
   port: 3000
});

const apiOrigin = dist.addOrigin(this, {
   target: myApiGatewayConstruct
});

dist.addBehavior(this, {
   path: '*',
   origin: ecsOrigin
   default: true
});

dist.addBehavior(this, {
   path: 'api*',
   origin: apiOrigin
   default: true
});

Hi @eladb, @PaulMaddox, @mindstorms6, @nathanpeck
I'm trying to add policy statements to my bucket, but it fails with the message Invalid principal in policy.
Any hint on how to reference the OID arn correctly?
    const originAccessIdentity = new cloudfront.CfnCloudFrontOriginAccessIdentity(
      this,
      'PublicSiteOAI',
      {
        cloudFrontOriginAccessIdentityConfig: {
          comment: 'OAI for adac-public-site-bucket'
        }
      }
    );

    const bucket = new s3.Bucket(this, 'Bucket', {
      blockPublicAccess: s3.BlockPublicAccess.BlockAll,
      bucketName: 'public-site-bucket'
    });

    const distribution = new cloudfront.CloudFrontWebDistribution(
      this,
      'Cloudfront',
      {
        comment: 'Public Site Cloudfront',
        originConfigs: [
          {
            behaviors: [{ isDefaultBehavior: true }],
            s3OriginSource: {
              originAccessIdentity,
              s3BucketSource: bucket
            }
          }
        ]
      }
    );

    const listBucketPolicy = new iam.PolicyStatement()
      .addAction('s3:ListBucket')
      .allow()
      .addResource(`arn:aws:s3:::${distribution.domainName}`)
      .addAwsPrincipal(originAccessIdentity.cloudFrontOriginAccessIdentityId);
    listBucketPolicy.sid = 'bucket_policy_site_root';
    bucket.addToResourcePolicy(listBucketPolicy);

    const getObjectPolicy = new iam.PolicyStatement()
      .addAction('s3:GetObject')
      .allow()
      .addResource(`arn:aws:s3:::${distribution.domainName}/*`)
      .addAwsPrincipal(originAccessIdentity.cloudFrontOriginAccessIdentityId);
    getObjectPolicy.sid = 'bucket_policy_site_all';
    bucket.addToResourcePolicy(getObjectPolicy);

authors: KnisterPeter eladb nathanpeck

repo: awslabs/aws-cdk


text:
args.region is not used for lambda client

Hi,
Currently, args.region is not used when a lambda client is created.
Therefore a lambda function in default environmental region is used.
$ scheduler-cli describe-periods --stack TestSchedule --region us-east-1
$ An error occurred (ResourceNotFoundException) when calling the Invoke operation: Function not found: arn:aws:lambda:ap-northeast-1:xxxxxxxxxx:function:TestSchedule-InstanceSchedulerMain

Thanks will add this as parameter in next update

It has been covered in v1.3.

These changes were incorporated into the latest release.

authors: arieleeuw georgebearden satoshihirose tapughose

repo: awslabs/aws-instance-scheduler


text:
Adding new account does not work as Amplify bucket is empty

Describe the bug
I am trying to add a new account to import resources from. This is the parent account of the AWS organization and Perspective is deployed in a sub-account. However, i can neither view the template or deploy the template in the step for beginning the Cloud Formation stack deployment into accounts you want to from. This is because the amplify bucket from which these templates are supposed to be deployed is completely empty.
To Reproduce
Steps to reproduce the behavior:

Go to 'Account and Regions' in the left menu.
Click on 'Import' tab.
Enter 'account number' and select 'region' for the region you want to import.
Click on 'Import' button.
Click on 'view template' button.
You get error message

NoSuchKey
The specified key does not exist.
public/cfn/import-templates/accounts/5b3674e3eb1129c175d1b591322c361ad24c856f.template
77DFBD373653757A
Lhrl+tjubVkk2mqZ/iCg/+ZMxsuA5b/Nvpn1IwnDw9oBg/NKUo1ZA+lwib5xFBR0HmT7V7EpGJM=

Expected behavior
I should be able to view the cloud formation template that needs to be deployed into my account that i am trying to add into Perspective.
Screenshots
If applicable, add screenshots to help explain your problem.
Browser (please complete the following information):

Name Chrome
Version 85.0.4183.121

Additional context
Add any other context about the problem here.
The reason we are getting this error because even through the AWS Perspective launch cloudformation stack completed successfully, including all the sub-stacks, the amplify buckets has no contents inside. And referring to the documentation for Automated Deployment, it does not have any additional steps to perform to populate the amplify bucket.

I believe this is related to #8. We have fixed the issue in v1.0.1. If you are happy to delete the v1.0.0 stack and redeploy using the latest from here that will fix your issue. We understand that might be quite painful and are working on instructions to be able to upgrade from v1.0.0. These will be available soon.
As a temporary workaround, the templates are stored in the DiscoveryBucket. You can download these replace the AccountId and AggregationRegion parameters with the account Dd and Region you deployed Perspective in. Then you can deploy this in the account and Region you are importing.

I believe this is related to #8. We have fixed the issue in v1.0.1. If you are happy to delete the v1.0.0 stack and redeploy using the latest from here that will fix your issue. We understand that might be quite painful and are working on instructions to be able to upgrade from v1.0.0. These will be available soon.
As a temporary workaround, the templates are stored in the DiscoveryBucket. You can download these replace the AccountId and AggregationRegion parameters with the account Dd and Region you deployed Perspective in. Then you can deploy this in the account and Region you are importing.

I will undeploy and redeploy. However, in this case the documentation link for the launch template should also be updated constantly to refer to the correct version of the cloudformation template.

I've read trough the issue #8 and this is not the same error. This error occurs because the amplify bucket is empty. If the step to enable "Public Access to Amplify Bucket" is a mandatory step, then we should make sure that it is documented appropriately as not being optional or provide the steps for a manual workaround if the optional step cannot be performed.

+1

Please create a new issue tagged with v1.1.0 if you are still experiencing this issue.

repo: awslabs/aws-perspective


text:
PerspectiveCostLamba JavaScript heap out of memory

Describe the bug
When working with large CUR reports the PerspectiveCostLamba throws a fatal error.
"FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory"
To Reproduce
We have a very large AWS organization with consolidated billing.
This means all Cost and Usage Reports are sent from a single account and thus are much larger then a single account CUR.
The PerspectiveCostLamba is able to unzip all CUR reports.
The newly invoked Lambda function which then parses the unzipped .csv file seems to fail on Large CUR reports.
The Lamba is configured by default to retry 2 times, this leads to 2 subsequent lambda invocations who fail due to an "AccessDenied" exception, since the first try fails after reading and deleting the unzipped .csv CUR report
Smaller CUR reports of around 40MB run succesfully.
Our larger CUR reports are somewhere between 100MB -150MB and always fail due to insufficient memory.
Expected behavior
PerspectiveCostLamba is able to parse the CUR files without throwing fatal error.
Screenshots
N/A
Browser (please complete the following information):
N/A
Additional context
Lambda log snippet with stack trace.
log-events-viewer-result.txt

Thanks for this. I have a feeling that lambda is probably not going to be sufficient for these very large reports. We are looking at improvements to the whole costing piece in a future release, most likely involving step functions.

I have a somewhat related questions: Is it supported to just have one CUR from the consolidated billing account covering all connected accounts instead of setting up one CUR for each account? This would save as a lot of work, having a 3 digit number of accounts.

#2

authors: ConnorKirk oli99sc svozza vaneyjo


text:
fix: Remove tests from .whl file (bdist_wheel)

Issue #, if available:
N/A
Description of changes:
Python wheels, .whl files, produced through python setup.py bdist_wheel do not read or understand MANIFEST.in files. Since we use this file to remove tests modules from the dists, tests end up in the .whl files that get uploaded to PyPi. This PR updates the exclude parameter in find_packages to exclude all test.* modules, following setuptools documentation.
Checklist:

[ ] Write Design Document (Do I need to write a design document?)
[ ] Write unit tests
[ ] Write/update functional tests
[ ] Write/update integration tests
[ ] make pr passes
[ ] Write documentation

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Hi, I don't totally understand this change, so I'm wondering if it addresses the problem I'm having right now. I just realized that when I do sam build -u somehow some Python unit tests that don't belong to me are getting put into my project's tests directory. They don't have any copyright notices or even code comments (shame, shame) but since they're importing things like serverlessrepo.parser I'm guessing they belong to SAM. True?

authors: jfuss tbradley-sans

repo: awslabs/aws-sam-cli


text:
Broken link to docs/stability.md

Describe the issue
The Getting Started guide includes a broken link to a stability guide. It's not obvious where this documentation has moved.
Links
https://github.com/awslabs/aws-sdk-kotlin/blob/main/docs/GettingStarted.md
https://github.com/awslabs/aws-sdk-kotlin/blob/main/docs/stability.md

The stability guide/information was merged into our versioning docs here. I've put up a PR for fixing the link.
Thanks for pointing this out.

authors: aajtodd zachary-etelie

repo: awslabs/aws-sdk-kotlin


text:
[request]: Support presigning for all S3 operations

A note for the community

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue, please leave a comment

Tell us about your request
All S3 requests should be presignable—this is the behavior of the Ruby SDK.
Tell us about the problem you're trying to solve.
Make a request to S3 from a context where the credentials are not directly available
Are you currently working around this issue?
routing requests through the server
Additional context
No response

Are there any plans when this will be done?
I am interested in pre-signing the ListObjectsV2 operation with a fixed prefix, will this be possible?

authors: rcoh stefanhoelzl

repo: awslabs/aws-sdk-rust


text:
Version 2.53.0 - It is removing the permissions granted by other resources to StateMachines

Since I have updated the dependencies in my project to v2.53.0 I am having issues when I use grant permissions.
Reproduction Steps
const definition = new sfnTasks.CallAwsService(this, 'Copy S3 object', {
  service: 's3',
  action: 'copyObject',
  iamResources: [
    this.sourceBucket.bucketArn,
    this.destinationBucket.bucketArn,
  ],
  parameters: {
    CopySource: sfn.JsonPath.format(
      '{}/{}',
      sfn.JsonPath.stringAt('$.sourceBucketName'),
      sfn.JsonPath.stringAt('$.sourceObjectKey')
    ),
    Bucket: this.destinationBucket.bucketName,
    Key: sfn.JsonPath.format(
      '{}/{}',
      sfn.JsonPath.stringAt('$.destinationFolder'),
      sfn.JsonPath.stringAt('$.sourceObjectKey')
    ),
  },
  resultPath: sfn.JsonPath.DISCARD,
});

const { stateMachine } = new S3ToStepfunctions(this, 'MockedMediaPipeline', {
  deployCloudTrail: false,
  createCloudWatchAlarms: false,
  stateMachineProps: {
    definition,
  },
  existingBucketObj: this.sourceBucket,
});

this.sourceBucket.grantRead(stateMachine);
this.destinationBucket.grantWrite(stateMachine);

Error Log
These are the differences, now my deployment is missing the policies on the right hand size of the screenshot

Environment

CDK CLI Version      : v2.128.0
CDK Framework Version: v2.128.0
AWS Solutions Constructs Version     : v2.53.0
OS                   : MacOS
Language             : Typescript

Other

This is :bug: Bug Report

We'll take a look. Can you tell me more about sourceBucket and originBucket? Are they created in this stack or imported?

Differences

Does the sample app perform correctly for Solutions Constructs 2.52.0? When I launch it, it doesn't appear to work. The eventBridgeEnabled property is not set on the source bucket, so the S3 events are not being sent to EventBridge which would prevent EventBridge from kicking off the state machine.

When I change the source bucket instantiation to this, the state machine executes:
    const sourceBucket = new s3.Bucket(this, 'SourceBucket', {
        eventBridgeEnabled: true
      });

(this is independent of the altered IAM policy, it's just something I found while digging deeply into the issue).

(sorry, shouldn't have closed - clicked the wrong button)

2.54.0 should fix the issue with the State Machine policy not accepting/retaining the bucket permissions.

When I change the source bucket instantiation to this, the state machine executes:
    const sourceBucket = new s3.Bucket(this, 'SourceBucket', {
        eventBridgeEnabled: true
      });

(this is independent of the altered IAM policy, it's just something I found while digging deeply into the issue).

I haven't been looking at this, sorry for the late response.
The point of this issue is IAM policies, not the entire thing working. I could have used any X to StepFunction construct, the point was the missing IAM Policies for the StepFunction instance to invoke other services.

2.54.0 should fix the issue with the State Machine policy not accepting/retaining the bucket permissions.

I'll take a look, thanks 😄

authors: biffgaut orekav

repo: awslabs/aws-solutions-constructs


text:
create instance profile when using a provided role

Issue #, if available:
addresses issue #600
Description of changes:
Creates a new instance profile container when a custom IAM role is provided.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Tested with a custom ec2_iam_role

authors: UnquietCode sean-smith

repo: awslabs/cfncluster


text:
Add profile support when creating new projects

Fixes #28

@jamesls - Changed to only pass a profile param to __init__, rebased and squashed.

Thanks!

authors: JamieCressey jamesls

repo: awslabs/chalice


text:
Explore Vite vs Next.js

Green Boost currently uses Vite as it's front-end tooling framework. Vite increases dev productivity being fast and easy to work with. However, it's not built directly for React. It supports React well, but Next.js (purpose built for React) may be a better option. With the recent release of Next.js 12 using swc (instead of webpack) it likely makes sense for GB to use Next.js instead of Vite.

Related: https://github.com/awslabs/green-boost/discussions/214

repo: awslabs/green-boost


text:
[uniffi] Setup CI for the Kotlin code

Issues:
Resolves #103.
Description of changes:
This sets up a simple CI pipeline for the generated Kotlin code. This would have allowed us to find and fix #101 and mozilla/uniffi-rs#2032 sooner.
Call-outs:
Note that this switches the UniFFI dependency to my clone: the PR that fixes the Exception inheritance loop is not yet merged.
Testing:
I still need to update the CI a bit to make the tests pass in all configurations.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 and MIT license.

Hi @mulmarta and @tomleavy, any chance that we could configure the workflow with Require approval for first-time contributors? My guess is that the setting is set to Require approval for all outside collaborators right now.
Since I'm no longer a first-time contributor to this repository, it should let the workflows run immediately when I create a PR — instead of having one of you approve them first.

We tried after your first contributions. If I understand correctly, this is a per-organization and not a per-repo setting (as your link suggests) and any settings for awslabs is beyond our control.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 89.51%. Comparing base (23a59ba) to head (9a9ee28).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #133   +/-   ##
=======================================
  Coverage   89.51%   89.51%
=======================================
  Files         173      173
  Lines       31450    31450
=======================================
  Hits        28151    28151
  Misses       3299     3299

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

We tried after your first contributions. If I understand correctly, this is a per-organization and not a per-repo setting (as your link suggests) and any settings for awslabs is beyond our control.

I see! Thanks for trying :smile: You've been super responsive on the PRs, so it's not a big deal for me to wait for the tests to trigger.

Ah, the kotlin-exception-loop branch was missing in my clone... because mozilla/uniffi-rs#2032 was merged yesterday. Let me update this PR to use the uniffi-rs repository instead then!

authors: codecov-commenter mgeisler mulmarta

repo: awslabs/mls-rs


text:
[Spike] Internationalization support using react-i18next

Description
Do not merge this PR, I just created it to discuss the implementation
Installed the react-i18next libary to add translation support and translated the Home string that says Last updated to Spanish and English. The default language will be selected based on your browser preference.
Testing
Ran the unit tests. Deployed to my personal environment.
https://fdingler.badger.wwps.aws.dev

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Closing as it was just a spike.

Closing as it was just a spike.

repo: awslabs/performance-dashboard-on-aws


text:
Rfc30/http

Motivation and Context
Part of RFC30.
Implements relevant logic for http/eventstream crate.
Description

add serde on external-type
add serde-serialize/serde-deserialize

Testing
Testing is not implemented yet.
Checklist
CHANGELOG is updated once all the PR is approved.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@rcoh
https://github.com/awslabs/smithy-rs/pull/1944#discussion_r1048766817
About this one, how do you think that it should handled?

yeah I agree with @jdisanti — it's a two-way-door to add ser/de in the future. I think it's best to hold off on this and consider adding it at the end

I don't really have a strong opinion about event streams to be honest.
I looked up the crates.io but sdk that uses event-stream got less than 3000 all-time downloads, which makes it difficult to justify implementing something that is potentially harmful.
https://github.com/awslabs/smithy-rs/pull/2198#issuecomment-1397586646
You could create a new struct which copies it's twin without the field for the event stream, if there is a demand.
e.g.
struct OriginalStruct {
   event_stream: EventStreamType,
   other_field: u64
}

#[derive(serde::Serialize)]
struct Twin {
   other_field: u64
}

@Velfi
@jdisanti
@rcoh
Can you guys check it out?
https://github.com/awslabs/smithy-rs/pull/2243

authors: rcoh thomas-k-cameron

repo: awslabs/smithy-rs


text:
Input Interrupt logic is broken

Misreading of data sheet caused miscoding of input interrupt functionality.

Fixed.  Adjust arbitrary Gpio Debouce Timeout


text:
refactor(its): change trusted address to trusted chain

AXE-6872

Codecov Report
Attention: Patch coverage is 96.87500% with 1 line in your changes missing coverage. Please review.

Project coverage is 92.70%. Comparing base (0b2d38a) to head (81e1f06).

Files with missing lines
Patch %
Lines

contracts/interchain-token-service/src/contract.rs
96.42%
1 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #110      +/-   ##
==========================================
- Coverage   92.72%   92.70%   -0.03%
==========================================
  Files          46       46
  Lines        2997     2988       -9
==========================================
- Hits         2779     2770       -9
  Misses        218      218

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

authors: AttissNgo codecov-commenter

repo: axelarnetwork/axelar-cgp-soroban


text:
feat: enable s3 filtering using prefixes

Fixes #8

good job!

authors: a-khaledf seiflotfy

repo: axiomhq/axiom-cloudfront-lambda


text:
XIVY-12548 Add browsertype to accept

I could adjust the 'accept' to include the browser type, and based on that, along with a flag to ensure that it only affects the ScriptArea, I was able to achieve the "import functionality."
Unfortunately, all the tests of the components with a combobox are failing. As you can see, I haven't made any changes to the combobox or anything similar, so I have no idea why this is suddenly failing, especially when it didn't fail in the PR where I added the editor to the combobox. What should I do now? Should I revert the macro functionality of the combobox, or should I mock the whole thing (as you briefly mentioned before)? If the mock approach is appropriate, we should have a quick call again, as I didn't quite understand how that works.

Okay, never mind, I just found out why the error occurs. To make sure in the useCodeEditor that the line "Import XY" always starts on the first line of the editor, I currently make "range: new monaco.Range(1, 1, 1, 1)" and I think here is the problem

repo: axonivy/inscription-client


text:
aya-log panics when logging [u8;6] without format argument

Using aya-log to log a [u8;6] compiles and leads to a runtime panic if no format argument (e.g. :mac) is provided.
Example:
info!(&ctx, "received a packet: {}", [0_u8;6]);
Error:
thread 'tokio-runtime-worker' panicked at 'called `Result::unwrap()` on an `Err` value: ()', /home/arctic-alpaca/.cargo/registry/src/github.com-1ecc6299db9ec823/aya-log-0.1.13/src/lib.rs:110:45
Used a newly created xdp project from the template to reproduce.

@vadorovsky when you get a chance could you look into this?

@alessandrod @vadorovsky  Hi, I think I understand why the error was thrown. Firstly, the write function is implemented for the [u8; 6] type, as shown here:
https://github.com/aya-rs/aya/blob/23ce42da6843b04fae4de40f4471565491b51109/aya-log-common/src/lib.rs#L191-L195
This writes the type of the field and some other information, which is then retrieved when formatting the string, as shown here:
https://github.com/aya-rs/aya/blob/23ce42da6843b04fae4de40f4471565491b51109/aya-log/src/lib.rs#L463-L465
Because {} is used in info!(&ctx, "received a packet: {}", [0_u8;6]);, the last_hint variable is None, and an error is thrown when formatting, as shown here:
https://github.com/aya-rs/aya/blob/23ce42da6843b04fae4de40f4471565491b51109/aya-log/src/lib.rs#L217-L231
And currently, it seems that only {:mac} or {:MAC} can be used for output. I think we could add a DebugFormatter with  {:?} format syntax or modify the DefaultFormatter to support this.

Please give me some feedback. If it would be helpful, I would be happy to take on the responsibility of addressing this issue.

I think the issue is here https://github.com/aya-rs/aya/blob/23ce42da6843b04fae4de40f4471565491b51109/aya-log/src/lib.rs#L115 ? We shouldn't unwrap

Indeed, unwrap is not appropriate here, but it is not the root cause of this bug because Err has already occurred. As I mentioned in my previous response, the error thrown by info!(&ctx, "received a packet: {}", [0_u8;6]); is due to the lack of a defined DefaultFormatter processing logic for the [u8;6] type. Therefore, I think resolving this issue depends on whether code like info!("{}", [0_u8;6]) is supported.

If it is valid, then a DefaultFormatter  should be added for the [u8;6] type, for example:

impl Formatter<[u8; 6]> for DefaultFormatter; {
    fn format(v: [u8; 6]) -> String {
        // e.g.
        format!("{:?}", v)
    }
}

If it is invalid, then the timing of throwing the error should be moved to the proc macro. The error should be thrown during compile time rather than runtime.

Here is an example code that moves the error timing to compile time:
https://github.com/aya-rs/aya/compare/main...Hanaasagi:aya:example

@Hanaasagi that's right, thanks for debugging that!
I think we shouldn't support info!("{}", [0_u8;6]) - std formatter in Rust doesn't support it either:
error[[E0277]](https://doc.rust-lang.org/stable/error_codes/E0277.html): `[u8; 6]` doesn't implement `std::fmt::Display`
 --> src/main.rs:2:20
  |
2 |     println!("{}", [0u8; 6]);
  |                    ^^^^^^^^ `[u8; 6]` cannot be formatted with the default formatter

So I like your second option - moving the error to proc macro. Happy to see that you started working on it! Would you like to submit a pull request?

Sure, I will open a pull request to validate this in proc macro.

authors: Hanaasagi alessandrod arctic-alpaca vadorovsky

repo: aya-rs/aya


text:
My thoughts

很高兴认识你。
第一次看到你的博客就被吸引了。
然后魔改，却变成四不像的样子，ahoy~
我也喜欢了pandoc，强迫我学会DIY。
我根据文档加入了default.html进行更精细的调控。
比如文章末尾的时间，可以用$if(date)$(跳过了pandoc.yaml)直接生成
还有自动生成文章列表，我用shell实现的，不知道你的怎么做到的。
还有你的博客真的很漂亮，样式简约，元素很少，看起来很舒服，很钦佩你。也给我提供了好思路！
我的博客也在翻修中，我觉得不是很好，但我喜欢构建它的过程！

谢谢！你修改的部分很有创意，我也会借鉴你的做法～

谢谢！你修改的部分很有创意，我也会借鉴你的做法～

Yesterday I added a feature that about light mode, although before always dark style. Now now color looks a bit odd. It need to repair, next vacation I guess. Hack ¶∆¶

It will changed depended on time and system-setting.

自动生成文章列表我没有做，是手工修改的，因为我觉得手工修改也不是很麻烦 XD

Now I am considering dark mode, but bright pictures seem to be a difficulty.

Now I am considering dark mode, but bright pictures seem to be a difficulty.

You can employ filter: brightness(70%) to decline brightness


text:
Kosovo Iban failing

Hello,
Implementing the unit tests I have found that it seems that ngIBan is not working for kosovo:
Just checked this two values:
XK051212012345678906
Source: http://formvalidation.io/validators/iban/
XK051000000000000053
Source: http://www.letterofcredit.biz/Business-Identifier-Code-(BIC)-and-International-Bank-Account-Number-(IBAN).html
Checking in some third parties sites, in some of them I get an ok in some other an error, maybe kosovo has joined recently?
In the ng-iban country I cannot found Kosovo.

@brauliodiez added Kosovo

authors: brauliodiez juvenpajares

repo: ayosdev/ng-iban-custom

license: mit


text:
[API] Rename distributions module to soft_labelling

Module distributions have been renamed to soft_labelling, as discussed in https://github.com/ayrna/dlordinal/issues/39

Code examples in docstrings have to be updated.

repo: ayrna/dlordinal


text:
devfreq ff3000000.gpu couldn't update frequency transition information

Not getting display on most HDMI devices. On the one device I do have that it does work on, when using emulationstation, the output will jump back and forth between HDMI and DVI (in the same resolution) - and I see this error pop up sometimes, which I assume is related.
devfreq ff3000000.gpu couldn't update frequency transition information

Hey, @mrfixit3000. How this was fixed?

The main rule of anyone seeking help - if you solved the problem yourself, take the solution with you to the grave


text:
Adding Binary Search Program Using JAva

Adding Binary Search program using Java.

How should i add description now ? Should i close this pull request and do it all over again ??  I'm sorry i am new here.

Added description for the pull request .


text:
Eventual crash when reading a lot of chunk data quickly

This issue appears first in version 0.5.0.
Here's the stack and analysis from WinDbg.
Here's a somewhat incomplete minimal reproduction: main.rs and regions.
Note: You might need more region files to reproduce this, depending on how powerful your machine is.

I should also mention that this doesn't happen in dev/debug builds(presumably because it's too slow). It does happen in release builds with the following settings:
# Commenting 'strip' and uncommenting 'debug' still causes a crash
[profile.release]
strip = true
opt-level = 3
codegen-units = 1
lto = true
# debug = "full"

I can't seem to reproduce this on my computer using your files, and I'm not really sure as to what the issue could be.
Your stack trace says the error is happening in this copy_nonoverlapping, and the code is a little confusing but as far as I can tell it should all work properly (and Miri doesn't complain either).
You said the error starts happening in 0.5.0, though the main thing that version changed is making some functions private and adding some other convenience functions.
If it's not too much of a hassle, could you try bisecting the commits to pin down exactly the one that started causing the issue?
Either way, I have a decently big refactor of simdnbt I mostly wrote a while ago that significantly improves performance again (it still has lots of unsafe but it reduces the allocations at least), so I'm assuming that'll probably fix this issue once I finish it up.

That's a bummer. Though, I'm glad to hear about the future improvements in hopes they'll get rid of the issue.
I'll try bisecting and get back to you.

According to git bisect 1ab27e531e14035286b0fe20e9fbf9b4f5c88108 is the first bad commit, although it doesn't compile so it's either that one and/or the one after it(which does compile) - 927df9b33bb34fb04ac60ec0a11b7a420817c0eb.
The stack and analysis don't seem too different either :/

If you'd like to have another go at trying to reproduce this, here's my crate.
I'd also like to emphasize that you should try to feed it a lot of chunks, just so it can run for at least a few seconds.
Here's a fairly big world. You can also try other worlds - there doesn't seem to be a difference as long as they're big enough.
Feel free to ask for more info if needed.

Alright so as I was working on my other branch I realized it'd be a good idea to add a fuzzer and it turned out to be way more effective than I anticipated. I ran cargo-fuzz on the master branch too, and it does actually find an input that triggers a buffer overflow (which I believe is the same as this issue).
This is the input cargo-fuzz generates:
10, 0, 0, 9, 0, 0, 10, 2, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 0, 8, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 0, 8, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 0, 8, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0

(I ran it through cargo-fuzz's minifier though I'm sure it could be minified more). Tbh though, figuring out the root cause of this is probably more effort than it's worth since I'm going to merge my other branch that doesn't have the same issue soon.

That's pretty cool!
I was just about to share that from my own testing that your tape branch fixes this issue, so, well done!

repo: azalea-rs/simdnbt


text:
fix: convert array len to u32 before writing

Closes #12

Thank you, I really should've had a test for this.


text:
Add some tests for the boundary API endpoint

I think this covers the existing custom boundary functionality. Any suggestions for tests I missed?

It'd be good to have a test or two for get_union() and make_multipolygon()

Based on recent HunchLab headaches, I feel like this is a very good level of abstraction for tests. I like that, for instance, the general entities (user, in this case) make an appearance in the base class's setup and there's not too much magic with kwargs and the like

+1 for the existing tests. I think it would be good to add a test for a shapefile with invalid geometries that shouldn't validate, and also some tests for shapefiles that are valid but aren't what we're expecting, e.g., multiple layers, no prj file, etc.

Thanks for the comments.
@ddohler: Going to hold off, as due to a discussion with @jbranigan and @kshepard those functions may not be necessary anymore...there's a bunch of other stuff to add here.
@kshepard: Will add those other tests in round two once I've made the changes we discussed yesterday.

authors: CloudNiner ddohler kshepard moradology

repo: azavea/ashlar


text:
Delete geometries and features

It would be nice to be able to delete map unit polys.
This could be as simple as a delete button next to the reset and submit buttons.

Updated this feature to note that we should be able to delete all geometries (lines, points, polys).

What if user wants to delete a contact that is also the defining element of a polygon (I think we should delete the contact and the polygon, but NOT delete other contacts of said polygon).

When deleting a polygon, should the underlying contacts be deleted? (I think the answer is no, which will make the implementation of ^ easier)

We have been using just the ArcPro built-in delete functionality to delete geometries. I think this is the right approach, but one downside is we don't have the ability to 'cascade delete' a polygon when the defining contact lines are deleted. We should be able to catch any issues during the topo check. Would that work?

authors: NoisyFlowers aazaff mcamp1

repo: azgs/geomapmaker


text:
discourse: handle trailing slash on url

Just remove it or treat it as optional
This works https://discourse.haskell.org/t/an-opportunity-that-i-couldnt-pass-up/7485
This doesn't https://discourse.haskell.org/t/an-opportunity-that-i-couldnt-pass-up/7485/

Also handle trailing number for the comment.
I guess just drop it since this thing doesn't have comment to comment navigation :/
Example: https://discourse.haskell.org/t/haskell-ci-group-meeting-notes/7315/3

repo: azimut/cli-view


text:
Add context properties

In aim to facilite apache proxy on Azkaban, add a propertie web.context=/azkaban

Sorry I miss to use the var in Context ...

replace by https://github.com/azkaban/azkaban/pull/579

repo: azkaban/azkaban


text:
Exceptions/Errors are not caught/handled in any queue items beyond the first.

Specifically in nodejs (v0.12.7).  I'm using Q for promises, but I think the same will happen with others.
Some sample code:
var Q = require('q');
var Queue = require('promise-queue');
Queue.configure(function (handler) {
	var deferred = Q.defer();
	try {
		handler(deferred.resolve, deferred.reject, deferred.notify);
	} catch (e) {
		console.error(e);
		deferred.reject(e);
	}
	return deferred.promise;
});

var maxConcurrent = 1;
var maxQueue = Infinity;
var serialQueue = new Queue(maxConcurrent, maxQueue);

serialQueue.add(function() {
     throw new Error('error that is caught');
});

serialQueue.add(function() {
     throw new Error('error that is not caught');
});

My thought is that the first one works because the _dequeue is directly in the try/catch.  However on subsequent calls when the _dequeue is called from the then/reject the scope changes and it's not in the try/catch block anymore.
I was going to take a stab at a PR, but not sure the best way of handling it or if it's even possible without changing the interface.

Resolved in promise-queue@2.2.0

authors: EricByers azproduction

repo: azproduction/promise-queue


text:
Add support to Application Insights for Central Diagnostics Log Analytics Workspace

Is there an existing issue for this?

[X] I have searched the existing issues

Community Note

Please vote on this issue by adding a :thumbsup: reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description
The "azurerm_application_insights" module allows you to specify a Log Analytics workspace but not to use a central LAW defined as a diagnostics destination in the launchpad, for example the central_logs that sit within the management subscription in CAF.
This feature request is to add that support.
New or Affected Resource(s
azurerm_application_insights
Potential Configuration file
azurerm_application_insights = {
  webapp = {
    name                                = "example-appinsights-web"
    resource_group_key                  = "rg1"
    application_type                    = "web"
    log_analytics_workspace_destination = "central_logs"
  }
}

References
No response

I will submit a PR for this

repo: aztfmod/terraform-azurerm-caf


text:
Added reporting of required memory to a call of nxd_mqtt_client_message_get() with insufficient memory

If nxd_mqtt_client_message_get() is called with insufficient memory, then currently there is no option to find out how much memory is required to perform a second call as suggested on https://docs.microsoft.com/en-us/azure/rtos/netx-duo/netx-duo-mqtt/chapter3#nxd_mqtt_client_message_get
This little PR adds reporting of the actual memory required in actual_topic_length and actual_message_length.
A caller can then adopt the strategy to first call nxd_mqtt_client_message_get() with both topic_buffer_size and message_buffer_size set to 0 and then use the values reported in actual_topic_length and actual_message_length to allocate the exact amount of memory required to store the message data.

Thanks for the suggestion. We will review this requirement internally and get back to you.

I like also like to add to the discussion the fact that a message can only be removed from the queue if sufficient memory has been allocated. So one needs to know how much is needed in advance. I don't see much value in providing this information only after the call has succeeded as per current implementation. In that case I had enough memory and would not care about how much would have been needed, I overallocated already.
The concept of providing zero length buffers to gather the size before allocation is a common practise in the standard C library and in this case adds little overhead.

Are there any updates on this review? It is a very simple change and would make mqtt usage much easier and more memory efficient as we can optimise the memory allocations to match what is actually needed rather over allocate all the time and hope we allocated enough.

Sorry there is no updates recently. As for AzureRTOS convention, output parameters are meaningless when the return value is not successful. We will consider to add API to retrieve message length in the future.

Ok, I accept that you stay by your convention. Would you accept a PR for such an API call?

@TiejunMS Yes, that would be a workaround, thank you for the suggestion. But I rather like to see that in an official API than accessing internal data structured directly. Compared to a lot of other NetX methods, MQTT does not use zero-copy buffers and a user has to allocate a message buffer before retrieving the message. This buffer must be of sufficient size, so you must know how big the message will be BEFORE calling nxd_mqtt_client_message_get(). So a function like nxd_mqtt_client_message_size_get() would be very useful. Otherwise how do you make sure the buffer is large enough?

You are right. MQTT does not support zero-copy. It was designed to send telemetry with small amount of data. While developing Azure IoT Middleware, we noticed JSON is used in some cases. So we have supported zero copy in Azure IoT Middleware. If your devices are connecting to Azure IoTHub, I would suggest you to use Azure IoT Middleware instead of MQTT directly. That is our first priority to simplify the user experience of connecting to Azure IoTHub.
See more details here: https://github.com/azure-rtos/netxduo/tree/master/addons/azure_iot

Unfortunately we have to support vanilla MQTT and digest arbitrary sized messages. So sensible and adaptive buffer sizing is important, we are on an embedded device and memory is finite.
I am right now in the process of implementing a sample nxd_mqtt_client_message_size_get() function. Once finished, tested and found workable, I publish it as a PR and will then close this PR hoping that one day it will make it into the official MQTT API.

Hello @TiejunMS,
here is a first implementation of a _nxd_mqtt_client_message_length_get() function to retrieve topic length and message length:
UINT _nxd_mqtt_client_message_length_get(NXD_MQTT_CLIENT *client_ptr, UINT *topic_length_ptr, UINT *message_length_ptr)
{

UINT                status;
NX_PACKET          *packet_ptr;
ULONG               topic_offset;
USHORT              topic_length;
ULONG               message_offset;
ULONG               message_length;
UINT                message_receive_queue_depth;

    tx_mutex_get(client_ptr -> nxd_mqtt_client_mutex_ptr, NX_WAIT_FOREVER);
    message_receive_queue_depth = client_ptr -> message_receive_queue_depth;
    packet_ptr = client_ptr -> message_receive_queue_head;

    while (message_receive_queue_depth)
    {
        status = _nxd_mqtt_process_publish_packet(packet_ptr, &topic_offset, &topic_length, &message_offset, &message_length);
        if (status == NXD_MQTT_SUCCESS)
        {
            {
                *topic_length_ptr = topic_length;
                *message_length_ptr = message_length;
                tx_mutex_put(client_ptr -> nxd_mqtt_client_mutex_ptr);
                return(NXD_MQTT_SUCCESS);
            }
        }

        packet_ptr = packet_ptr -> nx_packet_queue_next;
        message_receive_queue_depth--;
    }

    tx_mutex_put(client_ptr -> nxd_mqtt_client_mutex_ptr);
    return(NXD_MQTT_NO_MESSAGE);
}

Hello @hwmaier , your code looks good to me. Thanks for your contribution! We will try to incorporate this feature in the release this year. Before that, could you maintain it in your user application?

@TiejunMS Thank and sure I keep in the mean time my private extension.
While dealing with the memory allocation and the packet queue I also noticed one other potential issue which probably needs attention. Not sure if this is worth a separate issue report, but I'll mention it here for now.
If the application receives a large enough MQTT message which it fails to provision enough memory for, those messages will remain forever in the queue and the device won't be able to process MQTT any more. This leaves a device open to very simple DOS attacks by sending MQTT with large payloads. I believe there should be an official mechanism to clear the queue if _nxd_mqtt_client_message_get returns with NXD_MQTT_INSUFFICIENT_BUFFER_SPACE.

I would not treat large MQTT data as DOS attacks. Same situation applies to all TCP/UDP packets. For network security, we depends on TLS session to provide certain level of protection. But I agree with you that it is convenient to allow user application to skip copying specific MQTT data. One possible solution in the future is, we can allow user application to receive MQTT packet directly instead of copying data.

Ordinary TCP/UDP packets can simply be released if they don't comply by content or size. The TLS protects only between device and broker. MQTT data can travel between brokers and sometimes there is no control where it comes from. So the application should always be able to sanitise the data and reject any data it does not accept without deadlocking. Like you suggested, a simple API to reject the packets and release them would suffice.

authors: TiejunMS hwmaier yuxin-azrtos

repo: azure-rtos/netxduo


text:
Updated README to reflect docker-compose up

Docker compose command had a typo, adjusted to docker-compose up.

Thx :+1:


text:
Changes due to Willow/#113

This requires https://github.com/ba-st/Willow/pull/114 to be merged and released

Coverage remained the same at 97.981% when pulling a90fd1c0b4282d2aa5edfd720df0fa38a2e67baf on table_footer_support into 6afebd7df1d5387a69dee63dc40f08b7021f1626 on master.

authors: coveralls gcotelli

repo: ba-st/Willow-SemanticUI


text:
Add a parseExpression method

Hello,
I would like to use babylon as a fast expression parser to build an AST based code generator for the pug templating language. I was pointed to is-expression-babylon (https://github.com/pugjs/is-expression-babylon) that has a getExpression method.
But this module is now broken (I believe it broke after the migration to rollup).
Currently I use the code
var ast = babylon.parse('expr='+expr).program.body.pop().expression.right

it works but I suspect that this is not the most efficient way to simply parse an expression.
Would it be possible for someone with good knowledge of babylon internals to export an efficient parseExpression method ?
I could try an write a PR but would need directions. It is hard for me to know if what is-expression-babylon did is the most efficient use of babylon's internals.
Thanks for your help.

Try this:
import * as babylon from "babylon";
import traverse from "babel-traverse";

const ast = babylon.parse( 'expr='+exprfoo);
traverse(ast, {
    BinaryExpression(path){
        path.get('right') // log: exprfoo
    },
});

Personally I'd wrap it in parens and go with that, e.g.
babylon.parse('(' + expr + ')').program.body[0].expression

I'm somewhat hesitant to add an API for this since it's easy enough to do with the existing one.

@SerkanSipahi thanks for the snippet. Nevertheless, babel-traverse adds serious slowness compared to the access to raw fields. I did a small benchmark and had around 130K ops/sec with my version (same for @loganfsmyth's version) versus 8K ops/sec using babel-traverse.
I am trying to use a subset of what babylon does, trying to see if there can be an entry point inside babylon that would improve the performance of building the AST of something we know is an expression.
@loganfsmyth your solution with "(..)" is maybe more robust than mine with "expr=.." I'll try that. I would have to revise my javascript 101 to make sure which pattern would always be parsed as an expression.
do you think I could get faster parsing with access to internals ? (skip the File and Program that I do not need, go directly to "start of an expression"). I need to parse a lot of small snippets which is why I want to avoid the cost of setup as much as i can.

And, another disadvantage with such a solution is that inputs like a)( will be parsed as a valid expression.

@TimothyGu indeed ! "=a" leads to an expression with "expr=.." so I guess none of these solutions are foolproof except if we really find a pattern that would force the expression parsing (your example made me realize that may not exist).

Seeing the code on https://github.com/pugjs/is-expression-babylon/blob/master/src/index.js and on https://github.com/babel/babylon/blob/master/src/parser/index.js, is-expression-babylon was close enough to the tokenizer to be a nice solution to the problem.
do you think that babylon could export the Parser or the Tokenizer so that we can sub-class it again ?
or would you agree with a PR that would allow for parsing specific sub-parts of js like is-expression-babylon was doing ? I believe the main idea was to force the parser into Expression mode and check that everything was parsed when leaving Expression mode :
  assertExpression() {
    this.nextToken();
    const expr = this.parseExpression();
    if (!this.match(tokTypes.eof)) {
      this.unexpected();
    }
    return expr;
  }
}

It would be great if we could have that again ! Feel free to comment on this and tell me what kind of PR has the best chance of beeing accepted (exporting the parser/tokenizer or a new assertExpression method)
Thanks !

Done in https://github.com/babel/babylon/issues/213

authors: SerkanSipahi TimothyGu hzoo jeromew loganfsmyth

repo: babel/babylon


text:
Disallow '' in password based login

Signed-off-by: Shameem Ahmed shameemahmed20apr2000@gmail.com
Description
This PR disallows '\' in password based login. '' is only allowed for windows based authentication.
It also disallows '\' in role and SQL user.
Update the server_principal and database_principal for type "U".
Issues Resolved
Task: BABEL-3847
Sub-task: BABEL-3828, BABEL-3844
Test Scenarios Covered

Use case based - Yes

Boundary conditions - Yes

Arbitrary inputs - Yes

Negative test cases - Yes

Minor version upgrade tests -

Major version upgrade tests -

Performance tests -

Tooling impact -

Client tests -

Check List

[ ] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is under the terms of the Apache 2.0 and PostgreSQL licenses, and grant any person obtaining a copy of the contribution permission to relicense all or a portion of my contribution to the PostgreSQL License solely to contribute all or a portion of my contribution to the PostgreSQL open source project.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

I think, we can now get rid of get_roleform_ext() safely. It was initially there to safe check the login whether it is indeed a win login. But we can not safely assume based on whether login contains ''. For example current use of get_roleform_ext is as follow:
/* Let DropRole handle missing roles */
					if (HeapTupleIsValid(tuple))
						roleform = (Form_pg_authid) GETSTRUCT(tuple);
					else
					{
						/* Supplied login name might be in windows format i.e, domain\login form */
						tuple = get_roleform_ext(role_name);
						if (HeapTupleIsValid(tuple))
						{
							roleform = (Form_pg_authid) GETSTRUCT(tuple);
							/*
							 * This means that provided login name is in windows format
							 * so let's update role_name with UPN format.
							 */
							role_name = pstrdup((roleform->rolname).data);
							pfree(rolspec->rolename);
							rolspec->rolename = role_name;
						}
						else
							continue;
					}

This can now be changed to
					if (HeapTupleIsValid(tuple))
						roleform = (Form_pg_authid) GETSTRUCT(tuple);
					else
					{
						/* Supplied login name might be in windows format i.e, domain\login form */
						if (strchr(role_name, '\') != NULL)
						{
							/*
							 * This means that provided login name is in windows format
							 * so let's update role_name with UPN format.
							 */
							role_name = convertToUPN(role_name);
							tuple = SearchSysCache1(AUTHNAME,
											PointerGetDatum(role_name));
							if (HeapTupleIsValid(tuple))
							{
								roleform = (Form_pg_authid) GETSTRUCT(tuple);
								pfree(rolspec->rolename);
								rolspec->rolename = role_name;
							}
							else
							{
								continue;
							}
						}
						else
						{
							continue;
						}
					}


text:
Possible bug in evaluation script

When evaluating e.g. for SST2, the trainer seems to incorrectly select the best model checkpoint to be from step 1000 when the eval accuracy clearly continues to improve in subsequent steps. This results in a lower final accuracy being reported at the end than the actual one the model achieves.
Below is the output of trainer.log_history and trainer.best_model_checkpoint
log_history=[
{'eval_loss': 0.43397894501686096, 'eval_accuracy': 0.8090550899505615, 'eval_f1': 0.8101761252446184, 'eval_runtime': 0.3408, 'eval_samples_per_second': 1490.483, 'eval_steps_per_second': 187.777, 'epoch': 0.25, 'step': 200},
{'eval_loss': 0.40459370613098145, 'eval_accuracy': 0.8307086825370789, 'eval_f1': 0.833976833976834, 'eval_runtime': 0.3147, 'eval_samples_per_second': 1614.033, 'eval_steps_per_second': 203.343, 'epoch': 0.51, 'step': 400},
{'eval_loss': 0.3854110538959503, 'eval_accuracy': 0.8444882035255432, 'eval_f1': 0.8483685220729367, 'eval_runtime': 0.3154, 'eval_samples_per_second': 1610.677, 'eval_steps_per_second': 202.92, 'epoch': 0.76, 'step': 600},
{'eval_loss': 0.36048293113708496, 'eval_accuracy': 0.8602362275123596, 'eval_f1': 0.8599605522682446, 'eval_runtime': 0.3237, 'eval_samples_per_second': 1569.322, 'eval_steps_per_second': 197.71, 'epoch': 1.01, 'step': 800},
{'eval_loss': 0.36621662974357605, 'eval_accuracy': 0.8582677245140076, 'eval_f1': 0.855421686746988, 'eval_runtime': 0.3169, 'eval_samples_per_second': 1603.248, 'eval_steps_per_second': 201.984, 'epoch': 1.27, 'step': 1000},
{'eval_loss': 0.36977794766426086, 'eval_accuracy': 0.8622047305107117, 'eval_f1': 0.8594377510040161, 'eval_runtime': 0.3405, 'eval_samples_per_second': 1491.796, 'eval_steps_per_second': 187.943, 'epoch': 1.52, 'step': 1200},
{'eval_loss': 0.4045640230178833, 'eval_accuracy': 0.8622047305107117, 'eval_f1': 0.866412213740458, 'eval_runtime': 0.3215, 'eval_samples_per_second': 1580.063, 'eval_steps_per_second': 199.063, 'epoch': 1.77, 'step': 1400},
{'eval_loss': 0.3876116871833801, 'eval_accuracy': 0.8681102395057678, 'eval_f1': 0.8667992047713717, 'eval_runtime': 0.3189, 'eval_samples_per_second': 1592.866, 'eval_steps_per_second': 200.676, 'epoch': 2.03, 'step': 1600},
{'eval_loss': 0.4254477918148041, 'eval_accuracy': 0.8543307185173035, 'eval_f1': 0.8587786259541984, 'eval_runtime': 0.3182, 'eval_samples_per_second': 1596.405, 'eval_steps_per_second': 201.122, 'epoch': 2.28, 'step': 1800}...
]

best_model_checkpoint='.../finetune/sst2/checkpoint-1000'

On a related note, even though training is configured to run for 10 epochs, it stops after 3000 steps which amounts to 3.8 epochs. There is no apparent reason why that should be considering that the eval accuracy steadily improves up until that point.

Looks like it's choosing the best model based on loss, rather than the accuracy/f1 metrics. It also looks like the save_steps argument defaults to 500 rather than the eval_every default of 200, which is why it doesn't use step 800; thank you for catching this!
This also explains why it stops at step 3000. It thinks that step 1000 is best because it achieves the lowest loss of any step thus far (out of the steps that are a multiple of 500, I think). Because the patience is 10, it will stop after (10*200=)2000 steps if no improvement over the best loss is found.
Will update once these are fixed. Thanks!

Okay, you're right. I added prefix "eval_" to the metric_for_best_model arguments (don't know why that would be necessary but it seems to help) and explicitly set save_steps = data_args.eval_every, and things look alright now.
Do the baseline results need to be updated accordingly?

Hi, is there any progress on solving this issue? Thanks.

authors: YovaKem aaronmueller

repo: babylm/evaluation-pipeline


text:
🛑 SM Service is down

In 88734de, SM Service (http://smservice.de) was down:

HTTP code: 0
Response time: 0 ms

Resolved: SM Service is back up in 8b08133.

repo: bachmannschumacher/upptime


text:
🛑 SM Service is down

In ad37163, SM Service (http://smservice.de) was down:

HTTP code: 0
Response time: 0 ms

Resolved: SM Service is back up in 11abb4c.


text:
🛑 SM Service is down

In dd7a8a7, SM Service (http://smservice.de) was down:

HTTP code: 0
Response time: 0 ms

Resolved: SM Service is back up in 1a97364.


text:
ModuleNotFoundError: No module named 'jsonlines'

fresh Python virtualenv -> pip install -r requirements.txt -> python3 transcribe.py -h -> ModuleNotFoundError: No module named 'jsonlines'

fix: pip install jsonlines, -> ModuleNotFoundError: No module named 'torchaudio'

I have updated the requirements. Thanks!

authors: Gldkslfmsd backspacetg


text:
Docs to newrelic plugin

🔖 Need
Having a start guide for using a new relic plugin on the Backstage portal is of utmost importance because it provides users with a step-by-step approach to understanding and utilizing the plugin's features. It serves as a comprehensive reference to help new users familiarize themselves with the plugin and helps existing users to become more proficient in using it.
The current New relic plugin README isn't enough to to understand and use that plugin and When the docs isn't sufficient to understand a product or service, it can be frustrating and confusing for users.
🎉 Proposal
I recently stumbled upon an article that I believe can be used as documentation for this proposal. The article was well-written and provided clear, concise explanations of the plugin's use. Including the author is an active member of this community roadie.io.
We can improve this doc or just use the content of the article as doc.
〽 Alternatives
No response
❌ Risks
No response
👀 Have you spent some time to check if this RFC has been raised before?

[X] I checked and didn't find similar issue

🏢 Have you read the Code of Conduct?

[X] I have read the Code of Conduct

Think this makes sense, do you want to to contribute some better documentation to help us out? Maybe @timmywheels can help too as I think they wrote the documentation on the roadie article?

Sure, I can use this article as base and improve that docs

Up 👍

PR was accepted and merged,  I am closing this issue.

authors: benjdlambert juniorsdj plinioaugusto

repo: backstage/backstage


text:
Remove requirements for Group model

The current group model according to the docs requires ancestors, children and descendant.
But this means that if my org is:
root => Engineering -> Platform -> Devtools -> TeamA
I need to write any of these names in 3 places. Now I want to change "Devtools" to "Internal tools" and need to do modifications in multiple places.
If you want to build a tree of the org, the only required element should be the parent, which interestingly enough is the only optional element currently

It's optional only because the org root can't have a parent :)
Yep this is on the roadmap. Let's use this ticket to keep track of the progress of this aspect of it.
There is ongoing work (see recent PRs) to implement relations. The end result will be a flexible model where the input yaml can express either side of a relation, but both sides will be generated automatically.
For example, in some data sources, the parent is easily and immediately available as you say. In other data sources, the immediate children are instead easily available. Both should trivially be supported, and should lead to identical sets of relations to be produced.

Update:
Initial relations support has been implemented.
We are considering removing the ancestors and descendants fields.
There is some interesting discussion around relations and usage around users and groups for those who may be interested here: https://github.com/backstage/backstage/pull/3293

authors: freben jcarres-mdsol