text:
Improve running efficiency of Iterative Preference

Summary
This PR improves the efficiency of the current implementation of the Iterative Preference algorithm.
Changes
Modifications to func_iterative_preferences in scripts/functions.R to improve efficiency; changes are as follows:

Allocation by row (previously by columns)
If session is now full, assign 0 to all its preference values to minimise time spent in j-loop

Checks

[ ] Modified function runs with correct output in scripts/main.R
[ ] Modified function runs for <1 sec on bigger dataset in scripts/test scripts/main-test-big_data.R

To note
This "fixes #10".

Currently waiting for packrat to load all the packages used in this project.
In the meantime, some comments:
Very happy and pleased that you managed to implement a row-wise version of this algorithm and especially impressed at the speed improvements!
The code is neat!
Only one commit? I'd say there should at least be two commits (even that's being conservative) since there is also a test script you wrote that benchmarks the new function. We probably should have a discussion over the frequency of commits...
Branch name falls outside the git branch-naming style-guide. We probably should have a discussion about this too...

repo: avisionh/Preference-Allocation


text:
Automatic export when source file changes

Hello.
There are many small things to improve in this wonderful program...

Output in G21 (mm)
Use . as decimal separator instead of ,
offset output so that lowest left corner is at 0,0
set default S value
stop putting "M62 P0" - bCNC does not like it..

So I made a C# program that reads SVG-to-GCode generated file and corrects all these little things, so I would no bother you!
However, there is one thing I can´t do it with my program...
A checkbox that enables "Automatic Convert"
My C# program has one! Here´s how it works:
When the checkbox is checked, my C# program will

read the source file´s "last written date"
start a 1second timer

When the checkbox is unchecked, my C# program will stop the timer.
At the timer function, it reads the source file´s "last written date" and compares.
If the file has changed, it rebuilds.
This helps to make a workflow:
When I save a file in Inkscape...
SVG-to-GCode will notice file has changed and automatically reconvert.
My C# program will notice file has changed, "fix" it, and save to another file.
bCNC will notice file has changed and prompt to read again (it does!)
A few last words:
I did a timer because I´m lazy. I should use a notification... shame on me!
In the Export dialog (where there is the "Export Now" button) there could be an option of Rotation (0, 90, 180, 270) so that rotation will also be done automatically, and not because user pressed the "Rotate 90" button.
It would also be nice to have Xmirror and Ymirror..
Thank you

Hi, and thanks for your interest!
I don't want to add an 'automatic export when source file changes' because it would lose any changes you made in SVG to GCode, such as scaling, tiling, removing sections, etc.
I also don't really use this application anymore and currently don't have the time to maintain it - feel free to make the changes needed!
At some point I may convert this to a C# application, but that is a lot of work.
Cheers!

source: gharchive/issue

authors: amirm avwuff

repo: avwuff/SVG-to-GCode

license: mit

license_type: permissive

license_source: bigquery


text:
support sandbox secret remove --all

Environment information
n/a

Describe the feature
It can be tedious to remove all secrets related to a sandbox environment without removing each individual secret, or by navigating to the AWS console to remove secrets manually. Support the --all flag on sandbox secret remove to remove all secrets in a single command
Use case
see above

Marking as feature request

authors: josefaidt ykethan

repo: aws-amplify/amplify-backend

license_source: github-api


text:
feat: add support for allowDestructiveSchemaUpdates in defineData

Issue #, if available:
N/A
Description of changes:
When AMPLIFY_TABLE deployment strategy was enabled, we didn't add the requisite override flag to enable deploying destructive updates (e.g. key changes). This PR wires that through, and adds test for default, positive, and negative cases.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Closing this, and likely converting to a flag that comes in the pipeline-deploy command.

source: gharchive/pull-request


text:
vague error when amplify init fails

Before opening, please confirm:

[X] I have installed the latest version of the Amplify CLI (see above), and confirmed that the issue still persists.
[X] I have searched for duplicate or closed issues.
[X] I have read the guide for submitting bug reports.
[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
[X] I have removed any sensitive information from my code snippets and submission.

How did you install the Amplify CLI?
pnpm
If applicable, what version of Node.js are you using?
18
Amplify CLI Version
10.0.0
What operating system are you using?
mac
Did you make any manual changes to the cloud resources managed by Amplify? Please describe the changes made.
n/a
Amplify Categories
Not applicable
Amplify Commands
Not applicable
Describe the bug
If amplify init fails for some reason we are presented with a vague error (in my case I hit the limit for s3 buckets, but I needed to go to the CloudFormation console to figure out what went wrong)
aws-amplify/reproductions/854
➜  amplify init -y
Note: It is recommended to run this command from the root of your app directory
The following configuration will be applied:

Project information
| Name: 854
| Environment: dev
| Default editor: Visual Studio Code
| App type: javascript
| Javascript framework: none
| Source Directory Path: src
| Distribution Directory Path: dist
| Build Command: npm run-script build
| Start Command: npm run-script start

Using default provider  awscloudformation
Adding backend environment dev to AWS Amplify app: d3zwvxdkkdme5

Deployment failed.
Deploying root stack 854 [ ==========------------------------------ ] 1/4
	amplify-854-dev-82333          AWS::CloudFormation::Stack     ROLLBACK_COMPLETE              Fri
	AuthRole                       AWS::IAM::Role                 CREATE_FAILED                  Fri
	UnauthRole                     AWS::IAM::Role                 CREATE_FAILED                  Fri
	DeploymentBucket               AWS::S3::Bucket                CREATE_FAILED                  Fri

An error occurred when creating the CloudFormation stack

Expected behavior
error includes guidance for next steps, what went wrong
Reproduction steps

reach the bucket limit (or some other means of causing init to fail)
amplify init -y
observe failure and error

GraphQL schema(s)

# Put schemas below this line

Project Identifier
No response
Log output

# Put your logs below this line

Additional information
No response

This gets into a weird state where the app and root stack were created, but not the bucket or supporting roles.
aws-amplify/reproductions/854
➜  amplify init -y
Note: It is recommended to run this command from the root of your app directory
The following configuration will be applied:

Project information
| Name: 854
| Environment: dev
| Default editor: Visual Studio Code
| App type: javascript
| Javascript framework: none
| Source Directory Path: src
| Distribution Directory Path: dist
| Build Command: npm run-script build
| Start Command: npm run-script start

Using default provider  awscloudformation
Adding backend environment dev to AWS Amplify app: d3v5oleehwg3n2

Deploying resources into dev environment. This will take a few minutes. ⠧
Deploying root stack 854 [ ---------------------------------------- ] 0/4
	amplify-854-dev-83528          AWS::CloudFormation::Stack     ROLLBACK_IN_PROGRESS           Fri
	UnauthRole                     AWS::IAM::Role                 CREATE_FAILED                  Fri
	DeploymentBucket               AWS::S3::Bucket                CREATE_FAILED                  Fri
	AuthRole                       AWS::IAM::Role                 CREATE_FAILED                  Fri
^C⏎                                                                                ^Aborted!

aws-amplify/reproductions/854
➜  amplify version
10.0.0

aws-amplify/reproductions/854
➜  amplify delete --force
🛑
      No Amplify backend project files detected within this folder. Either initialize a new Amplify project or pull an existing project.
      - "amplify init" to initialize a new Amplify project
      - "amplify pull <app-id>" to pull your existing Amplify project. Find the <app-id> in the AWS Console or Amplify Admin UI.

repo: aws-amplify/amplify-cli


text:
field level @auth has two bugs

Describe the bug
Consider the following schema (taken from the docs):
type User @model {
  id: ID!
  username: String
  ssn: String @auth(rules: [{ allow: owner, ownerField: "username" }])
}

Now when creating a user (assuming we have a token with a username of username) like this:
mutation MyMutation {
  createUser(input: {id: "id", username: "username"}) {
    id
    ssn
  }
}

and then querying them like this:
query MyQuery {
  getUser(id: "id") {
    id
    ssn
  }
}

it throws an error:
{
  "data": {
    "getUser": {
      "id": "id",
      "ssn": null
    }
  },
  "errors": [
    {
      "message": "Unable to convert $util.toJson($context.source.ssn)\n to class com.amazonaws.deepdish.transform.model.lambda.LambdaVersionedConfig.",
      "errorType": "MappingTemplate",
      "data": null,
      "errorInfo": null,
      "path": [
        "getUser",
        "ssn"
      ],
      "locations": [
        {
          "line": 18,
          "column": 5,
          "sourceName": "GraphQL request"
        }
      ]
    }
  ]
}

Apparently, this is caused by the ssn being null.
Additionally, when updating the ssn and querying for the ssn field in this mutation:
mutation m {
  updateUser(input:{id:"id", ssn:"ssn"}) {
  	id
  	ssn
  }
}

it doesn't return the ssn properly but null instead:
{
  "data": {
    "updateUser": {
      "id": "id",
      "ssn": null
    }
  }
}

When querying for the ssn in a query afterwards, everything works as expected:
query MyQuery {
  getUser(id: "id") {
    id
    ssn
  }
}

Response:
{
  "data": {
    "getUser": {
      "id": "id",
      "ssn": "ssn"
    }
  }
}

Amplify CLI Version
4.24.3
To Reproduce
See above
Expected behavior
The queries and mutations to work properly
Desktop (please complete the following information):

OS: Ubuntu 19.10
Node version: 14.7.0

Hello @tobiasfeil
The first case should return null the value was not set. Though returning null as an error seems specific to mock which is something we can investigate into looking why it throws that specific error.
The second case returns null on the mutation so it can protect the subscription response.
Read more on that here: https://docs.amplify.aws/cli/graphql-transformer/directives#authorizing-subscriptions

Thanks for the response! OK I'll just ignore the error for now then.

Found out I can't just ignore it because it actually throws an error. Therefore, for now, I have to fetch my data like this:
try {
      await API.graphql(
        graphqlOperation(listItems)
      )
    } catch (e) {
      this.data = e.data
    }

It would be nice to have this taken care of sooner rather than later.

Dangit, if I do it like this, the other limitation kicks in where field-level @auth protected fields return null in mutations... This is a bit tricky

authors: SwaySway tobiasfeil


text:
Transformer v2 fails when converting a table with an index on a primaryKey

Before opening, please confirm:

[X] I have installed the latest version of the Amplify CLI (see above), and confirmed that the issue still persists.
[X] I have searched for duplicate or closed issues.
[X] I have read the guide for submitting bug reports.
[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.

How did you install the Amplify CLI?
npm -g
If applicable, what version of Node.js are you using?
14.18.1
Amplify CLI Version
7.6.3
What operating system are you using?
OSX 12
Amplify Categories
api
Amplify Commands
push
Describe the bug
I'm attempting to migrate to Transformer Version 2 (amplify api migrate) in order to perform some other GraphQL updates via amplify push.
Once the new schema has been generated, any amplify push (on a new environment) end up breaking with the following error:
✖ An error occurred when pushing the resources to the cloud
🛑 Could not fetch keySchema for Model.
InvalidDirectiveError: Could not fetch keySchema for Model.
    at AuthTransformer.protectListResolver (/Users/.nvm/versions/node/v14.18.1/lib/node_modules/@aws-amplify/cli/node_modules/@aws-amplify/graphql-auth-transformer/src/graphql-auth-transformer.ts:499:13)
    at AuthTransformer.generateResolvers (/Users/.nvm/versions/node/v14.18.1/lib/node_modules/@aws-amplify/cli/node_modules/@aws-amplify/graphql-auth-transformer/src/graphql-auth-transformer.ts:326:16)
    at GraphQLTransform.transform (/Users/.nvm/versions/node/v14.18.1/lib/node_modules/@aws-amplify/cli/node_modules/@aws-amplify/graphql-transformer-core/src/transformation/transform.ts:252:21)
    at _buildProject (/Users/.nvm/versions/node/v14.18.1/lib/node_modules/@aws-amplify/cli/node_modules/amplify-provider-awscloudformation/src/graphql-transformer/transform-graphql-schema.ts:564:37)
    at buildAPIProject (/Users/.nvm/versions/node/v14.18.1/lib/node_modules/@aws-amplify/cli/node_modules/amplify-provider-awscloudformation/src/graphql-transformer/transform-graphql-schema.ts:521:24)
    at transformGraphQLSchema (/Users/.nvm/versions/node/v14.18.1/lib/node_modules/@aws-amplify/cli/node_modules/amplify-provider-awscloudformation/src/graphql-transformer/transform-graphql-schema.ts:394:29)
    at Object.run (/Users/.nvm/versions/node/v14.18.1/lib/node_modules/@aws-amplify/cli/node_modules/amplify-provider-awscloudformation/src/push-resources.ts:166:5)
    at async Promise.all (index 0)
    at providersPush (/Users/.nvm/versions/node/v14.18.1/lib/node_modules/@aws-amplify/cli/src/extensions/amplify-helpers/push-resources.ts:139:3)
    at AmplifyToolkit.pushResources [as _pushResources] (/Users/.nvm/versions/node/v14.18.1/lib/node_modules/@aws-amplify/cli/src/extensions/amplify-helpers/push-resources.ts:96:9)
    at Object.executeAmplifyCommand (/Users/.nvm/versions/node/v14.18.1/lib/node_modules/@aws-amplify/cli/node_modules/@aws-amplify/amplify-category-api/src/index.ts:222:5)
    at executePluginModuleCommand (/Users/.nvm/versions/node/v14.18.1/lib/node_modules/@aws-amplify/cli/src/execution-manager.ts:204:3)
    at executeCommand (/Users/.nvm/versions/node/v14.18.1/lib/node_modules/@aws-amplify/cli/src/execution-manager.ts:30:5)
    at Object.run (/Users/.nvm/versions/node/v14.18.1/lib/node_modules/@aws-amplify/cli/src/index.ts:205:5)

Digging into the amplify cli code a bit and performing various console debugs, it looks like an @index has been created on the @primaryKey in the schema, but the @index gets ignored.
Original schema:
type Model
@model
@auth(rules: [
  {allow: public, provider: apiKey}
])
@key(name: "byUser", fields: ["user", "channel"], queryField: "listModelsByUser")
@key(name: "byChannel", fields: ["channel", "user"], queryField: "listModelsByChannel")
@key(fields: ["user", "channel", "token", "secret"])
{
  user: String!
  channel: String!
  token: String!
  secret: String
}

Transformed schema:
type Model @model @auth(rules: [{allow: public, provider: apiKey}]) {
  user: String! @primaryKey(sortKeyFields: ["channel", "token", "secret"]) @index(name: "byUser", queryField: "listModelsByUser", sortKeyFields: ["channel"])
  channel: String! @index(name: "byChannel", queryField: "listModelsByChannel", sortKeyFields: ["user"])
  token: String!
  secret: String!
}

The table.globalSecondaryIndexes only contains the byChannel index and throws the InvalidDirect exception.
Expected behavior
amplify push should succeed, finding all available indexes
Reproduction steps
Create the following table using v1 transformer:
type Model
@model
@auth(rules: [
  {allow: public, provider: apiKey}
])
@key(name: "byUser", fields: ["user", "channel"], queryField: "listModelsByUser")
@key(name: "byChannel", fields: ["channel", "user"], queryField: "listModelsByChannel")
@key(fields: ["user", "channel", "token", "secret"])
{
  user: String!
  channel: String!
  token: String!
  secret: String
}

Run amplify push
Run amplify api migrate
Run amplify push
GraphQL schema(s)

# Put schemas below this line

type Model
@model
@auth(rules: [
  {allow: public, provider: apiKey}
])
@key(name: "byUser", fields: ["user", "channel"], queryField: "listModelsByUser")
@key(name: "byChannel", fields: ["channel", "user"], queryField: "listModelsByChannel")
@key(fields: ["user", "channel", "token", "secret"])
{
  user: String!
  channel: String!
  token: String!
  secret: String
}

Log output

# Put your logs below this line

✖ An error occurred when pushing the resources to the cloud
🛑 Could not fetch keySchema for Model.
InvalidDirectiveError: Could not fetch keySchema for Model.
    at AuthTransformer.protectListResolver (/Users/.nvm/versions/node/v14.18.1/lib/node_modules/@aws-amplify/cli/node_modules/@aws-amplify/graphql-auth-transformer/src/graphql-auth-transformer.ts:499:13)
    at AuthTransformer.generateResolvers (/Users/.nvm/versions/node/v14.18.1/lib/node_modules/@aws-amplify/cli/node_modules/@aws-amplify/graphql-auth-transformer/src/graphql-auth-transformer.ts:326:16)
    at GraphQLTransform.transform (/Users/.nvm/versions/node/v14.18.1/lib/node_modules/@aws-amplify/cli/node_modules/@aws-amplify/graphql-transformer-core/src/transformation/transform.ts:252:21)
    at _buildProject (/Users/.nvm/versions/node/v14.18.1/lib/node_modules/@aws-amplify/cli/node_modules/amplify-provider-awscloudformation/src/graphql-transformer/transform-graphql-schema.ts:564:37)
    at buildAPIProject (/Users/.nvm/versions/node/v14.18.1/lib/node_modules/@aws-amplify/cli/node_modules/amplify-provider-awscloudformation/src/graphql-transformer/transform-graphql-schema.ts:521:24)
    at transformGraphQLSchema (/Users/.nvm/versions/node/v14.18.1/lib/node_modules/@aws-amplify/cli/node_modules/amplify-provider-awscloudformation/src/graphql-transformer/transform-graphql-schema.ts:394:29)
    at Object.run (/Users/.nvm/versions/node/v14.18.1/lib/node_modules/@aws-amplify/cli/node_modules/amplify-provider-awscloudformation/src/push-resources.ts:166:5)
    at async Promise.all (index 0)
    at providersPush (/Users/.nvm/versions/node/v14.18.1/lib/node_modules/@aws-amplify/cli/src/extensions/amplify-helpers/push-resources.ts:139:3)
    at AmplifyToolkit.pushResources [as _pushResources] (/Users/.nvm/versions/node/v14.18.1/lib/node_modules/@aws-amplify/cli/src/extensions/amplify-helpers/push-resources.ts:96:9)
    at Object.executeAmplifyCommand (/Users/.nvm/versions/node/v14.18.1/lib/node_modules/@aws-amplify/cli/node_modules/@aws-amplify/amplify-category-api/src/index.ts:222:5)
    at executePluginModuleCommand (/Users/.nvm/versions/node/v14.18.1/lib/node_modules/@aws-amplify/cli/src/execution-manager.ts:204:3)
    at executeCommand (/Users/.nvm/versions/node/v14.18.1/lib/node_modules/@aws-amplify/cli/src/execution-manager.ts:30:5)
    at Object.run (/Users/.nvm/versions/node/v14.18.1/lib/node_modules/@aws-amplify/cli/src/index.ts:205:5)

Additional information
Help me Obi Wan Kenobi ⚔

@alexhouse I'm not able to reproduce the issue. First, I created a new project and deployed the v2 schema you provided. That went through fine.
Next, I created another new project using the v1 schema you provided. There are a couple issues with the provided schema. There is no deviceToken or expoPushToken field in the schema. For the v1 primary key, I used the fields user, channel, and token, since the provided schema was unclear. Once I corrected those errors, I was able to migrate and push the schema just fine.

Hmm.
In my attempt to provide a simple table to reproduce the issue I messed up. I've edited the OP with the correct schemas.
I've just noticed that the byUser key is currently labelled as a local secondary instead of a global secondary within DynamoDB... that may well be the cause of the errors. I have a feeling that the only way of making a local key global is to fully recreate the table, right? Which would then require all data to be upserted somehow (batch operations?)

authors: alexhouse cjihrig


text:
PostConfirmation Trigger unable to add user to group, missing cognito-idp:GetGroup permission on attached auth resource

Before opening, please confirm:

[X] I have installed the latest version of the Amplify CLI (see above), and confirmed that the issue still persists.
[X] I have searched for duplicate or closed issues.
[X] I have read the guide for submitting bug reports.
[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.

How did you install the Amplify CLI?
pnpm
If applicable, what version of Node.js are you using?
v16.13.0
Amplify CLI Version
7.6.3
What operating system are you using?
Mac
Amplify Categories
auth
Amplify Commands
Not applicable
Describe the bug
Unable to successfully execute add-to-group PostConfirmation trigger upon creating a new user in UserPool due to missing permission. Trigger template also tries to create the group if it fails to get the group, which also fails due to a missing permission.

cognito-idp:GetGroup
cognito-idp:CreateGroup

add-to-group trigger code
/* eslint-disable-line */ const aws = require('aws-sdk')

const cognitoidentityserviceprovider = new aws.CognitoIdentityServiceProvider({
  apiVersion: '2016-04-18',
})

exports.handler = async event => {
  const groupParams = {
    GroupName: process.env.GROUP,
    UserPoolId: event.userPoolId,
  }
  const addUserParams = {
    GroupName: process.env.GROUP,
    UserPoolId: event.userPoolId,
    Username: event.userName,
  }
  /**
   * Check if the group exists; if it doesn't, create it.
   */
  try {
    await cognitoidentityserviceprovider.getGroup(groupParams).promise()
  } catch (e) {
    console.log('error is', e)
    await cognitoidentityserviceprovider.createGroup(groupParams).promise()
  }
  /**
   * Then, add the user to the group.
   */
  await cognitoidentityserviceprovider
    .adminAddUserToGroup(addUserParams)
    .promise()

  return event
}

Sample repo: https://github.com/josefaidt/amplify-postconf-auth-trigger-issue

After adding a console.log(e) to the try/catch block of the postconfirmation trigger (to understand why it is attempting to create the group) we see the following error in the Lambda's CloudWatch logs

AccessDeniedException: User: arn:aws:sts::814763596509:assumed-role/authtriggersf7a8d23ePostConfirmation-dev/authtriggersf7a8d23ePostConfirmation-dev is not authorized to perform: cognito-idp:GetGroup on resource: arn:aws:cognito-idp:us-east-1:814763596509:userpool/us-east-1_UzWz3I2rt because no identity-based policy allows the cognito-idp:GetGroup action

Expected behavior
default add-to-group template should add new users to group without error
Reproduction steps

amplify init -y
amplify add auth

create group users
add add-to-group functionality
add PostConfirmation trigger, accept template

amplify push -y
Create sample frontend, attempt to create a new user
Observe error

GraphQL schema(s)

# Put schemas below this line

Log output

# Put your logs below this line

Additional information
No response

fix #7673

I also faced this issue, but I fixed it, by manually adding an inline policy to the post-confirmation role.

Closing as a duplicate of https://github.com/aws-amplify/amplify-cli/issues/7582

authors: ammarkarachi hackrx josefaidt


text:
Amplify stuck at deploy step

App Id
d1j578lqqx5u9d
Region
eu-central-1
Describe the bug
My Next.js SSR app deployment is currently stuck at the Deploy Step with these logs:
2021-06-15T12:52:00 [INFO]: Starting Deployment 2021-06-15T12:52:00 [INFO]: Updating Edge config
Reproduction steps
The Deployment is triggered from Gitlab push event.
Build Settings
version: 1
frontend:
  phases:
    preBuild:
      commands:
        - npm ci
    build:
      commands:
        - npm run build
  artifacts:
    baseDirectory: .next
    files:
      - '**/*'
  cache:
    paths:
      - node_modules/**/*

I'm having the same issue, more info in https://github.com/aws-amplify/amplify-console/issues/1960

I had the same issue when my repo was nested inside the groups..what worked for me was moving the repo inside the group to the root level of the group..
My issue was #1941

Any updates on this one?

The deployment eventually succeed. I don't know how nor why.

@asungomez having same issue please any help would be much appreciated.. My app id=d1lxz7k7qngoq2 and region=eu-central-1(frankfurt)

authors: antoineandrieu asungomez zugger14

repo: aws-amplify/amplify-console


text:
Amplify Build failed

Before opening, please confirm:

[X] I have checked to see if my question is addressed in the FAQ.
[X] I have searched for duplicate or closed issues.
[X] I have read the guide for submitting bug reports.
[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.

App Id
aws:amplify:ap-southeast-2:698750000341:apps/d2pes57mg7f1ea
Region
ap-southeast-2
Amplify Console feature
Custom builds, Custom headers, Monorepo
Describe the bug
2021-06-25T03:45:27.761Z [WARNING]: !Error: ENOENT: no such file or directory, open '/codebuild/output/src817206887/src/transiot/amplify.yml'
2021-06-25T03:45:27.785Z [INFO]: # Retrieving environment cache...
2021-06-25T03:45:27.812Z [INFO]: # Retrieved environment cache
2021-06-25T03:45:27.812Z [WARNING]: !Cannot read property 'split' of undefined
2021-06-25T03:45:27.823Z [ERROR]: !!! TypeError: undefined is not a function
2021-06-25T03:45:27.823Z [INFO]: # Starting environment caching...
2021-06-25T03:45:27.823Z [INFO]: # Environment caching completed
Terminating logging...
Expected behavior
Build fails
Reproduction steps
2021-06-25T03:45:27.761Z [WARNING]: !Error: ENOENT: no such file or directory, open '/codebuild/output/src817206887/src/transiot/amplify.yml'
2021-06-25T03:45:27.785Z [INFO]: # Retrieving environment cache...
2021-06-25T03:45:27.812Z [INFO]: # Retrieved environment cache
2021-06-25T03:45:27.812Z [WARNING]: !Cannot read property 'split' of undefined
2021-06-25T03:45:27.823Z [ERROR]: !!! TypeError: undefined is not a function
2021-06-25T03:45:27.823Z [INFO]: # Starting environment caching...
2021-06-25T03:45:27.823Z [INFO]: # Environment caching completed
Terminating logging...
Build Settings
version: 1
applications:
  appRoot: dist/apps/portal
  frontend:
  phases:
    preBuild:
      commands:
        - npm install
    build:
      commands:
        # - npm run build:$BUILD_ENV
        # - echo $BUILD_ENV > .env
        - echo $API_HOST >> .env
        - echo $tmp >> .env
        - npm run ui:generate apps-ui/portal
  artifacts:
    # IMPORTANT - Please verify your build output directory
    baseDirectory: dist/apps/portal
    files:
      - '**/*'

Additional information
Not able to fix the issue
!!! TypeError: undefined is not a function

This is not related to custom domain
App ARN - arn:aws:amplify:ap-southeast-2:698750000341:apps/d2pes57mg7f1ea
Region - ap-southeast-2

Closing. If you run into this issue again please feel free to reopen.

repo: aws-amplify/amplify-hosting


text:
Amplify deployment not working, start with Provision step and stuck longtime

Before opening, please confirm:

[X] I have checked to see if my question is addressed in the FAQ.
[X] I have searched for duplicate or closed issues.
[X] I have read the guide for submitting bug reports.
[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
[X] I have removed any sensitive information from my code snippets and submission.

App Id
aaa
AWS Region
ap-south-1
Amplify Hosting feature
Deployments
Describe the bug
I just started deploy our app and the provision step started and stuck in same step long time.
Expected behavior
Actually the provision step not taking this much time
Reproduction steps
Few hours before same kind of issue experiences
Build Settings
No response
Log output

# Put your logs below this line

Additional information
No response

Now working


text:
fix: Support new list name pluralization

Issue #, if available:
https://github.com/aws-amplify/amplify-cli/issues/8350
Description of changes:

[x] Finish rest schema updates to use listPluralName and syncPluralName
[x] Run existing integration test using feature flag turned off
[x] Generate new models using the latest codegen process (https://github.com/aws-amplify/amplify-codegen/pull/255#issuecomment-949890655) and cross check with the test schemas we have
[x] Provision backend using improvepluralization true (this is on by default), and rerun integration tests

Check points:

[x] Added new tests to cover change, if needed
[x] All unit tests pass
[x] All integration tests pass
[ ] Documentation update for the change if required
[x] PR title conforms to conventional commit style

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

This has been released in https://github.com/aws-amplify/amplify-ios/releases/tag/v1.15.4

Thank you @lawmicha
What are the steps required to update existing projects ?
Here is what I did and it is not working

update pod to 1.15.4
update cli to 6.3.1
run  amplify codeine models

It still fails at runtime. I guess some change are required on the backend side. How can we fix this without deleting / re-creating an API backend ?

hey @sebsto, sorry I wasn't clear, the codegen changes still need to be released. What you have there is the correct workaround by modifying what the codegen would be generating

I thought that CLI 6.3.0 and this PR is fixing the pluralisation https://github.com/aws-amplify/amplify-cli/pull/8401
What is the CLI PR to track ?

that one sets the flag as false for new projects, this one https://github.com/aws-amplify/amplify-codegen/pull/255 adds code that generates the new fields based on the flag

authors: lawmicha sebsto

repo: aws-amplify/amplify-ios


text:
pod Amplify 版本不支持

Describe the bug
https://github.com/aws-amplify/amplify-swift
这个是Amplify代码仓库，为什么我用pod依赖管理工具pod 'Amplify' 得到的代码目录结构和git仓库的目录结构不一样？而用Swift Package Manager引入的库就和git仓库的目录结构一样的，而且用 Swift Package Manager 得到的版本比pod高很多，这是为什么？
Steps To Reproduce
Amplify iOS端 一共分两种引入方式：
1、SPM   这种方式亚马逊官方在用很适合美国那边的开发者，看了一下版本比较高都到2.28.0以上了
2、pod    这种方式很适合中国的开发者，但是最高版本是1.30.8，很显然这个版本比SPM低多了，我用这个方式引入后调用终端代码报错，说signInResult找不到isSignedI字段。
func signIn(username: String, password: String) async {
        do {
            let signInResult = try await Amplify.Auth.signIn(
                username: username,
                password: password
                )
            if signInResult.isSignedIn {
                print("Sign in succeeded")
            }
        } catch let error as AuthError {
            print("Sign in failed \(error)")
        } catch {
            print("Unexpected error: \(error)")
        }
    }
而SPM方式不会报错，我看源码发现这两种版本的代码确实不一样：
SPM的定义：
public func signIn(username: String? = nil,
                       password: String? = nil,
                       options: AuthSignInRequest.Options? = nil) async throws -> AuthSignInResult {
        return try await plugin.signIn(username: username, password: password, options: options)
    }

pod的定义：
func signIn(
        username: String? = nil,
        password: String? = nil,
        options: AuthSignInOperation.Request.Options? = nil
    ) -> AuthSignInOperation {
        signIn(username: username, password: password, options: options, listener: nil)
    }
很明显看到不一样的地方是：SPM版本的返回值是AuthSignInResult，所以有isSignedI字段。而pod版本的返回值是AuthSignInOperation，没有isSignedI字段。pod版本的Amplify太落后。

我的问题是：想请您帮我找一下有没有比较高的pod版本的Amplify，和SPM版本的Amplify SDK保持一致，同步更新的版本。

Expected behavior
不支持  pod Amplify最新版本。
Amplify Framework Version
1.30.8
Amplify Categories
Auth
Dependency manager
Cocoapods
Swift version
v5.8
CLI version
12+
Xcode version
14.2
Relevant log output
signInResult找不到isSignedI字段。

Is this a regression?
No
Regression additional context
No response
Platforms
iOS
OS Version
ios 13+
Device
any
Specific to simulators
any
Additional context
No response

Hi @zhangshouyin!
Amplify v2 (i.e. versions 2.x.x. and higher) is only distributed through SPM, and we have no plans of adding CocoaPods support.
Amplify v1 (i.e. versions 1.x.x) is currently in Maintenance Mode, so we advice you to use Amplify v2 instead.

Hi @zhangshouyin! Amplify v2 (i.e. versions 2.x.x. and higher) is only distributed through SPM, and we have no plans of adding CocoaPods support.
Amplify v1 (i.e. versions 1.x.x) is currently in Maintenance Mode, so we advice you to use Amplify v2 instead.

Hi @zhangshouyin! Amplify v2 (i.e. versions 2.x.x. and higher) is only distributed through SPM, and we have no plans of adding CocoaPods support.
Amplify v1 (i.e. versions 1.x.x) is currently in Maintenance Mode, so we advice you to use Amplify v2 instead.

Hi @zhangshouyin! Amplify v2 (i.e. versions 2.x.x. and higher) is only distributed through SPM, and we have no plans of adding CocoaPods support.
Amplify v1 (i.e. versions 1.x.x) is currently in Maintenance Mode, so we advice you to use Amplify v2 instead.

I saw that aws-sdk-ios is written in Objective-C, with 1.7K stars, it should be very stable, will it continue to be updated?
So should I give up Amplify v2 and use aws-sdk-ios instead? I don't know which solution is better for me?
What are the advantages of Amplify v2 compared to aws-sdk-ios?

Hi @zhangshouyin! Amplify v2 (i.e. versions 2.x.x. and higher) is only distributed through SPM, and we have no plans of adding CocoaPods support.
Amplify v1 (i.e. versions 1.x.x) is currently in Maintenance Mode, so we advice you to use Amplify v2 instead.

Hi @ruisebas
I saw that aws-sdk-ios is written in Objective-C, with 1.7K stars, it should be very stable, will it continue to be updated?
So should I give up Amplify v2 and use aws-sdk-ios instead? I don't know which solution is better for me?
What are the advantages of Amplify v2 compared to aws-sdk-ios?

The AWS SDK for iOS is still maintained, although we advice you to use Amplify v2 instead.
Amplify v2 is written fully in Swift and takes advantages of the language's features. It provides higher level APIs that support a wide range of common use cases out of the box. The SDK on the other hand is more of a low-level client that lets you interact directly with AWS APIs, but as such you will need a deeper understanding of how they work and how to handle them.
You can read more about it in this FAQ: How is the SDK different from the Amplify Libraries, and in Amplify's documentation main page: How Amplify works.

Hi @zhangshouyin! Amplify v2 (i.e. versions 2.x.x. and higher) is only distributed through SPM, and we have no plans of adding CocoaPods support.
Amplify v1 (i.e. versions 1.x.x) is currently in Maintenance Mode, so we advice you to use Amplify v2 instead.

Hi @ruisebas
Is it feasible for me to privatize the Amplify v2 code into my own repository and then add the corresponding podspec files to each library?

Hi @zhangshouyin! Amplify v2 (i.e. versions 2.x.x. and higher) is only distributed through SPM, and we have no plans of adding CocoaPods support.
Amplify v1 (i.e. versions 1.x.x) is currently in Maintenance Mode, so we advice you to use Amplify v2 instead.

Hi @ruisebas!
I'm accustomed to using CocoaPods to manage code libraries and not comfortable with SPM. I still want to use CocoaPods.
So, Is it feasible for me to privatize the Amplify v2 code into my own repository and then add the corresponding podspec files to each library?

You're free to fork the repo and make modifications to the code that fits your use cases better, but please be aware that we cannot offer any support on forks.
Alternatively, you might be able to wrap Amplify v2 in an xcframework and/or host that xcframework as a podspec in your build system.

You're free to fork the repo and make modifications to the code that fits your use cases better, but please be aware that we cannot offer any support on forks.
Alternatively, you might be able to wrap Amplify v2 in an xcframework and/or host that xcframework as a podspec in your build system.

Hi @ruisebas  Thanks for your advice I tried packaging into xcframework.

authors: ruisebas zhangshouyin

repo: aws-amplify/amplify-swift


text:
test(storage): fix copy_configuration script for stress test

Issue #

Description

General Checklist

[ ] Added new tests to cover change, if needed
[ ] Build succeeds with all target using Swift Package Manager
[ ] All unit tests pass
[ ] All integration tests pass
[ ] Security oriented best practices and standards are followed (e.g. using input sanitization, principle of least privilege, etc)
[ ] Documentation update for the change if required
[ ] PR title conforms to conventional commit style
[ ] New or updated tests include Given When Then inline code documentation and are named accordingly testThing_condition_expectation()
[ ] If breaking change, documentation/changelog update with migration instructions

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

storage stress tests workflow https://github.com/aws-amplify/amplify-swift/actions/runs/8840460451


text:
Fix prow presets configMap

Issue #, if available:
Description of changes:
While working on adding new prow plugins i came across this little dot. I believe it wasn't supposed to be here :)
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

It's surprising that this didn't break anything in our prow cluster. Or at least we didn't notice anything wrong..
/cc @RedbackThomson @jljaco


text:
Moving a tab to the background inflates LCP

The LCP measure uses a PerformanceObserver. As PerformanceObservers don't run in the background, if a user visits a page and minimizes / changes browser tab before the LCP is measured then the LCP value will be the time the browser tab is brought back to the foreground

Any updates?

Hi @dbreeze-veeqo, we bumped the web vitals version (from v1 to v3) in release 1.9.x. Could you check to see if that resolved the issue?

Hey @qhanam, are you able to clarify whether RUM detects and filters out LCP events for pages that were opened in the background? It looks like the web-vitals library will record LCP for all pages regardless of whether they were opened in the background or not, so I don't think bumping the version will have any effect on this issue. We're still seeing very large spikes in LCP (as high as 2 hours).
https://web.dev/lcp/#when-is-largest-contentful-paint-reported

It looks like the standard approach to handle this case is to discard the LCP values. Specifically, when the page was loaded in the background, the CloudWatch RUM web client should not record the LCP.

Hello, I've verified that after bumping the web-vitals library for the web client, LCP for pages opened in the background is not recorded as I was unable to reproduce this issue using web client release 1.11.x. As Quinn mentioned, this update is available from release 1.9.x and higher. Could you verify if this issue still persists with the newer web client releases? If not, feel free to close this issue!

authors: MyGuySi dbreeze-veeqo limhjgrace qhanam

repo: aws-observability/aws-rum-web


text:
[GST Plugin] Audio is garbled with WebRTC

I'm working on using the gstreamer + kvsplugin as a way to ingest some video streams into KVS for real-time monitoring. The gstreamer pipeline runs successfully, but if I preview the audio in the AWS web console or via the browser example, the audio is coming out scrambled/garbled.
This can be reproduced with this simple test-generator pipeline:
gst-launch-1.0 \
    videotestsrc is-live=TRUE ! queue ! video/x-raw,width=1280,height=720,framerate=30/1 ! \
    x264enc bframes=0 speed-preset=veryfast bitrate=512 byte-stream=TRUE tune=zerolatency ! \
    video/x-h264,stream-format=avc,alignment=au,profile=baseline ! \
    kvsplugin name=kvs aws-region=us-east-1 channel-name=audio-issue enable-streaming=false connect-webrtc=true log-level=3 \
    audiotestsrc is-live=TRUE ! queue ! audioconvert ! audioresample ! faac ! \
    audio/mpeg,rate=48000,channels=2 ! kvs.

I think the underlying cause is codec incompatibility between the kvsplugin sinks and the underlying KVS WebRTC service. No matter which format that is given to kvsplugin (audio/mpeg, audio/x-mulaw, or audio/x-alaw), the plugin always reports it's codec as audio/opus. So the end result is the browser is decoding AAC encoded audio using the opus decoder.
It feels if the GST plugin is meant to support both KVS Video Streams and WebRTC, then it should handle some codec conforming, so that a single audio/video input can be routed to both a Video Stream or Signaling channel. If kvsplugin is meant to support both WebRTC and VideoStreams but not in the same pipeline then it may make sense to add an audio sink that accepts audio/x-opus. What's the best path forward here?

Thank you for bringing this to our attention.  It appears we have hardcoded opus here, we are working on the change properly declare support for opus/mulaw/alaw.   We do not support AAC in WebRTC.  We'll use the gstreamer parameter that's passed in to select which of opus/mulaw/alaw to send.  You can find our supported audio/video codecs here:
https://github.com/awslabs/amazon-kinesis-video-streams-webrtc-sdk-c/tree/master/src/source/Rtp/Codecs

@adamthesax ,
I have PR out to fix the issue? Would you like to give it a try?

@adamthesax ,
the changes are merged. Feel free to reach out if you face any issues.

Thanks for putting in a fix and sorry for the delay in getting back to you. I'm still having a tough time getting audio working cleanly. With your latest changes, I can send mulaw or alaw audio, which KVS recognizes as such (audio/PCMU), but playback is still distorted (my test tone sounds like it has shifted down in frequency). Maybe it is something with my pipeline, but it's pretty straight forward. Here's the updated pipeline which is sending mulaw:
gst-launch-1.0 \
    videotestsrc is-live=TRUE pattern=ball ! queue ! video/x-raw,width=1280,height=720,framerate=30/1 ! \
    x264enc bframes=0 speed-preset=veryfast bitrate=512 byte-stream=TRUE tune=zerolatency ! \
    video/x-h264,stream-format=avc,alignment=au,profile=baseline ! \
    kvsplugin name=kvs aws-region=us-east-1 channel-name=audio-issue enable-streaming=false connect-webrtc=true log-level=3 \
    audiotestsrc wave=8 sine-periods-per-tick=40 marker-tick-period=3 is-live=TRUE ! queue ! audioconvert ! audioresample ! mulawenc ! \
    audio/x-mulaw,rate=48000,channels=2 ! kvs.

Also there isn't a declared sink on kvsplugin for audio/x-opus so even though internally the plugin is detecting if we are sending opus, alaw or mulaw, there is no way to pipe opus audio into the plugin.
Thanks again for the help!

@adamthesax ,
I tested with the pipeline you provided previously, juts replacing with mulaw/alaw and the allow sample rate (8000 or 12000)
Yes, audio/x-opus was removed in previous PR since WebRTC supports opus but not producer. Will need to look into it.
In the meantime, can you try this pipeline? You raudio sample rate is 48000 which mulawenc or alawenc does not support. Check this: https://gstreamer.freedesktop.org/documentation/mulaw/mulawenc.html?gi-language=c
gst-launch-1.0 videotestsrc is-live=TRUE ! queue ! video/x-raw,width=1280,height=720,framerate=30/1 ! x264enc bframes=0 speed-preset=veryfast bitrate=512 byte-stream=TRUE tune=zerolatency ! video/x-h264,stream-format=avc,alignment=au,profile=baseline ! kvsplugin name=kvs channel-name=audio-issue enable-streaming=false connect-webrtc=true log-level=3 audiotestsrc is-live=TRUE ! queue ! audioconvert ! audioresample ! mulawenc ! audio/x-mulaw,rate=8000,channels=2 ! kvs.

@disa6302,
Thanks for getting back to me! Moving to 8000 sample rate does fix this problem, but the audio quality isn't great. From my understanding of the link you sent, mulawenc supports a range from 8000 to 192000. Consider this pipeline:
gst-launch-1.0 audiotestsrc is-live=TRUE ! queue ! audioconvert ! audioresample ! mulawenc ! audio/x-mulaw,rate=48000 ! decodebin ! autoaudiosink

mulawenc is happy to encode at various sample rates between 8kHz and 192kHz, but kvsplugin is expecting it to be at 8kHz. I looks like 8 kHz is baked into the webrtc-sdk but it's a bit unclear from the WebRTC spec if 8kHz is a hard requirements for PCM audio.
From WebRTC Audio Codec and Processing Requirements

For all cases where the endpoint is able to process audio at a sampling rate higher than 8 kHz, it is RECOMMENDED that Opus be offered before PCMA/PCMU.

I recognize the long term solution here may be to move to Opus for better quality at a lower bitrate (recognizing that there's still work to integrate opus support into kvsplugin), but in the short term, do you know if the 8kHz is an KVS/SDK limitation or an actual limitation of the WebRTC spec?

@adamthesax ,
I do notice that the audio is not clear at a sampling rate higher than 8KHz. And yes, we hardcode the audio sample rate in the WebRTC C SDK. And I am assuming the limit was set based on the recommendation. It would be best to proceed with Opus for now.
I have a PR sitting to include Opus. Can you try it out?

@disa6302 Yes happy to test out Opus support, just point me at the PR

https://github.com/aws-samples/amazon-kinesis-video-streams-demos/pull/162

Alright I checked out your branch and built it. I think the big problem right now in terms fo Opus support is there is not declared sink for audio/x-opus so GST can't build a pipeline from opusenc to kvsplugin. At a minimum I think we need to declare that kvsplugin accepts audio/x-opus here but when I add that, I get:
2021-03-29 19:59:13 ERROR   gst_kvs_plugin_change_state(): Failed to initialize track with 0x00000002
ERROR: from element /GstPipeline:pipeline0/GstKvsPlugin:kvs: Could not initialize supporting library.
Additional debug info:
/Users/211351/code/thirdparty/amazon-kinesis-video-streams-demos/gst/gst-kvs-plugin/src/GstPlugin.c(1296): gst_kvs_plugin_change_state (): /GstPipeline:pipeline0/GstKvsPlugin:kvs:
Failed to initialize with 0x00000002

I thus I think we also need to support opus in initTrackData()

I think that is where the problem is. The code needs some refactoring. The issue is, the KVS producer does not support OPUS and the initTrackData uses producer to initialize. I am going to make some changes to outsource this to the plugin and then reference in WebRTC. Stay tuned...

@adamthesax ,
Can you test the PR?
I tested with a-law, mu-law and opus and it works. Would like o get it verified from you as well.

🙌  Yes! Opus is working for me. Ran into one little missing ;:
GstPlugin.c:1372:59: error: expected ';' after expression
```
After I patched that up opus is working and sounding great!

@adamthesax ,
PR is merged. Feel free to open a new issue if you have any questions.

authors: adamthesax disa6302 hassanctech


text:
Add MapLibre for Android 9.2.1

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Add the Bintray Maven URL at allprojects.repositories in build.gradle (Project: Amazon_Location_Service_Demo)
Update dependencies in build.gradle (Module: Amazon_Location_Service_Demo.app)

Figure 1:  The two build.gradle files to update

Figure 2:  Result after adding MapLibre

Figure: Demo of Amazon Location with MapLibre

Figure: Demo of Amazon Location with MapLibre

repo: aws-samples/amazon-location-samples


text:
RAG OpenSearch deployment fails with "image size exceeds 10 GiB limit)

deploying RAG OpenSearch and got:
AwsGenaiLllmChatbotStack: creating CloudFormation changeset...
9:49:14 AM | CREATE_FAILED        | AWS::Lambda::Function                           | OpenSearchVectorSe...exDocument36B
9A569
Resource handler returned message: "Lambda function AwsGenaiLllmChatbotStack-OpenSearchVectorSearchInd-ncsjPLa7PulW reac
hed terminal FAILED state due to InvalidImage(SizeLimitExceeded: Uncompressed container image size exceeds 10 GiB limit)
and failed to stabilize" (RequestToken: 015e710a-ca90-5ada-4a74-c4b7ca44b586, HandlerErrorCode: NotStabilized)

 ❌  AwsGenaiLllmChatbotStack failed: Error: The stack named AwsGenaiLllmChatbotStack failed to deploy: UPDATE_ROLLBACK_COMPLETE: Resource handler returned message: "Lambda function AwsGenaiLllmChatbotStack-OpenSearchVectorSearchInd-ncsjPLa7PulW reached terminal FAILED state due to InvalidImage(SizeLimitExceeded: Uncompressed container image size exceeds 10 GiB limit) and failed to stabilize" (RequestToken: 015e710a-ca90-5ada-4a74-c4b7ca44b586, HandlerErrorCode: NotStabilized)
    at FullCloudFormationDeployment.monitorDeployment (/home/ec2-user/.nvm/versions/node/v18.18.0/lib/node_modules/aws-cdk/lib/index.js:467:10232)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Object.deployStack2 [as deployStack] (/home/ec2-user/.nvm/versions/node/v18.18.0/lib/node_modules/aws-cdk/lib/index.js:470:179911)
    at async /home/ec2-user/.nvm/versions/node/v18.18.0/lib/node_modules/aws-cdk/lib/index.js:470:163159

closing because duplicate of issue #38

repo: aws-samples/aws-genai-llm-chatbot


text:
[Enhancement] Add Identity Perimeter Controls Part 1

Add VPC Endpoint policies w/PrincipalOrgID restriction
see quip with design specifications

Did not implement


text:
Update changelog

Issue #, if available:
Description of changes:
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Skipping code review since this pr is trivial.

repo: aws-solutions/aws-media-insights-engine


text:
Instance Scheduler should allow for instance type flexibility

Is your feature request related to a problem? Please describe.
Sometimes for a single specified instance type there is not enough capacity in the AZ. This results in an Insufficient Capacity Error (ICE). This occurs more frequently for peculiar types. May also occur during periods of high demand. If there is impact in another AZ or other region, there becomes a thundering herd of many requests to launch instances so demand may exceed available capacity. To avoid ICE's it's advisable for automation to be flexible as to which AZ and also flexible as to what type is deemed acceptable for launch.
Launch Templates can specify multiple possible instance types
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/create-launch-template.html#lt-instance-type
And Fleets in ec2 help flexibility by leveraging things like "attribute based instance type selection". https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-fleet-attribute-based-instance-type-selection.html
Describe the feature you'd like
Instance Scheduler solution should allow for optional use of ec2 Fleets and Launch Templates, as these are standard mechanisms that allow instance type flexibility and AZ flexibility. This will help keep everyone get back to work by launching their next preferred instance types on days when their first choice is not available.
Additional context

Hi @playphil
Thanks for submitting this FR! The correct handling of ICE errors is an ongoing discussion, and you present some interesting ideas that I will bring to the rest of the team for consideration.

Great ya, another way might also to be allow each to configure a small list of additional instance types, sizes or qualities to try if/when an ICE is received at time of attempting a re-launch of an existing instance that may not already be a part of a fleet nor launch template.

Hello
i have the same issue, you can maybe add a new tag that contain ec2 types catalog if the first encounter an error then it takes the second

Hello i have the same issue, you can maybe add a new tag that contain ec2 types catalog if the first encounter an error then it takes the second

Awesome idea, gives a simple way everyone could adopt incrementally. The optional additional tag values on some instances could then be attempted for launch anytime ICE is found in Cloudtrail. Example:
preferred-instance-types = m7i.xlarge,m6i.xlarge,m5a.xlarge,c7i.xlarge,r7i.xlarge

Hello, Any update?

Hi @playphil and @ashraf133, thanks for reaching out, and to help us prioritize items in our backlog, can you please let us know which company you are representing, and what your specific use cases are?

@shujacks the use case is explained in the issue. Please review the full contents here with your tech lead that understands the nature of ec2 physical capacity constraints and ICE. This is important for all large customers having thousands of instances or more. Instance type flexibility becomes even more important during world events, natural disasters, and with instance types that are in short supply in a given AZ. Instance type flexibility is a core principle of scalable, reliable use of ec2 instances. The companies we currently represent is of no relevance to this issue.

@shujacks , we encounter everyday ec2 capacity problem when starting instances
An error occurred (InsufficientInstanceCapacity) when calling the StartInstances operation
The solution that we suggest is adding a new tag that contains a list of ec2 instances types and try to start using the first type, if it is ok then everything is good, else try with the second type ...
as mentionned by @playphil preferred-instance-types = m7i.xlarge,m6i.xlarge,m5a.xlarge,c7i.xlarge,r7i.xlarge

Hi @playphil @ashraf133 thanks for the response. We certainly understand the ec2 capacity issue. However, the product team requested to get this information to prioritize this feature request since we have a long backlog to evaluate: 1. size of the deployment 2. how long have you been using the solution 3. use case (e.g. for dev account? for testing purpose?).
The answers to these questions will help us prioritize all customer asks, thank you.

i use it for around 2000 ec2 and 300 rds since 2022 for all environments

authors: CrypticCabub ashraf133 playphil shujacks

repo: aws-solutions/instance-scheduler-on-aws


text:
parseCognitoWebResponse does not return any indication that there was no code parameter in the URL

parseCognitoWebResponse will call the userhandler methods if it fails to retrieve cognito tokens after parsing the response URL, but that does not happen if it does not find a code parameter whatsoever. This makes it extremely hard to know if that succeeded or not.

Nevermind, just realized this is fixed in https://github.com/aws/amazon-cognito-auth-js/pull/135. Thanks!

repo: aws/amazon-cognito-auth-js


text:
Remove private subnets from environment CloudFormation template

After removing the NAT gateways it seems the private subnets are still being created, but I don't think we're using them anywhere so we can consider removing them as well.

This was a conscious decision as they don't cost anything and should be in the same level as public subnets. Resolving it

authors: efekarakus sonofachamp


text:
Emoji and Spinner glyphs don't work on Windows

Good news is, colors and the prompt work - but the emojis and spinner don't work as well.

#180 Removes emoji for Windows so we should be good for now

authors: hencrice kohidave


text:
Fix Bug in OTA

Fix Bug in OTA
Description
Since we are using QOS1, the broker can potentially send a job message while it is performing a stream.
In that case the stream get's interrupted and OTA fails.
This fix will discard job message while OTA is performing streaming operations.
Checklist:

[x ] I have tested my changes. No regression in existing tests.
[ ] My code is Linted.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/bot run check

repo: aws/amazon-freertos

license: MIT


text:
NullReferenceExecption when intializing AmazonSQSClient from EC2 instance with no assigned IAM role

Background:
Running a .NET Core client application on an EC2 instance.
Trying to initialize an AmazonSQSClient with only a region as a parameter
When no IAM role is assigned to the EC2 instance the initialization of the client fails with the following error
NullReferenceException -- Object reference not set to an instance of an object.
Exception.Data: ( )
Exception.TargetSite: Amazon.Runtime.ImmutableCredentials FetchCredentials()
Exception.Source: AWSSDK.Core
Exception.HResult: -2147467261
Exception.StackTrace:
at Amazon.Runtime.DefaultInstanceProfileAWSCredentials.FetchCredentials()
at Amazon.Runtime.DefaultInstanceProfileAWSCredentials.GetCredentials()
at Amazon.Runtime.Internal.CredentialsRetriever.PreInvoke(IExecutionContext executionContext)
at Amazon.Runtime.Internal.CredentialsRetriever.InvokeSync(IExecutionContext executionContext)
at Amazon.Runtime.Internal.RetryHandler.InvokeSync(IExecutionContext executionContext)
at Amazon.Runtime.Internal.CallbackHandler.InvokeSync(IExecutionContext executionContext)
at Amazon.Runtime.Internal.CallbackHandler.InvokeSync(IExecutionContext executionContext)
at Amazon.Runtime.Internal.ErrorCallbackHandler.InvokeSync(IExecutionContext executionContext)
at Amazon.Runtime.Internal.MetricsHandler.InvokeSync(IExecutionContext executionContext)
at Amazon.Runtime.Internal.RuntimePipeline.InvokeSync(IExecutionContext executionContext)
at Amazon.Runtime.AmazonServiceClient.Invoke[TResponse](AmazonWebServiceRequest request, InvokeOptionsBase options)

When I assign an IAM role, no error is thrown.
Expected Behavior
When no IAM role exists on the instance and the AmazonSQSClient is being initialized a more specific exception should get thrown, rather than the NullReferenceException.
Steps to Reproduce (for bugs)
Initialize an AmazonSQSClient on an EC2 instance without an IAM role
.NET Core Info
.NET Core SDK (reflecting any global.json):
 Version:   2.2.109
 Commit:    586f23c400

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.17763
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\2.2.109\

Host (useful for support):
  Version: 2.2.7
  Commit:  b1e29ae826

.NET Core SDKs installed:
  2.1.509 [C:\Program Files\dotnet\sdk]
  2.2.109 [C:\Program Files\dotnet\sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.1.13 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.13 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.13 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]

Hi @BlanchardT, what version of the SDK are you using?  I believe this has already been fixed.

AWSSDK.SQS v3.3.101.2
AWSSDK.Core v3.3.103.31
AWSSDK.EC2 v3.3.127

@klaytaybai any idea on this?

I assume you are not providing credentials at all then?  May I ask why?  If you are not providing credentials at all, you should have an associated IAM role if you want to connect to the service.

That's correct, there was no IAM role assigned to the EC2 instance that this ran on. This was during a test and the engineer running the test noticed this error in the log and was unaware of the necessary IAM roles. The null reference error didn't make it obvious as to what was missing.
In order to provide a nicer error for future engineers we could catch the null reference thrown from the AmazonSQSClient and rethrow a custom error but I think that the sdk could handle this a little better.

Is this something your team would consider fixing? We could catch the null reference but it's not nice to handle..

@BlanchardT you seem to be using older version of SDK. This was fixed couple of months ago.
https://github.com/aws/aws-sdk-net/commit/9b7e1f4bc7debb3ecea86b40c97c55bbdb6474c7#diff-19922f5d8acc62e7de2723c988c4e583

authors: BlanchardT klaytaybai runeetv

repo: aws/aws-sdk-net


text:
Unity 2019.x Errors with v3.3 on Android

Description
When initializing the AWS SDK for Amazon Mobile Analytics, we get an exception related to reflection.
Related: https://github.com/aws/aws-sdk-net/issues/1286
Reproduction Steps

Create a unity project with Unity 2019.x
Add Mobile Analytics package
Initialize the MobileAnalyticsManager
Build for Android
Use adb to watch the logs

Logs
ArgumentException: Object of type 'System.Object[]' cannot be converted to type 'UnityEngine.AndroidJavaObject[]'.
09-23 11:25:17.253 28732 28782 E Unity   :   at System.RuntimeType.CheckValue (System.Object value, System.Reflection.Binder binder, System.Globalization.CultureInfo culture, System.Reflection.BindingFlags invokeAttr) [0x00000] in <00000000000000000000000000000000>:0
09-23 11:25:17.253 28732 28782 E Unity   :   at System.Reflection.MonoMethod.ConvertValues (System.Reflection.Binder binder, System.Object[] args, System.Reflection.ParameterInfo[] pinfo, System.Globalization.CultureInfo culture, System.Reflection.BindingFlags invokeAttr) [0x00000] in <00000000000000000000000000000000>:0
09-23 11:25:17.253 28732 28782 E Unity   :   at System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x00000] in <00000000000000000000000000000000>:0
09-23 11:25:17.253 28732 28782 E Unity   :   at Amazon.Util.Internal.AmazonHookedPlatformInfo.Init () [0x00000] in <00000000000000000000000000000000>:0

Environment

"ProductVersion" : "3.3.802.0",
"CoreVersion"    : "3.3.107.40",
"MobileAnalytics"    "3.3.100.222",

OS Info: Android
Build Environment Unity 2019.x

Resolution

Solution is outlined here - replace the contents of each method in Core/Amazon.Util/Internal/_unity/AndroidInterop.cs with the following:

/*
 * Copyright 2010-2013 Amazon.com, Inc. or its affiliates. All Rights Reserved.
 *
 * Licensed under the Apache License, Version 2.0 (the "License").
 * You may not use this file except in compliance with the License.
 * A copy of the License is located at
 *
 *  http://aws.amazon.com/apache2.0
 *
 * or in the "license" file accompanying this file. This file is distributed
 * on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
 * express or implied. See the License for the specific language governing
 * permissions and limitations under the License.
 */

using System;
using System.Collections.Generic;
using System.Linq;
using System.Reflection;
using System.Runtime.InteropServices;
using System.Text;
using UnityEngine;

namespace Amazon.Util.Internal
{
    /// <summary>
    /// This class is used to make Android Java calls as an alternative to using Android macro's
    /// The class uses reflection but doesnto cache the PropertyInfo and MethodInfo,
    /// so it should be sparingly used so as not to impact performance.
    /// </summary>
    public class AndroidInterop
    {
        /// <summary>
        /// The API makes a call to a static java method on a class and returns a typed parameter
        /// </summary>
        /// <typeparam name="T"></typeparam>
        /// <param name="className"></param>
        /// <param name="methodName"></param>
        /// <param name="parameters"></param>
        /// <returns></returns>
        public static T CallStaticJavaMethod<T>(string className, string methodName, params object[] parameters)
        {
            AndroidJavaClass c = new AndroidJavaClass(className);
            return c.CallStatic<T>(methodName, parameters);
        }

        /// <summary>
        /// The API makes a call to a static java method on a class and returns an object of Type AndroidJavaObject
        /// </summary>
        /// <param name="className"></param>
        /// <param name="methodName"></param>
        /// <returns></returns>
        public static object GetJavaObjectStatically(string className, string methodName)
        {
            AndroidJavaClass c = new AndroidJavaClass(className);
            return c.CallStatic<AndroidJavaObject>(methodName);
        }

        /// <summary>
        /// This API makes a call to a method on an Object by passing the specified parameters and returns a typed parameter
        /// </summary>
        /// <typeparam name="T"></typeparam>
        /// <param name="androidJavaObject"></param>
        /// <param name="methodName"></param>
        /// <param name="parameters"></param>
        /// <returns></returns>
        public static T CallMethod<T>(object androidJavaObject, string methodName, params object[] parameters)
        {
            AndroidJavaObject j = (AndroidJavaObject)androidJavaObject;
            return j.Call<T>(methodName, parameters);
        }

        /// <summary>
        /// This API makes a call to a method on an Object by passing the specified parameters and returns an object of type AndroidJavaObject
        /// </summary>
        /// <param name="androidJavaObject"></param>
        /// <param name="methodName"></param>
        /// <param name="parameters"></param>
        /// <returns></returns>
        public static object CallMethod(object androidJavaObject, string methodName, params object[] parameters)
        {
            AndroidJavaObject j = (AndroidJavaObject)androidJavaObject;
            return j.Call<AndroidJavaObject>(methodName, parameters);
        }

        /// <summary>
        /// This API get a typed value from a static field
        /// </summary>
        /// <typeparam name="T"></typeparam>
        /// <param name="className"></param>
        /// <param name="methodName"></param>
        /// <returns></returns>
        public static T GetStaticJavaField<T>(string className, string methodName)
        {
            AndroidJavaClass c = new AndroidJavaClass(className);
            return c.GetStatic<T>(methodName);
        }

        /// <summary>
        /// This API returns a value of type AndroidJavaObject from a static field
        /// </summary>
        /// <param name="className"></param>
        /// <param name="methodName"></param>
        /// <returns></returns>
        public static object GetStaticJavaField(string className, string methodName)
        {
            AndroidJavaClass c = new AndroidJavaClass(className);
            return c.GetStatic<AndroidJavaObject>(methodName);
        }

        /// <summary>
        /// This API returns a value of type AndroidJavaObject from a field on an android java object
        /// </summary>
        /// <typeparam name="T"></typeparam>
        /// <param name="androidJavaObject"></param>
        /// <param name="methodName"></param>
        /// <returns></returns>
        public static T GetJavaField<T>(object androidJavaObject, string methodName)
        {
            AndroidJavaObject j = (AndroidJavaObject)androidJavaObject;
            return j.Get<T>(methodName);
        }

        /// <summary>
        /// Returns an AndroidJavaObject
        /// </summary>
        /// <returns></returns>
        public static object GetAndroidContext()
        {
            return GetStaticJavaField("com.unity3d.player.UnityPlayer", "currentActivity");
        }

    }
}

This solution gets rid of the reflection that causes the issue and uses the Unity methods directly.

This is a :bug: bug-report

Hey @GarethPikPok,
Thanks for reporting this, however as of SDK v3.5 we no longer separate Unity support from regular .NET support, if you are having trouble with this transition, please make sure you are targeting ^.NET 4.0 or ^.NET Standard 2.0 using Unity 2018.0 or later. This means that you can have full access to AWS services when using the .NET SDK with Unity, but if you would still like to use the legacy SDK for Unity, you can reference it from our archive repo.
If you would like to submit this fix to the archived repo, that would be excellent and someone will review your PR (although it may take some time since it is no longer officially supported).
Please let me know if you have any questions!
😸 😷


text:
Don't roll KCP when etcd not ready or no endpoints

Description of changes:
Coming from https://github.com/abhay-krishna/cluster-api/pull/26 and https://github.com/abhay-krishna/cluster-api/pull/25
This should be another layer of protection if something happens with the etcd nodes, at leats KCP won't roll new nodes with empty etcd endpoints.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

fixed checksums, adding lgtm manually

/approve

repo: aws/eks-anywhere-build-tooling


text:
update OWNERS

Issue #, if available:
Description of changes:
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/approve

/lgtm

/verify-owners

authors: natechensan vineeth-bandi

repo: aws/eks-distro-build-tooling


text:
Fix go mod to run linter

Issue #, if available:
Description of changes:
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/hold

/unhold
/approve

/lgtm

repo: aws/eks-distro-prow-jobs


text:
fix(python): correctly handle interface declarations on returned objects

When the returned type had a known base type, the interfaces tagged on the object
reference tag. This addresses this problem.
Fixes #976

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

AWS CodeBuild CI Report

Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@RomainMuller I tool the liberty of implementing this test in .NET and Java, just in case, and also noticed that a related test is missing from python, so I added it (all passed).

AWS CodeBuild CI Report

Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

using cdk 1.16.3 (build 8c7f593)
'Resource' object has no attribute ... error still exists.
vpc = ec2.Vpc.from_lookup(self, "myvpc", vpc_id=VPC_ID)

@dxunix I had to update all of the Python dependencies in my virtual env before the issue was resolved.

authors: RomainMuller aws-cdk-automation dxunix eladb jvolkman

repo: aws/jsii


text:
feat: Changed OCI tag format

Fixes #N/A
Description
This PR changes the OCI tag pattern to be the SemVer version without the v prefix.
How was this change tested?
Does this change impact docs?

[ ] Yes, PR includes docs updates
[ ] Yes, issue opened: #
[x] No

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Pull Request Test Coverage Report for Build 7933770498
Warning: This coverage report may be inaccurate.
This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 82.497%

Totals

Change from base Build 7927768624:
0.0%

Covered Lines:
5015

Relevant Lines:
6079

💛  - Coveralls

@jonathan-innis I've updated the docs and have standardised the pattern used to refer to Karpenter versions for consistency (see below).
Karpenter version: `0.34.1`
Karpenter minimum version: `0.34.0`+
Karpenter latest patch: `0.34.x` (this shouldn't be used where the minimum version would also work)

authors: coveralls stevehipwell

repo: aws/karpenter-provider-aws


text:
ci: setup release automation with release-please

Problem
To maintain SemVer versions of packages in Github and npm we rely on manual release process, which consists of next steps:

Collect and analyze change log
Choose next version number based on changes
Create and merging new release PR
Pushing new version tag to Github
Creating Github Release page
Publishing new version package to npm

Manual process becomes cumbersome when number of changes grows.
Solution
Replace manual process with automation using release-please Github Action.
With this setup, most of the steps above are automated. Automation relies on enforcement of Conventional Commits in git history. New process is implemented by using special pending Pull Request, which accumulates all changes committed to main branch automatically and maintained by release-please workflow. It can be separated in 2 main steps: 1) collecting pending release changes and 2) performing the release.
Initial configuration for release-please workflow maintains our existing git tags formats for all packages. Versioning pattern of unstable 0.x versions is also maintained.
Release Pull Request
Release Pull Request has github-actions bot as author, and will have chore(release): release packages from branch main title.
These Release PRs are kept up-to-date as additional work is merged to main. When we're ready to tag a release, simply merge the Release PR. Both squash-merge and merge commits work with Release PRs.
Collecting pending release changes
Collecting changes is done automatically for all packages, configured in release-please-config.json and .release-please-manifest.json files. release-please workflow is triggered on every merge to main branch and follows these steps:

release-please analyzes commits for each configured package by modified file paths.
For every updated package, release-please creates unreleased Changelog and computes new SemVer version based on impact of commit based on Conventional Commit type.
release-please updates pending Release Pull Request: commits changed files and updates Pull Request description.

Performing the release
To release accumulated changes, merge Release Pull Request. When the Release PR is merged, workflow takes the following steps:

Updates changelog file (for example CHANGELOG.md), along with other language specific files (for example package.json).
Tags the commit with the version number
Creates a GitHub Release based on the tag
Publishes released packages to npm registry

License
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Great to see this. I've had success with release-please on other projects.
Does this mean this repo will also adopt conventional commits (AFAIK release-please somewhat expects them)?
For reference, in aws-toolkit-vscode repo we have a single-file script that enforces our flavor of conventional commits. That may be useful as a lightweight, flexible alternative to the various GHA conventional-commit apps.

After we merge this PR, will the bot collet all the changes that have been merged to main since the last tag release for each package?

Should we only allow "Squash and merge" to include just the PR title in the changelog? I don't recall if we agreed on it.

After we merge this PR, will the bot collect all the changes that have been merged to main since the last tag release for each package?

Workflow collects all changes as they are merged to main, not when Release PR is merged. And yes, it does it by collecting all unreleased commits.

Should we only allow "Squash and merge" to include just the PR title in the changelog? I don't recall if we agreed on it.

Branches are already configured to only allow Squash and Rebase merge strategies.

After we merge this PR, will the bot collect all the changes that have been merged to main since the last tag release for each package?

Workflow collects all changes as they are merged to main, not when Release PR is merged. And yes, it does it by collecting all unreleased commits.

I meant this PR that sets up the release-please workflow. I'm wondering will the process look back at commits that are already merged to main or will it only start tracking from this point onward?

I meant this PR that sets up the release-please workflow. I'm wondering will the process look back at commits that are already merged to main or will it only start tracking from this point onward?

This is handled in initial configuration release-please-config.json in bootstrap-sha option - it tells from which commit should start looking

authors: justinmk3 natdurlik viktorsaws

repo: aws/language-server-runtimes


text:
Elaborated on how to work with IoT credentials.

It is not a straight-forward process to enable KVS and IoT credentials.
It is required to read and follow through https://docs.aws.amazon.com/kinesisvideostreams/latest/dg/how-iot.html.
Issue #, if available:
Description of changes:
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Thank you @byongwu ! Can you direct this change to develop branch instead and I can merge your contribution!


text:
Building a Near-Real-Time Discovery Platform

Added aws-blog-firehose-lambda-elasticsearch-near-real-time-discovery-platform folder for the post that will be published later this week

please ignore previous pull request for aws-blog-firehose-lambda-elasticsearch-near-real-time-discovery-platform. Added a new file

repo: awslabs/aws-big-data-blog


text:
Overriding the CA on Mac results in server certs to be always valid

I am using the basic_pub_sub example and passing a custom CA file results in the server certificate to be always valid.
Example:
./basic-pub-sub --endpoint "xxxxx-ats.iot.us-east-1.amazonaws.com" --cert device1004.pem --key privkey.pem --topic testTopic --ca_file self_signed_ca.pem

should fail, but works.

Merged fix. Thanks for the PR!

authors: justinboswell massi-ang

repo: awslabs/aws-c-io


text:
s3.read_parquet() handling NA value differently from Pandas

Describe the bug
When reading a parquet file containing Null values, awswrangler convert these values into pd.NA instead of None, which is the Pandas behavior.
How to Reproduce
Find a parquet file that contains null values and compare the loaded DataFrame using the following functions:

wr.s3.read_parquet()
pd.read_parquet()

For my case, the column with all string data was inferred to "string[python]" as dtype , with null values represented using pd.NA when using awswrangler. When the DF is loaded using pandas, the same column was inferred to "dtype('O')", with null values represented using None.
While i actually prefers the awswrangler behavior in this case, I wonder if this is intended as when migrating from using Pandas to WR this might be a unexpected surprise for some developers.
Expected behavior
No response
Your project
No response
Screenshots
No response
OS
Amazon Linux 2012
Python version
Python 3.9.5
AWS DataWrangler version
2.14.0
Additional context
No response

From the docs, wr.s3.read_parquet has a map_types parameter set to True by default. It converts pyarrow DataTypes to pandas ExtensionDtypes.
By default, pyarrow uses NA. Setting the parameter to False returns None
import pandas as pd
import awswrangler as wr

path = "s3://<my-bucket>/1197.parquet"

df = pd.DataFrame({
    "c0": [2, 1, 1, 2],
    "c1": ["test", None, None, None],
})

wr.s3.to_parquet(df=df, path=path)

df = wr.s3.read_parquet(path=path, map_types=False)
df
   c0    c1
0   2  test
1   1  None
2   1  None
3   2  None

df = wr.s3.read_parquet(path=path, map_types=True)
df
   c0    c1
0   2  test
1   1  <NA>
2   1  <NA>
3   2  <NA>

Can an argument like this be added to athena.read_sql_query?  It has the same issue.

Can an argument like this be added to athena.read_sql_query? It has the same issue.

Also addressed here:
https://github.com/awslabs/aws-data-wrangler/issues/1265

authors: aeryen jaidisido koberghe liquidcarbon

repo: awslabs/aws-data-wrangler


text:
pandas_sdk_layer Uses incorrect layer version us-east-1 LambdaLayer

Describe the bug
When using the pandas_sdk_layer() for a Lambda Layer added to a Lambda Function deployed in us-east-1 the utility does not attach that latest version of AWS SDK for pandas and instead adds the Lambda layer suffixed with :1
How to Reproduce
*P.S. Please do not attach files as it's considered a security risk. Add code snippets directly in the message body as much as possible.*

Use SQSToLambda Stage when defining your DDK Application and pass layers=[pandas_sdk_layer()]
Deploy the DDK App to us-east-1 region and check the lambda layer version of SDK for pandas attached

Expected behavior
Using the pandas_sdk_layer() to add a lambda layer should use the latest version unless a verison is explicitly specified
Your project
No response
Screenshots
No response
Environment
Provide your `pip list` output, particularly the version of the AWS DDK library you used. Providing this information may significantly improve resolution times.

OS
N/A
Python version
3.9
AWS DDK version
0.6.2
Additional context
No response

This feature is on hold in 1.x.x, closing this in favor of #212

authors: malachi-constant noah-paige

repo: awslabs/aws-ddk


text:
backcompat: cli

bugfix: make executable_search_paths computable if not specified and add it as
None.

explicitly check with a test with older RPC version and removal of
executable_search_paths param.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Lgtm

authors: TheSriram sanathkr

repo: awslabs/aws-lambda-builders


text:
TypeError when publishing from Kinesis to Redis

When fetching from a Kinesis stream and pushing to Redis (AWS Elasticache), I get the following error in CloudWatch:

Error occured while preparing Redis command: [TypeError: list argument must be an Array of Buffers.]

Not being familiar with Redis does some preparation need to be done before writing data?

I had not seen your issue before (GitHub configuration issue). Do you have any stack trace about this?

I think I have found the cause. Can you check again?

authors: billsinc julienlepine

repo: awslabs/aws-lambda-fanout


text:
Go debugging Delve API support

*Issue #886
Description of changes:
Update Usage documentation on how to run GO Delve debugger using API version 2 mode.
Checklist:

[x] Write documentation

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

This looks good to me.

This should be updated directly on our docs right, since that is the source of truth?
https://github.com/awsdocs/aws-sam-developer-guide
Side Note: @TheSriram and @sanathkr We need to make updates to push docs updates over there and remove the .md files that are no longer needed.

This should be updated directly on our docs right, since that is the source of truth?
https://github.com/awsdocs/aws-sam-developer-guide

Can I leave this for you guys to do, or would you like me to create an additional PR?

@jackmcguire1 Can you create the same PR on https://github.com/awsdocs/aws-sam-developer-guide. That repo should be our authoritative version of the docs.

authors: TheSriram jackmcguire1 jfuss

repo: awslabs/aws-sam-cli


text:
Additional args

Allows users to pass DEBUGGER_ARGS environment variable that will then be used upon execution of the function when debugging is enabled.

Happy to include this, but we should also document it somewhere in the debugging section of the readme.
"If you want to pass in additional arguments to the debugger you can....."

Perfect thanks! I will merge once the build is complete

authors: PaulMaddox bradydowling sanathkr

repo: awslabs/aws-sam-local


text:
chore: Codegen dev script takes args for manifest generator

Description of changes
Changes the scripts/codegen.sh development script to take arguments & pass them unchanged to the package manifest generator when it is called.
New/existing dependencies impact assessment, if applicable
No new dependencies were added to this change.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

/test

/test

/test

repo: awslabs/aws-sdk-swift