text:
Displaying data on tooltips that isn't X or Y data?

I'd like to grab data outside of the X and Y values for my tooltips in a Scatter Plot, because I'd like for each point to have a unique name pertaining to it's object. Is there any way to do this?
`
    @foreach (var data in ApData)
    {
        <ApexPointSeries  TItem="ApDataModel"
                          Items="ApData"
                          Name="@data.SSID"
                          SeriesType="SeriesType.Scatter"
                          XValue="@(x => x.PrimaryChannel)"
                          YValue="@(x => x.Signal)" />
    }

`
Currently this makes a new Series per object in the list ApData and plots the same data over and over on top of the last and all we see is the name of the most recent object.

Hi,
have you check sample: https://apexcharts.github.io/Blazor-ApexCharts/features/formatters#pass extra data

@joadan I have, but after a few hours of trying to figure it out, it seems overly complex for the simple task of adding a 3rd piece of data to a tooltip. I'm pretty much at a loss when trying to manipulate the code snippet you linked.

Clean up

repo: apexcharts/Blazor-ApexCharts


text:
x-axis tooltip hides the actual x value on hover

I'm using a custom formater and specificly this
tooltip: {
      enabled: true,
      x: {
        formatter: (val) => {
          return 'Total transactions'
        }
      }
    }

It works correctly but when i hover over a day, it hides that day and displays the value that it got. This is normal behavior but how can i fix it?

The problem in the above image is that Wednesday is hidden behind Total transactions in the X axis

It will override but not change dynamically while scoll, it's stick on the same situations

A new property xaxis.tooltip.formatter will be provided - which will override the default tooltip.x formatter function

It will override but not change dynamically while scoll, it's stick on the same situations

source: gharchive/issue

authors: jimfilippou mitul-srashtasoft

license: MIT

license_type: permissive

license_source: github-api


text:
Fuzzing Feature Request - Empty Required Parameter Tests

Original Payload:
{
    "engagement": {
        "crn": "crn{{$timestamp}}",
        ..
}

Assuming "crn" is the mandatory(aka required) field.
Fuzzing already has the feature to have negative tests to exclude mandatory parameters using the config:
"requiredFields": {
                      "enabled": true
                    },

which results in the crn field itself to be excluded
{
    "engagement": {
        ..
}

But, it would be great if we had a feature to allow the parameter to exist (either JSON or Query Parameter) but make it empty.
Expected Payload:
{
    "engagement": {
        "crn": "",
        ..
}

Workaround:
As of now, making minLength: 1 makes this possible by adding a test with no characters in the parameter. But, this doesn't work when the parameter is overwritten with Postman variables such as {{crn}}.

hi @aazizpc
That could be a valuable addition to generate a Fuzzing variation for the required field with an empty value.

@aazizpc
We got a similar request for "blank" values generated by Fuzzing and looking for more input from the users.
The fuzzy generation is based on the OpenAPI definition.
If the property has minLength option defined in OpenAPI, it will generate a fuzzy variance with a less characters.
If the property is marked as required, Fuzzing will drop the property.
For "blank", we don't have any support yet, but we have received this request already a couple of times.
How would you want configure this?
Would you add a certain x-portman-fuzz-blank: true to your OpenAPI or configure it in the Portman config, and define blanking for certain properties?

@thim81
Thanks for the consultation. We would not want to add any new annotation to the OpenAPI spec since we already have the information that the field is mandatory through the existing spec.
We already have the "requiredFields" Fuzzing option too that can be made to generate the two negative tests - one for skipping the field itself, and one for having the empty value, but if we do need an extra option to clarify - We would be satisfied with perhaps a new Fuzzing option in the portman-config.api.json file like "requiredFieldsEmpty" as shown in the config snippet below.
    "variationTests": [
      {
        "openApiOperation": "*::*",
        "variations": [
          {
            "name": "Unprocessable",
            "fuzzing": [
              {
                "requestBody": [
                  {
                    "requiredFields": {
                      "enabled": true
                    },
                    "requiredFieldsEmpty": {
                      "enabled": true
                    },
                    "minimumNumberFields": {
                      "enabled": true
                    },
                    "maximumNumberFields": {
                      "enabled": true
                    },
                    "minLengthFields": {
                      "enabled": true
                    },
                    "maxLengthFields": {
                      "enabled": true
                    }
                  }
                ],

And of course, this should apply to not just requestBody, but also requestQueryParams and requestHeaders.
And, this should also work when we overwrite the value in the overwrites section with anything - including a postman variable such as {{crn}}.
We understand that this will work only with quoted string data in JSON bodies, because with numbers and bool, we can't have double quoted values. But with headers, query parameters, it can apply to all field types.

@nicklloyd What do you think of introducing the extra option "requiredFieldsEmpty"?
Or should the empty value test generation be part of the generated "requiredFields"?

@aazizpc I'm leaning towards the additional "requiredFieldsEmpty", since it will allow you to control the fuzzing for "required" and "empty" separately.
Pulling in some Portman experts from the community: @savage-alex @danielkocot @tillig here to see what their opinion is on the Fuzzing topic?

Should we introduce an extra option "requiredFieldsEmpty"? Or should the empty value test generation be part of the generated "requiredFields"?

I think for request body separating "missing" from "empty" is interesting. It may be that totally missing or null is treated differently than empty string, in which case that might be valuable to test.
Some thoughts, in no particular order, but which may apply to how this might be considered.
I work primarily in .NET, and missing/null/empty gets handled somewhat differently depending on data type and whether it's in the request body, headers, and query string parameters.

If a query string parameter is missing or empty it's the same as if it wasn't passed at all. http://server?foo=1&bar= is treated as though bar wasn't provided. I'm not sure how to pass an empty string or null on a query string, but if you were to pass http://server?foo=1&bar="" then bar would be the literal "" string, including the quotes. We've already run into some weirdness with fuzzing and query string parameters where we have a schema saying "this parameter isn't required, but if it's provided, it has a minLength=1. The test that gets generated is to pass ?param= (a zero-length string) but this comes across as "the parameter wasn't provided, so use the behavior as though the parameter wasn't passed." We usually turn off required parameter fuzzing for query strings.
If you omit a parameter in the body, it's treated the same as if you passed null. There's no differentiation once the ASP.NET framework has processed it and bound it to a model, so there's no way to validate missing vs. null.
I haven't tried it, but I think if you have an empty request header it's actually invalid HTTP. Which doesn't mean it shouldn't be tested, I guess, but it seems more like testing your HTTP server's protocol adherence than API contract at that point.

Maybe what that boils down to is:
I think required-fields-empty separate from required-fields-being-present is interesting to control, but only for request bodies. For query strings, it's only a maybe depending on your web app framework. For headers it doesn't make sense.

@tillig This is exactly the kind of input, I was looking for.
Based on your input, it is clear that the fuzzing feature can be greatly improved to generate the mentioned cases.
It has also become apparent that the body, header & query param handling will become more different.
If I translate your input into expected cases, I have this list:
JSON Request body:

omit properties {"name": "John"}
empty values for properties  {"name": "John", "surname": ""}
null values for properties {"name": "John", "surname": null}

Question: Since Portman Fuzzing is based on OpenAPI info, should the (1) (2) cases be applied on "required", with the option to control both cases in the Portman config?
Example OpenAPI required property trigger:
components:
  schemas:
    User:
      type: object
      required:
        - name
        - surname
      properties:
        name:
          type: string
          example: "John"
        surname:
          type: string
          example: "Doe"

Portman Config:
{
  "requiredFields": {
    "enabled": true
  },
  "requiredFieldsEmpty": {
    "enabled": true
  }
}

And for case (3), Would it be based on "nullable: false" in OpenAPI?
Example OpenAPI null fuzzing trigger
components:
  schemas:
    User:
      type: object
      properties:
        name:
          type: string
          example: "John"
        surname:
          type: string
          nullable: false
          example: "Doe"

Portman Config:
{
  "nullFields": {
    "enabled": true
  }
}

Or what would you expect/desire to trigger & control the Fuzzing generation?

@tillig For the query params, the empty values are indeed handled depending on the backend framework.
Would it make sense to provide some additional options in Portman to define this?
I detected these cases for query parameters:
Request Query Params:

omit query params http://example.com/api?name=john
empty values for query params http://example.com/api?name=&surname= AND/OR http://example.com/api?name=""&surname="" (which can be defined by a Portman config)
null values for query params http://example.com/api?name=null&surname=null AND/OR http://example.com/api?name=&surname= (which can be defined by a Portman config)

At this point, for me the nullable case is getting more and more clear.
About the empty case for request body, how would you define which properties should be blank values?
I was thinking in the direction of the "required" openapi in combination with a Portman specific config, but after reading your very valid feedback, I wonder how you define this?

For request bodies I think empty string tests are for fields with type: string and one or more of...

minLength greater than zero.
pattern, but where the pattern specified does not match empty string.
any format annotation since none of those appear to allow empty string; though per the JSON schema I gather format is less about validation so maybe this one can be ignored.

@tillig Looking at the initial starting point of this, there was the request to fuzz "required" properties also with blank values.
The advice here would be that @aazizpc should add minLenght to his engagement  property.
Does this mean that there is no use-case anymore for Portman requiredFieldsEmpty config?
Or would you generate requiredFieldsEmpty cases for any string field?

I'll try to find time to setup a POC to show the behaviour and how the config would look like.

This is the little POC, I created using the latest Portman version:
OpenAPI
openapi: 3.0.1
info:
  title: Sample API
  description: API description in Markdown.
  version: 1.0.0
servers:
  - url: 'http://api.example.com/v1'
paths:
  /users:
    post:
      summary: Create a new user
      requestBody:
        content:
          application/json:
            schema:
              type: object
              required:
                - name
                - surname
              properties:
                name:
                  type: string
                  description: The user's name.
                surname:
                  type: string
                  description: The user's surname.
                  minLength: 1
      responses:
        '201':
          description: Created

Portman
{
  "version": 1.0,
  "tests": {
    "variationTests": [
      {
        "openApiOperation": "*::/*",
        "variations": [
          {
            "name": "Bad",
            "openApiResponse": "400",
            "fuzzing": [
              {
                "requestBody": [
                  {
                    "requiredFields": {
                      "enabled": true
                    },
                    "minLengthFields": {
                      "enabled": true
                    }
                  }
                ]
              }
            ],
            "tests": {
              "contractTests": [
                {
                  "statusCode": {
                    "enabled": true,
                    "code": 400
                  }
                },
                {
                  "jsonBody": {
                    "enabled": true
                  }
                },
                {
                  "contentType": {
                    "enabled": true
                  }
                },
                {
                  "schemaValidation": {
                    "enabled": true
                  }
                }
              ]
            }
          }
        ]
      }
    ]
  },
  "globals": {
    "stripResponseExamples": true
  }
}

Postman result

```
{
  "_": {
    "postman_id": "b9e37556-9698-4a04-b5e0-a6aa842061eb"
  },
  "item": [
    {
      "id": "835d5d08-8435-4a60-b568-cb1ede6e3582",
      "name": "Create a new user",
      "request": {
        "name": "Create a new user",
        "description": {
          "type": "text/plain"
        },
        "url": {
          "path": [
            "users"
          ],
          "host": [
            "{{baseUrl}}"
          ],
          "query": [],
          "variable": []
        },
        "header": [
          {
            "key": "Content-Type",
            "value": "application/json"
          }
        ],
        "method": "POST",
        "body": {
          "mode": "raw",
          "raw": "{\n  \"name\": \"dolore non in cupidatat\",\n  \"surname\": \"qui cillum ullamco et\"\n}",
          "options": {
            "raw": {
              "language": "json"
            }
          }
        }
      },
      "response": [
        {
          "_": {
            "postman_previewlanguage": "text"
          },
          "id": "6cb67cad-1ec6-449b-b88b-da4ef2e0f236",
          "name": "Created",
          "originalRequest": {
            "url": {
              "path": [
                "users"
              ],
              "host": [
                "{{baseUrl}}"
              ],
              "query": [],
              "variable": []
            },
            "method": "POST",
            "body": {
              "mode": "raw",
              "raw": "{\n  \"name\": \"dolore non in cupidatat\",\n  \"surname\": \"qui cillum ullamco et\"\n}",
              "options": {
                "raw": {
                  "language": "json"
                }
              }
            }
          },
          "status": "Created",
          "code": 201,
          "header": [
            {
              "key": "Content-Type",
              "value": "text/plain"
            }
          ],
          "body": "",
          "cookie": []
        }
      ],
      "event": [],
      "protocolProfileBehavior": {
        "disableBodyPruning": true
      }
    },
    {
      "id": "87cbffa2-131e-44ff-9ba7-ed7ac308c214",
      "name": "Variation Tests",
      "item": [
        {
          "id": "63de135a-2500-496a-bd7c-4caa670d605f",
          "name": "555-fuzz-blank Variations",
          "item": [
            {
              "id": "a9ecc22d-e512-4f8a-8db7-72e25d0370b8",
              "name": "Create a new user[Bad][required name]",
              "request": {
                "name": "Create a new user[Bad][required name]",
                "description": {
                  "type": "text/plain"
                },
                "url": {
                  "path": [
                    "users"
                  ],
                  "host": [
                    "{{baseUrl}}"
                  ],
                  "query": [],
                  "variable": []
                },
                "header": [
                  {
                    "key": "Content-Type",
                    "value": "application/json"
                  }
                ],
                "method": "POST",
                "body": {
                  "mode": "raw",
                  "raw": "{\n    \"surname\": \"qui cillum ullamco et\"\n}",
                  "options": {
                    "raw": {
                      "language": "json"
                    }
                  }
                }
              },
              "response": [],
              "event": [],
              "protocolProfileBehavior": {
                "disableBodyPruning": true
              }
            },
            {
              "id": "21e8c221-7cbf-4ea1-afd0-3b544355a84e",
              "name": "Create a new user[Bad][required surname]",
              "request": {
                "name": "Create a new user[Bad][required surname]",
                "description": {
                  "type": "text/plain"
                },
                "url": {
                  "path": [
                    "users"
                  ],
                  "host": [
                    "{{baseUrl}}"
                  ],
                  "query": [],
                  "variable": []
                },
                "header": [
                  {
                    "key": "Content-Type",
                    "value": "application/json"
                  }
                ],
                "method": "POST",
                "body": {
                  "mode": "raw",
                  "raw": "{\n    \"name\": \"dolore non in cupidatat\"\n}",
                  "options": {
                    "raw": {
                      "language": "json"
                    }
                  }
                }
              },
              "response": [],
              "event": [],
              "protocolProfileBehavior": {
                "disableBodyPruning": true
              }
            },
            {
              "id": "84379cfc-023d-4f83-a3cf-4c06ab52544c",
              "name": "Create a new user[Bad][minimum length surname]",
              "request": {
                "name": "Create a new user[Bad][minimum length surname]",
                "description": {
                  "type": "text/plain"
                },
                "url": {
                  "path": [
                    "users"
                  ],
                  "host": [
                    "{{baseUrl}}"
                  ],
                  "query": [],
                  "variable": []
                },
                "header": [
                  {
                    "key": "Content-Type",
                    "value": "application/json"
                  }
                ],
                "method": "POST",
                "body": {
                  "mode": "raw",
                  "raw": "{\n    \"name\": \"dolore non in cupidatat\",\n    \"surname\": \"\"\n}",
                  "options": {
                    "raw": {
                      "language": "json"
                    }
                  }
                }
              },
              "response": [],
              "event": [],
              "protocolProfileBehavior": {
                "disableBodyPruning": true
              }
            }
          ],
          "event": []
        }
      ],
      "event": []
    }
  ],
  "event": [],
  "variable": [
    {
      "type": "string",
      "value": "http://api.example.com/v1",
      "key": "baseUrl"
    }
  ],
  "info": {
    "_postman_id": "b9e37556-9698-4a04-b5e0-a6aa842061eb",
    "name": "555-fuzz-blank",
    "schema": "https://schema.getpostman.com/json/collection/v2.1.0/collection.json",
    "description": {
      "content": "API description in Markdown.",
      "type": "text/plain"
    }
  }
}
```

About the {{CNR}} handling, this also seems to work as expected.
Same OpenAPI as above
Portman config
{
  "version": 1.0,
  "tests": {
    "variationTests": [
      {
        "openApiOperation": "*::/*",
        "variations": [
          {
            "name": "Bad",
            "openApiResponse": "400",
            "fuzzing": [
              {
                "requestBody": [
                  {
                    "requiredFields": {
                      "enabled": true
                    },
                    "minLengthFields": {
                      "enabled": true
                    }
                  }
                ]
              }
            ],
            "overwrites": [
              {
                "openApiOperation": "*::/*",
                "overwriteRequestBody": [
                  {
                    "key": "surname",
                    "value": "{{CNR}}"
                  }
                ]
              }
            ],
            "tests": {
              "contractTests": [
                {
                  "statusCode": {
                    "enabled": true,
                    "code": 400
                  }
                },
                {
                  "jsonBody": {
                    "enabled": true
                  }
                },
                {
                  "contentType": {
                    "enabled": true
                  }
                },
                {
                  "schemaValidation": {
                    "enabled": true
                  }
                }
              ]
            }
          }
        ]
      }
    ]
  },
  "globals": {
    "stripResponseExamples": true
  }
}

Postman Results

It still makes me wonder, if there is a need for fuzzing where blank values are used? Using requiredFieldsEmpty ?

I'm not sure why - required doesn't imply not empty.

Summon @miriamgreis

authors: aazizpc danielkocot thim81 tillig

repo: apideck-libraries/portman


text:
docs: Move code snippets to separate files

Move code snippets to separate files.
Some fixes in deployment to Apify.
It was rendered & checked locally.

EDIT: yes you did, so I'll close that PR?

yes please, thanks

source: gharchive/pull-request

repo: apify/crawlee-python


text:
"RangeError: Map maximum size exceeded" when uploading file

Which package is this bug report for? If unsure which one to select, leave blank
@crawlee/core
Issue description
Hi there I have an issue with uploading a zip file to my Actor's key value store using Apify version 3.1 "apify@3.1.1" and Crawlee version 3.1 "crawlee@3.1.3" when uploading a file over 9 MB in size.
Attached example.

Code sample
import { Actor } from "apify";
await Actor.init();
let numbers = Array.from(([...Array(18_100_000).keys()]).map((i) => i * 3_000_000))

let zip = Buffer.from([...numbers]);
console.log(zip.byteLength);

await Actor.setValue("numbers.test", zip, { contentType: "application/zip" });
await Actor.exit();

Package version
apify@3.1.1, crawlee@3.1.3
Node.js version
v18.7.0
Operating system
Mac OS 10.15
Apify platform

[ ] Tick me if you encountered this issue on the Apify platform

I have tested this on the next release
No response
Other context
No response

Can you provide the full stack trace pls?

I just tested it
/Users/lukas/apify/demo2/node_modules/@sapphire/shapeshift/dist/index.js:1213
    const inputEntries = new Map(Object.entries(value));
                         ^

RangeError: Value undefined out of range for undefined options property undefined
    at Map.set (<anonymous>)
    at new Map (<anonymous>)
    at ObjectValidator.handleIgnoreStrategy (/Users/lukas/apify/demo2/node_modules/@sapphire/shapeshift/dist/index.js:1213:26)
    at ObjectValidator.handlePassthroughStrategy (/Users/lukas/apify/demo2/node_modules/@sapphire/shapeshift/dist/index.js:1294:25)
    at ObjectValidator.handleStrategy (/Users/lukas/apify/demo2/node_modules/@sapphire/shapeshift/dist/index.js:1112:47)
    at ObjectValidator.handle (/Users/lukas/apify/demo2/node_modules/@sapphire/shapeshift/dist/index.js:1205:17)
    at ObjectValidator.run (/Users/lukas/apify/demo2/node_modules/@sapphire/shapeshift/dist/index.js:187:23)
    at UnionValidator.handle (/Users/lukas/apify/demo2/node_modules/@sapphire/shapeshift/dist/index.js:1083:32)
    at UnionValidator.run (/Users/lukas/apify/demo2/node_modules/@sapphire/shapeshift/dist/index.js:187:23)
    at runPredicate (/Users/lukas/apify/demo2/node_modules/@sapphire/shapeshift/dist/index.js:1215:32)

It doesn't like buffer, converting to base64 is fine

@B4nan definitely something to fix upstream. I'll take a look, but its probably an easy fix temporarily on our side by making the validator check for instances of buffers before everything else

@vladfrangu  Thanks for your reply.
Is this an issue with the way Buffers are handled (in "Crawlee") within the validation code or is this an issue with objects of this size in general?

@metalwarrior665 Is guess that converting the Buffer to base64 would be a temporary solution to the problem. Thank you! I hope that this error gets fixed. I've pulling my hair out for weeks making a function to split my key-value store into smaller pieces to archive and then save to KVS for download... lol.
Good luck and have a nice new year!

Also I think there are other actors (on the Apify Store) using an older version of the Apify package and they do not have this issue. :) I could also use that instead I guess.

Update: Creating a file over 133 MB also crashes the program after converting that to a base64 string.
I've got it to work up to around 133 MB maximum.
Here it works:

Here it doesn't:

Oh it seems that the console failed immediately, but the debug session actually succeeded a long time after:

A new maximum found for uploading: 139.5 MB

@Umisyus This last error is that you run out of memory and is not caused by Crawlee, it is Node.js config. You need to assign more heap memory to your local process.
https://support.circleci.com/hc/en-us/articles/360009208393-How-Can-I-Increase-the-Max-Memory-for-Node-

@metalwarrior665 yes it is. The issue still appears when I set the limit to something like my maximum RAM amount (8192 MB), though it does work better with smaller sizes.

Another solution is to store

the ZIP file in the file-system and not in-memory.
the link to the ZIP file in the KV store.

@LeMoussel Have you tried this? Sounds like it would work! Thanks.

No I didn't try. But this type of solution is often used for DBMS. For example: The SQLite Zipfile Module.

Saving with fs.writeFiledies work, but after that kvs.getPublicUrl does not create an appropriate link.
Thank you all very much for your helpful suggestions.

you can use this code as a guide.
import fs from 'fs';
import {
    KeyValueStore,
    log
} from 'crawlee';
// https://github.com/cthackers/adm-zip
import AdmZip from 'adm-zip';

// Create outputs directory if it doesn't exist
if (!fs.existsSync("outputs")) {
    fs.mkdirSync("outputs");
}

log.info('Creating ZIP archive');
const numbersSave = Array.from(([...Array(100_000).keys()]).map((i) => i * 3_000_000));
const zipSave = new AdmZip();
zipSave.addFile("numbers.json", Buffer.from(JSON.stringify(numbersSave)));
log.info('ZIP archive created');
zipSave.writeZip('./outputs/numbers.zip');
log.info('ZIP archive saved to the file-system');

const store = await KeyValueStore.open('ZIP_Archive');
await store.setValue(
    'ZIP_Archive',
    {
        path: './outputs/numbers.zip' // path to a zip archive to read from the file-system.
    }
);
log.info('Path to a zip archive saved to KVS');

const zipFile = await store.getValue('ZIP_Archive');
log.info(`Loading numbers from ZIP file: ${zipFile.path}`);
const zipLoad = new AdmZip(zipFile.path);
const zipEntries = zipLoad.getEntries();
const numbersLoad = JSON.parse(zipEntries[0].getData().toString());
log.info(`Loaded ${numbersLoad.length} numbers from ZIP file`)

Thanks everyone.

Do you have a time for when the bug fix will be released?

Hello everyone. Is anyone else still having this issue?

My logs:

/Users/umit/.nvm/versions/node/v18.7.0/bin/node ./dist/src/kvs-test.js
INFO  System info {"apifyVersion":"3.1.1","apifyClientVersion":"2.6.1","crawleeVersion":"3.1.3","osType":"Darwin","nodeVersion":"v18.7.0"}
18100000
zip
Buffer(18100000) [0, 192, 128, 64, 0, 192, 128, 64, 0, 192, 128, 64, 0, 192, 128, 64, 0, 192, 128, 64, 0, 192, 128, 64, 0, 192, 128, 64, 0, 192, 128, 64, 0, 192, 128, 64, 0, 192, 128, 64, 0, 192, 128, 64, 0, 192, 128, 64, 0, 192, 128, 64, 0, 192, 128, 64, 0, 192, 128, 64, 0, 192, 128, 64, 0, 192, 128, 64, 0, 192, 128, 64, 0, 192, 128, 64, 0, 192, 128, 64, 0, 192, 128, 64, 0, 192, 128, 64, 0, 192, 128, 64, 0, 192, 128, 64, 0, 192, 128, 64, …]
Uncaught RangeError RangeError: Map maximum size exceeded
    at handleIgnoreStrategy (/Users/umit/Desktop/Github Test/pin_down/node_modules/@sapphire/shapeshift/dist/index.js:1173:26)
    at handlePassthroughStrategy (/Users/umit/Desktop/Github Test/pin_down/node_modules/@sapphire/shapeshift/dist/index.js:1254:25)
    at ObjectValidator.handleStrategy (/Users/umit/Desktop/Github Test/pin_down/node_modules/@sapphire/shapeshift/dist/index.js:1075:47)
    at handle (/Users/umit/Desktop/Github Test/pin_down/node_modules/@sapphire/shapeshift/dist/index.js:1165:17)
    at run (/Users/umit/Desktop/Github Test/pin_down/node_modules/@sapphire/shapeshift/dist/index.js:92:23)
    at handle (/Users/umit/Desktop/Github Test/pin_down/node_modules/@sapphire/shapeshift/dist/index.js:1046:32)
    at run (/Users/umit/Desktop/Github Test/pin_down/node_modules/@sapphire/shapeshift/dist/index.js:92:23)
    at runPredicate (/Users/umit/Desktop/Github Test/pin_down/node_modules/@sapphire/shapeshift/dist/index.js:1175:32)
    at handleIgnoreStrategy (/Users/umit/Desktop/Github Test/pin_down/node_modules/@sapphire/shapeshift/dist/index.js:1185:9)
    at ObjectValidator.handleStrategy (/Users/umit/Desktop/Github Test/pin_down/node_modules/@sapphire/shapeshift/dist/index.js:1068:47)
    at handle (/Users/umit/Desktop/Github Test/pin_down/node_modules/@sapphire/shapeshift/dist/index.js:1165:17)
    at parse (/Users/umit/Desktop/Github Test/pin_down/node_modules/@sapphire/shapeshift/dist/index.js:106:88)
    at setRecord (/Users/umit/Desktop/Github Test/pin_down/node_modules/@crawlee/memory-storage/resource-clients/key-value-store.js:196:12)
    at setValue (/Users/umit/Desktop/Github Test/pin_down/node_modules/@crawlee/core/storages/key_value_store.js:280:28)
    at setValue (/Users/umit/Desktop/Github Test/pin_down/node_modules/apify/actor.js:585:22)
    --- await ---
    at processTicksAndRejections (internal/process/task_queues:95:5)
    --- await ---
    at runMainESM (internal/modules/run_main:55:21)
    at executeUserEntryPoint (internal/modules/run_main:78:5)
    at  (internal/main/run_main_module:17:47)
Process exited with code 1

You're on an outdated Crawlee version! We've released this in v3.2.0 of Crawlee, so please upgrade to that and follow up if the issue persists

Oh okay!

Works!

As of Cralwee version 3.2.2 saving large files to the Key-Value Store does work. It will take a long time though! I am yet to see the program save a 50 MB file. As of writing, the process is still going after about 11 minutes.

Completed one run after tweaking the setup, took 21 minute to save a 45 MB file. any recommendations regarding performance? Thanks!

Code

let { Actor, log } = await import("apify");
async function newFunction(FILE_SIZE = 5000000) {
await Actor.init();
console.time()
let numbers = Array.from(([...Array(FILE_SIZE).keys()]).map((i) => i * 1));
let zip = Buffer.from([...numbers]);
console.log(zip.byteLength);
log.info(zip size: ${zip.byteLength / 1000000} MB);
await new Promise(async (resolve) => {
    log.info(`Saving zip to KVS...`)
    await Actor.setValue("TESTzip", zip, { contentType: "application/zip" });
    log.info(`Saved to KVS`);
    //@ts-ignore
    resolve();
});

console.timeEnd()
// .then(async () =>
// await Actor.exit();

};
await newFunction(45_000_000)

As of Cralwee version 3.2.2 saving large files to the Key-Value Store does work. It will take a long time though! I am yet to see the program save a 50 MB file. As of writing, the process is still going after about 11 minutes.

Is this while running locally or on platform? Also was this something you encountered in previous versions too? Can you send the code in a gist, please

by the way, creating a buffer that is so big will take a long while, which is not a crawlee issue! 😅 If you're going to work with big buffers, its best to use the streaming capability of KeyValueStores

Simple repro here on Apify. Finished almost instantly
https://console.apify.com/view/runs/7hTf8hCektIXxSrP8
The KV store was instant. The buffer creation took 6 seconds. So I think this has nothing to do with Crawlee.
import { Actor, log } from 'apify';

await Actor.init()

log.info(`Creating buffer`)
let numbers = Array.from(([...Array(45_000_000).keys()]).map((i) => i * 1));
let zip = Buffer.from([...numbers]);
console.log(zip.byteLength);
log.info(`zip size: ${zip.byteLength / 1000000} MB`);

log.info(`Saving zip to KVS...`)
await Actor.setValue("TESTzip", zip, { contentType: "application/zip" });
log.info(`Saved to KVS`);

await Actor.exit();

Hello @vladfrangu I am running is locally and this was something that I counted on previous versions (Crawlee version 3.1.3) as well. I have a gist here.

I see... Thing is, creating such massive arrays/buffers will be slow even if you don't bring in Crawlee/Apify in the mix! Try it out by removing all of the crawlee code and timing the function itself that makes the buffer - you'll see it takes a while.

@metalwarrior665 I'm glad the issue isn't with Apify then! I am testing on my machine and it hangs forever.

I see... Thing is, creating such massive arrays/buffers will be slow even if you don't bring in Crawlee/Apify in the mix! Try it out by removing all of the crawlee code and timing the function itself that makes the buffer - you'll see it takes a while.

Absolutely, @vladfrangu it does take a while to create the 'archive', however I see that the issue is happening while saving to the Key-Value Store on my machine. I will have to understand why.

Converting the data into a ReadableStream then running Actor.setValue() works for almost any file size!

authors: LeMoussel Umisyus metalwarrior665 vladfrangu

repo: apify/crawlee


text:
Proxy authentication error

Which package is this bug report for? If unsure which one to select, leave blank
@crawlee/cheerio (CheerioCrawler)
Issue description
I slightly modified the demo of the official website and used ProxyConfiguration to set a proxy list. Error 407 was reported, but my authentication information was carried in the url. I am not sure what operation to do
This is the exception log：
/usr/local/bin/node /Users/admin/Desktop/codebaes/crawlee_uitl/req.js
(node:93791) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
INFO  CheerioCrawler: Starting the crawler.
WARN  CheerioCrawler: Reclaiming failed request back to the list or queue. Detected a session error, rotating session...
Proxy responded with 407 Proxy Authentication Required: 29 bytes

 {"id":"35Zqyzcm4WOh60y","url":"https://www.zhihu.com/question/603336478/answer/3406593764","retryCount":1}
WARN  CheerioCrawler: Reclaiming failed request back to the list or queue. Detected a session error, rotating session...
Proxy responded with 407 Proxy Authentication Required: 29 bytes

 {"id":"35Zqyzcm4WOh60y","url":"https://www.zhihu.com/question/603336478/answer/3406593764","retryCount":2}
WARN  CheerioCrawler: Reclaiming failed request back to the list or queue. Detected a session error, rotating session...
Proxy responded with 407 Proxy Authentication Required: 29 bytes

 {"id":"35Zqyzcm4WOh60y","url":"https://www.zhihu.com/question/603336478/answer/3406593764","retryCount":3}
WARN  CheerioCrawler: Reclaiming failed request back to the list or queue. Detected a session error, rotating session...
Proxy responded with 407 Proxy Authentication Required: 29 bytes

 {"id":"35Zqyzcm4WOh60y","url":"https://www.zhihu.com/question/603336478/answer/3406593764","retryCount":4}
WARN  CheerioCrawler: Reclaiming failed request back to the list or queue. Detected a session error, rotating session...
Proxy responded with 407 Proxy Authentication Required: 29 bytes

 {"id":"35Zqyzcm4WOh60y","url":"https://www.zhihu.com/question/603336478/answer/3406593764","retryCount":5}
WARN  CheerioCrawler: Reclaiming failed request back to the list or queue. Detected a session error, rotating session...
Proxy responded with 407 Proxy Authentication Required: 29 bytes

 {"id":"35Zqyzcm4WOh60y","url":"https://www.zhihu.com/question/603336478/answer/3406593764","retryCount":6}
WARN  CheerioCrawler: Reclaiming failed request back to the list or queue. Detected a session error, rotating session...
Proxy responded with 407 Proxy Authentication Required: 29 bytes

 {"id":"35Zqyzcm4WOh60y","url":"https://www.zhihu.com/question/603336478/answer/3406593764","retryCount":7}
WARN  CheerioCrawler: Reclaiming failed request back to the list or queue. Detected a session error, rotating session...
Proxy responded with 407 Proxy Authentication Required: 29 bytes

 {"id":"35Zqyzcm4WOh60y","url":"https://www.zhihu.com/question/603336478/answer/3406593764","retryCount":8}
WARN  CheerioCrawler: Reclaiming failed request back to the list or queue. Detected a session error, rotating session...
Proxy responded with 407 Proxy Authentication Required: 29 bytes

 {"id":"35Zqyzcm4WOh60y","url":"https://www.zhihu.com/question/603336478/answer/3406593764","retryCount":9}
WARN  CheerioCrawler: Reclaiming failed request back to the list or queue. Detected a session error, rotating session...
Proxy responded with 407 Proxy Authentication Required: 29 bytes

 {"id":"35Zqyzcm4WOh60y","url":"https://www.zhihu.com/question/603336478/answer/3406593764","retryCount":10}
ERROR CheerioCrawler: Request failed and reached maximum retries. Error: Detected a session error, rotating session...
Proxy responded with 407 Proxy Authentication Required: 29 bytes

    at CheerioCrawler._requestFunction (/Users/admin/Desktop/codebaes/crawlee_uitl/node_modules/@crawlee/http/internals/http-crawler.js:448:23)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async wrap (/Users/admin/Desktop/codebaes/crawlee_uitl/node_modules/@apify/timeout/cjs/index.cjs:54:21) {"id":"35Zqyzcm4WOh60y","url":"https://www.zhihu.com/question/603336478/answer/3406593764","method":"GET","uniqueKey":"https://www.zhihu.com/question/603336478/answer/3406593764"}
INFO  CheerioCrawler: All requests from the queue have been processed, the crawler will shut down.
INFO  CheerioCrawler: Final request statistics: {"requestsFinished":0,"requestsFailed":1,"retryHistogram":[null,null,null,null,null,null,null,null,null,null,1],"requestAvgFailedDurationMillis":77,"requestAvgFinishedDurationMillis":null,"requestsFinishedPerMinute":0,"requestsFailedPerMinute":41,"requestTotalDurationMillis":77,"requestsTotal":1,"crawlerRuntimeMillis":1456}
INFO  CheerioCrawler: Error analysis: {"totalErrors":1,"uniqueErrors":1,"mostCommonErrors":["1x: Detected a session error, rotating session... (/Users/admin/Desktop/codebaes/crawlee_uitl/node_modules/@crawlee/http/internals/http-crawler.js:448:23)"]}
INFO  CheerioCrawler: Finished! Total 1 requests: 0 succeeded, 1 failed. {"terminal":true}

进程已结束，退出代码为 0

Code sample
async function getReqCheerio(url, push_redis, proxyUrls) {
    const start_time = Math.ceil(Date.now() / 1000);
    let crawlerOptions = {
        ...opt,
        async requestHandler({ request, response, body, proxyInfo }) {
            // console.log('URL:', request.url);
            console.log('Used proxy:', proxyInfo);
            push_redis({
                url: request.url,
                html: body.toString(),
                status: response.statusCode,
                rt: Math.ceil(Date.now() / 1000) - start_time,
            });
        },

    };

    if (proxyUrls && proxyUrls.length > 0) {
        crawlerOptions.proxyConfiguration = new ProxyConfiguration({
            proxyUrls:proxyUrls,
        })
    }

    const crawler = new CheerioCrawler(crawlerOptions);

    await crawler.run([url]);
}

const proxyUrls = ["http://**:**@***.225.30.161:11273"];
await getReqCheerio('https://www.zhihu.com/question/603336478/answer/3406593764', console.log, proxyUrls);

Package version
crawlee@3.10.0
Node.js version
v21.1.0
Operating system
macos
Apify platform

[ ] Tick me if you encountered this issue on the Apify platform

I have tested this on the next release
No response
Other context
No response

This agent, I only use other programming languages, or other modules is OK, you can disguise my ip, please take a look, thank you😭😭😭😭


text:
Question: Add validation for format

lib/validators.js validateTypeAndFormat
Define validation for format if type is String, the supported formats are date and dete-time.
How can I add with sagger-tools a Validation for a format like email or any other format?

What is happening now?

Hi Jeremy,
Just the question. How can I add with sagger-tools a Validation for a format like email or any other format?

Right now it does not and this would fall into #140.  In all honesty, we're in the process of breaking out the API parts of swagger-tools into a separate project which already has this: https://github.com/apigee-127/swagger-core-api/blob/master/docs/API.md#SwaggerApi+registerValidator

Ok, thank you Jeremy

authors: oliverhr whitlockjc

license: mit

license_source: bigquery


text:
Refactor project router files for consistency

Audit all project router.js files so they follow the same pattern:

each router file is located in a component
each component router file contains routes only for the component
component router files are in the /component/client/lib/router.js file
router onBefore or other hooks are either in core, or in the specific router file where they are used

@jykae thanks for going beyond the definition of done here. As per our discussion, and approval from @bajiat, please refactor the settings component into separate components, with the following structure:

/branding
/logo
/settings

/client

/menu

/proxies
/users

/account
/profile

@frenchbread and @nnn, please be aware that @jykae will be moving these files around, so that you have as few conflicts as possible.

Also note, this should not affect the user interface. It is just a code-level refactor. We want to keep the great work on the settings page, including the navigation menu.

@brylie Ok, did not notice to check out this issue, was following PR. I make further changes based on your comments & our discussion.

authors: brylie jykae

repo: apinf/api-umbrella-dashboard


text:
🛑 OSC is down

In 0daf339, OSC (https://openstorage.xyz) was down:

HTTP code: 0
Response time: 0 ms

Resolved: OSC is back up in 562493c.

repo: apinter/OSC-mon


text:
"Load" and "Install" should process http[s]://....zip

Both command should allow URLs that point to a ZIP file.
If the ZIP file's name conforms with the Tatin rules for package names:

the ZIP file should be downloaded into a TEMP location
the ZIP file should then be processed like any local package

That would allow the release of Tatin packages on GitHub, meaning that GitHub would act as a backend for Tatin packages.

It's much less attractive when you consider that the dependencies must be added in the same format: a dependency
group-foo-1.0.0

is usually considered to be found on the same Tatin server the original package is found on, otherwise it has to be a full URL like:
https://tatin.dev/group-f00-1.0.0

That implies that when the ZIP file is hosted on something like GitHub and the dependency lives on GitHub as well, then it must be specified as
https://github.com/{group}/{package}/releases/download/{version}/{packageID}.zip

Do we really want to go down this route? Does not look attractive to me.

It's much less attractive when you consider that the dependencies must be added in the same format: a dependency
group-foo-1.0.0

is usually considered to be found on the same Tatin server the original package is found on, otherwise it has to be a full URL like:
https://tatin.dev/group-f00-1.0.0

That implies that when the ZIP file is hosted on something like GitHub and the dependency lives on GitHub as well, then it must be specified as
https://github.com/{group}/{package}/releases/download/{version}/{packageID}.zip

Do we really want to go down this route? Does not look attractive to me.

No we don't!

No we don't!

repo: aplteam/Tatin


text:
SyntaxError: Unexpected token {

I'm getting the following error:
SyntaxError: Unexpected token {
    at new Script (vm.js:74:7)
    at createScript (vm.js:246:10)
    at Object.runInThisContext (vm.js:298:10)
    at Module._compile (internal/modules/cjs/loader.js:657:28)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:700:10)
    at Module.load (internal/modules/cjs/loader.js:599:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:538:12)
    at Function.Module._load (internal/modules/cjs/loader.js:530:3)
    at Module.require (internal/modules/cjs/loader.js:637:17)
    at require (internal/modules/cjs/helpers.js:20:18)

The error is from: node_modules/apollo-angular/ApolloModule.js:1 (function (exports, require, module, __filename, __dirname) { import { NgModule } from '@angular/core';
The complete code is at:
https://github.com/jonasarcangel/ModuleIssue

This is with dotnet core.
It happens when using "dotnet run" and navigating to any route. It does not
happen when I run in ClientApp folder using "npm run build:ssr && npm run
start".
On Mon, Aug 13, 2018 at 5:23 PM, Kamil Kisiela notifications@github.com
wrote:

@jonasarcangel https://github.com/jonasarcangel could you elaborate
more? When does it happen?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/apollographql/apollo-angular/issues/745#issuecomment-412430339,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ARtmV2dlWHOxO0tht4y_0IFVl0tb1D86ks5uQSl-gaJpZM4V5UIt
.

@jonasarcangel Could you check v1.2.0? We switched to use ng-packagr

I guess I can close it
https://github.com/kamilkisiela/apollo-angular-ssr/tree/ts-node

authors: jonasarcangel kamilkisiela

repo: apollographql/apollo-angular


text:
Can I prefetch a query which is called at server side on hovering a link or so

Can I prefetch a query which is called at server side on hovering a link or so
for example  we had like this for client queries from apollo docs

 const    client  =  useApolloClient()

<Link
            to={{
              pathname: `/${data.breed}/${data.id}`,
              state: { id: data.id }
            }}
            onMouseOver={() =>
              client.query({
                query: GET_DOG,
                variables: { breed: data.breed }
              })
            }
            style={{ textDecoration: "none" }}
          >
            <Dog {...data} url={data.displayImage} />
         </Link>

@phryneas can u help me in with to prefetch the query called at server side on hovering a link  when we are using this new "@apollo/experimental-nextjs-app-support/ssr" . Prefetching the query will help me to land to the page instantaneously

You would do exactly that. After the first page load, there is no further SSR in Next.js, so nothing changed here.

@phryneas  what do you suggest . what cud be done?

Ah, wait a second, you mean "server side", so we're talking RSC here, not SSR?
Next.js already does this for RSC page navigations if you use their Link component, nothing for you to do.

I'm doing some housekeeping so I'm closing some older issues that haven't seen activity in a while.
If this is still relevant, please feel free to reopen the issue.

authors: jinowac phryneas

repo: apollographql/apollo-client-nextjs


text:
Fixing issue with framework generation

Fixing an issue where the make command for building the apollo-ios framework was failing, also fixing issue with carthage not being able to find shared schemes to build from.
Closes apollographql/apollo-ios#3439
Closes apollographql/apollo-ios#3427

I updated the script to test it out with carthage.
pushd "$(mktemp -d)" || exit
echo 'github "apollographql/apollo-ios-xcframework" "fix/issue-3439"' > Cartfile
carthage bootstrap --platform iOS --use-xcframeworks

It appears it's building each scheme twice again because the shared scheme shared a name with a scheme Xcode generates for the targets from the swift package.
Would it be possible to rename the schemes tuist generates so they are unique? Maybe adding the suffix -xcframework? That was the workaround I found last time. Though we'd be fine continuing to do the in our project, we've already got that code in our makefile.

https://github.com/apollographql/apollo-ios/issues/3308
https://github.com/Carthage/Carthage/issues/3358

Hmmm ok, wondering if I create the xcode schemes manually in the tuist configuration and disable the auto-generation if it would avoid the issue with the spm package schemes, ill give that a shot and see, odd that Apollo is the only one being duplicated and not the other targets from the spm package

Actually no that wont work at least for the duplication, but doing the schemes manually and adjusting the names as you suggested seems like a good solution.

Oops, I should have included the logs, it builds every target twice.
*** No Cartfile.resolved found, updating dependencies
*** Fetching apollo-ios-xcframework
*** Checking out apollo-ios-xcframework at "a06f84a4ee2968f1c55dbc206b5163f441444cd1"
*** xcodebuild output can be found in /var/folders/_x/0yf9v78x1zzfd4pk03vx7f440000gq/T/carthage-xcodebuild.QilH1E.log
*** Building scheme "Apollo" in Apollo.xcworkspace
*** Building scheme "ApolloAPI" in Apollo.xcworkspace
*** Building scheme "Apollo" in Apollo.xcworkspace
*** Building scheme "ApolloAPI" in Apollo.xcworkspace
*** Building scheme "ApolloSQLite" in Apollo.xcworkspace
*** Building scheme "ApolloWebSocket" in Apollo.xcworkspace
*** Building scheme "ApolloSQLite" in Apollo.xcworkspace
*** Building scheme "ApolloWebSocket" in Apollo.xcworkspace

I also just noticed that the target for ApolloTestSupport isn't built. I assume it shouldn't be missing.

@CraigSiemens ok i updated the branch tuist now generates the schemes with a suffix like you suggested and the make commands are all working so I'm guessing this will solve your Carthage issue as well.
As for ApolloTestSupport its currently not supported as part of the xcframework setup, although its something I can look into adding.

That change seems to be working, each target was built once.
*** No Cartfile.resolved found, updating dependencies
*** Fetching apollo-ios-xcframework
*** Checking out apollo-ios-xcframework at "5cfce4fd24fd65107e500ef2f0fc78ab3375fa94"
*** xcodebuild output can be found in /var/folders/_x/0yf9v78x1zzfd4pk03vx7f440000gq/T/carthage-xcodebuild.XGit2r.log
*** Building scheme "ApolloWebSocket-xcframework" in Apollo.xcworkspace
*** Building scheme "ApolloSQLite-xcframework" in Apollo.xcworkspace
*** Building scheme "ApolloAPI-xcframework" in Apollo.xcworkspace
*** Building scheme "Apollo-xcframework" in Apollo.xcworkspace

Great, can get this merged into main, and it will officially release in a tagged release when we do our next apollo-ios release.

authors: BobaFetters CraigSiemens

repo: apollographql/apollo-ios-xcframework


text:
toFlow() and .watch() result in IllegalArgumentException: Flow has more than one element

Version
3.7.3
Summary
When using cache and network, both:

apollo.query(makeQuery()).fetchPolicy(FetchPolicy.CacheAndNetwork).toFlow()
apollo.query(makeQuery()).fetchPolicy(FetchPolicy.CacheAndNetwork).watch()

result in java.lang.IllegalArgumentException: Flow has more than one element.
If I use FetchPolicy.NetworkOnly no exception is thrown and the data is collected as expected. Am I using the FetchPolicy feature correctly? I assumed the flow would emit twice, once for the cache and one more time for the network. Likewise I assumed that .watch() will continuously emit whenever the cache updates, such as after a mutation in another part of the app that updates any of the data related to the original query.
I'm also noticing that after deleting the app and rerunning it, the sqlite database in the Device File Explorer is populated with data from a very old version of the schema, when the schema and data types were completely different. Could this be related? EDIT: For this cleaning and rebuilding the project fixed it. The flow exception still remains.

Thanks for reaching out. That sounds unexpected 🤔 . Can you copy paste the full stacktrace? Looks like there's a single() operator somewhere.

@martinbonnin Sure, does something like this help?

Thanks! It would be nice to have the lines too (as in exception.printStackTrace().
Looking around the code that's in there, I can find one single() there but I wouldn't expect this to emit multiple times...
How did you setup your interceptor chain/ApolloClient.Builder? Did you add anything else besides cache?

Ok! Thanks for your support. Here's what the stack trace printed:
HttpClient(_cause=java.lang.IllegalArgumentException: Flow has more than one element)
	at com.davidarvelo.app.core.interactors.BaseQueryInteractor$mapped$1.invokeSuspend(BaseQueryInteractor.kt:49)
	at com.davidarvelo.app.core.interactors.BaseQueryInteractor$mapped$1.invoke(Unknown Source:9)
	at com.davidarvelo.app.core.interactors.BaseQueryInteractor$mapped$1.invoke(Unknown Source:6)
	at kotlinx.coroutines.flow.FlowKt__ErrorsKt$catch$$inlined$unsafeFlow$1.collect(SafeCollector.common.kt:114)
	at kotlinx.coroutines.flow.FlowKt__ErrorsKt$catch$$inlined$unsafeFlow$1$1.invokeSuspend(Unknown Source:15)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.internal.ScopeCoroutine.afterResume(Scopes.kt:33)
	at kotlinx.coroutines.AbstractCoroutine.resumeWith(AbstractCoroutine.kt:102)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
	at android.os.Handler.handleCallback(Handler.java:942)
	at android.os.Handler.dispatchMessage(Handler.java:99)
	at android.os.Looper.loopOnce(Looper.java:201)
	at android.os.Looper.loop(Looper.java:288)
	at android.app.ActivityThread.main(ActivityThread.java:7898)
	at java.lang.reflect.Method.invoke(Native Method)
	at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:548)
	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:936)
Caused by: java.lang.IllegalArgumentException: Flow has more than one element
	at kotlinx.coroutines.flow.FlowKt__ReduceKt$single$2.emit(Reduce.kt:58)
	at kotlinx.coroutines.flow.internal.SafeCollectorKt$emitFun$1.invoke(SafeCollector.kt:15)
	at kotlinx.coroutines.flow.internal.SafeCollectorKt$emitFun$1.invoke(SafeCollector.kt:15)
	at kotlinx.coroutines.flow.internal.SafeCollector.emit(SafeCollector.kt:87)
	at kotlinx.coroutines.flow.internal.SafeCollector.emit(SafeCollector.kt:66)
	at com.apollographql.apollo3.cache.normalized.FetchPolicyInterceptors$CacheAndNetworkInterceptor$1$intercept$1$invokeSuspend$$inlined$map$1$2.emit(Emitters.kt:223)
	at kotlinx.coroutines.flow.FlowKt__ErrorsKt$catchImpl$2.emit(Errors.kt:158)
	at kotlinx.coroutines.flow.FlowKt__ChannelsKt.emitAllImpl$FlowKt__ChannelsKt(Channels.kt:62)
	at kotlinx.coroutines.flow.FlowKt__ChannelsKt.access$emitAllImpl$FlowKt__ChannelsKt(Channels.kt:1)
	at kotlinx.coroutines.flow.FlowKt__ChannelsKt$emitAllImpl$1.invokeSuspend(Unknown Source:14)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	... 9 more

This is the contents of my ApolloClient.Builder:
private var appSettingsStorage: SettingsStorage? = null

object DataClient {

    private const val AUTHORIZATION_HEADER = "Authorization"

    private val memoryFirstThenSqlCacheFactory = MemoryCacheFactory(10 * 1024 * 1024)
    // I used to use this but it's buggy for now.
    // Seems SQLite is returning duplicate records and
    // Apollo cancels the flow because it expects a single record.
    // TODO: Maybe in a later version of Apollo (as of this writing, 3.7.3) I can re-enable this.
//    .chain(SqlNormalizedCacheFactory("habbit.db"))

    val apollo = ApolloClient.Builder()
        .serverUrl("http://localhost:4000/graphql")
        .addHttpInterceptor(AuthorizationInterceptor())
        // Note that mutations will always be sent as POST requests, regardless of these settings, as to avoid hitting caches.
        .autoPersistedQueries(
            // For the initial hashed query that does not send the actual Graphql document
            httpMethodForHashedQueries = HttpMethod.Get,
            // For the follow-up query that sends the full document if the initial hashed query was not found
            httpMethodForDocumentQueries = HttpMethod.Get
        )
        .normalizedCache(normalizedCacheFactory = memoryFirstThenSqlCacheFactory)
        .build()

    fun configure(settingsStorage: SettingsStorage) {
        appSettingsStorage = settingsStorage
    }

    class AuthorizationInterceptor : HttpInterceptor {
        override suspend fun intercept(request: HttpRequest, chain: HttpInterceptorChain): HttpResponse {
            val requestBuilder = request.newBuilder().let {
                val authToken = appSettingsStorage?.authToken
                if (authToken != null) {
                    it.addHeader(AUTHORIZATION_HEADER, "Bearer $authToken")
                } else {
                    it
                }
            }
            return chain.proceed(requestBuilder.build())
        }
    }
}

Thanks for sending that! Coroutines do not help with investigating the stack trace unfortunately... Maybe try without autopersisted queries? Does that change anything?
I'll work on a small reproducer project, see if we can manage to reproduce something

I can reproduce and this seems to be linked to auto persisted queries indeed. I'll keep digging

Turns out this is the culprit single()
I'll investigate how to make the configuration more robust. In the meantime, you can workaround this issue by configuring the auto persisted queries after the cache:
    val apollo = ApolloClient.Builder()
        .serverUrl("http://localhost:4000/graphql")
        .addHttpInterceptor(AuthorizationInterceptor())
        // Configure cache first
        .normalizedCache(normalizedCacheFactory = memoryFirstThenSqlCacheFactory)
        // And APQ last
        // Note that mutations will always be sent as POST requests, regardless of these settings, as to avoid hitting caches.
        .autoPersistedQueries(
            // For the initial hashed query that does not send the actual Graphql document
            httpMethodForHashedQueries = HttpMethod.Get,
            // For the follow-up query that sends the full document if the initial hashed query was not found
            httpMethodForDocumentQueries = HttpMethod.Get
        )
        .build()

That was so fast! I added the workaround and it works as expected. Thank you so much!

authors: darvelo martinbonnin

repo: apollographql/apollo-kotlin


text:
Implement addTypename option and remove queryTransformer

Fixes #616

TODO:

[x] If this PR is a new feature, reference an issue where a consensus about the design was reached (not necessary for small changes)
[x] Make sure all of the significant new logic is covered by tests
[x] Rebase your changes on master so that they can be merged easily
[x] Make sure all tests and linter rules pass
[x] Update CHANGELOG.md with your change
[x] Add your name and email to the AUTHORS file (optional)
[ ] If this was a change that affects the external API, update the docs and post a link to the PR in the discussion

FYI: the jsdoc for the ApolloClient constructor still documents the unsupported queryTransformer rather than addTypename

authors: glasser stubailo

repo: apollostack/apollo-client


text:
Restructure repository to be a bit more sane

Fixes #14

thanks @cjtf0991

repo: apollozkp/zkp-subnet


text:
[FEATURE] hovering of login signup to be improved

Description
if shrinks on hover which doesnot look great , would like to improve it
Screenshots
No response
Additional information
No response

@harshita-2003 Someone already working on this in another issue. Please find an another one

repo: apoorvaron/Shorty


text:
Add img tag in allowed

allowedTags: [ 'h3', 'h4', 'h5', 'h6', 'blockquote', 'p', 'a', 'ul', 'ol',
  'nl', 'li', 'b', 'i', 'strong', 'em', 'strike', 'code', 'hr', 'br', 'div',
  'table', 'thead', 'caption', 'tbody', 'tr', 'th', 'td', 'pre', 'iframe',  --> 'img' <-- ],

img is missing in list of allowed tags

Hi, this list is a bit arbitrary, you should just configure your own. You can easily concatenate our defaults to your own list if you want, see the documentation on that.

authors: boutell mchampaneri

repo: apostrophecms/sanitize-html


text:
Theme UI Pages - Available for common users

All pages available for common users should be available for common users.
Argon Dashboard already provides this evolution. please use it as a reference.
Ty!

validated. closing


text:
Add stable as a supported branch name for SaaS template

https://issues.redhat.com/browse/RHCLOUD-27381
This should add support for stable as a valid branch name to deploy console.redhat.com Frontend applications from stable branches

Note: This is not just for insights, this is for every app that runs on console.redhat.com.

Note: This is not just for insights, this is for every app that runs on console.redhat.com.

Amended the description

this is different from #120 since release indicates production, and stable not as much IMO

authors: Victoremepunto bennyturns maorfr

repo: app-sre/qontract-schemas


text:
proxy shouldn't run on linux

Just had a support request where the user was unable to run a docker container for postgres in fuge on an ubuntu os, because of this error message:
fuge shell system.yml
compiling...
starting proxy...
  proxy postgres 5432 -> 192.168.1.6:5432
  proxy mesh 10000 -> 192.168.1.6:20000
  proxy concorda 10001 -> 192.168.1.6:20001
  proxy dashboard 10002 -> 192.168.1.6:20002
  proxy client 10003 -> 192.168.1.6:20003
starting shell..
? fuge> start postgres
running: postgres
? fuge> [postgres - 16162]: Error response from daemon: Cannot start container f2ef544bd8ff9e03af7b2fc74d1cae59365b68012cf87c181abd1951cfdf4a6d: failed to create endpoint gigantic_fermat on network bridge: Error starting userland proxy: listen tcp 0.0.0.0:5432: bind: address already in use
process exit [16162]: postgres

What it eventually ended up being was that the proxy was running on his ubuntu machine, trying to forward localhost:5432 to localhost:5432, and therefor brought down the process.
When he changed the {proxy: 'all'} in his fuge-config, it stopped happening.
we need to be detecting whether the proxy needs to run, instead of requiring users on different operating systems to change their config files (that doesn't scale).

Closing as proxy now removed as not required with Docker for mac/windows

authors: AdrianRossouw pelger

repo: apparatus/fuge


text:
feat: add vanilla js example

Note: onchange  event fires when the input loses focus so we need to press enter key or click outside the suggestion container.

@bietkul  This is the example with Vanilla JS

repo: appbaseio/searchbase


text:
Missing an example for how to verify an IAP transaction receipt

Either we don't need to do this and we can just look up transaction IDs via this library... wr we could use an example of how to do this with this API.  The docs are also unclear if you can look up transactioninfo for an unfinished transaction - for me so far, it doesn't work.

Hello @eedeebee, can you be a little more specific with what IAP transaction receipt you are referring to, is it a JWSTransaction (https://developer.apple.com/documentation/appstoreserverapi/jwstransaction)? Also, unfinished transactions are visible in the Get Transaction History or Get Transaction Info endpoints in the App Store Server API

Thanks I'm just learning here  and appreciate help.  My question has morphed a bit as I'm for now just testing against the sandbox and I'd like to use the SignedDataVerifier (essentially offline without talking to the App Store best I can tell) with the cert from my XCode to verify and decode a transactionReceipt I get from a purchase in the simulator against a StoreKit config I synced down from the App Store.  Can this be done?  I tried providing the cert and I couldn't get the receipt to verify/validate:
import { Environment } from "@apple/app-store-server-library"

const fs = require('fs');

function loadCAs() {
    return [
        fs.readFileSync('./StoreKitTestCertificate.cer'),
    ];
}

const bundleId = process.env.APPLE_APP_BUNDLE_ID!
const cas: Buffer[] = loadCAs()
const environment = Environment.SANDBOX
const enableOnlineChecks = false ; // 0 docs on what this does and it was true in the example

console.log(bundleId)
const verifier = new SignedDataVerifier(cas, enableOnlineChecks, environment, bundleId)

const doit = async function () {
    try {
        const receipt = process.argv[2];  // Passing the receipt on the cmd line
        console.log("receipt", receipt);
        const trans = await verifier.verifyAndDecodeTransaction(receipt);
        console.log(trans)
    } catch (e) {
        console.error(e)
    }
}

doit();```

I get an exception as if the receipt is some other data/format than is expected:

```VerificationException [Error]
    at SignedDataVerifier.verifyJWT (/Users/ericbloch/repos/notewize-web/node_modules/@apple/app-store-server-library/jws_verification.ts:175:17)
    at SignedDataVerifier.verifyAndDecodeTransaction (/Users/ericbloch/repos/notewize-web/node_modules/@apple/app-store-server-library/jws_verification.ts:70:67)
    ... 6 lines matching cause stack trace ...
    at doit (/Users/ericbloch/repos/notewize-web/store-tools/verification.ts:58:12)
    at Object.<anonymous> (/Users/ericbloch/repos/notewize-web/store-tools/verification.ts:36:1) {
  status: 1,
  cause: TypeError: Cannot read properties of null (reading 'originalTransactionId')
      at JWSTransactionDecodedPayloadValidator.validate (/Users/ericbloch/repos/notewize-web/node_modules/@apple/app-store-server-library/models/JWSTransactionDecodedPayload.ts:213:24)
      at SignedDataVerifier.verifyJWT (/Users/ericbloch/repos/notewize-web/node_modules/@apple/app-store-server-library/jws_verification.ts:140:24)
      at SignedDataVerifier.verifyAndDecodeTransaction (/Users/ericbloch/repos/notewize-web/node_modules/@apple/app-store-server-library/jws_verification.ts:70:67)
      at /Users/ericbloch/repos/notewize-web/store-tools/verification.ts:29:38
      at step (/Users/ericbloch/repos/notewize-web/store-tools/verification.ts:33:23)
      at Object.next (/Users/ericbloch/repos/notewize-web/store-tools/verification.ts:14:53)
      at /Users/ericbloch/repos/notewize-web/store-tools/verification.ts:8:71
      at new Promise (<anonymous>)
      at __awaiter (/Users/ericbloch/repos/notewize-web/store-tools/verification.ts:4:12)
      at doit (/Users/ericbloch/repos/notewize-web/store-tools/verification.ts:58:12)
}

For what it's worth, I got the receipt from a purchase in a sim against my storekit config (using react-native-iap if that matters, but lmk).
Thanks for any help!

You're probably looking for Environment.XCODE not Environment.SANDBOX as Xcode is used when testing via Xcode without the Sandbox environment that connects to the App Store. enableOnlineChecks enables checking for revocation on the certificates, which requires making an OCSP (RFC 2560) call to check the revocation status. This is documented in https://developer.apple.com/videos/play/wwdc2023/10143/. For Xcode testing that doesn't matter as the certificates you're working with are generated locally and certificate validation is skipped for the Xcode environment.
However, looking at this receipt, this is not an AppTransaction or signed transaction that the library could verify directly. This is an App Receipt from Original Storekit (https://developer.apple.com/documentation/storekit/in-app_purchase/original_api_for_in-app_purchase/validating_receipts_with_the_app_store). If you are unable to use StoreKit 2 for some reason, which provides the newer AppTransaction (https://developer.apple.com/documentation/storekit/apptransaction) and can be verified with the SignedDataVerifier, you can see an example from the README for how to work with legacy app receipts. A snippet is provided below.
const appReceipt = "MI..." const receiptUtil = new ReceiptUtility() const transactionId = receiptUtil.extractTransactionIdFromAppReceipt(appReceipt)

Thanks @alexanderjordanbaker ! I configured react-native-iap to use StoreKit2 (forced my app to iOS 15.0 and later) and with Environment.XCode and my cert, I can use SignedDataVerifier !

I guess my original question stands though - do I need to verify the receipt still or just look up a transaction info to see if it matches what I think it should?

@eedeebee If you verify the signed data with the SignedDataVerifier, you don't need to do anything else to verify the transaction.

Could you provide docs on what enableOnlineChecks does?

authors: alexanderjordanbaker eedeebee

repo: apple/app-store-server-library-node


text:
hidden property for arguments

Sometimes arguments are best handled in USAGE without needing their own section as in the following screen-shot.  I would like to be able to hide the ARGUMENTS section for the location hint(s).

You can use @Argument(help: .hidden) to hide an argument from the help screen, but then it isn't visible in the usage string either. Are you looking for the ability to hide just the ARGUMENTS section, but to keep the usage string intact?

Yes exactly that: hiding the ARGUMENTS section when it does not add helpful information to preserve vertical space.

Following up: Typically this would involve a single argument (instance or .remaining), such as file-names.

repo: apple/swift-argument-parser


text:
[BitSet] Fix a thinko in BitSet.isEqualSet

If the interval we're checking against happens to end on a storage word boundary, then we mistakenly returned false even if the comparison should've succeeded.
This was caught by exercising the 32-bit code paths on watchOS -- one of the tests happened to use 130..<160 as the range, and 160 == 5 * 32. Add new test cases that would've caught this on 64 bit systems, too.
Checklist

[X] I've read the Contribution Guidelines
[X] My contributions are licensed under the Swift license.
[X] I've followed the coding style of the rest of the project.
[X] I've added tests covering all new code paths my change adds to the project (if appropriate).
[ ] I've added benchmarks covering new functionality (if appropriate).
[X] I've verified that my change does not break any existing tests or introduce unexplained benchmark regressions.
[ ] I've updated the documentation if necessary.

@swift-ci test

repo: apple/swift-collections


text:
Merge darwin/trunk, including libdispatch-703.1.4 Apple source drop, to master

Merge tag darwin/darwin-006 on darwin/trunk, which includes the latest Apple source drop libdispatch-703.1.4 as well as all the github PR’s that we’ve pulled back into our internal repository since the last darwin/trunk merge, along with any changes made to these during internal review and testing on Darwin.

@swift-ci Please test

@swift-ci Please test

repo: apple/swift-corelibs-libdispatch


text:
Don't parse a character property containing a backslash

Add backslash to the list of characters we don't consider valid for a character property name (previous rules implemented in https://github.com/apple/swift-experimental-string-processing/pull/269). This means that we'll bail when attempting to lex a POSIX character property and instead lex a custom character class if we see a \. This allows e.g [:\Q :] \E] to be lexed as a custom character class. For \p{...} this just means we'll emit a truncated invalid property error, which is arguably more inline with what the user was expecting..
I noticed this when digging through the ICU source code. It will bail out of parsing a POSIX character property if it encounters one of its known escape sequences (e.g \a, \e, \f, ...). Interestingly this doesn't cover character property escapes e.g \d, but it's not clear that is intentional. Given backslash is not a valid character property character anyway, it seems reasonable to broaden this behavior to bail on any backslash.

@swift-ci please test


text:
Backport. #752 to nio 1

We should backport #752  to nio 1. Does  not matter in production but it just makes debugging unnecessarily hard...

@weissi does that mean backporting in all the nio-1.x branches?

@Palleas no, we only need that for nio-1.13

this is done in #987

shipped in 1.14.1

authors: Palleas weissi

repo: apple/swift-nio


text:
concurrency takeover safe for 6.0

Motivation:
Letting NIO take over the Concurrency thread pool is important for performance. We saw roughly 30% improvements on Linux for non-I/O code and much more for I/O code.
Modifications:
Allowlist Swift 6.0.
Result:
More perf for more versions.

Nightly failure is known and expected due to new warnings

authors: FranzBusch weissi


text:
SR-1298: [swift-test] Allow running specific tests

swift-test -h advertises the ability to test specific test cases, by passing a specifier. However the specifier is ignored and all tests are run. This change allows passing a specifier which is used. Tested on both OS X and Ubuntu 15.10.
See also SR-1298.

This is dupe of https://github.com/apple/swift-package-manager/pull/266
Consider closing

@aciidb0mb3r ah yes; didn't come across your PR in Jira. See my change on how to use a test specifier without introducing a flag.

Yeah I tried same thing first but the problem with that is, if you specify a mode it'll fail.

Ah yes; swift-test --chdir hello will turn both --chdir and hello into a mode. Hmm apart from rewriting OptionParser to have more knowledge on the arguments, I don't see an easy solution.

-s sounds reasonable

repo: apple/swift-package-manager


text:
Module aliasing: allow multiple root packages

Support module aliasing for multiple root packages
Add module alias tracker API
Resolves rdar://88518683

@swift-ci smoke test

@swift-ci smoke test

@swift-ci smoke test linux


text:
Add 'package sign' subcommand

Motivation:
Part of https://github.com/apple/swift-evolution/blob/main/proposals/0391-package-registry-publish.md
Modifications:

Add 'package sign' command
Refactor code to share between 'package sign' and 'package-registry publish'

@swift-ci smoke test

CMake Error: The source directory "/Users/ec2-user/jenkins/workspace/swift-package-manager-PR-macos-smoke-test/branch-main/swift-asn1" does not exist.

checkout of new libraries did not take place?

@swift-ci please smoke test

@swift-ci please smoke test Windows

@swift-ci smoke test linux

authors: tomerd yim-lee


text:
Add bundle version

iTunes Connect will reject frameworks without a CFBundleVersion. It was already set to CURRENT_PROJECT_VERSION but that was empty. I put in the value of 1 for now.

I guess we don't actually need to track the real version.

@tbkka Thanks!
@thomasvl I usually use a build phase script to count the number of commits and use that as the version number. I do that for apps since you have to have a higher unique version than the last one you uploaded to iTunes Connect. For frameworks, I usually just use one, since no one ever looks at this. I usually set the short version number to the actual version (like 0.9.27).
Happy to PR my script if you want though.

authors: soffes thomasvl

repo: apple/swift-protobuf


text:
Fix Kingfisher build by the new scheme settings

This is a fix for #527
It now succeeds as below:
--- Executing build actions ---
$ /Users/onevcat/Desktop/swift-source-compat-suite/runner.py --swift-branch swift-5.1-branch --swiftc /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swiftc --projects /Users/onevcat/Desktop/swift-source-compat-suite/projects.json --include-repos 'path == "Kingfisher"' --include-versions 'version == "5.1"' --include-actions 'action.startswith("Build")'
$ git -C /Users/onevcat/Desktop/swift-source-compat-suite/project_cache/Kingfisher rev-parse --show-toplevel
PASS: Kingfisher, 5.1, 44450a, Kingfisher, generic/platform=iOS
$ git -C /Users/onevcat/Desktop/swift-source-compat-suite/project_cache/Kingfisher rev-parse --show-toplevel
PASS: Kingfisher, 5.1, 44450a, Kingfisher, generic/platform=macOS
$ git -C /Users/onevcat/Desktop/swift-source-compat-suite/project_cache/Kingfisher rev-parse --show-toplevel
PASS: Kingfisher, 5.1, 44450a, Kingfisher, generic/platform=tvOS
$ git -C /Users/onevcat/Desktop/swift-source-compat-suite/project_cache/Kingfisher rev-parse --show-toplevel
PASS: Kingfisher, 5.1, 44450a, Kingfisher, generic/platform=watchOS
========================================
Action Summary:
     Passed: 4
     Failed: 0
    XFailed: 0
    UPassed: 0
      Total: 4
========================================
Repository Summary:
      Total: 1
========================================
Result: PASS
========================================
--- Kingfisher checked successfully against Swift 5.1 ---

@swift-ci test

authors: onevcat shahmishal

repo: apple/swift-source-compat-suite


text:
[forward C++ interop] FRT Optional type assignment causes Swift IRGen crash

Describe the bug
I am hitting a compiler crash when assigning a C++ imported FRT to an Optional type:
5  libsystem_c.dylib        0x00000001821d3340 abort + 168
swift-frontend           0x0000000106c86fe8 llvm::install_out_of_memory_new_handler() + 0
7  swift-frontend           0x00000001027e23d4 swift::irgen::IRGenFunction::emitStrongRetain(llvm::Value*, swift::ReferenceCounting, swift::irgen::Atomicity) + 0
8  swift-frontend           0x00000001027bbcf0 (anonymous namespace)::SinglePayloadEnumImplStrategy::consume(swift::irgen::IRGenFunction&, swift::irgen::Explosion&, swift::irgen::Atomicity) const + 544
9  swift-frontend           0x00000001028b48a8 swift::SILInstructionVisitor<(anonymous namespace)::IRGenSILFunction, void>::visit(swift::SILInstruction*) + 24436
10 swift-frontend           0x00000001028aa0c0 (anonymous namespace)::IRGenSILFunction::emitSILFunction() + 7344
11 swift-frontend           0x00000001028a7e7c swift::irgen::IRGenModule::emitSILFunction(swift::SILFunction*) + 896

Swift version 5.8-dev (LLVM 6ad20b8df5a475e, Swift 9ab055adce4a6e0)
Steps To Reproduce
Have the following FRT in C++:
// SentenceLib header
#define INTRUSIVE_REF_COUNTED                             \
     __attribute__((swift_attr("import_as_ref")))         \
     __attribute__((swift_attr("retain:retainContext")))  \
     __attribute__((swift_attr("release:releaseContext")))

// The engine of the game.
class INTRUSIVE_REF_COUNTED SentenceGameContext {
public:
    SentenceGameContext() = default;
    SentenceGameContext(const SentenceGameContext &) = delete;
    SentenceGameContext(SentenceGameContext &&) = delete;
    ~SentenceGameContext();
private:
    int retainCount = 1;
    friend void retainContext(SentenceGameContext &ctx);
    friend void releaseContext(SentenceGameContext &ctx);
};

inline void retainContext(SentenceGameContext &ctx) {
    ctx.retainCount++;
}

inline void releaseContext(SentenceGameContext &ctx) {
    ctx.retainCount--;
    if (ctx.retainCount == 0)
        delete &ctx;
}

Be used like this:
import SentenceLib

public func testSG(_ ctx: SentenceGameContext) {
    let s: SentenceGameContext? = ctx
}

Expected behavior
It should compile.

more info:
Ref counting should be handled in TypeInfo.
UNREACHABLE executed at /Users/ec2-user/jenkins/workspace/oss-swift-package-macos/swift/lib/IRGen/GenHeap.cpp:1026!
Please submit a bug report (https://swift.org/contributing/#reporting-bugs) and include the project and the crash backtrace.
Stack dump:

Fixed by https://github.com/apple/swift/pull/61533

authors: hyp zoecarver

repo: apple/swift


text:
[AST] Fix argument to diagnostic

A simple “%1” to “%0” for a diagnosis with just one arg.
This is a fix for the crash here: https://github.com/apple/swift/pull/1013

Ha, thanks. Can you try to find a non-fuzzing-based test case and add it to your commit?

@jrose-apple Sure thing! Done and squashed and tests run.

Thanks!

Ah, now that #1013 has been merged we need to update that test too.

authors: gregomni jrose-apple lattner


text:
[build-script] cmake: Use lldb-dotest to drive lldb testing

Using lldb-dotest to drive testing solves the short-term problem of
there being missing test dependencies when we invoke dotest.py for cmake
builds. See: https://ci.swift.org/job/oss-lldb-incremental-osx-cmake/139.
Long term, we really want to delete as much lldb-specific logic from
build-script as possible and replace all of it with a single call to
lldb-dotest.
Even with this first cut, there are some nice simplifications
(build-script no longer needs to know how to find an out-of-tree
debugserver, etc).

@swift-ci smoke test and merge

@dcci This addresses the -llldb failures we've been seeing in cmake builds (the add_custom_target/DEPENDS theory I mentioned off-line was incorrect).


text:
Temporarily disable an optimization to avoid some crashes.

We are seeing some crashes due to this optimization where a type
contains a single retainable pointer. Temporarily disable it while
we get to the root cause.
rdar://problem/39629937

@swift-ci please test


text:
[SILProfiler] Set up profiling for lazily-emitted functions

A SIL function that's initially only emitted as a declaration may later
be prepared for definition. When this happens, set up a profiler for the
definition.
This makes code coverage visible for private methods (the frontend
follows a declare-then-define pattern for these).
rdar://47759243

@swift-ci please smoke test

Bots have been failing with:
swift/stdlib/public/Darwin/QuartzCore/NSValue.swift.gyb:33:50: error: use of undeclared type 'Type'

@swift-ci please smoke test


text:
[5.1] Cherry-pick module interface fixes to 5.1

Cherry-pick of module interface fixes to 5.1 from:
https://github.com/apple/swift/pull/24225
https://github.com/apple/swift/pull/24212
Resolves rdar://problem/49856177
Resolves rdar://problem/49856927

@swift-ci please test


text:
SimplifyCFG: fix a compile time bug from exponential jump threading.

Change the way how to limit jump threading.  Instead of counting the
number of jump threading optimizations on a block, count the number of
copied instructions due to jump threading.
rdar://problem/51416939
Patch by Erik Eckstein!
(cherry picked from commit 75ff6216af36c9dc1a5cd5c413cd855e935ca933)

Replace this paragraph with a description of your changes and rationale. Provide links to external references/discussions if appropriate.

Resolves SR-NNNN.

@swift-ci test.

@swift-ci test source compatibility.


text:
[ConstraintSystem] Check availability as part of resolving overload c…

…hoice
This is a follow up to changes related to contextual availability
(https://github.com/apple/swift/pull/29921) which increased score
for unavailable declarations only if they were overloaded but
overlooked a case of a single unavailable choice.
Resolve: rdar://problem/60047439

@swift-ci please test

@swift-ci please test source compatibility


text:
GSB: Fix getMinimalConformanceSource() for top-level requirements in a requirement signature

Consider the following program:
protocol P1 {
  associatedtype A : P2
}

protocol P2 {
  associatedtype A
}

func f<T>(_: T) where T : P2, T.A : P1, T.A.A == T {}

There are two proofs of T : P2:

The explicit requirement in f()'s generic signature.

Since T.A.A == T, we can also prove T : P2 via T.A.A : P2:

First, we prove that T.A : P1 via the explicit requirement
in f()'s generic signature.
Second, we prove that T.A.A : P1 via Self.A : P2 in P1's
requirement signature.

However, the second proof does not render the explicit requirement
T : P2 redundant, because it relies on the existence of the
nested type T.A, which only exists if T : P2.
This is captured in getMinimalConformanceSource(), which returns
nullptr for the requirement source corresponding to the second proof
above. It does this by looking at the root type of the requirement
source, T.A.
Now consider the analogous situation but with protocols -- let's
replace f() with a protocol P3:
protocol P3 : P2 where Self.A : P1, Self.A.A == Self {}

Here, we also have two proofs of Self : P2:

The explicit requirement in P3's requirement signature.

First, we prove that Self.A : P1 via the explicit requirement
in P3's requirement siganture.
Second, we prove that Self.A.A : P1 via Self.A : P2 in P1's
requirement signature.

Once again, the second proof implicitly depends on the explicit
requirement, so we cannot use it to mark the explicit requirement
as redundant. However, since the requirement source root type here
is just 'Self', we were unable to recognize this, and we would
diagnose the requirement as redundant and drop it, resulting in
computing an invalid requirement signature for protocol P3.
To fix this, handle requirements at the top level of a protocol
requirement signature just like they are explicit requirements.
Fixes https://bugs.swift.org/browse/SR-13850 / rdar://problem/71377571.

@swift-ci Please smoke test

@swift-ci Please test source compatibility

@swift-ci Please smoke test macOS


text:
[Concurrency] Test showing that runDetached leaks

Detaching currently leaks and this is a simple reproducer for it.
rdar://74973364

This used to crash but was since fixed by @jckarter, merging so we have another test covering this leaking behavior.

@swift-ci please smoke test and merge


text:
libswift: add instructions and some analysis

This PR adds some infrastructure to libswift:

new instructions
the ApplySite/FullApplySite protocols
basic block predecessors/successors
AliasAnalysis
CalleeAnalysis

@swift-ci test

@swift-ci test

Build failed
Swift Test OS X Platform
Git Sha - 7ed04c3ba32d3365aef9ab3c57232fd1e3fe170d

Build failed
Swift Test Linux Platform
Git Sha - 7ed04c3ba32d3365aef9ab3c57232fd1e3fe170d

@swift-ci test

authors: eeckstein swift-ci


text:
[SILInliner] Added lexical lifetimes for arguments.

When the -enable-experimental-lexical-lifetimes flag is passed, added lexical borrow scopes for every non-trivial argument when inlining a function.

@swift-ci please test

@swift-ci please test

Build failed
Swift Test OS X Platform
Git Sha - 6bcbb04bc18274e9fa459882e6d48b0e0fe78525

@swift-ci please test macos platform

authors: nate-chandler swift-ci


text:
Opaque type fixes for interesting generic environments

Fix a few remaining issues with opaque result types that have "interesting" generic environments:

Only build opaque archetype types corresponding to opaque generic parameters, and do normal substitutions for outer generic parameters
Correct the IR generation logic for finding a particular witness table in an opaque type. It was hard-coded to only work with opaque type declarations that have a single opaque type in them and a "flat" set of conformances.

Although the tests here are described in terms of named opaque result types (which is hidden behind an experimental flag), the fixes also apply to structural opaque result types and any feature that allows us to add constraints to opaque result types.

@swift-ci please smoke test


text:
[SwiftSyntax] Adjustments because recursiveDescription is removed in favor of debugDescription

Companion of https://github.com/apple/swift-syntax/pull/1503

https://github.com/apple/swift-syntax/pull/1503
@swift-ci Please smoke test


text:
[AST] ASTScope: Pass parent module to all search methods

Addresses recently discovered performance regression.
findChildContaining has to handle macros and for that
it needs a parent module, looking it up every time is expensive
because it involves walking up back to a source file scope.
Resolves: rdar://114667408
Resolves: rdar://114667496

@swift-ci please test

@swift-ci please test Linux platform


text:
[gardening] Mark as regressions

These crashers are not yet fixed, so moving them from the compiler_crashers_fixed/ to the compiler_crashers/ dir :-)

@swift-ci please smoke test and merge


text:
Enable building on GCC 4.8

Our minimum GCC version to build is currently 5.4. With this PR, we can build on GCC 4.8, which comes out of the box in Ubuntu 14.04.

@ylow Comment addressed, please re-review, thanks!

repo: apple/turicreate


text:
Exceeding ChatGPT API Context Length

When I add too many files using git, the following error message appears:
Error: error, status code: 400, message: This model's maximum context length is 4097 tokens. However, you requested 4355 tokens (4055 in the messages, 300 in the completion). Please reduce the length of the messages or completion.

Two approaches:

try gpt-3.5-turbo-16k model
ignore unnecessary diff codes using git.exclude_list config.

codegpt config set git.exclude_list xxxx

repo: appleboy/CodeGPT


text:
Can't upload to server

Description
The JSON object is not uploaded to the server. The code cannot connect to the server when I try to establish the the connection with the server. Here I am using a local JSON server to store all the data in a JSON file. to connect the JSON server to the internet I am using Ngrok which gives me an URL through which other devices can use connect to my server. The logs does not even show failed POST request. So I am lost why it is not connecting.
To Reproduce
Here is the document to reproduce the Ngrok server
ngrokServer.docx
Follow the steps to reproduce the error after creating the server

Go to src/main/resources/base/element/string.json
Replace the value key with URL provided by Ngrok where name key is url
Run the application
Crash the application with uncaughtException.

Expected Behavior
The expected behavior was that the a JSON object was uploaded to the server with all the information that was selected.
Desktop
-OS: Windows 10
-DevEco Studio 2.1.0.501
-Harmony OS SDK 5
-Postman
Device Information
-Device Type: Remote emulator
-Device: MatePad Pro
-OS: Harmony OS 2.0.0.0
Additional context
I have already tested POST and GET method with postman and I was able to store the data in the  JSON server. I am attaching the code snippet which have used to establish the connection.
private void makeHttpConnectionAndUpload(){
		URL url = null;
		try {
			url = new URL(postUrl);
		} catch (MalformedURLException e1) {
			HiLog.error(new HiLogLabel(HiLog.LOG_APP, 0x00201 ,""+ability.getBundleName()), "MalformedURLException");
		}
		HttpURLConnection conn = null;
		try {
			HiLog.debug(new HiLogLabel(HiLog.LOG_APP, 0x00201 , "Test"), "Before connection");
			conn = (HttpURLConnection)url.openConnection();
		} catch (IOException e1) {
			HiLog.error(new HiLogLabel(HiLog.LOG_APP, 0x00201 ,""+ability.getBundleName()), "IOException");
		}
		try {
			conn.setRequestMethod("POST");
		} catch (ProtocolException e1) {
			HiLog.error(new HiLogLabel(HiLog.LOG_APP, 0x00201 ,""+ability.getBundleName()), "ProtocolException");
		}
		HiLog.debug(new HiLogLabel(HiLog.LOG_APP, 0x00201 ,""+ability.getBundleName()), "Sending request");
		conn.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
		conn.setDoInput(true);
		conn.setDoOutput(true);
		List<PostValuesPair> params1 = new ArrayList<>();
		params1.add(new PostValuesPair("error_report", jObjectData.toString()));
		try{
			HiLog.debug(new HiLogLabel(HiLog.LOG_APP, 0x00201 ,"Debugger"), "checking ");
			OutputStream os = conn.getOutputStream();
			BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(os, StandardCharsets.UTF_8));
			writer.write(getQuery(params1));
			writer.flush();
			writer.close();
			os.close();
		}
		catch(Exception ee)
		{
			HiLog.error(new HiLogLabel(HiLog.LOG_APP, 0x00201 ,""+ability.getBundleName()), "Buffer Write Exception");
		}
		try {
			conn.connect();
		} catch (IOException e1) {
			HiLog.error(new HiLogLabel(HiLog.LOG_APP, 0x00201 ,""+ability.getBundleName()), "IOException");
		}
	}

Hello @raj-bhagat ,
I cannot see the above method implementation in the latest code. Seems, you have now implemented changed the functionality of dumping log to the local file system and removed the upload to server code. Can you please attach the old code, for future reference and fixes?
~Kanak

Hi @kanaksony,
I am attaching the Exception Handler file below
ExceptionHandler.txt.

Thanks, WIll create a branch and store there.

authors: kanaksony raj-bhagat

repo: applibgroup/AppCrashTracker


text:
ImageRenderer with PNG image

Sorry, but i wonder how to draw PNG in view with background is transparent, my image aways render BLACK background,
Thanks

Hi @xicun9x , we've updated the ImageRenderer to allow transparent PNGs. Please update your version in build.gradle to 0.2.1

repo: appoly/ARCore-Location


text:
Add a demo app

Change description
Add a demo app with various models and views to test the manager on.

This app is sourced from https://github.com/mdn/django-locallibrary-tutorial

How to verify

Start with migrating your DB

python manage.py migrate

Generate models objects and store them in your local DB by using

python manage.py generate  # This will create a suppreuser for you with these credentials: `username='admin'` and `password='admin'`

Go to http://127.0.0.1:8000/ and see you objects in action.

Note
This app is created for the sole purpose of testing and simplifying the review for the actual commands in #2. No other functionality is expected out of this app.

Type of change

[ ] Bug fix (fixes an issue)
[x] New feature (adds functionality)

Development

[x] Lint rules pass locally
[x] Application changes have been tested thoroughly
[x] Automated tests covering modified code pass

Security

[ ] Security impact of change has been considered
[ ] Code follows company security practices and guidelines

Code review

[ ] Pull request has a descriptive title and context useful to a reviewer. Screenshots or screencasts are attached as necessary
[x] "Ready for review" label attached and reviewers assigned
[ ] Changes have been reviewed by at least one other contributor
[ ] Pull request linked to task tracker where applicable

Is there something we can read that gives this some context? It's a big PR and as you say, it's mostly sourced from that django-locallibrary-tutorial. But it's not clear if this is just copying that over as demoapp, or if there are changes made in the process that we should be reviewing and no easy way to see what those changes are without going line by line through 67 files. If it's just copying it in (along with the github ephemera to run tests, etc) so future work can make use of it, that's fine. Otherwise, it would help for us to know what we should actually be looking for in a review.

I went over the files and it's mostly a boilerplate and a demo app. If so, I think it's safe to merge without a review. Thanks Ahmed!

authors: OmarIthawi ahmedaljazzar thraxil

repo: appsembler/gestore


text:
Activation experiment#2: Offer building blocks along with templates in start from template flow.

Notion link

All tasks done.

repo: appsmithorg/appsmith


text:
[Feature]: Move away from environment variables and rely on db changes

Is there an existing issue for this?

[X] I have searched the existing issues

Summary
Currently, when a user makes changes in admin settings those changes go into docker.env which inturn goes in as an environment variable.
Rather, it will be great if we can add required entries into the database and avoid restarts and relying on environment variabls
Why should this be worked on?
This will help in increased user experience and avoid UN-necessary hassle of maintaining environment variables

Duplicate of https://github.com/appsmithorg/appsmith/issues/11855.


text:
[Bug]: Fullscreen control blocked with HLS and DASH streaming video files but not .mp4

Is there an existing issue for this?

[X] I have searched the existing issues

Description
HLS and Dash video types are not supported in the video widget. As such, I tried to add the video using an iframe widget and custom widget.
With both widget types, if I use an .mp4 video, everything works fine. I can make the video fullscreen just fine.
However, when I try to use an .mpd or .m3u8 video, although the video plays and I can control the audio, the fullscreen functionality is blocked/grayed out.
In both instances, I have used the Video.js video player. I have also tried the Dash.js video player and the result is the same.
Steps To Reproduce

Create an iframe or custom widget and add code to display an .mp4 video. Play the video and press the fullscreen button.
Update the iframe or custom widget to display an HLS (.m3u8) or DASH (.mpd) video. Play the video and try to press the fullscreen button.

Public Sample App
https://app.appsmith.com/app/streamer/develop-673605b65a542b4e182b70fd
Environment
Production
Severity
Medium (Frustrating UX)
Issue video log
https://www.loom.com/share/a178e7f070c44c8c829c74f0a6560c6b?sid=333d8961-2100-4bb8-a18e-909c9b5d69cd
Version
Cloud

This is also now no longer working for me with MP4 files as well. The fullscreen button is blocked.
Here is the most recent code I tried in an iframe with an MP4 video:
<!DOCTYPE html>
<html>
  <head>
    <link href="https://vjs.zencdn.net/8.16.1/video-js.css" rel="stylesheet" />
  </head>
  <body>
    <video-js id="my-video" class="video-js" controls preload="auto" width="640" height="360" data-setup='{}'>
      <!-- Replace with your MP4 URL -->
      <source src="https://<url>/switzerland.mp4" type="video/mp4">
    </video-js>

    <script src="https://vjs.zencdn.net/8.16.1/video.min.js"></script>
  </body>
</html>