text:
Fixed WorkerPool to handle remote actors properly

Currently WorkerPool:

Treats remote actors as weak and cleans them directly
Doesn't work on Swift 6 mode as WorkItem and WorkResult are not marked as Sendable

This PR fixes this, basically cherry picked changes from this PR https://github.com/apple/swift-distributed-actors/pull/1159 and updated tests

Hm, would be nice to add test to check remote actors also.

@swift-server-bot test this please

@swift-server-bot test this please

@swift-server-bot add to allowlist

authors: akbashev ktoso

repo: apple/swift-distributed-actors


text:
[Incremental, Testing] Implement 7 more removal tests.

Tests all combinations of removeInputFromInvocation,  removeSwiftDepsFile, & restoreBadPriors.
The last condition simulates a case where a file is removed but somehow the driver does not get to remove the priors, which has entries for the removed file.
For that condition, the test passes but prints out a warning.
A subsequent PR will fix this issue.

@swift-ci please test

@swift-ci please test

@swift-ci please test

rdar://77998890

source: gharchive/pull-request

repo: apple/swift-driver

license_type: permissive

license_source: bigquery


text:
Tests: make ExplicitModuleBuildTests.* pass on Windows

This fixes the ExplicitmoduleBuildTests to pass on Windows.  The
original port to Windows had setup the stdlib paths incorrectly.
Unfortunately, that is a larger change that needs to wait until other
stuff is complete.  Namely, we currently structure the SDK as:
SDKROOT/usr/lib/swift/[OS]/[Arch]/[Module].swiftmodule/[Triple].swift{module,doc,info}

The directory form of the swiftmodule was adopted to allow
multi-architecture variants of the modules and does allow the
intermediate architecture directory to be removed.  The architecture
specific directory would still contain the import libraries (TBD-like
content) which is not currently multi-architecture friendly (due to
tooling).  So while the directory needs to remain, the swiftmodules
could be hoisted a level.  This would need to be coordinated along the
driver, the frontend, and the installer.  So it is best to hold off on
this change for the time being.
Correcting the path computation and then wiring that along to the tests
allows the ExplicitModuleBuildTests to now pass on Windows.
Special thanks to @artemcm for pointing out that we weren't including
those paths in the search.

@swift-ci please test


text:
Refactor "diagnostics" into "findings".

This change addresses https://github.com/apple/swift-syntax/pull/309 and https://github.com/apple/swift-syntax/pull/325, which split SwiftSyntaxParser into its own module and then removed the DiagnosticEngine APIs (so only the Diagnostic type remains as the way diagnostics from the parser are sent back to the caller).
Rather than remain coupled to swift-syntax's Diagnostic type, this change creates a similar Finding type (and related types) to describe the "lint findings" that tree-based rules, the pretty-printer, and the whitespace linter encounter during
formatting/linting.
This Finding type is the currency type for these kinds of findings/diagnostics emitted by the formatter API layer. The
swift-format frontend tool now adopts DiagnosticsEngine from https://github.com/apple/swift-tools-support-core to manage and print these, but other clients using the API could use something different entirely. Since the usage of TSC is limited to the executable, API users don't have to take on a large and mostly unrelated dependency.
There are still some opportunities for renaming here -- mainly replacing instances of diagnose in the internal code with emitFinding or something similar -- but I'll leave that as future cleanup.

Tested against swift-DEVELOPMENT-SNAPSHOT-2021-01-28-a:
Test Suite 'All tests' passed at 2021-11-04 15:53:21.029.
	 Executed 578 tests, with 0 failures (0 unexpected) in 11.055 (11.084) seconds

cc @dylansturg

repo: apple/swift-format


text:
debian: add initial control files for debian packaging

The packaging on debian mostly requires the control files and a
pre-configured layout to package as a root.  This adds the control files
as a starting point.

CC: @shahmishal

We'd also consider replacing swift- prefix with swiftlang-, discussion here:
https://forums.swift.org/t/rpm-and-debs-for-swift-call-for-the-community/49117/35
swiftlang is also the official package name for Fedora, since 3 years ago.

@tomerd, there are a number of fundamental flaws in this PR all of which highlight the need for a package design document as we discussed in #27.
The package metadata should be unified across all platforms as much as possible. There will be small differences but the key things like package name, description, license etc should be the same.
package naming ?
The package naming was commented on in a code review by @stevapple but for some reason this has been ignored.
swift-lang cannot be used as there is an entire family of swift packages already available for debian/ubuntu.
Also, linux should not be included in any package name.
A better naming would be - swiftlang, swiftlang-runtime, swiftlang-dev.
package descriptions
The package descriptions for the different versions are a bit strange -
C/C++/Objective-C/Swift Toolchain ??
Swift Development Content for Linux  ??.
3 levels of swift install ?
The inclusion of multiple control files for different levels of swift install I feel is unnecessary at this point in time. There has been no discussion about different levels of install packages i.e. swift-runtime, swift-?, swift-dev.
Essentially the swift toolchain install is already swiftlang-dev as it includes a multitude of additional packages such as clang, llvm, lldb, lld, sourcekit-lsp. We even have a very old version (2016) of python-six included.
master control file template
There will be a need for many control files for all the different distributions supported but these will essentially be identical and can generated from a master control file template.
The only differences will be in the architecture, dependancies, and replaces fields.
missing metadata
This PR does not represent a working control file and is also not usable as a template as there are a number of missing and essential fields.
WIP
I know that this is all work in progress but if we don't start in the right place then this whole process will be a time consuming mess and we will never achieve the desired outcome.

hi @futurejones inline

@tomerd, there are a number of fundamental flaws in this PR all of which highlight the need for a package design document as we discussed in #27.

I agree and @shahmishal is putting one together. this repo is in a work-in-progress state and no packages are to be created from it until we finalize the design questions you highlight.

The package metadata should be unified across all platforms as much as possible. There will be small differences but the key things like package name, description, license etc should be the same.

totally agree, and I also put out #33 which attempts to start the unification process from a technical side of things, with the actual content pending the design mentioned above

package naming ?

The package naming was commented on in a code review by @stevapple but for some reason this has been ignored.
swift-lang cannot be used as there is an entire family of swift packages already available for debian/ubuntu.
Also, linux should not be included in any package name.
A better naming would be - swiftlang, swiftlang-runtime, swiftlang-dev.

agree - lets discuss this more when @shahmishal puts the design doc PR out

package descriptions

The package descriptions for the different versions are a bit strange -
C/C++/Objective-C/Swift Toolchain ??
Swift Development Content for Linux ??.

agree - lets discuss this more when @shahmishal puts the design doc PR out
3 levels of swift install ?

The inclusion of multiple control files for different levels of swift install I feel is unnecessary at this point in time. There has been no discussion about different levels of install packages i.e. swift-runtime, swift-?, swift-dev.
Essentially the swift toolchain install is already swiftlang-dev as it includes a multitude of additional packages such as clang, llvm, lldb, lld, sourcekit-lsp. We even have a very old version (2016) of python-six included.

agree - lets discuss this more when @shahmishal puts the design doc PR out. fwiw I imagined two kind (one for development - full toolchain, and one for runtime - just runtime support libraries) and looks like @compnerd has 3 in mind, we should hash this out more in the design phase

master control file template

There will be a need for many control files for all the different distributions supported but these will essentially be identical and can generated from a master control file template.
The only differences will be in the architecture, dependancies, and replaces fields.

not familiar enough with debian control files, but we should discuss more

missing metadata

This PR does not represent a working control file and is also not usable as a template as there are a number of missing and essential fields.

agree - lets discuss this more when @shahmishal puts the design doc PR out

WIP

I know that this is all work in progress but if we don't start in the right place then this whole process will be a time consuming mess and we will never achieve the desired outcome.

agree again, I think the next step here is for @shahmishal to puts the design doc PR out.

What is the status of this PR? We merged the Debs package control files already.

I don't think that this is needed anymore then.  I think that it got held up and people just redid the work 🤷♂

authors: compnerd futurejones shahmishal stevapple tomerd

repo: apple/swift-installer-scripts

license_source: github-api


text:
"Temporarily" disable some lldb-mi tests that are unreliable.

rdar://problem/50050012

@swift-ci test and merge

@eeckstein ^

I believe your commit upset lit
11:44:11 ValueError: couldn't parse text: '://50050012'
11:44:11 in expression: 'rdar://50050012'
11:44:11 in REQUIRES: directive on test line 1
11:44:11
11:44:11

@swift-ci test and merge

authors: adrian-prantl dcci

repo: apple/swift-lldb


text:
Fix a thinko in CommandObjectMemoryRegion.

Don't try to read the first argument till you've checked
that there is one.
git-svn-id: https://llvm.org/svn/llvm-project/lldb/trunk@329844 91177308-0d34-0410-b5e6-96231b3b80d8

rdar://problem/39337042

@swift-ci test and merge


text:
Added dependency update for Swift 5.2

Starting in Swift 5.2 the target dependency format changes, this PR updates the README with the new format.
Motivation:
When I added swift-log to my project it wouldn't work, after looking at the error I saw it was a straightforward update and wanted to help others with the same change,.
Modifications:
README only, no code updates.
Result:
Less confusion for Swift 5.2 users.

@swift-server-bot test this please

authors: dpedley ktoso

repo: apple/swift-log


text:
allegedly, DNS round-robin not working in SwiftNIO (because of getaddrinfo)

TIL that apparently, getaddrinfo does not do DNS round-robin'ing because apparently it sorts the results.
If true that would mean that SwiftNIO programs that don't change the default resolver -- which is to say most SwiftNIO programs -- will always pick the same IP address for a given host name.
I didn't check to see if that's true but it seems likely because the Happy Eyeballs code seems to take the IP addresses in order (merging IPv6 and IPv4 addresses).

SwiftNIO has never really done round-robin DNS, that's correct. I think this is ultimately a dupe of #536: getaddrinfo is out of scope for us to change, and locally shuffling the DNS results does not necessarily achieve a round-robin effect. We could in principle work around this by shuffling the results of the lookups, but it's not straightforwardly clear that the increased computation effort is worthwhile.

Thanks @Lukasa, that all makes sense. I personally wouldn't dupe it because at least to me it was news that GAI doesn't RR and this might be a problem that someone is actually experiencing.
Regarding the shuffling or not: I'd personally say that shuffling it is worth the effort (especially because it can be done off-EL) but I can understand your position too.

source: gharchive/issue

authors: Lukasa weissi

repo: apple/swift-nio


text:
remove a few deprecated apis

motivation: cleanup deprecated methods
changes: remove apis deprecated 6m+ ago

@swift-ci please smoke test

repo: apple/swift-package-manager


text:
[UtilityTests] Close master fd explicitly

It looks like if deinit wasn't called for pesudoterminal in time the
master won't be closed and OS would assign slave fd to new pipe (because it was already closed)
and that turns out to be a bad file descriptor somehow...
Closing explicitly fixes the issue.

rdar://problem/28411803 POSIX.popen() sometimes breaks the build
https://bugs.swift.org/browse/SR-2703

@swift-ci please test

Cool!  Thanks for the fix and the explanation.  Another example of why relying on deinit/dealloc to close files an descriptors is a bad idea (to be clear, using deinit to call the close-if-still-open method as robustness is fine — but the normal use case should be to actively release external resources at the right time).
We should take a pass through the code, actually, and make sure that we're using deinit for appropriate things.

@aciidb0mb3r Can we get this into the swift-3.0-branch as well?

Although it's only a bug in the unit tests, I agree, because of the annoyance of having the spurious test failures (and ugliness of having to disable unit tests).  Let's get it into master first, and then I think it's a strong candidate for the swift-3.0-branch.

@abertelrud This is in master and CI seems to be happy with it. Do you want me to make a PR on swift-3 branch or wait till current bugfix PR is merged?

I'm still looking at a mysterious CI failure on Linux for the other PR, but once I've merged that, I'll go ahead and cherry-pick this fix onto swift-3.0-branch.
I am still not 100% clear on why leaving a file descriptor open would cause it to be reused later when issuing the system call.  Presumably it would just hang around as a leaked file descriptor.  Granted, I am not very familiar with ptys, it's just not the symptom I would expect.  It would be nice to understand how the symptom stems from the problem before considering this completely put to rest.

authors: abertelrud aciidb0mb3r erg


text:
Revert "Exclude XCBuildSupport when building with CMake (#7358)"

This reverts commit 52870620e7b550b492086ae877d6eb9afd94c33f, which causes macOS toolchain builds to fail with:
main/main.swift:345: Fatal error: SwiftPM was built without XCBuild support

Also adds the DriverSupport dependency to XCBuildSupport, which was skipped in #7258 because #7358 was merged (but we're reverting that).

@swift-ci please build toolchain macOS platform

@swift-ci please test

@swift-ci please test Windows platform


text:
[6.0/Prepare] Don't skip non-exportable-decls when enable-testing (#7675)

Explanation: @testable doesn't work if you skip non-exportable-decls. Make sure we don't turn on that flag if the target is enable-testing.
Scope: Prepare for indexing only
Risk: isolated to prepare for indexing with is behind an experimental flag
Original PR: https://github.com/apple/swift-package-manager/pull/7675
Testing: Unit-Test

@swift-ci please test

@swift-ci test

@swift-ci test windows


text:
Non breaking CaseIterable conformance

This is part of a wider effort to make breaking changes introduced in main non-breaking.
The SwiftProtobuf.Enum protocol started inheriting from CaseIterable. This is a breaking change, since it's a new requirement of the protocol to make sure all conforming concrete types also conform to CaseIterable.
To allow conformance of generated enums to CaseIterable without being source-breaking, this PR modified the code generator to conform all types conforming to Enum to CaseIterable, but removes the inheritance of CaseIterable from the Enum protocol.
A separate commit regenerates the protos, and a third one regenerates the references for tests.

Looks like you might need to sync/rebase and rerun the regeneration & reference steps to get a few more things updated.

@thomasvl I'm getting the following error when running make regenerate:
conformance/test_protos/test_messages_edition2023.proto: This file uses editions, but --experimental_editions has not been enabled. This syntax is experimental and should be avoided.

File was added here. My protoc version is 26.1 (downloaded from https://github.com/protocolbuffers/protobuf/releases/tag/v26.1
Am I missing something, or should we change the command we run from the Makefile?

Use a main of protobuf to get the updated editions support.

@thomasvl thanks - tests are passing now

@thomasvl thanks - tests are passing now

That's was check about to check.  Merging.

authors: gjcairo thomasvl

repo: apple/swift-protobuf


text:
Generalize sanitizer tests.

Remove redundant XFAIL/REQUIRE, *_runtime variable should be
sufficient to specify whether the test can be run.
Provide required machinery to pass environment variables
to different operating systems in a generalized way
(e.g. iOS Simulator only passes environment variables starting with a
certain prefix).

@swift-ci please smoke test

LGTM

@swift-ci please smoke test Linux platform

Please smoke test OS X platform

@swift-ci please smoke test Linux platform

@swift-ci please smoke test OS X platform

I think you should run full test (please test) on OS X. Smoke test doesn't run tests for simulators.

@rintaro currently in the open-source swift none of the sanitizers are being compiled for the simulators, thus the tests would be disabled.

authors: AnnaZaks cheshire rintaro

repo: apple/swift


text:
[4.2][CSSimplify] When trying to simplify bind with error type fail gracefully

Since member lookup doesn't check requirements
it might sometimes return types which are not
visible in the current context e.g. typealias
defined in constrained extension, substitution
of which might produce error type for base, so
assignement should thead lightly and just fail
if it encounters such types.
Resolves: rdar://problem/39931339
Resolves: SR-5013
(cherry picked from commit 11ba7e0f4289869ac3d0fa9e19e52fd027a29462)

Build failed
Swift Test Linux Platform
Git Sha - 244854f074112f8151c6957e444dd5237d35a55d

Build failed
Swift Test OS X Platform
Git Sha - 244854f074112f8151c6957e444dd5237d35a55d

@swift-ci please test

@swift-ci please test source compatibility

Build failed
Swift Test Linux Platform
Git Sha - 244854f074112f8151c6957e444dd5237d35a55d

Build failed
Swift Test OS X Platform
Git Sha - 244854f074112f8151c6957e444dd5237d35a55d

@swift-ci please test source compatibility

authors: swift-ci xedin


text:
Fixes typo in var name in example snippet

instead of imagesPaths should be used imagePaths which is declared in line 320.

n/a

@swift-ci please smoke test

@swift-ci please smoke test Linux platform

Thanks for fixing this, @valeriyvan!

authors: compnerd natecook1000 valeriyvan


text:
[Type checker] Ensure that we record archetype anchors.

When we aren't going through proper resolution of an associated type
of 'Self', at least record an archetype anchor. Narrow fix for
rdar://problem/47605019.

Huh, LLDB test timeout:
201 out of 201 test suites processed - TestSwiftStepping.py
02:04:49 =============
02:04:49 Issue Details
02:04:49 =============
02:04:49 TIMEOUT: test_swift_stepping_dwarf (lang/swift/stepping/TestSwiftStepping.py)

@swift-ci please smoke test macOS


text:
[5.1 06/12][CSDiag] Always find and set correct declaration context for sub-expr type-check

Explanation:

Replace dedicated method with typeCheckChildIndependently always
setting the closest possible declaration context for type-check call.
This fixes a problem where sub-expression comes from multiple levels
or nested closures but CSDiag didn't re-typecheck parent closure.

Issue: rdar://problem/50869732

Scope: Diagnostics, in subset of cases where there multiple nested single-statement closures involved.

Risk: Low.

Testing: Added compiler regression tests.

Reviewed by: @DougGregor

Resolves: rdar://problem/50869732

@swift-ci please test


text:
IRGen: Public non-resilient types that reference resilient types from the same module must view those types as non-fixed

Otherwise, we don't emit certain parts of the metadata that is expected
when these types are used outside the module (payload size).
rdar://51422528

@swift-ci Please test

Build failed
Swift Test Linux Platform
Git Sha - 93e0cefa7be7ea2ec1d53e0ac9e6ebead7115c59

Build failed
Swift Test OS X Platform
Git Sha - 93e0cefa7be7ea2ec1d53e0ac9e6ebead7115c59

@swift-ci Please test

Build failed
Swift Test Linux Platform
Git Sha - 93e0cefa7be7ea2ec1d53e0ac9e6ebead7115c59

Build failed
Swift Test OS X Platform
Git Sha - 93e0cefa7be7ea2ec1d53e0ac9e6ebead7115c59

@swift-ci Please test

Build failed
Swift Test OS X Platform
Git Sha - 6f4c5c48ce109d702419fbbbabd4cd683733ba23

Build failed
Swift Test Linux Platform
Git Sha - 6f4c5c48ce109d702419fbbbabd4cd683733ba23

New version looks great, thank you very much!

@swift-ci Please test

Build failed
Swift Test Linux Platform
Git Sha - 2b5267046d0936d96e3b18c354ea4d0eeb35b268

Build failed
Swift Test OS X Platform
Git Sha - 2b5267046d0936d96e3b18c354ea4d0eeb35b268

@swift-ci Please test

Build failed
Swift Test Linux Platform
Git Sha - 3bc3f66d3f7856528d6d13a088c7b23dfdf178d1

Build failed
Swift Test OS X Platform
Git Sha - 3bc3f66d3f7856528d6d13a088c7b23dfdf178d1

Unrelated error:
/Users/buildnode/jenkins/workspace/swift-PR-osx/branch-master/swift/benchmark/single-source/ObjectiveCBridging.swift:753:43: error: use of unresolved identifier 'string'
07:49:35   let jsonString = "[\(String(reflecting: string))]"

@swift-ci Please test OS X platform

authors: aschwaighofer slavapestov swift-ci


text:
[build-script] Remove redundant test check in sourcekitlsp's build product description

I believe the test for args.test is redundant since if test is False then test_sourcekitlsp is also being set to False in https://github.com/apple/swift/blob/master/utils/build_swift/driver_arguments.py#L207.

@swift-ci Please test


text:
[NFC] Add DirectLookupRequest

There's no more Clang Importer cycles to stand in my way!

@swift-ci test

@swift-ci please test source compatibility

Build failed
Swift Test OS X Platform
Git Sha - 06448b50c569fcad6ddbab4a46be4aa07a2f0e47

@swift-ci please test macOS platform

@swift-ci please test source compatibility

Build failed
Swift Test OS X Platform
Git Sha - 06448b50c569fcad6ddbab4a46be4aa07a2f0e47

@swift-ci please test source compatibility

@swift-ci please test macOS platform

Build failed
Swift Test OS X Platform
Git Sha - 06448b50c569fcad6ddbab4a46be4aa07a2f0e47

@swift-ci please build toolchain

Linux Toolchain (Ubuntu 16.04)
Download Toolchain
Git Sha - 06448b50c569fcad6ddbab4a46be4aa07a2f0e47
Install command
tar zxf  swift-PR-28732-329-ubuntu16.04.tar.gz
More info

macOS Toolchain
Download Toolchain
Git Sha - 06448b50c569fcad6ddbab4a46be4aa07a2f0e47
Install command
tar -zxf  swift-PR-28732-451-osx.tar.gz --directory ~/

@swift-ci please test source compatibility

@swift-ci please test source compatibility

@swift-ci please smoke test macOS platform

@swift-ci please smoke test macOS platform

@swift-ci please clean smoke test macOS platform

@swift-ci test macOS

Build failed
Swift Test OS X Platform
Git Sha - 06448b50c569fcad6ddbab4a46be4aa07a2f0e47

@swift-ci please test source compatibility Debug

@swift-ci please test source compatibility Debug

@swift-ci please test source compatibility Debug

@swift-ci please smoke test

@swift-ci please smoke test

@swift-ci please test source compatibility

@swift-ci please test source compatibility

Let's see what the damages come out to.

@swift-ci please test

@swift-ci please test

@swift-ci please test source compatibility

@swift-ci please test source compatibility

@swift-ci please test compiler performance

@swift-ci please test compiler performance

@swift-ci please test compiler performance

Build failed
Swift Test OS X Platform
Git Sha - eed8d8b1df3fdea193034c5c6999fb7a5d5539c5

Infrastructure failed
@swift-ci please test macOS platform

Build failed
Swift Test OS X Platform
Git Sha - eed8d8b1df3fdea193034c5c6999fb7a5d5539c5

Summary for master full
Unexpected test results, excluded stats for RxCocoa, Base64CoderSwiftUI, SwifterSwift
Regressions found (see below)

Debug-batch

debug-batch brief
debug-batch detailed

Release

release brief
release detailed

Debug-batch

debug-batch brief

Regressed (0)

name
old
new
delta
delta_pct

Improved (0)

name
old
new
delta
delta_pct

Unchanged (delta < 1.0% or delta < 100.0ms) (3)

name
old
new
delta
delta_pct

Frontend.NumInstructionsExecuted
42,867,617,371,486
42,912,725,397,091
45,108,025,605
0.11%

LLVM.NumLLVMBytesOutput
1,801,781,976
1,801,787,836
5,860
0.0%

time.swift-driver.wall
3162.0s
3157.4s
-4.6s
-0.15%

debug-batch detailed

Regressed (0)

name
old
new
delta
delta_pct

Improved (11)

name
old
new
delta
delta_pct

AST.NumSourceLinesPerSecond
4,699,547
4,648,444
-51,103
-1.09% :white_check_mark:

Sema.AccessLevelRequest
10,942,525
10,743,906
-198,619
-1.82% :white_check_mark:

Sema.CollectOverriddenDeclsRequest
6,522,155
6,341,091
-181,064
-2.78% :white_check_mark:

Sema.GenericParamListRequest
10,661,550
10,521,164
-140,386
-1.32% :white_check_mark:

Sema.InterfaceTypeRequest
14,504,720
14,347,474
-157,246
-1.08% :white_check_mark:

Sema.NamedLazyMemberLoadSuccessCount
11,140,027
11,006,782
-133,245
-1.2% :white_check_mark:

Sema.NominalTypeLookupDirectCount
33,072,663
0
-33,072,663
-100.0% :white_check_mark:

Sema.NumConformancesDeserialized
8,176,740
8,085,431
-91,309
-1.12% :white_check_mark:

Sema.NumDeclsDeserialized
63,293,635
62,596,987
-696,648
-1.1% :white_check_mark:

Sema.ProvideDefaultImplForRequest
6,522,155
6,341,091
-181,064
-2.78% :white_check_mark:

Sema.USRGenerationRequest
7,815,094
7,623,348
-191,746
-2.45% :white_check_mark:

Unchanged (delta < 1.0% or delta < 100.0ms) (195)

name
old
new
delta
delta_pct

AST.ImportSetCacheHit
1,595,720
1,594,837
-883
-0.06%

AST.ImportSetCacheMiss
694,205
694,020
-185
-0.03%

AST.ImportSetFoldHit
372,880
372,857
-23
-0.01%

AST.ImportSetFoldMiss
321,325
321,163
-162
-0.05%

AST.ModuleShadowCacheHit
1,249,373
1,248,275
-1,098
-0.09%

AST.ModuleShadowCacheMiss
10,083
10,055
-28
-0.28%

AST.ModuleVisibilityCacheHit
28,905
28,905
0
0.0%

AST.ModuleVisibilityCacheMiss
6,638
6,638
0
0.0%

AST.NumASTBytesAllocated
51,638,804,526
51,174,699,989
-464,104,537
-0.9%

AST.NumASTScopeLookups
3,528,478
3,528,330
-148
-0.0%

AST.NumBraceStmtASTScopeExpansions
619,299
619,299
0
0.0%

AST.NumBraceStmtASTScopes
619,299
619,299
0
0.0%

AST.NumDecls
139,853
139,853
0
0.0%

AST.NumDependencies
301,271
301,269
-2
-0.0%

AST.NumInfixOperators
53,695
53,695
0
0.0%

AST.NumIterableTypeBodyASTScopeExpansions
227,890
227,886
-4
-0.0%

AST.NumIterableTypeBodyASTScopes
279,378
279,375
-3
-0.0%

AST.NumLinkLibraries
0
0
0
0.0%

AST.NumLoadedModules
281,053
281,053
0
0.0%

AST.NumLocalTypeDecls
250
250
0
0.0%

AST.NumModuleLookupClassMember
6,927
6,927
0
0.0%

AST.NumModuleLookupValue
41,342,894
41,309,818
-33,076
-0.08%

AST.NumObjCMethods
24,240
24,240
0
0.0%

AST.NumPostfixOperators
49
49
0
0.0%

AST.NumPrecedenceGroups
25,840
25,840
0
0.0%

AST.NumPrefixOperators
99
99
0
0.0%

AST.NumReferencedDynamicNames
189
189
0
0.0%

AST.NumReferencedMemberNames
6,309,711
6,309,711
0
0.0%

AST.NumReferencedTopLevelNames
476,239
476,239
0
0.0%

AST.NumSourceBuffers
338,107
338,107
0
0.0%

AST.NumSourceLines
4,717,684
4,717,684
0
0.0%

AST.NumTotalClangImportedEntities
4,881,806
4,856,086
-25,720
-0.53%

Driver.ChildrenMaxRSS
232,760,762,368
230,960,125,952
-1,800,636,416
-0.77%

Driver.DriverDepCascadingDynamic
0
0
0
0.0%

Driver.DriverDepCascadingExternal
0
0
0
0.0%

Driver.DriverDepCascadingMember
0
0
0
0.0%

Driver.DriverDepCascadingNominal
0
0
0
0.0%

Driver.DriverDepCascadingTopLevel
0
0
0
0.0%

Driver.DriverDepDynamic
0
0
0
0.0%

Driver.DriverDepExternal
0
0
0
0.0%

Driver.DriverDepMember
0
0
0
0.0%

Driver.DriverDepNominal
0
0
0
0.0%

Driver.DriverDepTopLevel
0
0
0
0.0%

Driver.NumDriverJobsRun
28,030
28,030
0
0.0%

Driver.NumDriverJobsSkipped
0
0
0
0.0%

Driver.NumDriverPipePolls
109,469
109,742
273
0.25%

Driver.NumDriverPipeReads
100,445
100,640
195
0.19%

Driver.NumProcessFailures
0
0
0
0.0%

Frontend.MaxMallocUsage
917,201,390,832
917,087,387,152
-114,003,680
-0.01%

Frontend.NumInstructionsExecuted
42,867,617,371,486
42,912,725,397,091
45,108,025,605
0.11%

Frontend.NumProcessFailures
0
0
0
0.0%

IRModule.NumIRAliases
196,319
196,319
0
0.0%

IRModule.NumIRBasicBlocks
6,862,691
6,862,691
0
0.0%

IRModule.NumIRComdatSymbols
0
0
0
0.0%

IRModule.NumIRFunctions
3,361,962
3,361,962
0
0.0%

IRModule.NumIRGlobals
3,672,227
3,672,227
0
0.0%

IRModule.NumIRIFuncs
0
0
0
0.0%

IRModule.NumIRInsts
87,965,671
87,965,671
0
0.0%

IRModule.NumIRNamedMetaData
135,380
135,380
0
0.0%

IRModule.NumIRValueSymbols
6,379,066
6,379,066
0
0.0%

LLVM.NumLLVMBytesOutput
1,801,781,976
1,801,787,836
5,860
0.0%

Parse.NumFunctionsParsed
268,976
268,976
0
0.0%

Parse.NumIterableDeclContextParsed
617,726
617,677
-49
-0.01%

Parse.ParseAbstractFunctionBodyRequest
246,109
246,109
0
0.0%

Parse.ParseMembersRequest
485,878
485,829
-49
-0.01%

SILModule.NumSILGenDefaultWitnessTables
0
0
0
0.0%

SILModule.NumSILGenFunctions
1,711,421
1,711,421
0
0.0%

SILModule.NumSILGenGlobalVariables
53,554
53,554
0
0.0%

SILModule.NumSILGenVtables
18,627
18,627
0
0.0%

SILModule.NumSILGenWitnessTables
72,395
72,395
0
0.0%

SILModule.NumSILOptDefaultWitnessTables
0
0
0
0.0%

SILModule.NumSILOptFunctions
2,463,088
2,463,088
0
0.0%

SILModule.NumSILOptGlobalVariables
55,187
55,187
0
0.0%

SILModule.NumSILOptVtables
31,428
31,428
0
0.0%

SILModule.NumSILOptWitnessTables
158,611
158,611
0
0.0%

Sema.AbstractGenericSignatureRequest
30,626
30,626
0
0.0%

Sema.AnyObjectLookupRequest
281
281
0
0.0%

Sema.AreAllStoredPropertiesDefaultInitableRequest
21,021
21,021
0
0.0%

Sema.AttachedFunctionBuilderRequest
3
3
0
0.0%

Sema.AttachedPropertyWrapperTypeRequest
537,877
537,877
0
0.0%

Sema.AttachedPropertyWrappersRequest
2,134,113
2,134,108
-5
-0.0%

Sema.CallerSideDefaultArgExprRequest
78,637
78,637
0
0.0%

Sema.ClassAncestryFlagsRequest
95,094
95,094
0
0.0%

Sema.CompareDeclSpecializationRequest
461,921
459,858
-2,063
-0.45%

Sema.CursorInfoRequest
0
0
0
0.0%

Sema.CustomAttrNominalRequest
3
3
0
0.0%

Sema.DefaultAndMaxAccessLevelRequest
53,508
53,511
3
0.01%

Sema.DefaultArgumentExprRequest
38,177
38,177
0
0.0%

Sema.DefaultArgumentInitContextRequest
454
454
0
0.0%

Sema.DefaultDefinitionTypeRequest
7,789
7,789
0
0.0%

Sema.DefaultTypeRequest
454,216
454,216
0
0.0%

Sema.DynamicallyReplacedDeclRequest
1,021,107
1,021,107
0
0.0%

Sema.EmittedMembersRequest
25,900
25,900
0
0.0%

Sema.EnumRawTypeRequest
21,806
21,806
0
0.0%

Sema.EnumRawValuesRequest
10,526
10,526
0
0.0%

Sema.ExistentialConformsToSelfRequest
12,543
12,513
-30
-0.24%

Sema.ExistentialTypeSupportedRequest
15,800
15,800
0
0.0%

Sema.ExpandASTScopeRequest
6,427,614
6,427,576
-38
-0.0%

Sema.ExtendedNominalRequest
516,158
516,158
0
0.0%

Sema.ExtendedTypeRequest
75,630
75,629
-1
-0.0%

Sema.FunctionBuilderTypeRequest
3
3
0
0.0%

Sema.FunctionOperatorRequest
51,327
51,327
0
0.0%

Sema.GenericSignatureRequest
4,503,223
4,485,657
-17,566
-0.39%

Sema.GetDestructorRequest
26,389
26,389
0
0.0%

Sema.HasCircularInheritanceRequest
22,567
22,567
0
0.0%

Sema.HasCircularInheritedProtocolsRequest
10,019
10,019
0
0.0%

Sema.HasCircularRawValueRequest
8,958
8,958
0
0.0%

Sema.HasDefaultInitRequest
51,652
51,652
0
0.0%

Sema.HasDynamicMemberLookupAttributeRequest
618,082
617,301
-781
-0.13%

Sema.HasMemberwiseInitRequest
19,263
19,263
0
0.0%

Sema.HasUserDefinedDesignatedInitRequest
51,662
51,662
0
0.0%

Sema.InferredGenericSignatureRequest
163,447
163,443
-4
-0.0%

Sema.InheritedDeclsReferencedRequest
5,706,645
5,650,851
-55,794
-0.98%

Sema.InheritedTypeRequest
277,531
277,364
-167
-0.06%

Sema.InheritsSuperclassInitializersRequest
26,251
26,251
0
0.0%

Sema.InitKindRequest
91,209
91,209
0
0.0%

Sema.IsABICompatibleOverrideRequest
135,234
135,234
0
0.0%

Sema.IsAccessorTransparentRequest
309,058
309,058
0
0.0%

Sema.IsCallableNominalTypeRequest
2,089
2,089
0
0.0%

Sema.IsDeclApplicableRequest
0
0
0
0.0%

Sema.IsDynamicRequest
1,694,136
1,694,136
0
0.0%

Sema.IsFinalRequest
2,450,304
2,430,778
-19,526
-0.8%

Sema.IsGetterMutatingRequest
415,665
415,665
0
0.0%

Sema.IsImplicitlyUnwrappedOptionalRequest
2,369,091
2,367,002
-2,089
-0.09%

Sema.IsObjCRequest
1,504,434
1,502,827
-1,607
-0.11%

Sema.IsSetterMutatingRequest
336,823
336,823
0
0.0%

Sema.IsStaticRequest
3,011,019
3,003,939
-7,080
-0.24%

Sema.LazyStoragePropertyRequest
2,197
2,197
0
0.0%

Sema.LookupInModuleRequest
5,624,934
5,599,516
-25,418
-0.45%

Sema.LookupPrecedenceGroupRequest
175,916
175,916
0
0.0%

Sema.MangleLocalTypeDeclRequest
500
500
0
0.0%

Sema.ModuleQualifiedLookupRequest
2,548,126
2,522,855
-25,271
-0.99%

Sema.NamedLazyMemberLoadFailureCount
15,152
15,117
-35
-0.23%

Sema.NamingPatternRequest
177,207
177,202
-5
-0.0%

Sema.NeedsNewVTableEntryRequest
661,585
661,585
0
0.0%

Sema.NumAccessorBodiesSynthesized
189,100
189,100
0
0.0%

Sema.NumAccessorsSynthesized
277,650
277,650
0
0.0%

Sema.NumConstraintScopes
27,616,503
27,606,622
-9,881
-0.04%

Sema.NumConstraintsConsideredForEdgeContraction
92,780,056
92,777,029
-3,027
-0.0%

Sema.NumCyclicOneWayComponentsCollapsed
0
0
0
0.0%

Sema.NumDeclsTypechecked
1,412,182
1,412,182
0
0.0%

Sema.NumFunctionsTypechecked
529,260
529,260
0
0.0%

Sema.NumGenericSignatureBuilders
1,190,888
1,184,872
-6,016
-0.51%

Sema.NumLazyIterableDeclContexts
7,592,226
7,554,583
-37,643
-0.5%

Sema.NumLazyRequirementSignatures
762,256
761,029
-1,227
-0.16%

Sema.NumLazyRequirementSignaturesLoaded
568,816
567,705
-1,111
-0.2%

Sema.NumLeafScopes
17,589,198
17,580,907
-8,291
-0.05%

Sema.NumTypesDeserialized
19,217,469
19,096,948
-120,521
-0.63%

Sema.NumTypesValidated
1,373,400
1,373,394
-6
-0.0%

Sema.NumUnloadedLazyIterableDeclContexts
4,638,960
4,637,582
-1,378
-0.03%

Sema.OpaqueReadOwnershipRequest
262,685
262,685
0
0.0%

Sema.OpaqueResultTypeRequest
0
0
0
0.0%

Sema.OperatorPrecedenceGroupRequest
724
724
0
0.0%

Sema.OverriddenDeclsRequest
2,702,458
2,683,861
-18,597
-0.69%

Sema.ParamSpecifierRequest
1,190,684
1,190,683
-1
-0.0%

Sema.PatternBindingEntryRequest
433,145
433,140
-5
-0.0%

Sema.PreCheckFunctionBuilderRequest
0
0
0
0.0%

Sema.PropertyWrapperBackingPropertyInfoRequest
532,859
532,859
0
0.0%

Sema.PropertyWrapperBackingPropertyTypeRequest
537,877
537,877
0
0.0%

Sema.PropertyWrapperMutabilityRequest
610,515
610,515
0
0.0%

Sema.PropertyWrapperTypeInfoRequest
1
1
0
0.0%

Sema.ProtocolRequiresClassRequest
59,377
59,178
-199
-0.34%

Sema.QualifiedLookupRequest
6,688,194
6,673,981
-14,213
-0.21%

Sema.RangeInfoRequest
0
0
0
0.0%

Sema.RequirementRequest
107,343
107,343
0
0.0%

Sema.RequirementSignatureRequest
629,205
627,826
-1,379
-0.22%

Sema.RequiresOpaqueAccessorsRequest
1,286,424
1,286,424
0
0.0%

Sema.RequiresOpaqueModifyCoroutineRequest
253,459
253,459
0
0.0%

Sema.ResilienceExpansionRequest
1,700,090
1,700,083
-7
-0.0%

Sema.ResolveImplicitMemberRequest
354,852
354,231
-621
-0.18%

Sema.ResolveProtocolNameRequest
0
0
0
0.0%

Sema.ResultTypeRequest
633,330
633,326
-4
-0.0%

Sema.RootAndResultTypeOfKeypathDynamicMemberRequest
0
0
0
0.0%

Sema.RootTypeOfKeypathDynamicMemberRequest
0
0
0
0.0%

Sema.SelfAccessKindRequest
2,465,538
2,454,951
-10,587
-0.43%

Sema.SelfBoundsFromWhereClauseRequest
1,495,039
1,483,778
-11,261
-0.75%

Sema.SetterAccessLevelRequest
141,907
141,907
0
0.0%

Sema.StorageImplInfoRequest
1,375,254
1,375,254
0
0.0%

Sema.StoredPropertiesAndMissingMembersRequest
29,894
29,894
0
0.0%

Sema.StoredPropertiesRequest
295,206
295,206
0
0.0%

Sema.StructuralTypeRequest
2,078
2,078
0
0.0%

Sema.SuperclassDeclRequest
396,409
395,415
-994
-0.25%

Sema.SuperclassTypeRequest
52,537
52,537
0
0.0%

Sema.SynthesizeAccessorRequest
277,650
277,650
0
0.0%

Sema.SynthesizeDefaultInitRequest
4,810
4,810
0
0.0%

Sema.SynthesizeMemberwiseInitRequest
3,059
3,059
0
0.0%

Sema.TypeCheckFunctionBodyUntilRequest
529,260
529,260
0
0.0%

Sema.TypeCheckSourceFileRequest
27,076
27,076
0
0.0%

Sema.TypeDeclsFromWhereClauseRequest
28,683
28,686
3
0.01%

Sema.TypeRelationCheckRequest
0
0
0
0.0%

Sema.TypeWitnessRequest
8,336
8,335
-1
-0.01%

Sema.UnderlyingTypeDeclsReferencedRequest
309,490
307,915
-1,575
-0.51%

Sema.UnderlyingTypeRequest
33,848
33,848
0
0.0%

Sema.UnqualifiedLookupRequest
3,671,421
3,671,273
-148
-0.0%

Sema.ValueWitnessRequest
63,289
63,289
0
0.0%

Release

release brief

Regressed (1)

name
old
new
delta
delta_pct

time.swift-driver.wall
5635.8s
5694.3s
58.4s
1.04% :no_entry:

Improved (0)

name
old
new
delta
delta_pct

Unchanged (delta < 1.0% or delta < 100.0ms) (2)

name
old
new
delta
delta_pct

Frontend.NumInstructionsExecuted
50,746,207,143,320
50,984,991,765,118
238,784,621,798
0.47%

LLVM.NumLLVMBytesOutput
1,924,860,036
1,924,858,140
-1,896
-0.0%

release detailed

Regressed (0)

name
old
new
delta
delta_pct

Improved (0)

name
old
new
delta
delta_pct

Unchanged (delta < 1.0% or delta < 100.0ms) (18)

name
old
new
delta
delta_pct

AST.NumLoadedModules
30,053
30,053
0
0.0%

AST.NumTotalClangImportedEntities
1,178,286
1,178,286
0
0.0%

IRModule.NumIRBasicBlocks
5,449,021
5,449,021
0
0.0%

IRModule.NumIRFunctions
2,823,846
2,823,846
0
0.0%

IRModule.NumIRGlobals
3,183,206
3,183,206
0
0.0%

IRModule.NumIRInsts
53,487,485
53,487,485
0
0.0%

IRModule.NumIRValueSymbols
5,649,647
5,649,647
0
0.0%

LLVM.NumLLVMBytesOutput
1,924,860,036
1,924,858,140
-1,896
-0.0%

SILModule.NumSILGenFunctions
1,193,600
1,193,600
0
0.0%

SILModule.NumSILOptFunctions
1,713,054
1,713,054
0
0.0%

Sema.NumConformancesDeserialized
3,604,849
3,604,849
0
0.0%

Sema.NumConstraintScopes
27,323,568
27,323,568
0
0.0%

Sema.NumDeclsDeserialized
10,551,454
10,551,454
0
0.0%

Sema.NumFunctionsTypechecked
532,882
532,882
0
0.0%

Sema.NumGenericSignatureBuilders
280,429
280,429
0
0.0%

Sema.NumLazyIterableDeclContexts
1,315,640
1,315,640
0
0.0%

Sema.NumTypesDeserialized
5,280,297
5,280,297
0
0.0%

Sema.NumTypesValidated
826,725
826,725
0
0.0%

@swift-ci please test macOS platform

Build failed
Swift Test OS X Platform
Git Sha - eed8d8b1df3fdea193034c5c6999fb7a5d5539c5

Wat
@swift-ci please clean smoke test macOS platform

@swift-ci please clean smoke test macOS platform

authors: CodaFi shahmishal swift-ci


text:
build: hide symbols using CMake

This uses the CMake settings to correctly select the visibility
attribute for the runtime.

Replace this paragraph with a description of your changes and rationale. Provide links to external references/discussions if appropriate.

Resolves SR-NNNN.

@edymtt - I think that this is a better approach to what you were trying to do.  I'd still like to understand how this is a performance thing.  These are not weak symbols, they are COMDATed symbols.  However both of those do not incur a penalty - the weak symbol just means that if one is not provided it will be used, but the resolution is done once at load time.

@swift-ci please test

CC: @drexin

Build failed
Swift Test OS X Platform
Git Sha - 941681fb5d550e0e7dd02447fb5357c76cbc7c22

Build failed
Swift Test Linux Platform
Git Sha - 941681fb5d550e0e7dd02447fb5357c76cbc7c22

@compnerd This came from an internal verifier which says that weak symbols can impact app launch time. I'm not completely clear as to why, but I guess dyld has to do more work.

@mikeash Yes, that’s absolutely correct, the original statement of the reason just made it seem like the change caused a regression at runtime.  Eric helped clear that up.  Just a miscommunication on the actual results of the change.

@compnerd Yeah, it's always a bit tricky and weird when internal stuff bubbles up like this. Glad we're all on the same page now.

@swift-ci please test

@edymtt Ill go ahead and merge this, and then follow up with the tests for ensuring that this doesn't happen again in the future.  Thanks for the help!

authors: compnerd edymtt mikeash swift-ci


text:
VisualC: do not require modification of the Visual Studio installation

Currently Swift toolchains on Windows require manual copying/symlinking of VisualC modulemap & apinotes into the Visual Studio headers directory. However, modifying the system include directories is not acceptable in many usage scenarios, for example on CI. In other cases, it adds additional burden to the manual installation.
This PR replaces it with an automatic installation into the Swift toolchain under %SDKROOT%/usr/lib/swift/visualc.
It adds forwarding headers for the system headers included by the VisualC modulemap to workaround the restriction that modulemap should be located in the same directory as the headers it references. The forwarding headers are purposefully named the same way as the system headers (and use #include_next because of that) to avoid breaking libdispatch build & potentially other usages (some backstory)

@compnerd could you please take a look? If you're okay with this change, I think it would be super nice to have it before the 5.4 release branching.

@swift-ci please smoke test

@swift-ci please smoke test

@swift-ci please test Windows platform

@swift-ci please test Windows platform


text:
Platform: add ActiveX module

Add the ActiveX module to the Windows SDK.  This is needed for the
IPropBag2 interface.

Replace this paragraph with a description of your changes and rationale. Provide links to external references/discussions if appropriate.

Resolves SR-NNNN.

CC: @egorzhdan

CC: @egorzhdan

@swift-ci please test Windows platform

@swift-ci please test Windows platform

@swift-ci please smoke test

@swift-ci please smoke test

@swift-ci please smoke test Linux platform

@swift-ci please smoke test Linux platform

@swift-ci please smoke test Linux platform

@swift-ci please smoke test Linux platform

@swift-ci please smoke test Linux platform

@swift-ci please smoke test Linux platform

@swift-ci please test Linux platform

@swift-ci please test Linux platform


text:
[Reflection] Support OpaqueExistential in RecordTypeInfo::readExtraInhabitantIndex

Support OpaqueExistential in RecordTypeInfo::readExtraInhabitantIndex().
This missing support was identified when calling projectEnumValue() with a type of Any?. In that case, the projectEnumValue was returning false because readExtraInhabitantIndex returned false. For OpaqueExistentials, readExtraInhabitantIndex should look at the type field (named "metadata"), and use that to determine extra inhabitants.

This is an initial working version for macOS x86-64, in a specific case I was debugging in lldb. Still todo are:

[ ] Verify other platform support (!macOS || !x86-64)
[ ] Verify opaque existential + nested optional (Any?? etc)

This is an initial working version for macOS x86-64, in a specific case I was debugging in lldb. Still todo are:

[ ] Verify other platform support (!macOS || !x86-64)
[ ] Verify opaque existential + nested optional (Any?? etc)

@swift-ci test

@swift-ci test

Nice!  This is simpler than I thought it would be.  (To be honest, I've not looked at this code in a little while...).  I see a couple of other unimplemented cases that we might be able to fill in fairly quickly.
I'll sketch out some test cases this afternoon to help push this forward.

Nice!  This is simpler than I thought it would be.  (To be honest, I've not looked at this code in a little while...).  I see a couple of other unimplemented cases that we might be able to fill in fairly quickly.
I'll sketch out some test cases this afternoon to help push this forward.

test cases sound great, thanks

test cases sound great, thanks

I've changed the base branch to main now, as requested.

I've changed the base branch to main now, as requested.

I'm hoping to merge this soon, and then to cherry pick it to release/5.4. Are you ok with doing the other cases in a separate change because of this? I can add some tests for this case. Based on #30635, is validation-test/Reflection still the place to write reflection tests?

I'm hoping to merge this soon, and then to cherry pick it to release/5.4. Are you ok with doing the other cases in a separate change because of this? I can add some tests for this case. Based on #30635, is validation-test/Reflection still the place to write reflection tests?

@swift-ci test

@swift-ci test

Build failed
Swift Test OS X Platform
Git Sha - 340b85fe03009ec9cfd579af9366b0aafed7983f

Build failed
Swift Test OS X Platform
Git Sha - 340b85fe03009ec9cfd579af9366b0aafed7983f

Ugh.  Looks like I made a lot of obvious dump copy/paste errors in the 32-bit version.  I'll work on that this afternoon...

Ugh.  Looks like I made a lot of obvious dump copy/paste errors in the 32-bit version.  I'll work on that this afternoon...

I'm surprised that Linux passed here.   I'll have to double-check, but I thought Linux only had 4k extra inhabitants in 64-bit pointers, where macOS has 4G.

I'm surprised that Linux passed here.   I'll have to double-check, but I thought Linux only had 4k extra inhabitants in 64-bit pointers, where macOS has 4G.

@swift-ci Please test macOS

@swift-ci Please test macOS

I did some fixes in 7ed7122, and I'll run the tests locally to iterate on them faster.

I did some fixes in 7ed7122, and I'll run the tests locally to iterate on them faster.

I see we're both on it haha

I see we're both on it haha

Build failed
Swift Test OS X Platform
Git Sha - 7f8bebb4676a059a3ca51a0282bd19158b9b0c10

Build failed
Swift Test OS X Platform
Git Sha - 7f8bebb4676a059a3ca51a0282bd19158b9b0c10

lookiung

lookiung

@swift-ci Please test macOS

@swift-ci Please test macOS

This looks right to me.

Thanks!

"Just" need to finish filling out the test suite for it.

@tbkka Are you intending for "filling out the test suite" to be done in this PR or in a follow up?

Thanks for pushing this forward…

thanks for your guidance.

This looks right to me.

Thanks!

"Just" need to finish filling out the test suite for it.

@tbkka Are you intending for "filling out the test suite" to be done in this PR or in a follow up?

Thanks for pushing this forward…

thanks for your guidance.

@swift-ci test Linux

@swift-ci test Linux

Sorry, I probably should have said "get this test suite to pass."  I think the test we have is sufficient coverage, but it may still have some warts.  More tests can follow in a subsequent PR, I think.

Sorry, I probably should have said "get this test suite to pass."  I think the test we have is sufficient coverage, but it may still have some warts.  More tests can follow in a subsequent PR, I think.

Th new test in your wrote pass for the different architectures:
PASS: Swift(macosx-x86_64) :: Reflection/reflect_Optional_Any.swift (678 of 14039)
PASS: Swift(iphonesimulator-i386) :: Reflection/reflect_Optional_Any.swift (679 of 14039)
PASS: Swift(iphonesimulator-x86_64) :: Reflection/reflect_Optional_Any.swift (679 of 14039)
PASS: Swift(watchsimulator-i386) :: Reflection/reflect_Optional_Any.swift (679 of 14039)

Th new test in your wrote pass for the different architectures:
PASS: Swift(macosx-x86_64) :: Reflection/reflect_Optional_Any.swift (678 of 14039)
PASS: Swift(iphonesimulator-i386) :: Reflection/reflect_Optional_Any.swift (679 of 14039)
PASS: Swift(iphonesimulator-x86_64) :: Reflection/reflect_Optional_Any.swift (679 of 14039)
PASS: Swift(watchsimulator-i386) :: Reflection/reflect_Optional_Any.swift (679 of 14039)

Removed swift5-branch-managers from reviewers list; you don't need their review for merging into main

Removed swift5-branch-managers from reviewers list; you don't need their review for merging into main

@rjmccall and @jckarter – Is there any reason why different Linux platforms would have different extra inhabitant counts? This new test isn't passing on my Fedora 33 (x86-64) box and the test seems to assume that the maximum number of extra inhabitants for 64-bit platforms are all the same (which isn't true).

@rjmccall and @jckarter – Is there any reason why different Linux platforms would have different extra inhabitant counts? This new test isn't passing on my Fedora 33 (x86-64) box and the test seems to assume that the maximum number of extra inhabitants for 64-bit platforms are all the same (which isn't true).

Now I'm wondering if the tests are executed on the Linux build bots at all (as opposed to just my "unified build" setup). Let's find out: #35512

Now I'm wondering if the tests are executed on the Linux build bots at all (as opposed to just my "unified build" setup). Let's find out: #35512

Ding. This test isn't triggering on the Linux bots but it is on my cmake build:
$ curl https://ci.swift.org/job/swift-PR-Linux/25240/consoleText 2>/dev/null|grep reflect_Optional_Any
UNSUPPORTED: Swift(linux-x86_64) :: Reflection/reflect_Optional_Any.swift (617 of 14002)

I'm going to commit a workaround that unbreaks unified builds: #35513

Ding. This test isn't triggering on the Linux bots but it is on my cmake build:
$ curl https://ci.swift.org/job/swift-PR-Linux/25240/consoleText 2>/dev/null|grep reflect_Optional_Any
UNSUPPORTED: Swift(linux-x86_64) :: Reflection/reflect_Optional_Any.swift (617 of 14002)

I'm going to commit a workaround that unbreaks unified builds: #35513

@swift-ci Please test Linux

@swift-ci Please test Linux

On Linux, pointers only have 4096 extra inhabitants.  that requires some finessing of the test.  I'll work on that.

On Linux, pointers only have 4096 extra inhabitants.  that requires some finessing of the test.  I'll work on that.

Thanks. Do you think you'll have a fix soon or should I merge the workaround?

Thanks. Do you think you'll have a fix soon or should I merge the workaround?

Ding. This test isn't triggering on the Linux bots but it is on my cmake / unified build:

Just in case this is something obvious — AFAIK the reflection tests are part of the validation tests. Is your bot running these?

Ding. This test isn't triggering on the Linux bots but it is on my cmake / unified build:

Just in case this is something obvious — AFAIK the reflection tests are part of the validation tests. Is your bot running these?

Yes, my bot always runs the validation tests. (They just take a few seconds.) I thought some of the Swift bots run the validation tests. Is that still true?

Yes, my bot always runs the validation tests. (They just take a few seconds.) I thought some of the Swift bots run the validation tests. Is that still true?

authors: adrian-prantl davezarzycki kastiglione swift-ci tbkka


text:
Sema: Ban forward references of captured values from 'defer' body

While 'defer' is implemented as a local function, it doesn't
behave as one. In particular, since SILGen runs it after
destroying all local bindings that appear after the 'defer'
definition, the body of a 'defer' cannot forward reference
captured bindings the way that local functions can.
Fixes rdar://problem/75088379.

@swift-ci Please smoke test

@swift-ci Please test source compatibility

@swift-ci Please smoke test

@swift-ci Please test source compatibility

@swift-ci Please smoke test

@swift-ci Please smoke test Linux

The source combat failure is an unrelated build system issue in Kickstarter-ReactiveExtensions.

authors: jckarter slavapestov


text:
[Concurrency] Perform userInitiated adjustment after determining priority

Based on feedback, perform the adjustment from userInteractive to
userInitiated all the time, and rely on qos_class_self whenever we
don't have a task.

@swift-ci please smoke test and merge

@swift-ci please smoke test macOS


text:
[Distributed] cleanup some warnings in SILGenDistributed

Some cleanups, removing warnings.

A few cleanups and warning removals.

@swift-ci please smoke test


text:
[move-function] Implement move checking for address only lets

This new pass works as follows:

Mandatory Inlining converts builtin "move" to a mark_unresolved_move_addr when inlining the function "_move" (the only place said builtin is invoked). mark_unresolved_move_addr is semantically a copy_addr [init] so we have valid IR.

This is then run through a special checker that
either proves that the mark_unresolved_move_addr can actually be a move in which
case it converts it to copy_addr [take][init] or if it can not be a move, emit
an error and convert the instruction to a copy_addr [init]. After this is done
for all instructions, we loop back through again and emit an error on any
mark_unresolved_move_addr that were not processed earlier allowing for us to
know that we have completeness.

NOTE: The move kills checker for addresses runs after Mandatory
Inlining, but before predictable memory opts and any other memory optimizations on purpose.

@swift-ci test

Build failed
Swift Test Linux Platform
Git Sha - 5d6c72a258e313c6cde7cf340c925e50a90530f5

Build failed
Swift Test OS X Platform
Git Sha - 5d6c72a258e313c6cde7cf340c925e50a90530f5

@swift-ci test

@swift-ci test

This seems like the best place to put a review of this pass even though it's based on the latest iteration of the code.
Notes on the design description
The generally looks like a great approach from the high level description. It's should be as similar to the value-based algorithm as possible.
The same diagnostic algorithm should be used for moveonly variables and copyable variables that are explicitly moved. The name indicates otherwise. I expected simply a MoveValueCheck.cpp and MoveAddressCheck.cpp.
As discussed offline, the complete form of this diagnostic is equivalent to DefiniteInitialization. DI can insert mark_initialized markers to help you, but that's a pretty big FIXME to call out.
Please check that the design handles all the variations on indirect call arguments. I'm a little concerned that SILGen might emit temporary alloc_stack copies for some things.

semantically this instruction is actually a copy_addr [init]

Why is mark_unresolved_move_addr equivalent to copy_addr [init]? What does the [init] have to do with the move? Do you mean copy_addr [take]?

Then it partitions those uses into three sets:

It all be simpler to understand if you partition uses into two sets: live uses vs. destroys. Then say that some subset of those live uses are "moves".

NOTE: We do not need to check for
moves here since any move in our block would have either resulted in the
destroy_addr being eliminated earlier by the single block form of the
diagnostic and us exiting early or us emitting an error diagnostic and
exiting early.

This is hard to parse. Isn't it just this?
We don't check for moves because a move followed by a destroy
without an intervening init is invalid SIL, caught be the memory
verifier.

NOTE: The reason why we do not track init information on a per block is that
in our dataflow, we are treating inits as normal uses and if we see an init
without seeing a destroy_addr, we will error on the use and bail. Once we
decide to support vars, this will change.

I don't understand what "see an init without seeing a destroy_addr" means. But It seems unfortunate to have an algorithm for addressable variables that doesn't support reinitialization.

NOTE: The reason why we do not track all consuming operations

For moveonly variables, consuming operations will need to be handled just like moves. I'm not sure why the algorithm isn't just designed that way from the start. This whole discussion and design is centered on the mark_unresolved_move_addr opcode. The only thing that's special about that instruction is that it should be treated like a consume even though it's OperandOwnership isn't consuming (yet). I don't think the design should be centered around that operation.

If b is a block associated with one of our converted
mark_unresolved_move_addr, continue.

When you say "continue" here I think you mean "stop".
Notes on the implementation
There are several different data flow algorithms and dozens of data structures, without clear boundaries between them. Each aspect of the pass should be encapsulated into its own internal state distinct from its result that feeds into other aspects:

Gather function uses
Upward liveness data flow
Downward exposed consume data flow
Closure use summary (based on those 2 data flows)
Scanning for local use-after-move
Destroy placement data flow
Diagnostic handling

For each data flow, clearly separate:

Seeding data flow
Basic block summary (based on local scan)
Global data flow propagation
Data flow results

Most of the large functions in this pass are trying to do multiple unrelated things at once and have access to all different pieces of state. These are a ton of data structures across this code. They should all be encapsulated into objects that are each the output of one stage and the input of following stages.

struct GatherClosureUseVisitor : public AccessUseVisitor {

This is easy to understand. But UseState has a large number of data structures that are directly exposed to the entire pass.
Please clarify somehow that UseState is the output of GatherClosureUseVisitor. The only modification to that output is that uses are "cleared" later as they are processed. The UseState and GatherClosureUseVisitor are currently separated by 500 lines of code.
  SmallVector<MarkUnresolvedMoveAddrInst *, 1> markMoves;
  SmallPtrSet<SILInstruction *, 1> seenMarkMoves;
  SmallSetVector<SILInstruction *, 2> inits;
  SmallSetVector<SILInstruction *, 4> livenessUses;
  SmallBlotSetVector<DestroyAddrInst *, 4> destroys;
  llvm::SmallDenseMap<SILInstruction *, unsigned, 4> destroyToIndexMap;
  SmallBlotSetVector<SILInstruction *, 4> reinits;
  llvm::SmallDenseMap<SILInstruction *, unsigned, 4> reinitToIndexMap;
  llvm::SmallMapVector<Operand *, ClosureOperandState, 1> closureUses;
  llvm::SmallDenseMap<Operand *, unsigned, 1> closureOperandToIndexMap;

Should look like
  // definition of this...
  SmallVector<MarkUnresolvedMoveAddrInst *, 1> markMoves;

  // definition of this...
  SmallPtrSet<SILInstruction *, 1> seenMarkMoves;

  // definition of this...
  SmallSetVector<SILInstruction *, 2> inits;

  // definition of this...
  SmallSetVector<SILInstruction *, 4> livenessUses;

  // definition of this...
  SmallBlotSetVector<DestroyAddrInst *, 4> destroys;
  llvm::SmallDenseMap<SILInstruction *, unsigned, 4> destroyToIndexMap;

  // definition of this...
  SmallBlotSetVector<SILInstruction *, 4> reinits;
  llvm::SmallDenseMap<SILInstruction *, unsigned, 4> reinitToIndexMap;

  // definition of this...
  llvm::SmallMapVector<Operand *, ClosureOperandState, 1> closureUses;
  llvm::SmallDenseMap<Operand *, unsigned, 1> closureOperandToIndexMap;

Why do you have both SmallBlotSetVector/SmallMapVector and DenseMap for the same thing?
You can write nullptr into a vector instead of using Blot/MapVector. The null-ness can be hidden behind the API.
I don't know why you need bitsets. But, if you do map instructions to integers, why aren't you just using a single instruction map, a single vector for all uses, and 6 bitsets to represent the sets? Instead of 9 vectors, 9 hashmaps, and dynamically populated bitsets?
None of these data structure details should be exposed. You can add an API inside UseState like:
clearDestroy(Destroy *)
Bitset &getDestroyBits()

static DownwardScanResult
downwardScanForMoveOut(MarkUnresolvedMoveAddrInst *mvi, UseState &useState,
SILInstruction **foundInst, Operand **foundOperand,

APIs like this are hard to make sense of. An analysis function takes some input and produces a result. DownwardScanResult, foundInst, foundOperand, and foundClosureInsts can all be encapsulated in DownwardScanResult.

ClosureArgDataflowState

I could not figure out how the closure analysis fits with the rest of the pass. It seems like exactly the same analysis should run on the main function and on the closure, then two separate pieces of information need to be summarized for the closure:
(1) upward liveness
(2) downward exposed consumes

downwardScanForMoveOut()...
auto iter = useState.reinitToIndexMap.find(&next);
if (iter != useState.reinitToIndexMap.end()) {

Code like this makes the function hard to read. This function doesn't care how the data structure is implemented. It should just ask useState.isReinit(inst).

DataflowState::init() and upwardScanForXXX()...

It might be simpler and more efficient to have a single BasicBlockSet for all blocks with uses (from GatherUses). And have a single upwardScan function that runs for each block initializes the block's data flow summary.

performSingleBasicBlockAnalysis(...

performSingleBasicBlockAnalysis conceptually has nothing to do with global data flow propagation. DataflowState should not be passed to it.

dataflowState.markMovesThatPropagateDownwards.emplace_back(mvi);
// Now scan up to see if mvi is also a use to seed the dataflow. This could
// happen if we have an earlier move.
if (upwardScanForUseOut(mvi, dataflowState.useState)) {
LLVM_DEBUG(llvm::dbgs() << "MVI projects a use up");
dataflowState.useBlocks[mvi->getParent()] = mvi;
}

This upward scan here seems unrelated to the purpose of performSingleBasicBlockAnalysis. All the per-block data flow initialization should be encapsulated in a single place.
Generally, try to avoid conflating the move-as-use vs. the move-as-lifetime-end. Just handle move-as-use as part of the regular use set.

{
auto diag =
diag::sil_movekillscopyablevalue_value_consumed_more_than_once;
StringRef name = getDebugVarName(address);
diagnose(astCtx, getSourceLocFromValue(address), diag, name);
}

Can you centralize all the diagnostics in a single object? Then, instead of passing in address, just pass in the diagnostic handler? The interface can be:
/// True if \p move is exposed at the end of its block without first being destroyed.
bool checkUseAfterMoveInBlock(SILInstruction *move, ErrorHandler &handler) {...}

In DataflowState::process, I can't figure out why bitsets are suddently used here. What does it means to "pair" an instruction, and why we would care about "pairing".

bool DataflowState::cleanupAllDestroyAddr(...

I don't understand why cleanupAllDestroyAddr is a member of DataflowState. It takes 8 arguments. Most of them data structures defined locally then passed around. These should all be encapsulated in a separate data flow algorithm for destroy placement.
It would be really great if each data flow was a separate class that only contained data flow state and data flow logic and produced a data flow result. Nothing else whatsoever. Otherwise I think it's practically impossible to relate the code to the file-level comment that explains the design.

authors: atrick gottesmm swift-ci


text:
[Test] Eased check assertion.

Eased filecheck expectations.

@swift-ci please smoke test and merge


text:
[5.9] [Macros] Add default plugin paths for Darwin SDKs and platforms.

Explanation: Corresponding to https://github.com/apple/swift-driver/pull/1377, this adds some default plugin paths for Darwin SDKs and platforms so that macro implementations provided their can be used by the compiler through the plugin server.
Scope: Narrow; applies to all macros that currently ship within the toolchain.
Risk: Low: only affects macros in the toolchain, sending them through the out-of-process path instead of loading them directly into the compiler.
Reviewed by: @bnbarham
Issue:  rdar://110819604
Original pull request: https://github.com/apple/swift/pull/66711

@swift-ci please test


text:
[cxx-interop][nfc] Disable test for strstr on Linux.

Replace this paragraph with a description of your changes and rationale. Provide links to external references/discussions if appropriate.

Resolves #NNNNN, fix apple/llvm-project#MMMMM.

@swift-ci please smoke test

@bnbarham I think you can just merge this without testing if you want to make the bot green. The change is trivial and test only.

Sure, I don't think there's any way to test this anyway at the moment - it's only on UBI and there's no way to specify that IIUC.

authors: bnbarham zoecarver


text:
[5.10] [Concurrency] Don't infer MainActor on top level globals in the task to thread concurrency model

Cherry-pick of https://github.com/apple/swift/pull/71039
Explanation: Global actors aren't permitted in the task-to-thread concurrency model, so don't infer them.
Scope: Narrow; suppresses inference of global actors in the task-to-thread model.
Issue: rdar://121329164
Risk: Very low; only affects task-to-thread model
Testing: Added tests, and already tested on main
Reviewer: Holly Borla (author) / Doug Gregor (reviewer)

@swift-ci please test


text:
[LICM] Bail on unreferenceable storage.

When computing an access path, if a struct with unreferenceable storage is encountered, bail out.  Otherwise, an attempt will be made to construct such a struct via the struct instruction which isn't possible.
rdar://127013278

@swift-ci please test

LGTM!

authors: meg-gupta nate-chandler


text:
[Exclusivity] Temporarily disable an assert in AccessEnforcementSelec…

…tor.
This pass doesn't do anything useful downstream yet, so it's safe to temporarily
disable the assert.
Reenable after fixing:
rdar://problem/31797132 [Exclusivity] lldb asserts in AccessEnforcementSelector

@swift-ci smoke test and merge.


text:
Can't get activity classification to work with MPS

In OS10.14, activity classification works fine with CPU. I tried to force it to use MPS by setting use_mps() as True in the file _mps_utils.py. But the test case is just crashing.
Image classification seems to be using MPS, but I am not sure why activity classification is not.
Has the activity classification been verified to work with MPS? If so, could you share the steps to get it working?

Sorry you are having this issue.
It should work by default. What Mac hardware are you using?

I am using a 2016 13 inch MacBook Pro with Intel Iris Graphics 550.
Below are the steps I followed:

Install MacOS 10.14 public beta version
Get Turi Create source code: https://github.com/apple/turicreate
Set use_mps() = True in _mps_utils.py file
Build Turi
Run unit test for activity classification:
~/turicreate/run_python_test.sh debug

If I skip step 3, then unit tests run training on CPU and the tests are passing.
But, with MPS forced, the test crashes.

Unfortunately, the GPU acceleration for activity classification (and object detection) requires an AMD GPU. Image classification uses a different framework (via CoreML) to leverage GPU resources.
Probably we should update our documentation to clarify the requirements, especially since they differ across toolkits

Thanks for the clarification. Are there plans to enable activity classification with Intel graphics since MPS can support it?

Some testing with our current MPS implementation using Intel graphics did not reveal performance improvements over our MXNet (CPU only) implementation. We do plan to do some more work on activity classification, so we can certainly revisit this question after we've iterated on the implementation some.

For future reference - _mps_utils.use_mps() is an internal function, that checks the user config plus relevant hardware availability, not a user facing API.
The APIish way for enforcing GPU usage would be
tc.config.set_num_gpus(1)

Ok thank you @nickjong and @igiloh . Please keep me updated when support with Intel graphics is available. Even if we do not see perf improvement with Intel graphics, it will be good to have the option of using it.

hi @prashpal ,
If you're building TC from source - you can try modifying has_fast_mps_support() in _mps_utils.py to return always true. If you're on mac OS 10.14+ - it would use the intel GPU.

Yes, I am setting use_mps() = True in _mps_utils.py file to force to use Intel graphics.
use_mps() seems to be a check for 2 things -  has_fast_mps_support() and _tc_config.get_num_gpus() != 0.
With the above change, the test seems to be going through Intel graphics, but the validation test crashes. So I wanted to check if the tests were verified to work with Intel graphics?
Is my understanding correct?

Image classification Training - can use CPU or discrete GPU (via MPS or MXNet)
Activity classification Training - can use CPU or discrete GPU (via MPS or MXNet)
Image classification Inference - can use CPU or discrete GPU (via CoreML or MPS or MXNet) or Intel GPU (via CoreML)
Activity classification Inference - can use CPU or discrete GPU (via MPS or MXNet)

I have not verified any tests for the Intel graphics MPS code path, since this code path is not currently supported.
Image classification has two phases: feature extraction using a neural network and logistic regression based on the extracted features. The logistic regression currently always runs on CPU. The feature extraction is the same for both training and inference, and always uses CoreML, which should use GPU or CPU, as available.
Activity classification training and inference both use either MPS (on Macs with AMD GPUs) or MXNet (using GPU or CPU, as available).

Thanks for clarifying @nickjong

We should probably just go ahead and use the Intel GPU anyway, since this is less confusing. Need to verify that this works end-to-end though

Ok, thanks for the update.

Hi @nickjong , I wanted to check if we have any updates on this.

Sorry, nothing concrete to report yet, although activity classification is something we're actively investigating now

We currently expect/hope to support Skylake Intel GPUs and later, in June

Ok thanks. We should see performance improvement when using Intel GPU (via MPS) compared to CPU.

Hi
Im curious if there is any public documentation on CoreML's device selection heuristic. With the addition of 10.15's CoreML preferredMetalDevice API for MLModelConfig, I imagined it would be possible to force the MTLDevice an MLModel / Vision request runs on.
In my testing with integrated, discrete and eGPU, it appears only the eGPU consistently runs the CoreML model. My CoreML Model is a pipeline model consisting of a Mobilenet classifier with multiple outputs (multi head classifiers attached to a custom feature extractor).
Im curious to understand device selection preference for a few reasons:
a) Id like to ensure my MLModel is fed images CIImages backed by textures local to the device inference will be run on, to limit PCI transfers and keep things local
b) my model is actually fed frames of video, and WWDC '19 / 10.15 introduces VideoToolbox and AVFoundation API's to help force particular video encoders and decoders on specific GPUs.
In theory, if all works well, I should be able to specify the same MTLDevice for video decode, preprocessing, CoreML/Vision inference, and subsequent encoding - keeping all IOSurface backed pixel buffers and textures resident on the same GPU.
Apple has a Pro Apps WWDC video suggesting this is the path forward to fast path Multi GPU support / Afterburner decoder support moving forward.
Does CoreML ACTUALLY allow suggested device placement to work?
I am running a retina MacBook Pro 2018 with Vega 20 GPU, and trying various methods to get the Vega 20 to light up.

Disabling automatic graphics switching

Disabling automatic graphics switching / setting NSSupportsAutomaticGraphicsSwitching to False

Disabling automatic graphics switching / setting NSSupportsAutomaticGraphicsSwitching to True

Enabling automatic graphics switching / setting NSSupportsAutomaticGraphicsSwitching to False

Enabling automatic graphics switching / setting NSSupportsAutomaticGraphicsSwitching to True

having a full battery and plugged into my Apple power adaptor

having full battery and plugged into my eGPU

I can only on occasion get the Vega 20 to 'light up' - but can consistently have CoreML run on the eGPU (Radeon 580)
I can inspect the CoreML model and see its MLConfig has a preferred device set to the Vega 20, but Instruments, Xcode, and Activity Monitor all report no GPU usage on the Vega no, and in fact, sometimes none at all.
Any insight would be most helpful.
Apologies if this is not the best repository to post my query to.
Thanks in advance.

@vade - this isn't the right place to ask this question. I suggest reporting the issue here: https://developer.apple.com/bug-reporting/

I hear you @TobyRoseman - however having these convo's in the open rather than behind closed feedback is helpful to other developers who have similar questions, and leaves a breadcrumb trail to answers. I'm sure you understand!
But yes, ive asked there and on S/O as well. Appreciate the response!

authors: TobyRoseman igiloh nickjong prashpal srikris vade

repo: apple/turicreate


text:
[DO NOT MERGE] Refactor distance sampling for volumes

Instead of calling march function in PathTracer each time the volume is entered, we treat volume scattering in the same way we treat surface scattering, using PathTracer::trace function.
As we need to obtain somehow a position of the next volume scattering, instead of tracing the ray to the next surface in PathTracer completely, we sample the distance and create volumetric shading point. Since importance distance sampling might depend on the scene lights (with backward tracing) or on camera (with forward tracing), the whole procedure of obtaining the next shading point is delegated to the visitors.
Each distance sample contains information about the phase function at the sampled point. Since now there is a BSDF wrapper around phase functions, both direct lighting integration and the bounce itself can be processed in a general way, just like normal surface BSDF.
It seems that integrating direct lighting over the entire ray via several distance samples does not fit well with the entire path tracing architecture and isn't applicable to non-homogeneous volumes, though it can decrease the variance. One distance sample which is used both for determining the bounce and for the direct lighting integration allows to trace more rays per second and the overall result looks not much worse that the original one.

repo: appleseedhq/appleseed

license: mit


text:
more misc work

OSL flip handedness.
Reverted removal of Ashikmin OSL closure.
Small stdosl.h changes + formatting.

Optimized a bit, addressed comments and rebased.

Fixed and rebased

Thanks!


text:
Migrate the demos/requirements.txt and demo to work with lakefs_sdk

As per title.

Addressed in https://github.com/appliedAI-Initiative/lakefs-spec/pull/117

Addressed in https://github.com/appliedAI-Initiative/lakefs-spec/pull/117

repo: appliedAI-Initiative/lakefs-spec


text:
Add page about Ecto/Telemetry configuration

Although still a bit of a stub, this page explains how the Ecto/Telemetry configuration works, and how each of the parameters passed to it work.

Moved most of the Ecto docs to the Ecto page, and am now linking from the Phoenix guide to the Ecto one for more information.

repo: appsignal/appsignal-docs


text:
feat: Adds video "How To Generate PDF Reports With Carbone.io"

Adds documentation and video guide for "How To Generate PDF Reports With Carbone.io"
Video link https://www.youtube.com/watch?v=xlthDth2S6Q

@Pranay105 @jnikhila please review this 🙏

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

authors: CLAassistant cokoghenun

repo: appsmithorg/appsmith-docs


text:
chore: Added content changes for #845

#845  changes.

closing as fixed by #926


text:
[Bug]: Unable to add GET parameters conditionally on an API request.

Is there an existing issue for this?

[X] I have searched the existing issues

Description
I have the following paginated API request:
https://api.vercel.com/v6/deployments?limit=4

The pagination is done with from and to parameters

If I wanted to get the next page, I should pass the next cursor from the response to from parameter of the request as follows:
https://api.vercel.com/v6/deployments?limit=4&from=1662731501677

If I wanted to get the previous page, I should pass prev cursor from response to to parameter of the request:
https://api.vercel.com/v6/deployments?limit=4&to=1662735248196

This is how I constructed the URL on Appsmith's API pane:
https://api.vercel.com/v6/deployments?limit=4{{appsmith.store.getDeploymentsPaginationQuery}}

where I conditionally set the appsmith store getDeploymentsPaginationQuery as either &from=1662731501677 or &to=1662735248196 or '' (for the first request)
But, what I end up having is args like this:
{
  "limit": "4&from=1662731501677"
}

Note: Vercel's Documentation for reference
Steps To Reproduce
Make the following API request in Appsmith and observe the args section of the response:
https://echo.hoppscotch.io/v6/deployments?limit=4{{Math.random() > 0.5 ? '&param1=5' : '&param2=6'}}

Public Sample App
No response
Version
Cloud

Hi @aswathkk , thanks for reporting this issue! would you please explain what the expected result should be instead of:
{ "limit" : "4&from= 1662731501677"
also, are you able to make this work by using the params?

@rohan-arthur
Here the value of limit parameter becomes 4&from=1662731501677.
I wanted that value to be only 4 and it should treat the string after 4 to be the next set of parameters.
So, the entire argument should be
{
  "limit": "4",
  "from": "1662731501677"
}

I was able to make this work using params since Vercel also accepts empty values like limit=4&from=&to=. Not sure whether all API endpoints work like this or not.
The step to Reproduce I wrote 4 days back is not working anymore!
Here's the steps to reproduce this issue:

Create a JSObject (JSObject1)

export default {
  myFun1: () => {
    return Math.random() > 0.5 ? '&param1=5' : '&param2=6';
  }
}

Make the following API request in Appsmith and observe the args section of the response:

https://echo.hoppscotch.io/abcd?limit=4{{ JSObject1.myFun1() }}

Response:
{
  "method": "GET",
  "args": {
    "limit": "4&param1=5"
  },
  "data": "",
  "headers": {...},
  "path": "/abcd",
  "isBase64Encoded": true
}

The args should've been:
 "args": {
    "limit": "4",
    "param1": "5"
  },

This seems related to #14706

authors: Nikhil-Nandagopal aswathkk rohan-arthur

repo: appsmithorg/appsmith


text:
[Bug]: Text input does not work with "[]"

Is there an existing issue for this?

[X] I have searched the existing issues

Description
Hi,
When using multi-line or normal text input field in a form created from a postgres table, and try to input a json string starting with "[]", something seems to happen that changes the string and inserts an altered string into the database removing the "[]". The aim here is to store the full stringified version of the json as a text in the column.
Examples:
[] is resulting in a null error (no string at all)
[{"a" : "b"}] in something like {"{"a" : "b"}"}

Steps To Reproduce

Create a form from a postgres database with a text field
try to insert a json string into the data base with a text input or multitext input that is an array e.g. "[]"

Public Sample App
No response
Version
Self hosted - Appsmith Community v1.8.3

@jondoering this should work if you turn off the prepared statement option in the query

authors: Nikhil-Nandagopal jondoering


text:
[Bug]:Issue with URL Column type in Table widget

Is there an existing issue for this?

[X] I have searched the existing issues

Description
For column type URL, the URL link is getting appende to the homepage link and hence incorrect behaviour is observed
Steps To Reproduce

DnD a table widget
Bind any DB and change anyone column type to URL
Click on that URL link
you'll notice the link was appended to the home page URL

Public Sample App
No response
Issue video log
https://www.loom.com/share/54fd482c08244f449be04c0b084942e5
Version
production

@kamakshibhat-appsmith can you check if this is same as #17022 because i think it is. If yes, can you please close it as a duplicate?

Closing this as duplicate of #17022

authors: dilippitchika kamakshibhat-appsmith


text:
[Bug]: Issues with centring of the widget in Edit mode

Is there an existing issue for this?

[X] I have searched the existing issues

Description

Issue with JSON form - When there are multiple fields in JSON form , and your click on the first one to add some details , the widget starts centring and the current input field goes out of view
Issues with dropdown (for select widgets, phone etc)
Issue with Modal, on clicking the child widget inside the modal, the main canvas starts scrolling

Steps To Reproduce
https://www.loom.com/share/17b287ca5e794602bb47c51c930fe7d8
Link to app-->
Untitled application 97 (2).json.zip
Public Sample App
No response
Environment
Production
Issue video log
No response
Version
Cloud/Self-hosted

Validated on release Appsmith v1.9.32-SNAPSHOT
Issue with JSON form - When there are multiple fields in JSON form , and your click on the first one to add some details , the widget starts centring and the current input field goes out of view
Issue with Modal, on clicking the child widget inside the modal, the main canvas starts scrolling
Working fine

authors: kamakshibhat-appsmith kavitasmoolya


text:
[Task]: Analytics event [One time query]: Get the number of JSObjects per page for users on cloud

Is there an existing issue for this?

[X] I have searched the existing issues

SubTasks

Check the number of JSObjects per page. The format of the return statement should be as follows:

JSObject count
Number of pages with these JS Objects

1
100

2
70

@hetunandu @kocharrahul7 is this a client side event

DB Collection: newAction
Aggregate Query:
[
{
$match: {
pluginType: "JS",
},
},
{
$group: {
_id: "$unpublishedAction.pageId",
count: {
$count: {},
},
},
},
{
$group: {
_id: "$count",
PageCount: {
$count: {},
},
},
},
{
$project: {
JSObjectCount: "$_id",
_id: 0,
PageCount: 1,
},
},
{
$sort: {
JSObjectCount: 1,
},
},
]

@kocharrahul7 @rohan-arthur
Output from Appsmith Prod:

JSObject count
Number of pages with these JS Objects

1
5532

2
6289

3
1842

4
1328

5
691

6
535

7
334

8
349

9
285

10
761

@kocharrahul7 @rohan-arthur
Output from Appsmith Prod:
Queried Collection:  ActionCollection
Total JSObjects: 2533995

JSObject count
Number of pages with these JS Objects

1
1220

2
248

3
68

4
28

5
15

6
4

7
4

8
1

9
2

10
2

DB Collection: actionCollection
Aggregate Query:
[   {     $match: {       "unpublishedCollection.pluginType": "JS",     },   },   {     $group: {       _id: "$unpublishedCollection.pageId",       count: {         $count: {},       },     },   },   {     $group: {       _id: "$count",       PageCount: {         $count: {},       },     },   },   {     $project: {       JSObjectCount: "$_id",       _id: 0,       PageCount: 1,     },   },   {     $sort: {       JSObjectCount: 1,     },   }, ]


text:
[Bug]: System generated commit drops a widget

Is there an existing issue for this?

[X] I have searched the existing issues

Description
When I pull my project from git remote, Appsmith generates a commit dropping one of my widgets. This happens only to selected widgets, after I do some changes to them.
The commit reads "System generated commit, for syncing changes with remote after git pull" and the changes from that commit are "drop file data_table.json".
What's weird is that on the source branch I develop with (dev), the widget is present in both "Edit" and "Launch" views, but the repo does NOT contain the widget JSON file.
When I manually revert this commit from git, and then pull the changes into Appsmith, it again generates the commit that's dropping the widget file.
When I import this project from Git on a different Appsmith instance (on my local machine) - the widget is gone.
What seems to solve the issue is copying the widget, deleting the old one, and renaming the new to something like data_table2. If I rename it to data_table - it once again goes missing.
So it seems the widget is somehow cached in Appsmith, and it would not let me return the widget file into git.
Steps To Reproduce
No idea how to trigger this, it's intermittent.
Public Sample App
No response
Environment
Production
Issue video log
No response
Version
Self Hosted v1.9.42

@phantom943 Since you have figured out a workaround and this is not consistently reproducible, we would appreciate it if you could get on a call and we can triage this together. You can schedule a call using this link.

@phantom943 we're closing this issue for now. Please let us know if this resurfaces.

authors: Nikhil-Nandagopal phantom943 riteshkew


text:
[Bug]: SSH Misconfiguration error when connecting to Forgejo repository

Is there an existing issue for this?

[X] I have searched the existing issues

Description
When connecting my app to Git it says SSH key misconfiguration. I added my deploy key to the repository in Forgejo with write access as directed.
The user sees this error in their logs related to this. Summary:
backend stdout | [2024-02-15 18:40:43,570] userEmail=REDACTED, sessionId=1c667c51-1449-4b0f-a8ee-3f351da9e58b, thread=boundedElastic-40, requestId=0a6721d1-70ff-4be8-ba55-e64e1370ef40 - boundedElastic-40: Cloning the repo from the remote git@coastalcommits.com:CornerstoneEngineering/ArchiveRehydrator.git
backend stdout |
backend stdout | [2024-02-15 18:40:44,158] userEmail=REDACTED, sessionId=1c667c51-1449-4b0f-a8ee-3f351da9e58b, thread=boundedElastic-40, requestId=0a6721d1-70ff-4be8-ba55-e64e1370ef40 - Error while cloning the remote repo,
backend stdout | org.eclipse.jgit.api.errors.TransportException: git@coastalcommits.com:CornerstoneEngineering/ArchiveRehydrator.git: Auth fail

...

backend stdout | [2024-02-15 18:40:44,179] userEmail=REDACTED, sessionId=1c667c51-1449-4b0f-a8ee-3f351da9e58b, thread=boundedElastic-40, requestId=0a6721d1-70ff-4be8-ba55-e64e1370ef40 -
backend stdout | com.appsmith.server.exceptions.AppsmithException: SSH key is not configured correctly. Did you forget to add the SSH key to your remote repository? Please try again by reconfiguring the SSH key with write access.
backend stdout |        at com.appsmith.server.services.ce.GitServiceCEImpl.lambda$connectApplicationToGit$40(GitServiceCEImpl.java:774)
backend stdout |        Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException:
backend stdout | Error has been observed at the following site(s):
backend stdout |        *__checkpoint ⇢ Handler com.appsmith.server.controllers.GitController#connectApplicationToRemoteRepo(String, GitConnectDTO, String) [DispatcherHandler]

I expect my app to be connected after following the instructions.
Steps To Reproduce

Spin up a Forgejo instance.
Create a new repository in Forgejo.
Open Appsmith and edit any app not connected to git.
Follow the steps to connect the app using the Forgejo repository up until adding the deploy key.
Add the deploy key to the Forgejo repository with write access.
Click Connect button.
Observe SSH key misconfiguration error in the UI.

Public Sample App
No response
Environment
Production
Issue video log
https://www.loom.com/share/8a3dea5ff29d429ea78bb2f90db03199
Version
Self Hosted 1.10
 Front conversations

Hey there! I'm the user who submitted the original support post. I was asked what version of Appsmith I'm using. Here's some details:

Appsmith v1.12 (Released 12 hours ago)
Image name & label: appsmith/appsmith-ee:latest
Image hash: 56678b56457ca75aac37cbb75907a900292ce5ef65c346e3cfd21478af50efa1

If anything else is required from me, let me know and I'll get back to ya'll.

authors: LagunaElectric SeaswimmerTheFsh


text:
Unexpected duplicated JSObject on update

We had customers reporting that sometimes the JSObject got duplicated on update action.
Since the team has been working on improving JSObjects stability and performance, we need to check if this issue is still happening. If not, we can close it. If yes, we need to understand the cause and the potential fix for it.
here it is the page of the evidence, with the link for more context.

Tried to reproduce this issue by following below steps:
trying to save the JSObject in a complex application

While it is saving, refresh the page
check if there is a duplicate JSObject created or not.
Do the same while trying to rename a function instead of updating the body of the function and verify the same.

Issue was not reproducible even for a large application and large JS object, hence closing it.
CC: @rishabhrathod01 @carinanfonseca @nerbos


text:
refactor: Shift loader component to common file

Description

Shifted loader component to common file so that it can be used in other components as well.

Fixes #16376
Type of change

Refactoring

How Has This Been Tested?

Tested if the loader on Admin settings screen is working as expected.

Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes

/ok-to-test sha=14634f0


text:
feat: text-wrap change

Description
Update text wrap to pretty. This will enable better wrapping of text and balance white space.
Fixes #33474
Automation
/ok-to-test tags="@tag.Sanity"
:mag: Cypress test results

[!CAUTION]
If you modify the content in this section, you are likely to disrupt the CI result for your PR.

Communication
Should the DevRel and Marketing teams inform users about this change?

[ ] Yes
[x] No

/build-deploy-preview skip-test=true


text:
[DO NOT MERGE] Hackathon: Feat/quicker building blocks experience

Description

[!TIP]
Add a TL;DR when the description is longer than 500 words or extremely technical (helps the content, marketing, and DevRel team).
Please also include relevant motivation and context. List any dependencies that are required for this change. Add links to Notion, Figma or any other documents that might be relevant to the PR.

Fixes #Issue Number
or
Fixes Issue URL

[!WARNING]
If no issue exists, please create an issue first, and check with the maintainers if the issue is valid.

Automation
/ok-to-test tags=""
:mag: Cypress test results

[!CAUTION]
If you modify the content in this section, you are likely to disrupt the CI result for your PR.

Communication
Should the DevRel and Marketing teams inform users about this change?

[ ] Yes
[ ] No

Summary by CodeRabbit

New Features

Introduced a pre-push Git hook to verify changes before pushing.
Added support for creating reusable building blocks with a new UI modal and associated form.
Enabled building block management with backend service and repository support.
Added context menu option to create building blocks in the editor.

Enhancements

Adjusted Cypress Sidebar timeout for better stability.

Infrastructure

Updated GitHub Action to use the latest Ubuntu environment for builds.

/build-deploy-preview skip-tests=true

/build-deploy-preview skip-tests=true


text:
fix: Added new row addition action for form value changes

Description
Adding multiple rows of where clause and then removal of one of those caused all the clauses to be deleted. This is because only the remove command was being recorded, not the add one. Fixed the same with this PR.
Fixes #7630
Type of change

Bug fix (non-breaking change which fixes an issue)

Checklist:

[ ] My code follows the style guidelines of this project
[ ] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes

Test coverage results :test_tube:
:green_circle: Total coverage has increased
// Code coverage diff between base branch:release and head branch: fix/7630-fixing-where-clause-add-delete-row

Status
File
% Stmts
% Branch
% Funcs
% Lines

:green_circle:
total
54.61 (0)
36.4 (0.01)
33.83 (0)
55.21 (0.01)

:green_circle:
app/client/src/utils/autocomplete/TernServer.ts
50.71 (0.24)
39.13 (0.87)
36.21 (0)
55.35 (0.27)

@ayushpahwa Can we please merge the latest release into this PR & prioritize closing this PR?

authors: ayushpahwa mohanarpit


text:
node-red: Run stack as non root

Checklist:

[x] Read the Code of Conduct and Contributing Guidelines.

[x] Followed the commit message guidelines.

[x] Stack adheres to Appsody stack structure.

Modifying an existing stack:

[x] Updated the stack version in stack.yaml

Changes to the stack to run as non root user during local development and production. This should avoid any permission problems and solves an issue with appsody build in Openshift. I was having issues setting permissions to the node user that comes with the node image and so I created a new user.
Related Issues:
Related to: https://github.com/appsody/stacks/issues/622

Fails to run with same issues as in https://github.com/appsody/stacks/pull/604#issuecomment-590523106

Needs to be synced with the other PRs: #589 #604 #624

Closing for now as we don't think we reached a consensus as to how this will be done. If others disagree we can reopen and reassess how to approach this.

repo: appsody/stacks


text:
fix: datetime hourcycle formatting

What does this PR do?

fixes date time hour cycle

Test Plan

add tests

Have you read the Contributing Guidelines on issues?
✅

Related issue:

https://github.com/appwrite/appwrite/issues/3918

repo: appwrite/console


text:
Update wizard steps to use formal language

What does this PR do?
@TGlide I missed some stuff when reviewing your PR, here's the fix 😅
Also, can we hide selfSigned for cloud projects @TGlide @ArmanNik
Test Plan
(Write your test plan here. If you changed any code, please provide us with clear instructions on how you verified your changes work.)
Related PRs and Issues
(If this PR is related to any other PR or resolves any issue or related to any issue link all related PR and issues here.)
Have you read the Contributing Guidelines on issues?
(Write your answer here.)

@gewenyu99 I think is out of date since you addressed these one in the fix-sentence-case branch

@ArmanNik yep


text:
Feature Request: making this package offline persistent natively

If this package could natively enable/disable the persistent storage of data (caching the loaded data for offline usage) then it would change the game. (Just like firebase :3)

You can pass storage as constructor param. You need to extend DirectusStorage class. and pass it's instance to constructor
Example:
// This way we will store data in this custom storage, that can be in memory
Directus('http://localhost', storage: someStorageInstance);

authors: apstanisic reev-create

repo: apstanisic/directus-dart

license: MIT


text:
Covnert the project into react js

I want to convert the project into react js so that the contributors find easy to implement the new features Please assign me this issue under GSSOC'24

@Vishnu1804K Welcome to METAVERSE!!!!!!!!!!
Please try to make raise issue with the proper Issue templates mentioned in the repo so that it will clarify all the things.


text:
BUG UI Enhancement of Image Gallery

Describe the bug

THE whole image gallery is not at all responsive and it is occupying a lot of space in the webpage and at the end the label is not visible.
And it is overlapping the testimonials.
Expected behavior

I will change it responsive and user friendly.
Screenshots (Mandatory)

Additional Context (Mandatory )
Are you contributing under any open-source program ?
GSSoC'24

Please Assign this to me under GSSOC'24.

@sharmi4590 this is really a very good issue . Try to design this trip gallery portion interesting . & I think there is some previously assigned issues you already have . So try to complete that at first

please assign me this issue. I want to work on this issue

please assign me this issue. I want to work on this issue

This issue is already assigned . Please try to raise new issues on which you want to work on


text:
bump tfsec version on chocolatey

Hi
can you upload a new version to Chocolatey
https://community.chocolatey.org/packages/tfsec
also check virus report at https://community.chocolatey.org/packages/tfsec#virus

Done - https://community.chocolatey.org/packages/tfsec/0.60.1 It's pending
Also, very little we can do about the virus report, its a false positive which seems to be shared by a lot of Go based applications

thanks @owenrumney
PS: there is a nice workaround for this
choco install tfsec
tfsec --update

authors: owenrumney vanmash

repo: aquasecurity/tfsec


text:
bug: wget: bad address '' for v1.28.1

Describe the bug
wget: bad address ''
When using latest tag, my tfsec-action is using the new v1.28.1 release. It appears that the release assets are named differently than in previous releases and the grep in the action is failing to return any file names from the wget output grep -o -E 'https://.+?tfsec-linux-amd64'
To Reproduce
Steps to reproduce the behavior:

Run tfsec-action with default latest version set.

Expected behavior
wget returns a json output, grep -o -E 'https://.+?tfsec-linux-amd64' returns a download url for the tar.gz, wget downloads that file.
Output of your tfsec command with --debug flag

System Info

tfsec version: v1.28.1
terraform version: 1.3.3
OS: linux_amd64

Example Code
N/A
Additional context
Reverting to v1.28.0 works just fine.

Noticed this bug as well with version 1.28.1

@mennlo How did you lock to version 1.28.0 until this gets resolved? I see a min required version directive in the configuration file docs, but I fear that will automatically bump up from 1.28.0 to 1.28.1.

@scottelundgren I'm using GitHub Actions, so I just set the version parameter.
      - name: Scan
        id: tfsec
        uses: aquasecurity/tfsec-action@v1.0.2
        with:
          version: 'v1.28.0'
          working_directory: ${{ inputs.environments_folder }}

I also noticed that the v1.28.1 release only has 9 assets related to it, while every release before it, including v1.28.0 has 34. @liamg was this planned?

I think this is the problem [failed release] (https://github.com/aquasecurity/tfsec/actions/runs/3322656189/jobs/5491987947) The release step failed. @liamg

Thanks all, rerunning now. Looks like docker.io responded weirdly for a bit.

...and we have artifacts :partying_face:

authors: ANicolucciTreeOfLife liamg mennlo scottelundgren