text:
Firebase gist not displaying UI properly

This gist does not display its UI when run:
https://dartpad.dev/f3d57a133b61f88aa938e480e1d57e9b?null_safety=true
There's a Debug banner that appears in the iframe, but no other output. The DartPad console displays a "Script error" message, and there's a generic message in the browser console:
dart_sdk.js:6676 Uncaught Error: Assertion failed: file:///Users/brettmorgan/Documents/GitHub/dart-services/flutter-sdk/packages/flutter/lib/src/widgets/framework.dart:2539:20
_debugCurrentBuildTarget == context
is not true
    at Object.throw_ [as throw] (dart_sdk.js:5041)
    at Object.assertFailed (dart_sdk.js:4980)
    at flutter_web.js:337735
    at framework.BuildOwner.new.buildScope (flutter_web.js:337739)
    at RenderObjectToWidgetAdapter.new.attachToRenderTree (flutter_web.js:326164)
    at binding$.WidgetsFlutterBinding.new.attachRootWidget (flutter_web.js:326036)
    at flutter_web.js:326030
    at internalCallback (dart_sdk.js:24253)

@RedBrogdon this came from Frank, right? A talk or a workshop? Do we know whether this works in Flutter Web, outside DartPad?

@goderbauer helped me debug, and we found that the dartfire.Database db = dartfire.database(); line in _MyHomepageState.initState() is causing a problem; if you wrap that line in a try/catch, you can see the following error: (for some reason, Flutter swallows the error):
NoSuchMethodError: tried to call a non-function, such as null: 'dart.global.firebase.database'
at Object.database$ [as database] (<anonymous>:4459:101)
at main._MyHomePageState.new.initState (<anonymous>:900:28)
at framework.StatefulElement.new.[_firstBuild] (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:343165:54)
at framework.StatefulElement.new.mount (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:343035:28)
at framework.SingleChildRenderObjectElement.new.inflateWidget (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:215128:16)
at framework.SingleChildRenderObjectElement.new.updateChild (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:214993:25)
at framework.SingleChildRenderObjectElement.new.mount (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:327839:33)
at framework.StatelessElement.new.inflateWidget (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:215128:16)
at framework.StatelessElement.new.updateChild (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:214993:25)
at framework.StatelessElement.new.performRebuild (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:343071:35)
at framework.StatelessElement.new.rebuild (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:215471:12)
at framework.StatelessElement.new.[_firstBuild] (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:343039:12)
at framework.StatelessElement.new.mount (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:343035:28)
at framework.SingleChildRenderObjectElement.new.inflateWidget (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:215128:16)
at framework.SingleChildRenderObjectElement.new.updateChild (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:214993:25)
at framework.SingleChildRenderObjectElement.new.mount (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:327839:33)
at framework.SingleChildRenderObjectElement.new.inflateWidget (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:215128:16)
at framework.SingleChildRenderObjectElement.new.updateChild (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:214993:25)
at framework.SingleChildRenderObjectElement.new.mount (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:327839:33)
at framework.StatefulElement.new.inflateWidget (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:215128:16)
at framework.StatefulElement.new.updateChild (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:214993:25)
at framework.StatefulElement.new.performRebuild (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:343071:35)
at framework.StatefulElement.new.performRebuild (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:343191:13)
at framework.StatefulElement.new.rebuild (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:215471:12)
at framework.StatefulElement.new.[_firstBuild] (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:343039:12)
at framework.StatefulElement.new.[_firstBuild] (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:343184:29)
at framework.StatefulElement.new.mount (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:343035:28)
at framework.StatefulElement.new.inflateWidget (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:215128:16)
at framework.StatefulElement.new.updateChild (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:214993:25)
at framework.StatefulElement.new.performRebuild (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:343071:35)
at framework.StatefulElement.new.performRebuild (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:343191:13)
at framework.StatefulElement.new.rebuild (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:215471:12)
at framework.StatefulElement.new.[_firstBuild] (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:343039:12)
at framework.StatefulElement.new.[_firstBuild] (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:343184:29)
at framework.StatefulElement.new.mount (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:343035:28)
at framework.SingleChildRenderObjectElement.new.inflateWidget (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:215128:16)
at framework.SingleChildRenderObjectElement.new.updateChild (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:214993:25)
at framework.SingleChildRenderObjectElement.new.mount (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:327839:33)
at framework.SingleChildRenderObjectElement.new.inflateWidget (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:215128:16)
at framework.SingleChildRenderObjectElement.new.updateChild (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:214993:25)
at framework.SingleChildRenderObjectElement.new.mount (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:327839:33)
at _InheritedNotifierElement.new.inflateWidget (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:215128:16)
at _InheritedNotifierElement.new.updateChild (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:214993:25)
at _InheritedNotifierElement.new.performRebuild (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:343071:35)
at _InheritedNotifierElement.new.rebuild (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:215471:12)
at _InheritedNotifierElement.new.[_firstBuild] (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:343039:12)
at _InheritedNotifierElement.new.mount (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:343035:28)
at framework.SingleChildRenderObjectElement.new.inflateWidget (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:215128:16)
at framework.SingleChildRenderObjectElement.new.updateChild (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:214993:25)
at framework.SingleChildRenderObjectElement.new.mount (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:327839:33)
at framework.StatefulElement.new.inflateWidget (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:215128:16)
at framework.StatefulElement.new.updateChild (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:214993:25)
at framework.StatefulElement.new.performRebuild (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:343071:35)
at framework.StatefulElement.new.performRebuild (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:343191:13)
at framework.StatefulElement.new.rebuild (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:215471:12)
at framework.StatefulElement.new.[_firstBuild] (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:343039:12)
at framework.StatefulElement.new.[_firstBuild] (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:343184:29)
at framework.StatefulElement.new.mount (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:343035:28)
at framework.InheritedElement.new.inflateWidget (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:215128:16)
at framework.InheritedElement.new.updateChild (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:214993:25)
at framework.InheritedElement.new.performRebuild (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:343071:35)
at framework.InheritedElement.new.rebuild (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:215471:12)
at framework.InheritedElement.new.[_firstBuild] (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:343039:12)
at framework.InheritedElement.new.mount (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:343035:28)
at framework.InheritedElement.new.inflateWidget (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:215128:16)
at framework.InheritedElement.new.updateChild (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:214993:25)
at framework.InheritedElement.new.performRebuild (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:343071:35)
at framework.InheritedElement.new.rebuild (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:215471:12)
at framework.InheritedElement.new.[_firstBuild] (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:343039:12)
at framework.InheritedElement.new.mount (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:343035:28)
at framework.StatefulElement.new.inflateWidget (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:215128:16)
at framework.StatefulElement.new.updateChild (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:214993:25)
at framework.StatefulElement.new.performRebuild (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:343071:35)
at framework.StatefulElement.new.performRebuild (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:343191:13)
at framework.StatefulElement.new.rebuild (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:215471:12)
at framework.StatefulElement.new.[_firstBuild] (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:343039:12)
at framework.StatefulElement.new.[_firstBuild] (https://storage.googleapis.com/nnbd_artifacts/2.14.0-301.2.beta/flutter_web.js:343184:29)
...

@RedBrogdon This makes me wonder if we're miissing a top-level object which is loaded via JS interop. Right now, Dart pad only loads in firebase-app.js and firebase-auth.js. Might something else be needed?
@goderbauer was able to launch the app without issues in Flutter Web (outside DartPad) with just these two JS SDKs, so probably a false trail.
Is this an older style of FlutterFire, using firebase.dart instead of firebase_core.dart and cloud_firestore.dart?

Actually, I had the following scripts in my index.html:
<script src="https://www.gstatic.com/firebasejs/8.6.1/firebase-app.js"></script>
<script src="https://www.gstatic.com/firebasejs/8.6.1/firebase-database.js"></script>
<script src="https://www.gstatic.com/firebasejs/8.6.1/firebase-firestore.js"></script>
<script src="https://www.gstatic.com/firebasejs/8.6.1/firebase-auth.js"></script>

Maybe dartpad IS missing the firebase-database.js to make this work?

Oh ho! Thanks for following up. This may be an easy fix. Will check ASAP.

On the questions about an older style: the firebase.dart library is a pure Dart implementation predating FlutterFire, and only support Auth and RTDB. Since FlutterFire's RTDB plugin doesn't support web yet, the firebase.dart package is the only option to use RTDB in Dartpad (and Flutter web in general).

Thanks @johnpryan
I found we may need the allow-same-origin policy on our iframes.

@RedBrogdon is this still a P1 issue?

This is probably out of date since we don't currently have Firebase support.

authors: RedBrogdon goderbauer johnpryan mosuem puf srawlins


text:
removing completion duplicates

removed duplicates when the completion hint is both a getter and a setter
merge the getter and a setter to 1 completion with classname type-getter_and_setter

Great idea! I like the look of it - it's easy for the user to identify what it is, in relation to the getter and setter icons.

Cool, will clean this up tomorrow.
I was also thinking in this filtering we could also do something like if,
result.text is stringToReplace, put the result at the beginning of the
list.
Op 31 mrt. 2015 04:07 schreef "Devon Carew" notifications@github.com:

Great idea! I like the look of it - it's easy for the user to identify
what it is, in relation to the getter and setter icons.
—
Reply to this email directly or view it on GitHub
https://github.com/dart-lang/dart-pad/pull/218#issuecomment-87899458.

We could also put the filtering logic in the initialisation of a CompletionResult:
  CompletionResult(completions, this.replacementString, {this.replaceOffset, this.replaceLength}) {
    List<Completion> filterCompletions = new List.from(completions);
    for (Completion completion in completions) {
      // If the text to be replaced matches the completion text, put it on top.
      if (completion.value == replacementString && filterCompletions[0].value != completion.value) {
        filterCompletions.remove(completion);
        filterCompletions.insert(0, completion);
      }
      for (Completion other in completions) {
        // Removes duplicates when a completion is both a getter and a setter.
        if (completion.isSetterAndMatchesGetter(other)) {
          filterCompletions.removeWhere((c) => completion == c);
          other.type = "type-getter_and_setter";
        }
      }
    }
    this.completions = filterCompletions;
  }
}

I think I'd like us to extract a bunch of these filters into a series of separate methods. That will make it much easier if we decide to move some of the sorting or filtering from the client back to the server. But I'll leave it up to @devoncarew and @kasperpeulen  whether you think that is too much architecture at this stage.

merging -

source: gharchive/pull-request

authors: devoncarew kasperpeulen lukechurch

repo: dart-lang/dart-pad

license_type: permissive

license_source: github-api


text:
blast_repo fixes

Dependabot
GitHub Action

@devoncarew PTAL

repo: dart-lang/dart-services

license_source: bigquery


text:
Ignore deprecations and upgrade gradle system

Suppress deprecation warning for package:jni and upgrade the gradle system based on https://docs.flutter.dev/release/breaking-changes/flutter-gradle-plugin-apply
Closes #1026.

coverage: 92.007% (-1.7%) from 93.677%
when pulling 9a61ef5d92dc96dabf512623f398f9000d80a5c2 on ignore-deprecation
into 71ada4b5c8001e6b8207ed40331d158e0912cd94 on main.

authors: HosseinYousefi coveralls

repo: dart-lang/native


text:
Bug in reflectable_mirror_based.dart

I was running test code and got this error:
type 'String' is not a subtype of type 'Symbol' of 'name'.
  package:reflectable/src/reflectable_mirror_based.dart 1704:24  MethodMirrorImpl._isArgumentListShapeAppropriate.<fn>

All reflectable API's use Strings instead of Symbols, so I guess somewhere along the way, it is not correctly converted into a Symbol.
This is the method itself:
  bool _isArgumentListShapeAppropriate(
      int numberOfPositionalArguments, Iterable<Symbol> namedArgumentNames) {
    if (numberOfPositionalArguments <
            numberOfPositionalParameters -
                numberOfOptionalPositionalParameters ||
        numberOfPositionalArguments > numberOfPositionalParameters) {
      return false;
    }
    if (namedArgumentNames == null) return true;
    return namedArgumentNames
        .every((Symbol name) => namesOfNamedParameters.contains(name));
  }

The last line throws the error.

It is true that the reflectable API in general takes arguments of type Symbol where 'dart:mirrors' takes arguments of type String. However, the namedArguments given to various kinds of invocation (several methods named invoke and newInstance) have type Map<Symbol, dynamic>:
  ...
  Object invoke(String memberName,
                List positionalArguments,
                [Map<Symbol, dynamic> namedArguments]);
  ...
  Object newInstance(String constructorName,
                     List positionalArguments,
                     [Map<Symbol, dynamic> namedArguments]);
  ...

Presumably you have been using checked mode and passing an object of type Map<dynamic, dynamic>. That object would be able to masquerade as a Map<Symbol, dynamic> (and its keys would be an Iterable<dynamic> which could masquerade as an Iterable<Symbol>), and then you wouldn't get any checked-mode check failures before you take out one of the elements and find a String rather than a Symbol.
In general, I prefer using explicitly typed map/list literals, exactly because they won't pretend to have types that they don't have. If you do this then you'll get the checked mode failure already at invoke or newInstance. [You may know that there is an ongoing effort to introduce a 'strong mode' which is associated with inference of type arguments to some list/map literals, but it is unfortunately not in compliance with the Dart semantics to create an actual List<int> when you encounter [2, 3], so the safe bet, so far, is to give the type arguments explicitly, as in <int>[2, 3].)
The reason why we have this particular occurrence of Symbol in the API is that we did not want to modify the values passed as arguments in invocations (that's a base level concern, not a reflective one), and the set of named arguments given in an invocation can reasonably be considered to be a Map<Symbol, dynamic>. (The compiler can add extra checks for each key based on the type annotation on each named parameter, so we actually have something like a heterogeneous map here, which is an interesting concept ;-).
The reason why the Dart language per se justifies the claim that named arguments form a map whose keys are Symbols is, among other things, that Function.apply accepts a map whose keys are Symbols, and noSuchMethod receives an Invocation whose namedArguments are given as a map whose keys are Symbols. Those things do not have any ties to 'dart:mirrors' (nor reflectable, of course), and they are specified in the language specification. Hence, the names of named parameters are symbols.
So I'll close the issue.

Hmm, it seems that you are right. However, I find the use of both Symbols and Strings quite confusing. Are there plans to clear that out, or will the API be fixed the way it is now?

I agree that it is a somewhat surprising case, and we discussed it for a while before we landed it as it is today. I don't think it will be easy to change the language in this respect (in particular, Invocation, noSuchMethod, and Function.apply), simply because changes to the semantics of Dart will break so many programs.
Some people have argued that Symbol should be eliminated altogether (exactly because of this confusion about strings and symbols). You could also argue that it makes sense to have this division of labor, as in "Symbol: mainly compile-time values, canonicalized, language related; String: general purpose, dynamic".
But we certainly haven't changed our mind: it is more convenient to use strings for reflective operations than symbols. So there is no danger that we'll go back to use all symbols. ;-)

source: gharchive/issue

authors: eernstg stevenroose

repo: dart-lang/reflectable


text:
Drop /vendor BUILD files

Instead of a handcoded BUILD file which pulls in sources from and
external repo, make the external repo have the dart_library target.

Add pub_deps to the calls to pub_repository rule - this is the
information which is currently only in the BUILD files
Instead of a filegroup exporting srcs, use a dart_libary target
Replace all references to //vendor/ with @vendor_
Don't drop vendor_ from lib paths in the layout_action. We will
point. To these packages with the prefix in generate .packages files

Confirmed that the vm snapshots, like DDC html generator, can be built.

Presubmit passes locally for me - I think it's an incompatible version of bazel. Supporting old bazel version is not a priority right now. Will push and @kevmoo  will look at updating travis.


text:
List should have a method #unshift and #unshiftAll

This issue was originally filed by william.es...@gmail.com

With List#add and List#addAll commands, theres never a "inverse" command when I would like to add items to the beginning of the List besides List#insert and List#insertAll, but I have to pass an index.
I propose doing something like Ruby(s) Array#unshift command. It would be good syntactic sugar for the language.
So I can just do:
List<int> list = [4,5,6];
list.unshift(0);
=> [1,4,5,6]
Or:
List<int> list = [4,5,6];
list.unshiftAll([1,2,3]);
=> [1,2,3,4,5,6]

I beleive that can easily be addressed by doing the following:
  var colors = ["red", "green", "blue", "orange", "pink"];
  var extras = ["white", "yellow"];
  extras.addAll(colors);
  print(extras);

Feel free to test in

authors: DartBot morfinismo

repo: dart-lang/sdk


text:
Meta issue tracking implementation of generic function type syntax in Dart 1.5

This is a tracking bug for implementing support for generic function type syntax (generic typedef) in Dart 1.5.
Proposal: https://gist.github.com/eernstg/ffc7bd281974e9018f10f0cb6cfee4aa
Dart 2 supports generic methods and generic tear-offs (or anonymous functions). This means that a function can request a function that is generic:
// Takes a function `f` that is generic on T.
void foo(A<T> constructorForA<T>()) {
  A<int> x = constructorForA<int>();
  A<String> x = constructorForA<String>();
}

It is straightforward to add the syntax to the existing syntax for function types for parameters (as was done in the example). However, we don't have an easy way to do the same for fields or local variables:
// Does *not* do what we want it to do:
typedef A<T> ConstructorForAFun<T>();

A f;  // A function that returns an `A<dynamic>`.

We already used the most consistent place for the generic method argument as a template argument to the typedef itself. If we could go back in time, we could change it as follows:
typedef<T> List<T> TemplateTypedef();
TemplateTypedef<int> f;  // A function that returns a List<int>.
TemplateTypedef f;  // A function that returns a List<dynamic>.

typedef List<T> GenericTypedef<T>();
GenericTypedef f;  // A function that is generic.
List<int> f<int>();
List<String> f<String>();

Given that this would be a breaking change we have considered alternatives and are now exploring new typedef syntax altogether. The idea is to provide syntax that can also be used for locals and fields at the same time.
For example:
typedef F = (int) -> void;  // Function from (int) to void.
typedef F<T> = () -> List<T>;  // Template Typedef.
typedef F = <T>(T) -> List<T>;  // Generic function from T to List<T>.

This way we would solve 3 issues at the same time:

users often forget the parameter name and expect typedef void F(int) to be a function that takes an int.
users can write the function type of fields and locals without needing to write a typedef.
of course: we can write the type of a generic function.

While investigating different syntax we found that the following points need consideration:

nullability: the syntax must be nullable without too much hassle:  (int)->int?;  // A function that is nullable, or that returns a nullable integer?
 Problem disappears with <-
 int <- (int)? ; vs int? <- (int)

readability
interactions with union types (if we ever want them).

The corresponding individual issues will be listed here:

[x] Language team decision on syntax (private issue): dart-lang/dart-lang-evolution#88
[ ] Final specification issue: #27970
[ ] Analyzer implementation issue: #27969
[ ] VM implementation issue: #27966
[x] Dart2js implementation issue: #27967
[ ] DDC implementation issue: #27971
[ ] dartfmt issue: https://github.com/dart-lang/dart_style/issues/563
[ ] dartdoc issue: https://github.com/dart-lang/dartdoc/issues/1321
[ ] Make sure changelog has an entry for this

@crelier

Status update:
I have updated the issue description for more context.
We are still mainly exploring the "how does it look and feel" parts of the syntax (while still paying attention to how the parser should work).
The following CLs show explorations:
https://codereview.chromium.org/2439573003/
https://codereview.chromium.org/2466393002/

lgtm with a slight preference for the <- variant.

status: Prototyping several options, @floitschG is driving.

Latest proposal:
typedef F = void Function(int);

// also works for field or local types:
final int Function(String, bool) myFun = ...;

works nicely with the current Function type.
very similar to the existing parameter types. Hopefully this also means that the parser can reuse parts of the parameter code.
longer than <- or ->.

Here is a CL that experiments with it: https://codereview.chromium.org/2482923002/
Note: we would still allow Function(int x). However, Function(int) should be a function that takes int (and not a function that takes a dynamic argument of name int).
Eventually we would like to change the old syntax to do the same, but that requires some migration effort.

I do not like this latest proposal. The name Function is promoted to some kind of keyword used for parsing. Also, Function is not declared to be generic, therefore, using it to declare a non-generic typedef for a generic function will look like an error.

It's not a perfect choice, but it has its advantages.
Dart already uses Function as general type for functions. It is true that Function was a real type, and the proposed function-type syntax shifts it into a keyword-like position, but there is a strong connection between the two. It's easy to see and understand that Function (without return and parameter types) is just like a reduced function-type syntax. It's not the same (as you correctly state), but it's very close to it.
It's true that the Function type/class isn't generic, but it also has no way to declare its return and parameter types. Clearly, the function-type syntax does more than just the Function type. It really is an anonymous subtype of Function. As such, it doesn't feel wrong to add generic functionality to it.
Personally, I'm not that much concerned about the fact, that Function now becomes a keyword for function types. On the contrary, I would expect this to improve the readability, since the new syntax is so close to the existing parameter types, and the meaning of the stand-alone Function fits the new interpretation.
Function f;  // a function.
Function(int x) f;  // a function that takes an int.
double Function(int) f;  // a function that takes an int and returns a double.

I'm more concerned about the fact that it is longer, and that we intend to interpret one-identifier parameters differently in the new syntax than in parameter types in the old syntax: Function(int) is a function that takes an int and not a function that takes one dynamic argument of name int. However, I believe that the benefits outweigh the cost here. In fact, we would advocate to change the meaning of one-identifier types for old-syntax parameter types, too. This would be a breaking change, though, and would require a careful migration effort.
Just for completeness sake, here are some examples of how the syntax would deal with the different generic arguments (to the typedef or for the function).
typedef F = void Function(int);  // Function from (int) to void.
typedef F<T> = List<T> Function();  // Template Typedef.
typedef F = List<T> Function<T>(T);  // Generic function from T to List<T>.

I agree that having a special marker for function types has advantages and is easier to read. I only find the already defined "Function" a questionable choice for this marker, especially since the formal argument syntax is modified and not what you would expect after an identifier (i.e. regular function signature syntax). That's the reason I'd prefer a symbol rather than an identifier (or keyword), as in the previous proposals.
I agree that "<-" of the previous proposal is a bit hard to read when generic.
T <- <T>(T)
Maybe reusing the just retired # symbol would work better.
T #<T>(T)
As a side note, you will also need to specify the syntax for optional arguments (no default values allowed), both positional and named, something like:
void Function(int, [int, bool]);  // optional positional
void Function(int, {int i, bool b});  // optional named

# is still used for symbols (which was one of the reasons we didn't like it for tear-offs. For example, it wasn't possible to tear off globals).
Yes. We will have to specify the syntax for the optional arguments. (This is the reason that Function<R, P1, P2> isn't even a real option). Your proposal looks right (and I don't think there are any corner-cases that are hiding).
I/We will think about using something else than Function, but I doubt that we will find something better. The obvious candidates, -> and <-, have already been considered, and most other choices are probably inferieur.
Personally, I also tend to avoid using symbols for uncommon functionality. Function types are not extremely rare, but they are not common enough to warrant their own symbol.

I/We will think about using something else than Function, but I doubt that we will find something better

Would it be possible to add a new contextual keyword like function or fun or something? I'm guessing that would be a breaking change, but maybe it's not actually ambiguous in the grammar?

Personally, I also tend to avoid using symbols for uncommon functionality.

+1. Punctuation is almost always the wrong choice for new syntax unless it represents something almost all users already know, like using + for addition.

The backwards-compatible advantage in using Function  is nice, but not essential. If you can't omit the parameters, so that int Function x is any function returning an int, then the syntaxes will be completely disjoint. If you can omit the parameters, then I think we should definitely use Function as the keyword - then you can write Function Function Function x; and have something thrice callable :)
We can make Function no longer be a class. Nobody should implement Function anyway, so it's not a loss. The only complication is Function.apply which needs a namespace, but if we remove call and make function objects completely distinct from class instances, then they can have apply as an instance method.

About omitted parameter lists in the <return-type> Function<formal-parameter-list> syntax: That would allow developers to specify a return type and leave the parameter list unspecified (that is: anything goes), which is exactly what was requested in, say, https://github.com/dart-lang/sdk/issues/26420.
We might decide to introduce syntax for variadic functions at some point, in which case this will become a shorthand for "the most general argument list", but even at that point it seems likely that such a shorthand would be convenient.

In the informal spec (which will be made public when it has matured a bit) I've proposed a rename of this feature to 'Generalized type aliases' (because a limitation of this mechanism to handle generic function types only is artificial, and presumably counter productive).

Could we add some tests cases for this so that the implementations can be verified.

Marking spec complete for implementation purposes based on https://github.com/dart-lang/sdk/issues/27970#issuecomment-266673190

Moving to 1.22 for implementation.

Made the informal specification available as a public gist.

Groovy, added a link to it up top in the issue description so it's easy to find.

Based on several requests I've updated the grammar in the informal spec to make int Function(int) style types a syntax error in various places (e.g., extends, implements, with). This was already a static semantic error, but the fact that it is now also a syntax error may simplify some tasks when dealing with programs as syntax only (that is, where no static analysis is performed).

The current informal spec changes the rule for type:
type:
    typeName typeArguments? |
    functionType // NEW

but type is used in other rules in the spec, not related to the typedefs, for example:
mixinApplication: type mixins interfaces?
superclass: extends type
typeTest: isOperator type

etc. I guess you are not going to change the syntax of these constructs, are you?
//cc @eernstg

Sorry, ignore my comment, I see that you updated spec while I was writing it :)

@eernstg  Well, I think there's still something to fix. type is used also in:
finalConstVarOrType
varOrType
typeParameter
declaration
returnType
typeList
onPart
typeTest
typeCast

I also could miss a couple of cases.
Also, 'It is a compile-time error if a functionType is used as a superclass...' can now be removed I guess.

It might be easier to leave type as is and change normalParameterType and typedIdentifier instead:
normalParameterType: type | functionType | typedIdentifier
typedIdentifier: type identifier | functionType identifier

Hello Alexander, thanks for your input!
Yes, we did intend to change those other locations: A function type wasn't inline expressible until now, but we want to make them expressible, such that developers aren't forced to use a typedef to express function types in all cases, they can just go ahead and write them where they need them.
Note that typedefs introduce aliases, not new types, so there is no semantic difference between being able to give a new name to a function type and then using that name where that type should be, versus being able to write that function type directly where it is needed.
With finalConstVarOrType, varOrType, returnType you'd get new possibilities like
int Function(int) myFunction = ..., and int Function(int) foo() => ..., with typeParameter and typeList you'd get List<int Function(int)> (but not .. extends int Function(int) nor .. with MyMixin, int Function(int), because that's been defined away with the new grammar rules). If you think that's too unwieldy you can always go back to the old style and typedef. Similarly, you can use f is int Function(int) and .. as ... So that's a feature rather than a bug. ;-)
You can't remove the sentence about compile-time errors because class C extends D could still have a function type as a superclass if D is introduced by a typedef.

The number of changes in the grammar doesn't differ much if you reorganize: There are several places where we need the general type (which includes functionType) and several places where we need typeWithoutFunction.

Ok, thanks for the explanation!

Bug fix applied to the informal spec: Added missing grammar update for redirectingFactoryConstructorSignature

Won't parsers have problems with the recursion type -> functionType -> returnType -> type ->...?
type: typeWithoutFunction | functionType
functionType: returnType? 'Function' typeParameters? parameterTypeList
returnType: void | type

I don't think the return-type/type distinction is a problem to parse.
After parsing any type, you have to check if it's followed by Function< or Function(, and if so, it's a return type of a function type, otherwise it's just a type by itself. There shouldn't be any ambiguity, and you only need a very short look-ahead.

A breaking change has been detected in google3 code, the following example captures the jist of it
import 'dart:core' hide Function;
class X {
X() { print(1); }
X.f() { print(2); }
}
class Function extends X {
Function() : super();
Function.f() : super.f();
}
main() {
new Function();
new Function.f();
}
Please clarify

This is an unfortunate consequence of this change. At some point I looked through the Google3 code but didn't see any class Function that wasn't in our own code. Maybe that changed...
In any case, please let us know where this happened and we will work on fixing it. Thanks for notifying us.

Punting to 1.23 per email discussion

We reverted the syntax for 1.22 and add a deprecation warning for class Function instead.
1.23 will then feature the actual syntax.

Is this still on track for 1.23?

I can't speak for anyone else, but I am actively working on the analyzer support.

I am not sure who you are asking, but the corresponding VM issue has been closed on January 27.

@leafpetersen see detailed status in the two unchecked items in the top checklist

For dartdoc, it turns out there's no need to do anything directly for the SDK release and the work that does need to be done isn't blocking any longer.   I've dropped the 1.23 label from dart-lang/dartdoc#1321 accordingly, and there are more details there as to why.

I had misunderstood a key piece of the feature, I recommend blocking this feature on dartdoc now.  Dartdoc in the SDK won't crash without the upcoming fix, but it will display the typedef with the generic parameter missing.

dartfmt support is in flight.

dartfmt is fixed and in the repo.

@leafpetersen looks like this one is still missing in changelog.md?

I left it out, since I wasn't 100% sure that it was landing, but it looks like maybe the dartdoc issue is resolved?  If so, I can go ahead and write something up.

There seems to still be quite a bit of missing functionality still in the analyzer implementation, see discussion at the end here:  https://github.com/dart-lang/sdk/issues/27969 .

Can we go ahead and update changelog.md now?

Any updates on this?

@leafpetersen @floitschG can we please finish this for 1.24?

authors: a-siva alexander-doroshko bwilkerson crelier dgrove eernstg floitschG jcollins-g leafpetersen lrhn mit-mit munificent


text:
language: Make it an error to have two mixin applications of the same mixin

It very difficult to reason correctly about a class that mixes in the same mixin class twice along the inheritance chain. Most correct uses of the mixin require a lot of super. calls to avoid logic errors. See https://github.com/dart-lang/sdk/issues/26775 for an example of how difficult it is to simply correctly use a field of a mixin class.
I think we should make the repeated mixing-in of the same class an error. We can lift the error when a compelling use cases can be found. Most code that uses a mixin twice has unintended behaviour, e.g.  https://github.com/dart-lang/sdk/issues/28771

[ ] Make repeated mixins an error in Analyzer
[ ] Make repeated mixins an error in VM
[ ] Make repeated mixins an error in dart2js

We still don't have a clear sketch of how to implement this correctly and efficiently in JavaScript without doing a lot of work to clone code. Any work in this area will be delayed until after Kernel is working fully. JavaScript idioms for mixins (e.g. Object.assign to prototype) not handle the repeated mixin case correctly.
We should help our customers not write code that will not be correctly compiled for multiple quarters, if at all.

I'm not sure most mixins use super calls at all. They just add methods that act on other (abstract) methods. Look for example on Iterable, FixedLengthListMixin or HtmlMixin (the last one is from package:analysis_server). They just add methods, the ones in Iterable all do virtual calls to the abstract iterator getter, but not a super call.
There are some mixins that do use super calls. Those need to work, and I don't think it's that much harder to make them work, whether they are mixed in once or twice, in different class chains or in the same one. In either case, they need to work on different dynamic supertypes, and yes, that might require copying.
I think Dart mixins are fundamentally about copying the methods into a new class. Any implementation that works without copying is doing an optimization, and it's great when that works, just as any other space-saving optimization. Still, two mixin applications, of the same mixin or different ones, do work on top of different supertype chains, and their super invocations see different superclasses.
I can see that it's easier when you can just compile the mixin function once, and store its super-dependency in a single private JS member that it can access by name, and that doesn't work if you have the same method twice in the same class chain. Maybe you'll need to make the function into a closure that closes over the name it uses for its super-reference. Maybe you can find something smarter, but it should be possible.

I can follow Lasse's argument in the direction that "we can make it work", but I tend to think that we should also be aware of the the cognitive load created by the need to manage multiple copies of a given instance variable, as well as managing identical overriding method implementations (potentially causing multiple "copies" of the same side-effects on reachable state). This is a non-trivial cost that we incur from having multiple applications of the same mixin, and it also affects clients who just know that a given object has type MyMixin, but may not know that there are three of them..
One thing that usually comes up is that we may also mix in different instantiations of the same generic class (with MyMixin<int>, MyMixin<String>, directly or indirectly), but that shouldn't be much worse than implements.

From what I'm gathering, the problem is mixing in the same (private?) field more than once.
Would it be sufficient to not allow a mixin to override a private field with another? (Or public too, if necessary)
I see no reason to restrict mixins that doesn't have fields.
Also, the restriction written in this issue doesn't preclude:
class X {
   int _x = 42;
   foo() => _x;
}
class Y {
  bar() => super._x;
}
class W = X with Y, X;  // Not mixing in X twice, just mixin it in once on top of itself.

So, are we sure the requirements here are both precise and sufficient?

@rakudrama Any thoughts about Lasse's comments?
@kmillikin Do you expect that kernel will fix the implementation problems for dart2js?

Lasse has a good point that the problem I characterized as two mixin applications of the same class also occurs with mixing a class on top of itself. We should ban that too.
Public fields are no different to private fields. The mess can occur in a library or across several libraries.
Multiple mixins with fields rarely do what the programmer expects. At best they introduce inaccessible fields.
Code-only mixins would shadow an implementation with the same implementation. I'm not sure when this would be useful or how to use it in a non-obscure way. If the class is not abstract and implements the mixin's interface, what is to be gained?
At first sight you might think that  UnmodifiableListView with IterableMixin replaces the efficient list length with the iteration-based one.  Perhaps you want to avoid the some property of the specialized subtype's implemention.
However, it does not work. get iterator returns a ListIterator, this calls get length because it is assumed that Lists can do that efficiently, resulting in unbounded recursion.
My conclusion with this experiment is that for a mixin to be applied twice requires the mixin and all classes using it to be designed for that.
Mixins containing only abstract methods are interfaces and could be re-affirmed via implements.

Kernel doesn't have a ready solution to the code duplication problem for dart2js.
Currently for the VM, we have a mixin elimination transformation that introduces normal classes instead of mixin applications and copies the fields and methods from the mixin.  Then we can resolve super calls in mixins so that super calls are always direct calls.
Unfortunately this transformation is not modular --- it needs access to method bodies and field initilizers for mixin classes in other libraries and requires recompilation if those bodies change.

The reason why I prefer to ban multiple applications of the same mixin in the same class is conceptual and semantic, so it's just as bad when one copy is a superclass and another one is a mixin as it is when they are both mixins. So I agree with Stephen that they should be treated the same.
For the conceptual part, I don't think it is very helpful to be able to "be" something more than once without having a notion of roles. You can be an employee with several employers, so you have several employee roles, but if phoneNumber is a property of an employee then you'd want to say "give me your phoneNumber as an employee of that employer", not just "give me your phoneNumber".
In more concrete Dart terms, let's assume that class C has 3 copies of mixin M and you wish to invoke a specific copy of some method M.foo(). We may use the well-known workaround where we declare methods like foo1() => super.foo(); in a subclass of the first copy of M, foo2() => super.foo(); for the second copy of M, etc. But it's not a very convincing way to "be an M three times", because we can't use the interface for M, we have to use the new names foo1/2/3.
On top of that, using 3 mixins to model 3 employments is far too rigid to work well in real life, we'd want to create and cancel employments dynamically. Multiple copies of a mixin just doesn't do the job, it's much more likely to work if we use separate objects (say, instances of an Employment role class), and we can even make it look the same with forwarders like foo1() => myEmployment1.foo();.
Another potential justification could be encapsulated implementation: We could chain up the methods of the mixin such that an invocation of myC.foo() would invoke every implementation of foo because each of them calls super.foo(). In this case neither C itself nor its clients need to access each copy of M.foo individually. This could make sense if foo has side effects, because they would then occur 3 times, which means that we don't have 3 copies of M for nothing. Still, it's not likely to be hugely useful..
My impression is that the conceptual value of this feature is so low (probably negative) that it's a bad idea to invest a lot of effort into solving whatever technical difficulties it may raise.
Do you have a lot of cool use cases?

authors: eernstg kmillikin leafpetersen lrhn rakudrama


text:
UnmodifiableMapBase documentation is incomplete

See https://api.dartlang.org/stable/1.22.1/dart-collection/UnmodifiableMapBase/addAll.html, etc.
UnmodifiableMapBase API Spec reads about addAll method:

Adds all key-value pairs of other to this map. If a key of other is already in this map, its value is overwritten.

However, actually UnmodifiableMapBase objects cannot be modified and addAll method throws UnsupportedError("Cannot modify unmodifiable map").
The same is true about clear, putIfAbsent, remove methods.

Duplicate of #28917

authors: anders-sandholm iarkh


text:
Async marker on function expression need special return types in fasta

Async marker on function expression need special return types in fasta:
() sync* {} // return type is Iterable
() async {} // return type is Future
() async* {} // return type is Stream

@stereotype441 have you already implemented this for strong mode?

@peter-ahe-google Yes, strong mode support for this is fully implemented in front_end.  See the test coverage in pkg/front_end/testcases/inference/block_bodied_lambdas_*.dart.

Can we get this working without strong mode?

Sure.  Should be addressed by https://dart-review.googlesource.com/4141

Thank you, Paul!

I realize that https://github.com/dart-lang/sdk/issues/31414 has not landed yet, but once it does, this bug will be visible again (the type will be handled correctly in strong mode, but not in legacy mode)

Unassigning myself because I’m not actively working on this issue right now.

This is obsolete

authors: johnniwinther peter-ahe-google sigmundch stereotype441


text:
Have to set interface properties to null to implement

This is invalid code because Dart requires MyInterface.foo to be initialized even though its abstract.
abstract class MyInterface {
  final String foo;
}

class MyClass implements MyInterface {
  final String foo;

  MyClass(this.foo);
}

This is invalid code because MyClass doesn't have a setter for MyInterface.foo
abstract class MyInterface {
 String foo;  // remove final
}

class MyClass implements MyInterface {
  final String foo;

  MyClass(this.foo);
}

This is valid code but it is confusing and ugly.
abstract class MyInterface {
 final String foo = null;  // add back final, add set to null
}

class MyClass implements MyInterface {
  final String foo;

  MyClass(this.foo);
}

What about
abstract class MyInterface {
  String get foo;
}

That does work but it's not obvious to me that a getter can be done with a final but a final cannot be done with a getter or a final. I'll defer to others to decide if this warrants a change.
My personal opinion is that the implementation should be able to match the abstract interface exactly to improve clarity.
Glad to know that the getter works, though! :)

Sounds like https://github.com/dart-lang/language/issues/45

It is indeed dart-lang/language#45.
The problem you run into here is that a final field must be initialized, either with an initializer expression or by all generative constructors.
The final field foo in your abstract MyInterface class is not initialized, so your program has an error.
It doesn't matter that you won't ever use the class for anything other than its interface, the rule is strict.
If someone somewhere decided to extend your abstract class, they would not be able to get the field initialized.
That's why changing the field to be non-final, initializing it or changing it to a getter signature all work, the class no longer contains an uninitialized final field. Using a getter signature is the recommended solution.
We could change rules to make this easier. The largest impact change would be to change grammar for property getters/setters and allow you to write something like:
abstract class MyInterface {
  String foo { get; }
}

It's still further from an abstract field.
Another option is to reintroduce the interface keyword, which would make it clear that it is not a class that can be extended, so the field must be abstract:
interface MyInterface {
  final String foo;
}

Another option is to allow abstract on a field declaration to mark it abstract (for all other members you can omit the body, but there is no body for a field to omit).
abstract class MyInterface {
  abstract final String foo;
}

Or, maybe we can say that having a non-initialized field is only an error for concrete (non-abstract) classes. Since MyInterface is abstract, it's allowed to have a non-initialized final field, but any concrete class extending it will be an error. It makes it easier to forget initializing a field in a class intended as an abstract superclass, but I guess proper testing would catch that.
That rule would have no other reason to exist than to allow abstract classes to be used only for their interfaces, but it can then be used to prevent extending the class (which isn't new, you could already just omit having a public generative constructor).
Currently, all you can do is use the getter.

I personally like how dart has implicit interfaces. I cannot provide a good reason why, it just seems cleaner. But my understanding is that abstract classes cannot use a constructor and thus there would be no way to actually initialize a final property. For that reason I just assumed that I could write my abstract classes like a blueprint, like a pre-paste copy-paste, with finals etc and consume it as an interface without issues.
Is there a scenario I am missing where an abstract class will initialize a final property?
Cheers

Besides a personal obsession with the idea of an abstract class as an interface working like a copy-paste, using the getter does make sense and does seem reasonable. It feels quite Darty.

Abstract classes can have a constructor. You can't call it if you use it as interface though.
Otherwise you invoke it with super() in the sub class.

Whoops. Yeah, I am a little embarrassed that I missed that.

One advantage of implicit interfaces is that it has low "migration cost". If you start something out as an "interface" (a fully abstract class), and you then decide to provide a default implementation for some functions, you can just do that in the same class. You don't have to invent a new class.
Having two ways to do the same thing, one simple and limited, and one complex and powerful, can indirectly drive API designs to stay within the limited capabilities of the simple approach rather than risk the cost of having to convert everything to the more complex design.
This is one place where our enum classes fail. Dart enums are very simplistic. If you want more than that, then you have to rewrite everything to a normal class where you declare each constant manually, even if it's only to add one method to every value. The Java enum feature allows organically adding more features to an initially simple syntax without forcing a sudden and expensive migration. That is why it is better, not just because it's more powerful (it's no more powerful than full classes anyway), but because it can evolve better.

@zoechi the value of those dev tools cannot be understated. That's basically all TypeScript is, one massive autocomplete engine, and it's incredibly successful. Improving dx is important to me because it improves productivity and thus the value of the language.

authors: lrhn lukepighetti zoechi


text:
loadLibrary() return type

What is the return type of deferredPrefix.loadLibrary() when NNBD?
Currently in analyzer it is Future<dynamic>.
But this does not match CFE.
@leafpetersen @lrhn Could you clarify this?
import self as self;
import "dart:async" as asy;
import "dart:core" as core;

import "dart:math" deferred as math;

static method main() → dynamic {
  asy::Future<dynamic>? v = LoadLibrary(math);
  core::print(v);
}

See the test log  and the CL where is causes issues.

The spec is (yet) silent on the type of the future returned, and doesn't specify the static type of the loadLibrary function. We are aware of that
and have a CL to fix it: https://github.com/dart-lang/language/pull/568
The change will make the static signature Future<void> loadLibrary() and it must return a future of void, dynamic or Object? (a top type) at run-time,  and it is completed with null.

Thank you.
So, lib.loadLibrary() returns a non-nullable type (Future<void>), and the null check in isolate tests is not necessary. Probably something to fix in CFE. @a-siva

/cc @johnniwinther (regarding errors from CFE when lib.loadLibrary().then is used).

authors: a-siva lrhn scheglov


text:
Errors reported by ConstantEvaluationValidator should be made more specific

Right now most errors use the default error, "Const variables must be initialized with a constant value." (const_initialized_with_non_constant_value). Each case can be given more information though, .e.g "constructor reference used in constant expression must be a const constructor."

Yes, and many of them should have context messages associated with them. I took a look at this a couple of years ago but didn't have time for the amount of work that I thought would be necessary. Happy to discuss if you're interested.

@kallentu tackled this and improved a lot of error categories. Closing this in favor of any remaining specific issues.

authors: bwilkerson srawlins


text:
compiler/test/impact/impact_test and compiler/test/inference/inference2_test fail with records and patterns enabled

I'm working on the CL to enable the "records" and "patterns" experiments and I see failures in these two tests. I'd include the failure output here but it's huge and I'm not sure what portions are relevant.
For now, I'm going to just approve these failures because as far as I can tell, this isn't critical.

impact_test seems to be a CFE issue - I guess the change to switch/case semantics now cause it to crash when it hits case B(const bool.fromEnvironment('x') ? 0 : 1).
The inference test needs new expectations which can be generated with dart pkg/compiler/test/inference/inference_test_helper.dart -g issue_46770.dart, and we can assess the changes more easily in code review. But maybe we should just pin the test to 2.19?

Note: the impact_test error is hitting the same underlying tests as https://github.com/dart-lang/sdk/issues/51676, so we can track it there.

authors: fishythefish munificent sigmundch


text:
Failures on [co19] Roll co19 to c7f412ab8b0ad444eea4d6dbd7616b475e050fab

There are new test failures on [co19] Roll co19 to c7f412ab8b0ad444eea4d6dbd7616b475e050fab.
The tests
co19/Language/Functions/element_type_A01_t03 RuntimeError (expected Pass)
co19/Language/Functions/element_type_A01_t04 RuntimeError (expected Pass)
co19/Language/Functions/element_type_A01_t06 RuntimeError (expected Pass)
co19/Language/Functions/element_type_A02_t03 RuntimeError (expected Pass)
co19/Language/Functions/element_type_A02_t04 RuntimeError (expected Pass)
co19/Language/Functions/element_type_A02_t06 RuntimeError (expected Pass)
co19/LanguageFeatures/Extension-types/static_analysis_extension_types_A21_t01 DartkCrash (expected Pass)
co19/LanguageFeatures/Extension-types/static_analysis_extension_types_A21_t02 DartkCrash (expected Pass)
co19/LanguageFeatures/Extension-types/static_analysis_extension_types_A24_t02 DartkCrash (expected Pass)

are failing on configurations
vm-mac-release-arm64

They shouldn't. Let's wait for all bots and I'll do a triage

Good thanks! Cf. https://github.com/dart-lang/sdk/issues/53237.

Failures triaged

Failure
Issue

analyzer-asserts-linux

new test -> CompileTimeError (expected Pass)

co19/LanguageFeatures/Extension-types/static_analysis_extension_types_A24_t02
https://github.com/dart-lang/sdk/issues/53206

new test -> MissingCompileTimeError (expected Pass)

co19/Language/Functions/element_type_A01_t01
https://github.com/dart-lang/sdk/issues/53054

co19/Language/Functions/element_type_A01_t02
https://github.com/dart-lang/sdk/issues/53054

co19/Language/Functions/element_type_A02_t01
https://github.com/dart-lang/sdk/issues/53054

co19/Language/Functions/element_type_A02_t02
https://github.com/dart-lang/sdk/issues/53054

cfe...

MissingCompileTimeError -> Crash (expected Pass)

co19/LanguageFeatures/Extension-types/static_analysis_extension_types_A23_t01
https://github.com/dart-lang/sdk/issues/53212

Pass -> MissingCompileTimeError (expected Pass)

co19/LanguageFeatures/Extension-types/static_analysis_extension_types_A08_t02
https://github.com/dart-lang/sdk/issues/53207

new test -> Crash (expected Pass)

co19/LanguageFeatures/Extension-types/static_analysis_extension_types_A21_t06
https://github.com/dart-lang/sdk/issues/53232

co19/LanguageFeatures/Extension-types/static_analysis_extension_types_A21_t07
https://github.com/dart-lang/sdk/issues/53213

new test -> MissingCompileTimeError (expected Pass)

co19/LanguageFeatures/Extension-types/static_analysis_extension_types_A13_t01
https://github.com/dart-lang/sdk/issues/53172

co19/LanguageFeatures/Extension-types/superinterfaces_of_extension_type_A01_t01
https://github.com/dart-lang/sdk/issues/53170

co19/LanguageFeatures/Extension-types/superinterfaces_of_extension_type_A01_t02
https://github.com/dart-lang/sdk/issues/53170

co19/LanguageFeatures/Extension-types/superinterfaces_of_extension_type_A04_t01
https://github.com/dart-lang/sdk/issues/53170

co19/LanguageFeatures/Extension-types/superinterfaces_of_extension_type_A04_t02
https://github.com/dart-lang/sdk/issues/53170

co19/LanguageFeatures/Extension-types/superinterfaces_of_extension_type_A04_t04
https://github.com/dart-lang/sdk/issues/53170

co19/LanguageFeatures/Extension-types/superinterfaces_of_extension_type_A04_t05
https://github.com/dart-lang/sdk/issues/53170

co19/LanguageFeatures/Extension-types/superinterfaces_of_extension_type_A04_t06
https://github.com/dart-lang/sdk/issues/53170

co19/LanguageFeatures/Extension-types/superinterfaces_of_extension_type_A04_t07
https://github.com/dart-lang/sdk/issues/53170

co19/LanguageFeatures/Extension-types/superinterfaces_of_extension_type_A05_t01
https://github.com/dart-lang/sdk/issues/53172

co19/LanguageFeatures/Extension-types/superinterfaces_of_extension_type_A05_t02
https://github.com/dart-lang/sdk/issues/53172

cfe...

dart2js...

dart2wasm...

ddc...

new test -> Crash (expected Pass)

co19/LanguageFeatures/Extension-types/static_analysis_extension_types_A21_t01
https://github.com/dart-lang/sdk/issues/53140

co19/LanguageFeatures/Extension-types/static_analysis_extension_types_A21_t02
https://github.com/dart-lang/sdk/issues/53140

co19/LanguageFeatures/Extension-types/static_analysis_extension_types_A24_t02
https://github.com/dart-lang/sdk/issues/53140

cfe...

dart2js...

dart2wasm...

ddc...

vm...

new test -> CompileTimeError (expected Pass)

co19/LanguageFeatures/Extension-types/static_analysis_extension_types_A24_t01
https://github.com/dart-lang/sdk/issues/53211

co19/LanguageFeatures/Extension-types/static_analysis_member_invocation_A05_t03
https://github.com/dart-lang/sdk/issues/53211

cfe...dart2js...dart2wasm...new test -> Crash (expected Pass)

co19/LanguageFeatures/Extension-types/dynamic_semantics_extension_type_A05_t02
https://github.com/dart-lang/sdk/issues/53238

co19/LanguageFeatures/Extension-types/static_analysis_extension_types_A26_t01
https://github.com/dart-lang/sdk/issues/53232

dart2js...CompileTimeError -> RuntimeError (expected Pass)

co19/LanguageFeatures/Extension-types/dynamic_semantics_member_invocation_A02_t04
https://github.com/dart-lang/sdk/issues/53233

co19/LanguageFeatures/Extension-types/dynamic_semantics_member_invocation_A03_t04
https://github.com/dart-lang/sdk/issues/53233

dart2js...dart2wasm...ddc...vm…

new test -> RuntimeError (expected Pass)

co19/Language/Functions/element_type_A01_t03
https://github.com/dart-lang/sdk/issues/53053

co19/Language/Functions/element_type_A01_t04
https://github.com/dart-lang/sdk/issues/53053

co19/Language/Functions/element_type_A01_t06
https://github.com/dart-lang/sdk/issues/53053

dart2js...dart2wasm-linux-d8ddc...vm…

new test -> RuntimeError (expected Pass)

co19/Language/Functions/element_type_A02_t03
https://github.com/dart-lang/sdk/issues/53053

co19/Language/Functions/element_type_A02_t04
https://github.com/dart-lang/sdk/issues/53053

co19/Language/Functions/element_type_A02_t06
https://github.com/dart-lang/sdk/issues/53053

dart2wasm...new test -> Crash (expected Pass)

co19/Language/Functions/element_type_A03_t02
https://github.com/dart-lang/sdk/issues/53234

co19/LanguageFeatures/Extension-types/static_analysis_member_invocation_A05_t03
https://github.com/dart-lang/sdk/issues/53239

vm...

new test -> DartkCrash (expected Pass)

co19/LanguageFeatures/Extension-types/static_analysis_extension_types_A21_t01
https://github.com/dart-lang/sdk/issues/53140

co19/LanguageFeatures/Extension-types/static_analysis_extension_types_A21_t02
https://github.com/dart-lang/sdk/issues/53140

co19/LanguageFeatures/Extension-types/static_analysis_extension_types_A24_t02
https://github.com/dart-lang/sdk/issues/53140

authors: eernstg sgrekhov


text:
[ddc/dart2js] Compilers assume constructors that are external, only contain named args, and are extension type members are object literal constructors

This is tracking cleanup of an assumption we have in both compilers.
With interop extension types, we're allowed to write code like this:
extension type Options._(JSObject _) implements JSObject {
  external Options({int a, int b});
}

where the Options constructor is an object literal constructor. This type of constructor is special because it's the only interop member that is lowered only in the backends in order to support a more performant syntax. Since we avoid using the JS<> foreign helper in our interop transformers, they do not have a way to lower something like Options({a: 1, b: 1}) to {a: 1, b: 1}. Instead, they would have to do some combination of creating a new object and using property setters to do this. So, we do this instead in the backends.
However, determining that this constructor is an object literal constructor can be expensive. It requires indexing extension types and doing a reverse lookup to determine that this is an interop member and a constructor. This indexing is already done in the interop transformer, but that's not where we do the lowering. Re-indexing just for this one case is likely expensive. So, we instead make the assumption that any extension type member that is external and only contain named args are object literal constructors. JS interop has checks to ensure that the only JS interop external members that have named args are object literal constructors, but naturally external can be used for other purposes, so this assumption may prove false sometime in the future.
Options are:

Avoid the need for indexing by having the AST providing members on extension type members to access their extension type.
Provide some mechanism to the interop transformer to create an object literal.
Re-index extension types in the backends.

For option 1, does Member.enclosingExtensionTypeDeclaration work?
Also, for context, why do we avoid using JS in our transformers?

For option 1, does Member.enclosingExtensionTypeDeclaration work?

Ooh, I didn't know this was added. We could probably optimize some of our existing indexing to use this (maybe). As for whether we could use this in the backends to avoid the more expensive lookup, it certainly helps, but it still doesn't quite get all the way there as we determine whether something is a JS interop type based on whether it wraps a JS type recursively. We can't quite use extensionTypeErasure for that as JS types are extension types themselves, so there's still a bit of cost there.

Also, for context, why do we avoid using JS in our transformers?

It's been a while since we've had this discussion, so I don't remember much beyond us wanting to limit the scope of where it's used. Maybe @rakudrama remembers why.

We've been talking about using Component.metadata in dart2js for various reasons. Could we apply it here? The initial interop transformer can add a metadata mapping to tag relevant nodes, and we can simply read it later in the backends rather than reindexing.

Yes. We know the node in question in the interop transformer so tagging it and having the backend read it later would definitely work.

authors: fishythefish srujzs


text:
multiple duplicate tooltips

fvm flutter doctor -v
[✓] Flutter (Channel stable, 3.24.1, on macOS 15.0.1 24A348 darwin-arm64, locale ru-RU)
• Flutter version 3.24.1 on channel stable at /Users/vasilijisajkin/fvm/versions/3.24.1
• Upstream repository https://github.com/flutter/flutter.git
• Framework revision 5874a72aa4 (9 weeks ago), 2024-08-20 16:46:00 -0500
• Engine revision c9b9d5780d
• Dart version 3.5.1
• DevTools version 2.37.2
[✓] Android toolchain - develop for Android devices (Android SDK version 35.0.0)
• Android SDK at /Users/vasilijisajkin/Library/Android/sdk
• Platform android-35, build-tools 35.0.0
• ANDROID_HOME = /Users/vasilijisajkin/Library/Android/sdk
• Java binary at: /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
• Java version OpenJDK Runtime Environment (build 21.0.3+-79915917-b509.11)
• All Android licenses accepted.
[✓] Xcode - develop for iOS and macOS (Xcode 16.0)
• Xcode at /Applications/Xcode.app/Contents/Developer
• Build 16A242d
• CocoaPods version 1.14.3
[✓] Chrome - develop for the web
• Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome
[✓] Android Studio (version 2024.2)
• Android Studio at /Applications/Android Studio.app/Contents
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
• Java version OpenJDK Runtime Environment (build 21.0.3+-79915917-b509.11)
[✓] VS Code (version 1.94.1)
• VS Code at /Applications/Visual Studio Code.app/Contents
• Flutter extension version 3.94.0
[✓] Connected device (4 available)
• iPhone (василий) (mobile)       • 00008101-001D69421E41001E • ios            • iOS 17.6.1 21G93
• macOS (desktop)                 • macos                     • darwin-arm64   • macOS 15.0.1 24A348 darwin-arm64
• Mac Designed for iPad (desktop) • mac-designed-for-ipad     • darwin         • macOS 15.0.1 24A348 darwin-arm64
• Chrome (web)                    • chrome                    • web-javascript • Google Chrome 129.0.6668.101
[✓] Network resources
• All expected network resources are available.
• No issues found!

I'm unsure if this is the same thing but I think @DanTup has worked on a similar issue before. I think it was maybe over at the vscode extension issue tracker though.

Yep, this is the same as https://github.com/dart-lang/sdk/issues/56320 which was recently fixed.

authors: ChaserVasya DanTup FMorschel


text:
rev to the latest protocol spec

redux of https://github.com/dart-lang/vm_service_drivers/pull/91

merging; we can revisit the java changes


text:
Feature Request: The colored cirlces don't have a point?

Hi I'm just exploring your plugin. I have a Feature Request. What's the point of the colored circles in the groupings? Why not have the opportunity to insert the Colored Circle in front of the Emotion? Otherwise it's just a word on a line, and doesn't stand out in any way. I know you have the option to identify it as a Tag or it's own page, but I don't need either of those. My use case would be to add an Emotion at the end of a statement, as I do an Emoticon. The emoticon stands out, as an icon. I would like this word to stand out too, and the colored circle would do this. Otherwise, what's the point of having the colored circles, or Dots if you will, in the groupings (in the Options)?
Actually I was looking for a Plugin that offered Emoticons, and yours is the nearest, so it does have great potential!
Many thanks!

@Clastodon the colored circle is a color picker for underlinings of an emotion group.

authors: Clastodon dartungar

repo: dartungar/obsidian-emotion-picker

license: MIT


text:
Adding a HTML div element (toolbar, etc ..)

First of all, thank you for this great work.
I want to add Html elements on top the map. (Control Panel, toolbar, etc..)
How can I add these overlay elements?

You can just add elements inside Viewer:
<Viewer>

hogehoge

</Viewer>

repo: darwin-education/resium


text:
Sort imports based on first and third party

As far as I can tell the new sort imports feature currently sorts imports alphabetically. Other similar formatters usually try and separate imports based on if they're first or third party, meaning something (contrived) like the following
import UIKit
import Bar
import Foundation
import Foo

would be sorted into this
import Foundation
import UIKit

import Bar
import Foo

This would probably mean having to maintain a list of first party modules or is that something that can maybe be read from the system?

Should work now.

authors: dasdom kiliankoe

repo: dasdom/SwiftTools


text:
Tweaks

Minor tweaks and fixes of Netbeans java warnings.

Reviewed & merged on master

authors: dgutierr jhrcek

repo: dashbuilder/dashbuilder


text:
feat!: rename sendTransaction and applyStateTransition to broadcast

Issue being fixed or feature implemented

In order to more accurately name DAPI and DAPI gRPC endpoints we need to rename sendTransaction and applyStateTransition methods.
What was done?

sendTransaction renamed to broadcastTransaction
applyStateTransition renamed to broadcastStateTransition
dapi-grpc updated to 0.15.0-dev.1
How Has This Been Tested?

Unit tests
Breaking Changes

sendTransaction renamed to broadcastTransaction
applyStateTransition renamed to broadcastStateTransition
Checklist:

[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have added or updated relevant unit/integration/functional/e2e tests
[x] I have made corresponding changes to the documentation

For repository code-owners and collaborators only

[x] I have assigned this pull request to a milestone

It seems, we need 0.15 test suite

authors: shumkov shuplenkov

repo: dashevo/dapi


text:
V0.12.0.x Slightly refactor masternode statuses

Slightly refactor masternode statuses:

better names (MASTERNODE_INITIAL was defined twice before that)
move to proper header

Hmm ,this isn't mergable

Rebased :)

repo: dashpay/dash

license: mit


text:
Add masternode tab in qt wallet

What's inside:

show all masternodes
filter masternode list (updates every MASTERNODELIST_UPDATE_SECONDS (5) seconds)
show "my" masternodes (from masternode.conf)
start alias/all/MISSING for "my" masternodes
status of "my" mastenordes updates every MY_MASTERNODELIST_UPDATE_SECONDS (60) seconds (could be cpu-heavy for large lists)

Tab will be active only if there is at least one legit line in masternode.conf, normal users should see no difference/performance impact.

concept ACK, i love it

Thanks @UdjinM6! Assuming the list of "All Masternodes" will be very long, should we make "My Masternodes" the default (and move to the left position)?
The user would need to take action (click) on "All Masternodes" in order to initiate display of the long list which seems like better UX.

I agree, "My Masternodes" should be default tab.

nit addressed https://github.com/dashpay/dash/pull/776/commits/3764e6869ad67ff75e7928ebe06c28cba667e250#diff-8e9b29a6133dd7bf43643c37c5c4689aR94, tabs swapped and I also removed few debug printfs (no idea how they sneaked here 😄 )

Love it as well :thumbsup:
Good thing is, I wanted to implement something like this myself and fortunately didn't yet start. Time saved to break other things :sunglasses:

Looking forward for a new icon for Masternodes tab from @snogcel  😄  (will require slight code changes also)

Builds and runs as expected on both operating systems (and Windows also :smiling_imp: , tested with 8.1 and 10).
Protocol column needs some more room, and making the column width adjustable would be nice (but I can live without that):

@crowning- : agree, pushed new fixes in additional commit (will squash all commits into one and rebase if needed later).
Also fixed a small bug (event binding), done some trivial refactoring (more verbose ui element names) and added my ugly theme-dependent masternodes icon (will replace it when we there is a good alternative - @salmon 😉 ).

Was not quite happy with no way to control Masternodes tab appearance so added a semi-automatic gui option.

Icon updated, thanks @salmion!

Shouldn't conceptually the "Show Masternodes Tab" checkbox be right below the "Enable coin control features" checkbox?

@crowning- probably... wasn't sure about where to place it but yeah, I guess that will be a better place. moved :)

found and fixed a dumb bug, see 86bca13

@UdjinM6 if we'd ever meet in real life having a beer and I would tell you just HOW often I have connected wrongly attributed signals in Qt you'd have a LOT of beers when I'm finished...

This is greeeat!!! 👍 👍 👍

Merged!

Bummer, i wanted this to be squashed first :) N/m


text:
Use label for upstream build on pull requests

While fixing the upstream CI build it's easy to forget to include test-upstream in the commit message. This PR proposes we also allow the upstream label to indicate the upstream build should run on PRs. This way one doesn't need to remember to always include test-upstream in commit messages.
cc @j-bennet @charlesbluca

Noting the upstream build is running here because I've applied the label https://github.com/dask/dask/actions/runs/4069325858/jobs/7008931869

Thanks for reviewing @j-bennet @charlesbluca

repo: dask/dask


text:
S3FileSystem.find appears not to respect max_depth

What happened:
When I call find(d, maxdepth=1) on a directory with 24 subdirectories, it returns not only the 24 subdirectories but also all the files contained in all those directories.  That means, the output is:
24
2304

What you expected to happen:
Due to the maxdepth=1 parameter, I expected it would only return the 24 subdirectories.  That means, I expect the output to be:
24
24

Minimal Complete Verifiable Example:
import s3fs
fs=s3fs.S3FileSystem(anon=True)
d = "noaa-goes16/ABI-L1b-RadF/2020/045/"
print(len(fs.ls(d)))
print(len(fs.find(d, maxdepth=1)))

Anything else we need to know?:
This makes globbing in higher level directories so slow as to be unusable.  For example, a glob of "noaa-goes16/ABI-L1b-RadF/*" will call .find("noaa-goes16/ABI-L1b-RadF", maxdepth=1) which (ignoring the maxdepth parameter) will list the entire contents of the subdirectory, millions of files, and will take a very long time to complete.
Environment:

s3fs version: s3fs-0.5.1+24.gf94478e (latest master)
Python version: 3.8.5
Operating System: OpenSUSE 15.0
Install method (conda, pip, source): pip install from git master

Sorry about this, a case of choosing one optimisation over another.
https://github.com/dask/s3fs/pull/360 changed find to not be recursive, so that you can complete the request in a single call for a nested directory tree, because find is usually used to get everything. The previous version was usually slower except in the case you have before you, or a small value for depth.
We could resurrect the old code, and use it for the case that depth is small. What do you think?

I haven't followed the history so it's difficult for me to comment on the optimisation of find (and I don't understand the linked PR; from its diff it looks like the method didn't exist at all before?), but the current situation in which there exists a max_depth argument that is ignored (even without warning) seems undesirable.  I'm not personally using find directory, but rather fs.glob("noaa-goes16/ABI-L1B-RadF/*") (or a different pattern, but with a single * only), so if find becomes too complicated when maintaining two approaches for small or large values of depth, then it would work for me if glob with a single * got its listing elsewhere (such as from ls?).

Oops, I just realised that the glob method implementation is not in s3fs at all, but inherited from s3fs.AbstractFileSystem.   So it would still need to be resolved here (changing the implementation of find seems then more sensible than overloading glob).

I can confirm that in v0.5.0 the MCVE (adding withdirs=True) works as expected, but in v0.5.1 there is a problem.  Slightly longer example:
import s3fs
fs=s3fs.S3FileSystem(anon=True)
d = "noaa-goes16/ABI-L1b-RadF/2020/045/"
print(len(fs.ls(d)))
print(len(fs.find(d, maxdepth=1, withdirs=True)))
print(len(fs.glob(d + "*")))

With 0.5.0:
24
24
24

With 0.5.1:
24
2304
0

i.e. it's not just trading one optimisation against another, I think we can call this broken.

I would appreciate if you would for this into a (failing) test function with expectations in a PR, so that a fix is easier to make.

Note that with current master, the last result is also 24, so something got fixed along the line.

See https://github.com/dask/s3fs/pull/379 for a unit test exposing the problem.  I need to focus on something else right now.

authors: gerritholl martindurant


text:
Please rev the major version instead of the minor version for new major versions of keycloak

Description
Please rev the major version instead of the minor version for new major versions of keycloak.
Teams that rely on automated dependency management tools (dependabot, etc) most often set these tools to update by minor version since the majority of projects follow Semver versioning schemes.
The current versioning scheme is incompatible with Semver leading teams to perform workarounds for testcontainers-keycloak.
Motivation
No response
Details
No response

You can mitigate version conflicts with setting an explicit Keycloak image when creating the container, e.g. new KeycloakContainer("quay.io/keycloak/keycloak:18.0.0");
This way, the testcontainer doesn't use the most recent default image, like specified in the code.
Additionally, when using the keycloak-admin-client dependency, you can exclude the default one and add the desired version to your dependency list.

That only works if the commands/flags in KeycloakContainer stay the same between versions.
To me it seems safer and more user friendly to keep MAJOR and MINOR versions incrementing with keycloak's.

I decided against aligning the testcontainers-keycloak extension with Keycloak versioning. I'd would clash if we need a patch/minor update of this lib and shortly after this, Keycloak would publish a patch or minor release.
But FWIW, I made the method getKeycloakDefaultVersion() public, as mentioned by @markbanierink

repo: dasniko/testcontainers-keycloak


text:
Ds sharing consumptionrole s3 policy

Feature or Bugfix

Bugfix

Detail
This PR will address the issue - 1226
Relates
Security
Please answer the questions below briefly where applicable, or write N/A. Based on
OWASP 10.

Does this PR introduce or modify any input fields or queries - this includes
fetching data from storage outside the application (e.g. a database, an S3 bucket)? N/A

Is the input sanitized? N/A
What precautions are you taking before deserializing the data you consume? N/A
Is injection prevented by parametrizing queries? N/A
Have you ensured no eval or similar functions are used? N/A

Does this PR introduce any functionality or component that requires authorization? N/A

How have you ensured it respects the existing AuthN/AuthZ mechanisms? N/A
Are you logging failed auth attempts? N/A

Are you using or adding any cryptographic features? N/A

Do you use a standard proven implementations? N/A
Are the used keys controlled by the customer? Where are they stored? N/A

Are you introducing any new policies/roles/users? No

Have you used the least-privilege principle? How?Yes, restricted the s3:* permissions to s3 readonly permissions

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

This is an extension of https://github.com/data-dot-all/dataall/pull/1262 to add s3:GetObject permissions and remove s3:* permissions

@mourya-33 while teting I found the following issues:

We still have kms* policy
We have s3* for share point access. May be we should revisit it as well? Now sure
You corrected bucket policy, but there is still s3* in iam policy (which is overruled by resource policy, but still not good). This is granted in add_missing_resources_to_policy_statement and related both to kms and s3 policies. Please, fix it as well

@SofiaSazonova , this also seem to require a change in the test assertions. I am testing this locally and will push the changes early next week.

@mourya-33 Sorry for not writing all at once, but I just realised:
we really should be changing the existing policies as well to patch the problem. Could you write migration script for that?

@dlpzx @noah-paige I need your advice here: is it ok to put the backfilling of iam policies in migration scripts?

@mourya-33 please, resolve the conflicts and I will merge this PR. Migrations will be done in separate PR, because it will take more time.

repo: data-dot-all/dataall


text:
maxRows not working for me as resolved in issue #86

I must be missing something, because I have the same issue and I just tried setting maxRows on executeStatement and neither this maxRows nor the maxRows on fetchChunk appear to be having an effect. The table has over 800k rows and I aborted the test after several iterations. The log statement inside the do loop is never executed. Below is the full test and the console output
const { DBSQLClient } = require('@databricks/sql')

const connectOptions = {
        token: process.env.DATABRICKS_TOKEN,
        host: process.env.DATABRICKS_SERVER_HOSTNAME,
        path: process.env.DATABRICKS_HTTP_PATH
}

if (!connectOptions.token || !connectOptions.host || !connectOptions.path) {
  throw new Error("Cannot find Server Hostname, HTTP Path, or personal access token. " +
                  "Check the environment variables DATABRICKS_TOKEN, " +
                  "DATABRICKS_SERVER_HOSTNAME, and DATABRICKS_HTTP_PATH.")
}

const client = new DBSQLClient()
console.log('client created')

async function poc() {
  await client.connect(connectOptions).then(async client => {
    console.log('client connected')
    const session = await new Promise((resolve,reject) => {
      client.on("error", reject).openSession().then(resolve)
    })

    console.log('session established')
    // statement = 'SELECT * FROM default.diamonds LIMIT 2',
    let statement = `select * from star_sandbox_sisense.dim_document`
    let options   = {
      runAsync: true,
      maxRows: 100 // This option enables the direct results feature.
    }
    const queryOperation = await session.executeStatement(statement,options)
    console.log('statement executed')

    let result = []
    do {
        result = await queryOperation.fetchChunk({maxRows: 10})
        console.log(`result has ${result.length} rows`)
    } while (await queryOperation.hasMoreRows())

    await queryOperation.close()
    await session.close()
    await client.close()

    console.log('POC completed')
  }).catch((error) => {
    console.log(error)
  })
}

poc()

{"level":"info","message":"Created DBSQLClient"}
client created
client connected
BaseCommand. client.output = defaultReadBufferSize,writeBufferSize,inBuf,readCursor,writeCursor,outBuffers,outCount,onFlush,client,_seqid
session established
BaseCommand. client.output = defaultReadBufferSize,writeBufferSize,inBuf,readCursor,writeCursor,outBuffers,outCount,onFlush,client,_seqid
statement executed
BaseCommand. client.output = defaultReadBufferSize,writeBufferSize,inBuf,readCursor,writeCursor,outBuffers,outCount,onFlush,client,_seqid
BaseCommand. client.output = defaultReadBufferSize,writeBufferSize,inBuf,readCursor,writeCursor,outBuffers,outCount,onFlush,client,_seqid
BaseCommand. client.output = defaultReadBufferSize,writeBufferSize,inBuf,readCursor,writeCursor,outBuffers,outCount,onFlush,client,_seqid
BaseCommand. client.output = defaultReadBufferSize,writeBufferSize,inBuf,readCursor,writeCursor,outBuffers,outCount,onFlush,client,_seqid
result has 3630 rows
BaseCommand. client.output = defaultReadBufferSize,writeBufferSize,inBuf,readCursor,writeCursor,outBuffers,outCount,onFlush,client,_seqid
result has 3616 rows
BaseCommand. client.output = defaultReadBufferSize,writeBufferSize,inBuf,readCursor,writeCursor,outBuffers,outCount,onFlush,client,_seqid
result has 4046 rows
BaseCommand. client.output = defaultReadBufferSize,writeBufferSize,inBuf,readCursor,writeCursor,outBuffers,outCount,onFlush,client,_seqid
result has 4092 rows
BaseCommand. client.output = defaultReadBufferSize,writeBufferSize,inBuf,readCursor,writeCursor,outBuffers,outCount,onFlush,client,_seqid
result has 4085 rows
BaseCommand. client.output = defaultReadBufferSize,writeBufferSize,inBuf,readCursor,writeCursor,outBuffers,outCount,onFlush,client,_seqid
result has 4064 rows
BaseCommand. client.output = defaultReadBufferSize,writeBufferSize,inBuf,readCursor,writeCursor,outBuffers,outCount,onFlush,client,_seqid
result has 4096 rows
BaseCommand. client.output = defaultReadBufferSize,writeBufferSize,inBuf,readCursor,writeCursor,outBuffers,outCount,onFlush,client,_seqid
result has 4076 rows
BaseCommand. client.output = defaultReadBufferSize,writeBufferSize,inBuf,readCursor,writeCursor,outBuffers,outCount,onFlush,client,_seqid
result has 4096 rows
BaseCommand. client.output = defaultReadBufferSize,writeBufferSize,inBuf,readCursor,writeCursor,outBuffers,outCount,onFlush,client,_seqid
result has 4033 rows
BaseCommand. client.output = defaultReadBufferSize,writeBufferSize,inBuf,readCursor,writeCursor,outBuffers,outCount,onFlush,client,_seqid
result has 4096 rows
BaseCommand. client.output = defaultReadBufferSize,writeBufferSize,inBuf,readCursor,writeCursor,outBuffers,outCount,onFlush,client,_seqid
result has 4096 rows
BaseCommand. client.output = defaultReadBufferSize,writeBufferSize,inBuf,readCursor,writeCursor,outBuffers,outCount,onFlush,client,_seqid
result has 4096 rows
BaseCommand. client.output = defaultReadBufferSize,writeBufferSize,inBuf,readCursor,writeCursor,outBuffers,outCount,onFlush,client,_seqid
result has 3867 rows
BaseCommand. client.output = defaultReadBufferSize,writeBufferSize,inBuf,readCursor,writeCursor,outBuffers,outCount,onFlush,client,_seqid
result has 3758 rows
BaseCommand. client.output = defaultReadBufferSize,writeBufferSize,inBuf,readCursor,writeCursor,outBuffers,outCount,onFlush,client,_seqid
result has 4074 rows
BaseCommand. client.output = defaultReadBufferSize,writeBufferSize,inBuf,readCursor,writeCursor,outBuffers,outCount,onFlush,client,_seqid
result has 4081 rows
BaseCommand. client.output = defaultReadBufferSize,writeBufferSize,inBuf,readCursor,writeCursor,outBuffers,outCount,onFlush,client,_seqid
result has 4084 rows
BaseCommand. client.output = defaultReadBufferSize,writeBufferSize,inBuf,readCursor,writeCursor,outBuffers,outCount,onFlush,client,_seqid
result has 4078 rows
BaseCommand. client.output = defaultReadBufferSize,writeBufferSize,inBuf,readCursor,writeCursor,outBuffers,outCount,onFlush,client,_seqid
result has 4072 rows
BaseCommand. client.output = defaultReadBufferSize,writeBufferSize,inBuf,readCursor,writeCursor,outBuffers,outCount,onFlush,client,_seqid
result has 4096 rows
BaseCommand. client.output = defaultReadBufferSize,writeBufferSize,inBuf,readCursor,writeCursor,outBuffers,outCount,onFlush,client,_seqid
result has 4096 rows
BaseCommand. client.output = defaultReadBufferSize,writeBufferSize,inBuf,readCursor,writeCursor,outBuffers,outCount,onFlush,client,_seqid
result has 3716 rows
BaseCommand. client.output = defaultReadBufferSize,writeBufferSize,inBuf,readCursor,writeCursor,outBuffers,outCount,onFlush,client,_seqid
result has 4028 rows
BaseCommand. client.output = defaultReadBufferSize,writeBufferSize,inBuf,readCursor,writeCursor,outBuffers,outCount,onFlush,client,_seqid
result has 4084 rows
BaseCommand. client.output = defaultReadBufferSize,writeBufferSize,inBuf,readCursor,writeCursor,outBuffers,outCount,onFlush,client,_seqid
result has 4096 rows
BaseCommand. client.output = defaultReadBufferSize,writeBufferSize,inBuf,readCursor,writeCursor,outBuffers,outCount,onFlush,client,_seqid
result has 4076 rows
BaseCommand. client.output = defaultReadBufferSize,writeBufferSize,inBuf,readCursor,writeCursor,outBuffers,outCount,onFlush,client,_seqid
result has 3668 rows
BaseCommand. client.output = defaultReadBufferSize,writeBufferSize,inBuf,readCursor,writeCursor,outBuffers,outCount,onFlush,client,_seqid
result has 4010 rows
BaseCommand. client.output = defaultReadBufferSize,writeBufferSize,inBuf,readCursor,writeCursor,outBuffers,outCount,onFlush,client,_seqid
result has 4096 rows
BaseCommand. client.output = defaultReadBufferSize,writeBufferSize,inBuf,readCursor,writeCursor,outBuffers,outCount,onFlush,client,_seqid
result has 4083 rows
BaseCommand. client.output = defaultReadBufferSize,writeBufferSize,inBuf,readCursor,writeCursor,outBuffers,outCount,onFlush,client,_seqid
result has 4096 rows
^C
[ec2-user@ip-172-31-57-122 mc]$

Originally posted by @edtguy in https://github.com/databricks/databricks-sql-nodejs/issues/86#issuecomment-1646584919

Hi @edtguy! First of all, I have to say that maxRows specifies rather a desired amount of rows. But server may return different count (less or in some cases more). After some discussion we decided to keep this behavior as is.
Second, when session.executeStatement returns it doesn't mean that your query was executed, but that it was enqueued. In some cases of course query may be already executed when session.executeStatement returns, but generally that's not true. Methods like fetchAll or fetchChunk later do wait for query being in fact executed so data can be fetched. In other words, your first call to fetchChunk most likely will take long because it will be waiting for warehouse to start and execute your query. Also, you may explicitly call await operation.finished() - it will just wait for query to execute and return onceit's done and results are awailable. Considering you're trying to fetch 800k rows, query execution may not be fast (I don't state it, but that's totally possible).
And the last thing to mention. I checked your logs. Log format is not familiar to me, it has to be something custom, but I definitely can see lines like result has 4083 rows. Which for mee look like your loop is actually executing, and it has to do 800k / 4000 = approx. 200 iterations before it fetches all the data. So I want to ask you to double check if there actually is any problem there. Thank you!

Hi @kravets-levko,
Thank you for your prompt and informative reply. You are correct that the log statements are executing in the output I copied, although the "Add Code" formatting option did not retain the line breaks. In my first run of the test script they were not output. I added more loggining statements, then ran it again. That time it did execute those statements within the loop, but I forgot to alter my comments. My apologies.
I'm not sure I understand  the purpose of maxRows if it is only a suggestion. My test script was based on example code and when I successfully connected to the target Databricks instance, I chose a table at random for my initial test. When I executed fetchAll, it caused node to abend with an out of memory exception. That's when I began experimenting with maxRows.
Is there more detailed documentation on best practices and suggested settings for parameters like maxRows when working with large data sets? Also for all the possible methods such as finished() or hasMoreRows(). I only discovered hasMoreRows() from issue 86 and finished() from your reply. Also, in the original example I had started from, the options object for fetchAll contained progress and callback. Is there documentation of the options for both fetchAll and fetchChunk?
All additional detail is most appreciated.
Regards,
Ed

Sorry for delayed reply @edtguy!
Our docs are not perfect, that's true. Currently we're working on improving them, and feedback like yours is very valuable to us.
When you need to fetch a large dataset - fetchAll may be not the best choise. Internally, it's just a fetchChunk called in a loop, all the data is collected into a single array and then returned. fetchAll mostly just passes its options to underlying fetchChunk. Obviously, if there are too much data - no matter if it will be fetched in smaller or bigger chunks. It will be fetched entirely anyway, and may cause OOM if dataset is large.
So your decision to use fetchChunk was totally correct. As for maxRows and why it works as it is - fetchChunk just sends this value to server, and server uses it somehow to decide how much records to return. But the library has no control over that decision, and when it receives the chunk - it just returns it. Yes, it's possible to implement it so operation would internally maintain some buffer, and prefetch enough data to later return chunks of exact size. But it would lead to unnecessary memory overhead (with a number of edge cases), so we decided to just return chunk as is and allow user to decide how to process them.
callback option is used only if you tried to fetch data but operation is not finished yet. In this case, the called method will enter a wait loop polling for operation status, and calling that callback on each iteration. The callback takes a single argument - the operation status on the current iteration (similar to what operation.status() returns). progress = true will ask server to provide extended status data (but not guaranteed that server would return it, so always check the actual status fields)

Thank you @kravets-levko, your reply was very helpful.

Hi @edtguy! We just released v1.7.0 which contains a fix to maxRows behavior (basically, it should now work as one would expect - fetch chunks of specified size). I want to ask you to try it and share your feedback. Thank you!

I'm closing this issue as it should be fixed now. Please feel free to re-open if needed

authors: edtguy kravets-levko

repo: databricks/databricks-sql-nodejs