text:
Analysis Server crash: UnimplementedError: GenericFunctionTypeImpl

From DartPad v1.23.0
AnalysisException: Cannot compute DART_ERRORS for StringSource (/tmp/analyzertbIDHG/main.dart)
Caused by Unexpected exception while performing VerifyUnitTask for source /tmp/analyzertbIDHG/main.dart
#0      AnalysisTask._safelyPerform (package:analyzer/task/model.dart:333:7)
#1      AnalysisTask.perform (package:analyzer/task/model.dart:220:7)
#2      AnalysisDriver.performWorkItem (package:analyzer/src/task/driver.dart:284:10)
#3      AnalysisDriver.performAnalysisTask (package:analyzer/src/task/driver.dart:255:11)
#4      AnalysisContextImpl.performAnalysisTask.<anonymous closure> (package:analyzer/src/context/context.dart:1199:27)
#5      _PerformanceTagImpl.makeCurrentWhile (package:analyzer/src/generated/utilities_general.dart:189:15)
#6      AnalysisContextImpl.performAnalysisTask (package:analyzer/src/context/context.dart:1197:50)
#7      Analyzer._ensureAnalysisDone (package:services/src/analyzer.dart:204:24)
#8      Analyzer.analyzeMulti (package:services/src/analyzer.dart:138:7)
#9      Analyzer.analyze (package:services/src/analyzer.dart:118:12)
#10     testAnalysis (file:///home/lukechurch/dart-services/tool/fuzz_driver.dart:251:41)
<asynchronous suspension>
#11     testPath (file:///home/lukechurch/dart-services/tool/fuzz_driver.dart:190:39)
<asynchronous suspension>
#12     main (file:///home/lukechurch/dart-services/tool/fuzz_driver.dart:109:13)
<asynchronous suspension>
#13     _startIsolate.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:263)
#14     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:151)

Caused by UnimplementedError: GenericFunctionTypeImpl
#0      ErrorVerifier._checkForNotInstantiatedBound (package:analyzer/src/generated/error_verifier.dart:5343:7)
#1      ErrorVerifier.visitTypeParameter (package:analyzer/src/generated/error_verifier.dart:1227:5)
#2      TypeParameterImpl.accept (package:analyzer/src/dart/ast/ast.dart:10834:15)
#3      NodeListImpl.accept (package:analyzer/src/dart/ast/ast.dart:8052:20)
#4      TypeParameterListImpl.visitChildren (package:analyzer/src/dart/ast/ast.dart:10895:21)
#5      RecursiveAstVisitor.visitTypeParameterList (package:analyzer/dart/ast/visitor.dart:1218:10)
#6      ErrorVerifier.visitTypeParameterList (package:analyzer/src/generated/error_verifier.dart:1234:40)
#7      TypeParameterListImpl.accept (package:analyzer/src/dart/ast/ast.dart:10891:15)
#8      FunctionExpressionImpl.visitChildren (package:analyzer/src/dart/ast/ast.dart:5358:22)
#9      RecursiveAstVisitor.visitFunctionExpression (package:analyzer/dart/ast/visitor.dart:869:10)
#10     ErrorVerifier.visitFunctionExpression (package:analyzer/src/generated/error_verifier.dart:814:43)
#11     FunctionExpressionImpl.accept (package:analyzer/src/dart/ast/ast.dart:5354:15)
#12     FunctionDeclarationImpl.visitChildren (package:analyzer/src/dart/ast/ast.dart:5204:26)
#13     RecursiveAstVisitor.visitFunctionDeclaration (package:analyzer/dart/ast/visitor.dart:857:10)
#14     ErrorVerifier.visitFunctionDeclaration (package:analyzer/src/generated/error_verifier.dart:795:44)
#15     FunctionDeclarationImpl.accept (package:analyzer/src/dart/ast/ast.dart:5197:15)
#16     NodeListImpl.accept (package:analyzer/src/dart/ast/ast.dart:8052:20)
#17     CompilationUnitImpl.visitChildren (package:analyzer/src/dart/ast/ast.dart:2523:21)
#18     RecursiveAstVisitor.visitCompilationUnit (package:analyzer/dart/ast/visitor.dart:707:10)
#19     ErrorVerifier.visitCompilationUnit (package:analyzer/src/generated/error_verifier.dart:569:38)
#20     CompilationUnitImpl.accept (package:analyzer/src/dart/ast/ast.dart:2516:15)
#21     VerifyUnitTask.internalPerform (package:analyzer/src/task/dart.dart:6335:10)
#22     AnalysisTask._safelyPerform (package:analyzer/task/model.dart:321:9)
#23     AnalysisTask.perform (package:analyzer/task/model.dart:220:7)
#24     AnalysisDriver.performWorkItem (package:analyzer/src/task/driver.dart:284:10)
#25     AnalysisDriver.performAnalysisTask (package:analyzer/src/task/driver.dart:255:11)
#26     AnalysisContextImpl.performAnalysisTask.<anonymous closure> (package:analyzer/src/context/context.dart:1199:27)
#27     _PerformanceTagImpl.makeCurrentWhile (package:analyzer/src/generated/utilities_general.dart:189:15)
#28     AnalysisContextImpl.performAnalysisTask (package:analyzer/src/context/context.dart:1197:50)
#29     Analyzer._ensureAnalysisDone (package:services/src/analyzer.dart:204:24)
#30     Analyzer.analyzeMulti (package:services/src/analyzer.dart:138:7)
#31     Analyzer.analyze (package:services/src/analyzer.dart:118:12)
#32     testAnalysis (file:///home/lukechurch/dart-services/tool/fuzz_driver.dart:251:41)
<asynchronous suspension>
#33     testPath (file:///home/lukechurch/dart-services/tool/fuzz_driver.dart:190:39)
<asynchronous suspension>
#34     main (file:///home/lukechurch/dart-services/tool/fuzz_driver.dart:109:13)
<asynchronous suspension>
#35     _startIsolate.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:263)
#36     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:151)

#0      AnalysisContextImpl.computeResult (package:analyzer/src/context/context.dart:737:7)
#1      AnalysisContextImpl.computeErrors (package:analyzer/src/context/context.dart:674:12)
#2      Analyzer.analyzeMulti.<anonymous closure> (package:services/src/analyzer.dart:142:18)
#3      List.forEach (dart:core-patch/growable_array.dart:256)
#4      Analyzer.analyzeMulti (package:services/src/analyzer.dart:141:19)
#5      Analyzer.analyze (package:services/src/analyzer.dart:118:12)
#6      testAnalysis (file:///home/lukechurch/dart-services/tool/fuzz_driver.dart:251:41)
<asynchronous suspension>
#7      testPath (file:///home/lukechurch/dart-services/tool/fuzz_driver.dart:190:39)
<asynchronous suspension>
#8      main (file:///home/lukechurch/dart-services/tool/fuzz_driver.dart:109:13)
<asynchronous suspension>
#9      _startIsolate.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:263)
#10     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:151)

The current location of the exception is (package:analyzer/src/generated/error_verifier.dart:5316), and I expect that it's being caused by code similar to
class C <T extends int Function(int)> {}

Going to close this; couldn't repro.
Tried variations of the following:
class C<T extends int Function(int)> {}

main() {
  new C<int Function(int Function(int))>();
}

Also, the method which is throwing not implemented was removed a few months ago, likely during the course of implementing generic function types or proper strong mode instantiation to bounds.

authors: MichaelRFairhurst bwilkerson lukechurch


text:
Unexpected commas on the end of argument completion for GestureDetector

Bit a strange one... I had an issue raised where the completion items where showing trailing commas:

I can repro, but when I try to reproduce it on my own class (which as far as I can tell is effectively the same as the GestureDetector defintion in flutter/flutter) I do not see it:

The comma seems to be included in the completion result from the analyzer:

This raises two questions:

Should the comma be there? (I think it looks weird)
Why does it appear to behave differently for these two similar (as far as I can tell) classes?

The source for GestureDetector is here and my version is like this:
class Danny {
  Danny({
    int key,
    this.onTap,
    this.excludeFromSemantics: false
  }) {}
  final DannyCallback onTap;
  final bool excludeFromSemantics;
}

typedef void DannyCallback(Object details);

@pq

Right.  This is scoped to Widget subclasses which explains why you don't always see it.  As for motivation, see https://github.com/flutter/flutter-intellij/issues/551 for some background.   Sorry for the confusion!

This feels really wonky for me for a few reasons:

It looks bad in the completion list
It's inconsistent - why does it only apply to these classes?
It seems more complicated to me having the cursor inserted because after typing the argument value I know how to move the cursor past the comma (it's more awkward to use the cursor keys while typing than it is to type the comma)

After reading that thread I'm still not really clear on the motivation - what's the advantage to having the comma inserted?

The comma is a (imo, really ugly) way of telling the formatter to use a different formatting style. It is the formatting style preferred for Flutter development in which each argument is on a separate line, even when some arguments could be on the same line.

Oh, I understand the original case now, I was confused about the mention of commas and also indenting.
I agree this is an ugly way to achieve this... if it's confusing to @sethladd it's only going to be more confusing for people outside Google! The experience is weird (with the comma inserted, it's now more effort to move on to the next argument than if the comma was not inserted) and it also seems bizzare to be formatting code differently just because it's a subclass of Widget! I won't turn this into a complaint about dartfmt but it kinda feels like another thing it doesn't handle how people would like :-(
I'm happy to strip the comma off in Dart Code if you don't want to change it (I really don't want to ship this for all the reasons above; plus it's already cost me a bug report and some time and it's not even out the door yet!) but I really think it could be handled better centrally (eg. can't dartfmt handle this better (in a non-flutter-specific way) and then the IDE can just format as the user is typing to fix indenting?).

@munificent For the formatter questions.

it also seems bizzare to be formatting code differently just because it's a subclass of Widget! I

The formatter doesn't know it's a subclass of Widget. It only knows syntax. It has different rules for formatting argument lists that contain trailing commas and those that don't. The logic for argument lists without a trailing comma is the original logic. It's how all Dart users inside Google formatted their code, as well as most other users. It was designed to be similar to the official styles used by Google for JS and Java, and similar to how users at the time were hand-formatting their code. This style puts the closing ) right after the final argument, even if it splits:
function(argument, argument,
    argument); // <--

The Flutter team has their own preferred formatting style. They requested that the language team add support for trailing commas in argument lists. The formatter's normal formatting rules don't make a lot of sense with that. The only point in having a trailing comma is that it makes it easier to reorganize arguments when each is on its own line — you can cut and paste the entire line. That benefit is lost if the ) is put on the same line as in:
function(argument, argument,
    argument,); // <--

So, grudgingly, I added separate formatting logic that kicks in when the argument list has a trailing comma. Now you basically get to pick your preferred formatting style on a case by case basis by deciding whether or not to put a trailing comma in your argument list. This undermines the entire point of dartfmt — having a single consistent style and free users from thinking about formatting — but dartfmt had to do something with those trailing commas since they are now valid syntax.

I guess it's not really what dartfmt is doing here that I object to (though it does frustrate me that the number of places in which it doesn't meet its goal is growing, yet it still isn't configurable), it's the comma-insertion.

Now you basically get to pick your preferred formatting style on a case by case basis by deciding whether or not to put a trailing comma in your argument list.

This doesn't seem completely bad, but in this case it feels like the formatting is being decided for us by the analyzer with the comma and the rules for it are not logical - I might prefer this formatting for my own classes, but I have to do it manually. It just seems weird that the analyzer has special behaviour for Widgets, Additionally, it feels like it's become harder to write code this way - when the comma is inserted it's now much clumsier to insert the next argument because you need to jump over the comma (which is more awkward than just typing the comma no matter whether you do it with cursor keys, tabstops, whatever), and because it's inconsistent the user needs to pay attention to this to not insert additional commas when typing quickly (I'm guessing IDEs don't "swallow" the comma if you type one with the cursor in front of it, but maybe they do).
I don't feel too strongly about changing the analyzer, but I probably will strip them out client-side to avoid more confusion. I also wonder if dartfmt could detect when a user appears to have formatted the args over multiple lines and preserve them, trailing comma or not? I appreciate that's both difficult and yet another thing going against the goal of a consistent style (though IMO I think it's kinda failing to deliver that anyway and it may be time to give up on that bit ;-))

@lukechurch Do we have any evidence to show whether this is a UX win or loss?

For the people that do like this behaviour, I'm curious - does your IDE do anything special to handle the comma (like overtyping it if you press a comma to avoid you having to use cursor keys to skip over it after typing a value)? And if not, are you happy with it or is this an annoyance you're living with in order to get the better formatting?

Ok, so I just noticed that when there's already a comma after the cursor, the completion does not include another one. This partially addresses my concern - it's fine for you to type another comma and then an argument, because you don't end up with multiple commas building up after the cursor.

So I guess my issues are just (and these are somewhat minor):

It's inconsistent; it's not clear to the user why it sometimes happens (Widget sub-classes?) and sometimes does not
The display of the autocomplete looks wonky when commas are included (see Seth's original issue in Dart Code - when I saw his screenshot I thought I'd messed something up)

I could fix (2) by stripping the trailing comma from the display value (but leaving it for the completion), though it would be inconsistent with other IDEs and I'm not sure how users would feel when the completion doesn't (quite) match up with what they see. It's already the case for snippets so maybe that's not a big deal; though I'm interesting in other opinions.
I can't fix (1) though; this is a decision for you guys. It feels wonky to me but I'm currently writing the least Dart of anyone here so my opinion is based on very little real experience. I don't want to keep bashing on dartfmt (even I'm getting bored of hearing me!) but it does seem to me like this issue is a result of it not allowing people to format arguments as they'd really like - maybe there's some better way of deciding to format this way (that is more easily explainable than "is a widget class") and then comma wouldn't be needed?

I also wonder if dartfmt could detect when a user appears to have formatted the args over multiple lines and preserve them, trailing comma or not?

That defeats the point of dartfmt. If you want a formatter that preserves your hand-formatting, cat works pretty well. :)

though IMO I think it's kinda failing to deliver that anyway and it may be time to give up on that bit ;-)

I think it's doing that better than might be apparent. I review tons of random Dart code internally, and it's a huge relief to never ever ever have to make comments about the formatting style.

On vacation so a bit late to the party...  Thanks for the thoughtful conversation!
@DanTup: you make some great points and to be clear these changes were settled on after a fair amount deliberation but not without some reservations.  That said, I still think it's not a terrible compromise assuming Flutter folks find the trailing comma generation valuable and that in practice the inconsistency isn't otherwise confusing.  My sense is that it's not that likely to be but I expect we'll get more feedback from @InMatrix and @lukechurch as they continue to study users.
Cheers everyone for the thoughtful back and forth! 🍻

@DanTup @pq Should we leave this open until we hear results from the user studies, or should we close this and open a new issue if the user studies reveal a problem?

I'm inclined to close and follow-up with an issue driven by user feedback but am open either way.

Now I understand it, I've no objection to closing this. I am interested in @sethladd's opinion on my question here on the Dart Code issue though. I'm tempted to hide the comma from the completion list (if it looked like a bug to both Seth and I, I presume many other users will think so too).

FWIW - due to no real opinions, I made a call on how to behave here in Dart Code.. I've trimmed the commas from the display text so everything looks normal:

But when you insert, you still get the original text including the comma (and the placeholder I injected):

I haven't used it much, but in a quick test this looks and feels really natural :-)

source: gharchive/issue

authors: DanTup bwilkerson munificent pq

repo: dart-lang/sdk

license_type: permissive

license_source: github-api


text:
case doesn't understand that both if and else end in return

switch (foo) {
  case 1:
    if (bar) {
      return 2;
    } else {
      return 3;
    }
}

The analyzer gives the following error for this code:
The last statement of the 'case' should be 'break', 'continue', 'rethrow', 'return' or 'throw'. #case_block_not_terminated
But I would argue that this case does have return as its last statement in any way that matters.

I use whatever blaze uses on google3.

Sorry, apparently you need a case after the offending case.
This fails on dartpad, although with a different error (Error: Switch case may fall through to the next case.)
int main()  {
  int foo = 1;
  bool bar = true;
  switch (foo) {
    case 1:
      if (bar) {
        return 2;
      } else {
        return 3;
      }
    case 2:
      return 4;
  }
}

Ok, now I can reproduce.

This is behaving as defined in the specification:

Let s be the last statement of the statement sequence sk. If s is a non-empty block statement, let s instead be the last statement of the block statement. It is a compile-time error if s is not a break, continue, rethrow or return statement or an expression statement where the expression is a throw expression.

It's working as specified. It's still highly annoying and I hope we can fix it at some point.

Duplicate of https://github.com/dart-lang/sdk/issues/28042

authors: bwilkerson dskloetg lrhn srawlins zoechi


text:
Default value of type Set in mock causes crash

With Analyzer using summary2 we support sets as constant values, and in google3 we have code {Set<String> fields = const {}}. This causes problems for DDC mock generation.
class A {
  void f({Set<int> a = const {}}) {
    print('a: $a');
  }
}

class B implements A {
  noSuchMethod(Invocation invocation) => super.noSuchMethod(invocation);
}

We're sorry, you've found a bug in our compiler.
You can report this bug at:
https://github.com/dart-lang/sdk/issues/labels/area-dev-compiler
Please include the information below in your report, along with
any other information that may help us track it down. Thanks!
dartdevc arguments: -o test.js test.dart
dart --version: 2.3.3-edge.c1b56a9ea0373b6498d17618f1bca6b4ffe5560a (Sat Jun 15 01:02:50 2019 +0000) on "macos_x64"
Unsupported operation: tried to generate an unreachable node: `Set<int> ({})`
#0      CodeGenerator._unreachable (package:dev_compiler/src/analyzer/code_generator.dart:6363:5)
#1      CodeGenerator._emitDartObject (package:dev_compiler/src/analyzer/code_generator.dart:4556:12)
#2      CodeGenerator._emitArgumentInitializers (package:dev_compiler/src/analyzer/code_generator.dart:2503:26)
#3      CodeGenerator._implementMockMember (package:dev_compiler/src/analyzer/code_generator.dart:1674:19)
#4      CodeGenerator._emitClassMethods.<anonymous closure> (package:dev_compiler/src/analyzer/code_generator.dart:1503:21)
#5      MappedIterator.moveNext (dart:_internal/iterable.dart:392:18)
#6      List.addAll (dart:core-patch/growable_array.dart:193:13)
#7      CodeGenerator._emitClassMethods (package:dev_compiler/src/analyzer/code_generator.dart:1502:15)
#8      CodeGenerator._emitClassDeclaration (package:dev_compiler/src/analyzer/code_generator.dart:819:21)
#9      CodeGenerator.visitClassDeclaration (package:dev_compiler/src/analyzer/code_generator.dart:778:12)
#10     ClassDeclarationImpl.accept (package:analyzer/src/dart/ast/ast.dart:1560:49)
#11     CodeGenerator._emitTypeDeclaration (package:dev_compiler/src/analyzer/code_generator.dart:511:26)
#12     CodeGenerator.visitCompilationUnit (package:dev_compiler/src/analyzer/code_generator.dart:565:9)
#13     List.forEach (dart:core-patch/array.dart:106:8)
#14     CodeGenerator.compile (package:dev_compiler/src/analyzer/code_generator.dart:318:22)
#15     compileWithAnalyzer (package:dev_compiler/src/analyzer/module_compiler.dart:108:33)
#16     _compile (package:dev_compiler/src/analyzer/command.dart:152:16)
#17     compile (package:dev_compiler/src/analyzer/command.dart:56:18)
#18     compile (package:dev_compiler/src/compiler/shared_command.dart:413:18)
#19     main (file:///b/s/w/ir/cache/builder/sdk/pkg/dev_compiler/bin/dartdevc.dart:32:24)
<asynchronous suspension>
#20     _startIsolate.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:297:32)
#21     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:172:12)

I think we need to:

Implement _emitConstSet as we do emitConstList but using constSet instead of constList.

Add a clause here to handle the constant set case and call the above method.

authors: scheglov vsmenon


text:
Someime analysys center crashes

Analyzer Feedback from IntelliJ
Version information

IDEA IC-192.7142.36
2.5.0-edge.1ef83b86ae637ffe7359173804cbc6d3fa25e6db
IC-192.7142.36, JRE 11.0.4+10-b304.77x64 JetBrains s.r.o, OS Windows 10(amd64) v10.0 , screens 1920x1080

Exception
Dart analysis server, SDK version 2.5.0-edge.1ef83b86ae637ffe7359173804cbc6d3fa25e6db, server version 1.27.2, error: Failed to handle completion domain request: {id: 7633, method: completion.getSuggestions, params: {file: C:\projects\myLuka\lib\chat\ui\chat_message_widget.dart, offset: 5991}, clientRequestTime: 1572603771377}
NoSuchMethodError: The getter 'offset' was called on null.
Receiver: null
Tried calling: offset
#0      Object.noSuchMethod (dart:core-patch/object_patch.dart:51:5)
#1      _OpTypeAstVisitor.visitSimpleFormalParameter (package:analyzer_plugin/src/utilities/completion/optype.dart:1059:30)
#2      _OpTypeAstVisitor.visitFormalParameterList (package:analyzer_plugin/src/utilities/completion/optype.dart:713:7)
#3      FormalParameterListImpl.accept (package:analyzer/src/dart/ast/ast.dart:4551:49)
#4      new OpType.forCompletion (package:analyzer_plugin/src/utilities/completion/optype.dart:152:16)
#5      DartCompletionRequestImpl.opType (package:analysis_server/src/services/completion/dart/completion_manager.dart:377:21)
#6      DartCompletionManager._addIncludedElementKinds (package:analysis_server/src/services/completion/dart/completion_manager.dart:198:26)
#7      DartCompletionManager.computeSuggestions (package:analysis_server/src/services/completion/dart/completion_manager.dart:133:7)
<asynchronous suspension>
#8      CompletionDomainHandler.computeSuggestions (package:analysis_server/src/domain_completion.dart:122:42)
<asynchronous suspension>
#9      CompletionDomainHandler.processRequest (package:analysis_server/src/domain_completion.dart:392:5)
<asynchronous suspension>
#10     CompletionDomainHandler.handleRequest.<anonymous closure> (package:analysis_server/src/domain_completion.dart:282:9)
#11     _rootRun (dart:async/zone.dart:1124:13)
#12     _CustomZone.run (dart:async/zone.dart:1021:19)
#13     _runZoned (dart:async/zone.dart:1516:10)
#14     runZoned (dart:async/zone.dart:1500:12)
#15     CompletionDomainHandler.handleRequest (package:analysis_server/src/domain_completion.dart:275:12)
#16     AnalysisServer.handleRequest.<anonymous closure>.<anonymous closure> (package:analysis_server/src/analysis_server.dart:325:45)
#17     _PerformanceTagImpl.makeCurrentWhile (package:analyzer/src/generated/utilities_general.dart:258:15)
#18     AnalysisServer.handleRequest.<anonymous closure> (package:analysis_server/src/analysis_server.dart:321:50)
#19     _rootRun (dart:async/zone.dart:1124:13)
#20     _CustomZone.run (dart:async/zone.dart:1021:19)
#21     _runZoned (dart:async/zone.dart:1516:10)
#22     runZoned (dart:async/zone.dart:1500:12)
#23     AnalysisServer.handleRequest (package:analysis_server/src/analysis_server.dart:320:5)
#24     ByteStreamServerChannel._readRequest.<anonymous closure> (package:analysis_server/src/channel/byte_stream_channel.dart:174:16)
#25     _PerformanceTagImpl.makeCurrentWhile (package:analyzer/src/generated/utilities_general.dart:258:15)
#26     ByteStreamServerChannel._readRequest (package:analysis_server/src/channel/byte_stream_channel.dart:165:47)
#27     ByteStreamServerChannel.listen.<anonymous closure> (package:analysis_server/src/channel/byte_stream_channel.dart:110:26)
#28     _rootRunUnary (dart:async/zone.dart:1132:38)
#29     _CustomZone.runUnary (dart:async/zone.dart:1029:19)
#30     _CustomZone.runUnaryGuarded (dart:async/zone.dart:931:7)
#31     _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:336:11)
#32     _BufferingStreamSubscription._add (dart:async/stream_impl.dart:263:7)
#33     _SinkTransformerStreamSubscription._add (dart:async/stream_transformers.dart:68:11)
#34     _EventSinkWrapper.add (dart:async/stream_transformers.dart:15:11)
#35     _StringAdapterSink.add (dart:convert/string_conversion.dart:236:11)
#36     _LineSplitterSink._addLines (dart:convert/line_splitter.dart:150:13)
#37     _LineSplitterSink.addSlice (dart:convert/line_splitter.dart:125:5)
#38     StringConversionSinkMixin.add (dart:convert/string_conversion.dart:163:5)
#39     _SinkTransformerStreamSubscription._handleData (dart:async/stream_transformers.dart:120:24)
#40     _rootRunUnary (dart:async/zone.dart:1132:38)
#41     _CustomZone.runUnary (dart:async/zone.dart:1029:19)
#42     _CustomZone.runUnaryGuarded (dart:async/zone.dart:931:7)
#43     _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:336:11)
#44     _BufferingStreamSubscription._add (dart:async/stream_impl.dart:263:7)
#45     _SinkTransformerStreamSubscription._add (dart:async/stream_transformers.dart:68:11)
#46     _EventSinkWrapper.add (dart:async/stream_transformers.dart:15:11)
#47     _StringAdapterSink.add (dart:convert/string_conversion.dart:236:11)
#48     _StringAdapterSink.addSlice (dart:convert/string_conversion.dart:241:7)
#49     _Utf8ConversionSink.addSlice (dart:convert/string_conversion.dart:312:20)
#50     _Utf8ConversionSink.add (dart:convert/string_conversion.dart:305:5)
#51     _ConverterStreamEventSink.add (dart:convert/chunked_conversion.dart:72:18)
#52     _SinkTransformerStreamSubscription._handleData (dart:async/stream_transformers.dart:120:24)
#5...

For additional log information, please append the contents of
file://C:\Users\Slava\AppData\Local\Temp\report.txt.

report.txt

Thanks for reporting this issue! This was previously reported at https://github.com/dart-lang/sdk/issues/38513; please follow along there.

Duplicate of https://github.com/dart-lang/sdk/issues/38513


text:
[vm/ffi] Structs API in NNBD

With NNBD, non-nullable fields need initializers or need to be late field.
Our structs API uses nun-nullable fields, without initializers - because they are replaced by getters and setters:
https://github.com/dart-lang/sdk/blob/ef0c7f16e609e84cad7411394cec9fd8b1e1e04b/tests/ffi/coordinate.dart#L11-L26
We could work around it by replacing them with late fields, but it isn't elegant.
 class Coordinate extends Struct {
   @Double()
   late double x;

   @Double()
   late double y;

   late Pointer<Coordinate> next;
 }

cc @mkustermann @lrhn

Reference to migrating breaking changes for NNBD: https://github.com/dart-lang/sdk/issues/40444.
For this API change, we can probably allow struct fields to be non-external for opted out code to ease the migration.

Proposal language: https://github.com/dart-lang/language/blob/master/working/00044 - Abstract-External Variables/proposal.md

This is not required for unforking the SDK.
For the time being, we can keep the FFI usages opted out of NNBD. (NNBD is an opt in feature.)

All tests/ffi/* now succeed on all configurations except for the analyzer.
SELECT configuration
FROM `dart-ci.results.latest_results`
WHERE suite = "ffi"
AND unexpected_result IS NOT NULL
GROUP BY configuration;

Row | configuration |
-- | -- | --
1 | analyzer-asserts-strong-linux |
2 | analyzer-asserts-weak-linux
Analyzer issue: https://github.com/dart-lang/sdk/issues/41940


text:
Passing a non-const value to a const constructor gives a confusing error message

Consider this code:
class Foo {
  const Foo(this.x);
  final int x;
}

void test(int x) {
  const Foo(x);
}

void main() { }

I would expect one error message, namely, "Arguments of a constant creation must be constant expressions" or some variant thereof. However, today we get two errors, and the first one is really confusing:
  error • A value of type 'Null' can't be assigned to a parameter of type 'int' in a const constructor • test.dart:7:13 • const_constructor_param_type_mismatch
  error • Arguments of a constant creation must be constant expressions • test.dart:7:13 • const_with_non_constant_argument

This is because const in Dart requires all values to be known at compile time. In this case, x is known at run-time, not compile-time because x is not known to the constructor until it is provided by the outer function.
If you're coming from a JavaScript background, const in Dart works much differently than it does in JavaScript.
final in Dart works more like const does in JavaScript. Here's a more thorough explanation.
https://stackoverflow.com/questions/50431055/what-is-the-difference-between-the-const-and-final-keywords-in-dart

This is because const in Dart requires all values to be known at compile time. In this case, x is known at run-time, not compile-time because x is not known to the constructor until it is provided by the outer function.
So const Foo(x) doesn't work but const Foo(9) does. If you want to pass a parameter, just remove const.
If you're coming from a JavaScript background, const in Dart works much differently than it does in JavaScript. final in Dart works more like const does in JavaScript. Here's a more thorough explanation:
https://stackoverflow.com/questions/50431055/what-is-the-difference-between-the-const-and-final-keywords-in-dart

I think you are missing the OP's point. While your answer is correct as to why that code is in error - the actual point is that the error message is super confusing, especially this part: "A value of type 'Null' can't be assigned to a parameter of type 'int' in a..." - why is Null mentioned? That is the part that could be improved. I am still getting this message today - and in my case it's within an enum (rather than a class) where the constructor must be const...

authors: Hixie eggzotic khari998


text:
[vm][flutter][android] Logs are truncated at 1024 characters

In Flutter, Dart VM logs are truncated at 1024 characters:
E/DartVM  ( 6593): *** BEGIN CFG
E/DartVM  ( 6593): Unoptimized Compilation
E/DartVM  ( 6593): ==== package:flutter/src/rendering/proxy_box.dart_RenderAnnotatedRegion_set_value (SetterFunction)
E/DartVM  ( 6593): B0[graph]:0
E/DartVM  ( 6593): B1[function entry]:2
E/DartVM  ( 6593):     Constant(#0)
E/DartVM  ( 6593):     CheckStackOverflow:8(stack=0, loop=0)
E/DartVM  ( 6593):     DebugStepCheck:10()
E/DartVM  ( 6593):     t0 <- LoadLocal(newValue @1)
E/DartVM  ( 6593):     t1 <- Constant(#TypeParameter: X0; bound: Object)
E/DartVM  ( 6593):     t2 <- LoadLocal(this @2)
E/DartVM  ( 6593):     t2 <- LoadField(t2 . :type_arguments {final})
E/DartVM  ( 6593):     t3 <- Constant(#null)
E/DartVM  ( 6593):     t0 <- AssertAssignable:12(t0, t1, 'newValue', instantiator_type_args(t2), function_type_args(t3))
E/DartVM  ( 6593):     StoreLocal(newValue @1, t0)
E/DartVM  ( 6593):     goto:48 B5
E/DartVM  ( 6593): B6[function entry]:46
E/DartVM  ( 6593):     Constant(#1)
E/DartVM  ( 6593):     CheckStackOverflow:40(stack=0, loop=0)
E/DartVM  ( 6593):     DebugStepCheck:42()
E/DartVM  ( 6593):     goto:50 B5
E/DartVM  ( 6593): B5[join]:44 pred(B1, B6)
E/DartVM  ( 6593):     t0 <- LoadLocal(this @2)
E/DartVM  ( 6593):     t0 <- InstanceCall:14( get:_value@469160605<0>, t0)
E/DartVM  ( 6593):     t1 <- LoadLocal(newValue @1)
E/DartVM  ( 6593):     t0 <- InstanceCall:16( ==<0>, t0, t1 using unchecked entrypoint)
E/DartVM  ( 6593):     t1 <- LoadLocal(:t0 @-2)
E/DartVM  ( 6593):     AssertBoolean:18(t1)
E/DartVM  ( 6593):     t1 <- Constant(#true)
E/DartVM  ( 6593):     Branch if St
E/DartVM  ( 6593): *** BEGIN CFG
E/DartVM  ( 6593): Unoptimized Compilation

*** BEGIN CFG
Unoptimized Compilation
==== package:flutter/src/rendering/proxy_box.dart_RenderAnnotatedRegion_set_value (SetterFunction)
B0[graph]:0
B1[function entry]:2
    Constant(#0)
    CheckStackOverflow:8(stack=0, loop=0)
    DebugStepCheck:10()
    t0 <- LoadLocal(newValue @1)
    t1 <- Constant(#TypeParameter: X0; bound: Object)
    t2 <- LoadLocal(this @2)
    t2 <- LoadField(t2 . :type_arguments {final})
    t3 <- Constant(#null)
    t0 <- AssertAssignable:12(t0, t1, 'newValue', instantiator_type_args(t2), function_type_args(t3))
    StoreLocal(newValue @1, t0)
    goto:48 B5
B6[function entry]:46
    Constant(#1)
    CheckStackOverflow:40(stack=0, loop=0)
    DebugStepCheck:42()
    goto:50 B5
B5[join]:44 pred(B1, B6)
    t0 <- LoadLocal(this @2)
    t0 <- InstanceCall:14( get:_value@469160605<0>, t0)
    t1 <- LoadLocal(newValue @1)
    t0 <- InstanceCall:16( ==<0>, t0, t1 using unchecked entrypoint)
    t1 <- LoadLocal(:t0 @-2)
    AssertBoolean:18(t1)
    t1 <- Constant(#true)
    Branch if St

The above is exactly 1024 characters.
Related Flutter issue with a Dart workaround: https://github.com/flutter/flutter/issues/22665
To reproduce this use a custom engine which allows all DartVM flags: flutter run --local-engine=android_debug_unopt_x86 --local-engine-src-path=$HOME/flt/engine/src -d emu --dart-flags --print-flow-graph,--print-flow-graph-filter=...

Workaround: flutter run --local-engine=android_debug_unopt_x86 --local-engine-src-path=$HOME/flt/engine/src -d emu --dart-flags --print-flow-graph,--print-flow-graph-filter=convertYUV420ToImage,--force-log-flush
However, that causes many extra newlines:
E/DartVM  (11212): *** BEGIN CFG
E/DartVM  (11212): Unoptimized Compilation
E/DartVM  (11212): ==== package:flutter_match/tflite/camera_utils.dart_CameraUtils_convertYUV420ToImage (RegularFunction
E/DartVM  (11212): )
E/DartVM  (11212): B0[graph]:0
E/DartVM  (11212):
E/DartVM  (11212): B1[function entry]:2
E/DartVM  (11212):
E/DartVM  (11212):
E/DartVM  (11212): CheckStackOverflow:8(stack=0, loop=0)
E/DartVM  (11212):
E/DartVM  (11212):
E/DartVM  (11212): DebugStepCheck:10()
E/DartVM  (11212):
E/DartVM  (11212):
E/DartVM  (11212): t0 <- LoadLocal(cameraImage @1)
E/DartVM  (11212):
E/DartVM  (11212):
E/DartVM  (11212): t0 <- InstanceCall:12( get:width<0>, t0)
E/DartVM  (11212):
E/DartVM  (11212):
E/DartVM  (11212): StoreLocal(width @-1, t0)
E/DartVM  (11212):
E/DartVM  (11212):
E/DartVM  (11212): t0 <- LoadLocal(cameraImage @1)
E/DartVM  (11212):
E/DartVM  (11212):
E/DartVM  (11212): t0 <- InstanceCall:14( get:height<0>, t0)
E/DartVM  (11212):
E/DartVM  (11212):
E/DartVM  (11212): StoreLocal(height @-2, t0)
E/DartVM  (11212):
E/DartVM  (11212):


text:
API to create deeply immutable typed data, from one or multiple chunks.

The VM has support for deeply immutable typed data. Those can be shared by-pointer across isolates. Though right now one can only create them using an embedder API.
Using UnmodifiableUint8ListView(bytes) is not deeply immutable, because bytes may be mutable and someone may write to them.
So I propose to make an API that can make an UnmodifiableUint8ListView from any Uint8List and it may make a copy of it if the underlying bytes are mutable.
Since data is often available as chunks (e.g. loading from network results in List<Uint8List>), the API should support making UnmodifiableUint8ListView from such a set of chunks.
A possible API would be
class UnmodifiableUint8ListView {
    factory UnmodifiableUint8ListView.fromChunks(List<Uint8List> chunks);
}

/cc @lrhn opinions about the API?

This API sounds unrelated to being unmodifiable.
We should have a modifiable ByteBuffer and an unmodifiable ByteBuffer, and then (seemingly) use the same typed-data views on top.
Building a buffer from chunks would apply to both kinds of buffers.
So, we can have both Uint8List.fromChunks(Iterable<Uint8List> chunks) and Uint8List.unmodifiableFromChunks(Iterable<Uint8List> chunks)/UnmodifiableUint8ListView.fromChunks.
Or even ByteBuffer.fromChunks(Iterable<...something...> chunks).
(The UnmodifiableThingListView types are mostly bad design. They should just be UnmodifiableThingList.)

Building a buffer from chunks would apply to both kinds of buffers.
So, we can have both Uint8List.fromChunks(Iterable<Uint8List> chunks) and
Uint8List.unmodifiableFromChunks(Iterable<Uint8List> chunks)/UnmodifiableUint8ListView.fromChunks.
Or even ByteBuffer.fromChunks(Iterable<...something...> chunks).

LGTM. @lrhn would this be something you'd like to do?
(Side note: Maybe fromList is better for consistency than fromChunks, since we have TransferableTypedData.fromList already?)

(The UnmodifiableThingListView types are mostly bad design. They should just be UnmodifiableThingList.)

Something for dart 3.0? (could make typedef now and migrate code)

LGTM. @lrhn would this be something you'd like to do?

Design the API, sure.
Can you elaborate a little more on how the native unmodifiable type data works?
Do you simply have an unmodifiable ByteBuffer which throws on modification attempts?
We can update the fromList to take a flag saying whether the result should be immutable or not:
external factory Uint8List.fromList(List<int> bytes, {bool unmodifiable = false});

The immutable list will have the same API and static type as the mutable list.
With that as a way to directly create an unmodifiable typed-data lis, I'd assume the .buffer of that would inherit the immutability, so immutableList.buffer.asUint16List() would give you an unmodifiable 16-bit list.
We can choose to only use UnmodifiableUint16ListView as views (what the unmodifiable ByteBuffer.asUint16ListView returns) so that it throws early on attempts to modify, or use the normal views and have the underlying buffer throw on modification attempts.
The second part, concatenating existing bytes, could be a separate constructor:
external factory Uint8List.fromLists(Iterable<List<int>> bytes, {bool unmodifiable = false});

Those two constructor changes should be the only needed API for this to work.
Then you can do everything you do today with the unmodiifable lists, like create views, as long as you don't try to change the contents.
We can consider adding a bool get isMutable to the lists, or to the buffer. It's not necessary, we don't do that for normal unmodifiable lists. (But it's technically possible because all typed-data types are sealed.)
WDYT?

authors: lrhn mkustermann


text:
[dart2wasm] Tear-offs of same instance are not equal

Extracted from a larger SDK test:
class C {
  void call() {}
}

void main() {
  final c = C();
  print(c.call == c.call);
}

AOT output: true
dart2wasm output: false

We could implement _Function.operator == to achieve this by comparing the contexts (which contain the this references for tear-offs) and the vtables. So just two reference comparisons.
Implementing a (non-trivial) hashCode that is consistent with this is tricky, though. For tear-offs, the this objects have identity hash codes we can use, but this is not the case for other contexts. The vtables current don't have anything hashable in them, and adding a hash for this obscure use case seems wasteful. We could hash the types, which are always equal whenever the vtables are.
So,
int get hashCode => runtimeType.hashCode;

could be an initial version which is consistent with == and not completely trivial.

Hmm, looking at the description for Function.operator ==:
  /// Test whether another object is equal to this function.
  ///
  /// Function objects are only equal to other function objects
  /// (an object satisfying `object is Function`),
  /// and never to non-function objects.
  ///
  /// Some function objects are considered equal by `==`
  /// because they are recognized as representing the "same function":
  ///
  /// - It is the same object. Static and top-level functions are compile time
  ///   constants when used as values, so referring to the same function twice
  ///   always give the same object, as does referring to a local function
  ///   declaration twice in the same scope where it was declared.
  /// - if they refer to the same member method extracted from the same object.
  ///   Repeatedly extracting an instance method of an object as a function value
  ///   gives equal, but not necessarily identical, function values.
  ///
  /// Different evaluations of function literals
  /// never give rise to equal function objects.
  /// Each time a function literal is evaluated,
  /// it creates a new function value that is not equal to any other function
  /// value, not even ones created by the same expression.

My suggestion above could sometimes characterize different evaluations of the same function literal as equal, in violation of the last paragraph.
A fix would be to only consider the function objects equal if their contexts are objects, i.e. subtypes of the class hierarchy top type, meaning the closures are tear-offs.
The hashCode could use a combination (e.g. xor) of the type hash and the this identity hash for tear-offs and the identity hash of the closure for other closures.

We could implement _Function.operator == to achieve this by comparing the contexts (which contain the this references for tear-offs) and the vtables. So just two reference comparisons.

Isn't this the same as my suggestion?

My suggestion above could sometimes characterize different evaluations of the same function literal as equal

You mean when the function doesn't have a context (doesn't capture variables), right?
We could allocate an empty struct for context in those cases and compare contexts by reference in all cases.
Allocation of function literals will be slightly more expensive, but equality check won't need any special cases and will be faster.
Would that work?

My suggestion above could sometimes characterize different evaluations of the same function literal as equal

You mean when the function doesn't have a context (doesn't capture variables), right?

This is an interesting question: We've specified that certain function objects must compare equal, or even identical, but not that they must be different. The underlying rationale is that it should be allowed for a compiler to optimize, for instance, a function literal that does not use its context:
void main() {
  Function functionFactory() => () => 3;
  var f1 = functionFactory(), f2 = functionFactory();
  print(identical(f1, f2)); // Can be 'true'.
}

This is because it's allowed to implement the above as follows:
int _g() => 3;

void main() {
  Function functionFactory() => _g;
  var f1 = functionFactory(), f2 = functionFactory();
  print(identical(f1, f2)); // 'true'.
}

Isn't this the same as my suggestion?

If by "Function indices" you mean the reference to the vtable, then yes. 😉

You mean when the function doesn't have a context (doesn't capture variables), right?

Not only that. When a function literal occurs inside a loop but only captures variables outside the loop, the loop context is optimized away, and the closure refers directly to the outer context. So closures produced by different iterations of the loop would compare equal.

Allocation of function literals will be slightly more expensive, but equality check won't need any special cases and will be faster.

Comparison of closures is an obscure use case. If we can avoid it, we should not make commonly used constructs slower to support it. Much rather make closure comparison slower.

We've specified that certain function objects must compare equal, or even identical, but not that they must be different.

Does this mean that the last paragraph of the documentation for Function.operator ==, which mandates some function objects to compare different, and thus also not identical, is incorrect?

Does this mean that the last paragraph of the documentation for Function.operator ==, which mandates some function objects to compare different, and thus also not identical, is incorrect?

Correct. We discussed this with Erik in chat and I created https://dart-review.git.corp.google.com/c/sdk/+/269721 to fix this.

Since function literals not being equal and identical is not part of the spec (and that part of the Function.operator == documentation will be removed soon with the CL above), we can now implement this simply by comparing vtable and context references. No special cases needed for functions without context.

https://dart-review.git.corp.google.com/c/sdk/+/269721 updates Function documentation and removes invalid tests
https://dart-review.googlesource.com/c/sdk/+/269882 fixes this issue in dart2wasm

Both CLs landed.

authors: askeksa-google eernstg osa1


text:
Spawning 16 (or more) [Isolate]s with [sleep] freeze the application

Hi! I'm in a situation where I'm using Isolates to "poll" for events. But as soon as there are more than 16 isolates in Dart SDK or 8 in Flutter, whole application freezes. Please refer to code below.
Expected Behavior
"Alive!" is printed out to the console every second.
Actual Behavior
No output is ever printed out to the console as if application is frozen.
When using the following code in Dart SDK & Flutter:

Dart SDK: "Alive!" is printed as long as [kIsolateSpawnCount] is less than 16.
Flutter: "Alive!" is printed as long as [kIsolateSpawnCount] is less than 8.

Once kIsolateSpawnCount is set to 15 in Dart SDK or 7 in Flutter, "Alive!" is printed every second as expected.

Dart Example
import 'dart:io';
import 'dart:async';
import 'dart:isolate';

/// Expected Behavior
/// -----------------
/// "Alive!" is printed out to the console every second.
///
/// Actual Behavior
/// ---------------
/// No output is ever printed out to the console as if application is frozen.
/// When using the following code in Dart SDK & Flutter:
/// 1. Dart SDK: "Alive!" is printed as long as [kIsolateSpawnCount] is less than 16.
/// 2. Flutter: "Alive!" is printed as long as [kIsolateSpawnCount] is less than 8.
///
/// Once [kIsolateSpawnCount] is set to 15 in Dart SDK or 7 in Flutter, "Alive!" is printed every second as expected.
///

const kIsolateSpawnCount = 16;

void isolate(message) {
  // [sleep]ing one [Isolate] should not affect rest of the [Isolate]s, main [Isolate] or the event loop.
  // Note: [Future.delayed] works as intended.
  sleep(const Duration(days: 1));
}

Future<void> main() async {
  for (int i = 0; i < kIsolateSpawnCount; i++) {
    await Isolate.spawn(isolate, null);
  }
  Timer.periodic(
    const Duration(seconds: 1),
    (e) {
      print('Alive!');
    },
  );
}

Flutter Example
import 'dart:io';
import 'dart:async';
import 'dart:isolate';

import 'package:flutter/material.dart';

/// Expected Behavior
/// -----------------
/// "Alive!" is printed out to the console every second.
///
/// Actual Behavior
/// ---------------
/// No output is ever printed out to the console as if application is frozen.
/// When using the following code in Dart SDK & Flutter:
/// 1. Dart SDK: "Alive!" is printed as long as [kIsolateSpawnCount] is less than 16.
/// 2. Flutter: "Alive!" is printed as long as [kIsolateSpawnCount] is less than 8.
///
/// Once [kIsolateSpawnCount] is set to 15 in Dart SDK or 7 in Flutter, "Alive!" is printed every second as expected.
///

const kIsolateSpawnCount = 8;

void isolate(message) {
  // [sleep]ing one [Isolate] should not affect rest of the [Isolate]s, main [Isolate] or the event loop.
  // Note: [Future.delayed] works as intended.
  sleep(const Duration(days: 1));
}

void main() {
  runApp(const MyApp());
}

class MyApp extends StatefulWidget {
  const MyApp({Key? key}) : super(key: key);

  @override
  State<MyApp> createState() => _MyAppState();
}

class _MyAppState extends State<MyApp> {
  @override
  void initState() {
    super.initState();
    WidgetsBinding.instance.addPostFrameCallback((_) async {
      for (int i = 0; i < kIsolateSpawnCount; i++) {
        await Isolate.spawn(isolate, null);
      }
      Timer.periodic(
        const Duration(seconds: 1),
        (e) {
          debugPrint('Alive!');
        },
      );
    });
  }

  @override
  Widget build(BuildContext context) {
    return const MaterialApp(
      home: HomeScreen(),
    );
  }
}

class HomeScreen extends StatelessWidget {
  const HomeScreen({Key? key}) : super(key: key);

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: const Text('test'),
      ),
      body: Center(
        child: ElevatedButton(
          onPressed: () {
            Navigator.of(context).push(
              MaterialPageRoute(
                builder: (context) => const HomeScreen(),
              ),
            );
          },
          child: const Text('Go!'),
        ),
      ),
    );
  }
}

dart --version
Dart SDK version: 2.18.6 (stable) (Tue Dec 13 21:15:14 2022 +0000) on "windows_x64"

I assume Flutter spawns it's own isolates. I don't know however, I have limited knowledge of this.

I'm working on package:media_kit & one of the users recently reported that as soon as they create 8 Player instances, everything freezes up. I could reduce down problem to the code I shared above. Still, you can see these lines which run on another Isolate for every instance of Player, where I'm calling mpv_wait_event, passing negative duration means waiting until next event comes up.
Now I'm considering to create a separate shared library which will do the "polling" using C++'s std::thread, but it will be a pain & hacky approach to ship another shared library separately for each platform (while rest of the code is in 100% Dart).
On the other hand, using Isolate already present in Dart SDK was handy & worked correctly.
Any guidance or explanation will be really helpful.
Thanks!

/cc @aam @mkustermann

The issue is well known: We only allow a certain number of threads to be active in an isolate group to avoid issues with the memory subsystem (see https://github.com/dart-lang/sdk/issues/51261 for more context)
There's a workaround to this, which is basically telling the Dart VM that the isolate isn't actively needed, which can currently be done via custom C code:
auto isolate = Dart_CurrentIsolate();
Dart_ExitIsolate();
{
  <... run blocking C Code, e.g. sleep() or mpv_wait_event() ...>
}
Dart_EnterIsolate(isolate);

Filed https://github.com/dart-lang/sdk/issues/51261 for possibly letting the VM do that automatically based on boolean.

@alexmercerind Do you have the option to write custom C code for the few places where you know C code can block for longer time? If so, is your use case standalone dart or flutter (or both)?
(Flutter is hiding all Dart C API symbols - such as Dart_ExitIsolate() - so we may need to add those two functions to our NativeApi.initializeApiDLData & include/dart_api_dl.h)

Hi @mkustermann!
Thanks a lot for the detailed explanation & information. I'm really glad.
Sure! Flutter recently added a new project template for FFI plugins, where one can build custom C/C++ shared library targets before build. I think I can include these Dart API headers there & export the required C methods for access inside Dart SDK.
Alternatively, I think I can (I believe this will save you people a lot of work as-well since my case seems very specific):
I'm thinking to call mpv_wait_event within C/C++ using std::thread (avoiding to use Isolate) & send the raw address (int) of the mpv_event* using SendPort/ReceivePort mechanism to Dart (where I can interpret it using dart:ffi). This will avoid any block within Dart. I have used this once before when dealing with https://github.com/dart-lang/sdk/issues/37022.
Infact, libmpv also offers a non-wait API for handling events, but since those event callbacks are invoked from another thread, it caused Dart VM to crash due to https://github.com/dart-lang/sdk/issues/37022,. So, I went with mpv_wait_event + Isolate, but it seems this has it's own limitations too.
Thanks again!

Hi!
I exported Dart_CurrentIsolate, Dart_EnterIsolate & Dart_ExitIsolate inside dart_api_dl.h like you guided above.
However, the whole project seems to experience some memory access violation when calling Dart_CurrentIsolate_DL.
Am I missing something here?
Thanks!
https://github.com/alexmercerind/dart_api_test

For now I just created a new FFI plugin package which spawns a std::thread & forwards data to Dart VM through SendPort. Inside original root package, I just load the native library if found otherwise fallback to Isolate based solution.
https://github.com/alexmercerind/media_kit/blob/b5b980e29b2f88f226a58c1c5eda416dcb19b598/media_kit_native_event_loop/src/media_kit_native_event_loop.cc#L43-L46
https://github.com/alexmercerind/media_kit/blob/main/media_kit/lib/src/libmpv/core/initializer.dart

This is a huge implementation overhead to a seemingly simple feature.
@mkustermann Will you be willing to accept a PR exposing these functions?

@mkustermann Is there a possibility to actually use Dart_CurrentIsolate etc. in the native code of an Flutter application? Since, as you mentioned, these functions are not exported it seems that the workaround cannot be applied.
@mkustermann Will you be willing to accept a PR exposing these functions?

The 3 needed API functions are quite stable and I don't see a big reason not to expose them. Would be happy to accept a PR - though our ffi testing setup has some particularities, so maybe it's easier if we do it: made cl/292722. Will add relevant reviews to see if they have objections.
Will think a bit more about whether we can do it in an automatic way (see https://github.com/dart-lang/sdk/issues/51261) that works nicely.

@SynSzakala We could easily expose these functions, but before that I want to check with you regarding the timelines. If we land this change today it is going to reach master or dev Flutter channel relatively quickly, but it is going to take more than 6 months to reach a stable Flutter release. So we have a choice - to land something quick and dirty which you can only use on a master channel for a while - or spend this time to design a more complete and future-proof solution.
Which Flutter channel do you use?

@mraleph I'm aware of that, it'll be completely fine for us to migrate to the master channel (or back-port the fix to a fork somehow, if that's possible). It's still less work than adding an additional asynchronous native layer (especially given large API surfaces).
Of course, designing and implementing solution as discussed in #51261 would be very desirable, since that completely removes the need for adding native code. But, from what I understand there's still some discussion on how to do that right.
So, from my perspective it would be reasonable to expose the Dart_*Isolate functions anyway, since the solution discussed in #51261 might not cover every use-case (e.g. some kind of conditional blocking in the native code).

I agree with @SynSzakala comment, that it is just way easier to write that kind of logic in Dart and TBH I'm into letting more advanced usecases for DL APIs. I currently don't have any issue that this change would resolve, but this change gives FFI users flexibility and it wouldn't already burden people that are currently using stuff like NativePorts.

This has landed now on main branch in the Dart SDK. Once it rolls up to flutter engine and flutter framework main branches, it will be available to use.
We'll continue the discussion on how to do this better / automatically in https://github.com/dart-lang/sdk/issues/51261 (to avoid the custom C code that exits + enters isolate).

I also experienced this, is there anything the same?
has it been fixed?

I also experienced this, is there anything the same?
has it been fixed?

@azkadev Generally if the isolate yield back to the event loop there should be no freezing of the application. This is mainly a problem if one e.g. calls out to C code in many isolates and that C code blocks. For this scenario one can use a new C API to exit the current isolate and re-enter when the blocking C call has finished. See https://github.com/dart-lang/sdk/commit/f9eb97fdff9517d7b7f0f1ddec936dd3920ce89c  (You can use dart:ffis NativeApi.initializeApiDLData pass it to C code, which can include dart_api_dl.h from the Dart SDK (can make a copy of it) initialize it and then call the 2 new API functions)

authors: Piero512 SynSzakala alexmercerind azkadev mkustermann mraleph


text:
Bluetooth serial

Hello,
I tried to use the example for bluetooth, but I always have "Bluetooth radio is not discoverable" message and "no devices found". Is there somethings to do to have discoverable enable ?
The main goal of this is to have serial communication over Bluetooth with a device. I have succeed on Android with package "flutter_bluetooth_serial" and I have to do the same on windows.
I don't no if I can do this with the "win32" package, but any help or advise would be very welcomed.
kind regards,
Nicolas Hallynck

Hi there! Thanks for the report. On investigating, I found a bug in the Bluetooth example where it wasn't setting the size of the struct correctly. I've updated it and now it seems to be working for me:
https://github.com/timsneath/win32/blob/main/example/bluetooth.dart
[C:\src\win32] dart example\bluetooth.dart
Found a radio with handle: 0x00000250
Radio name: TIM-SURFACE
Radio address: 03:55:55:8E:34:C8
Bluetooth radio is discoverable.
Device address: 52:D6:90:6B:88:70
  Name: GID6B
  Authenticated: True
  Connected: True
  Remembered: True
Device address: CC:66:75:CD:69:C8
  Name:
  Authenticated: False
  Connected: False
  Remembered: False

Give the updated version a try and see if it works for you?

Hi,
Thanks for the update. I've tested it and it 's working now for me also.

Great! Thanks again for spotting this!

Hi @timsneath , is it possible to search for a Bluetooth device that is not yet connected to a Windows device? because currently when I try to run the example at https://github.com/dart-windows/win32/blob/main/example/bluetooth.dart it only shows connected bluetooth devices.

authors: nico3587 tegarnugroho timsneath


text:
add " ' " around ticket id to make it a more valid JSON object

The TRADES is not a valid JSON because it is using ' and not ". This fix just add a ' around the ticket ID map so it can be parsed as JSON by replacing all ' with "

Hi,
thank you for your suggestion and sorry for the late reply.
You are right that the current solution is not valid JSON. Still, we prefer to keep it this way for now because changing it could break existing strategies. We also aimed for consistency between mql4 and python. Since order IDs are integers in mql4, we thought it more intuitive to also use integer values in python.

source: gharchive/pull-request

authors: elvinex gadieichhorn

repo: darwinex/dwx-zeromq-connector


text:
How to handle concurrent requests ?

Hi so i am getting parallel alerts from trading view like same time buy signal and it is placing 4 orders is there any way to place only one order when having concurrent requests ?

You could store the last trade time and only allow a new trade after x minutes.

authors: elvinex vivek-ram

repo: darwinex/dwxconnect


text:
@mixin annotation

Use PhpStorm's mixin annotation for classes like Query\Builder that forward calls to another class.

This will do for now, though will only auto-complete the storage methods.
Auto-complete can't change for Darya\Database\Storage::query() unless a separate Query\Builder class is made purely to change the @mixin. 🤔
Yeah, this will do for now.

repo: darya/framework

license: mit

license_source: bigquery


text:
feat: better rw locking of platform state

Issue being fixed or feature implemented

What was done?

How Has This Been Tested?

Breaking Changes

Checklist:

[ ] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have added or updated relevant unit/integration/functional/e2e tests
[ ] I have added "!" to the title and described breaking changes in the corresponding section if my code contains any
[ ] I have made corresponding changes to the documentation if needed

For repository code-owners and collaborators only

[ ] I have assigned this pull request to a milestone

Replaced by another PR

repo: dashpay/platform

license: MIT


text:
Pass a dict instead of ParquetFileWriteOptions

ParquetFileWriteOptions can't be pickled, so we can't pass it to distributed workers. But we can pass a Mapping to create the options from. There's no problem passing a pyarrow.Schema though.
Closes https://github.com/dask-contrib/dask-deltatable/issues/50
cc @fjetter

Codecov Report

Merging #51 (f24ee92) into main (b6a675e) will increase coverage by 0.15%.
The diff coverage is 50.00%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
@@            Coverage Diff             @@
##             main      #51      +/-   ##
==========================================
+ Coverage   74.45%   74.61%   +0.15%
==========================================
  Files           6        6
  Lines         321      323       +2
==========================================
+ Hits          239      241       +2
  Misses         82       82

Impacted Files
Coverage Δ

dask_deltatable/write.py
66.29% <50.00%> (+0.77%)
:arrow_up:

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

authors: codecov-commenter j-bennet

repo: dask-contrib/dask-deltatable


text:
Fix a bug in resource specification

Tracing the edited lines back to about 4 years ago, I believe this was a typo/error in how resources are specified. There is a related StackOverflow question here: https://stackoverflow.com/a/72581445/10693596

It's been a long time since this PR had any activity so I'm going to close it out.

authors: SultanOrazbayev jacobtomlinson

repo: dask/dask-jobqueue


text:
Slicing Dask arrays with NumPy scalars

Dask arrays can be sliced with Python integers, Python lists, NumPy arrays, etc. However, slicing with a NumPy scalar results in an AssertionError being raised (full traceback is further down below):
In [1]: import dask.array as da

In [2]: import numpy as np

In [3]: x = da.arange(10)

In [4]: x[0]   # Slicing with Python integer works
Out[4]: dask.array<getitem, shape=(), dtype=int64, chunksize=(), chunktype=numpy.ndarray>

In [5]: x[[0, 1]]   # Slicing with Python list works
Out[5]: dask.array<getitem, shape=(2,), dtype=int64, chunksize=(2,), chunktype=numpy.ndarray>

In [6]: x[np.array([0, 1])]   # Slicing with NumPy arrays
Out[6]: dask.array<getitem, shape=(2,), dtype=int64, chunksize=(2,), chunktype=numpy.ndarray>

In [7]: x[np.array(0)]   # Slicing with NumPy scalars raises an error
# AssertionError is raised here

Details:
In [1]: import dask.array as da

In [2]: import numpy as np

In [3]: x = da.arange(10)

In [4]: x[0]
Out[4]: dask.array<getitem, shape=(), dtype=int64, chunksize=(), chunktype=numpy.ndarray>

In [5]: x[[0, 1]]
Out[5]: dask.array<getitem, shape=(2,), dtype=int64, chunksize=(2,), chunktype=numpy.ndarray>

In [6]: x[np.array([0, 1])]
Out[6]: dask.array<getitem, shape=(2,), dtype=int64, chunksize=(2,), chunktype=numpy.ndarray>

In [7]: x[np.array(0)]
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-7-e3c33b6b5f59> in <module>
----> 1 x[np.array(0)]

~/projects/dask/dask/dask/array/core.py in __getitem__(self, index)
   1766
   1767         out = "getitem-" + tokenize(self, index2)
-> 1768         dsk, chunks = slice_array(out, self.name, self.chunks, index2, self.itemsize)
   1769
   1770         graph = HighLevelGraph.from_collections(out, dsk, dependencies=[self])

~/projects/dask/dask/dask/array/slicing.py in slice_array(out_name, in_name, blockdims, index, itemsize)
    169
    170     # Pass down to next function
--> 171     dsk_out, bd_out = slice_with_newaxes(out_name, in_name, blockdims, index, itemsize)
    172
    173     bd_out = tuple(map(tuple, bd_out))

~/projects/dask/dask/dask/array/slicing.py in slice_with_newaxes(out_name, in_name, blockdims, index, itemsize)
    191
    192     # Pass down and do work
--> 193     dsk, blockdims2 = slice_wrap_lists(out_name, in_name, blockdims, index2, itemsize)
    194
    195     if where_none:

~/projects/dask/dask/dask/array/slicing.py in slice_wrap_lists(out_name, in_name, blockdims, index, itemsize)
    247     # No lists, hooray! just use slice_slices_and_integers
    248     if not where_list:
--> 249         return slice_slices_and_integers(out_name, in_name, blockdims, index)
    250
    251     # Replace all lists with full slices  [3, 1, 0] -> slice(None, None, None)

~/projects/dask/dask/dask/array/slicing.py in slice_slices_and_integers(out_name, in_name, blockdims, index)
    300             )
    301
--> 302     assert all(isinstance(ind, (slice, Integral)) for ind in index)
    303     assert len(index) == len(blockdims)
    304

AssertionError:

This is not very helpful to this issue as posted, but I'd just like to put in a placeholder to note that array assignment with __setitem__ deliberately copied the __getitem__ capabilities. In def parse_assignment_indices, used by __setitem__, we have:
    # Disallow scalar boolean indexing, and also indexing by scalar
    # numpy or dask array.
    #
    # numpy allows these, but Array.__getitem__ does not yet implement
    # them properly, so disallow it for now in __setitem__

(https://github.com/dask/dask/blob/2021.07.1/dask/array/slicing.py#L1397-L1415)
So if indexing by numpy scalars were to be enabled in __getitem__ it would be good to allow it in __setitem__ as well. I would be happy make a PR for setitem in the event it is needed.
Thanks,
David

cc @afofa as you may find this issue interesting

authors: davidhassell jrbourbeau

repo: dask/dask


text:
Overhaul deprecations system in dask.config

The config deprecation system was problematic:

dask/dask used check_deprecations, which worked for dask.config.set but not for ~/.config/dask.yaml
dask/distributed used rename, which worked for  ~/.config/dask.yaml but not for dask.config.set

This PR overhauls and unifies the whole system.
Additionally, it adds support in rename for dot-separated old keys and None values.
Follow-up: https://github.com/dask/distributed/pull/8179

@jrbourbeau this PR is ready for review again

@jrbourbeau this is now blocking https://github.com/dask/dask/issues/10507

I think this may have introduced an accidental change in the behavior of dask.config.get (I think https://github.com/dask/dask-kubernetes/issues/816 is referring to the same thing, albeit somewhat cryptically 🙂):
# Previous behavior, 2023.9.1 and earlier
(.venv)  ➜  dask git:(2023.9.2) ✗ gco 2023.9.1
Previous HEAD position was 216c7c939 Release 2023.9.2 (#10514)
HEAD is now at 3316c38f0 bump version to 2023.9.1
(.venv)  ➜  dask git:(2023.9.1) ✗ python -c 'import dask; print(dask.config.get("array.chunk-size", override_with=None))'
128MiB

# New version after this PR
HEAD is now at 216c7c939 Release 2023.9.2 (#10514)
(.venv)  ➜  dask git:(2023.9.2) ✗ python -c 'import dask; print(dask.config.get("array.chunk-size", override_with=None))'
None

I believe this also contradicts the docstring so I think it's a clear bug and not just a neutral change

If override_with is not None this value will be passed straight back

cc @crusaderky @jrbourbeau @fjetter

authors: bnaul crusaderky


text:
Register assertion rewriting

Thoughts on this? May make debugging a bit easier, as a small comparison:
...
        if str(adt) != str(bdt):
            diff = difflib.ndiff(str(adt).splitlines(), str(bdt).splitlines())
            raise AssertionError('string repr are different' + os.linesep +
                                 os.linesep.join(diff))

        try:
>           assert a.shape == b.shape
E           AssertionError

dask/array/utils.py:99: AssertionError

vs.
...
        if str(adt) != str(bdt):
            diff = difflib.ndiff(str(adt).splitlines(), str(bdt).splitlines())
            raise AssertionError('string repr are different' + os.linesep +
                                 os.linesep.join(diff))

        try:
>           assert a.shape == b.shape
E           assert (4, 3) == (3, 4)
E             At index 0 diff: 4 != 3
E             Use -v to get the full diff

dask/array/utils.py:99: AssertionError

For arrays, it's a bit more (too?) verbose:
        try:
            assert a.shape == b.shape
>           assert allclose(a, b, **kwargs)
E           assert False
E            +  where False = allclose(array([[ 0.,  0.,  0.],\n       [ 0.,  0.,  0.],\n       [ 0.,  0.,  0.],\n       [ 0.,  0.,  0.]]), array([[ 1.,  1.,  1.],\n       [ 1.,  1.,  1.],\n       [ 1.,  1.,  1.],\n       [ 1.,  1.,  1.]]), **{})

dask/array/utils.py:100: AssertionError

although if I change assert_eq to use numpy.testing.assert_allclose, the output looks like:
E           AssertionError:
E           Not equal to tolerance rtol=1e-07, atol=0
E
E           (mismatch 100.0%)
E            x: array([[ 0.,  0.,  0.],
E                  [ 0.,  0.,  0.],
E                  [ 0.,  0.,  0.],
E                  [ 0.,  0.,  0.]])
E            y: array([[ 1.,  1.,  1.],
E                  [ 1.,  1.,  1.],
E                  [ 1.,  1.,  1.],
E                  [ 1.,  1.,  1.]])

No strong thoughts from me

Maybe we'll hold off then. I've had seemingly innocuous changes to pandas' conftest negatively affect downstream projects.

authors: TomAugspurger mrocklin


text:
[WIP] Do FS operations on remote workers

For discussion.
Solves #5380 in combination with https://github.com/intake/filesystem_spec/pull/244
For a start, only parquet is affected.
Parquet tests here pass with the fsspec branch above (not with the env in CI); however, the tests only use the local scheduler, so they only show that the "pass through" part works.

I don't think this is how we should go about doing this. Certain file operations (e.g. reading parquet metadata) may do lots of IO across lots of files. It seems to me that it would be better to think critically about what operations are done at graph-build-time and wrap those as tasks to run on the workers. This reduces bandwidth costs and exposes more opportunities for parallelizing IO and metadata inference (which users have complained about being slow).
For example, say we have some io thing (pseudocode):
def read_foo(path):
    fs = infer_fs_from_path(path)
    metadata = read_metadata(fs, path)
    return build_graph_for_metadata_path(fs, metdata, path)

instead of moving the io to the workers (changing only the infer_fs_from_path to use the dask filesystem instead), I think we should keep the raw filesystem and write this like:
def read_foo(path):
    fs = infer_fs_from_path(path)
    metadata = run_on_worker(read_metadata, args=(fs, path))
    return build_graph_for_metadata_path(fs, metdata, path)

I am not convinced that this is true: certainly for parquet metadata, the thrift binary representation is about as compact as you would get, so you would serialise and deserialise on the worker and be no better off. Note that the I from workers can already be parallelised across workers, with a little rewriting, but to move the metadata inference and/or graph build to the workers entirely would probably need a lot of work.

OK, so I'm going to change things in fsspec, such that this PR won't be necessary at all, you'll simply be able to supply a URL like "dask::s3://bucket/file" to read via a worker.
This solves the specific problem "how can I work on files that can be seen by workers and not the client".
Other broader topics that can be thought about, from the conversation above:

parallelising IO operations in the case that graph building requires access to multiple files (I think this is only the many parquet, no _metadata and yes statistics case)
parsing the bytes on workers to produce objects to send to the client rather than the bytes themselves
moving the whole graph-build process to the workers; either sending the graph back to the client or just the future representing the output

authors: jcrist martindurant


text:
find()/glob() with detail=True return a dict instead of a list (of dicts)

Weird I didn't find this before  - tested on 2021.04.0:
ls, find and glob are supposedly returning a list of either strings or dicts (in case detail=True).
However with detail=True, find() and glob() return a dict of dicts.
Found that out when switching from ls to glob,
sorted(files, key=lambda f: f['updated'])[-1] did not work anymore since the returned files is not a list anymore.

Hm, it seems to have been this way for quite some time! I don't know why the decision was this way.

Not sure it was intentional as the documentation says list either way.

xref https://github.com/intake/filesystem_spec/pull/814 (about what the dict keys are called)

repo: dask/gcsfs


text:
Issue from website page Get signals

From URL: https://www.daskeyboard.io/api-ressources/signal/get-signals.md
There is no documentation for using the localhost API. The page says Q Cloud Only but below that has a deprecation notice saying to use the localhost API or Q Applet API.
I have tried calling /api/1.0/signals and /signals and both resources return the following message:
Cannot GET /api/1.0/signals

Hi @rideron89 ,
https://www.daskeyboard.io/api-resources/signal/get-signals/
This route can be used only with the cloud API, which is now deprecated but still works.
To get the signals from the local API I advice you to use this endpoint instead:
https://www.daskeyboard.io/api-resources/signal/get-all-shadow-signals/

Do not hesitate to contact our support if any further help is needed.
Closing.

authors: Mubramaj fabiendv rideron89


text:
add: SQLAlchemy ORM models

related to #3

How to Docs
Regarding writing comments, there're many guidelines, but there exists no "perfect" way of writing docs.
When to write docs
Generally, it's a good guideline to document things that're unique to a codebase.
For instance, you wouldn't want to explain all details about how a database or querying works, every time you write a SQLAlchemy query.
This will be done dozens of times in your codebase and it would just be repetetive.
However, if you find yourself writing a workaround hack on a very specific query, this is definitely something you should document, as you'll probably not do this again and nobody will understand why you did this.
I think there're a few questions you can ask yourself, whether to write comments:
Contra comment:

Some logic you use in your codebase all the time.
For stuff like this, there exists docs in the libraries you're using.
Example:
- Querying stuff from the database. SQLAlchemy explains this stuff
- How to build Models -> SQLAlchemy docs
- What is the app.get thinky above routes -> FastAPI explains this stuff
Short, simple and straight-forward logic

Pro comment:

Some logic that's unique to the codebase.
Complex logic that's hard to understand.
Arcane implicit logic that's not obvious when reading.
Simple logic, which needs some very specific knowledge.
In these cases, I usually write a comment that links to the exact external documentation one needs to understand the code.
Example:
session = scoped_session(sessionmaker(bind=engine)).
This line looks innocent, but is packed with stuff you have to know.

How to structure docs
I usually suggest the following:
Try to write a headline and text that answer the following questions in your documentation.
For Classes

Headline: What does this class represent?
Something in the real world? A person, an animal, An abstract thing
Body: What is this class used for? Why is it needed?
Keep this short, there's no need to explain everything that can be done with a class.

For functions
The Python guideline tells us to write docs for every function.

Headline: What's the most basic description for this function.
Quite often, this is just the name of the function itself, but in normal text :D
I.e. get_user -> Get a user from our database

Body: This is usually only needed if the function does complex logic.
Skip if the function contains stuff that is straight forward (such as "get a user").
Otherwise, go ahead and write what part of the logic is not straight forward.
Why does it have to be complex, obfuscated or weird?
Sometimes, functions are very simple and only contain two lines of code.
However, there's looads of stuff happening in those two lines of code that really aren't obvious.
In that case, try to explain what happens in there or link some external docs that explain it.

Inline documentation
For inline documentation, I tend to overdo it a bit and this is very subjective :D.
I try to chunk each code into logical pieces.
Each chunk then get's a small explanation of what happens inside of it.
The following is a small example of a function with a few inline comments:
@job_wrapper
def perma_ban_checker(context: CallbackContext, session: scoped_session) -> None:
    """Perma-ban people that send more than 250 votes for at least 3 days in the last week."""
    vote_limit = config["telegram"]["max_user_votes_per_day"]
    stats = (
        session.query(UserStatistic)
        .filter(UserStatistic.votes >= vote_limit)
        .filter(UserStatistic.date == date.today())
        .all()
    )

    for stat in stats:
        # Check how often the user reached the limit in the last week
        days_above_limit = (
            session.query(UserStatistic)
            .filter(UserStatistic.votes >= vote_limit)
            .filter(UserStatistic.date >= date.today() - timedelta(days=6))
            .filter(UserStatistic.date <= date.today() - timedelta(days=1))
            .filter(UserStatistic.user == stat.user)
            .all()
        )

        # If the user reached the limit on two other days in the last week (three days in total)
        if len(days_above_limit) >= 2:
            stat.user.banned = True

As you can see, I even went ahead and wrote some inline docs between each call, since the calls are quite complex and do quite a bit of stuff.
The strings simply explain in natural language what it exactly is what these calls do.
This allows developers to understand the intended logic and then verify that the code indeed does what's described in the docs.
Otherwise developers would have to read to code to understand what happens, without any context!
This also helps a lot when navigating the code.
You don't have to actually read code to get to the place you want to go.
You rather have to read natural language docs until you reach the place you're interested in.

After a lot of reading, I see two solutions on how to do session handling :D
Via a library:
https://github.com/mfreeborn/fastapi-sqlalchemy
The main (potential) problem I see with this library is a isolated database test setup? Though I assume this might still be possible.
Manually:
https://github.com/tiangolo/fastapi/issues/726#issuecomment-557687526
Pretty raw approach, but it works with fastapi's dependencies and is pretty straight-forward.

https://docs.sqlalchemy.org/en/14/orm/query.html#sqlalchemy.orm.Query.get

Regarding doc strings on Schemas.
I'm not sure if it's necessary to write docs for every schema class there is? It feels like it's enough to document the database model, since this is the core class for each entity we'll work with.
-> I would be ok, if you just skip the docstrings on the schema classes :). But feel free to do so anyway, if you think that this is cleaner.

authors: Nukesor dasmerlon

repo: dasmerlon/tief-gang-list


text:
Add note to negative for signed integer dtype behavior

This PR

adds a note to negative concerning the behavior of computing the numerical negative of the minimum representable integer value for arrays having a signed integer data type. Namely, that behavior is implementation-dependent. This is similar to gh-250.

This was discussed in gh-250.

repo: data-apis/array-api


text:
Integration tests executed on a real deployment as part of the CICD - Datasets

Same as for https://github.com/data-dot-all/dataall/issues/1220.
This issue is to track the progress for the Datasets modules.
It has its own dedicated issue because of the challenge of pre-existing infrastructure needed to test datasets

Required tests for basic coverage
For fresh deployments
For each of the following API calls we need to test authorized and unauthorized scenarios as well as all possible configurations (e.g. autoapproval...)

[ ] Create Dataset - includes testing if dataset is indexed in Catalog
[ ] Import Dataset - includes testing if dataset is indexed in Catalog
[ ] List Datasets
[ ] Edit Dataset
[ ] Delete dataset
[ ] Start crawler
[ ] Sync tables
[ ] Preview table
[ ] Delete table
[ ] Create folder
[ ] Delete folder

For backwards compatibility

[ ] Update created Dataset stack
[ ] Update imported Dataset stack
[ ] Create Dataset in updated Environment
[ ] Import Dataset in updated Environment

Full coverage [WIP]
For fresh deployments

[ ] AWS access to Dataset - Credentials
[ ] AWS access to Dataset - S3 redirect
[ ] Upload data
[ ] Profile table

For backwards compatibility
For the updated Dataset stacks:

[ ] AWS access to Dataset - Credentials
[ ] AWS access to Dataset - S3 redirect
[ ] Start crawler
[ ] Sync tables

repo: data-dot-all/dataall


text:
fix adls handler

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Wrangler 4.1.0-SNAPSHOT ............................ SUCCESS [  2.728 s]
[INFO] Wrangler API ....................................... SUCCESS [  5.368 s]
[INFO] Wrangler REST Protocol Classes ..................... SUCCESS [  0.852 s]
[INFO] Wrangler Core ...................................... SUCCESS [ 20.946 s]
[INFO] Wrangler Storage ................................... SUCCESS [  1.863 s]
[INFO] Wrangler Service ................................... SUCCESS [ 40.545 s]
[INFO] Wrangler Testing Framework ......................... SUCCESS [  0.467 s]
[INFO] Wrangler Transform 4.1.0-SNAPSHOT .................. SUCCESS [ 21.791 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:36 min
[INFO] Finished at: 2019-05-25T09:51:43-07:00
[INFO] ------------------------------------------------------------------------

LGTM 👍

squashing and merging.

authors: CuriousVini nitinmotgi

repo: data-integrations/wrangler


text:
Add convenience to creating a downloader job for a single batch.

As per @mhuyck's comment here the DownloaderJob model can take a batch as an optional argument and add it to the DownloaderJobsToBatches table.

@mhuyck I've come up with the following function to do handle this issue. I wasn't able to add it as part of the __init__ function like I had originally thought I might because the DownloaderJob has to be saved before it can be saved as a foreign key to the DownloaderJobsToBatches table. Do you think this is about the best that can be done or do you know of any helpful tricks. (Maybe @rzelayafavila or @dongbohu might also have good input?)
    @classmethod
    def create_job_and_relationships(*args, **kwargs):
        """Inits and saves a job, its batches, and the relationships.

        Creates and saves a single DownloaderJob. For each batch
        passed in via the 'batches' keyword, saves that batch, creates
        a DownloaderJobsToBatches record, and saves that record.
        """
        batches = kwargs.pop('batches', None)
        this_job = super().__init__(*args, **kwargs)
        if batches is None:
            raise KeyError("The 'batches' argument must be specified.")
        else:
            this_job.save()
            for batch in batches:
                batch.save()
                downloader_job_to_batch = DownloaderJobsToBatches(batch=batch,
                                                                  downloader_job=this_job)
                downloader_job_to_batch.save()

👍
I don't know any special tricks for making __init__ do what we thought. It makes sense that you'd need to have the .save done first so there's a bit of an initialization chicken and egg issue there.
My primary observation during that code review was that a one-line object initialization was getting replaced with a template-like block of code in a bunch of places, so if this helper function can streamline that you'll end up with more DRY code and all the bounty that comes with that.

Yup I definitely feel that for sure.

Addressed by https://github.com/data-refinery/data_refinery/pull/26

authors: kurtwheeler mhuyck


text:
unit test build error for mac

Bug Report

error: This macro cannot be used on the current target.
You can prevent it from being used in other architectures by
guarding it behind a cfg(any(target_arch = "x86", target_arch = "x86_64")).
--> parser/mysql/src/lex.rs:675:16
|
675 |             if is_x86_feature_detected!("sse2") {
|                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: this error originates in the macro is_x86_feature_detected (in Nightly builds, run with -Z macro-backtrace for more info)
What version of Pisanix are you using?

What operating system and CPU are you using?

Steps to reproduce

What did you expect?
What did happened?

Thanks feeback, I will check it.

It seems like cfg!(any(target_arch = "x86", target_arch = "x86_64"))  didn't exclude the target arch, I updated it with a general macro  #[cfg(any(target_arch = "x86", target_arch = "x86_64"))] and compiled successfully.

@zhang-arvin Is this issue still reproducible ? This issue will be closed if it is fixed.

authors: mlycore xuanyuan300 zhang-arvin

repo: database-mesh/pisanix


text:
Thrift file missing

Hello,
I'm trying to create a .net sql connector but the Thrift file is missing.
Could you provide it please

Hi @aminebizid, we don't officially support .Net for Databricks thrift. Have you tried to use the open source version?
https://thrift.apache.org/lib/netstd.html

Hi @andrefurlan-db
Thanks for your response
I'm using https://thrift.apache.org/lib/netstd.html but I just need the .thrift file you are using to generate the code

closing this issue as we can't provide the file yet. Next year we'll probably work on a .Net connector for Databricks.

authors: aminebizid andrefurlan-db

repo: databricks/databricks-sql-go


text:
SQL Error: ERROR:  operator is not unique: smallint[] @> smallint[]

Hello, I'm getting this issue while trying to compact my table:

[Tue Feb 27 11:17:51 2018] (mydb:public.call_project_cases) SQL Error: ERROR:  operator is not unique: smallint[] @> smallint[]
LINE 26: ...SELECT string_to_array(indkey::text, ' ')::int2[] @> array[0...
^
HINT:  Could not choose a best candidate operator. You might need to add explicit type casts.
[Tue Feb 27 11:17:51 2018] (mydb:public.call_project_cases) Table handling interrupt.

Is there anything I can do to get rid of it?
postgresql-9.6      9.6.7-1.pgdg16.04+1 amd64
Ubuntu 16.04.3 LTS

Hello, are you using intarray extension?

Yes, we do

ok, I'll push a fix for this in an hour or two.

Thanks!

Could you please also explain what means next entries:
WARNING:  relation "mobile_phones" page 164340 is uninitialized --- fixing
WARNING:  relation "mobile_phones" page 164341 is uninitialized --- fixing
WARNING:  relation "mobile_phones" page 164342 is uninitialized --- fixing

please check if latest version has this problem.

Could you please also explain what means next entries:
most likely DB crashed during table extension leaving uninitialized pages at the end of table. may be it happened because there was no free space.

Works great, thanks a lot.

most likely DB crashed during table extension leaving uninitialized pages at the end of table

Can you recommend any tool to periodically use for checking against such pages?

Hello
Uninitialized pages are handled postgresql itself during vacuum or autovacuum. Here is no necessary further checking.

authors: Melkij alexius2 eskornev

repo: dataegret/pgcompacttable


text:
Correction for raw affiliation Division of Molecular Immunology, MRC, National Institute for Medical Research, London NW7 1AA, UK.

Correction needed for raw affiliation Division of Molecular Immunology, MRC, National Institute for Medical Research, London NW7 1AA, UK.
raw_affiliation_name: Division of Molecular Immunology, MRC, National Institute for Medical Research, London NW7 1AA, UK.
new_rors:
previous_rors: 04tnbqb63
works_examples: W2085871375;W2132408817
contact: bccd6fbe6816ac9e8f87a2d17ad1dcf6:42610a98580fc85727d144acadac1bbd6854136d @ crick.ac.uk

This issue was accepted and ingested by the OpenAlex team on 2024-10-10. The new affiliations should be visible within the next 7 days.

repo: dataesr/openalex-affiliations


text:
Can't be included in dylib crate type

Tested on Mac OS Sierra. Including datafusion as a dependency in a crate with
[lib]
crate-type = ["dylib"]

Causes the linker to fail:
error: linking with `cc` failed: exit code: 1
  |
  = note: "cc" "-m64" "-L" "/Users/jakebecker/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib" [really long command omitted]

  = note: duplicate symbol _XXH64_digest in:
              /var/folders/ff/stmyfx1s5yqg_0pl1fq6cft80000gn/T/rustcfOjx4t/libzstd_sys-7acf92beb8282696.rlib(xxhash.o)
              /var/folders/ff/stmyfx1s5yqg_0pl1fq6cft80000gn/T/rustcfOjx4t/liblz4_sys-e1d5bb1cebde788e.rlib(xxhash.o)
          duplicate symbol _XXH32_digest in:
              /var/folders/ff/stmyfx1s5yqg_0pl1fq6cft80000gn/T/rustcfOjx4t/libzstd_sys-7acf92beb8282696.rlib(xxhash.o)
              /var/folders/ff/stmyfx1s5yqg_0pl1fq6cft80000gn/T/rustcfOjx4t/liblz4_sys-e1d5bb1cebde788e.rlib(xxhash.o)
          duplicate symbol _XXH64_reset in:
              /var/folders/ff/stmyfx1s5yqg_0pl1fq6cft80000gn/T/rustcfOjx4t/libzstd_sys-7acf92beb8282696.rlib(xxhash.o)
              /var/folders/ff/stmyfx1s5yqg_0pl1fq6cft80000gn/T/rustcfOjx4t/liblz4_sys-e1d5bb1cebde788e.rlib(xxhash.o)
          duplicate symbol _XXH32_reset in:
              /var/folders/ff/stmyfx1s5yqg_0pl1fq6cft80000gn/T/rustcfOjx4t/libzstd_sys-7acf92beb8282696.rlib(xxhash.o)
              /var/folders/ff/stmyfx1s5yqg_0pl1fq6cft80000gn/T/rustcfOjx4t/liblz4_sys-e1d5bb1cebde788e.rlib(xxhash.o)
          duplicate symbol _XXH_versionNumber in:
              /var/folders/ff/stmyfx1s5yqg_0pl1fq6cft80000gn/T/rustcfOjx4t/libzstd_sys-7acf92beb8282696.rlib(xxhash.o)
              /var/folders/ff/stmyfx1s5yqg_0pl1fq6cft80000gn/T/rustcfOjx4t/liblz4_sys-e1d5bb1cebde788e.rlib(xxhash.o)
          duplicate symbol _XXH64_hashFromCanonical in:
              /var/folders/ff/stmyfx1s5yqg_0pl1fq6cft80000gn/T/rustcfOjx4t/libzstd_sys-7acf92beb8282696.rlib(xxhash.o)
              /var/folders/ff/stmyfx1s5yqg_0pl1fq6cft80000gn/T/rustcfOjx4t/liblz4_sys-e1d5bb1cebde788e.rlib(xxhash.o)
          duplicate symbol _XXH32_hashFromCanonical in:
              /var/folders/ff/stmyfx1s5yqg_0pl1fq6cft80000gn/T/rustcfOjx4t/libzstd_sys-7acf92beb8282696.rlib(xxhash.o)
              /var/folders/ff/stmyfx1s5yqg_0pl1fq6cft80000gn/T/rustcfOjx4t/liblz4_sys-e1d5bb1cebde788e.rlib(xxhash.o)
          duplicate symbol _XXH64_canonicalFromHash in:
              /var/folders/ff/stmyfx1s5yqg_0pl1fq6cft80000gn/T/rustcfOjx4t/libzstd_sys-7acf92beb8282696.rlib(xxhash.o)
              /var/folders/ff/stmyfx1s5yqg_0pl1fq6cft80000gn/T/rustcfOjx4t/liblz4_sys-e1d5bb1cebde788e.rlib(xxhash.o)
          duplicate symbol _XXH32_canonicalFromHash in:
              /var/folders/ff/stmyfx1s5yqg_0pl1fq6cft80000gn/T/rustcfOjx4t/libzstd_sys-7acf92beb8282696.rlib(xxhash.o)
              /var/folders/ff/stmyfx1s5yqg_0pl1fq6cft80000gn/T/rustcfOjx4t/liblz4_sys-e1d5bb1cebde788e.rlib(xxhash.o)
          duplicate symbol _XXH64_copyState in:
              /var/folders/ff/stmyfx1s5yqg_0pl1fq6cft80000gn/T/rustcfOjx4t/libzstd_sys-7acf92beb8282696.rlib(xxhash.o)
              /var/folders/ff/stmyfx1s5yqg_0pl1fq6cft80000gn/T/rustcfOjx4t/liblz4_sys-e1d5bb1cebde788e.rlib(xxhash.o)
          duplicate symbol _XXH32_copyState in:
              /var/folders/ff/stmyfx1s5yqg_0pl1fq6cft80000gn/T/rustcfOjx4t/libzstd_sys-7acf92beb8282696.rlib(xxhash.o)
              /var/folders/ff/stmyfx1s5yqg_0pl1fq6cft80000gn/T/rustcfOjx4t/liblz4_sys-e1d5bb1cebde788e.rlib(xxhash.o)
          duplicate symbol _XXH64_createState in:
              /var/folders/ff/stmyfx1s5yqg_0pl1fq6cft80000gn/T/rustcfOjx4t/libzstd_sys-7acf92beb8282696.rlib(xxhash.o)
              /var/folders/ff/stmyfx1s5yqg_0pl1fq6cft80000gn/T/rustcfOjx4t/liblz4_sys-e1d5bb1cebde788e.rlib(xxhash.o)
          duplicate symbol _XXH32_createState in:
              /var/folders/ff/stmyfx1s5yqg_0pl1fq6cft80000gn/T/rustcfOjx4t/libzstd_sys-7acf92beb8282696.rlib(xxhash.o)
              /var/folders/ff/stmyfx1s5yqg_0pl1fq6cft80000gn/T/rustcfOjx4t/liblz4_sys-e1d5bb1cebde788e.rlib(xxhash.o)
          duplicate symbol _XXH64_freeState in:
              /var/folders/ff/stmyfx1s5yqg_0pl1fq6cft80000gn/T/rustcfOjx4t/libzstd_sys-7acf92beb8282696.rlib(xxhash.o)
              /var/folders/ff/stmyfx1s5yqg_0pl1fq6cft80000gn/T/rustcfOjx4t/liblz4_sys-e1d5bb1cebde788e.rlib(xxhash.o)
          duplicate symbol _XXH32_freeState in:
              /var/folders/ff/stmyfx1s5yqg_0pl1fq6cft80000gn/T/rustcfOjx4t/libzstd_sys-7acf92beb8282696.rlib(xxhash.o)
              /var/folders/ff/stmyfx1s5yqg_0pl1fq6cft80000gn/T/rustcfOjx4t/liblz4_sys-e1d5bb1cebde788e.rlib(xxhash.o)
          duplicate symbol _XXH64_update in:
              /var/folders/ff/stmyfx1s5yqg_0pl1fq6cft80000gn/T/rustcfOjx4t/libzstd_sys-7acf92beb8282696.rlib(xxhash.o)
              /var/folders/ff/stmyfx1s5yqg_0pl1fq6cft80000gn/T/rustcfOjx4t/liblz4_sys-e1d5bb1cebde788e.rlib(xxhash.o)
          duplicate symbol _XXH32_update in:
              /var/folders/ff/stmyfx1s5yqg_0pl1fq6cft80000gn/T/rustcfOjx4t/libzstd_sys-7acf92beb8282696.rlib(xxhash.o)
              /var/folders/ff/stmyfx1s5yqg_0pl1fq6cft80000gn/T/rustcfOjx4t/liblz4_sys-e1d5bb1cebde788e.rlib(xxhash.o)
          duplicate symbol _XXH64 in:
              /var/folders/ff/stmyfx1s5yqg_0pl1fq6cft80000gn/T/rustcfOjx4t/libzstd_sys-7acf92beb8282696.rlib(xxhash.o)
              /var/folders/ff/stmyfx1s5yqg_0pl1fq6cft80000gn/T/rustcfOjx4t/liblz4_sys-e1d5bb1cebde788e.rlib(xxhash.o)
          duplicate symbol _XXH32 in:
              /var/folders/ff/stmyfx1s5yqg_0pl1fq6cft80000gn/T/rustcfOjx4t/libzstd_sys-7acf92beb8282696.rlib(xxhash.o)
              /var/folders/ff/stmyfx1s5yqg_0pl1fq6cft80000gn/T/rustcfOjx4t/liblz4_sys-e1d5bb1cebde788e.rlib(xxhash.o)
          ld: 19 duplicate symbols for architecture x86_64
          clang: error: linker command failed with exit code 1 (use -v to see invocation)

It appears that both libzstd and liblz4 include the C library xxhash, and this causes the duplicate symbol error. I'd like to write bindings for Elixir, and this requires that the crate be compilable as a dylib. I'm a novice at Rust but I'll attempt to find a fix.

I filed the issue on parquet-rs, since it's the library including both those dependencies: https://github.com/sunchao/parquet-rs/issues/145

repo: datafusion-rs/datafusion


text:
Pages équivalents : inverser les infographies et le graphique

Retour de Laura :

Pour expliquer la valeur en KgCO2e, on pourrait inverser les infographies et le graphique avec les détails -> visuel plus clair que du texte et complémentaire au textes pour expliquer l'impact
De + , les infographies sont coupées et passent sous la ligne de flottaison alors que le graphe pourrait être en entier sur la page

Je ne sais pas trop ce que ça rendrait mais ça mérite d'être testé ?

Je pense que ça mérite d'être discuté. C'était comme ça de base (ECV avant les infographies) et ça avait été changé suite au retour de Jérémie

authors: Clem-mrtn florianpanchout


text:
Vérifier l'accessibilité de la v2

Reprendre l'audit des autres simulateurs pour s'assurer d'un minimum de 50 % sur celui-ci

Done