text:
possible concurrent modification error in finalizeExports

@jakemac53 run into this exception, which seems related to the current support for exports:
Concurrent modification during iteration: _LinkedHashMap len:121.
#0      _CompactIterator.moveNext (dart:collection-patch/compact_hash.dart:431)
#1      _HashVMBase&MapMixin&&_LinkedHashMapMixin.forEach (dart:collection-patch/compact_hash.dart:354)
#2      DillLoader.finalizeExports (package:front_end/src/fasta/dill/dill_loader.dart:62)
#3      DillTarget.buildOutlines (package:front_end/src/fasta/dill/dill_target.dart:54)
<asynchronous suspension>
#4      generateKernelInternal (package:front_end/src/kernel_generator_impl.dart:91)
<asynchronous suspension>
#5      generateKernel.<anonymous closure> (package:front_end/src/kernel_generator_impl.dart:31)
<asynchronous suspension>
#6      CompilerContext.runInContext.<anonymous closure> (package:front_end/src/fasta/compiler_context.dart:89)
#7      _rootRun (dart:async/zone.dart:1120)
#8      _CustomZone.run (dart:async/zone.dart:1001)
#9      runZoned (dart:async/zone.dart:1467)
#10     CompilerContext.runInContext (package:front_end/src/fasta/compiler_context.dart:89)
#11     CompilerContext.runWithOptions (package:front_end/src/fasta/compiler_context.dart:96)
#12     generateKernel (package:front_end/src/kernel_generator_impl.dart:30)
<asynchronous suspension>
#13     summaryFor (package:front_end/summary_generator.dart:40)
<asynchronous suspension>

I know this is currently in flux, but it's possible that this issue exists both in the old and new version of the code. @jakemac53 - what commit of the sdk did you use when you got this error?
I don't have at this time a small repro, but I can work on one if we need one.

/cc @peter-ahe-google @scheglov who both have been doing work recently in this part of the code

Yes, I saw this too.
One of the motives for 234c4a7f493e1fc3dafc393472e39163ac2ad783 was to fix this.

I think this is the problem with the tree-shaker that Konstantin mentioned in #30448.
I can't see any other way of getting a concurrent modification to DillLoader.builders.

@jakemac53 is running this in an environment without the tree-shaker though.

I'll try to follow up with Jake to create a repro, I'm reopening while we investigate

I think this happens when there is reexport. Not 100% sure.
You reexport something reexported from somewhere else.
A declares a.dart, b.dart exports a.dart, c.dart exports b.dart, and there d.dart imports c.dart.
So, we attempt to add new builders while processing builders.

@scheglov I also think that is what is happening. What I can't figure out is why we add a new builder.
This happens in DillLoader, and it happens at a point where all builders should have been created.
If we can't get a repro, I have some ideas for some instrumentation I can add. However, it is also possible that it is fixed now that we've landed Konstantin's export changes again. If that's the case, my only regret is that we may not get a regression test.

The issue triggers when trying to do an export from one .dill file of a library that comes from a different .dill file. Turns out you can trigger this even without a double export.
Here is the repro:
import 'package:async_helper/async_helper.dart' show asyncTest;

import 'package:front_end/src/testing/compiler_common.dart';
import 'package:front_end/front_end.dart';

_onError(e) => (e) => throw '${e.severity}: ${e.message}';

main() {
  asyncTest(() async {
    var sources = <String, dynamic>{
      'a.dart': 'class A  {}',
    };
    var summary = await summarize(['a.dart'], sources,
        options: new CompilerOptions()..onError = _onError);

    sources = <String, dynamic>{
      'c.dart': 'export "a.dart";',
      'x.dill': summary,
    };

    summary = await summarize(['c.dart'], sources,
        inputSummaries: ['x.dill'],
        options: new CompilerOptions()..onError = _onError);

    sources = <String, dynamic>{
      'd.dart': 'import "c.dart"; class B extends A {}',
      'y.dill': summary,
    };

    await compileUnit(['d.dart'], sources,
        inputSummaries: ['y.dill'],
        options: new CompilerOptions()..onError = _onError);
  });
}

I see this failing both before and after @scheglov changes.

This CL diagnoses the internal problem that leads to this CL 3010963002.

friendly ping, still seeing this issue on 2.0.0-dev.4.0

I think the root cause of this is that the source loader may attempt to modify its map of builders when reading a dill-backed library it hasn't seen before.
This is currently number three on my personal task list.

I'm on vacation next week, but this it now at the top of my task list.

Great, thanks for the update!

Any update on this?  My understanding is that the bazel plumbing for kernel (and thus ddc-kernel) is blocked on this.

Sorry, I had to extend my vacation for another week. Good news is that I've been thinking about how to solve the problem while away on vacation, and I think I know how to fix it.

The problem I've been thinking about during my vacation was unrelated.
The problem in this bug report is that a.dart is marked external as of 2289ce12aeb. That implies that you need to load a.dill as well to provide the implementation of it.
So this program works:
import 'package:async_helper/async_helper.dart' show asyncTest;

import 'package:front_end/src/testing/compiler_common.dart';
import 'package:front_end/front_end.dart';

_onError(e) => (e) => throw '${e.severity}: ${e.message}';

main() {
  asyncTest(() async {
    var sources = <String, dynamic>{
      'a.dart': 'class A  {}',
    };
    var aSummary = await summarize(['a.dart'], sources,
        options: new CompilerOptions()
          ..verbose = true
          ..onError = _onError);

    sources = <String, dynamic>{
      'b.dart': 'export "a.dart";',
      'a.dill': aSummary,
    };

    var bSummary = await summarize(['b.dart'], sources,
        inputSummaries: ['a.dill'],
        options: new CompilerOptions()
          ..verbose = true
          ..onError = _onError);

    sources = <String, dynamic>{
      'c.dart': 'import "b.dart"; class B extends A {}',
      'a.dill': aSummary,
      'b.dill': bSummary,
    };

    await compileUnit(['c.dart'], sources,
        inputSummaries: [

Key difference:
          'a.dill',

          'b.dill',
        ],
        options: new CompilerOptions()
          ..verbose = true
          ..onError = _onError);
  });
}

I think the resolution of this bug is outside the front-end. I think bazel needs to provide all summaries, not just the result of the previous compilation.

I have tried supplying all transitive summaries in bazel as well, and ran into the same issue.

Here is a CL that avoids concurrent modifications and throws an exception with a bit more information: https://dart-review.googlesource.com/22620

Apologies, I actually had a bug in the code that was supposed to provide transitive summaries so it wasn't, I retried with a fix and now it seems to work, thanks!

No need to apologize. My only regret is that I confused this with a more obscure concurrent modification error instead of trying the repro directly. Hopefully, the internal errors of the above CL will make it easier to debug another time.

authors: jakemac53 peter-ahe-google scheglov sigmundch vsmenon


text:
tracking issue for dartanalyzer CLI test failures with --preview-dart-2

ExitCodesTest_UseCFE

[ ] test_enableAssertInitializer
[ ] test_fatalErrors
[ ] test_fatalHints
[ ] test_fatalWarnings
[ ] test_missingOptionsFile
[ ] test_notFatalHints
[ ] test_notFatalWarnings
[ ] test_partFile
[ ] test_partFile_extra

LinterTest_UseCFE

[ ] test_defaultLints_generatedLints
[ ] test_defaultLints_getsDefaultLints
[ ] test_lintsInOptions_generatedLints
[ ] test_lintsInOptions_getAnalysisOptions
[ ] test_noLints_lintsDisabled
[ ] test_noLints_noGeneratedWarnings
[ ] test_noLints_noRegisteredLints

OptionsTest

[ ] test_useCFE

OptionsTest_UseCFE

[ ] test_analysisOptions_excludes
[ ] test_analysisOptions_excludesRelativeToAnalysisOptions_explicit
[ ] test_analysisOptions_excludesRelativeToAnalysisOptions_inferred
[ ] test_analyzeFilesInDifferentContexts
[ ] test_basic_filters
[ ] test_basic_language
[ ] test_basic_strongMode
[ ] test_includeDirective
[ ] test_previewDart2
[ ] test_strongSdk
[ ] test_todo
[ ] test_withFlags_overrideFatalWarning

out of date

updated

more updates

The CFE integration work is suspended, so this will not be addressed.

source: gharchive/issue

authors: bwilkerson devoncarew

repo: dart-lang/sdk

license_type: permissive

license_source: github-api


text:
Different behavior of tear-off in VM vs JS

When doing tear-off methods in the JS it returns the same reference each time (identical == true), but in VM it does return different one each time.
Code: https://gist.github.com/rkj/a69f9b9b9cdc9349a6ca5f1570fe0b68
VM:
$ dart tearoff.dart
true
false

Web: https://imgur.com/a/ElZC0Kz

I believe this is working as intended.
The specification (section "16.22.3 Ordinary Member Closurization") requires that == is true in your example:

Assume that o1 and o2 are objects, m is an identifier, and c1 and c2 are function objects obtained by closurization of m on o1 respectively o2. Then c1 == c2 evaluates to true if and only if o1 and o2 is the same object.

Later it also clarifies that identical is not guaranteed to be true, but it allows some implementations to return true if they want:

In particular, two closurizations of a method m from the same object are equal, and two closurizations of a method m from non-identical objects are not equal. Assuming that vi is a fresh variable bound to an object, i ∈ 1..2, it also follows that identical(v1.m, v2.m) must be false when v1 and v2 are not bound to the same object. However, Dart implementations are not required to canonicalize function objects, which means that identical(v1.m, v2.m) is not guaranteed to be true, even when it is known that v1 and v2 are bound to the same object.

I've tagged this under area-specification in case some of the language team members disagree and would like to reopen.

FYI (and for my recollection): this is also different in Dart2js vs DDC, leads to tests & code behaving differently in those two envs. Would be awesome if at least they could be kept in sync.

Since the result of identical(a, b) is unspecified, dart2js will optimize two calls to a tear-off getter (i.e. foo._foo), reusing the first one.
If tearOff is annotated to not be inlined, then there are two tearoff closure objects (created at twice the cost):
class Foo {
  void _foo() => null;
  @pragma('dart2js:noInline')
  void Function() get tearFoo => _foo;
 }

void main() {
  var foo = Foo();
  var a = foo.tearFoo;
  var b = foo.tearFoo;
  print(a == b);
  print(identical(a, b));
}

This program prints
true
false

It is not going to be possible to make DDC have this optimization. It depends on exactly the same inlining to bring together the two calls to foo._foo.
The question is: should dart2js be changed to avoid making the program smaller and faster within the specification because it is detectable?

I think we should keep the current specification and the current behavior as it is (and I agree that there is no conflict, and hence it's fine that this issue has been closed with 'working as intended').
We just need to keep saying that the correct equality test on function objects obtained by tearing off an instance method is operator ==. An optimization might make identical return the same answer, but it is not safe and not portable to rely on that optimization.
The performance difference may be significant: identical is resolved statically and could be inlined as  very few instructions when applied to expressions whose static type is, e.g., a function type, whereas operator == is a regular instance method, subject to late binding. So it may be quite useful to be able to guarantee that identical will provide the correct comparison.
However, if we wish to invest in offering this kind of performance improvement I think we should aim for optimizations where f1 == f2 is compiled as identical(f1, f2) when that's known to be correct. Developers should stick to the rule that the correct test is written using operator ==.

Thanks for explanations, it all makes sense. I agree not worth making changes to spec.
I guess it's just the case of Angular being Angular. I run into this issue while trying to figure out why UnstableExpressionError is being thrown for some benign looking code. Angular uses identical exactly for performance reasons. Given the dart2js behavior is performant I think this is not a serious issue, just annoyance in testing.

authors: eernstg rakudrama rkj sigmundch


text:
Flaky use-after-free/uninitialized-read during IsolateKillerVisitor

When I run the full test suite with ASAN and MSAN, I see a few tests with bad memory access during IsolateKillerVisitor. I can't reproduce running the failing tests individually, so there is likely a race here too.
--- Command "vm" (took 15.000160s):
DART_CONFIGURATION=ReleaseASANX64 out/ReleaseASANX64/dart --short_socket_read --ignore-unrecognized-flags --packages=/usr/local/google/home/rmacnak/dart2/sdk/.packages /usr/local/google/home/rmacnak/dart2/sdk/tests/standalone_2/io/socket_close_test.dart

exit code:
-6

stdout:
unittest-suite-wait-for-done
unittest-suite-success

stderr:
=================================================================
==92914==ERROR: AddressSanitizer: heap-use-after-free on address 0x6030000191e0 at pc 0x5602ea02a11d bp 0x7fff9b56dfb0 sp 0x7fff9b56d738
READ of size 1 at 0x6030000191e0 thread T0
    #0 0x5602ea02a11c in __interceptor_strcmp /b/s/w/ir/kitchen-workdir/llvm-project/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:433:3
    #1 0x5602ea5dfabf in dart::Dart::VmIsolateNameEquals(char const*) ../../out/ReleaseASANX64/../../runtime/vm/dart.cc:966:11
    #2 0x5602ea6d720b in dart::Isolate::IsVMInternalIsolate(dart::Isolate const*) ../../out/ReleaseASANX64/../../runtime/vm/isolate.cc:3114:10
    #3 0x5602ea6d720b in dart::IsolateVisitor::IsVMInternalIsolate(dart::Isolate*) const ../../out/ReleaseASANX64/../../runtime/vm/isolate.cc:486
    #4 0x5602ea6d720b in dart::IsolateKillerVisitor::ShouldKill(dart::Isolate*) ../../out/ReleaseASANX64/../../runtime/vm/isolate.cc:3186
    #5 0x5602ea6d720b in dart::IsolateKillerVisitor::VisitIsolate(dart::Isolate*) ../../out/ReleaseASANX64/../../runtime/vm/isolate.cc:3174
    #6 0x5602ea6d848e in std::__2::function<void (dart::Isolate*)>::operator()(dart::Isolate*) const ../../out/ReleaseASANX64/../../buildtools/linux-x64/clang/lib/clang/8.0.0/include/c++/v1/functional:1923:12
    #7 0x5602ea6d848e in dart::IsolateGroup::ForEachIsolate(std::__2::function<void (dart::Isolate*)>) ../../out/ReleaseASANX64/../../runtime/vm/isolate.cc:2418
    #8 0x5602ea6d848e in dart::Isolate::VisitIsolates(dart::IsolateVisitor*)::$_4::operator()(dart::IsolateGroup*) const ../../out/ReleaseASANX64/../../runtime/vm/isolate.cc:3027
    #9 0x5602ea6d848e in decltype(std::__2::forward<dart::Isolate::VisitIsolates(dart::IsolateVisitor*)::$_4&>(fp)(std::__2::forward<dart::IsolateGroup*>(fp0))) std::__2::__invoke<dart::Isolate::VisitIsolates(dart::IsolateVisitor*)::$_4&, dart::IsolateGroup*>(dart::Isolate::VisitIsolates(dart::IsolateVisitor*)::$_4&, dart::IsolateGroup*&&) ../../out/ReleaseASANX64/../../buildtools/linux-x64/clang/lib/clang/8.0.0/include/c++/v1/type_traits:4350
    #10 0x5602ea6d848e in void std::__2::__invoke_void_return_wrapper<void>::__call<dart::Isolate::VisitIsolates(dart::IsolateVisitor*)::$_4&, dart::IsolateGroup*>(dart::Isolate::VisitIsolates(dart::IsolateVisitor*)::$_4&, dart::IsolateGroup*&&) ../../out/ReleaseASANX64/../../buildtools/linux-x64/clang/lib/clang/8.0.0/include/c++/v1/__functional_base:349
    #11 0x5602ea6d848e in std::__2::__function::__func<dart::Isolate::VisitIsolates(dart::IsolateVisitor*)::$_4, std::__2::allocator<dart::Isolate::VisitIsolates(dart::IsolateVisitor*)::$_4>, void (dart::IsolateGroup*)>::operator()(dart::IsolateGroup*&&) ../../out/ReleaseASANX64/../../buildtools/linux-x64/clang/lib/clang/8.0.0/include/c++/v1/functional:1572
    #12 0x5602ea6d2f1f in std::__2::function<void (dart::IsolateGroup*)>::operator()(dart::IsolateGroup*) const ../../out/ReleaseASANX64/../../buildtools/linux-x64/clang/lib/clang/8.0.0/include/c++/v1/functional:1923:12
    #13 0x5602ea6d2f1f in dart::IsolateGroup::ForEach(std::__2::function<void (dart::IsolateGroup*)>) ../../out/ReleaseASANX64/../../runtime/vm/isolate.cc:442
    #14 0x5602ea6d2f1f in dart::Isolate::VisitIsolates(dart::IsolateVisitor*) ../../out/ReleaseASANX64/../../runtime/vm/isolate.cc:3026
    #15 0x5602ea6ccbe8 in dart::Isolate::KillAllIsolates(dart::Isolate::LibMsgId) ../../out/ReleaseASANX64/../../runtime/vm/isolate.cc:3197:3
    #16 0x5602ea5dd90d in dart::Dart::Cleanup() ../../out/ReleaseASANX64/../../runtime/vm/dart.cc:501:3
    #17 0x5602eb8ea4ce in Dart_Cleanup ../../out/ReleaseASANX64/../../runtime/vm/dart_api_impl.cc:1043:10
    #18 0x5602ea0f7f03 in dart::bin::main(int, char**) ../../out/ReleaseASANX64/../../runtime/bin/main.cc:1230:11
    #19 0x5602ea0f9e5a in main ../../out/ReleaseASANX64/../../runtime/bin/main.cc:1258:3
    #20 0x7fe8a07b9bba in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26bba)
    #21 0x5602ea016028 in _start (/usr/local/google/home/rmacnak/dart2/sdk/out/ReleaseASANX64/dart+0x1d56028)

0x6030000191e0 is located 0 bytes inside of 23-byte region [0x6030000191e0,0x6030000191f7)
freed by thread T17 (DartWorker) here:
    #0 0x5602ea0bdc52 in __interceptor_free /b/s/w/ir/kitchen-workdir/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cc:124:3
    #1 0x5602ea6c433b in dart::Isolate::~Isolate() ../../out/ReleaseASANX64/../../runtime/vm/isolate.cc:1329:3
    #2 0x5602ea5de225 in dart::Dart::ShutdownIsolate() ../../out/ReleaseASANX64/../../runtime/vm/dart.cc:956:3
    #3 0x5602ea6cc367 in dart::ShutdownIsolate(unsigned long) ../../out/ReleaseASANX64/../../runtime/vm/isolate.cc:2033:3
    #4 0x5602ea75689e in dart::MessageHandler::TaskCallback() ../../out/ReleaseASANX64/../../runtime/vm/message_handler.cc:516:5
    #5 0x5602eaae663f in dart::ThreadPool::Worker::Loop() ../../out/ReleaseASANX64/../../runtime/vm/thread_pool.cc:382:11
    #6 0x5602eaae6212 in dart::ThreadPool::Worker::Main(unsigned long) ../../out/ReleaseASANX64/../../runtime/vm/thread_pool.cc:434:27
    #7 0x5602ea90baae in dart::ThreadStart(void*) ../../out/ReleaseASANX64/../../runtime/vm/os_thread_linux.cc:138:5
    #8 0x7fe8a0aa0fb6 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x8fb6)

previously allocated by thread T12 (DartWorker) here:
    #0 0x5602ea0a8e00 in __interceptor_strdup /b/s/w/ir/kitchen-workdir/llvm-project/compiler-rt/lib/asan/asan_interceptors.cc:447:3
    #1 0x5602ea6c5c04 in dart::Isolate::BuildName(char const*) ../../out/ReleaseASANX64/../../runtime/vm/isolate.cc:1574:13
    #2 0x5602ea6c5c04 in dart::Isolate::InitIsolate(char const*, dart::IsolateGroup*, Dart_IsolateFlags const&, bool) ../../out/ReleaseASANX64/../../runtime/vm/isolate.cc:1395
    #3 0x5602eb8ec8fe in dart::CreateIsolate(dart::IsolateGroup*, char const*, void*, char**) ../../out/ReleaseASANX64/../../runtime/vm/dart_api_impl.cc:1094:16
    #4 0x5602eb8eda25 in Dart_CreateIsolateGroup ../../out/ReleaseASANX64/../../runtime/vm/dart_api_impl.cc:1227:7
    #5 0x5602ea0f67e7 in dart::bin::CreateIsolateGroupAndSetupHelper(bool, char const*, char const*, char const*, char const*, Dart_IsolateFlags*, void*, char**, int*) ../../out/ReleaseASANX64/../../runtime/bin/main.cc:691:15
    #6 0x5602ea0f885a in dart::bin::CreateIsolateGroupAndSetup(char const*, char const*, char const*, char const*, Dart_IsolateFlags*, void*, char**) ../../out/ReleaseASANX64/../../runtime/bin/main.cc:747:10
    #7 0x5602ea4d1c84 in dart::SpawnIsolateTask::Run() ../../out/ReleaseASANX64/../../runtime/lib/isolate.cc:159:44
    #8 0x5602eaae663f in dart::ThreadPool::Worker::Loop() ../../out/ReleaseASANX64/../../runtime/vm/thread_pool.cc:382:11
    #9 0x5602eaae6212 in dart::ThreadPool::Worker::Main(unsigned long) ../../out/ReleaseASANX64/../../runtime/vm/thread_pool.cc:434:27
    #10 0x5602ea90baae in dart::ThreadStart(void*) ../../out/ReleaseASANX64/../../runtime/vm/os_thread_linux.cc:138:5
    #11 0x7fe8a0aa0fb6 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x8fb6)

Thread T17 (DartWorker) created by T8 (DartWorker) here:
    #0 0x5602ea0a6a8d in __interceptor_pthread_create /b/s/w/ir/kitchen-workdir/llvm-project/compiler-rt/lib/asan/asan_interceptors.cc:210:3
    #1 0x5602ea90b7ef in dart::OSThread::Start(char const*, void (*)(unsigned long), unsigned long) ../../out/ReleaseASANX64/../../runtime/vm/os_thread_linux.cc:157:12
    #2 0x5602eaae4e42 in dart::ThreadPool::Worker::StartThread() ../../out/ReleaseASANX64/../../runtime/vm/thread_pool.cc:338:16
    #3 0x5602eaae4e42 in dart::ThreadPool::RunImpl(std::__2::unique_ptr<dart::ThreadPool::Task, std::__2::default_delete<dart::ThreadPool::Task> >) ../../out/ReleaseASANX64/../../runtime/vm/thread_pool.cc:69
    #4 0x5602eb0965b4 in bool dart::ThreadPool::Run<dart::ConcurrentMarkTask, dart::GCMarker*, dart::Isolate* const&, dart::PageSpace*&, dart::MarkingVisitorBase<true>*&>(dart::GCMarker*&&, dart::Isolate* const&, dart::PageSpace*&, dart::MarkingVisitorBase<true>*&) ../../out/ReleaseASANX64/../../runtime/vm/thread_pool.h:43:12
    #5 0x5602eb0965b4 in dart::GCMarker::StartConcurrentMark(dart::PageSpace*) ../../out/ReleaseASANX64/../../runtime/vm/heap/marker.cc:797
    #6 0x5602eb0a610a in dart::PageSpace::CollectGarbageAtSafepoint(bool, bool, long, long) ../../out/ReleaseASANX64/../../runtime/vm/heap/pages.cc:1122:14
    #7 0x5602eb0a56d3 in dart::PageSpace::CollectGarbage(bool, bool) ../../out/ReleaseASANX64/../../runtime/vm/heap/pages.cc:1064:5
    #8 0x5602eb08db85 in dart::Heap::StartConcurrentMarking(dart::Thread*) ../../out/ReleaseASANX64/../../runtime/vm/heap/heap.cc:611:16
    #9 0x5602eb08a859 in dart::Heap::CheckStartConcurrentMarking(dart::Thread*, dart::Heap::GCReason) ../../out/ReleaseASANX64/../../runtime/vm/heap/heap.cc:604:5
    #10 0x5602eb0a0394 in dart::PageSpace::EvaluateConcurrentMarking(dart::PageSpace::GrowthPolicy) ../../out/ReleaseASANX64/../../runtime/vm/heap/pages.cc:454:16
    #11 0x5602eb0a0394 in dart::PageSpace::TryAllocateInFreshLargePage(long, dart::HeapPage::PageType, dart::PageSpace::GrowthPolicy) ../../out/ReleaseASANX64/../../runtime/vm/heap/pages.cc:502
    #12 0x5602eb089210 in dart::PageSpace::TryAllocate(long, dart::HeapPage::PageType, dart::PageSpace::GrowthPolicy) ../../out/ReleaseASANX64/../../runtime/vm/heap/pages.h:295:12
    #13 0x5602eb089210 in dart::Heap::AllocateOld(long, dart::HeapPage::PageType) ../../out/ReleaseASANX64/../../runtime/vm/heap/heap.cc:131
    #14 0x5602ea774e9a in dart::Heap::Allocate(long, dart::Heap::Space) ../../out/ReleaseASANX64/../../runtime/vm/heap/heap.h
    #15 0x5602ea774e9a in dart::Object::Allocate(long, long, dart::Heap::Space) ../../out/ReleaseASANX64/../../runtime/vm/object.cc:2544
    #16 0x5602ea778fe4 in dart::Array::New(long, long, dart::Heap::Space) ../../out/ReleaseASANX64/../../runtime/vm/object.cc:21909:9
    #17 0x5602ea778fe4 in dart::Array::New(long, dart::Heap::Space) ../../out/ReleaseASANX64/../../runtime/vm/object.cc:21881
    #18 0x5602eaaca2b2 in dart::RawArray* dart::HashTables::New<dart::HashSet<dart::UnorderedHashTable<dart::SymbolTraits, 0l> > >(long, dart::Heap::Space) ../../out/ReleaseASANX64/../../runtime/vm/hash_table.h:443:9
    #19 0x5602eaac9deb in void dart::HashTables::EnsureLoadFactor<dart::HashSet<dart::UnorderedHashTable<dart::SymbolTraits, 0l> > >(double, dart::HashSet<dart::UnorderedHashTable<dart::SymbolTraits, 0l> > const&) ../../out/ReleaseASANX64/../../runtime/vm/hash_table.h:501:21
    #20 0x5602eaacedac in dart::HashSet<dart::UnorderedHashTable<dart::SymbolTraits, 0l> >::EnsureCapacity() const ../../out/ReleaseASANX64/../../runtime/vm/hash_table.h:703:5
    #21 0x5602eaacedac in dart::RawObject* dart::HashSet<dart::UnorderedHashTable<dart::SymbolTraits, 0l> >::InsertNewOrGet<dart::ConcatString>(dart::ConcatString const&) const ../../out/ReleaseASANX64/../../runtime/vm/hash_table.h:666
    #22 0x5602eaac2e19 in dart::RawString* dart::Symbols::NewSymbol<dart::ConcatString>(dart::Thread*, dart::ConcatString const&) ../../out/ReleaseASANX64/../../runtime/vm/symbols.cc:588:21
    #23 0x5602eaac2425 in dart::Symbols::FromConcat(dart::Thread*, dart::String const&, dart::String const&) ../../out/ReleaseASANX64/../../runtime/vm/symbols.cc:475:12
    #24 0x5602eafb3838 in dart::kernel::TranslationHelper::DartGetterName(dart::kernel::NameIndex, dart::kernel::StringIndex) ../../out/ReleaseASANX64/../../runtime/vm/compiler/frontend/kernel_translation_helper.cc:487:10
    #25 0x5602eafb3838 in dart::kernel::TranslationHelper::DartGetterName(dart::kernel::NameIndex) ../../out/ReleaseASANX64/../../runtime/vm/compiler/frontend/kernel_translation_helper.cc:479
    #26 0x5602eafb3838 in dart::kernel::TranslationHelper::DartProcedureName(dart::kernel::NameIndex) ../../out/ReleaseASANX64/../../runtime/vm/compiler/frontend/kernel_translation_helper.cc:438
    #27 0x5602ea741749 in dart::kernel::KernelLoader::LoadProcedure(dart::Library const&, dart::Class const&, bool, long) ../../out/ReleaseASANX64/../../runtime/vm/kernel_loader.cc:1887:26
    #28 0x5602ea7456b2 in dart::kernel::KernelLoader::FinishClassLoading(dart::Class const&, dart::Library const&, dart::Class const&, long, dart::kernel::ClassIndex const&, dart::kernel::ClassHelper*) ../../out/ReleaseASANX64/../../runtime/vm/kernel_loader.cc:1700:5
    #29 0x5602ea746ad6 in dart::kernel::KernelLoader::FinishLoading(dart::Class const&) ../../out/ReleaseASANX64/../../runtime/vm/kernel_loader.cc:1755:17
    #30 0x5602ea527ce6 in dart::ClassFinalizer::FinalizeClass(dart::Class const&) ../../out/ReleaseASANX64/../../runtime/vm/class_finalizer.cc:1130:7
    #31 0x5602ea527dee in dart::ClassFinalizer::FinalizeClass(dart::Class const&) ../../out/ReleaseASANX64/../../runtime/vm/class_finalizer.cc:1149:5
    #32 0x5602ea52ac62 in dart::ClassFinalizer::LoadClassMembers(dart::Class const&) ../../out/ReleaseASANX64/../../runtime/vm/class_finalizer.cc:1194:5
    #33 0x5602ea7b732d in dart::Class::EnsureIsFinalized(dart::Thread*) const ../../out/ReleaseASANX64/../../runtime/vm/object.cc:4109:37
    #34 0x5602eaf2b23e in dart::kernel::ConstantReader::ReadConstantInternal(long) ../../out/ReleaseASANX64/../../runtime/vm/compiler/frontend/constant_reader.cc:221:49
    #35 0x5602eaf294c9 in dart::kernel::ConstantReader::ReadConstant(long) ../../out/ReleaseASANX64/../../runtime/vm/compiler/frontend/constant_reader.cc:81:15
    #36 0x5602eaf57bca in dart::kernel::StreamingFlowGraphBuilder::BuildConstantExpression(dart::TokenPosition*, dart::kernel::Tag) ../../out/ReleaseASANX64/../../runtime/vm/compiler/frontend/kernel_binary_flowgraph.cc:3920:46
    #37 0x5602eaf3545c in dart::kernel::StreamingFlowGraphBuilder::BuildExpression(dart::TokenPosition*) ../../out/ReleaseASANX64/../../runtime/vm/compiler/frontend/kernel_binary_flowgraph.cc:1309:14
    #38 0x5602eaf6cb03 in dart::kernel::StreamingFlowGraphBuilder::BuildArgumentsFromActualArguments(dart::Array*) ../../out/ReleaseASANX64/../../runtime/vm/compiler/frontend/kernel_binary_flowgraph.cc:2078:21
    #39 0x5602eaf4f80b in dart::kernel::StreamingFlowGraphBuilder::BuildArguments(dart::Array*, long*, long*) ../../out/ReleaseASANX64/../../runtime/vm/compiler/frontend/kernel_binary_flowgraph.cc:2068:10
    #40 0x5602eaf4f80b in dart::kernel::StreamingFlowGraphBuilder::BuildStaticInvocation(dart::TokenPosition*) ../../out/ReleaseASANX64/../../runtime/vm/compiler/frontend/kernel_binary_flowgraph.cc:3284
    #41 0x5602eaf352f6 in dart::kernel::StreamingFlowGraphBuilder::BuildExpression(dart::TokenPosition*) ../../out/ReleaseASANX64/../../runtime/vm/compiler/frontend/kernel_binary_flowgraph.cc:1250:14
    #42 0x5602eaf62642 in dart::kernel::StreamingFlowGraphBuilder::BuildReturnStatement() ../../out/ReleaseASANX64/../../runtime/vm/compiler/frontend/kernel_binary_flowgraph.cc:4611:33
    #43 0x5602eaf3c464 in dart::kernel::StreamingFlowGraphBuilder::BuildStatement() ../../out/ReleaseASANX64/../../runtime/vm/compiler/frontend/kernel_binary_flowgraph.cc:1374:14
    #44 0x5602eaf41342 in dart::kernel::StreamingFlowGraphBuilder::BuildFunctionBody(dart::Function const&, dart::LocalVariable*, bool) ../../out/ReleaseASANX64/../../runtime/vm/compiler/frontend/kernel_binary_flowgraph.cc:788:13
    #45 0x5602eaf43066 in dart::kernel::StreamingFlowGraphBuilder::BuildGraphOfFunction(bool) ../../out/ReleaseASANX64/../../runtime/vm/compiler/frontend/kernel_binary_flowgraph.cc:939:7
    #46 0x5602eaf444a2 in dart::kernel::StreamingFlowGraphBuilder::BuildGraph() ../../out/ReleaseASANX64/../../runtime/vm/compiler/frontend/kernel_binary_flowgraph.cc:1095:14
    #47 0x5602eaf8a321 in dart::kernel::FlowGraphBuilder::BuildGraph() ../../out/ReleaseASANX64/../../runtime/vm/compiler/frontend/kernel_to_il.cc:806:39
    #48 0x5602eb02b191 in dart::DartCompilationPipeline::BuildFlowGraph(dart::Zone*, dart::ParsedFunction*, dart::ZoneGrowableArray<dart::ICData const*>*, long, bool) ../../out/ReleaseASANX64/../../runtime/vm/compiler/jit/compiler.cc:140:30
    #49 0x5602eb02efaf in dart::CompileParsedFunctionHelper::Compile(dart::CompilationPipeline*) ../../out/ReleaseASANX64/../../runtime/vm/compiler/jit/compiler.cc:580:32
    #50 0x5602eb032d96 in dart::CompileFunctionHelper(dart::CompilationPipeline*, dart::Function const&, bool, long) ../../out/ReleaseASANX64/../../runtime/vm/compiler/jit/compiler.cc:763:46
    #51 0x5602eb031dfd in dart::Compiler::CompileFunction(dart::Thread*, dart::Function const&) ../../out/ReleaseASANX64/../../runtime/vm/compiler/jit/compiler.cc:903:10
    #52 0x5602eb02c05a in dart::DRT_HelperCompileFunction(dart::Isolate*, dart::Thread*, dart::Zone*, dart::NativeArguments) ../../out/ReleaseASANX64/../../runtime/vm/compiler/jit/compiler.cc:239:12
    #53 0x5602eb02c05a in dart::DRT_CompileFunction(dart::NativeArguments) ../../out/ReleaseASANX64/../../runtime/vm/compiler/jit/compiler.cc:215
LLVMSymbolizer: error reading file: No such file or directory
    #54 0x7fe89cd80ff7  (/memfd:dart-codespace (deleted)+0xff7)
    #55 0x7fe89cd81053  (/memfd:dart-codespace (deleted)+0x1053)
    #56 0x7fe892389679  (<unknown module>)
    #57 0x7fe892389503  (<unknown module>)
    #58 0x7fe892389344  (<unknown module>)
    #59 0x7fe892388ac8  (<unknown module>)
    #60 0x7fe892387dc8  (<unknown module>)
    #61 0x7fe892384255  (<unknown module>)
    #62 0x7fe89238107f  (<unknown module>)
    #63 0x7fe8923809f0  (<unknown module>)
    #64 0x7fe8923804df  (<unknown module>)
    #65 0x7fe89cd8148b  (/memfd:dart-codespace (deleted)+0x148b)
    #66 0x5602ea5f4aff in dart::DartEntry::InvokeFunction(dart::Function const&, dart::Array const&, dart::Array const&, unsigned long) ../../out/ReleaseASANX64/../../runtime/vm/dart_entry.cc:182:10
    #67 0x5602ea82d681 in dart::Library::Invoke(dart::String const&, dart::Array const&, dart::Array const&, bool, bool) const ../../out/ReleaseASANX64/../../runtime/vm/object.cc:12071:10
    #68 0x5602eb942b73 in Dart_Invoke ../../out/ReleaseASANX64/../../runtime/vm/dart_api_impl.cc:4395:16
    #69 0x5602ea152915 in dart::bin::SingleArgDart_Invoke(_Dart_Handle*, char const*, _Dart_Handle*) ../../out/ReleaseASANX64/../../runtime/bin/dartutils.cc:311:10
    #70 0x5602ea152915 in dart::bin::DartUtils::SetWorkingDirectory() ../../out/ReleaseASANX64/../../runtime/bin/dartutils.cc:375
    #71 0x5602ea152915 in dart::bin::DartUtils::PrepareBuiltinLibrary(_Dart_Handle*, _Dart_Handle*, bool, bool) ../../out/ReleaseASANX64/../../runtime/bin/dartutils.cc:469
    #72 0x5602ea153088 in dart::bin::DartUtils::PrepareForScriptLoading(bool, bool) ../../out/ReleaseASANX64/../../runtime/bin/dartutils.cc:567:12
    #73 0x5602ea0fafc1 in dart::bin::SetupCoreLibraries(_Dart_Isolate*, dart::bin::IsolateData*, bool, char const**) ../../out/ReleaseASANX64/../../runtime/bin/main.cc:185:12
    #74 0x5602ea0fa092 in dart::bin::IsolateSetupHelper(_Dart_Isolate*, bool, char const*, char const*, bool, Dart_IsolateFlags*, char**, int*) ../../out/ReleaseASANX64/../../runtime/bin/main.cc:309:12
    #75 0x5602ea0f680f in dart::bin::CreateIsolateGroupAndSetupHelper(bool, char const*, char const*, char const*, char const*, Dart_IsolateFlags*, void*, char**, int*) ../../out/ReleaseASANX64/../../runtime/bin/main.cc:706:23
    #76 0x5602ea0f885a in dart::bin::CreateIsolateGroupAndSetup(char const*, char const*, char const*, char const*, Dart_IsolateFlags*, void*, char**) ../../out/ReleaseASANX64/../../runtime/bin/main.cc:747:10
    #77 0x5602ea4d1c84 in dart::SpawnIsolateTask::Run() ../../out/ReleaseASANX64/../../runtime/lib/isolate.cc:159:44
    #78 0x5602eaae663f in dart::ThreadPool::Worker::Loop() ../../out/ReleaseASANX64/../../runtime/vm/thread_pool.cc:382:11
    #79 0x5602eaae6212 in dart::ThreadPool::Worker::Main(unsigned long) ../../out/ReleaseASANX64/../../runtime/vm/thread_pool.cc:434:27
    #80 0x5602ea90baae in dart::ThreadStart(void*) ../../out/ReleaseASANX64/../../runtime/vm/os_thread_linux.cc:138:5
    #81 0x7fe8a0aa0fb6 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x8fb6)

Thread T8 (DartWorker) created by T2 (DartWorker) here:
    #0 0x5602ea0a6a8d in __interceptor_pthread_create /b/s/w/ir/kitchen-workdir/llvm-project/compiler-rt/lib/asan/asan_interceptors.cc:210:3
    #1 0x5602ea90b7ef in dart::OSThread::Start(char const*, void (*)(unsigned long), unsigned long) ../../out/ReleaseASANX64/../../runtime/vm/os_thread_linux.cc:157:12
    #2 0x5602eaae4e42 in dart::ThreadPool::Worker::StartThread() ../../out/ReleaseASANX64/../../runtime/vm/thread_pool.cc:338:16
    #3 0x5602eaae4e42 in dart::ThreadPool::RunImpl(std::__2::unique_ptr<dart::ThreadPool::Task, std::__2::default_delete<dart::ThreadPool::Task> >) ../../out/ReleaseASANX64/../../runtime/vm/thread_pool.cc:69
    #4 0x5602ea4cd232 in bool dart::ThreadPool::Run<dart::SpawnIsolateTask, dart::Isolate*&, std::__2::unique_ptr<dart::IsolateSpawnState, std::__2::default_delete<dart::IsolateSpawnState> > >(dart::Isolate*&, std::__2::unique_ptr<dart::IsolateSpawnState, std::__2::default_delete<dart::IsolateSpawnState> >&&) ../../out/ReleaseASANX64/../../runtime/vm/thread_pool.h:43:12
    #5 0x5602ea4cd232 in dart::DN_HelperIsolate_spawnFunction(dart::Isolate*, dart::Thread*, dart::Zone*, dart::NativeArguments*) ../../out/ReleaseASANX64/../../runtime/lib/isolate.cc:294
    #6 0x5602ea4cd232 in dart::BootstrapNatives::DN_Isolate_spawnFunction(dart::Thread*, dart::Zone*, dart::NativeArguments*) ../../out/ReleaseASANX64/../../runtime/lib/isolate.cc:243
    #7 0x5602ea75af45 in dart::NativeEntry::BootstrapNativeCallWrapper(_Dart_NativeArguments*, void (*)(_Dart_NativeArguments*)) ../../out/ReleaseASANX64/../../runtime/vm/native_entry.cc:140:38
    #8 0x7fe89cd81158  (/memfd:dart-codespace (deleted)+0x1158)
    #9 0x7fe89bf34864  (<unknown module>)
    #10 0x7fe89bf32204  (<unknown module>)
    #11 0x7fe89bf2b546  (<unknown module>)
    #12 0x7fe89bf315f5  (<unknown module>)
    #13 0x7fe89bf30508  (<unknown module>)
    #14 0x7fe89bf2d8c7  (<unknown module>)
    #15 0x7fe89bf2d568  (<unknown module>)
    #16 0x7fe89bf2b92f  (<unknown module>)
    #17 0x7fe89bf2b706  (<unknown module>)
    #18 0x7fe89bf2b546  (<unknown module>)
    #19 0x7fe89bf2a84f  (<unknown module>)
    #20 0x7fe89bf09679  (<unknown module>)
    #21 0x7fe89bf2a53d  (<unknown module>)
    #22 0x7fe89cd8148b  (/memfd:dart-codespace (deleted)+0x148b)
    #23 0x5602ea5f4aff in dart::DartEntry::InvokeFunction(dart::Function const&, dart::Array const&, dart::Array const&, unsigned long) ../../out/ReleaseASANX64/../../runtime/vm/dart_entry.cc:182:10
    #24 0x5602ea5fa01c in dart::DartEntry::InvokeFunction(dart::Function const&, dart::Array const&) ../../out/ReleaseASANX64/../../runtime/vm/dart_entry.cc:36:10
    #25 0x5602ea5fa01c in dart::DartLibraryCalls::HandleMessage(dart::Object const&, dart::Instance const&) ../../out/ReleaseASANX64/../../runtime/vm/dart_entry.cc:662
    #26 0x5602ea6bfbed in dart::IsolateMessageHandler::HandleMessage(std::__2::unique_ptr<dart::Message, std::__2::default_delete<dart::Message> >) ../../out/ReleaseASANX64/../../runtime/vm/isolate.cc:991:30
    #27 0x5602ea75489a in dart::MessageHandler::HandleMessages(dart::MonitorLocker*, bool, bool) ../../out/ReleaseASANX64/../../runtime/vm/message_handler.cc:232:16
    #28 0x5602ea7566c2 in dart::MessageHandler::TaskCallback() ../../out/ReleaseASANX64/../../runtime/vm/message_handler.cc:446:20
    #29 0x5602eaae663f in dart::ThreadPool::Worker::Loop() ../../out/ReleaseASANX64/../../runtime/vm/thread_pool.cc:382:11
    #30 0x5602eaae6212 in dart::ThreadPool::Worker::Main(unsigned long) ../../out/ReleaseASANX64/../../runtime/vm/thread_pool.cc:434:27
    #31 0x5602ea90baae in dart::ThreadStart(void*) ../../out/ReleaseASANX64/../../runtime/vm/os_thread_linux.cc:138:5
    #32 0x7fe8a0aa0fb6 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x8fb6)

Thread T2 (DartWorker) created by T0 here:
    #0 0x5602ea0a6a8d in __interceptor_pthread_create /b/s/w/ir/kitchen-workdir/llvm-project/compiler-rt/lib/asan/asan_interceptors.cc:210:3
    #1 0x5602ea90b7ef in dart::OSThread::Start(char const*, void (*)(unsigned long), unsigned long) ../../out/ReleaseASANX64/../../runtime/vm/os_thread_linux.cc:157:12
    #2 0x5602eaae4e42 in dart::ThreadPool::Worker::StartThread() ../../out/ReleaseASANX64/../../runtime/vm/thread_pool.cc:338:16
    #3 0x5602eaae4e42 in dart::ThreadPool::RunImpl(std::__2::unique_ptr<dart::ThreadPool::Task, std::__2::default_delete<dart::ThreadPool::Task> >) ../../out/ReleaseASANX64/../../runtime/vm/thread_pool.cc:69
    #4 0x5602eaa89bb1 in bool dart::ThreadPool::Run<dart::RunServiceTask>() ../../out/ReleaseASANX64/../../runtime/vm/thread_pool.h:43:12
    #5 0x5602eaa89bb1 in dart::ServiceIsolate::Run() ../../out/ReleaseASANX64/../../runtime/vm/service_isolate.cc:494
    #6 0x5602ea5dcc59 in dart::Dart::Init(unsigned char const*, unsigned char const*, _Dart_Isolate* (*)(char const*, char const*, char const*, char const*, Dart_IsolateFlags*, void*, char**), bool (*)(void**, char**), void (*)(void*, void*), void (*)(void*, void*), void (*)(void*), void (*)(), void* (*)(char const*, bool), void (*)(unsigned char**, long*, void*), void (*)(void const*, long, void*), void (*)(void*), bool (*)(unsigned char*, long), _Dart_Handle* (*)(), bool, Dart_CodeObserver*) ../../out/ReleaseASANX64/../../runtime/vm/dart.cc:396:5
    #7 0x5602eb8ea32f in Dart_Initialize ../../out/ReleaseASANX64/../../runtime/vm/dart_api_impl.cc:1031:10
    #8 0x5602ea0f7e8f in dart::bin::main(int, char**) ../../out/ReleaseASANX64/../../runtime/bin/main.cc:1209:11
    #9 0x5602ea0f9e5a in main ../../out/ReleaseASANX64/../../runtime/bin/main.cc:1258:3
    #10 0x7fe8a07b9bba in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26bba)

Thread T12 (DartWorker) created by T2 (DartWorker) here:
    #0 0x5602ea0a6a8d in __interceptor_pthread_create /b/s/w/ir/kitchen-workdir/llvm-project/compiler-rt/lib/asan/asan_interceptors.cc:210:3
    #1 0x5602ea90b7ef in dart::OSThread::Start(char const*, void (*)(unsigned long), unsigned long) ../../out/ReleaseASANX64/../../runtime/vm/os_thread_linux.cc:157:12
    #2 0x5602eaae4e42 in dart::ThreadPool::Worker::StartThread() ../../out/ReleaseASANX64/../../runtime/vm/thread_pool.cc:338:16
    #3 0x5602eaae4e42 in dart::ThreadPool::RunImpl(std::__2::unique_ptr<dart::ThreadPool::Task, std::__2::default_delete<dart::ThreadPool::Task> >) ../../out/ReleaseASANX64/../../runtime/vm/thread_pool.cc:69
    #4 0x5602ea4cd232 in bool dart::ThreadPool::Run<dart::SpawnIsolateTask, dart::Isolate*&, std::__2::unique_ptr<dart::IsolateSpawnState, std::__2::default_delete<dart::IsolateSpawnState> > >(dart::Isolate*&, std::__2::unique_ptr<dart::IsolateSpawnState, std::__2::default_delete<dart::IsolateSpawnState> >&&) ../../out/ReleaseASANX64/../../runtime/vm/thread_pool.h:43:12
    #5 0x5602ea4cd232 in dart::DN_HelperIsolate_spawnFunction(dart::Isolate*, dart::Thread*, dart::Zone*, dart::NativeArguments*) ../../out/ReleaseASANX64/../../runtime/lib/isolate.cc:294
    #6 0x5602ea4cd232 in dart::BootstrapNatives::DN_Isolate_spawnFunction(dart::Thread*, dart::Zone*, dart::NativeArguments*) ../../out/ReleaseASANX64/../../runtime/lib/isolate.cc:243
    #7 0x5602ea75af45 in dart::NativeEntry::BootstrapNativeCallWrapper(_Dart_NativeArguments*, void (*)(_Dart_NativeArguments*)) ../../out/ReleaseASANX64/../../runtime/vm/native_entry.cc:140:38
    #8 0x7fe89cd81158  (/memfd:dart-codespace (deleted)+0x1158)
    #9 0x7fe89bf34864  (<unknown module>)
    #10 0x7fe89bf32204  (<unknown module>)
    #11 0x7fe89bf2b546  (<unknown module>)
    #12 0x7fe89bf315f5  (<unknown module>)
    #13 0x7fe89bf30508  (<unknown module>)
    #14 0x7fe89bf2d8c7  (<unknown module>)
    #15 0x7fe89bf2d568  (<unknown module>)
    #16 0x7fe89bf2b92f  (<unknown module>)
    #17 0x7fe89bf2b706  (<unknown module>)
    #18 0x7fe89bf2b546  (<unknown module>)
    #19 0x7fe89bf2a84f  (<unknown module>)
    #20 0x7fe89bf09679  (<unknown module>)
    #21 0x7fe89bf2a53d  (<unknown module>)
    #22 0x7fe89cd8148b  (/memfd:dart-codespace (deleted)+0x148b)
    #23 0x5602ea5f4aff in dart::DartEntry::InvokeFunction(dart::Function const&, dart::Array const&, dart::Array const&, unsigned long) ../../out/ReleaseASANX64/../../runtime/vm/dart_entry.cc:182:10
    #24 0x5602ea5fa01c in dart::DartEntry::InvokeFunction(dart::Function const&, dart::Array const&) ../../out/ReleaseASANX64/../../runtime/vm/dart_entry.cc:36:10
    #25 0x5602ea5fa01c in dart::DartLibraryCalls::HandleMessage(dart::Object const&, dart::Instance const&) ../../out/ReleaseASANX64/../../runtime/vm/dart_entry.cc:662
    #26 0x5602ea6bfbed in dart::IsolateMessageHandler::HandleMessage(std::__2::unique_ptr<dart::Message, std::__2::default_delete<dart::Message> >) ../../out/ReleaseASANX64/../../runtime/vm/isolate.cc:991:30
    #27 0x5602ea75489a in dart::MessageHandler::HandleMessages(dart::MonitorLocker*, bool, bool) ../../out/ReleaseASANX64/../../runtime/vm/message_handler.cc:232:16
    #28 0x5602ea7566c2 in dart::MessageHandler::TaskCallback() ../../out/ReleaseASANX64/../../runtime/vm/message_handler.cc:446:20
    #29 0x5602eaae663f in dart::ThreadPool::Worker::Loop() ../../out/ReleaseASANX64/../../runtime/vm/thread_pool.cc:382:11
    #30 0x5602eaae6212 in dart::ThreadPool::Worker::Main(unsigned long) ../../out/ReleaseASANX64/../../runtime/vm/thread_pool.cc:434:27
    #31 0x5602ea90baae in dart::ThreadStart(void*) ../../out/ReleaseASANX64/../../runtime/vm/os_thread_linux.cc:138:5
    #32 0x7fe8a0aa0fb6 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x8fb6)

SUMMARY: AddressSanitizer: heap-use-after-free /b/s/w/ir/kitchen-workdir/llvm-project/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:433:3 in __interceptor_strcmp
Shadow bytes around the buggy address:
  0x0c067fffb1e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c067fffb1f0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c067fffb200: fd fd fd fd fa fa fd fd fd fd fa fa fd fd fd fa
  0x0c067fffb210: fa fa 00 00 07 fa fa fa 00 00 00 00 fa fa 00 00
  0x0c067fffb220: 00 00 fa fa 00 00 00 00 fa fa fd fd fd fd fa fa
=>0x0c067fffb230: fd fd fd fd fa fa fd fd fd fa fa fa[fd]fd fd fa
  0x0c067fffb240: fa fa fd fd fd fa fa fa fd fd fd fd fa fa 00 00
  0x0c067fffb250: 00 00 fa fa 00 00 00 00 fa fa fd fd fd fd fa fa
  0x0c067fffb260: fd fd fd fd fa fa fd fd fd fd fa fa fd fd fd fd
  0x0c067fffb270: fa fa fd fd fd fa fa fa fd fd fd fd fa fa fd fd
  0x0c067fffb280: fd fd fa fa fd fd fd fd fa fa fd fd fd fd fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
==92914==ABORTING

--- Re-run this test:
python tools/test.py -n dartk-asan-linux-release-x64 standalone_2/io/socket_close_test/3

--- Command "vm" (took 01.000663s):
DART_CONFIGURATION=ReleaseMSANX64 out/ReleaseMSANX64/dart --enable-isolate-groups --ignore-unrecognized-flags --packages=/usr/local/google/home/rmacnak/dart2/sdk/.packages /usr/local/google/home/rmacnak/dart2/sdk/runtime/tests/vm/dart/transferable_throws_test.dart

exit code:
-6

stderr:
Uninitialized bytes in __interceptor_strcmp at offset 0 inside [0x701000001050, 1)
==64077==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x55c26ae31b4f in dart::Dart::VmIsolateNameEquals(char const*) ../../out/ReleaseMSANX64/../../runtime/vm/dart.cc:966:11
    #1 0x55c26af37300 in dart::Isolate::IsVMInternalIsolate(dart::Isolate const*) ../../out/ReleaseMSANX64/../../runtime/vm/isolate.cc:3114:10
    #2 0x55c26af37300 in dart::IsolateVisitor::IsVMInternalIsolate(dart::Isolate*) const ../../out/ReleaseMSANX64/../../runtime/vm/isolate.cc:486
    #3 0x55c26af37300 in dart::IsolateKillerVisitor::ShouldKill(dart::Isolate*) ../../out/ReleaseMSANX64/../../runtime/vm/isolate.cc:3186
    #4 0x55c26af37300 in dart::IsolateKillerVisitor::VisitIsolate(dart::Isolate*) ../../out/ReleaseMSANX64/../../runtime/vm/isolate.cc:3174
    #5 0x55c26af38a19 in std::__2::function<void (dart::Isolate*)>::operator()(dart::Isolate*) const ../../out/ReleaseMSANX64/../../buildtools/linux-x64/clang/lib/clang/8.0.0/include/c++/v1/functional:1923:12
    #6 0x55c26af38a19 in dart::IsolateGroup::ForEachIsolate(std::__2::function<void (dart::Isolate*)>) ../../out/ReleaseMSANX64/../../runtime/vm/isolate.cc:2418
    #7 0x55c26af38a19 in dart::Isolate::VisitIsolates(dart::IsolateVisitor*)::$_4::operator()(dart::IsolateGroup*) const ../../out/ReleaseMSANX64/../../runtime/vm/isolate.cc:3027
    #8 0x55c26af38a19 in decltype(std::__2::forward<dart::Isolate::VisitIsolates(dart::IsolateVisitor*)::$_4&>(fp)(std::__2::forward<dart::IsolateGroup*>(fp0))) std::__2::__invoke<dart::Isolate::VisitIsolates(dart::IsolateVisitor*)::$_4&, dart::IsolateGroup*>(dart::Isolate::VisitIsolates(dart::IsolateVisitor*)::$_4&, dart::IsolateGroup*&&) ../../out/ReleaseMSANX64/../../buildtools/linux-x64/clang/lib/clang/8.0.0/include/c++/v1/type_traits:4350
    #9 0x55c26af38a19 in void std::__2::__invoke_void_return_wrapper<void>::__call<dart::Isolate::VisitIsolates(dart::IsolateVisitor*)::$_4&, dart::IsolateGroup*>(dart::Isolate::VisitIsolates(dart::IsolateVisitor*)::$_4&, dart::IsolateGroup*&&) ../../out/ReleaseMSANX64/../../buildtools/linux-x64/clang/lib/clang/8.0.0/include/c++/v1/__functional_base:349
    #10 0x55c26af38a19 in std::__2::__function::__func<dart::Isolate::VisitIsolates(dart::IsolateVisitor*)::$_4, std::__2::allocator<dart::Isolate::VisitIsolates(dart::IsolateVisitor*)::$_4>, void (dart::IsolateGroup*)>::operator()(dart::IsolateGroup*&&) ../../out/ReleaseMSANX64/../../buildtools/linux-x64/clang/lib/clang/8.0.0/include/c++/v1/functional:1572
    #11 0x55c26af32b15 in std::__2::function<void (dart::IsolateGroup*)>::operator()(dart::IsolateGroup*) const ../../out/ReleaseMSANX64/../../buildtools/linux-x64/clang/lib/clang/8.0.0/include/c++/v1/functional:1923:12
    #12 0x55c26af32b15 in dart::IsolateGroup::ForEach(std::__2::function<void (dart::IsolateGroup*)>) ../../out/ReleaseMSANX64/../../runtime/vm/isolate.cc:442
    #13 0x55c26af32b15 in dart::Isolate::VisitIsolates(dart::IsolateVisitor*) ../../out/ReleaseMSANX64/../../runtime/vm/isolate.cc:3026
    #14 0x55c26af2bfcc in dart::Isolate::KillAllIsolates(dart::Isolate::LibMsgId) ../../out/ReleaseMSANX64/../../runtime/vm/isolate.cc:3197:3
    #15 0x55c26ae2ef92 in dart::Dart::Cleanup() ../../out/ReleaseMSANX64/../../runtime/vm/dart.cc:501:3
    #16 0x55c26a8b990d in dart::bin::main(int, char**) ../../out/ReleaseMSANX64/../../runtime/bin/main.cc:1230:11
    #17 0x55c26a8bbbe8 in main ../../out/ReleaseMSANX64/../../runtime/bin/main.cc:1258:3
    #18 0x7effef152bba in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26bba)
    #19 0x55c26a838028 in _start (/usr/local/google/home/rmacnak/dart2/sdk/out/ReleaseMSANX64/dart+0x1893028)

SUMMARY: MemorySanitizer: use-of-uninitialized-value ../../out/ReleaseMSANX64/../../runtime/vm/dart.cc:966:11 in dart::Dart::VmIsolateNameEquals(char const*)
Exiting

--- Re-run this test:
python tools/test.py -n dartk-msan-linux-release-x64 vm/dart/transferable_throws_test/0

@aam @mkustermann

Appears fixed at top-of-tree.


text:
Dart language tools should error if libraries and parts have inconsistent language versions

Per spec:

Part files must be marked with the same version as their library. They must always be treated the same way as the library they belong to, so it is a compile-time error if a part file has a different language version override than its library. It is also a compile-time error if a part file has no language version marker, and the importing library does, or vice versa. Tools that work on individual part files, like the formatter, need a marker in the part file. If there isn't one, the tool can assume that the default language version of the package applies.

There should be language tests here with implementations in our compilers (CFE) and analyzer.
CC @leafpetersen @lrhn

CC @scheglov @bwilkerson @devoncarew – It'd be nice to get this error into analyzer soonish.
Do you want to track it as a separate (child) issue?

Analyzer implementation landed as 849cdb2ab40218585e235c87cf8ba43bac7011f0.

Anything left to do here?

I believe needs to do work here – this should be an error in the compliers, etc

Is the needed work tests or implementation? As far as I know CFE has handled this for a long time, but maybe there is a case I'm missing.

If we have tests to validate this and they are passing, I guess we're good.
We can close it
On Tue, Jul 14, 2020, 12:18 johnniwinther notifications@github.com wrote:

Is the needed work tests or implementation? As far as I know CFE has
handled this for a long time, but maybe there is a case I'm missing.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/dart-lang/sdk/issues/42456#issuecomment-658364631,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAEFCXPIQRJ7LD6BG5UDHDR3SVRJANCNFSM4OGECKQA
.

I don't think we have test, say, in the language folder that are shared between analyzer and CFE.
CFE has it's own unittest-suite for language versioning.

Would it be good to have tests in the language folder?
Seems the type of thing we should validate across all impls – consistently, right?

Yes. That's probably for the language team. @lrhn can you look into this?

Will do.

Dropping priority to P2 since (we hope) everything is implemented correctly.

@kevmoo Is this something we still need to track for TP2?

Nope. For beta, certainly!

@lrhn are you working on tests for this?

Landed as https://dart-review.googlesource.com/c/sdk/+/154462

Can we mark the present issue resolved then?

A quick checks shows the front-end giving errors on the test, and the analyzer crashing (failing to read the some sdk/version file), both on a fresh build of the SDK.
@johnniwinther @scheglov  Are the bugs in the test or in the implementations?

All tests run, so can be closed.

authors: franklinyow johnniwinther kevmoo leafpetersen lrhn mit-mit scheglov


text:
Null safety feedback: [issue summary]

I maintain 2 packages on pub:

The first one is a "pure" Dart package with no dependencies on packages made by other people. I only relied on the Dart's API. No problems at all in the migration! A smooth transition. It was just a matter of replacing @required with required. I was also able to create some more const constructors thanks to late final.

The other package of mine relies on a package made by another person. He hasn't migrated to null safety yet so I'm "stuck" because I have to wait for him. A part from this issue, I was able to easily migrate most of the code too!

I never used null and tried to follow Dart's best practices as much as possible so in order to compile new code under null safety was literally a matter or replacing @required with required and recompile.
I am very satisfied. On the Flutter side, the situation is mostly the same since I only had to:

Replace @required with required
Add some ? here and there to fix some changes made on the API (such as Navigator)

There have been other minor "issues" (if we can call those "issues") but the migration page on the Flutter doc covered them all! I'd give a 8/10 overall.

Thanks much for the feedback @albertodev01, great to hear that it went well. cc @leafpetersen fyi


text:
Move semantics for SendPort

This tracker is for issues related to:

Dart VM

Description
We should be able to transfer any object over SendPort in constant time if we know:

Isolates share a heap
There is one reference to an object
The object is never used again after it is sent to a SendPort

To make this possible I propose we add a new class called Unique that when constructed sent over a SendPort asserts that there is one reference to each object in the object graph and nulls out its reference.  That way we can have constant-time communication of any object over SendPorts.  It isn't asserted at compile time, but this might be the best we can do with Dart.
The printed timestamps should almost be the same, today the difference is 12s locally:
// @dart=2.12
import 'dart:isolate';

class Unique<T> {
  T? get value => _value;
  T? _value;
  Unique(this._value) {
    // Crawl through the object tree and ensure there is only one reference to
    // each object, otherwise throw exception.
    // assert(isUnique(_value));
  }
}

Map<String, Object> makeLargeMap(int n) {
  Map<String, Object> result = {};
  for (int i = 0; i < n; ++i) {
    result[i.toString()] = i;
  }
  return result;
}

void isolateMain(SendPort sendPort) {
  ReceivePort receivePort = ReceivePort();
  sendPort.send(receivePort.sendPort);
  receivePort.listen((message) {
    var foo = Unique<Map<String, Object>>(makeLargeMap(message));
    print(DateTime.now().millisecondsSinceEpoch);
    sendPort.send(foo);
    // assert(foo.value == null);
  });
}

void main() async {
  ReceivePort receivePort = ReceivePort();
  Isolate.spawn(isolateMain, receivePort.sendPort);
  SendPort sendPort = await receivePort.first;
  sendPort.send(10000000);
  print(await receivePort.first);
  print(DateTime.now().millisecondsSinceEpoch);
}

Related language request for move semantics: https://github.com/dart-lang/language/issues/2355 (this has compile time checks for uniqueness)
cc @mkustermann
Alternatives considered

TransferableTypedData - this requires encoding and decoding time for most datatypes
Isolate.exit - this requires you to spawn an isolate for each result

Not sure how you would efficiently assert that there is only one reference to each object in the object graph. That sounds like something which needs a full GC-equivalent memory scan to ensure. (Mark the transitive closure of the unique object in one color, then mark from the roots in another color, stopping at the unique object, and see that you don't re-mark any object in both colors).
That's already better than "only referenced once" because it allows the unique object graph to refer to the same object more than once, as long as there are no references from outside that graph.
Also, it probably still won't work, since it means the graph cannot safely contain null, booleans, integers or strings, because you don't know whether those are shared. Same with constants.
I guess some objects (immutable ones mainly) can be freely shared, it's only references to mutable objects which matter.
I think simply making "immutable objects" a concept (anything created using const constructors with immutable arguments, even if the call is not const itself), and simply allowing those to be shared cheaply.
Trying to contain a Dart object reference, while providing access to the object, is something I very much doubt will ever work.

That sounds like something which needs a full GC-equivalent memory scan to ensure.

I guess it depends on how the garbage collector was implemented.  I was assuming it would be cheap to crawl the subsection of memory that is referenced from an object recursively and count those references per object.  I never looked at the Dart GC implementation, but it's probably not the case now that I consider it.  You'd probably need a doubly linked graph of objects that's readily available at any time, which we'd never have.

I guess some objects (immutable ones mainly) can be freely shared, it's only references to mutable objects which matter.

Yea, Martin has already talked about making these cheap for sending over SendPort, so they there is already an idea for detecting them.

Trying to contain a Dart object reference, while providing access to the object, is something I very much doubt will ever work.

I don't follow this statement.
Maybe my proposal for compile-time checks for uniqueness was a better way to go.  I already closed it thinking this would be easier to implement.

As @lrhn says it is a very hard problem to maintain the there is only one reference guarantee.
Rephrasing the requirement of there is only one reference to an object is basically saying, there's a unique owner. So we'd need to implement ownership tracking - either statically or dynamically - which is hard to do.
On top of that, sending across SendPort would then become limited to sending object trees, because DAGs or even graphs with cycles have more than one reference to an object.
So we're down to sending mutable trees at this point. Do they have to be mutable?

I was assuming it would be cheap to crawl the subsection of memory that is referenced from an object recursively and count those references per object.

This would not work, because you are looking for incoming references, rather than outgoing references.
You will need to crawl the whole heap to find all incoming references to objects within the subgraph referenced by the given root. It's going to be expensive.
That being said, I think there is a trick that might work here. If you know that you are creating objects for transfer, you could tell VM to allocate them all in a separate part of the heap and you use write-barrier to track when references to that part of the heap escape, i.e.
// o.f = v
def WriteBarrier(object, field, value):
  if pageOf(object).id != pageOf(value).id
    pageOf(value).selfContained = false
    if !value.isCanonical:
      pageOf(object).selfContained = false

Now when you are transfering an object to another isolate you could do the following:
def transfer(port, object):
  # Check if there is potential escape through thread stack or globals
  # Caveat: ignores current frame (object variable).
  iterateRoots(lambda o: pageOf(o).selfContained = false)
  if not pageOf(object).selfContained:
    raise 'Can't transfer object which potentially escaped'
  # Now we know that the object graph is self-contained within
  # a number of heap pages and there are no references (except object)
  # to these pages. This means we can transfer the object.

With this approach you would need to write the code like this:
Map<String, Object> makeLargeMap(int n) {
  // |allocateForTransfer| makes VM allocate all new objects in a separate page.
  return allocateForTransfer(() {
    Map<String, Object> result = {};
    for (int i = 0; i < n; ++i) {
      result[i.toString()] = i;
    }
    return result;
  });
}

Maybe something like this can work.

Using allocateForTransfer would probably be zone-based, otherwise it won't work with async functions.
Using a separate page with a write barrier needs to check both directions: When references to the presumed nonshared objects escape, and when an already shared (other-page) reference is written into that page. Either write loses the ability to share the page. Presumably permanently, because if you clear either of the cross-page references, you may need to do an expensive test to check that the object graph is closed again.
You'll have references into the page while working with the objects, often on the stack, but also in the heap if using async functions. Those will die automatically (stack popped, continuations GC'ed after use). Cross-page reference counting won't work with GC.
It still means needing to share common objects, like numbers, strings, booleans and null and enums. All immutable, so that's fine. Sharing immutable objects in general would be a good start.
Factory constructors and object caches become a problem, when you can't predict whether a call actually performs the allocation in the current zone.
Seems potentially very complicated, if at all viable. (Maybe just make allocateForTransfer run in a new isolate, that ensures the graph is closed).

Another probably silly idea (just for brainstorm): Can we convert objects between mutable and immutable, and ensure mutable XOR shared-across-isolate semantics?
A typical usage:

construct (mutable) object graph, and modify it freely
lock it (make it immutable), then share (indeed not move) across isolate
the second isolate can now read it, but no isolate can modify anything
after second isolate exits, the graph can be mutable again

@lrhn wrote:

I think simply making "immutable objects" a concept (anything created using const constructors with immutable arguments, even if the call is not const itself), and simply allowing those to be shared cheaply.

I'm curious, is/was this idea pursued? I have not found an issue specifically for it.

I'm curious, is/was this idea pursued? I have not found an issue specifically for it.

Kinda: https://github.com/dart-lang/sdk/blob/f62c78b52cfee7e2142cd3bc45076c76446034a3/runtime/docs/deeply_immutable.md

authors: fzyzcjy gaaclarke lrhn mkustermann mraleph ryanheise


text:
[dart fix] Allow changing base class and mixins

For https://github.com/flutter/flutter/pull/123352 I would like to write a dart fix that changes all classes extending RootRenderObjectElement to instead extend RenderObjectElement and mix in RootElementMixin.
Basically, I would like to write a fix that changes
class Foo extends RootRenderObjectElement { ... }

to
class Foo extends RenderObjectElement with RootElementMixin { ... }

Implementation note: I'd like to expand the capabilities of replacedBy to handle this case because it feels like we're replacing a class with another class and a mixin. I don't have a concrete proposal at this point for exactly how to do that.

This came up again, but this time in the other direction where we would want to fix
class Foo extends RenderObjectWidget with SlottedMultiChildRenderObjectWidgetMixin<S> {
  // ...
}

to
class Foo extends SlottedMultiChildRenderObjectWidget<S> {
  // ...
}

authors: bwilkerson goderbauer


text:
Dart 3.0.0-417.1.beta is breaking ffigen generated code

Context: https://github.com/flutter/codelabs/tree/main/ffigen_codelab/ and https://codelabs.developers.google.com/codelabs/flutter-ffigen
I'm attempting to update the codelabs for Flutter 3.10 and Dart 3, but I'm hitting a ffigen code generation issue. On attempting to build step_06 of the updated codelab, I'm seeing the following error logs:
In file included from /Users/brettmorgan/Documents/GitHub/codelabs/ffigen_codelab/step_06/macos/Classes/duktape.c:3:
duk_bi_date.c:311:34: warning: implicit conversion loses integer precision: 'unsigned long' to 'duk_uint_fast32_t' (aka 'unsigned int') [-Wshorten-64-to-32]
                        match_val = (1UL << part_idx) + (1UL << (sep_idx + 9)); /* match against rule part/sep bits */
                                  ~ ~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~
1 warning generated.
../lib/duktape_bindings_generated.dart:5155:7: Error: The type 'duk_thread_state' must be 'base', 'final' or 'sealed' because the supertype 'Struct' is 'base'.
Try adding 'base', 'final', or 'sealed' to the type.
class duk_thread_state extends ffi.Struct {
      ^
../lib/duktape_bindings_generated.dart:5160:7: Error: The type 'duk_memory_functions' must be 'base', 'final' or 'sealed' because the supertype 'Struct' is 'base'.
Try adding 'base', 'final', or 'sealed' to the type.
class duk_memory_functions extends ffi.Struct {
      ^
../lib/duktape_bindings_generated.dart:5186:7: Error: The type 'duk_function_list_entry' must be 'base', 'final' or 'sealed' because the supertype 'Struct' is 'base'.
Try adding 'base', 'final', or 'sealed' to the type.
class duk_function_list_entry extends ffi.Struct {
      ^
../lib/duktape_bindings_generated.dart:5212:7: Error: The type 'duk_hthread' must be 'base', 'final' or 'sealed' because the supertype 'Opaque' is 'base'.
Try adding 'base', 'final', or 'sealed' to the type.
class duk_hthread extends ffi.Opaque {}
      ^
../lib/duktape_bindings_generated.dart:5218:7: Error: The type 'duk_number_list_entry' must be 'base', 'final' or 'sealed' because the supertype 'Struct' is 'base'.
Try adding 'base', 'final', or 'sealed' to the type.
class duk_number_list_entry extends ffi.Struct {
      ^
../lib/duktape_bindings_generated.dart:5227:7: Error: The type 'duk_time_components' must be 'base', 'final' or 'sealed' because the supertype 'Struct' is 'base'.
Try adding 'base', 'final', or 'sealed' to the type.
class duk_time_components extends ffi.Struct {
      ^
Target kernel_snapshot failed: Exception

Command PhaseScriptExecution failed with a nonzero exit code
warning: Run script build phase 'Run Script' will be run during every build because it does not specify any outputs. To address this warning, either add output dependencies to the script phase, or configure it to run in every build by unchecking "Based on dependency analysis" in the script phase. (in target 'Flutter Assemble' from project 'Runner')
** BUILD FAILED **

Obligatory flutter doctor -v output:
$ flutter doctor -v
[✓] Flutter (Channel beta, 3.10.0-1.1.pre, on macOS 13.4 22F5037d darwin-arm64, locale en)
    • Flutter version 3.10.0-1.1.pre on channel beta at /Users/brettmorgan/flutter
    • Upstream repository https://github.com/flutter/flutter
    • Framework revision ecdb7841df (6 days ago), 2023-04-11 20:47:40 -0700
    • Engine revision 870e640e7e
    • Dart version 3.0.0 (build 3.0.0-417.1.beta)
    • DevTools version 2.23.1

[✓] Android toolchain - develop for Android devices (Android SDK version 33.0.0)
    • Android SDK at /Users/brettmorgan/Library/Android/sdk
    • Platform android-33, build-tools 33.0.0
    • Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 14.3)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 14E222b
    • CocoaPods version 1.11.3

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2021.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)

[✓] IntelliJ IDEA Community Edition (version 2022.2.2)
    • IntelliJ at /Applications/IntelliJ IDEA CE.app
    • Flutter plugin version 70.0.5
    • Dart plugin version 222.4167.21

[✓] VS Code (version 1.77.3)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.62.0

[✓] Connected device (2 available)
    • macOS (desktop) • macos  • darwin-arm64   • macOS 13.4 22F5037d darwin-arm64
    • Chrome (web)    • chrome • web-javascript • Google Chrome 112.0.5615.49

[✓] Network resources
    • All expected network resources are available.

• No issues found!

This is tracked in

https://github.com/dart-lang/ffigen/issues/538

I can make the change and a pre-release with the change.
(I could push a stable release with that behavior, it would require a 3.0 stable release lower bound. So I have to do that on release day instead.)

https://github.com/dart-lang/ffigen/pull/553

The PR passes on 3.0.0-417.0.dev, so nothing else needs to be fixed besides the class modifiers.

@domesticmouse could you check that everything works out when https://pub.dev/packages/ffigen/versions/8.0.0-dev.0 instead?

@domesticmouse could you check that everything works out when https://pub.dev/packages/ffigen/versions/8.0.0-dev.0 instead?

Yup working as intended: https://github.com/flutter/codelabs/pull/1561

authors: dcharkes domesticmouse


text:
Is there any document about ext.flutter.name or ext.flutter.inspector.name?

Hi, I'm looking into this document to make my Flutter inspector and vm client.
https://github.com/dart-lang/sdk/blob/main/runtime/vm/service/service.md#private-rpcs-types-and-properties
It's well-organized document about public RPCs like getVM (thank you!) but I couldn't find ext.flutter.name or ext.flutter.inspector.name methods although they look no private method.
I have looked into Flutter repository but I only found https://github.com/flutter/flutter/blob/7c860ddf970575d4ca6dd0ccc4bd823613a16359/packages/flutter/lib/src/widgets/widget_inspector.dart#L843
If I missed anything, I'm sorry but do you have any public document about them? and If no, why?

//cc @bkonyi

//cc @kenzieschmoll could you reply to this question.

@takassh I believe this is a Flutter issue, could you create a new issue for this under Flutter and close this.

It looks like you are referring to service extensions. Service extensions can be registered on the vm service and then called from another client. The Flutter inspector registers several service extensions in the Flutter framework that can be called from another VM service client like Dart DevTools. If you'd like to discuss further, please file a Flutter issue and tag me or reach out on the #hackers-devtools discord channel and we can discuss there (https://discord.com/channels/608014603317936148/1106667330093723668). It would be helpful to understand further what exactly you are trying to do.

authors: a-siva kenzieschmoll takassh


text:
Parser error with cascading

Not sure, but I think this is a parser error:
tests/language/crash_6725_test.dart:45:50: unexpected token '=='
  if (new DateTime.now()..millisecondsSinceEpoch == 42) {
                                                 ^^

Not a bug.

authors: mhausner peter-ahe-google


text:
Dart 2.0 problem

When I try to get packages, I get this error;
`The current Dart SDK version is 2.1.0-dev.9.4.flutter-f9ebf21297.
Because ***** depends on validator any which requires SDK version <2.0.0, version solving failed.
pub get failed (1)`

you need to specify the version to ^1.0.0 in your pubspec.yaml file:
dependencies:
  validators: ^1.0.0

you need to specify the version to ^1.0.0 in your pubspec.yaml file:
dependencies:
  validators: ^1.0.0

Because my_app depends on validator ^1.0.0 which doesn't match any versions, version solving failed.

The current Dart SDK version is 2.1.2-dev.0.0.flutter-0a7dcf17eb.
Because my_app depends on validator >=0.0.2 which requires SDK version <2.0.0, version solving failed.
pub get failed (1)

@willladislaw you need to upgrade to version 1.0.0 or greater. Previous versions don't support dart-2

authors: luisvt nomercy78 willladislaw

repo: dart-league/validators

license: MIT


text:
Optimistic scheme support for Ethereum/BSC <> Substrate interactive protocol

https://github.com/darwinia-network/darwinia-bridge-sol/pull/38#discussion_r654230826
Need draft design spec and protocol before implementation

Using Fee market to incentive relayer interact the protocol.
See https://github.com/darwinia-network/darwinia-common/issues/776

Using Fee market to incentive relayer interact the protocol.
See darwinia-network/darwinia-common#776

Different with the relayer fee market on the source chain, Optimistic scheme mentioned here is a sub protocol of beefy on target chain which need extra detail design.

In this document, describe some optimistic schema we need notice:
https://hackmd.io/ohOt4jAPT8uu-soJXHUq0Q#Optimistic-scheme

A protocol with competing initialization transactions for a given statement $S_n$ is only required whenever

the designated relayer for $S_n$ times out on the initialization transaction or submits a malicious statement $S'_n$
the designated relayer for $S_n$ times submits a valid initialization transaction, but times out on the finalization transaction

Malicious statement are expensive to detect since the full fault proof might need to include all grandpa sigs.
Currently, we simple take the strategy to slash the relayer who didn't do finalization transaction in time. or do finalization wrong.
For those initialization transaction the light client require, but no relayer do initialization transaction. we might require to take following actions. But I think it is unnecessary since other relayers will also help do it.
The only root cause for not doing it is due to the lack of incentive, or relay on demand design. Relayer fee market will help resolve the first one, and for relay on demand design, following actions are useful to optimize the gas rather than providing competing  initialization transactions.

Assuming that these costly situations are rare, we can take an optimistic approach where on the Polkadot side, we already commit to the relayer for $S_n$ much earlier and include their identity in the statement $S_{n-k}$, where $k$ is a tradeoff between the minimum distance between Polkadot blocks that we relay and the number of backup relayers, and the range of statements for which we must pre-commit to designated relayers.
This would then allocate a mutex for $S_n$ for a block range within which the smart contract will only accept initialization transactions from the pre-elected relayer. If and once the relayer chosen in $S_{n-k}$ times out on the initialisation transaction for $S_n$, the relayer for statement $S_{n-k+1}$ (who was elected for relaying statement $S_{n+1}$) attains the mutex for $S_n$ on the initialization transaction and submits this initialization transaction instead.
This scheme can be iterated for up to $k-1$ failures, at which point we must revert to a protocol with competing initialization transactions. As such, $k$ increases the number of backup relayers we can have to remain within the optimistic scheme, but thus also increases the impact a sequence of colluding designated relayers can have.
[^designation-motivation]: The motivation for having a designation procedure is that the second transaction will be very expensive, yet we will only refund one single relayer for sending it

repo: darwinia-network/darwinia-bridge-sol


text:
mvn compile error

I run mvn compile and find this error
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal com.googlecode.cmake-maven-project:cmake-maven-plugin:3.7.2-b1:generate (cmake-generate) on project dashj-bls: Return code: 1
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoExecutionException: Return code: 1
at com.googlecode.cmakemavenproject.GenerateMojo.execute (GenerateMojo.java:214)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)
who can help me ,greate thanks

Hi @leilaohu111!
First of all you need to update submodules:
git submodule update  --init --recursive

However it can be not enough. What OS are you working on?

repo: dashevo/dashj


text:
feat: withdrawals screen enhancements

Follow up of PR #17
This PR includes:

Auto-query at startup
Auto refresh when a filter is selected
Handling case when no withdrawal is matching the selected filters

Summary by CodeRabbit

New Features

Enhanced the Withdraws Status Screen with improved state management and user feedback during data loading.
Introduced a loading spinner to indicate data retrieval status.
Added pagination controls for better navigation of withdrawal data.

Improvements

Updated methods to return actions, allowing for better handling of user interactions and data requests.
Improved messaging for scenarios with no withdrawal data.

@coderabbitai review

https://github.com/coderabbitai review

source: gharchive/pull-request

repo: dashpay/dash-evo-tool


text:
Naively roundtrip parquet data from Spark

Currently it is not easy to roundtrip data between Spark and Dask Dataframe with Parquet.  There are a variety of details one needs to know to do this well.  This would be a good case study to improve our usability.
cc @martindurant

In particular I'm thinking that the following should probably just work for a variety of data types.
spark_df.write.parquet(fn)
dask_df = dd.read_parquet(fn)

dask_df.to_parquet(fn2)
spark.read.parquet(fn2)

Fastparquet does have a number of tests specifically for spark round-tripping: https://github.com/dask/fastparquet/blob/master/fastparquet/test/test_aroundtrips.py
(these are without dask's part, of course)

Understood.  To be clear, I'm not speaking about fastparquet, I'm talking about dask.dataframe.  My attempt to recreate the workflow above failed with dask dataframe.  I think that this is a fairly important issue.

I suspect that Martin feels your frustration :)
On Tue, Oct 16, 2018 at 5:23 AM Uwe L. Korn notifications@github.com
wrote:

We also have a flavor='spark' argument in pyarrow which handles some of
the limitations that the Spark Parquet implementation has. Sadly there are
still some features of Parquet that pyarrow uses but Spark cannot handle.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/dask/dask/issues/4096#issuecomment-430166110, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AASszLL-47f3N9EsCoLwfbBj7YcjzVK-ks5ulaWVgaJpZM4Xaqs5
.

Is this still of interest?

Could be contemplated in the context of #4336 , but interop with arrow more important now that spark per se (plus pyspark's inherent limitations) - so maybe the issue of rountripping to spark should be something that they care about more than we do. In any case, I don't think there are any plans to work on this. I am ambivalent on whether to close.

I care about this.  I think that it is important for the Dask project to be
able to easily move data back and forth.  I agree with @martindurant that
this might be partially solved by doing work in the Arrow codebase.  I'd
still like to keep track of it here.
On Tue, Apr 30, 2019 at 12:08 PM Martin Durant notifications@github.com
wrote:

Could be contemplated in the context of #4336
https://github.com/dask/dask/pull/4336 , but interop with arrow more
important now that spark per se (plus pyspark's inherent limitations) - so
maybe the issue of rountripping to spark should be something that they care
about more than we do. In any case, I don't think there are any plans to
work on this. I am ambivalent on whether to close.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/dask/dask/issues/4096#issuecomment-488035056, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AACKZTHLIZMSR5OOOIY6TOLPTB4HVANCNFSM4F3KVM4Q
.

authors: jakirkham martindurant mrocklin

repo: dask/dask


text:
from_dask_array with wrong dtype when passing index

Example:
import dask.dataframe as dd
import dask.array as da
import pandas as pd
import numpy as np

x = da.ones((10,), chunks=(5,))

def generate_df(size):
    d = {
        'dates':
        pd.date_range('1980-01-01', periods=size, freq='1T'),
        'signal1':
        np.random.normal(0, 0.3, size=size).cumsum() + 50,
    }

    ddf = dd.from_pandas(pd.DataFrame(d), npartitions=2).set_index('dates')
    return ddf

df = generate_df(10)
a = dd.from_dask_array(x, index=df.index, columns='ones')
a.reset_index().dtypes

Output:
index      int64
ones     float64
dtype: object

As far as I could understand from the source code, the index dtype is not populated to the meta index dtype. Is this a bug?

This is a bug - the index dtype should be propagated to the meta object (perhaps in dask.dataframe.io.io._meta_from_array). Would you be interested in fixing this issue?

@jcrist Yes, I will try to fix it

@jcrist I think this https://github.com/dask/dask/pull/4686 fix the issue.

authors: henriqueribeiro jcrist


text:
read_parquet with filters and arrow engine returns index in the columns

What happened:
When using dd.read_parquet with filters, the original index ends up in the columns
What you expected to happen:
Minimal Complete Verifiable Example:
import numpy as np
import pandas as pd
import dask.dataframe as dd
import dask.datasets
import shutil

N = 1_000
P = 4
b = np.arange(P).repeat(N // P)

df = pd.DataFrame({
    "A": np.random.random(size=N),
    "B": pd.Categorical(b)
})
ddf = dd.from_pandas(df, P)

shutil.rmtree("parquet-filters.parquet", ignore_errors=True)
ddf.to_parquet("parquet-filters.parquet", partition_on="B", engine="pyarrow")

dd.read_parquet("parquet-filters.parquet", engine="pyarrow", filters=[[("B", "=", 1)]]).compute()

Out[1]:
            A  index  B
0    0.410034    250  1
1    0.313085    251  1
2    0.848964    252  1
3    0.910770    253  1
4    0.922110    254  1
..        ...    ... ..
245  0.028301    495  1
246  0.392158    496  1
247  0.759204    497  1
248  0.122554    498  1
249  0.339143    499  1

[250 rows x 3 columns]

Anything else we need to know?:
Haven't debugged yet, but for the fastparquet engine the index is in the index, but the outupt is not filtered
   ...: ddf.to_parquet("parquet-filters.parquet", partition_on="B", engine="fastparquet")
   ...:
   ...: dd.read_parquet("parquet-filters.parquet", engine="fastparquet", filters=[[("B", "=", 1)]]).compute()

              A  B
index
0      0.304616  0
1      0.626666  0
2      0.986920  0
3      0.585114  0
4      0.663830  0
...         ... ..
995    0.999620  3
996    0.223592  3
997    0.796024  3
998    0.063832  3
999    0.621195  3

[1000 rows x 2 columns]

Environment:

Dask version: master
Python version: 3.7

cc @dask/io :)

A couple other notes:

Why is the column called index, when the original index didn't have a name? Is that expected?
If I set df.index.name = 'myindex before writing, and manually specify index='myindex', things are correct for pyarrow. But index=None doesn't auto-detect, which narrows it to PyArrowEngine.read_metadata.

The fastparquet version does work, but with this syntax:
dd.read_parquet("parquet-filters.parquet", engine="fastparquet", filters=[("B", "==", 1)]).compute()

(no double list, but double equals)

Thanks. Do you know if there's a standard for which expressions should be allowed there?

fastparquet did it first, so it must be right :) https://fastparquet.readthedocs.io/en/latest/api.html#fastparquet.ParquetFile.to_pandas

Why is the column called index

This is a default workaround, since the column in parquet cannot have no name. However, the original "None" should have been saved in the global pandas metadata.

Thanks for raising @TomAugspurger.  Martin is absolutely correct about the reason for the "index" name.  I'm not sure why the index is not being detected, but I will investigate a bit

The problem is likely in the _write_partitioned workaround to ARROW-8244. @jsignell  pointed out here that we can likely go back to using pyarrow's write_to_dataset.  I will try to fix the issue and work in write_to_dataset for newer pyarrow versions.

@rjzamora did you see https://issues.apache.org/jira/browse/ARROW-7782?

did you see https://issues.apache.org/jira/browse/ARROW-7782? Might be relevant.

Nope - Seems like the same bug we have since the logic is mostly copied :)

@rjzamora : OK to close?

@rjzamora : OK to close?

Yes  We should probably close for now.  However, it is worth mentioning that there are a two minor points that have not been addressed yet:

There is a slightly different filter syntax expected by the different engines, and I think this could be documented a bit better
We are still not using the pyrrow write_to_dataset implementation, and we may be able to simplify the dask code by using the improved version

Let's make those new issues. The former should be simple enough that we can accept either form for either backend, I imagine.

authors: TomAugspurger martindurant rjzamora


text:
block_info kwarg from map_blocks - error when mixing array input with dataframe/series output

What happened:
You cannot use the block_info keyword argument with map_blocks if the processing function returns anything other than an array. For example, you can't use the block_info kwarg with functions returning Series or DataFrame objects (similar to the examples in this blogpost).
This might be more of an edge case that isn't supported, rather than a bug. Still I feel obligated to mention it, now I've come across it.
What you expected to happen:
I expected things that worked with map_blocks to work the same way, regardless of whether the block_info keyword argument was passed in or not.
Minimal Complete Verifiable Example:
import dask.array as da
import pandas as pd

def func(x, block_info=None):
    return pd.DataFrame({'x': [1,2,3], "y": [4,5,6]})

arr = da.random.random((100,100), chunks=(50,50))
result = arr.map_blocks(func, drop_axis=1, dtype=float)

Error message (click to expand details):
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-22-dde43f1e2f6d> in <module>
----> 1 result = arr.map_blocks(func, drop_axis=1, dtype=float)

~/anaconda3/envs/dask/lib/python3.8/site-packages/dask/array/core.py in map_blocks(self, func, *args, **kwargs)
   2362     @wraps(map_blocks)
   2363     def map_blocks(self, func, *args, **kwargs):
-> 2364         return map_blocks(func, self, *args, **kwargs)
   2365
   2366     def map_overlap(self, func, depth, boundary=None, trim=True, **kwargs):

~/anaconda3/envs/dask/lib/python3.8/site-packages/dask/array/core.py in map_blocks(func, name, token, dtype, chunks, drop_axis, new_axis, meta, *args, **kwargs)
    766                     ]
    767                     num_chunks[i] = arg.numblocks
--> 768         out_starts = [cached_cumsum(c, initial_zero=True) for c in out.chunks]
    769
    770         block_info_name = "block-info-" + out.name

~/anaconda3/envs/dask/lib/python3.8/site-packages/dask/dataframe/core.py in __getattr__(self, key)
   3993             return self[key]
   3994         else:
-> 3995             raise AttributeError("'DataFrame' object has no attribute %r" % key)
   3996
   3997     def __dir__(self):

AttributeError: 'DataFrame' object has no attribute 'chunks'

If you take block_info=None out of the function signature, the code above works fine.
Anything else we need to know?:
We run into this error because there is no .chunks attribute with this kind of non-array output. Here is the relevant line of code:
https://github.com/dask/dask/blob/7772ee32352daf800561ac08fc05f4ba1546e88f/dask/array/core.py#L773
Environment:

Dask version: 2021.06.2'
Python version: 3.8
Operating System: Ubuntu 20.04 LTS
Install method (conda, pip, source): conda

Thanks for writing this up @GenevieveBuckley! I didn't know you could create dataframes from array.map_blocks like this. I'll take a look.

Huh. So block_id and block_info both use the output as well as the input and they both assume that the output is a dask.Array. I am wondering if this should be treated as a supported workflow. If not we should raise a warning as soon know that the output is not a dask.Array. If we do want to support it then we should generalize the block_id and block_info logic.
I am leaning towards explicitly forbidding this workflow.

This is right up the alley of something I ran into while working on dask-histogram. In that case, I was looking for a way to have the behavior of map_blocks while returning a custom (histogram) object. To me the map_blocks API is quite rigidly "array in and array out" (with the dtype, chunks, and meta argument docstrings making that pretty explicit). I reached for dask.delayed-- just calling to_delayed on the input array and passing the delayed blocks to another delayed function that returns a histogram object.
what I do:
@delayed
def _delayed_fill(data, axes)
    h = Histogram(axes)
    h.fill(data)
    return h

def fill(data, template):
    array_parts = data.to_delayed()
    axes = *template.axes
    fills = [_delayed_fill(a, axes) for a in array_parts]
    # could also be
    # fills = [_delayed_fill(a, axes) for a  in data.blocks]
    final_result = delayed(sum)(fills)
    return final_result

x = dask_array_factory(...)
h = Histogram(...)
h = fill(x, h)
h.compute()

Would it make sense to provide an API that allows map_blocks like behavior but instead of returning an array, explicitly return a Delayed object? My example above would become:
def fill(data, axes):
    h = Histogram(axes).fill(data)

x = dask_array_factory(...)
h = Histogram(...)
axes = *h.axes
h = x.map_blocks_custom(fill, axes)
h.compute()

At the end of the day this would be syntactic sugar, as explicitly taking advantage of delayed works fine.

A similar conversation has been happening on #4148. I have yet to see a really compelling reason why using delayed (like you are saying Doug) or iterating over the blocks. is not sufficient. But I am totally open to being shown such a reason :)

Yes, iterating over the blocks and using delayed is what I'm using instead.
I'm happy for https://github.com/dask/dask/issues/7921 (this) and https://github.com/dask/dask/issues/4148 to be closed as out of scope. I mostly just wanted to make sure to make a record of what I found (there are discussions elsewhere about why can't we just use $foo instead).
I would be less happy if there was a move to forbid mixed output entirely, particularly after publicizing that you can. (Maybe you hadn't realized I was going to highlight that, which is fine - I also hadn't realized you might want to ban it).

A similar conversation has been happening on #4148. I have yet to see a really compelling reason why using delayed (like you are saying Doug) or iterating over the blocks. is not sufficient. But I am totally open to being shown such a reason :)

@jakirkham might have an opinion here.
(Tagging him in since he mentioned memory issues as a potential problem elsewhere. I don't fully understand that concern though, so it might not be relevant here)

Fair enough. I think for now we can raise an error sooner that is very explicit about block_id and block_info not being supported for this kind of output.

authors: GenevieveBuckley douglasdavis jsignell


text:
DataFrame groupby+apply/transform/shift on index results in unwanted shuffle

What happened:
I'm working on pandas 1.5 compatibility (cf #8776) and came across this apparent bug in groupby + transform/apply/shift.
If we are grouping on the dataframe index, we should be able to avoid a shuffle, since the dataframe is already ordered in the way we want. Indeed, the groupby code has a check to avoid shuffling:
https://github.com/dask/dask/blob/99e260081f51ce368bf0456c66c9d3c2f20e8c9b/dask/dataframe/groupby.py#L1724
However, this check does not succeed when you pass in the dask dataframe index directly, as the check only looks for column names.
I think we can fix this by also checking if the dd.Index dask name matches the grouped-by object's name (pseudocode, since by could be a string):
 should_shuffle = not (
    df.known_divisions and
    (df._contains_index_name(self.by) or df.index._name == self.by._name)
)

What you expected to happen:
Grouping by ddf.index should avoid a shuffle.
Minimal Complete Verifiable Example:
import dask
import dask.dataframe as dd
import pandas

# use task-based rather than disk based for easier introspection.
# Not otherwise important here
dask.config.set({"shuffle": "tasks"})

df = pandas.DataFrame(
    {"x": [1, 2, 3, 4]},
    index=pandas.Index(["a", "a", "b", "b"], name="idx"),
)
ddf = dd.from_pandas(df, npartitions=2)

# Groupby index name avoids shuffle
ddf = ddf.groupby("idx").apply(lambda x: x, meta=ddf._meta)
assert not any(isinstance(l, dask.layers.SimpleShuffleLayer) for l in ddf.dask.layers.values())

# Groupby index object shuffles!
ddf = ddf.groupby(ddf.index).apply(lambda x: x, meta=ddf._meta)
assert not any(isinstance(l, dask.layers.SimpleShuffleLayer) for l in ddf.dask.layers.values())

Anything else we need to know?:
Environment:

Dask version: main
Python version: 3.8
Operating System: ubuntu
Install method (conda, pip, source): source

Ah, I'm realizing that this is closely related to #8442 (though perhaps this is a simpler case, where by is exactly ddf.index.


text:
Enable drop to support mismatched partitions

This PR sets enforce_metadata=False in drop (as we do in several other methods) to allow for cases where some partitions are missing columns (as happens sometimes with CSV data)

rerun tests


text:
ENH: dataframe.from_pandas ... accept npartitions or chunksize

Hello,
In the project I'm building, I've found it much more convenient if I can make a constant dataframe of the same size and with the same shape as my from_bcolz data frame. (So the dataframe is just a column of zero's or one's or whatnot.)
I wanted the divisions to be the same as well to make the processing easier.... and this was the easiest route to that.
Let me know your thoughts.
Thanks.

+1 on the content.  This looks great!
Can I ask you to rebase in the whoopsie commit?  Or perhaps all into one for something of this size?

So this is where I confess I'm not too good with git.
I think what you are suggesting is that I do the following
git checkout master
git fetch origin master
git checkout pd_dataframe_chunksize
git rebase master
Is that right? I've screwed up git a lot.

Wait wait...
git checkout master
git fetch upstream master
git checkout pd_dataframe_chunksize
git rebase master

Close.  I'm suggesting that you do the following:
git checkout master
git fetch upstream master
git checkout pd_dataframe_chunksize
git rebase -i master

Then, change pick on the left to squash on the commits that should arguably have been written in the above commit.  In your case it will probably look like the following:
pick a0b0a3c ENH: Allow dataframe.from_pandas to accept chunksize or npartitions
squash 0d9913b DOC: Wrote a little more documentation
squash 5c6dcb7 DOC: Whoopsie...

Then you save/exit and rewrite the docstring.
Finally you force push back up to your fork on github.  As a warning, this is where you can lose information.
git push -f origin pd_dataframe_chunksize

Alternatively, I can do all of this for you.  It's a decent thing to learn how to do though.

Yep I screwed it up.
Let me make a new pull request. I think I should have squashed the ones above this commit as well.

authors: mrocklin thequackdaddy