text:
Introduce alias for dapr app id

In what area(s)?

/area runtime

/area operator

/area placement

/area docs

/area test-and-release

Describe the feature
Currently it is is only possible to provide one app id.
This makes migration scenarios where the app id should change hard.
Currently it is only possible to define via annotation the app id.
dapr.io/app-id
For a migration scenario it would be great to have an additional annotation like.
dapr.io/app-id-alias
The app can be called via service invocation using the app-id or the app-id-alias.
This would make a migration easier especially in larger environments.
Release Note

ADD Add app id alias.

keep alive

keep alive


text:
add file extension check when loading subscription declaratively in selfhosted mode

Signed-off-by: wwbweibo wwbweibo@outlook.com
Description
add file extension check when loading subscription declaratively in selfhosted mode

Issue reference

Please reference the issue this PR will close: #4641
Checklist
Please make sure you've  completed the relevant tasks for this PR, out of the following list:

[x] Code compiles correctly
[x] Created/updated tests
[x] Unit tests passing
[x] End-to-end tests passing
[ ] Extended the documentation / Created issue in the https://github.com/dapr/docs/ repo: dapr/docs#[issue number]
[ ] Specification has been updated / Created issue in the https://github.com/dapr/docs/ repo: dapr/docs#[issue number]
[ ] Provided sample for the feature / Created issue in the https://github.com/dapr/docs/ repo: dapr/docs#[issue number]

If we check where os.ReadDir() gets called in runtime project, we'll notice there are three usages which all intend to load yaml config files:

standalone_loader.go:49 verify file extension:

// LoadComponents loads dapr components from a given directory.
func (s *StandaloneComponents) LoadComponents() ([]components_v1alpha1.Component, error) {
	files, err := os.ReadDir(s.config.ComponentsPath)
	if err != nil {
		return nil, err
	}

	list := []components_v1alpha1.Component{}

	for _, file := range files {
		if !file.IsDir() && s.isYaml(file.Name()) {
			components := s.loadComponentsFromFile(file.Name())
			if len(components) > 0 {
				list = append(list, components...)
			}
		}
	}

	return list, nil
}

resiliency.go:152

// LoadStandaloneResiliency loads resiliency configurations from a file path.
func LoadStandaloneResiliency(log logger.Logger, runtimeID, path string) []*resiliency_v1alpha.Resiliency {
	if _, err := os.Stat(path); os.IsNotExist(err) {
		return nil
	}

	files, err := os.ReadDir(path)
	if err != nil {
		log.Errorf("failed to read resiliences from path %s: %s", err)
		return nil
	}

subscriptions.go:225

// DeclarativeSelfHosted loads subscriptions from the given components path.
func DeclarativeSelfHosted(componentsPath string, log logger.Logger) []Subscription {
	var subs []Subscription

	if _, err := os.Stat(componentsPath); os.IsNotExist(err) {
		return subs
	}

	files, err := os.ReadDir(componentsPath)
	if err != nil {
		log.Errorf("failed to read subscriptions from path %s: %s", err)
		return subs
	}

We should make sure all these three places behave same, either more strict or more relax. I personally support adding file ext check.

That’s a good point, this could be a breaking change.
perhaps this could be handled by showing a warning first?

In Dapr 1.8 when we find a file without extension we continue loading it (in all places), but show a warning with a link to our preview features page ( https://docs.dapr.io/operations/support/support-preview-features/ )
In a future version (probably Dapr 1.10?) we stop loading those files entirely.

I agree with @beiwei30 that we should modify all methods that read files to have a file extension check.
@wwbweibo If you can please make the following changes:

Add file extension checks to the methods listed above
Add a warning log message whenever we load a file with the wrong extension, prompting that it will not be loaded

Re: breaking changes, I think we're ok here considering the backward compatibility guarantees are applied to our APIs and CRD schemas plus the warning message.

I have added the warning message for the three file listed above, and moved the IsYaml file to utils package.

@wwbweibo please address the comments so we can merge this.

I force pushed some code, so I can not address the requested change. I'm so sorry.

Thank you for this contribution @wwbweibo.

source: gharchive/pull-request

repo: dapr/dapr

license_type: permissive

license_source: github-api


text:
sentry: preserve original Secret metadata

When storeKubernetes writes a secret it overrides existing metadata. We expect secret to be created in advance, so we must preserve original metadata such as lables (and annotations), which is usefull to keep secret under helm management.
Signed-off-by: Dinar Valeev dinar.valeev@absa.africa
Description

Issue reference

Please reference the issue this PR will close: #[issue number]
Checklist
Please make sure you've  completed the relevant tasks for this PR, out of the following list:

[ ] Code compiles correctly
[ ] Created/updated tests
[ ] Unit tests passing
[ ] End-to-end tests passing
[ ] Extended the documentation / Created issue in the https://github.com/dapr/docs/ repo: dapr/docs#[issue number]
[ ] Specification has been updated / Created issue in the https://github.com/dapr/docs/ repo: dapr/docs#[issue number]
[ ] Provided sample for the feature / Created issue in the https://github.com/dapr/docs/ repo: dapr/docs#[issue number]

/ok-to-test

@yaron2 Ah, sorry for that. Fixed

Thanks for your contribution @k0da.
We will need to test and validate our K8s upgrade scenarios to make sure this doesn't break anything.


text:
docker exec -it dapr_redis redis-cli did not return anything

Describe the issue
after curl http://localhost:3500/v1.0/state/statestore/name
it returns the following
"Bruce Wayne"%
then  docker exec -it dapr_redis redis-cli
did not return what expected, it says:
127.0.0.1:6379>
then run : keys *
it returns : (empty array)
URL of the docs

https://docs.dapr.io/getting-started/get-started-api/
Expected content

Output:

"myapp||name"
Screenshots

Additional context

Thanks for your feedback @peachblossomgame - the docs example here works for me and I'm getting the expected output. Could you provide any more details so we can help resolve this for you?

@peachblossomgame  If you have more details here please add them and reopen this issues.

Same issue. On windows.
Also went to running container in docker desktop, opened cli, ran redis-cli, and looked for keys. no keys. A powershell request to the state store returned "Bruce Wayne" so it must be stored somewhere!
Solved: I had a non-containerized standalone redis-server already running on port 6379, the post to http://localhost:3500/v1.0/state/statestore had stored state there.
Is this expected behaviour? I would have thought Dapr would still have used the containerized redis?

source: gharchive/issue

authors: hhunter-ms msfussell peachblossomgame white-label-development

repo: dapr/docs


text:
GetStateAsync throws exception if key not found when using json contentType

Ask your question here
I am using redis for state store component with RedisJSON module, and when getting state with contentType metadata set to application/json, method GetStateAsync on the dapr client throws the following exception if the key doesn't exist:
Error when trying to Retrieve ... with Id: ..."Error","@x":"Dapr.DaprException:
State operation failed: the Dapr endpoint indicated a failure. See InnerException for details.
Grpc.Core.RpcException: Status(StatusCode=\"Internal\", Detail=\"fail to get <...> from state store local: redis: nil\")
   at Dapr.Client.DaprClientGrpc.GetStateAsync[TValue](String storeName, String key, Nullable`1 consistencyMode, IReadOnlyDictionary`2 metadata, CancellationToken cancellationToken)

Is this an intended behavior or more like a bug? When not using any metadata the return value from GetStateAsync is null with no exception thrown if the key doesn't exist.
Thanks!

@MargaretKrutikova - I think we're supposed to return null instead of erroring out. How exactly are you setting the metadata? The format of from state store local: redis: nil is concerning for me. It looks like something may not be set right in the actual request.

I apologize for the late reply, this how we are doing it:
Dictionary<string, string> metadata = new() { { "contentType", "application/json" } };
var data = await daprClient.GetStateAsync<SomeData>(config.StoreName, dataId, metadata: metadata);

Is there anything we are missing here?

Hello,
I have the same problem.
I had to change in my code to explicitly set the content type to json, to support the query api.
And now I get the surprise of getting errors when keys not found...
Thanks!

We have also started to get this issue, also after adding the "contentType", "application/json" metadata so we could use querying with redis

I encountered the same issue.
I used image docker.io/redis/redis-stack-server:latest instead of the Redis pulled by dapr init to enable ReJSON.
When specifying metadata.contentType=application/json, it throws an error if the key does not exist.
I noticed that the message "fail to get test from state store statestore: redis: nil" is returned from Dapr's state service.
curl 'http://localhost:3500/v1.0/state/statestore/test' -v
* Host localhost:3500 was resolved.
* IPv6: ::1
* IPv4: 127.0.0.1
*   Trying [::1]:3500...
* Connected to localhost (::1) port 3500
> GET /v1.0/state/statestore/test HTTP/1.1
> Host: localhost:3500
> User-Agent: curl/8.7.1
> Accept: */*
>
* Request completely sent off
< HTTP/1.1 204 No Content
< Content-Type: text/plain; charset=utf-8
< Traceparent: 00-4f62151a9cfab3eeb5fc6bc0c4c4889f-50b6560d85adeb3e-01
< Date: Fri, 06 Sep 2024 04:33:06 GMT
<
* Connection #0 to host localhost left intact

curl 'http://localhost:3500/v1.0/state/statestore/test?metadata.contentType=application/json' -v
* Host localhost:3500 was resolved.
* IPv6: ::1
* IPv4: 127.0.0.1
*   Trying [::1]:3500...
* Connected to localhost (::1) port 3500
> GET /v1.0/state/statestore/test?metadata.contentType=application/json HTTP/1.1
> Host: localhost:3500
> User-Agent: curl/8.7.1
> Accept: */*
>
* Request completely sent off
< HTTP/1.1 500 Internal Server Error
< Content-Type: application/json
< Traceparent: 00-b6bf3bdefcded5578f8aebada999418d-0d819f29f861d7b8-01
< Date: Fri, 06 Sep 2024 04:33:40 GMT
< Content-Length: 98
<
* Connection #0 to host localhost left intact
{"errorCode":"ERR_STATE_GET","message":"fail to get test from state store statestore: redis: nil"}

authors: CormacLennon MargaretKrutikova halspang kenchou robert2ship

repo: dapr/dotnet-sdk


text:
Update README.md

Updated the supported version

Resolved with https://github.com/dapr/samples/pull/74.

repo: dapr/samples


text:
Addition Subfolder created in Get-SFTPItem

SFTP Server:  OpenSSH 7.4 on RHEL 7.4
I would like to transfer the whole backup folder, its sub-directories, and files to a CIFS share eventually.  To start, make the destination to a local Windows folder.
$srcPath = "/folder/subfolderA "
$destPath = "c:\destFolder"
Get-SFTPItem -SessionId 0 -Path $srcPath -Destion $destPath
I expected the result to be:
c:\destFolder\file1.txt
c:\destFolder\file2.txt, etc.
Instead, it created an additional sub-folder from the srcPath and the result is like:
c:\destFolder\subfolderA\file1.txt
c:\destFolder\subfolderA\file2.txt, etc.
Intriguingly, if I ran Set-SFTPLocation first with $srcPath, then run
Get-SFTPItem -SessionId 0 -Path . -Destion $destPath
The additional subfolder would still be createdl, but this time named as "v1.0" (if executing from PowerISE) like:
c:\destFolder\v1.0\file1.txt
c:\destFolder\v1.0.\file2.txt, etc.
In both scenarios above, when executing Get-SFTPItem, the powershell session would crush and restart, but files transfer and creation of folders appear to complete before that happens.
In contrast, if executing in the powershell console, the additional sub-directory name created would be the login username (e.g. johndoe) instead of" v1.0", and powershell console shell would also crush and restart.
I wonder anyone experience the same symptoms and a fix may be available soon.  I plan to run ths powershell script in a schedule task.  If the powershell session keeps crushing, the schedule task would work right.  Thanks.

I think it is behaving like it should since it is "*Path" the parameter, you specify the location to place the file or folder, it behaves just like the Item cmdlets. by "crushing" you mean crashing?  Will look as to why it is crashing on ISE, I do most of my development in code so I rarely use the ISE.

You ca see on the first case it is behaving as expected. The example in help shows the way it behaves. on the second case can you do a -verbose?
PS C:\Users\carlos\Documents\Posh-SSH\Release> help Get-SFTPItem -Examples

NAME
    Get-SFTPItem

SYNOPSIS
    Downloads via SFTP an item from a SSH server.

    Example 1

    PS C:\>Get-SFTPItem -SessionId 0 -Path .ssh -Destination ./ -Verbose

    Downloads the .ssh folder from the server to the current directory.

authors: ScriptXaaS darkoperator

repo: darkoperator/Posh-SSH


text:
Video post

When I post video, I get error.
this error = array(3) { ["code"]=> int(390) ["message"]=> string(64) "There was a problem uploading your video file. Please try again." ["data"]=> array(0) { } }
	$file = "domain/path/SampleVideo.mp4";

	$description = "First video";

	$title = "first video title";

	$resutl = $this->facebook->publish_video($file, $description, $title);

	if ($resutl) {
		var_dump($resutl);
	}else {
		echo "error";
	}

The error is from Facebook, so it can be anything. Maybe incorrect file format, or the file size is to large or the encoding is not supported.
I would recommend looking into the Facebook documentation to see what the video graph endpoint support and not.

authors: cagdascakmakci darkwhispering

license: mit

license_source: bigquery


text:
Missing regcomp/regexec/regfree while compiling with Visual Studio

Hi,
i'm trying to compile the pgtt.c file with visual studio, but ends with unresolved external symbol for the function calls of regcomp/regexec/regfree.
I try to write a patch for these functions using pg_regcomp/pg_regexec/pg_regfree, but the parameters are not matching.
   static int regcomp(regex_t* re, char const  *string, int flags)
  {
     auto len = strlen(string); //right??
         auto collation = LC_COLLATE;//???
      return pg_regcomp(re, string, **len**, flags, **colation**);
  }
 static int regexec(const regex_t* restrict preg, const char* restrict string,
      size_t nmatch, regmatch_t pmatch[restrict], int eflags)
  {
      auto len = strlen(string); // right?
      auto search_start = ??;
          rm_detail_t * details = ????
      return pg_regexec(preg, string, len, **search_start, details ,**nmatch, pmatch, eflags);
  }

Do you have any idea, what values for collation in regcomp and search_start, details in regexec?
Regards,
Lanlan

What is your PostgreSQL version ?

What is your PostgreSQL version ?

13

Can you post the exact error reported by make?

Can you post the exact error reported by make?
I'm using visual studio under windows, it compiles the c file with msvc.

Build started...
1>------ Build started: Project: gptt, Configuration: Debug x64 ------
1>pgtt.c
1>C:\Program Files\PostgreSQL\13\include\server\nodes\pg_list.h(314,11): warning C4244: 'return': conversion from '__int64' to 'int', possible loss of data
1>C:\Program Files\PostgreSQL\13\include\server\libpq\pqformat.h(116,10): warning C4267: '=': conversion from 'size_t' to 'int', possible loss of data
1>C:\Program Files\PostgreSQL\13\include\server\libpq\pqformat.h(111,15): warning C4267: 'initializing': conversion from 'size_t' to 'int', possible loss of data
1>C:\Work\Program\pgtt-2.10\pgtt.c(297,6): warning C4013: 'regcomp' undefined; assuming extern returning int
1>C:\Work\Program\pgtt-2.10\pgtt.c(363,2): warning C4013: 'regfree' undefined; assuming extern returning int
1>C:\Work\Program\pgtt-2.10\pgtt.c(486,26): warning C4267: '=': conversion from 'size_t' to 'int', possible loss of data
1>C:\Work\Program\pgtt-2.10\pgtt.c(525,8): warning C4013: 'regexec' undefined; assuming extern returning int
1>C:\Work\Program\pgtt-2.10\pgtt.c(1120,22): warning C4244: 'return': conversion from '__int64' to 'int', possible loss of data
1>C:\Work\Program\pgtt-2.10\pgtt.c(1349,31): warning C4047: 'function': 'const char *' differs in levels of indirection from 'int'
1>C:\Work\Program\pgtt-2.10\pgtt.c(1349,27): warning C4024: 'GetGttByName': different types for formal and actual parameter 1
1>C:\Work\Program\pgtt-2.10\pgtt.c(1351,4): warning C4022: 'hash_search': pointer mismatch for actual parameter 2
1>C:\Work\Program\pgtt-2.10\pgtt.c(1348,9): warning C4047: 'initializing': 'int' differs in levels of indirection from 'char *'
1>   Creating library C:\postgresql_ext_pgtt\gptt\x64\Debug\gptt.lib and object C:\Work\workspaces\GC Products\postgresql_ext_pgtt\gptt\x64\Debug\gptt.exp
1>pgtt.obj : error LNK2019: unresolved external symbol regcomp referenced in function _PG_init
1>pgtt.obj : error LNK2019: unresolved external symbol regfree referenced in function exitHook
1>pgtt.obj : error LNK2019: unresolved external symbol regexec referenced in function gtt_check_command
1>C:\postgresql_ext_pgtt\gptt\x64\Debug\gptt.dll : fatal error LNK1120: 3 unresolved externals
1>Done building project "gptt.vcxproj" -- FAILED.
========== Build: 0 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========
========== Elapsed 00:01,794 ==========
I can try to build it with make. :)

Can you post the exact error reported by make?

now i tried with gnu make to compile it, but it complains that there's no target...

In commit 5a5381b I have rewritten pgtt.c to use the pg_reg* functions. Please give it a try and let me know.
Note that port to PG 16 is broken, I will try to fix that later if the fix is the right one.

In commit 5a5381b I have rewritten pgtt.c to use the pg_reg* functions. Please give it a try and let me know. Note that port to PG 16 is broken, I will try to fix that later if the fix is the right one.

Many thanks! It compiles under windows.
There's only one  small difference on the elog marco using #if #else #endif.
In windows should look like this:
#if PG_VERSION_NUM < 150000
elog(DEBUG1, "looking for dropping table: %s", relationNameValue->val.str);
#else
elog(DEBUG1, "looking for dropping table: %s", relationNameValue->sval);
#endif
#if PG_VERSION_NUM < 150000
elog(DEBUG1, "looking if table %s is registered as GTT",
relationNameValue->val.str);
gtt_unregister_gtt_not_cached(relationNameValue->val.str);
#else
elog(DEBUG1, "looking if table %s is registered as GTT",
relationNameValue->sval);
gtt_unregister_gtt_not_cached(relationNameValue->sval);
#endif
Otherwise it is perfect! ;-)
I will test the extension afterwards in postgresql.
Regards,
Lanlan

Commit 8d6851d applies your patch.

repo: darold/pgtt

license: isc


text:
ENH: validate the zoom level

xref https://github.com/darribas/contextily/issues/67
(doesn't fully close the issue, as I don't clip the zoom level if it is above automatically)

Coverage increased (+0.3%) to 93.261% when pulling f6e91637df778c3ccf588451b15838bd1b0e28d9 on jorisvandenbossche:max-zoom into 3e48cf94e7670deaf9c0f9e1c24557dcda9db1d9 on darribas:master.

This looks great @jorisvandenbossche, thanks very much!!! I think this in itself is useful functionality so would be happy to merge, unless you want to keep pushing and add a re-assignment of zoom into the closest possible level (with a warning that that's happened)? Happy either way.

@darribas I added the logic to re-assign zoom to the max level, in case a too large value was inferred and a max_zoom is known.

authors: coveralls darribas jorisvandenbossche

repo: darribas/contextily


text:
Stop button (in debugger) looks always enabled

The Stop button worked (it stopped by app), but now it looks like I can keep stopping it.
Suggestion: gray-out or disable the Stop button if the app isn't running.

This is tied into what it means for a flutter app to be running.

authors: devoncarew sethladd

repo: dart-atom/dartlang


text:
Remove problematic counter interface and prepare for 2.1.1.

The sharded counter support has bugs in it as it doesn't handle exceptions for cancelled datastore transactions (which can be part of normal operation).  This could be fixed, but everything we need counters for we can get better via stackdriver and some additional logging.  So, just remove it.
Also update version number for a 2.1.1 release.

example of log entry from a failed datastore transaction, that this PR will fix:
Uncaught error in request handler: The transaction was aborted.
#0      GrpcDatastoreImpl.commit (package:appengine/src/grpc_api_impl/datastore_impl.dart:105:9)
<asynchronous suspension>
#1      _commitHelper (file:///root/.pub-cache/hosted/pub.dartlang.org/gcloud-0.6.0+2/lib/src/db/db.dart:387:8)
#2      Transaction.commit (file:///root/.pub-cache/hosted/pub.dartlang.org/gcloud-0.6.0+2/lib/src/db/db.dart:93:12)
#3      Counter.increment.<anonymous closure>.<anonymous closure>.<anonymous closure> (package:dart_services/src/sharded_counter.dart:27:30)
#4      _rootRunUnary (dart:async/zone.dart:1132:38)
#5      _CustomZone.runUnary (dart:async/zone.dart:1029:19)
#6      _FutureListener.handleValue (dart:async/future_impl.dart:126:18)
#7      Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:639:45)
#8      Future._propagateToListeners (dart:async/future_impl.dart:668:32)
#9      Future._complete (dart:async/future_impl.dart:473:7)
#10     _SyncCompleter.complete (dart:async/future_impl.dart:51:12)
#11     _AsyncAwaitCompleter.complete (dart:async/runtime/libasync_patch.dart:28:18)
#12     _completeOnAsyncReturn (dart:async/runtime/libasync_patch.dart:294:13)
#13     GrpcDatastoreImpl.lookup (package:appengine/src/grpc_api_impl/datastore_impl.dart)
<asynchronous suspension>
#14     _lookupHelper (file:///root/.pub-cache/hosted/pub.dartlang.org/gcloud-0.6.0+2/lib/src/db/db.dart:407:8)
#15     Transaction.lookup (file:///root/.pub-cache/hosted/pub.dartlang.org/gcloud-0.6.0+2/lib/src/db/db.dart:41:12)
#16     Counter.increment.<anonymous closure>.<anonymous closure> (package:dart_services/src/sharded_counter.dart:22:28)
#17     DatastoreDB.withTransaction.<anonymous closure> (file:///root/.pub-cache/hosted/pub.dartlang.org/gcloud-0.6.0+2/lib/src/db/db.dart:305:32)
#18     _rootRunUnary (dart:async/zone.dart:1132:38)
#19     _CustomZone.runUnary (dart:async/zone.dart:1029:19)
#20     _FutureListener.handleValue (dart:async/future_impl.dart:126:18)
#21     Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:639:45)
#22     Future._propagateToListeners (dart:async/future_impl.dart:668:32)
#23     Future._complete (dart:async/future_impl.dart:473:7)
#24     _SyncCompleter.complete (dart:async/future_impl.dart:51:12)
#25     _AsyncAwaitCompleter.complete (dart:async/runtime/libasync_patch.dart:28:18)
#26     _completeOnAsyncReturn (dart:async/runtime/libasync_patch.dart:294:13)
#27     GrpcDatastoreImpl.beginTransaction (package:appengine/src/grpc_api_impl/datastore_impl.dart)
<asynchronous suspension>
#28     DatastoreDB.withTransaction (file:///root/.pub-cache/hosted/pub.dartlang.org/gcloud-0.6.0+2/lib/src/db/db.dart:302:10)
#29     Counter.increment.<anonymous closure> (package:dart_services/src/sharded_counter.dart:21:24)
#30     _rootRunUnary (dart:async/zone.dart:1132:38)
#31     _CustomZone.runUnary (dart:async/zone.dart:1029:19)
#32     _FutureListener.handleValue (dart:async/future_impl.dart:126:18)
#33     Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:639:45)
#34     Future._propagateToListeners (dart:async/future_impl.dart:668:32)
#35     Future._completeWithValue (dart:async/future_impl.dart:483:5)
#36     Future._asyncComplete.<anonymous closure> (dart:async/future_impl.dart:513:7)
#37     _rootRun (dart:async/zone.dart:1124:13)
#38     _CustomZone.run (dart:async/zone.dart:1021:19)
#39     _CustomZone.runGuarded (dart:async/zone.dart:923:7)
#40     _CustomZone.bindCallbackGuarded.<anonymous closure> (dart:async/zone.dart:963:23)
#41     _microtaskLoop (dart:async/schedule_microtask.dart:41:21)
#42     _startMicrotaskLoop (dart:async/schedule_microtask.dart:50:5)
#43     _Timer._runTimers (dart:isolate/runtime/libtimer_impl.dart:391:30)
#44     _Timer._handleMessage (dart:isolate/runtime/libtimer_impl.dart:416:5)
#45     _RawReceivePortImpl._handleMessage (dart:isolate/runtime/libisolate_patch.dart:171:12)

Coverage decreased (-0.2%) to 81.533% when pulling 83d3c24cedc371321c96368e8b27c51bf73b31c4 on remove-counter into 6d94895006c192080053678a032d063bc8873e85 on master.

authors: coveralls jcollins-g

repo: dart-lang/dart-services


text:
The test_package path dependency results in error message on pub upgrade

Error on line 1, column 1 of https://pub.dartlang.org/api/packages/initialize: Invalid description: "test_package" is a relative path, but this isn't a local pubspec.

I only use hosed dependencies in my pubspec.yaml.

I get the same also when upgrading to:
polymer: ^1.0.0-rc.10
reflectable: ^0.5.0

Ya I don't know what the issue is with this, the error message isn't clear and the path dependency has been there for a long time. Need to file an issue with pub to get more info.
That being said this hasn't actually caused any issues for me?

I filed https://github.com/dart-lang/pub/issues/1369

Can this be closed?

yep, thanks

authors: jakemac53 terrywarwar zoechi

repo: dart-lang/initialize


text:
Requesting more prescriptive README for example w.r.t. script

The README in the example says "create the scripts to run code generation", but I don't know how to take action on that. Can we be more prescriptive? I know you link to the tool/ directory but can you be more specific?
What's the MVP step-by-step I need to do in order to use this? Cut-n-paste style :)
For example, do I need all three of the scripts you mention? Do I pick one to start? Do I just copy and paste these three scripts into my app? Where would I paste these script into?

it's basically where you import the build_runner package along with the json_serializable's action.dart, then in main you just put build([whateverAction(['lib/**.dart'])]) or whatever.
or, y'know, just use dson. which actually tells you what to do.

We point everyone to the example package: https://github.com/dart-lang/json_serializable/tree/master/example
We try to be very clear/prescriptive.
The new build_runner logic also makes this much easier.

authors: dwaindibbly kevmoo sethladd


text:
Flutter tag links to incorrect page

The individual package pages show a tag for the package type, e.g. 'FLUTTER' here:
https://pub.dartlang.org/experimental/packages/url_launcher#-readme-tab-
For flutter packages this currently links to https://pub.dartlang.org/flutter/packages. I think it should link to https://pub.dartlang.org/experimental/flutter.

cc @isoos

This is fixed in code, but not deployed yet.

Great, thanks

Deployed!

authors: isoos kevmoo mit-mit

repo: dart-lang/pub-dartlang-dart


text:
Increase package size limit

Now that we have people starting to hack on mobile development in Dart, larger assets are becoming more common and more important. The Sky folks have a package with about 30MB of pngs.
10MB is starting to feel kind of antiquated. Maybe bump it up to 50 or 100?
We'll have to see how this affects the server storage for packages first, of course. Martin, any thoughts?

This issue has been moved to dart-lang/pub#1270.

authors: DartBot munificent

repo: dart-lang/sdk


text:
Strong mode hotlist

This is a hotlist issue for the main stuff the people hacking on strong mode are focusing on right now:
Generic methods (@jmesserly)

[ ] SDK needs to have generic method comments where appropriate
[ ] comment syntax allows reifying a tighter type than VM supports, e.g. new List<num/*=T*/>() (#25637)
[ ] implement missing warnings/errors (#25200)
[ ] implement missing AST nodes, e.g. f<int> and o.m<int> (#25175)
[ ] declaration and use of generic function types, e.g. <T>(T) ->T (#25179)
[ ] should we allow invalid use of comment form? (#24985)

Consistent use of upwards/downwards inference

[ ] Downwards inference on generic methods (#25477)
[ ] Upwards inference on constructors (#25220)
[ ] Upwards inference for list/map literals (#25220)
[ ] Consider upwards inference for block bodied closure returns & implicit unions (#25322)

Function parameter type and dynamic (top vs bottom)
Needs design.

[x] Top-level function type weirdness (#24507)
[ ] Instance method overrides -- warning? (#25578)

Closure / function parameter and return type inference (@leafpetersen)

[x] Non-arrow function return (open, easily closable, uninferred closure) (https://github.com/dart-lang/dev_compiler/issues/290)

Future / async typing

[x] Future<Future<T>> flattening
[ ] flattening diverges (https://github.com/dart-lang/sdk/issues/25611)

Generics and variance
@vsm: try making DefaultEquality generic, revisit package:collection - do we need this?

[ ] B <: A<bottom> (https://github.com/dart-lang/sdk/issues/25604)
[ ] Add flag for invariant generics?

Ground types and is/as
Needs design.

[ ] is on type parameter? (#24565)

Least upper bound

[ ] is unsound & EfficientLength problem (https://github.com/dart-lang/sdk/issues/25802)

Field overrides

[ ] do we want a @virtual?
[x] Disallow for now? (https://github.com/dart-lang/dev_compiler/issues/52)

Error / warning quality
Should work on phrasing.

[ ] Overhaul categorization
[x] Rephrase DDC wording
[ ] Make errors actionable
[ ] Separate out DDC errors unrelated to soundness

No dynamic

[ ] No reified dynamic (https://github.com/dart-lang/sdk/issues/25573)
[ ] Inferences fails / no static dynamic
[ ] No dynamic helpers generated

Improve type promotion
Various improvements that would help user productivity:

[ ] meta bug: https://github.com/dart-lang/sdk/issues/25565

Other bugs to fix

[x] Type of (x) => null (https://github.com/dart-lang/dev_compiler/issues/47)
[ ] F-bounded quantification (https://github.com/dart-lang/sdk/issues/25530)
[x] Private methods incorrectly treated as overrides even in different libs (https://github.com/dart-lang/dev_compiler/issues/366)
[ ] assignment in setter (https://github.com/dart-lang/sdk/issues/25642)
[ ] infer type from parameter default value (https://github.com/dart-lang/sdk/issues/25794)
[ ] not checking inferred field initializer (https://github.com/dart-lang/sdk/issues/25546)

I went through and updated this with current bug numbers (at this point, all Strong Mode bugs should be here, not in DDC repo). I tried to add open issues we were missing. Not sure if I captured all of them. I think we are going to want to trim this list down.

I'm going to close this now that we've gone through and triaged bugs. But we can reopen if needed

authors: jmesserly munificent


text:
dart2js: Compiler phases no longer show up in observatory

It used to be that compiler phases were available as user tage in observatory cpu profiles when compiling with -v.
Sometime in the past couple of weeks this stopped working.

@peter-ahe-google Do you know if your change did this?

Yes, I removed the use of UserTag (see dart:developer). It's relatively easy to add back, but be careful that measuring adds significant overhead. Also, there's a hard-coded limit on the number of UserTags supported in an application (64). This makes it problematic to use UserTags in production code.

@peter-ahe-google I'd like to see the user tags come back (under a flag if they really are that expensive.)
They are critical to understanding cpu profiles of multi-phase programs.
@iposva-google Talking of overhead, the innocuous looking map literal {measurer: this}, accounts for 2% of dart2js total run time with -v.
https://github.com/dart-lang/sdk/blob/master/pkg/compiler/lib/src/common/tasks.dart#L149
This is a known issue - https://github.com/dart-lang/sdk/issues/26046 @fsc8000, how is it going?
@johnmccutchan The cpu profile annotated source text is misleading. In my profile 0.24% (inclusive) is attributed to the line with the map literal, and 2.60% (inclusive) on the next line.  Looking at the disassembly, 1.96% (inclusive) is attributed to the return address of the call to _InternalLinkedHashMap._InternalLinkedHashMap, a pop rcx instruction that is within the inlining region Map.Map._fromLiteral@915557746 -> LinkedHashMap.LinkedHashMap.
I would not expect any part of the map literal to be attributed to a source position after the closing brace.

I'm sorry I wasn't clear, it's not UserTag that adds overhead, it's the --verbose option that adds overhead and this skews your results. We can't use UserTag without --verbose, but the numbers we get are skewed so UserTag doesn't seem like a good approach.
I'm hoping to get access to the profiler ticks somehow, and use this to create a tool similar to PerfAnal, this should give us exactly the profiling information we need.

Could we put all the --verbose overhead under a new UserTag?
Then we could ignore the overhead when looking at profiles in observatory.
For example, the 2% for allocating {measurer: this} would then be separable from the cost of other map allocations.

authors: peter-ahe-google rakudrama


text:
Flaky VM crash: language/vm/lazy_deopt_with_exception_and_stacktrace_test

An unrelated commit of me caused a crash of the language/vm/lazy_deopt_with_exception_and_stacktrace_test test:
FAILED: none-vm-checked debug_simmips language/vm/lazy_deopt_with_exception_and_stacktrace_test
Expected: Pass
Actual: Crash
CommandOutput[vm]:

stdout:
123

Command[vm]: DART_CONFIGURATION=DebugSIMMIPS out/DebugSIMMIPS/dart --enable_asserts --enable_type_checks --optimization-counter-threshold=10 --no-background-compilation --enable-inlining-annotations --ignore-unrecognized-flags --packages=/mnt/data/b/build/slave/vm-linux-debug-simmips-be/build/sdk/.packages /mnt/data/b/build/slave/vm-linux-debug-simmips-be/build/sdk/tests/language/vm/lazy_deopt_with_exception_and_stacktrace_test.dart
Took 0:00:01.659855

Short reproduction command (experimental):
    python tools/test.py -asimmips --write-debug-log --write-test-outcome-log --copy-coredumps --exclude-suite pkg --checked -t480 --packages /mnt/data/b/build/slave/vm-linux-debug-simmips-be/build/sdk/.packages language/vm/lazy_deopt_with_exception_and_stacktrace_test

/cc @fsc8000 Any ideas what might cause this?

This test just added in this commit: https://github.com/dart-lang/sdk/commit/3413e052b30ee12ad41cb73a7f1abb50bba2a3ae @rmacnak-google


text:
Remove generalized tear-offs from the VM

This is the VM issue for https://github.com/dart-lang/sdk/issues/27518.

This is a breaking change, and is therefore not suitable for the 1.50 milestone.

@dgrove specifically asked it to be added to 1.50. As I note in the meta bug, there are zero usages of it within Google. I don't know if there are any outside, but if there are, they are bound to be few.
There's a sort of "tree falls in the woods, does it make sound" question about whether removing a feature which has no users is a breaking change or not. If no programs are broken, I wouldn't classify it as a breaking change.

Completely agree with @zanderso . And apart from being a breaking change, what is the exact benefit of removing this language feature?

Reasons for its deletion:

it's not implemented across the toolchain
has no users.

There is a lot of work to get this implemented everywhere, and we do not
believe that it's worth this effort.
On Thu, Oct 6, 2016 at 1:00 PM, Florian Schneider notifications@github.com
wrote:

Completely agree with @zanderso https://github.com/zanderso . And apart
from being a breaking change, what is the exact benefit of removing this
language feature?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dart-lang/sdk/issues/27520#issuecomment-252072118,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACAsWzPkR-4DWNjnBWE2lpVOilSSr9Dsks5qxVNigaJpZM4KQQPj
.

Every feature adds complexity to the language, adds cognitive load to users, increases the effort it takes for a new user to learn Dart, must be implemented and optimized and maintained, needs to be documented, and occupies portions of the grammar that could be used for other things. It interacts with other features and gets in the way of other ways of accomplishing the same goal.
Every language feature begins its life quite deep in debt. It pays off that debt by making the lives of as many users possible as much better as possible.
Generalized tear-offs didn't meet that requirement. It's a fairly complex feature, and it sits right in the middle of the expression grammar and even takes up a unique token. It is redundant—and more verbose!—with the existing tear-off syntax for cases where both work. It provides no solution for many similar problems, such as wanting to create a "tear-off" that invokes a method with a given receiver.
The fact that it has no users, and no users have clamored for it to be fully implemented is a good sign that it does not meet a real user need. The fact that no one on the language team is excited about it is another good sign.
Since it is unused, we have an easy opportunity to just delete it now and unburden ourselves from cost of the finishing implementing it, maintaining, etc. This also frees us up to design a better solution in the future, which we have pretty good ideas for.

it's not implemented across the toolchain
has no users.

The second point could be be a consequence of the first line. There could be users of the Dart VM using it.

You are using a overly-narrow definition of "within Google", one that your colleagues working in the open source world may object to. For example, tear-offs are used in the Mojo bindings and embedder, e.g. here: https://github.com/domokit/mojo/blob/master/mojo/dart/embedder/io/mojo_patch.dart#L193. The API that suggests using tear-offs is public:
https://fuchsia.googlesource.com/mojo/+/master/dart/packages/mojo/src/proxy.dart#370
I don't feel it is my responsibility to audit all users of this API be they internal or external.
I'm not disagreeing with the decision to remove the feature, but I do object to the timing. The 1.50 plan states that it will have only "high-priority language enhancements (not breaking changes)". This change seems low-priority and breaking, which violates both of these requirements.

I think it would be great to add a warning for when it's used as soon as possible.
Is there an easy way to check if it's used?
@munificent how did you check the Google code base?
It might be that I use it somewhere. I know I had a need once but worked around it passing closures but I might have changed that later to use tear-offs.
I wouldn't mind changing the code to get rid of tear-offs.

Is there an easy way to check if it's used?

If you grep for something like \w+#\w+, it should turn them up, I believe.

I know I had a need once but worked around it passing closures but I might have changed that later to use tear-offs.

Yeah, you never need to use generalized tear-offs. You can always just make a closure.

Wish there was an easy way to check all of github for use of "#" but https://help.github.com/articles/searching-code/ says "You can't use the following wildcard characters as part of your search query: . , : ; / \ ` ' " = * ! ? # $ & + ^ | ~ < > ( ) { } [ ]. The search will simply ignore these symbols."

I just pulled down the latest version of every package on pub. With some grep hackery, I determined there are zero usages of generalized tear-offs on pub.dartlang.org.

After reviewing all of the data, we will make this change for 1.50 .

Moving to 1.22 per release discussion, deprecation warning will happen in 1.21 (see #27742).

After discussion with Dan, moving this to 1.22.
@a-siva let's get this removed!

@a-siva is someone working on this?

https://github.com/dart-lang/sdk/commit/c69cbb586f29ed67b949cf8661d4d22b922a2339

authors: a-siva anders-sandholm dgrove fsc8000 mit-mit munificent zanderso zoechi


text:
ddc should issue an error if an integer literal cannot be exactly represented as a double

See https://github.com/dart-lang/sdk/issues/33282 for context.

This is now covered / fixed in #33351 - @jmesserly can you verify?

I'm going to reopen this one to track DDC status.

blocked until I can figure out how to fix Analyzer:
  int z2 = -0x8000000000000000;

is rejected by Analyzer with:
[error] The integer literal -0x8000000000000000 can't be represented in 64 bits. (../../../scratch/test.dart, line 5, col 13)

Please fix all errors before compiling (warnings are okay).

But not by CFE (DDK/VM work).

WIP CL should fix Analyzer and DDC, need to verify tests behave as expected, and updated many status entries: https://dart-review.googlesource.com/c/sdk/+/63023
This brings Analyzer/DDC into alignment with CFE, using similar logic whenever possible to keep them in sync.

@jmesserly is this likely to land soon?

yes I am putting finishing touches on status files now :)  I was out of office last week

the CL is out for review

authors: dgrove jmesserly leafpetersen vsmenon


text:
dart2js: type inference of equality(==) operator

According to dart2js_info visualizer, comparing primitive vars (statically known to be strings or numbers) may lead to dependence on all implementations of overridden operator ==().
For example, dart.core._Uri._removeDotSegments somehow uses dart.core.DateTime.==.
Dart SDK 2.0.0-dev.59.0

Type inference is not yet benefiting much from Dart 2 semantics.
In Dart 1 we can't trust String.split returns a List containing only Strings - the return type List<String> could be satisfied by List<dynamic>, containing anything. So the comparison segment == ".." appears to possibly call any implementation of ==. Interestingly, the second comparison is written "." == segment which can only call String.==. With Dart 2, there will be less need to write comparisons 'backwards' to get tighter dependencies.
Once Dart 2 is released and Dart 1 is retired, we can work on taking advantage of the better type guarantees provided by Dart 2 semantics.

authors: lexaknyazev rakudrama


text:
Support --inline-source-map in ddc kernel backend

Sorry for such a late addition to the 2.3 milestone but it looks like to get source maps working fully (in package:test primarily), we will need the --inline-source-map option enabled in the kernel backend for ddc.
cc @jmesserly @vsmenon
We could decide to punt on this and accept js stack traces for tests, but it would be good to get in. There may also be other scenarios outside of testing where the inline source maps are required though.

fwiw the command line option doesn't seem to throw (like it used to I believe), but it doesn't seem to be outputting the source map either

Actually nevermind these are being included - we just need to map the uris to something that make sense (we already do this for normal source maps but need to do the same mapping in the dart stack trace mapper code, which we have control of outside of the sdk).

Nice! Yeah, I implemented that flag in the same CL as trackLibraries.


text:
support vm events available to systrace

For example,  API_TIMELINE_DURATION in dart_api_impl.cc
DART_EXPORT Dart_Handle Dart_Invoke(Dart_Handle target,
                                    Dart_Handle name,
                                    int number_of_arguments,
                                    Dart_Handle* arguments) {
  DARTSCOPE(Thread::Current());
  API_TIMELINE_DURATION(T);
  ...
}

(https://github.com/flutter/engine/pull/14323/files) this PR  make API stream events available to systrace. but only can record TimeBeginEndScope, not including TimelineDurationScope.
with this PR, Dart_Invoke event can support in systrace

so I choose to replace all uses of TimelineDurationScope with TimeBeginEndScope, and remove  unnecessary code to prevent future use of TimelineDurationScope method.

https://dart-review.googlesource.com/c/sdk/+/128200

relation to https://github.com/flutter/flutter/issues/47771


text:
Apply dart fixes to require_trailing_commas

Dart SDK Version Dart SDK version: 2.14.0 (stable) (Mon Sep 6 13:47:59 2021 +0200) on "macos_x64"
Reproduction
Add require_trailing_commas to analysis_options.yaml lints
TextStyle get hintTextFiesld => TextStyle(
      color: Colors.grey.shade500, debugLabel: 'lihjiolhoiukllinilobibib');
dart fix --dry-run

Result
Computing fixes in mobile (dry run)... 82.0s
Nothing to fix!

Expected
Offers to infer  the comma.
Display the fix on Fix, Fix.all on Vs code.

https://dart-review.googlesource.com/c/sdk/+/215403

I think this is happening again after I upgraded.
[✓] Flutter (Channel stable, 3.13.4, on macOS 13.4.1
    22F82 darwin-arm64, locale en-ID)
    • Flutter version 3.13.4 on channel stable at
      /Users/upbanx/dev/flutter
    • Upstream repository
      https://github.com/flutter/flutter.git
    • Framework revision 367f9ea16b (5 days ago),
      2023-09-12 23:27:53 -0500
    • Engine revision 9064459a8b
    • Dart version 3.1.2
    • DevTools version 2.25.0

[✓] VS Code (version 1.81.1)
    • VS Code at /Applications/Visual Studio
      Code.app/Contents
    • Flutter extension version 3.72.0

Somehow require_trailing_commas is detected again now. Could this be related to the package instead, such as  flutter_lints? Because my Flutter & Dart versions are still the same. Can't figure out what fixes the issue.
[✓] Flutter (Channel stable, 3.13.4, on macOS 13.4.1 22F82
    darwin-arm64, locale en-ID)
    • Flutter version 3.13.4 on channel stable at
      /Users/upbanx/dev/flutter
    • Upstream repository
      https://github.com/flutter/flutter.git
    • Framework revision 367f9ea16b (2 weeks ago), 2023-09-12
      23:27:53 -0500
    • Engine revision 9064459a8b
    • Dart version 3.1.2
    • DevTools version 2.25.0

[✓] VS Code (version 1.82.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.72.0

I'm facing the same issue

The require_trailing_commas lint does not appear to be included in flutter_lints. If you mean that the lint appears to be enabled now when it was disabled before, the only way I can think of that that could happen is if it's been added, directly or indirectly, to the analysis options file being used to analyze the code where it's being reported.

We will not be adding any work for the require_trailing_commas lint rule; instead leaning on the Dart formatter to provide commas, with the new Tall Style.

authors: asashour bounty1342 bwilkerson ggirotto gmaggio srawlins


text:
[Records] Analyzer implementation

The following is a list of the individual features that need to be considered. The features are listed roughly in dependency order.

[x] #50087
[x] Element model (scheglov, NA)
[x] Type system updates (scheglov)
[x] Summary support (scheglov)
[x] Resolution (scheglov)
[x] Constant evaluation (brianwilkerson)
[x] Index and search (scheglov)

@srawlins what items are needed to call this feature work done?

I updated it by copying pkg/analyzer/doc/process/new_language_feature.md, which might or might not be complete.

@bwilkerson have you found that these items cover most necessary work?  Is there anything in that list I should update?

@scheglov can confirm, but I believe that the list above is reasonably complete and that the analyzer, aside from a couple of known bugs, is complete.

Yes, I think the analyzer implementation is probably complete.
However there are still discussions about parsing ambiguous code, and corresponding parser changes, and there is a bucket of co19 tests failing because of the need to disambiguate. So, there might be something new to be uncovered with tests.

@bwilkerson and @scheglov thank you for providing clarity around this issue!

@scheglov how about now? Did the change regarding whitespace in an annotation land? And the change to make the first value in a record, $1, did that land?

I believe we can mark "records" done.

authors: bwilkerson itsjustkevin scheglov srawlins


text:
Debugger should step across the async gap when stepping into an async function call

When stepping into an async function call, VS Code's Dart debugger steps into _SuspendState._initAsync inside async_patch.dart -- and then proceeds to go through a series of very complex steps to handle the async call. It's impossible to step through an async call without putting the cursor on the async function name, hitting f12, then setting a breakpoint inside the function, then clicking the Continue button in the debugger. Then you have to remove the breakpoint again if there are other calls to the function that you don't want to debug.
All this should happen automatically if you step into an async function call -- the debugger should stop at the first line of the function that is being called, not at the first line of the async machinery.

Dart SDK Version (dart --version): 3.0.2
Whether you are using Windows, MacOSX, or Linux (if applicable): Android
Whether you are using Chrome, Safari, Firefox, Edge (if applicable)

//cc @derekxu16 @bkonyi

This is particularly a problem when the async method is an abstract method of a superclass, and the actual method call dispatched at runtime is based on the type of the receiver. To step into these async methods, you can't just hit F12 and set a breakpoint at the start of the async function. You have to find all the implementing subclasses, and manually set breakpoints on entry to each implementation of the abstract function.

One halfway workaround would be: in the debugger, if you hit F12 on a method, and the concrete receiver is a subclass of the class defining the method, then ask the user if they want to step into the concrete subclass, or the defining superclass.
But in general, it would be better if "Step in" worked fine, always, regardless of the async gap.

I found one way to do it -- "Step Into" (which enters async code) followed by "Step Over". This seems to take the debugger to the async method.

@derekxu16 You're right, that was set, and I have no idea how it got set. Thank you for your help!

authors: a-siva lukehutch


text:
Invalid code crashes the analyzer

Dart info
#### General info

- Dart 3.1.4 (stable) (Tue Oct 17 14:55:53 2023 +0000) on "macos_arm64"
- on macos / Version 14.1 (Build 23B74)
- locale is en-US

Code
If I remove the const in the constructor the analyzer no longer crashes
class NumberPool {
  final int capacity;
  final List<num> _values;

  const NumberPool(this.capacity, {List<num>? values})
      : _values = [...?values],
        assert(() {
          if (values != null && values.length > capacity) return false;
          return true;
        }(), 'The list provided to populate NumberPool has more elements(${values?.length}) than max($capacity}) allows');

  num get last => _values.last;

  void add(num value) {
    if (_values.length == capacity) _values.removeAt(0);
    _values.add(value);
  }

  bool get isEmpty => _values.isEmpty;

  bool get isFull => _values.length == capacity;

  List<num> get values => UnmodifiableListView(_values);

  num average() {
    double avg = 0;
    int t = 1;
    for (var x in _values) {
      avg += (x - avg) / t;
      ++t;
    }
    return avg;
  }
}

Logs
/Users/myuser/Developer/diagram_playground/lib/number_pool.dart
[Info  - 6:33:21 PM] Connection to server got closed. Server will restart.
true
[Error - 6:33:23 PM] The Dart Analysis Server server crashed 5 times in the last 3 minutes. The server will not be restarted. See the output for more information.

App: Visual Studio Code
App Host: desktop
Version: mac 1.83.1

Workspace type: Flutter (LSP)

Dart (3.1.4): /Users/myuser/fvm/versions/stable/bin/cache/dart-sdk
Flutter (3.13.8): /Users/myuser/fvm/versions/stable (No device)
[6:33:16 PM] [FlutterDaemon] [Info] <== [{"event":"daemon.connected","params":{"version":"0.6.1","pid":21971}}]
[6:33:16 PM] [FlutterDaemon] [Info] ==> [{"id":"1","method":"device.enable"}]
[6:33:16 PM] [FlutterDaemon] [Info] ==> [{"id":"2","method":"emulator.getEmulators"}]
[6:33:16 PM] [Analyzer] [Info] <== Content-Length: 641
Content-Type: application/vscode-jsonrpc; charset=utf-8
[6:33:16 PM] [Analyzer] [Info] <== {"id":0,"jsonrpc":"2.0","result":{"capabilities":{"executeCommandProvider":{"commands":["edit.sortMembers","edit.organizeImports","edit.fixAll","edit.sendWorkspaceEdit","refactor.perform","refactor.validate","dart.logAction","dart.refactor.convert_all_formal_parameters_to_named","dart.refactor.convert_selected_formal_parameters_to_named","dart.refactor.move_selected_formal_parameters_left","dart.refactor.move_top_level_to_file"],"workDoneProgress":true},"workspace":{"workspaceFolders":{"chan…
[6:33:16 PM] [Analyzer] [Info] ==> Content-Length: 86
[6:33:16 PM] [Analyzer] [Info] ==> {"jsonrpc":"2.0","method":"initialized","params":{},"clientRequestTime":1699313596501}
[6:33:16 PM] [Analyzer] [Info] <== Content-Length: 180
Content-Type: application/vscode-jsonrpc; charset=utf-8
[6:33:16 PM] [Analyzer] [Info] <== {"id":1,"jsonrpc":"2.0","method":"workspace/configuration","params":{"items":[{"scopeUri":"file:///Users/myuser/Developer/diagram_playground","section":"dart"},{"section":"dart"}]}}
[6:33:16 PM] [Analyzer] [Info] ==> Content-Length: 7241
[6:33:16 PM] [Analyzer] [Info] ==> {"jsonrpc":"2.0","id":1,"result":[{"additionalAnalyzerFileExtensions":[],"analysisExcludedFolders":[],"analyzeAngularTemplates":true,"analyzerAdditionalArgs":[],"analyzerVmAdditionalArgs":[],"analyzerDiagnosticsPort":null,"analyzerPath":null,"analyzerSshHost":null,"analyzerVmServicePort":null,"includeDependenciesInWorkspaceSymbols":true,"notifyAnalyzerErrors":true,"onlyAnalyzeProjectsWithOpenFiles":false,"showTodos":true,"useLegacyAnalyzerProtocol":false,"updateDevTools":true,"devToolsBrowse…
[6:33:16 PM] [Analyzer] [Info] <== Content-Length: 5158
Content-Type: application/vscode-jsonrpc; charset=utf-8

{"id":2,"jsonrpc":"2.0","method":"client/registerCapability","params":{"registrations":[{"id":"0","method":"textDocument/didOpen","registerOptions":{"documentSelector":[{"language":"dart","scheme":"file"},{"language":"yaml","pattern":"**/pubspec.yaml","scheme":"file"},{"language":"yaml","pattern":"**/analysis_options.yaml","scheme":"file"},{"language":"yaml","pattern":"**/lib/{fix_data.yaml,fix_data/**.yaml}","s…
[6:33:16 PM] [Analyzer] [Info] ==> Content-Length: 98
[6:33:16 PM] [Analyzer] [Info] ==> {"jsonrpc":"2.0","method":"$/setTrace","params":{"value":"off"},"clientRequestTime":1699313596615}
[6:33:16 PM] [Analyzer] [Info] ==> Content-Length: 98
[6:33:16 PM] [Analyzer] [Info] ==> {"jsonrpc":"2.0","method":"$/setTrace","params":{"value":"off"},"clientRequestTime":1699313596637}
[6:33:16 PM] [Analyzer] [Info] ==> Content-Length: 72
[6:33:16 PM] [Analyzer] [Info] ==> {"jsonrpc":"2.0","id":2,"result":null,"clientRequestTime":1699313596643}
[6:33:16 PM] [Analyzer] [Info] ==> Content-Length: 1098
[6:33:16 PM] [Analyzer] [Info] ==> {"jsonrpc":"2.0","method":"textDocument/didOpen","params":{"textDocument":{"uri":"file:///Users/myuser/Developer/diagram_playground/lib/number_pool.dart","languageId":"dart","version":1,"text":"class NumberPool {\n  final int capacity;\n  final List<num> _values;\n\n  const NumberPool(this.capacity, {List<num>? values})\n      : _values = [...?values],\n        assert(() {\n          if (values != null && values.length > capacity) return false;\n          return true;\n        }(), 'The list …
[6:33:16 PM] [FlutterDaemon] [Info] <== [{"event":"daemon.logMessage","params":{"level":"status","message":"Starting device daemon..."}}]
[6:33:16 PM] [Analyzer] [Info] ==> Content-Length: 98
[6:33:16 PM] [Analyzer] [Info] ==> {"jsonrpc":"2.0","method":"$/setTrace","params":{"value":"off"},"clientRequestTime":1699313596743}
[6:33:16 PM] [Analyzer] [Info] ==> Content-Length: 122
[6:33:16 PM] [Analyzer] [Info] ==> {"jsonrpc":"2.0","method":"workspace/didChangeConfiguration","params":{"settings":null},"clientRequestTime":1699313596743}
[6:33:16 PM] [Analyzer] [Info] <== Content-Length: 180
Content-Type: application/vscode-jsonrpc; charset=utf-8

{"id":3,"jsonrpc":"2.0","method":"workspace/configuration","params":{"items":[{"scopeUri":"file:///Users/myuser/Developer/diagram_playground","section":"dart"},{"section":"dart"}]}}
[6:33:17 PM] [Analyzer] [Info] <== Content-Length: 189
Content-Type: application/vscode-jsonrpc; charset=utf-8
[6:33:17 PM] [FlutterDaemon] [Info] <== [{"id":"1"}]
[6:33:17 PM] [FlutterDaemon] [Info] <== [{"id":"2","result":[{"id":"apple_ios_simulator","name":"iOS Simulator","category":"mobile","platformType":"ios"},{"id":"Pixel_3a_API_34_extension_level_7_arm64-v8a","name":"Pixel_3a_API_34_extension_level_7_arm64-v8a","category":"mobile","platformType":"android"}]}]
[6:33:17 PM] [Analyzer] [Info] ==> Content-Length: 202
[6:33:17 PM] [Analyzer] [Info] ==> {"jsonrpc":"2.0","id":1,"method":"textDocument/documentColor","params":{"textDocument":{"uri":"file:///Users/myuser/Developer/diagram_playground/lib/number_pool.dart"}},"clientRequestTime":1699313597034}
[6:33:17 PM] [Analyzer] [Info] ==> Content-Length: 322
[6:33:17 PM] [Analyzer] [Info] ==> {"jsonrpc":"2.0","id":2,"method":"textDocument/codeAction","params":{"textDocument":{"uri":"file:///Users/myuser/Developer/diagram_playground/lib/number_pool.dart"},"range":{"start":{"line":24,"character":17},"end":{"line":24,"character":17}},"context":{"diagnostics":[],"triggerKind":2}},"clientRequestTime":1699313597034}
[6:33:17 PM] [Analyzer] [Info] <== {"jsonrpc":"2.0","method":"textDocument/publishDiagnostics","params":{"diagnostics":[],"uri":"file:///Users/myuser/Developer/diagram_playground/android/app/src/profile/AndroidManifest.xml"}}Content-Length: 186
Content-Type: application/vscode-jsonrpc; charset=utf-8

{"jsonrpc":"2.0","method":"textDocument/publishDiagnostics","params":{"diagnostics":[],"uri":"file:///Users/myuser/Developer/diagram_playground/android/app/src/main/AndroidManifest.xml"}}
[6:33:17 PM] [Analyzer] [Info] ==> Content-Length: 7241
[6:33:17 PM] [Analyzer] [Info] ==> {"jsonrpc":"2.0","id":3,"result":[{"additionalAnalyzerFileExtensions":[],"analysisExcludedFolders":[],"analyzeAngularTemplates":true,"analyzerAdditionalArgs":[],"analyzerVmAdditionalArgs":[],"analyzerDiagnosticsPort":null,"analyzerPath":null,"analyzerSshHost":null,"analyzerVmServicePort":null,"includeDependenciesInWorkspaceSymbols":true,"notifyAnalyzerErrors":true,"onlyAnalyzeProjectsWithOpenFiles":false,"showTodos":true,"useLegacyAnalyzerProtocol":false,"updateDevTools":true,"devToolsBrowse…
[6:33:17 PM] [Analyzer] [Info] ==> Content-Length: 273
[6:33:17 PM] [Analyzer] [Info] ==> {"jsonrpc":"2.0","id":3,"method":"textDocument/inlayHint","params":{"textDocument":{"uri":"file:///Users/myuser/Developer/diagram_playground/lib/number_pool.dart"},"range":{"start":{"line":0,"character":0},"end":{"line":33,"character":1}}},"clientRequestTime":1699313597075}
[6:33:17 PM] [Analyzer] [Info] ==> Content-Length: 284
[6:33:17 PM] [Analyzer] [Info] ==> {"jsonrpc":"2.0","id":4,"method":"textDocument/semanticTokens/range","params":{"textDocument":{"uri":"file:///Users/myuser/Developer/diagram_playground/lib/number_pool.dart"},"range":{"start":{"line":0,"character":0},"end":{"line":33,"character":1}}},"clientRequestTime":1699313597075}
[6:33:17 PM] [Analyzer] [Info] <== Content-Length: 187
Content-Type: application/vscode-jsonrpc; charset=utf-8

{"jsonrpc":"2.0","method":"textDocument/publishDiagnostics","params":{"diagnostics":[],"uri":"file:///Users/myuser/Developer/diagram_playground/android/app/src/debug/AndroidManifest.xml"}}Content-Length: 167
Content-Type: application/vscode-jsonrpc; charset=utf-8

{"jsonrpc":"2.0","method":"textDocument/publishDiagnostics","params":{"diagnostics":[],"uri":"file:///Users/myuser/Developer/diagram_playground/analys…
[6:33:17 PM] [Analyzer] [Info] ==> Content-Length: 201
[6:33:17 PM] [Analyzer] [Info] ==> {"jsonrpc":"2.0","id":5,"method":"textDocument/foldingRange","params":{"textDocument":{"uri":"file:///Users/myuser/Developer/diagram_playground/lib/number_pool.dart"}},"clientRequestTime":1699313597142}
[6:33:17 PM] [Analyzer] [Info] <== {"id":1,"jsonrpc":"2.0","error":{"code":-32013,"data":"/Users/myuser/Developer/diagram_playground/lib/number_pool.dart","message":"Analysis failed for file"}}Content-Length: 157
Content-Type: application/vscode-jsonrpc; charset=utf-8

{"id":3,"jsonrpc":"2.0","error":{"code":-32013,"data":"/Users/myuser/Developer/diagram_playground/lib/number_pool.dart","message":"Analysis failed for file"}}Content-Length: 180
Content-Type: application/vscode-jsonrpc; charset=utf-8

{"id":5,"jsonrpc":"2.0…
[6:33:17 PM] [Analyzer] [Error] Unhandled exception:
Exception: Missing implementation of visitFunctionExpression
#0      ThrowingAstVisitor._throw (package:analyzer/dart/ast/visitor.dart:2859:5)
#1      ThrowingAstVisitor.visitFunctionExpression (package:analyzer/dart/ast/visitor.dart:2528:58)
#2      FunctionExpressionImpl.accept (package:analyzer/src/dart/ast/ast.dart:8680:50)
#3      AstBinaryWriter._writeNode (package:analyzer/src/summary2/ast_binary_writer.dart:944:10)
#4      AstBinaryWriter.visitFunctionExpressionInvoc…
[6:33:17 PM] [Analyzer] [Info] ==> Content-Length: 72
[6:33:17 PM] [Analyzer] [Info] ==> {"jsonrpc":"2.0","id":4,"result":null,"clientRequestTime":1699313597204}
[6:33:17 PM] [Analyzer] [Info] ==> Content-Length: 208
[6:33:17 PM] [Analyzer] [Info] ==> {"jsonrpc":"2.0","id":6,"method":"textDocument/semanticTokens/full","params":{"textDocument":{"uri":"file:///Users/myuser/Developer/diagram_playground/lib/number_pool.dart"}},"clientRequestTime":1699313597486}
[6:33:17 PM] [Analyzer] [Info] Spawning /Users/myuser/fvm/versions/stable/bin/cache/dart-sdk/bin/dart with args ["language-server","--protocol=lsp","--client-id=VS-Code","--client-version=3.76.1"]
[6:33:17 PM] [Analyzer] [Info]     PID: 22052
[6:33:17 PM] [Analyzer] [Info] ==> Content-Length: 6452
[6:33:17 PM] [Analyzer] [Info] ==> {"jsonrpc":"2.0","id":0,"method":"initialize","params":{"processId":21970,"clientInfo":{"name":"Visual Studio Code","version":"1.83.1"},"locale":"en","rootPath":"/Users/myuser/Developer/diagram_playground","rootUri":"file:///Users/myuser/Developer/diagram_playground","capabilities":{"workspace":{"applyEdit":true,"workspaceEdit":{"documentChanges":true,"resourceOperations":["create","rename","delete"],"failureHandling":"textOnlyTransactional","normalizesLineEndings":true,"changeAnnotationSuppor…

when building it does errors, but the analyzer should not crash.
Error: Constant expression expected.
      : _values = [...?values],
                  ^
lib/main.dart:7:16:
Error: Not a constant expression.
        assert(() {
               ^^

CC @kallentu

As far as I can see, this is the same issue as https://github.com/dart-lang/sdk/issues/53075.
This has been fixed as of https://github.com/dart-lang/sdk/commit/ba1988e0d3acb340402a9d549db484eae933b65e.

authors: fabiancrx kallentu srawlins


text:
Invalid constant causes analysis server to stop updating problems view

Ran into this problem while coding the other day, and after some experimentation I was able to come up with a narrow repro.
Using VSCode, if I open an empty file and paste in these contents:
class Foo extends Bar {
  static const instance = Foo._();

  const Foo._() : super();
}

class Bar extends Baz {
  const Bar({required super.i});
}

class Baz {
  final int i;

  const Baz({required this.i});
}

No errors show in the problems view.
But if I analyze the same file using dart analyze on the command line, I see:
Analyzing foo.dart...                  0.8s

  error • foo.dart:2:27 • Const variables must be initialized with a constant value. Try changing the initializer to be a constant expression. • const_initialized_with_non_constant_value
  error • foo.dart:2:27 • Invalid constant value. • invalid_constant
           - The error is in the super constructor invocation of 'Bar', and occurs here at foo.dart:1:0.
           - The evaluated constructor 'Bar' is called by 'Foo._' and 'Foo._' is defined here at foo.dart:4:13.
  error • foo.dart:4:19 • The named parameter 'i' is required, but there's no corresponding argument. Try adding the required argument. • missing_required_argument

3 issues found.

If I add an obviously bogus line to the end of the file (like late static int class await(@String);, to name a random example I tried), there are still no errors in the problems view.
The problem seems to be related to the invalid constant declaration static const instance = Foo._();. If I comment it out, then the other errors in the file appear in the problems view. If I then un-comment it, the other errors in the file don't go away.
Reproduced using the latest SDK as of 54abf5ea6b2b13732ae49a7338a1247df0afbb86.

@scheglov


text:
Analyzer Feedback from IntelliJ

Version information

IDEA AI-233.14808.21.2331.11709847
3.2.4
AI-233.14808.21.2331.11709847, JRE 17.0.10+0--11572160x64 JetBrains s.r.o., OS Windows 11(amd64) v10.0 , screens 2400.0x1350.0

@alexander-doroshko, virtually all of these Analyzer Feedback from IntelliJ issues are empty; I suspect most people are hitting the action unintentionally and / or aren't realizing they're filing issues. Can we just remove it? We have better mechanisms now for users to provide us feedback.
https://github.com/JetBrains/intellij-plugins/blob/master/Dart/src/com/jetbrains/lang/dart/ide/errorTreeView/AnalysisServerFeedbackAction.kt

@devoncarew Sure, I don't mind removing it if it hasn't proved to be useful. Shall we notify our email group just to check that there are no other ideas or objections?

@devoncarew Sure, I don't mind removing it if it hasn't proved to be useful. Shall we notify our email group just to check that there are no other ideas or objections?

Sounds good. As a data point, we get on the order of a dozen of these issues filed a month, and I don't remember when the last non-empty one was.

authors: abh2311 alexander-doroshko devoncarew


text:
feat: support superscope search in later initializer.

@stereotype441 @johnniwinther
https://github.com/dart-lang/sdk/issues/43137

Thanks for the pull request! I've created a gerrit CL for it and made some modifications: https://dart-review.googlesource.com/c/sdk/+/159842

This PR has landed as https://dart.googlesource.com/sdk/+/991f877e53d76e8ba5c400dddcb338d86e98aaaf

https://dart-review.googlesource.com/c/sdk/+/159842
I've submitted this PR before to add active search logic to later field reference's parent class, is this PR a viable solution?

authors: johnniwinther lfkdsk


text:
Enable ci testing on osx

Fixes #26.

Glad you're trying this out!

btw, it failed bacause we haven't updated .so file for iOS yet

I fixed the .so build with this change upstream in the tf repo https://github.com/tensorflow/tensorflow/pull/32788.

authors: kevmoo lambdabaa truongsinh


text:
ZMQ and python. DWX_MTX_GET_ALL_OPEN_TRADES() "None" output when 18 or more positions are opened.

Hi all
Thank you in advance for your support.
I have been operating with python code requesting y zmq market data and open positions. I can obtain open positions but when i have opened 18 or more trades, the method DWX_MTX_GET_ALL_OPEN_TRADES() gives me None as output instead of the open trades.
Have anybody had the same problem anytime. Could it be related with the waiting time of response?
Thank you in advance!

Hi all.  I am answering myself. The problems is that in MQL4 the length of the string received from metatrader has a limit number of characters.
You can solve the issue by requesting less information in the Darwinex code in the mql4 script. In the concrete method  around line 775. the following code is the orignal code from Darwinex:
// GET OPEN ORDERS
void DWX_GetOpenOrders(string &zmq_ret) {

   bool found = false;

   zmq_ret = zmq_ret + "'_action': 'OPEN_TRADES'";
   zmq_ret = zmq_ret + ", '_trades': {";

   for(int i=OrdersTotal()-1; i>=0; i--) {
      found = true;

      if (OrderSelect(i,SELECT_BY_POS)==true) {

         zmq_ret = zmq_ret + IntegerToString(OrderTicket()) + ": {";

         zmq_ret = zmq_ret + "'_magic': " + IntegerToString(OrderMagicNumber()) + ", '_symbol': '" + OrderSymbol() + "', '_lots': " + DoubleToString(OrderLots()) + ", '_type': " + IntegerToString(OrderType()) + ", '_open_price': " + DoubleToString(OrderOpenPrice()) + ", '_open_time': '" + TimeToStr(OrderOpenTime(),TIME_DATE|TIME_SECONDS) + "', '_SL': " + DoubleToString(OrderStopLoss()) + ", '_TP': " + DoubleToString(OrderTakeProfit()) + ", '_pnl': " + DoubleToString(OrderProfit()) + ", '_comment': '" + OrderComment() + "'";

         if (i != 0)
            zmq_ret = zmq_ret + "}, ";
         else
            zmq_ret = zmq_ret + "}";
      }
   }
   zmq_ret = zmq_ret + "}";

}

Just adjust to the parameters you need to trade. For example:
// GET OPEN ORDERS
void DWX_GetOpenOrders(string &zmq_ret) {

   bool found = false;

   zmq_ret = zmq_ret + "'_action': 'OPEN_TRADES'";
   zmq_ret = zmq_ret + ", '_trades': {";

   for(int i=OrdersTotal()-1; i>=0; i--) {
      found = true;

      if (OrderSelect(i,SELECT_BY_POS)==true) {

         zmq_ret = zmq_ret + IntegerToString(OrderTicket()) + ": {";

         zmq_ret = zmq_ret + "'_comment': '" + OrderComment() + "'";

         if (i != 0)
            zmq_ret = zmq_ret + "}, ";
         else
            zmq_ret = zmq_ret + "}";
      }
   }
   zmq_ret = zmq_ret + "}";

}

This is a tactic and temporary solution. The strategic solution is to make another method where the allowed number of characters is unlimited. However if the information you need when you make a request from metatrader is small it works

repo: darwinex/DarwinexLabs


text:
Implement middleware for responders / subscribers

Certain functions such as auth or request decoration would make use of middlewares, as in, virtually everywhere (Express & co).
The following code...
responder.on('request', function(req, cb) {
    // do something;
});

should still work with a middleware:
responder.on('request', middleware, function(req, cb) {
    // do something if middleware allows
});

Hi.
Can we use promise-based middleware? and what is the write way to implement it? thanks

// do something if middleware allows
not sure i understand this.
what signature are we expecting the middleware function to have?

It should be the same as the actual handler, and it should have a means to stop execution, either through a next() function or throwing inside a promise.

ah ok. so would this work?
responder.use('somerequest',(req,cb,next) => {
   // do something
});
responder.use('somerequest',(req,cb,next) => {
   // do something else
});
responder.on('somerequest',(req,cb) => {
   // ...
});

what would happen with the callback in a middleware? what if it gets called? can it be called again later down the line? what effect would that have?

Hey,
I implemented a promise based middleware just like app.use() in express. At any level in the middleware, we can do either res.resolve or res.reject and the requester will trigger .then() and .catch() accordingly.

@tericnexus cool! do you have it on a fork? does it work with callbacks as well? (because cote has to support callbacks for now.)

@dashersw I've come up with a pretty clear idea of middleware based on express which will require no breaking changes and still support callbacks.
src/components/responder.js
`
on(type, ...listeners) {
super.on(type, (...args) => {
let index = 0;
const next = () => {
const rv = listeners[index++](...args, next);
    if (rv && typeof rv.then == 'function') {
      const cb = args.pop();
      rv.then((val) => cb(null, val)).catch(cb);
    }
  };
  next();
});

}
`

requester.send({
type: 'ping',
},
(err, str) => {
console.log(str);
}
);

Greate solution. I've been using it for a week now, but recently I noticed that the next function becomes undefined when there is an await in the function.

responder.on(
  'ping',
    async (req, cb, next) => {
    const test = await Test();
    next();
  },
  async (req, cb, next) => {
    // next becomes undefined here
    console.log('responder middleware 2 - rate limiter');
  }
)

authors: Keramet Msiavashi dashersw mr-kashif orrgal1 tericnexus

repo: dashersw/cote


text:
Add - Collection Tabs

it fixes issue #6
Every collection link opens a new tab for itself and has a piece of last query information.

the active tab gets darker and the inactive tabs are brighter. could you please invert this theme so that the active tab is bright?

Thank you for the great work @ziftinpeki, this feature is so necessary! Could you please address my comments?

I've tried to fix all of them please check my updates and comments

authors: dashersw ziftinpeki

repo: dashersw/mogollar

license: MIT


text:
identities.register() returns identity with 0 credit balance

identity = await client.platform?.identities.register();
returns an identity with balance 0 on v.21 dashmate
I would expect the returned identity to have balance 9997899, the same as returned by identities.get()
Filed here for tracking and retesting once v.22 is released.

v.22 does the same. I think it's because the SDK doesn't retrieve the identity at the end of .register. If the ST broadcast is succesful, it simply returns the identity that it submitted (see https://github.com/dashevo/platform/blob/v0.22-dev/packages/js-dash-sdk/src/SDK/Client/Platform/methods/identities/register.ts#L33-L45)

looks like an issue with State Transition Acknowledgement which JS SDK doesn't fully support yet

https://github.com/dashevo/platform/tree/feat/identity-register-balance

temporary workaround is to refetch the identity:
identity = await client.platform?.identities.register();

identity = await client.platform?.identities.get(identity.getId()); // TODO remove this line once identities.register() returns correct identity.balance

retest with v0.22.3 please

I'm now receiving a balance when registering identities with v0.22.3 :+1:

Retest blocked due to new Dashmate issue [to be raised]

v0.22 is on testnet so you can test directly against that without running a dashmate node.

Actually that's a point. FYI @jaboski what do you think about that? Can you provide a set of steps that brings a proof the initial issue being fixed?

Yes, I agree with thephez - you can retest that directly on testnet, there is no point to run it on dashmate.
About a set of steps: just follow the instruction in description, e.g.:
const registeredIdentity = await client.platform.identities.register();  console.log('Balance: ', registeredIdentity.balance);

I agree with thephez, you can retest it directly on testnet, there's no point to run it on dashmate.
About steps - just follow the description, e.g.:
const registeredIdentity = await client.platform.identities.register(); console.log('Balance: ', registeredIdentity.balance);

retest passed on dashmate@.22.12

authors: SamKirby22 dash-maverick dashameter jaboski thephez

repo: dashevo/platform


text:
Use Taskspec fuse implementation

This is using the Task.fuse (see https://github.com/dask/dask/pull/11509) method for the Fused task. In theory this should've reduced complexity quite a bit but I encountered problems since the subgraphs are including dead nodes, i.e. there are expressions in _expr that are no longer reachable if we walk the tasks. The Task.fuse raises in such a case since it requires a subgraph that is uniquely reducing to a single task (I put that condition in to catch stuff like this).
Adding an explicit culling step in advance fixes this so everything is OK. However, where is this redundant node coming from and can we get rid of this??
cc @phofl

I'ver removed the culling, things should be good to go now

thanks

authors: fjetter phofl

repo: dask/dask-expr


text:
OAR: add async to _submit_job method

_submit_job is awaited by the Job.start.

Thanks a lot for fixing this! This bug was very likely added with the SpecCluster refactor (#307) and not noticed since the 0.7 release ...
I added an entry in the changelog, so merging this!

As discussed IRL, if you ever get the chance to add CI for OAR, it would be super appreciated!
I would also make OAR more of a first citizen in dask-jobqueue (as SLURM, SGE and PBS) and help avoid similar problems in the future. We use docker-compose for your CI, you can have a look how it is done for SLURM, SGE and PBS: https://github.com/dask/dask-jobqueue/tree/master/ci.
While I am at it, one of the limitation of OARCluster: in the OAR cluster I have access to, oarsub is not installed on the compute nodes. My understanding is that this is the default setup in OAR.
If you create your OARCluster on a compute node (rather than on the login node, which is easier but not completely best practice compliant), a work-around as the one mentioned https://github.com/dask/dask-jobqueue/issues/333#issuecomment-530263090 is probably necessary.
If it turns out that oarsub is not installed on the compute nodes on your OAR cluster either, we could add a parameter to OARCluster e.g. login_node_host or something similar to make OARCluster easier to use.

authors: lesteve msimonin

repo: dask/dask-jobqueue


text:
Setup unit test overview and test report

CI has been a little fragile the last couple of days. In distributed we put in some infrastructure that helps us identify failing tests.

Unit Test Results aggregation

Every job aggregates the unit test results over the test matrix, e.g. (sorry, bad example ince there is only one failing test :)

What's even more useful is the test report, see https://dask.github.io/distributed/test_short_report.html
For instance, this test is failing sporadically on py3.10 and below but it happens frequently so it is likely a problem (see https://github.com/dask/distributed/pull/8425)

while this one is something that is likely of no concern

Seems reasonable to me

authors: fjetter jrbourbeau

repo: dask/dask


text:
dask.dataframe.describe error with nullable data types

What happened:
Error with dask.dataframe.describe() when columns contain nullable data types.
What you expected to happen:
Similar output as pandas.dataframe.describe() (which works)
Minimal Complete Verifiable Example:
import pandas as pd
import dask.dataframe as dd

n = 10
test = pd.DataFrame({
    '1': pd.Series(['a', pd.NA]*n, dtype = pd.StringDtype()),
    '2': pd.Series([1, pd.NA]*n, dtype = pd.Int64Dtype()),
    '3': pd.Series([0.56, pd.NA]*n, dtype = pd.Float64Dtype())
})
test.describe() # works no problem

ddf = dd.from_pandas(test, npartitions=2)
ddf.describe().compute() # error

 full traceback
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
AttributeError: 'NAType' object has no attribute 'conjugate'

The above exception was the direct cause of the following exception:

TypeError                                 Traceback (most recent call last)
/var/folders/hf/2s7qjx7j5ndc5220_qxv8y800000gn/T/ipykernel_4176/3509056714.py in <module>
----> 1 dd_df[['2']].describe().compute()

~/mambaforge/envs/dask-mini-tutorial/lib/python3.9/site-packages/dask/dataframe/core.py in describe(self, split_every, percentiles, percentiles_method, include, exclude, datetime_is_numeric)
   2522                 bools_and_times = self._meta.select_dtypes(include=_include)
   2523                 if len(bools_and_times.columns) == 0:
-> 2524                     return self._describe_numeric(
   2525                         self,
   2526                         split_every,

~/mambaforge/envs/dask-mini-tutorial/lib/python3.9/site-packages/dask/dataframe/core.py in _describe_numeric(self, data, split_every, percentiles, percentiles_method, is_timedelta_column, is_datetime_column)
   2630             num.count(split_every=split_every),
   2631             num.mean(split_every=split_every),
-> 2632             num.std(split_every=split_every),
   2633             num.min(split_every=split_every),
   2634             num.quantile(percentiles, method=percentiles_method),

~/mambaforge/envs/dask-mini-tutorial/lib/python3.9/site-packages/dask/dataframe/core.py in wrapper(self, *args, **kwargs)
     94         elif kwargs.get("numeric_only") is True:
     95             self = self._get_numeric_data()
---> 96         return func(self, *args, **kwargs)
     97
     98     return wrapper

~/mambaforge/envs/dask-mini-tutorial/lib/python3.9/site-packages/dask/dataframe/core.py in std(self, axis, skipna, ddof, split_every, dtype, out, numeric_only)
   2148             return handle_out(out, result)
   2149         else:
-> 2150             v = self.var(skipna=skipna, ddof=ddof, split_every=split_every)
   2151             name = self._token_prefix + "std"
   2152             result = map_partitions(

~/mambaforge/envs/dask-mini-tutorial/lib/python3.9/site-packages/dask/dataframe/core.py in wrapper(self, *args, **kwargs)
     94         elif kwargs.get("numeric_only") is True:
     95             self = self._get_numeric_data()
---> 96         return func(self, *args, **kwargs)
     97
     98     return wrapper

~/mambaforge/envs/dask-mini-tutorial/lib/python3.9/site-packages/dask/dataframe/core.py in var(self, axis, skipna, ddof, split_every, dtype, out, numeric_only)
   2007
   2008             # pandas 1.0+ does not implement var on timedelta
-> 2009             result = self._var_numeric(skipna, ddof, split_every)
   2010
   2011             if isinstance(self, DataFrame):

~/mambaforge/envs/dask-mini-tutorial/lib/python3.9/site-packages/dask/dataframe/core.py in _var_numeric(self, skipna, ddof, split_every)
   2028         cols = num._meta.columns if is_dataframe_like(num) else None
   2029
-> 2030         var_shape = num._meta_nonempty.values.var(axis=0).shape
   2031         array_var_name = (array_var._name,) + (0,) * len(var_shape)
   2032

~/mambaforge/envs/dask-mini-tutorial/lib/python3.9/site-packages/numpy/core/_methods.py in _var(a, axis, dtype, out, ddof, keepdims, where)
    239     # numbers and complex types with non-native byteorder
    240     else:
--> 241         x = um.multiply(x, um.conjugate(x), out=x).real
    242
    243     ret = umr_sum(x, axis, dtype, out, keepdims=keepdims, where=where)

TypeError: loop of ufunc does not support argument 0 of type NAType which has no callable conjugate method

Anything else we need to know?:
Individually calling the methods in describe, std and quantile throw TypeError: Cannot interpret 'Int64Dtype()' as a data type

 full traceback
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
/var/folders/hf/2s7qjx7j5ndc5220_qxv8y800000gn/T/ipykernel_4176/418681936.py in <module>
----> 1 ddf['2'].quantile().compute()

~/mambaforge/envs/dask-mini-tutorial/lib/python3.9/site-packages/dask/base.py in compute(self, **kwargs)
    286         dask.base.compute
    287         """
--> 288         (result,) = compute(self, traverse=False, **kwargs)
    289         return result
    290

~/mambaforge/envs/dask-mini-tutorial/lib/python3.9/site-packages/dask/base.py in compute(traverse, optimize_graph, scheduler, get, *args, **kwargs)
    569         postcomputes.append(x.__dask_postcompute__())
    570
--> 571     results = schedule(dsk, keys, **kwargs)
    572     return repack([f(r, *a) for r, (f, a) in zip(results, postcomputes)])
    573

~/mambaforge/envs/dask-mini-tutorial/lib/python3.9/site-packages/dask/threaded.py in get(dsk, result, cache, num_workers, pool, **kwargs)
     77             pool = MultiprocessingPoolExecutor(pool)
     78
---> 79     results = get_async(
     80         pool.submit,
     81         pool._max_workers,

~/mambaforge/envs/dask-mini-tutorial/lib/python3.9/site-packages/dask/local.py in get_async(submit, num_workers, dsk, result, cache, get_id, rerun_exceptions_locally, pack_exception, raise_exception, callbacks, dumps, loads, chunksize, **kwargs)
    505                             _execute_task(task, data)  # Re-execute locally
    506                         else:
--> 507                             raise_exception(exc, tb)
    508                     res, worker_id = loads(res_info)
    509                     state["cache"][key] = res

~/mambaforge/envs/dask-mini-tutorial/lib/python3.9/site-packages/dask/local.py in reraise(exc, tb)
    313     if exc.__traceback__ is not tb:
    314         raise exc.with_traceback(tb)
--> 315     raise exc
    316
    317

~/mambaforge/envs/dask-mini-tutorial/lib/python3.9/site-packages/dask/local.py in execute_task(key, task_info, dumps, loads, get_id, pack_exception)
    218     try:
    219         task, data = loads(task_info)
--> 220         result = _execute_task(task, data)
    221         id = get_id()
    222         result = dumps((result, id))

~/mambaforge/envs/dask-mini-tutorial/lib/python3.9/site-packages/dask/core.py in _execute_task(arg, cache, dsk)
    117         # temporaries by their reference count and can execute certain
    118         # operations in-place.
--> 119         return func(*(_execute_task(a, cache) for a in args))
    120     elif not ishashable(arg):
    121         return arg

~/mambaforge/envs/dask-mini-tutorial/lib/python3.9/site-packages/dask/utils.py in __call__(self, arg, *args, **kwargs)
    619         """
    620         meth = self.dispatch(type(arg))
--> 621         return meth(arg, *args, **kwargs)
    622
    623     @property

~/mambaforge/envs/dask-mini-tutorial/lib/python3.9/site-packages/dask/dataframe/backends.py in percentile(a, q, interpolation)
    531 @percentile_lookup.register((pd.Series, pd.Index))
    532 def percentile(a, q, interpolation="linear"):
--> 533     return _percentile(a, q, interpolation)
    534
    535

~/mambaforge/envs/dask-mini-tutorial/lib/python3.9/site-packages/dask/array/percentile.py in _percentile(a, q, interpolation)
     29             a = a.values
     30
---> 31     if np.issubdtype(a.dtype, np.datetime64):
     32         values = a
     33         a2 = values.view("i8")

~/mambaforge/envs/dask-mini-tutorial/lib/python3.9/site-packages/numpy/core/numerictypes.py in issubdtype(arg1, arg2)
    417     """
    418     if not issubclass_(arg1, generic):
--> 419         arg1 = dtype(arg1).type
    420     if not issubclass_(arg2, generic):
    421         arg2 = dtype(arg2).type

TypeError: Cannot interpret 'Int64Dtype()' as a data type

Environment:

Dask version: 2021.11.2
Python version: 3.9.7
Operating System: MacOS
Install method (conda, pip, source): conda

Yeah this isn't terribly surprising. The pd nullable types are not super well supported. I think you can get through your first issue with this change:
diff --git a/dask/dataframe/core.py b/dask/dataframe/core.py
index 3a1e517b..d5a1f910 100644
--- a/dask/dataframe/core.py
+++ b/dask/dataframe/core.py
@@ -2068,7 +2068,7 @@ Dask Name: {name}, {task} tasks"""
         name = self._token_prefix + "var-numeric" + tokenize(num, split_every)
         cols = num._meta.columns if is_dataframe_like(num) else None

-        var_shape = num._meta_nonempty.values.var(axis=0).shape
+        var_shape = num._meta_nonempty.var(axis=0).shape
         array_var_name = (array_var._name,) + (0,) * len(var_shape)

         layer = {(name, 0): (methods.wrap_var_reduction, array_var_name, cols)}

but I haven't been able to get to the bottom of the next error yet. Are you planning on looking into this?

Thanks @jsignell! Yup, I can keep looking into this.


text:
Deprecate inplace in DataFrame::eval

[x] Closes #10744
[x] Tests added / passed
[x] Passes pre-commit run --all-files

set_index didn't seem to need it, as there is already a NotImplementedError on its use in that method.

We will also start raising if inplace=False in the future, so we should also deprecate this case

thx @milesgranger

authors: milesgranger phofl


text:
TST: Travis build matrix to specify numpy/pandas ver

Specifies older version of numpy / pandas explicitly.

I'm generally +1 on the idea.
Thoughts on dropping support for Pandas 0.16 and 0.17?

It's OK, but it means dropping support of py 3.3 as pandas 0.18 no longer supports it.
However, there looks no reason to exclude pandas 0.16/0.17 from the matrix because there is no failure.

OK, sounds like it's a separate issue.  What's here looks good to me.  Merging.

authors: mrocklin sinhrks


text:
testing functionality

I think we need some testing functionality, as we'll need some test suite to check that all implementations pass it and raise errors when appropriate (e.g. pd.DataFrame([[1, 2]]).__dataframe_standard__() should raise)
Shall we add

namespace.assert_frame_equal
namespace.assert_series_equal
?

Signature could be
def assert_frame_equal(df1, df2, *, exact, atol, rtol)

Missing values: 'nan' and null values should match?
Other than that, I'd force dtype and columns order to match, no need to add configuration for that

Wouldn't it be better to keep that completely separate, a la https://github.com/data-apis/array-api-tests/?

sure, not saying they need to be part of the standard, but I'm currently writing tests like
def test_get_column_by_name() -> None:
    df = PandasDataFrame(pd.DataFrame({"a": [1, 2, 3], "b": [4, 5, 6]}))
    result = df.get_column_by_name("a")._series
    expected = pd.Series([1, 2, 3], name="a")
    pd.testing.assert_series_equal(expected, result)

which isn't ideal, some way of asserting equality between Column/DataFrame objects would really help

taking this forwards in https://github.com/MarcoGorelli/impl-dataframe-api

authors: MarcoGorelli rgommers

repo: data-apis/dataframe-api


text:
add about Data Umbrella

PR Overview
Type of contribution

[x] Code
[ ] Documentation

Reference: Issue or Pull Request (PR)

[x] Fixes #70

Description: What does this implement/fix? Explain your changes.
Adds about Data Umbrella Page
Reminders

[x] Did you review the CONTRIBUTING GUIDE?
[x] Did you reference an issue or another pull request?

Any other comments?
None

@SangamSwadiK
one minor edit, and it looks good to merge.
Probably need to sync one more time, after accepting above changes.

@SangamSwadiK
One last sync and this is good to be merged in.

Thank you @SangamSwadiK

authors: SangamSwadiK reshamas

repo: data-umbrella/data-umbrella-website


text:
Low-rank, joint imputation layers

as a complement to the column-wise independent imputation layers we have now.

Conditioning on non-missing data will be potentially computationally complex and require a lot of linear algebra...

Let's leave this for now - it is a big undertaking, and I think the current imputation layers need a bit of updating


text:
Support locking in integration tests

Changes
Locking Interface
Integration tests run in parallel, both within a single action runner as well as across different runs of an action (e.g. multiple people running integration tests in parallel). Some tests modify certain resources and need exclusive access to those resources during the duration of the test. As a result, when two tests that modify a shared resource run concurrently, they fail.
This PR introduces an integration test locking framework for the Go SDK & tools built on the Go SDK. In general, for Databricks integration tests, usage will look like the example in git_credentials_test.go:

func acquireGitCredentialsLock(ctx context.Context, t *testing.T, w *databricks.WorkspaceClient) {
	me, err := w.CurrentUser.Me(ctx)
	require.NoError(t, err)
	lockable := lock.GitCredentials{
		WorkspaceHost: w.Config.Host,
		Username:      me.UserName,
	}
	_, err = lock.Acquire(ctx, lockable, lock.InTest(t))
	require.NoError(t, err)
}

func TestAccGitCredentials(t *testing.T) {
	ctx, w := workspaceTest(t)

	// skip-next-line-roll
	acquireGitCredentialsLock(ctx, t, w)
	...
}

Calling lock.Acquire() will attempt to lock a resource using a lock backend, returning a lock. When used in tests, unlocking the lock will be registered automatically as a test cleanup, so you only need to check that no error occurred while acquiring the lock. By default, the lock will be held for 60 seconds, and locks will be created in a workspace for which the serialized SDK config can be found in LOCK_CLIENT_CONFIG and the path to the lock directory in LOCK_DIRECTORY.
A lockable resource is any resource implementing the Lockable interface, which provides the lock ID to use. I've added several implementations of Lockable for cases that I know about, but any string or structure containing only string/int fields can be used as a Lockable through lock.NewLockableImpl(). If a string is provided, the lock ID is the string itself. If a struct is provided, the lock ID is the struct name followed by the names of each field & the value of each field, separated by semicolon. For example: testStruct:A=hello;B=1.
OpenAPI Code Generator
Additionally, I've added a directive // skip-next-line-roll in the integration test parser. This makes it possible to skip the next statement from the codegen parsing. This should make it easier to add other code to integration tests without needing to update the OpenAPI test compiler. Just add the comment right before the statement you want to skip from example generation, e.g.
func TestMwsAccAbc(t *testing.T) {
	...
	// skip-next-line-roll
	myCustomFunc()
	....
}

Finally, I've made a small bugfix in the comment parsing, which currently sets hint to the last comment before the current node, even if the current node is not immediately preceded by a comment.
Tests

[ ] make test passing
[ ] make fmt applied
[ ] relevant integration tests applied

Codecov Report
Attention: 67 lines in your changes are missing coverage. Please review.

Comparison is base (aad9abf) 18.27% compared to head (f5bc4a6) 18.69%.
Report is 1 commits behind head on main.

Files
Patch %
Lines

qa/lock/databricks/databricks_backend.go
57.14%
25 Missing and 8 partials :warning:

qa/lock/lock.go
62.96%
17 Missing and 3 partials :warning:

openapi/roll/tool.go
68.00%
7 Missing and 1 partial :warning:

qa/lock/internal/lockable.go
92.59%
1 Missing and 1 partial :warning:

qa/lock/lock_options.go
66.66%
2 Missing :warning:

qa/lock/lockables.go
50.00%
2 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #776      +/-   ##
==========================================
+ Coverage   18.27%   18.69%   +0.41%
==========================================
  Files         110      115       +5
  Lines       14834    15135     +301
==========================================
+ Hits         2711     2829     +118
- Misses      11901    12071     +170
- Partials      222      235      +13

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

authors: codecov-commenter mgyucht

repo: databricks/databricks-sdk-go


text:
fix(#244): Error When Freezing Array.prototype

In a particular file, the Array.prototype is being frozen which is causing compatibility issues. The issue arises when the at method is added to the Array.prototype after it has been frozen, resulting in an error.
Steps to reproduce:

Run the application.
Observe the error related to the at method.

Expected behavior:
The application should run without errors.
Actual behavior:
An error occurs when running the application due to the Array.prototype being frozen before the at method is added.
Proposed solution:
Modify the code to conditionally add the at method to the Array.prototype only if it doesn't already exist.

Fixes. #244

closing this PR since a request is already raised.

repo: databricks/databricks-sql-nodejs


text:
Remove the schedule for a released version.

Koalas 1.2.0 was released as scheduled.

I'd merge this now.

Nice!

authors: HyukjinKwon ueshin

repo: databricks/koalas


text:
Please clarify the License Policy

Hi DataBricks,
I am not sure which license shall be applied to the Dolly-v2-12b so that I cannot give proper credits to your guys when I distribute your model.
In your Model View(https://github.com/databrickslabs/dolly#model-overview), it states that, dolly-v2 -12b applies CC-BY-SA, but do not specify the specifical version of CC-BY-SA. As you know, there is four versions of CC-BY-SA (See https://opendefinition.org/licenses/cc-by-sa/). What makes me more confused is that the LICENSE shows Apache 2.0 License text.
So just clarify the license applied to the Dolly-v2-12b model and many thanks to you

Duplicate of https://huggingface.co/databricks/dolly-v2-12b/discussions/86
No, it says that the Dolly 15K dataset used for fine-tuning is licensed as CC-BY-SA 3.0, and that is elaborated at https://huggingface.co/datasets/databricks/databricks-dolly-15k
The model card says the model itself (weights) are MIT licensed.

authors: JayLiangs srowen

repo: databrickslabs/dolly


text:
[BUG] Issues with running test script locally

Describe the bug
In following the directions in https://github.com/datacommonsorg/website/blob/master/README.md, there look to be a couple of dep issues in the script. For sure pytest and flask are not currently being installed correctly, but also there is this bug:
___________ ERROR collecting server/tests/services/discovery_test.py ___________
ImportError while importing test module '/Users/jengishchokusu/website-1/server/tests/services/discovery_test.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/importlib/__init__.py:90: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
server/__init__.py:23: in <module>
    from flask_babel import Babel
E   ModuleNotFoundError: No module named 'flask_babel'

To Reproduce
Running on Mac in terminal.
Steps to reproduce the behavior:

Follow the directions in readme to run tests.

Expected behavior
Should run tests and not fail on dependencies.

I can't reproduce this with a checkout from this morning. My process was:

clone the repo
run ./run_tests.sh --setup_python
run ./run_tests.sh -a

Can you confirm you ran the setup_python step first? I'll send a PR to update the docs to require that, as it does seem unclear.

authors: KilimAnnejaro wittrock

repo: datacommonsorg/website