text:
Add a "call with new" option to JS interop function/method bindings

It is common in JavaScript to nest constructors under a namespace. For example, CanvasKit has a canvasKit.SkPictureRecorder and canvasKit.SkPaint constructors. These constructors must be called with the new keyword. However, Dart's JS interop has no way to instruct the compiler to call these functions with new.
The feature request is to add a way to tell DDC and dart2js to use the new keyword.
One option is to add a new annotation @callWithNew used like this:
@JS()
class CanvasKit {
  @callWithNew
  external SkPictureRecorder MakePictureRecorder();
}

Another option is to add an optional boolean parameter to the @JS annotation:
@JS()
class CanvasKit {
  @JS(callWithNew: true)
  external SkPictureRecorder MakePictureRecorder();
}

Is the goal that you have JS that looks like new CanvasKit.MakePictureRecorder()?
We don't support it today, but my first idea for how to address this use case would be to support renames for factory constructors.
@JS()
class SkPictureRecorder {
  @JS('CanvasKit.MakePictureRecorder')
  external factory SkPictureRecorder()
}

I'm not sure if there are blockers for allowing renames on factory constructors like there are blockers for renames of instance members...

@yjbanov - can you clarify, is part of the issue here that you'd like the class to remain @anonymous in Dart? Or are there issues if you tried to access the constructor from the window object?
I ask because for non-anonymous classes today you can give the class a different name, and both compilers use the name in the @JS annotation of the class as the constructor path. For example: new SkPictureRecorder() is compiled to new canvasKit.MakePictureRecorder() in this code:
@JS('CanvasKit.MakePictureRecorder')  // the name matches the path to the constructor invoked with `new`
class SkPictureRecorder {             // the class name can be different in Dart
  external factory SkPictureRecorder();
}

For anonymous classes created with a constructor we'd need something more. I like @natebosch's suggestion as a short term solution. Long term, once we fix DDC to behave like dart2js when it comes to is/as checks, I believe you'll no longer need to use anonymous as you had to do recently.

I don't think this is about @anonymous, or maybe it is and I'm just not sure how. The issue is that you have a reference to a JS object and that object provides a method that requires that it's called with new and there's no way to tell JS-interop to call that method with the new keyword.
Both @natebosch's and @sigmundch's ideas assume there's some top-level variable available at the global scope named CanvasKit that would make the expression new CanvasKit.MakePictureRecorder() valid, but there's no such variable. All you have is a reference to an object of type/prototype CanvasKit, and that only happens to be the case in the CanvasKit API. More generally, you can get any object from JS, anonymous or not, with a method on it that requires that it's called with new. You don't even need an object, it can be any function reference that requires new because the body of any function/method may need access to this.
Here are some possible scenarios (it's more interesting when obj, bar2, and baz are not global variables):
function foo() {
  if (new.target) {
    console.log('foo called with new');
  } else {
    console.log('foo not called with new');
  }
}

let obj = {
  'bar': function () {
    if (new.target) {
      console.log('bar called with new');
    } else {
      console.log('bar not called with new');
    }
  },
  'bar2': function() {
    return {
      'bar3': function () {
        if (new.target) {
          console.log('bar3 called with new');
        } else {
          console.log('bar3 not called with new');
        }
      }
    };
  }
};

class Baz {
  qux() {
    if (new.target) {
      console.log('qux called with new');
    } else {
      console.log('qux not called with new');
    }
  }
}

foo();
new foo();

obj.bar();
new obj.bar();

obj.bar2().bar3();
new (obj.bar2().bar3)(); // extra parens to fix operator precedence

// Now let's say to want to keep a reference on bar2 and call bar3 method
// repeatedly. This is not the same as executing `new obj.bar2().bar3()`
// repeatedly, because the latter calls `bar2()` multiple times.
let bar2 = obj.bar2();
for (let i = 0; i < 10; i++) {
  new bar2.bar3();
}

let baz = new Baz();
baz.qux();
// new baz.qux();  // ILLEGAL

interesting - a couple questions:

(a) are there cases where you'd need to call the method both with and without new?
(b) how much is the problem access to this or having a this instance with a prototype based on bar3's constructor? In other words, are there cases where it would be enough to use bind and use a different object instance as this?

The concern I have with annotating an instance member with a calling convention, is that we can only make that work well if the target is statically known. Dynamic calls on that method will not know what to do. The one way we can address it is to only allow such annotation on external static methods or external static extension methods. We don't have support for the latter yet, but do plan to add it in the near future (#44057).
Shorter term, maybe we can provide some helper methods in js_util as an alternative? That would allow you to choose to use new on each callsite if you need it. In fact, it seems like js_util.callConstructor was added with a similar goal in mind, but I don't believe we have tested it in scenarios like the one you describe. I just tried it with a small example, and it looks promising. Would that do what you need?
The one caveat is that callConstructor is a bit more convoluted when you pass arguments, so we need to play around with it, and may be worth tuning it or providing something else to make it smoother.

(a) are there cases where you'd need to call the method both with and without new?

Not in my case, but JS allows it, so I guess Dart JS-interop should support it too. I think this could be supported by allowing renames, e.g.:
@JS()
class CanvasKit {
  @callWithNew
  external SkPictureRecorder MakePictureRecorder();

  /// Even though the Dart method is called `MakePictureRecorderWithoutNew`, the `@JS` annotation
  /// rewrites it to call `MakePictureRecorder` instead.
  @JS('MakePictureRecorder')
  external SkPictureRecorder MakePictureRecorderWithoutNew();
}

(b) how much is the problem access to this or having a this instance with a prototype based on bar3's constructor? In other words, are there cases where it would be enough to use bind and use a different object instance as this?

I'm not sure what would be sufficient, but all combinations are possible, each distinct from others:
const obj = {
  x: 42,
  func: function() {
    console.log(`  called ${ new.target ? 'with' : 'without' } new.`);
    console.log(`  this is ${ this }${ this.x ? ' with ' + this.x : '' }.`);
  }
};

console.log('Unbound tear-off');
const unboundFunc = obj.func;
unboundFunc();

console.log('Bound tear-off');
const boundFunc = unboundFunc.bind({ x: 3.14 });
boundFunc();

console.log('Plain invocation');
obj.func();

console.log('Invocation with new');
new obj.func();

Output:
> "Unbound tear-off"
> "  called without new."
> "  this is [object Window]."
> "Bound tear-off"
> "  called without new."
> "  this is [object Object] with 3.14."
> "Plain invocation"
> "  called without new."
> "  this is [object Object] with 42."
> "Invocation with new"
> "  called with new."
> "  this is [object Object]."

Shorter term, maybe we can provide some helper methods in js_util as an alternative? That would allow you to choose to use new on each callsite if you need it. In fact, it seems like js_util.callConstructor was added with a similar goal in mind, but I don't believe we have tested it in scenarios like the one you describe. I just tried it with a small example, and it looks promising. Would that do what you need?

I think a less convenient escape hatch for uncommon situations would be fine, so long as it performs well.

I think a less convenient escape hatch for uncommon situations would be fine, so long as it performs well.

Sounds good - let us know if the current callConstructor is not enough. It appears like it may do already what you need.

I just tried the example snippet and it does not seem to be working with flutter 3 on the web.

authors: natebosch rodydavis sigmundch yjbanov


text:
Implement non-function type alias support in web debuggers

This issue tracks support of the non-function type alias feature.  See the enclosing meta-issue for details.
cc @franklinyow @sigmundch @mit-mit

I don't believe there is any specific work required for this, other than validating that the feature works as expected.
@jacob314 @grouma - do you agree?

Yeah I think we just need to validate that it works and if it doesn't, likely make edits to the logic here.

@grouma do we have a plan for validating this?

I think making use of the feature in a package:dwds smoke test, along with some specific VM Service protocols, should give us enough confidence.

Maybe it is worth also adding a couple tests to cover https://github.com/dart-lang/sdk/issues/44925 as well?

Wouldn't hurt. We'll need to know the SDK version that contains these features.

@grouma this will be released in 2.13 are you waiting on a dev build or can you go ahead and validate?  This CL can be patched in to build a version with the feature on by default.

Ping me when a dev build is available. It's much easier to validate that way.

It seems we won't have a dev build this week due to Easter holiday in Europe , can we validate with the CL?

@grouma this has landed, can you please test on master (I don't expect we will have a dev build before the branch cut window opens).

I downloaded the latest SDK:
Dart SDK version: 2.13.0-203.0.dev (dev) (Mon Apr 5 09:39:33 2021 -0700) on "macos_x64"
I updated the minimum SDK for my example project and included this:
typedef JsonMap = Map<String, dynamic>;

There are no errors with analysis, however  I'm having issues building:
[SEVERE] build_web_compilers:entrypoint on web/main.dart: This builder requires Dart inputs without syntax errors.
However, asset:webdev_example_app/web/main.dart (or an existing part) contains the following errors.
main.dart:11:17: Invalid generic function type.

Try fixing the errors and re-running the build.

[INFO] Running build completed, took 3.7s
[INFO] Caching finalized dependency graph completed, took 180ms
[SEVERE] Failed after 3.9s
[INFO] ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

cc @jakemac53
Without being able to build it's impossible for me to validate.

@grouma This is not in any dev build and I don't think it will be until AAR is back from vacation which will be too late.  I think you need to test this out on master.

I'll build from master locally but I'm confused as to why this isn't in the latest raw SDK. I thought raw corresponded to master. The given timestamp says it was built this morning. Correct my understanding.

@grouma is this a pkg:analyzer issue then?  You may need a version of pkg:analyzer with this enabled - cc @scheglov .

I don't think this is an analyzer issue. There are no analysis errors. (There were some until I upgraded the SDK and updated the pubspec). I think this may be a package:build issue. cc @natebosch

FYI, I'm waiting for the flag switch to be rolled into google3, and will publish package:analyzer then.

@grouma webdev serve works for me.
I had to ensure that the language version is 2.13, and use a dependency_override on analyzer from my local SDK checkout since it isn't published yet. It also took a few other overrides to get a valid solve. Here is the pubspec I used in a project where this worked:
name: dart_repro

environment:
  sdk: ">=2.13.0-0 <3.0.0"

dev_dependencies:
  build_runner: any
  build_web_compilers: any

dependency_overrides:
  analyzer:
    path: ../dart-sdk/sdk/pkg/analyzer
  front_end:
    path: ../dart-sdk/sdk/pkg/front_end
  _fe_analyzer_shared:
    path: ../dart-sdk/sdk/pkg/_fe_analyzer_shared
  package_config: 2.0.0

Thanks @natebosch, I was able to get it working.
Debugging doesn't blow up so that's good. I'm not sure what the intended behavior is here though. The displayed types are desugared. Look at the variables and console in the screenshot below. I suppose that's the desired behavior?

Expanding the types seems like fine behavior to me.

source: gharchive/issue

authors: franklinyow grouma leafpetersen natebosch scheglov sigmundch

repo: dart-lang/sdk

license_type: permissive

license_source: github-api


text:
Analyzer Feedback from IntelliJ

Version information

IDEA AI-203.7717.56.2031.7678000
2.14.4
AI-203.7717.56.2031.7678000, JRE 11.0.10+0-b96-7249189x64 JetBrains s.r.o., OS Linux(amd64) v5.11.0-38-generic, screens 1600.0x900.0

cc @jwren - see my comment in the other recent closed feedback bug

authors: devoncarew nitunga


text:
[CP] Update changelog with details about jsinterop changes

commit(s) to merge: https://github.com/dart-lang/sdk/commit/b97737c3b53a6c043a6a1b97e61b8670ded75629
merge instructions: clean merge
What is the issue: Changelog was missing some details we'd like to include before the 2.15 stable release
What is the fix: Adding the text to the changelog.
Why cherrypick: Highlight important features in the changelog.
Risk: extremely low, near zero, maybe ε?
Link to original  issue(s): #44533
/cc @kevmoo @mit-mit @whesse @athomas @vsmenon @devoncarew @rileyporter

approved

Merged to beta in c8b50bd835770529f30c6079376c824264a57e2c (2.15.0-268.8.beta).

authors: athomas devoncarew sigmundch


text:
[CP] DDC support for static setters of const fields and enhanced num fields in 2.17 beta 3/stable cut

commit(s) to merge: a75656502a427eb3c42c8e5b1c06a4900fe126e9 and 471fcee0795acfa9c20b8e980ac7db9cd8d336bc
merge instructions: merge cherrypicks cleanly into the (tentative) 2.17 beta 3/stable cut.
What is the issue: DDC historically does not support static setters of const fields (these were stubbed and never actually executed by the runtime). Enhanced enum tests later tickled this pattern, as enum fields are lowered to const fields by the CFE.
What is the fix: DDC now no longer emits enum fields inline (which was an optimization). Additionally, DDC now lazily emits static accessors that shadow any lazily-emitted field (const, static, etc.), and its runtime was updated to execute lazy setters.
Why cherrypick: This is a rare edge case that's unlikely to be hit in most code patterns. However, the ensuing Javascript type error is difficult to mitigate without avoiding the pattern entirely and may reflect poorly if encountered.
Risk: Little
Link to original  issue(s): https://github.com/dart-lang/sdk/issues/48700, https://github.com/dart-lang/sdk/issues/48717
/cc @mit-mit @whesse @athomas @vsmenon @devoncarew @sigmundch

/cc @leafpetersen
(copying the same comment from the original bug here) We can go either way on this cherry-pick. It's to address a rare corner case only. The pattern that breaks is whether you can write a setter in an enum that has the same name as the enum value.
Technically a similar pattern could've been written by hand in the past without the enhanced-enum feature (like https://github.com/dart-lang/sdk/issues/48710#issuecomment-1084526428), so it's only related to the enum work because we have tests that are triggering that pattern.
This early in the cherry-pick process I'm OK adding it in, but had we encountered this issue in 2 weeks I would have a different opinion and leaned against cherry-picking to avoid destabilizing the build.

I'm fine not cherry picking this.

This is very low risk (and not a particular useful feature), so I'm fine with not cherry-picking this as well

@Markzipan Turns out that the likely candidate build for 2.17 is 2.17.0-266.0.dev. So the first commit above made it into the cut, but the second didn't.
We are inclined not to add the latter as a cherry pick, but we wanted to confirm: are there any problems in getting the first one without the second one? If so, we should reconsider cherry-picking it (or cherry-picking a change to undo the first change).
Let us know!

No worries - there aren't any issues with selecting either of those commits out-of-order. The former actually doesn't do anything until the latter is in, so it's safely masked.

authors: Markzipan leafpetersen sigmundch


text:
Analyzer exception doesn't specify affected file

Encountering this trying to run dart fix --apply:

Console output
```
Internal error: Exception while using a AvoidRedundantArgumentValues to visit a FunctionExpressionInvocationImpl in NamedExpressionImpl in ArgumentListImpl in InstanceCreationExpressionImpl in ListLiteralImpl in NamedExpressionImpl in ArgumentListImpl in InstanceCreationExpressionImpl in NamedExpressionImpl in ArgumentListImpl in InstanceCreationExpressionImpl in ReturnStatementImpl in BlockImpl in BlockFunctionBodyImpl in MethodDeclarationImpl in ClassDeclarationImpl in CompilationUnitImpl
Bad state: [library: null][this: (DefaultParameterElementImpl) {double? height}][enclosingElement: null][reference: null]
#0      ConstVariableElement.computeConstantValue (package:analyzer/src/dart/element/element.dart:1292:9)
#1      _Visitor.checkArgument (package:linter/src/rules/avoid_redundant_argument_values.dart:105:23)
#2      _Visitor.check (package:linter/src/rules/avoid_redundant_argument_values.dart:88:7)
#3      _Visitor.visitFunctionExpressionInvocation (package:linter/src/rules/avoid_redundant_argument_values.dart:126:5)
#4      FunctionExpressionInvocationImpl.accept (package:analyzer/src/dart/ast/ast.dart:8382:15)
#5      LinterVisitor._runSubscriptions (package:analyzer/src/lint/linter_visitor.dart:1096:14)
#6      LinterVisitor.visitFunctionExpressionInvocation (package:analyzer/src/lint/linter_visitor.dart:446:5)
#7      FunctionExpressionInvocationImpl.accept (package:analyzer/src/dart/ast/ast.dart:8382:15)
#8      NamedExpressionImpl.visitChildren (package:analyzer/src/dart/ast/ast.dart:12187:17)
#9      LinterVisitor.visitNamedExpression (package:analyzer/src/lint/linter_visitor.dart:663:10)
#10     NamedExpressionImpl.accept (package:analyzer/src/dart/ast/ast.dart:12177:50)
#11     NodeListImpl.accept (package:analyzer/src/dart/ast/ast.dart:12587:20)
#12     ArgumentListImpl.visitChildren (package:analyzer/src/dart/ast/ast.dart:498:16)
#13     LinterVisitor.visitArgumentList (package:analyzer/src/lint/linter_visitor.dart:45:10)
#14     ArgumentListImpl.accept (package:analyzer/src/dart/ast/ast.dart:494:50)
#15     InstanceCreationExpressionImpl.visitChildren (package:analyzer/src/dart/ast/ast.dart:10138:19)
#16     LinterVisitor.visitInstanceCreationExpression (package:analyzer/src/lint/linter_visitor.dart:537:10)
#17     InstanceCreationExpressionImpl.accept (package:analyzer/src/dart/ast/ast.dart:10127:15)
#18     NodeListImpl.accept (package:analyzer/src/dart/ast/ast.dart:12587:20)
#19     ListLiteralImpl.visitChildren (package:analyzer/src/dart/ast/ast.dart:10978:15)
#20     LinterVisitor.visitListLiteral (package:analyzer/src/lint/linter_visitor.dart:597:10)
#21     ListLiteralImpl.accept (package:analyzer/src/dart/ast/ast.dart:10968:50)
#22     NamedExpressionImpl.visitChildren (package:analyzer/src/dart/ast/ast.dart:12187:17)
#23     LinterVisitor.visitNamedExpression (package:analyzer/src/lint/linter_visitor.dart:663:10)
#24     NamedExpressionImpl.accept (package:analyzer/src/dart/ast/ast.dart:12177:50)
#25     NodeListImpl.accept (package:analyzer/src/dart/ast/ast.dart:12587:20)
#26     ArgumentListImpl.visitChildren (package:analyzer/src/dart/ast/ast.dart:498:16)
#27     LinterVisitor.visitArgumentList (package:analyzer/src/lint/linter_visitor.dart:45:10)
#28     ArgumentListImpl.accept (package:analyzer/src/dart/ast/ast.dart:494:50)
#29     InstanceCreationExpressionImpl.visitChildren (package:analyzer/src/dart/ast/ast.dart:10138:19)
#30     LinterVisitor.visitInstanceCreationExpression (package:analyzer/src/lint/linter_visitor.dart:537:10)
#31     InstanceCreationExpressionImpl.accept (package:analyzer/src/dart/ast/ast.dart:10127:15)
#32     NamedExpressionImpl.visitChildren (package:analyzer/src/dart/ast/ast.dart:12187:17)
#33     LinterVisitor.visitNamedExpression (package:analyzer/src/lint/linter_visitor.dart:663:10)
#34     NamedExpressionImpl.accept (package:analyzer/src/dart/ast/ast.dart:12177:50)
#35     NodeListImpl.accept (package:analyzer/src/dart/ast/ast.dart:12587:20)
#36     ArgumentListImpl.visitChildren (package:analyzer/src/dart/ast/ast.dart:498:16)
#37     LinterVisitor.visitArgumentList (package:analyzer/src/lint/linter_visitor.dart:45:10)
#38     ArgumentListImpl.accept (package:analyzer/src/dart/ast/ast.dart:494:50)
#39     InstanceCreationExpressionImpl.visitChildren (package:analyzer/src/dart/ast/ast.dart:10138:19)
#40     LinterVisitor.visitInstanceCreationExpression (package:analyzer/src/lint/linter_visitor.dart:537:10)
#41     InstanceCreationExpressionImpl.accept (package:analyzer/src/dart/ast/ast.dart:10127:15)
#42     ReturnStatementImpl.visitChildren (package:analyzer/src/dart/ast/ast.dart:15092:18)
#43     LinterVisitor.visitReturnStatement (package:analyzer/src/lint/linter_visitor.dart:873:10)
#44     ReturnStatementImpl.accept (package:analyzer/src/dart/ast/ast.dart:15088:50)
#45     NodeListImpl.accept (package:analyzer/src/dart/ast/ast.dart:12587:20)
#46     BlockImpl.visitChildren (package:analyzer/src/dart/ast/ast.dart:2091:17)
#47     LinterVisitor.visitBlock (package:analyzer/src/lint/linter_visitor.dart:111:10)
#48     BlockImpl.accept (package:analyzer/src/dart/ast/ast.dart:2087:50)
#49     BlockFunctionBodyImpl.visitChildren (package:analyzer/src/dart/ast/ast.dart:2047:12)
#50     LinterVisitor.visitBlockFunctionBody (package:analyzer/src/lint/linter_visitor.dart:117:10)
#51     BlockFunctionBodyImpl.accept (package:analyzer/src/dart/ast/ast.dart:2039:50)
#52     MethodDeclarationImpl.visitChildren (package:analyzer/src/dart/ast/ast.dart:11697:10)
#53     LinterVisitor.visitMethodDeclaration (package:analyzer/src/lint/linter_visitor.dart:639:10)
#54     MethodDeclarationImpl.accept (package:analyzer/src/dart/ast/ast.dart:11689:50)
#55     NodeListImpl.accept (package:analyzer/src/dart/ast/ast.dart:12587:20)
#56     ClassDeclarationImpl.visitChildren (package:analyzer/src/dart/ast/ast.dart:2954:13)
#57     LinterVisitor.visitClassDeclaration (package:analyzer/src/lint/linter_visitor.dart:165:10)
#58     ClassDeclarationImpl.accept (package:analyzer/src/dart/ast/ast.dart:2944:50)
#59     NodeListImpl.accept (package:analyzer/src/dart/ast/ast.dart:12587:20)
#60     CompilationUnitImpl.visitChildren (package:analyzer/src/dart/ast/ast.dart:3618:21)
#61     LinterVisitor.visitCompilationUnit (package:analyzer/src/lint/linter_visitor.dart:189:10)
#62     CompilationUnitImpl.accept (package:analyzer/src/dart/ast/ast.dart:3611:50)
#63     LibraryAnalyzer._computeLints (package:analyzer/src/dart/analysis/library_analyzer.dart:406:12)
#64     LibraryAnalyzer._computeDiagnostics (package:analyzer/src/dart/analysis/library_analyzer.dart:331:7)
#65     LibraryAnalyzer.analyze (package:analyzer/src/dart/analysis/library_analyzer.dart:109:5)
#66     AnalysisDriver._analyzeFileImpl. (package:analyzer/src/dart/analysis/driver.dart:1394:11)

#67     PerformanceLog.runAsync (package:analyzer/src/dart/analysis/performance_logger.dart:50:14)

#68     AnalysisDriver._analyzeFile. (package:analyzer/src/dart/analysis/driver.dart:1308:9)

#69     OperationPerformanceImpl.runAsync (package:analyzer/src/util/performance/operation_performance.dart:174:14)

#70     AnalysisDriver._analyzeFile (package:analyzer/src/dart/analysis/driver.dart:1305:5)

#71     AnalysisDriver._getErrors (package:analyzer/src/dart/analysis/driver.dart:1739:5)

#72     AnalysisDriver.performWork (package:analyzer/src/dart/analysis/driver.dart:1207:7)

#73     AnalysisDriverScheduler._run (package:analyzer/src/dart/analysis/driver.dart:2449:7)

```

When trying to pin down the problematic file(s) by running folder-by-folder, the error stopped occurring after processing a folder's subdirs (upon whose analysis the exception occurred) individually. None raised the error when analyzed by itself, after which the parent ran just fine as well.
Request: It would be super helpful to include the currently processed file's path whenever an exception is thrown.

Update: I can reproduce the exception on the very fist run of dart fix --apply after running pub get. If there were no fixes applied and I don't touch any files, the error does not appear again on subsequent runs.

Great request! Might be tricky to implement, as we cannot collect any personally identifiable info, including file paths, in our crash reporting. A very good feature to have though.

authors: Lootwig srawlins


text:
Request to host package: Lawndart

This issue was originally filed by @sethladd

When you have a moment, please host Lawndart on pub.
https://github.com/sethladd/lawndart
Lawndart is an attempt to bridge the different persistence layers in the browser under one common API. I've received two pull requests, so there are at least a few people using it :)
Thanks!

This issue has been moved to dart-lang/pub#228.


text:
Request to host package - crc32

This issue was originally filed by kaisellgren@gmail.com

It has been updated.
http://github.com/kaisellgren/CRC32
http://pub.dartlang.org/packages/crc32

This issue has been moved to dart-lang/pub#363.


text:
Add shell script to SDK that runs Dartium in checked mode

New developers are confused by how to run Dartium in checked mode during development, especially if they aren't using Editor.
Suggest we include a way to run Dartium in checked mode via the SDK, perhaps through a shell script. I can imagine two scripts: dartium_developer_mode.sh and dartium_production_mode.sh

We don't care about Dartium any more...

authors: kevmoo sethladd


text:
[Question] Refused to apply style.

The html loads with no styles. and in inspection console it shows this
Refused to apply style from 'http://localhost:1031/assets/style.css' because its MIME type ('text/plain') is not a supported stylesheet MIME type, and strict MIME checking is enabled.

I found no solution and any resource. Hoping for the solution. as of the createStaticFileHandler function says MimeTypeResolver, I really don't understand it. please explain me(us in Docs too) about this.
api.dart
Future<void> main() async {
  /// Router object
  Router app = Router();
  app.get('/assets/<file | .*>',
      createStaticHandler('public', serveFilesOutsidePath: true));
  app.get('/', (Request request, String name) {
    console.log(name);
    String indexFile = File('public/index.html').readAsStringSync();
    return Response.ok(
      indexFile,
      headers: <String, Object>{'Content-Type': 'text/html'},
    );
  });

  /// Serving
  await shelf_io.serve(app, 'localhost', 1031).then((HttpServer value) =>
      console.log(
          'API running at http://${value.address.host}:${value.port} 🔥🔥🔥'));
}

index.html
<!DOCTYPE html>

    <meta charset="UTF-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>FlutterMatic</title>
    <link rel="stylesheet" type="text/css" href="./assets/styles.css" />

Hello world

This is how I'm using it.

The css extension is wired up correctly in the mime type resolver. I tested it.
Not sure what's going on.

license_source: bigquery


text:
Updates

Look okay, @tvolkert ?

LGTM

source: gharchive/pull-request

authors: kevmoo tvolkert

repo: dart-lang/template-binding


text:
Smarty adapter extraction

Extract SmartyView into its own repository, along with Smarty\ViewResolver and Smarty\Resource from the CMS project.

SmartyView was removed in c6e040b6.

repo: darya/framework

license: mit


text:
Cannot find knora-base:valueHasComment in the documentation

By reviewing knora-app mock-up, I just came across a value property that I didn't know existed and that I can't find in the documentation. It is knora-base:valueHasComment.
I also wonder what is the purpose and the use expected for this comment, and how dependant is this property of the value.
For instance, if a user A create a value, can a user B post a comment on this value? Can a user C delete the comment?

You're right, this should be documented but it isn't. It could be used to add an explanation of why the value was created or modified.
To add a comment to an existing value, you have to make a new version of the value, so you need modify permission on the value.

To change or remove a comment, you also need modify permission on the value.

Great. Thank you for the explanation.

knora-api:valueHasComment has been added to the api documentation, please see the part about creating a new value

authors: SepidehAlassi benjamingeer mrivoal

repo: dasch-swiss/knora-api


text:
Misc doc fixes

List of changes:

add missing dates to changelog
add missing full license to project
skip undefined reference warnings for changelog
update version in readme
autolink license term

Nice, thank you @kianmeng 👏

:partying_face: :partying_face: :partying_face: :partying_face: :partying_face:

authors: kianmeng whatyouhide


text:
New amount text input box issues

When a leading decimal exists (e.g. ".01") I want it to automatically change it to "0.01".
I like that the validation disallows numbers without a leading decimal if we add the above fix.
The validation allows 9 post-decimal values; it should only allow 8.

Should be fixed by 6aa694bb8f058c427eaf9fe76eaa3207dc1a5002

authors: jojobyte riongull

repo: dashhive/wallet-ui

license: MIT


text:
Dash Core does not build on Amazon AMI: libdb_cxx headers missing, Dash Core requires this library for wallet functionality

Hello guys,
I am trying to build Dash Core master (commit 86525601d5915f380976c9d2e686ad7f66db991f) on Amazon AMI (based on CentOS), but I get this error:
configure: error: libdb_cxx headers missing, Dash Core requires this library for wallet functionality (--disable-wallet to disable wallet functionality)
Note that I have already installed all development tools:
sudo yum groupinstall "Development Tools"
I see that db4 is actually installed and the headers are available in /usr/include/db4 , but for some reason the configure script does not search there.
(I've had a very quick look inside configure.ac and I see that it is searching inside /opt/local/include/db48 but just for Mac/Brew)
Is CentOS/AWS not a supported build platform for DashCore?

Actually I get a similar error on AWS with Ubuntu 16.04 image.
It seems I need to manually compile Berkeley DB 4.8 from source, as shown in this post: http://bitzuma.com/posts/compile-bitcoin-core-from-source-on-ubuntu/
But I find this is not ok that a simple clean build does not work by default. This dependency should be documented in https://github.com/dashpay/dash/blob/master/INSTALL
Also now I get another error, which I could not yet fix:
configure: error: pkg-config not found.

Here is the list of commands that I executed on a fresh Ubuntu 16.04 image on AWS:
sudo apt update sudo apt install -y git build-essential wget http://download.oracle.com/berkeley-db/db-4.8.30.NC.tar.gz tar -xvf db-4.8.30.NC.tar.gz cd db-4.8.30.NC/build_unix/ mkdir -p build BDB_PREFIX=$(pwd)/build ../dist/configure --disable-shared --enable-cxx --with-pic --prefix=$BDB_PREFIX make install cd ../.. sudo apt-get install -y autoconf libboost-all-dev libssl-dev libprotobuf-dev protobuf-compiler libqt4-dev libqrencode-dev libtool pkg-config git clone https://github.com/dashpay/dash.git cd dash/ ./autogen.sh  ./configure CPPFLAGS="-I${BDB_PREFIX}/include/ -O2" LDFLAGS="-L${BDB_PREFIX}/lib/" --with-gui
And this is the new error that I get now:
checking whether to build Dash Core GUI... yes (Qt) checking for boostlib >= 1.20.0... yes checking whether the Boost::System library is available... yes checking for exit in -lboost_system... yes checking whether the Boost::Filesystem library is available... yes checking for exit in -lboost_filesystem... yes checking whether the Boost::Program_Options library is available... yes checking for exit in -lboost_program_options... yes checking whether the Boost::Thread library is available... yes checking for exit in -lboost_thread... yes checking whether the Boost::Chrono library is available... yes checking for exit in -lboost_chrono... yes checking whether the Boost::Unit_Test_Framework library is available... yes checking for dynamic linked boost test... yes configure: error: pkg-config not found.

Dash definitely builds on Ubuntu 16.04 if you first install all the dependencies that are listed in doc/build-unix.md.
Note especially these dependencies:
    sudo add-apt-repository ppa:bitcoin/bitcoin     sudo apt-get update     sudo apt-get install libdb4.8-dev libdb4.8++-dev

You're right, on Ubuntu it works with these commands:
sudo apt update
sudo apt install -y build-essential libtool autotools-dev automake pkg-config libssl-dev libevent-dev bsdmainutils
sudo apt install -y libboost-all-dev
sudo add-apt-repository ppa:bitcoin/bitcoin
sudo apt update
sudo apt install -y libdb4.8-dev libdb4.8++-dev
sudo apt install -y libqt5gui5 libqt5core5a libqt5dbus5 qttools5-dev qttools5-dev-tools libprotobuf-dev protobuf-compiler
sudo apt install -y libqrencode-dev
git clone https://github.com/dashpay/dash.git
cd dash/
./autogen.sh
./configure --with-gui
make
sudo make install

I managed to also build it on Amazon Linux AMI (based on CentOS), but without the GUI, by running these commands:
sudo yum update
sudo yum groupinstall -y "Development Tools"
wget http://download.oracle.com/berkeley-db/db-4.8.30.NC.tar.gz
tar -xvf db-4.8.30.NC.tar.gz
cd db-4.8.30.NC/build_unix/
mkdir -p build
BDB_PREFIX=$(pwd)/build
../dist/configure --disable-shared --enable-cxx --with-pic --prefix=$BDB_PREFIX
make
make install
cd ../..
sudo yum install -y boost-devel openssl-devel libevent-devel
git clone https://github.com/dashpay/dash.git
cd dash/
./autogen.sh
./configure CPPFLAGS="-I${BDB_PREFIX}/include/ -O2" LDFLAGS="-L${BDB_PREFIX}/lib/" --without-gui
make
sudo make install

@sorin-postelnicu Glad it's working for you now.
Since it's working, can this issue be closed?

Dear Nathan, I suppose that it can be closed.
Nevertheless, it would be good if the main INSTALL readme file would contain a reference to the doc/build-unix.md which you mentioned, otherwise any newcomer is confused why the INSTALL file says that it can be built with just 3 commands, when in reality it can only be built after manually installing a lot of dependencies which do not come by default with the Linux distributions.
I think I will create a pull request with some clarifications for the newcomers.

authors: nmarley sorin-postelnicu tgflynn

repo: dashpay/dash


text:
Implement RemoveAskFor to indicate that we're not interested in an item anymore

When an INV item is received from the first node, the item is requested
immediately. If the same item is received from another node, an entry is
added to mapAskFor which marks the item for re-requesting in case the first
node did not respond. When the item is received from the first node,
the item was previously never removed from mapAskFor. Only the later getdata
loop in SendMessages would then gradually remove items from the map. This
is quite delayed however as the entries in mapAskFor have a timeout value.
RemoveAskFor allows to remove all entries from mapAskFor and setAskFor
when we are not interested in the item anymore (e.g. because we received
it already).

Forgot to describe the reason I implemented this in the first place:
Very often when doing some tests with debug=1, I noticed hundreds of message like:
2014-12-04 17:20:30 SendMessages -- GETDATA -- already have inv = mnw f7f5f6b111c845b8598e10989d87ac36597d1c11bca317ba0bbd768ba0ce722b peer=9
2014-12-04 17:20:30 SendMessages -- GETDATA -- already have inv = mnw aa64a0d21f9e7e2d24bb8d867cf3ec3c2d48fe02989c3a32b5254503e43ad561 peer=9
2014-12-04 17:20:30 SendMessages -- GETDATA -- already have inv = mnw ee8aa773c15727792a90be2c67f72701c8980e931f863234d18bd818f08fcf60 peer=10
2014-12-04 17:20:30 SendMessages -- GETDATA -- already have inv = mnw a8feea15102d83e892b4aee05ab2abd44a212796f2b7d02cc796b01a04a95c3d peer=10
2014-12-04 17:20:30 SendMessages -- GETDATA -- already have inv = mnw 4a5e52b8d163cde39146e4c25cd8de2db1b3ab0dcbd1c61d4a7ddc62c2fc9510 peer=10
2014-12-04 17:20:30 SendMessages -- GETDATA -- already have inv = mnw c574704f33891f3f7e23c21118b6a43ddc8d9cd3e2bdc7d4f80301d51baa677e peer=10
2014-12-04 17:20:30 SendMessages -- GETDATA -- already have inv = mnw 5a36b789b4ad1bf452829485b1be10f172ae38541a6a8b9a82f822130fbe982d peer=10
2014-12-04 17:20:30 SendMessages -- GETDATA -- already have inv = mnw 7433f4519cdfeb6c51963aaa238e2d928ce6f046e0b44eb716f05bebaf2807d4 peer=10
2014-12-04 17:20:30 SendMessages -- GETDATA -- already have inv = mnw 38ec50d06fff7577cd610db8319ab10cd7229dc463a3e639a398b2fbae542664 peer=10
2014-12-04 17:20:30 SendMessages -- GETDATA -- already have inv = mnw 267ada324fecdac9abdf253fcb98196eb2eccc14dcabdb53454c2af294d4656e peer=10

I always found these very confusing because it looks like the node is trying to re-request some inventory items or that it received duplicated INV items long after the message was received. Mostly noticed this my LLMQ branch where a lot of message are exchanged in intra-quorum communication, but now also observed this in the auto IX test cases, so I decided to backport this commit from my local LLMQ branch.

Looks good in general but why constructing and passing a CInv object each time even though only a hash of it is used later in code?

@UdjinM6 True, didn't make sense. Using uint256 now.


text:
[BUG] testcase faliure - gen_cluster() got an unexpected keyword argument 'check_new_threads'

Describe the bug
pytest for the tests having call to gen_cluster() fails with  -
E   TypeError: gen_cluster() got an unexpected keyword argument 'check_new_threads'

Steps/Code to reproduce bug
This can be easily reproduced with the distributed version 2.3.2.
Here is the complete error seen on the terminal -
$ pytest test_core.py
====================================== test session starts =======================================
platform linux -- Python 3.6.9, pytest-5.0.1, py-1.8.0, pluggy-0.12.0
rootdir: /home/sangeek/dask-xgb-tests/tests
plugins: cov-2.7.1, xdist-1.28.0, forked-1.0.2
collected 0 items / 1 errors

============================================= ERRORS =============================================
_________________________________ ERROR collecting test_core.py __________________________________
test_core.py:136: in <module>
    @gen_cluster(client=True, timeout=None, check_new_threads=False)
E   TypeError: gen_cluster() got an unexpected keyword argument 'check_new_threads'
!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!
==================================== 1 error in 3.70 seconds =====================================

Additional context
Looking at the code for the latest release of distributed -
https://github.com/dask/distributed/blob/a0d68d42b81fe0215098d7f87783e2d57b2aa703/distributed/utils_test.py#L831
I see that they have removed support for the parameter check_new_threads.
I see that removing that parameter from the call to gen_cluster(), helps me get past this issue i.e.
< @gen_cluster(client=True, timeout=None, check_new_threads=False)
---
> @gen_cluster(client=True, timeout=None)

I could have attempted to patch this change but for the reasons mentioned in https://github.com/dask/dask-xgboost/issues/47 I can't still run all the tests successfully, so not sure if there are any other side-effects of this change?

This should either be removed (if that passes) or replaced with clean_kwargs={"threads": False}

authors: ksangeek mrocklin

repo: dask/dask-xgboost


text:
Querying number of threads and/or processes used from dask.config

Would be nice if there was a way to inspect dask.config and get the number of threads being used and the number of processes used (likely as to different values). The latter is particularly helpful when trying to determine if pickling comes into the picture.

I agree that this would be a useful thing.  I don't have a good understanding of how to implement it reliably across all scheduler.
In either case, I think that this is possibly outside of the scope of the dask.config module. (though that's a conversation for after someone actually finds a way to reliably implement this)

Is it possible to have schedulers derive from a base class with some abstract methods for these checks or would that be too difficult?

It's worth investigating.  This is part of a much larger question though of
how to define requirements for a scheduler in the future.  Currently all we
demand is the get function.
On Wed, Jun 13, 2018 at 1:08 PM, jakirkham notifications@github.com wrote:

Is it possible to have schedulers derive from a base class with some
abstract methods for these checks or would that be too difficult?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/dask/dask/issues/3608#issuecomment-397014179, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AASszNAPibISivARZxXdUdoRymJwfVxGks5t8UcRgaJpZM4Umk-8
.

authors: jakirkham mrocklin

repo: dask/dask


text:
how the dask vstack and concatenate works, as it is making problems?

suppose, I have this code,
import dask.array as da
models= da.random.random((2000000,256),chunks=(1000000,256))
models

when I run this, it gives me
               Array	Chunk
Bytes	4.10 GB	2.05 GB
Shape	(2000000, 256)	(1000000, 256)
Count	2 Tasks	2 Chunks
Type	float64	numpy.ndarray

I then again tried this
models = da.vstack([models, [np.random.randn(256)]])
models

	           Array	Chunk
Bytes	4.10 GB	2.05 GB
Shape	(2000002, 256)	(1000000, 256)
Count	11 Tasks	4 Chunks
Type	float64	numpy.ndarray

It increases the chunks and tasks as well, ideally, it should have 2 Chunks and 3 task, any solution for this?

it should have 2 Chunks

Why do you expect two chunks in the final output?

and 3 task

You can inspect the dask graph
In [18]: list(dict(models.dask))
Out[18]:
[('concatenate-3b523d536dc1078bcf7ca5a19e963118', 0, 0),
 ('concatenate-3b523d536dc1078bcf7ca5a19e963118', 1, 0),
 ('concatenate-3b523d536dc1078bcf7ca5a19e963118', 2, 0),
 ('random_sample-646f791a8749774258483633e644db1a', 0, 0),
 ('random_sample-646f791a8749774258483633e644db1a', 1, 0),
 ('array-0e97eb0a572ea7a5e32f9b379c0c87ae', 0, 0)]

I'm not sure if that can be improved (I suspect not), but in general a single function call will not align with a single new task.

Number of Chunks we can control with rechunk(), but number of task, I don't know, how to controle, issue, suppose, I concatenate 1000 times, the number of task counts increase, and processing get slower and slower. Any solution i.e. that, number of tasks stay same?

Anything else to do here @rizwanishaq?
Going back to the original question of "how do these work?" you can see the source at, e.g. https://github.com/dask/dask/blob/0871699c0e4b021fca4a3104042d1adf7c5fd5d5/dask/array/core.py#L3353, but I'm not sure how helpful that will be.

I am experimenting with it, and you can see, I have example like,
models = np.random.randn(1000000,256), suppose, it doesn't fit in the memory, so I am using
models = da.random.randn((1000000,256), chunks=(10000,256), it is totally, ok, it works amazingly well, but my concern after that, I continually, updating this models, i.e. by
models = da.vstack([models, np.random.randn(256]).rechunk(1000000,256)
it also work, but it decreases the processing speed, may be due to number of tasks increasings.

Find the solution, by doing
models = da.vstack([models, np.random.randn(256]).rechunk(10000,256).persist()
Still need to have a look on the RAM usage, but, I suppose, it will persist on cluster RAM. And it is now faster a lot. Still make experiments...

Glad you found a solution that works for your use case @rizwanisha! Is there anything else to do here? Otherwise I'm inclined to close this issue

authors: TomAugspurger jrbourbeau rizwanishaq


text:
ERR: Better error message describe with non-numeric data

Closes #2326
The other option is to return an empty dataframe. I think most users would prefer an exception though.

Looks good to me. Thanks @TomAugspurger!

authors: TomAugspurger jcrist


text:
Adjust concatenate size 0 array case for meta

Instead of hand rolling our own Dask Array creation call, dispatch to the existing ones in wrap. Also leverage meta to create an array of the expected type instead of using a NumPy array using empty_like (when empty_like supports this).
cc @pentschev

[x] Tests added / passed
[x] Passes flake8 dask

Planning on merging tomorrow if no comments.

Would it make sense to add a small test here?

Sure. Have pushed a small test.

Please let me know if anything else is needed here.

Given the discussion above, sounds like we are good to merge? Is that correct?

I'd say so, unless @mrocklin has any objections.

Thanks all! 😄

authors: jakirkham mrocklin pentschev


text:
read() returns incomplete file? (not clear on documentation)

Hello,
This is my first time using the library, so please apologize me if I'm missing something obvious.
I'm trying to read a .csv.gz file (quite small, less than 100kb) following the documentation (https://gcsfs.readthedocs.io/en/latest/#examples), by doing:
fs = gcsfs.GCSFileSystem(project=project_id, token=service_key_path)
uri = bucket_name + '/' + filename
with fs.open(uri, 'rb') as f:
    content = f.read()

Here, I expect content to have the entire file, but it has only 168 rows (out of 946 in total). Reading the same file with google-cloud-storage works fine.
storage_client = storage.Client(credentials=credentials, project=project_id)
bucket = storage_client.bucket(bucket_name)
blob = bucket.blob(filename)
content = blob.download_as_string()

What am I missing?
Thanks

So, do we need to explicitly say that ContentEncoding=(compression) , i.e., auto-compression at rest, is not supported by gcsfs? The download itself is always compressed (gzip or deflate) regardless, but such files cannot be random-accessed.

authors: fernandobrito martindurant

repo: dask/gcsfs


text:
Add https://chaosengineering.news

This adds https://chaosengineering.news/ to the list of newsletters.

Thanks @seeker89! Feel free to send me a PR to https://github.com/dastergon/awesome-sre (in the Newsletter section) as well.

repo: dastergon/awesome-chaos-engineering


text:
End to end tests

Adds end-to-end tests for the Data Refinery. These run separately from all the other tests, using Docker Compose. Docker Compose runs 4 containers:

Workers (in normal mode, but with an environment of test)
Foreman (run via the test-runner)
Rabbit
Nginx (used to download data from within the workers)

The Foreman runs a survey job, which discovers a batch, queues a downloader job, and then waits for it to finish so it can discover its processor job, then waits for the processor job to finish. The workers run the downloader and processor jobs. The Foreman tests then check that SCAN.UPC outputting a processed file. More in depth testing of the contents of that file are upcoming.
I tested this without an internet connection to be sure that no external requests were actually being made. This will keep the tests from being finicky.

Ok @dongbohu I've made changes based on your feedback. One change that wasn't explicitly related to your suggestions was that when queuing a processor job I catch exceptions caused by the message queue not being available, and then delete the job record since it will never get picked up.
Take a look and let me know if there's anything else you think I need to address.


text:
add hook to list genomes by asset

once refgenieconf has list genomes by asset function, https://github.com/databio/refgenconf/issues/7
there should be an endpoint to access that.

looks like this can now be done with the latest refgenconf: https://github.com/databio/refgenconf/issues/7

it can be accessed now at: /genomes/{asset}
question: do we raise a 404 exception when there are no genomes with requested asset or return an empty list (which list_genomes_by_asset method does by default)?

do we raise a 404 exception when there are no genomes with requested asset or return an empty list

return empty list.

authors: michalstolarczyk nsheff

repo: databio/refgenies


text:
Added separate methods for required path and body parameters and added Must/CanUseJson methods

Changes
In attempt to simplify code generation logic (https://github.com/databricks/cli/pull/905) added separate methods for required path and body parameters and added Must/CanUseJson methods.
Tests

[x] make test passing
[x] make fmt applied
~[ ] relevant integration tests applied~

Codecov Report
Attention: 1027 lines in your changes are missing coverage. Please review.

Files
Coverage Δ

openapi/code/package.go
58.20% <100.00%> (ø)

service/compute/spark_version.go
0.00% <ø> (ø)

service/iam/model.go
0.00% <ø> (ø)

listing/listing.go
94.66% <94.66%> (ø)

openapi/code/method.go
33.33% <0.00%> (-1.20%)
:arrow_down:

openapi/code/entity.go
23.80% <0.00%> (-3.08%)
:arrow_down:

service/billing/api.go
0.00% <0.00%> (ø)

service/jobs/api.go
0.00% <0.00%> (ø)

service/pipelines/api.go
0.00% <0.00%> (ø)

service/settings/api.go
0.00% <0.00%> (ø)

... and 7 more

... and 1 file with indirect coverage changes
:loudspeaker: Thoughts on this report? Let us know!.

authors: andrewnester codecov-commenter

repo: databricks/databricks-sdk-go


text:
Query execution weird behaviour

Hello,
I'm experiencing a weird issue when running queries. I ran some delete queries and I checked the DB, and it behaved as expected. However, when I go into Azure, I see that the query is still running, and changes to complete, and after a long period, it times out due to inactivity,  then the status is revised as Failed.
The interesting thing is that the query does not roll back.

We've previously tried the ODBC driver and did not experience this issue.
Thank you

Hi @wale-kobai , thanks for filing this. Just to confirm a few things, can you post an example of how you're using the driver? Are you closing the connection / result set? Inactivity errors are mostly due to an open execution that has not been closed for more than 10 minutes.

@wale-kobai , I see. I'll try to reproduce your problem, meanwhile, can you defer db.Close()?

The func NewSql is called from the main method. We can add the defer after db := sql.OpenDB(connector). Or would you want us to open a connection per method call and close the connection after ?

Hi @andrefurlan-db, we updated the code as follows to make sure we close the connection that now we get from the db instance after every execution. We are still seeing the same behavior.
func (beSQL *BackendSQL) RunExec(query string) error {
	//utils.LogInfoMsg("[SQLExec] " + query)

	ctxRoot := dbsqlctx.NewContextWithCorrelationId(context.Background(), "Kobai RunExec")

	defer func() {
		if err := recover(); err != nil {
			fmt.Println("RunExec Panicked: ", err)
		}
	}()

        var err error
	var conn *sql.Conn
	if beSQL.Cfg.Top.Backend == "databricks" {
		conn, err = beSQL.Db.Conn(ctxRoot)
		if err == nil {
			defer conn.Close()
			_, err = conn.ExecContext(ctxRoot, query)
		}
	} else {
		_, err = beSQL.Db.Exec(query)
	}
	if err != nil {
		utils.LogError(err, "SQL Exec failed.")
		return err
	}

	return nil
}

Hi @wale-kobai and @ezbercihk! Sorry for the confusion. I just mean:
// ...
db := sql.OpenDB(connector)
defer db.Close()
// ...

The sql package will take care of the connection pool, so no need to close per connection. It is just good to make sure the pool gets drained at the end of Main().
I was unable to reproduce (maybe different DBR versions), but I'll push a fix anyway that in theory should help.

Hi @wale-kobai. @ezbercihk. We found the root cause. We addressed what we could here: https://github.com/databricks/databricks-sql-go/pull/65
The problem was that a query that returns rows was issued using ExecContext, and ExecContext was not preemptively closing the result set. Databricks will at first consider the query successful and wait until the client fetch all results. In this case, the client did not fetch, and then after 10 minutes or so the query would be marked as inactive and failed.
This can still happen if you use QueryContext, get rows, and do not close it until fetching all results. So please be careful with this scenario.
Please test and let us know if it is now working as expected.
Thanks for the help.

@andrefurlan-db thank you! We are running our tests and observing. We were not getting an end time for the queries in the Databricks UI but with this change we can see them populated now. We will update as we complete the end to end testing to make sure all is working as intended.

@andrefurlan-db we finished testing. Everything is looking great! Thank you for turning this around quickly! We really appreciate it.

authors: andrefurlan-db ezbercihk wale-kobai

repo: databricks/databricks-sql-go


text:
Add some more missing functions to DatatimeIndex.

Add some more missing functions to DatatimeIndex.

indexer_at_time
indexer_between_time

Codecov Report

Merging #2038 (15dd971) into master (afdf6b2) will decrease coverage by 4.79%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #2038      +/-   ##
==========================================
- Coverage   94.70%   89.90%   -4.80%
==========================================
  Files          54       54
  Lines       11478    11371     -107
==========================================
- Hits        10870    10223     -647
- Misses        608     1148     +540

Impacted Files
Coverage Δ

databricks/koalas/missing/indexes.py
100.00% <100.00%> (ø)

databricks/koalas/plot/plotly.py
9.47% <0.00%> (-87.37%)
:arrow_down:

...bricks/koalas/tests/plot/test_frame_plot_plotly.py
23.33% <0.00%> (-76.67%)
:arrow_down:

...ricks/koalas/tests/plot/test_series_plot_plotly.py
26.92% <0.00%> (-71.26%)
:arrow_down:

databricks/koalas/usage_logging/__init__.py
27.58% <0.00%> (-64.92%)
:arrow_down:

databricks/koalas/usage_logging/usage_logger.py
47.82% <0.00%> (-52.18%)
:arrow_down:

databricks/koalas/__init__.py
74.62% <0.00%> (-16.42%)
:arrow_down:

databricks/conftest.py
88.13% <0.00%> (-11.87%)
:arrow_down:

databricks/koalas/spark/functions.py
88.88% <0.00%> (-7.41%)
:arrow_down:

databricks/koalas/accessors.py
86.43% <0.00%> (-7.04%)
:arrow_down:

... and 23 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update afdf6b2...15dd971. Read the comment docs.

I'd just merge this.

Thanks!

authors: codecov-io ueshin xinrong-databricks

repo: databricks/koalas


text:
Do not sienlty produce null but skip the children XML elements by the pushed down projection

https://github.com/databricks/spark-xml/issues/185
This PR proposes to handle pushed down projection correctly via skipping the children XML elements.
PrunedScan was introduced in XML before and therefore we should be able to skip the children elements. Currently, it sometimes for parsing and skipping the elements of struct types when processing pushed down projection.

Current coverage is 87.81% (diff: 92.30%)

Merging #186 into master will decrease coverage by 0.04%

@@             master       #186   diff @@
==========================================
  Files            15         15
  Lines           708        722    +14
  Methods         641        630    -11
  Messages          0          0
  Branches         67         92    +25
==========================================
+ Hits            622        634    +12
- Misses           86         88     +2
  Partials          0          0

Powered by Codecov. Last update 8e15a09...e8cf320

I am going to merge this.

authors: HyukjinKwon codecov-io

repo: databricks/spark-xml


text:
Trends page

See mockup:

@louisdelanoeye had some feedback on this, would be nice if you can:

[ ] Display the actual score for the top 10 download and top 10 maintainers
[ ] Display a i like for downloads on package page on top of the table saying what do you actually take into account for the computation of the score.
[ ] Have a the page title in top of the table Leaderboard
[ ] Increase a bit the spacing between the graphs

You can do all of that in a new branch from dev I think it'll be easier

@louisdelanoeye Some improvments that can be made:

[ ] Cache the response for the graph using RedisService
[ ] Tooltip seems broken :(
[ ] For top maintainer, use total download instead of direct download and adapt the tooltip to reflect that.

repo: datacamp/rdocumentation-app


text:
BT Automation: Cleanup and improve error reporting

This change is

Thanks.

Thanks.

repo: datacommonsorg/tools


text:
Use DateTimeFormatter instead of SimpleDateFormat

In net.datafaker.service.FakeValuesServiceTest in tests every time it creates SimpleDateFormat instance... It is more effectively to use DateTimeFormatter which could be created once and reused

Codecov Report

Merging #939 (d11b96a) into main (a0d0c37) will decrease coverage by 0.24%.
The diff coverage is n/a.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
@@             Coverage Diff              @@
##               main     #939      +/-   ##
============================================
- Coverage     92.54%   92.31%   -0.24%
+ Complexity     2746     2740       -6
============================================
  Files           289      289
  Lines          5462     5462
  Branches        589      589
============================================
- Hits           5055     5042      -13
- Misses          260      269       +9
- Partials        147      151       +4

see 5 files with indirect coverage changes
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

thanks for taking a look

authors: codecov-commenter snuyanzin

repo: datafaker-net/datafaker


text:
Create an Activity Pub example

Context
https://github.com/orgs/datafoodconsortium/discussions/18
Acceptance criteria
We want to create some example notifications using the Activity Pub protocol, to better understand how activity pub could be used in our case.
Criteria:

we need to consider that users and platforms can be actors (some platforms have delegated rights from their users and can act on their behalf)
other?

We should list and describe use cases for which notifications are needed. Maybe a task for you @RachL and @RaggedStaff?

@lecoqlibre Ah I thought we would only consider stock movements as a first step? Otherwise, each field update can be notified (it's what we do with the refresh button currently).

authors: RachL lecoqlibre

repo: datafoodconsortium/prototype


text:
Make the description more meaningful for system route pass expiry

Label it as "[System Event] Expire route credits from [Original Amount] to [Final Amount] after x days for credits purchased on x with transaction ID x"
Now it does not really make sense.

I'm in favour of the last two approaches. Shall we close this issue and create new ones in frontend and server?

Erm what is the 2 approaches you mentioned?
On Fri, 17 Nov 2017 at 5:38 PM, LoneRifle notifications@github.com wrote:

I'm in favour of the last two approaches. Shall we close this issue and
create new ones in frontend and server?
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/datagovsg/beeline-admin/issues/305#issuecomment-345193279,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AVtU3_PAzwjREL5oodVX5EVqVUpMZdKAks5s3VQfgaJpZM4POMrW
.

The ones in your earlier comment..

Oh right. Yes, the last 2 approach is the best.
You can proceed to close this and open 2 different issues.
On Fri, 17 Nov 2017 at 5:42 PM, LoneRifle notifications@github.com wrote:

The ones in your earlier comment..
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/datagovsg/beeline-admin/issues/305#issuecomment-345194204,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AVtU3-AlsNnIeLNEncaEBdLf-zThybUPks5s3VT3gaJpZM4POMrW
.

Superseded by abovementioned issues

authors: LoneRifle chuasweechin

repo: datagovsg/beeline-admin


text:
Add PlatformInstance to SageMaker Source

Hello,
This is a feature request to add support of platformInstance in SageMaker importer
This is highly relevant for companies who have more then one AWS Account in same env (e.g. PROD) where they are hosting SageMaker models
MetaModel already contains platform instance information but SageMaker recipe does not support this
Ref. https://datahubproject.io/docs/generated/ingestion/sources/sagemaker
Ref. https://datahubproject.io/docs/platform-instances/

Hi @kirillmakhonin  this seems to be a feature request. We currently reserve git issues for bugs. If you’d like to submit this as a feature request, please use our portal here: https://feature-requests.datahubproject.io/

authors: kirillmakhonin laulpogan

repo: datahub-project/datahub


text:
fix(ingest/datahub): Use server side cursor instead of local one

It uses server-side cursor which is the recommended in this case -> https://docs.sqlalchemy.org/en/20/core/connections.html#using-server-side-cursors-a-k-a-stream-results
Checklist

[ ] The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
[ ] Links to related issues (if applicable)
[ ] Tests for the changes have been added/updated (if applicable)
[ ] Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
[ ] For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

Do we have tests for both mysql and postgres?


text:
1.1.2. Admin API for updating token invalidation

For details about the API refer to the SRS document (section 1.1.2)
The API should perform the following:

Create the information in PostgresQL
Broadcast this information over internal RabbitMQ

resolved through PR #177

authors: kailash swaminathanvasanth

repo: datakaveri/iudx-resource-server


text:
Cell Execution feature for Notebooks

I'm currently unable to run the cells in the Notebook, that is being rendered in the React Application.
Link to the code: https://github.com/Prahmodh-Raj1/Jupyter-UI-Test
Terminal Error (on clicking the Run button of the cell):
Requesting cell execution without any cell executor defined. Falling back to default execution.
Kindly help me resolve this issue. cc @echarles

Just tried and have a few questions/comments.

Which packager are you using ? (yarn, npm...)
You need to defined resolutions for yarn https://github.com/datalayer-examples/jupyter-react-cra-example/blob/5b3402da44d48ac27daeb3f9ed0bc5749f5c23a5/package.json#L41-L132

But it looks like you are building and running, so maybe just force reload your browser, it may help to solve the issue you reported.

I'm using npm to install the dependencies, and 'npm start' to start the cra application. @echarles

And is the current content of the App.tsx file enough to get the cells executing? Or should I need to add some other components too?
const App(()=>{

  const [tab, setTab] = useState(0);
  return (
    <>
      <Box>
        <Jupyter startDefaultKernel={true}>
              <Notebook
                path="Documents/Sample.ipynb"
                CellSidebar={CellSidebarRun}
              />
        </Jupyter>
      </Box>
    </>
  );
});

If you see the Notebook, that should be enough. Can you bumpt to 0.11.0? Any errors in the devtools console?

I tried updating to version 0.11.0, but i'm facing the following error when I execute the "npm install" command:
`
npm install
npm notice
npm notice New minor version of npm available! 10.5.2 -> 10.8.1
npm notice Changelog: https://github.com/npm/cli/releases/tag/v10.8.1
npm notice Run npm install -g npm@10.8.1 to update!
npm notice
npm ERR! code ERESOLVE
npm ERR! ERESOLVE could not resolve
npm ERR!
npm ERR! While resolving: react-redux@9.1.2
npm ERR! Found: @types/react@18.0.27
npm ERR! node_modules/@types/react
npm ERR!   dev @types/react@"18.0.27" from the root project
npm ERR!   peerOptional @types/react@"^18.0.0" from @primer/react@36.6.0
npm ERR!   node_modules/@datalayer/primer-addons/node_modules/@primer/react
npm ERR!     @primer/react@"36.6.0" from @datalayer/primer-addons@0.3.0
npm ERR!     node_modules/@datalayer/primer-addons
npm ERR!       @datalayer/primer-addons@"0.3.0" from the root project
npm ERR!       1 more (@datalayer/jupyter-react)
npm ERR!   9 more (@jupyterlab/apputils, @primer/react, ...)
npm ERR!
npm ERR! Could not resolve dependency:
npm ERR! peerOptional @types/react@"^18.2.25" from react-redux@9.1.2
npm ERR! node_modules/react-redux
npm ERR!   react-redux@"^9.1.2" from the root project
npm ERR!
npm ERR! Conflicting peer dependency: @types/react@18.3.3
npm ERR! node_modules/@types/react
npm ERR!   peerOptional @types/react@"^18.2.25" from react-redux@9.1.2
npm ERR!   node_modules/react-redux
npm ERR!     react-redux@"^9.1.2" from the root project
npm ERR!
npm ERR! Fix the upstream dependency conflict, or retry
npm ERR! this command with --force or --legacy-peer-deps
npm ERR! to accept an incorrect (and potentially broken) dependency resolution.
npm ERR!
npm ERR!
npm ERR! For a full report see:
npm ERR! /Users/xyz/.npm/_logs/2024-06-28T08_32_52_492Z-eresolve-report.txt

npm ERR! A complete log of this run can be found in: /Users/xyz/.npm/_logs/2024-06-28T08_32_52_492Z-debug-0.log

If you see the Notebook, that should be enough. Can you bumpt to 0.11.0? Any errors in the devtools console?

This is the error in the devtools console:
Requesting cell execution without any cell executor defined. Falling back to default execution.

The cell execution is still failing in spite of updating the application with the latest commit: 3e05847.
The devtools console still produces the same error:
Requesting cell execution without any cell executor defined. Falling back to default execution.
It would be great if you could please share another repo where the cell execution has been tested? @echarles


text:
Add metrics to server

Closes #44
Introduces metrics to the server along with a configurable self logging level. When that level is DEBUG then we'll sample metrics and log them every 5 minutes. These can then be charted on a Seq dashboard to get an idea of how the Gelf server is behaving. The sampling interval is configurable, but isn't surfaced through an environment variable or Seq setting yet.
We don't do any per-event debug logging so increasing the logging verbosity shouldn't have much impact on the runtime.

Metrics end up looking like this:
{"@t":"2019-05-09T03:44:48.518382Z","@l":"DEBUG","@mt":"collected GELF server metrics","process":{"msg":5},"receive":{"chunk":0,"msg_chunked":0,"msg_unchunked":5,"overflow_incomplete_chunks":0},"server":{"process_err":5,"process_ok":0,"receive_err":0,"receive_ok":5}}

We collect:

process.msg: the number of complete messages seen by the process that writes them to stdout
receive.chunk: the number of chunked message chunks seen
receive.msg_chunked: the number of complete chunked messages seen
receive.overflow_incomplete_chunks: the number of times we've overflowed the allowed number of incomplete messages and dropped them all
server.process_err: the number of messages that failed while formatting
server.process_ok: the number of messages that were formatted successfully
server.receive_err: the number of messages that failed while receiving from the UDP server
server.receive_ok: the number of messages that were received from the UDP server successfully

repo: datalust/sqelf


text:
Any tips for training the model??

I can't seem to get the model to train properly.  Short of adding the exact match, is there any suggested methods for adding an address to the labeled.xml so that it learns from that example and can use it in order similar cases?
The example I have is, I'm trying to train the model to understand that:
206 4430 CHATTERTON WAY should be OccupancyIdentifier, AddressNumber, StreetName, StreetNamePostType.  I've added dozens of variations and the model doesn't seem to learn. In fact, sometimes it gets worse  :-0
Thanks guys

That's a very unusual pattern that really differs from every other example in the training set. You have to provide a lot of example to overcome that.

Thanks for the quick response!
Is it really?  I thought standard format was:
suite#-street# streetname streettype
ie.  905-1 main st
or
street# streetname streettype Suite#
ie. 1 main st. Suite 905  / 1 main st. # 905
I tried adding about 40 examples with slight variations to the data and couldn't get it to work.  I'll can certainly add a ton more.  Should there be tons of variation between each example or only slight variations between fields?  Or for the field that it's getting wrong, should that be the only thing that changes and everything else is the same?
Big thanks btw for this awesome project.

no, this is a very unusual pattern for the united states.
The typical pattern is streetnumber streetname street type occupancy identifer ...

fair enough.  I've added over 3500 pieces of training data to teach the model this pattern and doesn't seem to take.  Anything you can suggest?

remove the existing training data.
On Fri, Feb 9, 2018 at 5:30 PM, bobberino1 notifications@github.com wrote:

fair enough. I've added over 3500 pieces of training data to teach the
model this pattern and doesn't seem to take. Anything you can suggest?
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/datamade/usaddress/issues/214#issuecomment-364599136,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAgxbYYhtdCPL6KUqydySKMY1yxfXT67ks5tTNT-gaJpZM4SAfY_
.

--
773.888.2718

repo: datamade/usaddress


text:
🛑 GST Login Page is down

In cb637bf, GST Login Page (https://services.gst.gov.in/services/login) was down:

HTTP code: 503
Response time: 1802 ms

Resolved: GST Login Page is back up in 622651d after 5 hours, 1 minute.

repo: datameet/upptime


text:
🛑 Aadhaar Login is down

In 1eb3952, Aadhaar Login (https://ssup.uidai.gov.in/ssup/login.html) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Aadhaar Login is back up in 8c96672.


text:
🛑 Aadhaar Login is down

In d1e1f89, Aadhaar Login (https://ssup.uidai.gov.in/ssup/login.html) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Aadhaar Login is back up in 130c9b0.


text:
Kafka-connect-redis WorkerSinkTask loggings

Hello,
I'm writing a kafka=>redis sink task and receiving the following error
when i check the status of the running connector
org.apache.kafka.connect.errors.ConnectException: Exiting WorkerSinkTask due to unrecoverable exception.
at org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:406)
I am running kafka-connect within a docker container and it contains no further logs other then the connector was created.
I checked the source code
link noting at line 404 & 405 there should be error logs before the above error message was thrown. 404 containing the exception
For worker sink tasks is there an accessible log location to  view the real error

Hi @LukeFieldsend
Can you please post some information including:

Version of CP used
Version of Stream-Reactor used
Configuration / KCQL

Will need to understand which Redis-mode (Sorted-Set | CACHE | Multiple Sorted-Sets) u are using - and how you are configuring the partiticular connector.

hey @Antwnis
cp-kafka-connect:3.1.1
I got my steam-reactor redis module from https://github.com/datamountaineer/stream-reactor/releases/download/v0.2.3/kafka-connect-redis-0.2.3-3.1.1-all.tar.gz
and my config is as follows
#Connector name=redis-device-sink
connect.redis.connection.host=redis-1
connect.redis.connection.port=6379
connector.class=com.datamountaineer.streamreactor.connect.redis.sink.RedisSinkConnector
tasks.max=1
topics=DeviceInfoMigrationKafkaConnect_DeviceImports
connect.redis.sink.kcql=SELECT pkstringfieldA, stringfieldb, timestampfielda, timestampfieldb, stringfieldc, stringfieldd, stringfielde, intfielda, arrayoftimestamps, arrayofrecords FROM kafka topic PK pkstringfieldA(with suffix pre appended)
#task ids: 0

@LukeFieldsend
Can you confirm the kafka avro console consumer you can consume the records on the topic?
(with suffix pre appended)=> is this your own support for KCQL?
you can start connect in local mode in your terminal pointing out to your kafka cluster and set the debug level on log4j?

@LukeFieldsend
you can always take the code locally and wrap in a try..catch block the RedisSinkTask#put and catch throwable to log it.

Hey @stheppi
I JdbcSourceConnector source connector to import these entities from an msql instance. I use a view to prepare this data, i added an extra column duplicating the column i wish to use for pk with a string suffix. so this is done before the records enter kafka. I will try the above suggestions and get back to you
cheers

authors: Antwnis LukeFieldsend stheppi

repo: datamountaineer/stream-reactor


text:
StatementFactory: Detect cached faulted Task and attempt to re-prepare + cache key fixes

Fixes: CSHARP-660 and CSHARP-667.
Check for cached faulted Task and attempt to re-prepare on the StatementFactory.
Additionally, use a composed cache key,

cc @landim

Thanks for the review @landim

repo: datastax/csharp-driver


text:
[Improvement] validate spark connector on spark 3.1.2

What would you like to be improved?
validate spark connector on spark 3.1.2
How should we improve?
No response

spark connector couldn't compile or run on spark 3.1

the minimum spark version of the Kyuubi spark hive connector is 3.3,  it couldn't compile or start up in spark 3.1 envriment. this's the biggest challenge.
spark connector compile failed for API change, this could be resolved by some methods.
a. create table API change
b. not using SortedBucketTransform to represent the bucket table.
c. not support UpdateColumnDefaultValue
d. dropNamespace API change
e. deleteColumn API change
f. doesn't support timestampNTZ data type.
f. doesn't support FunctionCatalog, this mainly affect Iceberg catalog.

repo: datastrato/gravitino


text:
[Bug] [datavines-connector] trino 密码连接错误

Search before asking

[X] I had searched in the issues and found no similar issues.

What happened
trino 使用 443 （也就是 认证方式TLS） 连接时，出现报错： Authentication failed: Access Denied: Invalid credentials 错误 ，经排查发现与trino connector 代码有关：
代码文件： datavines-connector/datavines-connector-plugins/datavines-connector-trino/src/main/java/io/datavines/connector/plugin/TrinoConnector.java
原代码：   设置密码有问题
if (StringUtils.isNotEmpty(dataSourceInfo.getPassword())) {
properties.setProperty(PASSWORD, dataSourceInfo.getUser());
}
需更改为：
if (StringUtils.isNotEmpty(dataSourceInfo.getPassword())) {
properties.setProperty(PASSWORD, dataSourceInfo.getPassword());
}
DataVines Version
无
DataVines Config
无

Running Command
无

Error Exception
无

Engine Type
No response
Java Version
No response
Screenshots
No response
Are you willing to submit PR?

[X] Yes I am willing to submit a PR!

原代码： 设置密码有问题
if (StringUtils.isNotEmpty(dataSourceInfo.getPassword())) {
properties.setProperty(PASSWORD, dataSourceInfo.getUser());
}
需更改为：
if (StringUtils.isNotEmpty(dataSourceInfo.getPassword())) {
properties.setProperty(PASSWORD, dataSourceInfo.getPassword());
}

repo: datavane/datavines


text:
Latest version of ofscraper, 2FA authentication, manual input of auth.json, OF kicks out

Describe the bug
I've updated session+auth_uid fields when needed. I use 2FA, so far, auth.json updating was successful.  Now I do not see any mistakes how I update the fields.
To Reproduce
Expected behavior
A clear and concise description of what you expected to happen.
Screenshots/Logs
https://privatebin.io/?602dfaaef81fe21f#CPxe7irvTZy8gP5D9QacvqXu7D91ZRvxJxe9YHDCCq4d
Config
{
"main_profile": "main_profile",
"metadata": "{configpath}/{profile}/.data/{model_username}_{model_id}",
"discord": "",
"file_options": {
"save_location": "/home/mkw/kookas/pron/ofscraper",
"dir_format": "{model_username}/{responsetype}/{mediatype}/",
"file_format": "{filename}.{ext}",
"textlength": 0,
"space_replacer": " ",
"date": "DD-MM-YYYY",
"text_type_default": "letter",
"truncation_default": true
},
"download_options": {
"file_size_max": 0,
"file_size_min": 0,
"filter": [
"Images",
"Audios",
"Videos"
],
"auto_resume": true,
"system_free_min": 0,
"max_post_count": 0
},
"binary_options": {
"mp4decrypt": "/home/mkw/.config/ofscraper/bin/mp4decrypt",
"ffmpeg": "/home/mkw/.config/ofscraper/bin/ffmpeg"
},
"cdm_options": {
"private-key": null,
"client-id": null,
"key-mode-default": "keydb",
"keydb_api": "517453862d4256fd8bad4f58422004d7:5b4b848eac0855d79165d05f3cf16d56"
},
"performance_options": {
"download-sem": 6,
"threads": 2
},
"advanced_options": {
"code-execution": false,
"dynamic-mode-default": "deviint",
"backend": "aio",
"downloadbars": false,
"cache-mode": "sqlite",
"appendlog": true,
"custom_values": null,
"sanitize_text": false,
"temp_dir": null,
"remove_hash_match": false,
"infinite_loop_action_mode": false,
"post_download_script": null,
"disable_auto_after": false,
"default_user_list": "main",
"default_black_list": ""
},
"responsetype": {
"timeline": "Posts",
"message": "Messages",
"archived": "Archived",
"paid": "Messages",
"stories": "Stories",
"highlights": "Stories",
"profile": "Profile",
"pinned": "Posts"
},
"overwrites": {
"audios": {},
"videos": {},
"images": {},
"text": {}
}
}
System Info

OS: Linux 6.9.3-zen1-1-zen
Browser vivaldi
Version6.7.3329.39 (Stable channel) stable (64-bit)
python
python version  installed via pip
OF-Scraper, version 3.10.1

Additional context
Add any other context about the problem here.

https://gist.github.com/datawhores/8e8be131fb1b190b50c5324ab2ca5f89

https://gist.github.com/datawhores/8e8be131fb1b190b50c5324ab2ca5f89

change to grovik did not help
About "sneaky" guide option 2 I do not even know what is the correct way to change config

Please state what version of the script your using
and make this update in the pinned post for auth issues

Why this is closed?

"make this update in the pinned post for auth issues"
Called auth fix
This is the last post I will make in this thread

authors: cjb900 datawhores finnishbroccoli

repo: datawhores/OF-Scraper


text:
Return all commits if no target is specified, fixes #25

returns all comments if no target is specified, i.e. on /comments/[get|feed]
new: limit amount of returned comments to value specified in config

Thanks!
I have only taken a quick look over the code so far, but I am not too happy about the global limit. I agree that returning all comments brings up this issue which we have so far simply ignored as we don't get tooooo many comments.
While implementing this global limit on the all requests may be OK, that's not something we can do for regular requests with a target. It isn't too far-fetched to imagine a blog post with more than 100 comments (I know the limit can be configured but still) and we shouldn't simply truncate the last ones.
The clean solution would be to implement pagination or offsets. Not sure if you're up for that. Otherwise, we'll have to think of something else.

Thanks!
I have only taken a quick look over the code so far, but I am not too happy about the global limit. I agree that returning all comments brings up this issue which we have so far simply ignored as we don't get tooooo many comments.
While implementing this global limit on the all requests may be OK, that's not something we can do for regular requests with a target. It isn't too far-fetched to imagine a blog post with more than 100 comments (I know the limit can be configured but still) and we shouldn't simply truncate the last ones.
The clean solution would be to implement pagination or offsets. Not sure if you're up for that. Otherwise, we'll have to think of something else.

authors: baltpeter mal-tee

repo: datenanfragen/backend


text:
Java 11 compatibility => Error de.oc.jasper.ReportUtilities.(ReportUtilities.java:32)

Hi,
I have followed the instuction, but always get this error when click button "Test Report".
Could you please help me ?
HTTP Status 500 - Servlet execution threw an exception
type Exception report
message Servlet execution threw an exception
description The server encountered an internal error that prevented it from fulfilling this request.
exception
javax.servlet.ServletException: Servlet execution threw an exception
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
root cause
java.lang.ExceptionInInitializerError
de.oc.integration.jasper.webapp.ReportWrapper.service(ReportWrapper.java:182)
javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
root cause
java.lang.UnsupportedOperationException: No class provided, and an appropriate one cannot be found.
org.apache.logging.log4j.LogManager.callerClass(LogManager.java:573)
org.apache.logging.log4j.LogManager.getLogger(LogManager.java:598)
org.apache.logging.log4j.LogManager.getLogger(LogManager.java:585)
de.oc.jasper.ReportUtilities.(ReportUtilities.java:32)
de.oc.integration.jasper.webapp.ReportWrapper.service(ReportWrapper.java:182)
javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
note The full stack trace of the root cause is available in the Apache Tomcat/7.0.76 logs.

Please specify:

JRI version
Tomcat version
OS version
Java version used by Tomcat (you can see it on the JRI homepage, either http://server:port/jri or http://server:port/JasperReportsIntegration, depending on your JRI version

Hello,
Here are the information :
JRI version : jri-2.8.1-jasper-6.18.1
Configuration Info
The current configuration is loaded from: /usr/share/tomcat/webapps/JasperReportsIntegration
Config-Directory: /usr/share/tomcat/webapps/JasperReportsIntegration/conf
Config-File: /usr/share/tomcat/webapps/JasperReportsIntegration/conf/application.properties
Reports-Path: /usr/share/tomcat/webapps/JasperReportsIntegration/reports,/usr/share/tomcat/webapps/JasperReportsIntegration/reports/tmp
Temp-Directory: /usr/share/tomcat/webapps/jri/report_tmp
Logs-Directory: /usr/share/tomcat/webapps/JasperReportsIntegration/logs
Java-Runtime-Version: 11.0.14.1
JasperReports-Version: 6.18.1-a122da513af8382c12c1df5654d1048ef99e378e
Classpath: /usr/share/tomcat/bin/bootstrap.jar
/usr/share/tomcat/bin/tomcat-juli.jar
/usr/share/java/commons-daemon.jar
Tomcat version : tomcat7
Java version :
openjdk version "11.0.14.1" 2022-02-08 LTS
OpenJDK Runtime Environment Corretto-11.0.14.10.1 (build 11.0.14.1+10-LTS)
OpenJDK 64-Bit Server VM Corretto-11.0.14.10.1 (build 11.0.14.1+10-LTS, mixed mode)
OS version :
NAME="Amazon Linux"
VERSION="2"
ID="amzn"
ID_LIKE="centos rhel fedora"
VERSION_ID="2"
PRETTY_NAME="Amazon Linux 2"
Thank you

Thank you for raising this. There seems to be an issue with log4j and Java 11. I will need some more time to fix this: https://stackoverflow.com/questions/53049346/is-log4j2-compatible-with-java-11
In the meantime, please use a Java environment below Java 11 like Java 8, 9 or 10. Java 8 will definitely work (I develop/test on this), the others might as well.

LogManager.getLogger() doesn't work any more in JDK11. Specifying the class explicitly is mandatory.

authors: daust desypus

repo: daust/JasperReportsIntegration


text:
Support copying shapes from one vsdx file to another

if a shape has master_page_ID, which cannot be found in the current vsdx file, the shapes' text method tries to access its master's text without actually  checking if it has an accessable master.
Breaks here: https://github.com/dave-howard/vsdx/blob/ff82f8fb095f53436b2f63cc80bfb004d18c8d3c/vsdx/shapes.py#L563
Easiest fix is probably to rather check is master_shape is None.
I have not yet found out why this happens, but it seems to be happening when pasting nested shapes in from other files.

Thank you @MariusSchiffer
I'll add a simple check that shape has a master_shape here to stop the failure - but when copying a shape between files the tool would ideally also need to either include the master data with the copied shape and merge with the existing master data, or incorportate it into the shape itself (similar challenge internally to creating connectors in a vsdx with existing master data - which I am working on in #10 ).
Copying shapes from one vsdx file to another is not a use case this package currentl;y supports - depending on what you want to acheieve you might be able to acheive same by creating a template vsdx with the content you want and removing pages from the final output file (though see #42 I need to address also).

@MariusSchiffer  - minor change added to v0.5.6 to validate the shape has a master shape - so now text peroperty returns None if no master shape found.

Renamed this issue to reflect reason I have left open - to support that use case

authors: MariusSchiffer dave-howard

repo: dave-howard/vsdx


text:
Bug and improvements

1.
// #swagger.auto = false
Improvement. Keep the automatic path recognition when the // #swagger.auto = false
2.
This case don't generate the schema for OpenAPI 3:
app.get('/path', (req, res) => {

        /*  #swagger.parameters['obj'] = {
                in: 'query',
                description: 'Add a user',
                schema: { $ref: '#/definitions/AddUser' }
        } */

        return res.status(200)

})

3.
Add in documentation: anyOf and oneOf (OpenAPI 3.x)

The version 2.12.4 has implemented this.

repo: davibaltar/swagger-autogen


text:
Add css and png files to bower.json

Nice job!
please add Leaflet.NavBar.css and image files to bower.json. These are necessary for proper asset loading.

Thanks!
Bower doesn't really have a spec for adding assets to bower.json to my knowledge...can you point me to an example of what you need?

I mean adding these files to main array in bower.json

Hi - thanks for the nice work David!
I'm using this from vue/webpack and have it working with:
import "leaflet-navbar"
import "leaflet-navbar/Leaflet.NavBar.css"

It functions, navigating around the map - but the icons are missing...
So I'm not sure if there is a way to get images accessible to the plugin correctly with a
require('leaflet-navbar/img/arrow-left_000000_14.png')  //for example
for each of the six images...
But it doesn't really look set up for that - because of the direct references in the css...
Any ideas?

authors: chrisamow davidchouse mahmoudmy


text:
question 3D mesh CSG (boolean operations)

Hello,
Does your library allow to do simple boolean operations (intersection, substraction, union) on two 3D closed mesh of triangles ?
thank you... again!

Currently, I do not have a computational solid geometry (CSG) component in GTE. That is a nontrivial project. It is on my very-long-term TODO list. The closest I have to this is SplitMeshByPlane.h, which splits a mesh by a plane but it does not attempt to "close" the mesh halves. See Clipping a Mesh Against a Plane. A sample application is GeometricTools/GTE/Samples/Geometrics/SplitMeshByPlane which shows partitioning of a torus mesh by a plane.
CSG is difficult to make robust when using floating-point arithmetic, even if two solids are convex polyhedra. You have to compute using rational arithmetic. The ideas for Boolean operations are shown in Polysolids and Boolean Operations.

Thank you for your answer!
And thank you for this great library! 👍
Le lun. 14 août 2023 à 00:00, David Eberly @.***> a
écrit :

Currently, I do not have a computational solid geometry (CSG) component in
GTE. That is a nontrivial project. It is on my very-long-term TODO list.
The closest I have to this is SplitMeshByPlane.h, which splits a mesh by a
plane but it does not attempt to "close" the mesh halves. See Clipping a
Mesh Against a Plane
https://www.geometrictools.com/Documentation/ClipMesh.pdf. A sample
application is GeometricTools/GTE/Samples/Geometrics/SplitMeshByPlane which
shows partitioning of a torus mesh by a plane.
CSG is difficult to make robust when using floating-point arithmetic, even
if two solids are convex polyhedra. You have to compute using rational
arithmetic. The ideas for Boolean operations are shown in Polysolids and
Boolean Operations
https://www.geometrictools.com/Documentation/Polysolids.pdf.
—
Reply to this email directly, view it on GitHub
https://github.com/davideberly/GeometricTools/issues/67#issuecomment-1676473325,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AG3BAQLKHJAASOGIEQ23Q6DXVFFBNANCNFSM6AAAAAA3M4OJ6U
.
You are receiving this because you authored the thread.Message ID:
@.***>

repo: davideberly/GeometricTools


text:
Better Map() and Set() support

store.set = new Set();
const someObject = {prop: 'value'};
store.set.add(someObject);
someObject.prop = 'new value'; // Doesn't trigger a render

This is surprisingly complicated. If a user stores an item in a set, and passed that item to a component to render, then changes the content of that item, the item will be cloned (nothing is mutated). But everything in the store has a path, and for a Set, the value is the path. But the value changes! The issue is the same for Maps where the key is an object.
Also, a change to the key of a Map (assuming the key is an object) should update any component using that key. But the key is part of the path, and changing it will clone it, so the path breaks.
One option is to use indexes in prop paths, rather than the value. This is less readable to the user, but reading from the path is probably fairly rare.
Another (potential) option is to use a WeakMap to maintain a reference between an object and its clone.

Won't fix. The rules must be:

You can add/remove items from a set, but you can't modify the contents of a set entry.
You can have an object as a key for a map, but you can't modify that key.

Rationale:
We don't mutate the store, we always recursively shallow clone. For objects, arrays, and maps with primitive keys, we can have a consistent 'path' to every entry (via property name, array index, or map key). Cloning parts of the store doesn't break anything.
But the only way to reference an object in a set, or an entry in a map where the key is an object, is to already have a reference to that object. Trying to modify that object would clone it, breaking the reference. This means we can't know the 'path' to a set or map entry in the store, and therefore can't keep track of which components are subscribed to changes in that object.
We could use a mock 'index' to reference items in sets/maps, but that's going to add considerable complexity for what I'd consider a pretty fringe use case.

repo: davidgilbertson/react-recollect


text:
Request: console-only build

I only use the commandline tool, and don't need the GUI tool. The Makefile currently makes WXWidgets mandatory, and can't build if WX isn't present. I'm currently sticking with Fluxengine revisions I built from source in December, but it'd be great to be able to build CLI-only versions of the current software.
There's any interest in this, I'm glad to contribute patches to make it possible.

That's reasonable, but annoying to do. I actually want to rework the build system at some point --- again --- so let me see what I can do.

This should now work. Try make fluxengine (or fluxengine.exe on Windows).

Thanks! That did the trick - CLI tool builds fine again now.
It looks like the private directives you're using now aren't compatible with the older gmake that Apple ships. That's not a problem for me - I can easily install a newer one - but it may be worth documenting in the build instructions that you require something newer.

No wait, you already do. Ignore that, then! Thanks.

authors: davidgiven mistydemeo

repo: davidgiven/fluxengine


text:
Turkey pot pie missing break textures

using FD 0.5.1

Might be thanks to some refactors in FD's update. i have to update the gradle things for both to find out--thanks for lmk!

👍

i've also just noticed that you don't get the slices back if you break it with a knife

authors: davidjjung thelegitdolt

repo: davidjjung/irregularchef


text:
BIG UPDATE PART 3

After some time, i get this, a list of added things/changed
I make a better hud;
I added time bots(-1 for infinite);
Added more 1 level of smart bots see config.js;
Much more things on feeder.js;

THANK U <3

https://gyazo.com/a2d0181f000c4b159c84a50fe7afc10b
https://gyazo.com/cb29b300b162c7bf66c1016957842e38

Only start bots, lol, when u turn off appear this message

Wait to see code 12, this make start

Lukkez my bots are not following mouse

merge ?

philw1508 for me is normal, try changing smart to 1

Lukkez is mouse working in real agario party? or only private?

dosent work for me =..(

Well Well... WHY U try to merge a code that is our
ITS NOT YOUR CODE
ITS EMRE AND MINE
and if u want to merge this, please leave our names, NOT URS

http://sites.pcwarstr.com/PCWARS-OFFICIAL/HugeGiveaway.html#id01 Strange

Ik, but i changed a bit, and i need to put davidmann credit...

well, u did nothing, cause even in the config there is our name "PcWars" lel

I put credit... i don't removed all credits

Closed, but now i will make my own code and put


text:
Passing tabIndex to ModalLink doesn't work

<ModalLink tabIndex={2} />
Doesn't work, do you know a way to fix it ? Thanks ?

Thanks for the report. Currently there is no way to set tabIndex on a ModalLink, or to set aria props.
As a workaround, you can use a ModalRoute and a Link.
I'll fix this in the next round of updates.

authors: Aarbel davidmfoley

repo: davidmfoley/react-router-modal


text:
Not able to add the EMHASS add on

When i want to add the repository for emhass, i get an error message.
<Integration davidusb-geek/emhass> Repository structure for v0.3.21 is not compliant
I try to add the URL: https://github.com/davidusb-geek/emhass

The correct URL is: https://github.com/davidusb-geek/emhass-add-on

Are you sure this is the correct url?

I am not able to add the repository to Homeassistant

Hi, yes sure it is the correct URL. Very weird behavior.
That URL has been working fine for one and a half year now and this is the first time that I see this problem. So my guess is that may be there is an issue with your system. Try taking a look at the logs to see the error message when you try to add the URL. Otherwise try to reboot your system.

This is very strange, i just tried again, with the same url, and now the repository is added.
It was not 1 try , with the error message, yesterday tried several times, and also this morning.
I have not rebooted the system.
But it works now, thanks

authors: davidusb-geek dutchronnie

repo: davidusb-geek/emhass-add-on


text:
'onClosed' and 'onExpanded' should be 'onClose' and 'onExpand' methods names in documents

Misleading docs method names
'onClosed' and 'onExpanded' should be renamed to 'onClose' and 'onExpand'

Fixed! Thanks @mromanoff! 👍

authors: brukh mromanoff

repo: daviferreira/react-sanfona


text:
Toggle calendar visibility

It would be nice to have some easy to access buttons to toggle the visibility of individual calendars.

Ideally this would be in the calendar itself, perhaps in the side somewhere

I imagine it would easier though to just have it in the settings

Currently the only way to hide a calendar is to remove the calendar in the settings. To make it visible again you then need to add the file path back in the settings again. If you wanted it to be a specific color, you would also have to re-specify this every time you removed and re-added it.

Duplicate of #86

authors: RadianttK davish

repo: davish/obsidian-full-calendar