text:
[sil] [irgen] make object instruction valid

This patch allows the object instruction to be used everywhere.
Why the change?
Why is this change beneficial? It may appear that it generates the same code as an alloc_ref on the stack. That is mostly true.
The problem with alloc_ref is that it behaves very differently from most other sil instructions. It pretends to be a value but it is actually a pointer. It makes it hard for both the sil optimizer and the llvm optimizer to reason about what's going on. This change allows objects (classes) to be created in a way that is much more semantically similar to structs, enums, and tuples which will allow our current sil otpimization passes to optimize class objects with little to no changes.
For example, mem2reg cannot optimize class objects at all. Adding support would mean re-writing almost the entire pass. However, if objects are created using the object instruction, it can be updated to support objects with only two lines changed in general utility files. That's great.
Additionally, it seems to make it easier for llvm to reason about what's going on as well. In another patch I added support for promoting alloc_refs to objects. And, after making no other change to the sil optimizer (i.e. the sil optimizer still didn't know that objects existed) the llvm optimizer was able to reason about what was going on much more easily.
For example

class Foo {
    var x : Int = 0
}

func a( _ f : Foo) {
    f.x += 1
}

func b( _ f : Foo) -> Int {
    return f.x + 2
}

public func foouser(_ check : Bool) -> Int {
    let f = Foo()
    if check {
        a(f)
    }
    return b(f)
}

Before this patch:
define swiftcc i64 @"$s3run7foouserySiSbF"(i1 %0) #1 {
entry:
  %1 = alloca %T3run3FooC, align 8
  %access-scratch = alloca [24 x i8], align 8
  %access-scratch2 = alloca [24 x i8], align 8
  %2 = tail call swiftcc %swift.metadata_response @"$s3run3FooCMa"(i64 undef) #10
  %3 = extractvalue %swift.metadata_response %2, 0
  %4 = getelementptr inbounds %T3run3FooC, %T3run3FooC* %1, i64 0, i32 0
  %reference.new = call %swift.refcounted* @swift_initStackObject(%swift.type* %3, %swift.refcounted* nonnull %4) #3
  %5 = getelementptr inbounds %swift.refcounted, %swift.refcounted* %reference.new, i64 1
  %._value = bitcast %swift.refcounted* %5 to i64*
  store i64 0, i64* %._value, align 8
  br i1 %0, label %6, label %entry._crit_edge

entry._crit_edge:                                 ; preds = %entry
  %.pre = bitcast %swift.refcounted* %5 to i8*
  br label %14

6:                                                ; preds = %entry
  %7 = getelementptr inbounds [24 x i8], [24 x i8]* %access-scratch2, i64 0, i64 0
  call void @llvm.lifetime.start.p0i8(i64 -1, i8* nonnull %7)
  %8 = bitcast %swift.refcounted* %5 to i8*
  call void @swift_beginAccess(i8* nonnull %8, [24 x i8]* nonnull %access-scratch2, i64 1, i8* null) #3
  %9 = load i64, i64* %._value, align 8
  %10 = call { i64, i1 } @llvm.sadd.with.overflow.i64(i64 %9, i64 1)
  %11 = extractvalue { i64, i1 } %10, 1
  br i1 %11, label %22, label %12, !prof !21, !misexpect !22

12:                                               ; preds = %6
  %13 = extractvalue { i64, i1 } %10, 0
  store i64 %13, i64* %._value, align 8
  br label %14

14:                                               ; preds = %entry._crit_edge, %12
  %.pre-phi = phi i8* [ %.pre, %entry._crit_edge ], [ %8, %12 ]
  %15 = getelementptr inbounds [24 x i8], [24 x i8]* %access-scratch, i64 0, i64 0
  call void @llvm.lifetime.start.p0i8(i64 -1, i8* nonnull %15)
  call void @swift_beginAccess(i8* nonnull %.pre-phi, [24 x i8]* nonnull %access-scratch, i64 0, i8* null) #3
  %16 = load i64, i64* %._value, align 8
  %17 = call { i64, i1 } @llvm.sadd.with.overflow.i64(i64 %16, i64 2)
  %18 = extractvalue { i64, i1 } %17, 1
  br i1 %18, label %21, label %19, !prof !21, !misexpect !22

19:                                               ; preds = %14
  %20 = extractvalue { i64, i1 } %17, 0
  ret i64 %20

21:                                               ; preds = %14
  call void asm sideeffect "", "n"(i32 0) #3
  call void @llvm.trap()
  unreachable

22:                                               ; preds = %6
  call void asm sideeffect "", "n"(i32 1) #3
  call void @llvm.trap()
  unreachable
}

After this patch:
define swiftcc i64 @"$s3run7foouserySiSbF"(i1 %0) {
entry:
  %1 = alloca %T3run3FooC, align 8
  %2 = tail call swiftcc %swift.metadata_response @"$s3run3FooCMa"(i64 undef) #4
  %3 = extractvalue %swift.metadata_response %2, 0
  %4 = getelementptr inbounds %T3run3FooC, %T3run3FooC* %1, i64 0, i32 0
  %reference.new = call %swift.refcounted* @swift_initStackObject(%swift.type* %3, %swift.refcounted* nonnull %4) #3
  %5 = getelementptr inbounds %swift.refcounted, %swift.refcounted* %reference.new, i64 1
  %._value = bitcast %swift.refcounted* %5 to i64*
  %spec.store.select = zext i1 %0 to i64
  store i64 %spec.store.select, i64* %._value, align 8
  %6 = select i1 %0, i64 3, i64 2
  ret i64 %6
}

Why stack?
I spent a few days trying to get this to work when visitObjectInst just emitted an Explosion. In the future, I think this is a thing we can (and should) do but, right now it would mean updating almost every use of a class to handle two paths: an alloc_ref object and an object object. This is because alloc_ref pretends to be a value when it's actually a pointer. The simplest solution was to make object essentially a stack version of alloc_ref and have it pretend to be a pointer as well.
Future plans
I have two more patches that I'll put up for review today. One updates StackPromotion to promote stack alloc_refs to objects and the other adds some basic support for object instructions across the sil optimizer.
Sorry for all the words here :P TL;DR: this patch lays the groundwork for extending struct optimizations to classes as well.

@zoecarver I'm not sure if this makes sense.
I would not say that alloc_refs behaves differently than other instructions. What you call 'to pretend to be a pointer' is a feature of the swift/SIL type system: that's what we call a 'reference type'. It's true that the implementation of a reference is done with a pointer, but that's just how it is implemented. In SIL, a reference type has a very well defined semantic and it can be copied, destroyed, etc. It just means something different: copy = increase ref count, destroy = decrease ref count.
If you want to represent the contents of a class as a value-type in SIL, then this would need a change in the type system rather than just using a different allocation instruction. In your proposal, the result of 'object' is still a reference type (a ref_element_addr of an object - what does this even mean?)
A straw man proposal: we could introduce a new SIL type, kind of a wrapper-type, _class_content(ClassType), which is a struct containing all the stored properties of the class ClassType.
Instead of
  %3 = object $Foo (%2 : $Int)

we can just use a struct instruction to construct a "value-type" class:
  %3 = struct  $_class_content(Foo) (%2 : $Int)

It also has the advantage that all SIL optimization would work with this out of the box.

@eeckstein I completely agree with your points.

I would not say that alloc_refs behaves differently than other instructions.

The problem I am trying to articulate is this (and I probably didn't do a great job articulating it): if we generalize some of the current passes to support alloc_ref instructions, it is likely that we will, for example, get a copy_value instruction where we want a copy_addr because the pass checked getType().isAddress(). As you pointed out, this patch doesn't fix that issue (in fact, it may make it worse).
My original solution to this problem was to emit an Explosion (what we do for struct instructions). That can work as an implementation for reference types by creating a stack_alloc and letting mem2reg do the heavy lifting.
Anyway, I love your proposal! It solves a lot of problems with this patch (doesn't require stack allocation, fixes the above problem, doesn't require any optimizer changes, etc.). What do you think the next steps should be? Do any wrapper types like that exist? Or would this be a new kind of thing? #30743 could probably be updated with minimal changes to create a _class_content struct and emit a store.

Thinking about it again: why not just use a tuple which contains all the class properties?

I suppose that would probably work. It would also remove the wrapper which would be nice. I think the best way to do this will be to create an alloc_stack and replace all stores and ref_element_addrs with copy_addrs and tuple_element_addrs respectively. Then we can just ref cast and copy_addr the tuple to the alloc_ref.
I really appreciate your suggestions! This will now be a much better implementation. Let me know if you have any other ideas.

Adding a new type in SIL optimizations might be a problem, because AFAIK, the AST is considered immutable after the frontend. @slavapestov or @DougGregor can comment on this.
An alternative would be to add this kind of "wrapper-type" just in SIL: add an API layer for struct types in SILType, which either redirect to the AST struct (in most cases) or map the class properties for the wrapper-type case.

I think your tuple idea will work well. I've been playing around with it and looking at the IR generated and it seems to have all the benefits of this patch without the drawbacks. In order to efficiently copy the tuple pointer to the class ref I've added a new sil instruction.
We don't necessarily need another sil instruction but it allows us to copy the whole object with a memcpy which I'm not sure we could otherwise do. I'll put it up for review later today and we can decide whether adding another instruction is worth the cost.

authors: eeckstein zoecarver


text:
[LTO] Setup LTO pipeline and swift-lto tool

Start to implement the LTO library and set up a basis of incremental testing tool named swift-lto. The swiftLTO library accepts SIB files for multi-modules, optimizes them at SIL level and emit LLVM bitcode. It will be dynamically loaded into linker to support language specific LTO.
CC: @compnerd

@compnerd I want to confirm which commit style is easier to review.

Always squash commit even while under reviewing
Commit for each addressing change while under reviewing and squash them before merge.

Its definitely easier to review (at least for me) when the change is squashed into a single change.  However, any change that is not directly related (e.g. refactoring or renaming) can be pulled into a separate change that can be merged prior to the change itself being in review is helpful in keeping the size of the change low.

@gottesmm Thanks for your comment. The dependent libraries are pretty much the same as sil-llvm-gen 👀 I'll combine them and make a symlink like swift-autolink-extract to separate the exec file.

Close as it's too stale for now.

source: gharchive/pull-request

authors: compnerd kateinoigakukun

repo: apple/swift

license_type: permissive

license_source: github-api


text:
[sourcekit] Make it possible to install both sourcekitdInProc and sourcekitd efficiently

This PR makes it possible to install both sourcekitd.framework and sourcekitdInProc.framework on Darwin (previously it was mutually exclusive). In order to make this configuration practical, when both are being installed, we share the implementation by having the XPC service dynamically link to the InProc framework.  When only the XPC service is being installed, it continues to statically link as before.  In order to enable dynamic linking, I fixed some library layering violations around notification handling and UID handling so that the "sourcekitdAPI" library does not implicitly depend on the framework that is supposed to contain it. Also fixed the runtime library path when installing sourcekitdInProc.framework, which seems to have bit-rotted.
Note: this PR does not change the default installation components, so it shouldn't change behaviour unless you manually add both "sourcekit-inproc" and "sourcekit-xpc-service" to your install components.  My intention is to propose installing sourcekitInProc by default as a follow-up.

@swift-ci please test

Build failed
Swift Test Linux Platform
Git Sha - d4a52b8cb791a81b6899841dc175f5ebad7c39b7

@swift-ci please test Linux

@swift-ci please test

Build failed
Swift Test Linux Platform
Git Sha - 16dd929959f857261700970453536548265c99f2

@swift-ci please clean test Linux

@swift-ci please test

@swift-ci please test

Build failed
Swift Test OS X Platform
Git Sha - 16dd929959f857261700970453536548265c99f2

Build failed
Swift Test Linux Platform
Git Sha - 16dd929959f857261700970453536548265c99f2

authors: benlangmuir swift-ci


text:
Revert "[Driver][Frontend] add the symbol graph dir to the supplementary file map"

Reverts apple/swift#36542
Resolves rdar://76815547
Due to the implementation differences between the old and new drivers, there were issues where trying to emit symbol graphs with whole-module optimization and the new driver would cause an error. This PR reverts the implementation in the old driver and frontend to bring the drivers in sync.

Using apple/swift-driver#611
@swift-ci Please test

@swift-ci Please build toolchain macOS platform

macOS Toolchain
Download Toolchain
Git Sha - 87fe71ec5a25d2186aa01b11ecc795c5c981415a
Install command
tar -zxf  swift-PR-37017-956-osx.tar.gz --directory ~/

authors: QuietMisdreavus swift-ci


text:
[Diagnostics] Offer 'is' replacement for unused 'if let' expression with optional operand

let foo: Any? = 42 as Int

// current fix-it: Replace 'let bar = foo as? Int' with '(foo as? Int) != nil'
// better fix-it: Replace 'let bar = foo as? Int' with 'foo is Int'

if let bar = foo as? Int {
    print("'foo' is of type 'Int'")
}

Currently, the above code does not produce the better fix-it for replacing the expression with is because the expression was wrapped inside OptionalEvaluationExpr.
This PR enables optional typed values to produce is replacement fixit.

Resolves SR-14646

One more thing - please use clang-format to format the code, you can do that via git-clang-format as well after git adding the files, that tool could be found in clang/tools/clang-format at your llvm-project checkout.

Thanks! I applied clang-format to format the code :)

@swift-ci please smoke test

@swift-ci please clean smoke test macOS platform

@swift-ci please clean smoke test macOS platform

@swift-ci please clean smoke test macOS platform

@swift-ci please smoke test macOS platform

@swift-ci please smoke test macOS platform

authors: LucianoPAlmeida mininny xedin


text:
[SIL][DebugInfo] PATCH 1/3: Add support for expr op_deref

This new SIL di-expression represents the dereference on the SSA value.
Similar to DW_OP_deref in DWARF. It is also going to replace the
existing debug_value_addr. Namely, replacing the following
instruction:
debug_value_addr %a : $*T, name "my_var"

with this one:
debug_value %a : $*T, name "my_var", expr op_deref

This is a PR only for review. DO NOT MERGE

@swift-ci please test


text:
Revert "Test type_sequence fails on incremental bot"

I cannot reproduce this locally. Using this to see if it still hits our infrastructure
rdar://86744286

@swift-ci test

preset=buildbot_incremental,tools=RA,stdlib=RA
@swift-ci Please test with preset macOS

⛵


text:
constant_vprintf: Add a mode where prints are silenced when -DPRINT_DISABLED is present on the client side

Added a test that showcases that this actually works.

@swift-ci please test


text:
[Mem2Reg] Skip load [take] of cast projections.

Already, load [take]s of struct_element_addr|tuple_element_addr projections resulted in Mem2Reg bailing.  Expand that to include load [take]s involving unchecked_addr_cast.
To handle load [take]s of (struct|tuple)_element_addr projections, it would be necessary to replace the running value with a value obtained from the original product by recursive destructuring, replacing the value at the load [take]n address with undef, and then restructuring.
To handle load [take]s of cast projections, it would be necessary to use unchecked_value_cast instead of unchecked_bitwise_cast.  But we would need to still use unchecked_bitwise_cast in the case of load [copy] because otherwise we would lose the original value--unchecked_value_cast forwards ownership, and not all casts can be reversed (because they may narrow).
For now, just bail out in the face of these complex load [take]s.

@swift-ci please test


text:
[stdlib] Fix mismatched constant name in static func == doc

I've found that mismatched constant name in static func == example.
Given the context, it should be numberToMatch, not numberToFind. 😄

@swift-ci Please smoke test

Thanks
@swift-ci smoke test and merge

authors: CodaFi Jager-yoo


text:
[CSBindings] Favor conjunctions over last resort existential bindings

If type variable's binding set is incomplete, let's not
bind it to a type inferred from conformance constraint
(that's the action of last resort because not all
protocols can conform to themselves), instead let's
solve conjunctions first which could end up providing
actual type to convert to a contextual existential.

@swift-ci please test

@swift-ci please test source compatibility

@swift-ci please test macOS platform

@swift-ci please test source compatibility

@swift-ci please test macOS platform

Apparently this break some existing code, so I'm going to close this for now and trying to figure out a different approach.


text:
SIL: Pass SubstFlags::PreservePackExpansionLevel in a few places

Description: In a few places we do a substitution with a callback that doesn't wrap a pack parameter T in a PackType{PackExpansionType{T}}. This no longer works with the PackElementType refactoring that landed recently because we get confused if we see a free pack parameter that is outside of an expansion. These callbacks should be refactored to do the pack wrapping correctly, but in the meantime the easiest fix is to pass a temporary flag which reverts to the old behavior.

Scope of the issue: There was an easy test case that crashed in the SIL optimizer.

Risk: Only impacts variadic generic code paths.

Radar: rdar://problem/111219086

Reviewed by: @hborla

@swift-ci Please test


text:
[embedded] Fix arm64e pointer signing in embedded heap object destruction

Some embedded tests today crash under arm64e -- we need to properly authenticate the ptrauth signed "destroy" function pointer when making the runtime call to destroy a class instance.

@swift-ci please test

@swift-ci please test

@swift-ci please test Linux platform

@swift-ci please test


text:
[DebugInfo] Re-enable x86 async tests

With https://github.com/apple/llvm-project/pull/7787 merged, we can now finally renable these tests, as the x86 swift async handling should be properly preserved by the x86 instruction selector.
These tests contained some incorrect checks from a time where it was expecting an entry value even in the entry funclet of coroutines. For example, it had this:
CHECK-NEXT: [[ASYNC_REG:DW_OP_.*], DW_OP_plus_uconst 0x10, DW_OP_plus_uconst 0x8,
DW_OP_deref

Which tries to find some "async reg", but really matches anything. It worked by accident on arm, but not on x86. The correct check is simply:
CHECK-NOT: OP_entry_value

rdar://114831502

https://github.com/apple/llvm-project/pull/7787/files
@swift-ci please test


text:
[Exclusivity] Improve static enforcement diagnostics

Static diagnostics now refer to the identifier for the variable requiring
exclusive diagnostics. Additionally, when two accesses conflict we now always
emit the main diagnostic on the first modifying access and the note on either
the second modifying access or the read.
The diagnostics also now highlight the source range for the expression
beginning the access.

@swift-ci Please smoke test and merge


text:
[Bug]: Grouping widgets in a Modal makes the resulting container move to the main canvas

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior
When I group multiple widgets which are in a modal, the resulting canvas  automatically moves to the main canvas, rather than staying in place
https://user-images.githubusercontent.com/20532920/148173902-7fa57daf-d300-4a32-bcd5-5de1e2301208.mov
Steps To Reproduce

Create a modal
Select the 2 existing buttons (close and confirm)
Group them

Environment
Production
Version
Cloud

@riodeuno @rahulramesha is this something we'll be able to cover as part of the Dynamic Height or  Copy Paste Improvements project?

@vuiets This seems like a bug with grouping. I have seen this in the Form widget as well. We should pick this up as a bug fix. @jsartisan Could you help triage this as this seems related to the grouping feature.

Stats

Stat
Values

Reach
120

Effort (months)
0.5

For reach,
People using modals in their apps in the last 6 months: 1223
Expect about 10% of these to group widgets in the modal: 120
Added effort as 2 person-weeks for this but feel free to correct me if wrong here.

This is a duplicate of https://github.com/appsmithorg/appsmith/issues/12320, Which was fixed in https://github.com/appsmithorg/appsmith/pull/13658. Closing this.
cc: @btsgh

source: gharchive/issue

authors: kocharrahul7 rahulramesha riodeuno vuiets

repo: appsmithorg/appsmith


text:
[Bug]: Enable TLS Protected Connection toggle in Email settings [Admin Settings] seems nonfunctional

Is there an existing issue for this?

[X] I have searched the existing issues

Description
The SMTP User name and SMTP Password fields are both available even if the Enable TLS Protected Connection is off.
Steps To Reproduce

Login as instance admin to release [CE or EE]
Navigate to Admin Settings, and click on the Email section
Here, note that the SMTP User name and SMTP Password fields are visible even though the Enable TLS Protected Connection is off.

Public Sample App
No response
Environment
Release
Issue video log
https://ddpa7mfuvv.vmaker.com/record/kBOpFfntxsDOSoMu
Version
Cloud

@btsgh this was intentionally changed by @sharat87 under this PR:
https://github.com/appsmithorg/appsmith/pull/17952

As per above comments, this is intended behavior

authors: ankitakinger btsgh


text:
[Bug]: On forking an application into a new workspace, modal opens inside the app, thus query errors pop up even before saving DS configs

Is there an existing issue for this?

[X] I have searched the existing issues

Description
When I fork an application containing a few Datasources into a new workspace, the reconnect modal opens inside the app, and thus query errors pop up even before saving DS configs
This is incorrect behavior.
The expected behavior is to have it behave similar to import behavrior in which case the reconnect modal opens on the home page itself. and thus only after saving DS configs, the app is opened, and queries pass.
Steps To Reproduce

Create an app with a few datasources like mongo/dynamo/s3/gsheets
Fork this application into a new workspace
Note that the reconnect modal opens inside the application, and thus query errors pop up even before saving DS configs
This is incorrect behaviour.
The expected behaviour is to have it behave similar to import, in which case the reconnect modal opens on the home page itself. And thus only after saving DS configs, the app is opened, and queries pass thereof

Public Sample App
No response
Environment
Production
Issue video log
No response
Version
Cloud

Still reproducible on Prod

Updated the labels as once the modal appears, the fork/import operation is complete.

authors: btsgh infinitetrooper


text:
[Bug]: Input widget should add a scroll when text gets longer rather than increasing the widget size

Is there an existing issue for this?

[X] I have searched the existing issues

Description
When input widget is set to multiline and input text starts to occupy more space than available, then the widget should add a scroll to handle the text overflow rather than increase the size of the widget
Steps To Reproduce

Set input widget to multiline
Drop/add a long text until the widget starts to expand
Keep adding text and notice the section height has increased causing an unsavory UI layout

Public Sample App
No response
Environment
Deploy Preview
Severity
Medium (Frustrating UX)
Issue video log
https://www.loom.com/share/f18b85c7f3c4456784aecadd37a71696
Version
cloud

@ichik should we add max-height for multiline input?

@KelvinOm TBH that seems like something that should be controlled by builders. For dense UI max-height should be default.

authors: KelvinOm ichik ramsaptami


text:
[Bug]: Unable to close Filter popup without choosing a filter selection

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior
On a table widget, clicking the Filter button at the top opens a movable Filter popup. But then this popup is impossible to close, unless a selection for filtering is made.
https://user-images.githubusercontent.com/120119/136984927-9efc9a04-5502-4d2d-b654-4049a0c5d463.mov
Steps To Reproduce
Just open Filter box for a Table widget and try to close it.

Environment
Production
Version
Cloud

Yes, it's poor experience.

This is fixed in #8734

authors: sharat87 somangshu tmdoit


text:
[Bug]:  Video widget supports only URL and not camera binding

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior
When the camera widget video binding is fetched under the entity explorer it is displayed as "camera1.video" when. tried to bind the same into Video it is not accepted. As video needs URL as binding
Expectation:
To make it easy to user we can display the exact binding needed for the video widget
Steps To Reproduce

add a camera widget
navigate to entity explorer
observe the binding for video
Copy the binding for the video
Add a video widget
paste the binding to the video
and observe

Environment
Deploy Preview
Version
Cloud

Fixed.

authors: prapullac shwetha-ramesh


text:
fix: handled content type none in API

Description

The default header (content-type:none) gets added by default in API Pane when the user switches between body and header tabs https://www.loom.com/share/0dd89cd153b54c71a11f86bc5820d968
The default header (content-type:none) gets added by default in API Pane when the user switches between HTTP method types

Fixes #14624
Type of change

Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?
Have updated the test case in API_DefaultContentType_spec
Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[x] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes

/ok-to-test sha=88fdec6

/ok-to-test sha=7a900cd

Tested on the above given Deploy Preview for - [Bug]: The header gets added by default in API Pane when the user switches between body and header tabs#14624.
The issue has been fixed. Though the default header will be added, Content-Type is not set to None when browsing between the Body and Headers tab. It is set to one of the options selected in the Body tab (other than None)

authors: ChandanBalajiBP btsgh


text:
Fix: action selector create flow

Description

[!TIP]
Add a TL;DR when the description is longer than 500 words or extremely technical (helps the content team).
Please also include relevant motivation and context. List any dependencies that are required for this change. Add links to Notion, Figma or any other documents that might be relevant to the PR.

Fixes #Issue Number
or
Fixes Issue URL

[!WARNING]
If no issue exists, please create an issue first, and check with the maintainers if the issue is valid.

Automation
/ok-to-test tags=""
:mag: Cypress test results

[!CAUTION]
If you modify the content in this section, you are likely to disrupt the CI result for your PR.

/build-deploy-preview skip-tests=true env=release

/build-deploy-preview skip-tests=true

authors: albinAppsmith hetunandu rishabhrathod01


text:
Only add appsmith functions in global scope during trigger evaluation

Description
We were adding appsmith trigger functions in the global scope for all types of evals, this was wasteful as they will never get executed outside a trigger based field. This change improves performance in the evaluation stage
Type of change

Performance

How Has This Been Tested?
All triggers should work as normal.
Using a function in any other non-trigger field should throw an evaluation error
Checklist:

[ ] My code follows the style guidelines of this project
[ ] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes

Test coverage results :test_tube:
:green_circle: Total coverage has increased
// Code coverage diff between base branch:release and head branch: perf/improve-evaluate-perf

Status
File
% Stmts
% Branch
% Funcs
% Lines

:green_circle:
total
53.84 (0.01)
35.82 (0.02)
32.45 (0.02)
54.43 (0.02)

:green_circle:
app/client/src/workers/evaluate.ts
100 (2.94)
80 (11.25)
100 (5.88)
100 (3.03)

/ok-to-test sha=442819b


text:
Upgraded the issue templates

I have ugraded our issue templates for reporting a bug, doccumenting an issue and to request a feature. I have also tested them all by creating a issue each. They are working fine..

@Ln11211  Great work. One more change before we can approve. We have made few changes in the language and also the format of Code of conduct. Please check https://github.com/appwrite/appwrite template and update accordingly.
Thank you.

@lohanidamodar , will do, thank you 😊

@lohanidamodar , so in the branch naming convention  " TYPE-ISSUE_ID-DESCRIPTION" , the type in this case will be DOC right ?

@lohanidamodar , I tried changing the main branch name and it doesn't seem to change it in the pr, can you please help me out ?  I named it as feat-#5-Add_Issues_Templates, so should I create a new branch and restart again ?

@lohanidamodar, I think I have to create a new pull request..

@Ln11211 I am not sure what mean. This is good. You don't need another PR just update the templates with the fixes and push it in your main branch. That's ok.

@lohanidamodar , I'm sorry but what exactly should I update in my templates ?, did I miss anything?

@lohanidamodar  so I have noticed that the templates were updated with "Code of conduct " and I have done the same. Please review...

@lohanidamodar , well, I also noticed that the original appwrite has a "workflows" folder in ".github" folder and it contains two .yml files, should I also add them  ?

repo: appwrite/docker-swagger

license: MIT


text:
update to appwrite 1.3.0

What does this PR do?
(Provide a description of what this PR does.)
Test Plan
(Write your test plan here. If you changed any code, please provide us with clear instructions on how you verified your changes work.)
Related PRs and Issues
(If this PR is related to any other PR or resolves any issue or related to any issue link all related PR and issues here.)
Have you read the Contributing Guidelines on issues?
(Write your answer here.)

LGTM

repo: appwrite/sdk-for-web


text:
empty app view (screen 203)

from #51.
The empty app view screen shows info on how to deploy an app.
Should look like:

@fancyremarker How would diesel know that an app had been deployed before? Would it be enough to know that app.lastDeployOperation is not empty?

If there's no lastDeployOperation, then no deploy has been attempted.
If there's no successful deploy operation or, alternatively, no services, then no deploy has been successful.

@bantic , I've thought about this further and think there is a minor mistake in the design I provided.  Even though the app is technically 'empty', I think there is still some value in a: allowing the user to see app activity, and b: allowing the user to deprovision the empty app.
So, I think we still need an app nav for this screen with 3 tabs: 'Deploy', 'Activity', and 'Deprovision'.  Activity and Deprovision are the same as what we currently have and Deploy is the 3 deploy step UI from the original design.
'Services' and 'Domains' tabs should still be removed from the empty app nav.

@sandersonet Makes sense — the "deploy" tab is only there for a not-previously-deployed app, then? And is it the default tab shown for a not-previously-deployed app? Current default is the "services" tab.
I am very nearly finished with an update to the HAL adapter to have it read in the embedded lastOperation and lastDeployOperation for an app. Once that is in this should be a fairly straightforward issue.

Yep, exactly.

@sandersonet Any objection to modifying the API to include this as a property of the app. Like last_successful_deploy_at or embedding another last_successful_deploy_operation operation object? It would simplify the checking we'd otherwise need to do when we enter the app show page.

I think embedding last_successful_deploy_operation sounds like a good fit.  Feel free to add this to API yourself, or I can handle it later this afternoon.

:+1: , SGTM.
Also, if it's easier, we can add a status field to App and set it to provisioned (or some other value) in Sweetness once a successful deploy happens.

@fancyremarker that would be the simplest. Would that be straightforward for me to add to the API? Else I can embed a last_successful_deploy_operation easily. Either way will make an API pr soon...

authors: bantic fancyremarker sandersonet


text:
Training Runtime Error: StopIteration

Hi,
I'm using the released training data on AWS and the latest main branch to train the model.

The directory structure of the released data is not recognized by the code.
After re-structuring the directories and put all the .hhr and .a3m under the alignment directory, the code crashes at File "/global/u2/b/bz186/openfold/openfold/data/data_modules.py", line 377, in reroll datapoint_idx = next(samples) with default settings.

Any idea to solve this?
Thanks,
Bo
The full trace back is as below:
Traceback (most recent call last):
  File "train_openfold.py", line 548, in <module>
    main(args)
  File "train_openfold.py", line 341, in main
    ckpt_path=ckpt_path,
  File "/global/homes/b/bz186/openfold/lib/conda/envs/openfold_venv/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 741, in fit
    self._fit_impl, model, train_dataloaders, val_dataloaders, datamodule, ckpt_path
  File "/global/homes/b/bz186/openfold/lib/conda/envs/openfold_venv/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 685, in _call_and_handle_interrupt
    return trainer_fn(*args, **kwargs)
  File "/global/homes/b/bz186/openfold/lib/conda/envs/openfold_venv/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 777, in _fit_impl
    self._run(model, ckpt_path=ckpt_path)
  File "/global/homes/b/bz186/openfold/lib/conda/envs/openfold_venv/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1199, in _run
    self._dispatch()
  File "/global/homes/b/bz186/openfold/lib/conda/envs/openfold_venv/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1279, in _dispatch
    self.training_type_plugin.start_training(self)
  File "/global/homes/b/bz186/openfold/lib/conda/envs/openfold_venv/lib/python3.7/site-packages/pytorch_lightning/plugins/training_type/training_type_plugin.py", line 202, in start_training
    self._results = trainer.run_stage()
  File "/global/homes/b/bz186/openfold/lib/conda/envs/openfold_venv/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1289, in run_stage
    return self._run_train()
  File "/global/homes/b/bz186/openfold/lib/conda/envs/openfold_venv/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1319, in _run_train
    self.fit_loop.run()
  File "/global/homes/b/bz186/openfold/lib/conda/envs/openfold_venv/lib/python3.7/site-packages/pytorch_lightning/loops/base.py", line 140, in run
    self.on_run_start(*args, **kwargs)
  File "/global/homes/b/bz186/openfold/lib/conda/envs/openfold_venv/lib/python3.7/site-packages/pytorch_lightning/loops/fit_loop.py", line 197, in on_run_start
    self.trainer.reset_train_val_dataloaders(self.trainer.lightning_module)
  File "/global/homes/b/bz186/openfold/lib/conda/envs/openfold_venv/lib/python3.7/site-packages/pytorch_lightning/trainer/data_loading.py", line 595, in reset_train_val_dataloaders
    self.reset_train_dataloader(model=model)
  File "/global/homes/b/bz186/openfold/lib/conda/envs/openfold_venv/lib/python3.7/site-packages/pytorch_lightning/trainer/data_loading.py", line 365, in reset_train_dataloader
    self.train_dataloader = self.request_dataloader(RunningStage.TRAINING, model=model)
  File "/global/homes/b/bz186/openfold/lib/conda/envs/openfold_venv/lib/python3.7/site-packages/pytorch_lightning/trainer/data_loading.py", line 611, in request_dataloader
    dataloader = source.dataloader()
  File "/global/homes/b/bz186/openfold/lib/conda/envs/openfold_venv/lib/python3.7/site-packages/pytorch_lightning/trainer/connectors/data_connector.py", line 300, in dataloader
    return method()
  File "/global/u2/b/bz186/openfold/openfold/data/data_modules.py", line 694, in train_dataloader
    return self._gen_dataloader("train")
  File "/global/u2/b/bz186/openfold/openfold/data/data_modules.py", line 671, in _gen_dataloader
    dataset.reroll()
  File "/global/u2/b/bz186/openfold/openfold/data/data_modules.py", line 377, in reroll
    datapoint_idx = next(samples)
StopIteration
srun: error: nid001680: task 0: Exited with exit code 1
srun: launch/slurm: _step_signal: Terminating StepId=2466693.0

Hi,
I dug a little deeper into this bug by add the exception handling:
try:
     datapoint_idx = next(samples)
except StopIteration:
     print("samples.length = {}, idx = {}".format(sum(1 for _ in samples), dataset_idx))

but the result shows that samples.length = 0, idx = 0
just FYI.
Bests,
Bo

Try increasing the length of the training epoch.

Try increasing the length of the training epoch.

So this means set --train_epoch_len to a large value?

Yes. As for the format of the AWS data, you'll need to reformat it slightly. For each chain, you'll want to flatten the hhr and a3m directories, such that the directory for each chain contains both the hhr files and the a3m files for that chain.

Yeah. I've done that. But I found some chains doesn't have all the 4 files(3 .a3m and 1 .hhr).
By the way, I've set the --train_epoch_len to 80000 but still not working. If I set it to <=1200, it works but doesn't make sense.

Could you share your training command?

srun python3 train_openfold.py \
/pscratch/sd/b/bz186/openfold/data/pdb_mmcif/mmcif_files \
/pscratch/sd/b/bz186/openfold/data/alignment_openfold \
/pscratch/sd/b/bz186/openfold/data/pdb_mmcif/mmcif_files \
/pscratch/sd/b/bz186/openfold/data/train_full_output \
2021-10-10 \
--template_release_dates_cache_path=/pscratch/sd/b/bz186/openfold/data/mmcif_cache.json \
--precision=32 \
--gpus=4 \
--replace_sampler_ddp=True \
--seed=42 \
--deepspeed_config_path=/global/homes/b/bz186/openfold/deepspeed_config.json \
--checkpoint_every_epoch \
--obsolete_pdbs_file_path=/pscratch/sd/b/bz186/openfold/data/pdb_mmcif/obsolete.dat \
--train_chain_data_cache_path=/pscratch/sd/b/bz186/openfold/data/chain_data_cache.json \
--train_epoch_len=80000

Is this being run on a single node?

A single node with 4 GPUs , I did it on interactive mode.

Hm not able to reproduce. Could you share a sample of the directory structure of /pscratch/sd/b/bz186/openfold/data/alignment_openfold?

/pscratch/sd/b/bz186/openfold/data/alignment_openfold
    -11as_A
        - bfd_uniclust_hits.a3m
        - mgnify_hits.a3m
        - pdb70_hits.hhr
        - uniref90_hits.a3m
    -11ba_A
        - bfd_uniclust_hits.a3m
        - mgnify_hits.a3m
        - pdb70_hits.hhr
        - uniref90_hits.a3m
    -11ba_B
    -11bg_A
    -11bg_B
    -11gs_A

The directory tree is similar as above for all chains ,except some doesn't have all 4 files.

How about the filter thing?

It prints some of the chain_data_cache_entry like this:
{'release_date': '2011-02-09', 'seq': 'MSAGKLPEGWVIAPVSTVTTLIRGVTYKKEQAINYLKDDYLPLIRANNIQNGKFDTTDLVFVPKNLVKESQKISPEDIVIAMSSGSKSVVGKSAHQHLPFECSFGAFCGVLRPEKLIFSGFIAHFTKSSLYRNKISSLSAGANINNIKPASFDLINIPIPPLAEQKIIAEKLDTLLAQVDSTKARFEQIPQILKRFRQAVLGGAVNGKLTEKWRNFEPQHSVFKKLNFESILTELRNGLSSKPNESGVGHPILRISSVRAGHVDQNDIRFLECSESELNRHKLQDGDLLFTRYNGSLEFVGVCGLLKKLQHQNLLYPDKLIRARLTKDALPEYIEIFFSSPSARNAMMNCVKTTSGQKGISGKDIKSQVVLLPPVKEQAEIVRRVEQLFAYADTIEKQVNNALARVNNLTQSILAKAFRGELTAQWRAENPDLISGENSAAALLEKIKAERAASGGKKASRKKS', 'resolution': 18.0, 'cluster_size': -1}

Could you count how many of your proteins are getting filtered out by that function? I see for example that the resolution of this protein is very high---this one gets filtered for that reason.

The count number of the if block is 26 before it crashes.

How about the probabilities that come out of the stochastic filter? Are those really small?

I print the p right after
p = get_stochastic_train_filter_prob(
                        chain_data_cache_entry,
                    )

almost all of them are > 0.5
p = 1.0
p = 0.609375
p = 0.880859375
p = 0.5
p = 0.5
p = 0.5
p = 0.693359375
p = 0.5
p = 0.5
p = 0.5
p = 0.5
p = 0.6640625
p = 0.822265625
p = 0.001953125
p = 0.5
p = 0.5
p = 0.9140625
p = 0.5
p = 0.5390625
p = 0.52734375
p = 0.5
p = 0.5
p = 0.5
p = 0.51953125
p = 1.0
p = 0.53125
p = 0.5
p = 0.5
p = 0.5
p = 0.5
p = 0.5

Hm so that's not it either. Whatever the issue is, it's probably happening somewhere in that function. Would you mind pinpointing where the samples are disappearing?

I'm trying to do that. But we have some difficulties to understand the code, since we'are only computer science research and do not have required protein knowledge.

In the reroll() function, could you verify that len(self._samples) == 1 and also that torch.sum(dataset_choices) == 0?

yes, they are. 1 and tensor(0).

since the self._samples only has 1 element inside it, I tried to investigate the self._samples[0] by doing print(sum(1 for _ in self._samples[0])). This time, it tells me that
Traceback (most recent call last):
  File "train_openfold.py", line 548, in <module>
    main(args)
  File "train_openfold.py", line 262, in main
    data_module.setup()
  File "/global/homes/b/bz186/openfold/lib/conda/envs/openfold_venv/lib/python3.7/site-packages/pytorch_lightning/core/datamodule.py", line 474, in wrapped_fn
    fn(*args, **kwargs)
  File "/global/u2/b/bz186/openfold/openfold/data/data_modules.py", line 654, in setup
    _roll_at_init=False,
  File "/global/u2/b/bz186/openfold/openfold/data/data_modules.py", line 358, in __init__
    print(sum(1 for _ in self._samples[0]))
  File "/global/u2/b/bz186/openfold/openfold/data/data_modules.py", line 358, in <genexpr>
    print(sum(1 for _ in self._samples[0]))
  File "/global/u2/b/bz186/openfold/openfold/data/data_modules.py", line 334, in looped_samples
    chain_data_cache_entry = chain_data_cache[chain_id]
KeyError: '3wwy_A'

But when I check the alignment_dir, this chain does exist:
(openfold_venv) bz186@nid001520:~/openfold> ls /pscratch/sd/b/bz186/openfold/data/alignment_openfold | grep 3wwy_A
3wwy_A
(openfold_venv) bz186@nid001520:~/openfold> ls /pscratch/sd/b/bz186/openfold/data/alignment_openfold/3wwy_A
bfd_uniclust_hits.a3m  mgnify_hits.a3m	pdb70_hits.hhr	uniref90_hits.a3m

I'm assuming there is something wrong with the file passed by --train_chain_data_cache_path. But I did that pre-processing step using the full mmcif dataset downloaded by the script and the PDB40 in the README link. Is there any potential error that could happens given that the AWS dataset is claimed to have ~130,000 chains, however the full mmcif dataset has ~190,000 .cif files

To clarify, self._samples contains one infinite iterator for each dataset you're using. Each entry thereof feeds you samples from the corresponding dataset shuffled + filtered in various ways forever.
Yes, it's looking like something is wrong with your train chain data cache. It should contain an entry for every chain for which you have alignments. The total number of .cif files doesn't matter; what controls the size of the dataset objects is the alignment_dir. I'll revisit the chain data cache script and see if I can find any errors. In the meantime, you can add a check in that function we were discussing earlier that skips chains that do not appear in your chain data cache.

I also ran into the same problem. I found 395 unique proteins in my msa_dir that wasn't in my mmcif_files dir and also in my chain_data_cache.json. I removed all the chains for the 395 proteins, and the training works for me. I'm guessing there is a difference in the data used to generate the msa.

Could you verify that every single chain in your alignment_dir has a corresponding .mmcif file in the data_dir?

Thank you so much for the quick response Gustaf. You're right -- 57 chains appear to be missing. I'm unsure what caused it.
The missing chains are 1erk 3d6s 3rvw 3rvw 3wwy 3wwz 4acq 4ftp 4gbo 4kyu 4pp1 4pp1 4pp1 4roq 4ror 5b7o 5fim 5hbg 5i4g 5l0x 6dfz 6iwl 6l7f 6nuy 6nuy 6nuz 6nuz 6nv0 6nv0 6ors 6qwj 6qwj 6qwj 6qwj 6qwj 6qwj 6qwj 6qwj 6qwj 6qwj 6qwj 6ubg 6ubg 6vig 6vj1 6vmu 6x8v 6xbp 6xdl 6xoe 7ay4 7ayl 7elo 7jnq 7kcd 7ra6 7v7y.
How do you recommend I proceed? Should I just delete their subdirectories in the template directory?
I doubt there were errors during the download and extraction of the PDB mmcif files. It's also weird that the user above reported 395 missing chains while in my case it's only 57...

Delete their alignment_dirs and rerun. I'll look into what's causing this.

Thank you again Gustaf for the quick response.
I moved all the subdirectories to a separate "missing_mmcif" directory as to make sure no data was lost. I tried rerunning it (same command as before), but I'm now getting a KeyError at line 338 of data_modules.py:

File "/fsx/openbioml/openfold/openfold/data/data_modules.py", line 338, in looped_samples
chain_data_cache_entry = chain_data_cache[chain_id]
KeyError: '6tif_AAA'

6tif.cif is present in the mmcif directory, and 6tif_AAA is present in the alignment/templates directory with what appear to be all necessary files: bfd_uniclust_hits.a3m  mgnify_hits.a3m  pdb70_hits.hhr  uniref90_hits.a3m. However, only 6tif_A and 6tif_B are present in chain_data_cache.json -- I'll check which chains are missing and move them away too. Will report back tomorrow.

My chain_data_cache has these chains, and I tried parsing my copy of 6tif.cif and found them there too.
Could you verify that this file does not match the copy of 6tif.cif the installation scripts downloaded for you?

Indeed, the 6tif.cif file (which I downloaded via the script) does not match yours. I have uploaded mine here. Still unsure what is causing this, but my 6tif.cif file only has A and B chains (just like in the generated chain_data_cache.json file. I'm not familiar with mmcif files, but the _audit_conform.dict_version, in my file, is 5.359 (while in yours it's 5.326). Maybe it has been updated? Indeed if I download the mmCIF file directly off of PDB the file matches mine (dict_version 5.359, only chains A and B present).

Sigh...
The way the data is set up assumes that PDB .cif files don't change over time. I'll need to do some thinking to figure out the best way to address this. I might end up having to upload my set of .cif files to RODA.
Thanks for running these diagnostics!

Nah, thank you for OpenFold! :)
I checked if any other chains had the same issue, but none appear to apart from the aforementioned 6tif_AAA. Removing it from the alignment directory seems to fix the issue, but I soon run into another. I don't think it's related -- please let me know if you'd prefer me opening a separate issue.
I'm first getting a warning UserWarning: One of given dataloaders is None and it will be skipped.
I'm then getting several more warnings about exact sequences missing. An example is the following:

WARNING:root:The exact sequence MTTPRRALIVIDVQNEYVTGDLPIEYPDVQSSLANIARAMDAARAAGVPVVIVQNFAPAGSPLFARGSNGAELHPVVSERARDHYVEKSLPSAFTGTDLAGWLAARQIDTLTVTGYMTHNADASTINHAVHSGLAVEFLHDATGSVPYENSAGFASAEEIHRVFSVVLQSRFAAVASTDEWIAAVQGGTPLA was not found in 3oqp_A. Realigning the template to the actual sequence.

Finally, I get a RuntimeError: expected scalar type BFloat16 but found Half (not sure if related to #180):

Traceback (most recent call last):
File "/fsx/openbioml/openfold/train_openfold.py", line 104, in training_step
outputs = self(batch)
File "/fsx/openbioml/openfold_venv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/fsx/openbioml/openfold/train_openfold.py", line 67, in forward
return self.model(batch)
File "/fsx/openbioml/openfold_venv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/fsx/openbioml/openfold/openfold/model/model.py", line 510, in forward
_recycle=(num_iters > 1)
File "/fsx/openbioml/openfold/openfold/model/model.py", line 243, in iteration
inplace_safe=inplace_safe,
File "/fsx/openbioml/openfold_venv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/fsx/openbioml/openfold/openfold/model/embedders.py", line 116, in forward
tf_emb_i = self.linear_tf_z_i(tf)
File "/fsx/openbioml/openfold_venv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/fsx/openbioml/openfold_venv/lib/python3.7/site-packages/torch/nn/modules/linear.py", line 114, in forward
return F.linear(input, self.weight, self.bias)
RuntimeError: expected scalar type BFloat16 but found Half

A cursory google search leards to https://discuss.pytorch.org/t/mixed-precision-training-on-cuda-with-bfloat16/156248, so I tried re-running with fp16 rather than bfloat16, and apart from the same warnings I now get a ValueError("Unsupported datatype"):

File "/fsx/openbioml/openfold/train_openfold.py", line 104, in training_step
outputs = self(batch)
File "/fsx/openbioml/openfold_venv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/fsx/openbioml/openfold/train_openfold.py", line 67, in forward
return self.model(batch)
File "/fsx/openbioml/openfold_venv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/fsx/openbioml/openfold/openfold/model/model.py", line 510, in forward
_recycle=(num_iters > 1)
File "/fsx/openbioml/openfold/openfold/model/model.py", line 367, in iteration
_mask_trans=self.config._mask_trans,
File "/fsx/openbioml/openfold_venv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/fsx/openbioml/openfold/openfold/model/evoformer.py", line 996, in forward
m, z = checkpoint_fn(b, m, z)
File "/fsx/openbioml/openfold_venv/lib/python3.7/site-packages/deepspeed/runtime/activation_checkpointing/checkpointing.py", line 743, in checkpoint
CheckpointFunction.apply(function, all_outputs, *args)
File "/fsx/openbioml/openfold_venv/lib/python3.7/site-packages/deepspeed/runtime/activation_checkpointing/checkpointing.py", line 582, in forward
outputs = run_function(*inputs_cuda)
File "/fsx/openbioml/openfold_venv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/fsx/openbioml/openfold/openfold/model/evoformer.py", line 519, in forward
self.ckpt if torch.is_grad_enabled() else False,
File "/fsx/openbioml/openfold_venv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/fsx/openbioml/openfold/openfold/model/msa.py", line 284, in forward
flash_mask=mask,
File "/fsx/openbioml/openfold_venv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/fsx/openbioml/openfold/openfold/model/primitives.py", line 488, in forward
o = attention_core(q, k, v, *((biases + [None] * 2)[:2]))
File "/fsx/openbioml/openfold/openfold/utils/kernel/attention_core.py", line 32, in forward
raise ValueError("Unsupported datatype")

Thinking that maybe something went wrong with flash attention's installation? Any tips? I would also greatly appreciate if you could share the DeepSpeed config you used for the reproduction run you reported. I'm running this on 64 A100s, so the setup should be similar to your one (which IIRC was on 45 A100s). All of this is for an open-source project that I would love to chat with you about at some point! :)

The "exact sequence" warning is expected and is nothing to worry about. As for the precision thing, could we move this to #180? I'm pretty sure it's the same thing.

I wonder if the issue is with the .pdb filenames? Here is what my distillation_data_dir looks like:

A0A009EX06_model_5_unrelaxed.pdb
A0A009EY75_model_5_unrelaxed.pdb
A0A009FAV8_model_5_unrelaxed.pdb
A0A009GC30_model_5_unrelaxed.pdb
A0A009GC83_model_5_unrelaxed.pdb
A0A009H2D3_model_5_unrelaxed.pdb
A0A009H847_model_5_unrelaxed.pdb
A0A009HRS4_model_5_unrelaxed.pdb
A0A009IPF9_model_5_unrelaxed.pdb
A0A009J662_model_5_unrelaxed.pdb

Those filenames also appear unmodified in the generated chain_data_cache json file. However, my distillation_alignment_dir looks like this:

A0A009EX06
A0A009EY75
A0A009FAV8
A0A009GC30
A0A009GC83
A0A009H2D3
A0A009H847
A0A009HRS4
A0A009IPF9
A0A009J662

Do you think this could indeed be the issue? In this case I guess the fix would be to simply renaming each .pdb file (removing _model_5_unrelaxed) and re-generating the distillation_chain_data_cache json file.
Since this is related to distillation, please let me know if you'd like me to move this discussion to #187.

repo: aqlaboratory/openfold


text:
feat: add tailscale/tailscale

Close #26715
tailscale/tailscale: The easiest, most secure way to use WireGuard and 2FA
$ aqua g -i tailscale/tailscale

Note that the latest version of tailscale is different by OS.

macOS - 1.72.2
Linux - 1.72.1

And macOS doesn't have 1.72.1, so I added 1.72.0 to pkg.yaml.

repo: aquaproj/aqua-registry


text:
Update docs to have PATH info, strip some trailing whitespace

Signed-off-by: Ashton Von Honnecke ashton@pixelstub.com

closing due to inactivity

authors: ahonnecke owenrumney

repo: aquasecurity/tfsec


text:
Fix typos in threat-alert-email-to-owner

Fix the typos to unblock #2078 and in the end #2070.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

#2084

superseded by https://github.com/aquasecurity/tfsec/pull/2100


text:
Change nvidia/cuda base image to 11.4.1

Describe your changes and why you are making these changes
This covers the common case use of using EKS with GPU configuration. The follow-up longterm fix is tracked here.
Related issue number (if any)
Eng-2483
Loom demo (if any)
Checklist before requesting a review

[x] I have created a descriptive PR title. The PR title should complete the sentence "This PR...".
[x] I have performed a self-review of my code.
[ ] I have included a small demo of the changes. For the UI, this would be a screenshot or a Loom video.
[ ] If this is a new feature, I have added unit tests and integration tests.
[ ] I have run the integration tests locally and they are passing.
[ ] I have run the linter script locally (See python3 scripts/run_linters.py -h for usage).
[ ] All features on the UI continue to work correctly.
[ ] Added one of the following CI labels:

run_integration_test: Runs integration tests
skip_integration_test: Skips integration tests (Should be used when changes are ONLY documentation/UI)

Awesome, thanks @hsubbaraj-spiral! Let me know once these images are pushed.

authors: hsubbaraj-spiral vsreekanti

repo: aqueducthq/aqueduct


text:
Document Expansion by Query Prediction

一言でいうと
文書から文書を検索する際に使われそうなクエリを予測して、文書＋予測クエリでインデックスを作成しておくことで検索性能を上げるという研究。非常にシンプルな手法ながら、MS MARCOの関連箇所検索(Passage Retrieval)でSOTAの性能。

論文リンク
https://arxiv.org/abs/1904.08375
著者/所属機関
Rodrigo Nogueira, Wei Yang, Jimmy Lin, Kyunghyun Cho

New York University
University of Waterloo
Facebook AI Research
CIFAR Azrieli Global Scholar

投稿日付(yyyy/MM/dd)
2019/4/17
概要
新規性・差分
手法
結果
コメント

実装
https://github.com/nyu-dl/dl4ir-doc2query

repo: arXivTimes/arXivTimes

license: mit

license_source: bigquery


text:
Array Usage In Next-Translate

Hello. I'm trying to do a language translation in my Next JS project. I have a configuration in my project as you can see below. I wanted to list my data here like the map method in JavaScript. But I could not succeed in this. How can I add my translations in the array to my project sequentially like in the map method?
{
"example": [
{
"title": "Title 1",
"description": "Desc 1"
},
{
"title": "Title 2",
"description": "Desc 2"
}
]
}
// An example usage in the project.
 {t("common:example.title", { returnObjects: true })}
// And this is how it looks on my page:
common:example.title (Yes. It looks exactly like this.)
I also tried to apply the methods mentioned in article 7 of the documentation. But I didn't get any result. Where do you think I am doing wrong? @aralroca

Thanks for your response. But I have the same problem when I use it as you mentioned. In addition, when I use it as you mentioned, I need to print a title value in the h2 tag and the desc value in the p tag, but I think I can't use it separately.
For now, if I can find a way that I can use, I will try to add each object in these arrays to the contatants folder by giving a separate name and call it with the map method. If I can't do that, unfortunately, my code count will increase.
I expect you to either improve your project to call such arrays within a map method, or develop a different method, as ChatGPT showed me.
// ChatGPT Example
import { useTranslation } from 'next-translate'
function Menu() {
const { t } = useTranslation('common')
return (

{t('menuItems', { returnObjects: true }).map((item) => (

{item.title}

))}

)
}
export default Menu
Actually, I think this type of usage can give excellent results for arrays. Seems like a very sensible use to me. Of course, I don't know much about the compatibility of this method with your project, unfortunately.
I wonder if there is another, shorter method I can use temporarily? I have to submit the project as soon as possible. Unfortunately, my time is somewhat limited. If there is no method for this, unfortunately, I will have to rearrange the codes. @aralroca

Sorry, but I don't understand your problem. What is wrong in this code?
import { useTranslation } from 'next-translate'

function Menu() {
  const { t } = useTranslation('common')
  const items = t('menuItems', { returnObjects: true })

  return (

      {items.map((item) => (

          {item.title}

      ))}

  )
}

export default Menu

Hello @aralroca . In order for you to better understand my problem, I deployed a simple version to the server and added my Github repo here .
Unfortunately, I couldn't solve my problem with the way you mentioned. Probably there is an error in my codes. If you are available, can you review it?

Mr. Aral, is there any solution for this problem? @aralroca

I also have a problem with array.

@joshuadesigner05 This problem is not related with the array, is related that you are trying to get translations in different languages at the same time. For that, you can overwrite the loadLocaleFrom:
{
  // ...rest of config
  "loadLocaleFrom": async (lang, ns) => ({
     es: await import(`./locales/es/${ns}.json`).then((m) => m.default),
     en: await import(`./locales/en/${ns}.json`).then((m) => m.default)
   }),
}

And then get for all languages:
const menuItemEN = t('en.menuItems', { returnObjects: true })
const menuItemES = t('es.menuItems', { returnObjects: true })
// ...

@aralroca I currently have a similar, very weird problem:
const { t } = useTranslation('common');
  const offerItems = t(
    'components.offers',
    {},
    { returnObjects: true }
  ); // will be an array of objects including properties `id` and `title`

This works but only on the first load. When the user refreshes, it will break -> if I log offerItems there will only be the key ('components.offers'). I only have one namespace and language yet.
i18n.json:
{
  "locales": ["de"],
  "defaultLocale": "de",
  "defaultNS": "common",
  "pages": {
    "*": ["common"]
  }
}

@sandrooco in which next-translate & next-translate-plugin version?

This happens with 2.5.2 in both packages.

it happens at first load,you have to condition for check is not null and is array like this:
const { t } = useTranslation('common');
const offerItems = t(
'components.offers',
{},
{ returnObjects: true }
);
if(offerItems  && Array.isArray(offerItems  )){
//do something
}

repo: aralroca/next-translate


text:
Incorrect COUNT on empty collection. 18,446,744,073,709,551,616

My Environment

ArangoDB Version:        3.6.1.1
Storage Engine:             RocksDB
Deployment Mode:       Single Server
Deployment Strategy:   ArangoDB Starter
Configuration:               default
Infrastructure:               AWS
Operating System:        Ubuntu 18.04
Total RAM in your machine:        32Gb
Disks in use:        SSD
Used Package:              Ubuntu .deb

Component, Query & Data
Affected feature:
AQL query using web interface
AQL query (if applicable): N/A
AQL explain (if applicable):
Dataset:
Any
Size of your Dataset on disk: Unknown
Number of records upto 20 000 000
Replication Factor & Number of Shards (Cluster only): N/A
Steps to reproduce

Created collection of several millions documents
All records were removed using AQL with LIMIT 200000 in several cycles.

Problem:
After removing all records page shows
18,446,744,073,709,551,616 doc(s)

Adding new documents to empty collection doesn't make any difference.

Running AQL on this empty collection
 return COUNT(sections)
Query String (22 chars, cacheable: false):
return COUNT(sections)

Execution plan:
Id   NodeType          Est.   Comment
   1   SingletonNode        1   * ROOT
   2   CalculationNode      1     - LET #0 = COLLECTION_COUNT("sections")   /* simple expression */
   3   ReturnNode           1     - RETURN #0

 Indexes used:
 none

 Functions used:
 Name               Deterministic   Cacheable   Uses V8
 COLLECTION_COUNT   false           false       false

 Optimization rules applied:
  none

Query String (22 chars, cacheable: false):
 return COUNT(sections)

Execution plan:
 Id   NodeType          Calls   Items   Runtime [s]   Comment
  1   SingletonNode         1       1       0.00000   * ROOT
  2   CalculationNode       1       1       0.00001     - LET #0 = COLLECTION_COUNT("sections")   /* simple expression */
  3   ReturnNode            1       1       0.00000     - RETURN #0

Indexes used:
 none

Optimization rules applied:
 none

Query Statistics:
 Writes Exec   Writes Ign   Scan Full   Scan Index   Filtered   Exec Time [s]
           0            0           0            0          0         0.00020

Query Profile:
 Query Stage           Duration [s]
 initializing               0.00000
 parsing                    0.00004
 optimizing ast             0.00001
 loading collections        0.00001
 instantiating plan         0.00001
 optimizing plan            0.00004
 executing                  0.00003
 finalizing                 0.00005

Produces result:
[
18446744073709552000
]
Expected result:
[
0
]
Running
return COUNT(FOR s IN  sections RETURN s)
Produces result as expected:
[
0
]
NOTE:
Trancating empty collections fixes the issue.

I have tried to replicate this issue creating 20M records.
1. created two simultaneously running threads.
Thread 1:
AQL:
FOR i IN 1..10000000
INSERT {'index': i, 'tag': '1'} IN sections

Thread 2:
AQL:
FOR i IN 1..10000000
INSERT {'index': i, 'tag': '1'} IN sections

2. Another two threads running simultaneously
Thread1:  running cycles until collection is empty
AQL:
RETURN LENGTH(
 FOR s IN sections
 FILTER s.tag=='1'
 LIMIT 2000000
 REMOVE s IN sections RETURN OLD
)

Thread2: running cycles until collection is empty
AQL:
RETURN LENGTH(
 FOR s IN sections
 FILTER s.tag=='2'
 LIMIT 2000000
 REMOVE s IN sections RETURN OLD
)

When done, during next 10-15 min on attempt to open this collection I always see "Fetching documents" dialog for a couple of seconds, like ArangoDB is processing some data.

This time it doesn't return bad count though.
I also investigated log file. I see  ArangoDB crashed couple of times during the original process of removing documents, which created COUNT issue.
2020-02-09T00:07:48Z [2505] WARNING [3ad54] {engines} slow background settings sync: 2.096967 s
2020-02-09T00:07:51Z [2505] WARNING [8bcee] {queries} slow query: 'RETURN LENGTH(FOR coll IN @@collection FILTER coll.catalogKey==@catalogKey LIMIT 200000 REMOVE coll IN @@collection RETURN OLD)', bind vars: {"@collection":"option_groups_option_group_details","catalogKey":"9"}, took: 58.003240 s
2020-02-09T00:08:05Z [2505] WARNING [8bcee] {queries} slow query: 'RETURN LENGTH(FOR coll IN @@collection FILTER coll.catalogKey==@catalogKey LIMIT 200000 REMOVE coll IN @@collection RETURN OLD)', bind vars: {"@collection":"options_option_group_details","catalogKey":"10"}, took: 52.098410 s
2020-02-09T00:08:49Z [2505] WARNING [8bcee] {queries} slow query: 'RETURN LENGTH(FOR coll IN @@collection FILTER coll.catalogKey==@catalogKey LIMIT 200000 REMOVE coll IN @@collection RETURN OLD)', bind vars: {"@collection":"option_groups_option_group_details","catalogKey":"9"}, took: 58.491383 s
2020-02-09T00:09:11Z [9487] INFO [e52b0] ArangoDB 3.6.1 [linux] 64bit, using jemalloc, build tags/v3.6.1-0-gfef8223049, VPack 0.1.33, RocksDB 6.2.0, ICU 58.1, V8 7.1.302.28, OpenSSL 1.1.1d  10 Sep 2019
2020-02-09T00:09:11Z [9487] INFO [75ddc] detected operating system: Linux version 4.15.0-38-generic (buildd@lcy01-amd64-023) (gcc version 7.3.0 (Ubuntu 7.3.0-16ubuntu3)) #41-Ubuntu SMP Wed Oct 10 10:59:38 UTC 2018
2020-02-09T00:09:11Z [9487] WARNING [118b0] {memory} maximum number of memory mappings per process is 192000, which seems too low. it is recommended to set it to at least 256000
2020-02-09T00:09:11Z [9487] WARNING [49528] {memory} execute 'sudo sysctl -w "vm.max_map_count=256000"'
2020-02-09T00:09:11Z [9487] INFO [43396] {authentication} Jwt secret not specified, generating...
2020-02-09T00:09:11Z [9487] INFO [144fe] using storage engine 'rocksdb'
2020-02-09T00:09:11Z [9487] INFO [3bb7d] {cluster} Starting up with role SINGLE
2020-02-09T00:09:11Z [9487] INFO [6ea38] using endpoint 'http+tcp://10.0.1.33:8529' for non-encrypted requests
2020-02-09T00:09:11Z [9487] INFO [a1c60] {syscall} file-descriptors (nofiles) hard limit is 131072, soft limit is 131072
2020-02-09T00:09:11Z [9487] INFO [3844e] {authentication} Authentication is turned on (system only), authentication for unix sockets is turned on
2020-02-09T00:09:11Z [9487] WARNING [b387d] found existing lockfile '/vol/arangodb3-rocksdb/LOCK' of previous process with pid 2505, but that process seems to be dead already
2020-02-09T00:09:14Z [9487] INFO [cf3f4] ArangoDB (version 3.6.1 [linux]) is ready for business. Have fun!
2020-02-09T00:05:05Z [2505] WARNING [8bcee] {queries} slow query: 'RETURN LENGTH(FOR coll IN @@collection FILTER coll.catalogKey==@catalogKey LIMIT 200000 REMOVE coll IN @@collection RETURN OLD)', bind vars: {"@collection":"option_groups_option_group_details","catalogKey":"9"}, took: 44.714396 s
2020-02-09T00:05:36Z [2505] WARNING [8bcee] {queries} slow query: 'RETURN LENGTH(FOR coll IN @@collection FILTER coll.catalogKey==@catalogKey LIMIT 200000 REMOVE coll IN @@collection RETURN OLD)', bind vars: {"@collection":"options_option_group_details","catalogKey":"10"}, took: 47.412764 s
2020-02-09T00:06:04Z [2505] WARNING [8bcee] {queries} slow query: 'RETURN LENGTH(FOR coll IN @@collection FILTER coll.catalogKey==@catalogKey LIMIT 200000 REMOVE coll IN @@collection RETURN OLD)', bind vars: {"@collection":"option_groups_option_group_details","catalogKey":"9"}, took: 58.308028 s
2020-02-09T00:06:22Z [2505] WARNING [8bcee] {queries} slow query: 'RETURN LENGTH(FOR coll IN @@collection FILTER coll.catalogKey==@catalogKey LIMIT 200000 REMOVE coll IN @@collection RETURN OLD)', bind vars: {"@collection":"options_option_group_details","catalogKey":"10"}, took: 46.224647 s
2020-02-09T00:06:53Z [2505] WARNING [8bcee] {queries} slow query: 'RETURN LENGTH(FOR coll IN @@collection FILTER coll.catalogKey==@catalogKey LIMIT 200000 REMOVE coll IN @@collection RETURN OLD)', bind vars: {"@collection":"option_groups_option_group_details","catalogKey":"9"}, took: 48.885522 s
2020-02-09T00:07:13Z [2505] WARNING [8bcee] {queries} slow query: 'RETURN LENGTH(FOR coll IN @@collection FILTER coll.catalogKey==@catalogKey LIMIT 200000 REMOVE coll IN @@collection RETURN OLD)', bind vars: {"@collection":"options_option_group_details","catalogKey":"10"}, took: 51.219891 s
2020-02-09T00:07:48Z [2505] WARNING [3ad54] {engines} slow background settings sync: 2.096967 s
2020-02-09T00:07:51Z [2505] WARNING [8bcee] {queries} slow query: 'RETURN LENGTH(FOR coll IN @@collection FILTER coll.catalogKey==@catalogKey LIMIT 200000 REMOVE coll IN @@collection RETURN OLD)', bind vars: {"@collection":"option_groups_option_group_details","catalogKey":"9"}, took: 58.003240 s
2020-02-09T00:08:05Z [2505] WARNING [8bcee] {queries} slow query: 'RETURN LENGTH(FOR coll IN @@collection FILTER coll.catalogKey==@catalogKey LIMIT 200000 REMOVE coll IN @@collection RETURN OLD)', bind vars: {"@collection":"options_option_group_details","catalogKey":"10"}, took: 52.098410 s
2020-02-09T00:08:49Z [2505] WARNING [8bcee] {queries} slow query: 'RETURN LENGTH(FOR coll IN @@collection FILTER coll.catalogKey==@catalogKey LIMIT 200000 REMOVE coll IN @@collection RETURN OLD)', bind vars: {"@collection":"option_groups_option_group_details","catalogKey":"9"}, took: 58.491383 s
2020-02-09T00:09:11Z [9487] INFO [e52b0] ArangoDB 3.6.1 [linux] 64bit, using jemalloc, build tags/v3.6.1-0-gfef8223049, VPack 0.1.33, RocksDB 6.2.0, ICU 58.1, V8 7.1.302.28, OpenSSL 1.1.1d  10 Sep 2019
2020-02-09T00:09:11Z [9487] INFO [75ddc] detected operating system: Linux version 4.15.0-38-generic (buildd@lcy01-amd64-023) (gcc version 7.3.0 (Ubuntu 7.3.0-16ubuntu3)) #41-Ubuntu SMP Wed Oct 10 10:59:38 UTC 2018
2020-02-09T00:09:11Z [9487] WARNING [118b0] {memory} maximum number of memory mappings per process is 192000, which seems too low. it is recommended to set it to at least 256000
2020-02-09T00:09:11Z [9487] WARNING [49528] {memory} execute 'sudo sysctl -w "vm.max_map_count=256000"'
2020-02-09T00:09:11Z [9487] INFO [43396] {authentication} Jwt secret not specified, generating...
2020-02-09T00:09:11Z [9487] INFO [144fe] using storage engine 'rocksdb'
2020-02-09T00:09:11Z [9487] INFO [3bb7d] {cluster} Starting up with role SINGLE
2020-02-09T00:09:11Z [9487] INFO [6ea38] using endpoint 'http+tcp://10.0.1.33:8529' for non-encrypted requests
2020-02-09T00:09:11Z [9487] INFO [a1c60] {syscall} file-descriptors (nofiles) hard limit is 131072, soft limit is 131072
2020-02-09T00:09:11Z [9487] INFO [3844e] {authentication} Authentication is turned on (system only), authentication for unix sockets is turned on
2020-02-09T00:09:11Z [9487] WARNING [b387d] found existing lockfile '/vol/arangodb3-rocksdb/LOCK' of previous process with pid 2505, but that process seems to be dead already
2020-02-09T00:09:14Z [9487] INFO [cf3f4] ArangoDB (version 3.6.1 [linux]) is ready for business. Have fun!

Regarding

When done, during next 10-15 min on attempt to open this collection I always see "Fetching documents" dialog for a couple of seconds, like ArangoDB is processing some data.

this is somewhat expected if a collection contains a lot of deleted documents. Note that a document deletion will add "removal" markers to the collection which are eventually cleaned up in compaction runs. Thus scanning collections with lots of removed documents can take longer than expected until the compaction has actually taken place and has removed not only the obsolete documents but also the removal markers.

The value of 18,446,744,073,709,551,616 documents is very likely caused by an underflow in the "number of documents" counter, which is kept internally for performance reasons.
The maximum value of that counter is 18,446,744,073,709,551,615 and the number you reported is sufficiently close to it to make me believe this.
The internal "number of documents" counter should of course always be in sync with the actual number of documents in the underlying collection, so I guess due to one of the crashes you mentioned the counter somehow got out of sync.
Regarding the reasons of these crashes I am not sure, but it could be that the arangod process simply got OOM-killed by the operating system. A query such as
RETURN LENGTH(
 FOR s IN sections
 FILTER s.tag=='2'
 LIMIT 2000000
 REMOVE s IN sections RETURN OLD
)

will create a huge in-memory result, and depending on RAM availability/scarcity the operating system may choose to kill processes which consume a lot of RAM. This could be found out by checking the OS logs for any invocation of the OOM killer.

Actually, real query that crashed arango was (based on log )
RETURN LENGTH(
  FOR coll IN @@collection
  FILTER coll.catalogKey==@catalogKey
  LIMIT 200000 REMOVE coll IN @@collection
RETURN OLD)',

It was slow, however I am deleting only 200K documents at a time. I think It is not that many.
Collection is indexed by catalogKey and it still takes 58.491383 s to remove
I also noticed that CPU usage is 100% during call. Is there any way to remove documents efficiently without crashing arangodb?

The query
RETURN LENGTH(
  FOR coll IN @@collection
  FILTER coll.catalogKey==@catalogKey
  LIMIT 200000 REMOVE coll IN @@collection
RETURN OLD)

will build up a huge in-memory result containing all the removed documents, but actually they are not needed for the query's final result, which is just a number.
Changing it to just
FOR coll IN @@collection
  FILTER coll.catalogKey==@catalogKey
  LIMIT 200000 REMOVE coll IN @@collection

and then looking at the query statistics (which are also returned as part of a query should be much more efficient.
The number of removed documents can be found in the writesExecuted attribute of the stats sub attribute of the the extra sub attribute, e.g.
$ curl -X POST http://127.0.0.1:8529/_api/cursor --user "root:" --data '{"query":"FOR coll IN @@collection LIMIT 2000000 REMOVE coll IN @@collection","bindVars":{"@collection":"test"}}' --dump -; echo
HTTP/1.1 201 Created
X-Content-Type-Options: nosniff
Server: ArangoDB
Connection: Keep-Alive
Content-Type: application/json; charset=utf-8
Content-Length: 274

{"result":[],"hasMore":false,"cached":false,"extra":{"stats":{"writesExecuted":1000012,"writesIgnored":0,"scannedFull":0,"scannedIndex":1000012,"filtered":0,"httpRequests":0,"executionTime":15.994641065597534,"peakMemoryUsage":98392},"warnings":[]},"error":false,"code":201}

Closing the issue for now. It is better to avoid problem by optimizing queries that are causing huge memory and CPU impact.

authors: alexanderkunin jsteemann

repo: arangodb/arangodb


text:
Please Please give .tar compiled version for arangodb like mongodb.

Hello , i am trying to understand the filesystem and all the programs and file associated with arangodb , but i don't know where you guys have stored it using .deb package , it is becoming more less controlled , and thats why is it very less likely that i am going to use. please make compiled .tar version which includes all the files and programs , so , i can do all the work easily and manually , with full control. trust me , people needs it for better exploration and control. thank you in advance guys.

you probably know that you can simply use alien -t to generate tarballs from .debs?
The clear advantage of .debs (or rpms) are that they will manage dependencies and stop you from running into subtle bugs from libraries having wrong symbols etc.
They also manage stuff like adding users, registering init scripts, setting file permissions and doing a good cleanup on removal.
Its rather heading in the different direction, users want complete VM images (that maintain an easy upgrade paths) or automated meta scripts like the docker library that will install ArangoDB.
They're all very good reason why most of the Linux world evolved from Slackwares system of simply rolling out tarballs.

With the installed .deb file you can do
dpkg -L arangodb
to see all installed files.
You can even do
dpkg --info <.deb file>
(if I remember correctly) without installing it. And finally, a .deb is just a cpio archive containing a .tar.gz and some more stuff, as far as I remember.

More short:
ar vx arangodb_*_amd64.deb
gives you: data.tar.gz

authors: dothebart neunhoef webgini


text:
Invalid ubuntu package name in download preview page (ubuntu section)

Hello,
Download and Install a Technical Preview (ubuntu section):

sudo apt-get install arangodb=3.0.0r3

Should be:

sudo apt-get install arangodb3=3.0.0r3

fixed, thanks for reporting!

authors: dothebart khamin


text:
Bug fix 3.7/remove sonderlocke

Scope & Purpose
Remove a Sonderlocke for patching double value in existing VPack chunks.
There was a potential optimized code branch for platforms that allow unaligned writes, but this only made the code more complex and can cause false-positive warnings with UBSan. So let's simplify this and rely on compilers to properly optimize it. It is called rarely anway.

[x] Bug-Fix for devel-branch (i.e. no need for backports?)
[x] The behavior change can be verified via automatic tests

Testing & Verification
This PR adds tests that were used to verify all changes:

[x] Added new C++ Unit Tests (Either GoogleTest or Catch-Test)

http://172.16.10.101:8080/view/PR/job/arangodb-matrix-pr/10846/

http://172.16.10.101:8080/view/PR/job/arangodb-matrix-pr/10848/

https://172.16.10.101/job/arangodb-matrix-pr/10851/

https://172.16.10.101/job/arangodb-matrix-pr/10852/

Tests blue


text:
[Pregel] Add recovering to conductor state machine

Fifth PR for adding a state machine to the pregel conductor.
Adds InError and Recovering states to conductor state machine. Recovering is triggered from outside and is just allowed when state is Running or InError, therefore I created a new state-machine method recover(), which starts the Recovery state only from InError and Running (not yet implemented) state.

TODO merge parent first


text:
SEARCH-342 fix includeAllFields handling, backport to 3.10

Scope & Purpose
(cherry picked from commit 1c2577b29a02a0e856804c87ab966c33ccc47f03)

[ ] :hankey: Bugfix
[ ] :pizza: New feature
[ ] :fire: Performance improvement
[ ] :hammer: Refactoring/simplification

Checklist

[ ] Tests

[ ] Regression tests
[ ] C++ Unit tests
[ ] integration tests
[ ] resilience tests

[ ] :book: CHANGELOG entry made
[ ] :books: documentation written (release notes, API changes, ...)
[ ] Backports

[ ] Backport for 3.10: (Please link PR)
[ ] Backport for 3.9: (Please link PR)
[ ] Backport for 3.8: (Please link PR)

Related Information
(Please reference tickets / specification / other PRs etc)

[ ] Docs PR:
[ ] Enterprise PR:
[ ] GitHub issue / Jira ticket:
[ ] Design document:

Only unrelated known failure on Windows in Jenkins.


text:
remove now-obsolete warning

Scope & Purpose
Remove storage engine warning, as requested in planning/#4407

[ ] Bug-Fix for devel-branch (i.e. no need for backports?)
[x] Bug-Fix for a released version (did you remember to port this to all relevant release branches?)
[x] The behavior in this PR can be (and was) manually tested (support / qa / customers can test it)

Related Information

[x] There is an internal planning ticket: https://github.com/arangodb/planning/issues/4407

Testing & Verification
This change is a trivial rework / code cleanup without any test coverage.
https://172.16.10.101/view/PR/job/arangodb-matrix-pr/5861/

Tests blue except unrelated failure.


text:
Add commas

Add commas to introductory clauses.
Add oxford comma. [^1]

[^1]: Strippers, JFK, Stalin, and the Oxford Comma

Strange. 🤔
There should be a CLA from me on file already. Perhaps it wasn't added to the cla-bot. On Saturday February 17th 2018 at 11:57 PM (CST), I sent my signed copy to cla@arangodb.com. As you can see from the screenshot below, I received a reply several days later on February 28th from Santo Leto.
@cw00dw0rd - Please let me know how I should proceed on this.
Cheers.

Email Details
Date: Sat, 17 Feb 2018 23:57:38 -0600
From: Matthew Petty <matt@kizmeta.com>
To: cla@arangodb.com
Message-ID: <600d42d8-4d42-4eb7-b9ba-7955fbb1a5a9@Spark>
Subject: CLA Matt Petty

Content-Type: application/octet-stream
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="CLA-signed.pdf"

Gmail Screenshot
I received this reply to my initial email, to which I replied with my Github username.

@cla-bot check

@lodestone thanks again for these updates, would you mind adding these fixes to the other versions as well (3.6, 3.7, etc..)?

... would you mind adding these fixes to the other versions as well (3.6, 3.7, etc..)?

I've updated files also named foxx-guides-browser.md in other docs/3.x directories.
@cw00dw0rd let me know if any additional work is needed!
Cheers

Thanks again @lodestone for the fixes!

repo: arangodb/docs


text:
String centralization/Localization support

I don't remember how it used to be done, but at least I think we can use a Resource Dictionary for strings. This would also make it easier to rename things in the future: we only make changes in that file

@flalexp what do we do with the qgrep strings?

authors: aranhil flalexp

repo: aranhil/qgrepSearchTool


text:
uperf plugin fails to build with docker-compose

Describe the bug
uperf plugin builds correctly with podman-compose but fails to build with docker-compose.
To reproduce
With podman-compose:
...
[2/2] COMMIT arcaflow-uperf
--> cc90295e6c9
Successfully tagged localhost/arcaflow-uperf:latest
cc90295e6c9408cd43612b8a4d86e1fb4c99abc5147c1476d7b064f5034a7bc9
exit code: 0

$ podman images
REPOSITORY                TAG         IMAGE ID      CREATED             SIZE
localhost/arcaflow-uperf  latest      cc90295e6c94  10 seconds ago      631 MB

With docker-compose:
Building server
[+] Building 8.4s (9/18)
 => [internal] load build definition from Dockerfile                       0.1s
 => => transferring dockerfile: 37B                                        0.0s
 => [internal] load .dockerignore                                          0.0s
 => => transferring context: 2B                                            0.0s
 => [internal] load metadata for quay.io/centos/centos:stream8             0.9s
 => [stage-1  1/11] FROM quay.io/centos/centos:stream8@sha256:de7327910e7  0.0s
 => [internal] load build context                                          0.0s
 => => transferring context: 147B                                          0.0s
 => CACHED [stage-1  2/11] RUN dnf module -y install python39 && dnf inst  0.0s
 => CACHED [builder 2/4] RUN dnf install -y git make gcc lksctp-tools-dev  0.0s
 => CACHED [builder 3/4] RUN git clone -b 1.0.7 https://github.com/uperf/  0.0s
 => ERROR [builder 4/4] RUN cd /uperf && ./configure && make && make inst  7.3s
------
 > [builder 4/4] RUN cd /uperf && ./configure && make && make install:
 ...
#8 7.231 CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/sh /uperf/missing aclocal-1.16
#8 7.234 /uperf/missing: line 81: aclocal-1.16: command not found
#8 7.238 WARNING: 'aclocal-1.16' is missing on your system.
#8 7.238          You should only need it if you modified 'acinclude.m4' or
#8 7.238          'configure.ac' or m4 files included by 'configure.ac'.
#8 7.238          The 'aclocal' program is part of the GNU Automake package:
#8 7.238          <https://www.gnu.org/software/automake>
#8 7.238          It also requires GNU Autoconf, GNU m4 and Perl in order to run:
#8 7.238          <https://www.gnu.org/software/autoconf>
#8 7.238          <https://www.gnu.org/software/m4/>
#8 7.238          <https://www.perl.org/>
#8 7.239 make: *** [Makefile:329: aclocal.m4] Error 127
------
executor failed running [/bin/sh -c cd /uperf && ./configure && make && make install]: exit code: 2
ERROR: Service 'server' failed to build : Build failed

Additional context
$ uname -a
Darwin jlema-mac 21.6.0 Darwin Kernel Version 21.6.0: Wed Aug 10 14:25:27 PDT 2022; root:xnu-8020.141.5~2/RELEASE_X86_64 x86_64

$ docker version
Client:
 Cloud integration: v1.0.25
 Version:           20.10.16
 API version:       1.41
 Go version:        go1.17.10
 Git commit:        aa7e414
 Built:             Thu May 12 09:20:34 2022
 OS/Arch:           darwin/amd64
 Context:           default
 Experimental:      true

Server: Docker Desktop 4.9.0 (80466)
 Engine:
  Version:          20.10.16
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.17.10
  Git commit:       f756502
  Built:            Thu May 12 09:15:42 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.4
  GitCommit:        212e8b6fa2f44b9c21b2798135fc6fb7c53efc16
 runc:
  Version:          1.1.1
  GitCommit:        v1.1.1-0-g52de29d
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

@jaredoconnell

Adding automake in https://github.com/arcalot/arcaflow-plugins/blob/main/python/uperf/Dockerfile#L2 does the trick. Preparing a PR.

repo: arcalot/arcaflow-plugins


text:
XML libraries not working with VS Code Diagrams.net Plugin

Workaround: Use older version in branch last_working_version.
Potential cause: a new XML export format for libraries introduced with some draw.io version newer than the one used in the plugin.
Fix 1: Draw.io plugin updates the version of embedded draw.io (not sure if this is really the problem).
Fix 2: Use older drawio desktop version to generate library XML. Tested to be working with drawio 20.2.8.

Since Fix 1 requires actions by the maintainer of the draw.io plugin, I applied fix 2.


text:
a-table有无doubleclick事件

目前有row-click鼠标单击事件，能否增加一个row-dblclick鼠标双击事件

另外建议row-click提供3个参数：
record: TableData
index: seq
ev: Event
比如序号相关数据不一定存在row里面的。。。

我们也遇到这个问题，没有双击事件很不方便，还请开发组暴露一下，谢谢！

repo: arco-design/arco-design-vue


text:
switch切换组件样式不正确，底色直接展示背景色了，白色背景色下看不见整个按钮

[x] I'm sure this does not appear in the issue list of the repository

Basic Info

Package Name And Version: @arco-design/web-vue@2.43.0
Framework version: vue3
Browser: chrome110.0.0.0

What is expected?
应当跟官网一致，现在是完全看不见按钮的背景色

Steps to reproduce
<a-card>
    <a-switch />
</a-card>

考虑是否有默认样式覆盖

除了这个我只引入了tailwind css，用的是acro vue pro
---- 回复的原邮件 ----
| 发件人 | @.> |
| 日期 | 2023年03月07日 15:36 |
| 收件人 | @.> |
| 抄送至 | @.>@.> |
| 主题 | Re: [arco-design/arco-design-vue] switch切换组件样式不正确，底色直接展示背景色了，白色背景色下看不见整个按钮 (Issue #2206) |
考虑是否有默认样式覆盖
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>

而且现象是他的底色跟随背景色，感觉样式覆盖应该是固定的
---- 回复的原邮件 ----
| 发件人 | @.> |
| 日期 | 2023年03月07日 15:36 |
| 收件人 | @.> |
| 抄送至 | @.>@.> |
| 主题 | Re: [arco-design/arco-design-vue] switch切换组件样式不正确，底色直接展示背景色了，白色背景色下看不见整个按钮 (Issue #2206) |
考虑是否有默认样式覆盖
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>

考虑覆盖样式是不是把背景色变成透明

我先关闭了，目前我手动设置组件底色属性了，不过没排查到有样式覆盖。因为我这边没有手写的css，除了引入tailwindcss

我现在也有这个问题,感觉是 和 tailwindcss 冲突了

#759 里面其实提到的也是这个问题，tailwindcss的引用放在arco css引用下方就有这个问题，调转就没有，但是调转以后arco的样式高，会导致有时候设置tailwindcss的类不起作用

authors: Saber-Kurama minxiang51574 mojerro pzgz weishaoy


text:
a-table sticky-header属性无效

[x] I'm sure this does not appear in the issue list of the repository

基本信息

依赖包名 及 版本: @arco-design/web-vue@2.52.0
框架版本: vue@3.3.4
浏览器版本: chrome117.0.0.0
复现地址: https://arco.design/vue/component/table#sticky

预期结果
表头吸顶
复现步骤
打开链接，也就是官方文档就可以复现

最新版2.54.1也没有修复，是不是不维护了

这个问题22年就有人提了( #1852 ), 到现在还没有修复

authors: Cheerwhy cyrilluce starzeng


text:
级联选择器的搜索有问题，目前只能搜索叶子结点，这是非预期的

预期
input 有值的时候把 options 扁平化，例如输入南京或江苏，叶子节点的路径中国 / 江苏 / 南京都会出现在可选项中
// 原始值
const options = [
  {
    label: '中国',
    value: '01',
    children: [
      {
        label: '江苏',
        value: '0101',
        children: [
          {
            label: '南京',
            value: '010101',
          }
        ]
      },
      {
        label: '上海',
        value: '0102',
      }
    ]
  }
]

@DingRui12138  可以开启严格选择模式来让路径的可选择 https://arco.design/vue/component/cascader#check-strictly

@Flsion 不是路径选择，是路径支持搜索
// 源码（只筛选了叶子结点）
const filteredLeafOptions = computed(() =>
  Array.from(leafOptionSet).filter(
    (item) =>
      props.filterOption?.(computedInputValue.value, item) ??
      item.label
        ?.toLocaleLowerCase()
        .includes(computedInputValue.value?.toLocaleLowerCase())
  )
);

@DingRui12138 懂了，这个版本改一下


text:
google analytics set up

Google analytics is now configured. You may have to configure the below in Google analytics though. Source
This plugin will fire a new event called gatsby-route-change (or as in the gatsby-config.js configured routeChangeEventName) whenever a route is changed in your Gatsby application. To record this in Google Tag Manager, we will need to add a trigger to the desired tag to listen for the event:
Visit the Google Tag Manager console and click on the workspace for your site.
Navigate to the desired tag using the ‘Tags’ tab of the menu on the right hand side.
Under “Triggering”, click the pencil icon, then the ”+” button to add a new trigger.
In the “Choose a trigger” window, click on the ”+” button again.
Choose the trigger type by clicking the pencil button and clicking “Custom event”. For event name, enter gatsby-route-change (or as in the gatsby-config.js configured routeChangeEventName).
This tag will now catch every route change in Gatsby, and you can add Google tag services as you wish to it.

This seems helpful as well: https://andre-wibbeke.dev/posts/how-to-add-google-analytics-to-your-gatsby-site-over-google-tag-manager

This seems helpful as well: https://andre-wibbeke.dev/posts/how-to-add-google-analytics-to-your-gatsby-site-over-google-tag-manager

authors: ardalis ilyanaDev

repo: ardalis/DevIQ-gatsby


text:
How to use another router?

Hello! No issues with httptreemux, nor am I suggesting it be replaced. I'm just more familiar with Chi and was curious if it was even possible to use it instead? In lecture video 5.6, its explained how httptreemux returns a map in the Params function. I'm not certain that Chi behaves the same way under the hood. At first glace, this is how the two differ when it comes to defining parameters:
httptreemux: /api/v1/post/:postid
chi: /api/v1/post/{postid}
httptreemux  returns a map of string, chi doesn't have a  function that behaves like that. I think the closest would be chi.URLParamFromCtx, but that doesn't seem right either.  If anyone here is familiar with Chi and has used in it their own branch of the project, do you have any suggestions?
request.go - old
// Params returns the web call parameters from the request.
func Params(r *http.Request) map[string]string {
	return httptreemux.ContextParams(r.Context())
}

request.go - new
func Params(ctx context.Context, key string) string {
	return chi.URLParamFromCtx(ctx, key)
}

No issues in the app.go file:
app. go
type App struct {
	mux      *chi.Mux
	otmux    http.Handler
	shutdown chan os.Signal
	mw       []Middleware
}

func NewApp(shutdown chan os.Signal, mw ...Middleware) *App {
	mux := chi.NewRouter()

	return &App{
		mux:      mux,
		otmux:    otelhttp.NewHandler(mux, "request"),
		shutdown: shutdown,
		mw:       mw,
	}
}

func (a *App) handle(debug bool, method string, path string, handler Handler, mw ...Middleware) {
	if debug {
		if _, exists := registered[method+path]; exists {
			return
		}
		registered[method+path] = true
	}

	handler = wrapMiddleware(mw, handler)

	handler = wrapMiddleware(a.mw, handler)

	h := func(w http.ResponseWriter, r *http.Request) {

		ctx := r.Context()
		ctx, span := trace.SpanFromContext(ctx).Tracer().Start(ctx, r.URL.Path)
		defer span.End()

		v := Values{
			TraceID: span.SpanContext().TraceID().String(),
			Now:     time.Now(),
		}
		ctx = context.WithValue(ctx, KeyValues, &v)

		if err := handler(ctx, w, r); err != nil {
			a.SignalShutdown()
			return
		}
	}

	if debug {
		f := func(w http.ResponseWriter, r *http.Request) {
			switch {
			case r.Method == method:
				h(w, r)
			default:
				w.WriteHeader(http.StatusNotFound)
			}
		}
		http.DefaultServeMux.HandleFunc("/debug"+path, f)
		return
	}
	a.mux.MethodFunc(method, path, h)
}

The whole point of this code is that it’s NOT a framework so you can change out anything you want.
I think you want this API call instead
  userID := chi.URLParam(r, "userID")

So something like this
func Params(r *http.Request, key string) string {
	return chi.URLParam(r, key)
}

This is as close to what I have.
func Params(r *http.Request) chi.RouteParams {
	rctx := RouteContext(r.Context())
	return rctx.URLParams
}

Thanks for putting me on the the right track! I used your snippet, and there's an issue in /sales-api/handlers/user.go where we call web.Params:
line 33: pageNumber, err := strconv.Atoi(params["page"])
[compiler NonIndexableOperand] [E] invalid operation: cannot index params (variable of type chi.RouteParams)

I'm still learning my Go chops, but I think this is because before we were returning map[string]string for Params, and now Params is returning chi.RouteParams.  So I need to change the way I wrap ["page"] in user.go?

That code needs to change. You will need to use the methods for the RouteParams. The return type is different.

Looking at this more closely, you need this instead
chi.URLParam(r, key)
Change web.Params to be web.Param
func Param(r *http.Request, key string) string {
	return chi.URLParam(r, key)
}

Then this
pageNumber, err := strconv.Atoi(params["page"])

becomes
pageNumber, err := strconv.Atoi(web.Param(r, "page"))

Works like a charm! Thank you for taking the time to look at this for me!
Cheers

authors: ardan-bkennedy jonleopard

repo: ardanlabs/service