text:
bugfix for multiple shadow threads rendered at the same time, small probability crash.

Summary
bugfix for multiple shadow threads rendered at the same time, small probability crash.
Changelog
Android
Fixed
ViewManagersPropertyCache.class
Background:

4 tab(RN Page) has 4 ReactInstanceManager.
4 ReactInstanceManager has 4 shadow threads and 4 JS threads.
4 RN Page if rendered at the same time. small probability crash
the key crash log(full log at the end):
java.lang.IllegalArgumentException: method com.facebook.react.uimanager.LayoutShadowNode.setWidth argument 1 has type com.facebook.react.bridge.Dynamic, got java.lang.String
need Dynamic and got String.
Reasons of crash: PropSetter class field : VIEW_MGR_ARGS, VIEW_MGR_GROUP_ARGS, SHADOW_ARGS, SHADOW_GROUP_ARGS is static.
one shadow thread put data in static array, it was changed by another shadow thread before method invoke.
No.1 shadow thread put "Dynamic" in static array.
No.2 shadow thread put "String" in the same static array(replace the "Dynamic").
No.1 shadow  thread invoke method with the static array as params. the crash..
The solution:
use ThreadLocal instead of static array.
Test Plan
I make 4 tab page rendered at the same time and test 100 times. it is not crash again.
about 15% chance of crash before fixed.
Crash stack:
2021-09-08 11:56:07.392 16776-17062/com.shopeepay.merchant.id.debug E/unknown:ViewManager: Error while updating prop width
java.lang.IllegalArgumentException: method com.facebook.react.uimanager.LayoutShadowNode.setWidth argument 1 has type com.facebook.react.bridge.Dynamic, got java.lang.String
at java.lang.reflect.Method.invoke(Native Method)
at com.facebook.react.uimanager.ViewManagersPropertyCache$PropSetter.updateShadowNodeProp(ViewManagersPropertyCache.java:111)
at com.facebook.react.uimanager.ViewManagerPropertyUpdater$FallbackShadowNodeSetter.setProperty(ViewManagerPropertyUpdater.java:161)
at com.facebook.react.uimanager.ViewManagerPropertyUpdater.updateProps(ViewManagerPropertyUpdater.java:65)
at com.facebook.react.uimanager.ReactShadowNodeImpl.updateProperties(ReactShadowNodeImpl.java:320)
at com.facebook.react.uimanager.UIImplementation.createView(UIImplementation.java:251)
at com.facebook.react.uimanager.UIManagerModule.createView(UIManagerModule.java:469)
at java.lang.reflect.Method.invoke(Native Method)
at com.facebook.react.bridge.JavaMethodWrapper.invoke(JavaMethodWrapper.java:372)
at com.facebook.react.bridge.JavaModuleWrapper.invoke(JavaModuleWrapper.java:151)
at com.facebook.react.bridge.queue.NativeRunnable.run(Native Method)
at android.os.Handler.handleCallback(Handler.java:938)
at android.os.Handler.dispatchMessage(Handler.java:99)
at com.facebook.react.bridge.queue.MessageQueueThreadHandler.dispatchMessage(MessageQueueThreadHandler.java:27)
at android.os.Looper.loop(Looper.java:236)
at com.facebook.react.bridge.queue.MessageQueueThreadImpl$4.run(MessageQueueThreadImpl.java:226)
at java.lang.Thread.run(Thread.java:923)
2021-09-08 11:56:07.488 16776-17062/com.shopeepay.merchant.id.debug E/unknown:ReactNative: CatalystInstanceImpl caught native exception
com.facebook.react.bridge.JSApplicationIllegalArgumentException: Error while updating property 'width' in shadow node of type: RCTView
at com.facebook.react.uimanager.ViewManagersPropertyCache$PropSetter.updateShadowNodeProp(ViewManagersPropertyCache.java:125)
at com.facebook.react.uimanager.ViewManagerPropertyUpdater$FallbackShadowNodeSetter.setProperty(ViewManagerPropertyUpdater.java:161)
at com.facebook.react.uimanager.ViewManagerPropertyUpdater.updateProps(ViewManagerPropertyUpdater.java:65)
at com.facebook.react.uimanager.ReactShadowNodeImpl.updateProperties(ReactShadowNodeImpl.java:320)
at com.facebook.react.uimanager.UIImplementation.createView(UIImplementation.java:251)
at com.facebook.react.uimanager.UIManagerModule.createView(UIManagerModule.java:469)
at java.lang.reflect.Method.invoke(Native Method)
at com.facebook.react.bridge.JavaMethodWrapper.invoke(JavaMethodWrapper.java:372)
at com.facebook.react.bridge.JavaModuleWrapper.invoke(JavaModuleWrapper.java:151)
at com.facebook.react.bridge.queue.NativeRunnable.run(Native Method)
at android.os.Handler.handleCallback(Handler.java:938)
at android.os.Handler.dispatchMessage(Handler.java:99)
at com.facebook.react.bridge.queue.MessageQueueThreadHandler.dispatchMessage(MessageQueueThreadHandler.java:27)
at android.os.Looper.loop(Looper.java:236)
at com.facebook.react.bridge.queue.MessageQueueThreadImpl$4.run(MessageQueueThreadImpl.java:226)
at java.lang.Thread.run(Thread.java:923)
Caused by: java.lang.IllegalArgumentException: method com.facebook.react.uimanager.LayoutShadowNode.setWidth argument 1 has type com.facebook.react.bridge.Dynamic, got java.lang.String
at java.lang.reflect.Method.invoke(Native Method)
at com.facebook.react.uimanager.ViewManagersPropertyCache$PropSetter.updateShadowNodeProp(ViewManagersPropertyCache.java:111)
at com.facebook.react.uimanager.ViewManagerPropertyUpdater$FallbackShadowNodeSetter.setProperty(ViewManagerPropertyUpdater.java:161)
at com.facebook.react.uimanager.ViewManagerPropertyUpdater.updateProps(ViewManagerPropertyUpdater.java:65)
at com.facebook.react.uimanager.ReactShadowNodeImpl.updateProperties(ReactShadowNodeImpl.java:320)
at com.facebook.react.uimanager.UIImplementation.createView(UIImplementation.java:251)
at com.facebook.react.uimanager.UIManagerModule.createView(UIManagerModule.java:469)
at java.lang.reflect.Method.invoke(Native Method)
at com.facebook.react.bridge.JavaMethodWrapper.invoke(JavaMethodWrapper.java:372)
at com.facebook.react.bridge.JavaModuleWrapper.invoke(JavaModuleWrapper.java:151)
at com.facebook.react.bridge.queue.NativeRunnable.run(Native Method)
at android.os.Handler.handleCallback(Handler.java:938)
at android.os.Handler.dispatchMessage(Handler.java:99)
at com.facebook.react.bridge.queue.MessageQueueThreadHandler.dispatchMessage(MessageQueueThreadHandler.java:27)
at android.os.Looper.loop(Looper.java:236)
at com.facebook.react.bridge.queue.MessageQueueThreadImpl$4.run(MessageQueueThreadImpl.java:226)
at java.lang.Thread.run(Thread.java:923)
2021-09-08 11:56:07.488 16776-17062/com.shopeepay.merchant.id.debug E/com.shopeepay.merchant.id.debug: com.shopee.app.react.util.ReactJSException: Error while updating property 'width' in shadow node of type: RCTView
at com.facebook.react.uimanager.ViewManagersPropertyCache$PropSetter.updateShadowNodeProp(ViewManagersPropertyCache.java:125)
at com.facebook.react.uimanager.ViewManagerPropertyUpdater$FallbackShadowNodeSetter.setProperty(ViewManagerPropertyUpdater.java:161)
at com.facebook.react.uimanager.ViewManagerPropertyUpdater.updateProps(ViewManagerPropertyUpdater.java:65)
at com.facebook.react.uimanager.ReactShadowNodeImpl.updateProperties(ReactShadowNodeImpl.java:320)
at com.facebook.react.uimanager.UIImplementation.createView(UIImplementation.java:251)
at com.facebook.react.uimanager.UIManagerModule.createView(UIManagerModule.java:469)
at java.lang.reflect.Method.invoke(Native Method)
at com.facebook.react.bridge.JavaMethodWrapper.invoke(JavaMethodWrapper.java:372)
at com.facebook.react.bridge.JavaModuleWrapper.invoke(JavaModuleWrapper.java:151)
at com.facebook.react.bridge.queue.NativeRunnable.run(Native Method)
at android.os.Handler.handleCallback(Handler.java:938)
at android.os.Handler.dispatchMessage(Handler.java:99)
at com.facebook.react.bridge.queue.MessageQueueThreadHandler.dispatchMessage(MessageQueueThreadHandler.java:27)
at android.os.Looper.loop(Looper.java:236)
at com.facebook.react.bridge.queue.MessageQueueThreadImpl$4.run(MessageQueueThreadImpl.java:226)
at java.lang.Thread.run(Thread.java:923)
2021-09-08 11:56:07.571 16776-16776/com.shopeepay.merchant.id.debug E/com.shopeepay.merchant.id.debug: com.shopee.app.react.util.ReactJSException: Trying to add unknown view tag: 1519
at com.facebook.react.uimanager.NativeViewHierarchyManager.manageChildren(NativeViewHierarchyManager.java:487)
at com.facebook.react.uimanager.UIViewOperationQueue$ManageChildrenOperation.execute(UIViewOperationQueue.java:209)
at com.facebook.react.uimanager.UIViewOperationQueue$1.run(UIViewOperationQueue.java:917)
at com.facebook.react.uimanager.UIViewOperationQueue.flushPendingBatches(UIViewOperationQueue.java:1028)
at com.facebook.react.uimanager.UIViewOperationQueue.access$2600(UIViewOperationQueue.java:48)
at com.facebook.react.uimanager.UIViewOperationQueue$DispatchUIFrameCallback.doFrameGuarded(UIViewOperationQueue.java:1088)
at com.facebook.react.uimanager.GuardedFrameCallback.doFrame(GuardedFrameCallback.java:29)
at com.facebook.react.modules.core.ReactChoreographer$ReactChoreographerDispatcher.doFrame(ReactChoreographer.java:175)
at com.facebook.react.modules.core.ChoreographerCompat$FrameCallback$1.doFrame(ChoreographerCompat.java:85)
at android.view.Choreographer$CallbackRecord.run(Choreographer.java:1056)
at android.view.Choreographer.doCallbacks(Choreographer.java:880)
at android.view.Choreographer.doFrame(Choreographer.java:809)
at android.view.Choreographer$FrameDisplayEventReceiver.run(Choreographer.java:1043)
at android.os.Handler.handleCallback(Handler.java:938)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.os.Looper.loop(Looper.java:236)
at android.app.ActivityThread.main(ActivityThread.java:7876)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:656)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:967)
2021-09-08 11:56:07.615 16776-16776/com.shopeepay.merchant.id.debug E/unknown:ReactContextBaseJavaModule: Unhandled SoftException
java.lang.RuntimeException: Catalyst Instance has already disappeared: requested by Timing
at com.facebook.react.bridge.ReactContextBaseJavaModule.getReactApplicationContextIfActiveOrWarn(ReactContextBaseJavaModule.java:67)
at com.facebook.react.modules.core.TimingModule.access$000(TimingModule.java:22)
at com.facebook.react.modules.core.TimingModule$BridgeTimerManager.callTimers(TimingModule.java:28)
at com.facebook.react.modules.core.JavaTimerManager$TimerFrameCallback.doFrame(JavaTimerManager.java:84)
at com.facebook.react.modules.core.ReactChoreographer$ReactChoreographerDispatcher.doFrame(ReactChoreographer.java:175)
at com.facebook.react.modules.core.ChoreographerCompat$FrameCallback$1.doFrame(ChoreographerCompat.java:85)
at android.view.Choreographer$CallbackRecord.run(Choreographer.java:1056)
at android.view.Choreographer.doCallbacks(Choreographer.java:880)
at android.view.Choreographer.doFrame(Choreographer.java:809)
at android.view.Choreographer$FrameDisplayEventReceiver.run(Choreographer.java:1043)
at android.os.Handler.handleCallback(Handler.java:938)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.os.Looper.loop(Looper.java:236)
at android.app.ActivityThread.main(ActivityThread.java:7876)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:656)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:967)
2021-09-08 11:56:07.626 16776-16776/com.shopeepay.merchant.id.debug E/unknown:DeviceInfo: Unhandled SoftException
com.facebook.react.bridge.ReactNoCrashSoftException: No active CatalystInstance, cannot emitUpdateDimensionsEvent
at com.facebook.react.modules.deviceinfo.DeviceInfoModule.emitUpdateDimensionsEvent(DeviceInfoModule.java:99)
at com.facebook.react.ReactRootView$CustomGlobalLayoutListener.emitUpdateDimensionsEvent(ReactRootView.java:755)
at com.facebook.react.ReactRootView$CustomGlobalLayoutListener.checkForDeviceDimensionsChanges(ReactRootView.java:713)
at com.facebook.react.ReactRootView$CustomGlobalLayoutListener.onGlobalLayout(ReactRootView.java:664)
at android.view.ViewTreeObserver.dispatchOnGlobalLayout(ViewTreeObserver.java:1061)
at android.view.ViewRootImpl.performTraversals(ViewRootImpl.java:3093)
at android.view.ViewRootImpl.doTraversal(ViewRootImpl.java:2054)
at android.view.ViewRootImpl$TraversalRunnable.run(ViewRootImpl.java:8383)
at android.view.Choreographer$CallbackRecord.run(Choreographer.java:1058)
at android.view.Choreographer.doCallbacks(Choreographer.java:880)
at android.view.Choreographer.doFrame(Choreographer.java:813)
at android.view.Choreographer$FrameDisplayEventReceiver.run(Choreographer.java:1043)
at android.os.Handler.handleCallback(Handler.java:938)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.os.Looper.loop(Looper.java:236)
at android.app.ActivityThread.main(ActivityThread.java:7876)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:656)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:967)
2021-09-08 11:56:07.674 16776-16776/com.shopeepay.merchant.id.debug E/unknown:ReactContextBaseJavaModule: Unhandled SoftException
java.lang.RuntimeException: Catalyst Instance has already disappeared: requested by Timing
at com.facebook.react.bridge.ReactContextBaseJavaModule.getReactApplicationContextIfActiveOrWarn(ReactContextBaseJavaModule.java:67)
at com.facebook.react.modules.core.TimingModule.access$000(TimingModule.java:22)
at com.facebook.react.modules.core.TimingModule$BridgeTimerManager.callTimers(TimingModule.java:28)
at com.facebook.react.modules.core.JavaTimerManager$TimerFrameCallback.doFrame(JavaTimerManager.java:84)
at com.facebook.react.modules.core.ReactChoreographer$ReactChoreographerDispatcher.doFrame(ReactChoreographer.java:175)
at com.facebook.react.modules.core.ChoreographerCompat$FrameCallback$1.doFrame(ChoreographerCompat.java:85)
at android.view.Choreographer$CallbackRecord.run(Choreographer.java:1056)
at android.view.Choreographer.doCallbacks(Choreographer.java:880)
at android.view.Choreographer.doFrame(Choreographer.java:809)
at android.view.Choreographer$FrameDisplayEventReceiver.run(Choreographer.java:1043)
at android.os.Handler.handleCallback(Handler.java:938)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.os.Looper.loop(Looper.java:236)
at android.app.ActivityThread.main(ActivityThread.java:7876)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:656)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:967)
2021-09-08 11:56:07.677 16776-16776/com.shopeepay.merchant.id.debug E/unknown:DeviceInfo: Unhandled SoftException
com.facebook.react.bridge.ReactNoCrashSoftException: No active CatalystInstance, cannot emitUpdateDimensionsEvent
at com.facebook.react.modules.deviceinfo.DeviceInfoModule.emitUpdateDimensionsEvent(DeviceInfoModule.java:99)
at com.facebook.react.ReactRootView$CustomGlobalLayoutListener.emitUpdateDimensionsEvent(ReactRootView.java:755)
at com.facebook.react.ReactRootView$CustomGlobalLayoutListener.checkForDeviceDimensionsChanges(ReactRootView.java:713)
at com.facebook.react.ReactRootView$CustomGlobalLayoutListener.onGlobalLayout(ReactRootView.java:664)
at android.view.ViewTreeObserver.dispatchOnGlobalLayout(ViewTreeObserver.java:1061)
at android.view.ViewRootImpl.performTraversals(ViewRootImpl.java:3093)
at android.view.ViewRootImpl.doTraversal(ViewRootImpl.java:2054)
at android.view.ViewRootImpl$TraversalRunnable.run(ViewRootImpl.java:8383)
at android.view.Choreographer$CallbackRecord.run(Choreographer.java:1058)
at android.view.Choreographer.doCallbacks(Choreographer.java:880)
at android.view.Choreographer.doFrame(Choreographer.java:813)
at android.view.Choreographer$FrameDisplayEventReceiver.run(Choreographer.java:1043)
at android.os.Handler.handleCallback(Handler.java:938)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.os.Looper.loop(Looper.java:236)
at android.app.ActivityThread.main(ActivityThread.java:7876)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:656)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:967)
2021-09-08 11:56:07.686 16776-16776/com.shopeepay.merchant.id.debug E/unknown:DeviceInfo: Unhandled SoftException
com.facebook.react.bridge.ReactNoCrashSoftException: No active CatalystInstance, cannot emitUpdateDimensionsEvent
at com.facebook.react.modules.deviceinfo.DeviceInfoModule.emitUpdateDimensionsEvent(DeviceInfoModule.java:99)
at com.facebook.react.ReactRootView$CustomGlobalLayoutListener.emitUpdateDimensionsEvent(ReactRootView.java:755)
at com.facebook.react.ReactRootView$CustomGlobalLayoutListener.checkForDeviceDimensionsChanges(ReactRootView.java:713)
at com.facebook.react.ReactRootView$CustomGlobalLayoutListener.onGlobalLayout(ReactRootView.java:664)
at android.view.ViewTreeObserver.dispatchOnGlobalLayout(ViewTreeObserver.java:1061)
at android.view.ViewRootImpl.performTraversals(ViewRootImpl.java:3093)
at android.view.ViewRootImpl.doTraversal(ViewRootImpl.java:2054)
at android.view.ViewRootImpl$TraversalRunnable.run(ViewRootImpl.java:8383)
at android.view.Choreographer$CallbackRecord.run(Choreographer.java:1058)
at android.view.Choreographer.doCallbacks(Choreographer.java:880)
at android.view.Choreographer.doFrame(Choreographer.java:813)
at android.view.Choreographer$FrameDisplayEventReceiver.run(Choreographer.java:1043)
at android.os.Handler.handleCallback(Handler.java:938)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.os.Looper.loop(Looper.java:236)
at android.app.ActivityThread.main(ActivityThread.java:7876)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:656)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:967)
2021-09-08 11:56:07.698 16776-16776/com.shopeepay.merchant.id.debug E/unknown:ReactContextBaseJavaModule: Unhandled SoftException
java.lang.RuntimeException: Catalyst Instance has already disappeared: requested by Timing
at com.facebook.react.bridge.ReactContextBaseJavaModule.getReactApplicationContextIfActiveOrWarn(ReactContextBaseJavaModule.java:67)
at com.facebook.react.modules.core.TimingModule.access$000(TimingModule.java:22)
at com.facebook.react.modules.core.TimingModule$BridgeTimerManager.callTimers(TimingModule.java:28)
at com.facebook.react.modules.core.JavaTimerManager$TimerFrameCallback.doFrame(JavaTimerManager.java:84)
at com.facebook.react.modules.core.ReactChoreographer$ReactChoreographerDispatcher.doFrame(ReactChoreographer.java:175)
at com.facebook.react.modules.core.ChoreographerCompat$FrameCallback$1.doFrame(ChoreographerCompat.java:85)
at android.view.Choreographer$CallbackRecord.run(Choreographer.java:1056)
at android.view.Choreographer.doCallbacks(Choreographer.java:880)
at android.view.Choreographer.doFrame(Choreographer.java:809)
at android.view.Choreographer$FrameDisplayEventReceiver.run(Choreographer.java:1043)
at android.os.Handler.handleCallback(Handler.java:938)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.os.Looper.loop(Looper.java:236)
at android.app.ActivityThread.main(ActivityThread.java:7876)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:656)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:967)
2021-09-08 11:56:08.406 16776-16776/com.shopeepay.merchant.id.debug E/unknown:ReactContextBaseJavaModule: Unhandled SoftException
java.lang.RuntimeException: Catalyst Instance has already disappeared: requested by Timing
at com.facebook.react.bridge.ReactContextBaseJavaModule.getReactApplicationContextIfActiveOrWarn(ReactContextBaseJavaModule.java:67)
at com.facebook.react.modules.core.TimingModule.access$000(TimingModule.java:22)
at com.facebook.react.modules.core.TimingModule$BridgeTimerManager.callTimers(TimingModule.java:28)
at com.facebook.react.modules.core.JavaTimerManager$TimerFrameCallback.doFrame(JavaTimerManager.java:84)
at com.facebook.react.modules.core.ReactChoreographer$ReactChoreographerDispatcher.doFrame(ReactChoreographer.java:175)
at com.facebook.react.modules.core.ChoreographerCompat$FrameCallback$1.doFrame(ChoreographerCompat.java:85)
at android.view.Choreographer$CallbackRecord.run(Choreographer.java:1056)
at android.view.Choreographer.doCallbacks(Choreographer.java:880)
at android.view.Choreographer.doFrame(Choreographer.java:809)
at android.view.Choreographer$FrameDisplayEventReceiver.run(Choreographer.java:1043)
at android.os.Handler.handleCallback(Handler.java:938)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.os.Looper.loop(Looper.java:236)
at android.app.ActivityThread.main(ActivityThread.java:7876)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:656)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:967)
2021-09-08 11:56:08.423 16776-16776/com.shopeepay.merchant.id.debug E/unknown:ReactContextBaseJavaModule: Unhandled SoftException
java.lang.RuntimeException: Catalyst Instance has already disappeared: requested by Timing
at com.facebook.react.bridge.ReactContextBaseJavaModule.getReactApplicationContextIfActiveOrWarn(ReactContextBaseJavaModule.java:67)
at com.facebook.react.modules.core.TimingModule.access$000(TimingModule.java:22)
at com.facebook.react.modules.core.TimingModule$BridgeTimerManager.callTimers(TimingModule.java:28)
at com.facebook.react.modules.core.JavaTimerManager$TimerFrameCallback.doFrame(JavaTimerManager.java:84)
at com.facebook.react.modules.core.ReactChoreographer$ReactChoreographerDispatcher.doFrame(ReactChoreographer.java:175)
at com.facebook.react.modules.core.ChoreographerCompat$FrameCallback$1.doFrame(ChoreographerCompat.java:85)
at android.view.Choreographer$CallbackRecord.run(Choreographer.java:1056)
at android.view.Choreographer.doCallbacks(Choreographer.java:880)
at android.view.Choreographer.doFrame(Choreographer.java:809)
at android.view.Choreographer$FrameDisplayEventReceiver.run(Choreographer.java:1043)
at android.os.Handler.handleCallback(Handler.java:938)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.os.Looper.loop(Looper.java:236)
at android.app.ActivityThread.main(ActivityThread.java:7876)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:656)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:967)
2021-09-08 11:56:09.594 16776-16776/com.shopeepay.merchant.id.debug E/unknown:DeviceInfo: Unhandled SoftException
com.facebook.react.bridge.ReactNoCrashSoftException: No active CatalystInstance, cannot emitUpdateDimensionsEvent
at com.facebook.react.modules.deviceinfo.DeviceInfoModule.emitUpdateDimensionsEvent(DeviceInfoModule.java:99)
at com.facebook.react.ReactRootView$CustomGlobalLayoutListener.emitUpdateDimensionsEvent(ReactRootView.java:755)
at com.facebook.react.ReactRootView$CustomGlobalLayoutListener.checkForDeviceDimensionsChanges(ReactRootView.java:713)
at com.facebook.react.ReactRootView$CustomGlobalLayoutListener.onGlobalLayout(ReactRootView.java:664)
at android.view.ViewTreeObserver.dispatchOnGlobalLayout(ViewTreeObserver.java:1061)
at android.view.ViewRootImpl.performTraversals(ViewRootImpl.java:3093)
at android.view.ViewRootImpl.doTraversal(ViewRootImpl.java:2054)
at android.view.ViewRootImpl$TraversalRunnable.run(ViewRootImpl.java:8383)
at android.view.Choreographer$CallbackRecord.run(Choreographer.java:1058)
at android.view.Choreographer.doCallbacks(Choreographer.java:880)
at android.view.Choreographer.doFrame(Choreographer.java:813)
at android.view.Choreographer$FrameDisplayEventReceiver.run(Choreographer.java:1043)
at android.os.Handler.handleCallback(Handler.java:938)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.os.Looper.loop(Looper.java:236)
at android.app.ActivityThread.main(ActivityThread.java:7876)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:656)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:967)

Looks good, thanks for updating!
CI build is failing now with:
/root/react-native/ReactAndroid/src/main/java/com/facebook/react/uimanager/ViewManagersPropertyCache.java:588: error: local variable size is accessed from within inner class; needs to be declared final
        return new Object[size];

Otherwise, should be good to go.

Pinging @chenmo187 🤗

@ShikaSD would you consider co-authoring the last few changes in or should we keep on waiting for @chenmo187 to finish the PR?

Pinging @chenmo187 🤗

Sorry for the late reply, I will handle the

PR build artifact for 2c7cb7e is ready. To use, download tarball from "Artifacts" tab in this CircleCI job then run yarn add <path to tarball> in your React Native project.

I download the tgz file already, but how to use the file ....

I have met the error : "Trying to add unknown view tag" , but i have no idea to deal with it . pls give me some advice . thanks!

authors: ArcherLucky ShikaSD chenmo187 rikur zhairui


text:
Merge the two Gradle Plugins

Summary:
This diff merges the react-native-gradle-plugin and the react-codegen/android into a single plugin.
This will allow us to iterate faster on a single plugin, will create less confusion for our users (react vs reactApp)
and will help us avoid race conditions when the two plugins are applied together (as we will control the whole lifecycle of it).
Changelog:
[Internal] [Changed] - Merged the two Gradle Plugins
allow-large-files
Differential Revision: D30765147

👀 This change looks like comes with bundle size optimization. Compared with PR https://github.com/facebook/react-native/pull/32176#issuecomment-916131891

👀 This change looks like comes with bundle size optimization. Compared with PR #32176 (comment)

Thanks for pointing it out. I believe the bundle size should actually be unaffected so that might be a false reporting. I'll look into it.

@cortinico  I would suggest com.facebook.react.gradle as plugin name

@cortinico I would suggest com.facebook.react.gradle as plugin name

I believe we will end up with com.facebook.react as plugin name, but it's still up for debate. cc @ShikaSD

@cortinico I would suggest com.facebook.react.gradle as plugin name

I believe we will end up with com.facebook.react as plugin name, but it's still up for debate. cc @ShikaSD

It's even better 👍

source: gharchive/pull-request

authors: cortinico dulmandakh ecreeth

repo: facebook/react-native

license: MIT

license_type: permissive

license_source: github-api


text:
Refactor __flushRenderedRootElement/__flushElementChildren

these contain duplicate code
these are buggy - see #85 and example below

I think a recursive strategy for most of their responsibilities is likely to be a better approach
  public function testNested(): void {
    $x =
      <test:xhphelpers class="herp">
        <test:xhphelpers class="derp" />
      </test:xhphelpers>;
    // Fails - actual output is

    $this->assertSame('

', $x->toString());
  }

My plan is to create a new '__renderAndProcess' method on :x:composable-element that:

validates attributes
validates children
transfers attributes
transfers contexts
awaits children
returns Awaitable
is the only thing that calls render()

Reduction to :primitive should be handled by ::toString(). I don't think this would be a significant user-visible API change.

:+1:

not entirely sure that 'awaits children' belongs there. Will experiment.

Examples:

https://github.com/fredemmott/xhp-lib/blob/failing-tests/tests/XHPHelpersTest.php#L115-120
https://github.com/fredemmott/xhp-lib/blob/9165443a311ec0c9f53b4e4cc67de2f43dad0dc4/tests/ChildRuleTest.php#L280-286

Test for XHPHelpersTest was incorrect (missing {$this->getChildren()}), but still fails when corrected.

source: gharchive/issue

repo: facebook/xhp-lib

license_source: bigquery


text:
How do I add more than one proxy?

How do I add more than one proxy. In the documentation, only one way is shown:
"proxy": "http://localhost:4000"
But I need add a few patterns for my proxy like:
"proxy": {     "/api/*": "http://localhost:4000",     "/forms/*": "http://localhost:4000",     "/content/*": "http://localhost:4000"   }
I would normally write this like so in my webpack.config file but this isn't working:
devServer: { 	proxy: { 		"/api/*": "http://localhost:4000", 		"/forms/*": "http://localhost:4000", 		"/content/*": "http://localhost:4000" 	} }

This is currently a WIP and you can comment on the proposal here: https://github.com/facebookincubator/create-react-app/pull/1790.
The original issue is https://github.com/facebookincubator/create-react-app/issues/1780 (of which this issue is a duplicate).
You can see if you can help expedite https://github.com/facebookincubator/create-react-app/pull/1790 by commenting there. 😄

authors: Timer kwingat

repo: facebookincubator/create-react-app

license: mit


text:
Upgrade to webpack 3

Webpack 3 has released.
This article by @TheLarkinn highlight new features in this release of webpack.

I updated on my local machine and ran on default app.

This is a pretty straight upgrade, no change in the config. The only thing hindering is chunck.modules has been deprecated and is used by extract-text-plugin, thus throws a depreciation warning.
https://github.com/webpack-contrib/extract-text-webpack-plugin/issues/494

uglifyjs-plugin is moved into separate repository

I don't think babili is ready yet, but hopefully this would be a step in that direction.

Just to set precedence, we won't be in any rush to upgrade to webpack 3 until it has been battle tested for some time.

@Timer Sure, i was just pointing out it released and has straight upgrade path, though definitely would have to wait for a while, as some plugins have issues.

Did you have the ModuleConcatenationPlugin plugin enabled in this test @shrynx?

@ianschmitz yes i did, but since this is a tiny app (default), you don't see size getting affected much.

Facing the same problem

extract-text-webpack-plugin released v3.0.0 a few hours ago. webpack 3 is now supported.

latest: https://github.com/facebookincubator/create-react-app/pull/2850

authors: Timer arnabrahman ianschmitz miraage shrynx themre wtgtybhertgeghgtwtg


text:
Add support for a static folder

It is a nice escape hatch, ref: #28
Verified by going through the entire flow (in all three states, i.e. node_modules, development and ejected) with npm start.

Can you please rebase this? I want to take a look this week.

Put it on my todo list, will do this evening!

Rebased!

What happens if I just reference /src/image.png in my HTML? If it works now we need to make sure it doesn’t because that would be confusing.

Don't have time to try it out at the moment I'm afraid!

Interesting thread! I was just experiencing this "short coming" today while testing out create-react-app and I needed to load Apple touch icons. I just happened to solve this the exact same way (only calling it assets which I understand was an unpopular choice), and silly me didn't check to see if there already was an open PR. I think this would be a great addition.
@gaearon I tested out referencing the /src/logo.svg directly from index.html and that didn't work. Neither did creating an arbitrarily file, /bar/foo.svg, and that could not be referenced from the static files either.
As a maybe "happy accident", by calling this static, it works as expected when referencing the files from the index.html it feels natural both in build and start mode as one can refer to the JS, CSS and media files from the same location. I also verified that a folder media in the template/static directory, got merged with the build/static/ directory and not overwritten.
Now it might feel "magical" that you have to create the static directory yourself and users need to confer with documentation by own initiative to check "if something like this just happens to exist". I think the template should have this folder by default by using something like PNG favicons or something.
Having something like this:
static/favicon/favicon-16.png
static/favicon/favicon-32.png

And in index.html:
...
<title>React App</title>
<link rel="icon" href="static/favicon/favicon-16.png" sizes="16x16" type="image/png">
<link rel="icon" href="static/favicon/favicon-32.png" sizes="32x32" type="image/png">
...

This way it would be easy to understand how to use it, and that it even exists. Just wanted to chime in and managed to write an entire essay doing so.

@mikaelbr Haha thanks for feedback on this!

For a Service Worker to control a site, it has to be placed at the root. I.e., a service worker at /static/sw.js could only control pages that started with /static. There are other systems like Google's Website Verification that work by looking for a file at the root of a domain.
Any concern with simply removing the to option from CopyWebpackPlugin(), so that the contents of static are copied directly into the output?

@sco that'd be confusing if you put files into static/image.png but then have to reference /image.png. It might also override over files that are in the root? (imagine putting an index.html into static/…)

copy-webpack-plugin actually won't overwrite any previously staged files, by default.
I can see how it might be a little confusing, but I think @mikaelbr's suggestion of putting the favicons there, and referencing them from index.html, would help to illustrate it pretty clearly.
I suspect putting arbitrary files at the root of a project will be a fairly common need. (robots.txt comes to mind, for instance.)

There might be several files that are required to be in the root, so there might be a need to do this as well. What is the possible downsides of copying files from root to root?
By just copying files it'll be easier to restrict what files to actually copy and reduce the build-directory, but it might be more more confusing; "why do this files work, but I can't find /some-dir/bar.svg." Are there any huge side-effects by transferring all extra static files, except build size?

If you have a backend to your app, serving arbitrary static files from the root url that your webserver serves generally is not a good idea, because then you can't use a regex in your nginx/apache config to decide whether to route things to your static file service mechanism or to your backend. You usually want to special-case a small number of files, like robots.txt, crossdomain.xml, sw.js, and so on. So I suggest we continue serving assets in /static/ from a url containing /static/.
Perhaps you could actually work around this quite easily if you don't have a backend, by proxying to localhost:3000/static. That will proxy localhost:3000/sw.js to localhost:3000/static/sw.js IIUC. If you do have a backend then this won't work, but if you do have a backend then you already have the problem that the default logic will proxy /sw.js to your backend. You might just want to serve a handful of static files from your existing backend in that case.

I don't think proxying like that is a good idea. Having 2 different routes that route to the same assets is needlessly confusing.
I think it's much better to move index.html and favicon.ico into the static file and have everything in there copied into dist.

I think it's much better to move index.html

This would be a much better solution if index.html wasn't passed through webpack with the HTMLWebpackPlugin – but alas, it is. That'd only make things more confusing if suddenly one single file in the static/ folder was passed through webpack but all the others aren't.

Depends on whether the associated images in there would be passed though
webpack as well to have hashes added for cdn cache busting reasons.
I wouldn't mind building a pr for all this but have no clue if it's what
the maintainers want.
On Mon, Aug 29, 2016 at 4:01 AM Max Stoiber notifications@github.com
wrote:

I think it's much better to move index.html
This would be a much better solution if index.html wasn't passed through
webpack with the HTMLWebpackPlugin – but alas, it is. That'd only make
things more confusing if suddenly one single file in the static/ folder
was passed through webpack but all the others aren't.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/facebookincubator/create-react-app/pull/226#issuecomment-243057958,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABJ2igg8U4OqFJMyqeGmlDfRTZDG2BGks5qkpHhgaJpZM4JVdxr
.

Depends on whether the associated images in there would be passed though webpack as well to have hashes added for cdn cache busting reasons.

Nope, then they should go into the src folder next to the respective components. The whole point of having a static/ folder is that it's an escape hatch out of all of that behaviour.

Then index.html should be in src
On Mon, Aug 29, 2016 at 8:13 AM Max Stoiber notifications@github.com
wrote:

Depends on whether the associated images in there would be passed though
webpack as well to have hashes added for cdn cache busting reasons.
Nope, then they should go into the src folder next to the respective
components. The whole point of having a static/ folder is that it's an
escape hatch out of all of that behaviour.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/facebookincubator/create-react-app/pull/226#issuecomment-243106997,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABJ2rrU9h8qvQRURSbYUVKdWu5MgFoOks5qksz0gaJpZM4JVdxr
.

This is tangential to this PR, but if that's the case then index.html should be inside of src anyways.
And with that context I'd say that having static is a mistake that would lead to more confusion than it would solve. It gives 2 locations for things to be stored rather than just one. For those who need the more advanced  behavior they can always eject and do themselves.

@tal I personally have no useful opinion on layout or even what should or shouldn't go through the webpack transformation.
On the topic of capabilities, though: I do think that it's good to have a fairly ambitious definition of what can be done without ejecting.  So, in particular, serving favicons for all major OSes and as much of the progressive web app feature set as possible (good service workers defaults, etc.), should not require ejecting.
If those capabilities are accessible (but not required) in the default setup, then more people will implement them, and more webapps will be better webapps, which is good for users and good for the web.

What i was meaning was that those favicon or icons of whatever sort for all
the OSes are specified by meta tags. Not convention. That means that they
should be cache busted, this probably means going though webpack in some
way.
On Mon, Aug 29, 2016 at 12:45 PM David Ascher notifications@github.com
wrote:

@tal https://github.com/tal I personally have no useful opinion on
layout or even what should or shouldn't go through the webpack
transformation.
On the topic of capabilities, though: I do think that it's good to have
a fairly ambitious definition of what can be done without ejecting. So, in
particular, serving favicons for all major OSes and as much of the
progressive web app feature set as possible (good service workers defaults,
etc.), should not require ejecting.
If those capabilities are accessible (but not required) in the default
setup, then more people will implement them, and more webapps will be
better webapps, which is good for users and good for the web.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/facebookincubator/create-react-app/pull/226#issuecomment-243180755,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABJ2rwYSvWX7TlyHBe3QVKaCxlVFWQUks5qkwyggaJpZM4JVdxr
.

@tal, in that case, maybe a package like webpack-manifest-plugin would be interesting. It works by adding this to your webpack loader config:
{
    test: /manifest.json$/,
    loader: 'file?name=manifest.json!web-app-manifest',
},

Sorry for the long turnaround on this.
I’ve been weighing pros and cons and I’m coming to realization that this is not the approach I want to take.
Instead, #428 should satisfy most use cases, and if not, we can address them on case by case basis (file issues!)
Please see #551 for more comments on this, justification for not supporting this, and additional documentation that I believe adequately explains how #428 works as well as the behavior I ultimately want to have.
Sorry 😥.

We got some more feedback and I know more about the use cases people have for this. I couldn't find any better alternative also let's do it in the next minor.

Thought some more about it. It won't work for non-relative roots (like GH Pages).
People will reference /static in their HTML and it will 404 after hosting.
Any ideas how to solve that?

would love to help with this issue. it relates to the open graph meta tag issue i'm wanting resolve too.
what should i try?

I’m going to close this because I ended up with a different solution that I think addresses these concerns better: https://github.com/facebookincubator/create-react-app/pull/703. Let me know what you think!

This is now supported in 0.5.0.
Read about using the new public folder.
See also migration instructions and breaking changes in 0.5.0.

authors: SpaceK33z davidascher gaearon kahneraja lacker mikaelbr mxstbr sco tal


text:
Eager loading doesn't seem to work for M2M

Hello,
I think there's a bug in the generated code for M2M eager loading. I don't have a simple test case to share, but I've done some digging and I think I found the issue:
I have two entities Person and Email (ignore for a second the fact that this shouldn't be a M2M to begin with 😬). I'm trying to query a specific Person with id 124:
client.Person.Query().WithEmails().Where(person.ID(124)).Only(ctx)

That query will return the person, but the Emails will not be populated on the PersonEdges (the flag for the edge is set on the loadedTypes however). Looking at the queries it seems to me like it's picking the wrong column from the second query to use for the third. (I removed the explicit field lists to make it easier to read):
SELECT DISTINCT * WHERE "persons"."id" = $1 LIMIT $2 -- args=[124 2]
SELECT "person_id", "email_id" FROM "person_emails" WHERE "person_id" IN ($1) -- args=[124]
SELECT DISTINCT * FROM "emails" WHERE "emails"."id" IN ($1, $2) -- args=[124 124]

If we check out the generated code, we can see why:
// in person_query.go
Assign: func(out, in interface{}) error {
	eout, ok := out.(*sql.NullInt64)
	if !ok || eout == nil {
		return fmt.Errorf("unexpected id value for edge-out")
	}
	ein, ok := in.(*sql.NullInt64)
	if !ok || ein == nil {
		return fmt.Errorf("unexpected id value for edge-in")
	}
	outValue := int(eout.Int64)
	inValue := int(eout.Int64)        // <---- this should be ein
	node, ok := ids[outValue]
	if !ok {
		return fmt.Errorf("unexpected node id in edges: %v", outValue)
	}
	edgeids = append(edgeids, inValue)
	edges[inValue] = append(edges[inValue], node)
	return nil
},

I tried fixing it in the code generator, but I didn't immediately understand what the template code is supposed to do, so I'll leave that to someone else 😬 (my guess would be changing the "eout" to $arg)
When I changed it in the generated code it started working for me, so I'm going to assume that should fix it.

Thanks for reporting this issue @ligustah.
Can you please paste here your schema definition for Person and Email?

Can you please paste here your schema definition for Person and Email?

Don't need. I found the issue.
We missed this, because we run codegen with a different config in integration - I'll try to add a better CI coverage for eager-loading this week.
Thanks for help!

Awesome, that was a quick fix!
Looking good to me, but I'll give it a test run
Thanks

repo: facebookincubator/ent


text:
VectorHasher's value ID caching logic makes certain queries unnecessarily slow

Description
Hi,
(I believe this issue is more of a performance bug report rather than an enhancement suggestion. However, since it is not a functional bug, I have chosen to classify it under the 'enhancement' category.)
I am building an OLAP DBMS system that uses Velox for the execution engine. In this issue, two executor processes exchange intermediate results. The query, which has been slightly modified to hide the real table name, is as follows:
SELECT ip, name
FROM (
	SELECT name
	FROM t1
	WHERE tm_col >= timestamp '2022-07-11 03:00:00'
	AND tm_col < timestamp '2022-07-11 03:10:00'
) JOIN (
	SELECT ip, name AS name2
	FROM t2
  	WHERE tm_col >= timestamp '2022-09-30 05:30:00'
	AND tm_col < timestamp '2022-09-30 06:00:00'
) ON name = name2
GROUP BY ip, name
;

The subqueries' resulting record count is 2,763,772 and 5,600 respectively.
The query became very slow after applying #7404 to my local Velox repository.
I investigated it and found the source code line that causes the problem is in ExchangeQueue.cpp.
The line is as follows:
 if (pageBytes > 0 && pageBytes + queue_.front()->size() > maxBytes) {

The code link is as follows:
https://github.com/facebookincubator/velox/blob/3a7f8a88afa21cd482fe2bc970473ddd296fcf0a/velox/exec/ExchangeQueue.cpp#L121
After I changed the code to the following, the query became significantly faster (from 176 secs to 43 secs).
if (pageBytes > 0 && pageBytes + queue_.front()->size() > 1) {

(I will call this modification as M1.)
The modified code effectively disables what #7404 tries to achieve. Strange.
So, I've run valgrind with callgrind on the original code and the resulting performance graph of one of the two executor processes was as follows:

memset() was taking a big portion of runtime. It is called by std::fill() by VectorHasher::makeValueIdsDecoded().
The code link is as follows:
https://github.com/facebookincubator/velox/blob/3a7f8a88afa21cd482fe2bc970473ddd296fcf0a/velox/exec/VectorHasher.cpp#L252
M1's valgrind result graph is as follows:

memset()'s portion is now negligible.
Upon further investigation, I found that the DecodedVector's base vector size becomes disproportionately large relative to the SelectivityVector's size in the unmodified code. For example, the DecodedVector's base vector size is 1,066,500, while the SelectivityVector's size is 1024. For M1, the DecodedVector's base vector size is 11,900, while the SelectivityVector's size is 1024.
As a result, the cost of clearing the value ID cache outweighs the benefits of caching.
When I removed the caching logic, the query performed as quickly as M1.
Since I am working with a modified version of Velox's source code and not the current official source, I cannot be completely certain that this issue is present in the current version. However, I believe there is a high probability that it exists.
It would be nice if VectorHasher could be made more intelligent to avoid this kind of issue. (For example, disable the caching logic if DecodedVector's size is too big for the SelectivityVector's size.)
Thank you.

@zeodtr Thank you for reporting this issue with so much detail.

I found that the DecodedVector's base vector size becomes disproportionately large relative to the SelectivityVector's size in the unmodified code. For example, the DecodedVector's base vector size is 1,066,500, while the SelectivityVector's size is 1024.

This sounds similar to https://github.com/facebookincubator/velox/pull/9843
CC: @Yuhta @xiaoxmeng

If VectorHasher is the place we spend most of the time, this can be fixed with a similar approach as #7150.  I can prepare something so you can try.

@Yuhta Jimmy, let's first find out which code produced such a dictionary vector. It might be better to change that code to avoid producing such vectors (similar to Unnest).

@mbasmanova Agree that we should find out the code produce this dictionary and selectivity in this case (out of exchange), because peeling can be inefficient as well on the data.
But as general case I see this can happen legitimately, for example whenever we use dictionary to filter rows (remaining filter / join filter).  As in VectorHasher there is no downside to handle this case separately and it is much easier to implement than encoding peeling, we should fix it in VectorHasher at the same time investigating how the data is generated.

@zeodtr #10084 is a fix for VectorHasher you can try it.  But as @mbasmanova pointed out this is probably not the only place it is inefficient.  Can you

Try to diagnose where the dictionary vectors are generated?
Or you can try create a unit test to reproduce it using PlanBuilder (example: https://github.com/rui-mo/velox/commit/e05691a6648ebd5a540ef3b3baae8868a48b38fd)
Or just print out the plan node tree in LocalPlanner and paste here

@Yuhta Thank you very much for your fix. Since I am out of the office this week, I will try it next week.
I have this for the plan printout for now. (I have changed names and deleted the column names in ROW<>) Two executor processes were given the same plans.
velox plan[0] :
 -- PartitionedOutput[partitionFunction: HASH(0) with 2 partitions] -> n2_2:VARCHAR
  -- Project[expressions: (n2_2:VARCHAR, ROW["n0_1"])] -> n2_2:VARCHAR
    -- Filter[expression: and(and(not(is_null(ROW["n0_1"])),gte(ROW["n0_0"],"2022-07-11T03:00:00.000000000")),lt(ROW["n0_0"],"2022-07-11T03:10:00.000000000"))] -> n0_0:TIMESTAMP, n0_1:VARCHAR
      -- TableScan[table: hive_table, data columns: ROW<...>] -> n0_0:TIMESTAMP, n0_1:VARCHAR

velox plan[1] :
 -- PartitionedOutput[partitionFunction: HASH(1) with 2 partitions] -> n2_3:VARCHAR, n2_4:VARCHAR
  -- Project[expressions: (n2_3:VARCHAR, ROW["n0_0"]), (n2_4:VARCHAR, ROW["n0_2"])] -> n2_3:VARCHAR, n2_4:VARCHAR
    -- Filter[expression: and(and(not(is_null(ROW["n0_2"])),gte(ROW["n0_1"],"2022-09-30T05:30:00.000000000")),lt(ROW["n0_1"],"2022-09-30T06:00:00.000000000"))] -> n0_0:VARCHAR, n0_1:TIMESTAMP, n0_2:VARCHAR
      -- TableScan[table: hive_table, data columns: ROW<...>] -> n0_0:VARCHAR, n0_1:TIMESTAMP, n0_2:VARCHAR

velox plan[2] :
 -- PartitionedOutput[partitionFunction: HASH(0, 1) with 2 partitions] -> n3_3:VARCHAR, n3_4:VARCHAR
  -- Aggregation[SINGLE [n3_3, n3_4] ] -> n3_3:VARCHAR, n3_4:VARCHAR
    -- Project[expressions: (n3_3:VARCHAR, ROW["t2.1.ip#0"]), (n3_4:VARCHAR, ROW["t1.0.name#0"])] -> n3_3:VARCHAR, n3_4:VARCHAR
      -- HashJoin[INNER t1.0.name#0=t2.1.name2#1] -> "t1.0.name#0":VARCHAR, "t2.1.ip#0":VARCHAR, "t2.1.name2#1":VARCHAR
        -- Exchange[] -> "t1.0.name#0":VARCHAR
        -- Exchange[] -> "t2.1.ip#0":VARCHAR, "t2.1.name2#1":VARCHAR

velox plan[3] :
 -- Project[expressions: (ip:VARCHAR, "#anonymous#.2.ip#0"), (name:VARCHAR, "#anonymous#.2.name#1")] -> ip:VARCHAR, name:VARCHAR
  -- Aggregation[SINGLE [#anonymous#.2.ip#0, #anonymous#.2.name#1] ] -> "#anonymous#.2.ip#0":VARCHAR, "#anonymous#.2.name#1":VARCHAR
    -- Exchange[] -> "#anonymous#.2.ip#0":VARCHAR, "#anonymous#.2.name#1":VARCHAR

Also, I will try to diagnose the dictionary vector. However, given my current knowledge of Velox's internals, I may have difficulty locating the place.
Creating a unit test case might not be possible due to the large amount of data in the tables.
Thank you.

@zeodtr Thanks for the detail.  Just a question, which plan tree are you observing the slowness?  Is it plan[2] or plan[3]?  Both are taking data from exchange.

@Yuhta I'm not entirely sure, but I guess it's plan[2].
The task for plan[2] ran for more than 2 minutes logging the following debugging messages many times.
{Driver: running Exchange(0)<xdb_cpu_executor_task_3:0.0 0x7f91a9896700> HashProbe(1)<xdb_cpu_executor_task_3:0.0 0x7f91a9845a00> FilterProject(2)<xdb_cpu_executor_task_3:0.0 0x7f91a9835c00> Aggregation(3)<xdb_cpu_executor_task_3:0.0 0x7f91a9834000> PartitionedOutput(4)<xdb_cpu_executor_task_3:0.0 0x7f91a9865900> CallbackSink(5)<xdb_cpu_executor_task_3:0.0 0x7f91a9896a80> {OpCallStatus: executing HashProbe::getOutput for 0ms}}
{Driver: running Exchange(0)<xdb_cpu_executor_task_3:0.0 0x7f91a9896700> HashProbe(1)<xdb_cpu_executor_task_3:0.0 0x7f91a9845a00> FilterProject(2)<xdb_cpu_executor_task_3:0.0 0x7f91a9835c00> Aggregation(3)<xdb_cpu_executor_task_3:0.0 0x7f91a9834000> PartitionedOutput(4)<xdb_cpu_executor_task_3:0.0 0x7f91a9865900> CallbackSink(5)<xdb_cpu_executor_task_3:0.0 0x7f91a9896a80> {OpCallStatus: executing Aggregation::addInput for 0ms}}
Each executor process spent its most time on different plans, (maybe) one process was busy executing plan[2] while the other process was waiting for the results of its peer.
Thank you.

@Yuhta For your information, the version of Velox for my executor process was taken from the upstream code as of the end of January. Therefore, it's possible that the issue related to plan[2] has already been resolved in the latest upstream version.

Plan 2 has Aggregation and Join. It seems likely that "bad" dictionary was produced by the Join and is causing trouble during Aggregation.

I think it's likely due to HashProbe plan[2] wrapping input in dictionary to filter out non-matching rows.  The ratio of 5600/2763772 has the same order of magnitude as 1024/1066500.  @mbasmanova I think the solution should belong to the same story as #7801

@Yuhta I've tried #10084.
I've run the code with the combinations of the following modifications:

M1: Change a line of ExchangeQueue::dequeueLocked() in ExchangeQueue.cpp to if (pageBytes > 0 && pageBytes + queue_.front()->size() > 1) { to disable #7404
M2: Apply #10084 you supplied (but since my Velox code base is older, #8579, #9803, and #9963 were not applied)
M3: Upon M2, change a line of VectorHasher::makeValueIdsDecoded in VectorHasher.cpp to if (rows.countSelected() * 20 <= decoded_.base()->size()) { so give the cache logic more chance to handle somewhat larger (sparse) decoded vectors. This effectively applies the cache logic to the M1 code.

The query running times are as follows:

No modification: 180~182 secs
M1: 42~43 secs
M2: 53~54 secs
M1 + M2: 53~53 secs
M2 + M3: 51~52 secs
M1 + M2 + M3: 41~42 secs (this is effectively the same as M1)

My guess is as follows:

The value ID cache logic can handle decoded vectors larger than SelectivityVector to some degree (at least in my case).
Small decoded vectors are still preferable so that the value ID cache logic can be used (at least in my case).

Thank you.

The exact condition to use cache is tricky and depending on the data.  The current solution makes sure there is no large regression if we are using dictionary for filtering.
For more investigation, it would be nice if you can find out what the dictionary vector is wrapping around.  If it is from the probe side rows, I would imagine it's used exclusively for filtering so cache should not improve performance here.  For build side rows, they are extracted from row container so should not be in dictionary.  So it's a little mystery here why cache is beneficial.  Maybe the join you have duplicate the probe side rows in some cases?

I've investigated further.
The sparse dictionary vectors are returned from HashProbe::getOutput().
I checked the shape of the data of the two tables. In short, the query was executed on poorly shaped data, resulting in many join mismatches and many duplications in the join results. Therefore, this query might not be suitable for Velox code optimization. I believe the fix in #10084 is sufficient.
The shape of the data for the two tables (t1, t2) is as follows.
The record counts are filtered by the query's WHERE clause.

t1's count(name): 2,763,772
t1's count(distinct name): 457
t2's count(name): 5,600
t2's count(distinct name): 54
count(name) after join: 533,305,844
count(distinct name) after join: 52
t1's count(name) that do not exist in t2: 837,094
t1's count(distinct name) that do not exist in t2: 405
t1's count(name) that exist in t2: 1,926,678
t1's count(distinct name) that exist in t2: 52

The queries for each count result (slightly modified to hide the real table and column name) are as follows:
-- 1.
SELECT count(name)
FROM t1
WHERE tm_col >= timestamp '2022-07-11 03:00:00'
AND tm_col < timestamp '2022-07-11 03:10:00'
;

-- 2.
SELECT count(DISTINCT name)
FROM t1
WHERE tm_col >= timestamp '2022-07-11 03:00:00'
AND tm_col < timestamp '2022-07-11 03:10:00'
;

-- 3.
SELECT count(name)
FROM t2
WHERE tm_col >= timestamp '2022-09-30 05:30:00'
AND tm_col < timestamp '2022-09-30 06:00:00'
;

-- 4.
SELECT count(DISTINCT name)
FROM t2
WHERE tm_col >= timestamp '2022-09-30 05:30:00'
AND tm_col < timestamp '2022-09-30 06:00:00'
;

-- 5., 6.
SELECT count(name), count(DISTINCT name)
FROM (
	SELECT name AS name
	FROM t1
	WHERE tm_col >= timestamp '2022-07-11 03:00:00'
	AND tm_col < timestamp '2022-07-11 03:10:00'
) JOIN (
	SELECT ip, name AS name2
	FROM t2
  	WHERE tm_col >= timestamp '2022-09-30 05:30:00'
	AND tm_col < timestamp '2022-09-30 06:00:00'
) ON name = name2
;

-- 7., 8
SELECT count(name1), count(DISTINCT name1)
FROM (
	SELECT name AS name1
	FROM t1
	WHERE tm_col >= timestamp '2022-07-11 03:00:00'
	AND tm_col < timestamp '2022-07-11 03:10:00'
)
WHERE NOT EXISTS
(
	SELECT 1
	FROM t2
  	WHERE tm_col >= timestamp '2022-09-30 05:30:00'
	AND tm_col < timestamp '2022-09-30 06:00:00'
	AND name = name1
)
;

-- 9., 10
SELECT count(name1), count(DISTINCT name1)
FROM (
	SELECT name AS name1
	FROM t1
	WHERE tm_col >= timestamp '2022-07-11 03:00:00'
	AND tm_col < timestamp '2022-07-11 03:10:00'
)
WHERE EXISTS
(
	SELECT 1
	FROM t2
  	WHERE tm_col >= timestamp '2022-09-30 05:30:00'
	AND tm_col < timestamp '2022-09-30 06:00:00'
	AND name = name1
)
;

I see so the build side are both duplicating and filtering.  Agree that the current solution should be enough, unless there is very important use case that requires us to optimize for this data shape.

authors: Yuhta mbasmanova zeodtr

repo: facebookincubator/velox


text:
Add more Substrait function mappings in velox

Description
For functions that have different names, the mapping is handled here: https://github.com/facebookincubator/velox/blob/main/velox/substrait/SubstraitParser.h#L87-L93
However, substrait has some functions that are not functions in Velox, so the mapping is unclear (if there is any current mapping at all).
Velox Conjunctions

[ ] and
[ ] or
[ ] not
https://github.com/facebookincubator/velox/blob/main/velox/dwio/common/MetadataFilter.cpp
https://github.com/facebookincubator/velox/blob/main/velox/expression/ConjunctExpr.cpp

Velox AST Node Expressions

[ ] coalesce (https://github.com/facebookincubator/velox/blob/main/velox/experimental/codegen/ast/ASTNode.h#L423)

Substrait Functions not in Velox

[ ] is_not_distinct_from (This is not a function in presto, but is handled like this: https://prestodb.io/docs/current/functions/comparison.html#is-distinct-from-and-is-not-distinct-from
[ ] is_not_null (https://prestodb.io/docs/current/functions/comparison.html#is-null-and-is-not-null)

There may need to be some special way of doing the Substrait -> Velox mappings for these.

cc : @zhejiangxiaomai @rui-mo

cc @mbasmanova

Hi @richtia, we have supported more functions locally, and the upstream process to Velox is still WIP. Maybe this is useful to you: https://github.com/oap-project/velox/blob/main/velox/substrait/SubstraitParser.h#L115-L129.
We use "coalesce" for CoalesceExpr so it is not in the map.
https://github.com/facebookincubator/velox/blob/main/velox/experimental/codegen/code_generator/ExprCodeGenerator.cpp#L221-L224

@richtia Mapping Substrait functions to Velox functions by name is fragile. The functions may have the same name, but different semantics (behavior) which will cause incorrect results.

@richtia Mapping Substrait functions to Velox functions by name is fragile. The functions may have the same name, but different semantics (behavior) which will cause incorrect results.

Substrait functions should have well defined semantics.  I don't think we are there yet.  Many of the function desriptions are too vague / incomplete.  I agree it would be a bug if a mapping led to a velox implementation that did not respect the Substrait semantics.  I'm not sure that makes the mapping process itself fragile though.
In other words, do you see this more as a lack of definition and tests?  Or as a flaw in the design itself?

authors: kgpai mbasmanova richtia rui-mo westonpace


text:
Add FileHandleTest to the build

This change includes the FileHandleTest in the build process.
It also improves the FileTest to use TempFilePath and the linkage.

Fixed the format check


text:
Add support for DICTIONARY encoding to PrestoSerializer

Extend PrestoSerializer to support deserializing dictionary-encoded data.
The format of dictionary-encoded columns is:

4 bytes: number of rows
N bytes: dictionary column
4*numRows bytes: indices
24 bytes: 'instance id' (used by Presto, but not present in Velox)

Part of #6595

@pedroerp Pedro, thank you for review. Addressed comments and updated the PR.


text:
Download source code archive directly for ubuntu and macos setup

It is better to use wget_and_untar to download the source code archive
directly, instead of using git to clone the source code. Because direct download
is much faster than using git clone. The setup-centos8.sh script has been
consistently downloading source code archive directly.

In our test environment, for boost, it took about 11 min before (from 2024-03-21T06:20:47 to 2024-03-21T06:31:13):
2024-03-21T06:20:47.3262778Z + run_and_time install_boost
2024-03-21T06:20:47.3263537Z + install_boost
2024-03-21T06:20:47.3264743Z + github_checkout boostorg/boost boost-1.84.0 --recursive
2024-03-21T06:20:47.3271539Z + local REPO=boostorg/boost
2024-03-21T06:20:47.3272490Z + shift
2024-03-21T06:20:47.3273275Z + local VERSION=boost-1.84.0
2024-03-21T06:20:47.3274005Z + shift
2024-03-21T06:20:47.3274716Z + local GIT_CLONE_PARAMS=--recursive
2024-03-21T06:20:47.3275628Z ++ basename boostorg/boost
2024-03-21T06:20:47.3276434Z + Finished running install_fmt
2024-03-21T06:20:47.3277281Z + local DIRNAME=boost
2024-03-21T06:20:47.3278123Z + SUDO='sudo --preserve-env'
2024-03-21T06:20:47.3279167Z + cd /opt/gluten/ep/build-velox/build/velox_ep
2024-03-21T06:20:47.3280153Z + '[' -z boost ']'
2024-03-21T06:20:47.3281462Z + '[' -d boost ']'
2024-03-21T06:20:47.3282170Z + '[' '!' -d boost ']'
2024-03-21T06:20:47.3283545Z + git clone -q -b boost-1.84.0 --recursive https://github.com/boostorg/boost.git
2024-03-21T06:21:00.5554970Z Note: switching to 'ad09f667e61e18f5c31590941e748ac38e5a81bf'.
2024-03-21T06:21:00.5556728Z
2024-03-21T06:21:00.5557598Z You are in 'detached HEAD' state. You can look around, make experimental
2024-03-21T06:21:00.5559211Z changes and commit them, and you can discard any commits you make in this
2024-03-21T06:21:00.5560889Z state without impacting any branches by switching back to a branch.
2024-03-21T06:21:00.5561865Z
2024-03-21T06:21:00.5562552Z If you want to create a new branch to retain commits you create, you may
2024-03-21T06:21:00.5564281Z do so (now or later) by using -c with the switch command. Example:
2024-03-21T06:21:00.5565210Z
2024-03-21T06:21:00.5565585Z   git switch -c <new-branch-name>
2024-03-21T06:21:00.5667647Z
2024-03-21T06:21:00.5668525Z Or undo this operation with:
2024-03-21T06:21:00.5669179Z
2024-03-21T06:21:00.5669430Z   git switch -
2024-03-21T06:21:00.5788570Z
2024-03-21T06:21:00.5869925Z Turn off this advice by setting config variable advice.detachedHead to false
2024-03-21T06:21:00.5948424Z
2024-03-21T06:31:13.8519110Z + cd boost

It took about 20s now (from 2024-03-21T08:25:15 to 2024-03-21T08:25:32)
2024-03-21T08:25:15.1901014Z + run_and_time install_boost
2024-03-21T08:25:15.1902104Z + install_boost
2024-03-21T08:25:15.1904364Z + wget_and_untar https://github.com/boostorg/boost/releases/download/boost-1.84.0/boost-1.84.0.tar.gz boost
2024-03-21T08:25:15.1907965Z + local URL=https://github.com/boostorg/boost/releases/download/boost-1.84.0/boost-1.84.0.tar.gz
2024-03-21T08:25:15.1909631Z + local DIR=boost
2024-03-21T08:25:15.1910348Z + mkdir -p boost
2024-03-21T08:25:15.1922276Z + pushd boost
2024-03-21T08:25:15.1924758Z + curl -L https://github.com/boostorg/boost/releases/download/boost-1.84.0/boost-1.84.0.tar.gz
2024-03-21T08:25:15.1927531Z /opt/gluten/ep/build-velox/build/velox_ep/fmt/boost /opt/gluten/ep/build-velox/build/velox_ep/fmt
2024-03-21T08:25:15.2047924Z   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
2024-03-21T08:25:15.2049463Z                                  Dload  Upload   Total   Spent    Left  Speed
2024-03-21T08:25:15.2050292Z
2024-03-21T08:25:16.2292144Z   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
2024-03-21T08:25:16.2293827Z   0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0
2024-03-21T08:25:16.2295320Z   0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0
2024-03-21T08:25:17.3928559Z
2024-03-21T08:25:18.1827510Z   0  118M    0  7421    0     0   3391      0 10:08:26  0:00:02 10:08:24  3391
2024-03-21T08:25:19.1764844Z   0  118M    0  191k    0     0  65753      0  0:31:22  0:00:02  0:31:20  232k
2024-03-21T08:25:20.1637281Z   4  118M    4 5913k    0     0  1489k      0  0:01:21  0:00:03  0:01:18 3310k
2024-03-21T08:25:21.1679483Z  15  118M   15 18.4M    0     0  3807k      0  0:00:31  0:00:04  0:00:27 6811k
2024-03-21T08:25:22.1508048Z  26  118M   26 31.2M    0     0  5364k      0  0:00:22  0:00:05  0:00:17 8469k
2024-03-21T08:25:23.1499963Z  37  118M   37 44.0M    0     0  6490k      0  0:00:18  0:00:06  0:00:12 9473k
2024-03-21T08:25:24.1504551Z  48  118M   48 56.9M    0     0  7343k      0  0:00:16  0:00:07  0:00:09 11.4M
2024-03-21T08:25:25.1513762Z  59  118M   59 69.8M    0     0  8000k      0  0:00:15  0:00:08  0:00:07 12.8M
2024-03-21T08:25:26.1492539Z  70  118M   70 82.8M    0     0  8525k      0  0:00:14  0:00:09  0:00:05 12.9M
2024-03-21T08:25:27.1490347Z  81  118M   81 95.8M    0     0  8966k      0  0:00:13  0:00:10  0:00:03 12.9M
2024-03-21T08:25:27.7938709Z  92  118M   92  108M    0     0  9332k      0  0:00:12  0:00:11  0:00:01 12.9M
2024-03-21T08:25:27.7941828Z 100  118M  100  118M    0     0  9603k      0  0:00:12  0:00:12 --:--:-- 13.1M
2024-03-21T08:25:27.8029221Z + tar -xz --strip-components=1 -f boost.tar.gz
2024-03-21T08:25:32.2022371Z + popd
2024-03-21T08:25:32.2023384Z + cd boost

@kgpai Could you help to review?

cc @assignUser. Could you help to review?

@assignUser Could you help review again? I have added the functionality to change directory to a specific path in the cmake_install method.

Sorry for the delay, could you rebase again?

@assignUser Rebased to the latest main branch.

@liujiayi771 I havent noticed any particular slowness with git and it seems roughly equivalent to download. The advantage of using git is that its probably a little more secure than plain old downloads of tar.gz's (FYI the checksums we calculate arent really secure anyway).
Do other folks see significant differences b/w git and downloading tar.gz's ?
@majetideepak ?

@kgpai In my test, the time difference for Boost is quite significant.

Sorry, I missed this PR. I think we should uniformly use tar.gz for all because 1) We can then use a common shared file to download and install all the dependencies. 2) Downloading a single file should be faster than git (involves extra metadata + processing)?
We can also include the checksum in a single file to validate.
@liujiayi771 can you please rebase this PR?

@majetideepak Rebased to the latest main branch.

@Yuhta, can you please import this PR? Thanks!

authors: assignUser kgpai liujiayi771 majetideepak


text:
training ESM-2 using bf16 or fp16 or tf32 or fp32?

Hi,
Congrats on your great work on ESM-2.
I wonder what float point you used in training ESM-2, since I find using fp16 and using bf16 yields different numerical numbers as follows:

bf16['representations'][1]
tensor([[ 0.0825, -0.1167, -0.0654,  ..., -0.1011, -0.0023, -0.1523],
[ 0.0603, -0.0815,  0.0459,  ..., -0.0557,  0.0320, -0.1641],
[ 0.1045, -0.0854, -0.0698,  ..., -0.1143,  0.1016, -0.1533],
...,
[ 0.0464, -0.0386, -0.0425,  ..., -0.0144,  0.1045, -0.0649],
[ 0.0273, -0.1060, -0.1289,  ..., -0.0115,  0.1157, -0.0693],
[ 0.0273, -0.1060, -0.1289,  ..., -0.0115,  0.1157, -0.0693]],
dtype=torch.bfloat16)
fp16['representations'][1]
tensor([[ 0.0814, -0.1161, -0.0657,  ..., -0.1006, -0.0029, -0.1522],
[ 0.0607, -0.0811,  0.0446,  ..., -0.0558,  0.0321, -0.1647],
[ 0.1031, -0.0839, -0.0707,  ..., -0.1141,  0.1027, -0.1550],
...,
[ 0.0468, -0.0378, -0.0432,  ..., -0.0145,  0.1046, -0.0642],
[ 0.0263, -0.1041, -0.1304,  ..., -0.0126,  0.1151, -0.0701],
[ 0.0264, -0.1042, -0.1304,  ..., -0.0125,  0.1152, -0.0698]],
dtype=torch.float16)

ESM-2 was trained in fp16. The examples you show seem totally reasonable numerical precision deviations

Thanks!

repo: facebookresearch/esm


text:
how to use the scripts in wav2vec repository to generate manifest files .ltr for LibriSpeech.

Prepare manifest of the dataset
Use the scripts in wav2vec repository to generate manifest files (.tsv and .ltr) for LibriSpeech. The manifest of other datasets follows the similar format as that of the Librispeech. The .tsv file contains one extra column for class labels

The .ltr file contains columns for class labels, prompts and transcriptions

i use manifest.py of wav2vec to generate train.tsv,but i don't konw how to generate files about .ltr.

https://github.com/facebookresearch/fairseq/blob/main/examples/wav2vec/README.md#prepare-training-data-manifest

https://github.com/facebookresearch/fairseq/blob/main/examples/wav2vec/README.md#prepare-training-data-manifest python examples/wav2vec/wav2vec_manifest.py /path/to/waves --dest /manifest/path --ext $ext --valid-percent $valid

thank you ，i already finished it.

hey how did you do it, I am stuck at this.

authors: guobabaya noitanec srksaurabh1

repo: facebookresearch/fairseq


text:
how to implement this: torchvision.transforms.functional.perspective

Area Select
react-native-pytorch-core (core package)
Description
Hello! thanks for contributions!
I have a problem while developing my project.
I need a function like torchvision.transforms.functional.perspective
Could you add this implementation for torchvision.transforms.functional.perspective?
or
Can i implement this function?
There is no implementation of perspective function in playtorch docs
Another solution that i proceed is making pytorch mobile model for this function.
This idea came from @raedle of this issue.
but it has a error at react-native app like this:
{"message": "Calling torch.linalg.lstsq on a CPU tensor requires compiling PyTorch with LAPACK. Please use PyTorch built with LAPACK support.

  Debug info for handle(s): debug_handles:{-1}, was not found.

Exception raised from apply_lstsq at ../aten/src/ATen/native/BatchLinearAlgebraKernel.cpp:559 (most recent call first):
(no backtrace available)"}

Should i question to pytorch github about this error?
My perspective model is like this:
This model is successful at python code.
import torch, torchvision
from typing import List, Dict
import torchvision.transforms.functional as F

class WrapPerspectiveCrop(torch.nn.Module):
    def __init__(self):
        super().__init__()

    def forward(self, inputs: torch.Tensor, points: List[List[int]]):
        size_points = [[0,0], [inputs.shape[2],0] , [inputs.shape[2],inputs.shape[1]], [0,inputs.shape[1]]]
        inputs = F.perspective(inputs, points, size_points)
        return inputs

crop = WrapPerspectiveCrop()
scripted_model = torch.jit.script(crop)
scripted_model.save("wrap_perspective.pt")

import torch
from torch.utils.mobile_optimizer import optimize_for_mobile

optimized_scripted_module=optimize_for_mobile(scripted_model)
optimized_scripted_module._save_for_lite_interpreter("wrap_perspective.ptl")

How can i solve this problem?
Many many thanks for anyone help!

Hi @nh9k, unfortunately there is no quick workaround. The PyTorch Mobile shared object library is not linked against LAPACK. It would require the PyTorch team to add an options to compile the PyTorch Mobile libraries with LAPACK for Android/iOS.
As suggested, you could post in the PyTorch repo and ask for support. If it's added to the PyTorch Mobile shared object libraries, we can pull in the new libraries into PlayTorch!

Hi @nh9k, unfortunately there is no quick workaround. The PyTorch Mobile shared object library is not linked against LAPACK. It would require the PyTorch team to add an options to compile the PyTorch Mobile libraries with LAPACK for Android/iOS.
As suggested, you could post in the PyTorch repo and ask for support. If it's added to the PyTorch Mobile shared object libraries, we can pull in the new libraries into PlayTorch!
Closing the issue as it can't be resolved without PyTorch Mobile being compiled with LAPACK support. Feel free to reopen if this changes

Thanks @raedle, If I solve this problem from asking the PyTorch team for LAPACK, I report back!

@raedle the posted issue also talks about XNNPACK but that should not be the issue. Are you not building with USE_XNNPACK=1 for pytorch?

@kimishpatel, the PlayTorch API doesn't build PyTorch Mobile from source but uses the following PyTorch Mobile build artifacts

Android: org.pytorch:pytorch_android_lite:1.12.2
iOS: LibTorch-Lite ~> 1.12.0

Are the released PyTorch Mobile build artifacts build with USE_XNNPACK=1?

I am not entirely sure. Let me get back to you on this. Also is this on ios or android?

Thanks @kimishpatel!
@nh9k, is the issue with XNNPACK on both platforms Android and iOS or just on one of the two platforms?

Thanks @kimishpatel @raedle!
The issue is occured on Android. I haven't tested it in IOS.

Yeah for android it should be on by default.

@raedle, so, can i solve this problem?
How can i build react-native app with XNNPACK?
Writing dependencies implementation "org.pytorch:pytorch_android_lite:1.12.2"  at build.gradle file can solve this problem?

@nh9k, if symbols are part of org.pytorch:pytorch_android_lite:1.12.2, then you shouldn't need to do anything. The latest react-native-pytorch-core release v0.2.1 uses libpytorch_jni_lite.so from org.pytorch:pytorch_android_lite:1.12.2.
What is the react-native-pytorch-core version that you tested?
Can you share a simple model + python export code for us to test as well?

@raedle, Thank you so much!
My react-native-pytorch-core version is v1.0.0, because i have some errors in windows to install packages.
I have another problem with the inference of quantized model, which is probably the same problem.
Thank you so much again, I will change pytorch core version to v0.2.1 and report back!

@nh9k, are you using torch.utils.mobile_optimizer.optimize_for_mobile on the model that throws the XNNPACK error (see below)?
XNNPACK Convolution not usable! Reason: The provided input tensor is either invalid or unsupported by XNNPACK

If that's the case, can you try adding optimization_blocklist for INSERT_FOLD_PREPACK_OPS for the optimize_for_mobile function?
optimization_blocklist = {
  MobileOptimizerType.INSERT_FOLD_PREPACK_OPS,
}

optimized_model = torch.utils.mobile_optimizer.optimize_for_mobile(model, optimization_blocklist)

Insert and Fold prepacked ops (blocklisting option MobileOptimizerType::INSERT_FOLD_PREPACK_OPS): This optimization pass rewrites the graph to replace 2D convolutions and linear ops with their prepacked counterparts. Prepacked ops are stateful ops in that, they require some state to be created, such as weight prepacking and use this state, i.e. prepacked weights, during op execution. XNNPACK is one such backend that provides prepacked ops, with kernels optimized for mobile platforms (such as ARM CPUs). Prepacking of weight enables efficient memory access and thus faster kernel execution. At the moment optimize_for_mobile pass rewrites the graph to replace Conv2D/Linear with 1) op that pre-packs weight for XNNPACK conv2d/linear ops and 2) op that takes pre-packed weight and activation as input and generates output activations. Since 1 needs to be done only once, we fold the weight pre-packing such that it is done only once at model load time. This pass of the optimize_for_mobile does 1 and 2 and then folds, i.e. removes, weight pre-packing ops.

More details: https://pytorch.org/docs/stable/mobile_optimizer.html

@raedle, Thank you so much!!
I have the same error (xnnpack) chaning react-native-pytorch-core version to v0.2.1.
The code with from torch._C import MobileOptimizerType you recommended got a error,
{"message": "expected scalar type Byte but found Float

  Debug info for handle(s): debug_handles:{-1}, was not found.

Exception raised from data_ptr at aten/src/ATen/core/TensorMethods.cpp:20 (most recent call first):
(no backtrace available)"}

I don't know yet, why it is appear, it is probably my fault.
I don't work long time from now on, about a week, i would like to share my projects but i have no enough time for organizing my code. Sorry about this, If you would like to know the model code, My model is related about this repository.
Thank you so much again, I will back next week..

@kimishpatel yes, converted .ptl model works fine from python code!

@nh9k, can you share the model and the code used to export the model for the lite interpreter runtime?

@raedle, sorry i am late..
Can i share you my project using my private repository?
And my final aim is exporting quantization model with react-native app.
But it also has a problem. so can you test with quantization model?
If possible, i will share this quantized model .pt file(before making .ptl file) from my google drive.
The quantized model works well from python code.

@nh9k, instead of sharing the private repo, can you please create a reproducible example publicly? This way, the community can also benefit if anyone has a similar issue :)

Alright! The code used to export the model is here.
CODE:
import torch
from torch.utils.mobile_optimizer import optimize_for_mobile

model = torch.jit.load('model_scripted.pt')
model.eval()

device = torch.device('cpu')
x = torch.randn(1, 3, 768, 768).to(device)

# Use torch.jit.trace to generate a torch.jit.ScriptModule via tracing.
traced_script_module = torch.jit.trace(model, x)

from torch._C import MobileOptimizerType

optimization_blocklist = {
    MobileOptimizerType.INSERT_FOLD_PREPACK_OPS,
}

optimized_scripted_module = optimize_for_mobile(traced_script_module, optimization_blocklist)

optimized_scripted_module._save_for_lite_interpreter("model.ptl")

The model .pt file is here(my google drive)
Thank you so much, raedle.

@nh9k, I was somewhat successful. The exported model loads on Android in the PlayTorch app, and it can run inference with a random tensor as input. There is an issue on iOS that I need to look into (i.e., iOS crashes with this model).
Model export
I used the export similar to what you provided. The only change is that the module is already a ScriptModule and doesn't need to be traced but can be used directly. I exported the model with and without optimization_blocklist.
import torch
from torch.utils.mobile_optimizer import optimize_for_mobile

model = torch.jit.load('model_dl.pt')
model.eval()

device = torch.device('cpu')
x = torch.randn(1, 3, 768, 768).to(device)

from torch._C import MobileOptimizerType

optimization_blocklist = {
    MobileOptimizerType.INSERT_FOLD_PREPACK_OPS,
}

optimized_scripted_module = optimize_for_mobile(model)
optimized_scripted_module_with_blocklist = optimize_for_mobile(model, optimization_blocklist)

optimized_scripted_module._save_for_lite_interpreter("optimized_scripted_module.ptl")
optimized_scripted_module_with_blocklist._save_for_lite_interpreter("optimized_scripted_module_with_blocklist.ptl")

In python, I then reloaded the lite interpreter model and ran inference with a random tensor. It outputs a tuple with two tensors (assuming the tensors are in correct shape).
from torch.jit.mobile import _load_for_lite_interpreter

model2 =_load_for_lite_interpreter("optimized_scripted_module_with_blocklist.ptl")

with torch.no_grad():
  a, b = model2(torch.randn(1, 3, 768, 768))

print("a.shape", a.shape)
print("b.shape", b.shape)

I also logged the export_opnames for the input model, the optimized model, and the optimized model with the blocklist to show the ops.
Example:
torch.jit.export_opnames(optimized_scripted_module_with_blocklist)

Output:
['aten::cat',
 'aten::conv2d',
 'aten::max_pool2d',
 'aten::permute',
 'aten::relu_',
 'aten::size.int',
 'aten::upsample_bilinear2d']

Colab notebook with code from above: https://colab.research.google.com/drive/1JzjL7RZd4_ldgoc-7cJ02RUl53sIMhjr
Use model with PlayTorch
async function main() {
  try {
    console.log('loading model');
    const filePath = await MobileModel.download(
      'https://example.com/path/to/optimized_scripted_module_with_blocklist.ptl'
    );
    // or loading the model as project asset
    //const filePath = await MobileModel.download(
    //  require('./path/to/optimized_scripted_module_with_blocklist.ptl')
    //);
    const model = await torch.jit._loadForMobile(filePath);
    const output = await model.forward(torch.randn([1, 3, 768, 768]));
    console.log('output value', output);
  } catch (error) {
    console.error(error);
  }
}
main();

loading model
output value [{"dtype":"float32","shape":[1,384,384,2]},{"dtype":"float32","shape":[1,32,384,384]}]

@raedle, thank you so much!!, i am successful too.
I'm not sure why the previous model didn't work.
Thank you so much!!

@raedle, Is it still being tested on iOS? I haven't tested it on iOS yet.
What issues was there? I think i will test it within a month, too.

Hi, @raedle,
I have a problem, would you help me?
I expected a float32 tensor output(e.g., 0.         0.01859372 0.         ... 0.), but I got the output data 0 or 1 of integer when i printed the outputTensor.data() using console.log function. It is very strange.

@raedle, I figure out the blocklist was problem. When i removed this blocklist argument optimization_blocklist, the model output works fine as float output on react-native app. Thanks a lot for your help! I need to study more about this blocklist.

authors: kimishpatel nh9k raedle

repo: facebookresearch/playtorch


text:
🛑 fiscaliadeestado.misiones.gob.ar is down

In 83a7531, fiscaliadeestado.misiones.gob.ar (https://fiscaliadeestado.misiones.gob.ar/) was down:

HTTP code: 0
Response time: 0 ms

Resolved: fiscaliadeestado.misiones.gob.ar is back up in aa9d5ff.


text:
Can't load 'apply form' because of HTTPS

Roll a script to replace it or suffer the consequences @reederz
It's on {host}/da/apply

@danmindru I've added the form directly into the app and built the backend for it, which is running on our server. Can you do input validation (name, email required) and success, failure feedback?

Don't you wanna add a field for website/github? Otherwise good to go.

@danmindru Regarding fields - no, I don't think so. To make it even more generic, we could remove the education field. Can you do it in frontend? I'll take care of the backend.

All good now. Closing the issue.

authors: danmindru reederz

repo: fadeit/fadeitdk


text:
Pod creation delayed in UI after changing to gateway URL

After switching the URL to the gateway URL, I attempted to create a new pod. Blossom did create it, but the new pod only appeared in the Fairdrive UI after multiple reloads.
Steps to reproduce:

Change URL to gateway URL.
Try creating a new pod.
Notice that the pod is not immediately visible in the Fairdrive UI.

Expected behavior:
The new pod should appear in the UI immediately after creation.
Actual behavior:
The new pod only shows up after several reloads of the Fairdrive UI.
Environment:

Blossom version: https://github.com/fairDataSociety/blossom/pull/156

https://github.com/fairDataSociety/blossom/assets/1595061/ae97970a-c3fa-4d54-ad37-ef04db53749f

This was a Blossom issue and it has been fixed https://github.com/fairDataSociety/blossom/pull/156.
fdp-storage cache is disabled for now, because in order to support caching additional work is required.

authors: IgorShadurin tomicvladan

repo: fairDataSociety/blossom


text:
Validate application after angular 17 upgrade.

Validate the application after the upgrade to angular version 17.

Application validated.


text:
Revisit RequestOptions.auto_parse_form_urlencoded

Create a media handler for application/x-www-form-urlencoded that can be used via req.media. As part of this work, deprecate the use of the auto_parse_form_urlencoded (which merges form params into query string params) by adding a note to the docs to that effect.
The implementation can be patterned after Request._parse_form_urlencoded()

Work remaining after https://github.com/falconry/falcon/pull/1537:

Decide whether we want to add this to default handlers (@nZac expressed support for this in another discussion IIRC, I agree)
Deprecate auto_parse_form_urlencoded in the docs

repo: falconry/falcon


text:
update(pkg/driverbuilder): debian support for cloud-amd64 extraversion

Co-Authored-By: Leonardo Di Donato leodidonato@gmail.com
Signed-off-by: Lorenzo Fontana fontanalorenz@gmail.com
What type of PR is this?
/kind feature
Any specific area of the project related to this PR?
/area pkg
What this PR does / why we need it:
@danpopSD reported that they wanted to try Falco on a  4.19.0-10-cloud-amd64 kernel.
This PR makes sure that driverkit does not ignore the cloud extraversion and thus can build for said kernel group.

Which issue(s) this PR fixes:
Fixes #
Special notes for your reviewer:
Does this PR introduce a user-facing change?:
new: build support for debian cloud-amd64 kernels

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To complete the pull request process, please assign fntlnz
You can assign the PR to them by writing /assign @fntlnz in a comment when ready.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

LGTM label has been added.  Git tree hash: 14e2e4f035d936da4c101020ee64f3fc526d1fc8

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: leodido
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [leodido]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

authors: fntlnz poiana

repo: falcosecurity/driverkit


text:
Falco upcoming events

What type of PR is this?

Uncomment one (or more) /kind <> lines:

/kind bug

/kind cleanup

/kind design

/kind user-interface

/kind content

/kind event
Any specific area of the project related to this PR?

Uncomment one (or more) /area <> lines:

/area blog

/area documentation

/area community
What this PR does / why we need it:
We can help the community sign-up to out Falco party if they are present at Kubecon NA in Salt Lake City
Which issue(s) this PR fixes:

Fixes #
Special notes for your reviewer:

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: nigel-falco
Once this PR has been reviewed and has the lgtm label, please assign jasondellaluce for approval. For more information see the Kubernetes Code Review Process.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

LGTM label has been added.  Git tree hash: 735dc28387de747f76deea600c4b6b0014378e25

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: leogr, nigel-falco, vjjmiras
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [leogr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

authors: nigel-falco poiana

repo: falcosecurity/falco-website


text:
Improve getting started minikube docs

What type of PR is this?
/kind cleanup
Any specific area of the project related to this PR?
/area documentation
What this PR does / why we need it:
Improves the wonderful documentation
Which issue(s) this PR fixes:
Fixes #399
Special notes for your reviewer:

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To complete the pull request process, please assign rajakavitha1 after the PR has been reviewed.
You can assign the PR to them by writing /assign @rajakavitha1 in a comment when ready.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Welcome @mjlshen! It looks like this is your first PR to falcosecurity/falco-website 🎉

authors: mjlshen poiana


text:
rule(falco_rules): update evt.type for spawned process macro

What type of PR is this?
/kind rule-update
Any specific area of the project related to this PR?
/area rules
What this PR does / why we need it:
As suggested here by we must monitor also execveat events to check if a new process is spawned.
Which issue(s) this PR fixes:
Ref. #1857
Special notes for your reviewer:
Does this PR introduce a user-facing change?:
rule(macro spawned_process): monitor also processes spawned by `execveat`

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: Andreagit97
To complete the pull request process, please assign mstemm after the PR has been reviewed.
You can assign the PR to them by writing /assign @mstemm in a comment when ready.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

rules/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

This is great @Andreagit97 ! Thanks. This LGTM, just make sure of using the rule(...): ... notation in the release note. Take some older rule-related PR as reference, as that line will be used to generate the changelog of the upcoming release.

I fixed it, thank you!

/milestone 0.31.0

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: Andreagit97, leogr
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

rules/OWNERS [leogr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

LGTM label has been added.  Git tree hash: 533d8f471c4f1c62bd2633ced3a3fcd333c255ff

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: Andreagit97, jasondellaluce, leogr
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

rules/OWNERS [jasondellaluce,leogr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

authors: Andreagit97 jasondellaluce leogr poiana

repo: falcosecurity/falco


text:
Collaborative Coding with Teletype

Currently teletype doesn't have any features to send custom events.
so we have to send VEDA commands ourselves using some services like Puhser
https://pusher.com/

Pusher requires user authentication for client events...

repo: fand/atom-veda


text:
Add documentation regarding testing

Following discussion #672, we should add details in the documentation about how you can properly override the user dependency during testing.

Is this the same as https://fastapi-users.github.io/fastapi-users/cookbook/create-user-programmatically/?

@luisroel91 No, it will be slightly different.
The goal is to add some guidance to properly write unit tests with FastAPI Users, especially how to use the dependency_overrides mechanism of FastAPI to "mock" the current_user dependency.
I've added some insights here: https://github.com/fastapi-users/fastapi-users/discussions/830#discussioncomment-1874201

@frankie567 added pr to show the example of testing fastapi-users

repo: fastapi-users/fastapi-users


text:
✏ Fix import typo in reference example for Security

Description:
Seems the example does not import Security from fastapi instead points to Depends

Good catch, thank you! :mag:

repo: fastapi/fastapi


text:
Authorization headers not passed on request from UI

Prerequisites

[X] I have written a descriptive issue title
[X] I have searched existing issues to ensure the bug has not already been reported

Fastify version
4.18.0
Plugin version
1.9.0
Node.js version
16
Operating system
Linux
Operating system version (i.e. 20.04, 11.3, 10)
22.04
Description
Authorization headers not passed on request from UI:

Steps to Reproduce
import { fastify } from "fastify";
import fs from "@fastify/swagger";
import fsu from "@fastify/swagger-ui";

const app = fastify();

await app.register(fs, {
  openapi: {
    openapi: "3.1.0",
  },
});

await app.register(fsu);

app.get(
  "/",
  {
    schema: {
      headers: {
        type: "object",
        properties: {
          authorization: { type: "string" },
        },
        required: ["authorization"],
      },
    },
  },
  async (request, reply) => {
    return { hello: "world" };
  }
);

app
  .listen({ port: 3001 })
  .then(() => {
    console.log("listening");
  })
  .catch((err) => console.log(err));

Expected Behavior
curl request should include authorization header:
curl -X 'GET' \
  'http://localhost:3001/' \
  -H 'accept: */*' \
  -H 'test: t'
  -H 'authorization: a'

Based on openapi spec it's required to use securitySchemes to allow authorization headers
      components: {
        securitySchemes: {
          bearerAuth: {
            type: 'http',
            scheme: 'bearer',
          },
        },
      },

repo: fastify/fastify-swagger-ui


text:
Screenshot resume feature

Requesting a resume feature. Screenshots can take hours. One time my computer ran out of juice in the process. It was painful.

Taking screenshots or uploading screenshot?
How should this work for the users?
How could this work on the tech side?
(The issue template is there for a reason, it helps you create issues with the information necessary.)

Sorry about that.
During the taking screenshots part.
For the user, the default behavior could be:

Existing screenshots found, skipping these configurations:
en-US—iPhone 5s, en-US—iPhone 6...
To recreate those screenshots, delete them from the directory or add "clear_previous_screenshots(true)" to your Snapfile.

On the tech side, this could be implemented by using a glob to check for existing files before beginning to target a device+language combination.

I have the vague feeling that we already have something similar somewhere, but I can't pinpoint it right now.
Your request looks reasonable and useful.
The current suggested implementation details might be a breaking change, so we would probably prefer to add an option like e.g. skip_existing_screenshots or similar - but I would have to look deeper into the details of this.

authors: fulldecent janpio

repo: fastlane/fastlane


text:
ERROR ITMS-90725: "SDK Version Issue. This app update was built with the iOS SDK. New apps for iPhone or iPad must be built with the iOS 13 SDK or later.

New Issue Checklist

[ ] Updated fastlane to the latest version
[ ] I read the Contribution Guidelines
[ ] I read docs.fastlane.tools
[ ] I searched for existing GitHub issues

Issue Description

Command executed

Complete output when running fastlane, including the stack trace and command used

 [REPLACE THIS WITH YOUR INFORMATION]

Environment

 [REPLACE THIS WITH YOUR INFORMATION]

How is your question related to fastlane?
Seems you need to use the iOS 13 SDK as the error message tells you to.

Looks like something wrong in Apples end? We are hitting this as well.

Anyone had any luck find what is causing that issue?

Not yet, currently running the uploads manually. Will need to dig deeper into this. We are building using Unity as well, if that's of any relevance.

I'm hitting this issue as well

authors: apanagar darwingittest enggrakeshcse janpio jjlehtinen