text:
vm is always provisioned with alpine box

Hello,
I am trying to use this driver to test my role in a centos box but it is always using an alpine box, here is molecule.yml file:
dependency:
  name: galaxy
driver:
  name: vagrant
  provider:
    name: libvirt
    type: libvirt
platforms:
  - name: myrole
    box: centos/8
    memory: 512
    cpus: 1
    instance_raw_config_args:
      - vagrant.plugins = ["vagrant-libvirt"]
    config_options:
      ssh.keep_alive: yes
      ssh.remote_user: 'vagrant'
      synced_folder: false
    interfaces:
      - network_name: forwarded_port
        guest: 80
        host: 8080
    provision: no
provisioner:
  name: ansible
verifier:
  name: ansible

Output of molecule --version:
molecule 3.2.0a0
    ansible:2.10.3 python:3.6
    delegated:3.2.0a0 from molecule
    docker:0.3.3 from molecule_docker
    vagrant:0.6.1 from molecule_vagrant

Am I missing something or is the box property being used in the wrong place?
Thanks!

Since there's no release with a fix for that (it's only in HEAD), the line:
ssh.remote_user: 'vagrant'

is wrong. Please use:
ssh.remote_user: "'vagrant'"

I'm surprised that you managed to get a VM running, since the missing double quotes will lead to an invalid Vagrantfile and an error from vagrant.

ha, that did the trick, thanks a million :)

authors: apatard odra

repo: ansible-community/molecule-vagrant


text:
Https support

@sabre1041 it's less fine-grained than I would have like, but I would rather share it now, like that, than later on. Please review, let me know what you think,  but don't merge before a few days.

I'm closing this, because we need to rework other things and then this PR will be out of date. I also want to extract some changes done in this branch and merge them quickly into master.

source: gharchive/pull-request

license_type: permissive

license_source: github-api


text:
Add a RootView in order for service-index to be listed on /v1/

To be listed on /api/gateway/v1/ the applications must provide a root view named -view or -list responding with a list of nested endpoint
Gateway RootView expects registered applications to provide a rootview named -view/-list as implemented on https://github.com/ansible/aap-gateway/blob/devel/aap_gateway_api/views/api/v1/init.py

App can use EndpointEnumerator to filter out detail views

Extra Actions are shown at the top button for detail and list views.

Links don't work for test_app

Other PR is merging

authors: AlanCoding rochacbruno

repo: ansible/django-ansible-base


text:
Turek hron test

Isolated turek-hron example for documentation build.
Note:
Other examples have been deleted for this test.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 84.58%. Comparing base (0f0187e) to head (ac3238b).
Report is 33 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #378      +/-   ##
==========================================
- Coverage   85.21%   84.58%   -0.64%
==========================================
  Files          41       41
  Lines        2618     2634      +16
==========================================
- Hits         2231     2228       -3
- Misses        387      406      +19

@iboyd-ansys,
Updated as per review comments, It's more clear now for user to get understanding on end_time parameter & simulation run time as well!

repo: ansys/pysystem-coupling

license: MIT


text:
🐛[BUG]设定表格密度后，无法使用toolbar中密度控件再次调节

使用文档示例，如代码所示，在组件初始化时设定好表格密码后，使用toolbar中的密度条件控件，表格将无法调节密度

Ant Design Pro 版本: 4.1.0
umi 版本：3.1.1
浏览器环境：Chrome
开发环境 ： Windows

他是受控的，需要配合 onSizeChange 设置

source: gharchive/issue

repo: ant-design/pro-table


text:
use the plugin with vite-plugin-import-style, but style not loaded

config as follows:
// main.ts
import { createApp } from 'vue'
// import { Button } from 'ant-design-vue'
// import Antd from 'ant-design-vue'
import App from './App.vue'

import 'virtual:windi.css'

const app = createApp(App);

// app.use(Button)
// app.use(Antd)

app.mount('#root', true)

// vite.config.ts
import { defineConfig } from 'vite'
import vue from '@vitejs/plugin-vue'
import ViteComponents, {
  AntDesignVueResolver
} from 'vite-plugin-components'
// import ViteCompression from 'vite-plugin-compression'
import ViteStyleImport from 'vite-plugin-style-import'
import WindiCSS from 'vite-plugin-windicss'

// https://vitejs.dev/config/
export default defineConfig({
  // 1. If you are using the ant-design series, you need to configure this
  // 2. Make sure less is installed in the dependency `yarn add less -D`
  css: {
    preprocessorOptions: {
      less: {
        // modifyVars: {},
        javascriptEnabled: true,
      },
    },
  },
  plugins: [
    vue(),

    // https://github.com/antfu/vite-plugin-components
    ViteComponents({
      globalComponentsDeclaration: true,
      customComponentResolvers: [
        AntDesignVueResolver(),
      ]
    }),

    // https://github.com/windicss/vite-plugin-windicss
    WindiCSS(),

    // https://github.com/anncwb/vite-plugin-compression
    // ViteCompression(),

    // https://github.com/anncwb/vite-plugin-style-import
    ViteStyleImport({
      libs: [
        {
          libraryName: 'ant-design-vue',
          esModule: true,
          resolveStyle: (name) => {
            return `ant-design-vue/es/${name}/style/index`;
          },
        }
      ]
    }),
  ]
})

<template>
  <a-button type="primary">Ant Button</a-button>
</template>

style not generate

what should i do

whole repo
https://github.com/agoni1212/vite-plugins-test

You can use customComponentResolvers with sideEffects option to import the style (no need to use ViteStyleImport).
Here is my config:
customComponentResolvers: [
  (name) => {
    if (name.match(/^A[A-Z]/)) { // Ant Design Vue
      const partialName = name.slice(1)
      return {
        importName: partialName,
        path: 'ant-design-vue/es',
        sideEffects: `ant-design-vue/es/${partialName}/style`,
      }
    }
  }
]

The full example of this can be found here https://github.com/ngekoding/vite-project-explore

This is very helpful
appreciate @ngekoding

@ngekoding
need to handle the dirName in case of menuitem in menu ,layout-sider in layout, etc
otherwise error
[vite] Internal server error: Failed to resolve import "ant-design-vue/es/layout-sider/style" from "src\App.vue". Does the file exist?

@agoni1212 Ah you are right, after trying that component I got errors.
Actually currently I use ElementUI library, so don't know that problem before, maybe I need make a test for adv next time.

repo: antfu/vite-plugin-components


text:
Ralentissements affichage

après plusieurs secondes de fonctionnement, le rafraîchissement des  graphiques semble faire des micro-pauses régulières. (presque toutes les secondes)

l'implémentation d'un buffer circulaire n'a pas amélioré  la situation.
le SpaceXwidget tourne bien quand il est seul

le gagnant est : def chart_unit(self):
for i in range(0, 10):
mylabel = CoreLabel(text=str(i - 9), font_size=15, color=(1, 1, 1, 1))
# Force refresh to compute things and generate the texture
mylabel.refresh()
# Get the texture and the texture size
texture = mylabel.texture
texture_size = list(texture.size)
self.g.append(Rectangle(pos=(17 + i * 30, 100), texture=texture, size=texture_size))

c'était bien ça... le 'append' est passé dans l'init


text:
jewler is messy and doesn't allow for many things, remove it

Jewler blows for a lot of reasons, but mostly because it dumps a whole lot of junk in your gem when it's really easy to make your own spec.
I suggest removing it, the stuff it generated, and making having a hand made one.

(Spoiler: I didn't)

repo: anthonator/skittles

license: mit

license_source: bigquery


text:
Swift Implementation

Hi, I'm trying to implement this great lib on swift, by writing this code:
    let fluidView = BAFluidView(frame: self.view.frame, startElevation: 0.9)!

    fluidView.strokeColor = UIColor.green
    fluidView.fillColor = Constants.COLOR_BLUE
    fluidView.keepStationary()
    fluidView.startAnimation()

    self.view.addSubview(fluidView)

and here is the preview:

I've been changing the value of startElevation from 0.1 to 9000, but the height of wave didn't change. I want to achieve that the height of wave is 1/4 of the height screen.
Thank you.

Hey I'm not sure if this will help you, but are you by chance implementing the code in viewDidLoad? If so, try implementing it in viewDidAppear. I was having the same problem, implemented it in viewDidAppear and now its working as expected.

Apologies for the late response! I agree with @ImmanuelKannan, have you checked where you initialize? It could be the case that your final frame isn't set yet if you're in viewDidLoad.

authors: ImmanuelKannan antiguab gemamegantara

repo: antiguab/BAFluidView


text:
Pass custom tfvar file to terraform validate

I'd like terraform validate to run with a -var-file=test/terraform.tfvars parameter.
Is there an easy way to pass this as an option to the terraform_validate.sh script?

It seems it should be possible to pass args into the script as follows:
-   repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v1.2.3
    hooks:
    -   id: flake8
        args: [--max-line-length=131]

See the docs: https://pre-commit.com/#passing-arguments-to-hooks
Is there any chance you could add support something like that in terraform-validate?

repo: antonbabenko/pre-commit-terraform


text:
Make separate .scss file

Is there any reason the scss is embedded in the components? It would be nice to have a separate scss file that I can customize with the rest of the scss in my project.

Closed for inactivity.
If someone else needs this and can guide me I will do it.
I could not find documentation about it.

authors: Paul-Todd antoniandre

repo: antoniandre/vue-cal


text:
UITableViewController

Hi, this doesn't work on when presenting from a UITableViewController :(

Hi 👋
Could you be more specific? What is not working?

On 5. Oct 2018, at 17:13, hobbesthetige notifications@github.com wrote:
Hi, this doesn't work on when presenting from a UITableViewController :(
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

It looks like the constraints don't work against the tableview in a UITableViewController subclass. I was able to work around by changing my controller to a UIViewController and adding a tableview to it.

authors: antoniocasero hobbesthetige

repo: antoniocasero/Panels


text:
Is it possible to useusername and password based verification , rather than public key

I do not want to use key based verification .  Rather than I would like to use user name and key based verification system

Currently, there is only support for single password (defined by --authentication-password) or public keys as you've said. To limit sish's complexity, I have intentionally omitted the use of a persistent data store, so specific user/password combinations aren't currently possible.
If there is significant interest, I may consider adding support. That being said, I do not currently plan on doing so. Would be happy to accept a PR if you're interested in adding support!

repo: antoniomika/sish


text:
BotBuilder and BotConnector architecture

Investigate Microsoft BotBuilder and BotConnector architecture

[ ] Create MarkupBuilder
[ ] Escaping message

Define replyTo. If message.to contains @thread.skype or @p2p.thread.skype then need to use mesage.to otherwise message.from

Create TextBuilder instead of Markupbuilder where each part of text should be represented as TextPart.
Create Smilies as one of implementation of available smilies in skype.
Bold as TextPart which will wrap text into bold
Italic as TextPart which will wrap text into italic

Link as TextPart which will print a link

Also need to create a TextParser which will be parse text of message Andrey create TextBuilder

Send typing request before each message activity

Implement a sending and receiving quotes
<quote author=\"LOGIN\" authorname=\"LOGIN\" conversation=\"ID\" guid=\"xdedd1401e88ff300ec7303 6b9661085b15c683021741a32a042178c230debd97\" timestamp=\"1475750831\"><legacyquote>[13:47:11] LOGIN: </legacyquote>на их исправление много времени уйдет?<legacyquote>\r\n\r\n<<< </legacyquote></quote>"

https://docs.oracle.com/javase/tutorial/i18n/text/boundaryintro.html

<URIObject type="Video.1/Flik.1" uri="https://static.asm.skype.com/pes/v1/items/c8d2e2ae0da64a4d8e3bf4fc854c8d25\" url_thumbnail="https://static.asm.skyp
e.com/pes/v1/items/c8d2e2ae0da64a4d8e3bf4fc854c8d25/views/thumbnail"> Moji <a href="https://login.skype.com/login/sso?go=xmmfallback&c8d2e2ae0da64a4d8e3bf4fc854c8d25
">https://login.skype.com/login/sso?go=xmmfallback&c8d2e2ae0da64a4d8e3bf4fc854c8d25<OriginalName v="The Secret Life of Pets" />

Implement sending and receiving {code}

repo: antonleliuk/spring-social-botFramework


text:
Vertical Perdent (Gradient) Effect Layer

Would be nice to have a vertical oriented gradient option in the Vertical Perdent (Gradient) Effect Layer.
Thanks in advance.

You can Rotate the Freestyle selection.

Didn't know about this. Thanks!

authors: Cheerpipe gitmacer

repo: antonpup/Aurora


text:
[Game request] Battlerite

http://store.steampowered.com/app/504370/

Duplicate of #348

authors: Ziomastorm simon-wh


text:
Lesson 162 Part 3: Connection to the database timeouts.

When connecting to the PostgreSQL database deployed from lessons/162/3-part it timeouts.
I've added SG group with inbound rule for PostgreSQL port 5432 and it solved the issue.
Terraform code is missing the part for SG definition.

thanks, @Brain2life. i think i missed it and didn’t test it because the primary goal was to show secrets. if you can, please create a PR; that would be great!

Hi @antonputra , created PR https://github.com/antonputra/tutorials/pull/340
Please have a look.

Thank you @Brain2life , just merged

repo: antonputra/tutorials


text:
Disable debug data on production

https://docs.angularjs.org/guide/production

this will come in handy: https://github.com/guzart/gulp-ng-constant


text:
antrea stopped working after a few hours

Describe the bug
Installed Kubernetes with antrea on Windows nodes, it works well in the beginning, pod can ping other Linux nodes' pods, Windows nodes show healthy.
After a few hours, the Windows nodes become unhealthy, and unreachable. I use another network connected to this Windows machine, found antrea-agent stopped running.
To Reproduce
Install Kubernetes with antrea on Windows nodes, wait for a few hours (6 to 20 hours), antrea-agent stopped running, windows machine become unreachable.
Expected
Antrea keep running, and Windows nodes keep reachable.
Actual behavior
Windows nodes become unhealthy, and unreachable.
Versions:
Antrea version: v1.2.0
OVS version: ovs-pkg_2_14_2
Windows version: 10.0.17763.2366
containerd: 1.5.4
crictl version: v1.20.0
Additional context
antrea-agent logs: https://gist.github.com/yongxiu/11bb7b882770beeb9d20b51a19dba51a
ovs-vswitchd logs: https://gist.github.com/yongxiu/714e27832d0d24ba326b057edbf7990f
ovsdb-server logs: https://gist.github.com/yongxiu/9ccc6c7270d4dac256d882d847112927
ipconfig and other commands output: https://gist.github.com/yongxiu/a64fbb3e4ce5bfe867b77692aec77898

@wenyingd : could you check this?

Hi all,
From Antrea Agent log, we see that the Windows host was experiencing shutdown at 2022/01/08 03:53:57
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0108 03:53:57.986370    2052 agent.go:68] Starting Antrea agent (version v1.2.0)
F0108 03:53:57.991249    2052 main.go:58] Error running agent: error with OS specific initializations: A system shutdown is in progress.

Then at 03:54:11, Antrea Agent was starting, I guess at this time the Windows host is restarted. But at this time Antrea Agent fails to connect to the kube-apiserver, which causes agent crashes again.
E0108 03:54:11.530552    3604 agent.go:644] Failed to get node from K8s with name vm-win-0-29c10b8f034bf8c-ade831d14b8a59b6: Get "https://10.128.0.15:443/api/v1/nodes/vm-win-0-29c10b8f034bf8c-ade831d14b8a59b6": dial tcp 10.128.0.15:443: connectex: A socket operation was attempted to an unreachable network.
F0108 03:54:11.531037    3604 main.go:58] Error running agent: error initializing agent: Get "https://10.128.0.15:443/api/v1/nodes/vm-win-0-29c10b8f034bf8c-ade831d14b8a59b6": dial tcp 10.128.0.15:443: connectex: A socket operation was attempted to an unreachable network.

When Antrea Agent crashes because of the initialization failure, HNS Network is not created yet.
From OVS logs, we see that the OVS Extension is not enabled on any VMSwitch after the Windows is restarted, which caused OVS failed to recreate the network interfaces for the internal ports saved in OVSDB.
2022-01-08T03:54:10.638Z|00034|bridge|INFO|bridge br-int: added interface antrea-tun0 on port 1
2022-01-08T03:54:11.018Z|00035|wmi|WARN|Open vSwitch Extension is not enabled on any switch
2022-01-08T03:54:11.020Z|00037|dpif_netlink|ERR|Could not create wmi internal port with name:antrea-gw0
2022-01-08T03:54:11.020Z|00038|bridge|WARN|could not add network device antrea-gw0 to ofproto (Invalid argument)
2022-01-08T03:54:11.020Z|00039|bridge|INFO|bridge br-int: added interface Ethernet on port 3
2022-01-08T03:54:11.037Z|00040|wmi|WARN|Open vSwitch Extension is not enabled on any switch
2022-01-08T03:54:11.037Z|00041|wmi|ERR|Success

I doubt the HNSNetwork is removed after Windows host restarted, which caused the VMSwitch (created along with the HNSNetwork) on that OVS Extension was enabled in antrea agent last run was gone. So OVS datapath is not working at that time. Besides, the Windows network adapter "Ethernet" was not removed from OVS bridge from OVS status, and the "Hyper-V Extensible Virtual Switch" property is possibly not removed (which was set by Windows host when using it to create the Transparent HNSNetwork), that adapter is not able to work as a normal NIC that can process L3 packets.  This caused the Windows host network is not accessible.
To verify it, please help provide the HNSNetwork (powershell > Get-HNSNetwork ) or VMSwitch (powershell > Get-VMSwitch ) configuration, and also print the network adapter "Ethernet" binding configuration (powershell > Get-NetAdapterBinding -Name "Ethernet" -ComponentID vms_pp)

To quickly recover the host, the operations should include:

Remove OVS configurations. ovs-vsctl.exe del-br br-int
Disable Hyper-V Extensible Virtual Switch on Ethernet. powershell > Set-NetAdapterBinding -Name "Ethernet" -ComponentID vms_pp -Enabled $false

After the above two steps, the Windows host network should be recovered, then antrea agent is possibly restarted, and recreate the HNSNetwork/OVS bridge
To avoid similar issue in future, a cleanup scripts should be registered to run before Windows host shutdown, including, 1) remove OVS bridge, 2) remove HNSNetwork. So that the Windows host is clean after restarted.

I tested it, and it works! Thank you so much!

Will it be better to call  1) remove OVS bridge, 2) remove HNSNetwork during machine starting up? In case the machine shutted down unexpectedly.

Will it be better to call 1) remove OVS bridge, 2) remove HNSNetwork during machine starting up? In case the machine shutted down unexpectedly.

Personally, I would prefer to call them when shutting down. It is because the HNSNetwork configuration is saved in memory or a temp file on Windows host by default which is possibly deleted before starting up, and some connection data might be lost at this time, so if we do cleanup at this time, I am afraid some legacy network interfaces (or registry data) is difficult to remove. Actually we used to hit such issues.

authors: jianjuns wenyingd yongxiu

repo: antrea-io/antrea


text:
Revert CI changes

Revert CI changes in #5440

We can continue to use the OVS container as the testing environment, it won't change the agent behavior in CI tests.


text:
Automated cherry pick of #6280: Add documentation for the sameLabels feature in ACNP (#6280)

Cherry pick of #6280 on release-2.0.
#6280: Add documentation for the sameLabels feature in ACNP (#6280)
For details on the cherry pick process, see the cherry pick requests page.

/skip-all

authors: Dyanngg tnqn


text:
comboId字段无法自定义吗?

问题描述
comboId字段无法自定义吗?,如果后端返回的数据不同,数据多的话,前端遍历一遍添加修改属性会又不必要的性能支出

重现链接
https://g6.antv.antgroup.com/manual/middle/elements/combos/defaultcombo
重现步骤
无
预期行为
无
平台

操作系统: [macOS, Windows, Linux, React Native ...]
网页浏览器: [Google Chrome, Safari, Firefox]
G6 版本: [4.5.1 ... ]

屏幕截图或视频（可选）
No response
补充说明（可选）
No response

不支持，这些字段都是固定的

authors: Aicmortal Yanyan-Wang


text:
Migrate CreateEdge Behavior

AntV Open Source Contribution Plan（可选/Optional）

[X] 我同意将这个 Issue 参与 OSCP 计划 / I agree to participate in the OSCP plan

Issue 类型 / Issue Type
中级任务 / Intermediate Task
任务介绍 / Task Description
迁移创建边交互 Demo

Migrate Create Edge behavior Demo
参考说明 / Reference Description
您可以参考旧代码并考虑复用其中的部分逻辑。但请注意以下几点：

需要调整配置项的命名，以保持与当前交互命名风格的一致性。
根据您的理解，您可以对部分逻辑进行修改或优化。
不要迁移 History 相关的逻辑（例如：graph.startHistoryBatch）

在正式开始之前，建议您简要了解新版交互实现的逻辑和配置项命名风格，例如：Drag Canvas和Drag Element。
在迁移过程中，如果遇到任何问题，欢迎在Issues区留言寻求帮助。

You are encouraged to consult the old code and consider reusing some of its logic. However, please keep in mind the following points:

The naming of configuration items needs to be adjusted to remain consistent with the current interaction naming conventions.
Based on your understanding, you are welcome to modify or enhance parts of the logic as you see fit.
Do not migrate History related logic (e.g. graph.startHistoryBatch)

Before you begin, it is advisable to familiarize yourself briefly with the logic and naming conventions of configuration items used in the newer versions of interaction implementations, such as: Drag Canvas and Drag Element.
Should any issues arise during the migration process, you are welcome to seek assistance by posting a message in the Issues section.

认领

authors: Aarebecca ai-qing-hai


text:
3.0.1 registerBehavior的监听在chrome v68, v70偶现交互监听没有绑定成功

G6 Version:  3.0.1
Platform: window 10 x64  chrome v68， chrome v70
Mini Showcase(like screenshots):  无
CodePen Link:  无

---------------- 检索了关闭的issues, 都并未实质解决， 希望能重视此类问题 ------------------
之前的同学都有类似的问题出现，是偶现的，不容易复现
#677 v3.0.1 node:click edge:click all don't work
#700 v3.0.2 的graph中设置node:click 无效，在chrome
#708 v3.0.4版本在chrome浏览器，edge:click点击事件有时会失效

我使用vue3.0cli全家桶开发， registerBehavior的监听在chrome v68, v70版本偶现交互监听没有绑定成功，chrome打断点debug不进入，devServer模式/npm run build模式（dist）都有问题。
window10 64位的开发环境（若干同window系统环境，机器测试都有此现象）
伪代码结构如下
let graph = new G6.Graph()
// 官方实例，点击节点添加连线的那个例子
G6.registerBehavior()
偶尔就会出现绑定不上的情况，
排查：
1 怀疑是webpack打包构建缓存的锅， npm run build还是出现，不是webpack的问题
2 怀疑是vue的mounted钩子没有触发（mounted调用的g6）， 实际测试mounted每次都会触发。

click 不 work 与注册行为没有关联。click 问题是电脑触摸板 force touch 等会出现，是已知问题。behavior 绑定问题请附源码 codepen

的确存在这个问题，原因是底层对事件处理那块考虑不全，后面会发一个版本，发布了后通知你，再更新下G就可以了

更新G版本到3.4.6

@baizn @elaine1234 更新G到3.4.6。使用内置behavior: click-select，仍然会出现onClick偶尔没有触发。
环境是 MacOS 10.14.2, Chrome 75。
但是在macbook touch上，tap to click 没有问题，click 会出现问题。

@wxy1995 G外网还没有发布新版本，你可以先按这个#pr改下node_modules中G对应的文件，你关注下G的版本，发布了后你更新下就OK了。

@baizn 什么时候g6 release一个新版解决这个问题？

这个问题什么时候修复呢

。。。

@baizn 什么时候g6 release一个新版解决这个问题？

目前G4.0正在在开发中, G6下个大版本会升级底层渲染引擎G。

@baizn G6啥时候升级大版本啊？😂

请关注开发计划文档，后续的发布计划都会更新进去的。

G6 3.4 已经发布，底层 G 已经升级。

authors: Manjushaka Raywh Shawn-Cui Yanyan-Wang avennn baizn elaine1234 tm-roamer wxy1995


text:
行政区中如果使用的是直辖市的省份地图，他的popup的code应该是110100而不是110000

L7 Version:
Platform:
Mini Showcase(like screenshots):
CodePen Link:

例如：
省级地图
选择的是北京市（直辖市）
depth ： 2
这个时候 ： popup HTML 回调返回 它会掉的code是 110000 即 北京市，而不是 110100北京市 直辖市
和其它的省级地图回调不一致

相关例子：https://codesandbox.io/s/gifted-pike-ysl6w?file=/index.js

@jefferyssy district 没有维护了，数据没有更新
请使用新版行政区域 ，使用案例
使用方式

按图表方式

import { Choropleth } from '@antv/l7plot';

const choropleth = new Choropleth('container', options);

按场景方式

import { Scene, Mapbox } from '@antv/l7';
import { Choropleth } from '@antv/l7plot';

const scene = new Scene({
  id: 'container',
  map: new Mapbox({
    style: 'light',
    center: [102.447303, 37.753574],
    zoom: 5,
  }),
});

const choropleth = new Choropleth(options);

scene.on('loaded', () => {
  choropleth.addToScene(scene);
});

API
Choropleth 具体 API 文档移步 L7Plot 官网。

authors: jefferyssy liuvigongzuoshi


text:
升级版本到2x之后，表格的双击事件经常不触发，没有1x的双击效果灵敏，能不能在这里做一个优化？🤔

🏷 Version

Package
Version

@antv/s2

@antv/s2-react

@antv/s2-vue

Sheet Type

[√ ] PivotSheet
[√ ] TableSheet
[ ] GridAnalysisSheet
[ ] StrategySheet
[ ] EditableSheet

🖋 Description

升级版本到2x之后，表格的双击事件经常不触发，没有1x的双击效果灵敏，能不能在这里做一个优化？
🔗 Reproduce Link

😊 Expected Behavior
😅 Current Behavior

表格的双击事件经常不触发，没有1x的双击效果灵敏

如何定义 [灵敏] ?
目前 2.x 版本使用 G 底层 出于跨平台考虑没有使用 dbclick, 而是使用的 pointerdown 模拟的双击实现 (200 ms 内的两次连续点击认为是双击), 原生的 dbclick 大约是 500 ms 左右: https://g.antv.antgroup.com/api/event/faq#鼠标双击事件
如果你觉得不灵敏, 可以自行监听 click 事件判断前后时间间隔.

可以自己设置两次点击的间隔时间吗？
------------------ 原始邮件 ------------------
发件人:                                                                                "antvis/S2"                                                                                @.>;
发送时间: 2024年6月17日(星期一) 下午5:48
@.>;
@.@.>;
主题: Re: [antvis/S2] 升级版本到2x之后，表格的双击事件经常不触发，没有1x的双击效果灵敏，能不能在这里做一个优化？🤔 (Issue #2775)
表格的双击事件经常不触发，没有1x的双击效果灵敏
如何定义 [灵敏] ?
目前 2.x 版本使用 G 底层 出于跨平台考虑没有使用 dbclick, 而是使用的 pointerdown 模拟的双击实现 (200 ms 内的两次连续点击认为是双击), 原生的 dbclick 大约是 500 ms 左右: https://g.antv.antgroup.com/api/event/faq#鼠标双击事件
如果你觉得不灵敏, 可以自行监听 click 事件判断前后时间间隔.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>

通常来说, S2 的文档没写, 那就是不可以.
目前我看渲染引擎那边是写死的 200, 没开放参数, 近期给 G 那边反馈下, 应该是可以支持的, 目前你暂时自己判断下吧.

https://github.com/antvis/G/blob/277abff24936ef6f7c43407a16c5bc9260992511/packages/g-lite/src/services/EventService.ts#L316-L323

https://github.com/antvis/G/issues/1710

通常来说, S2 的文档没写, 那就是不可以.
目前我看渲染引擎那边是写死的 200, 没开放参数, 近期给 G 那边反馈下, 应该是可以支持的, 目前你暂时自己判断下吧.

https://github.com/antvis/G/blob/277abff24936ef6f7c43407a16c5bc9260992511/packages/g-lite/src/services/EventService.ts#L316-L323
还有种方式你可以直接监听 <canvas /> dom 的原生 dbclick 事件
s2.getCanvasElement().addEventListener('dblclick',(event) => {
  const cell = s2.getCell(event.target)
})

如果是s2-react呢，如何获取到s2示例啊？


text:
zoom-canvas监听的鼠标滚轮事件放大缩小与用户习惯不一致

滚轮事件放大，缩小，一般情况下，鼠标滚轮向上是缩小，向下是放大，插件实现的方式正好相反

可通过修改缩放比例计算方式修复(src/behavior/zoom-canvas.js)

用负数就行...

感谢您的回复，问题已解决，

authors: SongOfLightAndShadow lanzhenxiang


text:
Add due date handling

Add a calendar datepicker for setting due dates.
Conditionally highlight tasks that are near/overdue.

completed

repo: anusontarangkul/taskmaster-pro


text:
Requesting Support for Token Chaining JWT's

We have a use case were we want to be able to pass the ID Token down a chain of services and each service that the token is passed to they would verify the authenticty with the server and then call the userinfo endpoint to get the user's data.  For example:

Client Browser --> Web Application --> Web Service --> Data Service

Each of these steps should be able to call Anvil, verify the token is still good, and then call the userinfo service to get info about the user.
So far there are three standards proposed to make this work with none of them really taking the lead.  They are:

https://tools.ietf.org/html/draft-richer-oauth-chain-00
https://tools.ietf.org/html/draft-hunt-oauth-chain-01
http://tools.ietf.org/html/draft-campbell-oauth-sts-02
https://www.ietf.org/archive/id/draft-jones-oauth-token-exchange-01.txt

We don't have an opinion yet, but thought this would be a good place to get feedback from the rest of the community if we're the only crazies that need this or if other customers have similar needs.

Some earlier threads on the IETF discussions around this:
https://www.ietf.org/mail-archive/web/oauth/current/msg14433.html

Also some more recent discussions https://www.ietf.org/mail-archive/web/oauth/current/msg14750.html

repo: anvilresearch/connect


text:
2x faster code generation

Fixes #948. Simply replacing the pascal function because it is a bottleneck.
I added some test cases to make sure we don't break anything, do not hesitate to tell me if I missed something.
Don't take my word for it, here's a demo:
https://www.loom.com/share/0f822963b55542d8998077ce75252068
https://github.com/anymaniax/orval/assets/46993939/c4472624-77b8-485d-932a-8b85e79a9c28

You're welcome !
Do you know when the next version will be available? I'd like to take advantage of these changes as soon as possible 😉

@EwenQuim I will try to do that tomorrow

authors: EwenQuim anymaniax

repo: anymaniax/orval


text:
Conda package

I think it would be good to have a package available in one of the popular conda channels.
A lot of people use Anaconda to handle python enviroments and sometimes pip installing is not an option. I am aware that it is possible to compile it from source but I think it wuld be good to have something on conda forge, for example.

There is a package in conda-forge (I am one of the maintainers): https://github.com/conda-forge/pymoo-feedstock. It's synced with the latest pypi release (v0.6.1.1). Are you not able to install it?

Hello! I see, maybe I have a problem with my local setup and that's why it is not visible from my conda installation.

It is compiled for Python versions 3.8 through 3.12 on win64, osx-arm64, osx-64 and linux-64. If you find a specific issue, you can open an issue on the pymoo-feedstock repo. I'd be happy to help.

@cyrilpic Thanks for taking care of it!

authors: Fuminides blankjul cyrilpic

repo: anyoptimization/pymoo


text:
Broken Links to Documentation on docs.rs for Multiple Solana Crates

Description
I have encountered multiple broken links on docs.rs for various Solana crates. These broken links lead to not a library page , making it difficult to understand certain parts if the codebase .
List of Broken Links
Here are the broken documentation links i could find :

Solana Accounts DB

URL: https://docs.rs/solana-acounts-db

Solana Cargo Registry

URL: https://docs.rs/solana-cargo-registry

Solana Stake Accounts

URL: https://docs.rs/solana-stake-accounts

Solana RPC Test

URL: https://docs.rs/solana-rpc-test

Solana Nonce Client

URL: https://docs.rs/solana-nonce-client

Solana PoH Bench

URL: https://docs.rs/solana-poh-bench

Solana Ledger Tool

URL: https://docs.rs/solana-ledger-tool

Solana Client Test

URL: https://docs.rs/solana-client-test

Expected Behavior
All links should direct to the appropriate and current documentation pages on docs.rs.
Suggested Action
Please verify whether the crates are still actively maintained and whether their documentation URLs need updating or if there are typos in the crate names in the URLs. If the documentation has been moved or if the crates have been deprecated, please update the links or the documentation references to reflect the current status.

do you mean this one?

If yes, I don't think they are broken links. they are truly bin crates. btw, the crates that you listed, only the first one is a lib (and you have typo. it is https://docs.rs/crate/solana-accounts-db). All others are either bin crates or haven't been published for a while.
could you please link me to some appropriate documentation that you have seen for bin crates or crates that haven't been published for a long time?

authors: Ifechukwudaniel yihau

repo: anza-xyz/agave


text:
sdk: Extract borsh helpers into solana-borsh

Problem
The borsh helpers in solana-program are used by program implementations, which makes it impossible to re-write certain programs without depending on solana-program.
Summary of changes
Pretty straightforward -- just move all of the borsh helpers into a new solana-borsh crate.

✅

authors: joncinque yihau


text:
Tester for outdoor camera and doorbell

I can be a tester for outdoor camera and doorbell uses, but my plugin seems to crash on trying to read the properties of 'home'.
my home is not named 'home' in the netatmo security app, is this hardcoded?
Is there not a place I need to enter a netatmo's access token?
Logs:
[10/22/2022, 9:17:10 PM] [Netatmo Security Platform] Child bridge process ended
[10/22/2022, 9:17:10 PM] [Netatmo Security Platform] Process Ended. Code: 1, Signal: null
[10/22/2022, 9:17:17 PM] [Netatmo Security Platform] Restarting Process...
[10/22/2022, 9:17:19 PM] [Netatmo Security Platform] Launched child bridge with PID 10605
[10/22/2022, 9:17:19 PM] Registering platform 'homebridge-netatmo-security.netatmo-security'
[10/22/2022, 9:17:19 PM] [Netatmo Security Platform] Loaded homebridge-netatmo-security v1.2.1 child bridge successfully
[10/22/2022, 9:17:19 PM] [Netatmo Security Platform] Netatmo API constructed.
[10/22/2022, 9:17:20 PM] Homebridge v1.5.0 (HAP v0.10.2) (Netatmo Security Platform) is running on port *****.
TypeError: Cannot read properties of undefined (reading 'home')
    at NetatmoAPI.getHomeStatus (/homebridge/node_modules/homebridge-netatmo-security/src/util/NetatmoAPI.js:102:25)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at NetatmoAPI.getHomeDevices (/homebridge/node_modules/homebridge-netatmo-security/src/util/NetatmoAPI.js:110:20)
[10/22/2022, 9:17:22 PM] [Netatmo Security Platform] Child bridge process ended```

**Plugin Config:**

```json
        {
            "name": "Netatmo Security Platform",
            "client_id": "***",
            "client_secret": "***",
            "hide_client_secret": true,
            "username": "***",
            "password": "***",
            "hide_password": true,
            "platform": "netatmo-security",
            "_bridge": {
                "username": "***",
                "port": ***
            }
        }```

**Environment:**

* **Plugin Version**: homebridge-netatmo-security v1.2.1
* **Homebridge Version**:  1.5.0
* **Node.js Version**: v16.13.2
* **NPM Version**: 8.1.2
* **Operating System**: Docker Alpine Linux (3.12.9)

This plugin does not currently support Outdoor Camera. I do not have the accessory myself, so I cannoy test alone. Are you making a plugin? I can take a look, if you send me the source code?

Hello Anzure and thank you for this plugin.
I think I’m not to far to succes :), the authentication is ok but I have the same error as jmjjlai about "reading ‘home’".
It’s maybe because I have 2 homes in netatmo? But I’m unable to delete on of them.
could you help me with this please ? Maybe add the possibility to choose the good home ? I’m sorry I’m really a beginner here…
Thank you.
here are my errors:
TypeError: Cannot read properties of undefined (reading 'home')
at NetatmoAPI. getHomeStatus (/homebridge/node modules/homebridge-netatmo-security/src/util/NetatmoAPI. js
:102:25 )
at processticksAndRejections (node: internal/process/task queues :95:5)
at NetatmoAPI. getHomeDevices (/homebridge/node modules /homebridge-netatmo-security/src/util/NetatmoAPI. j
s:110:20 )

authors: anzure appolonien jmjlai

repo: anzure/homebridge-netatmo-security


text:
Consider adding static method friendly alias' for common type constructions

In particular
  ReactiveSeq.fromIterable(myCollection);

Could become
   asReactiveSeq(myCollection);

and
   ListX.fromIterable(myCollection);

would be
    asListX(myCollection);

Keep with one way to create these types.

repo: aol/cyclops-react


text:
add preprocessing function for decoding events

should be able to generate a list of events something like:
TIME_ZERO=238
EXP_START = 1
CENTER_TARGET_ON = 16
CURSOR_ENTER_CENTER_TARGET = 80
CENTER_TARGET_OFF = 32
REWARD = 48
DELAY_PENALTY = 66
TIMEOUT_PENALTY = 65
HOLD_PENALTY = 64
TRIAL_END = 239

using the sync_event_dict metadata from bmi3d

closed by #268

repo: aolabNeuro/analyze


text:
[CMakelists] Remove hypervisor build step as obsolete

After moving to the zephyr-v3.3.0-xt branch on zephyr qemu build was changed so it starts baremetal setup by default due to the SHIELD limitations.
From now qemu should be run externally using prebuilt xen and one of the dts files, provided in the scripts folder.

After some discussions I've got to the conclusion that this PR is valid. That's because zephyr xen_dom0 shield includes instructions about how xen should be built and started on qemu. Also, I don't see any point of leavig xen as external project here because zephyr app is meant to be built using mouling, which is responsible for building such external dependencies. @al1img


text:
Metadata Table recovery failure

Describe the bug
Ran cingest on a 2.1.0-SNAPSHOT cluster with per-table crypto configured. Specifically, the following properties are set in accumulo.properties:
instance.crypto.opts.factory=org.apache.accumulo.core.spi.crypto.PerTableCryptoServiceFactory
general.custom.crypto.recovery.service=org.apache.accumulo.core.spi.crypto.AESCryptoService
general.custom.crypto.wal.service=org.apache.accumulo.core.spi.crypto.AESCryptoService
general.custom.crypto.key.uri=file:///opt/accumulo-testing/accumulo/accumulo-2.1.0-SNAPSHOT/conf/per-table-enc-key

and the ci table had the property table.crypto.opts.service=org.apache.accumulo.core.spi.crypto.AESCryptoService. I loaded the table with a bunch of data, then I deleted it via the shell. The deletetable command never returned and it appears that one of the metadata tablets was re-assigned during the delete operation. When trying to load the metadata tablet, the tserver logged the following error:
2022-09-28T16:50:44,468 [log.RecoveryLogsIterator] DEBUG: Opening recovery log dir 11946b4a-99b6-4b7e-b0d4-ccc1f981783a
2022-09-28T16:50:44,469 [crypto.CryptoFactoryLoader] DEBUG: Creating new crypto factory class org.apache.accumulo.core.spi.crypto.GenericCryptoServiceFactory
2022-09-28T16:50:44,472 [tserver.AssignmentHandler] WARN : exception trying to assign tablet !0<;~ null
java.lang.RuntimeException: Error recovering tablet !0<;~ from log files
        at org.apache.accumulo.tserver.tablet.Tablet.<init>(Tablet.java:379) ~[accumulo-tserver-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.accumulo.tserver.AssignmentHandler.run(AssignmentHandler.java:165) ~[accumulo-tserver-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.accumulo.tserver.ActiveAssignmentRunnable.run(ActiveAssignmentRunnable.java:63) ~[accumulo-tserver-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.accumulo.core.trace.TraceWrappedRunnable.run(TraceWrappedRunnable.java:52) ~[accumulo-core-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
        at org.apache.accumulo.core.trace.TraceWrappedRunnable.run(TraceWrappedRunnable.java:52) ~[accumulo-core-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at java.lang.Thread.run(Thread.java:829) ~[?:?]
Caused by: java.io.IOException: java.io.UncheckedIOException: java.io.IOException: incorrect header check
        at org.apache.accumulo.tserver.log.TabletServerLogger.recover(TabletServerLogger.java:544) ~[accumulo-tserver-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.accumulo.tserver.TabletServer.recover(TabletServer.java:1204) ~[accumulo-tserver-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.accumulo.tserver.tablet.Tablet.<init>(Tablet.java:334) ~[accumulo-tserver-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        ... 7 more
Caused by: java.io.UncheckedIOException: java.io.IOException: incorrect header check
        at org.apache.accumulo.core.file.blockfile.impl.CachableBlockFile$Reader$BCFileLoader.load(CachableBlockFile.java:223) ~[accumulo-core-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.accumulo.core.client.rfile.RFileScanner$NoopCache.getBlock(RFileScanner.java:165) ~[accumulo-core-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.accumulo.core.client.rfile.RFileScanner$NoopCache.getBlock(RFileScanner.java:158) ~[accumulo-core-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.accumulo.core.file.blockfile.impl.CachableBlockFile$Reader.getMetaBlock(CachableBlockFile.java:376) ~[accumulo-core-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.accumulo.core.file.rfile.RFile$Reader.<init>(RFile.java:1164) ~[accumulo-core-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.accumulo.core.file.rfile.RFile$Reader.<init>(RFile.java:1256) ~[accumulo-core-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.accumulo.core.client.rfile.RFileScanner.iterator(RFileScanner.java:354) ~[accumulo-core-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.accumulo.tserver.log.RecoveryLogsIterator.validateFirstKey(RecoveryLogsIterator.java:156) ~[accumulo-tserver-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.accumulo.tserver.log.RecoveryLogsIterator.<init>(RecoveryLogsIterator.java:77) ~[accumulo-tserver-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.accumulo.tserver.log.SortedLogRecovery.findMaxTabletId(SortedLogRecovery.java:107) ~[accumulo-tserver-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.accumulo.tserver.log.SortedLogRecovery.findLogsThatDefineTablet(SortedLogRecovery.java:147) ~[accumulo-tserver-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.accumulo.tserver.log.SortedLogRecovery.recover(SortedLogRecovery.java:291) ~[accumulo-tserver-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.accumulo.tserver.log.TabletServerLogger.recover(TabletServerLogger.java:542) ~[accumulo-tserver-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.accumulo.tserver.TabletServer.recover(TabletServer.java:1204) ~[accumulo-tserver-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.accumulo.tserver.tablet.Tablet.<init>(Tablet.java:334) ~[accumulo-tserver-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        ... 7 more
Caused by: java.io.IOException: incorrect header check
        at org.apache.hadoop.io.compress.zlib.ZlibDecompressor.inflateBytesDirect(Native Method) ~[hadoop-client-api-3.3.4.jar:?]
        at org.apache.hadoop.io.compress.zlib.ZlibDecompressor.decompress(ZlibDecompressor.java:225) ~[hadoop-client-api-3.3.4.jar:?]
...

One tablet for the metadata table is in an offline state and the ci table is in a deleting state.

I'm wondering if the issue is here. The RecoveryLogsIterator creates an RFileScanner with the table properties of the metadata table. RFileScanner gets a CryptoService using the Table scope, not the Recovery scope.

What is the internal File reader classes ?

What is the internal File reader classes ?

https://github.com/apache/accumulo/blob/42a6a9f4c54a60e73ec5b20841f883353c95d2e8/core/src/main/java/org/apache/accumulo/core/clientImpl/bulk/BulkImport.java#L352-L355

authors: dlmarion milleruntime

repo: apache/accumulo


text:
ARTEMIS-1813 DB2 BLOB size should be set to match its SQL definition

max-blob-size.db2 and create-file-table.db2 are changed to match the
2 GB max blob size limit allowed by DB2.

Please wait to merge it that I'm performing several tests on DB2 to see if is a sane value/default :+1:

I'm closing it until the tests on it will pass :+1:

@franz1981  to reopen this PR successfully on github.
Reopen it with the same git you have now... then push -f...
If you reopen it with a different git-ID it will fail and you would need to open a new one.

repo: apache/activemq-artemis


text:
drop_graph('graph_name') fails to Delete Graph

Describe the bug
SELECT drop_graph('graph_name') is not deleting recently created graph.
How are you accessing AGE (Command line, driver, etc.)?

psql interface, command line.

What is the necessary configuration info needed?

Fresh install of PostgreSQL 15 and Apache AGE master, on Debian stable-slim Docker image.

What is the command that caused the error?
psql (15.4)
Type "help" for help.

test=# CREATE EXTENSION age;
CREATE EXTENSION
test=# LOAD 'age';
SET search_path TO ag_catalog;
LOAD
SET
test=# SELECT * FROM ag_catalog.ag_graph;
 graphid | name | namespace
---------+------+-----------
(0 rows)

test=# SELECT create_graph('agload');
NOTICE:  graph "agload" has been created
 create_graph
--------------

(1 row)

test=# SELECT drop_graph('agload');
2023-08-23 10:33:23.574 -03 [28787] ERROR:  cannot drop schema agload because other objects depend on it
2023-08-23 10:33:23.574 -03 [28787] DETAIL:  table agload._ag_label_vertex depends on schema agload
        table agload._ag_label_edge depends on schema agload
2023-08-23 10:33:23.574 -03 [28787] HINT:  Use DROP ... CASCADE to drop the dependent objects too.
2023-08-23 10:33:23.574 -03 [28787] STATEMENT:  SELECT drop_graph('agload');
ERROR:  cannot drop schema agload because other objects depend on it
DETAIL:  table agload._ag_label_vertex depends on schema agload
table agload._ag_label_edge depends on schema agload
HINT:  Use DROP ... CASCADE to drop the dependent objects too.
test=#

Expected behavior
After SELECT drop_graph('graph_name'); the graph should be dropped along with all its vertexes and edges.
Environment (please complete the following information):

AGE Version: 1.3.0

I actually forgot one argument, so I should be calling SELECT drop_graph('graph_name', true);, which works.
So now I raise a question: in what cases would setting the cascade option to false would be useful?
Calling the function with false, on any graph I tested, raised the same error.
Should we change the function so it always considers the cascade argument as true?

When there are multiple graphs with foreign key relationships between them, deleting the "main" graph with the cascade option set to True could cause errors or data integrity issues in the dependent graphs.
but as you mentioned it would be better if we set it to "true" and pop up a message in case we found a relation or dependency

cascade is A boolean that will will delete labels and data that depend on the graph. So, it is recommended to set the cascade option to true, otherwise everything in the graph must be manually dropped with SQL DDL commands.
See references here

authors: Amr-Shams Munmud markgomer

repo: apache/age


text:
Using a label assigned to another graph will crash the database

Describe the bug
A clear and concise description of what the bug is.
How are you accessing AGE (Command line, driver, etc.)?

[e.g. JDBC]
AGE 1.6.0 on PG15

What is the command that caused the error?
I will explain it via example:

create_graph('graph1');
create_graph('graph2);

create_vlabel('graph1', 'label_exclusive_to_graph1');

SELECT * FROM cypher('graph2', $$CREATE (a:label_exclusive_to_graph1 {name:'a'}),
                            $$) AS (a agtype);

When you drop the schemas, this will crash your database.
DROP SCHEMA graph2 CASCADE;
psql:age_install.sql:2: ERROR:  label (relation) cache corrupted
psql:age_install.sql:2: FATAL:  terminating connection because protocol synchronization was lost
psql:age_install.sql:2: server closed the connection unexpectedly
    This probably means the server terminated abnormally
    before or while processing the request.
psql:age_install.sql:2: error: connection to server was lost

Expected behavior
No database crash. Label implementation needs to be reconsidered/fixed

So far, I tried setting up the database through the commands mentioned in the issue using both JDBC and Postgres.
Other than that, I tried
Manually creating a vertex with a label for 1 graph, creating vertex using the same label for other graph, then DROP SCHEMA graph2 CASCADE;
That also does not crash the database but only gives an error.
test=# SELECT * FROM ag_catalog.create_graph('graph1');
NOTICE:  graph "graph1" has been created
 create_graph
--------------

test=# SELECT * FROM ag_catalog.create_graph('graph2');
NOTICE:  graph "graph2" has been created
 create_graph
--------------

(1 row)

test=# SELECT * FROM cypher('graph1', $$ CREATE (x:Person {name:'X'}) RETURN x $$) AS (x agtype);
                                        x

--------------------------------------------------------------------------------
-
 {"id": 844424930131969, "label": "Person", "properties": {"name": "X"}}::vertex
(1 row)

test=# SELECT * FROM cypher('graph2', $$ CREATE (x:Person {name:'Y'}) RETURN x $$) AS (x agtype);
                                        x

--------------------------------------------------------------------------------
-
 {"id": 844424930131969, "label": "Person", "properties": {"name": "Y"}}::vertex
(1 row)

test=# DROP SCHEMA graph2 CASCADE;
NOTICE:  drop cascades to 4 other objects
DETAIL:  drop cascades to sequence graph2._label_id_seq
drop cascades to table graph2._ag_label_vertex
drop cascades to table graph2._ag_label_edge
drop cascades to table graph2."Person"
2023-10-15 16:51:24.432 EEST [160811] ERROR:  table "Person" is for label "Person"
2023-10-15 16:51:24.432 EEST [160811] STATEMENT:  DROP SCHEMA graph2 CASCADE;
ERROR:  table "Person" is for label "Person"

I also tried switching the order of commands such as:
Creating graph 1 first, then vertex, then graph 2, then vertex.
Creating graph 1 first, then graph 2, then vertex 1, then vertex 2.
Both having mixed up labels (graph2 first labels in graph1, and graph1 labels in graph2).
The database did not crash
@dehowef Can you provide more information about the bug?
I'm using the master branch age with pg15.

@CapnSpek Could you try with JDBC? With PSQL, server does not crash.

Like @rafsun42 stated, no crash with psql -
psql-15.4-5432-pgsql=# drop extension age cascade; create extension age; load 'age'; set search_path TO ag_catalog;
ERROR:  extension "age" does not exist
CREATE EXTENSION
LOAD
SET
psql-15.4-5432-pgsql=#
psql-15.4-5432-pgsql=# select * from create_graph('graph1');
NOTICE:  graph "graph1" has been created
 create_graph
--------------

(1 row)

psql-15.4-5432-pgsql=# select * from create_graph('graph2');
NOTICE:  graph "graph2" has been created
 create_graph
--------------

(1 row)

psql-15.4-5432-pgsql=#
psql-15.4-5432-pgsql=# select * from create_vlabel('graph1', 'label_exclusive_to_graph1');
NOTICE:  VLabel "label_exclusive_to_graph1" has been created
 create_vlabel
---------------

(1 row)

psql-15.4-5432-pgsql=# select * from create_vlabel('graph2', 'label_exclusive_to_graph2');
NOTICE:  VLabel "label_exclusive_to_graph2" has been created
 create_vlabel
---------------

(1 row)

psql-15.4-5432-pgsql=#
psql-15.4-5432-pgsql=# select * from cypher('graph2', $$CREATE (a:label_exclusive_to_graph1 {name:'a'}) $$) as (a agtype);
 a
---
(0 rows)

psql-15.4-5432-pgsql=# select * from cypher('graph1', $$CREATE (a:label_exclusive_to_graph2 {name:'a'}) $$) as (a agtype);
 a
---
(0 rows)

psql-15.4-5432-pgsql=# drop schema graph2 cascade;
NOTICE:  drop cascades to 5 other objects
DETAIL:  drop cascades to sequence graph2._label_id_seq
drop cascades to table graph2._ag_label_vertex
drop cascades to table graph2._ag_label_edge
drop cascades to table graph2.label_exclusive_to_graph2
drop cascades to table graph2.label_exclusive_to_graph1
ERROR:  table "label_exclusive_to_graph1" is for label "label_exclusive_to_graph1"
psql-15.4-5432-pgsql=# drop schema graph1 cascade;
NOTICE:  drop cascades to 5 other objects
DETAIL:  drop cascades to sequence graph1._label_id_seq
drop cascades to table graph1._ag_label_vertex
drop cascades to table graph1._ag_label_edge
drop cascades to table graph1.label_exclusive_to_graph1
drop cascades to table graph1.label_exclusive_to_graph2
ERROR:  table "label_exclusive_to_graph2" is for label "label_exclusive_to_graph2"
psql-15.4-5432-pgsql=#

Resolved: User Error on my part.
Note, graphs must be dropped via drop_graph(text graph_name, bool cascade) before dropping the AGE Extension and rebuilding to prevent a database crash. Since the CREATE Extension command is a PG command, this care must be taken by AGE users.

authors: CapnSpek dehowef jrgemignani rafsun42


text:
Fix bug #339 - entities in WHERE clause have wrong Expr

Whenever a node or an edge within a WHERE clause uses a Var type expr retrieved from a different cypher parsestate, the Var->varlevelsup is adjusted.

Please correct these extra spaces at the end of lines -

@rafsun42 Other than the minor cosmetic issue above, the patch builds without error and looks good.

@jrgemignani Sorry! I missed the extra spaces again. I updated the PR now.

@jrgemignani Hi John, could you also check if whitespace issue still persist for this PR as well whenever you have time?

@rafsun42 Reviewing it.


text:
IVY-1651 Augment 'Child elements’ section of 'File System Resolver' documentation

PR to address https://issues.apache.org/jira/browse/IVY-1651

Hey ^^ Just wanted to ask when we can expect this change to be merged as it seems it's the only one left for the next version 2.5.3 and we are eagerly waiting for the fix in https://issues.apache.org/jira/browse/IVY-1647

I think we should link to https://ant.apache.org/ivy/history/2.5.2/concept.html where the possible tokens inside a pattern are discussed. In addition, the https://ant.apache.org/ivy/history/2.5.2/concept.html should be extended with additional tokens for maven dependencies.

Fixed by #103 , thanks for your contribution!

authors: Seros lewismc maartenc

repo: apache/ant-ivy


text:
fix: golang:1.19 base image vulnerability

Signed-off-by: fengshunli 1171313930@qq.com

Type of change:

[ ] Bugfix
[ ] New feature provided
[ ] Improve performance
[ ] Backport patches

What this PR does / why we need it:

Pre-submission checklist:

[ ] Did you explain what problem does this PR solve? Or what new features have been added?
[ ] Have you added corresponding test cases?
[ ] Have you modified the corresponding document?
[ ] Is this PR backward compatible? If it is not backward compatible, please discuss on the mailing list first

golang:1.19 image has many high-risk vulnerabilities and security risks. It is recommended to switch to golang:1.19-alpine3.17

It is only used during the build phase, not the final image.

there are also many high-risk vulnerabilities in centos, for details, refer to https://github.com/apache/apisix-ingress-controller/issues/1649

repo: apache/apisix-ingress-controller


text:
feat: add externalname support for upstream

Please answer these questions before submitting a pull request

Why submit this pull request?

[ ] Bugfix

[x] New feature provided

[ ] Improve performance

[ ] Backport patches

Related issues
https://github.com/apache/apisix-ingress-controller/issues/645

Bugfix

Description

How to fix?

New feature or improvement

Describe the details and related test reports.

Adding kubernetes service type ExternalName support
For kind: ApisixUpstream, add two more options passHost and upstreamHost

Let me know if there's anything missing here, thanks.

Backport patches

Why need to backport?

Source branch

Related commits and pull requests

Target branch

@elvis-cai Please add some test cases to cover these changes.

@elvis-cai Please add some test cases to cover these changes.

sure, doc & test added.

Codecov Report

Merging #726 (ab32fd3) into master (dc196ef) will decrease coverage by 0.44%.
The diff coverage is 20.79%.

:exclamation: Current head ab32fd3 differs from pull request most recent head 8670494. Consider uploading reports for the commit 8670494 to get more accurate results

@@            Coverage Diff             @@
##           master     #726      +/-   ##
==========================================
- Coverage   32.30%   31.86%   -0.45%
==========================================
  Files          65       65
  Lines        6500     6675     +175
==========================================
+ Hits         2100     2127      +27
- Misses       4149     4289     +140
- Partials      251      259       +8

Impacted Files
Coverage Δ

pkg/ingress/compare.go
0.00% <0.00%> (ø)

pkg/ingress/namespace.go
0.00% <0.00%> (ø)

pkg/ingress/controller.go
1.01% <5.00%> (-0.05%)
:arrow_down:

pkg/config/config.go
62.50% <13.79%> (-23.95%)
:arrow_down:

cmd/ingress/ingress.go
75.82% <50.00%> (-1.45%)
:arrow_down:

pkg/kube/translation/translator.go
48.18% <55.55%> (+1.18%)
:arrow_up:

pkg/kube/translation/util.go
32.16% <58.82%> (-0.56%)
:arrow_down:

pkg/kube/translation/apisix_upstream.go
88.47% <88.23%> (-0.03%)
:arrow_down:

test/e2e/e2e.go

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dc196ef...8670494. Read the comment docs.

Please fix the CI errors.

my bad, last commit forget to update test code, updated

Sorry for delay. Can you resolve the conflict? Then we run the test and merge this PR. Thanks!

Sorry for delay. Can you resolve the conflict? Then we run the test and merge this PR. Thanks!

all good, conflict resolved and feel free to rerun the test, thanks.

@tokers @tao12345666333     Why this feature not merge to master, Can you go ahead to process this feature ?

There are too many conflicts to resolve.

@elvis-cai Could you merge the newest master branch? Thanks!

This looks like a great feature and I'm really looking forward to having it merged. Is there something I can do to help move it along?

I am more looking forward to the feature https://github.com/apache/apisix-ingress-controller/issues/927, but sounds good to me , before that's implemented, will look at if I could fix the conflict and merge the latest master branch.

@tao12345666333 how does this look? I'd love to see this PR merged

Just adding my +1 for this feature :)

Sorry for delay reply.
The maintenance team is actively working on the remainder tasks of the v1.5 release window.
We plan to do a pre-release of v1.5 this month.
#726 #927 and some related tasks, I will put them in the v1.6 version to complete.
Currently we have too many things to do, including GitHub issues, PRs, some features and bugfixes, etc., and Slack channel replies. This leaves us almost at full capacity.
If anyone would like to see some new features added, please join us and help us make this project better, thanks

#1306 has been merged. I will close this one, thanks!

@tao12345666333 It looks like there's a discrepancy in the docs and what's actually possible when using the ApisixUpstream crd, i.e the passHost feature doesn't work- see https://github.com/apache/apisix-ingress-controller/issues/2010.
I'm not sure where the confusion happened but it looks like this pr would have added that feature, but #1306 was merged instead and does include passHostfunctionality?

authors: BrandonArp NMichas codecov-commenter elvis-cai mbuotidem purekeeper tao12345666333 tokers


text:
fix: improve link checker availability

Changes:
Fix the link checker artifact path error so that it can be uploaded to GitHub correctly.

Update: Add a timeout for Axios to prevent the detection process from executing indefinitely.
cc @liuxiran @juzhiyuan

repo: apache/apisix-website


text:
Snowflake create stage to include use_vectorized_scanner

What feature or improvement would you like to see?
https://github.com/apache/arrow-adbc/blob/main/go/adbc/driver/snowflake/bulk_ingestion.go#L48C2-L48C164
Change to
createTemporaryStageStmt = "CREATE OR REPLACE TEMPORARY STAGE " + bindStageName + " FILE_FORMAT = (TYPE = PARQUET USE_LOGICAL_TYPE = TRUE BINARY_AS_TEXT = FALSE USE_VECTORIZED_SCANNER=TRUE)"
See details about new option: https://medium.com/snowflake/faster-parquet-data-ingestion-with-snowflake-use-vectorized-scanner-28679bcff450
Temporary internal stages have SNOWFLAKE_FULL encryption by default which is not yet supported by USE_VECTORIZED_SCANNER. However it will be supported in the next month, at which point, the vectorized scanner performance improvement will kick in.
It's still possible to set this option now but it won't take effect as it will fall back to the old scanner. When the encryption support is added, it will work automatically.

@zeroshade

authors: ianmcook sfc-gh-xhuang

repo: apache/arrow-adbc


text:
optimize limit-full join in the limit push down rule

    I think we can also optimize `limit - FULL Join` pattern. @jackwener

Originally posted by @liukun4515 in https://github.com/apache/arrow-datafusion/issues/4257#issuecomment-1319729561
Now we have support the limit-left join and limit-right join, I think the limit-full join also can be optimized

We can do it after #4276

After careful consideration, this optimization has bug.

repo: apache/arrow-datafusion


text:
add aarch64 linux version datafusion-cli

Is your feature request related to a problem or challenge?
when I run pip install, it download the source code, but failed tow compile as my host missing some tools
Describe the solution you'd like
pip install the binary verion
Describe alternatives you've considered
No response
Additional context
pip install datafusion-cli
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Collecting datafusion-cli
  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/ba/02/2bb067f1c5bb4c16852767fe4b9cd7744d4b7cbc849d3e61aa9fe0398a86/datafusion_cli-36.0.0.tar.gz (1.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.7/1.7 MB 3.9 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [6 lines of output]

      Cargo, the Rust package manager, is not installed or is not on PATH.
      This package requires Rust and Cargo to compile extensions. Install it through
      the system's package manager or via https://rustup.rs/

      Checking for Rust toolchain....
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

Are datafusion-cli binaries distributed or is it always built from source? My understanding was it was built from source only.
I'd recommend just installing rust by following the directions in the error message.  Otherwise if you have docker installed on the aarch64 machine you can build the cli that way. Otherwise you can build the cli on another machine with docker using something like this.
Dockerfile:
FROM --platform=$TARGETPLATFORM rust:1.72-bullseye as builder

COPY . /usr/src/arrow-datafusion
COPY ./datafusion /usr/src/arrow-datafusion/datafusion

COPY ./datafusion-cli /usr/src/arrow-datafusion/datafusion-cli

WORKDIR /usr/src/arrow-datafusion/datafusion-cli

RUN rustup component add rustfmt

RUN cargo build --release

FROM --platform=$TARGETPLATFORM debian:bullseye-slim

COPY --from=builder /usr/src/arrow-datafusion/datafusion-cli/target/release/datafusion-cli /usr/local/bin

ENTRYPOINT ["datafusion-cli"]

CMD ["--data-path", "/data"]

docker buildx create --use
docker buildx build ./ -f ./datafusion-cli/Dockerfile --platform=linux/arm64

The above changes to the Dockerfile and the commands I ran seemed to build both binaries. This may not be ideal though - the build process is horribly slow (~5000s locally) and the image I think may run under emulation which for the CLI may just be fine.

thanks @Omega359


text:
Re-export modules individually to fix rustdocs

Which issue does this PR close?
Closes https://github.com/apache/arrow-datafusion/issues/6648
Rationale for this change
When I am trying to use rustdocs to find certain classes it often fails to do so (see https://github.com/apache/arrow-datafusion/issues/6648 for more details)
What changes are included in this PR?

Directly pub use the content of submodules rather than just the modules themselves

This means that rustdoc can now find all the classes directly:

Are these changes tested?
Existing tests

Are there any user-facing changes?
Better rust docs

cc @dadepo


text:
Parsing a string column containing JSON values into a typed array

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
I have a nullable StringArray column that contains JSON object literals.
I need to JSON parse the column into a StructArray of values following some schema, and NULL input values should become NULL output values.
This can almost be implemented using arrow_json::reader::ReaderBuilder::build_decoder and then feeding in the bytes of each string. But the decoder has no concept of record separators in the input stream. Thus, invalid inputs such as blank strings (""), or truncated records ("{\"a\":1"), or multiple objects ("{\"a\": 1} {\"a\": 2}") will confuse the decoding process. If we're lucky, it will produce the wrong number of records, but an adversarial input could easily seem to produce the correct number of records even tho no single input string represented a valid JSON object. Thus, if I want such safety, I'm forced to parse each string as its own RecordBatch (which can then be validated independently), and then concatenate them all. Ugly, error-prone, and inefficient:

pub fn parse_json(
    json_strings: StringArray,
    schema: SchemaRef,
) -> Result<RecordBatch, ArrowError> {
    // Use batch size of 1 to force one record per string input
    let mut decoder = ReaderBuilder::new(output_schema.clone())
        .with_batch_size(1)
        .build_decoder()?;

    // Feed a single string into the decoder and flush it to a record batch
    let mut parse_one = |json_string: Option<&str>| -> Result<RecordBatch, ArrowError> {
        // NOTE: null input becomes empty object (all fields null)
        let s = json_string.unwrap_or("{}");
        let mut reader = BufReader::new(s.as_bytes());
        let buf = reader.fill_buf()?;
        let read = buf.len();
        let decoded = decoder.decode(buf)?;
        assert_eq!(decoded, read);
        Ok(decoder.flush()?.unwrap())
    };
    let output: Vec<_> = json_strings
        .iter()
        .map(parse_one)
        .try_collect()?;
    concat_batches(&schema, output.iter())
}

(example code, has panics instead of full error handling)

Describe the solution you'd like
Ideally, the JSON Decoder could define public methods that say how many buffered rows the decoder has, and whether the decoder is currently at a record boundary or not. This is essentially a side effect-free version the same check that Tape::finish already performs when Decoder::flush is called:

impl TapeDecoder {
      ...
    /// The number of buffered rows this decoder has, including any in progress if [`has_partial_record()`].
    pub fn num_buffered_rows(&self) -> usize {
        self.cur_row
    }
    /// True if the decoder is part way through decoding a record. If so, calling [`finish`] would return an error.
    pub fn has_partial_row(&self) -> bool {
        !self.stack.is_empty()
    }

and
impl Decoder {
      ...
    /// The number records currently buffered in this decoder.
    /// A successful call to [`flush`] would produce this many rows.
    pub fn num_buffered_records(&self) -> usize {
        self.tape_decoder.num_buffered_rows()
    }
    /// True if the decoder is part way through decoding a record.
    /// Calling [`flush`] on a partial record would return an error.
    pub fn has_partial_record(&self) -> bool {
        self.tape_decoder.has_partial_record()
    }

That way, the above implementation becomes a bit simpler and a lot more efficient:

pub fn parse_json(
    json_strings: StringArray,
    schema: SchemaRef,
) -> Result<RecordBatch, ArrowError> {
    let mut decoder = ReaderBuilder::new(output_schema.clone())
        .with_batch_size(json_strings.len())
        .build_decoder()?;

    // Feed a single string into the decoder and flush it to a record batch
    let mut parse_one = |json_string| -> Result<(), ArrowError> {
        let mut reader = BufReader::new(json_string.as_bytes());
        let buf = reader.fill_buf()?;
        let read = buf.len();
        let decoded = decoder.decode(buf)?;
        assert_eq!(decoded, read);
        assert!(!decoder.has_partial_record());
    };

    // Make sure each string produces exactly one record.
    let mut num_records = 0;
    for json_string in json_strings {
        // NOTE: null input becomes empty object (all fields null)
        parse_one(json_string.unwrap_or("{}"))?;
        assert_eq!(decoder.num_buffered_records(), num_records+1);
        num_records += 1;
    }
    assert_eq!(num_records, json_strings.len());
    decoder.flush().transpose().unwrap()
}

It would be even nicer if the parse_json method could just become part of either arrow-json or arrow-compute, if parsing strings to JSON is deemed a general operation that deserves its own API call.
Describe alternatives you've considered
Tried shoving each string manually into a Decoder to produce a single RecordBatch, but the above-mentioned safety issues made it very brittle (wrong row counts, incorrect values, etc). Currently using the ugly/slow solution mentioned earlier, that creates and validates one RecordBatch per row, before concatenating them all into a single RecordBatch.

Sorry this one managed to slip through, adding num_buffered_rows and has_partial_record seems perfectly reasonable to me

take

Hi,
@scovich  @tustvold  I am currently looking into this.
Do we need to duplicate the changes, which are implemented in https://github.com/delta-io/delta-kernel-rs/pull/373/files
in the arrow-rs, along with num_buffered_rows has_partial_record functions ?

Do we need to duplicate the changes, which are implemented in https://github.com/delta-io/delta-kernel-rs/pull/373/files
in the arrow-rs, along with num_buffered_rows has_partial_record functions ?

Good question. We're happy to tweak the delta-kernel-rs code to match a new arrow-rust API, as long the new API covers the use case. I tried to factor that out in the "details" sections of this issue description.
If you refer to the parse_json_impl method in that PR, it corresponds to my comment in this issue's description:

It would be even nicer if the parse_json method could just become part of either arrow-json or arrow-compute, if parsing strings to JSON is deemed a general operation that deserves its own API call.

Seems like the low-level support can go independently of the decision to expose a new public parse_json method in arrow-compute or arrow-json?

authors: jatin510 scovich tustvold

repo: apache/arrow-rs


text:
feat: support reading and writingStringView and BinaryView in parquet (part 2)

Which issue does this PR close?

Closes #5530
Rationale for this change

Use view_buffer instead of offset_buffer for view type parquet reader
What changes are included in this PR?

Use view_buffer instead of offset_buffer for view type parquet reader
Are there any user-facing changes?
Yes

The parquet build for wasm32 check always failed, but it works on my machine...

Hi @alamb  @tustvold , this PR is not really ready for review, however, I may need some advice, and then I'll continue.

Thanks @ariesdevil  -- I think @tustvold  is away for a few days. I will try and give this PR a look over the next day or two. Very exciting!
cc @XiangpengHao

This is really nice step forward -- thank you @ariesdevil

I plan to give this another review today

The integration test https://github.com/apache/arrow-rs/actions/runs/8553472637/job/23436722309?pr=5557 seems to have failed for reasons unrelated to this PR. I have restarted it
################# FAILURES #################
FAILED TEST: union Rust producing,  Rust consuming
<class 'RuntimeError'>: Command failed: /build/rust/debug/flight-test-integration-client --host localhost --port=34799 --path /tmp/arrow-integration-92iwbbih/generated_union.json
With output:
--------------
Error: tonic::transport::Error(Transport, hyper::Error(Connect, ConnectError("tcp connect error", Os { code: 99, kind: AddrNotAvailable, message: "Cannot assign requested address" })))

I am sorry @ariesdevil  -- I need more time and a fresh set of eyes to review the new implementation in this PR. I will find time over the next few days.

Hi @alamb ,the new implementation has indeed changed a lot. It should have been implemented in another PR, but the original performance is really poor.
Thanks for your patience and have a nice weekend.

Perhaps we can either rebase this PR against main or maybe start a new PR

From the talk here ( https://github.com/apache/arrow-rs/pull/5618#issuecomment-2045839896 ) Views type would keeps views schema. However, I think write by StringView/BinaryView doesn't means a read type is BinaryView/StringView. Should we just use Binary/String?

From the talk here ( #5618 (comment) ) Views type would keeps views schema. However, I think write by StringView/BinaryView doesn't means a read type is BinaryView/StringView. Should we just use Binary/String?
Otherwise, we need extra checking for handling written-by view and read by string. If StringView/BinaryView is keep in metadata, hope a parquet-testing file for this could be add to testing legacy parquet reader could reading this to string/binary without casting↳

It's a design choices that require @alamb and @tustvold  judgment.

I hope to review this PR later today and if not I will do it tomorrow. I need to make sure I have enough contiguous time

From the talk here ( https://github.com/apache/arrow-rs/pull/5618#issuecomment-2045839896 ) Views type would keeps views schema. However, I think write by StringView/BinaryView doesn't means a read type is BinaryView/StringView. Should we just use Binary/String?

Otherwise, we need extra checking for handling written-by view and read by string. If StringView/BinaryView is keep in metadata, hope a parquet-testing file for this could be add to testing legacy parquet reader could reading this to string/binary without casting

I agree with @mapleFU 's observation that the special handling StringView / BinaryView will be substantially more code.
The reason that it would be valuable is that i think it can save a copy
For example as I understand it, from the parquet  encodings doc
...
BYTE_ARRAY: length in 4 bytes little endian followed by the bytes contained in the array
...

So the  data looks like this (length prefix, followed by the bytes):
\3\0\0\0foo\26\0\0\0abcdefghijklmnoprstuvwxyz

To make a StringArray, those bytes must be copied to a new buffer so they are contiguous:
offets: [0, 3, 29]
data: fooabcdefghijklmnoprstuvwxyz

However, for a StringView array, the raw bytes can be used without copying
views: [(len: 3, data:"foo"), (len:26, offset8)]
\3\0\0\0foo\26\0\0\0abcdefghijklmnoprstuvwxyz

So we should write without views schema and use a view reader to read to view array?

So we should write without views schema and use a view reader to read to view array?

I am not sure -- I am trying to figure it out / what the current PR does. I am not as familar with this code as I would like

Validate utf8 data (I left comments). I think we need a test for that too -- I'll make a PR to make that easier to add.

I wrote new tests in https://github.com/apache/arrow-rs/pull/5639

I am on vacation this week but I hope to find some time later today or tomorrow to give this another look. Thanks @ariesdevil

No need to rush, enjoy your vacation, just so I can have time to rethink how to continue optimization :smiley:

I see some new updates -- I plan to look at this PR later this week (probably won't be until Wednesday)

Hi @ariesdevil, thanks for the great work! I'm looking at this PR these days and trying to push it forward. However, I do feel the code change is too large for a single PR that it is quite difficult to understand every detail of the implementation.
I suggest we break the PR into even smaller pieces and incrementally add the features/optimizations so it poses less mental burdens on the reviewers, what do you think?
I'll try to consolidate a few low-hanging fruit changes and try to get them merged first; let me know your thoughts!

Hi @XiangpengHao, thanks for your reply. I don't think this PR is very large, the previous reviewers are already very familiar with this PR, this PR has been submitted for almost three months, and the reviewer has had enough time to get familiar with it.

Hi @XiangpengHao, thanks for your reply. I don't think this PR is very large, the previous reviewers are already very familiar with this PR, this PR has been submitted for almost three months, and the reviewer has had enough time to get familiar with it.

Indeed -- I am sorry this PR is stalled @ariesdevil  -- it has been very hard for me to find enough contiguous time to get familar enough with this PR and really understand it (and ensure the test coverage is adequate, etc).
@ariesdevil do you think we should attempt to merge this entire PR at once? I do think it would help speed up review if we could break it into some smaller PRs

Hi @alamb, I understand how busy you must be maintaining multiple projects and writing an important paper simultaneously.
As we now have try_append_view implemented, we no need ViewBuffer any more, I'll rebase this PR after #5877 merged. After that, this PR should be smaller.

Hi @alamb, I understand how busy you must be maintaining multiple projects and writing an important paper simultaneously.

Thank you for understanding -- I feel bad whenever PRs get stuck.

As we now have try_append_view implemented, we no need ViewBuffer any more, I'll rebase this PR after https://github.com/apache/arrow-rs/pull/5877 merged. After that, this PR should be smaller.

That is awesome. THank you.  I will make every effort to help get this over the line and now that @XiangpengHao is helping I am optimistic we can get it in shortly

https://github.com/apache/arrow-rs/pull/5877 is merged now!

closed by #5877

authors: XiangpengHao alamb ariesdevil mapleFU


text:
Update MSRVs to be accurate

Which issue does this PR close?
Closes #6741
What changes are included in this PR?
This changes MSRVs of a lot of crates in this repo to accurately reflect their actual MSRV, as they've fallen out of date.
There's an MSRV check in CI, but it currently only checks a few specific crates instead of all of them. It also doesn't have any way to tell if the workspace's rust-version field has fallen out of date due to all other crates in the workspace using a specifically different MSRV. This fixes both of those issues by checking every Cargo.toml that exists in this repo and also making sure at least one of them uses rust-version = { workspace = true }.
The MSRVs that these were changed to were all found by running cargo msrv find --manifest-path $cargo_toml_path for each Cargo.toml in the repo. If it failed due to a dependency that we could downgrade, I downgraded it as low as our Cargo.toml allowed it to go to see if that would allow us to get the MSRV lower.
Rationale for this change
The repo is easier to use and manage if this task is automated
Also, at time of writing, the arrow-flight/gen crate has a broken MSRV. This is to ensure the CI checks that I added actually work. I'll change it back once CI has verified it's bad.
Are there any user-facing changes?
No; these crates already wouldn't compile with their current (incorrect) MSRVs so we aren't affecting users at all with these changes.

The MSRV check failed, and although the output was too long to see what exactly the error was, I'm going to update the purposefully-broken package's MSRV to see if things pass now.

I took the liberty of pushing some commits to this branch to get this PR unstuck and hopefully ready to merge

Updated object_store stated msrv
removed workspace check
merged from apache/main

I struggle to approve this as a number of these crates were being tested with MSRV and were passing (e.g. arrow-flight)... Am I missing something?

I struggle to approve this as a number of these crates were being tested with MSRV and were passing (e.g. arrow-flight)... Am I missing something?

Yes - everything was passing before, but it simply wasn't testing every crate in this repo and we also had multiple MSRVs set too high (which msrv testing wouldn't catch, since we don't try to verify that the MSRV is set as low as it can go)
For example, arrow-flight was passing before, and we didn't change anything about it - however, the arrow-flight/gen crate had the MSRV set too low. It was set to 1.70.0, when the lowest version it actually supported was 1.71.1. We didn't catch this because the arrow-flight crate was already set to 1.71.1, and that was the only thing we were testing.
In the same vein, the arrow-integration-testing and arrow-pyarrow-integration-testing weren't being checked by CI at all either, so their MSRVs had fallen out of sync. This probably won't impact dependents at all, as I doubt anyone is using these crates besides this repo alone, but it's just nice to have them more up-to-date with this change now.
Besides these sorts of things, the other main change is updating the workspace's MSRV to 1.70 (from 1.62) since it had just fallen out-of-date with what the minimum MSRV is within the workspace. Since the workspace MSRV is now 1.70, all the crates that previously had their MSRV explicitly set to "1.70.0" can now just use { workspace = true }.
This PR fixes the above issues by:

Using a find loop to test every Cargo.toml in the repo for CI, instead of just a few hand-picked specific ones
Making sure at least one crate in the repo uses rust-version = { workspace = true } so that the workspace's MSRV doesn't get out-of-date anymore.

I wonder if we should even have per-crate MSRVs or rather opt for a workspace-wide one. To me, the different ranges are close to impossible to keep track off.

I wonder if we should even have per-crate MSRVs or rather opt for a workspace-wide one. To me, the different ranges are close to impossible to keep track off.

I think it would make sense for a single MSRV for arrow and a single MSRV for object_store (likely the same)

authors: alamb crepererum itsjunetime tustvold


text:
ARROW-13163: [C++][Gandiva] Implement REPEAT function on Gandiva

Implement REPEAT function on Gandiva which concatenate a string "n" times.

REPEAT(str, int)

@augustoasilva can you review it please?

lgtm. Can you fix the checkstyle error

authors: jpedroantunes projjal

repo: apache/arrow


text:
GH-36280: [Python][Parquet] Export C++ WriteRecordBatch in Python API

Rationale for this change

Export C++ WriteRecordBatch in Python API to allow memory saving
What changes are included in this PR?

[x] Add write_record_batch in pyx
[ ] Export to ParquetWriter api

Are these changes tested?

[ ] Will do

Are there any user-facing changes?
user can using it

Closes: #36280

Export C++ WriteRecordBatch in Python API to allow memory saving

Can you explain which memory saving you're talking about? Can you show an example?

Can you explain which memory saving you're talking about? Can you show an example?

When user writes to parquet table row-by-row, we usally batch the rows, then converting the batch to arrow, and write arrow batch to parquet, and when file is large enough, we will close writer and write to oss. Usally when we write to parquet, parquet::arrow::WriteTable is finally called, it will finish RowGroup once. So our files end with hundred or even thousands of RowGroups, which causing Metadata grows huge.
This patch exports parquet::arrow::FileWriter::WriteBatch, which will buffer the input in one RowGroup. Comparing to direct buffer the records in memory or in arrow file, buffering to parquet might be more straightforward @pitrou

So our files end with hundred or even thousands of RowGroups, which causing Metadata grows huge.

Hundres of row groups for which file size?

This patch exports parquet::arrow::FileWriter::WriteBatch, which will buffer the input in one RowGroup. Comparing to direct buffer the records in memory or in arrow file, buffering to parquet might be more straightforward @pitrou

But does it increase performance? Decrease file size? By how much?

@pitrou When RowGroup number grows, there're two problems:

size of Metadata in a File grows large, specially when there are lots of columns
2.during reading, RowGroup(idx) will be called many times, which making reading from file slow

At lease I just want a write_record_batch_without_flush_rowgroup to make writing to parquet more lightweight

I've a case, for example, we batch every 2000 rows. And write a file with 200,000 rows, the meta would be like:
  "TotalRows": "205434",
  "NumberOfRowGroups": "103",
  "NumberOfRealColumns": "97",
  "NumberOfColumns": "97",

The file is 3.5Mib, after changing to one row-group, the file can be just 450kb. This also making reading from it slower

Should I just change it to write_record_batch, or adding a argument to control it to avoid breaking previous behavior?

I think we could also consider just "breaking" the current behaviour. It will still honor the max row_group_size, just no longer create one row group per batch that you are writing.
Or only do this if row_group_size keyword is None.
What happens if you do a write_table after a write_batch if the write_batch didn't yet finalize the current row group?

@jorisvandenbossche
  Status NewRowGroup(int64_t chunk_size) override {
    if (row_group_writer_ != nullptr) {
      PARQUET_CATCH_NOT_OK(row_group_writer_->Close());
    }
    PARQUET_CATCH_NOT_OK(row_group_writer_ = writer_->AppendRowGroup());
    return Status::OK();
  }

parquet::arrow::FileWriterImpl::WriteTable will call NewRowGroup for every chunk, which means it will first close previous RowGroup, then split input table to chunk by users row group size, and call AppendRowGroup to create non-buffered row-group writer for every chunk.
parquet::arrow::FileWriterImpl::WriteRecordBatch will close previous rowgroup if previous row-group is not a "buffered" row-group, and append RecordBatch to the buffered row-group.
So:

If only WriteTable is called, every call will create at lease one new row group.
If only WriteRecord is called, every call might not create row group.
If WriteRecordBatch is called after WriteTable, it will find that there is not in-flight row-group, so it will create a buffered row-group and write to it.
If WriteTable is called after WriteRecordBatch, it will find that there is a in-flight row-group, so it will close the buffered row-group, create a non-buffered one, and write to it

Thanks for the explanation @mapleFU . Perhaps we can add an optional argument to parquet::arrow::FileWriterImpl::WriteTable to tell it whether to act more like WriteRecordBatch?
Something like:
  /// \brief Write a Table to Parquet.
  ///
  /// If `use_buffering` is false, then any pending row group is closed
  /// at the beginning and at the end of this call.
  /// If `use_buffering` is true, this function reuses an existing
  /// buffered row group until the chunk size is met, and leaves
  /// the last row group open for further writes.
  /// It is recommended to set `use_buffering` to true to minimize
  /// the number of row groups, especially when calling `WriteTable`
  /// with small tables.
  ///
  /// \param table Arrow table to write.
  /// \param chunk_size maximum number of rows to write per row group.
  /// \param use_buffering Whether to potentially buffer data.
  virtual ::arrow::Status WriteTable(
      const ::arrow::Table& table, int64_t chunk_size = DEFAULT_MAX_ROW_GROUP_LENGTH,
      bool use_buffering = false) = 0;

Yes, that is what I suggested above as well, to just change the behaviour, at least for the current write_batch

I'll add a WriteRecordBatch with hint row-group size, and then bump it to Python script.

authors: jorisvandenbossche mapleFU pitrou


text:
GH-36720: [R] stringr modifier functions cannot be called with namespace prefix

Rationale for this change
Bug in implementation of string modified functions caused them to be swallowed when prefixed with stringr namespace
What changes are included in this PR?
Strips out the stringr:: prefix when expressions contain stringr::fixed, stringr::coll, string::boundary or stringr::regex
Are these changes tested?
Yes
Are there any user-facing changes?
Yes

Closes: #36720

CI failure is due to #36787 and not changes in this PR, so I'll merge


text:
GH-39210: [C++][Parquet] Avoid WriteRecordBatch from produce zero-sized RowGroup

Rationale for this change

WriteRecordBatch might produce zero-sized row-group, which is mentioned in https://github.com/apache/arrow/issues/39210 . This patch avoid WriteRecordBatch from produce zero-sized RowGroup.
What changes are included in this PR?

adding a check for zero-sized row-group
Are these changes tested?

Yes
Are there any user-facing changes?
no

Closes: #39210

@mapleFU Can you rebase from main?

done

done

authors: mapleFU pitrou


text:
ARROW-6839: [Java] Add APIs to read and write "custom_metadata" field of IPC file footer

Related to ARROW-6839.
This is the java implementation of format change in ARROW-6836.

@wesm @jacques-n @emkornfield Please help take a look when you have time, thanks!

cc @lidavidm also

Thanks @rymurr for the review.
@emkornfield Do you have other comments? otherwise I'll merge this in several days later.

No please go ahead and merge.

@tianchen92 did you use the merge tool to merge this (if you haven't please read https://cwiki.apache.org/confluence/display/ARROW/Guide+for+Committers+and+Project+Maintainers)

@tianchen92 did you use the merge tool to merge this (if you haven't please read https://cwiki.apache.org/confluence/display/ARROW/Guide+for+Committers+and+Project+Maintainers)

Hmm, I directly merged this by click "squash and merge" button, should we revert this?

Not familiar enough with the implications @kszucs @kou what are the correct steps here?

The commit looks OK but @tianchen92 please use the merge tool not the GitHub UI to merge patches.

The commit looks OK but @tianchen92 please use the merge tool not the GitHub UI to merge patches.

I see, sorry for the incorrect merge operation :)

authors: emkornfield tianchen92 wesm


text:
ARROW-11964: [Rust][DataFusion] Extend constant folding and parquet filtering support

This PR enables to use some more constant folding and adds support for Between in parquet filtering.
Additionally, it adds some more debug logging.
Before:
[2021-03-14T17:23:20Z DEBUG datafusion::execution::context] Optimized logical plan:
     Projection: #SUM(l_extendedprice Multiply l_discount) AS revenue
      Aggregate: groupBy=[[]], aggr=[[SUM(#l_extendedprice Multiply #l_discount)]]
        Filter: #l_shipdate GtEq CAST(Utf8("1994-01-01") AS Date32) And #l_shipdate Lt CAST(Utf8("1995-01-01") AS Date32) And #l_discount BETWEEN Float64(0.06) Minus Float64(0.01) AND Float64(0.06) Plus Float64(0.01) And #l_quantity Lt Int64(24)
          TableScan: lineitem projection=Some([4, 5, 6, 10]), filters=[#l_shipdate GtEq CAST(Utf8("1994-01-01") AS Date32), #l_shipdate Lt CAST(Utf8("1995-01-01") AS Date32), #l_discount BETWEEN Float64(0.06) Minus Float64(0.01) AND Float64(0.06) Plus Float64(0.01), #l_quantity Lt Int64(24)]

After:
[2021-03-14T17:23:20Z DEBUG datafusion::execution::context] Optimized logical plan:
     Projection: #SUM(l_extendedprice Multiply l_discount) AS revenue
      Aggregate: groupBy=[[]], aggr=[[SUM(#l_extendedprice Multiply #l_discount)]]
        Filter: #l_shipdate GtEq Date32("8766") And #l_shipdate Lt Date32("9131") And #l_discount BETWEEN Float64(0.049999999999999996) AND Float64(0.06999999999999999) And #l_quantity Lt Int64(24)
          TableScan: lineitem projection=Some([4, 5, 6, 10]), filters=[#l_shipdate GtEq Date32("8766"), #l_shipdate Lt Date32("9131"), #l_discount BETWEEN Float64(0.049999999999999996) AND Float64(0.06999999999999999), #l_quantity Lt Int64(24)]

Todo: address/fix some tests

@yordan-pavlov I am trying to use the filtering in parquet - however on any query the row groups being filtered seems to be 0. Do you know what could be happening here?

@Dandandan I have mixed feelings about this change -  I get the performance improvement side, but in my opinion it makes the explain output less readable by replacing the string to date cast with a date: #l_shipdate GtEq CAST(Utf8("1994-01-01")  vs #l_shipdate GtEq Date32("8766"); for me the string to date cast is much more readable and I can't really tell what date is  Date32("8766")

@Dandandan I have mixed feelings about this change - I get the performance improvement side, but in my opinion it makes the explain output less readable by replacing the string to date cast with a date: #l_shipdate GtEq CAST(Utf8("1994-01-01")  vs #l_shipdate GtEq Date32("8766"); for me the string to date cast is much more readable and I can't really tell what date is Date32("8766")

The Date32("8766") I don't find very elegant, but it's not really part of this PR I would say. The formatting of constant dates could be improved to not show the integer value of the date but a formatted yyyy-mm-dd date instead. I agree the non-folded version is much easier to understand than this integer value. The idea of the PR is of course to evaluate the constant expressions, which in some cases might enable other optimizations.

The Apache Arrow Rust community is moving the Rust implementation into its own dedicated github repositories arrow-rs and arrow-datafusion. It is likely we will not merge this PR into this repository
Please see the mailing-list thread for more details
We expect the process to take a few days and will follow up with a migration plan for the in-flight PRs.

https://github.com/apache/arrow/pull/10096 has removed the arrow implementation from this repository (it now resides in https://github.com/apache/arrow-rs and https://github.com/apache/arrow-datafusion) in the hopes of streamlining the development process
Please re-target this PR (let us know if you need help doing so) to one/both of the new repositories.
Thank you for understanding and helping to make arrow-rs and datafusion better

authors: Dandandan alamb yordan-pavlov


text:
[BEAM-11695] Combiner packing in Dataflow

Previously, combiner packing did not work in Dataflow because the optimizations were only applied to the proto_pipeline object and not the pipeline object. This fixes combiner packing, and also makes it disabled by default unless enabled using the pre_optimize option.

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[x] Choose reviewer(s) and mention them in a comment (R: @username).
[x] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
Post-Commit Tests Status (on master branch)

Lang
SDK
Dataflow
Flink
Samza
Spark
Twister2

Go

---

---

---

Java

Python

---

---

XLang

---

---

Pre-Commit Tests Status (on master branch)

---
Java
Python
Go
Website
Whitespace
Typescript

Non-portable

Portable
---

---
---
---
---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

Ready for review, PTAL
R: @robertwb @tvalentyn

Ready for review, PTAL
R: @robertwb @tvalentyn

Do we have test coverage exercising these branches? If not could we try to add? This would make code cov happy and help us avoid indentation bugs. LGTM otherwise. Thanks!

Do we have test coverage exercising these branches? If not could we try to add? This would make code cov happy and help us avoid indentation bugs. LGTM otherwise. Thanks!

Thanks, @yifanmai !

Thanks, @yifanmai !

This was merged after release branch was cut.
If this is a blocker for 2.28.0 please send in a cherry-pick to the release branch: https://github.com/apache/beam/tree/release-2.28.0

This was merged after release branch was cut.
If this is a blocker for 2.28.0 please send in a cherry-pick to the release branch: https://github.com/apache/beam/tree/release-2.28.0

I think we will want to cherry-pick this.

I think we will want to cherry-pick this.

OK, #13829 is the cherry-pick for 2.28.

OK, #13829 is the cherry-pick for 2.28.

authors: chamikaramj robertwb tvalentyn yifanmai

repo: apache/beam


text:
[BEAM-12033] Change unimplemented exception to OUT_OF_RANGE.

Please add a meaningful description for your change here

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Choose reviewer(s) and mention them in a comment (R: @username).
[ ] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
Post-Commit Tests Status (on master branch)

Lang
SDK
ULR
Dataflow
Flink
Samza
Spark
Twister2

Go

---
---

---

---

Java

Python

---

---

---

XLang

---

---

---

Pre-Commit Tests Status (on master branch)

---
Java
Python
Go
Website
Whitespace
Typescript

Non-portable

Portable
---

---
---
---
---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

R: @apilloud