text:
superclass doesn't expect arguments

Hi got this error after I change from python2 to python3
line 77, in __init__
    super(TelloNode, self).__init__(
TypeError: __init__() got an unexpected keyword argument 'local_cmd_client_port'

It seems like the init function of the superclass doesn't expect the local_cmd_client_port argument. The argument is coming from this snippet
super(TelloNode, self).__init__(
            local_cmd_client_port=self.local_cmd_client_port,
            local_vid_server_port=self.local_vid_server_port,
            tello_ip=self.tello_ip,
            tello_cmd_server_port=self.tello_cmd_server_port,
            log=log)

i got same issue plz help me
my setting is
(docker)
ros noetic
ubuntu 20.04
and i changed from python2 to 3
root@cc506e2e2b5f:~/catkin_ws# roslaunch tello_driver tello_node.launch
... logging to /root/.ros/log/4156f33e-3a48-11ee-a22e-0242ac110002/roslaunch-cc506e2e2b5f-38675.log
Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://cc506e2e2b5f:43243/

SUMMARY
========

PARAMETERS
 * /rosdistro: noetic
 * /rosversion: 1.16.0
 * /tello/tello/connect_timeout_sec: 10.0
 * /tello/tello/local_cmd_client_port: 8890
 * /tello/tello/local_vid_server_port: 6038
 * /tello/tello/stream_h264_video: True
 * /tello/tello/tello_cmd_server_port: 8889
 * /tello/tello/tello_ip: 192.168.10.1

NODES
  /tello/
    image_compressed (image_transport/republish)
    tello (tello_driver/tello_driver_node.py)

ROS_MASTER_URI=http://localhost:11311

process[tello/tello-1]: started with pid [38696]
process[tello/image_compressed-2]: started with pid [38697]
Traceback (most recent call last):
  File "/root/catkin_ws/src/tello_driver/src/tello_driver_node.py", line 306, in <module>
    main()
  File "/root/catkin_ws/src/tello_driver/src/tello_driver_node.py", line 300, in main
    robot = TelloNode()
  File "/root/catkin_ws/src/tello_driver/src/tello_driver_node.py", line 76, in __init__
    super(TelloNode, self).__init__(
TypeError: __init__() got an unexpected keyword argument 'local_cmd_client_port'
[tello/tello-1] process has died [pid 38696, exit code 1, cmd /root/catkin_ws/src/tello_driver/src/tello_driver_node.py __name:=tello __log:=/root/.ros/log/4156f33e-3a48-11ee-a22e-0242ac110002/tello-tello-1.log].
log file: /root/.ros/log/4156f33e-3a48-11ee-a22e-0242ac110002/tello-tello-1*.log
^C[tello/image_compressed-2] killing on exit
shutting down processing monitor...
... shutting down processing monitor complete
done

authors: jeonsion mgrallos


text:
Use loop_control/loop_var with all loops

This avoids Ansible shadowing warnings for the item variable when this
role is used inside of a loop.

Fixes #434

It seems that the tests failed on CI, I'm guessing because item wasn't replaced in some of the Jinja2 template files. I've fixed this in f3a80cb and will squash once the tests are passing.
BTW, how does one run the tests locally? I didn't find any documentation on this and couldn't get molecule to run with molecule --base-config molecule/_shared/base.yml test --scenario-name ubuntu-18.04, as I got an error:
ansible_compat.errors.InvalidPrerequisiteError: Computed fully qualified role name of consul does not follow current galaxy requirements.
Please edit meta/main.yml and assure we can correctly determine full role name

@bbaassssiiee is it possible for you to re-trigger the CI tests?

source: gharchive/pull-request

repo: ansible-community/ansible-consul

license_type: permissive

license_source: github-api


text:
Light-Copy-Edit HM803

7 "a demo interface can be found on youtube" : capitalize "YouTube."
30 "of Ansible collections)." : Period should be inside parenthesis "collections.)"
58 "data in real-time" : Hyphen between "real" and "time" not needed
89 "Or, use the" : Comma after "Or" not needed

Thank you all, I'm new to GitHub, thank you for your patience.

repo: ansible/ansible-navigator


text:
Workflow Schedulers are not exported

Running AWX 2.0.1 / Ansible 2.7.1 / Tower-CLI 3.3.0
I am trying to export my configuration - including schedulers for my workflows
Everything seems fine, but unfortunately with "tower-cli receive --all" I do not get any of my configured schedulers - as well as receiving the workflow itselft does not show up any entry... ("tower-cli receive --workflow JOBNAME")
Meanwhile schedulers for Job Templates are included.
Did I miss any update? / Is this planned?
Cheers,
Matthias

With this version I tested:

export [✓]
import [✓]

meanwhile a site notice: the version 3.3.0 from this master branch seems not to be identical to the version 3.3.0 which is distributed via pip...... not so nice.....

merged in master - thanks!

source: gharchive/issue

repo: ansible/tower-cli


text:
Getting issue with PyFluent Workflow with monitors, report definitions

🔍 Before submitting the issue

[X] I have searched among the existing issues
[X] I am using a Python virtual environment

🐞 Description of the bug
Getting following traceback while executing pyfluent workflow.
Workflow includes monitor data
Exception in thread Thread-5:
Traceback (most recent call last):
  File "C:\Users\host_name\AppData\Local\Programs\Python\Python38\lib\threading.py", line 932, in _bootstrap_inner
Exception in thread Thread-15:
Traceback (most recent call last):
  File "C:\Users\host_name\AppData\Local\Programs\Python\Python38\lib\threading.py", line 932, in _bootstrap_inner
Exception in thread Thread-12:
Traceback (most recent call last):
  File "C:\Users\host_name\AppData\Local\Programs\Python\Python38\lib\threading.py", line 932, in _bootstrap_inner
    self.run()
    self.run()
  File "C:\Users\host_name\AppData\Local\Programs\Python\Python38\lib\threading.py", line 870, in run
  File "C:\Users\host_name\AppData\Local\Programs\Python\Python38\lib\threading.py", line 870, in run
    self.run()
  File "C:\Users\host_name\AppData\Local\Programs\Python\Python38\lib\threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
    self._target(*self._args, **self._kwargs)
  File "d:\Research\test\dev\.venv\lib\site-packages\ansys\fluent\core\streaming_services\events_streaming.py", line 52, in _process_streaming
  File "d:\Research\test\dev\.venv\lib\site-packages\ansys\fluent\core\streaming_services\events_streaming.py", line 52, in _process_streaming
    self._target(*self._args, **self._kwargs)
  File "d:\Research\test\dev\.venv\lib\site-packages\ansys\fluent\core\streaming_services\events_streaming.py", line 52, in _process_streaming
    call_back(
    call_back(
  File "d:\Research\test\dev\.venv\lib\site-packages\ansys\fluent\core\streaming_services\monitor_streaming.py", line 143, in refresh
  File "d:\Research\test\dev\.venv\lib\site-packages\ansys\fluent\core\streaming_services\monitor_streaming.py", line 143, in refresh
    call_back(
  File "d:\Research\test\dev\.venv\lib\site-packages\ansys\fluent\core\streaming_services\monitor_streaming.py", line 143, in refresh
    self.start()
    self.start()
  File "d:\Research\test\dev\.venv\lib\site-packages\ansys\fluent\core\streaming_services\streaming.py", line 63, in start
  File "d:\Research\test\dev\.venv\lib\site-packages\ansys\fluent\core\streaming_services\streaming.py", line 63, in start
    self.start()
  File "d:\Research\test\dev\.venv\lib\site-packages\ansys\fluent\core\streaming_services\streaming.py", line 63, in start
    self._prepare()
    self._prepare()
  File "d:\Research\test\dev\.venv\lib\site-packages\ansys\fluent\core\streaming_services\monitor_streaming.py", line 146, in _prepare
  File "d:\Research\test\dev\.venv\lib\site-packages\ansys\fluent\core\streaming_services\monitor_streaming.py", line 146, in _prepare
    self._prepare()
    self._update_dataframe()
    self._update_dataframe()
  File "d:\Research\test\dev\.venv\lib\site-packages\ansys\fluent\core\streaming_services\monitor_streaming.py", line 193, in _update_dataframe
  File "d:\Research\test\dev\.venv\lib\site-packages\ansys\fluent\core\streaming_services\monitor_streaming.py", line 193, in _update_dataframe
  File "d:\Research\test\dev\.venv\lib\site-packages\ansys\fluent\core\streaming_services\monitor_streaming.py", line 146, in _prepare
    monitors_name = list(monitor_set_info["monitors"]) + ["xvalues"]
    monitors_name = list(monitor_set_info["monitors"]) + ["xvalues"]
KeyError: 'monitors'
KeyError: 'monitors'
    self._update_dataframe()
  File "d:\Research\test\dev\.venv\lib\site-packages\ansys\fluent\core\streaming_services\monitor_streaming.py", line 193, in _update_dataframe
    monitors_name = list(monitor_set_info["monitors"]) + ["xvalues"]
KeyError: 'monitors'

📝 Steps to reproduce
Set report definitions, e.g.,

solver.solution.report_definitions.volume["max-temp"] = {
	"report_type": "volume-max",
	"zone_names": ["cell_zone"],
	"field": "temperature",
}
solver.solution.monitor.report_plots["max-temp-track"] = {
	"report_defs": ["max-temp"],
}
solver.solution.monitor.report_files["max-temp-track"] = {
	"file_name": r".\\max-track-temp-over-time.out",
	"report_defs": ["max-temp"],
}
solver.solution.monitor.report_files["max-temp-track"].report_defs = [
	"max-temp",
	"flow-time",
]
solver.solution.report_definitions.single_val_expression["charge-into-track"] = {
	"define": "charge_in"
}
solver.solution.monitor.report_plots["charge-into-track"] = {}

💻 Which operating system are you using?
Windows
📀 Which ANSYS version are you using?
V2023R1
🐍 Which Python version are you using?
3.10
📦 Installed packages
ansys-api-fluent==0.3.12
ansys-api-platform-instancemanagement==1.0.0
ansys-fluent-core==0.16.0
ansys-platform-instancemanagement==1.1.2
asttokens==2.2.1
backcall==0.2.0
black==23.7.0
certifi==2023.7.22
charset-normalizer==3.2.0
click==8.1.7
colorama==0.4.6
comm==0.1.4
contourpy==1.1.0
cycler==0.11.0
debugpy==1.6.7.post1
decorator==5.1.1
docker==6.1.3
executing==1.2.0
fonttools==4.42.1
grpcio==1.57.0
h5py==3.9.0
idna==3.4
importlib-metadata==6.8.0
importlib-resources==6.0.1
ipykernel==6.25.1
ipympl==0.9.3
ipython==8.12.2
ipython-genutils==0.2.0
ipywidgets==8.1.0
jedi==0.19.0
jupyter_client==8.3.0
jupyter_core==5.3.1
jupyterlab-widgets==3.0.8
kiwisolver==1.4.5
lxml==4.9.3
matplotlib==3.7.2
matplotlib-inline==0.1.6
mypy-extensions==1.0.0
nest-asyncio==1.5.7
numpy==1.24.4
packaging==23.1
pandas==2.0.3
parso==0.8.3
pathspec==0.11.2
pickleshare==0.7.5
Pillow==10.0.0
platformdirs==3.10.0
prompt-toolkit==3.0.39
protobuf==3.20.3
psutil==5.9.5
pure-eval==0.2.2
Pygments==2.16.1
pyparsing==3.0.9
python-dateutil==2.8.2
pytz==2023.3
pywin32==306
PyYAML==6.0.1
pyzmq==25.1.1
requests==2.31.0
six==1.16.0
stack-data==0.6.2
tomli==2.0.1
tornado==6.3.3
traitlets==5.9.0
typing_extensions==4.7.1
tzdata==2023.3
urllib3==2.0.4
wcwidth==0.2.6
websocket-client==1.6.2
widgetsnbextension==4.0.8
zipp==3.16.2

Issue Resolved!
Closing this issue! There is an issue in Report definition.
Thanks!

repo: ansys/pyfluent

license: MIT


text:
test_launch_remote_instance keeps failing in 25.1 CI

<previous messages>

        # Start fluent with launch_fluent
        # Note: This is mocking to start Fluent, but actually reusing the common one
        # Thus cleanup_on_exit is set to false
>       fluent = launcher.launch_fluent(cleanup_on_exit=False, mode="solver")
tests/test_launcher_remote.py:74:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../../.cache/pypoetry/virtualenvs/ansys-fluent-core-bzdQ0Z5z-py3.9/lib/python3.9/site-packages/ansys/fluent/core/utils/deprecate_args.py:54: in wrapper
    return func(*args, **kwargs)
../../../.cache/pypoetry/virtualenvs/ansys-fluent-core-bzdQ0Z5z-py3.9/lib/python3.9/site-packages/ansys/fluent/core/launcher/launcher.py:267: in launch_fluent
    return launcher()
../../../.cache/pypoetry/virtualenvs/ansys-fluent-core-bzdQ0Z5z-py3.9/lib/python3.9/site-packages/ansys/fluent/core/launcher/pim_launcher.py:160: in __call__
    return launch_remote_fluent(
../../../.cache/pypoetry/virtualenvs/ansys-fluent-core-bzdQ0Z5z-py3.9/lib/python3.9/site-packages/ansys/fluent/core/launcher/pim_launcher.py:235: in launch_remote_fluent
    fluent_connection = FluentConnection(
../../../.cache/pypoetry/virtualenvs/ansys-fluent-core-bzdQ0Z5z-py3.9/lib/python3.9/site-packages/ansys/fluent/core/fluent_connection.py:374: in __init__
    self.health_check.check_health()
../../../.cache/pypoetry/virtualenvs/ansys-fluent-core-bzdQ0Z5z-py3.9/lib/python3.9/site-packages/ansys/fluent/core/services/health_check.py:62: in check_health
    response = self._stub.Check(request, metadata=self._metadata)
../../../.cache/pypoetry/virtualenvs/ansys-fluent-core-bzdQ0Z5z-py3.9/lib/python3.9/site-packages/grpc/_interceptor.py:277: in __call__
    response, ignored_call = self._with_call(
../../../.cache/pypoetry/virtualenvs/ansys-fluent-core-bzdQ0Z5z-py3.9/lib/python3.9/site-packages/grpc/_interceptor.py:329: in _with_call
    call = self._interceptor.intercept_unary_unary(
../../../.cache/pypoetry/virtualenvs/ansys-fluent-core-bzdQ0Z5z-py3.9/lib/python3.9/site-packages/ansys/fluent/core/services/interceptors.py:122: in intercept_unary_unary
    return self._intercept_call(continuation, client_call_details, request)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <ansys.fluent.core.services.interceptors.GrpcErrorInterceptor object at 0x7fbe086592b0>
continuation = <function _UnaryUnaryMultiCallable._with_call.<locals>.continuation at 0x7fbe08c519d0>
client_call_details = _ClientCallDetails(method='/grpc.health.v1.Health/Check', timeout=None, metadata=[], credentials=None, wait_for_ready=None, compression=None)
request =
    def _intercept_call(
        self,
        continuation: Any,
        client_call_details: grpc.ClientCallDetails,
        request: Any,
    ) -> Any:
        response = continuation(client_call_details, request)
        if response.exception() is not None and response.code() != grpc.StatusCode.OK:
            grpc_ex = response.exception()
            ex = RuntimeError(grpc_ex.details())
            ex.__context__ = grpc_ex
>           raise ex from None
E           RuntimeError: The request failed to complete because the connection has not been validated. You can validate the connection by connecting to a connection service with a password or by sending a password as a key, value pair in the context of each request. The key should be the password.

For now I'm disabling it in #2886.

@mkundu1
This is PyPIM issue as it is not able to validate the password for IP and port of given Fluent session.

@mkundu1 @seanpearsonuk
If we skip self.health_check.check_health() in FluentConnection then this test is passing otherwise we get the above error message.

We can not skip self.health_check.check_health() and this test is also uses mock PyPIM instance therefore we can skip this test.
We have already confirmed that Fluent is launching successfully in the Ansys Lab. (with both latest dev release on PyPI and GitHub cloned version)

Fast-loading "/ansys_inc/v242/fluent/fluent24.2.0/addons/afd/lib/hdfio.bin"
Done.

Reading from fluent-3ddp-242-7b0gvnwdlcpipal8vmgqppg7tbnpl50qnpdgk-c5p8p:"/work/mixing_elbow.msh.h5" in NODE0 mode ...
  Reading mesh ...
       17822 cells,     1 cell zone  ...
          17822 mixed cells,  zone id: 87
       91581 faces,     7 face zones ...
           2168 polygonal wall faces,  zone id: 34
            268 polygonal wall faces,  zone id: 33
            155 polygonal pressure-outlet faces,  zone id: 32
            152 polygonal velocity-inlet faces,  zone id: 31
             55 polygonal velocity-inlet faces,  zone id: 30
           2001 polygonal symmetry faces,  zone id: 29
          86782 mixed interior faces,  zone id: 89
       66417 nodes,     3 node zones ...

Building...
     mesh
     materials,
     interface,
     domains,
     zones,
	Skipping thread 20 of domain 1 (not referenced by grid).
	Skipping thread 21 of domain 1 (not referenced by grid).
	Skipping thread 22 of domain 1 (not referenced by grid).
	Skipping thread 23 of domain 1 (not referenced by grid).
	Skipping thread 24 of domain 1 (not referenced by grid).
	Skipping thread 25 of domain 1 (not referenced by grid).
	Skipping thread 26 of domain 1 (not referenced by grid).
	Skipping thread 27 of domain 1 (not referenced by grid).
	Skipping thread 28 of domain 1 (not referenced by grid).
	wall-elbow
	wall-inlet
	outlet
	cold-inlet
	hot-inlet
	symmetry-xyplane
	interior--elbow-fluid
	elbow-fluid
     parallel,
Done.
Mesh is now scaled to meters.

 Domain Extents:
   x-coordinate: min (m) = -2.000000e-01, max (m) = 2.000000e-01
   y-coordinate: min (m) = -2.250000e-01, max (m) = 2.000000e-01
   z-coordinate: min (m) = 0.000000e+00, max (m) = 4.992264e-02
 Volume statistics:
   minimum volume (m3): 2.443789e-10
   maximum volume (m3): 5.720378e-07
     total volume (m3): 2.500657e-03
 Face area statistics:
   minimum face area (m2): 3.226185e-08
   maximum face area (m2): 7.873458e-05
 Checking mesh............................
Done.

Note: Settings to improve the robustness of pathline and
      particle tracking have been automatically enabled.


text:
picker组件，事件穿透

组件
picker
设备机型
iphoneSE
手机系统
IOS
APP
钉钉小程序
组件库版本
1.7.0
预期行为
picker组件下的选项，划动的时候不应该影响主页面
复现步骤

新增一个小程序页面，页面超过100vh
添加一个picker组件，设置选项
划动选项，可以看到选项滑动的同时，下方页面也在滑动

在IOS钉钉测试好像没问题呢，但是没有测试iphoneSE。是只是iphoneSE才有问题？

在IOS钉钉测试好像没问题呢，但是没有测试iphoneSE。是只是iphoneSE才有问题？

小程序开发者工具中是存在这样的问题，我想当然的认为手机上也这样。刚开始写小程序（抱歉抱歉抱歉）

IDE 3.7.2 修复了这个问题：https://opendocs.alipay.com/mini/ide/beta_log

authors: bytemain jc9702507 pijiupaowanzi

repo: ant-design/ant-design-mini


text:
下载模板加载less错误

@import "~antd/lib/style/themes/default.less";
./src/style/index.less
Module build failed:
// https://github.com/ant-design/ant-motion/issues/44
.bezierEasingMixin();
^
Inline JavaScript is not enabled. Is it set in your options?
in /Users/mac/work/singularity/sigly/node_modules/antd/lib/style/color/bezierEasing.less (line 110, column 0)

参考 https://github.com/ant-design/ant-design/issues/7927#issuecomment-372513256

@yesmeck
const path = require('path');
let disableDynamicImport = false;
let hostUrl = 'https://app.yellowmessenger.com';
if (process.env.NODE_ENV && process.env.NODE_ENV.toLowerCase() === 'development') {
disableDynamicImport = true;
hostUrl = 'https://staging.yellowmessenger.com';
}
export default {
entry: 'src/index.js',
extraBabelPlugins: [['import', { libraryName: 'antd', libraryDirectory: 'es', style: true }]],
env: {
development: {
extraBabelPlugins: ['dva-hmr'],
},
},
alias: {
"@babel/runtime": path.resolve(__dirname,"node_modules/@babel/runtime"),
components: path.resolve(__dirname, 'src/components/'),
},
ignoreMomentLocale: true,
theme: './src/theme.js',
html: {
template: './src/index.ejs',
},
disableDynamicImport,
publicPath: '/',
hash: true,
define: {
CDN_URL: process.env.CDN_URL || 'https://cdn.yellowmessenger.com',
HOST_URL: process.env.HOST_URL || hostUrl,
XMPP_DOMAIN: process.env.XMPP_URL || 'xmpp.yellowmssngr.com',
INSIGHTS_STORAGE: process.env.INSIGHTS_STORAGE
}
};
Where to add that config ?

@yesmeck
const path = require('path');
let disableDynamicImport = false;
let hostUrl = '';
if (process.env.NODE_ENV && process.env.NODE_ENV.toLowerCase() === 'development') {
disableDynamicImport = true;
hostUrl = '';
}
export default {
entry: 'src/index.js',
extraBabelPlugins: [['import', { libraryName: 'antd', libraryDirectory: 'es', style: true }]],
env: {
development: {
extraBabelPlugins: ['dva-hmr'],
},
},
alias: {
"@babel/runtime": path.resolve(__dirname,"node_modules/@babel/runtime"),
components: path.resolve(__dirname, 'src/components/'),
},
ignoreMomentLocale: true,
theme: './src/theme.js',
html: {
template: './src/index.ejs',
},
disableDynamicImport,
publicPath: '/',
hash: true,
define: {
CDN_URL: process.env.CDN_URL || '',
HOST_URL: process.env.HOST_URL || hostUrl,
XMPP_DOMAIN: process.env.XMPP_URL || '',
INSIGHTS_STORAGE: process.env.INSIGHTS_STORAGE
}
};
where to add that configuration ?

authors: agarwal00s osinglee yesmeck

repo: ant-design/ant-motion


text:
🐛[BUG] 搜索栏样式错误，希望小版本更新修改

🐛 bug 描述 [详细地描述 bug，让大家都能理解]
1.0.18 版本开始出现样式错误
📷 复现步骤 [清晰描述复现步骤，让别人也能看到问题]

🏞 期望结果 [描述你原本期望看到的结果]

💻 复现代码 [提供可复现的代码，仓库，或线上示例]
umi  antd pro 模版 更新 pro-table 到最新版本
© 版本信息

Ant Design Pro 版本: [e.g. 4.0.0] 最新
umi 版本  最新
浏览器环境 chrome
开发环境 [e.g. mac OS]

🚑 其他信息 [如截图等其他信息可以贴在这里]

@Wildlifes 已经修了，更新一下就好了

thx

repo: ant-design/pro-table


text:
Introduce new external facing domain, and kubernetes-sigs/external-dns

Resolves: https://github.com/anthr76/infra/issues/42

[ ] Create a CNAME to NWK1 dyndns on Kutura with external DNS CRD
[x] Write ansible roles to sync tsig key for internal kubernetes zone between servers
[x] Deploy internal external-dns
[ ] Deploy external external-dns
[ ] Create origin-ca-issuer cert with cert-manager for new domain
[ ] Remove all old entries
[ ] Set appropriate annotations/crds on services
[ ] Smoke test

ansible role has been added though unsure if I should be reloading the daemon for bind to see the key. The role is very messy and plan on iterating in the future when ansible and non-k8s infra get's a refresh.

ansible role has been added though unsure if I should be reloading the daemon for bind to see the key. The role is very messy and plan on iterating in the future when ansible and non-k8s infra get's a refresh.

More safety should be added to ansible before finishing this PR. Internal is successfully setup and needs DNSEndpoints/Annotations on services

More safety should be added to ansible before finishing this PR. Internal is successfully setup and needs DNSEndpoints/Annotations on services

Flux alerts have been suspended while testing

Flux alerts have been suspended while testing

Considering deploy lstio in place for traefik given these constraints:

Only one TLSStore in traefik.
CRDs are simply not working
TLS options are equally as rigid as TLS stores
HTTPs upgrades at the entrypoint requires root privileges

Considering deploy lstio in place for traefik given these constraints:

Only one TLSStore in traefik.
CRDs are simply not working
TLS options are equally as rigid as TLS stores
HTTPs upgrades at the entrypoint requires root privileges

This can be merged.
Takeaways
lstio soon.

This can be merged.
Takeaways
lstio soon.


text:
Default values for width and height

In V1 we didn't need to specify a height and width of the image to show. In V2 it is specified but it doesn't work if don't specify one. I get the error:
Error while updating property 'transform' of a view managed by: 'RTCImageView'

You are right and I am going to fix it in next minor release. Anyway I recommend to specify width and heigh to make rendering more smooth and without delay.

i thick width and height is not required, i just have a url, i dont know its width and height.
if i want to know the info, i must to download the image.
if i download the image before image view, loading will never used.

Fixed in version 2.0.2

authors: Hunsu antonKalinin musicode

repo: antonKalinin/react-native-image-view


text:
Connection refuse issue

Hi,
I get the following when trying to build a database.
[bgzip] Could not open ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/000/430/045/GCF_000430045.1_ASM43004v1/GCF_000430045.1_ASM43004v1_genomic.fna.gz: Connection refused
haystac: error: Unable to download assembly ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/000/430/045/GCF_000430045.1_ASM43004v1/GCF_000430045.1_ASM43004v1_genomic.fna.gz
None
Any thoughts?
Also the step "Job 4: Splitting the representative RefSeq table in smaller tables." takes too long every time that I need to make a different database. Since it is the same, is there a way to perform it once, store the smaller tables locally and use them in next database builds?
Finally, please consider to incorporate a new argument that will output the number and which genomes are going to be downoaded at the end, without actually downloading anything. This will help evaluate if a taxon is missing.

Thanks Antony,
the connection error persists but I think it is due to my institute's firewall. Is there a way to work around it own my own? If not, and if I just have to ask my admin, then you may close this issue.
Regarding the second point made, I guess that an option may be added for someone that does not care about the plasmids in order to save time.
Best,
Nikos

Hello Nikos,
I hope you are well !
The best I can think of is if you download the sequence you need manually from NCBI and then point to its location using the --sequences-file flag.
That's a great suggestion actually, and we'll definitely try our best to incorporate it in a future version of haystac. Again thank you for your patience, your suggestions and of course for using haystac.
Please do let me know if you run into any other issues !
Best,
Antony

authors: antonisdim npsonis

repo: antonisdim/haystac


text:
Add date operations

It would be great  to be able to use date operations like transaction.Date > now+5d
Perhaps using https://github.com/timberio/go-datemath ?

Hi,
This is what I was thinking of adding to the language at first but decided to keep it simple for a while. But now, maybe it's time to add this functionality now.
Currently, this can be done by operator overloading:
	type Segment struct {
		Date time.Time
	}
	type Request struct {
		Segments []Segment
		Before   func(a, b time.Time) bool
		Date     func(s string) time.Time
	}

	code := `Date("2001-01-01") < Segments[0].Date`

	program, err := expr.Compile(code, expr.Env(&Request{}), expr.Operator("<", "Before"))
	if err != nil {
		fmt.Printf("%v", err)
		return
	}

	request := &Request{
		Segments: []Segment{
			{Date: time.Date(2019, 7, 1, 0, 0, 0, 0, time.UTC)},
		},
		Before: func(a, b time.Time) bool {
			return a.Before(b)
		},
		Date: func(s string) time.Time {
			date, err := time.Parse("2006-01-02", s)
			if err != nil {
				panic(err)
			}
			return date
		},
	}

	output, err := expr.Run(program, request)
	if err != nil {
		fmt.Printf("%v", err)
		return
	}

	fmt.Printf("%v", output)

	// Output: true

You're right, I can get 90% of the way there with the operator overloading and that package.
import (
	"log"
	"testing"
	"time"

	"github.com/antonmedv/expr"
	"github.com/stretchr/testify/require"
	"github.com/timberio/go-datemath"
)

type DateTransaction struct {
	ReportDate time.Time
}

type Request struct {
	TX DateTransaction
}

func (*Request) Today() time.Time {
	return time.Now().Truncate(24 * time.Hour)
}

func (*Request) Date(d string) time.Time {
	t, err := time.Parse("20060102", d)
	if err != nil {
		panic(err)
	}
	return t
}

func (*Request) Add(t time.Time, amount string) time.Time {
	log.Printf("adding %s to %s", amount, t)
	expr, _ := datemath.Parse("now+" + amount)
	return expr.Time(datemath.WithNow(t))
}

func (*Request) Subtract(t time.Time, amount string) time.Time {
	log.Printf("subtracting %s from %s", amount, t)
	expr, _ := datemath.Parse("now-" + amount)
	return expr.Time(datemath.WithNow(t))
}

func (*Request) Round(t time.Time, amount string) time.Time {
	expr, _ := datemath.Parse("now/" + amount)
	return expr.Time(datemath.WithNow(t))
}

func (*Request) Before(a, b time.Time) bool {
	return a.Before(b)
}

func (*Request) After(a, b time.Time) bool {
	return a.After(b)
}

func TestToday(t *testing.T) {
	// Is there a holiday in the next three days?
	rule, err := expr.Compile(`Today()+'3b' != Today()+'3d'`, expr.Env(&Request{}), expr.Operator("<", "Before"), expr.Operator(">", "After"), expr.Operator("+", "Add"), expr.Operator("+", "Subtract"), expr.Operator("/", "Round"))
	require.NoError(t, err)
	require.NotNil(t, rule)

	output, err := expr.Run(rule, &Request{})
	require.NoError(t, err)
	require.Equal(t, true, output)
}

func TestDateMath(t *testing.T) {

	// Is the report date within two business days?
	rule, err := expr.Compile(`TX.ReportDate+'2b' > Date('20191020')`, expr.Env(&Request{}), expr.Operator("<", "Before"), expr.Operator(">", "After"), expr.Operator("+", "Add"), expr.Operator("+", "Subtract"), expr.Operator("/", "Round"))
	require.NoError(t, err)
	require.NotNil(t, rule)

	type testcase struct {
		tx    DateTransaction
		match bool
	}

	d := func(d string) time.Time {
		t, err := time.Parse("20060102", d)
		if err != nil {
			panic(err)
		}
		return t
	}

	for _, c := range []testcase{
		{
			tx: DateTransaction{
				ReportDate: d("20191016"),
			},
			match: false,
		},
		{
			tx: DateTransaction{
				ReportDate: d("20191017"),
			},
			match: true,
		},
	} {
		output, err := expr.Run(rule, &Request{
			TX: c.tx,
		})
		require.NoError(t, err)

		require.Equal(t, c.match, output)
	}

}

I think to add support for time.Time in next release. With new builtins date and duration.

authors: antonmedv elliots

repo: antonmedv/expr

license: mit

license_source: bigquery


text:
Issue with shortcut assistant

Small issue with the shortcut assistance preset. Aurora seems to always think I am holding down alt. has been working fine for the past week, then yesterday it starting doing this, resulting my my, ctrl, tab, f4, v and E always being yellow. not sure why.
Anyone else have this issue and know how to resolve it? Tried reseting the profile and that didn't work, issue still persisted on the default.

More information on this. Seems to happen whenever I first boot it up, or if I open a new application. For example, I launch Hearthstone and the issue occurs. Hit alt and it goes away.

Hm, I'm not sure, I haven't been able to replicate it. I'll try and have a look at it to see if I can see anything out of place.

Could you try this build and let me know if it solves your issue.

I've downloaded and extracted the file, but what am I supposed to do with
it from here? I don't see any executable, do I simply over write the
existing aurora files with this? Where is the existing aurora app stored? I
can't find it locally.
On Wed, May 30, 2018 at 3:53 AM, Simon W notifications@github.com wrote:

Could you try this build
https://ci.appveyor.com/project/antonpup/aurora/build/0.6.1-dev4-319/artifacts
and let me know if it solves your issue.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/antonpup/Aurora/issues/929#issuecomment-392829504,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AhSZHdbrF36stvlU17tE3bQUjwMjfCjCks5t3W7ngaJpZM4RP-FR
.

Apologies, must have missed that (no coffee yet today). Will check again.
On Thu, May 31, 2018 at 9:26 AM, Simon W notifications@github.com wrote:

@BeardyMcBeardFace https://github.com/BeardyMcBeardFace I just
downloaded it and there is "Aurora.exe", all you need to do is extract the
zip into a folder and run Aurora.exe.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/antonpup/Aurora/issues/929#issuecomment-393324844,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AhSZHUhYkQYsV6i3M-_ntJ5Lpk4Xpi3iks5t3w51gaJpZM4RP-FR
.

Seems to fix the issue when I load up hearthstone, which was the most
common cause. I'll keep at it and if it acts up again I'll let you know.
Many thanks :)
On Thu, May 31, 2018 at 9:27 AM, Andrew Cockell andrew.cockell@gmail.com
wrote:

Apologies, must have missed that (no coffee yet today). Will check again.
On Thu, May 31, 2018 at 9:26 AM, Simon W notifications@github.com wrote:

@BeardyMcBeardFace https://github.com/BeardyMcBeardFace I just
downloaded it and there is "Aurora.exe", all you need to do is extract the
zip into a folder and run Aurora.exe.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/antonpup/Aurora/issues/929#issuecomment-393324844,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AhSZHUhYkQYsV6i3M-_ntJ5Lpk4Xpi3iks5t3w51gaJpZM4RP-FR
.

Awesome! I'll close this for now, it can be reopened if you run into the same issues as before again.

authors: BeardyMcBeardFace simon-wh

repo: antonpup/Aurora


text:
Compilation errors for PACKAGE BODY AS_XLSX - Oracle Database 23c Free

Hi @antonscheffer,
Thanks for this great package!
Having this error compiling the package body in Oracle Database 23c Free:

Compilation errors for PACKAGE BODY AS_XLSX

Error: PL/SQL: ORA-00932: expression is of data type BLOB, which is incompatible with expected data type -
Line: 2401
Text: select ora_hash( p_img ) into l_hash from dual;

Error: PL/SQL: SQL Statement ignored
Line: 2401
Text: select ora_hash( p_img ) into l_hash from dual;

Database info:
OCI: version 12.1  (21.13.0.0.0)
Oracle Database 23c Free Release 23.0.0.0.0

Compilation works on previous releases.

Should be fixed

That was fast and elegant.. Thanks!


text:
Cherry pick of #5592: Remove redundant log in fillPodInfo/fillServiceInfo, #5731: Use labels to enhance records filtering in FA e2e tests, #5770: Improve flow-visibility e2e test on upstream release 1.14

Cherry pick of https://github.com/antrea-io/antrea/pull/5592 https://github.com/antrea-io/antrea/pull/5731 https://github.com/antrea-io/antrea/pull/5770 on release-1.14.
https://github.com/antrea-io/antrea/pull/5592: Remove redundant log in fillPodInfo/fillServiceInfo
https://github.com/antrea-io/antrea/pull/5731: Use labels to enhance records filtering in FA e2e tests
https://github.com/antrea-io/antrea/pull/5770: Improve flow-visibility e2e test

/test-all

authors: antoninbas yuntanghsu

repo: antrea-io/antrea


text:
Upgrade AWS and Azure SDK

Upgrade AWS SDK to v1.44.201 and Azure SDK v68.0

/nephe-test-e2e-kind, /nephe-test-e2e-aks, /nephe-test-e2e-eks

/nephe-test-e2e-kind

/nephe-test-e2e-azure

/nephe-test-e2e-azure

/nephe-test-e2e-azure
/nephe-test-e2e-aks

/nephe-test-e2e-aks

/nephe-test-e2e-azure, /nephe-test-e2e-aks, /nephe-test-e2e-aws

/nephe-test-e2e-azure

repo: antrea-io/nephe


text:
Upgrading controller runtime to v0.12.2

Controller runtime v0.12.1 had issues in which, controller logging
format was overwritten with last caller Name/Value. Please refer
to PR https://github.com/kubernetes-sigs/controller-runtime/pull/1911
Signed-off-by: Rahul Jain rahulj@vmware.com

/nephe-test-e2e-aws


text:
v2-galaxybase

👀 PR includes
✨ Feature

add v2-galaxybase

🐛 Bugfix

windows start error

What did you have any problems start in windows? Everything worked fine in my Windows environment.
My environment:

Arch: x64
System: Windows 11 21H2 @ 22000.1574
Node: v16.16.0

Start Steps
Install
pnpm i

remove assert

delete as any assertion in v2-galaxybase

start
cd gi-site
pnpm start

result

Things looks good

authors: Aarebecca JiKhun


text:
Marker在移动端浏览器，点击事件不生效

移动端浏览器无法出发点击事件 "click"

是否使用了GaodeMapV2？着急的话先尝试使用GaodeMap


text:
refactor: 精简 usePagination hooks deps

👀 PR includes
🎨 Enhance

[x] Refactoring

🔧 Chore

[x] Test case

📝 Description

useSpreadSheet 保存 s2 实例时，除了 ref，增加 state 保存，让下游 hooks deps 依赖更新时更加精准
结合 1 精简了 usePagination 内 hooks 的 dep 依赖
补充 https://github.com/antvis/S2/pull/1090 的 test

:tada: This PR is included in version @antv/s2-react-v1.9.1 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


text:
Update the  logo conflict in Navbar

Related Issue
Fixes:  #1171
Description
Logo was not positioned correctly within the navbar. The problem occurred due to a conflict between  two  logo elements in the HTML , causing overlapping and incorrect positioning.
Type of PR

[ x] Bug fix
[ ] Feature enhancement
[ ] Documentation update
[ ] Other (specify): _______________

Screenshots / videos (if applicable)
Before
https://github.com/anuragverma108/SwapReads/assets/121204667/97d4bfcc-7a31-43a2-a010-f32eb6d9daa8
After

Checklist:

[ x] I have made this change from my own.
[ x] My code follows the style guidelines of this project.
[x ] I have performed a self-review of my own code.
[x ] My changes generate no new warnings.
[ x] I have tested the changes thoroughly before submitting this pull request.
[ x] I have provided relevant issue numbers and screenshots after making the changes.

The dark mode button is coming out of Navbar. Also its height is so les.

okay , will work on it , should i create a new issue


text:
BUG:Footer Overlapping Issue #1615

BUG:Footer Overlapping Issue #1615
Fixes:  #1615
Description
fixed social icons overlapping

Type of PR

[y ] Bug fix
[ ] Feature enhancement
[ ] Documentation update
[ ] Other (specify): _______________

Screenshots / videos (if applicable)
before:

Now:

Checklist:

[ y] I have made this change from my own.
[ ] I have taken help from some online resources.
[y ] My code follows the style guidelines of this project.
[y ] I have performed a self-review of my own code.
[y ] I have commented my code, particularly in hard-to-understand areas.
[y ] I have made corresponding changes to the documentation.
[y ] My changes generate no new warnings.
[y ] I have tested the changes thoroughly before submitting this pull request.
[ y] I have provided relevant issue numbers and screenshots after making the changes.

Resolve the conflicts in the mentioned files. Ensure you pull all the latest changes before pushing your feature/bug fixes to the main branch. After resolving the conflicts, please open a new PR for review.
Thank you!


text:
fixed margin create account

Related Issue
fixed margin-top to create account text
Fixes:  ##3881
Description
fixed margin-top to create account text

Type of PR

[X ] Bug fix
[X ] Feature enhancement
[ X] Documentation update
[ X] Other (specify): _______________

Checklist:

[X ] I have made this change from my own.
[ ] I have taken help from some online resources.
[X ] My code follows the style guidelines of this project.
[X ] I have performed a self-review of my own code.
[X ] I have commented my code, particularly in hard-to-understand areas.
[ X] I have made corresponding changes to the documentation.
[ ] My changes generate no new warnings.
[X ] I have tested the changes thoroughly before submitting this pull request.
[X ] I have provided relevant issue numbers and screenshots after making the changes.

this is redundant issue. similar to #3884


text:
issue in unity 2022 lts

Library\PackageCache\com.avt.tool.scene.selection@b447b58e0e\Editor\ToolbarCallback.cs(57,17): error CS0103: The name 'm_currentToolbar' does not exist in the current context

thank for your feedback. The issue has been fixed in newer version by installing requirement package from https://github.com/marijnz/unity-toolbar-extender.git

authors: Samuelincoom anviettrung

repo: anviettrung/Unity-Quick-Scene-Selection


text:
Consider using go modules to implement Sysl modules.

Like https://gohugo.io/hugo-modules/

Dupe

Well spotted. #415

repo: anz-bank/sysl


text:
Fix make grammar to work again

make grammar stopped working when grammar.proto was removed in 71e3057. This PR adds back that file. However, there were other issues with the ANTLR code generation:

patching the generated code to make it thread-safe was fragile
code was generated in a temp dir and copied into the target, but some files were missed (*.tokens)
the SyslParser.tokens file was out-of-date - it was not updated for the last parser change.

Fix all these things and make a more make-oriented Makefile for the ANTLR build
@anz-bank/sysl-developers

PTAL @anz-gordonj7


text:
extract reward-info crate

Problem
solana_sdk::reward_type and solana_sdk::reward_info impose a solana-sdk dependency on solana-rpc-client-api
Summary of Changes

Move them into a single solana-reward-info crate.
Re-export in their old modules and add deprecation notices
Make serde optional in the new crate

Maybe I'm missing something, but where are these types used in solana-rpc-client-api?

The dependency is indirect, via solana-transaction-status-client-types

Seems like we're pretty close to being able to just swap in the smaller crates for all of solana-rpc-client-api.

Yeah this is being tracked in #3027

repo: anza-xyz/agave


text:
micro-async-data-loader default cron is not in Quartz supported format

Should be
"asyc.data.schedular.cron.loader": "0 * * * * ?"
DataCleaner in micro-async-data-writer likely to suffer from the same issue.

Released in 0.87.1 https://github.com/aol/micro-server/pull/230

repo: aol/micro-server


text:
Relocate accumulo-proxy to separate repository on its own release schedule

accumulo-proxy is really a separate project from the core Accumulo distribution, and I think it should be shipped separately, and maintained separately in its own repo.
This will also help isolate the maintenance interest, so that if there isn't any current interest, no time is wasted on it, and if there is, work can be done independently from the releases of the core Accumulo distribution.

@cjmctague thanks!

Nope sorry, this is partially done.  Still some proxy tests that need to be migrated.

Yeah, sorry, the new repo was created, but the old proxy code is still there. It needs to be removed, and cross-referenced with the new repo, to make sure everything made it over before it is removed. I think some tests were still dependent on the proxy, maybe some minicluster features were using it also. Not sure.

@cjmctague Are you interested in working on this?

@milleruntime Yea sure. So tests need to be moved over and the current tests using the proxy need to be ported to use the new repo?

@milleruntime Yea sure. So tests need to be moved over and the current tests using the proxy need to be ported to use the new repo?

Tests which test the proxy should be moved over. Tests which aren't testing the proxy shouldn't be using the proxy. The proxy repo should depend on the main Accumulo repo, but not the other way around.

Working this ticket instead of 1180

Status:  Updated accumulo-proxy project from the accumulo proxy module including version updates and was able to get it to build and test runs.  Now I'll look into using the jar with the accumulo project instead of proxy module.

@ctubbsii  the accumulo-proxy contrib subdirectory has a create-release-candidate.sh script which does not exist in the accumulo project.  I'm  not sure what it does and why it is there.  Just curious.

Running a verify gave me this [ERROR] Exception is caught when Exception is not thrown in org.apache.accumulo.proxy.ProxyServer.updateRowsConditionally(String, Map) [org.apache.accumulo.proxy.ProxyServer] At ProxyServer.java:[line 2207] REC_CATCH_EXCEPTION
the code line is
} catch (Exception e) {
handleException(e);
return null;
}
This apparently verifies okay in the accumulo project so why is it failing here - I'm not savvy with verify.  I changed the catch to the following and it verifies but not sure if you have better suggestions.
} catch (Throwable e) {
handleException(new Exception(e));
return null;
}

authors: cjmctague ctubbsii hkeebler milleruntime

repo: apache/accumulo


text:
Use internal RFile scan code for RecoveryLogsIterator

RecoveryLogsIterator was using the client side RFile scanner which does not allow for the Recovery scope to be used. This change modifies the RecoveryLogsIterator to use the server side RFile reader code (FileOperations) with the Recovery crypto environment
Closes #2979

It's unfortunate we have two separate RFile read paths. It'd be really nice to just have one that was easier for users to use, and also supported all our internal needs.

It's unfortunate we have two separate RFile read paths. It'd be really nice to just have one that was easier for users to use, and also supported all our internal needs.

Yeah that would be nice. For this situation, we would need to have the RFile API take a CryptoService or something to differentiate it from the table properties. Maybe once crypto graduates from being experimental, we can add something to the RFile API. But there are likely to be other situations like this though, where we don't won't to expose internals and still need FileSKVIterator.

I think having the RFile API accept an optional CryptoService was something I had previously suggested, but yeah, maybe it's premature if it's still marked experimental.

authors: ctubbsii dlmarion milleruntime


text:
Add a new AccumuloStore for FATE

This adds support for storing FATE data in an Accumulo table instead of storing in Zookeeper for #3559
This is currently a work in progress and is not finished but the main operations are implemented for the AcccumuloStore. Common code has been refactored and moved out of ZooStore and into AbstractFateStore. Most of the common code moved has to do with the code that handles signaling and tracking reserved/defered transactions. The store just uses a batch writer for now but we will likely need to use conditional mutations at some point.
There is a test called AccumuloStoreReadWriteIT which tests out the various operations on the new store. The test only tests read/writing to the store...the store hasn't been tested with the overall FATE framework yet as some more work probably needs to be done to test it for real (such as handling which store to use so we can still use the ZK store for root/metadata).
The next steps (besides addressing any review comments) are to try testing it for real with FATE (and finish up any supporting tasks for that) and then eventually support conditional mutations.

@keith-turner - I did some refactoring and now FateIT is abstract and there are test classes to run the 4 tests for both Zookeeper and Accumulo stores. One thing I noticed was one of the tests seems to have a race condition that I think needs fixing as I saw an occasional failure with the accumuo store version and i added a comment here.
This should be close to being able to go out of draft status if we are ok doing some follow ons for things like conditional mutations.


text:
APEXMALHAR-2276 over writing the value of key only when the time is g…

…reater than equal to latest time of the entry

@ilooner @davidyan74 @chaithu14  please have a look

repo: apache/apex-malhar


text:
docs: Add weekly-report (0814-0827)

weekly-report (0814-0827)

Revise the Chinese version accordingly @JYan00

Done.

repo: apache/apisix-website


text:
GH-359: [Release] Update versions for 16.1.0

Closes https://github.com/apache/arrow-cookbook/issues/359.

After merging, I ran dev/release/02-update.sh 16.1.0 successfully.

repo: apache/arrow-cookbook


text:
Improve codebase readability and error messages by and consistently handle downcasting

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
It is a common code pattern in DataFusion to downcast an ArrayRef to a concrete arrow array type.
There are actually at least three patterns to do so in the datafusion codebase https://github.com/apache/arrow-datafusion/search?l=Rust&q=downcast
I think this leads to

Poor user experience (as a DataFusion bug might panic and quit the process, which is impolite)
less readable code
less efficient feature development and review (as people have to figure out which of the three patterns to use and they can't just look at what convention is used in the rest of the codebase

Pattern 1: downcast_ref()
let array = array.as_any().downcast_ref::<Date32Array>().unwrap();

panics on error
Pattern 2: use as_XXX_array functions from arrow
let array = as_boolean_array(&array);

panics on error
Pattern 3: check type and return an error:
let array = array.as_any().downcast_ref::<Date32Array>().ok_or(
    DataFusionError::Execution("Cast Date32Array error".to_string()),
)?;

Returns an error to the user (a better behavior I think)
Describe the solution you'd like
I would like to use one consistent pattern throughout the codebase.  Specifically I propose functions that do the error checking / error message generation:
/// Return `array` cast to `Int32Array` otherwise an error
pub fn as_int32_array(array: &dyn Array) -> Result<&Int32Array, DataFusionError> {
  array.as_any().downcast_ref::<Int32Array>().ok_or(
    DataFusionError::Execution(format!("Expected a Int32Array, got: ", array.data_type()),
  )
}

/// And similar functions for the remaining array types

And then change all dynamic casts in the codebase:

If they are fallable, to use the new functions that make a nice error message
If they are infallible / somewhere were passing out the result is hard, use arrow functions

Describe alternatives you've considered
Leave the status quo
Additional context
Inspired by @avantgardnerio 's suggestion  https://github.com/apache/arrow-datafusion/pull/3110/files#r945847871 (but this is not the first time it has come up)

cc @avantgardnerio and @Jimexist and @Dandandan

I think this is a good first issue because it will be largely mechanical and will give potential contributors wide exposure to the codebase

I think this is a good first issue because it will be largely mechanical and will give potential contributors wide exposure to the codebase

this is attractive 👍  let me try this

Hi. While this issue is getting old, I might help but have some questions. . I will give as_int32_array as a generic example so you can think of those questions for all other types.

In which file we should implement as_int32_array function? I have implemented it in datafusion\common\src\lib.rs but I am not quite sure about the architecture.
Before opening pr, I want to be sure that this sample is implemented correctly. I added this function:

pub fn as_int32_array(array: &dyn Array) -> Result<&Int32Array> {
    array.as_any().downcast_ref::<Int32Array>().ok_or(
      DataFusionError::Execution(format!("Expected a Int32Array, got: {}", array.data_type()),
    ))
}

And change this:
https://github.com/apache/arrow-datafusion/blob/10e64dc013ba210ab1f6c2a3c02c66aef4a0e802/datafusion/optimizer/src/simplify_expressions/expr_simplifier.rs#L851-L854
To this:
let arg0 = as_int32_array(&args[0])?;

Is this the correct way? simplify_expressions::test_evaluator_udfs test runs successfully after implementation.

In which file we should implement as_int32_array function? I have implemented it in datafusion\common\src\lib.rs but I am not quite sure about the architecture.

I think that is reasonable. Or perhaps something like datafusion\common\src\cast.rs ?

Before opening pr, I want to be sure that this sample is implemented correctly. I added this function:

Thanks @retikulum  -- that looks good to me
What I would suggest is doing this in a series of smaller PRs -- perhaps add a few functions like as_int32_array, as_uint32_array, etc and consolidate some of the uses.
That way we can then get feedback from the wider community before you spend a large amount of your time changing the entire codebase

In which file we should implement as_int32_array function? I have implemented it in datafusion\common\src\lib.rs but I am not quite sure about the architecture.

I think that is reasonable. Or perhaps something like datafusion\common\src\cast.rs ?

datafusion\common\src\cast.rs is my first idea but I just don't want to create a new file :) cast  fits better.

Before opening pr, I want to be sure that this sample is implemented correctly. I added this function:

Thanks @retikulum -- that looks good to me
What I would suggest is doing this in a series of smaller PRs -- perhaps add a few functions like as_int32_array, as_uint32_array, etc and consolidate some of the uses.
That way we can then get feedback from the wider community before you spend a large amount of your time changing the entire codebase

This is exactly what I planned. Moreover, I will not get lost in the ocean. Thanks for feedback @alamb

authors: alamb retikulum waitingkuo

repo: apache/arrow-datafusion


text:
Getting started guide for new users (who want to use DatatFusion in their proejct)

Is your feature request related to a problem or challenge?
If we want to have DataFusion used as the core of many new systems, we need it to be as easy as possible for someone to get their idea working on top of DataFusion.
The current user guide I think helps setup the basics of the project and get a "hello world" style program going but then kind of leave the reader in a "now what" type situation:   https://arrow.apache.org/datafusion/user-guide/example-usage.html
Describe the solution you'd like
I would like a document, perhaps similar in style to the polars user guide: https://pola-rs.github.io/polars-book/user-guide/

This User Guide is an introduction to the Polars DataFrame library. Its goal is to introduce you to Polars by going through examples and comparing it to other solutions. Some design choices are introduced here. The guide will also introduce you to optimal usage of Polars.

Basically I am thinking of something that would have helped @BubbaJoe get up to speed
The examples directory holds a bunch of examples: https://github.com/apache/arrow-datafusion/tree/main/datafusion-examples
Potential outline:

Using the SQL API
Working with Exprs
Using the DataFrame API
Building LogicalPlans
Catalogs:  tables / schemas / tables / catalogs
Adding User Defined Functions: Scalar/Window/Aggregate/
Adding custom TableProviders
Extending DataFusion's operators: custom LogicalPlan andExecutionPlans
ExecutionPlans

Describe alternatives you've considered
No response
Additional context
This idea was suggested by @MrPowers

If someone wanted to help out the DataFusion project helping with this one would be awesome. A good first step would be to make the skeleton of the topics above in https://github.com/apache/arrow-datafusion/tree/main/docs and leave placeholder text (like "Coming Soon")
Then we can work together on writing the content in a few different PRs

This sounds great, really excited!
We'll either want two user guides or one user guide that's half in Python / half in Rust.
I guess that 99% of the users that want to query data via an API will want to do so in SQL / Python.  The Python DataFrame user guide is way more important than the Rust one.
Users leveraging DataFusion to build tools for other engines (e.g. delta-rs) are much more likely to be using Rust.
Perhaps we divide the documentation as follows:

https://arrow.apache.org/datafusion-python/ for the DataFrame/Python user guide
https://arrow.apache.org/datafusion/ for the Rust guide on building with DataFusion

I don't think we should invest in building out the DataFusion Rust DataFrame API docs yet because it's a lower ROI activity.  We should build a URL structure that allows for this however.

The Python DataFrame user guide is way more important than the Rust one.

I agree this is more important for "end users" rather than developers who are building with Rust

Perhaps we divide the documentation as follows:

That sounds great -- I filed  https://github.com/apache/arrow-datafusion-python/issues/432 to track the work for the python bindings

I filed a bunch of tickets for follow on work and update the description of this ticket
https://github.com/apache/arrow-datafusion/issues/7302
https://github.com/apache/arrow-datafusion/issues/7304
https://github.com/apache/arrow-datafusion/issues/7305
https://github.com/apache/arrow-datafusion/issues/7306
https://github.com/apache/arrow-datafusion/issues/7307
https://github.com/apache/arrow-datafusion/issues/7308

authors: MrPowers alamb


text:
Use Vectorized Partition Kernel for Window Frame Calculation

Is your feature request related to a problem or challenge?
BuiltInWindowExpr currently makes use of the vectorised partition function when evaluating window frames with ranks, but falls back to a very inefficient mechanism based on performing scans using Vec<ScalarValue> as a comparison key for window functions that rely on the window frame, including user defined window functions.
Describe the solution you'd like
I may be missing something but it looks like window functions are only evaluated on data that is already sorted, as such it should be possible to identify the partition boundaries in a single pass, and then use this information to efficiently extract the window boundaries.
Tagging @mustafasrepo and @alamb who may have more context on this design
Describe alternatives you've considered
No response
Additional context
#7517 tracks remove ScalarValue's implementation of Ord to cleanup the code and discourage this sort of approach in future

When a window function with rank is used.  A single pass on the whole data is sufficient. What we need is finding partition boundaries in the data. However, for window frames (with groups, and range) for each row we find the preceding and following boundary (corresponding to each row, determined by values of the row.) Hence this calculation should be done for each row(storing last boundaries found, helps in decreasing search space) on the subset of the table(around row of the interest.)
I am not sure whether window frame boundary search can be vectorized. That being said, I guess instead of Vec<ScalarValue> we can use Row type.

I may be missing something but it looks like window functions are only evaluated on data that is already sorted, as such it should be possible to identify the partition boundaries in a single pass, and then use this information to efficiently extract the window boundaries.

The window functions are indeed always evaluated on data that is sorted.
We might be able to operate directly on Array for windows that are defined as a row size, and as @mustafasrepo  suggests operate on Rows for windows defined by value

authors: alamb mustafasrepo tustvold


text:
Support IS NOT NULL predicates in PruningPredicate

Is your feature request related to a problem or challenge?
the PruningPredicate currently does not handle IS NOT NULL predicates
Describe the solution you'd like
No response
Describe alternatives you've considered
No response
Additional context
@appletreeisyellow  added an initial implementation in https://github.com/apache/arrow-datafusion/pull/9208 but I believe it didn't work in all cases, as described in https://github.com/apache/arrow-datafusion/issues/9230

take

Perhaps @appletreeisyellow  is more suitable for handling this issue😄

One thing I should have noted in the description here is that I think implementing IS NOT NULL will require a new API to PruningStatistics (the total_row_count) . Conveniently, @appletreeisyellow  is adding this API as part of https://github.com/apache/arrow-datafusion/issues/9171  I think
So in other words, this ticket is likely blocked until #9171 is done
BTW there is a bunch more backstory on https://github.com/apache/arrow-datafusion/issues/9230

I believe that @Ted-Jiang  implemented this in https://github.com/apache/datafusion/pull/9989 + https://github.com/apache/datafusion/pull/10051 so closing this issue

authors: alamb guojidan


text:
datafusion-cli cursor isn't on the right position in windows 7 cmd

Describe the bug
use pageup key to get the history command,
if I move the cursor to the left position of g and press (, it insert the ( after g
❯ select g(enerate_series(1,10);

To Reproduce
datafusion-cli

input
select g(enerate_series(1,10);
+-------------------------------------+
| generate_series(Int64(1),Int64(10)) |
+-------------------------------------+
| [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]     |
+-------------------------------------+
```

then
use pageup key to get the history command,
if I move the cursor to the left position of g and press `(`

### Expected behavior

insert `(` before g

### Additional context

_No response_

maybe the root of the problem is the prompt ❯ is a utf-8 character, it occupied multi-bytes, suggest use an ASCII charactor, such as
>

this prompt displayed as a box in cmd of windows 7

I think this is a good first issue for someone with a windows machine to test on

@alamb Hi! I have a windows machine and changed all those ❯ to >. Is this OK?
And I wanna test if the change does work how can I rebuild the datafusion to update the change by using the command datafusion-cli

Great @colommar !
To test the change locally you can do something like this
cd datafusion/datafusion-cli
cargo run

@l1t1 I guess it was fine now. Can you help me test it on your pc?

could you attach the binary file?

could you attach the binary file?

@l1t1 Sure! Is this OK? Does it help you? datafusion-cli.zip

it works, very thanks

authors: alamb colommar l1t1


text:
Add proper support for null literal by introducing ScalarValue::Null

Which issue does this PR close?

Closes #2363 .
Rationale for this change

To solve Null constants issues listed in #1184 , and since /apache/arrow-rs#1572 Null casted from and to most of types in arrow-rs kernel, it's reasonable that introduce Null type to df for type coercion.
What changes are included in this PR?

Introduce ScalarValue::Null type to df
Are there any user-facing changes?

No.

For someone reviews this pr,
since arrow = { version = "12" } doesn't contain patch /apache/arrow-rs#1572, thanks for @alamb suggestion, I create winkerdu/test_null_cast in my private repo like:
git checkout master -b winkerdu/test_null_cast
git reset --hard dbc47e030c3038d40dbef39fbf3b39ae41f9e98a                 # prepare for version 12.0.0 release
git cherry-pick b50cc737efa2a4f2b1e27c0f3da3bc0403c6a2b6

cc @alamb @andygrove @yjshen @xudong963

cc @alamb  PTAL, thank you ❤

This looks great @WinkerDu  -- thank you -- I am working to get arrow 13 -- see  https://github.com/apache/arrow-datafusion/pull/2382. It should only be a few more days now.

This looks great @WinkerDu -- thank you -- I am working to get arrow 13 -- see #2382. It should only be a few more days now.

Thanks @alamb look forward to it !

Since #2382 is merged, I switch this pr to open status for code reviewing

cc @alamb @andygrove @yjshen @xudong963
Please have a review, thank you

This is looking great. Thanks @WinkerDu! I don't have any additional comments beyond what @alamb has already raised

cc @alamb  PTAL, thank you

🎉  we are going to have real null support 😍

Thank you all @alamb @andygrove

authors: WinkerDu alamb andygrove


text:
Refactor Expr::Cast to use a struct.

Which issue does this PR close?

Part of #2175 and #3807
Rationale for this change

What changes are included in this PR?

Refactor Expr::Cast to use a struct.
Are there any user-facing changes?

@ygf11 thanks, have resolved


text:
Make datafusion-physical-expr compatible with blake3/traits-preview feature.

Which issue does this PR close?
Closes #4781.
What changes are included in this PR?
Makes datafusion-physical-expr compatible with blake3/traits-preview feature.
Are there any user-facing changes?
No

Thanks @BoredPerson  -- if this PR passes CI it looks good to me

authors: BoredPerson alamb


text:
Backport: Update version and Changelog for 21.1.0

Which issue does this PR close?
re https://github.com/apache/arrow-datafusion/issues/5744
Rationale for this change
Prepare for 22.1.0 release as described on https://github.com/apache/arrow-datafusion/issues/5744
What changes are included in this PR?
Update version to 22.1.0 and add (manual) Changelog entry on branch-21
Are these changes tested?
No
Are there any user-facing changes?
Eventually they will be

CI is not running on this PR for some reason -- I will merge to the branch and verify that the checks pass there.


text:
flight_sql_client always failed with error "Received RecordBatch prior to Schema" when run a query

Which part is this question about

arrow-flight
Describe your question

I am using arrow-flightsql-odbc as a flightsql server. I am using flight_sql_client from arrow-rs/arrow-flight to act as a client to test it. It could connect to the server successfully. But when I run a simple query select 1, the client always failed with an error "Received RecordBatch prior to Schema". I am trying to figure out why this error could be generated. At server side I printed the RecordBatch value as below. It includes a schema and result in one RecordBatch. At client side the flight_data is also printed. We can see the FlightDataDecoder state is None which is causing this failure. From arrow-rs/arrow-flight/src/decode.rs we can see the flight data handle for RecordBatch. When the state is None, error "Received RecordBatch prior to Schema" is printed.  So my question is how I could make the state to be not None. The RecordBatch already included the schema,  should I send the schema in another standalone RecordBatch? The schema is already sent at the first FlightInfo message. Also the FlightInfo is handled at execute_flight of arrow-rs/arrow-flight/src/bin/flight_sql_client.rs. So I am not sure what is wrong. Could someone help here? Thanks in advance!
-- arrow-flightsql-odbc server side RecordBatch value.
[src/odbc_command_handler.rs:176] batch = RecordBatch {
    schema: Schema {
        fields: [
            Field {
                name: "",
                data_type: Int32,
                nullable: false,
                dict_id: 0,
                dict_is_ordered: false,
                metadata: {},
            },
        ],
        metadata: {},
    },
    columns: [
        PrimitiveArray<Int32>
        [
          1,
        ],
    ],
    row_count: 1,
}

--FlightInfo at client side
[arrow-flight/src/bin/flight_sql_client.rs:198] &info = FlightInfo {
    schema: b"\xff\xff\xff\xffx\0\0\0\x10\0\0\0\0\0\n\0\x0c\0\n\0\t\0\x04\0\n\0\0\0\x10\0\0\0\0\x01\x04\0\x08\0\x08\0\0\0\x04\0\x08\0\0\0\x04\0\0\0\x01\0\0\0\x14\0\0\0\x10\0\x14\0\x10\0\0\0\x0f\0\x04\0\0\0\x08\0\x10\0\0\0\x18\0\0\0 \0\0\0\0\0\0\x02\x1c\0\0\0\x08\0\x0c\0\x04\0\x0b\0\x08\0\0\0 \0\0\0\0\0\0\x01\0\0\0\0\0\0\0\0\0\0\0\0",
    flight_descriptor: Some(
        FlightDescriptor {
            r#type: Cmd,
            cmd: b"\nCtype.googleapis.com/arrow.flight.protocol.sql.CommandStatementQuery\x12\n\n\x08SELECT 1",
            path: [],
        },
    ),
    endpoint: [
        FlightEndpoint {
            ticket: Some(
                Ticket {
                    ticket: b"\nCtype.googleapis.com/arrow.flight.protocol.sql.CommandStatementQuery\x12\n\n\x08SELECT 1",
                },
            ),
            location: [],
        },
    ],
    total_records: -1,
    total_bytes: -1,
    ordered: true,
}

--flight_data at client side
[arrow-flight/src/bin/flight_sql_client.rs:210] &flight_data = FlightRecordBatchStream {
    headers: MetadataMap {
        headers: {
            "content-type": "application/grpc",
            "date": "Fri, 02 Feb 2024 22:30:31 GMT",
        },
    },
    trailers: Some(
        LazyTrailers {
            trailers: Mutex {
                data: None,
                poisoned: false,
                ..
            },
        },
    ),
    inner: FlightDataDecoder {
        response: "<stream>",
        state: None,
        done: false,
    },
}

-- code from arrow-rs/arrow-flight/src/decode.rs
impl FlightDataDecoder {
     ......
    /// Extracts flight data from the next message, updating decoding
    /// state as necessary.
    fn extract_message(&mut self, data: FlightData) -> Result<Option<DecodedFlightData>> {
        use arrow_ipc::MessageHeader;
        let message = arrow_ipc::root_as_message(&data.data_header[..])
            .map_err(|e| FlightError::DecodeError(format!("Error decoding root message: {e}")))?;

        match message.header_type() {
            MessageHeader::RecordBatch => {
                println!("Value of self.state before if: {:?}", self.state);
                let state = if let Some(state) = self.state.as_ref() {
                    state
                } else {
                    return Err(FlightError::protocol(
                        "Received RecordBatch prior to Schema",
                    ));
                };

-- FlightInfo handle at arrow-rs/arrow-flight/src/bin/flight_sql_client.rs
async fn execute_flight(
    client: &mut FlightSqlServiceClient<Channel>,
    info: FlightInfo,
) -> Result<Vec<RecordBatch>> {
    let schema = Arc::new(Schema::try_from(info.clone()).context("valid schema")?);
    dbg!(&schema);
    let mut batches = Vec::with_capacity(info.endpoint.len() + 1);
    batches.push(RecordBatch::new_empty(schema));
    info!("decoded schema");

    for endpoint in info.endpoint {
        let Some(ticket) = &endpoint.ticket else {
            bail!("did not get ticket");
        };

        let mut flight_data = client.do_get(ticket.clone()).await.context("do get")?;
        dbg!(&flight_data);
        log_metadata(flight_data.headers(), "header");

        let mut endpoint_batches: Vec<_> = (&mut flight_data)
            .try_collect()
            .await
            .context("collect data stream")?;
        batches.append(&mut endpoint_batches);

        if let Some(trailers) = flight_data.trailers() {
            log_metadata(&trailers, "trailer");
        }
    }
    info!("received data");

    Ok(batches)
}

Additional context

Running on Ubuntu 20.04 64-bit, I also updated  flightsql server proto files and arrow and arrow-odbc dependency to newest versions.

Hey @RoyZhang2022
I have left a comment on arrow-flightsql-odbc since I believe the error originates there, https://github.com/timvw/arrow-flightsql-odbc/issues/17#issuecomment-1925193622
Don't think this is an issue with how arrow-rs handles reading the FlightData

Closing this as looks to be issue in arrow-flightsql-odbc, see https://github.com/timvw/arrow-flightsql-odbc/issues/17#issuecomment-1928457997

repo: apache/arrow-rs


text:
fix: Stop losing precision and scale when casting decimal to dictionary

Which issue does this PR close?

Closes https://github.com/apache/arrow-rs/issues/6381
Rationale for this change

Fix a correct issue described in https://github.com/apache/arrow-rs/issues/6381
What changes are included in this PR?

Fix cast from decimal to dictionary so that the resulting array has the correct precision and scale
Add new tests

Are there any user-facing changes?
Yes, fixes a correctness issue

@viirya @parthchandra this PR fixes a correctness issue that I ran into when working on dictionary support in Comet. I expect there is a more efficient way to do this though so could use some guidance.

Are parquet-testing and testing changes intentional?

oops, no .. reverted. Thanks for catching that.


text:
Make pyarrow installable in Jupyterlite

Describe the enhancement requested
I'm trying to use pyarrow with Panel in Panelite. Panelite is a custom build of Jupyterlite that works with Panel.
The issue is that piplite cannot install pyarrow. It would be great if pyarrow also worked with Pyodide and in Jupyterlite/ Panelite.
Thanks.

import piplite
await piplite.install(['panel', 'xlsxwriter', 'pyarrow'])

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[16], line 2
      1 import piplite
----> 2 await piplite.install(['panel', 'xlsxwriter', 'pyarrow'])

File /lib/python3.10/asyncio/futures.py:284, in Future.__await__(self)
    282 if not self.done():
    283     self._asyncio_future_blocking = True
--> 284     yield self  # This tells Task to wait for completion.
    285 if not self.done():
    286     raise RuntimeError("await wasn't used with future")

File /lib/python3.10/asyncio/tasks.py:304, in Task.__wakeup(self, future)
    302 def __wakeup(self, future):
    303     try:
--> 304         future.result()
    305     except BaseException as exc:
    306         # This may also be a cancellation.
    307         self.__step(exc)

File /lib/python3.10/asyncio/futures.py:201, in Future.result(self)
    199 self.__log_traceback = False
    200 if self._exception is not None:
--> 201     raise self._exception
    202 return self._result

File /lib/python3.10/asyncio/tasks.py:234, in Task.__step(***failed resolving arguments***)
    232         result = coro.send(None)
    233     else:
--> 234         result = coro.throw(exc)
    235 except StopIteration as exc:
    236     if self._must_cancel:
    237         # Task is cancelled right before coro stops.

File /lib/python3.10/site-packages/piplite/piplite.py:102, in _install(requirements, keep_going, deps, credentials, pre)
    100 """Invoke micripip.install with a patch to get data from local indexes"""
    101 with patch("micropip._micropip._get_pypi_json", _get_pypi_json):
--> 102     return await _micropip.install(
    103         requirements=requirements,
    104         keep_going=keep_going,
    105         deps=deps,
    106         credentials=credentials,
    107         pre=pre,
    108     )

File /lib/python3.10/site-packages/micropip/_micropip.py:573, in install(requirements, keep_going, deps, credentials, pre)
    563 wheel_base = Path(getsitepackages()[0])
    565 transaction = Transaction(
    566     ctx=ctx,
    567     ctx_extras=[],
   (...)
    571     fetch_kwargs=fetch_kwargs,
    572 )
--> 573 await transaction.gather_requirements(requirements)
    575 if transaction.failed:
    576     failed_requirements = ", ".join([f"'{req}'" for req in transaction.failed])

File /lib/python3.10/site-packages/micropip/_micropip.py:333, in Transaction.gather_requirements(self, requirements)
    330 for requirement in requirements:
    331     requirement_promises.append(self.add_requirement(requirement))
--> 333 await gather(*requirement_promises)

File /lib/python3.10/asyncio/futures.py:284, in Future.__await__(self)
    282 if not self.done():
    283     self._asyncio_future_blocking = True
--> 284     yield self  # This tells Task to wait for completion.
    285 if not self.done():
    286     raise RuntimeError("await wasn't used with future")

File /lib/python3.10/asyncio/tasks.py:304, in Task.__wakeup(self, future)
    302 def __wakeup(self, future):
    303     try:
--> 304         future.result()
    305     except BaseException as exc:
    306         # This may also be a cancellation.
    307         self.__step(exc)

File /lib/python3.10/asyncio/futures.py:201, in Future.result(self)
    199 self.__log_traceback = False
    200 if self._exception is not None:
--> 201     raise self._exception
    202 return self._result

File /lib/python3.10/asyncio/tasks.py:232, in Task.__step(***failed resolving arguments***)
    228 try:
    229     if exc is None:
    230         # We use the `send` method directly, because coroutines
    231         # don't have `__iter__` and `__next__` methods.
--> 232         result = coro.send(None)
    233     else:
    234         result = coro.throw(exc)

File /lib/python3.10/site-packages/micropip/_micropip.py:340, in Transaction.add_requirement(self, req)
    337     return await self.add_requirement_inner(req)
    339 if not urlparse(req).path.endswith(".whl"):
--> 340     return await self.add_requirement_inner(Requirement(req))
    342 # custom download location
    343 wheel = WheelInfo.from_url(req)

File /lib/python3.10/site-packages/micropip/_micropip.py:435, in Transaction.add_requirement_inner(self, req)
    432 metadata = await _get_pypi_json(req.name, self.fetch_kwargs)
    434 try:
--> 435     wheel = find_wheel(metadata, req)
    436 except ValueError:
    437     self.failed.append(req)

File /lib/python3.10/site-packages/micropip/_micropip.py:303, in find_wheel(metadata, req)
    300     if best_wheel is not None:
    301         return wheel
--> 303 raise ValueError(
    304     f"Can't find a pure Python 3 wheel for '{req}'.\n"
    305     f"See: {FAQ_URLS['cant_find_wheel']}\n"
    306     "You can use `micropip.install(..., keep_going=True)`"
    307     "to get a list of all packages with missing wheels."
    308 )

ValueError: Can't find a pure Python 3 wheel for 'pyarrow'.
See: https://pyodide.org/en/stable/usage/faq.html#micropip-can-t-find-a-pure-python-wheel
You can use `micropip.install(..., keep_going=True)`to get a list of all packages with missing wheels.

Component(s)
Python

Related to https://github.com/pyodide/pyodide/issues/2933

+1
pyarrow is becoming more crucial with pandas 2.0 as well. Would be great to have it supported.

I'm not super familiar with pyodide.  Is the only solution to have a pure python wheel?  There have been other efforts to get arrow-c++ to be usable with emscripten.  If this were done would the C++ then be usable with pyodide?

I am 90% there on arrow in pyodide. I have a running build of pyarrow here, but it needs a little work to be generally buildable by anyone. And a little more work so that it loads straight into pyodide nicely.
This PR:
https://github.com/apache/arrow/pull/35471
is the bulk of the work, because for pyodide / browsers we require a build of arrow cpp that works okay without threads.
Once that is merged, there are a few minor build file changes to add support and cmake presets for emscripten & pyodide - this isn't that hard because we've done a bunch of work in pyodide-build which simplifies stuff. Oh and that all needs adding to the github CI here or else people will break it over time.
Then finally, once there is a sensible build process for emscripten pyarrow, it will probably make sense to add a recipe to pyodide so that it is distributed with core pyodide. That is pretty straightforward (I've done a few of them before) -  basically pretty much a matter of pointing it at the git repository tag and telling it the build command to make arrow-cpp and pyarrow.

Are there any updates on this? Thanks for your incredible work

It'll be included as an available package in the next pyodide release: https://github.com/pyodide/pyodide/pull/4950. Or you can use wheels from https://github.com/joemarshall/pyarrow-pyodide/releases/tag/0.26.2 if you'd like to try it sooner.

I just tried to import from pyodide repl. But its still not working.

Hi @MarcSkovMadsen, you are using Pyodide 0.26.4 – PyArrow has been merged and will be available as a part of the 0.27.0 release, which we are planning soon: https://github.com/pyodide/pyodide/issues/5064
If you wish to use it right now, you would need to serve https://github.com/joemarshall/pyarrow-pyodide/releases/tag/0.26.2 or https://github.com/pyodide/pyodide-numpy-2.0-rebuilds/releases/tag/v1.0.0 through a CORS proxy.

@maartenbreddels. Do you know how we can get the py.cafe link for the .whl file in https://github.com/joemarshall/pyarrow-pyodide/releases/tag/0.26.2? I've tried all combinations of ids I can find in the actions section without luck. Thx.

authors: MarcSkovMadsen agriyakhetarpal elehcimd eliasdabbas joemarshall kylebarron westonpace

repo: apache/arrow


text:
[Python] Partitioned Datasets are not correctly written if partitions differ by case

Describe the bug, including details regarding any error messages, version, and platform.
R and Python Dataset writers share the same issue #38804:
partitions are case-sensitive, this conflicts with case-insensitive filesystems.
import pyarrow as pa
import pyarrow.dataset as ds

tbl = pa.table({
    'var_1': ("arrow", "arrow", "arrow"),
    'var_2': ("arrow", "arrow", "arroW")
})

part = ds.partitioning(
    pa.schema([("var_2", pa.string())]),
    flavor="hive"
)

ds.write_dataset(tbl, "dataset_output", format="parquet", partitioning=part)

ds_out = ds.dataset("dataset_output")

Only one partition is written:
ds_out.files
# ['dataset_output/var_2=arroW/part-0.parquet']

ds_out.to_table()
# pyarrow.Table
# var_1: string
# ----
# var_1: [["arrow"]]

Originally posted by @lgaborini in https://github.com/apache/arrow/issues/38804#issuecomment-1844899886
(Opening a new issue to add more labels)
Component(s)
Parquet, Python

Hi @lgaborini, thanks for the report. Since you can reproduce this with R and Python, it's probable C++ is affected too and may even be the place to put a patch. Could you close this issue and edit the tile and body of https://github.com/apache/arrow/issues/38804 just to have one issue for us to work from? I can go add the labels now.

Done, thanks!

authors: amoeba lgaborini


text:
Roadmap of Arrow.

Hi all,
Where can I find the roadmap of the Arrow project?
I may contribute to this project but I don't want to re-implement components you are working on, and I don't want to go in the wrong direction.
Specifically, do you have the plan to support groupby, join etc. in the future?  Maybe I can help I guess.

Hi @daadaada let's discuss on the mailing list dev@arrow.apache.org?
We do intend to implement an in-memory query engine. There is already DataFusion for Rust and we will be doing such work in C++ (for Python, R, etc. also). For example: I have written a document indicating some of the development areas that I and my team intend to work on in the project:
https://docs.google.com/document/d/12dWBniKW2JQ-5djE3SPjyQXVquCAEmLXVlb1dnhLhQ0/edit

authors: daadaada wesm


text:
[C#][Flight] Add DoExchange initial implementation

Describe the enhancement requested
Add initial implementation for DoExchange method.
PRs and other links:

https://github.com/apache/arrow/pull/40691

Component(s)
C#, FlightRPC

Issue resolved by pull request 40691
https://github.com/apache/arrow/pull/40691

authors: CurtHagenlocher qmmk


text:
ARROW-17496: [Go] Fix Nightly Build

Turns out that the pragma_table_info function in modernc.org/sqlite's package doesn't work correctly in go1.17 either, only in go1.18. As this is only used for testing and the example sqlite flightsql server, rather than anything needed in the flightsql package itself, the bulid failure is easily solved by marking the example and its tests to be only built in go1.18.
As we already have a git workflow that runs with go1.18, the CI will still continue to test the example code, but mamba builds using go1.17 won't break anymore.

@github-actions crossbow submit verify-rc-source-go-*

@github-actions crossbow-submit verify-rc-source-integration-*

@github-actions crossbow submit verify-rc-source-integration-*

The failing verify-rc-source-integration-macos-conda-amd64 is failing with a flightsql timeout on the C++ build, it's unrelated to this issue. @raulcd is this a different issue you filed? or otherwise know about? ie. should it prevent this fix from being submitted?

@github-actions crossbow submit verify-rc-source-integration-macos-conda-amd64

@raulcd is this a different issue you filed? or otherwise know about? ie. should it prevent this fix from being submitted?

I recall seeing this test time out before but I am not sure whether it is tracked. I do not think is related to the fix.

authors: raulcd zeroshade


text:
Update run-test.sh

According to the description in the comments (and observation of how this is invoked), RUN_TYPE should be $2

Ah, this change isn't needed.
Because we use shift. So, we can refer the second argument by $1.

Whoops!  You're right.  Sorry for the false alarm...


text:
MINOR: [C++] Return a BufferVector from CleanListOffsets

Rationale for this change
CleanListOffsets is taking output parameters when it could return multiple buffers in a vector exactly how they are supposed to be used to construct the list-like array.
Besides that, having the logic for setting the validity buffer split between it and the caller is unnecessarily confusing.
What changes are included in this PR?

Addition of a new constructor for MapArray that can accept a pre-allocated BufferVector
Removal of output parameter in CleanListOffsets and change of return type

Are these changes tested?
Yes, by existing tests. A pre-existing MapArray ctor delegates to this new ctor, so all tests that exercise the old ctor also exercise the new.

Could you open an issue for non MINOR change?
Our MINOR definition: https://github.com/apache/arrow/blob/main/CONTRIBUTING.md#Minor-Fixes
(This is also linked from the auto generated message: https://github.com/apache/arrow/pull/36316#issuecomment-1608586156 )

Could you open an issue for non MINOR change?
Our MINOR definition: https://github.com/apache/arrow/blob/main/CONTRIBUTING.md#Minor-Fixes (This is also linked from the auto generated message: #36316 (comment) )

Done.

authors: felipecrv kou


text:
GH-23221: [C++] webassembly / pyodide / emscripten build support

This PR adds a cmake build preset for emscripten / pyodide, and updates the setup for pyarrow so that it is possible to build a webassembly version of libarrow, and a version of pyarrow which works on pyodide.
Things that changed:

Cmake preset ninja-release-emscripten-python
Platform/toolchain override for cmake on emscripten, to set various build parameters
Some fixes to cmake files - e.g. pyarrow can build with static libarrow now, and third party dependencies pass the toolchain file through.
pyarrow knows about ARROW_ENABLE_THREADING and doesn't use threads if they are not enabled in libarrow.
pyarrow sets defaults for inclusion of submodules based on their inclusion in the arrow build. e.g. pyarrow.parquet is built only if ARROW_PARQUET is set. This makes it possible to build in situations where you don't have access to set the build environment variables (e.g. in cross compiling situations like pyodide). It also seems to me to make sense - I can't see why you'd want to build pyarrow without parquet if you've built arrow with it, and the inverse is obviously not possible.

Closes: #23221

Also fixes #34996 once this is in the pyodide package tree.

@kou I'm not sure who is the right person to review this - it's a smidgen of python / python packaging changes and cmake changes.

Can we split this pull request to the C++ part and then the Python part for easy to review?

@kou - I've split out the cmake changes into #37821. Once they're in, I can update this one to be just the pyodide build side of things - I think the python build depends on them, so it isn't worth reviewing the python changes until 37821 is in?

Closing this as all changes are in the other PRs

authors: joemarshall kou


text:
ARROW-6061: [C++] Add ARROW_JSON feature flag for configuring arrow builds without RapidJSON

Tested this in our build environment (which doesn't have rapidjson) and confirmed this patch works.

cc: @pitrou I took a stab at adding this.  I'm not sure if there are other cases to test for but here is how this looks if you try to enable the tests without JSON support:
cmake .. -DARROW_BUILD_TESTS=ON -DARROW_JSON=OFF
...
CMake Error at CMakeLists.txt:266 (message):
  JSON parsing of arrays is required for Arrow unit tests

@hatemhelal Looks good. I'll take a look at the PR, thank you!

Thanks for reviewing this @pitrou, I've tried to address your comments in the latest commit.

Looks good on the principle. I'm assuming you checked passing -DARROW_JSON=off and it works?

I've tested this as far as I'm capable of and happy it works for our use case.

authors: hatemhelal pitrou


text:
AVRO-2343 - Invalid json string caused by logic type timestamp millis

…a valid json string.

I'm using Avro 1.9.1 and this problem rises with logical types :

Time (millisecond precision)
Timestamp (millisecond precision)
Date

There is simply no way to handle dates

@pibou Just a quick note -- the workaround would be to use JsonEncoder/JsonDecoder to generate JSON.
Other than that, I think this PR is a dead end for now because of joda.  Thanks for taking a look at this!

authors: RyanSkraba henrywu2019 pibou

repo: apache/avro


text:
Python container builds always create Python 3.6 environment

When I run ./gradlew sdks:python:container:py37:docker, instead of the command creating a Python 3.7 environment as expected, Python 2.7 (the default) is used.
> Task :sdks:python:setupVirtualenv FAILED
created virtual environment CPython2.7.18.candidate.1-64 in 418ms
Update: Now that Python 2.7 and 3.5 are removed, something similar happens for Python 3.6, the new minimum version.
Imported from Jira BEAM-10316. Original Jira may contain additional context.
Reported by: ibzib.

.close-issue not_planned

authors: Abacn damccorm

repo: apache/beam


text:
[BEAM-8019]  Cherry pick 11844

Without this cross-language KafkaIO users may have to do
pipeline.run(False)
instead of
pipeline.run()
when executing a pipeline using Dataflow.
@TheNeuralBit this should be a pretty safe change to cherry-pick if you are OK with it.

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Choose reviewer(s) and mention them in a comment (R: @username).
[ ] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
Post-Commit Tests Status (on master branch)

Lang
SDK
Apex
Dataflow
Flink
Gearpump
Samza
Spark

Go

---
---

---
---

Java

Python

---

---
---

XLang
---
---
---

---
---

Pre-Commit Tests Status (on master branch)

---
Java
Python
Go
Website

Non-portable

Portable
---

---
---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

R: @TheNeuralBit

Thanks!


text:
[BEAM-7390] Add combineperkey code snippets

Add CombinePerKey code snippets.
R: @aaltay
R: @rosetn
Staged: http://apache-beam-website-pull-requests.storage.googleapis.com/12277/documentation/transforms/python/aggregation/combineperkey/index.html

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Choose reviewer(s) and mention them in a comment (R: @username).
[ ] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
Post-Commit Tests Status (on master branch)

Lang
SDK
Dataflow
Flink
Samza
Spark
Twister2

Go

---

---

---

Java

Python

---

---

XLang

---

---

---

Pre-Commit Tests Status (on master branch)

---
Java
Python
Go
Website

Non-portable

Portable
---

---
---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

It looks like my IDE reformatted the xml automatically, I'll get rid of all the unnecessary edits

The correct staging link seems to be (http://apache-beam-website-pull-requests.storage.googleapis.com/12277/documentation/transforms/python/aggregation/combineperkey/index.html). I reviewed that it LGTM.
Please revert the changes to other files. It makes the history much more cleaner and focused.
A question to both you and @rosetn - There is a quite a bit of content duplication between this page and CombineGlobally page. Is there a more de-duplicated way of presenting this content?

The correct staging link seems to be (http://apache-beam-website-pull-requests.storage.googleapis.com/12277/documentation/transforms/python/aggregation/combineperkey/index.html). I reviewed that it LGTM.

Thanks, forgot to update that part.

Please revert the changes to other files. It makes the history much more cleaner and focused.

Will do

A question to both you and @rosetn - There is a quite a bit of content duplication between this page and CombineGlobally page. Is there a more de-duplicated way of presenting this content?

They are different transforms and so they both have their own code snippets, we could try to merge both into a single page but that might be a little too long since it would have 2 code snippets for each example. @rosetn any thoughts?

Also, similarly there is CombineValues which also has its own set of code snippets. I'll wait for this to resolve before opening a PR for it.

I don't think we should merge the snippets into a single page for the following reasons:

The merged page is too long. The goal is to provide a friendly, quick reference catalog where you can immediately run transforms and understand various ways to use them. Scrolling through content that you are not looking for can be intimidating.
It'd be harder to quickly spot a specific transform if we start combining pages. If you're exploring different transforms, it's easier to find if there are individual listings in the left navigation bar. The transforms Overview page also provide a way to do this, but the left nav exists on every documentation page.

Ack. Thank you @rosetn.
@davidcavazos - please revert the unrelated changes, then we can merge this.

@aaltay I thought I had commits changing history on unrelated files, but now that I'm double checking I think there aren't any.

I'm linking from CombineGlobally since they are related (and I'll probably do the same for CombineValues once I create that PR)
combineperkey.md is a new file since it didn't exist earlier (I'll also have to create combinevalues.md in a later PR)
overview.md adds the CombineGlobally link to the overview page since it didn't exist earlier (and a couple style changes)
documentation.html adds the CombineGlobally link to the side navigation bar, plus my editor automatically formatted the HTML mostly for indentations, but with no content changes other than adding the new page

I can revert the style changes if that's what you're referring to, but it's mostly indenting the HTML so I think it's fine as a one time only thing. Would you prefer me to go back and revert those changes back and only leave adding the new link?

I think it would be good to revert changes to style (mostly in the documentation.html), or separate it to a different commit, or to a new pr.

@aaltay style changes on documentation.html are reverted.

authors: aaltay davidcavazos rosetn


text:
[BEAM-13015] Use a DirectExecutor for state since we are just completing futures on the callback.

Baseline:
Result "org.apache.beam.fn.harness.jmh.ProcessBundleBenchmark.testStateWithCaching":
  4588.967 ±(99.9%) 51.317 ops/s [Average]
  (min, avg, max) = (4518.876, 4588.967, 4778.099), stdev = 68.507
  CI (99.9%): [4537.650, 4640.284] (assumes normal distribution)
Result "org.apache.beam.fn.harness.jmh.ProcessBundleBenchmark.testStateWithoutCaching":
  2725.915 ±(99.9%) 17.143 ops/s [Average]
  (min, avg, max) = (2660.523, 2725.915, 2768.138), stdev = 22.885
  CI (99.9%): [2708.773, 2743.058] (assumes normal distribution)

With DirectExecutor (About a 1-2% change for state heavy workloads):
Result "org.apache.beam.fn.harness.jmh.ProcessBundleBenchmark.testStateWithCaching":
  4653.439 ±(99.9%) 33.903 ops/s [Average]
  (min, avg, max) = (4580.343, 4653.439, 4733.079), stdev = 45.260
  CI (99.9%): [4619.536, 4687.342] (assumes normal distribution)
Result "org.apache.beam.fn.harness.jmh.ProcessBundleBenchmark.testStateWithoutCaching":
  2728.673 ±(99.9%) 24.360 ops/s [Average]
  (min, avg, max) = (2671.593, 2728.673, 2777.028), stdev = 32.520
  CI (99.9%): [2704.313, 2753.034] (assumes normal distribution)

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Choose reviewer(s) and mention them in a comment (R: @username).
[ ] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
ValidatesRunner compliance status (on master branch)

Lang
ULR
Dataflow
Flink
Samza
Spark
Twister2

Go
---

---

Java

Python
---

---

XLang

---

Examples testing status on various runners

Lang
ULR
Dataflow
Flink
Samza
Spark
Twister2

Go
---
---
---
---
---
---
---

Java
---

---
---
---
---
---

Python
---
---
---
---
---
---
---

XLang
---
---
---
---
---
---
---

Post-Commit SDK/Transform Integration Tests Status (on master branch)

Go
Java
Python

Pre-Commit Tests Status (on master branch)

---
Java
Python
Go
Website
Whitespace
Typescript

Non-portable

Portable
---

---
---
---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

R: @scwhittle

Run Java PreCommit

Run Java PreCommit

What is the next step on this PR?

If this is blocked on experimenting with ManagedChannelBuilder.html#directExecutor perhaps we should just submit as is since this improved things and leave a comment to try that later.

@lukecwik LGTM, friendly ping to merge

Run Spotless PreCommit

Run Spotless PreCommit

Run Spotless PreCommit

Run Java PreCommit

Run Java PreCommit

Run Java PreCommit

Run Java_PVR_Flink_Batch PreCommit

It looks like this broke Java PreCommit, see BEAM-14008

authors: TheNeuralBit aaltay lukecwik scwhittle


text:
[BEAM-1717] maven-javadoc-plugin execution id should match the id in …

…apache parent.
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

[x] Make sure the PR title is formatted like:
[BEAM-<Jira issue #>] Description of pull request
[x] Make sure tests pass via mvn clean verify. (Even better, enable
Travis-CI on your fork and ensure the whole test matrix passes).
[x] Replace <Jira issue #> in the title with the actual Jira issue
number, if there is one.
[x] If this contribution is large, please file an Apache
Individual Contributor License Agreement.

R: @davorbonaci

CC: @jbonofre

Coverage increased (+0.006%) to 70.156% when pulling c910d09cf85a40edca4cae24a0b30f7190923859 on amitsela:change-javadoc-execution-id into 5e1be9fa7a55e8c1d20a62576588c5a7f95de92f on apache:master.

authors: amitsela coveralls jbonofre


text:
[Python] Create BigQuery wrapper with pipeline options for ReadAllFromBigQuery

Like #27583, but for ReadAllFromBigQuery. Haven't seen cases where this read method is used, but including it anyways for completion.

R: @riteshghorse can you look at this one too? its similar to #27583

Failures are unrelated to this PR. Merging!


text:
JStorm-runner: Implementation of processing timer

Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Make sure there is a JIRA issue filed for the change (usually before you start working on it).  Trivial changes like typos do not require a JIRA issue.  Your pull request should address just this issue, without pulling in other changes.
[ ] Each commit in the pull request should have a meaningful subject line and body.
[ ] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue.
[ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
[ ] Run mvn clean verify to make sure basic checks pass. A more thorough check will be performed on your pull request automatically.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

@peihe

Merged

Changes Unknown when pulling 9def8c96f8eb228e39bdcbcc5dd12ccfde441b82 on bastiliu:jstorm-runner into ** on apache:jstorm-runner**.

authors: bastiliu coveralls peihe