text:
MatFormFieldAppearance "standard" is missing

Is this a regression?

[X] Yes, this behavior used to work in the previous version

The previous version in which this bug was not present was
14
Description
With v15 the Type MatFormFieldAppearance was refactored to lose the properties legacy and standard.
Hereby standard was replaced by undefined, so the absence of an appearance should represent standard now.
There is a problem when are using provide MAT_FORM_FIELD_DEFAULT_OPTIONS though. When we set the default to something else we will not be able to opt-in for standard anywhere else anymore since  we can not set the appearance explicity to standard anymore
Even if I "trick" TypeScript into taking the string standard by casting it to any, Angular Material will throw since it seems to check the value at runtime and tell me that this is not a valid value.
This used to be possible in v14 and blocks me from migrating to v15.
Reproduction
Steps to reproduce:

{provide: MAT_FORM_FIELD_DEFAULT_OPTIONS, useValue: {appearance: 'outline'}} in your AppComponent
try to set the appearance to standard for any Angular Material Component in your codebase

Expected Behavior
MatFormFieldAppearance should accept standard as a value again
Actual Behavior
MatFormFieldAppearance is typed in a way that standard is absent as a value - and it throws at runtime if you somehow manage to still pass it otherwise.
Environment

Angular: 15
CDK/Material: 15
Browser(s): all
Operating System (e.g. Windows, macOS, Ubuntu): all

The MDC spec do not support these legacy/standard appearances anymore, so Angular team deprecated and removed them.

legacy and standard appearances were deprecated in version 9.0.0

The comments above are correct. The standard and legacy appearances aren't in the Material Design spec anymore.

authors: Totati crisbeto mrmokwa pkelleter


text:
Toolbar Style not applied when not placed in styles.scss

Bug, feature request, or proposal:
Bug
What is the expected behavior?
I have two material components on my test app: a toolbar and a card. Both located in app.component.html.

  <md-toolbar color="primary">

Angular 2

      <button md-icon-button [md-menu-trigger-for]="menu">
        <md-icon class="material-icons">menu</md-icon>
      </button>

  </md-toolbar>

  <md-menu x-position="before" #menu="mdMenu">
    <button md-menu-item>Option 1</button>
    <button md-menu-item>Option 2</button>
  </md-menu>

  <md-card>
       Content
  </md-card>

In app.component.scss I have the following simple styles:
md-toolbar-row {
  justify-content: space-between;
}

md-card {
  max-width: 95%;
  margin: 0.5em auto;
  text-align: center;
}

The toolbar renders with "Angular 2" on the left side and the menu icon on the right.
The card renders normally.
What is the current behavior?
The md-card renders with the applied styles. However the md-toolbar-row does not even though the style is in the generated

source: gharchive/issue

authors: DevVersion JDillon522 codeepic

license: MIT

license_type: permissive

license_source: github-api


text:
How to Use SideNav as an Actual Router Linked Page based on Click

Bug, feature request, or proposal:
This is more of a question
What is the expected behavior?
When using a side nave... I want to toggle the sidenav open and have information that would otherwise be a link from a router i.e. //some-page/employee-id1/department2
to go to the corresponding linked information... in this case a prepopulated form page
Are there methods or properties for this to access?
What is the current behavior?
N/A
What are the steps to reproduce?
N/A
Providing a Plunker (or similar) is the best way to get the team to see your issue.
Plunker template: https://goo.gl/DlHd6U
What is the use-case or motivation for changing an existing behavior?
Using a modal popup isn't always the best user experience.  A sidenav that is half the page works nicely on getting information infront of the end user and allow that to slide back off of the screen when they are finished
Which versions of Angular, Material, OS, browsers are affected?
2.4.8+, 2.0.0-beta-2, Mac sierre
Is there anything else we should know?
Nope

put router-outlet in it?

@fxck why would that work?  the url isn't or hans't been changed

@xtianus79, since this is question rather than a bug report / feature request, it really belongs on stack overflow, gitter, or similar. If you post a question on SO and link to it from here, I can try to help you there.
FWIW, if I understand your question, maybe ActivatedRoute can help you achieve what you're after.

@willshowell thank you.  I am testing a few things and if I come to the point of needing it I will post this on SO... I looked at ActivateRoute and that might be useful/what I need. I was thinking perhaps I don't need this routed at all... (rework you know)  I will do a little investigation first.  thanks

@willshowell I accomplished what i wanted by simply not using the route for anything.  This is what happens in recode life I guess.  I found that the end result was better suited as a modal or slider.  I know in router's you can still use a route for a popup type router link that would extrapolate url info but in this case I just didn't think it was necessary...
I used ngSwitch case to use different components depending on what action on the parent page I wanted to pursue.  Worked out well.

authors: fxck willshowell xtianus79


text:
Mat-chip selected input is broken

Bug, feature request, or proposal:
Bug
What is the expected behavior?
When an md-chip/mat-chip is selected, class mat-chip-selected should be applied
What is the current behavior?
selected state has no effect
What are the steps to reproduce?
See here
Which versions of Angular, Material, OS, TypeScript, browsers are affected?
Broken in  2.0.0-beta.11 (working in previous Beta)

Have you checked #7261?

Closing as duplicate of #7261

authors: donroyco josephperrott rubenlagus


text:
docs(getting-started): fix link to theming

Since this file was moved from root to guides/,
this link should be updated too.

@apuyou There is already a PR for this (https://github.com/angular/material2/pull/2400).

Thank you @dahaupt, I'm closing this one.

source: gharchive/pull-request

authors: apuyou dahaupt


text:
Please expose the prototype's of ElementFinder and ElementArrayFinder

I would love to see the prototype's of ElementFinder and ElementArrayFinder exposed, so it could be extended dynamically from within a pageObject or a test.
Just a suggestion!

+1, this would be super useful for adding methods to elements such as the above.

@cjroth It's actually there. See https://github.com/angular/protractor/issues/1584

Oh. Awesome. Thanks @hankduan.

authors: Joel-Kornbluh cjroth hankduan

repo: angular/protractor


text:
fix(element chaining): make element chaining work when the control fl…

…ow is disabled
Also added some tests to spec/ts/noCF/smoke_spec.ts double checking that the control flow is off
@juliemr I know we talked offline and said it'd be best not to mess with getWebElement and have  chaining only work on value.  But given functions like isElementPresent and equals and count, it seems like there are going to be a lot of edge cases on that route and no clear dividing line on which functions should work and which shouldn't.  This implementation is relatively clean, and should be very rigorously tested by our suite.  So let me know.

This looks good to me and the tests are happy, so I'm good. Can you remind me which test was flaky without this? Is there a way we can force that test to always fail w/o this change, to improve testing?

@juliemr The old flaky test was "should chain element actions".  The new test "should run chained element actions in sequence" should always fail without this change.

(you may have missed the new test because clang apparently reformatted the whole file)

k, LGTM

authors: juliemr sjelin


text:
fix(sauce): bring back sauceProxy as a configuration option

Reverts the change to PR #3868. Protractor should connect to Sauce Labs
via the sauceProxy. This is different than the webDriverProxy. The
webDriverProxy is tied to the browser session where we define where the
traffic will go through.
closes #4405

@kenisteward could you review my comments and PR?

The change looks good. In the comments I'd specify the proxy is used
specifically with the saucelabs library to make rest API calls for saucelabs
On Tue, Aug 1, 2017, 5:03 PM Craig notifications@github.com wrote:

@kenisteward https://github.com/kenisteward could you review my
comments and PR?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/angular/protractor/pull/4419#issuecomment-319510167,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AMPLtQK_rIzPTSOiaTgaty7RVttgVhohks5sT6CUgaJpZM4OqWkh
.

Tha KS for this revision
On Wed, Aug 2, 2017, 5:55 PM Kenneth Steward keni.steward@gmail.com wrote:

The change looks good. In the comments I'd specify the proxy is used
specifically with the saucelabs library to make rest API calls for saucelabs
On Tue, Aug 1, 2017, 5:03 PM Craig notifications@github.com wrote:

@kenisteward https://github.com/kenisteward could you review my
comments and PR?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/angular/protractor/pull/4419#issuecomment-319510167,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AMPLtQK_rIzPTSOiaTgaty7RVttgVhohks5sT6CUgaJpZM4OqWkh
.

authors: cnishina kenisteward


text:
Cannot read property 'length' of null in _updateTaskCount

When trying to add Bing maps to an Angular 2 and 4 app, the map freezes the second time you try to zoom.
I'm submitting a ...  (check one with "x")
[x] bug report
[ ] feature request
[ ] support request

Zone.js version: 0.7.7 and 0.7.6
Current behavior
Throws error and freezes the second time the map is zoomed (either by clicking the zoom button or scrolling).
Expected behavior
The map should continue to zoom.
Minimal reproduction of the problem with instructions
The issue can be seen at https://plnkr.co/edit/xF2rtWHPWyf4KcRdihzs?p=preview. After the map loads, click the zoom button (in or out) twice. The first time it zooms properly, the second time it throws an error and freezes. If using the scroll wheel, zoom once and allow the map to stop. Then scroll again to see the error.
The error is thrown in _updateTaskCount. The zoneDelegates variable is null when it's length is checked. I know a check could be wrapped around that for loop, but I'm not sure what the implications of that would be.

It looks like this issue is being handled. Looks like it was mentioned here: https://github.com/angular/zone.js/issues/638#issuecomment-280926207
And the pull request is pending: https://github.com/JiaLiPassion/zone.js/commit/cfb5c696c88af5ca1ef8587faf6fe95ebdcdf32c

license: mit

license_source: bigquery


text:
支持边框和credit

能不能像之前一样支持添加白边框
以及能不能加个作者的字段(
以及实景图缩小后用白色(可选) 填充空缺，也就是设置白色为背景颜色
用起来挺不错就是操作的时候会有一些卡顿

好快的加

白边框：已加✅，但是文字定位上不太完美，待后期更新
背景色：已加✅，之前其实考虑过这个问题，但是感觉很少会出现实景图比画框小的情况，通常都是放大调整视角，出于性能考虑就没做。但是刚才试了一下好像影响不大（都很卡orz），就补上了
作者字段：您是指原图巡礼信息的提供者，还是拍摄/对比图制作者（用户）的credit？是否有排版方案可以分享，因为目前底下四角已经放满了，暂时没有特别好的想法
性能：canvas处理图片好像难以避免地卡，未来重构

作者字段：您是指原图巡礼信息的提供者，还是拍摄/对比图制作者（用户）的credit？是否有排版方案可以分享，因为目前底下四角已经放满了，暂时没有特别好的想法

奥我指的就是用户的credit, 不过放哪里确实是个好问题。
可能可以加成三行? 最后一行放名字 + 一段话 (作为可选), 不填就默认两行这种

白边框：已加✅，但是文字定位上不太完美，待后期更新 背景色：已加✅，之前其实考虑过这个问题，但是感觉很少会出现实景图比画框小的情况，通常都是放大调整视角，出于性能考虑就没做。但是刚才试了一下好像影响不大（都很卡orz），就补上了

奥对, 移动实景照片的y轴往下可以遮住边框(( 这个是预期的吗

不是预期的。如果实景图向上移动遮住中间边框是之前的bug，请刷新网页缓存再试一下？向下移动应该不会遮住，

还遇到了一个问题: 从有带作品截图的巡礼地点切换到没有作品截图的巡礼地点的时候上传按钮还是禁用的

代码逻辑上的问题，目前建议是直接刷新）后期会作调整，比如改成刷新键

authors: Nambers ihkk

repo: anitabi/IMHere


text:
[Idea] Show loading after updateData(newData)

Hi,
I use a Chart with an own endpoint, and initially it displays "Loading..." until the Chart has been rendered. On my page, I sometimes switch to another endpoint by using updateData(newEndpoint).
Now my question/idea:

How to display "Loading..." during this second rendering? (Backend chart data generation may take a while). This would be a nice feature because now the chart seems stuck until the data has been received.
Is there a way to translate the "Loading..." string?

Hey @diesl, thanks for the suggestions! Added these to the plan for Chartkick 4. For now, instead of calling updateData, you can destroy the chart, show the loading message, and recreate it to accomplish this.

Hey @diesl, just pushed Chartkick 4 with a loading option that should work for both those cases. Thanks again for the suggestion!

Thanks for the notice! I will take a look once we will update to the new version.

authors: ankane diesl


text:
Learning Github

After receiving my request, just ignore or delete it. I am just learning how to use the fork method through your project.

Love to have more PR.. don't worry.

authors: ankitskvmdam subhashdixit

repo: ankitskvmdam/ecommerce-react


text:
Скласти блок контактної інформації

Contact me
email: annzita19@gmail.com

you can contact me

repo: annbazykina/homepage


text:
feat(component): collapse and collapse-item component

close #54

太强了🤩

#58 是不是不需要了？

#58是不是不需要了？

需要噢，一般 commit 太多，会分多个 merge 来合并，分离不相同的模块，便于审查

#58是不是不需要了？

需要噢，一般 commit 太多，会分多个 merge 来合并，分离不相同的模块，便于审查

好滴🥳，现在的unocss-applet属性模式有些问题，尽量不要用了🤣

authors: hairyf zguolee

repo: ano-ui/ano-ui


text:
Major DTO implementation

Majority of the DTO have been implemented in this pull request, along side addressing:

#5
#8
#14
#10
#13
#18

Turning this PR into "ready for submission", this has been going for a while so I wan to merge these changes into the main branch so I can close off some of the operational tickets like #30 and #16
Some of those will be done on the main branch itself, post which I recommend protecting the main branch and then moving back into the feature branches with PRs.

repo: anomaly/gallagher


text:
demo

DEMO isuue

DONE


text:
feat(init) add vault init ability

Add the ability to initialize Vault (cluster) when needed, it's a preview as I'll implement the output of unseal keys and the root token properly.

Maybe this inspires? https://github.com/dockpack/keybase_unseal

Thanks for the inpu @bbaassssiiee . I'll add support for keybase and eventually GPG key for initialization.
To be honest I don't see how it will be more secured in another role, or copied from the table output manually.
The person deploying Vault with ansible-vault will (and want) initialize it, usually just copying the table output for unseal keys and root token. It would be a minimum to have that feature in ansible-vault IMO.

Not everything needs to go in one role. For comparison https://github.com/dockpack/vault_dojo uses this role and an couple of others to initialize vault create policies. Most of these thing depend on the organization using Vault, hence my objection to implement initialization in this role. This role is already quite complex.
Shamir secret sharing is one of the foundations of Vault. If you initialize and get unencrypted unseal keys, then that violates that principle.

I know composition and it's a good engineering principle.
However the promise of ansible-vault is to deploy a Vault machine,cluster. Thus it has Raft, Consul... backends support and also auto-unsealing.
There is a dependency error while you don't even support initialization, which is a necessary step for these. It's also an engineering principle violation, but I won't fight for it, as you said the role is quite complex and needs a refactorization.
Thanks for your time,
Regards

authors: bbaassssiiee eagleusb

repo: ansible-community/ansible-vault


text:
Remove message acknowledgement to fix issues

Messages were queuing behind an unacknowledged message. This attempts to remove the use of acknowledgement.

Fixed in #5, thank you!


text:
Unable to build Dockerfile

Issue Type

Bug report

Unable to build Molecule Dockerfile
Desired Behavior
Build the Docker Image, install the python packages list with pip

ENV PACKAGES="
gcc
make
bash
shadow
libffi-dev
musl-dev
openssl-dev
py-pip
python
python-dev
linux-headers
sudo
rsync
curl
ruby
ruby-dev
ruby-etc
ruby-rdoc
ruby-irb
git
"

Actual Behaviour
Fails in pip install stage pip install --no-cache-dir ${PIP_PACKAGES}

Exception:
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/pip/_internal/cli/base_command.py", line 176, in main
status = self.run(options, args)
File "/usr/lib/python2.7/site-packages/pip/_internal/commands/install.py", line 346, in run
session=session, autobuilding=True
File "/usr/lib/python2.7/site-packages/pip/_internal/wheel.py", line 886, in build
assert have_directory_for_build
AssertionError
The command '/bin/sh -c apk update     && apk add --update --no-cache ${PACKAGES}     && rm -rf /var/cache/apk/*     && pip install --upgrade pip     && pip install --no-cache-dir ${PIP_PACKAGES}     && rm -rf /root/.cache     && gem install ${GEM_PACKAGES}     && adduser -D -h ${TEST_HOME_DIR} ${TEST_USER}     && usermod -aG dockremap ${TEST_USER}     && echo "${TEST_USER} ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers' returned a non-zero code: 2

Potentially related to: https://github.com/pypa/pip/issues/6197

@glcx, any ideas on a fix? Thanks for the solid detective work.

@lwm One of the fix would be to use  && pip install --no-cache-dir **--no-use-pep517** -r ${PIP_PACKAGES} \
A new version of pip will be released with the fix. (Anytime soon) so it's probably not needed unless we have a new molecule release coming.

So, right, we have a fix incoming via https://github.com/pypa/pip/issues/6197#issuecomment-459947065.

Did you try to build from the master branch ?
In that branch the Dockerfile is referencing ENV PIP_PACKAGES="molecule[azure,docker,docs,ec2,gce,lxc,openstack,vagrant,windows]     virtualenv "
But the new pip package has not been released yet, and it is necessary because it will include all the extras ([azure,docker,docs,ec2,gce,lxc,openstack,vagrant,windows]) that are not present in the 2.19 pip package.

Actually, you should have these errors in the logs, previous to the python failure:
  molecule 2.19.0 does not provide the extra 'azure'
  molecule 2.19.0 does not provide the extra 'docker'
  molecule 2.19.0 does not provide the extra 'docs'
  molecule 2.19.0 does not provide the extra 'ec2'
  molecule 2.19.0 does not provide the extra 'gce'
  molecule 2.19.0 does not provide the extra 'lxc'
  molecule 2.19.0 does not provide the extra 'openstack'
  molecule 2.19.0 does not provide the extra 'vagrant'
  molecule 2.19.0 does not provide the extra 'windows'

True this is caused by the fact that documentation is on master and not on last release. Things like calling as python module and extras are not working with last release but they work with master. Another reason to release more often.

OK, great, thanks for clearing up. Let's keep this open to track until we make the next release.

19.0.2 released. Original issue is now fixed.

Thank you @glcx!

Original issue is fixed, so closing.
If there are other problems please raise individual issues for those.

authors: glcx gundalow lwm ssbarnea themr0c

repo: ansible/molecule


text:
Implement provisioning callback

Connect #249.

Coverage decreased (-0.2%) to 90.581% when pulling e9cab5a3bbb40a80c27118f3758a3c0a8c3200d4 on jangsutsr:issue249_implement_provisioning_callback into fb6540b605fab703307c11f4544ba6c3e4ad5e1c on ansible:master.

Coverage decreased (-0.02%) to 89.526% when pulling a3cd4219e359e2aa0096ce9618278fa42e278134 on jangsutsr:issue249_implement_provisioning_callback into a40d0bed35b963506c18cd7b457715dfce92cf8d on ansible:master.

authors: coveralls jangsutsr

repo: ansible/tower-cli


text:
Fix add-license-headers to keep edits from previously run hooks

Closes #82

Feel free to merge @klmcadams

authors: RobPasMue klmcadams

repo: ansys/pre-commit-hooks


text:
ScrollAnim 的 OverPack和Link标签冲突

问题代码
链接部分代码

<Link to={"content_2_0"} toHash={false}>关于1
<OverPack
replay
playScale={[0.3, 0.1]}
{...props}
>
<Link to={"content_2_0"} toHash={false}>关于2
</OverPack>

链接目标代码：
  <Content1 id="content_2_0" key="content_2_0" isMobile={this.state.isMobile}/>,

问题：
点击【关于1】 动效正常，能完整显示 Content1
点击【关于2】只到了 Content1 的顶部，
去掉 playScale={[0.3, 0.1]} 点击 【关于2】到  Content1  中间

能提供完整代码吗？这样看不出什么问题的。。。

是不是这个问题？？？https://github.com/react-component/scroll-anim/issues/38

您好，和您刚才发的网址的不太一样，完整代码我怎么发给您

直接贴上来啊

请看文件Content0.jsx，关于1是正常的，关于2点击显示不正常，请帮忙指导下，辛苦！
import React, {PropTypes} from 'react';
import {Button, Icon,Modal} from 'antd';
import QueueAnim from 'rc-queue-anim';
import TweenOne from 'rc-tween-one';
// import OverPack from 'rc-scroll-anim/lib/ScrollOverPack';
import logoCenter from '../../assets/img/logo48.png'
import ScrollAnim from 'rc-scroll-anim';
const Link = ScrollAnim.Link;
const OverPack = ScrollAnim.OverPack;
class Content extends React.Component {
/*  state = {visible:false}
showModal = ()=> {
this.setSate=({
visible:true
})
}*/
render() {
const props = {...this.props};
delete props.isMobile;
return (

<Link
to={"content_2_0"}
toHash={false}
>
关于1

  <OverPack
    replay
    playScale={[0.3, 0.1]}
   {...props}
  >
    <QueueAnim
      type={['bottom', 'top']}
      delay={200}
      className={`${props.className}-wrapper`}
      key="text"
      id={`${props.id}-wrapper`}
    >
      <span
        className="title"
        key="title"
        id={`${props.id}-title`}
      >

      <p
        key="content"
        id={`${props.id}-content`}
      >
        动岚未来，将一切化繁为简，打造智能的健身健身服务系统

        <Link style={{"color":"#fff"}}
          to={"content_2_0"}
          toHash={false}
        >
          关于2
        </Link>

      <Button type="ghost" key="button" id={`${props.id}-button`} onClick={()=>{
      // props.webRouter.push("/#content_2_0");
      }}  >
      在线咨询
      </Button>
      {/*<Button type="ghost" key="button" id={`${props.id}-button`} onClick={()=>{*/}
      {/*// props.webRouter.push("/#content_2_0");*/}
      {/*}}  >*/}
      {/*关于我们*/}
      {/*</Button>*/}
    </QueueAnim>

    <TweenOne
      animation={{y: '-=20', yoyo: true, repeat: -1, duration: 1000}}
      className={`${props.className}-icon`}
      key="icon"
    >
      <Icon type="down"/>
    </TweenOne>
  </OverPack>

);

}
}
Content.propTypes = {
className: PropTypes.string,
};
Content.defaultProps = {
className: 'banner0',
};
export default Content;

您好，我已经上传git上
git网址：https://github.com/503710847/ant-motion-dva-cli-example-2.0
位置：https://github.com/503710847/ant-motion-dva-cli-example-2.0/blob/master/src/routes/Home/Content0.jsx

              <Link
                style={{ color: '#fff' }}
                to="content_2_0"
                toHash={false}

              >
                关于2
            </Link>

加 key, 可能是 OverPack 里将不带 key 的元素直接移除了，，所以导致动画也被直接移除，，加下 key 会在出场结束后再去移除，出场的动画时间跟滚动的时间相同就没有问题。。。

还有 Link 是有个 component 的,,可以这么写
<Link
  style={{ color: '#fff' }}
  to="content_2_0"
  toHash={false}
  component="h1"
  key="h1"
>
  关于2
</Link>

模板这块用 landing 吧，，那是最新的规范。。。。

嗯呐 感谢

repo: ant-design/ant-motion


text:
Added French language support

Nothing crazy, just French locale added 🙏  🇫🇷

Codecov Report

Merging #630 into master will decrease coverage by 0.01%.
The diff coverage is 50.00%.

@@            Coverage Diff             @@
##           master     #630      +/-   ##
==========================================
- Coverage   17.43%   17.42%   -0.02%
==========================================
  Files          21       21
  Lines        1147     1148       +1
  Branches      374      375       +1
==========================================
  Hits          200      200
- Misses        947      948       +1

Impacted Files
Coverage Δ

src/Table.tsx
20.47% <ø> (-0.28%)
:arrow_down:

src/component/toolBar/index.tsx
6.77% <0.00%> (-0.12%)
:arrow_down:

src/component/intlContext/index.tsx
60.71% <100.00%> (+1.45%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 11c3709...6cb997b. Read the comment docs.

authors: codecov-commenter sylvainpe

repo: ant-design/pro-table


text:
Add protected strategy.

Hello Antek,
I had to send a new PR because I force pushed. For my API, every end point is protected. I wanted to keep my code dry so I created a new strategy that by default protects every end point. I've updated the documentation to include an example. Let me know if there's any interest in this PR. If it's unneeded, feel free to leave it unmerged.

Hey @whatasunnyday, i like it! I used to have a request like this before. Will merge it ;) thanks!

authors: antek-drzewiecki whatasunnyday


text:
如何解决 eslint 报错问题？

如何让 eslint 不产生校验错误

#69

repo: antfu/unplugin-auto-import


text:
Updating messages results in jittery scroll

When scrolling to top and fetching more messages, the scroll will be jittery. This only happens when there's a system message, such as the date separating the messages, and as long as that date is present the problem remains.
I manually removed the system message DOM and the problem vanished.

The stopgap I use for now:
const scroller = document.getElementsByClassName('vac-container-scroll')[0];

self.messages = [...fetched, ...self.messages]

for (const element of document.getElementsByClassName('vac-card-info')) {
    scroller.scrollTop -= (element.offsetHeight * fetched.length)
}

It's acceptable, but not as smooth.

The stopgap I use for now:
const scroller = document.getElementsByClassName('vac-container-scroll')[0];

self.messages = [...fetched, ...self.messages]

for (const element of document.getElementsByClassName('vac-card-info')) {
    scroller.scrollTop -= (element.offsetHeight * fetched.length)
}

It's acceptable, but not as smooth.

I cannot reproduce. On which browser are you testing?
Could you share a minimal component that reproduces the issue?

I cannot reproduce. On which browser are you testing?
Could you share a minimal component that reproduces the issue?

https://user-images.githubusercontent.com/7769125/103484056-154e3680-4dec-11eb-99e8-3a585d2ca9b6.mp4
On Chrome

https://user-images.githubusercontent.com/7769125/103484056-154e3680-4dec-11eb-99e8-3a585d2ca9b6.mp4
On Chrome

Can see as soon as I remove the system message the scrolling goes back to normal, and also the code snippet above does make it smoother.

Can see as soon as I remove the system message the scrolling goes back to normal, and also the code snippet above does make it smoother.

Should be fixed in https://github.com/antoine92190/vue-advanced-chat/commit/882352979e2c08c237ebc9a7e7ea4f5b5ba860c4
Please confirm.

Should be fixed in https://github.com/antoine92190/vue-advanced-chat/commit/882352979e2c08c237ebc9a7e7ea4f5b5ba860c4
Please confirm.

It's resolved 👍🏻

It's resolved 👍🏻


text:
Add changes made by terraform_fmt

Hi
It would be cool if terraform_fmt would add the changes done to the file to the commit. Right know, if you don't format the files, the process is:

git add
git commit
the hook modifies the unformatted file, formatting it
git add
git commit

It may be:

git add
git commit
the hook modifies the unformatted file, formatting it, and adds it to the commit

I remember this happening automatically with this hook a couple of months ago, but I can't reproduce it, so maybe I'm just wrong.
PD: Thanks for your work, it's awesome.

https://github.com/pre-commit/pre-commit/issues/532

¡Hola Àlex!
As @lewandom said... Less magic = fewer surprises when something is committed unintentionally :)
PS: You are welcome!

I see. Thanks!

authors: alexppg antonbabenko lewandom

repo: antonbabenko/pre-commit-terraform


text:
Date ranges crashing browser

Not sure exactly what the cause is, but it has something to do with the date range.
Reproduction:

https://jsfiddle.net/wp39kL7u/3/

Example:
This works
start: '2019-11-03 10:00',
end: '2019-11-04 14:00'

This crashes the browser
start: '2019-11-02 10:00',
end: '2019-11-04 14:00'

It seems to be if it crosses 12-1am. and it crosses the end of a week/month

@TheJaredWilcurt  I had the same issue, In my case I think it was about the time change in germany. After splitting my event on 26.10 24:00/27.10 00:00 the issue was gone.
In your example it crashes on 27.10 too (at least when I tried), so maybe that could be helpful.

Hi @TheJaredWilcurt,
Mmm that's weird. And it does not crash for me - or even slow down - when I try both end dates on your fiddle and go to month view 🤔
But it's been reported before that browser was hanging like in your case: #194.
In that case the event was ending at 00:00 which you should not as per this issue #139.
I'm keen to find the cause but for now I can't reproduce, so I will leave this open and let's continue tracking this down.

In Firefox it freezes the entire tab. In Chrome it freezes the iframe (in JSFiddle).
I'm in USA Eastern time. Though my team in USA Central time also can reproduce it. Doesn't seem to be OS dependent (happens on Win/Lin/OSX), but perhaps it is the US locale of the system defaulting things in vue-cal differently.

I can also confirm that it happened at the local DST change date. In North America DST changed last weekend...

Cause found
Yep! That's what's causing it to crash. There is an hour of the day that it crashes during.
On March 10, 2019 we skip an hour (2:00 AM - 2:59 AM does not happen, it goes straight from 1:59 AM, then one minute later it is 3:00 AM).
On November 3, 2019 we repeat an hour (2AM-3AM happens, but then we go back to 2AM and repeat the same hour).
This is incredibly stupid and should be banned, but "we live in a society". So.

Impact
Interestingly, vue-cal has no problem skipping the hour (Events crossing March 10th do not crash).
Problem solved! Just handle a special case for this exact hour of dates after 2019-11-02 23:59 and before 2019-11-03 01:00.
Except... no. Because DLS happens on "The first Sunday of November" 🙄 .... but that's only in USA. And only in some cases. 💀
So you really just need to pull in a library like Moment or something. to detect if a date range passes over the repeated hour for the current user in whatever their timezone/locale is, and if so, then fix the bug for that event's date range.

Fix how? Change the end time of the event? In my case it was an event that spanned the DST night, not finishing or ending at the 'extra' hour.

I meant fix internally in vue-cal. It's the code that is crashing in this scenario, so it should be detecting the scenario and preventing the crashing.

I hear you guys - does not seems like an easy fix.
Thanks for your explanation @TheJaredWilcurt and @kstankov, I could reproduce the browser hang in Australia/Melbourne timezone only happening on DLS change, only when turning to winter, and only if it is a multiple-day event.
That's a good step already.
Detecting a timezone change is fairly easy with pure JS, but fixing the code is another story.
I will investigate what causes that in the code and let you know.

@TheJaredWilcurt, @kstankov,
Good news here, I found the problem and fixed it, I need to couple more tests and cleanup. Will release a patch soon.

Hi @TheJaredWilcurt, @kstankov,
I have released the patch in version 2.13.2, please try to add your breaking event in this codepen https://codepen.io/antoniandre/pen/mddLEeO?editors=1010 and let me know if this is fixed in your timezone too.
(Make sure there is no cache in the codepen hitting the right version of vue-cal@2.13.2)
Thanks!

Thanks a lot, and thanks for this great component!

Yep that seems to have fixed it. Great work!

authors: Spanching TheJaredWilcurt antoniandre kstankov

repo: antoniandre/vue-cal


text:
Streamlit uploading

The app cannot upload

Everything looks cool now

repo: antonie-riziki/air-quality


text:
Antrea supports setting network interface for inter-host communication

Describe the problem/challenge you have
// GetNodeAddr gets the available IP address of a Node. GetNodeAddr will first try to get the NodeInternalIP, then try
// to get the NodeExternalIP.
// Note: Although K8s supports dual-stack, there is only a single Internal address per Node because of issue (
// kubernetes/kubernetes#91940 ). The Node might have multiple addresses after the issue is fixed, and one per address
// family. And we should change the return type at that time.
func GetNodeAddr(node *v1.Node) (net.IP, error) {
	addresses := make(map[v1.NodeAddressType]string)
	for _, addr := range node.Status.Addresses {
		addresses[addr.Type] = addr.Address
	}
	var ipAddrStr string
	if internalIP, ok := addresses[v1.NodeInternalIP]; ok {
		ipAddrStr = internalIP
	} else if externalIP, ok := addresses[v1.NodeExternalIP]; ok {
		ipAddrStr = externalIP
	} else {
		return nil, fmt.Errorf("Node %s has neither external ip nor internal ip", node.Name)
	}
	ipAddr := net.ParseIP(ipAddrStr)
	if ipAddr == nil {
		return nil, fmt.Errorf("<%v> is not a valid ip address", ipAddrStr)
	}
	return ipAddr, nil
}

Antrea gets NodeInternalIP or NodeExternalIP for  inter-host communication, sometimes we need set special network interface for inter-host communication.
Describe the solution you'd like
Maybe need  antrea-agent --config /etc/antrea/antrea-agent.conf --iface eno1 or  antrea-agent --config /etc/antrea/antrea-agent.conf --iface-regex 172.16.100.0/24
Use iface or iface-regex, which config param is not empty, when setting up node  inter-host communication ip addr, otherwise use NodeInternalIP or NodeExternalIP.

@Jexf thanks for the suggestion. This is definitely a valid scenario Antrea should support. We received similar feature suggestion in https://github.com/antrea-io/antrea/issues/2344, though it's for windows.
@wenyingd has https://github.com/antrea-io/antrea/pull/2370 (not merged yet) which should covers the scenario. It adds an option "transportInterface" to agent configuration file. Would you check if it will work for you?

Thanks for reply @tnqn, the patch https://github.com/antrea-io/antrea/pull/2370 adds an option "transportInterface" to config interface name, maybe we need add option "transportCIDRRange"  for different interface name on each node.

Good idea. @jianjuns @antoninbas what's your opinion?
@Jexf would you like to create a PR for it after #2370 is merged?

I did not get the idea. transportCIDRRange is for supporting different interface names on different Nodes? How it helps?

I did not get the idea. transportCIDRRange is for supporting different interface names on different Nodes? How it helps?

I think it's to identify the interface based on its subnet and not based on its name which can vary across Nodes.
I understand the problem we are trying to solve but this doesn't seem like a very elegant solution, especially when not all Nodes are on the same L2 network. Can you think of any other project using a solution like this one? Any alternatives we could consider?

A k8s-antrea has three nodes:
[root@tos-04 ~]# kubectl  get nodes NAME     STATUS   ROLES                  AGE   VERSION tos-04   Ready    control-plane,master   15d   v1.21.2 tos-05   Ready    <none>                 15d   v1.21.2 tos-06   Ready    <none>                 15d   v1.21.2
The interface for inter-host communicatio on tos-04 is:
[root@tos-04 ~]# ip a s eth1
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 52:54:00:bd:db:31 brd ff:ff:ff:ff:ff:ff
    inet 172.18.123.15/22 brd 172.18.123.255 scope global noprefixroute eth1
       valid_lft forever preferred_lft forever
    inet6 fe80::5054:ff:febd:db31/64 scope link
       valid_lft forever preferred_lft forever

The interface for inter-host communicatio on tos-05 is:
[root@tos05 ~]# ip a s eno1
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 3c:ec:ef:4e:15:06 brd ff:ff:ff:ff:ff:ff
    inet 172.18.123.16/22 brd 172.18.123.255 scope global eno1
       valid_lft forever preferred_lft forever
    inet6 fe80::3eec:efff:fe4e:1506/64 scope link
       valid_lft forever preferred_lft forever

We can update transportInterface to transportInterfaces, and set the transportInterfaces to eth1,eno1`, then antrea match the first interface from transportInterfaces list. It will work well with diferrent interface name for inter-host communication. @tnqn @jianjuns
The subnet option for inter-host communication only work on L2 network. Maybe we can add an option with []subnet, then antrea will match the first interface from subnets list. @antoninbas

Interface or subnet list do not sound ideal either, as two Nodes can have the same set of interfaces or subnets, but want to choose different ones.
If we step back, do you think you can create two Antrea DaemonSets (each should select different Nodes) for the two Nodes in your example, with different transportInterface configuration? It is not very convenient but should work?

authors: Jexf antoninbas jianjuns tnqn

repo: antrea-io/antrea


text:
Add --random-fully to SNAT IP Table Rules

Describe the problem/challenge you have
Default SNAT behavior is to preserve the source whenever possible. We are seeing a precentage of traffic using SNATs traffic fail due to  TCP Port Numbers being reused too quickly. As the referenced in a-reason-for-unexplained-connection-timeouts-on-kubernetes-docker below where there is latency in the insert into the conntrack table which can cause the same Source Port to be given to multiple pods.
Thus we are seeing in the TCP Dump output, there are multiple [TCP Port Numbers reused] on POD.
Describe the solution you'd like
Expose an option to randomize the Source Pods Ports to use --random / --random-fully in iptables
Anything else you would like to add?
See the Netfilter NAT & Conntrack kernel modules section on the following link.
Note the following link is Kubernetes / Docker, but looks like it would hold true with Kubernetes/Containerd level
a-reason-for-unexplained-connection-timeouts-on-kubernetes-docker
time-wait article
Cilium SNAT issue

I personally support adding this as a configuration parameter for the antrea-agent, as I don't think there is any drawback (we would keep the current behavior as the default) and it could potentially help in some situations.
@Scoobed for your specific issue, there may be better ways to address it on the client side (we discussed this on Slack).
Regarding the links you shared: if memory served me right, the primary concern was race conditions in iptables / netfilter which impacted UDP specifically and therefore impacted DNS queries in K8s clusters. I am not sure TCP connections were affected. See https://quentin.machu.dev/2018/06/24/5-15s-dns-lookups-on-kubernetes/. In your case (based on our Slack conversation), there is no race condition as the implementation behaves correctly, but some network device in the path is not happy about rapid port reuse.
By the way, this may be a "good first issue" if you (or someone else) are interested in taking a stab at it. The only thing to look out for AFAIK is that some older kernel versions may not support --random-fully and that a check may be desired in the antrea-agent when the configuration parameter is set.

While this is not mentioned explicitly in the original post, I believe that @Scoobed experienced the issue in the context of Egress SNAT, given that the Slack post mentioned "SNAT IP per namespace".
However, the ability for users to configure --random-fully could apply both to Egress SNAT and to Node SNAT.
Node SNAT (masquerade): https://github.com/antrea-io/antrea/blob/e921a6ec179c3b666ec484e94801291cdda3d37e/pkg/agent/route/route_linux.go#L979-L987
Egress SNAT: https://github.com/antrea-io/antrea/blob/e921a6ec179c3b666ec484e94801291cdda3d37e/pkg/agent/route/route_linux.go#L969-L978
It is not completely clear to me whether we would want 2 separate configuration parameters or a unified one. I would personally prefer the latter.

I would prefer two settings if possible as we might one but not the other one

I would prefer two settings if possible as we might want the one but not the other one

Do you mean that you would want source port selection to be random for Egress SNAT but not for Node SNAT (or the reverse)? Could you provide more information as to why?

I mean I would just like to control them  independent of each other. Like only enable with Egress and not the node. Sorry if that came off confusing.

authors: Scoobed antoninbas


text:
Add mutex to prevent resource reconciler and stale controller race

Fixes #4854

/test-multicluster-e2e

/test-multicluster-e2e

/test-multicluster-e2e

/test-multicluster-e2e

/skip-all

authors: Dyanngg tnqn


text:
Add Hairpin traffic limitation doc

This is a follow-up PR of PR #5687.

Skip allowHairpinSerivce test when Antrea-Proxy is disabled.
Add documentation including the behavior of the hairpin service when Antrea-Proxy is enabled and disabled.

/skip-all

authors: GraysonWu tnqn


text:
refactor(mark): inline shape and infer transform

Refactor Mark
为了未来满足如下的写法：
import { Runtime, Interval, corelib, extend } from '@antv/g2';

const Chart = extend(Runtime, { ...corelib(), 'mark.interval': Interval });

const chart = new Chart();

chart.interval().data(data).encode('x', 'name').encode('y', 'value');

chart.render();

将 mark 的 shape 和 infer transform（Maybe 开头的 transform） 对应的组件直接放入了 mark。
stdlib 去掉了 shape（label shape 还是保存着） 和 infer transform。

Pull Request Test Coverage Report for Build 5819464364

127 of 127 (100.0%)  changed or added relevant lines in 27 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.03%) to 89.409%

Totals

Change from base Build 5806672521:
0.03%

Covered Lines:
9603

Relevant Lines:
10401

💛  - Coveralls

authors: coveralls pearmini


text:
绘制组件bug

官方绘制组件Demo(https://l7.antv.vision/zh/examples/draw/UI#amap)出现多个bug：
1、绘制线、多边形、圆出现错乱、消失现象。
2、绘制完编辑出现错乱编辑现象。
3、无法删除绘制。

除了了这里，LineLayer 也有问题，更新包，没有提升版本，都出现了这个问题。在绘制线的时候，回出现标点缺失！
比如本该绘制一个正方形的线，绘制后就会损失一个节点，导致只有三条线。

新版本已修复

authors: Liisp yanyu510 zchuhui


text:
autoResize: true isn't setting the height and width correctly with Vue shape nodes

Describe the bug
I have created an app with x6-vue-shape, where in when you click on the node everytime it increases the height by 100px. I have noticed that at some point the graph is getting cut. When inspecting, it looks like the height is small, and it isn't accommodating the entire svg.
Your Example Website or App
NA
Steps to Reproduce the Bug or Issue
1, Create an X6 graph with full x6-vue-shape nodes
2. Add an event to the node where for every click on it, it increases the height of the node by 100px
3, At one point of time, the graph will get cut.
Expected behavior
Ideally the graph should compute the height and width perfectly after the rending is fully complete. I suppose it is calculating the height before rendering.
Screenshots or Videos
No response
Platform

OS: [e.g. macOS, Windows, Linux]
Browser: [e.g. Chrome, Safari, Firefox]
Version: [e.g. 91.1]

Additional context
No response

What version of x6 are you using and do you have a reproducible demo.

I tried using both v1.x and also tried v2.x. In both the places there was no resolutions.
I don't have a demoable version.
Basically it about a few nodes on the left and right (dagree layout) built from x6-vue-shape. On click of the node, it expands it and append 5 entries as its port. (this in turn increases the size of the node). When there are multiple nodes in the same level, we do add a manual function to adjust the position( y coordinate) of the below node, based on the height of the previous one. After all these computations, the autoResize does it's function. And it gets cut off

I have tried manually increasing the height inorder to check if it is a height issue. Tried increasing it and it looks fine

autoResize is not specific to the node, it is specific to the canvas, and your scenario requires recalling the node.resize method every time you change the node size.

authors: NewByVector harshaktg


text:
g6中如何更新子节点html节点上的文本内容

如题，求解

update时候获取到具体的节点，拿到HTML元素更新就可以了。

authors: baizn frontHu


text:
DOC:Would u like to add Snake Animation to Readme.md file?

What's wrong with the existing documentation
Ntg , Liked to add snake animation of Contribution graph!
Add ScreenShots
https://github.com/GameSphere-MultiPlayer/Spaceship-escort/assets/154777864/dee077c5-6d1e-40cb-9cd2-824f24c59982
Record

[X] I agree to follow this project's Code of Conduct
[X] I'm a GSSOC'24 contributor
[X] I want to work on this issue

/assign


text:
Animation Database

Experience
AnimationDB.zip
Title
Animation Database
Screenshots

Description
The experience can help you manage and track anime, character, music, related news. Version 1, base and test on Japanese animation.
Categories
Collection, Hobbies
Language
en-US
Terms

[X] I agree that this Experience is licensed under the MIT License.
[X] I have intellectual property rights and/or permission to contribute the Experience and its content.
[X] I confirm that the Experience does not contain any harmful or malicious content.

/gallery approve

authors: Fradeet KirillSto

repo: anyproto/gallery


text:
Update zh-CN.json

Chinese translation of the first version

@any contributor @zxiilw63 l10n zh-CN


text:
Issue 526

It is a temp PR used to check PR #566 CI issue.

This is unused PR.

repo: anz-bank/sysl


text:
Unavailable tracks should be marked differently

Is your feature request related to a problem? Please describe.
When a track is unavailable by spotify, it is not greyed out, marked and generally does not show any sort of error when one tries to play it. As it happens, my current top track is one such track.
Describe the solution you'd like
An error should display, the song should be greyed out or otherwise 'marked'. Worst case, it should not be displayed.
Additional context

The top track, Lyfe Guitar Remix by itsnot.theis is the unavailable track in question.

Tried to take a crack at this in #207

authors: Xevion rileyallyn


text:
Add an uninstall command

This command removes the install and data directories, deletes the "brainframe" group, and removes all BrainFrame Docker images from the machine. It's effectively meant as the opposite of the "install" command. However, the command does not remove Docker, Docker Compose, rsync, or curl, even if the CLI was originally used to install them. It also does not uninstall the BrainFrame CLI itself.

This looks good. My only comment is to run the uninstall command in the test_installation CI pipeline, and do a check or two to see that it worked.

Do we want to split this operation into two varieties (a la apt remove and apt purge), only removing  the data directories if someone purges? It's possible that someone wants to uninstall BrainFrame without removing their data.

I'm open to that. So an apt purge-esque command would do what the current uninstall does now, which is:

Deletes the install directory
Deletes the data directory
Removes the "brainframe" group
Deletes the server Docker images

Which of these would an apt remove-esque command do? Maybe:

Deletes the install directory
Deletes the server Docker images

Do you think it should remove the "brainframe" group as well? The purpose of that group is to allow the user to manage BrainFrame data without needing sudo, so maybe it shouldn't.

Do you think it should remove the "brainframe" group as well? The purpose of that group is to allow the user to manage BrainFrame data without needing sudo, so maybe it shouldn't.

I would have thought it should remove the BrainFrame group if all files owned by the group are removed as well. However I found this on the Fedora wiki: https://fedoraproject.org/wiki/Packaging:UsersAndGroups#Allocation_Strategies

Okay, maybe I won't remove the group at all, then.

authors: BryceBeagle apockill velovix


text:
Add support for basic escaping of characters inside Shell commands

Is your feature request related to a problem? Please describe.
Original Jira ticket: https://issues.apache.org/jira/browse/ACCUMULO-2929
Excerpt from ticket:
For commands that have a -c option (e.g. InsertCommand, GrepCommand, ScanCommand),
 there is currently no way to specify a column family that contains a colon as it is
 interpreted as the delimiter between the column family and column qualifier 'cf[:cq]'.

It would be very simple to support a basic escape '\' character that would allow
users to use column families which contain colons in the shell.

Describe the solution you'd like
It would be a useful feature to allow escaping of special characters, specifically a ' : ' in this instance. Alternatively, extra options such as -cf and -cq could be added instead.
Additional context
The specific function inside ScanCommand (utilized by GrepCommand and others) that fetches the Columns.
https://github.com/apache/accumulo/blob/3da639ac6c49ad1b9671230e3443632d27887f75/shell/src/main/java/org/apache/accumulo/shell/commands/ScanCommand.java#L274-L275

Please assign in to me. Would love to work on this

repo: apache/accumulo


text:
NO-JIRA Specifying Maven Deploy Plugin as 2.4

Using the latest (3.2.0) here will break the executables authorization for some reason

@gemmellr the latest version of the maven deployment plugin is not reading the executor's argument on dep.xml, as a result a binary distribution would be broken.
Do you want to figure out why that happened? or can we just keep 2.4 here?

*assembly plugin, rather than deploy.
I'll look into the issue first. 2.4 is about heading for 9 years old now, we should try to stay more current so that upgrades don't need to be as wide ranging or extreme as this one was. Essentially the issue is just that I spent so long on the rest that I obviously didn't run the binary again in the end, which I should have, and have broken it along the way without noticing.

I have dropped the version back to 3.1.1 in https://github.com/apache/activemq-artemis/commit/7733a76649403193e0e0e7e1a6279095fdbf426c which got things working for me.
It seems that changes in 3.2.0 around reproducible builds changed some potentially undefined behaviour. The new behaviour exists in 3.3.0 as well. It seems perhaps some of the files in the assembly are relying on the old plugin behaviour, and so coupled with the descriptor not fully defining the permissions it wants this lead to the breakage on newer versions.
I think we should try ensuring the descriptor is updated to fully reflect what it wants and then update again, but for now the slightly-older but-not-entirely-ancient plugin version seems to be doing the trick.

authors: clebertsuconic gemmellr

repo: apache/activemq-artemis


text:
bug: rpc error: code = Internal desc = server closed the stream without sending trailers

Current Behavior
apisix: apache/apisix:2.15.0-alpine
plugin-runner: apache/apisix-go-plugin-runner v0.4.0
run apisix-go-plugin-runner ,in func RequestFilter, WriteHeader then return, grpc client will report an error(code = Internal desc = server closed the stream without sending trailers)
I studied for a day and didn't know how to solve the problem.  can anybody help solve the problem? thanks a lot!
Expected Behavior
grpc client not report error,and can get trailers
Error Logs
rpc error: code = Internal desc = server closed the stream without sending trailers
Steps to Reproduce
apisix: apache/apisix:2.15.0-alpine
plugin-runner: apache/apisix-go-plugin-runner v0.4.0

modify code in apisix-go-plugin-runner-master\cmd\go-runner\plugins\say.go, as follows:

func (p *Say) RequestFilter(conf interface{}, w http.ResponseWriter, r pkgHTTP.Request) {
	w.Header().Set("Content-Type", r.Header().Get("Content-Type"))
	num := rand.Int()
	if num%2 == 1 {
		w.WriteHeader(http.StatusOK)
		w.Header().Set("mykey", "test")
		w.Header().Set("Grpc-Status", "0")
		w.Header().Set("Grpc-Message", "ok")
		w.Header().Set("Grpc-Status-Details-Bin", "")
		return
	}
}

run a grpc server in 192.168.56.19:8301
2. then run apisix, config upstream in APISIX Dashboard，as follows:
{
  "nodes": [
    {
      "host": "192.168.56.19",
      "port": 8301,
      "weight": 1
    }
  ],
  "timeout": {
    "connect": 6,
    "send": 6,
    "read": 6
  },
  "type": "roundrobin",
  "scheme": "grpc",
  "pass_host": "pass",
  "name": "testa",
  "keepalive_pool": {
    "idle_timeout": 60,
    "requests": 1000,
    "size": 320
  }
}

then add a route config, as follows:

{
  "uri": "/routeguide.RouteGuide/GetFeature",
  "name": "testa1",
  "methods": [
    "GET",
    "POST",
    "PUT",
    "DELETE",
    "PATCH",
    "HEAD",
    "OPTIONS",
    "CONNECT",
    "TRACE"
  ],
  "plugins": {
    "ext-plugin-post-req": {
      "conf": [
        {
          "name": "say",
          "value": "{\"body\":\"123\"}"
        }
      ],
      "disable": false
    }
  },
  "upstream_id": "425744995673178819",
  "status": 1
}

then use client grpc call apisix, but have an error (rpc error: code = Internal desc = server closed the stream without sending trailers)，as follows:

2022/09/21 19:45:47.860 [I] [main.go:59]  header: map[content-type:[application/grpc] date:[Wed, 21 Sep 2022 11:45:47 GMT] mykey:[test] server:[APISIX/2.15.0]]
2022/09/21 19:45:47.860 [I] [main.go:60]  trailer: map[]
2022/09/21 19:45:47.861 [E] [main.go:62]  rpc error: code = Internal desc = server closed the stream without sending trailers

Environment
os: win10
apisix: apache/apisix:2.15.0-alpine  (use apisix-docker\example\docker-compose.yml  docker-compose up)
plugin-runner: apache/apisix-go-plugin-runner v0.4.0

If you want to use the grpc client to access APISIX, you need to enable the grpc-transcode plugin.
"scheme": "grpc" in upstream configures the protocol for accessing upstream to be grpc.

If you want to use the grpc client to access APISIX, you need to enable the grpc-transcode plugin.
"scheme": "grpc" in upstream configures the protocol for accessing upstream to be grpc.

I've read this grpc-transcode.md, but it doesn't seem to be relevant to this problem.
Can you contact me via WeChat 'program_no_error'? thanks a lot!

If you want to use the grpc client to access APISIX, you need to enable the grpc-transcode plugin.
"scheme": "grpc" in upstream configures the protocol for accessing upstream to be grpc.

my call chain like this:
1.[RequestFilter return] --?--> GrpcClient (report error)
GrpcClient --grpc--> Apisix --?--> Go-plugin-runner(ext-plugin-post-req)
2.[RequestFilter continue] --grpc--> GrpcServer --grpc--> Apisix --grpc--> GrpcClient(ok)
in Go-plugin-runner, func RequestFilter like this:
func (p *Say) RequestFilter(conf interface{}, w http.ResponseWriter, r pkgHTTP.Request) {
	w.Header().Set("Content-Type", r.Header().Get("Content-Type"))
	num := rand.Int()
	if num%2 == 1 {
		w.WriteHeader(http.StatusOK)
		w.Header().Set("mykey", "test")
		w.Header().Set("Grpc-Status", "0")
		w.Header().Set("Grpc-Message", "ok")
		w.Header().Set("Grpc-Status-Details-Bin", "")
		return   // here retrun to GrpcClient, is above chain 1, client report "server closed the stream without sending trailers"
	}
        return // here continue to GrpcServer then return to GrpcClient, is above chain 2, ok
}

above not use grpc-transcode，grpc-transcode ot suitable for our scene。Please have a look and help， thanks a lot!

Regarding this step GrpcClient --grpc--> Apisix, how did you do it? What tool or command was used ?

Regarding this step GrpcClient --grpc--> Apisix, how did you do it? What tool or command was used ?

client use grpc protocol request Apisix. code as follows:
	conn, err := grpc.Dial("192.168.0.1:9080", grpc.WithTransportCredentials(insecure.NewCredentials()))
	if nil != err {
		return
	}
	defer func() {
		conn.Close()
	}()
	client := routeguide.NewRouteGuideClient(conn)
	md := metadata.MD{}
	md.Set("header1", "test")
	ctx = metadata.NewOutgoingContext(ctx, md)
	var rspheader metadata.MD
	var rsptrailer metadata.MD
	req := routeguide.Point{Latitude: 99, Longitude: 88}
	rsp, err := cli.GetFeature(ctx , &req, grpc.Header(&rspheader), grpc.Trailer(&rsptrailer))
	if err != nil {
		logs.Error(err)      //here report error "server closed the stream without sending trailers"
		return
	}
	logs.Info("rsp:", rsp.String())
        logs.Info("rspheader:", rspheader)
        logs.Info("rsptrailer:", rsptrailer)

It seems that it is the same issue as https://github.com/apache/apisix/issues/7973.
Please do not submit the same issue in the two projects, it will be inconvenient to track, and information will be lost.

ok, sorry, i close this

authors: gitcome soulbird

repo: apache/apisix-go-plugin-runner


text:
bug: When the spec field of the ApisixUpstream resource is empty, it will panic

An error occurred when I added this resource.
version: 1.3.0
apiVersion: apisix.apache.org/v1
kind: ApisixUpstream
metadata:
  name: apisix-dashboard

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x50 pc=0x15fcef5]
goroutine 517 [running]:
github.com/apache/apisix-ingress-controller/pkg/kube/translation.(*translator).TranslateUpstream(0xc000010ce8, 0xc0001c8140, 0xe, 0xc0001c8230, 0x10, 0x0, 0x0, 0xc000000050, 0x8, 0x8, ...)
/build/controller/pkg/kube/translation/translator.go:196 +0x395
github.com/apache/apisix-ingress-controller/pkg/kube/translation.(*translator).translateUpstreamFromIngressV1(0xc000010ce8, 0xc0001c8140, 0xe, 0xc0006d08a0, 0x1, 0xc0005033b0, 0x0)
/build/controller/pkg/kube/translation/ingress.go:235 +0x94
github.com/apache/apisix-ingress-controller/pkg/kube/translation.(*translator).translateIngressV1(0xc000010ce8, 0xc000766170, 0xd, 0x15e44f0, 0x1b3d900)
/build/controller/pkg/kube/translation/ingress.go:79 +0x92f
github.com/apache/apisix-ingress-controller/pkg/kube/translation.(*translator).TranslateIngress(0xc000010ce8, 0x1fe3be0, 0xc0006d03f0, 0xc00066d36f, 0x10, 0x1fe3be0)
/build/controller/pkg/kube/translation/translator.go:255 +0x24c
github.com/apache/apisix-ingress-controller/pkg/ingress.(*ingressController).sync(0xc0003cd5e0, 0x1fdabe0, 0xc00003ca40, 0xc00081d050, 0x8, 0xc000604f80)
/build/controller/pkg/ingress/ingress.go:128 +0x2d3
github.com/apache/apisix-ingress-controller/pkg/ingress.(*ingressController).runWorker(0xc0003cd5e0, 0x1fdabe0, 0xc00003ca40)
/build/controller/pkg/ingress/ingress.go:78 +0x57
created by github.com/apache/apisix-ingress-controller/pkg/ingress.(*ingressController).run
/build/controller/pkg/ingress/ingress.go:67 +0x223

This is because we are not verifying its spec. Are you interested in helping to participate in the contribution?

ok.

#794 has been merged. Thanks @Brhetty

repo: apache/apisix-ingress-controller


text:
snowflake: driver is not returning ConstraintColumnNames, ConstraintColumnUsage

What happened?
I have a table (PRODUCT2) that has a relationship with ORDER2:
create or replace TABLE DEMODB.PUBLIC.ORDERS2 (
	ORDER_ID VARCHAR(16777216) NOT NULL,
	QUANTITY_SOLD NUMBER(11,2),
	primary key (ORDER_ID)
);

create or replace TABLE DEMODB.PUBLIC.PRODUCT2 (
	PRODUCT_ID VARCHAR(16777216) NOT NULL,
	PRODUCT_NAME VARCHAR(16777216),
	ORDER_ID VARCHAR(16777216),
	primary key (PRODUCT_ID),
	foreign key (ORDER_ID) references DEMODB.PUBLIC.ORDERS2(ORDER_ID)
);

When I run the SQL queries for GetObjects("DEMODB","PUBLIC","PRODUCT2", nil, nil), I get results from Snowflake that the constraint_column_names are present:
GET_OBJECTS
"{
  ""catalog_db_schemas"": [
    {
      ""db_schema_name"": ""PUBLIC"",
      ""db_schema_tables"": [
        {
          ""table_columns"": [
            {
              ""column_name"": ""PRODUCT_ID"",
              ""ordinal_position"": 1,
              ""xdbc_char_octet_length"": 16777216,
              ""xdbc_column_size"": 16777216,
              ""xdbc_is_nullable"": ""NO"",
              ""xdbc_nullable"": 0,
              ""xdbc_type_name"": ""TEXT""
            },
            {
              ""column_name"": ""PRODUCT_NAME"",
              ""ordinal_position"": 2,
              ""xdbc_char_octet_length"": 16777216,
              ""xdbc_column_size"": 16777216,
              ""xdbc_is_nullable"": ""YES"",
              ""xdbc_nullable"": 1,
              ""xdbc_type_name"": ""TEXT""
            },
            {
              ""column_name"": ""ORDER_ID"",
              ""ordinal_position"": 3,
              ""xdbc_char_octet_length"": 16777216,
              ""xdbc_column_size"": 16777216,
              ""xdbc_is_nullable"": ""YES"",
              ""xdbc_nullable"": 1,
              ""xdbc_type_name"": ""TEXT""
            }
          ],
          ""table_constraints"": [
            {
              ""constraint_column_names"": [
                ""PRODUCT_ID""
              ],
              ""constraint_column_usage"": [],
              ""constraint_name"": ""SYS_CONSTRAINT_386a9022-ad6e-47ed-94b6-f48501938f5f"",
              ""constraint_type"": ""PRIMARY KEY""
            },
            {
              ""constraint_column_names"": [
                ""ORDER_ID""
              ],
              ""constraint_column_usage"": [
                {
                  ""fk_catalog"": ""DEMODB"",
                  ""fk_column_name"": ""ORDER_ID"",
                  ""fk_db_schema"": ""PUBLIC"",
                  ""fk_table"": ""ORDERS2""
                }
              ],
              ""constraint_name"": ""SYS_CONSTRAINT_cfb3b763-4a97-4c0b-8356-a493c03b7e66"",
              ""constraint_type"": ""FOREIGN KEY""
            }
          ],
          ""table_name"": ""PRODUCT2"",
          ""table_type"": ""BASE TABLE""
        }
      ]
    }
  ],
  ""catalog_name"": ""DEMODB""
}"

However, when I inspect the results in the debugger:

it appears there are no ConstraintColumnNames or ConstraintColumnUsage returned:

I see there is some code in driver_base to parse these, but it appears only the FlightSQL driver calls that now. I don't see where the Snowflake driver would be parsing these values.
Stack Trace
No response
How can we reproduce the bug?
No response
Environment/Setup
No response

fyi @zeroshade

The problem appears to be the use of requiredList in the definition:
type ConstraintInfo struct {
	ConstraintName        *string                 `json:"constraint_name,omitempty"`
	ConstraintType        string                  `json:"constraint_type"`
	ConstraintColumnNames requiredList[string]    `json:"constraint_column_names"`
	ConstraintColumnUsage []ConstraintColumnUsage `json:"constraint_column_usage,omitempty"`
}

If I change it to a string array:
type ConstraintInfo struct {
	ConstraintName        *string                 `json:"constraint_name,omitempty"`
	ConstraintType        string                  `json:"constraint_type"`
	ConstraintColumnNames []string                `json:"constraint_column_names"`
	ConstraintColumnUsage []ConstraintColumnUsage `json:"constraint_column_usage,omitempty"`
}

then I get the expected results:

but I'm not sure the ramifications of this.

@davidhcoe I'll take a look into this and see what I can figure out. Looking at the code I can't see why the change between requiredList[string] vs []string would change anything in this scenario, I'll see if I can reproduce this issue myself so I can debug it.

authors: davidhcoe zeroshade

repo: apache/arrow-adbc


text:
[C] Research ConnectorX/pgeon for optimizing libpq driver

Pgeon: https://github.com/0x0L/pgeon
ConnectorX: https://sfu-db.github.io/connector-x/intro.html

ConnectorX

Can partition the query along a given column, then fetch the partitions in parallel
"Copy-exactly-once" architecture
Uses preallocated buffers where possible (also, appears to do things like implement its own conversion to Python strings)

These optimizations would probably be difficult to support, though we should preallocate where possible.
Turbodbc
ConnectorX's docs compare it to Turbodbc which tends to trail it, though Turbodbc does not appear to implement parallelization (that might explain the difference).
Turbodbc also lists some optimizations:
https://turbodbc.readthedocs.io/en/latest/pages/advanced_usage.html
In particular, it can interleave I/O and conversion. That may be interesting for us, though libpq seems to only either give you a choice between row-at-a-time or getting all query results at once.
Turbodbc also implements some memory optimizations: dictionary-encoding string fields, and dynamically determining the minimum integer width.
pgeon

Uses COPY (DuckDB appears to do this too, though note DuckDB's postgres extension is GPL) That honestly seems to be the main optimization
Queries some metadata tables up front to determine proper types

@lidavidm for pgeon I have experimented with FETCH instead of COPY. COPY was the fastest method in my [limited] testings

Ah, thanks. I noticed that, and it seems like FETCH also requires you to manage a server-side cursor which isn't great.

Async APIs are somewhere down on the list of things I would like to explore! But the 'base' API is all blocking. (I also haven't tried binding async C/C++ interfaces to Python's async APIs yet - I need to look at whether callbacks, polling, or something else is preferred/ergonomic.)
Thanks for the heads up - I'll make sure to support the broader ecosystem (I quite like trio's ideas, even if I haven't gotten a chance to use it in practice).

This benchmark (in slides) found that the libpq driver is very slow: https://www.clear-code.com/blog/2023/5/8/rubykaigi-2023-announce.html

I think that it's "after".
The benchmark is https://github.com/apache/arrow-flight-sql-postgresql/tree/main/benchmark/integer .

Ah, so the 'libpq' column is https://github.com/apache/arrow-flight-sql-postgresql/blob/main/benchmark/integer/select.c ? In that case I would expect it to be slower by definition since we're doing extra work to convert the result set to Arrow. And the Flight SQL server has an advantage since it can grab the data directly from PostgreSQL without going through the PostgreSQL wire protocol.

Yes.
(https://github.com/apache/arrow-flight-sql-postgresql/blob/main/benchmark/integer/select.sql is the same performance as select.c.)

Another optimization would be to attempt parallelizing the "read from connection" and "convert to arrow" operations.

FWIW, this is mentioned in the issue above. I think when I looked at it, it seemed like libpq would read the entire response before returning to you.

authors: 0x0L kou lidavidm


text:
Cluster state refactor Part 2

Which issue does this PR close?

Closes #554
Rationale for this change

See original ticket for full description but the tl;dr; is:

The existing interface for managing shared state among schedulers is too low-level and both pushes too much complexity into other layers and prevents us from taking advanatage of data store specific freatures (atomics, transactions, etc) to avoid locking in the application layer.
By forcing everything through the KV interface we force all the serialization overhead even if we are only using in-memory state.

What changes are included in this PR?

There are a lot of changes because the StateBackendClient was baked in everywhere but at a high level:

Create JobState trait to complement ClusterState created in last PR. This in the interface for storing/managing global state w/r/t jobs and sessions.
Refactor in-memory implementation to not serialize things to protobuf
Provide KeyValueState implementation based on existing KV interface so we can continue to use etcd and sled for state without any changes.
Take advantage of curated job architecture to minimize distributed locking. Since jobs are owned by a single scheduler we don't need to lock beyond the node level.
Cleanup configs.
Move everything related to state the ballista_scheduler::cluster module

I removed the separate configs for cluster-backend and config-backend because it seems confusing. So currently if you want to use different implementations for the ClusterState and JobState you would need to write your own entrypoint. We might introduce standard "profiles" (standalone/ha/etc) for setting up the two state backends on different implementations but for now it seems too noisy config wise.
NOTE: I was not able to run the integration tests using the standard configuration due to weird timeouts. When using push scheduling everything worked fine so not sure what's going on. I also checked on main and see the same issue so maybe something related to the M1 macbook?
Are there any user-facing changes?

The storage layout for etcd and sled are simplified so existing state would not be valid after upgrade.

@mingmwang @yahoNanJing @Dandandan

Not sure why this is failing (https://github.com/apache/arrow-ballista/actions/runs/4123347318/jobs/7121701363) but it doesn't seem to be an issue with the actual formatting

I'll plan on merging this weekend unless someone else would like more time to review.

repo: apache/arrow-ballista


text:
Add option to datafusion-cli to execute SQL from file and keep session open

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
(This section helps Arrow developers understand the context and why for this feature, in addition to  the what)
I would like to be able to write DDL statements once and save them to a file that could be read by datafusion-cli (similar to the existing --file command which closes after executing the file) this way I could reuse them in future sessions and quickly use my "database" again.
Describe the solution you'd like
A clear and concise description of what you want to happen.
Add a command line option to datafusion-cli that runs the SQL in a file and then leaves you in the sql session to continue writing queries.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.

@Jimexist FYI - as you've shown interest in datafusion-cli topics in the past.

I'm working on the ExecutionContext -> SessionContext refactoring now. We will have a clear session concept in the system.
https://github.com/apache/arrow-datafusion/issues/1862

authors: matthewmturner mingmwang

repo: apache/arrow-datafusion


text:
Create next DataFusion release (after 7.0) - 7.1

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
We released datafusion 7.0.0 about a month ago https://crates.io/crates/datafusion/7.0.0
We should figure out when to release the next one
Describe the solution you'd like
Plan out the next release(s) of DataFusion. Also figure out if we want to do a maintenance release (e.g. 7.0.1 / 7.1.0) or a release from master (8.0.0).
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Brought up by @silence-coding  here: https://github.com/apache/arrow-datafusion/pull/2066#issuecomment-1078715516
See notes on 7.0 release https://github.com/apache/arrow-datafusion/issues/1587

I think datafusion can release a small version (e.g. 7.0.1) once a month and a large version (e.g. 8.0.0-alpha) when there are major changes. The reason is that the pull request of DataFusion is frequent. Therefore, a stable release plan is required.

I would be happy to help support more incremental releases of datafusion, but I probably don't have time to manage the whole thing
What I think would be needed is:

A stable branch (release_7.x) -- which I can make
people to make PRs that cherry-pick changes from master to that stable branch
A regular release from the release branch

I am happy to do the mechanics of creating a branch and release artifacts, but I would need help from the community backporting / cherry-picking backwards compatible changes to it.

I agree with you very much. I suggest that you can post a bulletin in ReadMe to recruit volunteers to help manage the publishing. It may not be obvious to track the issue.

Maybe we could imitate the style of Apache Spark.

Major release (such as 8.0.0)(irregularly, but may be 2~3 years):  significant new features, optimizations in architecture, or the back compatibility is broken.
Minor release (such as 7.1, 7.2 ...)(around half a year): Performance improvement and small features
Maintenance  release (such as 7.0.1, 7.0.2)(regularly released monthly) : focus on bugs and stability. New features should not be introduced.

Arrow (non-rust) seems to have major release quarterly and as of 2022-02, it is 7.0.0. As of 2022-03, Arrow Rust have reach 11.0.0 and Arrow Datafusion is 7.0.0.
I think Datafusion can have similar release plan as Arrow (non-rust)

major release (from master), every 3 month
minor release (from master), every 1 month
patch release (from previous minor release branch) for bugs (review whether need release every week)

Agree to make 3-layer releases: major, minor, bug fix.
Another question is whether it is necessary to maintain the same version for different modules, like Ballista, datafusion-data-access (newly splitted one).

Agree to make 3-layer releases: major, minor, bug fix.

Arrow C++ does major quarterly releases; I have not seen a minor release (e.g. 6.1.0) in the last year. Occasionally there are patch releases but it is infrequent and typically once per major release.
I agree the three release sounds ideal as well.

minor release (from master), every 1 month

If we intended to conform to "semantic versioning" in the rust style, it is a challenge to release minor versions from master. For the minor (e.g. 7.0.0 to 7.1.0) release to be semantically versioned no breaking API changes can be introduced which would restrict what we can put on master

Another question is whether it is necessary to maintain the same version for different modules, like Ballista, datafusion-data-access (newly splitted one).

I do not think it is necessary to keep the same versions. I keep the versions of arrow-rs/arrow-flight/parquet in sync because it lowers the release overhead.

The challenge I predict we will encounter is getting the time to manage the releases (aka reviewing PRs, decide what to backport, backporting, making release notes and version bumps).
I don't think the work is "hard" per se but it does take sustained time and effort
Maybe we could start with

major release (from master), every 3 month
minor/patch release (from previous minor release branch) for bugs and minor features (released on on demand / every month)

Does anyone want to volunteer to manage such release(s)?

Does anyone want to volunteer to manage such release(s)?

I would love to

Agree that backporting patches to a stable branch is a very time consuming work so better not commit to it until we see strong need from our users or we have a maintainer who can allocate dedicated time to maintain the stable branch.

Ok, since @jychen7 has volunteered, let's give it a try for a release or two of datafusion 7.x
I have created a 7.x maintenance branch

https://github.com/apache/arrow-datafusion/tree/maint-7.x

The next steps would be to decide on some content to backport (via cherry-pick) that are semantically compatible.
To do so I suggest:

Create a new PR for each change you would like to release in the 7.x line against the maint-7.x branch
Tag me on the PR -- I'll review and merge
When we are ready to release this next version, we can update the release notes / changelog and I'll propose an official release.

Sound good @jychen7 ?

@alamb if I understand correctly, our next major release wil be around 2022-05-14 (2nd weekend of May). And next possible minor release will be around 2022-04-09 (2nd weekend of Apr).
we ask contributor who want minor/patch release to create PR to maint-* branch after original PR is merged to master? I draft the doc update at https://github.com/apache/arrow-datafusion/pull/2110
As volunteer, I would help to

every week, check how many PRs to maintainance branch (e.g. search by base:maint-7.x). If any,

confirm commit is cherry-pick from master
confirms is minor (non API breaking) change
tag you for review

draft changelog for release every month if major/minor release need

ps: 1 may be automate in Github workflow in future if need

@alamb if I understand correctly, our next major release wil be around 2022-05-14 (2nd weekend of May). And next possible minor release will be around 2022-04-09 (2nd weekend of Apr).

I think that would be reasonable

we ask contributor who want minor/patch release to create PR to maint-* branch after original PR is merged to master? I draft the doc update at https://github.com/apache/arrow-datafusion/pull/2110

Yes, thank you

As volunteer, I would help to

❤  thank you so much!

Question related, do you plan to release the datafusion-cli as a crate as well ? I see that the 7.0.0 datafusion-cli crate has been yanked (for reasons that I ignore).

Hi @happysalada  --I don't expect we'll release datafusion-cli to crates.io.
The reason that the datafusion-cli crate was not published (to crates.io) for 7.0.0 is that it depends on ballista which did not have a 7.0 release.
For now, you can probably install datafusion-cli from source / github if you want
As backstory,  datafusion-cli is mostly a debugging / development tool for datafusion and ballista -- and clients such as https://github.com/roapi/roapi or https://github.com/datafusion-contrib/datafusion-python  were more appropriate for end users.
If you wanted to make a datafusion-cli crate that was publishable (or break something similar into https://github.com/datafusion-contrib) I think it could be useful.

@happysalada shameless plug - im working on a more full featured datafusion cli client https://github.com/datafusion-contrib/datafusion-tui if youre interested.  its a new project that still has bugs but im getting close to a 0.1 release.  My plan is to publish on crates and to homebrew.

I think the easiest way to package this for now is to build from source.
With the PR that you nicely merged, it should be good. I hit some compilation error on macos x86_64, but I'm going to test on linux and release it for that platform (for nixos) if it all goes well.
Matthew, nice project! Watching out for releases, will package it for nixos as well when I play with it!

Release candidate was created and voting started here: https://lists.apache.org/thread/kvk7688gpfofrc46zso306rdnqxfdcdc

Mailing list approval: https://lists.apache.org/thread/h75wtqcvc1w64x2p4tb9lc6nc2t2zhwm
The release is available here:
https://dist.apache.org/repos/dist/release/arrow/arrow-datafusion-7.1.0
I have also released it to crates.io:
https://crates.io/crates/datafusion/7.1.0

Thanks @jychen7  for the assist getting this out 👍

authors: HaoYang670 alamb happysalada houqp jychen7 matthewmturner silence-coding yahoNanJing


text:
Add ClickBench queries to DataFusion benchmark runer

~Draft as it builds on https://github.com/apache/arrow-datafusion/pull/7054~
Which issue does this PR close?
closes https://github.com/apache/arrow-datafusion/issues/6994

Rationale for this change
see https://github.com/apache/arrow-datafusion/issues/6994 -- tldr is to optimize clickbench queries it needs to be easier to run them
What changes are included in this PR?

Add new dfbench clickbench command to run ClickBench queries
Update bench.sh to run clickbench queries
Update benchmarks/README.md -- see rendered version https://github.com/alamb/arrow-datafusion/tree/alamb/clickbench_runner/benchmarks

Are these changes tested?
I tested them manually
Run clickbench q1 directly (e.g. for profiling):
cargo run  --bin dfbench -- clickbench --query 1
Running benchmarks with the following options: RunOpt { query: Some(1), common: CommonOpt { iterations: 3, partitions: 2, batch_size: 8192 }, path: "benchmarks/data/hits.parquet", queries_path: "benchmarks/queries/clickbench/queries.sql", output_path: None }
Q1: SELECT COUNT(*) FROM hits;
Query 1 iteration 0 took 305.7 ms and returned 1 rows
Query 1 iteration 1 took 13.6 ms and returned 1 rows
Query 1 iteration 2 took 13.6 ms and returned 1 rows

run with hits_partitioned (100 parquet files):
cargo run  --bin dfbench -- clickbench --query 1 --path=benchmarks/data/hits_partitioned

Run with bench.sh:
./bench.sh run clickbench_1

See help
cargo run  --bin dfbench  -- clickbench --help

dfbench-clickbench 27.0.0
Run the clickbench benchmark

The ClickBench[1] benchmarks are widely cited in the industry and
focus on grouping / aggregation / filtering. This runner uses the
scripts and queries from [2].

[1]: https://github.com/ClickHouse/ClickBench
[2]: https://github.com/ClickHouse/ClickBench/tree/main/datafusion

USAGE:
    dfbench clickbench [OPTIONS]

FLAGS:
    -h, --help
            Prints help information

    -V, --version
            Prints version information

OPTIONS:
    -s, --batch-size <batch-size>
            Batch size when reading CSV or Parquet files [default: 8192]

    -i, --iterations <iterations>
            Number of iterations of each test run [default: 3]

    -o, --output <output-path>
            If present, write results json here

    -n, --partitions <partitions>
            Number of partitions to process in parallel [default: 2]

    -p, --path <path>
            Path to hits.parquet (single file) or `hits_partitioned` (partitioned, 100 files) [default:
            benchmarks/data/hits.parquet]
    -r, --queries_path <queries-path>
            Path to queries.sql (single file) [default: benchmarks/queries/clickbench/queries.sql]

    -q, --query <query>
            Query number. If not specified, runs all queries

Are there any user-facing changes?

@tustvold or @Dandandan  do you have time to review this PR (I hope to use this benchmark runner to drive / test further groupby peformance improvements)

One comment about the clickbench numbers, the rest looks good to me! Thanks for driving this forward

authors: Dandandan alamb


text:
Update documentation for COPY command

Which issue does this PR close?
closes https://github.com/apache/arrow-datafusion/issues/9927
Rationale for this change
Looks like I missed a spot while updating the docs in https://github.com/apache/arrow-datafusion/pull/9754
What changes are included in this PR?

Remove old options from COPY write options documentation
Update COPY

Are these changes tested?
CI doc checks
Are there any user-facing changes?
Docs

Thank you for the review @hveiga 🙏


text:
Add neq_dyn_utf8_scalar kernel and tests

Which issue does this PR close?
Closes #1113 task neq_dyn_utf8_scalar
Rationale for this change

What changes are included in this PR?

Are there any user-facing changes?

Codecov Report

Merging #1125 (4cc2716) into master (0d825c1) will increase coverage by 0.00%.
The diff coverage is 87.09%.

@@           Coverage Diff           @@
##           master    #1125   +/-   ##
=======================================
  Coverage   82.32%   82.32%
=======================================
  Files         168      168
  Lines       49577    49608   +31
=======================================
+ Hits        40812    40838   +26
- Misses       8765     8770    +5

Impacted Files
Coverage Δ

arrow/src/compute/kernels/comparison.rs
90.75% <87.09%> (-0.13%)
:arrow_down:

parquet_derive/src/parquet_field.rs
65.98% <0.00%> (-0.46%)
:arrow_down:

arrow/src/array/transform/mod.rs
85.56% <0.00%> (ø)

arrow/src/datatypes/datatype.rs
66.80% <0.00%> (+0.42%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0d825c1...4cc2716. Read the comment docs.

Codecov Report

Merging #1125 (4cc2716) into master (0d825c1) will increase coverage by 0.00%.
The diff coverage is 87.09%.

@@           Coverage Diff           @@
##           master    #1125   +/-   ##
=======================================
  Coverage   82.32%   82.32%
=======================================
  Files         168      168
  Lines       49577    49608   +31
=======================================
+ Hits        40812    40838   +26
- Misses       8765     8770    +5

Impacted Files
Coverage Δ

arrow/src/compute/kernels/comparison.rs
90.75% <87.09%> (-0.13%)
:arrow_down:

parquet_derive/src/parquet_field.rs
65.98% <0.00%> (-0.46%)
:arrow_down:

arrow/src/array/transform/mod.rs
85.56% <0.00%> (ø)

arrow/src/datatypes/datatype.rs
66.80% <0.00%> (+0.42%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0d825c1...4cc2716. Read the comment docs.

authors: codecov-commenter matthewmturner

repo: apache/arrow-rs


text:
Implement RecordBatch::concat

Which issue does this PR close?
Closes #461 .
Rationale for this change
As described in the issue, tries to implement RecordBatch::concat according to https://github.com/apache/arrow-datafusion/blob/master/datafusion/src/physical_plan/coalesce_batches.rs#L232 .
What changes are included in this PR?
Adds a new function concat to struct RecordBatch, and test cases.
Are there any user-facing changes?
With this fix, a new RecordBatch could be created by concatenating multiple RecordBatches.

Codecov Report

Merging #537 (e1f24d1) into master (f1fb2b1) will increase coverage by 0.01%.
The diff coverage is 91.42%.

:exclamation: Current head e1f24d1 differs from pull request most recent head eec5c90. Consider uploading reports for the commit eec5c90 to get more accurate results

@@            Coverage Diff             @@
##           master     #537      +/-   ##
==========================================
+ Coverage   82.60%   82.61%   +0.01%
==========================================
  Files         167      167
  Lines       45984    46054      +70
==========================================
+ Hits        37984    38048      +64
- Misses       8000     8006       +6

Impacted Files
Coverage Δ

arrow/src/record_batch.rs
90.14% <91.42%> (+0.88%)
:arrow_up:

parquet/src/encodings/encoding.rs
94.85% <0.00%> (-0.20%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f1fb2b1...eec5c90. Read the comment docs.

Although @nevi-me seems to like RecordBatch::concat -- https://github.com/apache/arrow-rs/issues/461#issuecomment-877784891

Hi @alamb, refer to @jorgecarleitao latest comment #461 (comment), it seems that arrow::compute::concat::concat_batches may be better (even not good enough for thread model).
I will follow the discussion and try to update this PR.

thanks @silathdiir !

Unless I hear different, I plan to merge this PR tomorrow (and include it in 5.0.0)

Thanks again @silathdiir !

authors: alamb codecov-commenter silathdiir


text:
[Integration][Archery] Can't disable Java

Describe the bug, including details regarding any error messages, version, and platform.
We have an option to disable Java integration test: --with-java=0
If we use it, the following error is occurred:
$ archery docker run -e ARROW_INTEGRATION_JAVA=OFF conda-integration
...
Traceback (most recent call last):
  File "/opt/conda/envs/arrow/bin/archery", line 33, in <module>
    sys.exit(load_entry_point('archery', 'console_scripts', 'archery')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/arrow/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/arrow/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/arrow/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/arrow/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/arrow/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/arrow/dev/archery/archery/cli.py", line 833, in integration
    from .integration.runner import write_js_test_json, run_all_tests
  File "/arrow/dev/archery/archery/integration/runner.py", line 37, in <module>
    from .tester_java import JavaTester
  File "/arrow/dev/archery/archery/integration/tester_java.py", line 54, in <module>
    _arrow_version = load_version_from_pom()
                     ^^^^^^^^^^^^^^^^^^^^^^^
  File "/arrow/dev/archery/archery/integration/tester_java.py", line 37, in load_version_from_pom
    tree = ET.parse(os.path.join(ARROW_BUILD_ROOT, 'java', 'pom.xml'))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/arrow/lib/python3.11/xml/etree/ElementTree.py", line 1219, in parse
    tree.parse(source, parser)
  File "/opt/conda/envs/arrow/lib/python3.11/xml/etree/ElementTree.py", line 570, in parse
    source = open(source, "rb")
             ^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/build/java/pom.xml'

Component(s)
Archery, Integration

Issue resolved by pull request 44301
https://github.com/apache/arrow/pull/44301

authors: kou raulcd

repo: apache/arrow


text:
[C++][Flight] Support dictionary replacement in DoGet

Describe the enhancement requested
Flight C++ doesn't support dictionary replacement in DoGet calls and support should be added.
A minimal reproduction of this follows:
Server:
import pandas as pd
import pyarrow as pa
import pyarrow.flight
import pyarrow.parquet

schema = pa.schema(
    [
        pa.field("numbers", pa.int8(), nullable=True),
        pa.field("letters", pa.dictionary(pa.int8(), pa.utf8()), nullable=True),
    ]
)

batches = [
    pa.RecordBatch.from_pandas(
        pd.DataFrame({"numbers": [10, 11, 12], "letters": ["a", "b", "c"]}),
        schema=schema,
    ),
    pa.RecordBatch.from_pandas(
        pd.DataFrame({"numbers": [13, 14], "letters": ["d", "e"]}), schema=schema
    ),
]

flights = {"./table": batches}

class FlightServer(pa.flight.FlightServerBase):
    def __init__(self, location="grpc://0.0.0.0:8815", **kwargs):
        super(FlightServer, self).__init__(location, **kwargs)
        self._location = location

    def do_get(self, context, ticket):
        dataset_name = ticket.ticket.decode("utf-8")
        reader = pa.RecordBatchReader.from_batches(schema, flights[dataset_name])

        return pa.flight.RecordBatchStream(reader)

if __name__ == "__main__":
    server = FlightServer()
    server.serve()

And with the following client code:
import pandas as pd
import pyarrow as pa
import pyarrow.flight

client = pa.flight.connect("grpc://0.0.0.0:8815")

for fl in client.list_flights():
    response = client.do_get(fl.endpoints[0].ticket)
    print(response.read_all().to_pandas())

An incorrect result is printed:
   numbers letters
0       10       a
1       11       b
2       12       c
3       13       a
4       14       b

(a b c d a b instead of a b c d e f)
Component(s)
C++, FlightRPC

@lidavidm pointed out the probable cause here,
https://github.com/apache/arrow/blob/aab7d81aeec1e8f106bdda953cdeb00f7f78b355/cpp/src/arrow/flight/transport_server.cc#L163-L164
And you can see the trick mentioned above in the client code, accessing the arrow::ipc::IpcPayloadWriter directly:
https://github.com/apache/arrow/blob/2bd2e35a97c28cbe32e542d4ba3b8f64e6592e42/cpp/src/arrow/flight/client.cc#L413-L418


text:
ARROW-1542: [C++] Install packages in temporary directory in MSVC build verification script

I found that the script did not work due to the remnants of the last time I ran it.

+1


text:
ARROW-3336: [JS] Fix IPC writer serializing sliced arrays

Fixes https://issues.apache.org/jira/browse/ARROW-3336. Realized too late I branched from the https://github.com/apache/arrow/pull/2616, so that's why there's some extra commits at the front.
Check out the relevant tests here: https://github.com/trxcllnt/arrow/blob/860f61046fca0081dbe0ce986a97408ed5934e22/js/test/unit/writer-tests.ts#L26. This test covers the primitive, nested, and dictionary vectors, but it'd be worth adding more (for example, Unions).

looks like JS is passing, but the R build has failed

@TheNeuralBit also this commit here makes the Table constructor arg handling significantly less confusing: https://github.com/apache/arrow/pull/2638/commits/49c601c03b46bd7821cf1fa600b38b32bad62cf0

Just rebased this after merging #2616, #2635

No more comments from me, let me know if you want me to merge this or anything else

@wesm I'm happy to do the merge also, if for no other reason than to get my PR merge stats up :-)

OK, I'll leave it to you both

authors: trxcllnt wesm


text:
GH-29705: [Python] Clean-up no-longer-used pandas dataframe serialization helpers

Small follow-up on https://github.com/apache/arrow/pull/34926, which removed the pyarrow.serialization functionality, making those functions obsolete.

Closes: #29705

Yay! Always fun to hit delete. LGTM

authors: danepitkin jorisvandenbossche


text:
MINOR: [Docs][Java] Add Apache Staging url to docs

Rationale for this change

The JAR staging location is not documented, let's fix that.

What changes are included in this PR?

Add Arrow Java Apache Staging URL to docs.
Remove lingering JDK 8 references.

Are these changes tested?
Yes - screenshot shared below
Are there any user-facing changes?
Yes, improved documentation on the Arrow website.

Screenshot:


text:
AXIS2-5804: Service removal within OSGi environments is not handled b…

…y WSTracker.

Can you also add a test case?

authors: cfiehe veithen


text:
Fix Pub/Sub Lite IO and SDF performance issues with shuffles

Imported from Jira BEAM-14116. Original Jira may contain additional context.
Reported by: dpcollins-google.

Unable to assign user @dpcollins-google . If able, self-assign, otherwise tag @damccorm so that he can assign you. Because of GitHub's spam prevention system, your activity is required to enable assignment in this repo.

repo: apache/beam


text:
[Feature Request]: Composite triggers for Go SDK

What would you like to happen?
Add composite triggers like AfterAll, AfterEach, AfterAny, OrFinally triggers to Go SDK.
Issue Priority
Priority: 2
Issue Component
Component: sdk-go

I've an PR opened for this #21756

.take-issue


text:
[Bug]: Make the lulz logging messages consistent across all flavors of Dataflow

What happened?
Format strings look like this, but are not exactly the same/consistent.
"Processing stuck in step {step name} for at least {duration} without outputting or completing in state process at {stack trace}".
Goals (As much as possible)

Ensure lulz logging is implemented in all flavors
Consistent user step name format (might be the most difficult, I think some places display "s4" style, and some display "User/Step/Name/Format"
Use same format string
Stack trace is displayed in all flavors.

Issue Priority
Priority: 3 (minor)
Issue Components

[X] Component: Python SDK
[ ] Component: Java SDK
[ ] Component: Go SDK
[ ] Component: Typescript SDK
[ ] Component: IO connector
[ ] Component: Beam examples
[ ] Component: Beam playground
[ ] Component: Beam katas
[ ] Component: Website
[ ] Component: Spark Runner
[ ] Component: Flink Runner
[ ] Component: Samza Runner
[ ] Component: Twister2 Runner
[ ] Component: Hazelcast Jet Runner
[ ] Component: Google Cloud Dataflow Runner

.take-issue

Thanks for filing and fixing this. should we also update the 'go' wording, see: https://github.com/apache/beam/blob/98210d99b8530346b66fcffe66b893924c910bea/sdks/go/pkg/beam/core/metrics/sampler.go#L64 cc: @lostluck

That is a good point. I'll do that, thanks @tvalentyn.

Reopening since Go still needs to be addressed

authors: bzablocki damccorm tvalentyn


text:
Revert "[BEAM-9014] CachingShuffleBatchReader use bytes to limit cache size."

This change negatively impacts shuffle performance for large iterables.
R: @tudorm @tysonjh

CC: @ibzib Another change to cherrypick into release 2.21 if possible.

lgtm

authors: lukecwik tudorm


text:
[BEAM-10821] Add Python SDK typing improvements to CHANGES.md

This PR documents two improvements to the typehints module of the Python SDK in CHANGES.md.

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[x] Choose reviewer(s) and mention them in a comment (R: @username).
[x] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
Post-Commit Tests Status (on master branch)

Lang
SDK
Dataflow
Flink
Samza
Spark
Twister2

Go

---

---

---

Java

Python

---

---

XLang

---

---

---

Pre-Commit Tests Status (on master branch)

---
Java
Python
Go
Website
Whitespace

Non-portable

Portable
---

---
---
---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

R: @udim


text:
[BEAM-970] add side-input support to gearpump-runner

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

[x] Make sure the PR title is formatted like:
[BEAM-<Jira issue #>] Description of pull request
[ ] Make sure tests pass via mvn clean verify. (Even better, enable
Travis-CI on your fork and ensure the whole test matrix passes).
[x] Replace <Jira issue #> in the title with the actual Jira issue
number, if there is one.
[x] If this contribution is large, please file an Apache
Individual Contributor License Agreement.

R: @kennknowles

Jenkins failure is the same dependency problem that affected many PRs. I will review as green.

@kennknowles  First of all, thanks for taking a look during your holidays 👍.
For the tests,

I was under the wrong impression that three-way merge hadn't been supported in Gearpump and rejected multiple side inputs (embarrassing...)
PAssertTest#testContainsInAnyOrderFalse and PAssertTest#testEmptyFalse don't pass because of assertion should have failed. I haven't figured out why.
I lazily copied RawUnionValue to gearpump-runner. It's also used in flink-runner so it could be valuable to move RawUnionValule to values.

Do the gearpump local tests take too long for Jenkins? One thing that helped a lot for Apex was to have them running in precommit while things were under development. You could also add them to precommit or to Travis in one commit just for the PR and then I can remove before the merge. Obviously it cannot be on all the time, but right now sharing the failure stack traces, etc, would be a good way to collaborate.

I expect that the problem with PAssert is that the AssertionFailure is not being propagated.

Just noting that this PR has acquired merge conflicts.

Thanks, I will rebase

superseded by https://github.com/apache/beam/pull/2150

authors: kennknowles manuzhang