text:
Conflict with Better Rolls for 5e

When I select a token and hotbar (page 5 in my case) pops up the rolls from it are from a different character than the selected token. Additionally, when there are multiple tokens of the same actor on board, charges and spell slots are consumed across them instead of on the ones that made the roll.
An example, let's say that I have an Acolyte on the board, with Bless and Cure Wounds on the hotbar. I cast Bless and one spell slot is consumed and an appropriate chat message appears with correct name of the token, all good for now. Add another Acolyte on the board and what happens is that the Acolyte 1 spell slots are reset (I did some testing without the module and consumed spell slots remained consumed if I followed the same scenario) and if I cast another spell with Acolyte 1 the message in chat and consumed slots are as if Acolyte 2 rolled.
I did some testing, turned off and on few modules, and it appears that Token Hotbar and Better Rolls for 5e don't like each other. Without Better Rolls for 5e it seems to be working as intended. If I were to guess I think it has something to do with how Better Rolls for 5e creates macros.
Token Hotbar version 4.2.0
Better Rolls for 5e version 1.2.1
Foundry version 0.7.8 (was happening on 0.7.7 as well)
DnD5e system version 1.1.1

Please Note that Better Rolls is incompatible with 1.2.X of dnd5e as of yet.

Any news on this? It still behaves weirdly, sometimes even showing macros that the actor does not have, transferring them between tokens. And resources are also consumed on the wrong actor. Bunch of weird stuff going on but I am still hoping.
Last tested on Foundry 0.8.6

Unfortunately this module is no longer under active development. This issue might be resolved if another developer in the community wants to pick it up.
Apologies for the inconvenience. :(

@InnilToo the revamped version of the module is available for testing: https://github.com/janssen-io/foundry-tokenhotbar-js
Nothing in particular was changed with regards to this issue, but you might find it works better.

Thank you for the update! I tested it and unfortunately the behavior persists.
I had to decrease the "minimumCoreVersion": "9" to 0.8.9 as that is the highest version available on Forge.

authors: InnilToo MrVauxs janssen-io


text:
if using 'match origin protocol',occasional .match of undefined

If using 'match origin protocol' I've noticed occasionally that the origin will be null, in which case .match is called on something undefined and things break. This just makes sure origin is at least http and not null.

should it be == null instead of === ?

Here's the error by the way:
TypeError: Cannot call method 'match' of undefined
data:       at WebSocket.onSocketConnect (/home/node/node_modules/socket.io/lib/transports/websocket/hybi-07-12.js:107:30)
data:       at WebSocket.handleRequest (/home/node/node_modules/socket.io/lib/transport.js:71:10)
data:       at WebSocket.Transport (/home/node/node_modules/socket.io/lib/transport.js:31:8)
data:       at new WebSocket (/home/node/node_modules/socket.io/lib/transports/websocket/hybi-07-12.js:60:13)
data:       at new WebSocket (/home/node/node_modules/socket.io/lib/transports/websocket.js:31:17)
data:       at Manager.handleClient (/home/node/node_modules/socket.io/lib/manager.js:630:19)
data:       at Manager.handleUpgrade (/home/node/node_modules/socket.io/lib/manager.js:590:8)
data:       at HTTPServer. (/home/node/node_modules/socket.io/lib/manager.js:119:10)
data:       at HTTPServer.emit (events.js:81:20)
data:       at Socket. (http.js:1045:14)

Get rid of my pull request, it handles the match undefined issue, but causes other problems with flashsocket and IE, gonna find a better way of checking for this.

source: gharchive/issue

license: MIT

license_type: permissive

license_source: github-api


text:
🛑 CompanyGuide1 is down

In 90e7830, CompanyGuide1 (https://comp2.fnguide.com/server_variables.asp) was down:

HTTP code: 0
Response time: 0 ms

Resolved: CompanyGuide1 is back up in af1f97a.


text:
🛑 Naver2 is down

In 21944e2, Naver2 (https://navercomp2.wisereport.co.kr/v2/servercheck.aspx) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Naver2 is back up in b5af29b.


text:
🛑 CompanyGuide2 is down

In 4a52216, CompanyGuide2 (https://comp3.fnguide.com/server_variables.asp) was down:

HTTP code: 0
Response time: 0 ms

Resolved: CompanyGuide2 is back up in 3ae53ab.


text:
🛑 CompanyWise3 is down

In 0caba27, CompanyWise3 (https://comp3.wisereport.co.kr/servercheck.aspx) was down:

HTTP code: 0
Response time: 0 ms

Resolved: CompanyWise3 is back up in f3d6b0b after 23 minutes.


text:
🛑 MobileETFGlobal3 is down

In 9cdd901, MobileETFGlobal3 (https://mglobaletf3.wisereport.co.kr/Home/ServerCheck) was down:

HTTP code: 0
Response time: 0 ms

Resolved: MobileETFGlobal3 is back up in 5275289 after 20 minutes.


text:
Discussion: scrollToIndex precision

Currently scrollToItem works precisely only when getEstimatedItemSize callback is provided for each list item. This is either not possible or not very convenient for most cases.
On the following picture, let's imagine we need to scroll to "Item 9" in the list. Obtaining y position of item 9 is not straightforward, because to have it we need to render and measure "item6", "item 7" and "item 9".

Naive solution here can be to delay scroll, until those missing items are rendered and measured. But it also looks bit complicated to implement. It will also be a problem, if there is 500 items before current and scrollToItem position. Therefore I'm opening this issue to brainstorm possible solution for a scrollTo algorithm.

Thats very interesting thread.
I'm wondering how hard it would be if we could scroll to estimated location(based on estimatedItemSize) and enforce that item to be rendered there.
In that case we would need some kind of bidirectional list which would allow us to scroll back to first item.
The issue I'm seeing which may happen is some wired jumping while we will mount prepending items.
Actually quite curious if that could be possible.

@MateWW very precisely described! Either we need to correct position when user scrolls closer to the screen edges(which will result in the content jump. Or to have some sort of bidirectional scrollview where we can adjust top and bottom edges dynamically.
One of implementations of bidirectional scrollview can be found there: https://github.com/GetStream/react-native-bidirectional-infinite-scroll

This is a tough one. I attempted it in an early prototype but it only worked in certain scenarios, and only on iOS (it depended on contentInset).
The specific problem I was trying to solve was when starting with an initialScrollIndex, to be able to scroll up without any visual jumping. I managed to get it to work on iOS by adding a padding element to the top when size was less than estimated size and offsetting it by contentInset, but that only worked on iOS because Android doesn't support contentInset. And it didn't work at all if size was greater than estimated size.
I also tried adjusting scroll to compensate, but it seemed to sometimes be 1 frame off so it would flash to an incorrect scroll position for 1 frame and then to the correct place.
But then I tried the other list libraries and they didn't solve it either, so I punted it to focus on the basics first.
So I think the best solution to this question would be to scroll to the estimated position, and when actual sizes come in above the current scroll positions, either adjust scroll or a top padding (or something) so that it visually looks correct.
I'll revisit this idea after a v1 release, or if someone wants to attempt a solution to it, I would be happy to advise!

and enforce that item to be rendered there

Love this idea!
A potential solution for the weird jumping could be a virtual window where when scrolling to an index, the item is rendered anywhere possible and the list just scrolls to it. Previous and next items are then rendered before and after to accommodate.
FlashList does a decent enough job but if the index is way out of draw distance, it will just scroll based on estimatedItemSize and calls it a day. FlatList just fails.

Yeah as mentioned, Flashlist has this issue as well, it will just estimate where it should land and usually fails if you have items of different sizes, it does well with equal size items.
Flatlist can actually do it precisely even for different sizes, but you have set initialNumToRender to the index + 1 your are looking for, which is obviously performance hell in long lists. Difficult one to crack, fingers crossed 🤞

I think I've cracked it 🎉. I'm just finalizing some edge cases around it before releasing it.

authors: AronBe MateWW alizahid jmeistrich mbilenko-florio

repo: LegendApp/legend-list


text:
armv7 support

I have been doing development in the simulator. When I ran into the file upload problem that I posted a bit ago, I decided to try it on a device to see if it made any difference. When I run on the device, I get a linker error:ld: warning: ignoring file .../NMSSH.framework/NMSSH, missing required architecture armv7 in file .../NMSSH.framework/NMSSH (2 slices). Can anyone explain? Does it really only run in arm64?

If built for the simulator it might not work on a device since the simulator uses a completely different architecture. You need to rebuild the framework for the device.

I have the opposite problem, I think mine was previously build for device so now I need to rebuild for the simulator.  Can you please tell me how to rebuild the framework?

I will let you know when I succeed.

On Feb 23, 2015, at 6:16 AM, solarissf notifications@github.com wrote:
I have the opposite problem, I think mine was previously build for device so now I need to rebuild for the simulator. Can you please tell me how to rebuild the framework?
—
Reply to this email directly or view it on GitHub https://github.com/Lejdborg/NMSSH/issues/114#issuecomment-75547334.

You should follow the wiki guide, the framework should be universal...

I ended up installing from PODS and both simulator and device both work now

To maybe help someone else.
I had not realised that I needed to build one copy of the NMSSH.framework for a simulator target AND another for the iOS device target.

The wiki guide show the steps to build a framework for both the device and the simulator, otherwise Xcode will build the framework only for the active architecture.

authors: Frugghi Lejdborg amckean iCyberPaul solarissf

repo: Lejdborg/NMSSH

license: mit

license_source: bigquery


text:
gatsby develop errors out

After installing the starter, going into the starter dir, and running gatsby develop I get errors locally
Output:
/Users/.../minimal-blog/node_modules/yoga-layout-prebuilt/yoga-layout/build/Release/nbind.js:53
        throw ex;
        ^

Error: Cannot find module '@gatsbyjs/reach-router/lib/utils'
Require stack:
- /Users/.../minimal-blog/node_modules/gatsby/dist/utils/find-page-by-path.js
- /Users/.../minimal-blog/node_modules/gatsby/dist/utils/websocket-manager.js
- /Users/.../minimal-blog/node_modules/gatsby/dist/utils/page-data.js
- /Users/.../minimal-blog/node_modules/gatsby/dist/utils/webpack-status.js
- /Users/.../minimal-blog/node_modules/gatsby/dist/commands/develop-process.js
- /Users/.../minimal-blog/.cache/tmp-36290-H2f75Rh4Kv4S
    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:927:15)
    at Function.Module._load (node:internal/modules/cjs/loader:772:27)
    at Module.require (node:internal/modules/cjs/loader:999:19)
    at require (node:internal/modules/cjs/helpers:93:18)
    at Object.<anonymous> (/Users/ragozzine/saga/minimal-blog/node_modules/gatsby/src/utils/find-page-by-path.ts:2:1)
    at Module._compile (node:internal/modules/cjs/loader:1095:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1124:10)
    at Module.load (node:internal/modules/cjs/loader:975:32)
    at Function.Module._load (node:internal/modules/cjs/loader:816:12)
    at Module.require (node:internal/modules/cjs/loader:999:19) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/Users/.../minimal-blog/node_modules/gatsby/dist/utils/find-page-by-path.js',
    '/Users/.../minimal-blog/node_modules/gatsby/dist/utils/websocket-manager.js',
    '/Users/.../minimal-blog/node_modules/gatsby/dist/utils/page-data.js',
    '/Users/.../minimal-blog/node_modules/gatsby/dist/utils/webpack-status.js',
    '/Users/.../minimal-blog/node_modules/gatsby/dist/commands/develop-process.js',
    '/Users/.../minimal-blog/.cache/tmp-36290-H2f75Rh4Kv4S'
  ]
}

+1.

Hi! Please see https://github.com/LekoArts/gatsby-themes/discussions/613 for a solution. Thanks!

Worked like a charm. Thanks so much!

authors: LekoArts boomspot ragozzine

repo: LekoArts/gatsby-themes


text:
Take care of the security issues

Not urgent because it's only dev dependencies, the website is just static HTML.

Aye, but it looks ugly ;)

authors: ralphtheninja vweevers


text:
添加镜像站点

网址
https://3am.com.hk
属性

基本属性（单选）

[x] 🆓完全免费，打开即用
[ ] 🔓有免费额度
[x] 🔒需要登陆
[ ] 🔑需要输入 API Key
[x ] 💰需要充值
[ ] 👀需要关注公众号

扩展属性（多选）

[ x] 💪支持 GPT4
[x ] 🧰不仅仅是 Chat
[ ] 🌎需要国际网络

附加信息
註冊後完全免費無限制使用GPT3.5Turbo 以及 Claude3 Haiku，專注繁體中文AI 體驗
介面簡潔，支持微信/支付宝 國際信用卡；企業級騰訊雲香港服務器集群，服務SLA 可靠

网址
https://3am.com.hk
属性
基本属性（单选）

[x]  🆓完全免费，打开即用
[ ]  🔓有免费额度
[x]  🔒需要登陆
[ ]  🔑需要输入 API Key
[x ] 💰需要充值
[ ]  👀需要关注公众号

扩展属性（多选）

[ x] 💪支持 GPT4
[x ] 🧰不仅仅是 Chat
[ ]  🌎需要国际网络

附加信息
註冊後完全免費無限制使用GPT3.5Turbo 以及 Claude3 Haiku，專注繁體中文AI 體驗 介面簡潔，支持微信/支付宝 國際信用卡；企業級騰訊雲香港服務器集群，服務SLA 可靠
完全免費Relax模式 Midjourney 繪圖功能

repo: LiLittleCat/awesome-free-chatgpt


text:
error: command '/usr/bin/g++' failed with exit code 1

running build
running build_ext
building 'tree_filter_cuda' extension
Emitting ninja build file /home/1/mmdetection-2.11.0/mmdetection-2.11.0/mmdet/ops/tree_filter/build/temp.linux-x86_64-3.7/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
1.10.2
g++ -pthread -B /home/1/anaconda3/envs/hongmoumou/compiler_compat -Wl,--sysroot=/ -pthread -shared -B /home/1/anaconda3/envs/hongmoumou/compiler_compat -L/home/1/anaconda3/envs/hongmoumou/lib -Wl,-rpath=/home/1/anaconda3/envs/hongmoumou/lib -Wl,--no-as-needed -Wl,--sysroot=/ /home/1/mmdetection-2.11.0/mmdetection-2.11.0/mmdet/ops/tree_filter/build/temp.linux-x86_64-3.7/home/1/mmdetection-2.11.0/mmdetection-2.11.0/mmdet/ops/tree_filter/src/bfs/bfs.o /home/1/mmdetection-2.11.0/mmdetection-2.11.0/mmdet/ops/tree_filter/build/temp.linux-x86_64-3.7/home/1/mmdetection-2.11.0/mmdetection-2.11.0/mmdet/ops/tree_filter/src/mst/boruvka.o /home/1/mmdetection-2.11.0/mmdetection-2.11.0/mmdet/ops/tree_filter/build/temp.linux-x86_64-3.7/home/1/mmdetection-2.11.0/mmdetection-2.11.0/mmdet/ops/tree_filter/src/mst/mst.o /home/1/mmdetection-2.11.0/mmdetection-2.11.0/mmdet/ops/tree_filter/build/temp.linux-x86_64-3.7/home/1/mmdetection-2.11.0/mmdetection-2.11.0/mmdet/ops/tree_filter/src/refine/refine.o /home/1/mmdetection-2.11.0/mmdetection-2.11.0/mmdet/ops/tree_filter/build/temp.linux-x86_64-3.7/home/1/mmdetection-2.11.0/mmdetection-2.11.0/mmdet/ops/tree_filter/src/tree_filter.o -L/home/anaconda3/envs/lib -L/home/1/anaconda3/envs/hongmoumou/lib/python3.7/site-packages/torch/lib -L/usr/local/cuda/lib64 -lc10 -ltorch -ltorch_cpu -ltorch_python -lcudart -lc10_cuda -ltorch_cuda -o build/lib.linux-x86_64-3.7/tree_filter_cuda.cpython-37m-x86_64-linux-gnu.so
g++: error: /home/1/mmdetection-2.11.0/mmdetection-2.11.0/mmdet/ops/tree_filter/build/temp.linux-x86_64-3.7/home/1/mmdetection-2.11.0/mmdetection-2.11.0/mmdet/ops/tree_filter/src/bfs/bfs.o: 没有那个文件或目录
g++: error: /home/1/mmdetection-2.11.0/mmdetection-2.11.0/mmdet/ops/tree_filter/build/temp.linux-x86_64-3.7/home/1/mmdetection-2.11.0/mmdetection-2.11.0/mmdet/ops/tree_filter/src/mst/boruvka.o: 没有那个文件或目录
g++: error: /home/1/mmdetection-2.11.0/mmdetection-2.11.0/mmdet/ops/tree_filter/build/temp.linux-x86_64-3.7/home/1/mmdetection-2.11.0/mmdetection-2.11.0/mmdet/ops/tree_filter/src/mst/mst.o: 没有那个文件或目录
g++: error: /home/1/mmdetection-2.11.0/mmdetection-2.11.0/mmdet/ops/tree_filter/build/temp.linux-x86_64-3.7/home/1/mmdetection-2.11.0/mmdetection-2.11.0/mmdet/ops/tree_filter/src/refine/refine.o: 没有那个文件或目录
g++: error: /home/1/mmdetection-2.11.0/mmdetection-2.11.0/mmdet/ops/tree_filter/build/temp.linux-x86_64-3.7/home/1/mmdetection-2.11.0/mmdetection-2.11.0/mmdet/ops/tree_filter/src/tree_filter.o: 没有那个文件或目录
error: command '/usr/bin/g++' failed with exit code 1
@LiWentomng
when I used the script (python setup.py build develop) to comply the tree_filter,the problem was occuring，I have tried to comply the bfs to bfs.o ,but there was lack of the file of bfs.cpp ,how can I  solve the problem?

I have same questions. Have u fixed?

authors: Wenlve-Zhou liusurufeng

repo: LiWentomng/boxlevelset


text:
请问基于transformers的中文转写可以加标点吗？

请问基于transformers的中文转写可以加标点吗？whisper可以通过prompt引导，咱们这个transformers可以加吗？

目前基于AISHELL、WENETSPEECH等数据做的微调，弱化了标点能力。
解决方法：

可以基于标点模型 对转写文本加标点。例如 https://modelscope.cn/models/damo/punc_ct-transformer_cn-en-common-vocab471067-large/summary
微调带标点的数据，这块开源数据可能比较少。

感谢，受益匪浅

repo: LianjiaTech/BELLE


text:
fix flutter 3.x warning

fix binding.instance may cause warning.
Compatible with both flutter 3.x and earlier versions.
refer: https://github.com/flutter/website/blob/main/src/development/tools/sdk/release-notes/release-notes-3.0.0.md#your-code

感谢提交，目前还没有进行Flutter 3.0的适配，待开新分支会将这笔提交合并过去

LGTM~

我发现 #232 已经修复这个问题，我会关闭这个PR 谢谢

source: gharchive/pull-request

authors: laiiihz zhoujuanjuan

repo: LianjiaTech/bruno


text:
Add alias to cmd line options

cmd line options can be accessed with shorter alternatives
Contributor Checklist:

[x] I have added a changelog entry for my contribution
[x] I have added/updated documentation for all user-facing changes

Thanks, merged!

repo: LiberTEM/LiberTEM


text:
What license?

Is this code production ready and what license?

Hi @EloB!
very interesting question.. I'll try to confirm licence ASAP

I've added MIT license to the project, have a fun!

Boom! <3

authors: EloB osigida

repo: LibertyGlobal/ReactLiberty


text:
SMA connector package: 142-0701-801

SMA connector, edge mount
https://www.digikey.ch/product-detail/de/cinch-connectivity-solutions-johnson/142-0701-801/J502-ND/35280

I would place the {{NAME}} label on the right side too, above the {{VALUE}} label. This makes the package much more compact. Otherwise it looks good :+1:

Fixed!

I also renamed the package to 142-0701-80X to include both plating variants.

authors: dbrgn ubruhin


text:
Code Organization

Hi @luojw-dwr,
Thanks for your quick reply and suggestions.
For the floorplanning part, I'm not sure which parts you think are inappropriately coupled? Currently the floorplan class takes in a dataflow graph and returns a mapping from the slots to the vertices assigned to this slot. I think we could have multiple classes, each for a different floorplanning algorithm?
BTW, previously you mentioned that you formulate the problem in another non-linear model. I'm not sure this is a good idea. With an ILP model, we already run into a scalability issue, which I'm working on right now. If some slots are completely out, as in the F1 case, we could set the available resource to 0 to handle the situation. Thus I do not see a solid motivation in making the model more complex, maybe you could explain more?
I agree the user interface is a mess right now, though I don't have the bandwidth for that recently. Ideally, the users should only provide the target device and their source code and nothing else. Then we will parse the mapping from the source code to the RTL. On the other hand, if they want to hack with the tool (e.g. add additional board, specify extra floorplanning constraints), they are supposed to be somewhat familiar with the CAD flow and I believe clock regions are relatively basic concepts? Let me know if you disagree.
My plan is to leave the user interface aside for a while (3 months, before my next deadline), and if users (rare for now) run into problems I could provide direct help.
Best,
Licheng

Hi @Licheng-Guo ,
About the HLSParser package, it is nice to see that tapa frontend has been added. It is quite a good beginning.
About the Device package, my idea is to leave the concrete device information to plain-text files, e.g. yaml or csv. We may have further discussion about how to represent the 'branches' on the resource variations induced by composition of instantiations of IPs, e.g. DDR, NoC infrastructures, etc. What choices come to me at this early stage:

record different resource situation as you've done in the old code base, but this may be a problem when the compositions are of a terrifying number.
let users pick the correct 'branch' in the resource file. (then supporting on comments can be a must for the file format)

About the Opt package, the components are still heavily coupled with the details of the proposed floorplanning algorithm. I'd like to start a new git branch to get them decoupled as I've mentioned in the last few e-mails. Hopefully I will keep the interfaces consistent (one exception is the coordination of floorplanner and slots, as the organizaton of slots may vary a lot depending on the floorplanner implementation).
About the Flow package, the Floorplan (optional) setion in def help seems to be confusing. It is really a problem for both of us to allow user to make partial assignment with a tidy interface without knowledge about the floorplanning implementation details on naming conventions.

This may be a problem if users want to connect vhls and autobridge directly as a whole workflow. (Some manual coding is not too big a realistic problem at this stage though).

The naming convention (the CLOCKREGION stuff) is not quite obvious even after I have read the Device package. This can be a real problem as normally user won't dive into the Opt package for namings (though the floorplanner itself and the constraint stuffs are there).

Maybe either

expose the namings to somewhere more obvious, or
let users bind to slot graphs and leave us to manage the name mappings. (It is not a choice if the slot graph is dynamically generated during floorplanning though. Several round trips between abstract coordinates and slot-during-iteration is otherwise needed.)

By the way, discussing in a closed issue seems not to be a correct way using GitHub. Maybe start a new issue ;-P
It is nice to be on a consistent track with you!
Best,
Jianwen

Hi Licheng,User interface is not of a high priority at present, let's put it aside.About the coupling, the dataflow graph contains vertical cuts and horizontal cuts, which may not fit other floorplanning algorithms. It can be better if we first generate an algorithm-independent representatio and then leave the floorplanner to add additional information (via new classes). On the other hand, the pipeliner still consumes the cuts, which maybe better to be extended to consume slot paths.For my model, there can be:* Differently weighted connections through die/slots* NoCs introducing quite a different interconnection topology/distance-metrics among slots.Besides, I think it not quite ideal to keep splitting slots into equal parts when handling with strongly heterogeneous situations. We may discuss this further.Best,Jianwen-------- 原始邮件 --------发件人： Licheng Guo @.>日期： 2021年3月16日周二 下午2:25收件人： Licheng-Guo/AutoBridge @.>抄送： Jianwen Luo @.>, Mention @.>主    题： [Licheng-Guo/AutoBridge] Code Organization (#9)
Hi @luojw-dwr,
Thanks for your quick reply and suggestions.
For the floorplanning part, I'm not sure which parts you think are inappropriately coupled? Currently the floorplan class takes in a dataflow graph and returns a mapping from the slots to the vertices assigned to this slot. I think we could have multiple classes, each for a different floorplanning algorithm?
BTW, previously you mentioned that you formulate the problem in another non-linear model. I'm not sure this is a good idea. With an ILP model, we already run into a scalability issue, which I'm working on right now. If some slots are completely out, as in the F1 case, we could set the available resource to 0 to handle the situation. Thus I do not see a solid motivation in making the model more complex, maybe you could explain more?
I agree the user interface is a mess right now, though I don't have the bandwidth for that recently. Ideally, the users should only provide the target device and their source code and nothing else. Then we will parse the mapping from the source code to the RTL. On the other hand, if they want to hack with the tool (e.g. add additional board, specify extra floorplanning constraints), they are supposed to be somewhat familiar with the CAD flow and I believe clock regions are relatively basic concepts? Let me know if you disagree.
My plan is to leave the user interface aside for a while (3 months, before my next deadline), and if users (rare for now) run into problems I could provide direct help.
Best,
Licheng
Hi @Licheng-Guo ,
About the HLSParser package, it is nice to see that tapa frontend has been added. It is quite a good beginning.
About the Device package, my idea is to leave the concrete device information to plain-text files, e.g. yaml or csv. We may have further discussion about how to represent the 'branches' on the resource variations induced by composition of instantiations of IPs, e.g. DDR, NoC infrastructures, etc. What choices come to me at this early stage:
record different resource situation as you've done in the old code base, but this may be a problem when the compositions are of a terrifying number.let users pick the correct 'branch' in the resource file. (then supporting on comments can be a must for the file format)
About the Opt package, the components are still heavily coupled with the details of the proposed floorplanning algorithm. I'd like to start a new git branch to get them decoupled as I've mentioned in the last few e-mails. Hopefully I will keep the interfaces consistent (one exception is the coordination of floorplanner and slots, as the organizaton of slots may vary a lot depending on the floorplanner implementation).
About the Flow package, the Floorplan (optional) setion in def help seems to be confusing. It is really a problem for both of us to allow user to make partial assignment with a tidy interface without knowledge about the floorplanning implementation details on naming conventions.
This may be a problem if users want to connect vhls and autobridge directly as a whole workflow. (Some manual coding is not too big a realistic problem at this stage though).
The naming convention (the CLOCKREGION stuff) is not quite obvious even after I have read the Device package. This can be a real problem as normally user won't dive into the Opt package for namings (though the floorplanner itself and the constraint stuffs are there).
Maybe either
expose the namings to somewhere more obvious, orlet users bind to slot graphs and leave us to manage the name mappings. (It is not a choice if the slot graph is dynamically generated during floorplanning though. Several round trips between abstract coordinates and slot-during-iteration is otherwise needed.)
By the way, discussing in a closed issue seems not to be a correct way using GitHub. Maybe start a new issue ;-P
It is nice to be on a consistent track with you!
Best,
Jianwen
—You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or unsubscribe.

About the coupling, the dataflow graph contains vertical cuts and horizontal cuts, which may not fit other floorplanning algorithms.

Those are obsolete code, I will remove them. Right now I believe the data flow graph is independent of the floorplanning algorithm.

On the other hand, the pipeliner still consumes the cuts, which maybe better to be extended to consume slot paths.

How the pipeline registers distribute should be handled in global routing?

For my model, there can be:

Differently weighted connections through die/slots
NoCs introducing quite a different interconnection topology/distance-metrics among slots.

Do you have a motivating example for your proposal? For the NoC case, I think every segment of your NoC could be viewed as a vertex?

It sounds a strike to me that you've met a scability problem. My plan was to test my model with AutoBridge as a framework. As my model consists of integer-valued parameters and binary-valued solutions, I am now considering to use a SMT solver for my model (but still embed in AutoBridge).

Gurobi kind of conceals the scalability problem because it is too powerful, but many do not have access to the commercial tool.

Besides, I think it not quite ideal to keep splitting slots into equal parts when handling with strongly heterogeneous situations.

Theoretically yes, but you may need actual examples to justify your motivation (if you are targeting a paper)

To be frank, I am working on my undergraduate thesis. I really appreciate your advice on my proposal a lot as this is not part of AutoBridge yet.

Gurobi kind of conceals the scalability problem because it is too powerful, but many do not have access to the commercial tool.

I am now considering use z3 solver to handle my model (which only involves integers and binaries) in case gurobi is absent. As I have not tested my model in real usage cases, I am not quite sure about the outcomes in the resulting running time.

Besides, I think it not quite ideal to keep splitting slots into equal parts when handling with strongly heterogeneous situations.

Theoretically yes, but you may need actual examples to justify your motivation (if you are targeting a paper)

For the time being I can only give a rough thought experiment by combining extremely unbalanced components (If you feel this make sense, I will look for real cases.):

Assuming you have a large CPU connected with several small co-processors through FIFOs, which can still be viewed as 'dataflow graphs' we are handling. Keep splitting the grids may leave too few resources for the large core or constrain the small cores in too constraint an area at even the first cut.
The formulation of your iteratively-cut grid model requires that all the slot are cut. Keep cutting on the assigned slots yields potential resources shortage for the large core. In other words, even we stick to a grid model, we do not have to make all the slots in the grid equal, but an imbalanced grid does not fit your present optimization model.

My major motivation for introducing my model is to allow a flexible distance metric among slots.

In the case with a hardened but coarse NoC, distant dies may be connected through NoC links while nearby dies can still be considered to be connected with pipelined fabrics.

Vertices can be regions surrounded by NoC access points (I'm not sure about the terms :-P). Edges are NoC links or die-die paths.
In this case, as latency insensitive as if connected through FIFOs, the weight of an NoC link can similar to that of a pipelined path between adjacent dies, yielding a metric which cannot be achieved by measuring the Manhattan distance from a grid.
As my advisor has some interest on FPGA with NoC, this idea may be validated with some tool like VPR in the future, but this won't happen very soon.

I am also wondering what if the FPGA itself is in an imbalanced grid-like geometry. My thought is still mature enough :-(

Thanks a lot.

Hi Licheng,
I went over Opt package of the new code base, I must apologize that I heavily misunderstood the codes before. The in-develop code does yield a clean interface as we have discussed.
One of my remained problem is that the Opt package still assumes the topology to be a grid with equal slots. I am not sure though, I think it cannot deal with something like the following:
-----------
|         |
-----------
|    |    |
-----------

Thanks for your patience!
Best,
Jianwen

You could develop your own floorplan algorithm for non-uniform slot sizes. The Slot, Dataflow Graph and Floorplan are all decoupled. Though you may want to first have some concrete cases rather than high-level intuitions to justify your motivation.
GlobalRouting is still under development. Should be similar to traditional grid routing.
For hardened NoC like Versal, using the hardened link may have a performance impact due to the limited width of the bus. Taking performance into consideration seems very complicated and is beyond my plan. But it is an interesting problem worth thinking about.

authors: Licheng-Guo luojw-dwr

repo: Licheng-Guo/AutoBridge


text:
RBS-1395 Create Download File component

Changes
Create new Download File component

[keep | delete] branch

Changelog entry

[BREAKING | WARNING | remove if none] Description of change in Component-name

Additional information

You may refer to this ticket
Any other information you would like to include

hi, I don't have access to the ticket, is this Figma page the finalised specs?
https://www.figma.com/design/us90jOpWBahsqK2VAluPvD/Flagship-Design-System?node-id=8475-54069&t=zMk9HHBbKUbDmDcy-0
are there any requirements on how a list of items are to be displayed?
do include file-download.mdx for the stories with the corresponding prop table, and remove any unnecessary CSFs

hi, I don't have access to the ticket, is this Figma page the finalised specs? https://www.figma.com/design/us90jOpWBahsqK2VAluPvD/Flagship-Design-System?node-id=8475-54069&t=zMk9HHBbKUbDmDcy-0
are there any requirements on how a list is to be displayed?
do include file-download.mdx for the stories with the corresponding prop table, and remove any unnecessary CSFs

while updating the storybook documentation, I found a display issue for long file names without word breaks. could this be addressed?

repo: LifeSG/react-design-system


text:
The Artefacts are stored in the proper way

🐛 Bug

To Reproduce
You are connected to the local Lightning App.
📂 root
┣━━ 📂 artifacts
┃   ┗━━ 📂 drive
┃       ┗━━ 📂 code
┃           ┗━━ 📂 root.file_server
┃               ┗━━ 📄 thomas-0ed22042
┣━━ 📂 sweep_controller.sweeps.thomas-1fdc34f9.w_0.ws.0
┃   ┗━━ 📂 Users
┃       ┗━━ 📂 thomas
┃           ┗━━ 📂 Documents
┃               ┗━━ 📂 GitHub
┃                   ┗━━ 📂 LAI-lightning-hpo-App
┃                       ┗━━ 📂 wandb
┃                           ┗━━ 📂 lightning_logs
┃                               ┗━━ 📂 trial_0
┃                                   ┗━━ 📂 1z93ywba
┃                                       ┗━━ 📂 checkpoints
┃                                           ┗━━ 📄 epoch=0-step=20.ckpt
┣━━ 📂 w_0.ws.0
┃   ┗━━ 📂 Users
┃       ┗━━ 📂 thomas
┃           ┗━━ 📂 Documents
┃               ┗━━ 📂 GitHub
┃                   ┗━━ 📂 LAI-lightning-hpo-App
┃                       ┗━━ 📂 wandb
┃                           ┗━━ 📂 lightning_logs
┃                               ┗━━ 📂 Optimizing a Simple CNN over MNIST with Lightning HPO
┃                                   ┗━━ 📂 version_None
┃                                       ┗━━ 📂 checkpoints
┃                                           ┣━━ 📄 epoch=0-step=20-v7.ckpt
┃                                           ┗━━ 📄 epoch=1-step=40.ckpt
┣━━ 📂 w_0.ws.1
┃   ┗━━ 📂 Users
┃       ┗━━ 📂 thomas
┃           ┗━━ 📂 Documents
┃               ┗━━ 📂 GitHub
┃                   ┗━━ 📂 LAI-lightning-hpo-App
┃                       ┗━━ 📂 lightning_logs
┃                           ┣━━ 📂 version_11
┃                           ┃   ┗━━ 📂 checkpoints
┃                           ┃       ┗━━ 📄 epoch=0-step=20.ckpt
┃                           ┗━━ 📂 version_12
┃                               ┗━━ 📂 checkpoints
┃                                   ┗━━ 📄 epoch=1-step=40.ckpt
┣━━ 📂 w_1.ws.0
┃   ┗━━ 📂 Users
┃       ┗━━ 📂 thomas
┃           ┗━━ 📂 Documents
┃               ┗━━ 📂 GitHub
┃                   ┗━━ 📂 LAI-lightning-hpo-App
┃                       ┗━━ 📂 wandb
┃                           ┗━━ 📂 lightning_logs
┃                               ┗━━ 📂 Optimizing a Simple CNN over MNIST with Lightning HPO
┃                                   ┗━━ 📂 version_None
┃                                       ┗━━ 📂 checkpoints
┃                                           ┗━━ 📄 epoch=0-step=20-v9.ckpt
┗━━ 📂 w_1.ws.1
    ┗━━ 📂 Users
        ┗━━ 📂 thomas
            ┗━━ 📂 Documents
                ┗━━ 📂 GitHub
                    ┗━━ 📂 LAI-lightning-hpo-App
                        ┗━━ 📂 lightning_logs
                            ┗━━ 📂 version_13
                                ┗━━ 📂 checkpoints
                                    ┗━━ 📄 epoch=0-step=20.ckpt

Steps to reproduce the behavior:

Go to '...'
Run '....'
Scroll down to '....'
See error

Code sample

Expected behavior

Environment

PyTorch Version (e.g., 1.0):
OS (e.g., Linux):
How you installed PyTorch (conda, pip, source):
Build command you used (if compiling from source):
Python version:
CUDA/cuDNN version:
GPU models and configuration:
Any other relevant information:

Additional context

Fixed by https://github.com/Lightning-AI/lightning-hpo/pull/127

repo: Lightning-AI/lightning-hpo


text:
Refactor bucketing of AllReduce to be applied to a generic trace

🚀 Feature
Today bucketing of AllReduce operation is done when the following conditions are met:

disable_torch_autograd is set to False
https://github.com/Lightning-AI/lightning-thunder/blob/139cc2246b87f3e58d80b13897286a61d55826d8/thunder/__init__.py#L328
compile_data.fn.use_ddp is set to True
https://github.com/Lightning-AI/lightning-thunder/blob/139cc2246b87f3e58d80b13897286a61d55826d8/thunder/executors/torch_autograd.py#L280-L283

We need to move the bucketing transformation out of the "torch autograd" code path to be applied in the generic thunder.jit pipeline with no information about "forward" or "backward".
This transformation could be placed to happen before "transform_for_execution" as part of the "additional_transforms":
https://github.com/Lightning-AI/lightning-thunder/blob/139cc2246b87f3e58d80b13897286a61d55826d8/thunder/__init__.py#L608-L610
or as the first step inside "transform_for_execution".
Currently, in the torch_autograd.py file, the allreduce bucketing transform is placed right before the "transform_for_execution":
https://github.com/Lightning-AI/lightning-thunder/blob/139cc2246b87f3e58d80b13897286a61d55826d8/thunder/executors/torch_autograd.py#L283-L292
cc @carmocca @awaelchli @crcrpar

This transformation could be placed to happen before "transform_for_execution" as part of the "additional_transforms":

I'm not quite sure how this can be achieved other than removing the check of https://github.com/Lightning-AI/lightning-thunder/blob/139cc2246b87f3e58d80b13897286a61d55826d8/thunder/init.py#L580-L601

or as the first step inside "transform_for_execution".

this might be a bit easier than the other approach.

authors: IvanYashchuk crcrpar

repo: Lightning-AI/lightning-thunder


text:
Update tests that mock cuda availability and extend it for mps

Proposed refactor
We have tests that fail locally on MPS supported devices because their mocking of CUDA does not extend to MPS. After #13947 landed, the availability check actually works now and we need to update some tests. These are mainly the tests that have this combination of settings:

has accelerator="gpu" (which includes both cuda and mps)
has function mocks like torch.cuda.is_available as True or torch.cuda.device_count > 0.

An example of such a test:
https://github.com/Lightning-AI/lightning/blob/e6a8283e9cd9df53fb661c64bbf2037e1391a16d/tests/tests_pytorch/trainer/connectors/test_accelerator_connector.py#L245-L262
Motivation
Let MPS users run the tests locally (me).
Pitch
Extend the mocking to MPS where applicable. In some tests it may make sense to actually only test for cuda, in which case we should change accelerator="gpu" to "cuda". Some other tests may need to be skipped entirely on MPS.
Note that simply slapping a RunIf(min_cuda=x) on top of these tests is not an option, as for most of the tests we want to run them with mocks on the CPU.
Additional context
Rather do this sooner than later. Right now, I can't differentiate from tests failing because of bugs in my branch vs. them failing due to this.

If you enjoy Lightning, check out our other projects! ⚡

Metrics: Machine learning metrics for distributed, scalable PyTorch applications.

Lite: enables pure PyTorch users to scale their existing code on any kind of device while retaining full control over their own loops and optimization logic.

Flash: The fastest way to get a Lightning baseline! A collection of tasks for fast prototyping, baselining, fine-tuning, and solving problems with deep learning.

Bolts: Pretrained SOTA Deep Learning models, callbacks, and more for research and production with PyTorch Lightning and PyTorch.

Lightning Transformers: Flexible interface for high-performance research using SOTA Transformers leveraging PyTorch Lightning, Transformers, and Hydra.

cc @borda @akihironitta @justusschock

I'd like to help with this; I am on an M1 series mac and have some experience writing tests (for flash).

Thanks @JustinGoheen. Let me know if you have any questions.

apologies, I'm no longer able to work this.

authors: JustinGoheen awaelchli

repo: Lightning-AI/lightning


text:
Re-enable convenient FSDP configuration for wrapping and activation checkpointing

Description & Motivation
The pull request #18045 is about to change the way activation checkpointing gets configured in FSDP. Previously, https://github.com/Lightning-AI/lightning/issues/15825 enabled a convenient way to configure activation checkpointing without importing boilerplate code from PyTorch FSDP:
fabric = Fabric(strategy=FSDPStrategy(..., activation_checkpointing=MyTransformerBlock))

However, if #18045 it will now be necessary again to configure it like so:
from torch.distributed.fsdp.wrap import ModuleWrapPolicy

strategy = FSDPStrategy(activation_checkpointing_policy=ModuleWrapPolicy({MyTransformerBlock}))

or via a functional policy. This is now more boilerplate code than necessary, since Lightning could easily accept the class or list of classes directly and wrap it internally, which is what this issue proposes.
Pitch
Part 1
Allow passing the module classes to wrap directly to the argument:
strategy = FSDPStrategy(activation_checkpointing_policy=MyTransformerBlock)

Part 2
In addition,  #18045 also proposes to rename activation_checkpointing to activation_checkpointing_policy with the argument that the newly provided functionality is a policy and consistent with a similarly functioning argument auto_wrap_policy. Given my pitch above, and the fact that it is no longer necessary to explicitly construct a policy (it is optional), we could go for a simplified and convenient API like this:
strategy = FSDPStrategy(
    auto_wrap={MyTransformerBlock, ...}  # automatically converted to a policy under the hood
    activation_checkpointing_policy={MyTransformerBlock, ...}  # automatically converted to a policy under the hood
)

Alternatives
No response
Additional context
I strongly vote for this pitch because the purpose of Lightning is to make these tools accessible to as many users as possible, whether they are distributed training experts or not. Making complex features in PyTorch easily configurable is at the core of Lightning's philosophy :)
cc @borda @awaelchli @carmocca

I don't have a preference here. My only suggestion is that either both arguments are suffixed by _policy or none of them are.

authors: awaelchli carmocca


text:
Fix specification for minimum PyTorch version in requirements file

What does this PR do?
Fixes #15183
Before submitting

[x] Was this discussed/approved via a GitHub issue? (not for typos and docs)
[x] Did you read the contributor guideline, Pull Request section?
[x] Did you make sure your PR does only one thing, instead of bundling different changes together?
[x] Did you make sure to update the documentation with your changes? (if necessary)
[x] Did you write any new necessary tests? (not for typos and docs)
[x] Did you verify new and existing tests pass locally with your changes?
[x] Did you update the CHANGELOG? (not for typos, docs, test updates, or internal minor changes/refactorings)

PR review
Anyone in the community is free to review the PR once the tests have passed.
Before you start reviewing make sure you have read Review guidelines. In short, see the following bullet-list:

[x] Is this pull request ready for review? (if not, please submit in draft mode)
[x] Check that all items from Before submitting are resolved
[x] Make sure the title is self-explanatory and the description concisely explains the PR
[x] Add labels and milestones (and optionally projects) to the PR so it can be classified

Did you have fun?
I made sure I had fun coding 🙃
cc @borda @tchaton @rohitgr7

Thank you so much @awaelchli. I think it does it!

It would be nice to have it somehow not being introduced again, like a test or a comment. But that's certainly just a nice to have.

OK, I have updated the other pip issue with my testing. It seems that using 1.9.* also works.
My new requirements.txt now looks like this and it can install everything correctly:
--find-links https://download.pytorch.org/whl/torch_stable.html
torch==1.10.2+cu113

# runtime deps
hydra-core == 1.2.0
pytorch-lightning == 1.7.7
fairscale == 0.4.12

# dev deps
pytest == 7.1.3
black ==  22.10.0
isort == 5.10.1
flake8 == 5.0.4

It might still be good to not use * in >= expression though. Personally I think using * in == makes more sense than using it in >=.

Closing as per the above.
The .* will be gone anyways once we update to minimum PyTorch 1.10, it's only set for the reasons that Akihiro mentioned.

authors: awaelchli carmocca min-xu-ai


text:
Add support for encrypting / decrypting the chunks + API Keys

🚀 Feature

Motivation

Add support for encrypting & decrypting the chunks to enable users to consume the data outside anywhere safely.
The data would be decrypted in RAM.
Pitch

Alternatives

Additional context

Hi @tchaton,
I'm interested in learning more about this feature and exploring how I might be able to contribute.
Thanks for your help!

Hey @bhimrazy. Go for it.

Hey @bhimrazy,
I think we could have something like this.
optimize(

	encryption=Entryption(password="...", level="sample|chunk")

And we could offer other encryption format such as RSA, etc..

Thank you, @tchaton, for providing the examples. They are extremely helpful.
For the symmetric encryption option, I am considering using Fernet (symmetric encryption).
What are your thoughts on using Fernet for this purpose?

Hi @tchaton,
Just checking if we need to move the 'API Keys' part from this issue to a new one, as #219 got closed for handling the encryption/decryption.

authors: bhimrazy tchaton

repo: Lightning-AI/litdata


text:
ld.optimize stuck at 0 for large datasets

🐛 Bug
Running ld.optimize(...) get's stuck at 0 samples for large datasets.
To Reproduce
If you set size to 10000 it will work, but with larger 1000000 it just stuck.
from functools import partial
import litdata as ld

def get_data(dataset, index):
    return dataset[index]

def main():
    global dataset_hf
    size = 1000000
    dataset_hf = [{"x": x, "y": x**2} for x in range(size)]
    output_dir = "./lit_dataset"

    print(get_data(dataset_hf, 5))

    get_data_with_dataset = partial(get_data, dataset_hf)

    ld.optimize(
        fn=get_data_with_dataset,
        inputs=list(range(len(dataset_hf))),
        output_dir=output_dir,
        num_workers=4,
        chunk_bytes="64MB",
    )

if __name__ == "__main__":
    main()

Output
❯ python experiments/litdata_generation.py
{'x': 5, 'y': 25}
Create an account on https://lightning.ai/ to optimize your data faster using multiple nodes and large machines.
Storing the files under /Users/wpc/Projects/cfdx/too/lit_dataset
Setup started with fast_dev_run=False.
Setup finished in 0.011 seconds. Found 1000000 items to process.
Starting 4 workers with 1000000 items. The progress bar is only updated when a worker finishes.
Workers are ready ! Starting data processing...

Progress:   0%|                                                                                | 0/1000000 [00:00<?, ?it/s]

Expected behavior
Expected behaviour is to process data irrespectively of the size
Environment

PyTorch 2.4.0
MacOS M1
pip/poetry
Python 3.12.4

Additional context
I came across the issue when trying to convert large datasets.Dataset object. Simply nothing would happen. The dataset I have is much much bigger than 1M items which is not unusual.

The same problem exists with exampe in the readme if you increase number of samples. The below does not process too, no data is being written in files.
import numpy as np
from PIL import Image
import litdata as ld

def random_images(index):
    fake_images = Image.fromarray(np.random.randint(0, 256, (32, 32, 3), dtype=np.uint8))
    fake_labels = np.random.randint(10)

    # You can use any key:value pairs. Note that their types must not change between samples, and Python lists must
    # always contain the same number of elements with the same types.
    data = {"index": index, "image": fake_images, "class": fake_labels}

    return data

if __name__ == "__main__":
    # The optimize function writes data in an optimized format.
    ld.optimize(
        fn=random_images,                   # the function applied to each input
        inputs=list(range(1000000)),           # the inputs to the function (here it's a list of numbers)
        output_dir="my_optimized_dataset",  # optimized data is stored here
        num_workers=4,                      # The number of workers on the same machine
        chunk_bytes="64MB"                  # size of each chunk
    )

Hi @AugustDev , unfortunately, I'm not able to reproduce the issue. I used lightning ai studio to run the script and it works fine.
So, I don't expect there's something to be fixed. Can you please share your device specifications?
Also, it would help if you used num_workers = os.cpu_count().
ld.optimize(
        fn=random_images,                   # the function applied to each input
        inputs=list(range(1000000)),           # the inputs to the function (here it's a list of numbers)
        output_dir="my_optimized_dataset",  # optimized data is stored here
        num_workers=os.cpu_count(),                      # The number of workers on the same machine
        chunk_bytes="64MB"                  # size of each chunk
    )

This PR should help: https://github.com/Lightning-AI/litdata/pull/301.
The issue was that we were pushing 1M index in a multiprocessing queue and Mac is quite slow to do so compared to Linux. When this isn't required, I am using a mock queue.

authors: AugustDev deependujha tchaton


text:
Add whitespace after "Instruct:"

In generate/base.py, the output is
Instruct:Fix typos in the following sentence: Exampel input
Output: Example input.

which looks a bit ugly.

It looks like they have it though. Maybe it just got stripped via the line break above.

repo: Lightning-AI/litgpt


text:
chat/base.py: extend checkpoint_dir before accessing it

Hi there 👋
Inside auto_download_checkpoint the code also extends the path by prepending checkpoints/ if it's not provided.
Since currently it's done after check_file_size_on_cpu_and_warn, the command
litgpt chat google/gemma-2-9b-it

will fail.

The reason why auto-download is so far down there (compared to the other places) is that I had to move it below the LoRA merging, because otherwise it will download a model if you want to use LoRA weights.

I also didn't notice the missing CPU warning because I was running it on GPU 😅. The following reorg might work ...

I think this should be good now. Feel free to merge if you agree

There is a problem.
checkpoint_path is created before auto_download_checkpoint is called, which means that the checpoint_dir wasn't extended at the moment of creation of checkpoint_path.
So when we do load_checkpoint(fabric, model, checkpoint_path) it will throw an error (if the path doesn't contain checkpoints/).
This needs to be fixed and the test should be updated.

Are we good to merge? What do you think?

Yes, I think so. But one last question, after the update, have you double-checked / tested it on custom paths that don't start with `"checkpoints", like
litgpt chat ~/Desktop/google/gemma-2-9b-it

have you double-checked / tested it on custom paths that don't start with `"checkpoints"

No 😊.
Only just now checked with a Pythia model.
fix_chat_checkpoint_dir_extension ~/lit-gpt litgpt chat custom_dir/$repo_id
{'access_token': None,
 'checkpoint_dir': PosixPath('custom_dir/EleutherAI/pythia-70m'),
 'compile': False,
 'multiline': False,
 'precision': None,
 'quantize': None,
 'temperature': 0.8,
 'top_k': 200,
 'top_p': 1.0}
Now chatting with pythia-70m.
To exit, press 'Enter' on an empty prompt.

Seed set to 1234
>> Prompt: Hello
>> Reply: *
Time for inference: 0.28 sec total, 3.52 tokens/sec, 1 tokens

Nice, thanks for checking! Looks all good to me now :)

authors: Andrei-Aksionov rasbt


text:
TrainingEpochLoop._should_check_val_fx discrepancy between continued run  <> restore from ckpt

🐛 Bug
Found a discrepancy between a continued run after checkpointing, and restoring from checkpoint
Observation:
training_batch / val_loop ordering upon checkpoint restoration not the same as original run after checkpoint saving.
There are still the same amount of train steps, but the validation loops are interleaved at a single step later, which can cause the restored run to end up with one less validation loop (see colab)
Assumption / expectation:
Zero difference between a training run after a checkpoint and a run continued from said checkpoint
Investigation so far:
Im new to some of this lightning code, but IIUC:
Key:
TrainingEpochLoop's self.batch_progress.increment_completed() is called after "on_train_batch_end" hooks, the latter kicking off checkpoint saving.

upon restoring, the TrainingEpochLoop.batch_progress.current.reset_on_restart() will reset the ready back to completed
yet the global_step, which refers to TrainingEpochLoop.batch_loop.optimizer_loop.optim_progress.optimizer.step.total.completed, has been increment_completed() (called within TrainingEpochLoop.batch_loop.run) and thus upon restoring, ..optimizer.step.total.ready is set to an up to date optimizer.step.total.completed, out of sync with the above
[simplification] in "val_check_interval mode", validation is triggered when TrainingEpochLoop.batch_progress.current.ready % val_check_interval == 0 (through TrainingEpochLoop.on_advance_end -> TrainingEpochLoop._should_check_val_fx
combining the above three, the same batch_progress.current ready/completed counter for the continued and restored runs, end up aligned with different global_steps, and hence validation triggers at different global_steps

Another observation:
The following if statement seems to allow for a zero-difference restart, except that just like 4. above, _should_check_val_fx wouldnt trigger where in the original run on the checkpointing step it did (although there called in on_advance_end). Not sure if the original intention of this snippet included the current scope
class TrainingEpochLoop(loops.Loop[_OUTPUTS_TYPE]):
    ...
    def advance(self, data_fetcher: AbstractDataFetcher) -> None:  # type: ignore[override]
        ...
        if self.restarting and self._should_check_val_fx(self.batch_idx, self.batch_progress.is_last_batch):
            # skip training and run validation in `on_advance_end`
            return

PR's relevant to this line:

#9681
#9563

Potential impact:
Assuming not too worrisome for the more default Lightning use cases:

With val_check_interval >> 3 (colab example = 3), or that turned off relying instead on check_val_every_n_epoch

However, in theory it can influence all of the following:

no 1:1 deterministic reproducibility
affect the latest/best validation loss

affects any code flow / decision making based on that

cause a "different usage order" of rngs (<- how I initially caught the issue, even with correctly restored rng states, if both validation and training steps use one theyll each end up with different random numbers as compared to the continued run)
other

To Reproduce

customized google colab bug_report_model.ipynb with same observation on BoringModel
Expected behavior
Zero difference between a training run continued after a checkpoint and a run continued from said checkpoint
Environment

Note:

The below is from original investigation in our own code base, with pytorch lightning v1.6.4.
The environment details from the BoringModel's reproduction are listed in the colab, with pytorch lighting v1.7.4
I also browsed through the master branch last weeks and the relevant code seems unchanged

Details

CUDA:

GPU:

NVIDIA RTX A4000
NVIDIA RTX A4000
NVIDIA RTX A4000
NVIDIA RTX A4000

available:         True
version:           11.0

Lightning:

efficientnet-pytorch: 0.7.1
pytorch-lightning: 1.6.4
torch:             1.11.0.post1103
torchmetrics:      0.7.0
torchvision:       0.12.0a1110.post1103

Packages:

absl-py:           0.15.0
adal:              1.2.7
adlfs:             2021.10.0
aiohttp:           3.7.4
applicationinsights: 0.11.10
argcomplete:       1.12.3
async-timeout:     3.0.1
attrdict:          2.0.0
attrs:             21.1.0
av:                8.0.3
azure-cli-core:    2.38.0
azure-cli-telemetry: 1.0.6
azure-common:      1.1.27
azure-core:        1.20.0
azure-datalake-store: 0.0.52
azure-identity:    1.10.0
azure-keyvault-secrets: 4.2.0
azure-mgmt-core:   1.2.2
azure-storage-blob: 12.11.0
backcall:          0.2.0
backoff:           1.10.0
bcrypt:            3.2.0
cachetools:        4.2.2
certifi:           2020.12.5
cffi:              1.14.5
chardet:           3.0.4
charset-normalizer: 2.0.12
click:             7.1.2
confluent-kafka:   1.7.0
cryptography:      3.4.8
cycler:            0.10.0
datadog:           0.44.0
decorator:         5.0.7
deepdiff:          5.5.0
deltalake:         0.5.8
docker-pycreds:    0.4.0
efficientnet-pytorch: 0.7.1
einops:            0.4.1
filelock:          3.7.1
fonttools:         4.37.1
frozendict:        2.3.2
fsspec:            2022.1.0
gitdb:             4.0.7
gitpython:         3.1.14
google-auth:       1.30.0
google-auth-oauthlib: 0.4.4
grpcio:            1.37.1
htmlmin:           0.1.12
humanfriendly:     10.0
idna:              2.10
imagehash:         4.2.1
inplace-abn:       1.1.0a1110.post1103
ipdb:              0.13.9
ipython:           7.23.1
isodate:           0.6.0
jedi:              0.18.0
jinja2:            3.1.2
jmespath:          0.10.0
joblib:            1.0.1
kafka-python:      2.0.2
kiwisolver:        1.3.1
knack:             0.9.0
markdown:          3.3.4
markupsafe:        2.0.1
matplotlib:        3.5.3
matplotlib-inline: 0.1.2
methodtools:       0.1.2
missingno:         0.5.0
msal:              1.16.0
msal-extensions:   0.3.0
msrest:            0.6.21
msrestazure:       0.6.4
multidict:         5.1.0
multimethod:       1.6
networkx:          2.5.1
numpy:             1.22.4
oauthlib:          3.1.0
opencv-python:     4.4.0.44
ordered-set:       4.0.2
packaging:         21.3
pandas:            1.4.3
pandas-profiling:  3.1.0
paramiko:          2.7.2
parso:             0.8.2
pathtools:         0.1.2
pexpect:           4.8.0
phik:              0.12.0
pickleshare:       0.7.5
pillow:            9.2.0
pip:               22.0.3
pkginfo:           1.7.0
polyline:          1.4.0
portalocker:       1.7.1
prometheus-client: 0.8.0
promise:           2.3
prompt-toolkit:    2.0.10
protobuf:          3.15.8
psutil:            5.9.1
psycopg2:          2.8.3
ptyprocess:        0.7.0
py:                1.10.0
py3nvml:           0.2.7
pyarrow:           9.0.0
pyasn1:            0.4.8
pyasn1-modules:    0.2.8
pycparser:         2.20
pydantic:          1.8.2
pydeprecate:       0.3.1
pygame:            2.1.2
pygments:          2.9.0
pyjwt:             1.7.1
pynacl:            1.4.0
pyntcloud:         0.1.6
pyopenssl:         20.0.1
pyparsing:         2.4.7
pyquaternion:      0.9.9
pysocks:           1.7.1
python-dateutil:   2.8.2
python-json-logger: 2.0.2
pytorch-lightning: 1.6.4
pytz:              2022.1
pywavelets:        1.1.1
pyyaml:            6.0
qrcode:            6.1
requests:          2.27.1
requests-oauthlib: 1.3.0
retry:             0.9.2
rsa:               4.7.2
runai:             0.3.0
scipy:             1.6.2
seaborn:           0.11.2
semver:            2.13.0
sentry-sdk:        1.9.4
setproctitle:      1.2.2
setuptools:        59.5.0
shapely:           1.8.0
shortuuid:         1.0.1
simplejpeg:        1.4.1
six:               1.16.0
slackclient:       2.9.4
smmap:             4.0.0
sqlalchemy:        1.3.24
tabulate:          0.8.9
tangled-up-in-unicode: 0.1.0
tensorboard:       2.6.0
tensorboard-data-server: 0.6.1
tensorboard-plugin-wit: 1.8.0
timm:              0.4.5
toml:              0.10.2
torch:             1.11.0.post1103
torchmetrics:      0.7.0
torchvision:       0.12.0a1110.post1103
tqdm:              4.60.0
traitlets:         5.3.0
transforms3d:      0.3.1
typing-extensions: 4.1.1
urllib3:           1.26.11
visions:           0.7.4
wandb:             0.12.14
wcwidth:           0.2.5
werkzeug:          1.0.1
wheel:             0.36.2
wirerope:          0.3.1
wrapt:             1.14.1
xmltodict:         0.12.0
xxhash:            1.4.1
yarl:              1.6.3

System:

OS:                Linux
architecture:

64bit
ELF

processor:         x86_64
python:            3.8.12
version:           /#138~18.04.1-Ubuntu SMP Fri Jun 24 14:14:03 UTC 2022

Additional context

cc @awaelchli @ananthsub @ninginthecloud @rohitgr7 @otaj @carmocca @justusschock

I'd be happy to help fix if agreed this is a bug, but like the opinion + take on a solution from folks more involved with pytorch lightning.
Some simple ideas, but none of them great:

Give the model checkpoint callback a special status compared to others, call it at the end of training_epoch_loop.on_advance_end

Sounds like this really shouldnt be required + makes codes fragile/ harder to maintain

have a relevant hook called at the beginning of training_epoch_loop.on_advance_end that checkpoints

why would we need yet another hook? And if we move one, I assume any hooks location in code flow was well thought out already

call on_train_batch_end after the increment_completed

definition based: does on_train_batch_end mean it wont change training module anymore and could be run after the increment? Is there logic in the on_train_batch_end that expects "completed" not to be incremented yet?

Hi, @Anner-deJong - Thank you for creating this issue, and helping with all the context around this. Probably @awaelchli or @carmocca can help you on this one.

Getting back to this issue, as it came up recently in a different context. This is definitely a behavior that needs fixing.
My take would be to not add an extra hook, but invoke increment_completed prior to on_train_batch_end. Which one comes before is very debatable from a definition standpoint. Having increments already set up correctly when I'm in my on_train_batch_end hook is a fair expectation IMO.
The sequence would go from:
self.batch_progress.increment_ready()
on_train_batch_start
self.batch_progress.increment_started()
...
self.batch_progress.increment_processed()
...
on_train_batch_end
self.batch_progress.increment_completed()

to
self.batch_progress.increment_ready()
on_train_batch_start
self.batch_progress.increment_started()
...
self.batch_progress.increment_processed()
...
self.batch_progress.increment_completed()
on_train_batch_end

Not sure what do think about the started part, but that's for another time.

On further thought, I'll take a more conservative approach since the implications are pretty wide.

authors: Anner-deJong krshrimali lantiga

repo: Lightning-AI/pytorch-lightning


text:
Other rules are not being checked when customizing one of the rules

I have created following flow that is violating a bunch of Flow Scanner rules:
<?xml version="1.0" encoding="UTF-8"?>
<Flow xmlns="http://soap.sforce.com/2006/04/metadata">
    <apiVersion>49.0</apiVersion>
    <environments>Default</environments>
    <interviewLabel>BadFlow {!$Flow.CurrentDateTime}</interviewLabel>
    BadFlow
    <loops>
        <name>Loop_through_Contacts</name>
        Loop through Contacts
        <locationX>176</locationX>
        <locationY>431</locationY>
        <collectionReference>Get_Contacts</collectionReference>
        <iterationOrder>Asc</iterationOrder>
        <nextValueConnector>
            <targetReference>Update_Contact</targetReference>
        </nextValueConnector>
    </loops>
    <processMetadataValues>
        <name>BuilderType</name>
        <value>
            <stringValue>LightningFlowBuilder</stringValue>
        </value>
    </processMetadataValues>
    <processMetadataValues>
        <name>CanvasMode</name>
        <value>
            <stringValue>AUTO_LAYOUT_CANVAS</stringValue>
        </value>
    </processMetadataValues>
    <processMetadataValues>
        <name>OriginBuilderType</name>
        <value>
            <stringValue>LightningFlowBuilder</stringValue>
        </value>
    </processMetadataValues>
    <processType>AutoLaunchedFlow</processType>
    <recordLookups>
        <name>Get_Contacts</name>
        Get Contacts
        <locationX>176</locationX>
        <locationY>323</locationY>
        <assignNullValuesIfNoRecordsFound>false</assignNullValuesIfNoRecordsFound>
        <connector>
            <targetReference>Loop_through_Contacts</targetReference>
        </connector>
        <filterLogic>and</filterLogic>
        <filters>
            <field>AccountId</field>
            <operator>EqualTo</operator>
            <value>
                <elementReference>$Record.Id</elementReference>
            </value>
        </filters>
        <getFirstRecordOnly>false</getFirstRecordOnly>
        <object>Contact</object>
        <storeOutputAutomatically>true</storeOutputAutomatically>
    </recordLookups>
    <recordUpdates>
        <name>Copy_1_of_Update_Contact</name>
        Copy 1 of Update Contact
        <locationX>264</locationX>
        <locationY>647</locationY>
        <connector>
            <targetReference>Loop_through_Contacts</targetReference>
        </connector>
        <inputReference>Loop_through_Contacts</inputReference>
    </recordUpdates>
    <recordUpdates>
        <name>Update_Contact</name>
        Update Contact
        <locationX>264</locationX>
        <locationY>539</locationY>
        <connector>
            <targetReference>Copy_1_of_Update_Contact</targetReference>
        </connector>
        <inputReference>Loop_through_Contacts</inputReference>
    </recordUpdates>
    <start>
        <locationX>50</locationX>
        <locationY>0</locationY>
        <connector>
            <targetReference>Get_Contacts</targetReference>
        </connector>
        <object>Account</object>
        <recordTriggerType>Create</recordTriggerType>
        <triggerType>RecordAfterSave</triggerType>
    </start>
    <status>Draft</status>
</Flow>

When I run sf flow scan command without any additional parameters and with no .flow-scanner.json file defined (so I am using default rules only) I am getting following output:
=== Flow: BadFlow (9 results)
Type: AutoLaunchedFlow

RULE                      TYPE           NAME
────────────────────────  ─────────────  ────────────────────────
Copy API Name             recordUpdates  Copy_1_of_Update_Contact
DML Statement In A Loop   recordUpdates  Update_Contact
DML Statement In A Loop   recordUpdates  Copy_1_of_Update_Contact
Missing Flow Description  description    undefined
Flow Naming Convention    name           BadFlow
Missing Fault Path        recordLookups  Get_Contacts
Missing Fault Path        recordUpdates  Update_Contact
Missing Fault Path        recordUpdates  Copy_1_of_Update_Contact
Missing Null Handler      recordLookups  Get_Contacts

This looks as expected. However, if I run the Flow with my custom config like this:
{
    "rules": {
        "FlowName": {
            "severity": "error",
            "expression": "[A-Za-z0-9]+_(SCR|SFL|SCH|EVT|EML|BeforeCreate|BeforeUpdate|BeforeDelete|BeforeCreateBeforeUpdate|AfterCreate|AfterUpdate|AfterDelete|AfterCreateAfterUpdate)_[A-Za-z0-9]+"
        }
    }
}

Where I only update what should be the pattern for the Flow Naming Convention, then if I run sf flow scan command again, I am getting following output:
=== Flow: BadFlow (1 results)
Type: AutoLaunchedFlow

RULE                    TYPE  NAME
──────────────────────  ────  ───────
Flow Naming Convention  name  BadFlow

So all of the other rules, that I have not defined in my flow-scanner.json file (because I don't want to override them) are not being checked. Not sure if it is working this way as designed, but this is definitely because of the logic in GetRuleDefinitions where custom config does not use DefaultRuleStore. I think ideally selectedRules should be initialized with default rules and then custom config should be processed and provided configuration should be overidden.

Thank you for this detailed feedback. The core module is indeed intended to execute all default rules when no rules are explicitly specified. Your suggestion for initializing selectedRules with default rules and then processing custom configurations to override them sounds like a sensible approach and could enhance the functionality of the tool.
In the current setup, users can specify rules without providing an "Expression" attribute value, effectively avoiding any unintended overrides of default rules. While this workaround exists, implementing your proposed enhancement could streamline the configuration process and ensure a more intuitive user experience.

I guess this is the matter of how this plugin should be used. I can list all of the rules in my configuration and not provide any additional details like this:
{
    "rules": {
        "APIVersion": {},
        "CopyAPIName": {},
        "DMLStatementInLoop": {},
        "DuplicateDMLOperation": {},
        "FlowDescription": {},
        "HardcodedId": {},
        "MissingFaultPath": {},
        "MissingNullHandler": {},
        "SOQLQueryInLoop": {},
        "UnconnectedElement": {},
        "UnusedVariable": {},
        "FlowName": {
            "severity": "error",
            "expression": "[A-Za-z0-9]+_(SCR|SFL|SCH|EVT|EML|BeforeCreate|BeforeUpdate|BeforeDelete|BeforeCreateBeforeUpdate|AfterCreate|AfterUpdate|AfterDelete|AfterCreateAfterUpdate)_[A-Za-z0-9]+"
        }
    }
}

And that will do the trick for me. However it is a little unintuitive for me that when I want to override a single rule, I need to specify all of them. I was thinking more of how configuration works in eslint where you extend default configuration and then you can provide your own overrides and keep default / recommended configuration. Current approach in lightning-flow-scanner is more similar to PMD where you just list all of the rules you want to use with additional configuration if necessary, but if you don't list anything, it will use default configuration.
I guess this is where the decision has to be made on what is expected and how it should be used, and documentation should be updated to mention these things.

I think your suggestion makes a lot of sense going forward when introducing more rules and custom rules. But as is, we are not there yet, so please bear with us! If you feel comfortable working this out further and we'd be more than happy to connect and take your help on making this more intuitive!

@filiprafalowicz that's a good suggestion, usually linters have a set of default rules, and the local configuration file is "merged" in this default set of rules, meaning that the day new rules appear, you don't have to update your locally defined config to benefit of them :)

@filiprafalowicz We are definitely keeping this suggestion in mind. I am however closing it as this is technically a relatively big undertaking for which personally I dont have time given the other issues. I will reopen this once we find the resources to work on it. I hope you understand that for now, it is what it is=)

@filiprafalowicz We are definitely keeping this suggestion in mind. I am however closing it as this is technically a relatively big undertaking for which personally I dont have time given the other issues. I will reopen this once we find the resources to work on it. I hope you understand that for now, it is what it is=)

authors: RubenHalman filiprafalowicz nvuillam


text:
🛑 LinerlyBot (Discord) is down

In ba6344e, LinerlyBot (Discord) (https://linerlybot-discord.linerly.tk) was down:

HTTP code: 0
Response time: 0 ms

Resolved: LinerlyBot (Discord) is back up in 987fb9f.

repo: Linerly/status


text:
🛑 SearXNG is down

In 8863809, SearXNG (https://searxng.linerly.tk) was down:

HTTP code: 0
Response time: 0 ms

Resolved: SearXNG is back up in dca4ed6.


text:
🛑 Nextcloud is down

In 2022c49, Nextcloud (https://cloud.linerly.xyz) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Nextcloud is back up in 096f598 after 19 days, 14 hours, 36 minutes.


text:
🛑 Nextcloud is down

In 8358ec0, Nextcloud (https://cloud.linerly.xyz) was down:

HTTP code: 503
Response time: 6176 ms

Resolved: Nextcloud is back up in e16196d after 2 hours, 50 minutes.


text:
openssl: libcurl.so.4: undefined symbol: SSL_CTX_set_alpn_protos

OS: Ubuntu 14.04 64
brew:
Homebrew 1.2.4-2-g59a338502
Homebrew/homebrew-core (git revision 7d09d; last commit 2017-07-19)

Updating Homebrew...
==> Auto-updated Homebrew!
Updated 2 taps (caskroom/cask, homebrew/core).
==> Updated Formulae
expat ✔

==> Upgrading 1 outdated package, with result:
openssl 1.0.2l
==> Upgrading openssl
/home/tiankai/.linuxbrew/opt/curl/bin/curl: symbol lookup error: /home/tiankai/.linuxbrew/lib/libcurl.so.4: undefined symbol: SSL_CTX_set_alpn_protos
/home/tiankai/.linuxbrew/opt/curl/bin/curl: symbol lookup error: /home/tiankai/.linuxbrew/lib/libcurl.so.4: undefined symbol: SSL_CTX_set_alpn_protos
==> Downloading https://www.openssl.org/source/openssl-1.0.2l.tar.gz
/home/tiankai/.linuxbrew/opt/curl/bin/curl: symbol lookup error: /home/tiankai/.linuxbrew/lib/libcurl.so.4: undefined symbol: SSL_CTX_set_alpn_protos
Trying a mirror...
==> Downloading https://dl.bintray.com/homebrew/mirror/openssl-1.0.2l.tar.gz
/home/tiankai/.linuxbrew/opt/curl/bin/curl: symbol lookup error: /home/tiankai/.linuxbrew/lib/libcurl.so.4: undefined symbol: SSL_CTX_set_alpn_protos
Trying a mirror...
==> Downloading https://www.mirrorservice.org/sites/ftp.openssl.org/source/openssl-1.0.2l.tar.gz
/home/tiankai/.linuxbrew/opt/curl/bin/curl: symbol lookup error: /home/tiankai/.linuxbrew/lib/libcurl.so.4: undefined symbol: SSL_CTX_set_alpn_protos
Error: Failed to download resource "openssl"
Download failed: https://www.mirrorservice.org/sites/ftp.openssl.org/source/openssl-1.0.2l.tar.gz

Please enclose copy-and-paste blocks in triple back ticks.
See https://help.github.com/articles/basic-writing-and-formatting-syntax/#quoting-code

Please report the output of brew gist-logs openssl
Please check the output of brew doctor and heed its warnings.
Please ensure that the environment variable LD_LIBRARY_PATH is not set.
unset LD_LIBRARY_PATH

Try…
brew fetch openssl
brew upgrade openssl

@sjackman Thank you! I solved the problem according to your method!

mark

authors: gttiankai qc1iu sjackman

repo: Linuxbrew/brew


text:
Did you mean?  prepend_path_in_profile

Please note that we will close your issue without comment if you delete, do not read or do not fill out the issue checklist below and provide ALL the requested information. If you repeatedly fail to use the issue template, we will block you from ever submitting issues to Homebrew again.

[ ] ran a brew command and reproduced the problem with multiple formulae? If it's a problem with a single, official formula (not cask) please file this issue at Linuxbrew/homebrew-core: https://github.com/Linuxbrew/homebrew-core/issues/new/choose. If it's a brew cask problem please file this issue at https://github.com/Homebrew/homebrew-cask/issues/new/choose. If it's a tap (e.g. Homebrew/homebrew-php) problem please file this issue at the tap.
[X] ran brew update and can still reproduce the problem?
[X] ran brew doctor, fixed all issues and can still reproduce the problem?
[X] ran brew config and brew doctor and included their output with your issue?

What you were trying to do (and why)
Due to the fact, 'brew' command, after re-login, was not recognized in my ENV after successful installation of LinuxBrew, I tried/hacked to get it working that way using an additional .bash_profile file to export related paths. (See file in archive)
What happened (include command output)
After executing twice 'brew update' and after that, 'brew doctor', the following output on terminal was shown:

Command output

vagrant@StretcH:~$ brew update
Already up-to-date.
vagrant@StretcH:~$ brew update
Already up-to-date.
vagrant@StretcH:~$ brew doctor
Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry or file an issue; just ignore this. Thanks!
Warning: You have HOMEBREW_BUILD_FROM_SOURCE set. This environment variable is
intended for use by Homebrew developers. If you are encountering errors,
please try unsetting this. Please do not file issues if you encounter
errors when using this environment variable.
Error: undefined method prepend_variable_in_profile' for Utils::Shell:Module Did you mean?  prepend_path_in_profile Please report this bug:   https://github.com/Linuxbrew/brew/wiki/troubleshooting /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/extend/os/linux/diagnostic.rb:48:in check_xdg_data_dirs'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/cmd/doctor.rb:69:in block in doctor' /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/cmd/doctor.rb:61:in each'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/cmd/doctor.rb:61:in doctor' /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/brew.rb:89:in '
vagrant@StretcH:~$

What you expected to happen
Well, no error message after executing 'brew doctor'.
Step-by-step reproduction instructions (by running brew commands)
'brew update'; executed twice as requested, after that:
'brew doctor' to check my LinuxBrew environment.
Following my bash files: (echo's are for my orientation on execution chronology)
brew_bug.tar.gz

Can you please include output of brew config, thanks.

Something like the following untested fix may work.
-#{Utils::Shell.prepend_variable_in_profile("XDG_DATA_DIRS", HOMEBREW_PREFIX/"share")}
+#{Utils::Shell.set_variable_in_profile("XDG_DATA_DIRS", HOMEBREW_PREFIX/"share:$PATH")}

@jonchang:
Hello Jonathan
Well, wasn't it in the archive? So, sure, here you are:
brew_config.txt
Thank you for your support.
@sjackman:
Hello Shaun
Thank you for your support by reporting a possible workaround. Well, since I am not really a SW-Dev and very unexperienced on SW-Dev steps/tools; where do I have to add your mentioned workaround?
Thanks for your reply.
Regards to you both,
Roger

This bug should hopefully be resolved by PR https://github.com/Linuxbrew/brew/pull/895 once it's merged.

This issue should be resolved. Run brew update

authors: RogerWeihrauch jonchang sjackman


text:
Unmarshal: invalid character ’.’ looking for beginning of value

using the downloader with internet connection from EU causes a "Unmarshal: invalid character ’.’ looking for beginning of value" and the program is unable to download any files
works fine with connection from USA

Do I have to use a VPN to change my connection to USA?

Yes, I think a connection anywhere that's not EU will work.

repo: Liru/tumblr-downloader


text:
Add video downloading functionality

Currently, only photo downloading is supported. Video downloading has a more odd structure. Parse the response of the API to find the video source and download it.

Very interested to have that since pretty much no Tumblr downloaders cater to video.

authors: Liru apoapostolov


text:
15 Critical vulnerabilities found

The application has 15 critical and 4 high vulnerabilities:
55 vulnerabilities found - Packages audited: 2796
Severity: 8 Low | 28 Moderate | 4 High | 15 Critical

Consider using dependabot.

Interesting, can you ensure that the vulnerabilities found (especially those with a high and critical level of severity) can be exploited, or are you just throwing this source code into the scanner and not understanding the results displayed?

Have checked each, and none of those are exploitable.
Am using Snyk (similar to Dependabot)

authors: KostLinux Lissy93 dshtmr


text:
SDK for Huawei Watch GT

Hi!
Is the LiteOS version in the Huawei Watch GT available to external developers? Is there a SDK for developers to build apps or customise the software for the Watch GT?
Thanks.

I notice there's no answer yet. You guys have some nice hardware but it's worthless without software. There's probably a few of us quite interested in developing 3rd party watch faces, apps and integrations but we can't do that without any docs, tools or SDK's. How about a simple guide in English covering: how to get hold of the firmware and get it running in QEMU and then how to package it up and deploy changes to the watch ? We can then build the apps whilst you're working out how to distribute them to end users.

I agree, I want to develop stuff for my watch, kinda disappointed in the software of the Watch GT...
Not even a possibility for developer. Sadly I already read many comment of people who are disappointed in the software :/

An SDK for such a watch is unrelated to the kernel the watch uses.
You can likely build apps for Lite OS by targeting this kernel, but the facility to target the GUI and deploy to the watch is unrelated to LiteOS.
As such, this issue should be closed.

Any idea if there will be a way of developing stuff for Huawei Watch GT 2 then?

I want to build an application for me 😥😣

pls post your question in https://gitee.com/LiteOS/LiteOS

I know why they dont answer. They dont know english 🤣

They don't answer because it is already answered. This is about the OS (LiteOS), which already allows apps. Not the watches, which have no SDK or method of deploying apps.

SDK is coming December 12th AFAIK. Here is the docs:
https://developer.harmonyos.com/en/docs/documentation/doc-guides/wearable-developing-0000001052265837

@xsacha This is related to HarmonyOS and not LiteOS. The users (developers) of Huawei Watch GT2, GT2e & GT2 Pro cannot be benefited. The reason being no USB or WLAN connectivity options. The only possible way is via Bluetooth. I believe that LiteOS is in its early stages. We have to wait this one out.

@xsacha This is related to HarmonyOS and not LiteOS. The users (developers) of Huawei Watch GT2, GT2e & GT2 Pro cannot be benefited. The reason being no USB or WLAN connectivity options. The only possible way is via Bluetooth. I believe that LiteOS is in its early stages. We have to wait this one out.

@manoj997 You'll find that HarmonyOS uses LiteOS (the kernel). https://github.com/Awesome-HarmonyOS/HarmonyOS
As far as I know, the new apps SDK is a streamed system from the phone OS. The apps do not run natively on the watch.

@manoj997 You'll find that HarmonyOS uses LiteOS (the kernel). https://github.com/Awesome-HarmonyOS/HarmonyOS
As far as I know, the new apps SDK is a streamed system from the phone OS. The apps do not run natively on the watch.

@xsacha
So is it possible to somehow run any custom app? If so, how (steps??)

@xsacha
So is it possible to somehow run any custom app? If so, how (steps??)

@manoj997 no, of course not :P Although it looks like the GT2 Pro also does not run any apps. It just streams them using an EMUI feature. It requires constant connection to an EMUI-running phone.

@manoj997 no, of course not :P Although it looks like the GT2 Pro also does not run any apps. It just streams them using an EMUI feature. It requires constant connection to an EMUI-running phone.

@xsacha
I own a Huawei Watch GT2e. LiteOS. This watch is capable of ton of things. I can customize apps and install. Sadly... It's not possible. Any workarounds for this?

@xsacha
I own a Huawei Watch GT2e. LiteOS. This watch is capable of ton of things. I can customize apps and install. Sadly... It's not possible. Any workarounds for this?

@xsacha
I own a Huawei Watch GT2e. LiteOS. This watch is capable of ton of things. I can customize apps and install. Sadly... It's not possible. Any workarounds for this?

@xsacha
I own a Huawei Watch GT2e. LiteOS. This watch is capable of ton of things. I can customize apps and install. Sadly... It's not possible. Any workarounds for this?

Any update? I've just bought gt2 pro and need to write an app for the watch

Any update? I've just bought gt2 pro and need to write an app for the watch

@nuplo
Not yet. :(

@nuplo
Not yet. :(

Apps are developed using huawei wear engine. Apparently you need an enterprise developer account.
https://developer.huawei.com/consumer/en/hms/huawei-wearengine/

Update: My Huawei GT2e got the firmware update 11.0.14.26 (216 MB) and in the changelog, it says it is possible to install 3rd party apps.
However, there are some limitations.

I have a Huawei Nova 3i with Android 9.1 and EMUI 9.0 and in the Huawei health (12.0.12.310) there is no option for installing apps yet.
Even if you get the install app option for other watches, there is only a handful of apps you can install and it means that you cannot build your own custom app, publish it to AppGallery and install it into your watch.
With every potential firmware update, the noticable changes are UI changes such as icon colors and updates to weather. Other than that, you can improve things like Support for Emojis, Support for app icons from the device. (It does not even recognize the Huawei Health App icon), Option to keep notifications even if they're viewed and possiblity to apply quick replies, Answer a call if a earbud is linked to the watch and transfer the call to the watch (This is mainly GT2e), Accuracy of compass and barometer readings, Ability to add custom always on watch faces.

Dear LiteOS devs, please take a note of this and try to improve it for the betterment.

I just got the same update.
You might need a newer app gallery

installed AppGallery HUAWEI AppGallery 11.6.1.200_beta and restarted the phone. Still the same :(

authors: NibbleOfAnOctet RafaelPuerta alduder azw413 blchinezu manoj997 markyin nuplo vipyoshi wlad xsacha

repo: LiteOS/LiteOS


text:
deleteAsync无效

使用 DataSupport.deleteAsync(Food::class.java, 5)方法不能删除数据，
delete方法可以成功删除
系统 MIUI9 android 7.1.1  和android5.0.2都不行

所以Async方法都要后接listen才会执行，要不然是不会执行的。你可以不做具体的回调处理，但是必须得写。
DataSupport.deleteAsync(Food::classs.java, 5).listen(null)

authors: guolindev imxy

repo: LitePalFramework/LitePal


text:
New feature Proposal : User login and logout

Child of #17
What i plan to do is

using local strategy of passport i will authenticate the username and password that the user is entering
If it is valid details then i will redirect the page to user profile and if its a failure then i will redirect it again to /login page
for logout i will use req.logout() and will redirect it to the home page

You'll have to setup passport for this project.

yes i will do it

updates ?

@YashKumarVerma I have sent the PR .
Should i delete the previous PR i had sent?

Ensure that only one active PR is there for one issue.

@YashKumarVerma  Okay i will keep that in mind

repo: LockedUp-Coders/nostalgify


text:
feat: Create CSV Export with lock "short names" instead of make & mod…

feat: Create CSV Export with lock "short names" instead of make & model data
Added handler(handleExportCsvShortNames) for exporting CSV with the short name (e.g. "ASSA Twin Combi, Triton, Neptun 4900 / TrioVing System 10, Twin Control") instead of the raw make and model data.
lpubeltsdata-new.csv
lpubeltsdata-old.csv

Came up with better solution

repo: Lockpickers-United/lpu-belt-explorer

license: Unlicense


text:
🛑 LogSentinel SIEM API is down

In 2cd0138, LogSentinel SIEM API (https://api.logsentinel.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: LogSentinel SIEM API is back up in 5948975.

repo: LogSentinel/status


text:
IOError crash

Issue description (if this or anything else isn't provided, your issue will be closed)

https://github.com/python/cpython/commit/55fe1ae9708d81b902b6fe8f6590e2a24b1bd4b0
Steps to reproduce the issue (every step, please, and in detail

IOError and EnvironmentError are no longer used so try except IOError
OS and versions

Lolex Tools:9.0nann3
OS (eg Windows 10):ALL
Python Version (eg 3.5.2):3.7 alpha 1

What error do you get? Include error types, line numbers and filenames

NameError

Fixed

repo: LolexOrg/Lolex-Tools

license: unlicense


text:
🛑 jpvir is down

In 880cd9f, jpvir ($JPVIR) was down:

HTTP code: 0
Response time: 0 ms

Resolved: jpvir is back up in 50b0d13.

repo: LonelyJupiter/UPPTIME


text:
Resolving private repositories

Description

I've been using buckaroo quite some time with my private repos and I'm constantly running into the same issue. I'm using buckaroo in "SSH" mode exclusively as it easier with private repos.
When using buckaroo resolve on a buckaroo project with private-repo dependencies, buckaroo results in an unresolvable error the first time, the second time it manages to resolve it and to create a lock file.
Expected Behavior

I would expect buckaroo to work the same with private repos as with public repos.
Actual Behavior

The first resolve:
info Resolve start: 2020-07-13T19:35:07
info Resolving dependencies using quick strategy...
[git] info Fetching refs from git@github.com:roey-e/six-p.git
[git] success Fetched 24 refs in 3.334
[git] info Shallow cloning git@github.com:roey-e/six-p.git
[solver] warning Unresolvable: github.com/roey-e/six-p@branch=master
info Resolve end: 2020-07-13T19:35:16
info Resolve time: 00:00:09.0553388
error The package github.com/roey-e/six-p at branch=master is unresolvable.

The second resolve:
info Resolve start: 2020-07-13T19:39:33
info Resolving dependencies using quick strategy...
[git] info Fetching refs from git@github.com:roey-e/six-p.git
[git] success Fetched 24 refs in 2.661
[solver] info Trying github.com/roey-e/six-p at branch=master
info Resolve end: 2020-07-13T19:39:36
info Resolve time: 00:00:03.4153366
success A solution to the constraints was found.
success The lock-file was updated.

Reading the log, I understand it has something to do with cloning the repo the first time. Deleting '~/.buckaroo' (which holds the cache if I am not mistaken) after the first resolve keeps the error.
Possible Fix

I tried to find what causes it. I debugged buckaroo and found out it might be calling sourceExplorer.FetchManifest from Solver.fs:186 which ultimately calls fetchFile of GitHubApi.fs:6. fetching a file from a private repo using 'https://raw.githubusercontent.com/' is not possible (unless using a token of course).
I looked up for a way to fetch a single file using SSH, but I did not succeed (git archive --remote doesn't work with GitHub). So maybe the fix is not trivial, I'm not familiar with buckaroo's code enough to come up with a solution easily.
Steps to Reproduce

Have a buckaroo project with at least one private repo as a dependency.
Clone your buckaroo project.
Remove the buckaroo.lock.toml file.
buckaroo resolve (or buckaroo resolve --upgrade if the lock file is not up to date) -> error :(
buckaroo resolve -> buckaroo.lock.toml generated :)

I used my roey-e/protobuf (branch: python-protobuf-buck) which uses my roey-e/six repo as a dependency which I privatized (roey-e/six-p).
Your Environment

Version used: The latest buckaroo (3.0.1)
Operating System and Architecture: It happens on Ubuntu 18.04 and on Windows 10

Thanks @roey-e  for investing time into investigating this issue. Your analysis was quite on point and was very helpful. I made a new release and you can find more context in the associated PR.
Please let me know if the issue still persists in the new release

authors: nikhedonia roey-e

repo: LoopPerfect/buckaroo


text:
Stop working

The Whatsapp changed the API. Any ideas in how back to work again?

function names changed again

https://github.com/rizwansoaib/whatsapp-monitor

Please check https://github.com/LoranKloeze/WhatsAllApp/issues/104 😊

authors: LoranKloeze RoelWKramer julianoo rizwansoaib

repo: LoranKloeze/WhatsAllApp


text:
Fabric 1.19 server crashes on startup with REI

REI doesn't seem to like this mod, server crashes on startup with the following error:
java.lang.RuntimeException: Could not execute entrypoint stage 'main' due to errors, provided by 'roughlyenoughitems'!
 at net.fabricmc.loader.impl.entrypoint.EntrypointUtils.lambda$invoke0$0(EntrypointUtils.java:51) ~[fabric-loader-0.14.7.jar:?]
 at net.fabricmc.loader.impl.util.ExceptionUtil.gatherExceptions(ExceptionUtil.java:33) ~[fabric-loader-0.14.7.jar:?]
 at net.fabricmc.loader.impl.entrypoint.EntrypointUtils.invoke0(EntrypointUtils.java:49) ~[fabric-loader-0.14.7.jar:?]
 at net.fabricmc.loader.impl.entrypoint.EntrypointUtils.invoke(EntrypointUtils.java:35) ~[fabric-loader-0.14.7.jar:?]
 at net.fabricmc.loader.impl.game.minecraft.Hooks.startServer(Hooks.java:62) ~[fabric-loader-0.14.7.jar:?]
 at net.minecraft.server.Main.main(Main.java:101) [server-intermediary.jar:?]
 at net.fabricmc.loader.impl.game.minecraft.MinecraftGameProvider.launch(MinecraftGameProvider.java:461) [fabric-loader-0.14.7.jar:?]
 at net.fabricmc.loader.impl.launch.knot.Knot.launch(Knot.java:74) [fabric-loader-0.14.7.jar:?]
 at net.fabricmc.loader.impl.launch.knot.KnotServer.main(KnotServer.java:23) [fabric-loader-0.14.7.jar:?]
 at net.fabricmc.loader.impl.launch.server.FabricServerLauncher.main(FabricServerLauncher.java:69) [fabric-loader-0.14.7.jar:?]
 at net.fabricmc.installer.ServerLauncher.main(ServerLauncher.java:69) [fabric-server-launch.jar:0.11.0]
Caused by: java.lang.RuntimeException: Failed to initialize REI entry point: me.shedaniel.rei.RoughlyEnoughItemsCore
 at me.shedaniel.rei.RoughlyEnoughItemsInitializer.initializeEntryPoint(RoughlyEnoughItemsInitializer.java:88) ~[RoughlyEnoughItems-9.0.475.jar:?]
 at me.shedaniel.rei.RoughlyEnoughItemsInitializer.onInitialize(RoughlyEnoughItemsInitializer.java:48) ~[RoughlyEnoughItems-9.0.475.jar:?]
 at java.lang.invoke.MethodHandleProxies$1.invoke(MethodHandleProxies.java:198) ~[?:?]
 at com.sun.proxy.jdk.proxy3.$Proxy27.onInitialize(Unknown Source) ~[?:?]
 at net.fabricmc.loader.impl.entrypoint.EntrypointUtils.invoke0(EntrypointUtils.java:47) ~[fabric-loader-0.14.7.jar:?]
 ... 8 more
Caused by: java.lang.RuntimeException: Rerached side issue when loading REI plugin by Musica. Please use "rei_server", "rei_client" or "rei_common" instead.
 at me.shedaniel.rei.RoughlyEnoughItemsState.error(RoughlyEnoughItemsState.java:53) ~[RoughlyEnoughItems-9.0.475.jar:?]
 at me.shedaniel.rei.fabric.PluginDetectorImpl.loadPlugin(PluginDetectorImpl.java:107) ~[RoughlyEnoughItems-9.0.475.jar:?]
 at me.shedaniel.rei.fabric.PluginDetectorImpl.detectCommonPlugins(PluginDetectorImpl.java:136) ~[RoughlyEnoughItems-9.0.475.jar:?]
 at me.shedaniel.rei.PluginDetector.detectCommonPlugins(PluginDetector.java) ~[RoughlyEnoughItems-9.0.475.jar:?]
 at me.shedaniel.rei.RoughlyEnoughItemsCore.onInitialize(RoughlyEnoughItemsCore.java:153) ~[RoughlyEnoughItems-9.0.475.jar:?]
 at me.shedaniel.rei.RoughlyEnoughItemsInitializer.initializeEntryPoint(RoughlyEnoughItemsInitializer.java:84) ~[RoughlyEnoughItems-9.0.475.jar:?]
 at me.shedaniel.rei.RoughlyEnoughItemsInitializer.onInitialize(RoughlyEnoughItemsInitializer.java:48) ~[RoughlyEnoughItems-9.0.475.jar:?]
 at java.lang.invoke.MethodHandleProxies$1.invoke(MethodHandleProxies.java:198) ~[?:?]
 at com.sun.proxy.jdk.proxy3.$Proxy27.onInitialize(Unknown Source) ~[?:?]
 at net.fabricmc.loader.impl.entrypoint.EntrypointUtils.invoke0(EntrypointUtils.java:47) ~[fabric-loader-0.14.7.jar:?]
 ... 8 more

I initially looked into where the error message sourced from, thinking it was something I had to do. But it appears to be a tip for developers? So I thought you might want to see this. Specifically the Rerached side issue when loading REI plugin by Musica. Please use "rei_server", "rei_client" or "rei_common" instead.
REI version 9.0.475
Wraith Musica version 1.5.2
MC version 1.19
Fabric loader version 0.14.7
Fabric launcher version 0.11.0

Hmm, odd... I thought I took care of that...
Noted, will look into it when I have time in the following days.

authors: LordDeatHunter Zennii

repo: LordDeatHunter/Musica


text:
🛑 Link Shortener is down

In a36a28f, Link Shortener (https://lr-link.vercel.app) was down:

HTTP code: 504
Response time: 10805 ms

Resolved: Link Shortener is back up in 67bc0f6.

repo: LordRonz/status


text:
Corrections of the font spacing causing issue with the Notice box outside of the menus.

When changing 0x0123 to 0x0023 at offset 0x7b5e in the eboot, the notice ouside the main menu have trouble adjusting to the correct text length. This only affect the notices outside the game menu.

fixed

repo: LostTemplarRH/innocence-r


text:
🛑 Casta is down

In ac00a28, Casta (https://casta.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Casta is back up in a24da83.


text:
Fix typos

This PR fixes various typos I spotted in the project.

All checks are passing now :smile:.

Since the format linter was only triggering for two of the longer doc comments with fixed typos, I think those should be the only ones for which formatting should have been fixed. Making other formatting changes seems out of scope for the PR.

You are right. Just reverted all changes from my second commit not related to typo fixing.

ty!

authors: LucasPickering deining

repo: LucasPickering/slumber


text:
Fix UB when calling get() after clear() for InlineStableVec

The test added here was failing, this adds manual bounds checking on has_element_at,
for reasons I don't yet understand this manual bounds checking is not equivalent to self.data.get(idx).is_some(),
which appears to actually just break everything.

I spent some time digging into the other miri failure I was seeing compact_tiny.
It looks like it is just because we take 2 mutable (non-overlapping) pointers out of a single mutable reference.
Which is UB in the stacked-borrows model.  If we run it with tree-borrows however all tests now pass.
MIRIFLAGS='-Zmiri-tree-borrows' cargo +nightly miri test

This one actually was what caused me to look into miri, because I ran into the UB from entirely safe code.
It was caught one of the existing tests in Lox, when switching one of the StableVecs there to use InlineStableVec when I was playing with an implementation of Lox #38

repo: LukasKalbertodt/stable-vec


text:
🛑 Bank of China Australia is down

In 025bc59, Bank of China Australia (https://obdevp.bank-of-china.net.au/cds-au/v1/banking/products) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Bank of China Australia is back up in 8ebadde.

repo: LukePrior/OpenBankingUptime


text:
🛑 Greater Bank is down

In 27377e1, Greater Bank (https://public.cdr-api.greater.com.au/cds-au/v1/banking/products) was down:

HTTP code: 403
Response time: 486 ms

Resolved: Greater Bank is back up in 0b38741.


text:
A quick modification

Added one more tag
Added optional step
Fixed some grammer

It seems like lots of deploys have been broken after this... maybe it was the tag I added?