text:
Looking for maintainers / contributors / code reviewers...

The project was dormant for a while because none of the current nominal maintainers has enough time/motivation to do the job. Hence, the project is looking for a new maintainer (or several).
Involvement
Ideally I would gladly hand the project over completely to a new principal maintainer, provided they're interested in keeping it roughly within the same principles it's been following the past 10 years. It would be a real letdown for existing users if the project suddenly pivoted into something else.
Realistically though, since I am probably still the only person intimately familiar with the core engine, we could probably make it work with shared maintenance. We can work out the details down the road.
My ultimate goal here is to stop being a bottleneck.
Revival plan
Another reason why the project got hard to maintain is that it's outgrown a few old assumption and is in need of reworking to bring it back into maintainable state. Roughly:

Our build process with several build targets and customizable sets of languages is a pain for both maintainers and users. Since we now live in the world with HTTP/2 and evolved JavaScript packaging it needs to be replaced. I can be instrumental in describing the pain points and historical perspective, but I don't know shit not familiar with modern JS infrastructure.

It might make sense to severely limit the set of languages supported within the main repository, with others moving outside for community support, hosted somewhere else. For that the API between the library and languages needs to (finally) be stabilized.

Migration

We should migrate the project under the org https://github.com/highlightjs/, forking it from here. I don't have a good idea what to do with all the open issues on this repo under /isagalaev/. One way could be defaulting on all of them here and asking interested people to reopen them at the new location.

I own highlightjs.org and happy to keep it running for now, but if we decide on handing over maintenance completely, the domain should probably go to the new maintainer as well.

There's also a Twitter account @highlightjs which I don't want to care about anymore and a Google group which might be simply deleted due to lack of traffic/purpose.

Anyone?
A few people have already pinged me over email, so please reiterate your intention here, to which degree you're interested in helping out/taking over.
And thank you!

Are there any news on this issue? My real question is: how plausible it is that the queue of PR adding new languages will be flushed soonish? Note that I'm not complaining about anything, I only try to get some insight.

I wonder if the best course of action would be to establish a GitHub org and grant all of the volunteers the rights necessary to execute maintainership duties? There needn't necessarily be a single maintainer - more would make the workload easier to handle.

I wonder if the best course of action would be to establish a GitHub org and grant all of the volunteers the rights necessary to execute maintainership duties?

Yes, that's the plan. I haven't yet gotten to it because I wanted to write a minimal maintainer's guide with dos and don'ts accumulated over the years. Without one I am bound to remain a bottleneck, this time as a reviewer.
Please be patient! :-)

+@mjbvz

@Tyriar, has @mjbvz spoken with @isagalaev ?

Sorry but this is not something I'm interested in. Our usage in highlight js is quite limited. We only use it in the markdown preview, and use it there mainly because many websites also use it for syntax coloring

Yes, that's the plan. I haven't yet gotten to it because I wanted to write a minimal maintainer's guide with dos and don'ts accumulated over the years. Without one I am bound to remain a bottleneck, this time as a reviewer.

I feel like we might have figured it out by now

True, and sorry for letting everyone down. Let me find some free evening this week (Wednesday most likely) to just move it over and unblock things.

Thank you!

OK, let me try this again…
I moved the project over to the new place https://github.com/highlightjs/highlight.js and invited @SirCmpwn as a new maintainer (since he prompted me to finally do it, so he's clearly still interested — thank you!).  I kindly ask all the others who volunteered (@ldct, @idleberg, @shiya, @sangdth, @DuncanmaMSFT, @tracker1) to shout out if they're still interested, it's been half a year ago after all…
I put a very basic maintainer's guide in place, it'll probably grow to include something about governance. We didn't need it before, but will clearly need from now on.
Finally, I created a ticket for a new build system: #1759. I wouldn't go as far as to say that it's essential for the future maintenance of the project, but it's certainly something which only grows in importance with time.

@isagalaev I'm still interested.

@isagalaev I'm still interested.

Thanks @isagalaev! I invited @sangdth and @idleberg as well.
Let me restate my original offer:

I would be happy to volunteer. I do not have time to work on extended development on the project, but I do have time for maintainership - holding the vision in my head and applying to reviewing pull requests, providing feedback on them, and merging and releasing them.

I still don't have time to do extended development on the project, but I'm going to turn my focus towards the huge backlog of pull requests (122!) we have to get through.

@isagalaev I'm still interested

I’m also available for code review.

@marcoscaceres I see someone added you to the org already, welcome aboard :-) Feels really good to have people doing things that would otherwise be blocked on me!

@isagalaev, could you kindly add @gmmorris to the contributors? Also, could you give "marcosc" the ability publish on npm?

Hi folks who offered to volunteer. Could I get a show of hands from those of you who are still willing to help here?

@gusbemacbe, January is totally ok :)

Is there any place to discuss, other than this “issue”? I was thinking of Gitter, Slack, Rocket Chat… you name it.

Ok, I set up a slack workspace:
https://highlightjs.slack.com
But I'll need to send individual invite - folks, please email me at mcaceres@mozilla.com so I can add you.

@marcoscaceres have you thought about using gitter instead? one advantage would be that the discussions would be publicly readable without registration

@IIIIllllIIIIllllIIIIllllIIIIllllIIIIll I have, but it's Slack is fine... we just need to redirect things back to bugs on Github to keep a record of things. Pull requests and bug details serve as better records for technical discussion, while Slack is good for planning and community building.

Hey guys! I just noticed this thread accidentally, and while I'm mostly away from the project, I'm going to set up my notifications in a way so that I will actually pay attention to direct messages. So if you need me to update highlightjs.org in any way, please tell me!

@isagalaev, much appreciated.

@isagalaev, could you kindly add @gmmorris to the contributors?

Done. I also fixed your role to owner, so you can do that too. I was fairly sure everyone I added was oener already, but apparently I missed something.

Also, could you give "marcosc" the ability publish on npm?

This is supposed to happen automatically when a new release is created (https://github.com/highlightjs/highlight.js/blob/master/docs/maintainers-guide.rst#release-process). And I think it makes sense to keep it this way so all updates happen everywhere simultaneously.

Ok, cool. I'd missed that bit of the maintainers guide.

No interest in being "principal" maintainer, but I don't mind helping out from time to time with PRs and issues as I can.

And just FYI: if becoming a maintainer is "scary" for anyone who's maybe never done that before or a little commitment shy... then you don't have to step up and say "I'll be a maintainer" or make a huge commitment...   There is no official swearing in process required to make a pull request.
There are plenty of PRs tagged with help welcome or beginner... if all you do is drop by and contribute a single PR or two, that would still be greatly appreciated... and maybe you end up staying longer and contributing more, maybe not.
Personally I started with one PR, then another, then 20 more, and then eventually it made sense to add myself as a listed maintainer in AUTHORS.
Just wanted to encourage others: you can start small if you want and see where it takes you. :-)

If it's still possible. I am ready to help and maintain. 🦄

Cool.  You could start by cutting your teeth on some of the beginning tagged issues:

https://github.com/highlightjs/highlight.js/issues/2217
https://github.com/highlightjs/highlight.js/issues/1616
https://github.com/highlightjs/highlight.js/issues/1559

Or if there is anything specific you're interested in, let me know. The huge CSS consolidation work could use some time invested.

Sure mate

Any objections to posting on https://adoptoposs.org ?  I just saw this.

Any objections to posting on https://adoptoposs.org ? I just saw this.

I wouldn't expect it to help find enthusiasts, but why not?

Any objections to posting on adoptoposs.org ? I just saw this.

No objections here. I hope something comes out of this :sweat_smile:

Happy to help with some basic PR maintenance: approving PRs runs, closing PRs for new core grammars (with suggestion to add to the third party list) and validating/closing issues.

Hi, seeing that this issue is still open are you still looking for maintainers/contributors/code reviewers?

Hey! I'm not in a position to answer your question myself, but last I know the discussion was now happening in Discord https://discord.gg/M24EbU7ja9, so you may have more luck asking over there.

authors: IIIIllllIIIIllllIIIIllllIIIIllllIIIIll NotMoni PatrickMassot SirCmpwn Tyriar allejo bradleymackey egor-rogov gusbemacbe idleberg isagalaev ldct marcoscaceres mjbvz sangdth yyyc514


text:
Allow passing string as element name

See tomkralidis/csw4js#1.
Currently element references require an object like this:
{
        "namespaceURI": "http://www.opengis.net/cat/csw/2.0.2",
        "localPart": "GetRecords",
        "prefix": "csw"
}

This is too verbose. If the csw prefix is already defined, then it should be enough to write
'csw:GetRecords'

instead. Or '{http://www.opengis.net/cat/csw/2.0.2}GetRecords' or '{http://www.opengis.net/cat/csw/2.0.2}csw:GetRecords'.
Good question what to do on unmarshalling. Return string? Or object like now? Or have an option to switch it? Which namespace prefix to use?

+1 for allowing option.

@tomkralidis Yes, option is probably the safest. Backwards compatibility and so on.

Started the documentatione here:
https://github.com/highsource/jsonix/wiki/Qualified-Names

Due to #56, I've decided not to implement QName -> string on unmarshalling for element references. The simplified mappings style solves this perfectly from my PoV.

source: gharchive/issue

authors: highsource tomkralidis

repo: highsource/jsonix

license_type: permissive

license_source: bigquery


text:
hgflask.client or higlass.client  when working in Jupyter notebook ?

Commit:
Steps to reproduce

I don't understand if I should be importing higlass.client or hgflask.client, the docs show both and higlass.client throws flask error.
import higlass
import higlass.client as hgc
view1 = hgc.View([
    hgc.Track(track_type='top-axis', position='top'),
    hgc.Track(track_type='heatmap', position='center',
             tileset_uuid='CQMd6V_cRw6iCI_-Unl3PQ',
              server="http://higlass.io/api/v1/",
              height=250,
             options={ 'valueScaleMax': 0.5 }),
])
(display, server, viewconf) = higlass.display([view1])
display
import higlass
import higlass.client as hgc
view1 = hgc.View([
    hgc.Track(track_type='top-axis', position='top'),
    hgc.Track(track_type='heatmap', position='center',
             tileset_uuid='CQMd6V_cRw6iCI_-Unl3PQ',
              server="http://higlass.io/api/v1/",
              height=250,
             options={ 'valueScaleMax': 0.5 }),
])
(display, server, viewconf) = higlass.display([view1])
display

Observed behavior

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-24-d44aa4ca60c5> in <module>
      9              options={ 'valueScaleMax': 0.5 }),
     10 ])
---> 11 (display, server, viewconf) = higlass.display([view1])
     12 display

/home/groups/hoolock/u1/jvc/miniconda3/envs/higlass/lib/python3.7/site-packages/higlass/viewer.py in display(views, location_sync, zoom_sync, server_port)
      6     Instantiate a HiGlass display with the given views
      7     '''
----> 8     from .server import Server
      9     from .client import ViewConf
     10     tilesets = []

/home/groups/hoolock/u1/jvc/miniconda3/envs/higlass/lib/python3.7/site-packages/higlass/server.py in <module>
     13
     14 # from flask_restful import reqparse, abort, Api, Resource
---> 15 from flask_cors import CORS
     16
     17 from fuse import FUSE

ModuleNotFoundError: No module named 'flask_cors'

Expected behavior

https://github.com/higlass/higlass/blob/develop/docs/jupyter.rst

Hey, should be higlass.client. That's a typo in the docs that we overlooked. It'll be fixed in the next docs release.
Your error is likely because we neglected to add flask-cors to the requirements. Can you try doing pip install -U flask-cors and then running your code again?

Okay great. Also, should this repo be disregarded then? https://github.com/higlass/higlass-jupyter
After installing flask-cors, and re-running the same chunk i get:
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-36-6b1223a16fc1> in <module>
      9              options={ 'valueScaleMax': 0.5 }),
     10 ])
---> 11 (display, server, viewconf) = higlass.display([view1])
     12 display
     13 import higlass

/home/groups/hoolock/u1/jvc/miniconda3/envs/higlass/lib/python3.7/site-packages/higlass/viewer.py in display(views, location_sync, zoom_sync, server_port)
      6     Instantiate a HiGlass display with the given views
      7     '''
----> 8     from .server import Server
      9     from .client import ViewConf
     10     tilesets = []

/home/groups/hoolock/u1/jvc/miniconda3/envs/higlass/lib/python3.7/site-packages/higlass/server.py in <module>
     15 from flask_cors import CORS
     16
---> 17 from fuse import FUSE
     18 import requests
     19 import slugid

ModuleNotFoundError: No module named 'fuse'

Yes, use higlass-python.
The fuse issue is because you need to install fusepy. Check out the requirements.txt here: https://github.com/higlass/higlass-python/blob/master/requirements.txt and just install everything there.
Sorry about those issues and thanks for reporting. That package is still really new and we're still ironing out all those bugs and installation requirements.

Okay great thanks! So now I've got some action, but it's not quite displaying the HiGlassDisplay, and there are some warnings:
self.diskcache_directory /tmp/higlass-python/dc True
starting fuse
Process Process-1:
Traceback (most recent call last):
  File "/home/groups/hoolock/u1/jvc/miniconda3/envs/higlass/lib/python3.7/site-packages/multiprocess/process.py", line 297, in _bootstrap
    self.run()
  File "/home/groups/hoolock/u1/jvc/miniconda3/envs/higlass/lib/python3.7/site-packages/multiprocess/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/home/groups/hoolock/u1/jvc/miniconda3/envs/higlass/lib/python3.7/site-packages/higlass/server.py", line 281, in start_fuse
    foreground=False,
  File "/home/groups/hoolock/u1/jvc/miniconda3/envs/higlass/lib/python3.7/site-packages/fuse.py", line 711, in __init__
    raise RuntimeError(err)
RuntimeError: 1
self.diskcache_directory /tmp/hgflask/dc True
starting fuse
Process Process-2:
Traceback (most recent call last):
  File "/home/groups/hoolock/u1/jvc/miniconda3/envs/higlass/lib/python3.7/site-packages/multiprocess/process.py", line 297, in _bootstrap
    self.run()
  File "/home/groups/hoolock/u1/jvc/miniconda3/envs/higlass/lib/python3.7/site-packages/multiprocess/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/home/groups/hoolock/u1/jvc/miniconda3/envs/higlass/lib/python3.7/site-packages/higlass/server.py", line 281, in start_fuse
    foreground=False,
  File "/home/groups/hoolock/u1/jvc/miniconda3/envs/higlass/lib/python3.7/site-packages/fuse.py", line 711, in __init__
    raise RuntimeError(err)
RuntimeError: 1
 * Serving Flask app "higlass.server" (lazy loading)
 * Environment: production
   WARNING: Do not use the development server in a production environment.
   Use a production WSGI server instead.
 * Debug mode: on
self.diskcache_directory /tmp/hgflask/dc True
starting fuse
Process Process-4:
Traceback (most recent call last):
  File "/home/groups/hoolock/u1/jvc/miniconda3/envs/higlass/lib/python3.7/site-packages/multiprocess/process.py", line 297, in _bootstrap
    self.run()
  File "/home/groups/hoolock/u1/jvc/miniconda3/envs/higlass/lib/python3.7/site-packages/multiprocess/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/home/groups/hoolock/u1/jvc/miniconda3/envs/higlass/lib/python3.7/site-packages/higlass/server.py", line 281, in start_fuse
    foreground=False,
  File "/home/groups/hoolock/u1/jvc/miniconda3/envs/higlass/lib/python3.7/site-packages/fuse.py", line 711, in __init__
    raise RuntimeError(err)
RuntimeError: 1
terminating: F8zPKgTFT1CeX9s-rWKSlQ
 * Serving Flask app "higlass.server" (lazy loading)
 * Environment: production
   WARNING: Do not use the development server in a production environment.
   Use a production WSGI server instead.
 * Debug mode: on
HiGlassDisplay(viewconf={'editable': True, 'views': [{'uid': 'WRftVwOEQkq86l3RBGq5ag', 'tracks': {'top': [{'ty…

OK, cool. What OS are you using?
Could you also pop open your browser's console and paste the contents? I suspect that you don't FUSE enabled in your OS but even if you don't that shouldn't be a problem and we can run around it.

Sorry about late response. The problem I think, is related to how I was running in a jupyter hub, and needed some extra js plugings enabled. I am able to work it from a locally running jupyter notebook.

That was my workaround to trying to get higlass loading data from a public URL, which I have not yet has success with. #434

authors: jakevc pkerpedjiev

repo: higlass/higlass

license: MIT

license_source: github-api


text:
动作探索选择的问题

我比较了一下 pymarl 和 pymarl2 的代码
发现在 pymarl 的 basic_controller.py 中的这个动作探索选择
https://github.com/oxwhirl/pymarl/blob/c971afdceb34635d31b778021b0ef90d7af51e86/src/controllers/basic_controller.py#L40-L48
if not test_mode:
    # Epsilon floor
    epsilon_action_num = agent_outs.size(-1)
    if getattr(self.args, "mask_before_softmax", True):
        # With probability epsilon, we will pick an available action uniformly
        epsilon_action_num = reshaped_avail_actions.sum(dim=1, keepdim=True).float()

    agent_outs = ((1 - self.action_selector.epsilon) * agent_outs
                   + th.ones_like(agent_outs) * self.action_selector.epsilon/epsilon_action_num)

被移动到了 action_selectors.py 中
https://github.com/hijkzzz/pymarl2/blob/d0aaf583605b2b012a1fd080eb6880a00954ed28/src/components/action_selectors.py#L94-L97
而且计算方式貌似在是否mask上有所不同，请问一下为什么要这样改动哇

基本上是一样的~~~只是原来那样太乱了，探索统一放到 selectors 里面。在VMIX下测试貌似更稳定

好的，另外还有一个小问题哈，我发现在 episode_runner.py 你们改动了一下，将 action 放到 cpu上再 update 到 batch 里面
https://github.com/hijkzzz/pymarl2/blob/d0aaf583605b2b012a1fd080eb6880a00954ed28/src/runners/episode_runner.py#L68-80
但其实在 batch 的 update 函数里有一行是把变量放到 gpu 上（如果使用 gpu），这样你们前面先从 gpu 放回 cpu 是不是有点冗余，还是出于什么原因考虑的呢？
https://github.com/hijkzzz/pymarl2/blob/d0aaf583605b2b012a1fd080eb6880a00954ed28/src/components/episode_buffer.py#L103

因为buffer 的数据是放在cpu内存中的，所以移过来释放gpu显存。

我debug跟了一下代码，self.batch 是在这里用 self.new_batch() 定义的
https://github.com/hijkzzz/pymarl2/blob/d0aaf583605b2b012a1fd080eb6880a00954ed28/src/runners/episode_runner.py#L44
而 self.new_batch() 的初始化使用了 self.args.device，如果当前 device 是 cuda 的话就会使用 cuda
https://github.com/hijkzzz/pymarl2/blob/3f894bc24a8dbdf7d642dcfb5f107cd15d02d200/src/runners/episode_runner.py#L30
从而导致这一步的 self.device 是 cuda
https://github.com/hijkzzz/pymarl2/blob/d0aaf583605b2b012a1fd080eb6880a00954ed28/src/components/episode_buffer.py#L103
然后我看了一下 buffer 的定义是用 cpu 存储的
https://github.com/hijkzzz/pymarl2/blob/d0aaf583605b2b012a1fd080eb6880a00954ed28/src/run/run.py#L113-L115
这里的逻辑应该是先得到 episode_batch （在 gpu 上），再加入到 buffer 中（加入buffer时会将变量放到 cpu 上）
https://github.com/hijkzzz/pymarl2/blob/d0aaf583605b2b012a1fd080eb6880a00954ed28/src/run/run.py#L177-L178
那么在生成 episode_batch 的过程中先从 gpu 转 cpu，又被转回 gpu 就有点冗余了好像？在后面加入 buffer 时都会放 cpu上，不知道我理解有没有问题～

你观察的很仔细 这么看 new_batch 的时候要用cpu~

嗯嗯有道理，谢谢啦

authors: hijkzzz liushunyu


text:
Improve docs and tests for model.save() / model.load()

[ ] Document which state is preserved and not preserved when model is reloaded
[ ] Add tests to ensure that the state that is documented to be preserved is actually preserved

Original question:
Is there a test to make sure that if I stop training, save the model, reload it from file, and continue training, the result will be identical compared to what would happen if I didn't stop training at all? I am mostly concerned about obscure state such as optimizer's internal variables.

Hello,

I am mostly concerned about obscure state such as optimizer's internal variables.

that is a good question. I think there are different things that are not currently stored which prevent a perfect recover from previous training:

the content of replay buffer is not saved for off-policy methods (for disk usage reason)
the state of the optimizer (e.g. Adam) is not saved
the learning rate schedule may be reset when reloading a model / resuming training

However, despite those caveats, I did not experience huge drop in performance (yet) when resuming training.

Hm. Does the following sound good:

Close this issue
Open more issues:

"Document which state is preserved and not preserved when model is reloaded"
"Add tests to ensure that the state that is documented to be preserved is actually preserved"

?

I don't know, I would maybe keep that issue, what do you think @hill-a , @erniejunior or @AdamGleave ?

I think it makes sense to keep this issue open but maybe break it into a task list.

Any update for resuming training in an unexpected crash case?
And I also found that set reset_num_timesteps=False in the learn function cannot create a continuous Tensorboard curve after broken, any suggestion?
This is my code
env = gym.make('CartPole-v1')

trpo = TRPO(
    MlpPolicy, env,
    timesteps_per_batch=512,
    max_kl=0.001,
    cg_iters=10,
    cg_damping=1e-3,
    entcoeff=0.0,
    gamma=0.99,
    lam=1.0,
    vf_stepsize=1e-4,
    vf_iters=3,
    tensorboard_log='./tmp_tb_logs',
    verbose=1, n_cpu_tf_sess=None)
trpo.learn(total_timesteps=5e4)
trpo.save('./tmp_models/trpo_cartpole')

del trpo

trpo = TRPO.load('./tmp_models/trpo_cartpole', env=env)
trpo.learn(total_timesteps=10e4, reset_num_timesteps=False)

+1 to continuation of tensorboard logging. when reset_num_timesteps=False

Has there been any update to this? Checkpoint files are still missing significant data required to resume training, defeating the purpose of a checkpoint file.

@mpgussert
Not in stable-baselines, but in stable-baselines3 the code saves all parameters (including optimizer parameters and replay buffer, if so desired).

yep!  i realized very quickly that i was mistaken.  sorry!

Fixed in SB3: https://github.com/DLR-RM/stable-baselines3/issues/435

authors: AdamGleave Miffyli Pastafarianist YeeCY araffin dniku jtromans mpgussert

repo: hill-a/stable-baselines

license: mit


text:
changed the img[0] to img.

In this example, images are a list of the image rendered in the environment, and here we should use img instead of img[0] which is 3 (the RGB channels). Using img[0] will make a wrong GIF file.

Hello,
thanks for the PR, please fill in the the PR template completely.

Using img[0] will make a wrong GIF file.

The environment will be a VecEnv (because created automatically in the constructor) so render will return a list of image, it does not seem wrong.

I'll make a new pr and fill in the pr template completely.

source: gharchive/pull-request

authors: KuKuXia araffin


text:
🛑 Main Page is down

In 35b6800, Main Page (https://hinzwifi.xyz) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Main Page is back up in 9a75344 after 16 minutes.

repo: hinzwifi/uptime-hinz


text:
Fix equip weapon

fix equip weapon

Closed due to #21

repo: hiperesp/DragonFable-Private-Server


text:
consider creating a repo with results of transformation

Since we want to encourage open data and the contributions of the community I think the time was arrived to put those to the public.
Even if they are not yet 100% good I think that having something to work with would encourage participation.
PS: we need to device on a format for organizing the feeds. I would suggest following an OSM relation convention such that we have the English name + the relation id. Example: Frankfurt am Main (62400). The idea is that you could search directly on the github page with your browser for the city of interest. Since we encourage osm participation, the relation id would allow that to be inmediately available for fixes.
What do you think?

https://www.openstreetmap.org/relation/62400

A new repo is very good idea. We have to double check Github and Github LFS limits on file sizes though.

Why would GitHub lfs matter in this case?

If we upload too many files I think the repo will grow over Github's quota which is 1GB now (and we might want to use LFS). However, we can also distribute the files in more than one repository which will mitigate the problem.

Each file would be quite small.

They would be relatively small if we don't apply the dummy flag. Even in that case some files would be megabytes in size. Do you think we should apply the dummy flag? It makes little sense to me.

Of course it would have to be decompressed otherwise we would not be able to see the changes of the PR.

We have to also keep in mind that as the files grow in size, viewing the diff on Github becomes harder and slower.

Do you think that the size of the trips and stop times file can be a problem?

I think if we treat them as source files we will gradually run into trouble with repo size. Moreover, I think people would rather like to download a zip file instead of cloning a large repo in order to pickup some files. One idea would be to release zip files in release pages. Or we can make regional repositories with their own set of released feeds.

@mehdisadeghi I think you got me wrong. I dont mean to use that repo as a osmtogtfs output dump. But rather to use it as a generic GTFS storage that is publicly available to everyone. Whether those files come from osmtogtfs or from an external provider is another topic. Certainly for the time being our only source is osmtogtfs but that shouldnt be like that for the future.
Therefore I dont think that we will run into the github limit anytime soon. Actually I am also storing the saarland osm file on my repo but I have never ran into that problem.
Once we get to the point were the repo grows too big then we can think about scaling it, but I think that creating a big design for 2 or 3 cities might be an overkill.

For a limited number of files it sounds good to me. For more files we have to find another solution for hosting them however.

@carocad

I would suggest following an OSM relation convention such that we have the English name + the relation id. Example: Frankfurt am Main (62400)

Each city has many relations, how could we use this naming convention?

@mehdisadeghi after knowing that there this GTFS feed server editor I think now that having a repo in GitHub would not be such a good idea.
Although it does improve the visibility and ease of participation, there is no tooling around it that we can use directly.
Now I think that it would be better to just use the editor with the output from osmtogtfs
What do you think?
PS: regarding the original comment, I think they organize things in "projects" such that you can name it and rename it as you see fit

authors: carocad mehdisadeghi

repo: hiposfer/osmtogtfs


text:
Faster Sidekiq counts ⚡

Areas I'm addressing in this PR:

Making Sidekiq counts fast out of the box⚡
Fixing workers count

Faster Sidekiq counts
This update should make Hirefire Sidekiq counts super fast for users with zero additional config (simply calling HireFire::Macro::Sidekiq.queue with no arguments, like we do).
Currently, when scheduled counts are included and even with max_scheduled specified (5,000 in our case), the count takes a very long time when many jobs are present, e.g. my scenario:

Using a simple benchmark:
def benchmark
  t1 = Time.now
  yield
  t2 = Time.now
  p "Time taken: #{((t2 - t1) * 1000.0).to_i} ms"
end

# Current
benchmark do
  val = Sidekiq::ScheduledSet.new.inject(0) do |memo, job|
    memo += 1 if job.at <= Time.now
    memo
  end
  p "#{val} matching jobs."
end

# Updated
benchmark do
  val = Sidekiq.redis { |c| c.zcount('schedule', '-inf', Time.now.to_f) }
  p "#{val} matching jobs."
end

I verified the difference is massive: current 45419 ms vs 2 ms when the count is offloaded to Redis. This is the approach used internally by Sidekiq. Equally fast approach can be used for all other counts.
One more issue I uncovered here is that the current approach results in a wrong (higher) count simply because the loop takes so long.
Workers count
I'm using Sidekiq 4.1.1 where the objects yielded to Workers instance are as follows:
"sidekiq.4:3:441559cd6484"
"ou0w76y5w"
{"queue"=>"integration", "payload"=>{"class"=>"Integrations::GenericWorker", "args"=>["Integrations::Apis::LinkedInLearning", "8bee650e-25fb-4829-a733-a3aab353ab5e"], "retry"=>true, "queue"=>"integration", "jid"=>"5d53fd6dc7d33994f27f6b25", "created_at"=>1535439610.2583237, "enqueued_at"=>1535439610.2583618}, "run_at"=>1535439620}
"sidekiq.4:3:441559cd6484"

given that, queues.include?(job[i]["queue"]) in the original implementation simply returned nil so
job[i]["run_at"] <= Time.now.to_i was never evaluated (otherwise would throw an error - undefined method '<=' for nil:NilClass).
We'll need to verify for other versions of Sidekiq, I only tested on 4.1.1.
@mrrooijen this a first draft. I wanted to hear your feedback first before we can polish this up. No spec coverage makes me a little bit uneasy, I could look into setting up a basic suite covering this if you see value in it.

Cool. Looks good. I know this is only a draft, so here's just some general feedback based solely on the draft. Perhaps you'd want to create a private function fast_lookup_capable?(queues) to determine whether or not to fast check. Then create two separate private functions (i.e. dynamic_lookup(queues) and fast_lookup(queues)) based on which strategy to take, with perhaps a short description of when fast_lookup would and wouldn't work.
if fast_lookup_capable?(queues)
  fast_lookup(queues)
else
  dynamic_lookup(queues)
end

Regarding test suite: I've never really bothered initially because the macros were very simple. They're getting slightly more complex than I'd like, but for good reasons (performance) of course, and that's fine. Due to the limitations of Redis versus for example an RDB, you have to go through a few aggregation hoops on the client-side to figure out the correct queue sizes in certain scenarios as you've noticed.
I'm not against having a test suite, and if you want to add something feel free to do so. However, if you do, please try to keep it as simple and minimal as possible, and avoid adding dependencies (i.e. use minitest that's built into MRI.)
Does the draft work with both Sidekiq 3 and 4?
Thanks!

@mrrooijen pushed some updates. I compared Sidekiq API between 3 and 4 and it didn't change for the functionality this depends on. Do you plan on continuing to support Sidekiq 2?
Regarding the test suite, it gets rather complicated to cover what I wanted to cover, so I chose not to add that to the codebase to avoid complexity.

Other than that it looks good to me. I'm not sure how important it'll be to continue to support Sidekiq 2.0. It's 3 major versions behind, and as far as I can remember upgrading Sidekiq to 3.0, 4.0 and 5.0 didn't require a significant amount of effort. So I'd say we can drop support for Sidekiq 2.0 and continue with >= 3.0, since 3.0 was released all the way back in Q1/2014.
Did you see if Sidekiq 5.0 is also compatible with the fast_check strategy?

Looking at the current API (v5), it hasn't changed since v3 for what we depend on here 👍

Cool. I think it's fine to drop support for Sidekiq 2.x. Many improvements have been made in Sidekiq since 2.x so upgrading to the latest version should be encouraged. If users don't want to upgrade they can continue to make use of a compatible hirefire-resource version, of course.

Looks good to me. Since you're updating the file anyway, can you change the .inject(:+) entries to simply .sum? Not sure why it never occurred to me to just use the less verbose sum method. It should produce equivalent results but looks more clean, and is probably more efficient too.
After that I think we should be able to release a new gem version with the changes in place.

@mrrooijen I wanted to do that initially, but then I learned that it wasn't until Ruby 2.4 that sum was released. I suppose that's why you went with inject. I think that support for Ruby versions < 2.4 is still important.

From what I can see the PR should be ready now. What do you think? I can merge it back to master and release a new gem version.

Yes @mrrooijen I've finished all the updates. Ready to merge when you are 👍

Pushed v0.6.0. Thanks!

authors: janklimo mrrooijen

repo: hirefire/hirefire-resource


text:
feat: add platform ssh docs

This is a draft version of the documentation for the platform ssh or how to setup a local editor for the Hiro Platform.
Not to be merged yet as the feature is not yet released on the platform.

@mefrem would you be able to review this one, please?

@andresgalante @ryanwaits Shall I merge this ?

@sabbyanandan I made the changes you requested, but I can't merge it until you approve. Let me know if there are still things missing.

authors: andresgalante beguene

repo: hirosystems/docs


text:
Feature request: Showcase a handful of a most popular deployed contracts on a given principal's explorer page

Is your feature request related to a problem? Please describe.
It would be great to be able to visit a principal's page on Explorer and see a list of that principal's deployed contracts. So often that's what I'm looking for when I visit a principal's page, either looking for a specific deployment or jumping between multiple deployed contracts.
Relatedly, but distinct, it would be a nice feature to have autocomplete or partial results within search for a partial search term, like "SP3D6PV2ACBPEKYJTCMH7HEN02KP87QSP8KTEH335.stx2" would return "SP3D6PV2ACBPEKYJTCMH7HEN02KP87QSP8KTEH335.stx20"

@eugeniadigon This too, should be considered for the new design

authors: andresgalante mefrem

repo: hirosystems/explorer


text:
fix: view all blocks and view all transactions incorrect redirect

To reproduce

Open Explorer, select Network -> Testnet
Under “Recent Blocks”, click “View all blocks”
Observe that the /blocks page is now showing Mainnet blocks instead of testnet

Same issue with "View all transactions"

:tada: This PR is included in version 1.52.3 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

authors: beguene blockstack-devops


text:
Alertコンポーネントの拡張

概要
Alertコンポーネントにステータス=infoのように設定できるところから、Admonition的な使い方を想定しているように、見受けられるのですが、Alert status=info のようなコードは可読性的に、誤解を招く(システム上Alertという文言は強い意味合いを持つため)不適切なように感じます。
問題の主張/正当化
コーディングの概念として、人間にも機械にも可読性を高めるという原則があるので、誤用される可能性を少なくなるようにしたほうがいいかと思います。
提案する解決策やAPI
Alertではなく、MessageBlock, Admonitionのようなコンポーネント名にしたほうがいいかと思います。また、デザインにかんしても、

(i) 注釈です
----------------
詳細詳細

のようなサンプルも追加してほしいです。
代替案
No response
追加情報
No response

UIコンポーネントライブラリとして、他のUIコンポーネントライブラリとコンポーネント名を合わせることは責務だと感じます。他のUIコンポーネントライブラリで同機能のものは、Alertと呼称しているのでYamada UIも同様に呼称していこうと思います。

Chakra UI - Alert
Mantine - Alert
Material UI - Alert

ただ、status="info"とお知らせのような表示をAlertコンポーネントで表現するのは不適切だと私も感じるので、v2あたりでコミュニティと協議し方向性を決めたいと感じます。

authors: hajime-matsumoto hirotomoyamada

repo: hirotomoyamada/yamada-ui


text:
Is this script still works 4 Core with 24GB ram?

First of all, thanks for the developing this script. I got a new Oracle Instance.
But I just want to know is this script still works if I set to 4 Core with 24GB ram.
About 2 weeks ago, I run this script on my server, but it seems it only shows the 'Out of host limit' message.
And someone replied that it works with 3 Core with 18GB ram. So I tried with this settings and now it works well.
Still, 3 Core and 18GB instance is big enough but I just want to know is this still works 4 core and 24GB ram.

Hey, you’re welcome!
If you have Free Tier only account, you must destroy our 3/18.
Of course, it works with 4/24 as it’s said in README, but availability depends only on Oracle’s side…

authors: hitrov writingdeveloper

repo: hitrov/oci-arm-host-capacity


text:
chatglm2多卡训练报错

请教一下设备不一的问题如何解决，这是我的训练脚本，谢谢解答
CUDA_VISIBLE_DEVICES=0,1 python src/train_sft.py --do_train --dataset chatglm2-train --dataset_dir chatglm2-train --finetuning_type lora --output_dir output --per_device_train_batch_size 1 --gradient_accumulation_steps 2 --lr_scheduler_type cosine --logging_steps 1 --save_steps 1000 --learning_rate 5e-5 --num_train_epochs 3.0 --fp16 --use_v2 --num_layer_trainable 3

请仔细阅读Readme.

好嘞，我看下

可以了多谢作者

authors: LemonZyb hiyouga

repo: hiyouga/ChatGLM-Efficient-Tuning


text:
昇腾910B npu8卡训练显存不足

Reminder

[X] I have read the README and searched the existing issues.

System Info
环境信息如下：

llamafactory version: 0.9.1.dev0
Platform: Linux-5.10.0-60.18.0.50.oe2203.aarch64-aarch64-with-glibc2.27
Python version: 3.10.13
PyTorch version: 2.2.0 (NPU)
Transformers version: 4.44.0
Datasets version: 2.19.1
Accelerate version: 0.30.1
PEFT version: 0.12.0
TRL version: 0.8.6
NPU type: Ascend910B3 （8卡）
CANN version: 8.0.RC2.alpha001
DeepSpeed version: 0.13.2

Reproduction
训练参数如下
model
#model_name_or_path: /model-data
model_name_or_path: /mnt/0913/Qwen1.572B
do_sample: false
method
stage: sft
do_train: true
finetuning_type: lora
lora_target: all
ddp
ddp_timeout: 180000000
deepspeed: examples/deepspeed/ds_z0_config.json
dataset
dataset: alpaca_en_demo.json   #alpha_zh_demo  # video: mllm_video_demo
template: qwen
cutoff_len: 1024
max_samples: 1000
overwrite_cache: true
#preprocessing_num_workers: 16
preprocessing_num_workers: 8
output
output_dir: /mnt/llama-facotry-store/output
logging_steps: 10
save_steps: 500
plot_loss: true
overwrite_output_dir: true
train
per_device_train_batch_size: 1
gradient_accumulation_steps: 8
learning_rate: 1.0e-4
num_train_epochs: 3.0
lr_scheduler_type: cosine
warmup_ratio: 0.1
bf16: true
ddp_timeout: 180000000
eval
val_size: 0.1
per_device_eval_batch_size: 1
eval_strategy: steps
eval_steps: 500
Expected behavior
报错信息：
root@hw-osc-ai:/mnt/llama-facotry-store/LLaMA-Factory-main#  ASCEND_RT_VISIBLE_DEVICES=0,1,2,3,4,5,6,7  NPROC_PER_NODE=4 llamafactory-cli train /mnt/llama-facotry-store/LLaMA-Factory-main/examples/train_lora/qwen2_lora_sft_test.yaml
[2024-09-20 01:18:56,433] [INFO] [real_accelerator.py:191:get_accelerator] Setting ds_accelerator to npu (auto detect)
09/20/2024 01:19:00 - INFO - llamafactory.cli - Initializing distributed tasks at: 127.0.0.1:24287
[2024-09-20 01:19:02,616] torch.distributed.run: [WARNING]
[2024-09-20 01:19:02,616] torch.distributed.run: [WARNING] *****************************************
[2024-09-20 01:19:02,616] torch.distributed.run: [WARNING] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
[2024-09-20 01:19:02,616] torch.distributed.run: [WARNING] *****************************************
[2024-09-20 01:19:18,204] [INFO] [real_accelerator.py:191:get_accelerator] Setting ds_accelerator to npu (auto detect)
[2024-09-20 01:19:19,052] [INFO] [real_accelerator.py:191:get_accelerator] Setting ds_accelerator to npu (auto detect)
[2024-09-20 01:19:19,310] [INFO] [real_accelerator.py:191:get_accelerator] Setting ds_accelerator to npu (auto detect)
[2024-09-20 01:19:19,479] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-09-20 01:19:19,551] [INFO] [real_accelerator.py:191:get_accelerator] Setting ds_accelerator to npu (auto detect)
09/20/2024 01:19:19 - WARNING - llamafactory.hparams.parser - ddp_find_unused_parameters needs to be set as False for LoRA in DDP training.
09/20/2024 01:19:19 - INFO - llamafactory.hparams.parser - Process rank: 3, device: npu:3, n_gpu: 1, distributed training: True, compute dtype: torch.bfloat16
09/20/2024 01:19:20 - INFO - llamafactory.data.template - Replace eos token: <|im_end|>
[2024-09-20 01:19:20,317] [INFO] [comm.py:637:init_distributed] cdb=None
09/20/2024 01:19:20 - WARNING - llamafactory.hparams.parser - ddp_find_unused_parameters needs to be set as False for LoRA in DDP training.
09/20/2024 01:19:20 - INFO - llamafactory.hparams.parser - Process rank: 2, device: npu:2, n_gpu: 1, distributed training: True, compute dtype: torch.bfloat16
[2024-09-20 01:19:20,572] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-09-20 01:19:20,572] [INFO] [comm.py:668:init_distributed] Initializing TorchBackend in DeepSpeed with backend hccl
09/20/2024 01:19:20 - WARNING - llamafactory.hparams.parser - ddp_find_unused_parameters needs to be set as False for LoRA in DDP training.
09/20/2024 01:19:20 - INFO - llamafactory.hparams.parser - Process rank: 0, device: npu:0, n_gpu: 1, distributed training: True, compute dtype: torch.bfloat16
[INFO|configuration_utils.py:731] 2024-09-20 01:19:20,588 >> loading configuration file /mnt/0913/Qwen1.572B/config.json
[INFO|configuration_utils.py:800] 2024-09-20 01:19:20,590 >> Model config Qwen2Config {
"_name_or_path": "/mnt/0913/Qwen1.572B",
"architectures": [
"Qwen2ForCausalLM"
],
"attention_dropout": 0.0,
"bos_token_id": 151643,
"eos_token_id": 151645,
"hidden_act": "silu",
"hidden_size": 8192,
"initializer_range": 0.02,
"intermediate_size": 24576,
"max_position_embeddings": 32768,
"max_window_layers": 70,
"model_type": "qwen2",
"num_attention_heads": 64,
"num_hidden_layers": 80,
"num_key_value_heads": 64,
"rms_norm_eps": 1e-06,
"rope_theta": 1000000.0,
"sliding_window": null,
"tie_word_embeddings": false,
"torch_dtype": "float16",
"transformers_version": "4.44.0",
"use_cache": true,
"use_sliding_window": false,
"vocab_size": 152064
}
[INFO|tokenization_utils_base.py:2267] 2024-09-20 01:19:20,592 >> loading file vocab.json
[INFO|tokenization_utils_base.py:2267] 2024-09-20 01:19:20,592 >> loading file merges.txt
[INFO|tokenization_utils_base.py:2267] 2024-09-20 01:19:20,592 >> loading file tokenizer.json
[INFO|tokenization_utils_base.py:2267] 2024-09-20 01:19:20,592 >> loading file added_tokens.json
[INFO|tokenization_utils_base.py:2267] 2024-09-20 01:19:20,592 >> loading file special_tokens_map.json
[INFO|tokenization_utils_base.py:2267] 2024-09-20 01:19:20,592 >> loading file tokenizer_config.json
[2024-09-20 01:19:20,829] [INFO] [comm.py:637:init_distributed] cdb=None
[INFO|tokenization_utils_base.py:2513] 2024-09-20 01:19:20,858 >> Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
[INFO|configuration_utils.py:731] 2024-09-20 01:19:20,859 >> loading configuration file /mnt/0913/Qwen1.572B/config.json
[INFO|configuration_utils.py:800] 2024-09-20 01:19:20,861 >> Model config Qwen2Config {
"_name_or_path": "/mnt/0913/Qwen1.572B",
"architectures": [
"Qwen2ForCausalLM"
],
"attention_dropout": 0.0,
"bos_token_id": 151643,
"eos_token_id": 151645,
"hidden_act": "silu",
"hidden_size": 8192,
"initializer_range": 0.02,
"intermediate_size": 24576,
"max_position_embeddings": 32768,
"max_window_layers": 70,
"model_type": "qwen2",
"num_attention_heads": 64,
"num_hidden_layers": 80,
"num_key_value_heads": 64,
"rms_norm_eps": 1e-06,
"rope_theta": 1000000.0,
"sliding_window": null,
"tie_word_embeddings": false,
"torch_dtype": "float16",
"transformers_version": "4.44.0",
"use_cache": true,
"use_sliding_window": false,
"vocab_size": 152064
}
[INFO|tokenization_utils_base.py:2267] 2024-09-20 01:19:20,862 >> loading file vocab.json
[INFO|tokenization_utils_base.py:2267] 2024-09-20 01:19:20,862 >> loading file merges.txt
[INFO|tokenization_utils_base.py:2267] 2024-09-20 01:19:20,862 >> loading file tokenizer.json
[INFO|tokenization_utils_base.py:2267] 2024-09-20 01:19:20,862 >> loading file added_tokens.json
[INFO|tokenization_utils_base.py:2267] 2024-09-20 01:19:20,862 >> loading file special_tokens_map.json
[INFO|tokenization_utils_base.py:2267] 2024-09-20 01:19:20,862 >> loading file tokenizer_config.json
09/20/2024 01:19:21 - WARNING - llamafactory.hparams.parser - ddp_find_unused_parameters needs to be set as False for LoRA in DDP training.
09/20/2024 01:19:21 - INFO - llamafactory.hparams.parser - Process rank: 1, device: npu:1, n_gpu: 1, distributed training: True, compute dtype: torch.bfloat16
09/20/2024 01:19:21 - INFO - llamafactory.data.template - Replace eos token: <|im_end|>
[INFO|tokenization_utils_base.py:2513] 2024-09-20 01:19:21,116 >> Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
09/20/2024 01:19:21 - INFO - llamafactory.data.template - Replace eos token: <|im_end|>
09/20/2024 01:19:21 - INFO - llamafactory.data.template - Replace eos token: <|im_end|>
Traceback (most recent call last):
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/launcher.py", line 23, in
Traceback (most recent call last):
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/launcher.py", line 23, in
launch()
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/launcher.py", line 19, in launch
run_exp()
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/train/tuner.py", line 50, in run_exp
launch()
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/launcher.py", line 19, in launch
run_sft(model_args, data_args, training_args, finetuning_args, generating_args, callbacks)
run_exp()  File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/train/sft/workflow.py", line 47, in run_sft
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/train/tuner.py", line 50, in run_exp
Traceback (most recent call last):
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/launcher.py", line 23, in
dataset_module = get_dataset(template, model_args, data_args, training_args, stage="sft", **tokenizer_module)
run_sft(model_args, data_args, training_args, finetuning_args, generating_args, callbacks)  File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/data/loader.py", line 248, in get_dataset
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/train/sft/workflow.py", line 47, in run_sft
dataset_module = get_dataset(template, model_args, data_args, training_args, stage="sft", **tokenizer_module)
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/data/loader.py", line 248, in get_dataset
dataset = _get_merged_dataset(data_args.dataset, model_args, data_args, training_args, stage)launch()
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/data/loader.py", line 154, in _get_merged_dataset
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/launcher.py", line 19, in launch
dataset = _get_merged_dataset(data_args.dataset, model_args, data_args, training_args, stage)
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/data/loader.py", line 154, in _get_merged_dataset
run_exp()for dataset_attr in get_dataset_list(dataset_names, data_args.dataset_dir):
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/train/tuner.py", line 50, in run_exp
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/data/parser.py", line 107, in get_dataset_list
for dataset_attr in get_dataset_list(dataset_names, data_args.dataset_dir):
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/data/parser.py", line 107, in get_dataset_list
Traceback (most recent call last):
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/launcher.py", line 23, in
run_sft(model_args, data_args, training_args, finetuning_args, generating_args, callbacks)raise ValueError("Undefined dataset {} in {}.".format(name, DATA_CONFIG))
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/train/sft/workflow.py", line 47, in run_sft
raise ValueError("Undefined dataset {} in {}.".format(name, DATA_CONFIG))ValueError
: Undefined dataset alpaca_en_demo.json in dataset_info.json.ValueError
: Undefined dataset alpaca_en_demo.json in dataset_info.json.
dataset_module = get_dataset(template, model_args, data_args, training_args, stage="sft", **tokenizer_module)
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/data/loader.py", line 248, in get_dataset
launch()
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/launcher.py", line 19, in launch
run_exp()dataset = _get_merged_dataset(data_args.dataset, model_args, data_args, training_args, stage)
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/train/tuner.py", line 50, in run_exp
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/data/loader.py", line 154, in _get_merged_dataset
run_sft(model_args, data_args, training_args, finetuning_args, generating_args, callbacks)
for dataset_attr in get_dataset_list(dataset_names, data_args.dataset_dir):  File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/train/sft/workflow.py", line 47, in run_sft
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/data/parser.py", line 107, in get_dataset_list
dataset_module = get_dataset(template, model_args, data_args, training_args, stage="sft", **tokenizer_module)
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/data/loader.py", line 248, in get_dataset
raise ValueError("Undefined dataset {} in {}.".format(name, DATA_CONFIG))
ValueError: Undefined dataset alpaca_en_demo.json in dataset_info.json.
dataset = _get_merged_dataset(data_args.dataset, model_args, data_args, training_args, stage)
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/data/loader.py", line 154, in _get_merged_dataset
for dataset_attr in get_dataset_list(dataset_names, data_args.dataset_dir):
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/data/parser.py", line 107, in get_dataset_list
raise ValueError("Undefined dataset {} in {}.".format(name, DATA_CONFIG))
ValueError: Undefined dataset alpaca_en_demo.json in dataset_info.json.
[2024-09-20 01:19:37,663] torch.distributed.elastic.multiprocessing.api: [ERROR] failed (exitcode: 1) local_rank: 0 (pid: 21112) of binary: /usr/local/python3.10.13/bin/python3.10
Traceback (most recent call last):
File "/usr/local/python3.10.13/bin/torchrun", line 8, in
sys.exit(main())
File "/usr/local/python3.10.13/lib/python3.10/site-packages/torch/distributed/elastic/multiprocessing/errors/init.py", line 347, in wrapper
return f(*args, **kwargs)
File "/usr/local/python3.10.13/lib/python3.10/site-packages/torch/distributed/run.py", line 812, in main
run(args)
File "/usr/local/python3.10.13/lib/python3.10/site-packages/torch/distributed/run.py", line 803, in run
elastic_launch(
File "/usr/local/python3.10.13/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 135, in call
return launch_agent(self._config, self._entrypoint, list(args))
File "/usr/local/python3.10.13/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 268, in launch_agent
raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/launcher.py FAILED
Failures:
[1]:
time      : 2024-09-20_01:19:37
host      : hw-osc-ai
rank      : 1 (local_rank: 1)
exitcode  : 1 (pid: 21113)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
[2]:
time      : 2024-09-20_01:19:37
host      : hw-osc-ai
rank      : 2 (local_rank: 2)
exitcode  : 1 (pid: 21114)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
[3]:
time      : 2024-09-20_01:19:37
host      : hw-osc-ai
rank      : 3 (local_rank: 3)
exitcode  : 1 (pid: 21115)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
Root Cause (first observed failure):
[0]:
time      : 2024-09-20_01:19:37
host      : hw-osc-ai
rank      : 0 (local_rank: 0)
exitcode  : 1 (pid: 21112)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
^C
root@hw-osc-ai:/mnt/llama-facotry-store/LLaMA-Factory-main# vi /mnt/llama-facotry-store/LLaMA-Factory-main/examples/train_lora/qwen2_lora_sft_test.yaml
root@hw-osc-ai:/mnt/llama-facotry-store/LLaMA-Factory-main# clear
root@hw-osc-ai:/mnt/llama-facotry-store/LLaMA-Factory-main#  ASCEND_RT_VISIBLE_DEVICES=0,1,2,3,4,5,6,7  NPROC_PER_NODE=4 llamafactory-cli train /mnt/llama-facotry-store/LLaMA-Factory-main/examples/train_lora/qwen2_lora_sft_test.yaml
[2024-09-20 01:20:51,901] [INFO] [real_accelerator.py:191:get_accelerator] Setting ds_accelerator to npu (auto detect)
09/20/2024 01:20:56 - INFO - llamafactory.cli - Initializing distributed tasks at: 127.0.0.1:21770
[2024-09-20 01:20:58,065] torch.distributed.run: [WARNING]
[2024-09-20 01:20:58,065] torch.distributed.run: [WARNING] *****************************************
[2024-09-20 01:20:58,065] torch.distributed.run: [WARNING] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
[2024-09-20 01:20:58,065] torch.distributed.run: [WARNING] *****************************************
[2024-09-20 01:21:14,111] [INFO] [real_accelerator.py:191:get_accelerator] Setting ds_accelerator to npu (auto detect)
[2024-09-20 01:21:14,295] [INFO] [real_accelerator.py:191:get_accelerator] Setting ds_accelerator to npu (auto detect)
[2024-09-20 01:21:15,114] [INFO] [real_accelerator.py:191:get_accelerator] Setting ds_accelerator to npu (auto detect)
[2024-09-20 01:21:15,159] [INFO] [real_accelerator.py:191:get_accelerator] Setting ds_accelerator to npu (auto detect)
[2024-09-20 01:21:15,351] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-09-20 01:21:15,351] [INFO] [comm.py:668:init_distributed] Initializing TorchBackend in DeepSpeed with backend hccl
09/20/2024 01:21:15 - WARNING - llamafactory.hparams.parser - ddp_find_unused_parameters needs to be set as False for LoRA in DDP training.
09/20/2024 01:21:15 - INFO - llamafactory.hparams.parser - Process rank: 0, device: npu:0, n_gpu: 1, distributed training: True, compute dtype: torch.bfloat16
[INFO|configuration_utils.py:731] 2024-09-20 01:21:15,368 >> loading configuration file /mnt/0913/Qwen1.572B/config.json
[INFO|configuration_utils.py:800] 2024-09-20 01:21:15,369 >> Model config Qwen2Config {
"_name_or_path": "/mnt/0913/Qwen1.572B",
"architectures": [
"Qwen2ForCausalLM"
],
"attention_dropout": 0.0,
"bos_token_id": 151643,
"eos_token_id": 151645,
"hidden_act": "silu",
"hidden_size": 8192,
"initializer_range": 0.02,
"intermediate_size": 24576,
"max_position_embeddings": 32768,
"max_window_layers": 70,
"model_type": "qwen2",
"num_attention_heads": 64,
"num_hidden_layers": 80,
"num_key_value_heads": 64,
"rms_norm_eps": 1e-06,
"rope_theta": 1000000.0,
"sliding_window": null,
"tie_word_embeddings": false,
"torch_dtype": "float16",
"transformers_version": "4.44.0",
"use_cache": true,
"use_sliding_window": false,
"vocab_size": 152064
}
[INFO|tokenization_utils_base.py:2267] 2024-09-20 01:21:15,371 >> loading file vocab.json
[INFO|tokenization_utils_base.py:2267] 2024-09-20 01:21:15,371 >> loading file merges.txt
[INFO|tokenization_utils_base.py:2267] 2024-09-20 01:21:15,371 >> loading file tokenizer.json
[INFO|tokenization_utils_base.py:2267] 2024-09-20 01:21:15,371 >> loading file added_tokens.json
[INFO|tokenization_utils_base.py:2267] 2024-09-20 01:21:15,371 >> loading file special_tokens_map.json
[INFO|tokenization_utils_base.py:2267] 2024-09-20 01:21:15,371 >> loading file tokenizer_config.json
[2024-09-20 01:21:15,540] [INFO] [comm.py:637:init_distributed] cdb=None
[INFO|tokenization_utils_base.py:2513] 2024-09-20 01:21:15,638 >> Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
[INFO|configuration_utils.py:731] 2024-09-20 01:21:15,639 >> loading configuration file /mnt/0913/Qwen1.572B/config.json
[INFO|configuration_utils.py:800] 2024-09-20 01:21:15,640 >> Model config Qwen2Config {
"_name_or_path": "/mnt/0913/Qwen1.572B",
"architectures": [
"Qwen2ForCausalLM"
],
"attention_dropout": 0.0,
"bos_token_id": 151643,
"eos_token_id": 151645,
"hidden_act": "silu",
"hidden_size": 8192,
"initializer_range": 0.02,
"intermediate_size": 24576,
"max_position_embeddings": 32768,
"max_window_layers": 70,
"model_type": "qwen2",
"num_attention_heads": 64,
"num_hidden_layers": 80,
"num_key_value_heads": 64,
"rms_norm_eps": 1e-06,
"rope_theta": 1000000.0,
"sliding_window": null,
"tie_word_embeddings": false,
"torch_dtype": "float16",
"transformers_version": "4.44.0",
"use_cache": true,
"use_sliding_window": false,
"vocab_size": 152064
}
[INFO|tokenization_utils_base.py:2267] 2024-09-20 01:21:15,641 >> loading file vocab.json
[INFO|tokenization_utils_base.py:2267] 2024-09-20 01:21:15,641 >> loading file merges.txt
[INFO|tokenization_utils_base.py:2267] 2024-09-20 01:21:15,641 >> loading file tokenizer.json
[INFO|tokenization_utils_base.py:2267] 2024-09-20 01:21:15,641 >> loading file added_tokens.json
[INFO|tokenization_utils_base.py:2267] 2024-09-20 01:21:15,641 >> loading file special_tokens_map.json
[INFO|tokenization_utils_base.py:2267] 2024-09-20 01:21:15,641 >> loading file tokenizer_config.json
09/20/2024 01:21:15 - WARNING - llamafactory.hparams.parser - ddp_find_unused_parameters needs to be set as False for LoRA in DDP training.
09/20/2024 01:21:15 - INFO - llamafactory.hparams.parser - Process rank: 2, device: npu:2, n_gpu: 1, distributed training: True, compute dtype: torch.bfloat16
[INFO|tokenization_utils_base.py:2513] 2024-09-20 01:21:15,910 >> Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
09/20/2024 01:21:15 - INFO - llamafactory.data.template - Replace eos token: <|im_end|>
09/20/2024 01:21:15 - INFO - llamafactory.data.loader - Loading dataset alpha_zh_demo.json...
num_proc must be <= 4. Reducing num_proc to 4 for dataset of size 4.
Converting format of dataset (num_proc=4): 100%|███████████████████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 23.85 examples/s]
09/20/2024 01:21:16 - INFO - llamafactory.data.template - Replace eos token: <|im_end|>
[2024-09-20 01:21:16,384] [INFO] [comm.py:637:init_distributed] cdb=None
[2024-09-20 01:21:16,427] [INFO] [comm.py:637:init_distributed] cdb=None
09/20/2024 01:21:16 - WARNING - llamafactory.hparams.parser - ddp_find_unused_parameters needs to be set as False for LoRA in DDP training.
09/20/2024 01:21:16 - INFO - llamafactory.hparams.parser - Process rank: 1, device: npu:1, n_gpu: 1, distributed training: True, compute dtype: torch.bfloat16
09/20/2024 01:21:16 - WARNING - llamafactory.hparams.parser - ddp_find_unused_parameters needs to be set as False for LoRA in DDP training.
09/20/2024 01:21:16 - INFO - llamafactory.hparams.parser - Process rank: 3, device: npu:3, n_gpu: 1, distributed training: True, compute dtype: torch.bfloat16
09/20/2024 01:21:17 - INFO - llamafactory.data.template - Replace eos token: <|im_end|>
09/20/2024 01:21:17 - INFO - llamafactory.data.template - Replace eos token: <|im_end|>
09/20/2024 01:21:23 - INFO - llamafactory.data.loader - Loading dataset alpha_zh_demo.json...
09/20/2024 01:21:23 - INFO - llamafactory.data.loader - Loading dataset alpha_zh_demo.json...
num_proc must be <= 4. Reducing num_proc to 4 for dataset of size 4.
09/20/2024 01:21:23 - INFO - llamafactory.data.loader - Loading dataset alpha_zh_demo.json...
num_proc must be <= 4. Reducing num_proc to 4 for dataset of size 4.
num_proc must be <= 4. Reducing num_proc to 4 for dataset of size 4.
num_proc must be <= 4. Reducing num_proc to 4 for dataset of size 4.
Running tokenizer on dataset (num_proc=4): 100%|███████████████████████████████████████████████████████████████████████| 4/4 [00:00<00:00,  4.21 examples/s]
training example:
input_ids:
[151644, 8948, 198, 2610, 525, 264, 10950, 17847, 13, 151645, 198, 151644, 872, 198, 102450, 62926, 104136, 89012, 22382, 44177, 101047, 100369, 99891, 101911, 5122, 102150, 101911, 33108, 8903, 63109, 36587, 1773, 151645, 198, 151644, 77091, 198, 102150, 101911, 20412, 100206, 99891, 104111, 101911, 3837, 99652, 100140, 55338, 100702, 31914, 100132, 67071, 48934, 30709, 105166, 106251, 8545, 102150, 31838, 104384, 1773, 100346, 114651, 104111, 99896, 101911, 3837, 100140, 102150, 20412, 55338, 100206, 105166, 100166, 33108, 98380, 75317, 3837, 104152, 100206, 100132, 67071, 46944, 57191, 101213, 102150, 101286, 3837, 102150, 101097, 67338, 102150, 110935, 100394, 100676, 102150, 1773, 100147, 101911, 67071, 106929, 22382, 120806, 5373, 99330, 101190, 31843, 33108, 100167, 100809, 34204, 16, 23, 18, 24, 7948, 104181, 101080, 3407, 8903, 63109, 36587, 104442, 101281, 20412, 101281, 38176, 9370, 99488, 3837, 105884, 3837, 113837, 102074, 101281, 108215, 9370, 101911, 1773, 99487, 101911, 112479, 105062, 29490, 63109, 36587, 101313, 3837, 100140, 102493, 102095, 105339, 9370, 99488, 1773, 8903, 63109, 36587, 9370, 101080, 28946, 20412, 99685, 99470, 72225, 13935, 99826, 99243, 99685, 3837, 104677, 16, 21, 101186, 84607, 102098, 108124, 101712, 26940, 35727, 31914, 104001, 67831, 87243, 109268, 34187, 101281, 38176, 113837, 102074, 101281, 104001, 9370, 104949, 3837, 17714, 35727, 104179, 103949, 107759, 102334, 102007, 1773, 151645]
inputs:
<|im_start|>system
You are a helpful assistant.<|im_end|>
<|im_start|>user
识别并解释给定列表中的两个科学理论：细胞理论和日心说。<|im_end|>
<|im_start|>assistant
细胞理论是生物科学的一个理论，它认为所有生命体都是由微小的基本单元——细胞所构成。这是生物学的一个基础理论，认为细胞是所有生物的基本结构和功能单位，所有的生物都是由一个或多个细胞组成，细胞只能通过细胞分裂产生新的细胞。这一理论由薛定谔、施瓦内和雪莱于1839年首次提出。
日心说是指太阳是太阳系的中心，也就是说，行星围绕太阳旋转的理论。这个理论打破了传统的地心说观点，认为地球并不是宇宙的中心。日心说的提出者是尼古拉·哥白尼，他在16世纪初发表了他的著作《天体运行论》，阐述了太阳系行星围绕太阳运行的模型，为天文学的发展做出了巨大贡献。<|im_end|>
label_ids:
[-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 102150, 101911, 20412, 100206, 99891, 104111, 101911, 3837, 99652, 100140, 55338, 100702, 31914, 100132, 67071, 48934, 30709, 105166, 106251, 8545, 102150, 31838, 104384, 1773, 100346, 114651, 104111, 99896, 101911, 3837, 100140, 102150, 20412, 55338, 100206, 105166, 100166, 33108, 98380, 75317, 3837, 104152, 100206, 100132, 67071, 46944, 57191, 101213, 102150, 101286, 3837, 102150, 101097, 67338, 102150, 110935, 100394, 100676, 102150, 1773, 100147, 101911, 67071, 106929, 22382, 120806, 5373, 99330, 101190, 31843, 33108, 100167, 100809, 34204, 16, 23, 18, 24, 7948, 104181, 101080, 3407, 8903, 63109, 36587, 104442, 101281, 20412, 101281, 38176, 9370, 99488, 3837, 105884, 3837, 113837, 102074, 101281, 108215, 9370, 101911, 1773, 99487, 101911, 112479, 105062, 29490, 63109, 36587, 101313, 3837, 100140, 102493, 102095, 105339, 9370, 99488, 1773, 8903, 63109, 36587, 9370, 101080, 28946, 20412, 99685, 99470, 72225, 13935, 99826, 99243, 99685, 3837, 104677, 16, 21, 101186, 84607, 102098, 108124, 101712, 26940, 35727, 31914, 104001, 67831, 87243, 109268, 34187, 101281, 38176, 113837, 102074, 101281, 104001, 9370, 104949, 3837, 17714, 35727, 104179, 103949, 107759, 102334, 102007, 1773, 151645]
labels:
细胞理论是生物科学的一个理论，它认为所有生命体都是由微小的基本单元——细胞所构成。这是生物学的一个基础理论，认为细胞是所有生物的基本结构和功能单位，所有的生物都是由一个或多个细胞组成，细胞只能通过细胞分裂产生新的细胞。这一理论由薛定谔、施瓦内和雪莱于1839年首次提出。
日心说是指太阳是太阳系的中心，也就是说，行星围绕太阳旋转的理论。这个理论打破了传统的地心说观点，认为地球并不是宇宙的中心。日心说的提出者是尼古拉·哥白尼，他在16世纪初发表了他的著作《天体运行论》，阐述了太阳系行星围绕太阳运行的模型，为天文学的发展做出了巨大贡献。<|im_end|>
num_proc must be <= 4. Reducing num_proc to 4 for dataset of size 4.
num_proc must be <= 4. Reducing num_proc to 4 for dataset of size 4.
[INFO|configuration_utils.py:731] 2024-09-20 01:21:25,218 >> loading configuration file /mnt/0913/Qwen1.572B/config.json
num_proc must be <= 4. Reducing num_proc to 4 for dataset of size 4.
[INFO|configuration_utils.py:800] 2024-09-20 01:21:25,219 >> Model config Qwen2Config {
"_name_or_path": "/mnt/0913/Qwen1.572B",
"architectures": [
"Qwen2ForCausalLM"
],
"attention_dropout": 0.0,
"bos_token_id": 151643,
"eos_token_id": 151645,
"hidden_act": "silu",
"hidden_size": 8192,
"initializer_range": 0.02,
"intermediate_size": 24576,
"max_position_embeddings": 32768,
"max_window_layers": 70,
"model_type": "qwen2",
"num_attention_heads": 64,
"num_hidden_layers": 80,
"num_key_value_heads": 64,
"rms_norm_eps": 1e-06,
"rope_theta": 1000000.0,
"sliding_window": null,
"tie_word_embeddings": false,
"torch_dtype": "float16",
"transformers_version": "4.44.0",
"use_cache": true,
"use_sliding_window": false,
"vocab_size": 152064
}
[INFO|modeling_utils.py:3653] 2024-09-20 01:21:25,280 >> loading weights file /mnt/0913/Qwen1.572B/model.safetensors.index.json
[INFO|modeling_utils.py:1584] 2024-09-20 01:21:25,281 >> Instantiating Qwen2ForCausalLM model under default dtype torch.bfloat16.
[INFO|configuration_utils.py:1038] 2024-09-20 01:21:25,282 >> Generate config GenerationConfig {
"bos_token_id": 151643,
"eos_token_id": 151645
}
Loading checkpoint shards:  42%|██████████████████████████████████████▋                                                     | 16/38 [00:12<00:17,  1.24it/s]
Traceback (most recent call last):
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/launcher.py", line 23, in
launch()
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/launcher.py", line 19, in launch
run_exp()
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/train/tuner.py", line 50, in run_exp
run_sft(model_args, data_args, training_args, finetuning_args, generating_args, callbacks)
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/train/sft/workflow.py", line 48, in run_sft
model = load_model(tokenizer, model_args, finetuning_args, training_args.do_train)
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/model/loader.py", line 162, in load_model
model = load_class.from_pretrained(**init_kwargs)
File "/usr/local/python3.10.13/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 564, in from_pretrained
return model_class.from_pretrained(
File "/usr/local/python3.10.13/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3941, in from_pretrained
) = cls._load_pretrained_model(
File "/usr/local/python3.10.13/lib/python3.10/site-packages/transformers/modeling_utils.py", line 4415, in _load_pretrained_model
new_error_msgs, offload_index, state_dict_index = _load_state_dict_into_meta_model(
File "/usr/local/python3.10.13/lib/python3.10/site-packages/transformers/modeling_utils.py", line 936, in _load_state_dict_into_meta_model
set_module_tensor_to_device(model, param_name, param_device, **set_module_kwargs)
File "/usr/local/python3.10.13/lib/python3.10/site-packages/accelerate/utils/modeling.py", line 400, in set_module_tensor_to_device
new_value = value.to(device)
RuntimeError: NPU out of memory. Tried to allocate 386.00 MiB (NPU 3; 60.97 GiB total capacity; 59.57 GiB already allocated; 59.57 GiB current active; 16.99 MiB free; 60.06 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.
Loading checkpoint shards:  42%|██████████████████████████████████████▋                                                     | 16/38 [00:14<00:19,  1.14it/s]
Traceback (most recent call last):
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/launcher.py", line 23, in
launch()
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/launcher.py", line 19, in launch
run_exp()
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/train/tuner.py", line 50, in run_exp
run_sft(model_args, data_args, training_args, finetuning_args, generating_args, callbacks)
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/train/sft/workflow.py", line 48, in run_sft
model = load_model(tokenizer, model_args, finetuning_args, training_args.do_train)
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/model/loader.py", line 162, in load_model
model = load_class.from_pretrained(**init_kwargs)
File "/usr/local/python3.10.13/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 564, in from_pretrained
return model_class.from_pretrained(
File "/usr/local/python3.10.13/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3941, in from_pretrained
) = cls._load_pretrained_model(
File "/usr/local/python3.10.13/lib/python3.10/site-packages/transformers/modeling_utils.py", line 4415, in _load_pretrained_model
new_error_msgs, offload_index, state_dict_index = _load_state_dict_into_meta_model(
File "/usr/local/python3.10.13/lib/python3.10/site-packages/transformers/modeling_utils.py", line 936, in _load_state_dict_into_meta_model
set_module_tensor_to_device(model, param_name, param_device, **set_module_kwargs)
File "/usr/local/python3.10.13/lib/python3.10/site-packages/accelerate/utils/modeling.py", line 400, in set_module_tensor_to_device
new_value = value.to(device)
RuntimeError: NPU out of memory. Tried to allocate 386.00 MiB (NPU 0; 60.97 GiB total capacity; 59.57 GiB already allocated; 59.57 GiB current active; 25.36 MiB free; 60.06 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.
Loading checkpoint shards:  42%|██████████████████████████████████████▋                                                     | 16/38 [00:13<00:19,  1.15it/s]
Traceback (most recent call last):
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/launcher.py", line 23, in
launch()
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/launcher.py", line 19, in launch
run_exp()
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/train/tuner.py", line 50, in run_exp
run_sft(model_args, data_args, training_args, finetuning_args, generating_args, callbacks)
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/train/sft/workflow.py", line 48, in run_sft
model = load_model(tokenizer, model_args, finetuning_args, training_args.do_train)
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/model/loader.py", line 162, in load_model
model = load_class.from_pretrained(**init_kwargs)
File "/usr/local/python3.10.13/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 564, in from_pretrained
return model_class.from_pretrained(
File "/usr/local/python3.10.13/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3941, in from_pretrained
) = cls._load_pretrained_model(
File "/usr/local/python3.10.13/lib/python3.10/site-packages/transformers/modeling_utils.py", line 4415, in _load_pretrained_model
new_error_msgs, offload_index, state_dict_index = _load_state_dict_into_meta_model(
File "/usr/local/python3.10.13/lib/python3.10/site-packages/transformers/modeling_utils.py", line 936, in _load_state_dict_into_meta_model
set_module_tensor_to_device(model, param_name, param_device, **set_module_kwargs)
File "/usr/local/python3.10.13/lib/python3.10/site-packages/accelerate/utils/modeling.py", line 400, in set_module_tensor_to_device
new_value = value.to(device)
RuntimeError: NPU out of memory. Tried to allocate 386.00 MiB (NPU 2; 60.97 GiB total capacity; 59.57 GiB already allocated; 59.57 GiB current active; 16.54 MiB free; 60.06 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.
Loading checkpoint shards:  42%|██████████████████████████████████████▋                                                     | 16/38 [00:14<00:19,  1.13it/s]
Traceback (most recent call last):
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/launcher.py", line 23, in
launch()
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/launcher.py", line 19, in launch
run_exp()
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/train/tuner.py", line 50, in run_exp
run_sft(model_args, data_args, training_args, finetuning_args, generating_args, callbacks)
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/train/sft/workflow.py", line 48, in run_sft
model = load_model(tokenizer, model_args, finetuning_args, training_args.do_train)
File "/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/model/loader.py", line 162, in load_model
model = load_class.from_pretrained(**init_kwargs)
File "/usr/local/python3.10.13/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 564, in from_pretrained
return model_class.from_pretrained(
File "/usr/local/python3.10.13/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3941, in from_pretrained
) = cls._load_pretrained_model(
File "/usr/local/python3.10.13/lib/python3.10/site-packages/transformers/modeling_utils.py", line 4415, in _load_pretrained_model
new_error_msgs, offload_index, state_dict_index = _load_state_dict_into_meta_model(
File "/usr/local/python3.10.13/lib/python3.10/site-packages/transformers/modeling_utils.py", line 936, in _load_state_dict_into_meta_model
set_module_tensor_to_device(model, param_name, param_device, **set_module_kwargs)
File "/usr/local/python3.10.13/lib/python3.10/site-packages/accelerate/utils/modeling.py", line 400, in set_module_tensor_to_device
new_value = value.to(device)
RuntimeError: NPU out of memory. Tried to allocate 386.00 MiB (NPU 1; 60.97 GiB total capacity; 59.57 GiB already allocated; 59.57 GiB current active; 17.39 MiB free; 60.06 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.
[2024-09-20 01:22:03,132] torch.distributed.elastic.multiprocessing.api: [WARNING] Sending process 24246 closing signal SIGTERM
[2024-09-20 01:22:03,132] torch.distributed.elastic.multiprocessing.api: [WARNING] Sending process 24247 closing signal SIGTERM
[2024-09-20 01:22:03,396] torch.distributed.elastic.multiprocessing.api: [ERROR] failed (exitcode: 1) local_rank: 2 (pid: 24248) of binary: /usr/local/python3.10.13/bin/python3.10
Traceback (most recent call last):
File "/usr/local/python3.10.13/bin/torchrun", line 8, in
sys.exit(main())
File "/usr/local/python3.10.13/lib/python3.10/site-packages/torch/distributed/elastic/multiprocessing/errors/init.py", line 347, in wrapper
return f(*args, **kwargs)
File "/usr/local/python3.10.13/lib/python3.10/site-packages/torch/distributed/run.py", line 812, in main
run(args)
File "/usr/local/python3.10.13/lib/python3.10/site-packages/torch/distributed/run.py", line 803, in run
elastic_launch(
File "/usr/local/python3.10.13/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 135, in call
return launch_agent(self._config, self._entrypoint, list(args))
File "/usr/local/python3.10.13/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 268, in launch_agent
raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
/mnt/llama-facotry-store/LLaMA-Factory-main/src/llamafactory/launcher.py FAILED
Failures:
[1]:
time      : 2024-09-20_01:22:03
host      : hw-osc-ai
rank      : 3 (local_rank: 3)
exitcode  : 1 (pid: 24249)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
Root Cause (first observed failure):
[0]:
time      : 2024-09-20_01:22:03
host      : hw-osc-ai
rank      : 2 (local_rank: 2)
exitcode  : 1 (pid: 24248)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
Others
尝试过了Qwen2 72B和Qwen1.5 72B，使用npu-smi info查看显卡信息，确实存在爆显存的情况，训练数据集也很小，4k大小的样子，请教下是什么原因，该如何解决。

我也遇到了类似的问题~想问问为什么

+1

authors: Linuxstyle LtroiNGU RY-lxf

repo: hiyouga/LLaMA-Factory


text:
🛑 Reime ohne Sinn und Verstand is down

In e66c191, Reime ohne Sinn und Verstand (https://www.reimeohnesinnundverstand.de) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Reime ohne Sinn und Verstand is back up in 03a9252 after 8 minutes.

repo: hjstrauss/MonitorMySites


text:
🛑 Griffo is down

In 9353041, Griffo (https://www.griffo.de) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Griffo is back up in ee19c4f after 10 minutes.


text:
🛑 LM Bogen WSV is down

In 539f54d, LM Bogen WSV (https://www.lmbogenwsv.de) was down:

HTTP code: 0
Response time: 0 ms

Resolved: LM Bogen WSV is back up in c8181cb after 8 minutes.


text:
🛑 LM Bogen WSV is down

In 8fd8329, LM Bogen WSV (https://www.lmbogenwsv.de) was down:

HTTP code: 0
Response time: 0 ms

Resolved: LM Bogen WSV is back up in 0a458e4 after 7 minutes.


text:
Support to select different heuristics

Thinking to use this implementation to draw lines between connectors, which means we favour nice looking lines.
At the moment I can see it is hard coded to use Manhattan distance when diagonal support is disabled.
Is it possible to also add below heuristics and update to allow selecting the heuristic from python?

only y diff

abs(current.y - goal.y)

only x diff

abs(current.x - goal.x)

Breaking ties

dx1 = current.x - goal.x
dy1 = current.y - goal.y
dx2 = start.x - goal.x
dy2 = start.y - goal.y
cross = abs(dx1dy2 - dx2dy1)
heuristic += cross*0.001
See http://theory.stanford.edu/~amitp/GameProgramming/Heuristics.html
Not sure if possible to even allow defining the heuristic in python without affecting performance.
The code change is minimal as you would know.

Thanks for taking the time to think about this. I see two different considerations regarding performance:

Asking the Python interpreter to evaluate every call to the heuristic function would clearly be too expensive. Thus, the heuristic passed from Python would have to be evaluated in C++, but I don't know how to marshall an arbitrary Python function object to C++ or if this is even possible. (A Python function object is much more complex than the simple array I marshall in the current implementation.) Let me know if you know of any way to do this.
Changing the heuristic may introduce a performance penalty if, for example, the new heuristic underestimates the cost and becomes less effective at eliminating expensive paths. I see the motivation for breaking ties above (I guess you have a uniform costs and prefer straight paths?), but what is the motivation for considering only the distance in x or y?

Thanks @hjweide
Will check further on 1, I doubt it is possible.
Regarding 2.
Yes, we prefer straight lines. We need to trade some performance for what looks nice in drawings.
You can't get anymore straight then just x or y. It's really just about giving options for users to pick. I don't think we can ever have a single heuristic that looks nice to everyone. Beauty is in the eye of the beholder...
Just x or y is easy to reason about. For the drawings we have, it's not just about connecting a single source and destination, there will be lots of sources, destinations and obstables, and we also want to avoid crosses.

Yes, we prefer straight lines. We need to trade some performance for what looks nice in drawings.

This makes sense. I suppose we could have a few heuristic functions implemented in C++ and let the user select one from Python by passing an argument. This wouldn't allow arbitrary heuristics, but would allow the user some flexibility without a performance penalty.
I don't expect that I'll have much time to work on this soon, would you be interested in submitting a PR? I'd be happy to provide guidance if needed.

Thanks @hjweide
Yes, that was the exact approach I was thinking.
Will send through a PR when ready. I wonder if also there penalty to passing more arguments to the Heuristic, the breaking ties one in particular needs to know alot more properties.
Cheers

I wonder if also there penalty to passing more arguments to the Heuristic, the breaking ties one in particular needs to know alot more properties.

In terms of computational overhead I think any arguments you might add are negligible compared to the np.ndarray, but of course we should (a) keep the interface backwards-compatible and (b) try not to add unnecessary arguments.
I haven't thought about this in detail, but I was thinking something like:

new argument indicating which heuristic to use (defaults to existing choices of heuristic based on allow_diagonal)
new argument indicating tiebreak strategy (default to nothing)
new argument indicating tiebreak penalty coefficient (defaults to 0.)

@hjweide, sorry for the delay didn't get much time
Can you check below PR
https://github.com/hjweide/pyastar2d/pull/35
and let me know if you want to make some other changes?
I will probably need to add one more heuristic, that prefers going outward first and then making turns rather than making turns straight away i.e. not optimal path.

I made a program to get through obstacles, and this project already brought down the python method I was using from 5 seconds to 0.05 seconds and without having to resize the original to 1/10th. If you added a heuristic that preferred going outwards for a bit first, that would make it even better because it takes a few seconds for me to turn. I appreciate your guys work.

Not sure if outward is actually needed, you can always create some obstacles in the weights to make sure it goes outward.

what about distance consequences for being N pixels close to borders, to encourage staying away from black pixels?

what about distance consequences for being N pixels close to borders, to encourage staying away from black pixels?

That would be expensive to calculate inside a heuristic function, best to just add that as obstacles in the grid.

If you added a heuristic that preferred going outwards for a bit first, that would make it even better because it takes a few seconds for me to turn. I appreciate your guys work.

Updated, orthogonal x and y, so that it changes direction in the middle rather than at the end. For drawing good outward lines from one node to another node, check what side is the starting point:

If left or right side, select orthogonal x
if top or bottom side, select orthogonal y

This should give good outward lines and work well with lines coming back as well.

what about distance consequences for being N pixels close to borders, to encourage staying away from black pixels?

best to just add that as obstacles in the grid.

Right. The heuristic is meant to eliminate bad solutions quickly to find the optimal solution faster. It is not meant to define what the optimal solution looks like. If you don't want to go near walls, raise the cost of those areas.

Closing this as it is supposed now

authors: blackdimund hjweide peterchenadded


text:
Challenges with GROUPS

Hi there,
great progress you have achieved with your node. Well done... I really like it and use it everywhere
I'm now starting to manage the group in my automation as well via the UI but have a few challenges where I would need your help:

Creating Groups (player.join.group)

I want to select the MASTER and SLAVE to select from a dropdown list (this is working already)
I want to send the MASTER and SLAVE info to your Universal node
I'm unable to send the MASTER information with an external message (solved)
==> just found the solution
=====> msg.topic = group.join.player
=====> msg.payload = Küche
=====> msg.playerName = Bad
I can only send one SLAVE to the Universal node at a time (Is there a way to send an array of SLAVES?)

Leaving a Group (player.leave.group)

I'm not able to send the MASTER information with an external message to the node (solved)
==> Same as above
I can only remove 1 SLAVE player at a time (Is there a way to send an array of SLAVES?)
Is there a way to clear a GROUP just by providing the MASTER?

Versions and Infrastructure
what system:  Raspberry Pi
node-red-contrib-sonos-plus version: 4.1.1
Node-RED version: 1.1.2
see: https://flows.nodered.org/flow/a5c42641bf33dd0da9bfd4426a06950a

Hi. SONOS provides only 2 actions related to group management: AddMember, RemoveMember (and change coordinator, become coordinator, change coordinator) - and they only work on a single player!
What I can do is:
household.separate.group: input any group member => will remove all players from group step by step and as a result all are stand alone players.
household.create.group: input of strings with names ==> will group all together with first name as coordinator step by step.
(maybe as prereq remove at least coordinator from any group otherwise its members will belong to the new group automatically)
I will start my vacation Monday - so that could be released  end of September.
You might be able to do the same in flows using household.get.groups, player.join.group, player.leave.group

4.2.0 is published with the above commands. You can define a group with household.create.group and state: "Küche,Bad,Wohnzimmer" - Küche will become the new coordinator.
You can use that "create" command also for any changes. To add a player just submit ""Küche,Bad,Wohnzimmer,Büro" and the program will add Büro to that group.
I hope my algorithm for handling changes is smart and efficient - and close this topic. Feel free to open a new one in case of any issues.

authors: hklages tsprivate

repo: hklages/node-red-contrib-sonos-plus


text:
BeServiceRequest

I see that some examples for an Observation resource (eg. http://build.fhir.org/ig/hl7-be/lab/Bundle-hematologyStudiesAsFHIRDocument.json.html) have a reference to a ServiceRequest in their "basedOn" field. The "profile" field in the "meta" field of that ServiceRequest refers to "https://www.ehealth.fgov.be/standards/fhir/lab/StructureDefinition/be-servicerequest" suggesting that this is a custom definition, similar to the BeDiagnosticReport and BePatient.
However, I was unable to locate a definition for such a "BeServiceRequest". In the BeObservationLaboratory resource profile, the "basedOn" field refers to a reference which could be of type "ServiceRequest", which is a generic FHIR type.
Is there a specification for the "BeServiceRequest"? Any must-support fields?

Hi @FlorianBonte-Moonchase ,
At the moment, there is no such definition. The example you indicate is erroneous and should be adapted. There will be a labrequest in the future, see: www.github.com/hl7-be/referral, but this effort has not been completed yet.
The only requirement in the current profile is for a general "ServiceRequest".

authors: FlorianBonte-Moonchase bdc-ehealth


text:
Micro-organisme external comment

Dear
We don't see any micro-organism external comment in the Fhir report.
What can be the reason for that please ?
best regards
Patricia Vandecandelaere

@pvandecandelaere : the version of the standard you are using has been officially replaced since 08/09/2022, please use the official version of the standard https://www.ehealth.fgov.be/standards/fhir/lab/index.html .

authors: bdc-ehealth pvandecandelaere


text:
Author should validate against display name for the user

Right now the Author check validates against the github login, not the user's display name. This means in the PowerToys UI the author gets displayed as something like "neilenns" instead of the more appropriate "Neil Enns".
The display name for a user is available via the REST API at https://api.github.com/users/username (e.g. https://api.github.com/users/neilenns):
{
  "login": "neilenns",
  "id": 9524118,
  "name": "Neil Enns",
}

Maybe this check can be updated to validate against either the login or the display name? The display name is more user friendly.

Yes, I agree!

This will be fixed in the next release

I confirmed this passes now against my DiscordTimestamp repo, which has the following in the plugin.json:
"Author": "Neil Enns",

This is now implemented in 0.3.0

authors: hlaueriksson neilenns


text:
+default/search-project does not work under windows

What did you expect to happen?
"M-x +default/search-project" should start ripgrep
What actually happened?
It errors out with
Debugger entered--Lisp error: (wrong-type-argument characterp "rg")
  counsel-rg(nil "c:/Users/guraltsev/.config/doom/" " " "rg  [./]: ")
  (let* ((this-command (quote counsel-rg)) (project-root (or (doom-project-root) default-directory)) (directory (or in project-root)) (args (concat (if all-files " -uu") (if recursive nil " --maxdepth 1") " " (mapconcat (function shell-quote-argument) args " ")))) (setq deactivate-mark t) (counsel-rg (or query (if (doom-region-active-p) (progn (replace-regexp-in-string "[! |]" (function (lambda ... ...)) (rxt-quote-pcre (doom-thing-at-point-or-region)))))) directory args (or prompt (format "rg%s [%s]: " args (cond ((equal directory default-directory) "./") ((equal directory project-root) (projectile-project-name)) ((file-relative-name directory project-root)))))))
  (progn (if (executable-find "rg") nil (user-error "Couldn't find ripgrep in your PATH")) (require (quote counsel)) (let* ((this-command (quote counsel-rg)) (project-root (or (doom-project-root) default-directory)) (directory (or in project-root)) (args (concat (if all-files " -uu") (if recursive nil " --maxdepth 1") " " (mapconcat (function shell-quote-argument) args " ")))) (setq deactivate-mark t) (counsel-rg (or query (if (doom-region-active-p) (progn (replace-regexp-in-string "[! |]" (function ...) (rxt-quote-pcre ...))))) directory args (or prompt (format "rg%s [%s]: " args (cond ((equal directory default-directory) "./") ((equal directory project-root) (projectile-project-name)) ((file-relative-name directory project-root))))))))
  (progn (let ((--cl-keys-- --cl-rest--)) (while --cl-keys-- (cond ((memq (car --cl-keys--) (quote (:query :in :all-files :recursive :prompt :args :allow-other-keys))) (setq --cl-keys-- (cdr (cdr --cl-keys--)))) ((car (cdr (memq ... --cl-rest--))) (setq --cl-keys-- nil)) (t (error "Keyword argument %s not one of (:query :in :all-files :recursive :prompt :args)" (car --cl-keys--)))))) (progn (if (executable-find "rg") nil (user-error "Couldn't find ripgrep in your PATH")) (require (quote counsel)) (let* ((this-command (quote counsel-rg)) (project-root (or (doom-project-root) default-directory)) (directory (or in project-root)) (args (concat (if all-files " -uu") (if recursive nil " --maxdepth 1") " " (mapconcat (function shell-quote-argument) args " ")))) (setq deactivate-mark t) (counsel-rg (or query (if (doom-region-active-p) (progn (replace-regexp-in-string "[! |]" ... ...)))) directory args (or prompt (format "rg%s [%s]: " args (cond (... "./") (... ...) (...))))))))
  (let* ((query (car (cdr (plist-member --cl-rest-- (quote :query))))) (in (car (cdr (plist-member --cl-rest-- (quote :in))))) (all-files (car (cdr (plist-member --cl-rest-- (quote :all-files))))) (recursive (car (cdr (or (plist-member --cl-rest-- (quote :recursive)) (quote (nil t)))))) (prompt (car (cdr (plist-member --cl-rest-- (quote :prompt))))) (args (car (cdr (plist-member --cl-rest-- (quote :args)))))) (progn (let ((--cl-keys-- --cl-rest--)) (while --cl-keys-- (cond ((memq (car --cl-keys--) (quote ...)) (setq --cl-keys-- (cdr ...))) ((car (cdr ...)) (setq --cl-keys-- nil)) (t (error "Keyword argument %s not one of (:query :in :all-files :recursive :prompt :args)" (car --cl-keys--)))))) (progn (if (executable-find "rg") nil (user-error "Couldn't find ripgrep in your PATH")) (require (quote counsel)) (let* ((this-command (quote counsel-rg)) (project-root (or (doom-project-root) default-directory)) (directory (or in project-root)) (args (concat (if all-files " -uu") (if recursive nil " --maxdepth 1") " " (mapconcat ... args " ")))) (setq deactivate-mark t) (counsel-rg (or query (if (doom-region-active-p) (progn ...))) directory args (or prompt (format "rg%s [%s]: " args (cond ... ... ...))))))))
  +ivy-file-search(:query nil :in nil :all-files nil)
  +ivy/project-search(nil)
  funcall-interactively(+ivy/project-search nil)
  call-interactively(+ivy/project-search)
  (let* ((projectile-project-root nil) (disabled-command-function nil) (default-directory (if arg (let* ((projects (and t ...))) (if projects (completing-read "Search project: " projects nil t) (user-error "There are no known projects"))) default-directory))) (call-interactively (cond (t (function +ivy/project-search)) (nil (function +helm/project-search)) ((function projectile-ripgrep)))))
  +default/search-project(nil)
  funcall-interactively(+default/search-project nil)
  call-interactively(+default/search-project record nil)
  command-execute(+default/search-project record)
  counsel-M-x-action("+default/search-project")

Additional details:
I believe it can be related to Issue 3215  and
commit (https://github.com/hlissner/doom-emacs/commit/f548ab600c95b36790c018c6027862e5d01e7d70)
My private config is essentially the default template (I keep a clean environment for bug testing).
doom.zip
Steps to reproduce:

Run emacs on Windows 10 with standard configuration
'M-x +default/search-project`
Observe error

System information:

emacs   version    26.3
        features   XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND NOTIFY ACL GNUTLS LIBXML2 ZLIB TOOLKIT_SCROLL_BARS MODULES THREADS LCMS2
        build      Dec 25, 2019
        buildopts  (--without-compress-install --without-dbus --with-modules 'CFLAGS=-O2 -g3')
        windowsys  w32
        daemonp    server-running
doom    version    2.0.9
        build      HEAD -> develop
        dir        ~/.config/doom/
system  type       windows-nt
        config     x86_64-w64-mingw32
        shell      C:/bin/emax64-26.3/libexec/emacs/26.3/x86_64-w64-mingw32/cmdproxy.exe
        uname      n/a
        path       (c:/bin/ C:/Program Files (x86)/Common Files/Oracle/Java/javapath C:/Windows/system32 C:/Windows C:/Windows/System32/Wbem C:/Windows/System32/WindowsPowerShell/v1.0/ C:/Windows/System32/OpenSSH/ C:/ProgramData/chocolatey/bin C:/Program Files/Calibre2/ C:/Program Files (x86)/gnupg/bin C:/Program Files/TortoiseGit/bin C:/Program Files (x86)/ZeroTier/One/ C:/Program Files (x86)/NVIDIA Corporation/PhysX/Common C:/Program Files (x86)/Intel/Intel(R) Management Engine Components/DAL C:/Program Files/Intel/Intel(R) Management Engine Components/DAL C:/Program Files/ImageMagick-7.0.10-Q16 C:/texlive/2020/bin/win32 C:/Program Files (x86)/Wolfram Research/WolframScript/ ~/AppData/Local/Microsoft/WindowsApps C:/tools/Cmder C:/Program Files/Oracle/VirtualBox ~/AppData/Local/Pandoc/ . c:/bin/emax64-26.3/libexec/emacs/26.3/x86_64-w64-mingw32)
config  envfile    nil
        elc-files  0
        modules    (:completion company ivy :ui doom doom-dashboard doom-quit hl-todo modeline ophints (popup +defaults) vc-gutter vi-tilde-fringe workspaces :editor file-templates fold snippets :emacs dired electric undo vc :checkers syntax :tools (eval +overlay) lookup magit :lang emacs-lisp markdown org sh :config (default +bindings +smartparens))
        packages   (n/a)
        unpin      (n/a)
        elpa       (n/a)

A little more investigation shows that probably it is not due to windows.
The guilty commit is
https://github.com/hlissner/doom-emacs/commit/235ad55ba12cb1ea47217894ce3af53cfb4075b6
** Force counsel-rg-base-command to be a list **
And this does not work with this line
(concat counsel-rg-base-command (counsel--rg-targets))

in build/counsel/counsel.el. Thus it seems it is an upstream bug. Counsel doesn't truely support counsel-rg-base-command to be a list.

Have you updated Doom + packages? I don't see a (concat counsel-rg-base-command (counsel--rg-targets)) in the pinned commit of counsel (swiper).

Yeah, just noticed that was my fault. I have been issuing doom upgrade but it doesn't seem to update packages as stated in the manual. Now I ran doom update and apparently I fell really behind on package updating.

Everything works now. I don't know why doom upgrade was not pulling in new packages. I'll investigate and if this reoccurs I'll post a new bug report.
Sorry for bothering.

authors: guraltsev hlissner

repo: hlissner/doom-emacs


text:
Need a code guidelines

Hello, and sorry for my poor English.
I want to contribute some code to doom-emacs. But I was in trouble when I read existing code.
I don't understand the difference between (defun doom-xxx), (defun doom|xxx) and (defun doom/xxx).
I think you need a code guidelines like spacemacs/CONVENTIONS.org.But I am also an emacs-lisp rookie, so I can't submit PR for you, sorry.
Thank you.

Thanks.

With 3f6decf we have contributing guidelines (though documentation is an ongoing effort), so I'll consider this resolved. Thanks for bringing it to my attention!

authors: hlissner re-mx yuzumx


text:
Add new symbols.

Thanks!

authors: hlissner mdssjc


text:
Clean up and refactor css variables

Currently the css variables are a hot mess, because of multiple people just merging things. They are not consistent and have multiple duplicates. This needs to be refactored and cleaned up.

when we do this, we need to add a reset.css to get rid off all the default user agent settings.

Pretty sure this is done.

authors: Kronnox bstopp

repo: hlxsites/famous-smoke-cigaradvisor


text:
Return json of ecall

See https://github.com/hm-riscv/venusbackend/pull/3

this has been manually resolved by this commit

repo: hm-riscv/venus


text:
Remove react from peer-dependencies?

Hello and thanks for your awesome work!
I am toying witch a "vanilla-typescript" type game and was surprised to see that statery installed react into my project.
Everything works as expected of course, so no real issues here - just curious if that could be removed? I would be happy to shoot a PR 👍

Since this issue was created, react was made a peerDependency (which should cause it to not automatically be installed alongside statery), and a recent commit explicitly marked it as optional:
  "peerDependencies": {
    "react": ">=18.0"
  },
  "peerDependenciesMeta": {
    "react": {
      "optional": true
    }
  }

I'm closing this issue for the time being, but please feel free to let me know if this doesn't solve the issue for you.
Since the core of Statery is framework agnostic and there could theoretically be additional framework bindings in the future, it's possible that the project will at some point be split into multiple micropackages, or at least provide separate per-framework entrypoints.

authors: DennisSmuda hmans

repo: hmans/statery