text:
Apps Version Report - Space in app name - Forbidden, You don't have permission to access this resource.

Having issues with data in reports when the app to track has a space in the name. Which is most Adobe products, DetectX Swift, Cricut Design Space and a few others. The apps will appear in the Apps Version Report with no data. Clicking on the menu to show "known versions of the app" results in a page that says "Forbidden. You don't have permission to access this resource."
If I change my .yml file and replace the space with an underscore the apps list will show the various versions available in the module, but then clicking on the menu or any of the versions available to see the devices results in an empty list.
Before: (with space)
0:
    app1: {appName: "Adobe Acrobat", widget: "app"}
    app2: {appName: "Adobe Acrobat Reader DC", widget: "app"}
    app25: {appName: "Creative Cloud", widget: "app"}

Result:
After: (without space)
0:
    app1: {appName: "Adobe_Acrobat", widget: "app"}
    app2: {appName: "Adobe_Acrobat_Reader_DC", widget: "app"}
    app25: {appName: "Creative_Cloud", widget: "app"}

Result:

I'm experiencing this issue too.

Hmm, I am using 5.8.0.4284 with PHP 8.2.7 on macOS 13.6 and don't see this issue.

If I put apps into the .env for "App Version Report" I have no issues with apps with spaces in the names.
APPS_TO_TRACK="Creative Cloud, Adobe Acrobat Reader, ...
If I put use a local dashboard with an app listed I have no issues with apps with spaces in the names:
app1: {appName: "DetectX Swift", widget: "app"}
If I use the example url provided in https://github.com/munkireport/munkireport-php/issues/1536 I still have no issues:
sample url:
https://munkireport.server/module/inventory/items/AppleScript Editor

Hmm, I wonder if it is time to compare Apples to Oranges. I am using the Docker image with PHP Version 8.2.7, MySql version 5.7.42 on a Ubuntu Server.

I too do not have this issue when using the provided sample URL. My guess is that your web server doesn't like it or is configured incorrectly. For reference, I'm using nginx

@davisriglsd and I are both using Docker, so it's possible something needs to be adjusted for the bundled version of Apache.

authors: davisriglsd flammable miawri precursorca tuxudo

repo: munkireport/munkireport-php


text:
Update README.md

asdf asd

Can one of the admins verify this patch?

source: gharchive/pull-request

repo: muratcabuk/dotnet-docker-examples

license: MIT

license_type: permissive

license_source: github-api


text:
Bug:  ReferenceError: i is not defined

Is there an existing issue for this?

[X] I have searched the existing issues

Which Mux Elements/Packages does this apply to? Select all that apply
mux-player, mux-player-react
Which browsers are you using?
Chrome, Firefox, Edge ("Edgeium")
Which operating systems are you using?
Other (add details below)
Description
We recently switched over to the React Mux Player (@mux/mux-player-react 1.10.1). Ever since we started using it, we've been seeing errors in Sentry for a subset of our users that look like this:
ReferenceError: i is not defined
The errors have no stack trace so it's hard for me to track down. As a clue, it seems to happen right after a GET request like this:
https://chunk-gcp-us-east4-vop1.fastly.mux.com/v1/chunk/yv02gDneyboLxusAyAbulcRXJrl00tWiNx9wh3Ccnw58PHTtB1yTqCKSiOfsM143hK01ycp02801HlFc6XjbGHD4mSxI02eQVznv1wOfb4TigLxd8/0.ts?skid=default&signature=NjQ2M2QyYjBfYTY4NjA4ODA2NDVlOWFkY2RmM2FhY2ZhYWM4YWVjZWJhOGVhYTk0YzRlODVlYjU0ZDZiYmZmZjhkNDJiZWFiNA==&zone=0&CMCD=br%3D2516%2Ccid%3D4419%2Cd%3D5000%2Cmtp%3D500%2Cot%3Dav%2Csf%3Dh%2Csid%3D"e6e87cc3-b0eb-4638-8fcc-edebbce99885"%2Csu%2Ctb%3D4755
And it seems to involve a URL like this: blob:https://ourdomain.co/5e8ec423-d265-4e83-996d-8d295cb557fc at line 1:371986
I have attached a screenshot from Sentry in case there is other debug info in there that you find useful.

Reduced test case
No response
Steps to reproduce
Intermittent and i've had trouble reproducing it. We're seeing it happen w/multiple end-users but haven't been able to replicate ourselves
Current Behavior
See description
Expected Behavior
See description
Errors
ReferenceError: i is not defined
What version of the package are you using?
@mux/mux-player-react 1.10.1

Hi @eddiesiegel, I'm facing the exact same issue and can't pin down the cause.
Were you able to find a fix for it?

Hmmm that is odd.
For some context: blob:... in the video src is expected, that's how Media Source Extensions work.
The fact that there's no stack trace is going to make this quite difficult to debug.
Can you find the Mux Data view in your Mux Data dashboard that corresponds to this error? If you're passing in the user ID as metadata you should be able to use that as a filter to trace it back

Yep I was able to find it. Nothing jumps out at me (no errors, etc.), but here are some screenshots. Please let me know if there is any other info I can share to help you investigate.

Hey @eddiesiegel and @kartikcho - I'm actually investigating this right now. It's looking like the issue is pretty deep in the weeds of layered builds + how hls.js generates its web worker-based transmuxer (https://github.com/video-dev/hls.js/blob/master/src/demux/inject-worker.ts#L15-L31). If you all don't mind, could you share any build tools (including versions) and/or app frameworks (including versions) that you are using when you see this error?

Not sure what's relevant so I dumped some of my config files into ChatGPT and it produced this summary:
Build Tools and Versions:

Babel: Not explicitly listed, but used within the project (Inferred from Babel plugins and presets)
Webpack: 4.46.0
Webpack-cli: 3.3.12
Fork-ts-checker-webpack-plugin: 6.3.3
Sentry Webpack plugin: 1.20.0
Webpack-dev-server: 4.2.1 (devDependency)
Rails webpacker: 5.4.3

App Frameworks and Versions:

React: 18.0.0
React-dom: 18.0.0
Mux-player-react: 1.11.1
React-bootstrap: 2.0.0-beta.4
Rails UJS: 6.0.0

Typescript Version:

Typescript: 4.2.4

Key Babel Configurations:

Presets used: @babel/preset-env, @babel/preset-react, @babel/preset-typescript
UseBuiltIns: entry for @babel/preset-env and true for @babel/preset-react
Modules: commonjs if testing else false for @babel/preset-env
Corejs: 3.30 for @babel/preset-env and false for @babel/plugin-transform-runtime

Key Typescript Configurations:

Target: es5
Module: es6
ModuleResolution: node
JSX: react

Webpack Module Rules:

.mjs file handling: Explicitly configuring Webpack to treat '.mjs' files as ECMAScript modules by setting the 'type' to 'javascript/auto'. This is specifically for the 'node_modules/@mux/mux-player-react' directory, which includes '.mjs' files.

Webpack Resolve Configurations:

extensions: '.mjs' files are added in the resolved extensions to ensure they are properly imported.

Babel Loader Exclusion:

node_modules exclusion: 'mux-player-react' is added to Babel loader exclusion list. This prevents Babel from transpiling certain node_modules packages that were causing console errors when playing videos.

Webpack Optimization:

SplitChunks: Enabled to split the code into separate chunks, which helps improve the loading time of the app.

This is great, thanks @eddiesiegel. I'll be sure to keep you posted on any updates from my end.

@eddiesiegel I think we saw this with another customer and it might have been solved with v1.11.4 when we changed how Hls.js is bundled. Can you upgrade and see if you're still experiencing the issue?

@dylanjha @cjpillsbury I can confirm upgrading to 1.11.4 fixed it. Thank you!

source: gharchive/issue

authors: cjpillsbury dylanjha eddiesiegel kartikcho

repo: muxinc/elements


text:
metadata and limits

Added extra auth data
Added game metadata
Added library limits for certain calls

I would like the new overrides to have a name denoting they take a GameID (like AuthenticateWithPINAndGameID) which is wordy but more explicit.
The behavior to ignore an empty one is fine, and the existing APIs can forward to the GameID variants with a null / empty string.
Otherwise 👍

authors: albert-wang soulfoam

repo: muxy/gamelink-cpp


text:
Is it possible for lower and dress virtual try on?

Hi, Thanks for your work.
I have a question for your work.
Is it possible for lower and dress virtual try on?
Or it is possible for only upper body?

Thanks, IMAGDressing-v1 support lower and dress.  You can try it.

authors: DeepAlgoExpert muzishen

repo: muzishen/IMAGDressing


text:
Doesn't work when not building in release mode

Hey not sure if it is usbd-serial or usb-device or maybe even the stm32f4xx-hal
but after much testing if found that when I build without the release flag Windows can't read the address of the device. Usbdeview shows me "Unknown USB Device (Device Descriptor Request Failed)"
Using poormans debugging with semihosting I saw that it gets stuck in the SerialPort::new call. Trying to debug using gdb it just land in asm/lib.rs (I also have no idea what I am doing never used gdb before :) )
I am totally new to embedded and rust, so this might just be a complete fail on my side. Also my instincts tell me that the compiler optimizing stuff should not be the reason for the code to work. But then it's super low level stuff I guess maybe optimizations change some register interactions. (But again shouldn't optimizations break code)
Editing the Release Profile:
just adding debug = true works
while debug = true playing with opt-level:
opt-level=0 - does not work
opt-level=1 - does not work
opt-level=2 - does work
opt-level=2 - also works
didn't test "z" and "s"
That are my findings :)
I still have the feeling it's just some stupid setting I added and totally my fault (Or I missed some note in the documentation) :)))
Config:

rustc 1.50.0 (cb75ad5db 2021-02-10)
building for thumbv7em-none-eabihf

Probably you missed a warning during the build: https://github.com/stm32-rs/synopsys-usb-otg/blob/master/build.rs#L6
The reason behind it is a combination of low opt-level (that makes your code really slow) and high expectations of USB re: response time for requests during the device enumeration process. If the firmware is not fast enough, USB device fails to enumerate.

Thanks,
that makes total sense, didn't think about timing constrains.
But I didn't get the build warning, my cargo.lock states that I am using the 0.2.4 which should include your change. (atleast the github tagged version does)
Adding them to my build.rs for testing shows me the warning

As explained this isn't directly related to usbd-serial so I'm closing this. Please open a new issue at the relevant repository if the problem persists.

authors: Disasm Lukas-Sturm mvirkkunen

repo: mvirkkunen/usbd-serial


text:
Window is open and checkbox is checked added to ui.rs example

Added demonstration how to use egui::Window::open(self, &'open mut bool). Since I am a newcomer to rust I have been struggling with passing a mutable bool. Should be useful for other newcomers as well since closing or opening a window is such a common use case.
Hope that helps.

Yes, it can. But then the default state will be a closed window unless we provide a manual Default implementation for UiState. Is there a way to make is_open default to true and leave #[derive(Default)] on UiState?

Using only UiState now. Left default as is but introduced configure_ui_state startup function to set is_window_open to true.

can we merge it or is there something else to be fixed?

Merged. Thank you!

authors: mvlabat tomuxmon


text:
14.4.5 grid broken - GridSorterComponentRenderer not found

Since 14.4.5 I get:
Caused by: java.lang.ClassNotFoundException: com.vaadin.flow.component.grid.GridSorterComponentRenderer
	at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:264)
	at com.github.mvysny.kaributesting.v10.GridKt.<clinit>(Grid.kt:464)

This is caused by this MR: https://github.com/vaadin/vaadin-flow-components/pull/535

Thanks! Yes, it has been fixed in git commit 132a6d48016428b4af415c5251208fb9709d8829 and should be present in Karibu-Testing 1.2.7 or higher. Please try it out and let me know :+1:

Thanks! Yes, it has been fixed in git commit 132a6d48016428b4af415c5251208fb9709d8829 and should be present in Karibu-Testing 1.2.7 or higher. Please try it out and let me know :+1:

According to https://github.com/mvysny/karibu-testing/tree/master/karibu-testing-v10 14.4.5 is not compatibel to 1.2.x?

According to https://github.com/mvysny/karibu-testing/tree/master/karibu-testing-v10 14.4.5 is not compatibel to 1.2.x?

It is, as long as you're not using the compatibility (bower) mode.

It is, as long as you're not using the compatibility (bower) mode.

Ah thanks, didn't see that.
Is there a migration guide from 1.1 to 1.2?
I rewrote some imports and added exclusions to some dependencies, now it seems to work, but I get:
com.vaadin.flow.router.NotFoundException: No route found for given navigation target!

Ah thanks, didn't see that.
Is there a migration guide from 1.1 to 1.2?
I rewrote some imports and added exclusions to some dependencies, now it seems to work, but I get:
com.vaadin.flow.router.NotFoundException: No route found for given navigation target!

I don't think there's a migration guide yet, I was hoping that all issues would be straightforward to fix :+1:
Regarding the NotFoundException: please see FAQ https://github.com/mvysny/karibu-testing#faq . Are you using autoDiscoverViews()?

I don't think there's a migration guide yet, I was hoping that all issues would be straightforward to fix :+1:
Regarding the NotFoundException: please see FAQ https://github.com/mvysny/karibu-testing#faq . Are you using autoDiscoverViews()?

Yes, we're specifying 2 packages. I assume this is working, as it was working before I bumped the karibu version

Yes, we're specifying 2 packages. I assume this is working, as it was working before I bumped the karibu version

I'm currently checking if its showing the same behavior on jenkins, if it does, I'm coming back with more info

I'm currently checking if its showing the same behavior on jenkins, if it does, I'm coming back with more info

Sure, but please open a new bug report for this. Also you can check whether the route in question is present in the Routes.routes set.

Sure, but please open a new bug report for this. Also you can check whether the route in question is present in the Routes.routes set.

authors: mvysny thigg

repo: mvysny/karibu-testing


text:
add 'ignore-pinned' and 'ignore-unfound' to outdated command

Added feature requested subcommand flags for outdated

Thanks, I'll merge this into my current dev branch - the necessary functionality should be easy to implement, will try to provide a new release soon

authors: mwallner virtualex-itv

repo: mwallner/rocolatey


text:
Problem with MWPhotoBrowser and UIWebView

Hi,
I really like MWPhotoBrowser, but i have one problem:
I create the photos with the photoWithURL: method. I also have an UIWebView in my application. The problem appears, as soon as a scroll quick through the photos: Instead of some photos, the activity indicator appears and the photos must be reload. When I remove the UIWebView, everything works fine.
You can try this very easy, just create in the Menu class of the demo project a UIWebView.
Sorry, but my english is not so good.
Thanks,
Mikkell

Hi Mikkell, I'm unable to recreate this issue as we use MWPhotoBrowser and UIWebViews on our own projects and don't suffer from the same problem. If you can provide a sample project that recreates this issue then I'll happily take a look.
— Michael

authors: ghost mwaterfall

repo: mwaterfall/MWPhotoBrowser

license: mit

license_source: bigquery


text:
When you have 2 rte's on one page only the first one is updating the selection

When you have multiple rte's at the same time the second rte doesn't update the selection-state as it should. Only the first one on the page is updated correctly.

I'm not sure what's causing your issue, but here's a sample repository that shows two editors on one page where as far as I can tell selection state updates as normal.  The repo uses create-elm-app if you want to run it locally.
Two editors demo: https://mweiss.github.io/elm-editor-two-on-one-page/build/
Project repo: https://github.com/mweiss/elm-editor-two-on-one-page

authors: mweiss quintenkasteel

repo: mweiss/elm-rte-toolkit


text:
Update JSFiddle examples in blog post

Looks like the new API in 0.5.0 broke your excellent blog post.
I would suggest either pinning the JSFiddles to a commit, or reworking them to use mobservable.ObservingComponent. I have done so in this update.

Awesome, thanks! I had it as pending todo, but then I lost my internet connection and forgot about it. Thanks for fixing :)

authors: STRML mweststrate

repo: mweststrate/MOBservable


text:
Character Missing Unit Tests

Being and child classes don't yet have unit tests.

Done as of #83 .

repo: mxashlynn/ParquetClassLibrary


text:
🛑 Website Load Balancers is down

In c92af0f, Website Load Balancers ($WEB_SERVER) was down:

HTTP code: 522
Response time: 15417 ms

Resolved: Website Load Balancers is back up in 7d58ff3.

repo: myavsuper/upptime


text:
sqlite Long type value convert bug

MyBatis version
3.5.14
Database vendor and version
sqlite  3.45.0
sqlite-jdbc 3.46.1.0
Test case or example project
example project:  https://github.com/sailingsky/sqlite-test.git
Steps to reproduce
there are two records in table test:
id       name
22	test22
850361281191579648	 tt

run post request: https://localhost:8083/test/maps
Expected result
[{"name":"test22","id":22},{"name":" tt","id":850361281191579648}]

Actual result
[{"name":"test22","id":22},{"name":" tt","id":1881903104}]

the second record's id value is wrong. the related issue had been submit to sqlite-jdbc: https://github.com/xerial/sqlite-jdbc/issues/1177 . FYI

Hello @sailingsky ,
You have already found the real cause, it seems.
When you are using java.util.Map as the result type, MyBatis calls ResultSet#getColumnClassName() to determine the Java type of the column.
And the driver determines the result based on the column value of the first row which clearly is a flawed logic and causes the problem you are experiencing.
https://github.com/xerial/sqlite-jdbc/blob/02bac1a4a5d28a08334d197d3a3bb246fab9cb05/src/main/java/org/sqlite/jdbc3/JDBC3ResultSet.java#L583-L588
I'll watch the issue https://github.com/xerial/sqlite-jdbc/issues/1177 .
Until it gets fixed, you may have to use a custom class (POJO) instead of java.util.Map.
I am going to close this as it is a driver issue and there is nothing to fix in MyBatis side.

authors: harawata sailingsky


text:
The better way to add native paging support for MyBatis.

The main source are:

PagingExecutor
Dialect

modify source:

XMLConfigBuilder
Configuration

others are three implementation class for Dialect:

DerbyDialect
HSQLDBDialect
MySQLDialect

enhance RowBounds:

PageRowBounds

six test for PagingExecutor:

PagingBatchExecutorTest
PagingCachingBatchExecutorTest
PagingCachingReuseExecutorTest
PagingCachingSimpleExecutorTest
PagingReuseExecutorTest
PagingSimpleExecutorTest

settings config:
<configuration>
    <settings>
        <setting name="usePagingExcutor" value="true"/>
        <setting name="dialect" value="MySQL"/>
    </settings>
</configuration>

I think this is a better way for mybatis to support paging.
It will make stronger mybatis.
Allows developers to more easily.

@kazuki43zoo @harawata What do you think?
If this PR been accept, I'll add support for  mariadb, sqlite, oracle,  postgresql, sqlserver2005+, db2, informix, h2, sqlserver2012+

Somthing about this PR
public interface Dialect {

  boolean doCount(MappedStatement ms, Object parameterObject, RowBounds rowBounds);

  String getCountSql(MappedStatement ms, BoundSql boundSql, Object parameterObject, RowBounds rowBounds, CacheKey countKey);

  void afterCount(long count, Object parameterObject, RowBounds rowBounds);

  boolean doPage(MappedStatement ms, Object parameterObject, RowBounds rowBounds);

  Object processParameterObject(MappedStatement ms, Object parameterObject, BoundSql boundSql, CacheKey pageKey);

  String getPageSql(MappedStatement ms, BoundSql boundSql, Object parameterObject, RowBounds rowBounds, CacheKey pageKey);

  <E> List<E> afterPage(List<E> pageList, Object parameterObject, RowBounds rowBounds);

}

When we want to paging a sql statement; We want not only the paging result, but also the total count of sql statement;
So the Dialect support two kinds of methods, count methods and page methods.
Although we often need to obtain the total number, but not all cases are like this.
For example, obtain the latest 10 news, at this point we don't need to know how many news are.
So we can control PageExecutor logic by the interface .
Why most methods hava a MappedStatement paremeter, but did not use in the given implementation class.
Because the implementation class is given based on RowBounds and PageRowBounds simple implementation, so there's no use.
MappedStatement contains much useful information, we can use MappedStatement#getId matching the specified method and automatically paged.
PageExecutor is support L1 Cache and L2 Cache. You can also use countKey and pageKey to control caching.
By default, the paging parameters do not affect the count query cache, but will affect the paging query is cached.
When you use the same criteria paging through a query, count will use the cache , and the paging query will excute in database.

Thanks for the PR!
Another form of #694 , it seems.
MyBatis (core) does not generate SQL by itself and there is no plan to change that design.
I haven't had the time to review the details, but is there any difficulty with implementing this as a plugin?

It's easy  in implementing this as a plugin：
https://github.com/pagehelper/Mybatis-PageHelper/tree/5.0
This #694 PR have some copied code from this project.
I will continue to enhance this plugin.

MyBatis (core) does not generate SQL by itself and there is no plan to change that design.

+1
I think the better way to be used is a plug-in  extension rather than the MyBatis (core) .

Hi @abel533 , thanks for your aggressive contribution !!
But we won't merge this PR because it does not match with design of MyBatis Core feature.

I will continue to enhance this plugin.

👍

Hi @harawata, it might be good creating a new wiki page that introduces useful plugins. What do you think ?

@abel533

It's easy in implementing this as a plugin

Glad to hear that. It probably is the best approach at the moment.
(1k stars...!? Impressive! Thank you for sharing! =D)
@wuwen5 , Thanks for the feedback! =)
@kazuki43zoo , +1

Hi @abel533, Can you add an English reference page for page helper plugin ? If you can be prepared it , i want to consider to introduce your page helper plugin on MyBatis wiki page.
Thanks.

@kazuki43zoo I am pleased to provide an English language reference documentation.
After Pagehelper 5.0 version released. I will provide an English document.

Plugin need to parse SQL statements, but the SQL statement is very complicated, sometimes it is hard to parse, plugin cannot support.
SQL statements in an XML file into the "select" and "from" two parts, no matter how complex SQL statements, can easily generate paging statements.


text:
Response to timeRequest (C_INTERNAL->I_TIME) not handled anymore?

Hi,
I have a few nodes requesting time from MyController, this has worked great in the past but now when running latest SNAPSHOT. I have no idea how far back this stopped working to be honest. Now only see node requesting time but no reponse in the logs:

14229:2017-01-12 13:53:16,459 DEBUG [Thread-508] [org.mycontroller.standalone.gateway.serialport.SerialDataListenerjSerialComm:64] Received a message:[0;255;3;0;9;TSF:MSG:READ,35-35-0,s=255,c=3,t=1,pt=0,l=0,sg=0:]
14230-2017-01-12 13:53:16,462 DEBUG [Thread-508] [org.mycontroller.standalone.gateway.serialport.SerialDataListenerjSerialComm:64] Received a message:[35;255;3;0;1;]

Is this a known issue?

@Redferne this issue was fixed recently. You can test this latest SNAPSHOT version. Thank you!

authors: Redferne jkandasa

repo: mycontroller-org/mycontroller


text:
Insight.io badge?

Hi mastani,
Thanks for include an insight.io link to your README.md, I just want to point out that we have badge you can use to directly include in your md file:

Here are some examples of other repositories using our badge:
https://github.com/kaonashi-tyc/zi2zi
https://github.com/tesseract-ocr/tesseract

cool, thanks!

authors: chongzhe mydearxym

repo: mydearxym/mastani


text:
🛑 instagram is down

In e4529d5, instagram (https://instagram.com) was down:

HTTP code: 429
Response time: 734 ms

Resolved: instagram is back up in 45cfd5f.

repo: mymovielist/uptime


text:
当我使用DeepSpeed试图多卡训练时，出现错误Expected all tensors to be on the same device, but found at least two devices, cuda:1 and cuda:0!

Cupy Buffers Initialized Successfully.
Pop out errors
Finished the initialization step at rank 0
Pop out errors
Finished the initialization step at rank 1
Using /root/.cache/torch_extensions/py310_cu116 as PyTorch extensions root...
Using /root/.cache/torch_extensions/py310_cu116 as PyTorch extensions root...
Emitting ninja build file /root/.cache/torch_extensions/py310_cu116/utils/build.ninja...
Building extension module utils...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
ninja: no work to do.
Loading extension module utils...
Time to load utils op: 0.07267284393310547 seconds
Loading extension module utils...
Time to load utils op: 0.1021263599395752 seconds
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /apdcephfs_cq3/share_1567347/share_info/mingxiaoli/chatglm_finetune_test/finetune.py:180 in      │
│                                                                                │
│                                                                                │
│   177                                                                                │
│   178                                                                                │
│   179 if name == "main":                                                                 │
│ ❱ 180 │   main()                                                                                │
│   181                                                                                │
│                                                                                │
│ /apdcephfs_cq3/share_1567347/share_info/mingxiaoli/chatglm_finetune_test/finetune.py:173 in main │
│                                                                                │
│   170 │   │   # callbacks=[TensorBoardCallback(writer)],                                         │
│   171 │   │   data_collator=data_collator,                                                       │
│   172 │   )  # 初始化训练器，用于训练模型                                                        │
│ ❱ 173 │   trainer.train()  # 训练模型                                                            │
│   174 │   # writer.close()  # 关闭TensorBoard写入器                                              │
│   175 │   # save model                                                                           │
│   176 │   model.save_pretrained(training_args.output_dir)  # 保存微调后的模型到指定目录          │
│                                                                                │
│ /opt/conda/lib/python3.10/site-packages/transformers/trainer.py:1662 in train                    │
│                                                                                │
│   1659 │   │   inner_training_loop = find_executable_batch_size(                                 │
│   1660 │   │   │   self._inner_training_loop, self._train_batch_size, args.auto_find_batch_size  │
│   1661 │   │   )                                                                                │
│ ❱ 1662 │   │   return inner_training_loop(                                                       │
│   1663 │   │   │   args=args,                                                                    │
│   1664 │   │   │   resume_from_checkpoint=resume_from_checkpoint,                                │
│   1665 │   │   │   trial=trial,                                                                  │
│                                                                                │
│ /opt/conda/lib/python3.10/site-packages/transformers/trainer.py:1929 in _inner_training_loop     │
│                                                                                │
│   1926 │   │   │   │   │   with model.no_sync():                                                 │
│   1927 │   │   │   │   │   │   tr_loss_step = self.training_step(model, inputs)                  │
│   1928 │   │   │   │   else:                                                                     │
│ ❱ 1929 │   │   │   │   │   tr_loss_step = self.training_step(model, inputs)                      │
│   1930 │   │   │   │                                                                             │
│   1931 │   │   │   │   if (                                                                      │
│   1932 │   │   │   │   │   args.logging_nan_inf_filter                                           │
│                                                                                │
│ /opt/conda/lib/python3.10/site-packages/transformers/trainer.py:2699 in training_step            │
│                                                                                │
│   2696 │   │   │   return loss_mb.reduce_mean().detach().to(self.args.device)                    │
│   2697 │   │                                                                                │
│   2698 │   │   with self.compute_loss_context_manager():                                         │
│ ❱ 2699 │   │   │   loss = self.compute_loss(model, inputs)                                       │
│   2700 │   │                                                                                │
│   2701 │   │   if self.args.n_gpu > 1:                                                           │
│   2702 │   │   │   loss = loss.mean()  # mean() to average on multi-gpu parallel training        │
│                                                                                │
│ /apdcephfs_cq3/share_1567347/share_info/mingxiaoli/chatglm_finetune_test/finetune.py:86 in       │
│ compute_loss                                                                                │
│                                                                                │
│    83                                                                                │
│    84 class ModifiedTrainer(Trainer):                                                            │
│    85 │   def compute_loss(self, model, inputs, return_outputs=False):                           │
│ ❱  86 │   │   return model(                                                                      │
│    87 │   │   │   input_ids=inputs["input_ids"],                                                 │
│    88 │   │   │   labels=inputs["labels"],                                                       │
│    89 │   │   ).loss                                                                             │
│                                                                                │
│ /opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py:1194 in _call_impl            │
│                                                                                │
│   1191 │   │   # this function, and just call forward.                                           │
│   1192 │   │   if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks o  │
│   1193 │   │   │   │   or _global_forward_hooks or _global_forward_pre_hooks):                   │
│ ❱ 1194 │   │   │   return forward_call(*input, **kwargs)                                         │
│   1195 │   │   # Do not call functions when jit is used                                          │
│   1196 │   │   full_backward_hooks, non_full_backward_hooks = [], []                             │
│   1197 │   │   if self._backward_hooks or _global_backward_hooks:                                │
│                                                                                │
│ /opt/conda/lib/python3.10/site-packages/deepspeed/utils/nvtx.py:11 in wrapped_fn                 │
│                                                                                │
│    8 │   function call."""                                                                       │
│    9 │   def wrapped_fn(*args, **kwargs):                                                        │
│   10 │   │   get_accelerator().range_push(func.qualname)                                     │
│ ❱ 11 │   │   ret_val = func(*args, **kwargs)                                                     │
│   12 │   │   get_accelerator().range_pop()                                                       │
│   13 │   │   return ret_val                                                                      │
│   14                                                                                │
│                                                                                │
│ /opt/conda/lib/python3.10/site-packages/deepspeed/runtime/engine.py:1846 in forward              │
│                                                                                │
│   1843 │   │   if self.fp16_auto_cast():                                                         │
│   1844 │   │   │   inputs = self._cast_inputs_half(inputs)                                       │
│   1845 │   │                                                                                │
│ ❱ 1846 │   │   loss = self.module(*inputs, **kwargs)                                             │
│   1847 │   │                                                                                │
│   1848 │   │   if self.zero_optimization_partition_weights():                                    │
│   1849 │   │   │   # Disable automated discovery of external parameters                          │
│                                                                                │
│ /opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py:1194 in _call_impl            │
│                                                                                │
│   1191 │   │   # this function, and just call forward.                                           │
│   1192 │   │   if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks o  │
│   1193 │   │   │   │   or _global_forward_hooks or _global_forward_pre_hooks):                   │
│ ❱ 1194 │   │   │   return forward_call(*input, **kwargs)                                         │
│   1195 │   │   # Do not call functions when jit is used                                          │
│   1196 │   │   full_backward_hooks, non_full_backward_hooks = [], []                             │
│   1197 │   │   if self._backward_hooks or _global_backward_hooks:                                │
│                                                                                │
│ /workspace/LLM-Adapters/peft/src/peft/peft_model.py:532 in forward                               │
│                                                                                │
│   529 │   │   **kwargs,                                                                          │
│   530 │   ):                                                                                │
│   531 │   │   if not isinstance(self.peft_config, PromptLearningConfig):                         │
│ ❱ 532 │   │   │   return self.base_model(                                                        │
│   533 │   │   │   │   input_ids=input_ids,                                                       │
│   534 │   │   │   │   attention_mask=attention_mask,                                             │
│   535 │   │   │   │   inputs_embeds=inputs_embeds,                                               │
│                                                                                │
│ /opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py:1194 in _call_impl            │
│                                                                                │
│   1191 │   │   # this function, and just call forward.                                           │
│   1192 │   │   if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks o  │
│   1193 │   │   │   │   or _global_forward_hooks or _global_forward_pre_hooks):                   │
│ ❱ 1194 │   │   │   return forward_call(*input, **kwargs)                                         │
│   1195 │   │   # Do not call functions when jit is used                                          │
│   1196 │   │   full_backward_hooks, non_full_backward_hooks = [], []                             │
│   1197 │   │   if self._backward_hooks or _global_backward_hooks:                                │
│                                                                                │
│ /opt/conda/lib/python3.10/site-packages/accelerate/hooks.py:165 in new_forward                   │
│                                                                                │
│   162 │   │   │   with torch.no_grad():                                                          │
│   163 │   │   │   │   output = old_forward(*args, **kwargs)                                      │
│   164 │   │   else:                                                                              │
│ ❱ 165 │   │   │   output = old_forward(*args, **kwargs)                                          │
│   166 │   │   return module._hf_hook.post_forward(module, output)                                │
│   167 │                                                                                │
│   168 │   module.forward = new_forward                                                           │
│                                                                                │
│ /root/.cache/huggingface/modules/transformers_modules/chatglm-6b/modeling_chatglm.py:1160 in     │
│ forward                                                                                │
│                                                                                │
│   1157 │   │   use_cache = use_cache if use_cache is not None else self.config.use_cache         │
│   1158 │   │   return_dict = return_dict if return_dict is not None else self.config.use_return  │
│   1159 │   │                                                                                │
│ ❱ 1160 │   │   transformer_outputs = self.transformer(                                           │
│   1161 │   │   │   input_ids=input_ids,                                                          │
│   1162 │   │   │   position_ids=position_ids,                                                    │
│   1163 │   │   │   attention_mask=attention_mask,                                                │
│                                                                                │
│ /opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py:1194 in _call_impl            │
│                                                                                │
│   1191 │   │   # this function, and just call forward.                                           │
│   1192 │   │   if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks o  │
│   1193 │   │   │   │   or _global_forward_hooks or _global_forward_pre_hooks):                   │
│ ❱ 1194 │   │   │   return forward_call(*input, **kwargs)                                         │
│   1195 │   │   # Do not call functions when jit is used                                          │
│   1196 │   │   full_backward_hooks, non_full_backward_hooks = [], []                             │
│   1197 │   │   if self._backward_hooks or _global_backward_hooks:                                │
│                                                                                │
│ /root/.cache/huggingface/modules/transformers_modules/chatglm-6b/modeling_chatglm.py:907 in      │
│ forward                                                                                │
│                                                                                │
│    904 │   │   │   raise ValueError("You have to specify either input_ids or inputs_embeds")     │
│    905 │   │                                                                                │
│    906 │   │   if inputs_embeds is None:                                                         │
│ ❱  907 │   │   │   inputs_embeds = self.word_embeddings(input_ids)                               │
│    908 │   │                                                                                │
│    909 │   │   if past_key_values is None:                                                       │
│    910 │   │   │   if self.pre_seq_len is not None:                                              │
│                                                                                │
│ /opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py:1212 in _call_impl            │
│                                                                                │
│   1209 │   │   │   bw_hook = hooks.BackwardHook(self, full_backward_hooks)                       │
│   1210 │   │   │   input = bw_hook.setup_input_hook(input)                                       │
│   1211 │   │                                                                                │
│ ❱ 1212 │   │   result = forward_call(input, **kwargs)                                           │
│   1213 │   │   if _global_forward_hooks or self._forward_hooks:                                  │
│   1214 │   │   │   for hook in (_global_forward_hooks.values(), *self.forward_hooks.values())  │
│   1215 │   │   │   │   hook_result = hook(self, input, result)                                   │
│                                                                                │
│ /opt/conda/lib/python3.10/site-packages/accelerate/hooks.py:165 in new_forward                   │
│                                                                                │
│   162 │   │   │   with torch.no_grad():                                                          │
│   163 │   │   │   │   output = old_forward(*args, **kwargs)                                      │
│   164 │   │   else:                                                                              │
│ ❱ 165 │   │   │   output = old_forward(*args, **kwargs)                                          │
│   166 │   │   return module.hf_hook.post_forward(module, output)                                │
│   167 │                                                                                │
│   168 │   module.forward = new_forward                                                           │
│                                                                                │
│ /opt/conda/lib/python3.10/site-packages/torch/nn/modules/sparse.py:160 in forward                │
│                                                                                │
│   157 │   │   │   │   self.weight[self.padding_idx].fill(0)                                     │
│   158 │                                                                                │
│   159 │   def forward(self, input: Tensor) -> Tensor:                                            │
│ ❱ 160 │   │   return F.embedding(                                                                │
│   161 │   │   │   input, self.weight, self.padding_idx, self.max_norm,                           │
│   162 │   │   │   self.norm_type, self.scale_grad_by_freq, self.sparse)                          │
│   163                                                                                │
│                                                                                │
│ /opt/conda/lib/python3.10/site-packages/torch/nn/functional.py:2210 in embedding                 │
│                                                                                │
│   2207 │   │   #   torch.embedding_renorm                                                       │
│   2208 │   │   # remove once script supports set_grad_enabled                                    │
│   2209 │   │   no_grad_embedding_renorm(weight, input, max_norm, norm_type)                    │
│ ❱ 2210 │   return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)        │
│   2211                                                                                │
│   2212                                                                                │
│   2213 def embedding_bag(                                                                        │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:1 and cuda:0! (when checking argument for argument index in method wrapper__index_select)
[2023-04-08 16:00:51,018] [INFO] [launch.py:318:sigkill_handler] Killing subprocess 9721
[2023-04-08 16:00:51,398] [INFO] [launch.py:318:sigkill_handler] Killing subprocess 9722
[2023-04-08 16:00:51,399] [ERROR] [launch.py:324:sigkill_handler] ['/opt/conda/bin/python', '-u', 'finetune.py', '--local_rank=1'] exits with return code = 1

我也遇到了这个问题，请问你解决了吗

并没有，我尝试了别人的多卡训练的代码也是同样的错误，我开始是怀疑可能是环境问题，于是我重新配置了docker环境是python3.8,cuda11.0+nvidia的cuda11.6的兼容包,然后cupy是11.0的，其他的环境都重新配置了一遍，但是仍然是有这个错误，但是只用accelerate是能跑起来的，比单纯不用要快很多，可是显存主要是一张用的比较多。按道理Trainer内置了deepspeed跟accelerate，但是正常配置在TrainerAugments中并不起作用也报的这个错误。

@Mr-lonely0
我解决了这个问题，但是解决的方式非常玄学。。。
我的model是用autoModel下载到本地的，于是当我在一个新的容器上没用使用automodel下载过的模型去加载我共享盘下好的chatglm进行ddp的多卡训练时候，就会报这个错误，但是当我把模型modeling_chatglm跟tokenizer_chatglm源代码拉出来，然后改成ChatGLM的tokenizer跟model的类型再跑一遍会报一个维度错误的error，但是当我进行了以上两步之后再改回去AutoModel跟AutoTokenizer，神奇的是居然不报错了，顺利的ddp跑起来了。。
这个流程我试了两遍，简直玄学。。。

@Mr-lonely0 我解决了这个问题，但是解决的方式非常玄学。。。
我的model是用autoModel下载到本地的，于是当我在一个新的容器上没用使用automodel下载过的模型去加载我共享盘下好的chatglm进行ddp的多卡训练时候，就会报这个错误，但是当我把模型modeling_chatglm跟tokenizer_chatglm源代码拉出来，然后改成ChatGLM的tokenizer跟model的类型再跑一遍会报一个维度错误的error，但是当我进行了以上两步之后再改回去AutoModel跟AutoTokenizer，神奇的是居然不报错了，顺利的ddp跑起来了。。
这个流程我试了两遍，简直玄学。。。

chatglm好像是不支持层并行的

怎么解决？我是在本地离线训练的，也是报这个错误！！！求指导

accelerate

请问下这个怎么用啊

请问怎么实现多卡训练啊，求指导

repo: mymusise/ChatGLM-Tuning


text:
documentation for batch mode XML files

Is your feature request related to a problem? Please describe.
I'm planning to work on wrapping MZmine for the Galaxy platform. And wondering if there is documentation on the XML file used for the batch mode (e.g. a schema definition or any other documentation).
Btw. as a first step I'm working on a bioconda package
Describe the solution you'd like
Any kind of docs might be of help.

That's great.
I can invite you to our mzmine slack if you are interested. That might be easier to coordinate.
I thought about this for a while now actually so great that you are proactive.
It's not documented.
All Parameters are saved depending on their datattypes and names in mzmine.
Easiest would be to lock to a version and then have a batch file for that version and just change some parameters based on user input.
Maybe we can meet and discuss

I can invite you to our mzmine slack if you are interested.

Seems like a good idea.

Easiest would be to lock to a version and then have a batch file for that version and just change some parameters based on user input.

Maybe we can identify in a first iteration the most important parameters to keep it simple and add more advanced ones in the next iteration.

Easiest would be to lock to a version and then have a batch file for that version and just change some parameters based on user input.

Since I got 3.2.8 running in bioconda I would go for this version for now. Would be happy about some input test data that I could try.

related to #1310

authors: bernt-matthias robinschmid


text:
Support for different relay selection strategies

It would be nice to be able to customize the relay selection strategy. For example, to randomly select from the best n by latency rather than just the single one with lowest latency.
This would be useful when running many smaller-capacity relays to ensure clients are distributed (somewhat) evenly across them rather than overloading a single one

Another idea that has been floated internally was to have relays be aware of their current load, and if exceeding a threshold, redirect clients to another node.
That said, this may require some coordination between relays.


text:
Fix issue # 32: Restart on disconnect

Issue #32

I can also add an option to easily toggle this, if you like.

I need to get an api on blockchain how do i do it


text:
TypeError while loading package in sublime3

Installed SublimeHighlight from https://github.com/n1k0/SublimeHighlight/tree/python3 and get following error when package is loading. https://github.com/n1k0/SublimeHighlight does not work either, ends with error Package desktop missing.
It is installed on Sublime3, running on Linux.
reloading plugin SublimeHighlight.SublimeHighlight
Traceback (most recent call last):
  File "/opt/sublime_text_3/sublime_plugin.py", line 73, in reload_plugin
    m = importlib.import_module(modulename)
  File "./importlib/__init__.py", line 90, in import_module
  File "<frozen importlib._bootstrap>", line 1584, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1565, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1532, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 584, in _check_name_wrapper
  File "<frozen importlib._bootstrap>", line 1022, in load_module
  File "<frozen importlib._bootstrap>", line 1003, in load_module
  File "<frozen importlib._bootstrap>", line 560, in module_for_loader_wrapper
  File "<frozen importlib._bootstrap>", line 868, in _load_module
  File "<frozen importlib._bootstrap>", line 313, in _call_with_frames_removed
  File "/home/michael/.config/sublime-text-3/Packages/SublimeHighlight/SublimeHighlight.py", line 17, in <module>
    if desktop.get_desktop() == 'Windows':
  File "/home/michael/.config/sublime-text-3/Packages/SublimeHighlight/HighlightLib/desktop/__init__.py", line 167, in get_desktop
    elif _is_xfce():
  File "/home/michael/.config/sublime-text-3/Packages/SublimeHighlight/HighlightLib/desktop/__init__.py", line 138, in _is_xfce
    return _readfrom(_get_x11_vars() + "xprop -root _DT_SAVE_MODE", shell=1).strip().endswith(' = "xfce4"')
TypeError: endswith first arg must be bytes or a tuple of bytes, not str

Same error here. I've submited a pull request with the fix.
You can see the fix here: https://github.com/n1k0/SublimeHighlight/pull/57/files

Thank you, this fixed my problem.
I just realised, also with pull request https://github.com/n1k0/SublimeHighlight/pull/53/files the issue will no longer occur.

authors: RubenDjOn kressi


text:
do not display empty error message

#1252

@jefleponot as this part of casperjs is actualy disabled, I don't see the value of this contribution (for now).
What do you think ? Could be vey interesting if we can re enable this part, and then enjoy your enhancement.
Mickaël

authors: jefleponot mickaelandrieu


text:
Routes in module folder doesn't work

Hey,
Sorry if this is simple and stupid, but when my routes are in Modules/MyModule/Http/routes.php, I can see them with php artisan routes:list, but it shows me a 404 page. If I move theses routes in the app/Http/routes.php, it's working perfectly.
I followed the documentation, anything i'm not doing right ? I'm on laravel 5.2 and I use laravel built-in serve.
Thanks for your work !

What do you have in your routes.php file from module's folder?

Default generated code more or less :
`Route::group(['middleware' => 'web', 'prefix' => 'backoffice', 'namespace' => 'Modules\BackOffice\Http\Controllers'], function()
{
Route::get('/test', ['as' => 'back.dashboard', 'uses' => 'BackOfficeController@index']);
});`

What url are you trying to  access, are you prepending the url with the prefix?
In your case the url to the back.dashboard route would be
yourhost.dev/backoffice/test

Try to define more obvious route in your module's route file, after clearing previous ones :
Route::get('hello', function() {
   return "John Doe";
});

and go to this url:  yourhost.dev/hello and check if this routing works via the module.

Hi Nicolas,
Did you have some issue when declaring resource routes in modules?
If i declare get, post, any, etc. routes its works.
But when I declare resources routes for eg.
Route::group(['middleware' => ['web','role:admin'], 'as' => 'users.' ,'prefix' => 'users', 'namespace' => 'Modules\Auth\Http\Controllers'], function()
{
Route::resource('/', 'UsersController');
});
It doesnt work. But the interesting is that in the route list the routes the id is not declared.
|        | DELETE                         | users/{}                       | users.destroy      | Modules\Auth\Http\Controllers\UsersController@destroy                      | web,role:admin      |
|        | PUT|PATCH                      | users/{}                       | users.update       | Modules\Auth\Http\Controllers\UsersController@update                       | web,role:admin      |
|        | GET|HEAD                       | users/{}                       | users.show         | Modules\Auth\Http\Controllers\UsersController@show                         | web,role:admin      |
|        | GET|HEAD                       | users/{}/edit                  | users.edit         | Modules\Auth\Http\Controllers\UsersController@edit                         | web,role:admin
Regards

I'm having the same issue with the latest version, that is, v7.1

I tried uninstallin and reinstalling but still no luck. I also created a new midule but all in vein,

What is the specific error?

I didn't get any error.
Sorry I had to switch to another way(another package actually) to load my packages because of a deadline.

Any solution to this issue? I'm having the same issue.

same issue here, @vishal2pc did you find any solution to this ?

What laravel-modules version is? Did you check if routes are in Modules/YourModules/Routes/web.php ?

@enextiarg nWidart/laravel-modules ~8.2, routes in ModuleNameA/Routes/web.php do not get loaded

Could you check if ModuleNameA/Providers/RouteServiceProvider.php is pointing to the correct route file?
Please try to remove bootstra/cache/* files some time the cache desnt update the routes.

Another question. Is the module active? you can check withphp artisan module:list

it's active @enextiarg

authors: Fagl4 JunaidQadirB eness enextiarg gillesgw ossycodes vishal2pc

repo: nWidart/laravel-modules


text:
Use Resources view for notifications?

how use custom views fro mail notify with modules, markdown/view  is not working with Module folder Resources.

I would assume same as normal laravel.

Inside the build() function of a mailable, trying to reference a view that's inside of the module, even with the module's namespace, returns the following:

View [viewname.blade.php] not found.

Code:
public function build()
    {
        return $this->subject($this->data['subject'])
            ->view('module::viewname');
    }

So is not possible to load the markup from the module ?
public function build(): static {         return $this->subject('test')             ->markdown('Modules/Blog/Resources/views/emails/newPost');  }
.... it throws an error: View not found.
I am using laravel/framework: 10 and nwidart/larevel-modules: 10

authors: ArtyomBist NickBourque bci24 nWidart


text:
Revert "Add functionality to my to do application to make it interactive (REOPENED FOR Javascript Best practices tasks)"

In this milestone pull request number three was reopend for JS best practices task.

Hi @naanmohammed ,
This is Jawad, Your code reviewer for this exercise.

Well Done and Congratulations 👍 😄

I have reviewed your code base of the Javascript best practices and DRY, KISS, and YAGNI rules and your code looks well-organized and readable.
Highlights:

DRY: There is no unnecessary duplication of code.
KISS : Your code is relatively simple and easy to understand.
YAGNI : Your code does what it is supposed to do without any unnecessary features.

authors: Jawad-Nazari naanmohammed


text:
用docker-compose运行mysql5.7的启动方式后，默认登录账号密码是多少？

开启 nacos.core.auth.enabled=true
运行
docker-compose -f example/nacos/standalone-mysql-5.7.yaml up -d
之后，管理面板登录 尝试了账号密码 nacos/nacos 报用户名或者密码错误

在mysql 的dockerfile里面修改sql连接为https://raw.githubusercontent.com/alibaba/nacos/2.3.2/distribution/conf/mysql-schema.sql

repo: nacos-group/nacos-docker


text:
New examples page proposal

Hello! Great react-datasheet. Really cool =)
Recently we release first public version of this project.
https://github.com/OpusCapitaBES/js-react-showroom-client
It allows to write documentation in markdown and run all docs in sandbox with code examples.
It often useful to have possibility to play with components at web page before use it in project.
Do you interested in adding interactivity to your examples page?)
I can make it and send you pull-request. Of course it can use your brand styles.

Feel free to make a gh-page I can link to within the doc, thanks

authors: kvolkovich-sc nadbm

repo: nadbm/react-datasheet


text:
withCustomActions decorator overides  not allow to use custom methods

Hi,
I tried to switch from https://github.com/Exelord/ember-custom-actions to this add-on & had some trouble with the decorator.
The decorators always redefine custom actions methods & does not allow to customize these methods on the class using the decorators...
So I end up by using the decorator in the application adapter to allow child adapter to overrides custom actions methods...
When reading, the doc, it does not seems to be the behavior wanted 🤔
Maybe the decorator should replace the class methods only when they are undefined to prevent this ?
Just for information the code I use :
import ApplicationAdapter from './application'
import { withCustomActions } from '@naeka/ember-custom-actions'

@withCustomActions
export default class RuleBaseAdapter extends ApplicationAdapter {
  urlForCustomAction(modelName, id, snapshot, actionId) {
    // handling some custom url here
  }
}

import Model from '@ember-data/model'
import { customAction } from '@naeka/ember-custom-actions'

export default class RuleBaseModel extends Model {
  ...
  updateCategory = customAction('updateCategory')
}

& when calling model.updateCategory() it never call the adapter.urlForCustomAction but call the default method created in the decorator (here)

@romgere Thanks for reporting ! The bug has been fixed in release 0.1.1

authors: romgere tomdrn

repo: naeka/ember-custom-actions


text:
Add ko.yml for waifu2x GUI

hello!
I did Korean translation for waifu2x GUI.
There is a reference image below.

We plan to add windows_package/docs/README_en.md soon.

same reason as before.

authors: DevNergis nagadomi

repo: nagadomi/nunif


text:
cublas initialization error

Hi, first thanks for this great software :)
I am trying to run the examples, but I am getting runtime errors, namely:
caffe_neural_models/dataset_01 (master ✘)✹ ᐅ bash train.sh
E0716 15:20:28.804553 11835 common.cpp:123] Cannot create Cublas handle. Cublas won't be available.
E0716 15:20:28.805637 11835 common.cpp:130] Cannot create Curand generator. Curand won't be available.
F0716 15:20:28.908949 11835 common.cpp:384] Check failed: status == CUBLAS_STATUS_SUCCESS (1 vs. 0)  CUBLAS_STATUS_NOT_INITIALIZED
*** Check failure stack trace: ***
    @     0x7f6e37237a1d  google::LogMessage::Fail()
    @     0x7f6e372398bd  google::LogMessage::SendToLog()
    @     0x7f6e3723760c  google::LogMessage::Flush()
    @     0x7f6e3723a1de  google::LogMessageFatal::~LogMessageFatal()
    @     0x7f6e37bf7f88  caffe::Caffe::SetDevice()
    @           0x45bb7b  main
    @     0x7f6e358e976d  (unknown)
    @           0x411f99  (unknown)
train.sh: line 1: 11835 Aborted                 (core dumped) ./../../caffe_neural_tool/build/caffe_neural_tool --gpu 3 --train 4 --proto 'train_process_usk_2.prototxt'

I suppose there is an issue with cublas, but I am not sure how to fix this.
I am linking against ViennaCL, but I am not sure where cublas comes from.
do you have any suggestions about what to try next? thanks!

First of all, how did you compile Caffe itself? It seems as if you have enabled CUDA compilation there. Remove it if no nVidia GPU is present, this might be one of the issues.
Second, make sure what device IDs are available by running:
./caffe_neural_tool --devices
and then edit the train.sh to use the correct GPU. At the moment it is GPU no. 3, which might be wrong.
Also make sure you use either an nVidia GPU, AMD GPU or non-NUMA processor, as NUMA processors will have cache invalidation issues with the multicore-backend of Caffe. Device fission might fix this in the future but currently, single-CPU systems are way faster than any NUMA (dual CPU) systems.

Thanks. I was looking at the Makefile.config files again, and I realized that I was linking against the wrong cuda version for caffe_neural_tool.
Now, to try it out, which is the easiest example to run, to try training and testing?
When I try to run ./tran.sh I get some errors, but I guess there is somewhere a simple example that should work.
One last question: in the ground truth images, is it possible to ignore a particular label. This is to deal with partially labeled data (for example, most pixels in a training image may not be labeled).

Yes this is possible. The value for pixel labels to ignore can be set into the network prototxt, in the softmax loss param as "ignore_label".
In this case though, don't use the masking parameter provided by my tool, as that will set pixel labels to -1 in order to ignore them.
The easiest example is to run the existing train/process for the SK network. Always make sure the relative paths fit and all folders necessary exist.
Just send me the error logs if you can't figure out something on your own.
Many more details though will only be available after the 24th of August after I'm done writing my thesis.

Thanks. For example, I get the following error:
caffe_neural_models/dataset_01 (master ✘)✹ ᐅ bash train.sh
I0717 15:07:19.546991 10671 caffe_neural_tool.cpp:113] Training mode.
F0717 15:07:19.547129 10671 train.cpp:17] Train parameter index does not exist.
*** Check failure stack trace: ***
    @     0x7fa32102da1d  google::LogMessage::Fail()
    @     0x7fa32102f8bd  google::LogMessage::SendToLog()
    @     0x7fa32102d60c  google::LogMessage::Flush()
    @     0x7fa3210301de  google::LogMessageFatal::~LogMessageFatal()
    @           0x4a3ed3  caffe_neural::Train()
    @           0x46087d  main
    @     0x7fa31f6df76d  (unknown)
    @           0x412e49  (unknown)
train.sh: line 1: 10671 Aborted                 (core dumped) ./../../caffe_neural_tool/build/caffe_neural_tool --gpu 3 --train 4 --proto 'train_process_usk_2.prototxt'

I am not sure where to find the SK network train.sh, so I tried with the first example I found.

Use in dataset_01:
--gpu 0 --train 0 --proto 'train_process_sk_9.prototxt'
Then use the snapshot after for example 10'000 steps:
--gpu 0 --process 0 --proto 'train_process_sk_9.prototxt'
and make sure there is something to process in a folder called 'input' as I only provide training/ground truth in the repository and no test data.

that's great, thanks, I will give it a try.

It seems to be working.
I am trying to adapt it to another example, where the ignore label is 1, so I have (among other things):
train {
  # solverstate: "neuraltissue_sk_2out_iter_16000.solverstate"
  solver: "../net_sk_2out/neuraltissue_solver.prototxt"
  input {
    padding_size: 102
    patch_size: 64
    channels: 3
    labels: 2
    batch_size: 1
    raw_images: "train/raw"
    label_images: "train/gt"
    preprocessor {
      label_consolidate {
        # 0 -
        label: 0
        # 1 /
        label: -1
        # 2 |
        label: 1
      }
      normalization: true
      rotation: true
      mirror: true
      clahe {
        clip: 4.0
      }
      crop {
        imagecrop: 1
        labelcrop: 0
      }
    }
  }
}

The problem is that I get a nan value at train time:
I0717 17:06:38.393384 30689 solver.cpp:43] Solver scaffolding done.
Label: 0, 4096
Label: 1, 0
Label: 2, 0
Label: 3, 0
I0717 17:06:44.165413 30689 solver.cpp:224] Iteration 0, loss = nan
I0717 17:06:44.165509 30689 solver.cpp:501] Iteration 0, lr = 0.001
Label: 0, 8192
Label: 1, 0
Label: 2, 0
Label: 3, 0
Label: 0, 12288
Label: 1, 0
Label: 2, 0
Label: 3, 0
Label: 0, 16384
Label: 1, 0
Label: 2, 0
Label: 3, 0
Label: 0, 20480

is there a way to enable additional debugging output information? The output above is with --debug already.

I managed to make it work with the examples.
I am trying with my data now, which has labels 0,1 and 255.
And I should ignore 255, but use 0 and 1 for a binary problem.
If I exclude the label from the label_consolidate configuration, will it still try to use it?
I tried with the ignore_label, which is -1, but in that case I was getting nan as loss.

OK I actually never thought of trying it that way - let me figure out where the bug stems from. Maybe I can fix it quickly or give you the correct settings.

thanks, that's great.
Also, if you have a paper I can cite for your work, or it could be the thesis afterwards, let me know.

There will maybe be a paper, but that's going to take a while. Until then, you can cite by name and github URL. And after the 24th of August I'll also post a link to the thesis somewhere.

Are your labels integer values in the image?
The tool will typically assign 0 to the lowest integer value label, 1 to the next one and so on. In grayscale images! Something seems to be going wrong then.
Best if you exclude label consolidate if you don't use it - just remove the whole block from the processing. And change the ignore_label to the actual label you want to ignore in the neuraltissue_net.prototxt here:
layer {
  include: {phase: TRAIN}
  name: "loss"
  type: "SoftmaxWithLoss"
  bottom: "ip3"
  bottom: "label"
  loss_param {
    ignore_label: -1
    normalize: true
  }
}

Also remember to set the correct number of output labels (last layer output number) in both prototxt files.

oh ok I see, I think I am understanding now.
Would 255 in the image be taken as label 255, or label 2?

@cbecker
You're right, I fixed it now.

How can I check how many labels it is detecting, and how many samples per label?
I am training now and my dataset is very skewed, so the loss goes way up, then down:
I0717 18:10:23.551026  7473 solver.cpp:224] Iteration 2200, loss = 25.6082
I0717 18:10:23.551105  7473 solver.cpp:501] Iteration 2200, lr = 0.00086145
I0717 18:10:37.191097  7473 solver.cpp:224] Iteration 2250, loss = 27.6978
I0717 18:10:37.191189  7473 solver.cpp:501] Iteration 2250, lr = 0.000858812
I0717 18:10:50.838942  7473 solver.cpp:224] Iteration 2300, loss = 43.4337
I0717 18:10:50.839028  7473 solver.cpp:501] Iteration 2300, lr = 0.000856192
I0717 18:11:04.493456  7473 solver.cpp:224] Iteration 2350, loss = 32.9644
I0717 18:11:04.493532  7473 solver.cpp:501] Iteration 2350, lr = 0.000853591
I0717 18:11:18.124866  7473 solver.cpp:224] Iteration 2400, loss = 49.9584
I0717 18:11:18.124938  7473 solver.cpp:501] Iteration 2400, lr = 0.000851008
I0717 18:11:31.758568  7473 solver.cpp:224] Iteration 2450, loss = 33.9239
I0717 18:11:31.758651  7473 solver.cpp:501] Iteration 2450, lr = 0.000848444
I0717 18:11:45.394291  7473 solver.cpp:224] Iteration 2500, loss = 13.3905
I0717 18:11:45.394366  7473 solver.cpp:501] Iteration 2500, lr = 0.000845897
I0717 18:11:58.996084  7473 solver.cpp:224] Iteration 2550, loss = 52.0906
I0717 18:11:58.996172  7473 solver.cpp:501] Iteration 2550, lr = 0.000843368
I0717 18:12:12.502898  7473 solver.cpp:224] Iteration 2600, loss = 50.3209
I0717 18:12:12.502974  7473 solver.cpp:501] Iteration 2600, lr = 0.000840857
I0717 18:12:25.900024  7473 solver.cpp:224] Iteration 2650, loss = 21.3011
I0717 18:12:25.900107  7473 solver.cpp:501] Iteration 2650, lr = 0.000838363
I0717 18:12:39.306751  7473 solver.cpp:224] Iteration 2700, loss = 4.43506
I0717 18:12:39.306835  7473 solver.cpp:501] Iteration 2700, lr = 0.000835886
I0717 18:12:52.684272  7473 solver.cpp:224] Iteration 2750, loss = 85.4389
I0717 18:12:52.684355  7473 solver.cpp:501] Iteration 2750, lr = 0.000833427
I0717 18:13:06.066045  7473 solver.cpp:224] Iteration 2800, loss = 11.0876
I0717 18:13:06.066126  7473 solver.cpp:501] Iteration 2800, lr = 0.000830984
I0717 18:13:19.453717  7473 solver.cpp:224] Iteration 2850, loss = 28.2095
I0717 18:13:19.453801  7473 solver.cpp:501] Iteration 2850, lr = 0.000828558
I0717 18:13:32.837424  7473 solver.cpp:224] Iteration 2900, loss = 5.7144
I0717 18:13:32.837505  7473 solver.cpp:501] Iteration 2900, lr = 0.000826148

I0717 18:13:46.211272  7473 solver.cpp:224] Iteration 2950, loss = 4.69093
I0717 18:13:46.211349  7473 solver.cpp:501] Iteration 2950, lr = 0.000823754

Enable debugging for the caffe_neural_tool if you want to see the label counts.
add --debug and --graphic if you want to see a bit of what's going on. Maybe you should also decrease the learning rate (0.0001) and increase momentum (0.99) in the neuraltissue_solver.prototxt for this data set.
Feel free to fork and add new stuff to this tool if it helps for you. I will review and accept pull requests.

Thanks ;)
I'm confused, because I see as if label 1 and 2 are there, but 0 is not:
Label: 0, 0
Label: 1, 6534
Label: 2, 574

and my labels TIF file has values 0,1 and 255.

If you enable the patch prior and masking function, what initial statistics does the tool output when starting the training?

Hi, Thanks.
Actually, if I set the number of classes to 3 the training loss never goes down.
So I am using 2. But then, if I disable CLAHE normalization, it doesn't converge either.
do you have any idea of what could be happening?
The config files are here: http://pastebin.com/L0NphqbP
thanks

Yes you should disable masking on your dataset. Remove the histogram equalization block. Histeq turned out to be useful on the ISBI 2012 dataset and also on FlyEM data with 9 labels.
If the data is already only partially masked or labelled for some reason, the histogram equalization will interfere heavily.
I apologize for the difficulty to set it up currently. The parameters need to be vastly different on different kinds of datasets.

Thanks. When disabling histeq, I get nan after 50 iterations. Do you think this is related to patch normalization and learning rate?
I0718 23:03:19.342078 10622 solver.cpp:224] Iteration 0, loss = 0.693077
I0718 23:03:19.342238 10622 solver.cpp:501] Iteration 0, lr = 0.001
I0718 23:03:29.735518 10622 solver.cpp:224] Iteration 50, loss = nan
I0718 23:03:29.735569 10622 solver.cpp:501] Iteration 50, lr = 0.000996266
I0718 23:03:40.074090 10622 solver.cpp:224] Iteration 100, loss = nan
I0718 23:03:40.074154 10622 solver.cpp:501] Iteration 100, lr = 0.000992565
I0718 23:03:50.432718 10622 solver.cpp:224] Iteration 150, loss = nan
I0718 23:03:50.432777 10622 solver.cpp:501] Iteration 150, lr = 0.000988896
I0718 23:04:01.384943 10622 solver.cpp:224] Iteration 200, loss = nan
I0718 23:04:01.385005 10622 solver.cpp:501] Iteration 200, lr = 0.000985258

Are there patches in your dataset that lack labels 0 and 1 completely and only expose label 2, or any other such odd combination? If that is the case this might well be a problem because the Caffe library might divide by zero at some point.
In that case, keeping patch_prior to true and masking to false should not give NaN, is that correct?

exactly, some parts have only label 2 (ignore).
yes, if I put patch_prior to true and masking to false, it seems to work.
will it do histogram eq. if masking = false? I want to make sure I disable histogram eq. as I think it is hurting in my case.

exactly, some parts have only label 2 (ignore).
yes, if I put patch_prior to true and masking to false, it seems to work.
will it do histogram eq. if masking = false? I want to make sure I disable histogram eq. as I think it is hurting in my case.

It will prioritize patches with rare labels if patch prior is enabled. This will equalize the histogram of labels slightly but not completely. It could be that this even sets the priority of patches with label 2 only to zero. But beware, I did not exactly test this behavior.
If you want to be very sure, then the best thing would be to either fix the loss function in Caffe to avoid division by zero and then disable the patch prior again, or otherwise fix the neural tool to not expose Caffe to patches with invalid labels.
I could fix the loss function if you want.

Actually fixed it just now in my Caffe branch if you want to try again.

I see, I am starting to understand better now.
ok great. So I should just disable the histeq block?

Yes remove it and try again with the updated code.

actually now it still throws nan, now it does it at the beginning already.
let me know if I can help with any other debugging info.
I can also send you an example image and files, if that helps.

I tried something else now.
I don't know if that would help as I pretty much know where the error comes from. It really is the division by zero or generally zero valid labels present, which is not handled by Caffe currently.

Thanks, though it's still not working.
In case it helps, here you have a working example for training: http://cvlabwww.epfl.ch/~cjbecker/tmp/test.tar.gz

I used it like that now, with the newest version of Caffe & the tool, and not getting NaN anymore:
# The training protocol buffer definition
train_net: "neuraltissue_net.prototxt"
########################################################################
# The testing protocol buffer definition
# test_net: "../net_sk_2out/neuraltissue_net.prototxt"
########################################################################
# Test_iter specifies how many forward passes the test should carry out.
# it is the number of batches shown, then
# examples shown = 'test_iter'*batch_size
# Carry out testing every 'test_interval' training iterations.
# test_iter: 1000
# test_interval: 500
########################################################################
# The base learning rate, momentum and the weight decay of the network.
# base_lr: 0.05
base_lr: 0.001
momentum: 0.9
weight_decay: 0.0005
########################################################################
# The learning rate policy
lr_policy: "inv"
gamma: 0.0001
power: 0.75
#lr_policy: "step"
#gamma: 0.1
#stepsize: 20000
########################################################################
# The maximum number of iterations
max_iter: 100000
########################################################################
# Snapshot intermediate results
snapshot: 2000
snapshot_prefix: "neuraltissue_sk_2out"
########################################################################
# Display every 'display' iterations
display: 5
########################################################################

train {
  # solverstate: "neuraltissue_sk_2out_iter_16000.solverstate"
  solver: "neuraltissue_solver.prototxt"
  input {
    padding_size: 102
    patch_size: 64
    channels: 3
    labels: 3
    batch_size: 1
    raw_images: "train/raw2"
    label_images: "train/gt2"
    preprocessor {
      normalization: true
      rotation: true
      mirror: true
      clahe {
      	clip: 4.0
      }
      crop {
        imagecrop: 1
        labelcrop: 0
      }
      histeq {
        patch_prior: true
        masking: false
      }
    }
  }
}

process {
  process_net: "neuraltissue_net.prototxt"
  # caffemodel: "neuraltissue_sk_2out_iter_100000.caffemodel"
  input {
    padding_size: 102
    patch_size: 128
    channels: 3
    labels: 2
    batch_size: 1
    raw_images: "input"
    preprocessor {
      normalization: true
      clahe {
        clip: 4.0
      }
      crop {
        imagecrop: 1
        labelcrop: 0
      }
    }
  }
  filter_output {
    output_filters: false
    output: "sk_filters"
  }
  output {
    format: "tif"
    fp32_out: false
    output: "output"
  }
}

and:
layer {
  include: {phase: TRAIN}
  name: "loss"
  type: "SoftmaxWithLoss"
  bottom: "ip3"
  bottom: "label"
  loss_param {
    ignore_label: 2
    normalize: true
  }
}

Thanks, that's great!
yes, I see about loss 0, I will give it a try and let you know, thanks again!

one more question: it is strange to me that the histogram equalization block is only in the training part. Does this mean to histogram equalize the input patch? because then I would expect to see it in the predict part as well.

It only prioritizes picking of training patches during training. Histogram equalization and masking has only an effect on what kind of errors the network will be seeing how often during SGD.
In prediction mode, it will just go through the patches linearly and give a prediction for each pixel.

I think it is working now, thanks.
What I notice is that the output probabilities are very 'thick' (in my case I have synapses).
Typically I would expect to see a transition zone from the border of the synapse, to the outside, with decaying probability. Have you experienced this issue as well? also, is it possible to output, instead of the probability output of the last sigmoid, the input to that sigmoid? In the case of a two class problem one typically plots the classifier score, before reaching the sigmoid, as the sigmoid may squeeze things too much (especially for unbalanced training data like this one).

Yes, that might happen.
You can just remove the Softmax/Sigmoid in the end which is labelled with a phase: TEST include clause at the end of the network. In this case, it should pick up the pre-softmax output in the tool.
Remove this block in the network prototxt:
layer {
  include: {phase: TEST}
  name: "prob"
  type: "Softmax"
  bottom: "ip3"
  top: "prob"
}

Great, that works.
I talked to Jan and he showed me your results for the dataset at https://github.com/unidesigner/groundtruth-drosophila-vnc
Do you have the parameters you used to train that network? and how many iterations approximately?

The best results I have on ISBI 2012 now are with:
Network: SK
Training: Softmax + Malis (loss function, 10'000 iterations each)
The previously best results were on SK with Softmax loss only, approximately 30'000 iterations is what we used.
I think it's about the same for the other dataset. But Malis can only be used properly if label 0 is background that separates objects.

Thanks. I did some tests, and performance is still a bit low.
I was trying to see the filters learned, to see if something weird is happening, but I am getting an error while loading the model and parameters in python:
IndexError                                Traceback (most recent call last)
<ipython-input-4-5f4574a759ce> in <module>()
----> 1 net = caffe.Classifier("net_sk_2out/neuraltissue_net.prototxt", "hipp/neuraltissue_sk_2out_iter_200000.caffemodel" )

/home/cjbecker/filer/jan-caffe/caffe/python/caffe/classifier.pyc in __init__(self, model_file, pretrained_file, image_dims, mean, input_scale, raw_scale, channel_swap)
     27
     28         # configure pre-processing
---> 29         in_ = self.inputs[0]
     30         self.transformer = caffe.io.Transformer(
     31             {in_: self.blobs[in_].data.shape})

IndexError: list index out of range

I suppose this is an issue with patch size and input layers, but I am not sure how to solve it.

Do you want to see the individual filtered stages after each layer or the filter kernels themselves?
The first point can be done in the tool itself. The second is not implemented (yet).
In any case, you cannot do forward/backward processing with the python interface, the memory data interface used in the networks only work together with the C++ interface currently.
You'd have to use python data layers to do so, and change the networks.

I meant to show the filter kernels, as they would look like noise if there is an issue during learning.
If there was a way to extract the coefficients from the model file, then that would be enough.
I can also try to look at the output of the first layer, that could give me a hint. This is done by removing or commenting out the other layers, right?

No, you can output everything during processing by using:
  filter_output {
    output_filters: true
    output: "sk_filters"
  }

inside the process {} block of train_process_sk_2.prototxt

ah great, thanks ;)
another observation from looking at the test output: I think normalization is not working very well, because, depending on the slice, there is a shift in the output of the network, and it is causing the output to vary too much between consecutive slices.
If normalization is disabled, then the code here https://github.com/naibaf7/caffe_neural_tool/blob/31c5a2c635062e9e0887719f20a77f674ee9c709/src/image_processor.cpp#L70 doesn't make the pixels go between -1 and +1, but instead between 0 and 1, right?  I am going to play with this because the variations in the output are significant with the current min/max scheme.

Yes, you should definitely check what min/max normalization and CLAHE does to your data, and whether it helps or hurts the classification.

Currently, the ISBI dataset with CLAHE and Min/Max normalization scores 12th:
http://brainiac2.mit.edu/isbi_challenge/leaders-board
(the INI entry)

Great, thanks.
I tried on my dataset (synapses) and I get ok results, though much worse than a random forest trained on some custom simple features. When testing on the training images, I see that there are negatives that are misclassified as positives, probably because they have little chance to appear during SGD. Have you ever tried hard negative mining? or is there a way to favor sampling some samples (or regions) more than others?

You'd probably have to change to minibatch training instead of patch training. As it is currently, to save computations and speed up, it always trains SK with 64 by 64 pixels that are not sampled i.i.d. but correlate very strongly (they are in the same local patch).
Depending on the dataset, this gives worse results, as you noticed, and in this case you need to prepare a HDF5 dataset with the samples as you wish them to be picked (i.i.d. from a distribution that you think will result in better training). Then you need to swap out the MemoryDataLayer for a HDF5 data layer and change the input size to 102x102, label size to 1x1 and batch size to 256 and train it without my tool (which is for patch training rathern than minibatch training).
Afterwards, processing can happen again with a patch label size of 128x128 and the speedup is given again. The results will be numerically identical to batch processing.

I see. but it would also be possible to modify the histeq module and the patch_prior, to weight certain patches more than others. Then I could have an image that has a weight per pixel, that 'guides' learning, I think that could be a first approximation to achieve this.

Yes there are actually many things I would have liked to include in the tool, but currently do not have time to program:

Support for i.i.d. minibatch training
Additional pixel error weighting/scaling metrics (via external weight map or internally precomputed)
Additional masking via external weight map
Support for 3D networks

Hi there. I am done making a few experiments, thanks for all the support.
I managed to implement loading the external 'weight map' for sampling, which helped a bit in some cases.
In terms of performance, the CNNs do much worse than our approach, even when trained on the largest training set we have. I don't think this is an issue of your implementation, as we saw this when using Caffe as well. To get better performance I think we need to add some prior to the network.
Just to be sure, the sk_2 network we talked about, included in the examples, has an 'equivalent patch size' or 'context' of 100x100 pixels, right? I mean, it would be equivalent to running a per-patch trained CNN whose input layer is of 100x100.

okay, interesting...
the context is 102 by 102

Thanks. One more thing: despite CNNs not doing so well, it is amazing the speed improvement, especially for prediction on whole images. Thanks again :)

Yes and that was only SK-Net (factor 50 speedup). USK and U-Net have speedups of up to 400.

Wow, sounds good. I've read a paper about U-net, but do you have some references so I can read about USK and SK-net? or is that the contribution that is part of your thesis?

SK-net was designed by Julien Martel at the UZH INI, and there exists no publication about it. USK-Net is my own contribution. Both will be described in detail in my thesis, which will be available for interested readers after the 24th of August.

I would certainly like to get a copy ;)
I've been trying the u-net prototxt files you provided, and adapted them to my problem.
The performance is significantly better, now it does close to one of my baselines, which is much more reasonable. I suppose this is because of the location-preserving characteristics of u-net.

According to my own tests on data I got from Jan Funke, USK-net performed best by quite some margin. Maybe also try that if you have enough GPU RAM (8 GiB headless or more).
It is definitely interesting that U-Net works better for you, it actually did worst on the training data from Jan. Seems to depend a lot on training and data, which makes sense.
The U-Net would have an "SK equivalent" context of about 184, but it's not entirely the same. USK operates similarly on a 180 by 180 context.

I see. I will give USK net a try as well, I will let you know of my findings.
Thanks!

I am looking at the USK network, and the loss layer is the malis loss. I added to ignore label 2, but I am wondering if that is causing some problems, for example:
0805 09:25:59.352416 29865 solver.cpp:224] Iteration 0, loss = 0.789748
I0805 09:25:59.352504 29865 solver.cpp:501] Iteration 0, lr = 0.0001
I0805 09:27:04.040350 29865 solver.cpp:224] Iteration 50, loss = 0.796925
I0805 09:27:04.040423 29865 solver.cpp:501] Iteration 50, lr = 9.96266e-05
I0805 09:28:04.154019 29865 solver.cpp:224] Iteration 100, loss = 0
I0805 09:28:04.154093 29865 solver.cpp:501] Iteration 100, lr = 9.92565e-05
I0805 09:29:03.739804 29865 solver.cpp:224] Iteration 150, loss = 0
I0805 09:29:03.739882 29865 solver.cpp:501] Iteration 150, lr = 9.88896e-05
I0805 09:30:03.227816 29865 solver.cpp:224] Iteration 200, loss = 0
I0805 09:30:03.227887 29865 solver.cpp:501] Iteration 200, lr = 9.85258e-05
I0805 09:31:02.691009 29865 solver.cpp:224] Iteration 250, loss = 0
I0805 09:31:02.691082 29865 solver.cpp:501] Iteration 250, lr = 9.81651e-05
I0805 09:32:01.873886 29865 solver.cpp:224] Iteration 300, loss = 0
I0805 09:32:01.873956 29865 solver.cpp:501] Iteration 300, lr = 9.78075e-05
I0805 09:33:01.481889 29865 solver.cpp:224] Iteration 350, loss = 0
I0805 09:33:01.481959 29865 solver.cpp:501] Iteration 350, lr = 9.74529e-05
I0805 09:34:00.891755 29865 solver.cpp:224] Iteration 400, loss = 0

The loss is 0 most of the time.

Yes, please use the USK network with Softmax.
The Malis loss is only good for foreground/background object separation and cannot use the ignore label parameter.

Great. From what I see so far, USK is doing significantly worse, but I wonder if this has to do with the size of the input training patches, as in my case the positives are very few, so that training with smaller patches yields more positives sampled than with larger patches (beyond the extreme of having too small patches).

Hard to tell... could be. It certainly did best with membrane classification both when using malis and softmax on the dataset 01 that is included in the examples.
I guess your kind of labeling needs something else then... you should have a look at the filters of the U-Net to see what it actually learns that makes it better than USK and SK for this kind of data set.

yes, I think it is quite particular. I think my problem as well is that I typically don't have so much training data, as it is expensive to collect. If you even want to try on this dataset I can pass you the data and evaluation scripts, just let me know. We will make it publicly available soon, as well.


text:
一点UI的小建议

感谢作者您的付出，确实很好用，但是刚开始用的时候对于UI确实有点不太适应，虽然我知道UI做到这样已经很不易了，但还是想提一下，供参考（应该都是比较容易实现的）:

感谢反馈，我会在下个版本中修改UI

authors: naivenaive nanlandetian

repo: naivenaive/EasyPubMed


text:
Entering a big year crashes Browser

Since you can manually change the year with key inputs, entering a date like this: 01/01/123454321, will crash the page or the browser.

+1

this was fixed in 56129c0e05b4f8cfe7c8317177654959bbd4e793


text:
Enable to switch viewController

forcibly switch to a viewController

:ok_woman:

repo: nakajijapan/NKJPagerViewController


text:
Make mail related functions work without AWS in offline dev

Currently reset password and email confirmation need AWS SES to work. This is not necessarily nice in offline dev, because it requires some unnecessary setup.
Not sure yet what the solution would be. Maybe a setting in the dot file to auto-confirm the email addresses and regarding password reset it would immediately return the password reset id, so the password can be reset.

Subtasks are done so closing this.


text:
Movement keys broken (W & S)

i know its fixed
but i doesnt work for me

i have the same

try compile on v142

repo: namazso/nSkinz


text:
InteractJS v1.9.5+ breaks react-calendar-timeline: Cannot read property 'docEvents' of undefined

Describe the bug
InteractJS v1.9.5+ breaks react-calendar-timeline.
To Reproduce
v1.9.7 as of the time of writing.
npm i interactjs@latest
Steps to reproduce the behavior:

Create a project with react-calendar-timeline
npm i interactjs
run your application
See error

Expected behavior
Application should run as intended
Screenshots
v1.9.4

v1.9.5

Library Version
v0.26.7
Desktop:

OS: Windows 10 1909 18363.720
Browser Edge (Chromium)
Version  83.0.461.1
Browser Chrome
Version  80.0.3987.149
Browser FireFox
Version  74.0

Additional context
Maybe this issue has something to do with it:
https://github.com/taye/interact.js/issues/803

I installed interactjs@1.9.4 and that seemed to fix my build issue. Still can't get the responsive drag n drop stuff working though :/

This seems to be fixed with 1.9.10

authors: dlowkeen stahlmanDesign willemliu

repo: namespace-ee/react-calendar-timeline


text:
🛑 yoma-design.ch is down

In 3ae5470, yoma-design.ch (https://yoma-design.ch) was down:

HTTP code: 0
Response time: 0 ms

Resolved: yoma-design.ch is back up in b8648ff.

repo: nanhosting/monitoring


text:
🛑 assafi.org is down

In f1a81c4, assafi.org (https://assafi.org) was down:

HTTP code: 500
Response time: 1129 ms

Resolved: assafi.org is back up in 84f65e4.


text:
Add Keystone authentication

Add authentication for OpenStack Identity Service to the basic ceph method. For this change, the update to the last version of joss is needed.

thanks！

authors: aidaph wangp-nhlab

repo: nanhu-lab/hadoop-ceph


text:
Admin UI not loading resources

Unable to access using the tunnel command:
nanobox tunnel data.rethink 8081:8080
Probably related:
https://github.com/rethinkdb/rethinkdb/issues/6476
https://github.com/rethinkdb/rethinkdb/issues/6522

Experience the same issue as sagif, please fix this!!

Give Chateau a try, it works but unfortunately you cant query the db

I had this same issue when I was trying to use proxy_pass with the RethinkDB admin panel on Nginx.

authors: PassTheMayo ShailenNaidoo sagivf

repo: nanobox-io/nanobox-docker-rethinkdb


text:
Missing instructions for requiring

Running nanoc deploy without explicitly requiring nanoc-git leads to the following error:

The specified deploy target has an unrecognised kind “git” (expected one of fog, rsync).

The README should be updated to reflect that nanoc-git needs to be required first (somewhere in lib/).
CC @samstarling

Fixed in 248d22e9ed46059144411d4540aabc3c5dca4708.

Hurrah, thanks!

authors: ddfreyne lifepillar samstarling

repo: nanoc/nanoc-git


text:
Update project

Description

Update dependency of CoreLib
Update AppVeyor
Add GitVersion for CI

Motivation and Context
How Has This Been Tested?

Screenshots
Types of changes

[ ] Improvement (non-breaking change that improves a feature, code or algorithm)
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[x] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes.
[x] All new and existing tests passed.

Signed-off-by: José Simões jose.simoes@eclo.solutions

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

authors: CLAassistant josesimoes


text:
using a bytes field as proxy for arbitrary messages

Hello nano developers,
I'd like to realize the following proto:
message container {
    enum MessageType {
        TYPE_UNKNOWN =  0;
        evt_resultStatus  =  1;
    }
    required MessageType mt = 1;
    optional bytes cmd_evt_transfer = 2;
}

message evt_resultStatus {
    required int32 operationMode = 1;
}

...

The dots denote, there are more messages with (multiple) primitive containing datatypes to come. The enum will grow likewise, just wanted to keep it short.
The container gets generated as:
typedef struct _container {
    container_MessageType mt;
    pb_callback_t cmd_evt_transfer;
} container;

evt_resultStatus is:
typedef struct _evt_resultStatus {
    int32_t operationMode;
} evt_resultStatus;

The field cmd_evt_transfer should act as a proxy of subsequent messages like evt_resultStatus holding primitive datatypes.
evt_resultStatus shall be encoded into bytes and be placed into the cmd_evt_transfer field.
Then the container shall get encoded and the encoding result will be used for subsequent transfers.
The background why to do so, is to shorten the proto definition and avoid the oneof thing. Unfortunately syntax version 3 is not fully supported, so we can not make use of any fields.
The first question is: will this approach be possible?
What I've got so far is the encoding including the callback which seems to behave fine. But on the other side, decoding somehow skips the callback. I've read issues here, that this happened also when using oneof and bytes fields.
Can someone please clarify on how to proceed with this?
Thx

I'll close this issue and answer to your Stack Overflow post instead. In general it's a good idea to link posts together if you feel the need to double-post, as otherwise people will end up wasting time answering your question multiple times.

authors: PetteriAimonen woodz-

repo: nanopb/nanopb

license: Zlib


text:
Remove nanshe-viewer

This removes the nanshe-viewer. It is no longer in use and after some discussion we have decided it doesn't belong here any more. We will add its history to an archive repo so it remains available. However, no development is planned on for the foreseeable future. Perhaps there are changes from it that can be taken up in Ilastik.
There are some doc updates needed as a consequence. Though those doc updates are long overdue anyways.

Coverage decreased, but that is what happens when you remove code. 😄

Removing the remaining Qt dependency in PR ( https://github.com/nanshe-org/nanshe/pull/395 ).

repo: nanshe-org/nanshe