text:
imap brute force

Checklist

[x] I have followed the Contributor Guidelines.
[x] The code has been thoroughly tested in my local development environment with flake8 and pylint.
[x] The code is Python 3 compatible.
[x] The code follows the PEP8 styling guidelines with 4 spaces indentation.
[x] This Pull Request relates to only one issue or only one feature
[x] I have referenced the corresponding issue number in my commit message
[x] I have added the relevant documentation.
[x] My branch is up-to-date with the Upstream master branch.

Changes proposed in this pull request
imap brute force module
Your development environment

OS: x
OS Version: x
Python Version: x

@securestep9 , @Ali-Razmjoo any update?

@Mrinank-Bhowmick  - this module failed my test, it failed to find the correct user/password pair when bruteforcing. I had no time to investigate further, likely the exceptions are not handled correctly

repo: OWASP/Nettacker


text:
Upgrade dependency

Upgrade jsonwebtoken@8.5.1 to version

Please ignore this PR and close it. Generated by TestIM

source: gharchive/pull-request

license: MIT

license_type: permissive

license_source: github-api


text:
Upgrade dependency

Upgrade jsonwebtoken@8.5.1 to version 9.0.2

Please ignore this PR and close it. Generated by TestIM


text:
create a basic popup text for aframe

Roassal3D shapes needs to add a basic popup when they have a model.

@ThisIsAntonio the popup in AFrame should use the current model of each shape as string.

The PopUp are Ok and running.
Add Code about this, and images.
visitCanvas: aRSCanvas
	stream << '<a-camera position="0 1.6 4">';
		    << '<a-text id="idPopUP" ';
		    << ' align="center" width="2.5" color="#FFF"';
		    << 'visible="false" position="2 2 -4" geometry="primitive: plane; width: 1.75"';
		    << 'material="color: #333"></a-text> ';
		    << '<a-cursor></a-cursor></a-camera>'; crlf.
	aRSCanvas shapes do: [ :shape | shape accept: self ] separatedBy: [ stream crlf ].

Works well

source: gharchive/issue

authors: ThisIsAntonio akevalion


text:
scale_negative_tracers fails on GPU

Everything else seems to work okay on GPU now except the utilities for stopping tracers going negative. When a GPU node becomes available I'll try and get this working.

Could fix this? I did something similar and it worked in GPU, maybe it helps.

This might be not applicable to your case as I had a way simpler model, but I am posting here in case it helps. I have used this for V100 and A100 GPUs.
# zeroing negative values
@inline function zeroing(sim)
    @unroll for tracer in [:P, :N, :Nr]
        parent(sim.model.tracers[tracer]) .= max.(0, parent(sim.model.tracers[tracer]))
    end
end
simulation.callbacks[:zeroing] = Callback(zeroing)

https://github.com/iuryt/NorthAtlanticBloom/blob/4d0d20cd9549c8eb78ae8b827518ee2756ae6807/src/01-model/02-submesoscale.jl#L223-L229

I've got this sorted now, thank you for offering though!
Mine didn't origionally work because kernel functions can't take symbols so I chaneged todo what Oceananigans tendencies do and used indices instead. Tests are failing still but its a depdnecy version issue rather than anything else.
Thanks though!

Nice! Yeah, I also had a problem with unicode characters while using GPU.
Not sure if there is an open issue for this on Oceananigans, but maybe this is something we could add there.

I think its more of a general issue with KernelAbstractions not being able to compile something.

authors: iuryt jagoosw


text:
[Feature][Improvement] possibility to filter by multiple tags and deselect a tag!

Describe the solution you'd like
I would like to filter by pressing in more than  one tag at a time, and deselect a tag that was previously selected

@clebsonf Autarquia, you agreed?

Yes, assign to us @clebsonf!

authors: MarcosLudgerio VitorAmrm


text:
Eliminate dependency on frame-metadata

Followup to https://github.com/Off-Narrative-Labs/Tuxedo/issues/82
Now that https://github.com/paritytech/substrate/pull/14398 landed in Substrate, we should be able to remove frame-metadata, the last frame-specific logic from our dependency tree.
This is a pretty small change, we just need to do the corresponding Substrate update (probably next monthly release) and we will magically get the smaller dependency graph.

@JoshOrndorff up for visibility 😄

Solved in #86

authors: JoshOrndorff muraca

repo: Off-Narrative-Labs/Tuxedo


text:
Missing clases

Hi. I downloaded the code for this repository but it seems that are some classes that are missing from project. The classes are: "Attendee", "Recipient" and many others. Do you know if it;s from me or there are not included in the project ?
Thank you

Sorry, my bad. It was because nuget connot rester the packages. Sorry for this post, you cand delete it.
Thank you and thanks for this project

license: mit

license_source: bigquery


text:
Breadcrumb cannot be modified

If items are added to a Breadcrumb after initial creation, it is not possible to recreate the component to redistribute the items between the main list of items and overflow list.
Also, only one breadcrumb can be created on a page.
Here's a demo of both problems: https://codepen.io/dyepishin/pen/JXMxgz

@dmitriyyepishin Thanks for pointing out these bugs. We'll take a look.

Here's a fix that seems to work: https://codepen.io/dyepishin/pen/WwzbwK/?editors=1010

@dmitriyyepishin Thanks for offering up a fix. We have a fix in PR currently, #487 .
The scope of objects running in the prototype needed to be refactored. Also, three new methods have been added which allow the Breadcrumb to be updated on the fly.

addItem(itemLabel, itemLink, tabIndex)
removeItemByLabel(itemLabel)
removeItemByPosition(position)

You can see the demo of it here: http://codepen.io/anon/pen/qZoWap

Nice. Thank you.

There are couple issues in the new code:

If no link is specified when using addItem(), then href is set to “undefined”
Code comment in removeItemByPosition is incorrect

I also would like to suggest the following additions to the Breadcrumb component, which I used in my app:

New public method clear, which removes all items from the breadcrumb
New public method removeItemsAfterPosition, which removes all breadcrumb items after a specified position (this seems to be something often needed)
Add new style js-auto-remove to breadcrumb div, which, if specified, indicates that when an breadcrumb item is clicked, all items after it should be removed.

Here's a demo: https://codepen.io/dyepishin/pen/LNJXBO

This issue was moved to OfficeDev/office-ui-fabric-js#63

authors: dmitriyyepishin mikewheaton wdo3650

repo: OfficeDev/Office-UI-Fabric


text:
App.config missing from Core/OfficeDevPnP.Core.Tests

Category
[X] Bug
[ ] Enhancement
Environment
[X] Office 365 / SharePoint Online
[X] SharePoint 2016
[X] SharePoint 2013
Expected or Desired Behavior
After the initial clone the solution should build without any errors.
Observed Behavior
After the initial clone I get a build error because of a missing App.config file in OfficeDevPnP.Core.Tests which reads:
Severity	Code	Description	Project	File	Line	Suppression State
Error		Could not copy the file "App.config" because it was not found.	OfficeDevPnP.Core.Tests
The App.config is part of the project but missing on disk and in the repository under https://github.com/OfficeDev/PnP-Sites-Core/tree/master/Core/OfficeDevPnP.Core.Tests (only App.config.sample is present.
Steps to Reproduce
Do a fresh clone to a new repository location. Open the solution and build it. Simple workaround by copying App.config.sample to a new file App.config.
Thx in advance.

This is by design. We explicitly exclude the app.config to prevent accidental commits containing the app.config which might contain usernames and/or passwords. The steps you take (copy app.config.sample to app.config) is the right way forward. I will close this issue.

authors: andreashebeisen erwinvanhunen

repo: OfficeDev/PnP-Sites-Core


text:
Provisioning - validate taxonomy field default value

AddFieldAsXml does not validate default value for Taxonomy fields that results in situation when term item is not added to taxonomy hidden list and default value is not applied to items. It is also not possible to set correct WssId for the default field value at creation time, since there is client API to get Wss Id of the term.
I've added routine to validate default value for taxomony fields (and set correct Wss Id for the value) after field is created or updated. It actually works only if you specify -1 as Wss Id for the value.

Thx Ivan, clean and nice implementation.

authors: VesaJuvonen ivanvagunin


text:
dotenv with imports does not work

The template uses the following pattern in server.ts to load env vars before initializing the app:
require('dotenv').config();
import * as allComponents from './TeamsAppsComponents';

However, this does not work because all imports are resolved first before any top-level code is executed in the file. Thus, a bot component imported via TeamsAppsComponents, which uses the @BotDeclaration wrapper to provide the app ID and password from process.env, will not find those env vars.
The workaround is to modify the gulpfile's nodemon task to pass --require dotenv/config as node args.

Thanks for your comment. In fact it is working, TypeScript converts the import to a require statement which is required inline and assigned to a variable/const (allComponents) which in turn will invoke the declarators. So switching the order of those will not work.
You can do an easy test by adding a simple log statement into the bot constructor and log the value of the connector (for instance log(this.Connector);). With the order above you will see that botConnectorAudience has a value, but if you change the order the value of it will be undefined.
Is your experience different? If so, do you use any other versions of TypeScript or Webpack?

I stepped through it with a debugger as well as log statements. The import was definitely being processed before dotenv was being invoked.
I went digging because my bot's replies had mysteriously stopped working. There were no errors being logged to the console but when I stepped through it with a debugger, the Teams REST API was returning a 401 Unauthorized response when the bot tried to send a reply. When I looked back at the botSettings in the debugger, the appId and appPassword fields were empty. I added logging just before the bot class definition to show the env variables and they were not set either.
I'm using Typescript 3.1.2 and Webpack 4.16.3. I believe that's a newer version of Typescript than what the generator originally used. I upgraded it because I wanted some React-related features like element fragments (<></>).

Tried with Typescript 3.1.2 and 3.1.3 and still can't reproduce it. Have you made any other project changes (TS or Webpack config or gulp file changes?). Could you share your code?

authors: doxxx wictorwilen

repo: OfficeDev/generator-teams


text:
Author unable to select all options to send messages

One of our users has been added to a Teams channel contacting the App, however when they create a message, they only receive the following option for sending:

I have confirmed their UPN is added in the Azure app config.
To compare, I get this:

Hi @terryseddonpm ,
Thanks for raising the query.  Please verify the below points.

Since the user is added to the author list, Open the teams on web client and may be on incognito/InPrivate window would be better to avoid cache issue if not ready already.

So I managed to find the issue. You can't see it on the screenshot, but the scroll bar has disappeared! The app's page automatically focuses towards the bottom or button of the page, and sometimes the scrollbar is not apparent, and/or disappears after a short amount of time, so the user does not realise they need to scroll up to see the other options.
Could the app be altered to focus on the top of the form?

Hi @terryseddonpm ,
This issue has not been reported from any other users. However we will replicate the scenario from our end and check the screen.


text:
Typo "** Applies to: **"→"Applies to:"

https://docs.microsoft.com/en-us/skype-sdk/ucwa/contactdevicetype_ref

Docs Build status updates of commit 948b3e3:
:warning: Validation status: warnings

File
Status
Preview URL
Details

Skype/UCWA/ContactDeviceType_ref.md
:warning:Warning

Details

Skype/UCWA/ContactDeviceType_ref.md

[Warning] Missing attribute: title. Add a title string to show in search engine results.

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

repo: OfficeDev/skype-docs


text:
Automatically logged out after less then 2 minutes.

Hello all,
When I use this package, it throws me out of the backoffice every two minutes or so.
I don't even have this plugin enabled, only installed.
When i monitor the console, I see that after it requests https://localhost:44326/umbraco/backoffice/UmbracoTrees/ContentTree/GetNodes?id=1094&application=content&tree=&isDialog=false, it gives the following 401:
)]}',
{"Message":"Authorization has been denied for this request."}

is there some kind of timeout parameter I need to set for it to not throw me out every two minutes?

Also experiencing this issue. It doesn't appear to be cookie expiration. I'll see if I can debug, but any thoughts would be appreciated!

The only thing that I can see at the moment that might be this:
https://github.com/Offroadcode/Umbraco-2FA/blob/master/Umbraco2FA/Umbraco/Fortress/Startup/FortressOwinStartup.cs#L66
app.UseTwoFactorSignInCookie(global::Umbraco.Core.Constants.Security.BackOfficeTwoFactorAuthenticationType, TimeSpan.FromMinutes(5));

will try adjusting the 5 minute cookie timeout

The only thing that I can see at the moment that might be this:
https://github.com/Offroadcode/Umbraco-2FA/blob/master/Umbraco2FA/Umbraco/Fortress/Startup/FortressOwinStartup.cs#L66
app.UseTwoFactorSignInCookie(global::Umbraco.Core.Constants.Security.BackOfficeTwoFactorAuthenticationType, TimeSpan.FromMinutes(5));

will try adjusting the 5 minute cookie timeout

I saw that too about a week ago. But since it times out within two minutes, I don't think that is our problem.. I will also continue investigating this :)

@tommyw  Did you find anything yet? I'm still searching but i cant find the problem..

@reense nope, afraid not. haven't spent a huge amount of time on it though. will hopefully be able to find a bit more time next week.

@reense I just realised that there's mean to be an umbraco2FA admin section, but when I assign it to the administrators group, it causes the back office to fall over.

So.... I'm really not sure where to go from here. Neither of the Fortress DB tables offer any settings for timeout. I think it must be something to do with the authenticate API call, and found this: https://github.com/Offroadcode/Umbraco-2FA/blob/master/Umbraco2FA/Umbraco/Fortress/Controllers/TwoFactorAuthController.cs#L70
but that's not included in the .csproj, so I guess it's old.
I'll try to get the whole solution, with the JS/grunt running locally, might provide some clues.

@tommyw  Hmm, that's what I found as well. However, I'm still clueless why this issue is occurring. There is nothing in the code that suggests it automatically logs out. I think it could also be some exception that is silently occurring, and then logging out because of that. Still investigating..

Hey folks, its been pretty flat out crazy busy here at Offroadcode Towers so sorry for the lack of response.
For a little history the 2FA stuff was originally part of a bigger project which had a collection of security goodies in it that we called Fortress hence all the references. We chose to release the 2FA stuff separately but to do that we had to rip out some of the other goodies but we did it in a bit of a hurry. As a result there are some left over artefacts in there which might prove to be red herrings, sorry.
Timeout wise I can't think of anything off the top of my head but that cookie timeout would certainly be worth bumping up to see if it did anything. I'll ask around in the office this week and see what we can come up with.
Thanks for your interest in the package though and your keenness to help us get to the bottom of these issues :)

Cheers Pete, appreciate it. Would be a great addition to U7, and hopefully will be rolled into U8!

se the 2FA stuff separately but to do that we had to rip out some of the other goodies but we did it in a bit of a hurry. As a result there are some left over artefacts in there which might prove to be red herrings, sorry.
Timeout wise I can't think of anything off the top of my head but that cookie timeout would certainly be worth bumping up to see if it did anything. I'll ask around in the office this week and see what we

Great, thank you! Looking forward to hearing what your colleagues said ;)

Any Solution for the above issue?

Having the same issue as stated above, being logged out of the backoffice after about a minute.
When can we expect this bug to be fixed?

Sorry folks, it's been put on my backburner for a little while. I'll see what I can do about it this week.

@benjaminhowarth1 have you had any time to fix this bug? I want to implement this package for a client but in the current state it's a no go for us.

Hello @benjaminhowarth1 - do you have an update on this issue? This is an issue that is stopping us from completing a project. Is there a commercial support contract that you offer for this package and if so we are interested in that. Please let me know if that is a viable option and that case, I can reach out to you directly.
Thank you.

Hi folks, I'm taking another look at this tonight. @naresh-msc commercial support for OSS components is always appreciated, do feel free to drop me a line about helping to financially support the project going forward :)

Benjamin - Thank you for the efforts to fix this issue. Please let me know if we can setup some type of support contract for this component and we can look into it.
Thank you.
From: Benjamin Howarth notifications@github.com
Sent: Tuesday, February 19, 2019 11:37 AM
To: Offroadcode/Umbraco-2FA Umbraco-2FA@noreply.github.com
Cc: Naresh Kumar Narendran nnaresh@multimediasolutions.com; Mention mention@noreply.github.com
Subject: Re: [Offroadcode/Umbraco-2FA] Automatically logged out after less then 2 minutes. (#16)
Hi folks, I'm taking another look at this tonight. @naresh-mschttps://github.com/naresh-msc commercial support for OSS components is always appreciated, do feel free to drop me a line about helping to financially support the project going forward :)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/Offroadcode/Umbraco-2FA/issues/16#issuecomment-465209221, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AKKjLHfpkTbyfvbB1gE9BfaPo84BLAhFks5vPCgkgaJpZM4Yb6ia.

Hey all,
I found some nice standalone project that here - https://github.com/nul800sebastiaan/YubiKey2Factor
but with some bugs. so I fork it and remove all the unnecessary code (yubik key), you are welcome to give it a review.  [((https://github.com/ng-soft/YubiKey2Factor))]

Hi folks, I'm taking another look at this tonight. @naresh-msc commercial support for OSS components is always appreciated, do feel free to drop me a line about helping to financially support the project going forward :)

Do you have any update yet?

@ng-soft I looked at your repo. It would be much nicer to have just the dll's\code we would need for our own install's.

@kgiszewski, That true, but I tried to do it as a standalone solution that all you have to do is just download it without any installation. I guess that to do it as a plugin is a much better approach, but this it would take some time. In the meantime, all you have to do is copy the 2FactorAuthentication folder into your solution and it should work too without a lot of configurations.

Has any progress been made to rectify the user timeout issue? Or perhaps a recommended work around. Current the plugin is unusable. Thanks

See my the comments above

Thanks very much @ng-soft, I will try your forked repo https://github.com/ng-soft/umbraco-2fa-with-google-authenticator as a workaround. Is there any update on the progress of the issue for this plugin however?

I can confirm @ng-soft 's solution is working. In order to fix this package, remove the FortressOwinStartup.cs file and add the TwoFactorEventHandler.cs  code from @ng-soft to your ApplicationStartup.cs file.

I can confirm @ng-soft 's solution is working. In order to fix this package, remove the FortressOwinStartup.cs file and add the TwoFactorEventHandler.cs  code from @ng-soft to your ApplicationStartup.cs file.

@filipesousa20 After doing this I have to revert the owin:appStartup value back to UmbracoDefaultOwinStartup - correct? After doing this I no longer get prompted for my 2FA.

I can confirm @ng-soft 's solution is working. In order to fix this package, remove the FortressOwinStartup.cs file and add the TwoFactorEventHandler.cs  code from @ng-soft to your ApplicationStartup.cs file.

@filipesousa20 After doing this I have to revert the owin:appStartup value back to UmbracoDefaultOwinStartup - correct? After doing this I no longer get prompted for my 2FA.

I have a number of existing Umbraco7 sites that I need to add 2FA to.
They all have an existing complex custom applicationstartup config (overriding ApplicationStarting and ApplicationStarted).
I've tried to deal with the timeout issue as detailed above, but at the risk of sounding like a dummy, I am struggling to follow what needs to be copied from the forked repo and how it should be applied to this plugin.
Can anyone please advise what is the minimum changes required to make this package work and not timeout ?

Hello from the future!
Is there a working solution for this that is build as a Umbraco PlugIn package for V7 ? I'm also getting the 60sec logout issue.

It's been a long time so I can't easily tell what I did but I do have a working solution. However, taking a quick look it looks more like it is based on the repository here.

Having the same issue here with a brand new Umbraco 13 installation. Note: I have two Umbraco projects running locally, at the same time.

authors: Lukic PeteDuncanson ProNotion benjaminhowarth1 c9mbundy filipesousa20 iKomettech judgestone kgiszewski koenvanras lgroh naresh-msc ng-soft reense tommyw


text:
unicode error

I'm having a problem with the code (I just copied the one from https://medium.com/@guymodscientist/image-prediction-with-10-lines-of-code-3266f4039c7a and set the correct path for the sample image). Pycharm already warns me because of 7 different invalid escape sequences and when i execute it, following error message pops up:
SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated \UXXXXXXXX escape
I didn't find any issue that would solve this, so I'm sorry if it has already been solved, I must've overlooked it.
And just in case it helps, these are the invalid escape sequences:
\B \D \O \P \T \i \s

Hello  @Alililon, You need to copy the image into the directory where your code is. This would help resolve the issue.
As an example:
predictions, percentage_probabilities = prediction.predictImage("car.jpg", result_count=5)
for index in range(len(predictions)):
print(predictions[index] , " : " , percentage_probabilities[index])
You can see here I just wrote the name of my image ('car.jpg') because I made sure it is in the same directory as my code.

authors: AbimbolaOO Alililon

repo: OlafenwaMoses/ImageAI


text:
Odd interaction between eval-at-mark and vim-repeat

Howdy!
Ran into this while running conjure-school: At lesson 5, when I attempt to invoke the evaluate-at-mark command, I type <LL>em and before I can supply the mark, some evaluation happens immediately. This string gets inlined at the cursor position:
ilent! call repeat#set('\em', v:count)

In gif form:

If difficult to reproduce, I can come up with a minimal init.vim.
Cheers!

Fixed this bug on develop! Thank you for pointing it out, so sorry it impacted your ConjureSchool :disappointed: that's the literal worst time for this sort of thing to happen. Just having a poke around to see how hard it would be to get vim-repeat actually repeating your "eval at mark" with the mark you selected.

Did it :slightly_smiling_face: so you can now use eval at mark again and it actually works with vim-repeat. Releasing!

Thanks a ton!

authors: Olical arthropodSeven

repo: Olical/conjure

license: Unlicense


text:
Disable default filetype mappings

On 06/04/23, @nmkip asked on Discord::

Is there a way to disable all the default mappings in one line or do I have to go one by one?

This PR adds an option to disable all of the default mappings for the clients listed in g:conjure#filetypes.
I hope I didn't misunderstand what was wanted.

BTW, I forgot to mention that I arbitrarily chose conjure#mapping#enable_ft_mappings and am not terribly wedded to it. Please choose something that better aligns with your liking.

Thanks a lot! I like the name, it's as good as any and I'd only pick something similar. Just a colour of a bike shed really 😄 much appreciated!

authors: Olical russtoku


text:
🛑 Helix Intranet API - Dev is down

In e4bade7, Helix Intranet API - Dev (https://hbi-intranet-backend.onrender.com/status) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Helix Intranet API - Dev is back up in 1c45a0f after 9 minutes.

repo: OluwaninsolaAO/uptime-monitoring


text:
Duplicate using directives in Metadata as source

To reproduce:

Create a new console app
Open the containing folder in VS Code
One the project is loaded "go to definition" on string

OmniSharp log:
Starting OmniSharp server at 5/12/2020, 10:03:29 AM
    Target: /Volumes/Data/Source/console100

OmniSharp server started with Mono 6.10.0.
    Path: /Users/joeyrobichaud/.vscode-insiders/extensions/ms-dotnettools.csharp-1.21.19/.omnisharp/1.35.1/omnisharp/OmniSharp.exe
    PID: 44882

[info]: OmniSharp.Stdio.Host
        Starting OmniSharp on MacOS 10.15.4 (x64)
[info]: OmniSharp.Services.DotNetCliService
        DotNetPath set to dotnet
[info]: OmniSharp.MSBuild.Discovery.MSBuildLocator
        Located 2 MSBuild instance(s)
            1: Mono 16.4 - "/Library/Frameworks/Mono.framework/Versions/6.10.0/lib/mono/msbuild/Current/bin"
            2: StandAlone 16.4 - "/Users/joeyrobichaud/.vscode-insiders/extensions/ms-dotnettools.csharp-1.21.19/.omnisharp/1.35.1/omnisharp/.msbuild/Current/Bin"
[info]: OmniSharp.MSBuild.Discovery.MSBuildLocator
        Registered MSBuild instance: Mono 16.4 - "/Library/Frameworks/Mono.framework/Versions/6.10.0/lib/mono/msbuild/Current/bin"
            CscToolExe = csc.exe
            CscToolPath = /Users/joeyrobichaud/.vscode-insiders/extensions/ms-dotnettools.csharp-1.21.19/.omnisharp/1.35.1/omnisharp/.msbuild/Current/Bin/Roslyn
[info]: OmniSharp.Cake.CakeProjectSystem
        Detecting Cake files in '/Volumes/Data/Source/console100'.
[info]: OmniSharp.Cake.CakeProjectSystem
        Could not find any Cake files
[info]: OmniSharp.MSBuild.ProjectSystem
        No solution files found in '/Volumes/Data/Source/console100'
[info]: OmniSharp.MSBuild.ProjectManager
        Queue project update for '/Volumes/Data/Source/console100/console100.csproj'
[info]: OmniSharp.Script.ScriptProjectSystem
        Detecting CSX files in '/Volumes/Data/Source/console100'.
[info]: OmniSharp.Script.ScriptProjectSystem
        Could not find any CSX files
[info]: OmniSharp.WorkspaceInitializer
        Invoking Workspace Options Provider: OmniSharp.Roslyn.CSharp.Services.CSharpFormattingWorkspaceOptionsProvider, Order: 0
[info]: OmniSharp.MSBuild.ProjectManager
        Loading project: /Volumes/Data/Source/console100/console100.csproj
[info]: OmniSharp.WorkspaceInitializer
        Invoking Workspace Options Provider: OmniSharp.Roslyn.CSharp.Services.RenameWorkspaceOptionsProvider, Order: 100
[info]: OmniSharp.WorkspaceInitializer
        Invoking Workspace Options Provider: OmniSharp.Roslyn.CSharp.Services.ImplementTypeWorkspaceOptionsProvider, Order: 110
[info]: OmniSharp.WorkspaceInitializer
        Invoking Workspace Options Provider: OmniSharp.Roslyn.CSharp.Services.BlockStructureWorkspaceOptionsProvider, Order: 140
[info]: OmniSharp.WorkspaceInitializer
        Configuration finished.
[info]: OmniSharp.Stdio.Host
        Omnisharp server running using Stdio at location '/Volumes/Data/Source/console100' on host 44799.
[info]: OmniSharp.MSBuild.ProjectManager
        Successfully loaded project file '/Volumes/Data/Source/console100/console100.csproj'.
[info]: OmniSharp.MSBuild.ProjectManager
        Adding project '/Volumes/Data/Source/console100/console100.csproj'
[info]: OmniSharp.MSBuild.ProjectManager
        Update project: console100
[info]: OmniSharp.Roslyn.CSharp.Services.Diagnostics.CSharpDiagnosticWorkerWithAnalyzers
        Solution initialized -> queue all documents for code analysis. Initial document count: 3.

we call Roslyn's metadata as service with reflection so I guess the issue would be there? unless I messed up some refactoring recently 😂

Indeed it does reproduce in VS with Roslyn. Sigh... Moving this to the rolsyn repo.

authors: JoeRobich filipw

repo: OmniSharp/omnisharp-roslyn


text:
Ubuntu: "The SDK 'Microsoft.Net.Sdk' specified could not be found."

I've installed the net core 2.1 SDK and the projects load correctly. I then installed the 3.0-preview and changed the solution to use this instead and now they can no longer load in VSCode. It loads perfectly fine on my Windows machine.
I've tried setting the SDK path environment variable explicitly but no difference: MsBuildSDKsPath=/usr/share/dotnet/sdk/3.0.100-preview-010184/Sdks/ as well as changing omnisharp.path to latest, setting the path inside omnisharp.json directly and using the insiders build of VSCode.
Is there anything off in my environment? I've created a VM with Linux specifically to test .net core 3 on it so there shouldn't be anything unusual interfering with things. Happy to try out whatever you can suggest.
Example csproj file:
<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>netcoreapp3.0</TargetFramework>
    <LangVersion>8.0</LangVersion>
    <IsPackable>false</IsPackable>
  </PropertyGroup>
  <ItemGroup>
      <PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.3.0" />
      <PackageReference Include="Mongo2Go" Version="2.2.8" />
      <PackageReference Include="Moq" Version="4.7.145" />
      <PackageReference Include="xunit" Version="2.2.0" />
      <PackageReference Include="xunit.runner.visualstudio" Version="2.2.0" />
  </ItemGroup>
  <ItemGroup>
    <ProjectReference Include="..\..\..\src\MyApp.Client\MyApp.Client.csproj" />
    <ProjectReference Include="..\..\..\src\MyApps\MyApp.csproj" />
  </ItemGroup>
</Project>

Environment data
dotnet --info output:
.NET Core SDK (reflecting any global.json):
 Version:   3.0.100-preview-010184
 Commit:    c57bde4593

Runtime Environment:
 OS Name:     ubuntu
 OS Version:  18.04
 OS Platform: Linux
 RID:         ubuntu.18.04-x64
 Base Path:   /usr/share/dotnet/sdk/3.0.100-preview-010184/

Host (useful for support):
  Version: 3.0.0-preview-27324-5
  Commit:  63a01b08e5

.NET Core SDKs installed:
  2.2.103 [/usr/share/dotnet/sdk]
  3.0.100-preview-010184 [/usr/share/dotnet/sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.2.1 [/usr/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.2.1 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.0-preview-19075-0444 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.2.1 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.0.0-preview-27324-5 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

VS Code version:
Version: 1.30.2
Commit: 61122f88f0bf01e2ac16bdb9e1bc4571755f5bd8
Date: 2019-01-07T22:49:48.319Z
Electron: 2.0.12
Chrome: 61.0.3163.100
Node.js: 8.9.3
V8: 6.1.534.41
OS: Linux x64 4.15.0-32-generic

C# Extension version: 1.17.1
Steps to reproduce

Install 2.2.1 SDK
Install 3.0-preview SDK
dotnet new console
Open via code .

Expected  behavior
The project loads correctly and code lens options show.
Actual behavior
Errors in the output and no code lens available. However, using dotnet run runs the project as intended.
Starting OmniSharp server at 27/01/2019, 20:04:35
Target: /home/jeroen/Documents/github/my-app/MyApp.Webapp.sln

OmniSharp server started.
Path: /home/jeroen/.vscode/extensions/ms-vscode.csharp-1.17.1/.omnisharp/1.32.8/run
PID: 20475

[info]: OmniSharp.Stdio.Host
Starting OmniSharp on ubuntu 18.4 (x64)
[info]: OmniSharp.Services.DotNetCliService
DotNetPath set to dotnet
[info]: OmniSharp.MSBuild.Discovery.MSBuildLocator
Located 1 MSBuild instance(s)
1: StandAlone 15.0 - "/home/jeroen/.vscode/extensions/ms-vscode.csharp-1.17.1/.omnisharp/1.32.8/omnisharp/msbuild/15.0/Bin"
[info]: OmniSharp.MSBuild.Discovery.MSBuildLocator
MSBUILD_EXE_PATH environment variable set to '/home/jeroen/.vscode/extensions/ms-vscode.csharp-1.17.1/.omnisharp/1.32.8/omnisharp/msbuild/15.0/Bin/MSBuild.dll'
[info]: OmniSharp.MSBuild.Discovery.MSBuildLocator
Registered MSBuild instance: StandAlone 15.0 - "/home/jeroen/.vscode/extensions/ms-vscode.csharp-1.17.1/.omnisharp/1.32.8/omnisharp/msbuild/15.0/Bin"
MSBuildExtensionsPath = /home/jeroen/.vscode/extensions/ms-vscode.csharp-1.17.1/.omnisharp/1.32.8/omnisharp/msbuild
BypassFrameworkInstallChecks = true
CscToolPath = /home/jeroen/.vscode/extensions/ms-vscode.csharp-1.17.1/.omnisharp/1.32.8/omnisharp/msbuild/15.0/Bin/Roslyn
CscToolExe = csc.exe
MSBuildToolsPath = /home/jeroen/.vscode/extensions/ms-vscode.csharp-1.17.1/.omnisharp/1.32.8/omnisharp/msbuild/15.0/Bin
[info]: OmniSharp.Cake.CakeProjectSystem
Detecting Cake files in '/home/jeroen/Documents/github/my-app'.
[info]: OmniSharp.Cake.CakeProjectSystem
Could not find any Cake files
[info]: OmniSharp.WorkspaceInitializer
Project system 'OmniSharp.DotNet.DotNetProjectSystem' is disabled in the configuration.
[info]: OmniSharp.MSBuild.ProjectSystem
Detecting projects in '/home/jeroen/Documents/github/my-app/MyApp.Webapp.sln'.
[info]: OmniSharp.MSBuild.ProjectManager
Queue project update for '/home/jeroen/Documents/github/my-app/src/MyApp.Webapp/MyApp.Webapp.csproj'
[info]: OmniSharp.MSBuild.ProjectManager
Queue project update for '/home/jeroen/Documents/github/my-app/src/MyApp.Client/MyApp.Client.csproj'
[info]: OmniSharp.MSBuild.ProjectManager
Queue project update for '/home/jeroen/Documents/github/my-app/src/MyApp/MyApp.csproj'
[info]: OmniSharp.MSBuild.ProjectManager
Queue project update for '/home/jeroen/Documents/github/my-app/tests/unit/MyApp.Tests/MyApp.Tests.csproj'
[info]: OmniSharp.Script.ScriptProjectSystem
Detecting CSX files in '/home/jeroen/Documents/github/my-app'.
[info]: OmniSharp.Script.ScriptProjectSystem
Could not find any CSX files
[info]: OmniSharp.WorkspaceInitializer
Invoking Workspace Options Provider: OmniSharp.Roslyn.CSharp.Services.CSharpWorkspaceOptionsProvider
[info]: OmniSharp.WorkspaceInitializer
Configuration finished.
[info]: OmniSharp.Stdio.Host
Omnisharp server running using Stdio at location '/home/jeroen/Documents/github/my-app' on host 20382.
[info]: OmniSharp.MSBuild.ProjectManager
Loading project: /home/jeroen/Documents/github/my-app/src/MyApp.Webapp/MyApp.Webapp.csproj
[warn]: OmniSharp.MSBuild.ProjectManager
Failed to load project file '/home/jeroen/Documents/github/my-app/src/MyApp.Webapp/MyApp.Webapp.csproj'.
/home/jeroen/Documents/github/my-app/src/MyApp.Webapp/MyApp.Webapp.csproj(1,1)
Microsoft.Build.Exceptions.InvalidProjectFileException: The SDK 'Microsoft.NET.Sdk.Web' specified could not be found. /home/jeroen/Documents/github/my-app/src/MyApp.Webapp/MyApp.Webapp.csproj
at Microsoft.Build.Shared.ProjectErrorUtilities.ThrowInvalidProject (System.String errorSubCategoryResourceName, Microsoft.Build.Shared.IElementLocation elementLocation, System.String resourceName, System.Object[] args) [0x00040] in <2338fdb301034c1d9c6b95e564d514b0>:0
at Microsoft.Build.Shared.ProjectErrorUtilities.VerifyThrowInvalidProject[T1] (System.Boolean condition, System.String errorSubCategoryResourceName, Microsoft.Build.Shared.IElementLocation elementLocation, System.String resourceName, T1 arg0) [0x00003] in <2338fdb301034c1d9c6b95e564d514b0>:0
at Microsoft.Build.Shared.ProjectErrorUtilities.ThrowInvalidProject[T1] (Microsoft.Build.Shared.IElementLocation elementLocation, System.String resourceName, T1 arg0) [0x00000] in <2338fdb301034c1d9c6b95e564d514b0>:0
at Microsoft.Build.Evaluation.Evaluator`4[P,I,M,D].ExpandAndLoadImportsFromUnescapedImportExpressionConditioned (System.String directoryOfImportingFile, Microsoft.Build.Construction.ProjectImportElement importElement, System.Collections.Generic.List`1[Microsoft.Build.Construction.ProjectRootElement]& projects, System.Boolean throwOnFileNotExistsError) [0x00254] in <2338fdb301034c1d9c6b95e564d514b0>:0
at Microsoft.Build.Evaluation.Evaluator`4[P,I,M,D].ExpandAndLoadImports (System.String directoryOfImportingFile, Microsoft.Build.Construction.ProjectImportElement importElement) [0x00024] in <2338fdb301034c1d9c6b95e564d514b0>:0
at Microsoft.Build.Evaluation.Evaluator`4[P,I,M,D].EvaluateImportElement (System.String directoryOfImportingFile, Microsoft.Build.Construction.ProjectImportElement importElement) [0x0000d] in <2338fdb301034c1d9c6b95e564d514b0>:0
at Microsoft.Build.Evaluation.Evaluator`4[P,I,M,D].PerformDepthFirstPass (Microsoft.Build.Construction.ProjectRootElement currentProjectOrImport) [0x000e6] in <2338fdb301034c1d9c6b95e564d514b0>:0
at Microsoft.Build.Evaluation.Evaluator`4[P,I,M,D].Evaluate (Microsoft.Build.BackEnd.Logging.ILoggingService loggingService, Microsoft.Build.Framework.BuildEventContext buildEventContext) [0x00103] in <2338fdb301034c1d9c6b95e564d514b0>:0
at Microsoft.Build.Evaluation.Evaluator`4[P,I,M,D].Evaluate (Microsoft.Build.Evaluation.IEvaluatorData`4[P,I,M,D] data, Microsoft.Build.Construction.ProjectRootElement root, Microsoft.Build.Evaluation.ProjectLoadSettings loadSettings, System.Int32 maxNodeCount, Microsoft.Build.Collections.PropertyDictionary`1[T] environmentProperties, Microsoft.Build.BackEnd.Logging.ILoggingService loggingService, Microsoft.Build.Evaluation.IItemFactory`2[S,T] itemFactory, Microsoft.Build.Evaluation.IToolsetProvider toolsetProvider, Microsoft.Build.Evaluation.ProjectRootElementCache projectRootElementCache, Microsoft.Build.Framework.BuildEventContext buildEventContext, Microsoft.Build.Execution.ProjectInstance projectInstanceIfAnyForDebuggerOnly, Microsoft.Build.BackEnd.SdkResolution.ISdkResolverService sdkResolverService, System.Int32 submissionId, Microsoft.Build.Evaluation.Context.EvaluationContext evaluationContext) [0x0001a] in <2338fdb301034c1d9c6b95e564d514b0>:0
at Microsoft.Build.Evaluation.Project.Reevaluate (Microsoft.Build.BackEnd.Logging.ILoggingService loggingServiceForEvaluation, Microsoft.Build.Evaluation.ProjectLoadSettings loadSettings) [0x0004c] in <2338fdb301034c1d9c6b95e564d514b0>:0
at Microsoft.Build.Evaluation.Project.ReevaluateIfNecessary (Microsoft.Build.BackEnd.Logging.ILoggingService loggingServiceForEvaluation, Microsoft.Build.Evaluation.ProjectLoadSettings loadSettings) [0x00034] in <2338fdb301034c1d9c6b95e564d514b0>:0
at Microsoft.Build.Evaluation.Project.ReevaluateIfNecessary (Microsoft.Build.BackEnd.Logging.ILoggingService loggingServiceForEvaluation) [0x00000] in <2338fdb301034c1d9c6b95e564d514b0>:0
at Microsoft.Build.Evaluation.Project.ReevaluateIfNecessary (Microsoft.Build.Evaluation.Context.EvaluationContext evaluationContext) [0x00023] in <2338fdb301034c1d9c6b95e564d514b0>:0
at Microsoft.Build.Evaluation.Project.Initialize (System.Collections.Generic.IDictionary`2[TKey,TValue] globalProperties, System.String toolsVersion, System.String subToolsetVersion, Microsoft.Build.Evaluation.ProjectLoadSettings loadSettings, Microsoft.Build.Evaluation.Context.EvaluationContext evaluationContext) [0x00126] in <2338fdb301034c1d9c6b95e564d514b0>:0
at Microsoft.Build.Evaluation.Project..ctor (System.String projectFile, System.Collections.Generic.IDictionary`2[TKey,TValue] globalProperties, System.String toolsVersion, System.String subToolsetVersion, Microsoft.Build.Evaluation.ProjectCollection projectCollection, Microsoft.Build.Evaluation.ProjectLoadSettings loadSettings, Microsoft.Build.Evaluation.Context.EvaluationContext evaluationContext) [0x0009e] in <2338fdb301034c1d9c6b95e564d514b0>:0
at Microsoft.Build.Evaluation.Project..ctor (System.String projectFile, System.Collections.Generic.IDictionary`2[TKey,TValue] globalProperties, System.String toolsVersion, System.String subToolsetVersion, Microsoft.Build.Evaluation.ProjectCollection projectCollection, Microsoft.Build.Evaluation.ProjectLoadSettings loadSettings) [0x00000] in <2338fdb301034c1d9c6b95e564d514b0>:0
at Microsoft.Build.Evaluation.Project..ctor (System.String projectFile, System.Collections.Generic.IDictionary`2[TKey,TValue] globalProperties, System.String toolsVersion, Microsoft.Build.Evaluation.ProjectCollection projectCollection, Microsoft.Build.Evaluation.ProjectLoadSettings loadSettings) [0x00000] in <2338fdb301034c1d9c6b95e564d514b0>:0
at Microsoft.Build.Evaluation.Project..ctor (System.String projectFile, System.Collections.Generic.IDictionary`2[TKey,TValue] globalProperties, System.String toolsVersion, Microsoft.Build.Evaluation.ProjectCollection projectCollection) [0x00000] in <2338fdb301034c1d9c6b95e564d514b0>:0
at Microsoft.Build.Evaluation.ProjectCollection.LoadProject (System.String fileName, System.Collections.Generic.IDictionary`2[TKey,TValue] globalProperties, System.String toolsVersion) [0x000f5] in <2338fdb301034c1d9c6b95e564d514b0>:0
at Microsoft.Build.Evaluation.ProjectCollection.LoadProject (System.String fileName, System.String toolsVersion) [0x00000] in <2338fdb301034c1d9c6b95e564d514b0>:0
at OmniSharp.MSBuild.ProjectLoader.EvaluateProjectFileCore (System.String filePath) [0x0003f] in <410de907c9654d6593c7a8d4ded6b20a>:0
at OmniSharp.MSBuild.ProjectLoader.BuildProject (System.String filePath) [0x0000d] in <410de907c9654d6593c7a8d4ded6b20a>:0
at OmniSharp.MSBuild.ProjectFile.ProjectFileInfo.Load (System.String filePath, OmniSharp.MSBuild.ProjectLoader loader) [0x00015] in <410de907c9654d6593c7a8d4ded6b20a>:0
at OmniSharp.MSBuild.ProjectManager+<>c__DisplayClass25_0.<LoadProject>b__0 () [0x00000] in <410de907c9654d6593c7a8d4ded6b20a>:0
at (wrapper delegate-invoke) System.Func`1[System.ValueTuple`3[OmniSharp.MSBuild.ProjectFile.ProjectFileInfo,System.Collections.Immutable.ImmutableArray`1[OmniSharp.MSBuild.Logging.MSBuildDiagnostic],OmniSharp.MSBuild.Notification.ProjectLoadedEventArgs]].invoke_TResult()
at OmniSharp.MSBuild.ProjectManager.LoadOrReloadProject (System.String projectFilePath, System.Func`1[TResult] loader) [0x0001b] in <410de907c9654d6593c7a8d4ded6b20a>:0

FYI if it helps anyone. I've got a Hyper-V VM running Ubuntu 19.04 and the snap vscode is installed, and I had this same issue, which I seem to have resolved by installing the dotnet-sdk snap and doing a "sudo ln -sv /snap/dotnet-sdk/current/dotnet /usr/local/bin" to manually create a symbolic link where Omnisharp would find it. In a perfect world, all I'd have to do is something like "sudo snap alias dotnet-sdk.dotnet dotnet" but I never could get Omnisharp to work that way as it appears to ignore /snap/bin or something (not sure). You can also just create a "~/bin" folder and make the symbolic link there (no sudo required, obviously, but I went ahead and made it system-wide for the VM).

@Vannevelj Could you try the suggestion from @byroniac and see if it works.
Also you can try adding omnisharp.json file as suggested here

@akshita31 I no longer am using Linux/VSCode and the VM is gone as well, so I'm afraid I won't be able to test that out.

@Vannevelj In that case I will close the issue here. Please reopen if the you encounter the problem again

@akshita31 I'm having the same issue.  My OS (Solus) doesn't use /usr/local .. I tried the symlink to ~/bin/dotnet but I still get the issue.
Even if the .csproj references Microsoft.NET.Sdk, the error still shows Microsoft.NET.Sdk.Web
The SDK 'Microsoft.NET.Sdk.Web' specified could not be found.
<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>netcoreapp3.0</TargetFramework>
    <RootNamespace>coreapp</RootNamespace>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.AspNetCore.Mvc.NewtonsoftJson" Version="3.0.0-preview-19075-0444" />
    <PackageReference Include="Microsoft.EntityFrameworkCore" Version="3.0.0-preview.19074.3" />
    <PackageReference Include="Microsoft.EntityFrameworkCore.Design" Version="3.0.0-preview.19074.3" />
    <PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" Version="3.0.0-preview.19074.3" />
  </ItemGroup>

</Project>

.NET Core SDK (reflecting any global.json):
 Version:   3.0.100-preview-010184
 Commit:    c57bde4593

Runtime Environment:
 OS Name:     solus
 OS Version:  3.9999
 OS Platform: Linux
 RID:         linux-x64
 Base Path:   /snap/dotnet-sdk/28/sdk/3.0.100-preview-010184/

Host (useful for support):
  Version: 3.0.0-preview-27324-5
  Commit:  63a01b08e5

.NET Core SDKs installed:
  3.0.100-preview-010184 [/snap/dotnet-sdk/28/sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.App 3.0.0-preview-19075-0444 [/snap/dotnet-sdk/28/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 3.0.0-preview-27324-5 [/snap/dotnet-sdk/28/shared/Microsoft.NETCore.App]

@arcooke Could you try setting MSBuildSdkPath as stated here

@akshita31
I can confirm that works.  I added export MSBuildSDKsPath=/snap/dotnet-sdk/current/sdk/$(dotnet --version)/Sdks to bashrc and deleted the legacysdkresolver file and it builds successfully.

@arcooke Glad you got it working

My vote would be to reopen this issue, why?
This initial UX for Linux-based devs trying out .NET dev in VSCode via OmniSharp is sub-par, here's how it goes if .NET Core isn't in your distro's package manager:

one downloads the .NET SDK, extracts it somewhere
sets env-var DOTNET_ROOT and ensures dotnet CLI is in PATH
all works, one can dotnet build and dotnet run and dotnet new projects just fine
try out the VScode C# extension aka OmniSharp!
"The SDK 'Microsoft.Net.Sdk' specified could not be found."

Well? Your Nodejs extension likely executes msbuild or other processes via standard Nodejs process / command-exec APIs. You can set your own process' env vars and they should be inherited by processes you spawn. You can also expressly elaborate the env for the process being spawned.
So what you do is: notice that MSBuildSDKsPath isn't set but DOTNET_ROOT is set, then set the former on-the-fly to what @arcooke outlined, but basing on DOTNET_ROOT instead of /snap/.... of course.

This should really be automated, or at the very least documented properly somewhere.

Seems for me removing Visual Studio 2019 and their paths and dependencies solves the problem.

@ralphchristianeclipse under Ubuntu?  :rofl:

authors: Vannevelj akshita31 arcooke byroniac danielcooperxyz metaleap ralphchristianeclipse

repo: OmniSharp/omnisharp-vscode


text:
No IntelliSense in C# file that is not included in a project

Seriously? What kind of editor is this?
If I need to follow certain instructions in order to get basic Intellisense on C# just to edit a single .cs file, then this needs to be really freaking obvious.

This is something we'll definitely be working on. Today, you can't get IntelliSense for a file without it belonging to a project file, which is not a great experience.

And I can't create a project too, because there is no "New Project" option on the menu, which is also not a great experience either.

@leocrespo -- bear in mind that this is an editor and not an IDE. "New Project" isn't something that you find in many other editors (like, say Sublime Text).

So there's no Intellisense because the file doesn't belong to a project, and yet I can't create a project. Sure, makes sense.

As I said, we'll be working on IntelliSense for project-less scenarios. At the moment, the most common way to create a C# project for VS Code scenarios is to use a scaffolding tool like yeoman or to create a project.json manually. Eventually, you'll be able to use the .NET command-line tools to create a project with "dotnet new".

@DustinCampbell  'Dotnet new' creates simple Console app template. vscode can't reate a project. Yeoman is probably outdated, it doesn't work with latest CLI. Is there any way of creating web api or class library?

Sorry if this is obvious, but the problem is that there is a major shift in how ASP.NET Core works, and it hasn't been fully released. I think your best bet is https://github.com/aspnet/cli-samples. If you have trouble, you may want to follow up with @sayedihashimi, and I believe he owns the yoman generator.

yo aspnet is typically updated a few days before the release. We don't have a good mechanism to get pre-release templates out today. The main issue on my side here is that we have to author templates twice. Once for VS and once for yo aspnet. The Microsoft team authors templates in aspnet/templates initially for VS, and then we post the final result to get added in yo aspnet. I'm working to define a new approach to templates where we can have one source of truth for templates which then get added to both VS and yo aspnet. Once I make good progress there then we will have a better story for pre-release templates. That will likely take me at least a few months as I'm doing all the work on the side at nights.

this looks like the same as this https://github.com/OmniSharp/omnisharp-roslyn/issues/207
I would like to mention that you can get project-less intellisense in CSX files already in omnisharp

This is an old issue but what needs to be done to accomplish this? At my job I review and trace code more than I write it. Usually I'll clone multiple repos and go through code, tracing things by hovering over a symbol and going to definition.
For example, I cloned https://github.com/aspnet/mvc. In Sublime with no additional setup I can just hover over a symbol and see where it's defined:

In VS Code the story is different:

What would need to be done in order to match Sublime's out-of-box experience with projects like this?

@landaire: This issue is different than what you've reported here. This issue is about opening just a single file with no project file. However, in your scenario (cloning MVC), you certainly have a project file.

Hmmm. True. I was under the impression I was using a different C# plugin that was not backed by Omnisharp but that's the plugin I'm using. Is there a separate issue already that you're aware of for what I'm referring to?

Without more information, it's hard to determine what's going on and whether there's an existing issue that matches what you're seeing. Have you tried clicking the OmniSharp icon at the bottom right of the status bar and picking a particular solution or project to launch OmniSharp on? Also, what does the OmniSharp Log display? (You can see this by selecting View->Output and then picking "OmniSharp Log" from the dropdown in the top-right of the Output pane).

Will a good use when you want to create small script to be use with dotnet script (https://github.com/filipw/dotnet-script). Otherwise i have to create a console application just to test something out.

So this still is the case, that in order to use OmniSharp's services, one needs to create a full project, instead of using a single file?

Nice - when will v1.16 be released with this?

The latest version of the c# extension which includes intellisense support for standalone c# files is out. Please do give it a try.

authors: Davilink DustinCampbell akshita31 dhnnjy filipw gregg-miskelly hifall landaire leocrespo mzhukovs sayedihashimi


text:
Daniel/card grid

Issue Type
[ ] Bug
[x] Feature
[ ] Tech Debt
Description
Created the foundation of our CardGrid and MediaCards which will ultimately hold the photos, descriptions, ingredients, and instructions pulled from the backend.
Expected behavior
Expect Cards to render within the card grid with dummy data. (Not fully implemented as I am trying ot get Sharon the most up-to-date code to work on.)

Looks good!

repo: One-Stop-Shopping/One-Stop-Shopping


text:
add auth to dev.onearmy.world

Create an basic htaccess auth with user & psw to access dev.onearmy.world

what's the purpose for this?

To keep the dev site for internal use, and we'll have the alpha.onearmy.world to deploy the release on a public url.

That would mean creating another database and auth system which I am not keen to do. We need a site that anybody who wants to contribute to the project can easily use, this is the purpose of the staging site. For each URL we have to essentially define the database, entrypoint and access, we currently have
Staging DB - localhost dev or dev.onearmy.world live, for any developer
Production DB - beta.onearmy.world, to display the latest production (/tagged) release
If you want the dev site not to be exposed publicly we can take down or change the url, but I really don't see the harm in having it open for anyone who wants to use it. If you really wanted I could add a script that deletes content periodically to keep it a bit cleaner, but for now the usage is still really low.

authors: BenGamma chrismclarke

repo: OneArmyWorld/onearmy


text:
OSError: [Errno 98] Address already in use`

I am using Google Colab.
I installed onedrive-sdk-python by doing:
pip install https://github.com/OneDrive/onedrive-sdk-python/archive/master.zip

And this is my authentication code:
import onedrivesdk
from onedrivesdk.helpers import GetAuthCodeServer
from onedrivesdk.helpers.resource_discovery import ResourceDiscoveryRequest

redirect_uri = 'http://localhost:8080'
client_id = "my client id"
client_secret = "my client secret"
discovery_uri = 'api.office.com/discovery'
auth_server_url = 'login.microsoftonline.com/common/oauth2/authorize'
auth_token_url = 'login.microsoftonline.com/common/oauth2/token'

http = onedrivesdk.HttpProvider()
auth = onedrivesdk.AuthProvider(http, client_id,
                                auth_server_url=auth_server_url,
                                auth_token_url=auth_token_url)
auth_url = auth.get_auth_url(redirect_uri)
code = GetAuthCodeServer.get_auth_code(auth_url, redirect_uri)
auth.authenticate(code, redirect_uri, client_secret, resource=discovery_uri)

And I got this error:
OSError Traceback (most recent call last)
in
code = GetAuthCodeServer.get_auth_code(auth_url, redirect_uri)

/usr/lib/python3.6/socketserver.py in server_bind(self)
self.socket.bind(self.server_address)

OSError: [Errno 98] Address already in use`

Originally posted by @KTibow in https://github.com/OneDrive/onedrive-sdk-python/issues/179#issuecomment-624064609

Try rebooting your computer, and running your code as administrator/root/super user.

Well I can't. I am using Google Colab.
My notebook on google colab is runing on 64 bit, 12 GB ram and python 3.6. That all what i know.

@ashish453 You won't be able to do that, then. Even if this worked, you'd need to be able to type things into a web browser that popped up on the Google Colab computer. This was meant to work on machines with an accessible desktop GUI, but a Google Colab computer is basically a command prompt that can run your code. Consider closing, as this is not related to onedrive-sdk-python but more related to Google Colab.
If you continue running into this error on a computer that you can access, then delete your app on Azure and make it use the redirect URI http://localhost:9740 instead. Change your code to use that address too.

so isn't there any way to upload files in onedrive  business from google colab?

@ashish453 use the other code sample instead:
import onedrivesdk

redirect_uri = 'http://localhost:8080/'
client_secret = 'your_client_secret'
client_id='your_client_id'
api_base_url='https://api.onedrive.com/v1.0/'
scopes=['wl.signin', 'wl.offline_access', 'onedrive.readwrite']

http_provider = onedrivesdk.HttpProvider()
auth_provider = onedrivesdk.AuthProvider(
    http_provider=http_provider,
    client_id=client_id,
    scopes=scopes)

client = onedrivesdk.OneDriveClient(api_base_url, auth_provider, http_provider)
auth_url = client.auth_provider.get_auth_url(redirect_uri)
# Ask for the code
print('Paste this URL into your browser, approve the app\'s access.')
print('Copy everything in the address bar after "code=", and paste it below.')
print(auth_url)
code = input('Paste code here: ')

client.auth_provider.authenticate(code, redirect_uri, client_secret)

The above code requires copy-pasting into your browser and back into your console. If you want to remove some of that manual work, you can use the helper class GetAuthCodeServer. That helper class spins up a webserver, so this method cannot be used on all environments.
I didn't get this instruction.
can you please explain me what to do before and after running the code..

@ashish453 it will generate a URL that you open in another tab. It gives you some stuff, and you copy and paste it back in.

@ashish453 why don't you just run it on your own computer?

Well I am using Google colab to download the large files from internet. As it gets downloaded very fast and then i need to upload it in onedrive. Using my PC it will consume terrible amount of time as i didn't have server as fast as google has to download 80 gb of files daily.

@ashish453 do it on your own computer, save your session to a pickle, and upload the pickle file and use it instead.

I can't log in into my account via the url generated by the code.
When I enter my email It says
**That Microsoft account doesn't exist. Enter a different account or get a new one.
**
But I do have onedrive account with my email.

@ashish453 try it again. If it still doesn't work see if you can log into online onedrive with the same email and password.

Yes i did.
I can login by onedrive official page but i can't access my account from code generated link.

I have Onedrive For Business.

Try
import onedrivesdk
from onedrivesdk.helpers import GetAuthCodeServer
from onedrivesdk.helpers.resource_discovery import ResourceDiscoveryRequest

redirect_uri = 'http://localhost:8080'
client_id = "my client id"
client_secret = "my client secret"
discovery_uri = 'api.office.com/discovery'
auth_server_url = 'login.microsoftonline.com/common/oauth2/authorize'
auth_token_url = 'login.microsoftonline.com/common/oauth2/token'

http = onedrivesdk.HttpProvider()
auth = onedrivesdk.AuthProvider(http, client_id,
                                auth_server_url=auth_server_url,
                                auth_token_url=auth_token_url)
auth_url = auth.get_auth_url(redirect_uri)
print("Paste this in, and copy/paste the thing after 'code='.")
print(auth_url)
code = input('Paste code here: ')
auth.authenticate(code, redirect_uri, client_secret, resource=discovery_uri)

Now I can Sign up but got another problem.

@ashish453 that is success! Copy the address. Run this code and paste in the address:
print(input("Paste the address and press enter: ").replace("http://localhost:8080/?code=", ""))

Copy what it outputs, and put it into the first code. I'll walk you through the next part.

Now I do Run this :
import onedrivesdk
from onedrivesdk.helpers import GetAuthCodeServer
from onedrivesdk.helpers.resource_discovery import ResourceDiscoveryRequest

redirect_uri = 'http://localhost:8080'
client_id = "**********"
client_secret = "************"
discovery_uri = 'api.office.com/discovery'
auth_server_url = 'login.microsoftonline.com/common/oauth2/authorize'
auth_token_url = 'http://login.microsoftonline.com/common/oauth2/token?'

http = onedrivesdk.HttpProvider()
auth = onedrivesdk.AuthProvider(http, client_id,
                                auth_server_url=auth_server_url,
                                auth_token_url=auth_token_url)
auth_url = auth.get_auth_url(redirect_uri)
print("Paste this in, and copy/paste the thing after 'code='.")
print(auth_url)
code = input('Paste code here: ')
auth.authenticate(code, redirect_uri, client_secret, resource=discovery_uri)

paste the url int the code:
print(input("Paste the address and press enter: ").replace("http://localhost:8080/?code=", ""))
And I do get a code which i paste in above code.
And got this error:
Paste this in, and copy/paste the thing after 'code='.
login.microsoftonline.com/common/oauth2/authorize?client_id=4dca28ce-c82a-42d6-8690-f892663999d7&response_type=code&redirect_uri=http%3A%2F%2Flocalhost%3A8080
Paste code here: AQABAAIAAAAm-06blBE1TpVMil8KPQ41N-w_JLfgB61-UoHEAs9xPfRHOFdPSvuOHs_NTsesCrgUwRP_wV8NeP6fnmhqxJd2Fj01uQk8pfnDo4MjXd4_TrBtdsDF2lhYbKEqTzg-kk2JF8pIsxETwPmsQ9pGQPiN7ZpjptOrzRs0guLQP9rLAs2yPgLMDAFK-1Dcpb6GDPJeexYnsKfmHVFdJpahrAaoO4k2Jikm1ehhfJupqmZ64zbAIlHE95I5oNrxCbFwnxEfKYyJ1a8KwB2G26rC2PQV0K6n_6Nc_M1CWobRJNW-dukBfFw475gUC8pKCTE0eAR1pbseSuQM5UPi8XHXUWIRT4ccqPKjDCclXvDpPrhpp8ONtDGxa2lvM-_DRa39FD5-CAHIyRZ0j1bis6g77aVzsf-R1OIvxZaQwg_t4Ko_MzVj-MOuPV2thUt-n1EYmukueXPpmbnCxnDMvqXfM6WjYLHWrGW_5KEFCq7INx0HlqsdjJ7V3AqTAY20hL4RfWeIUO92GHIuvqlhg6MZdpBzw85BRdXQ54vjQhWktztlLyAA&session_state=e093b529-c981-4e01-b095-8db161662c73
---------------------------------------------------------------------------
JSONDecodeError                           Traceback (most recent call last)
<ipython-input-12-4303e874c9a5> in <module>()
     18 print(auth_url)
     19 code = input('Paste code here: ')
---> 20 auth.authenticate(code, redirect_uri, client_secret, resource=discovery_uri)

3 frames
/usr/lib/python3.6/json/decoder.py in raw_decode(self, s, idx)
    355             obj, end = self.scan_once(s, idx)
    356         except StopIteration as err:
--> 357             raise JSONDecodeError("Expecting value", s, err.value) from None
    358         return obj, end

JSONDecodeError: Expecting value: line 3 column 1 (char 4)

What can I do now?

@KTibow I'm waiting for your reply.

@ashish453 I don't know what this is. I lost internet, so I couldn't see that you said that. Please edit your code out; it lets anyone have access to your account. You'll also need to delete the previous iteration. As for that, remove &session_state=e093b529-c981-4e01-b095-8db161662c73 so you just have the code:
AQABAAIAAAAm-06blBE1TpVMil8KPQ41N-w_JLfgB61-UoHEAs9xPfRHOFdPSvuOHs_NTsesCrgUwRP_wV8NeP6fnmhqxJd2Fj01uQk8pfnDo4MjXd4_TrBtdsDF2lhYbKEqTzg-kk2JF8pIsxETwPmsQ9pGQPiN7ZpjptOrzRs0guLQP9rLAs2yPgLMDAFK-1Dcpb6GDPJeexYnsKfmHVFdJpahrAaoO4k2Jikm1ehhfJupqmZ64zbAIlHE95I5oNrxCbFwnxEfKYyJ1a8KwB2G26rC2PQV0K6n_6Nc_M1CWobRJNW-dukBfFw475gUC8pKCTE0eAR1pbseSuQM5UPi8XHXUWIRT4ccqPKjDCclXvDpPrhpp8ONtDGxa2lvM-_DRa39FD5-CAHIyRZ0j1bis6g77aVzsf-R1OIvxZaQwg_t4Ko_MzVj-MOuPV2thUt-n1EYmukueXPpmbnCxnDMvqXfM6WjYLHWrGW_5KEFCq7INx0HlqsdjJ7V3AqTAY20hL4RfWeIUO92GHIuvqlhg6MZdpBzw85BRdXQ54vjQhWktztlLyAA

I will delete the working code in 1 day, or as soon as you reply, so please copy and paste this quickly.

@ashish453 Please confirm that you've gotten the code.

@ashish453 I am deleting this today. Please confirm you have the code.

@ashish453 @Ashish45351 in 1 hour, I am removing the code.

@KTibow  This doesn't work. I am still gting the same error again after removing &sesi.............
By the I do find an alternative for my task. I found Mover.io by Microsoft did the same work without any code. And That was extermely fast.
But I still want Onedrive-sdk to do this work.

@ashish453 your code has expired. Please start from the beginning.

I had started from very much beginning. Still same error. It worked for OneDrive personal but it's not working for OneDrive business.

@ashish453 Try using the same code but with onedrivesdk_fork. You'll need to change everywhere it says onedrivesdk in the code to onedrivesdk_fork.

@KTibow  This is giving me same error. please help bro.

### I decided to make a change:
auth_server_url = 'login.microsoftonline.com/common/oauth2/authorize'
auth_token_url = 'http://login.microsoftonline.com/common/oauth2/token?'

In above code i just made the http to https
auth_server_url = 'https://login.microsoftonline.com/common/oauth2/authorize'
auth_token_url = 'https://login.microsoftonline.com/common/oauth2/token?'

### and now i get some different error:
Paste this in, and copy/paste the thing after 'code='.
https://login.microsoftonline.com/common/oauth2/authorize?client_id=8186bbd3-5cdc-41ca-8f4a-707f86b8cebd&response_type=code&redirect_uri=http%3A%2F%2Flocalhost%3A8080
Paste code here: ********Zmx2kDH_KdCAA
---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
<ipython-input-19-1efc20771e9d> in <module>()
     18 print(auth_url)
     19 code = input('Paste code here: ')
---> 20 auth.authenticate(code, redirect_uri, client_secret, resource=discovery_uri)

2 frames
/usr/local/lib/python3.6/dist-packages/onedrivesdk_fork/http_response.py in __init__(self, status, headers, content)
     59                     raise OneDriveError(message["error"], self.status)
     60                 else:
---> 61                     raise Exception(str(message["error"]))
     62
     63     def __str__(self):

Exception: invalid_resource

Possibly related to #113
Sent from Outlook Mobilehttps://aka.ms/blhgte

From: ashish453 notifications@github.com
Sent: Tuesday, May 12, 2020 8:10:34 PM
To: OneDrive/onedrive-sdk-python onedrive-sdk-python@noreply.github.com
Cc: kendell kendell.r@outlook.com; Mention mention@noreply.github.com
Subject: Re: [OneDrive/onedrive-sdk-python] OSError: [Errno 98] Address already in use` (#180)
I decided to make a change:
auth_server_url = 'login.microsoftonline.com/common/oauth2/authorize'
auth_token_url = 'http://login.microsoftonline.com/common/oauth2/token?'
In above code i just made the http to https
auth_server_url = 'https://login.microsoftonline.com/common/oauth2/authorize'
auth_token_url = 'https://login.microsoftonline.com/common/oauth2/token?'
and now i get some different error:
Paste this in, and copy/paste the thing after 'code='.
https://login.microsoftonline.com/common/oauth2/authorize?client_id=8186bbd3-5cdc-41ca-8f4a-707f86b8cebd&response_type=code&redirect_uri=http%3A%2F%2Flocalhost%3A8080
Paste code here: ********Zmx2kDH_KdCAA
Exception                                 Traceback (most recent call last)
 in ()
18 print(auth_url)
19 code = input('Paste code here: ')
---> 20 auth.authenticate(code, redirect_uri, client_secret, resource=discovery_uri)
2 frames
/usr/local/lib/python3.6/dist-packages/onedrivesdk_fork/http_response.py in init(self, status, headers, content)
59                     raise OneDriveError(message["error"], self.status)
60                 else:
---> 61                     raise Exception(str(message["error"]))
62
63     def str(self):
Exception: invalid_resource
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FOneDrive%2Fonedrive-sdk-python%2Fissues%2F180%23issuecomment-627717968&data=02|01||3d34c7942f3e47f430b708d7f6eb3469|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637249362357325514&sdata=lR8WIulSgLkrFNp04xvbuXTmQB8MhfIuB%2BTZfOoh1Iw%3D&reserved=0, or unsubscribehttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FACR3DNWDVZEWHCK55XL5JCDRRIFSVANCNFSM4MZXJ2XQ&data=02|01||3d34c7942f3e47f430b708d7f6eb3469|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637249362357335509&sdata=8xykmFlPxQeYFFHraVBV6%2FEKFQ5wxui4DRlhjXtix7A%3D&reserved=0.

So this error can,t be fixed
:(

Well, I don't know how. Since you're using onedrivesdk_fork though, try mentioning @earonesty to see if he knows any better than me.

repo: OneDrive/onedrive-sdk-python


text:
FilePicker v8 requires multiple clicks to select 'places' when in compact mode

When the filepicker is in "compact mode" (i.e. the the width is small, and rather than displaying the 'places' to the left, and the 'directory listing' to the right, it only displays one at a time.) it can require multiple clicks to choose a new 'place'. see the following video:
https://youtu.be/Uy8hyXxq3ds
it always requires two clicks to access SharePoint places (the first account demoed in the above video), however the situation is worse with the consumer onedrive.live.com personal accounts (the second account demoed) ... you can see that it will not navigate to 'My Files' at all, until you first navigate to 'Photos' and then back again.
i don't believe this behaviour is a problem with my implementation, because when not running in "compact mode" it all works fine.
with thanks

@jonathon-love good find! This is a fairly-recent regression in the Picker's behavior. We will have a fix deployed in the next coupld of weeks.

authors: ThomasMichon jonathon-love

repo: OneDrive/samples


text:
cfx: fill block timestamp

What does this implement/fix? Explain your changes.
conflux的交易记录时间现在全是0，需要补上...
Does this close any currently open issues?
If it fixes a bug or resolves a feature request, be sure to link to that issue.
No
Pull request type

Put an x in the boxes that apply

[x] Bugfix
[ ] Feature
[ ] Code style update (formatting, renaming)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] Documentation content changes
[ ] Other (please describe):

Where has this been tested?
No
Any other comments?
No

这个要合吗？

authors: huazhouwang taimanhui

repo: OneKeyHQ/electrum


text:
🛑 Baidu is down

In 2f6c3f4, Baidu (https://www.baidu.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Baidu is back up in 1ea632f after 9 minutes.


text:
add nn-Graph bert

支持 nn-Graph bert

[x] 跑通 train&eval in nn-Graph version
[x] 对齐 eager mode 和 nn-Graph 误差
[x] 完成 ofrecord dataset
[x] Graph 中加入 dataset Module
[x] 加入 lr scheduler
[x] 完成 README
[x] 完成 inference.py 代码

初步跑了 1000 step 训练，400 step 测试结果如下

在 Graph 中加入了 OfRecord，跑通训练和测试，还未验证精度

在训练过程中按 ctrl-c 退出时会出现 core dump，异常退出，生成 core.* 文件

支持 OfRecord 数据读入，训练 20 epochs 对齐结果如下

初步对齐 graph 和 eager 的结果，不过还未进行正确性验证，因为 eager 的代码是基于非官方的 pytorch 版本进行复现的，未和 lazy 版本以及官方的 tf 版本进行对齐

NLLLoss(ignore_index=0) 在 graph 中不支持，正在尝试解决

和 @CPFLAME 一起解决了 NLLLoss(ignore_index=0) 的问题
发现 eager 实现下 next_sent_loss 计算存在问题，也使用了 ignore_index，而 next_sent_label 输出只有 [0, 1]，所以不应该和 mask_loss 一样使用 ignore_index

这个 PR 暂时废弃了，后续的工作在 https://github.com/Oneflow-Inc/models/pull/117 继续开展

repo: Oneflow-Inc/models


text:
Short solution needed: "Get key-value pairs from table using PHP PDO" (php-pdo)

Please help us write most modern and shortest code solution for this issue:
Get key-value pairs from table using PHP PDO (technology: php-pdo)
Fast way
Just write the code solution in the comments.
Prefered way

Create pull request with a new code file inside inbox folder.
Don't forget to use comments to make solution explained.
Link to this issue in comments of pull request.

Get key-value pairs from table using PHP PDO

repo: Onelinerhub/onelinerhub


text:
[cbridge] Add message when Generator is missing

The Problem To Be Solved
When a generator is not installed, output a message using the download-url that is in global.generator.yml.
generator:
  - id: CubeMX
    description: Global Registered Generator
    download-url: https://www.st.com/en/development-tools/stm32cubemx.html#st-get-software
    run: ../bin/cbridge
    path: $SolutionDir()$/STM32CubeMX/$TargetType$

Suggest A Solution
Output an error, for example with this format:
error cbridge: generator `<id>` missing. Install from `<download-url>`

Example:
error cbridge: generator `CubeMX` missing. Install from `https://www.st.com/en/development-tools/stm32cubemx.html#st-get-software`

implemented and merged into main branch

repo: Open-CMSIS-Pack/generator-bridge


text:
Manage Database Connections Across Entire System

MMS is maintaining 2 database connections per project.  On a system with hundreds of projects, this becomes a problem.  This is an attempt to enforce some kind of limitation on the number of idle connections without sacrificing too much performance.

There will most likely be a couple conflicts once the release branch is merged into develop. This PR will have to wait for the release.

Ok, let's just fix the formatting and it looks good to me.

Requested changes completed.  Thanks!

authors: BA-MBSE HuiJun

repo: Open-MBEE/mms


text:
Fix setup bugs

There is some issues with basic local setup and installation, so I would work off GitHub. I am not sure why this isn't compiling here.
Hi @anibalsolon, are you familiar with this setup? Can you help me? if not, I will dump this and find another template :)

I went through docs: but installing Ruby etc seems to be a pain on my mac.. I can walk you through this structure so you can help debug this stuff if your time permits. I'd understand if you are busy.

hey @raamana I managed to build it locally- could it be a wrong setup on Github Pages?

these are the commands I've used on Ubuntu to setup everything (didnt even had ruby-dev installed)
sudo apt-get install ruby-dev
sudo gem install bundler
sudo bundler
bundle exec jekyll serve

yeah, glad to see it worked on your machine! Yes, there is an issue on github pages or a setup of this repo w.r.t getting it out on the web.. are you familiar with this template, to try identify the issue? thanks,

so there is something at https://open-minds-lab.github.io/openminds/
but the layout is all messed up etc :)

Ah I see now- I was checking this URL https://openmindslab.github.io/
So you just need to set the URL to the correct one at:
https://github.com/Open-Minds-Lab/openminds/blob/master/_config.yml#L14
i.e.
url: "https://open-minds-lab.github.io/openminds/"

I changed it but the layout isn't changed for some reason

@raamana the org name contains hyphens, so it should be open-minds-lab

damn, how did I miss that! that did it! thanks..
its kinda weird to have to hyphens in the URL though.. will try to see how to remove that (and I forgot why the repo name has hyphens in the first place)

thanks for the help mate, it worked.

authors: anibalsolon raamana

repo: Open-Minds-Lab/openminds


text:
Problem while pretraining on custom dataset

Can you please provide the fairseq version you are using?

wasnt working on it for a while.
i was infrencing and came across this error
from fairseq import utils
ModuleNotFoundError: No module named 'fairseq'
this version i am using
https://github.com/Open-Speech-EkStep/fairseq -b v2-hydra

I was not working on it for a while, now that I have resumed I came across another problem.
there is another problem I came across, the flashlight has a separate class for processing text and it is in a seperate branch.
how you install the flashlight library.

authors: harveenchadha mujhenahiata


text:
Error running in Lambda Labs VM using instructions in docs

After upgrading to Python 3.9 and setting everything up, PyTorch becomes unusable, so the training script fails:
ubuntu@209-20-159-38:~/axolotl$ python
Python 3.9.17 (main, Jun  6 2023, 20:11:04)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3/dist-packages/torch/__init__.py", line 443, in <module>
    raise ImportError(textwrap.dedent('''
ImportError: Failed to load PyTorch C extensions:
    It appears that PyTorch has loaded the `torch/_C` folder
    of the PyTorch repository rather than the C extensions which
    are expected in the `torch._C` namespace. This can occur when
    using the `install` workflow. e.g.
        $ python setup.py install && python -c "import torch"

    This error can generally be solved using the `develop` workflow
        $ python setup.py develop && python -c "import torch"  # This should succeed
    or by running Python from a different directory.

I condensed everything into one script, if we can get it working we could add it inside scripts/ for ease of access:
#!/bin/bash

set -e

# Function to gracefully exit if a command fails
abort() {
    echo >&2 '
***************
*** ABORTED ***
***************
'
    echo "An error occurred. Exiting..." >&2
    exit 1
}

trap 'abort' 0

# Update system
sudo apt update

# Install python3.9
sudo apt install -y python3.9
sudo update-alternatives --install /usr/bin/python python /usr/bin/python3.9 1
sudo update-alternatives --config python # user must pick 3.9 if given option

# Verify python version
version=$(python -V 2>&1 | grep -Po '(?<=Python )(.+)')
if [[ -z "$version" ]]
then
    echo "Failed to detect python version"
    exit 1
fi
echo "Python version $version installed."

# Install pip
wget https://bootstrap.pypa.io/get-pip.py
python get-pip.py
rm get-pip.py

# Install torch
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118

# Ensure setuptools installed
pip3 install -U setuptools

# Install Axolotl
pip3 install -e .

# Install Axolotl dependencies
pip3 install protobuf==3.20.3
pip3 install -U requests scipy
pip3 install --ignore-installed psutil
pip3 install git+https://github.com/huggingface/peft.git # not for gptq

# Set path
export LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu:$LD_LIBRARY_PATH

# If we've reached this point, all commands were successful
trap : 0

echo >&2 '
************
*** DONE ***
************
'

cc @NanoCode012, any advice appreciated to workaround in meantime, will PR this script also when it's stable.

cc @NanoCode012, any advice appreciated to workaround in meantime, will PR this script also when it's stable.
This is confusing because I did not have this issue running on a Colab VM, somehow it was able to make Python 3.9+ work with PyTorch.

Hey, is this lambdalabs?
The error sounds like you failed to install torch correctly. Maybe try to uninstall then reinstall. If all fails, you can use docker image.

Yes, this is on a Lambda Labs H100 instance. I tried that, but I can't get torch to uninstall.
ubuntu@209-20-159-38:~/axolotl$ pip uninstall torch
Found existing installation: torch 2.0.1
Uninstalling torch-2.0.1:
  Would remove:
    /usr/bin/torchrun
    /usr/lib/python3/dist-packages/functorch
    /usr/lib/python3/dist-packages/nvfuser
    /usr/lib/python3/dist-packages/torch
    /usr/lib/python3/dist-packages/torch-2.0.1.egg-info
    /usr/lib/python3/dist-packages/torchgen
Proceed (Y/n)? y
ERROR: Exception:
Traceback (most recent call last):
  File "/usr/lib/python3.9/shutil.py", line 825, in move
    os.rename(src, real_dst)
PermissionError: [Errno 13] Permission denied: '/usr/bin/torchrun' -> '/tmp/pip-uninstall-nd6b1zrl/torchrun'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ubuntu/.local/lib/python3.9/site-packages/pip/_internal/cli/base_command.py", line 169, in exc_logging_wrapper
    status = run_func(*args)
  File "/home/ubuntu/.local/lib/python3.9/site-packages/pip/_internal/commands/uninstall.py", line 105, in run
    uninstall_pathset = req.uninstall(
  File "/home/ubuntu/.local/lib/python3.9/site-packages/pip/_internal/req/req_install.py", line 680, in uninstall
    uninstalled_pathset.remove(auto_confirm, verbose)
  File "/home/ubuntu/.local/lib/python3.9/site-packages/pip/_internal/req/req_uninstall.py", line 381, in remove
    moved.stash(path)
  File "/home/ubuntu/.local/lib/python3.9/site-packages/pip/_internal/req/req_uninstall.py", line 272, in stash
    renames(path, new_path)
  File "/home/ubuntu/.local/lib/python3.9/site-packages/pip/_internal/utils/misc.py", line 313, in renames
    shutil.move(old, new)
  File "/usr/lib/python3.9/shutil.py", line 846, in move
    os.unlink(src)
PermissionError: [Errno 13] Permission denied: '/usr/bin/torchrun'
ubuntu@209-20-159-38:~/axolotl$ sudo pip uninstall torch
Found existing installation: torch 2.0.1
Not uninstalling torch at /usr/lib/python3/dist-packages, outside environment /usr
Can't uninstall 'torch'. No files were found to uninstall.
ubuntu@209-20-159-38:~/axolotl$ sudo -H pip uninstall torch
Found existing installation: torch 2.0.1
Not uninstalling torch at /usr/lib/python3/dist-packages, outside environment /usr
Can't uninstall 'torch'. No files were found to uninstall.

Hey, seems like somehow the permissions got stuck? I would recommend just rebooting a new one to start from scratch. I would recommend Miniconda then to prevent this issue. I don't think you need to use sudo at all.
Lastly, I just wanted to give fyi, I also have issues with H100 on lambdalabs particularly with bitsandbytes and xformers in case you're using those!

I see what's happening - pytorch is being provided by apt packages:
ubuntu@209-20-159-38:~/axolotl$ dpkg -l | grep torch
ii  python3-torch-cuda                     2.0.1+ds-0lambda0.20.04.1                 amd64        Tensors and Dynamic neural networks GPU accelerated (Python 3)
ii  python3-torchvision-cuda               0.15.1-0lambda0.20.04.1                   amd64        Image and video datasets and models for PyTorch (Python 3, CUDA)

We should be good to wipe these out and handle it via PyTorch install instructions, right?
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118

Which is already in the script. Let's see.

You can pass a -U to upgrade or a --ignore-installed so it force installs. I was planning to add that flag but didn't get to.

We got it 😎

Will finalize this tomorrow:
https://gist.github.com/ctjlewis/7540d88f4ddb93d36e7515fb1b911833

This is my first few minutes of training loss with xformers disabled (we need PyTorch nightly for H100, can't use xformers with PyTorch nightly, have to disable xformers):

Which seems like an issue but I'll give it an hour or so.

Use bf16 @ctjlewis
Regarding pytorch, I don't think you need nightly. Though, if there's any new improvement in nightly, please do tell.

I had to use nightly to get rid of errors about H100 support (size sm_90).
With bf16 I ran into that "function" error. I tried all config permutations per that issue about it:

bf16: true in config AND --mixed-precision bfloat16 in accelerate launch command
bf16: true in config ONLY
--mixed-precision bfloat16 in accelerate launch command ONLY

But couldn't get rid of that error.

I had to use nightly to get rid of errors about H100 support (size sm_90).

I use an older version of transformers 4.29.2. However, you'll need to modify source code to comment out 4bit.

I had to use nightly to get rid of errors about H100 support (size sm_90).

I use an older version of transformers 4.29.2. However, you'll need to modify source code to comment out 4bit.

Could the Axolotl library just provide that instead?
I got the job started with nightly and H100 support, no xformers but if I could fix the bf16 error. How do I properly set that?

Yes, I think the 4bit part can be improved.
Unfortunately, mainly only H100 or so may be facing this issue, so there weren't many eyes.
I'm not sure why xformers or more specifically triton does not work as well on H100.
You can set bf16: true within config.

When the only config changes I make are:

Set xformers_attention: false
Set bf16: true

I get that function error:
ubuntu@209-20-159-22:~/axolotl$ accelerate launch scripts/finetune.py examples/openllama-3b/config.yml
2023-06-26 18:24:35.086204: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2023-06-26 18:24:35.146711: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI AVX512_BF16 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-06-26 18:24:35.921006: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
The following values were not passed to `accelerate launch` and had defaults used instead:
	`--num_processes` was set to a value of `1`
	`--num_machines` was set to a value of `1`
	`--mixed_precision` was set to a value of `'no'`
	`--dynamo_backend` was set to a value of `'no'`
To avoid this warning pass in values for each of the problematic parameters or run `accelerate config`.
2023-06-26 18:24:40.826320: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT

===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please run

python -m bitsandbytes

 and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
================================================================================
bin /home/ubuntu/.local/lib/python3.9/site-packages/bitsandbytes/libbitsandbytes_cuda118.so
CUDA SETUP: CUDA runtime path found: /usr/lib/x86_64-linux-gnu/libcudart.so
CUDA SETUP: Highest compute capability among GPUs detected: 9.0
CUDA SETUP: Detected CUDA version 118
CUDA SETUP: Loading binary /home/ubuntu/.local/lib/python3.9/site-packages/bitsandbytes/libbitsandbytes_cuda118.so...
INFO:root:loading tokenizer... openlm-research/open_llama_3b
Using pad_token, but it is not set yet.
INFO:root:Loading prepared dataset from disk at last_run_prepared/c6b6388039831944360f60b07eaffe22...
INFO:root:Prepared dataset loaded from disk...
INFO:root:loading model and peft_config...
WARNING:accelerate.utils.modeling:The model weights are not tied. Please use the `tie_weights` method before using the `infer_auto_device` function.
Using the `WANDB_DISABLED` environment variable is deprecated and will be removed in v5. Use the --report_to flag to control the integrations used for logging result (for instance --report_to none).
INFO:root:Compiling torch model
INFO:root:Starting trainer...
Traceback (most recent call last):
  File "/home/ubuntu/axolotl/scripts/finetune.py", line 352, in <module>
    fire.Fire(train)
  File "/home/ubuntu/.local/lib/python3.9/site-packages/fire/core.py", line 141, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
  File "/home/ubuntu/.local/lib/python3.9/site-packages/fire/core.py", line 475, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
  File "/home/ubuntu/.local/lib/python3.9/site-packages/fire/core.py", line 691, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
  File "/home/ubuntu/axolotl/scripts/finetune.py", line 337, in train
    trainer.train(resume_from_checkpoint=resume_from_checkpoint)
  File "/home/ubuntu/.local/lib/python3.9/site-packages/transformers/trainer.py", line 1645, in train
    return inner_training_loop(
  File "/home/ubuntu/.local/lib/python3.9/site-packages/transformers/trainer.py", line 1756, in _inner_training_loop
    model, self.optimizer = self.accelerator.prepare(self.model, self.optimizer)
  File "/home/ubuntu/.local/lib/python3.9/site-packages/accelerate/accelerator.py", line 1182, in prepare
    result = tuple(
  File "/home/ubuntu/.local/lib/python3.9/site-packages/accelerate/accelerator.py", line 1183, in <genexpr>
    self._prepare_one(obj, first_pass=True, device_placement=d) for obj, d in zip(args, device_placement)
  File "/home/ubuntu/.local/lib/python3.9/site-packages/accelerate/accelerator.py", line 1022, in _prepare_one
    return self.prepare_model(obj, device_placement=device_placement)
  File "/home/ubuntu/.local/lib/python3.9/site-packages/accelerate/accelerator.py", line 1311, in prepare_model
    torch.autocast(device_type=self.device.type, dtype=torch.bfloat16)(model.forward.__func__), model
AttributeError: 'function' object has no attribute '__func__'
Traceback (most recent call last):
  File "/home/ubuntu/.local/bin/accelerate", line 8, in <module>
    sys.exit(main())
  File "/home/ubuntu/.local/lib/python3.9/site-packages/accelerate/commands/accelerate_cli.py", line 45, in main
    args.func(args)
  File "/home/ubuntu/.local/lib/python3.9/site-packages/accelerate/commands/launch.py", line 941, in launch_command
    simple_launcher(args)
  File "/home/ubuntu/.local/lib/python3.9/site-packages/accelerate/commands/launch.py", line 603, in simple_launcher
    raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
subprocess.CalledProcessError: Command '['/usr/bin/python', 'scripts/finetune.py', 'examples/openllama-3b/config.yml']' returned non-zero exit status 1.

@NanoCode012 I moved off Lambda Labs, now on 8x A100 80GB.
xformers off, bf16: true, but the model still does not learn. I wonder if the hyperparams are messed up?

That's weird. That could be. You're also training for quite a lot of epochs.

@NanoCode012 @winglian Would it be possible for us to check that the vanilla Falcon/OpenLLaMA configs are working? I can't identify what's going wrong.

Just a note, your dataset might also be a factor in this. If you can run the configs successfully, then it's usually the hyperparameters or the dataset.

Any news on this? I tried getting Axolotl to run on Llamalabs for hours now without success...
Using the provided script with some fixed I got additional errors with Tensorflow and psutil and also to force reinstall both to get anything running.
Now basically everything fails with diverse errors
e.g. with the examples:
accelerate launch scripts/finetune.py examples/openllama-3b/qlora.yml
ERROR:root:Exception raised attempting to load model, retrying with AutoModelForCausalLM
ERROR:root:/home/ubuntu/.local/lib/python3.9/site-packages/bitsandbytes/libbitsandbytes_cpu.so: undefined symbol: cquantize_blockwise_fp16_nf4
Traceback (most recent call last):

or
accelerate launch scripts/finetune.py examples/openllama-3b/lora.yml
results in
ERROR:root:Exception raised attempting to load model, retrying with AutoModelForCausalLM
ERROR:root:/home/ubuntu/.local/lib/python3.9/site-packages/bitsandbytes/libbitsandbytes_cpu.so: undefined symbol: cget_col_row_stats

I´m not very experienced - are these axolotl issues or issues of the downstream libraries?
Thanks!

Hey @jpdus , I think you can raise a separate issue as this one seems to be solved. The author's last few comments were just on performance.
@ctjlewis , feel free to close this one and discuss in discord or in separate issue if your training is not improving.

Closing this as it seems to be solved. Please re-open if problem comes back.

authors: NanoCode012 ctjlewis jpdus

repo: OpenAccess-AI-Collective/axolotl


text:
ShareGPT datasets very strict -- data rows with non-standard role throw AssertionError

I found that one of the datasets I was consuming from HuggingFace with multi-language entries occasionally had roles other than "user" and "gpt". For example, "bing" or "chatgpt". This results in the entire dataset being rejected with an Assertion error and no context.
The offending code is:
  File "src/axolotl/prompters.py", line 319, in build_prompt
    assert role == conv.roles[j % 2]
AssertionError

Ideally additional context would be given to allow for debugging, such as the offending line or perhaps the rest of the datafile can be processed and a warning thrown.
People running into this issue can either remove or edit the lines with abnormal role names or add those roles to line 302 of prompters.py:
        roles = {"human": conv.roles[0], "gpt": conv.roles[1]}
Note that the particular file I was using then had other issues (e.g. system messages as "gpt") that resulted in IndexError as well
In general would be good for this parsing to provide more context when issues are encountered, but opening this issue in part so that others can know that these issues are likely specific types of data issues.

Thanks for this clarification! Would you be interested in opening a PR to improve the assertion message?