text:
ExecuteSqlCommand keeps using IsolationLevel of previous transaction

Steps to reproduce
Start the following program (tested with .NET FW 4.7.2 & .NET Core 2.2):
using Microsoft.EntityFrameworkCore;
using System;
using System.Threading;

namespace IsolationLevelTest
{
    class Program
    {
        static void Main(string[] args)
        {
            var opts = new DbContextOptionsBuilder<MyContext>().UseSqlServer("Data Source=.;Initial Catalog=Test;Integrated Security=True", o => o.CommandTimeout(900)).Options;

            using (var context = new MyContext(opts))
            using (var transaction = context.Database.BeginTransaction(System.Data.IsolationLevel.Serializable))
            {
                context.Database.ExecuteSqlCommand("SELECT 1");
                transaction.Commit();
            }
            var cts = new CancellationTokenSource();
            for (int i = 0; i < 10; i++)
            {
                var t = new Thread(new ThreadStart(() =>
                {
                    while (!cts.IsCancellationRequested)
                        using (var context = new MyContext(opts))
                        {
                            context.Database.ExecuteSqlCommand("WAITFOR DELAY '00:00:01'");
                        }
                }));
                t.Start();
            }
            Console.WriteLine("Started!");
            Console.ReadLine();
            cts.Cancel();
        }
    }

    class MyContext : DbContext
    {
        public MyContext(DbContextOptions options) : base(options)
        {
        }
    }
}

While the program is running, query the isolation levels of the active sessions in SQL:
select s.transaction_isolation_level, t.*, *
from sys.dm_exec_sessions s
join sys.dm_exec_requests r on s.session_id = r.session_id
cross apply sys.dm_exec_sql_text(r.sql_handle) t
where host_name is not null
and s.session_id != @@SPID
and program_name like '%.Net SqlClient Data Provider'

Expected: all the sessions executing the WAITFOR DELAY command should use isolation level 2 (ReadCommitted).
Actual: one of the sessions uses isolation level 4 (Serializable). The other sessions use level 2 (ReadCommitted).

transaction_isolation_level
dbid
objectid
number
encrypted
text
session_id
login_time
program_name
host_process_id
client_version
client_interface_name

4
8
NULL
NULL
0
WAITFOR DELAY '00:00:01'
52
2019-08-02 01:41:44.937
.Net SqlClient Data Provider
7708
7
.Net SqlClient Data Provider

2
8
NULL
NULL
0
WAITFOR DELAY '00:00:01'
54
2019-08-02 01:41:44.997
.Net SqlClient Data Provider
7708
7
.Net SqlClient Data Provider

2
8
NULL
NULL
0
WAITFOR DELAY '00:00:01'
56
2019-08-02 01:41:45.060
.Net SqlClient Data Provider
7708
7
.Net SqlClient Data Provider

2
8
NULL
NULL
0
WAITFOR DELAY '00:00:01'
57
2019-08-02 01:41:45.110
.Net SqlClient Data Provider
7708
7
.Net SqlClient Data Provider

2
8
NULL
NULL
0
WAITFOR DELAY '00:00:01'
58
2019-08-02 01:41:45.137
.Net SqlClient Data Provider
7708
7
.Net SqlClient Data Provider

2
8
NULL
NULL
0
WAITFOR DELAY '00:00:01'
59
2019-08-02 01:41:45.160
.Net SqlClient Data Provider
7708
7
.Net SqlClient Data Provider

2
8
NULL
NULL
0
WAITFOR DELAY '00:00:01'
60
2019-08-02 01:41:45.170
.Net SqlClient Data Provider
7708
7
.Net SqlClient Data Provider

2
8
NULL
NULL
0
WAITFOR DELAY '00:00:01'
61
2019-08-02 01:41:45.180
.Net SqlClient Data Provider
7708
7
.Net SqlClient Data Provider

2
8
NULL
NULL
0
WAITFOR DELAY '00:00:01'
62
2019-08-02 01:41:45.187
.Net SqlClient Data Provider
7708
7
.Net SqlClient Data Provider

2
8
NULL
NULL
0
WAITFOR DELAY '00:00:01'
63
2019-08-02 01:41:45.210
.Net SqlClient Data Provider
7708
7
.Net SqlClient Data Provider

Workaround: surround ExecuteSqlCommand with a transaction created with the correct isolation level:
using (var transaction = context.Database.BeginTransaction(System.Data.IsolationLevel.ReadCommitted))
{
    context.Database.ExecuteSqlCommand("WAITFOR DELAY '00:00:01'");
    transaction.Commit();
}

Further technical details
EF Core version: 2.2.6
Database Provider: Microsoft.EntityFrameworkCore.SqlServer
Operating system: Windows 10.0.18362.239
IDE: Visual Studio 2019 16.2.0

@roji Do you want to take a look?

Yep, will do.

This is an issue with SqlClient and not related to Entity Framework in any way, see https://github.com/dotnet/SqlClient/issues/96.
tl;dr when SqlConnections are returned to the pool, their isolation level isn't reset. The next time the connection is handed out, it has whatever isolation level it last had in use. Until this issue is fixed, the workaround is as you've done - never assume that a newly-opened connection has any particular isolation level, and always set it explicitly.

authors: AndriySvyryd Kloizdena roji


text:
Enable-migrations error

When executing migrations commands such as Enable-Migrations, Add-Migration, etc. the following error is encountered.

Exception calling "LoadFrom" with "1" argument(s): "The specified path, file name, or both are too long. The fully qualified file name must be les
s than 260 characters, and the directory name must be less than 248 characters."

The issue is due to a NuGet bug which is tracked by https://github.com/NuGet/Home/issues/528.
:star2: Workarounds :star2:

:star: Install an updated NuGet client that resolves this issue https://nuget.codeplex.com/releases/view/615507
As a workaround, you can download this patched version of EntityFramework.psm1, copy it into the packages\EntityFramework.6.1.3\tools directory, restart Visual Studio, and try again.
An alternate workaround is to unload other projects that reference Entity Framework (so that the only loaded project that references Entity Framework is the one that contains your EF model).

Details
Full details of the error are...
Exception calling "LoadFrom" with "1" argument(s): "The specified path, file name, or both are too long. The fully qualified file name must be les
s than 260 characters, and the directory name must be less than 248 characters."
At D:\helloworld\packages\EntityFramework.6.1.3\tools\EntityFramework.psm1:780 char:5
+     $utilityAssembly = [System.Reflection.Assembly]::LoadFrom((Join-Path $ToolsP ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : PathTooLongException

You cannot call a method on a null-valued expression.
At D:\helloworld\packages\EntityFramework.6.1.3\tools\EntityFramework.psm1:781 char:5
+     $dispatcher = $utilityAssembly.CreateInstance(
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [], RuntimeException
    + FullyQualifiedErrorId : InvokeMethodOnNull

Exception calling "CreateInstanceFrom" with "8" argument(s): "The specified path, file name, or both are too long. The fully qualified file name m
ust be less than 260 characters, and the directory name must be less than 248 characters."
At D:\helloworld\packages\EntityFramework.6.1.3\tools\EntityFramework.psm1:809 char:5
+     $domain.CreateInstanceFrom(
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : PathTooLongException

No path is longer than 200 on my harddisk, so I can't understand why this problem will happen.

This also happens when I tried to upgrade from 6.1.1 -> 6.1.3. I even moved my git repo to c: to gain few extra chars but to no avail.

This seems to happen only in VS 2015 (RC) which makes me believe its more of a new NuGet issue. VS 2013 (Update 4) doesn't have this. Installation went fine. And Enable-Migrations also seem to be working fine.

if create new project, all are fine, with my which worked in VS2013, in VS2015RC same error.

Same issue here.

Everyone, thanks for reporting this.
It seems we'll need more details to be able to repro and forward the information to the NuGet folks, E.g. was yours a clean setup of Visual Studio 2015 RC or an upgrade form a previous preview?
cc @yishaigalatzer

I had uninstalled CTP 6 to install the 2015 RC.
Also tried to no avail:

Repairing 2015 RC install
Deleting packages dir
Uninstalling nuget extension and reinstalling from https://visualstudiogallery.msdn.microsoft.com/5d345edc-2e2d-4a9c-b73b-d53956dc458d
Deleting nuget dirs from AppData  (http://stackoverflow.com/questions/27377094/cannot-be-installed-nuget-package-in-vs2015)
Downgrading to earlier entity framework versions (http://stackoverflow.com/questions/30020239/ef6-migrations-stopped-working-on-vs-2015-rc)

Works from VS 2013.

@divega
Having same issue, in multiple environments.

Windows 10 Preview, Build 10074, where VS 2015 was initially the CTP and upgraded all the way to the RC.
Windows 8.1, where VS 2015 RC was installed clean, with VS 2013 also present on the machine.

Solution was targeting .net 4.5.1, and I also changed it to 4.6, did not resolve the issue.
The project was using Entity Framework 6.1.3, but I have also tried downgrading to 6.1.2, 6.1.1 and 6.1.0 (to at least add a migration to the project) but I continue to get the same error.
Error from running Add-Migration:

New-Object : The value supplied is not valid, or the property is read-only. Change the value, and then try again.
At C:\Users\bgreen\Source\Paygle\packages\EntityFramework.6.1.3\tools\EntityFramework.psm1:757 char:13

$info = New-Object System.AppDomainSetup -Property @{

        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

CategoryInfo          : InvalidData: (:) [New-Object], Exception
FullyQualifiedErrorId : SetValueException,Microsoft.PowerShell.Commands.NewObjectCommand

The property 'PrivateBinPathProbe' cannot be found on this object. Verify that the property exists and can be set.
At C:\Users\bgreen\Source\Paygle\packages\EntityFramework.6.1.3\tools\EntityFramework.psm1:772 char:9

    $info.PrivateBinPathProbe += ';lib\net45'

    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

CategoryInfo          : InvalidOperation: (:) [], RuntimeException
FullyQualifiedErrorId : PropertyNotFound

Exception calling "LoadFrom" with "1" argument(s): "The specified path, file name, or both are too long. The fully qualified file name must be less than 260 characters, and the directory name must be less than
248 characters."
At C:\Users\bgreen\Source\Paygle\packages\EntityFramework.6.1.3\tools\EntityFramework.psm1:780 char:5

$utilityAssembly = [System.Reflection.Assembly]::LoadFrom((Join-Path $ToolsP ...

  + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
  + FullyQualifiedErrorId : PathTooLongException

You cannot call a method on a null-valued expression.
At C:\Users\bgreen\Source\Paygle\packages\EntityFramework.6.1.3\tools\EntityFramework.psm1:781 char:5

$dispatcher = $utilityAssembly.CreateInstance(

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

CategoryInfo          : InvalidOperation: (:) [], RuntimeException
FullyQualifiedErrorId : InvokeMethodOnNull

Exception calling "CreateInstanceFrom" with "8" argument(s): "The specified path, file name, or both are too long. The fully qualified file name must be less than 260 characters, and the directory name must be
less than 248 characters."
At C:\Users\bgreen\Source\Paygle\packages\EntityFramework.6.1.3\tools\EntityFramework.psm1:809 char:5

$domain.CreateInstanceFrom(

~~~~~~~~~~~~~~~~~~~~~~~~~~~

CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
FullyQualifiedErrorId : PathTooLongException

Same issue here.

What are the project types you are using? Can someone share a repro project?

https://github.com/hcavazin/vs2015err
Very simple. Just run "Enable-Migrations" to see errors.
I noticed that the error message is not always the same.

Same issue here when running Add-Migration. Simple class library which was on EF6.1.2, upgraded to 6.1.3 and received lots of these errors.
I upgraded to VS2015 RC from CTP6. Windows 7 Enterprise SP1 (x64) machine.
Should be noted that I tried uninstalling & re-installing EF6.1.3 and I get these errors during the installation process too:

Exception calling "CreateInstanceFrom" with "8" argument(s): "The specified path, file name, or > both are too long. The
fully qualified file name must be less than 260 characters, and the directory name must be less > than 248 characters."
At X:\MyProjectName\packages\EntityFramework.6.1.3\tools\EntityFramework.psm1:809 > char:5

$domain.CreateInstanceFrom(

~~~~~~~~~~~~~~~~~~~~~~~~~~~

CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
FullyQualifiedErrorId : PathTooLongException

Same issue here

Thanks everyone. @bricelam and @emgarten investigated and found the root cause. More details in https://github.com/NuGet/Home/issues/528.

@divega can u make an example how to fix it?

Ultimately the fix will have to go in NuGet bits.
Since the scenario that is broken is when the solution contains multiple projects with references to the same EF package, I suspect if you temporarily unload projects until only one loaded project in the solution references the EF package, that might help as a workaround. I haven't had a chance to verify this myself yet though.

for a now i fixed that error in this way: i go to Manager Nuget, delete from all projects inside my solution EF, and where is a main project with Code First i Install nuget 6.1.3, inside all others i add a Reference: System.Data.Entity, and pick with "quick fix" reference: "EntityFramework", and now all are worked, i can add an migration and all other.

@divega I confirm, unloading other projects using EF makes (at least) migrations commands work again.

As a workaround, you can download
this patched version of EntityFramework.psm1, copy it into the packages\EntityFramework.6.1.3\tools directory, restart Visual Studio, and try again.

If comment line  <package id =" EntityFramework "version =" 6.1.3 "targetFramework =" net46 "userInstalled =" true "/> in "packages.config" of other projects already solved.

You ALL are TERRIFIC!!  - Thank you very much for your effort
Yes @bricelam 's workaround works like a charm.
Thanks.

@bricelam  :heart:

@bricelam worked a treat.  Really appreciate it!

bricelam's script is not signed, don't forget to unblock it and probably even do
(admin cmd)> powershell Set-ExecutionPolicy Unrestricted
while its not completely fixed, to allow execution in Package manager console

:thumbsup: for @bricelam, your workaround works like a charm, thank you!

NuGet/Home#528 has been fixed (thanks @yishaigalatzer and @AndriySvyryd) so the workaround should become unnecessary the next time NuGet clients is refreshed.

:+1: For such quick turnaround!

When is gonna be refreshed the NuGet client? I have this problme today

Going back to Visual Studio Community 2013...

@sebrojas14 or....you could use the workaround that was posted by @bricelam.

We just dropped a new nuget client on codeplex https://nuget.codeplex.com/releases/view/615507 give it at try

@yishaigalatzer, great! Using NuGet 3.0 RC2 heals the issue. Actually this problem held me away from moving from VS 2013 to VS 2015 RC.

@Casimodo72 thanks for confirming!
I've added the updated client to the list of workarounds

Nice to know, but I already dongrade my company PCs to VS 2013. Maybe I will wait until stable release came, vecause uninstall, download and install its a long process

@sebrojas14 for sure waiting for the stable release of VS 2015 is reasonable, but if you want to continue trying the pre-release version, it should actually work side-by-side with VS 2013 on the same PC.

And of course if you find more bugs there is still some time to fix them

@bricelam solution worked in VS 2015 RC - Need to Unblock the file if it's replaced.

I am getting this error on VS2015 RC and the workarounds have not worked. I have found that the error occurs when you have multiple projects referencing EF. So I have a DataContext project with EF 6.1.3, then I have a client project(console, unit test, web, etc.) that references the DataContext project and EF 6.1.3. If I remove the client project from the solution the error goes away. Someone correct me if I am wrong, but I don't think this is a viable workaround. When you have a project that references your DataContext project it also needs the EF reference. Also when I try the workaround to use the patched psm file I get this "Import-Module : File ..\EntityFramework.6.1.3\tools\EntityFramework.psm1 cannot be loaded. The file
...\EntityFramework.6.1.3\tools\EntityFramework.psm1 is not digitally signed. You cannot run this script on the current system. For
more information about running scripts and setting execution policy, see about_Execution_Policies "

Having the same issue with Visual Studio 2013 Update 5. Any fix avl. for that?

Similar issue: https://github.com/NuGet/NuGetGallery/issues/2592

Still having the same issue. VS2015 on a new project of ASP.NET 4 MVC 5.2, trying to call Enable-Migrations. This is unbelievable. It's like the ASP.NET dropped the support for the released version focusing on the unreleased beta.
I tried the workaround of the first post - none worked. I have a solution that contains a single project.

@weitzhandler - which workaround?  The new nuget, or the patches psm1?

Both. I have the latest NuGet (3.2), and tried replacing the psm1 file. Neither worked.

Post your error message.  It's possible that there may be a different issue?
On 25/09/2015 11:17:53 AM, Shimmy notifications@github.com wrote:
Still having the same issue. VS2015 on a new project of ASP.NET 4 MVC 5.2, trying to call Enable-Migrations. This is unbelievable. It's like the ASP.NET dropped the support for the released version focusing on the unreleased beta.
I tried the workaround of the first post [https://github.com/aspnet/EntityFramework/issues/1950#issue-65169768] - none worked. I have a solution that contains a single project.
—
Reply to this email directly or view it on GitHub [https://github.com/aspnet/EntityFramework/issues/1950#issuecomment-143096434].

@brendangreen
This was all brought above but to make you happy here you go:
enable-migration : The term 'enable-migration' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path
was included, verify that the path is correct and try again.
At line:1 char:1
+ enable-migration
+ ~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (enable-migration:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

BTW, all the related EF commands (i.e. get-help entityframework) don't work either.

When I was using the work around, I had to Import-Module the psm1 file.  You did that?

@brendangreen
Well after explicitly uninstalled EntityFramework (Uninstall-Package EntityFramework) and reinstalled it it works now.

Anyone else still getting this problem?  I have created a new ASP.NET 5 solution with 5 different projects, one is using EntityFramework 6.1.3 and is being referenced by the other projects.  If i try to do enable-migrations i just get this error
enable-migrations : The term 'enable-migrations' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:1
+ enable-migrations
+ ~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (enable-migrations:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

I have tried uninstalling and reinstalling EntityFramework, but still doesnt work?

@Gillardo If you are using ASP.NET 5 then the PowerShell commands do not work (due to limitation in the ASP.NET 5 project system). You need to drop out to the command line, see here for more details http://ef.readthedocs.org/en/latest/getting-started/aspnet5.html#create-your-database.

This is referencing EF7, not EF6.  I managed to get round this, by creating a .csproj for my EF6 project, and then added all the files from my .xproj, by using the "Add By Link" option.  This way i havent got 2 copies of the files.  Now using my .csproj i can run all my commands for EF6, and when i build my .xproj i have the latest versions etc.
But support for EF6 natively in .xproj would be great

Ah, sorry I missed that. The same limitation holds true (as you discovered). Sounds like you have the right work around for the moment.

For deletion or handling of long name i check that "Long Path Tool" software should be try, many articles refer this for such solutions.

in package manager console
Import-Module (your project path)\packages\EntityFramework.6.1.3\tools\EntityFramework.psm1

source: gharchive/issue

authors: 613038475 Arser Casimodo72 Gillardo NooNameR amirhaghajani brendangreen bricelam divega gamicklisafisher hcavazin ilmax imarrero jkmuka maartenba mafshin mrchief rgripper rowanmiller ryan-codingintrigue sebrojas14 unt1tled weitzhandler yishaigalatzer

repo: aspnet/EntityFrameworkCore

license: MIT

license_type: permissive

license_source: github-api


text:
DependencyInjection Extentions crashes on Xamarin Android.

Describe the bug
Using AddSingleton to setup IoC container, crashes on Android.  Using TryAddSingleton works.
However AddSignleton works as expected on the desktop.  The call to serviceCollection.BuildServiceProvider triggers the crash.
To Reproduce
This is occurring in an application with a large number of interfaces, too many for me to reproduce.  I don't have the time to set up a small project with a few interfaces to see if I can repro the problem.  The best way to repro this issue is to create a Xamarin app and register interfaces using AddSingleton and another one using TryAddSingleton to see the difference.  Also, you must register your interfaces before resolving.  Trying to register after resolution cause the application to crash.
Expected behavior
Expected the DI to work on Android device exactly as it would on the desktop.
Additional context
Below is the stack track received from the debug window of the Xamarin Android app.
bug.txt

This is occurring in an application with a large number of interfaces, too many for me to reproduce. I don't have the time to set up a small project with a few interfaces to see if I can repro the problem. The best way to repro this issue is to create a Xamarin app and register interfaces using AddSingleton and another one using TryAddSingleton to see the difference. Also, you must register your interfaces before resolving. Trying to register after resolution cause the application to crash.

How many interfaces and how deep is the graph?

There are 48 Interfaces.  Well actually 30 interfaces and 18 class types.  I've did my best to remove any cyclical definitions.  I don't think I'm encountering a cycle.  In fact I know I'm not because when I did hit a cycle I got a stack overflow.  Therefore I don't think that the graph is very deep.  Like I mentioned, the application works when using TryAddXXX but as soon as I use AddXXX it crashes however I can use AddXXX and even replace interfaces on the desktop but not when running on the device.
Cheers and thanks for the quick response.

authors: cwaldron davidfowl

repo: aspnet/Extensions


text:
[Question] Get Options immediately after Configure

From @billpratt on Tuesday, August 22, 2017 9:41:23 AM
I have a need for configure options to be accessed via DI as well as needing them right away to do some additional configuration. Below is what I have so far. Is this right? Is there a better way? Am I way off?
var section = configuration.GetSection("CacheSettings");
services.Configure<CacheSettings>(section);

var cacheSettings = section.Get<CacheSettings>();

if(cacheSettings.Use.....)

Is there a way to do this instead?
var section = configuration.GetSection("CacheSettings");
var cacheSettings = section.Get<CacheSettings>();

services.Configure<CacheSettings>(cacheSettings);

Copied from original issue: aspnet/Options#226

From @HaoK on Tuesday, September 5, 2017 1:02:41 PM
Can you clarify what you are trying to do exactly?
It looks like you are trying to store some settings in configuration, and binding them to your options, which should work fine.  You also want to tweak the options after reading the initial settings from config?
That should work fine as well, what isn't working for you?

From @billpratt on Tuesday, September 5, 2017 2:11:40 PM
@HaoK Everything is working fine. I was wondering if there was a better way. Since I'm creating the cache settings to use locally, it would be nice to have an overload that takes in an instance of the type I'm configuring in DI.
Something like...
var section = configuration.GetSection("CacheSettings");
var cacheSettings = section.Get<CacheSettings>();

services.Configure<CacheSettings>(cacheSettings);

From @HaoK on Tuesday, September 5, 2017 2:56:10 PM
If you really want to avoid that hit, you can always do some kind of nesting:
class CacheOptions {
   public CacheSettings Settings { get; set; }
}

Then services.Configure<CacheOptions>(o => o.Settings = cacheSettings)

From @HaoK on Tuesday, September 5, 2017 2:58:34 PM
Or you can always just hammer in the options instance as a singleton wrapped in IOptions: services.AddSingleton<IOptions<CacheSettings>>(Options.Create(cacheSettings))

repo: aspnet/Home


text:
No addresses in IServerAddressesFeature

Could this be related to https://github.com/aspnet/Announcements/issues/224?
I'm binding to port 0 as described in the docs and trying to use the IServerAddressesFeature to find out what port was actually used - but the addresses array is empty.
VS: 15.6.6
dotnet --version: 2.1.104
Microsoft.AspNetCore.Server.Kestrel: 2.1.0-preview2-final

When are you checking? IServerAddressesFeature won't be populated until after Startup.

Ah, yeh... checking in Configure in Startup.cs

You'll have to delay that. Here's a middleware that checks it on first request:
https://github.com/aspnet/BasicMiddleware/blob/354cb18d6304b24063d460e0a41c6c1d51ea4000/src/Microsoft.AspNetCore.HttpsPolicy/HttpsRedirectionMiddleware.cs#L139

IHostedService's are started after the server, but IServerAddressesFeature isn't a service in DI so I'm not sure how you'd acquire it.

You're going to have to do some experimenting here. IServerAddressesFeature can't be registered as a service because it's not available until after services are built. You'd have to register some proxy service with a IServerAddressesFeature property that you could assign later.

We had a similar problem. Now we just generate the port ourselves from the emphemeral pool and assign it to asp.net as we can't get it from aspnet when we want it.
First request is too late, as no one can find your service if you don't advertise your address.

Yeh exactly... generating our own port is not ideal as you can't guarantee the port is available, but better than nothing.

The other place we extract these addresses is in Program.Main after StartAsync. https://github.com/aspnet/Hosting/blob/a63932a492513cdeb4935661145084cad2ae5521/src/Microsoft.AspNetCore.Hosting/WebHostExtensions.cs#L104
Is that something you can do?

Ok it looks like creating a proxy object with a property to hold the IServerAddressesFeature and injecting it into an IHostedService works.

... for now :)

If you want to pre allocate 60% of the time this works every time.
https://github.com/Drawaes/CondenserDotNet/blob/master/src/CondenserDotNet.Client/ServiceManagerConfig.cs#L22

@Drawaes we were doing that for our functional tests but it was very flaky on OSX, it would often return duplicate ports.

Cool that the other method works might have to take a look and put it in my lib as well.

Also with the finding the port yourself, it could in theory be taken by something else in the small space between finding it and binding to it.

Don't use osx so don't care ;) and certainly don't use osx in prod.
Unlikely that another app will get it if you look at how the ports are allocated it won't reallocate you port unless you used the entire rest of the emphemeral ports in the time between closing and opening. (On windows) and an entire bucket on Linux which is theorically possible but I doubt it will happen ever (and certainly not enough to be a problem).

That said I like the proxy object idea and will investigate it for my consul lib.

We periodically close 'discussion' issues that have not been updated in a long period of time.
We apologize if this causes any inconvenience. We ask that if you are still encountering an issue, please log a new issue with updated information and we will investigate.

authors: Drawaes Tratcher aspnet-hello tomstreet

repo: aspnet/Hosting


text:
ASPNETCORE_ENVIRONMENT seems to be ignored using Generic Host

Using the ASP.NET Background Tasks Sample (Generic Host)
https://github.com/aspnet/Docs/tree/master/aspnetcore/fundamentals/host/hosted-services/samples/2.x/BackgroundTasksSample-GenericHost
I am unable to set the Environment to "Development", it remains set to "Production" despite the ASPNETCORE_ENVIRONMENT env variable being set to Development.
I updated the vscode launch.json as instructed in the readme, i also added the ASPNETCORE_ENVIRONMENT env var to it as follows:
"console": "integratedTerminal",
            "env": {
                "ASPNETCORE_ENVIRONMENT": "Development"
            },

When running in debug in vscode on Mac and also when using ASPNETCORE_ENVIRONMENT=Development dotnet run
or when using export ASPNETCORE_ENVIRONMENT=Development, then dotnet run, the output still reports the Hosting Environment to be Production

Application started. Press Ctrl+C to shut down.
Hosting environment: Production

I am using the latest 2.1 RC1 framework and have not changed anything in the csproj file, just performed a restore and build.
I also added the following to the top of Program.cs to check:
Console.WriteLine("Running Main() ...");
string environment = Environment.GetEnvironmentVariable("ASPNETCORE_ENVIRONMENT");
Console.WriteLine("ASPNETCORE_ENVIRONMENT: {0}", environment);

var host = new HostBuilder()
........

Which outputs:

Running Main() ...
ASPNETCORE_ENVIRONMENT: Development

despite the Hosting Environment still showing Production.
The only other possibly related issue i could find was this: https://github.com/aspnet/Hosting/issues/1366
I actually first noticed the issue on a Generic Host app i built myself and then when looking for a sample to make sure i had not missed anything important. I also tried this on Windows using Visual Studio 2017 and updating the launchSettings.json to include the env var. I haven't yet tried this using v2.0
As a workaround to fix this added a line fo code to update hostContext.HostingEnvironment.EnvironmentName
.ConfigureAppConfiguration((hostContext, config) =>
                {
                    hostContext.HostingEnvironment.EnvironmentName = environment;
                    config.AddEnvironmentVariables();
                     ........

HostBuilder has no default config providers, including environment variables.
https://github.com/aspnet/Hosting/blob/23e3ab6555acf68715672f930b01b9a6a8b8134b/src/Microsoft.Extensions.Hosting/HostBuilder.cs#L102-L127
Call ConfigureHostConfiguration to add config providers:
https://docs.microsoft.com/en-us/aspnet/core/fundamentals/host/generic-host?view=aspnetcore-2.1#host-configuration
configHost.AddEnvironmentVariables("DOTNET_");
@guardrex

I'll open a doc issue to address this soon.

Found another sample here:
https://github.com/aspnet/Docs/tree/master/aspnetcore/fundamentals/host/generic-host/samples/2.x/GenericHostSample
this has the section:
.ConfigureHostConfiguration(configHost =>
                {
                    configHost.AddEnvironmentVariables();
                    configHost.AddJsonFile("hostsettings.json", optional: true);
                    configHost.AddCommandLine(args);
                })

but still does not work, also tried editing it to AddEnvironmentVariables("DOTNET_"); and AddEnvironmentVariables("ASPNETCORE_");
Currently both samples are broken as they suggest it should work due to the inclusion of:
config.AddJsonFile($"appsettings.{hostContext.HostingEnvironment.EnvironmentName}.json", optional: true);
but it is not possible to use appsettings.Development.json

I'll see if I can repro what you're seeing @chrisckc. I don't recall whether or not I specifically tested setting an env var, but I probably didn't test that scenario. I'll get back to you in perhaps an hour or two.

Were you running that GenericHostSample as-is? Or copying pieces into your own project?

Running it as is, just edited console entry inside the vscode generated
launch.json as instructed in the readme
On Thu, 31 May 2018, 17:28 Chris Ross, notifications@github.com wrote:

Were you running that GenericHostSample as-is? Or copying pieces into your
own project?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/aspnet/Hosting/issues/1440#issuecomment-393591588,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACVSDQWU0CznOwJfL54tdlDA_jNkcNQ2ks5t4BoegaJpZM4UU0jj
.

I'm able to repro the env var behavior here with the sample. @Tratcher, anything else you'd like me to try?
hostsettings.json
{
  "environment": "Development"
}

Works ... BUT ... there's a 🐞 in the sample. It needs ...
configHost.SetBasePath(System.IO.Directory.GetCurrentDirectory());

... to work. I'll patch that right now.
VSC env var
{
    "name": ".NET Core Launch (console)",
    "type": "coreclr",
    "request": "launch",
    "preLaunchTask": "build",
    "program": "${workspaceRoot}/bin/Debug/netcoreapp2.1/GenericHostSample.dll",
    "args": [],
    "cwd": "${workspaceRoot}",
    "stopAtEntry": false,
    "console": "externalTerminal",
    "env": {
        "ASPNETCORE_ENVIRONMENT": "Development"
    }
},

Doesn't work.
System env var
Doesn't work. However, I didn't do a restart. I don't think it's required in this case, as the console window should take it's snapshot of env vars when VSC opens the console window ... is that right @Tratcher?
Command line
dotnet run --environment Development

Works!
UseEnvironment
.UseEnvironment("Development")

Works!

Does this work?
    "env": {
        "ENVIRONMENT": "Development"
    }

You're not specifying a prefix on AddEnvironmentVariables and there is no default.

Ah ... I see. Stand-by ... I'll check that.

Yep ... that got it. When I get this patch in, the hostsettings.json file will work. I'm putting the same call to set the base path down on the configApp prior to setting up the appsettings files.

If you set the host contentroot then you shouldn't have to do anything for appsettings.

@Tratcher Which is simpler/best ...

Tell the reader that AddEnvironmentVariables doesn't specify the prefix, so they should use un-prefixed env vars when setting them for the debugger in an IDE.
Change the sample so that AddEnvironmentVariables does specify the prefix. We then instruct to use the prefix when setting the env var.

Specify a prefix in the sample, that's more discoverable.

remove the environment variable from the docs.

If we go the other way and make the change across the docs to specify the prefix every time ...
.AddEnvironmentVariables(prefix: "ASPNETCORE_")

... then we'd get out of this mess, right? :smile:

You'd still want to all out that env vars aren't added by default, and if you add them then you get to choose the prefix. I'd almost want to change all of the docs to use an alternate prefix to make it more obvious. "YOURPREFIX_"...

Ok, I'll open a docs issue so that we can do that work.
For now on this one patch PR to the Generic Host, let's set this one up with the ASPNETCORE_ prefix on AddEnvironmentVariables. Let's just patch it for now until we figure out the final, permanent fix for all topics+samps.
Besides, I'm neck-deep in 2.1 updates. After the 2.1 updates are complete, I'll have more time.

Great work in getting this progressed so quickly!
I spotted the merge into master so pulled the repo again to test it, i also updated to the release version of the 2.1.300 SDK which i believe was released yesterday.
Using the GenericHost Sample as is:

dotnet run

Works, Hosting Environment is set to Development using the hostsettings.json file
I then removed   "environment": "Development" from hostsettings.json, this caused Hosting Environment to be set to Production which i believe is the default.
With "environment": "Development" still removed from hostsettings.json, i tried this:

ASPNETCORE_ENVIRONMENT=Development dotnet run
Works, Hosting Environment is set to Development

I then added ASPNETCORE_ENVIRONMENT to launch.json:
            "console": "integratedTerminal",
            "env": {
                "ASPNETCORE_ENVIRONMENT": "Development"
            }

Works running vscode debug, Hosting Environment is set to Development.
Other observations:
I then added "environment": "Development" back into hostsettings.json and discovered that as expected, in the sample, this overrides any other configuration method.
For example if i do this:

ASPNETCORE_ENVIRONMENT=Staging dotnet run
Hosting Environment is still set to Development
Same if i try using Staging inside the launch.json file.

I then edited the Program.cs in the sample so that the call to AddEnvironmentVariables comes after AddJsonFile:
                .ConfigureHostConfiguration(configHost =>
                {
                    configHost.SetBasePath(Directory.GetCurrentDirectory());
                    configHost.AddJsonFile("hostsettings.json", optional: true);
                    configHost.AddEnvironmentVariables(prefix: "ASPNETCORE_");
                    configHost.AddCommandLine(args);
                })

This fixes that issue, i can now override the hostsettings.json file using env vars, is this behaviour intended for the sample? Would it not be better to have it like the above so that env vars always override configuration files which is what i have seen in most of the MVC examples?
Also i believe that the Web Host CreateDefaultBuilder method uses the following order:
appsettings.json
appsettings.{Environment}.json
User secrets when in Development mode
Environment variables
Commandline args
This is an inconsistency which i can't see any reason for?
Maybe Generic Host should have an equivalent CreateDefaultBuilder method which behaves the same way?

Sounds good, ConfigureAppConfiguration should also be the edited to be same order as ConfigureHostConfiguration.

@chrisckc Already done. :+1:

Great, found some another samples here that i had open in one of my many web browser tabs...
https://github.com/aspnet/Hosting/tree/release/2.1/samples
Looks like they will have the same issue.

I'll catch those on https://github.com/aspnet/Docs/issues/6760. I'm neck-deep in 2.1 sample updates. I'll get on that PR first thing when these samps are under control.

I'm running the sample at Docs\aspnetcore\fundamentals\host\hosted-services\samples\2.x\BackgroundTasksSample-GenericHost and I'm seeing this issue.  I've just pulled the latest and I'm running straight from VS2017.  I'm attempting to set the environment variables through the debug settings in the Project Properties/Debug/Environment variables box.

We periodically close 'discussion' issues that have not been updated in a long period of time.
We apologize if this causes any inconvenience. We ask that if you are still encountering an issue, please log a new issue with updated information and we will investigate.

authors: Tratcher aspnet-hello chrisckc guardrex mattspicer1


text:
Adds app-veyor!

Uses the default one for now.
@Eilon you mentioned that app veyor should at least try to run on the PR even without the .appveyor file, correct?

@jkotalik,
Thanks for having already signed the Contribution License Agreement. Your agreement was validated by .NET Foundation. We will now review your pull request.
Thanks,
.NET Foundation Pull Request Bot

Yup. And it looks like AppVeyor doesn't like you 😄

@jkotalik,
Thanks for having already signed the Contribution License Agreement. Your agreement was validated by .NET Foundation. We will now review your pull request.
Thanks,
.NET Foundation Pull Request Bot

@Eilon seems it is passing now 😄

source: gharchive/pull-request

authors: Eilon dnfclas jkotalik

repo: aspnet/IISIntegration


text:
Pawelka/aspnetcore

Issue #95 (point 2, part 1)

So this is a straight copy and rename of both the tool and tests? I assume you're planning to merge it at leave the old tool in place for a while?
You missed updating the SLN file and the nugetpackageverifier.json.

Yes. 1st commit is a copy. 2nd commit is changing to aspnetcoremodule. 3rd commit is reacting to changes in env variables. dotnet-publish-iis stays as is for now. Once aspnetcoremodule ships we will be able to remove it.
Good point about sln and package verifier. I will update both.

You'll also need to update the packages.csv file. This is used by the split-packages tool (categorizing packages to shipping/non-shipping/external).
File is here: https://github.com/aspnet/specs/blob/master/packages/packages.csv

:up::date:

:shipit:

authors: Tratcher moozzyk muratg


text:
Fixed UserValidator failing when UserName contains a hyphen

UserValidator would fail to validate usernames containing an email address with a hyphen in either the address or domain part

Hi @TsengSR, I'm your friendly neighborhood .NET Foundation Pull Request Bot (You can call me DNFBOT). Thanks for your contribution!

In order for us to evaluate and accept your PR, we ask that you sign a contribution license agreement. It's all electronic and will take just minutes. I promise there's no faxing. https://cla2.dotnetfoundation.org.

TTYL, DNFBOT;

@TsengSR, Thanks for signing the contribution license agreement so quickly! Actual humans will now validate the agreement and then evaluate the PR.
Thanks, DNFBOT;

Thanks for the PR, but we are going to remove regex entirely soon and will add hypen to the whitelist of characters.

See https://github.com/aspnet/Identity/pull/533

authors: HaoK TsengSR dnfclas

repo: aspnet/Identity


text:
Remove unused Key and IsResourceNotFound from LocalizedHtmlString

In addition, this type should not be exposed in IHtmlLocalizer. The Html() methods there should return IHtmlContent.

See also similar aspnet/Localization#158 issue.

repo: aspnet/Mvc


text:
AuthenticateResult.Fail(failreason) fails to propagate failreason through the stack

I am writing my own authentication middleware and upon returning AuthenticateResult.Fail("fail reason") in HandleAuthenticateAsync() I am expecting this to result in a challenge and somehow use this information during the challenge.
the "fail reason" string gets lost at some point when I would like to be able to see it passed to HandleForbiddenAsync or HandleAuthenticateAsync during the challenge. I've narrowed it down to perhaps having some properties passed in the challenge result from the result of authentication. At https://github.com/aspnet/Mvc/blob/c7a46e4caf0d1f25dd5f8bcd385af50b34b0f5dd/src/Microsoft.AspNetCore.Mvc.Core/Authorization/AuthorizeFilter.cs#L118 I wonder if the challenge result shouldn't be authenticated with properties derived from the authentication step at
https://github.com/aspnet/Mvc/blob/c7a46e4caf0d1f25dd5f8bcd385af50b34b0f5dd/src/Microsoft.AspNetCore.Mvc.Core/Authorization/AuthorizeFilter.cs#L92
it seems that this would resolve the issue I'm seeing as the rest of the stack seems to properly pass along this informationg to HandleForbiddenAsync and HandleAuthenticateAsync.

/cc @HaoK

Also I would like to be able to select between different deny behaviors in my middleware (Unauthorized vs Forbidden), it seems that upon executing the challenge the behavior that is passed is always Automatic, and this always leads to HandleUnauthorizedAsync. According to https://github.com/aspnet/Security/blob/5cd236c1b069cebecf53dbe6380e39933986eb47/src/Microsoft.AspNetCore.Authentication/AuthenticationHandler.cs#L317 the Principal should be null. But there is no way to create a ticket with a null principal since the property setter is private.

You shouldn't be setting the ticket directly, but you can call AuthenticateResult.Fail to return a result with a null ticket in your HandleAuthenticateAsync() method.
To change the behavior of Authorize, you should be able to IAsyncAuthorizationFilter and implement OnAuthorizationAsync however you like I believe.

For your original question of how to flow the failure message out of Authenticate, that's not something we are doing today, as its mostly an internal error for the handler (logging/debugging remote auth errors mostly).  As you mention you could certainly flow your error information in the AuthenticationProperties that is returned by your Auth middleware, and then pass it along in your call to challenge.

I haven't been able to trigger Forbidden, if the ticket is null, then the behavior becomes UnAuthorized.

Yep, that is the behavior for Automatic that comes from the AuthenticationHandler base class.  You can always implement an Auth middleware without using that base class too.

@HaoK is this still relevant?

Nope, challenge behavior was removed in preview 2, so Challenge/ForbidResult give you full control now

OK, closing this then since it doesn't seem to relevant for the future.

authors: HaoK gvessere rynowak


text:
Exception or log wrong asp-controller asp-action

Hi,
It is not a bug but I think it is a request.
Currently, we can generate a link like:
...

But if asp-controller or asp-action are wrong (not found). There are no warning.
I do similar error several time: I write a wrong asp-action and my link generate is by default /. If I don't test the link manually, I wannot know my error.
Perhaps, it should generate an exception ?

Thanks @zokiad.
@rynowak, @NTaylorMullen is there a way we can improve this experience? With razor compilation can we just add turn this into a build error or warning? Thanks!

Thanks for contacting us. We believe that the question you've raised have been answered. If you still feel a need to continue the discussion, feel free to reopen it and add your comments.

authors: mkArtakMSFT zokiad


text:
Remove virtual on a bunch of methods on RazorProjectItem

There's no meaningful way to override a lot of these

2b9c69bed8272ef09f70e419008556ca9f8d73ba

repo: aspnet/Razor


text:
Feature Question/Request: Razor Pages free view model property annotations

This isn't (probably) a thing, but I tried it because it would be cool if it were an actual feature.
Of course, old skool annotations work fine with a data-bound class.
For the free view model that comes with a Razor Page, I wish I could just do this ...
<form method="post" enctype="multipart/form-data">

        <input asp-for="Query" type="text">

        <button type="submit">Run Query</button>

</form>

... with (in the public class IndexModel : PageModel) ...
[BindProperty, Required, StringLength(50)]
public string Query { get; set; }

Is there an MVC theory that argues against this pattern? ... or is it just not a feature? ... If not, would u consider offering it?
Note: I couldn't get remote validation to work either on a Razor Pages free view model prop.

We plan to enable things like this in 2.1. This isn't possible today due to some limitations in the validation system.

@rynowak - do you know what issue (if any) is tracking this work for 2.1? I want to make sure this doesn't get lost.

https://github.com/aspnet/Mvc/issues/6790 tracks this work

FWIW, I just hit this too, and just assumed that it would work this way.
It seems you can bind properties, but validation doesn't work?

MVC has never supported validation attributes on 'top-level' items (parameters, properties). This was less of a problem before we did property binding on controllers (and then pages).
We're adding this in 2.1 because it's an obvious gap.

To my knowledge, this issue didn't make it to 2.1 --  rynowak said Issue #6790 tracks this work and it was scheduled for 2.1.0 preview 1 but I don't see it in the change log for 2.1.0 (https://github.com/aspnet/Home/releases/tag/2.1.0)
Can someone confirm?  Would be nice to get validation working on bound properties in the PageModel class.

The linked issue https://github.com/aspnet/Mvc/issues/6790 was fixed in 2.1.0-preview1. The support for validation attributes on Razor Page model properties and parameters was added in 2.1.0-preview2 (See https://github.com/aspnet/Mvc/issues/6790). If you're seeing issues with it, please file a new issue in https://github.com/aspnet/Mvc/issues

@pranavkm -- Sorry I'm not seeing it.  You said to refer to #6790 to see that it was added in preview2 -- but I don't see anywhere on preview2...

In my testing, this works as expected in 2.1

@ctolkien @pranavkm  -- Ok it does seem to work;  I have created a fresh AspNetCore 2.1 project to test and it's functioning as expected.  In the web application that I'm having trouble with I upgraded to 2.1 from 2.0 and it's still exhibiting issues so I'm not sure if it's for some reason using any 2.0 libraries still or not.  Anyone else having issues when upgrading an app from 2.0 to 2.1?

Found it!  Looks like you need to set the MVC compatibility version in ConfigureServices in the Startup.cs:
public void ConfigureServices(IServiceCollection services)
{
    services.AddMvc();
}

to
public void ConfigureServices(IServiceCollection services)
    {
        services.AddMvc().SetCompatibilityVersion(CompatibilityVersion.Version_2_1);
    }

Hope this helps others.

authors: Eilon RKDMax ctolkien guardrex pranavkm rynowak


text:
VS 2017 15.3 Required ?

For me loading this solution resulted in a protracted stream of load errors and package restores. I gave up after 20 minutes thinking it must be in a loop of some kind. I did notice that one of the first errors was as follows:
The version of Microsoft.NET.Sdk used by this project is insufficient to support this package.  Please ensure you are using the .NET Core SDK 2.0 or higher, which is included in Visual Studio 2017 Update 15.3.  This error may be suppressed by setting PackagingToolsInsufficientSDKVersion=false.
CookieSample
C:\Users\me\nuget\packages\microsoft.packaging.tools\1.0.0-preview2-25316-01\build\Microsoft.Packaging.Tools.targets	34
Suppressing the error seems silly if in fact there is a mismatch of capabilities . I am using VS 2017 Community 15.2 and upon investigation discover that 15.3 is in pre-release preview. It seems incongruous that we would want samples to only be supported by a prerelease IDE.
I am perhaps misinterpreting all of this and perhaps more patience is required but it seems to me something is amiss.
Any thoughts on a resolution appreciated.

Those are pre-release samples. VS is out of sync with pre-release runtime we're using. We have a hard time making it work on a day to day basis, even with 15.3-pre. The only guaranteed way to build the dev branch is to run build.cmd from the command line. VS can sometimes be used after a command line restore/build.

Yeah pretty much what @Tratcher said. Unfortunately sometimes the latest changes in the dev branch require features that aren't yet public in Visual Studio 2017. But, with the rapid pace of Visual Studio 2017 public updates, the wait generally isn't long. We try to avoid doing anything that prevents using the public VS2017, but sometimes it just isn't possible.

authors: Eilon Serexx Tratcher

repo: aspnet/Security


text:
Missing python requirement blocks starting the service from a Docker build

I ran the following and ran into the error below.
#!/bin/sh
 git clone git@github.com:assafelovic/gpt-researcher.git
 cd gpt-researcher
 docker compose build
 docker compose up

NOTE: I have my .env populated
Adding  lxml[html_clean] to the requirements.txt solved the problem.

WARN[0000] /home/smoney/src/aigency/gpt-researcher/gpt-researcher/docker-compose.yml: version is obsolete
gpt-researcher-1  | Traceback (most recent call last):
gpt-researcher-1  |   File "/usr/local/bin/uvicorn", line 8, in
gpt-researcher-1  |     sys.exit(main())
gpt-researcher-1  |              ^^^^^^
gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1157, in call
gpt-researcher-1  |     return self.main(*args, **kwargs)
gpt-researcher-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^
gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1078, in main
gpt-researcher-1  |     rv = self.invoke(ctx)
gpt-researcher-1  |          ^^^^^^^^^^^^^^^^
gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
gpt-researcher-1  |     return ctx.invoke(self.callback, **ctx.params)
gpt-researcher-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/click/core.py", line 783, in invoke
gpt-researcher-1  |     return __callback(*args, **kwargs)
gpt-researcher-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/uvicorn/main.py", line 409, in main
gpt-researcher-1  |     run(
gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/uvicorn/main.py", line 575, in run
gpt-researcher-1  |     server.run()
gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/uvicorn/server.py", line 65, in run
gpt-researcher-1  |     return asyncio.run(self.serve(sockets=sockets))
gpt-researcher-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
gpt-researcher-1  |   File "/usr/local/lib/python3.11/asyncio/runners.py", line 190, in run
gpt-researcher-1  |     return runner.run(main)
gpt-researcher-1  |            ^^^^^^^^^^^^^^^^
gpt-researcher-1  |   File "/usr/local/lib/python3.11/asyncio/runners.py", line 118, in run
gpt-researcher-1  |     return self._loop.run_until_complete(task)
gpt-researcher-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
gpt-researcher-1  |   File "/usr/local/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
gpt-researcher-1  |     return future.result()
gpt-researcher-1  |            ^^^^^^^^^^^^^^^
gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/uvicorn/server.py", line 69, in serve
gpt-researcher-1  |     await self._serve(sockets)
gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/uvicorn/server.py", line 76, in _serve
gpt-researcher-1  |     config.load()
gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/uvicorn/config.py", line 433, in load
gpt-researcher-1  |     self.loaded_app = import_from_string(self.app)
gpt-researcher-1  |                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/uvicorn/importer.py", line 19, in import_from_string
gpt-researcher-1  |     module = importlib.import_module(module_str)
gpt-researcher-1  |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
gpt-researcher-1  |   File "/usr/local/lib/python3.11/importlib/init.py", line 126, in import_module
gpt-researcher-1  |     return _bootstrap._gcd_import(name[level:], package, level)
gpt-researcher-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
gpt-researcher-1  |   File "", line 1204, in _gcd_import
gpt-researcher-1  |   File "", line 1176, in _find_and_load
gpt-researcher-1  |   File "", line 1147, in _find_and_load_unlocked
gpt-researcher-1  |   File "", line 690, in _load_unlocked
gpt-researcher-1  |   File "", line 940, in exec_module
gpt-researcher-1  |   File "", line 241, in _call_with_frames_removed
gpt-researcher-1  |   File "/usr/src/app/main.py", line 1, in
gpt-researcher-1  |     from backend.server import app
gpt-researcher-1  |   File "/usr/src/app/backend/server.py", line 7, in
gpt-researcher-1  |     from gpt_researcher.utils.websocket_manager import WebSocketManager
gpt-researcher-1  |   File "/usr/src/app/gpt_researcher/init.py", line 1, in
gpt-researcher-1  |     from .master import GPTResearcher
gpt-researcher-1  |   File "/usr/src/app/gpt_researcher/master/init.py", line 1, in
gpt-researcher-1  |     from .agent import GPTResearcher
gpt-researcher-1  |   File "/usr/src/app/gpt_researcher/master/agent.py", line 4, in
gpt-researcher-1  |     from gpt_researcher.master.functions import *
gpt-researcher-1  |   File "/usr/src/app/gpt_researcher/master/functions.py", line 7, in
gpt-researcher-1  |     from gpt_researcher.scraper.scraper import Scraper
gpt-researcher-1  |   File "/usr/src/app/gpt_researcher/scraper/init.py", line 3, in
gpt-researcher-1  |     from .newspaper.newspaper import NewspaperScraper
gpt-researcher-1  |   File "/usr/src/app/gpt_researcher/scraper/newspaper/newspaper.py", line 1, in
gpt-researcher-1  |     from newspaper import Article
gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/newspaper/init.py", line 10, in
gpt-researcher-1  |     from .api import (build, build_article, fulltext, hot, languages,
gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/newspaper/api.py", line 14, in
gpt-researcher-1  |     from .article import Article
gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/newspaper/article.py", line 15, in
gpt-researcher-1  |     from . import network
gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/newspaper/network.py", line 14, in
gpt-researcher-1  |     from .configuration import Configuration
gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/newspaper/configuration.py", line 15, in
gpt-researcher-1  |     from .parsers import Parser
gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/newspaper/parsers.py", line 12, in
gpt-researcher-1  |     import lxml.html.clean
gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/lxml/html/clean.py", line 18, in
gpt-researcher-1  |     raise ImportError(
gpt-researcher-1  | ImportError: lxml.html.clean module is now a separate project lxml_html_clean.
gpt-researcher-1  | Install lxml[html_clean] or lxml_html_clean directly.

I tried to add a PR but I can't
index f9d2110..28c666b 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -29,3 +29,4 @@ mistune
python-docx
htmldocx
langchain-google-genai
+lxml[html_clean]

+1 just tried to build now and getting the same error when I try to run it

I tried to add a PR but I can't
index f9d2110..28c666b 100644 --- a/requirements.txt +++ b/requirements.txt @@ -29,3 +29,4 @@ mistune python-docx htmldocx langchain-google-genai +lxml[html_clean]

I was able to add a PR

Looks like this may be duplicate of #418

I ran the following and ran into the error below.
#!/bin/sh
 git clone git@github.com:assafelovic/gpt-researcher.git
 cd gpt-researcher
 docker compose build
 docker compose up

NOTE: I have my .env populated
Adding lxml[html_clean] to the requirements.txt solved the problem.

WARN[0000] /home/smoney/src/aigency/gpt-researcher/gpt-researcher/docker-compose.yml: version is obsolete
gpt-researcher-1  | Traceback (most recent call last):
gpt-researcher-1  |   File "/usr/local/bin/uvicorn", line 8, in
gpt-researcher-1  |     sys.exit(main())
gpt-researcher-1  |              ^^^^^^
gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1157, in call
gpt-researcher-1  |     return self.main(*args, **kwargs)
gpt-researcher-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^
gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1078, in main
gpt-researcher-1  |     rv = self.invoke(ctx)
gpt-researcher-1  |          ^^^^^^^^^^^^^^^^
gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
gpt-researcher-1  |     return ctx.invoke(self.callback, **ctx.params)
gpt-researcher-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/click/core.py", line 783, in invoke
gpt-researcher-1  |     return __callback(*args, **kwargs)
gpt-researcher-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/uvicorn/main.py", line 409, in main
gpt-researcher-1  |     run(
gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/uvicorn/main.py", line 575, in run
gpt-researcher-1  |     server.run()
gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/uvicorn/server.py", line 65, in run
gpt-researcher-1  |     return asyncio.run(self.serve(sockets=sockets))
gpt-researcher-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
gpt-researcher-1  |   File "/usr/local/lib/python3.11/asyncio/runners.py", line 190, in run
gpt-researcher-1  |     return runner.run(main)
gpt-researcher-1  |            ^^^^^^^^^^^^^^^^
gpt-researcher-1  |   File "/usr/local/lib/python3.11/asyncio/runners.py", line 118, in run
gpt-researcher-1  |     return self._loop.run_until_complete(task)
gpt-researcher-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
gpt-researcher-1  |   File "/usr/local/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
gpt-researcher-1  |     return future.result()
gpt-researcher-1  |            ^^^^^^^^^^^^^^^
gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/uvicorn/server.py", line 69, in serve
gpt-researcher-1  |     await self._serve(sockets)
gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/uvicorn/server.py", line 76, in _serve
gpt-researcher-1  |     config.load()
gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/uvicorn/config.py", line 433, in load
gpt-researcher-1  |     self.loaded_app = import_from_string(self.app)
gpt-researcher-1  |                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/uvicorn/importer.py", line 19, in import_from_string
gpt-researcher-1  |     module = importlib.import_module(module_str)
gpt-researcher-1  |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
gpt-researcher-1  |   File "/usr/local/lib/python3.11/importlib/init.py", line 126, in import_module
gpt-researcher-1  |     return _bootstrap._gcd_import(name[level:], package, level)
gpt-researcher-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
gpt-researcher-1  |   File "", line 1204, in _gcd_import
gpt-researcher-1  |   File "", line 1176, in _find_and_load
gpt-researcher-1  |   File "", line 1147, in _find_and_load_unlocked
gpt-researcher-1  |   File "", line 690, in _load_unlocked
gpt-researcher-1  |   File "", line 940, in exec_module
gpt-researcher-1  |   File "", line 241, in _call_with_frames_removed
gpt-researcher-1  |   File "/usr/src/app/main.py", line 1, in
gpt-researcher-1  |     from backend.server import app
gpt-researcher-1  |   File "/usr/src/app/backend/server.py", line 7, in
gpt-researcher-1  |     from gpt_researcher.utils.websocket_manager import WebSocketManager
gpt-researcher-1  |   File "/usr/src/app/gpt_researcher/init.py", line 1, in
gpt-researcher-1  |     from .master import GPTResearcher
gpt-researcher-1  |   File "/usr/src/app/gpt_researcher/master/init.py", line 1, in
gpt-researcher-1  |     from .agent import GPTResearcher
gpt-researcher-1  |   File "/usr/src/app/gpt_researcher/master/agent.py", line 4, in
gpt-researcher-1  |     from gpt_researcher.master.functions import *
gpt-researcher-1  |   File "/usr/src/app/gpt_researcher/master/functions.py", line 7, in
gpt-researcher-1  |     from gpt_researcher.scraper.scraper import Scraper
gpt-researcher-1  |   File "/usr/src/app/gpt_researcher/scraper/init.py", line 3, in
gpt-researcher-1  |     from .newspaper.newspaper import NewspaperScraper
gpt-researcher-1  |   File "/usr/src/app/gpt_researcher/scraper/newspaper/newspaper.py", line 1, in
gpt-researcher-1  |     from newspaper import Article
gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/newspaper/init.py", line 10, in
gpt-researcher-1  |     from .api import (build, build_article, fulltext, hot, languages,
gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/newspaper/api.py", line 14, in
gpt-researcher-1  |     from .article import Article
gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/newspaper/article.py", line 15, in
gpt-researcher-1  |     from . import network
gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/newspaper/network.py", line 14, in
gpt-researcher-1  |     from .configuration import Configuration
gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/newspaper/configuration.py", line 15, in
gpt-researcher-1  |     from .parsers import Parser
gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/newspaper/parsers.py", line 12, in
gpt-researcher-1  |     import lxml.html.clean
gpt-researcher-1  |   File "/usr/local/lib/python3.11/site-packages/lxml/html/clean.py", line 18, in
gpt-researcher-1  |     raise ImportError(
gpt-researcher-1  | ImportError: lxml.html.clean module is now a separate project lxml_html_clean.
gpt-researcher-1  | Install lxml[html_clean] or lxml_html_clean directly.

Look here, a temporary fix for now
https://github.com/assafelovic/gpt-researcher/issues/418#issuecomment-2029066867

authors: arces scottmoney

repo: assafelovic/gpt-researcher


text:
Multi-Agents will have redundant content across different sections

Hello, I have tried the multi-agent implementation found in the multi-agents folder. I also read the multi-agent blog first. I discovered that even though each subsection focuses on different topics, it is easy for the content to overlap and discuss the same concepts across different subsections. As a result, the final report often contains a lot of redundant content, which is not useful at all.
I initially tried to add guidelines like "Each subsection must not have redundant content across different subsections." However, since the reviewer and reviser agents only work on individual subsections, the guidelines can only be applied at the subsection level, not to the entire report. Consequently, the reviewers and revisers are unaware of what is happening in other subsections, making it difficult to resolve the redundancy problem.
I am considering a solution where we have a chief reviewer and reviser. After each subsection completes its research, the chief reviewer and reviser would evaluate the final research across all subsections, ensure there is no redundant content, and provide revision directions to each subsection group to restart their research.
I think this kind of workflow will make the whole process more complex, increase token waste, and cause higher latency. However, I believe that if we can set global guidelines, such as "Each subsection must not have redundant content across different subsections," it can improve the final report's robustness and usefulness.

Hey @DandinPower that's a great discovery and definitely a huge improvement to the experience. At the moment (mostly for cost purposes) indeed the report does not take into consideration previous sub topics. To enable this every iteration would need to see the entire report generated before that which would be very costly in terms LLMs calls. I guess it's a tradeoff but definitely something we can consider adding as an option

Hey, I played with the researcher and noticed same issue. IMO having the tradeoff is well worth it, since the alternative is to remove redundant sections by hand. Maintaining coherence across the entire report seems paramount for real-world applications and complex subjects.
Maybe it could be an option so users could choose. Excited about possible architecture iterations in this direction and would love to help!

Hello @antoremin, I have forked the repo and written a brief draft about the modification design. If you are interested, you are welcome to check the multi_agent_v2 folder for details !

Hey @DandinPower would love help with a PR for this! Currently working on a new front end UX/UI

Hello @assafelovic, thank you for your invitation! I am willing to help with a PR after I finish the feature about the workflow I mentioned before.

@DandinPower @antoremin Thought about it deeper and I think there's a pretty decent approach that can work well:
Basically we can save all written content in a vector db and for every new section retrieve relevant information from it and consider it when generating new content. This way we're not overloading redundant content in the context for each section generation and it will most likely solve the issue. Wdyt?

Hello @assafelovic @antoremin,
I think this is a very good approach to reduce redundant content across different subsections. It also allows for generating content in complete parallel since we can retrieve the data independently. Additionally, moving extra LLMs calls into embedding models and vector searches is generally quicker and cheaper. However, I have some concerns about using the vector similarity-based approach:

How to set the similarity threshold to ensure the subsection content is covered enough.
How to set the similarity threshold to ensure it doesn't cover other subsection content.
If we use each subsection draft topic to retrieve relevant content, since the subsections are all related to the same topic, it is likely that all three "subsection draft topic" embeddings in the vector space will be very close. This makes it difficult to use the vector similarity-based method to retrieve the correct content we want.

In this figure, it shows that the three subsection topic embeddings (red points) are close to each other, and the other gray points are all written content chunks. The shallow yellow circles represent the relevant range based on the similarity threshold we set. This demonstrates the difficulty in finding a perfect way to retrieve the data without redundancy while getting all the content we want.

This is a great point! But consider that what we want is the LLM to take previously written sections into consideration when writing the next sections. So it might be enough to get similar chunks just for reasoning of generating new content. I'd be happy to see how we can push this forward anyone want to take a stab at it? @antoremin @DandinPower

Hey @assafelovic @antoremin,
Sorry for the mix-up earlier. Here’s my updated take on your idea:
We’re looking at creating content section by section. For each part, we’ll base the writing on the gpt_researcher’s detailed report, its topic, and some subsection guidelines. Plus, we’ll pull in similar content from previous sections to avoid repeating ourselves.
This method keeps things simple and cost-effective. We won’t need to feed all previous sections into the next one, which would otherwise make things way more complex and expensive. By tackling each section one at a time and only pulling in a bit of relevant info from before, we get good results with fewer calls to the embedding model, even if it takes a bit longer overall. It might take more time, but it’s worth it for a high-quality, non-redundant final product.
If I’ve got this right, I’m up for helping push this idea forward. Here’s a quick plan on how we could tweak the current multi_agent workflow:

After planning each subsection, we get researchers to do depth_report research in parallel for the first draft.
Start the subsection writing process. We’ll get a reviewer to check the draft and look at previous relevant results by pulling content from the embedding storage. With the draft and relevant contents, the reviewer will give notes for the revisor, who will make changes based on those notes (similar to the current workflow). Once the reviewer confirms it meets all guidelines, we can split the report into chunks, call the embedding model, and save to embedding storage for future subsections.
Once all sections are done, we can use the original workflow to write and publish the final report.
This is just my initial idea on how to implement this. Do you think it’s better to handle redundant content directly at the revisor stage, or do you have any other thoughts?

This sounds like a good plan, @DandinPower!
I have experience with converting transcripts of interviews, which might jump between topics, into a coherent report that keeps details (unlike a "summarize this meeting" prompt would).
My workflow was:

Chunk the interview into pieces (as it can be hours long, and we use GPT-3.5-turbo).
Create headlines
Create a structure based on the headlines.
Then split into chunks and write each chapter using the transcript chunked from a vectorDB.
Then compile all into final report.

There is still some redundancy, but not as much, as if not using the vectordb, and token efficiency increases a lot.
In your proposed workflow I could imagine the editor (or someone else) gathering abstract summary knowledge from the researchers, and writing a guided outline with headlines, and what facts/questions need to be addressed where.
This outline could be chunked and processed separately with knowledge from the vectorDB.

This sounds great guys, who is helping with leading this PR? :)

@danieldekay  Thank you for sharing your experience and workflow! Your approach with interview transcripts is quite interesting and offers some valuable insights.
@assafelovic I'm willing to take the lead on this PR and implement the improvements we've been discussing. Is there anything specific you'd like me to focus on or consider as I develop this feature?

@DandinPower, LangGraph also has a human-in-the-loop feature, and we might also want to ask a human on the highest abstraction level of the report structure to provide editorial feedback. This could be what it takes to bring from 60% quality to 85%.

@danieldekay That sounds like a good feature! In my opinion, I think there are two approaches to incorporate human feedback into the workflow:
(1) After the planner outlines each subsection topic, we allow humans to provide high-level feedback on the direction of each section.
(2) After each subsection revisor finishes its revision, not only can the reviewer give review notes, but it can also ask humans to provide direction for further revision.
Perhaps we can include a configuration option to let users decide whether they want to activate human feedback or not.

@DandinPower ping me on Discord if you'd like or we can open a channel for this feature and invite whomever would like to contribute/test it. Generally take in mind that GPT Researcher is in change of generating a research report based on research tasks, and the long detailed report leverages it multiple times. I assume the logic should be backend/report_type/detailed_report/ path. Looking forward and excited to see what comes out of it!

@assafelovic Okay, I'll ping you on Discord later! Thanks for your guidelines. I will make sure I understand the detailed report logic before pushing forward with the progress.

@DandinPower , I am also reading the STORM paper (https://arxiv.org/pdf/2402.14207) which has loads more insights into possible processes.

@danieldekay Hey! Thanks for introducing this paper. I think the current GPT researcher's detailed report type is also inspired by this paper. I will take a look at it.

Hey @assafelovic, I was previously busy, but now I have more time to push this forward! I am eager to see this move forward too!

Additionally I noticed that by having long descriptions (follow_guidelines does not work for me, throws error which is reported in discussion and also in #684 ) with explicit section logic helps.

Hey all, we've release a proposal solution for this. Please try it out and reopen this issue if you still find it an issue!

authors: DandinPower antoremin assafelovic danieldekay emilmirzayev


text:
remove $log.debug disable

Why disable $log.debug here: https://github.com/assisrafael/angular-input-masks/blob/2ea7504b30056e34e872ac872d564e847e83cadd/src/masks.js#L18
When included as a dependency, this disabled $log.debug in my app. I had to override the line in my own app to get back to the default state (enabled).

---
Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/11723963-remove-log-debug-disable?utm_campaign=plugin&utm_content=tracker%2F1022469&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F1022469&utm_medium=issues&utm_source=github).

Landed in v1.5.1.

authors: assisrafael simpixelated

repo: assisrafael/angular-input-masks

license: mit

license_source: bigquery


text:
issue with nested conditional patterns and non-linear patterns

There is a bug in the non-linear pattern detector where it does not recognize non-linear patterns when embedded in conditional patterns,
ast> let (name,name) = ("a","b").
error: non-linear pattern error: multiple instances of 'name' found within pattern.
ast> let (t1 if t1 is name, t2 if t2 is name) = ("a","b").  -- bug!
ast>

The non-linear pattern recognizer does not flag the second line correctly

the new implementation of conditional patterns fixes this.

repo: asteroid-lang/asteroid


text:
I feel you mean "active window" instead of "focused one"

Thanks for this software. It works great in my multimonitor setup with xmonad (as window manager).
My setup is a bit special (because I always prefer keyboard approaches) in the sense that my xmonad configuration does not change the active window when I move the mouse outside the active window. In other words, it allows that the active window is not the focused one (with the term focused I refer to the window where the mouse is).
In such case, I can confirm you that your software gives information about the active window (which is indeed what I prefer) and not about the focused one. Perhaps you can clarify this in the readme to avoid misinterpretations (since the name of your program might suggest it deals with "focus").

Short answer
I guess we have a misunderstanding what constitutes a focused and active window. Short answer:

Generally active = focus
If wm changes focus when mouse enters new window then: mouse = focus = active
If wm doesn't change focus when mouse enters new window then: mouse != focus and active = focus

my xmonad configuration does not change the active window when I move the mouse outside the active window

In that case it also shouldn't change the focus.

it allows that the active window is not the focused one (with the term focused I refer to the window where the mouse is)

Mouse doesn't equal focus necessarily (read long answer for explanation). Either xmonad changes the focus when the mouse enters another window, in which case _NET_ACTIVE_WINDOW should be changed to that window, or xmonad keeps the focus  on the same window even though the mouse enters another window, in which case _NET_ACTIVE_WINDOW shouldn't change.
Long answer
Based on my understanding a window is focused if the keyboard is attached to this window. Meaning all the keyboard input goes do that specific window (although it can be intercepted by other clients which manage your hotkeys like your wm or hotkey-daemons like sxhkd). It is not defined by where the mouse points to.
Some window-managers have a focus-follows-mouse feature which means that whenever the mouse enters a window it sets the focus to that window (in C by calling XSetInputFocus on that window). Meaning all the keyboard-input goes from now on to that window. That however is a wm-specific behavior and is not the default behavior of X. The wm can do whatever it wants when the mouse enters another window. It's up to the developers.
The active window, as I understand, is just the focused window (well not always actually, read more here, on the low-level it might not be the same window but logically it should be the window which your wm is pretending to give focus). Again it doesn't follow mouse but rather the input-focus.
If xmonad changes the focus (and therefore the keyboard-input) to the window where the mouse points to by calling XSetInputFocus on that window, than _NET_ACTIVE_WINDOW should also be set to that window. If xmonad keeps the focus on a window even though the mouse enters another window than also the active window and _NET_ACTIVE_WINDOW shouldn't change.
For clarification. _NET_ACTIVE_WINDOW is just a convention which a wm SHOULD follow if it wants that other programs which rely on _NET_ACTIVE_WINDOW work properly. The wm doesn't have to set _NET_ACTIVE_WINDOW. Or it can set it to whatever it wants. Again it's up to the developers of the wm.

https://tronche.com/gui/x/xlib/events/input-focus/

The keyboard is always attached to some window (typically, the root window or a top-level window), which is called the focus window. The focus window and the position of the pointer determine the window that receives keyboard input.

Long story short the active window is the focused window. If they are not the same the wm is doing something wrong. I think the issue here is that you think that the focused window is defined by where the mouse points to. Which can happen but than _NET_ACTIVE_WINDOW has to be set to the window under the mouse. But maybe I misunderstand something here.

Besides that does the program solve your problem or do you need to know when your mouse enters a new window? For this there is the so called EnterNotify-event.

astier, everything is fine.
I just wrote my remark because it took me some time to realize why the program was not following, in my settings, the mouse. This was due to my confusion between active and focus windows. After reading the previous comments it seems pretty clear I was on the minority side (and you are the tradition of X11).

Glad everything is fine.

authors: astier fboumol

repo: astier/xfocusnotify


text:
Install node 4 (lts)

Lounge no longer supports 0.12.

Done in a very crappy way in #14.

authors: astorije xPaw

repo: astorije/ansible-lounge


text:
Jupyter support?

Ruff 0.6 introduced Jupyter support by default. But that's not been updated here.

cc @dhruvmanila / @MichaReiser

authors: henryiii zanieb

repo: astral-sh/ruff-pre-commit


text:
Ruff requires multiple calls for certain import configurations

I noticed that I need to call ruff multiple times for the following configuration/code:
Configuration:
      lint: {
        select: [
          'I001',
          'I002',
        ],
        isort: {
          'required-imports': [
            "from typing import Dict"
          ],
          'lines-after-imports': 2,
          'combine-as-imports': true,
        },

Code:
from typing import Any

def func() -> Any:
  pass

After first pass of ruff:
from typing import Dict
from typing import Any

def func() -> Any:
    pass

After second pass of ruff:
from typing import Any, Dict

def func() -> Any:
    pass

Not sure if there is a config flag to add that would allow this to work in one go.  If not, seems like a bug.

Hmm... I'm not able to reproduce this. Does it looks like I'm doing everything you're doing?
❯ uv run ruff --version
ruff 0.8.2

❯ cat ruff.toml
[lint]
select = ["I001","I002"]

[lint.isort]
required-imports = ["from typing import Dict"]
lines-after-imports = 2
combine-as-imports = true

❯ cat ex.py
from typing import Any

def func() -> Any:
  pass%
❯ uv run ruff check --fix ex.py
Found 3 errors (3 fixed, 0 remaining).

❯ cat ex.py
from typing import Any, Dict

def func() -> Any:
  pass%

Ah I see. I'm using the ruff-wasm-nodejs npm package.
Any chance that can be updated to do the same?

I consider this out of scope for the JS API. It only provides an API to get all diagnostics for the initial source text. It doesn't even provide an official API for applying fixes. Instead, that's left to the caller.
You have to keep calling check until it returns no more diagnostics with fixes. You also have to be careful, fixes can overlap and some fixes shouldn't be applied when other fixes are present (e.g. one fix might try to remove an import when another edit adds it or even requires it).
Overall, the JS api has no support for fixes.

authors: MichaReiser dylwil3 leep-frog

repo: astral-sh/ruff


text:
Move Q004 to AST based checker

Summary
Continuing with #7595, this PR moves the Q004 rule to the AST checker.
Test Plan

[x] Existing test cases should pass
[x] No ecosystem updates

Thank you for the suggestion. That's a good idea. I'll just do the change for the string and bytes literal here and merge it.


text:
remove dynamic linking to liblzma from pip binary

If i install uv through pip, it dynamically links to /usr/local/opt/xz/lib/liblzma.5.dylib
# create an empty virtual env
~/Development/uv-dynlink ❯ virtualenv .venv
created virtual environment CPython3.12.4.final.0-64 in 381ms
  creator CPython3macOsBrew(dest=/Users/corygrinstead/Development/uv-dynlink/.venv, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, via=copy, app_data_dir=/Users/me/Library/Application Support/virtualenv)
    added seed packages: pip==24.1.2
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
# activate virtual env (source .venv/bin/activate)
~/Development/uv-dynlink ❯ overlay use .venv/bin/activate.nu
# install uv
(.venv) ~/Development/uv-dynlink ❯ pip install uv
(.venv) ~/Development/uv-dynlink ❯ which uv
╭───┬─────────┬──────────────────────────────────────────────┬──────────╮
│ # │ command │                        path                  │   type   │
├───┼─────────┼──────────────────────────────────────────────┼──────────┤
│ 0 │ uv      │ /Users/me/Development/example/.venv/bin/uv   │ external │
╰───┴─────────┴──────────────────────────────────────────────┴──────────╯
(.venv) ~/Development/uv-dynlink ❯ python3 --version
Python 3.12.4
(.venv) ~/Development/uv-dynlink ❯ python --version
Python 3.12.4
(.venv) ~/Development/uv-dynlink ❯ pip install uv
Collecting uv
  Using cached uv-0.2.31-py3-none-macosx_10_12_x86_64.whl.metadata (35 kB)
Using cached uv-0.2.31-py3-none-macosx_10_12_x86_64.whl (11.6 MB)
Installing collected packages: uv
Successfully installed uv-0.2.31

[notice] A new release of pip is available: 24.1.2 -> 24.2
[notice] To update, run: pip install --upgrade pip
(.venv) ~/Development/uv-dynlink ❯ overlay use .venv/bin/activate.nu
(.venv) ~/Development/uv-dynlink ❯ which uv | get path | otool -L ...$in | grep 'liblzma'
        /usr/local/opt/xz/lib/liblzma.5.dylib (compatibility version 12.0.0, current version 12.2.0)

However, if I build it from source, it does not
> cargo run python install
> otool -L ./target/debug/uv | grep 'liblzma'
# empty

Running into a related issue with shell install on  when running uv pip install ..., I get:
downloading uv 0.2.31 aarch64-apple-darwin
...
dyld[1134]: Library not loaded: /opt/homebrew/opt/xz/lib/liblzma.5.dylib
  Referenced from: <297928D5-54B8-3A44-B1B4-41CE488775BD> /private/var/root/.cargo/bin/uv
  Reason: tried: '/opt/homebrew/opt/xz/lib/liblzma.5.dylib' (no such file), '/System/Cryptexes/OS/opt/homebrew/opt/xz/lib/liblzma.5.dylib' (no such file), '/opt/homebrew/opt/xz/lib/liblzma.5.dylib' (no such file)

Interesting, thanks for reporting. I believe we can statically link it

Just can across this upgrading uv today.

Apologize, going to release shortly.

No worries! Part of being on the edge :) Thanks for making uv awesome.

authors: charliermarsh manzt rhenzel universalmind303

repo: astral-sh/uv


text:
uv run supports python package

Summary
Allow uv run ./package runs a Python package with a __main__.py script.
Resolves #7275

They serve different purposes. This PR (along with #7289) aims to make uv run works like a python launcher. This PR allows uv run to execute a directory containing a __main__.py file. Refer to https://docs.python.org/3/using/cmdline.html#interface-options for more details. On the other hand, python -m is used to run a module from the sys.path, and it's not equivalent to running a directory containing a __main__.py file.


text:
docs: add documentation on e2e rollup data flow and verification

Summary

add documentation on e2e rollup data flow and verification
see doc

Related Issues
closes #339

I'm looking into figuring out why it didn't, but I believe this should fail our markdown cli lint.

authors: joroshiba noot

repo: astriaorg/astria


text:
Tunnels

Tunnels support added.

Omg, thanks! I'll check it out as soon as possible.

authors: abadonna astrochili

repo: astrochili/narrator


text:
Added methods to load the image collection and set foreground/background

@ojustino - just so you are aware, I've added a few methods related to the foreground/background. With this, you can see what imagery is available by checking wwt.available_layers and then set the foreground/background with e.g. wwt.foreground = ...

Oops I messed up this PR, will open a new one (this wasn't really merged)

repo: astrofrog/pywwt-web


text:
add prestop hook to prevent logging issue

There is an issue in EKS where pods are deleted too quickly.
This preStopHook gives fluentd time to flush all logs before the
pod dies

Issue link: WILL BE INSERTED BY boring-cyborg
Make sure to mark the boxes below before creating PR: [x]

[ ] Description above provides context of the change
[ ] Unit tests coverage for changes (not needed for documentation changes)
[ ] Commits follow "How to write a good git commit message"
[ ] Relevant documentation is updated including usage instructions.
[ ] I will engage committers as explained in Contribution Workflow Example.

In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.
Read the Pull Request Guidelines for more information.

Codecov Report

Merging #471 into v1-10-6 will decrease coverage by 0.01%.
The diff coverage is 87.50%.

@@             Coverage Diff             @@
##           v1-10-6     #471      +/-   ##
===========================================
- Coverage    79.52%   79.50%   -0.02%
===========================================
  Files          519      519
  Lines        35335    35342       +7
===========================================
- Hits         28099    28098       -1
- Misses        7236     7244       +8

Impacted Files
Coverage Δ

airflow/contrib/kubernetes/istio.py
47.05% <0.00%> (ø)

airflow/contrib/kubernetes/worker_configuration.py
96.21% <ø> (ø)

airflow/contrib/executors/kubernetes_executor.py
57.38% <100.00%> (+0.09%)
:arrow_up:

...etes_request_factory/kubernetes_request_factory.py
99.16% <100.00%> (+0.02%)
:arrow_up:

.../kubernetes_request_factory/pod_request_factory.py
100.00% <100.00%> (ø)

airflow/contrib/kubernetes/pod.py
100.00% <100.00%> (ø)

airflow/plugins_manager.py
93.51% <0.00%> (-3.71%)
:arrow_down:

airflow/utils/dag_processing.py
58.53% <0.00%> (-0.33%)
:arrow_down:

airflow/www_rbac/views.py
75.51% <0.00%> (-0.07%)
:arrow_down:

airflow/www/views.py
44.95% <0.00%> (-0.06%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5bcb469...feedeaf. Read the comment docs.

authors: codecov-io dimberman

repo: astronomer/airflow


text:
Tasks failing without writing any logs

Some tasks are failing without writing any logs.
Possible reason might be
Imports of heavy libraries at module level might cause DAG processing timeout errors.
To avoid this, we need to delay of imports to the specific operator scope.
Reference: https://airflow.apache.org/docs/apache-airflow/stable/best-practices.html#top-level-python-code
However we need to investigate in detail.

We have a Zendesk ticket created for this issue: https://astronomer.zendesk.com/agent/tickets/9862

Since issue is not being observed for weeks, so closing the issue

repo: astronomer/astronomer-providers


text:
📌GSSOC EXT NOTICE: Improper Labeling in the Project

Dear Project Admin, @aswathcm29
It has come to our attention that your repository not applying the necessary labels to issues and merged PRs, which is affecting contributor scores on the leaderboard.
Please ensure you use the correct labels:
gssoc-ext
level1, level2, level3
hacktoberfest-accepted (only if your repository have Hacktoberfest topic in description)
NOTE: Level labels should not have any dashes between them. Spaces are allowed, but no dashes should be used.
Kindly review and update the labels on your issues and PRs to ensure fairness for all contributors. Failure to address this may result in your project being flagged.
Thank you,
Core Team
GSSoC '24 Extended

@aswathcm29 Please correct your lebels 'gssoc-ext' instead of gssoc-etd
Do it asap


text:
Implement SimpleObject for SystemTime

Description of the feature
I am using a database which returns time in std::time::SystemTime. I would like to re-use the same struct to query the database as well as use it for my resolvers. This can only be possible if SimpleObject has an implementation for SystemTime.
Code example (if possible)
#[derive(Debug, SimpleObject, Deserialize, Queryable)] // Queryable from edgedb
pub struct QuestionPartialsDetails {
    pub created_at: SystemTime,
}

You can use time::OffsetDateTime or chrono::DateTime<T>

You can use time::OffsetDateTime or chrono::DateTime<T>

When you don't need timezones, pulling in time or chrono just to serialize what is effectively a UNIX timestamp is a lot to ask.

authors: ColonelThirtyTwo IgnisDa sunli829

repo: async-graphql/async-graphql


text:
Error extensions cause stack overflow

Expected Behavior
When using error extensions, I should be able to implement the ErrorExtensions trait for my errors and then call error.extend() inside resolvers. When querying a field that returns an error, I should be able to see the error along with the code and other custom fields I may add
Actual Behavior
The program crashes when calling the graphql endpoint. Here is the stacktrace
thread 'actix-rt|system:0|arbiter:0' has overflowed its stack
fatal runtime error: stack overflow

Steps to Reproduce the Problem

Run the error-extensions example contained in the examples folder
Call the extend query on the schema
The server crashes

Basically the code goes in a loop on this line, and then ultimately crashes
Specifications

Version: Reproduced with the branches actix-web-v4-beta and master
Platform:
Subsystem:

Fixed in v3.0.7 😁


text:
JVM crash with frame Profiler::getJavaTraceAsync

Describe the bug
Crash occurred while the service was running.
pkuuid:[GX19001243] agent:[GX19001243] ip:[10.218.132.158] organ:[86510000] source:[2]  version:[] bundle:[] mainThread:[2409061029518267] orderUuid:[2831878424238424066]  2024-09-06 INFO

A fatal error has been detected by the Java Runtime Environment:

SIGSEGV (0xb) at pc=0x00007f89164312ac, pid=1, tid=0x00007f871bdfd700

JRE version: Java(TM) SE Runtime Environment (8.0_202-b08) (build 1.8.0_202-b08)
Java VM: Java HotSpot(TM) 64-Bit Server VM (25.202-b08 mixed mode linux-amd64 compressed oops)
Problematic frame:
C  [libasyncProfiler.so+0x312ac]  Profiler::getJavaTraceAsync(void*, ASGCT_CallFrame*, int, StackContext*)+0x65c

Core dump written. Default location: //core or core.1 (max size 5242880 kB). To ensure a full core dump, try "ulimit -c unlimited" before starting Java again

An error report file with more information is saved as:
//hs_err_pid1.log

If you would like to submit a bug report, please visit:
http://bugreport.java.com/bugreport/crash.jsp

[error occurred during error reporting , id 0xb]
[END] 2024/9/6 11:23:54
Expected vs. actual behavior
No response
Reproduction Steps
No response
Additional Information/Context
No response
Async-profiler version
2.10
Environment details
NAME="Anolis OS"
VERSION="8.6"
ID="anolis"
ID_LIKE="rhel fedora centos"
VERSION_ID="8.6"
PLATFORM_ID="platform:an8"
PRETTY_NAME="Anolis OS 8.6"
ANSI_COLOR="0,31"
HOME_URL="https://openanolis.cn/"

Version 2.x is no lonter supported.
Please try an up-to-date version of async-profiler - preferably, from master.
The issue should have been fixed already.

repo: async-profiler/async-profiler


text:
refactor: add CommonModel conversion to MetaModel

Description
This PR adds the conversion of CommonModel to MetaModel.
The reason I did not change the entire Interpreter functionality to use MetaModel instead, is because the complexity of the interpreter it's not possible without the CommonModel.
Related issue(s)
Blocked by #655

Remaking this.

repo: asyncapi/modelina


text:
fix: add support to activate the extension on json files

closes #123

/rtm

repo: asyncapi/vs-asyncapi-preview


text:
fix: preserve scroll position on preview reload

Signed-off-by: nikhilkalburgi nikhilkalburgi19@gmail.com
Fixes #168
Description

Used message passing feature of webview to preserve the position
load event and settimeout is used as we need to wait until the asynapistandalone completes
the panel retains the position of webview and passes that when the openasynapi is called again

https://github.com/asyncapi/vs-asyncapi-preview/assets/70331875/6f17d2a7-60c3-42cf-96fb-d341cf469c77

Awesome!! Thanks for this @nikhilkalburgi !!

rtm

/rtm

authors: ivangsa nikhilkalburgi


text:
docs: re-add message validation guide and change code snippet

Description

Re-added the message validation guide in the next-major-spec branch.
Changed the code snippet to match v3.

Related issue(s)
Fixes #1865

@AnimeshKumar923 thanks a 💯 mate. I thought I will have to do all these PRs. Just please update code sample

Just did what I was able to did. Glad to help! :grin:

/rtm

repo: asyncapi/website


text:
🛑 Wedding HTTPS is down

In 4fb8c98, Wedding HTTPS ($WEDDINGHTTPS) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Wedding HTTPS is back up in 97bf278.

repo: atatkin/milos-uptime


text:
🛑 Wedding HTTPS is down

In a59abf1, Wedding HTTPS ($WEDDINGHTTPS) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Wedding HTTPS is back up in 2102b21.


text:
🛑 Wedding HTTPS is down

In 72ac811, Wedding HTTPS ($WEDDINGHTTPS) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Wedding HTTPS is back up in 14a3978.


text:
🛑 Wedding HTTPS is down

In 0bef377, Wedding HTTPS ($WEDDINGHTTPS) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Wedding HTTPS is back up in 5bf04c5.


text:
🛑 Wedding HTTPS is down

In 6979727, Wedding HTTPS ($WEDDINGHTTPS) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Wedding HTTPS is back up in 0f655cc.