text:
FSharp.Formatting error if I include FSharp.Data and use CSV type provider

Description
Adding type Stocks = CsvProvider<"./data/MSFT.csv"> to https://github.com/fsprojects/ProjectScaffold/blob/master/src/FSharp.ProjectTemplate/Library.fs causes a FSharp.Formatting Error
Repro steps
Please provide the steps required to reproduce the problem
1.a Use forge new scaffold to create a new Scaffold.
1.b Add nuget FSharp.Data to group Build in paket.dependencies: https://github.com/fsprojects/ProjectScaffold/blob/master/paket.dependencies#L18
1.c Modify https://github.com/fsprojects/ProjectScaffold/blob/master/src/FSharp.ProjectTemplate/Library.fs and call CsvProvider (something like this):
namespace ProcessingData

open FSharp.Data

/// Stock prices example
type Stocks = CsvProvider<"./data/MSFT.csv">

/// Documentation for my library
///
/// ## Example
///
///     let h = Library.hello 1
///     printfn "%d" h
///
module Library =
  /// Returns 42
  ///
  /// ## Parameters
  ///  - `num` - whatever
  let hello num = 42

1.d Add a data folder inside project folder (in my case [root project folder]/src/ProcessingData) and put http://fsharp.github.io/FSharp.Data/data/MSFT.csv inside it.

Use .\build.cmd to run all the targets.

Expected behavior
Build correctly.
Actual behavior
I am getting the following error for GenerateReferenceDocs target:
Starting Target: GenerateReferenceDocs (==> RunTests, GenerateHelp)
Building documentation (Default), this could take some time, please wait...
C:\Users\oscarvarto\gitRepos\dotnetWork\ProcessingData\packages\build\FAKE\tools\FAKE.exe  --fsiargs -d:FAKE -d:RELEASE -d:REFERENCE "generate.fsx"
FSharp.Formatting Information: 0 : FSharp.Formatting Logging setup!
Yaaf.FSharp.Scripting Information: 0 : Yaaf.FSharp.Scripting Logging setup!
Copying file: C:\Users\oscarvarto\gitRepos\dotnetWork\ProcessingData\docs\img\logo-template.pdn
Copying file: C:\Users\oscarvarto\gitRepos\dotnetWork\ProcessingData\docs\img\logo.png
Copying styles and scripts: C:\Users\oscarvarto\gitRepos\dotnetWork\ProcessingData\docs\content\style.css
Copying styles and scripts: C:\Users\oscarvarto\gitRepos\dotnetWork\ProcessingData\docs\content\style.css.bak
Copying styles and scripts: C:\Users\oscarvarto\gitRepos\dotnetWork\ProcessingData\docs\content\style_light.css
Copying styles and scripts: C:\Users\oscarvarto\gitRepos\dotnetWork\ProcessingData\docs\content\tips.js
Copying styles and scripts: C:\Users\oscarvarto\gitRepos\dotnetWork\ProcessingData\docs\content\img\github-blue.png
Copying styles and scripts: C:\Users\oscarvarto\gitRepos\dotnetWork\ProcessingData\docs\content\img\github.png
Deleting contents of C:\Users\oscarvarto\gitRepos\dotnetWork\ProcessingData\docsrc\tools\../../docs\reference
FSharp.Formatting Error: 0 : Could not read comments from entity 'ProcessingData.Stocks': Microsoft.FSharp.Compiler.ErrorLogger+ReportedError: Exception of type 'Microso
ft.FSharp.Compiler.Tast+InternalUndefinedItemRef' was thrown.
   at Microsoft.FSharp.Compiler.ErrorLogger.ErrorLoggerExtensions.ErrorLogger.Error[b](ErrorLogger x, Exception exn) in C:\Users\robin\dev\FSharp.Compiler.Service\src\fs
harp\ErrorLogger.fs:line 313
   at Microsoft.FSharp.Compiler.Tast.EntityRef.get_Deref() in C:\Users\robin\dev\FSharp.Compiler.Service\src\fsharp\tast.fs:line 2684
   at Microsoft.FSharp.Compiler.Tastops.stripTyEqnsA(TcGlobals g, Boolean canShortcut, TType ty) in C:\Users\robin\dev\FSharp.Compiler.Service\src\fsharp\TastOps.fs:line
 627
   at Microsoft.FSharp.Compiler.Tastops.extensionInfoOfTy(TcGlobals g, TType ty) in C:\Users\robin\dev\FSharp.Compiler.Service\src\fsharp\TastOps.fs:line 1464
   at Microsoft.FSharp.Compiler.Tastops.metadataOfTy(TcGlobals g, TType ty) in C:\Users\robin\dev\FSharp.Compiler.Service\src\fsharp\TastOps.fs:line 1489
   at Microsoft.FSharp.Compiler.Infos.GetImmediateIntrinsicPropInfosOfType(FSharpOption`1 optFilter, AccessorDomain ad, TcGlobals g, ImportMap amap, range m, TType typ)
in C:\Users\robin\dev\FSharp.Compiler.Service\src\fsharp\infos.fs:line 3293
   at <StartupCode$FSharp-Compiler-Service>.$Symbols.get_MembersFunctionsAndValues@412-1.GenerateNext(IEnumerable`1& next) in C:\Users\robin\dev\FSharp.Compiler.Service\
src\fsharp\vs\Symbols.fs:line 426
   at Microsoft.FSharp.Core.CompilerServices.GeneratedSequenceBase`1.MoveNextImpl()
   at Microsoft.FSharp.Core.CompilerServices.GeneratedSequenceBase`1.System-Collections-IEnumerator-MoveNext()
   at Microsoft.FSharp.Collections.SeqModule.ToList[T](IEnumerable`1 source)
   at <StartupCode$FSharp-Compiler-Service>.$Symbols.get_MembersFunctionsAndValues@411.Invoke(Unit unitVar0) in C:\Users\robin\dev\FSharp.Compiler.Service\src\fsharp\vs\
Symbols.fs:line 412
   at Microsoft.FSharp.Compiler.SourceCodeServices.Impl.protect[a](FSharpFunc`2 f) in C:\Users\robin\dev\FSharp.Compiler.Service\src\fsharp\vs\Symbols.fs:line 27
   at Microsoft.FSharp.Compiler.SourceCodeServices.FSharpEntity.get_MembersFunctionsAndValues() in C:\Users\robin\dev\FSharp.Compiler.Service\src\fsharp\vs\Symbols.fs:li
ne 411
   at FSharp.MetadataFormat.Reader.getMembers@1059-1.GenerateNext(IEnumerable`1& next) in c:\Tomas\Public\tpetricek\FSharp.Formatting\src\FSharp.MetadataFormat\Main.fs:l
ine 0
   at Microsoft.FSharp.Core.CompilerServices.GeneratedSequenceBase`1.MoveNextImpl()
   at Microsoft.FSharp.Core.CompilerServices.GeneratedSequenceBase`1.System-Collections-IEnumerator-MoveNext()
   at Microsoft.FSharp.Collections.SeqModule.ToList[T](IEnumerable`1 source)
   at FSharp.MetadataFormat.Reader.readType@1055.Invoke(String cat, IDictionary`2 cmds, Comment comment) in c:\Tomas\Public\tpetricek\FSharp.Formatting\src\FSharp.Metada
taFormat\Main.fs:line 1070
   at FSharp.MetadataFormat.Reader.readCommentsInto[a](FSharpSymbol sym, ReadingContext ctx, String xmlDoc, FSharpFunc`2 f) in c:\Tomas\Public\tpetricek\FSharp.Formattin
g\src\FSharp.MetadataFormat\Main.fs:line 944
FSharp.Formatting Information: 0 : Starting razor engine
FSharp.Formatting Information: 0 : Generating: index.html
FSharp.Formatting Information: 0 : Generating modules...
FSharp.Formatting Information: 0 : Generating module: processingdata-library
FSharp.Formatting Information: 0 : Finished module: processingdata-library
FSharp.Formatting Information: 0 : Generating types...
Finished Target: GenerateReferenceDocs

Known workarounds
None yet :(
Related information

Operating system
Windows 10 Pro 64 bits (running on top of Parallels 12 for Mac).

Branch

.NET Runtime, CoreCLR or Mono Version

** Verifying the fsc version:
fsc --help
Microsoft (R) F# Compiler version 14.0.23020.0

I have already installed VS 2017 15.3.2

PATH Environment variables:

PS C:\Users\oscarvarto> $env:path.split(";")
C:\Program Files\ConEmu\ConEmu\Scripts
C:\Program Files\ConEmu
C:\Program Files\ConEmu\ConEmu
C:\Program Files (x86)\Parallels\Parallels Tools\Applications
C:\Windows\system32
C:\Windows
C:\Windows\System32\Wbem
C:\Windows\System32\WindowsPowerShell\v1.0\
C:\Program Files\Microsoft SQL Server\Client SDK\ODBC\110\Tools\Binn\
C:\Program Files (x86)\Microsoft SQL Server\120\Tools\Binn\
C:\Program Files\Microsoft SQL Server\120\Tools\Binn\
C:\Program Files\Microsoft SQL Server\120\DTS\Binn\
C:\Program Files (x86)\Microsoft SQL Server\120\Tools\Binn\ManagementStudio\
C:\Program Files (x86)\Microsoft SQL Server\120\DTS\Binn\
C:\Program Files\Microsoft SQL Server\130\Tools\Binn\
C:\Program Files\dotnet\
C:\Program Files (x86)\Xoreax\IncrediBuild
C:\Program Files\Git\cmd
C:\Program Files (x86)\Microsoft SQL Server\Client SDK\ODBC\130\Tools\Binn\
C:\Program Files (x86)\Microsoft SQL Server\140\Tools\Binn\
C:\Program Files (x86)\Microsoft SQL Server\140\DTS\Binn\
C:\Program Files (x86)\Microsoft SQL Server\140\Tools\Binn\ManagementStudio\
C:\ProgramData\chocolatey\bin
C:\Program Files (x86)\Microsoft SDKs\F#\4.0\Framework\v4.0
C:\Users\oscarvarto\scoop\shims
C:\Users\oscarvarto\AppData\Local\Microsoft\WindowsApps

I added a MSBUILD_PATH environment variable following instructions from https://github.com/ionide/ionide-vscode-fsharp/ pointing to C:\Program Files (x86)\MSBuild\14.0\Bin

I think my installation is not respecting/using this variable, because at the top of the build, I get a message saying that version 15.3.409.57025 of MSBuild is being used:
Starting Target: Build (==> AssemblyInfo)
Building project: C:\Users\oscarvarto\gitRepos\dotnetWork\ProcessingData\ProcessingData.sln
  C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\MSBuild\15.0\Bin\MSBuild.exe  C:\Users\oscarvarto\gitRepos\dotnetWork\ProcessingData\ProcessingData.sln /t:Rebuild /m  /nodeReuse:False    /p:RestorePackages="False" /p:Configuration="Release" /p:Configuration="Release" /p:Platform="Any CPU" /logger:Fake.MsBuildLogger+ErrorLogger,"C:\Users\oscarvarto\gitRepos\dotnetWork\ProcessingData\packages\build\FAKE\tools\FakeLib.dll"
Microsoft (R) Build Engine version 15.3.409.57025 for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.

FSHARPINSTALLDIR pointing to C:\Program Files (x86)\Microsoft SDKs\F#\4.1\Framework\v4.0\ (I guess VS 2017 installation added that).

Maybe this variable is messing with my build, because I am also getting a message about F# 4.1 being used:
 CoreCompile:
         C:\Program Files (x86)\Microsoft SDKs\F#\4.1\Framework\v4.0\fsc.exe -o:obj\Release\ProcessingData.dll

Related issues:

https://github.com/fsprojects/FSharp.Formatting/issues/423
https://github.com/fsprojects/FSharp.Formatting/issues/325

So, here are my initial thoughts:

This is almost certainly not an issue with ProjectScaffold (that having been said...)
Have you tried deleting the FAKE cache (it's the .fake folder at the root of your local repository)?
Have you tried pinning (in Paket) the versions of FSharp.Compiler.Service and/or FSharp.Formatting (note: you may have to play around with going up or down a version or two in each project)?
Also, I'm pretty sure you do want to be using F# 4.1. But I agree the MSBuild versions are... troubling.
Oh, also, what version of the runtime are you targeting (e.g. .NET Framework 4.6.1, .NET Stanard 2.0, et cetera)?

authors: oscarvarto pblasucci

repo: fsprojects/ProjectScaffold


text:
AssertionError: Found chunk size mismatch

Hello guys!
I'm trying to build kerchunk index for ERA5 full dataset.
Everythg goes well when I'm merging daily data, but when I try to build combined index for months with different amount of days like January and February for example:
e5.oper.an.sfc.128_134_sp.ll025sc.2019010100_2019013123.nc
e5.oper.an.sfc.128_134_sp.ll025sc.2019020100_2019022823.nc
from kerchunk.combine import MultiZarrToZarr
mzz = MultiZarrToZarr(
    singles,
    remote_protocol="s3",
    remote_options={'anon': False},
    concat_dims=["time"],
    identical_dims = ['latitude', 'longitude'],
)

I get the AssertionError: Found chunk size mismatch.
Probably I'm doing something wrong?

@peterm790 , you looked at era5, did you use these specific files? This is probably explicitly in one of your tutorials already.
@agrigoriev , it would be useful to show the xarray text view of the two datasets opened with chunks={}, so we can see the native chunk size.

Hi @agrigoriev I have a tutorial for the AWS ERA5-pds available here: https://github.com/peterm790/ERA5_Kerchunk_tutorial/blob/master/ERA5_tutorial.ipynb , this is considering hourly data in monthly files.
I assume your dataset is Monthly files of daily means? which would result in mismatched chunks. If that is the case I am not certain that there is a solution at present. Unless it were chunked as 1 chunk per time step.

It is certainly correct that kerchunk cannot change the fundamental chunk sizes in the underlying files, and also that zarr requires all the chunks along a given axis to be the same size (except the last). I am hoping to remove this requirement, but that need agreement within the zarr project itself.

Using scripts (appended in the end) I found out that, indeed, some source NetCDF files from DOI:10.5676/EUM_SAF_CM/SARAH/V003,  differ in their Block definition (as per GDAL) for example between year 2020 ( 2600 x 1) and year 2021, 2022 (2600 x 2600) :
❯ gdalinfo NETCDF:"SDUds2022010100000042310001I1MA.nc":SDU |grep Block  # First day

❯ gdalinfo NETCDF:"SDUds202001010000004231000101MA.nc":SDU |grep Block  # Last day
Band 1 Block=2600x1 Type=Int16, ColorInterp=Undefined

❯ gdalinfo NETCDF:"SDUds202012310000004231000101MA.nc":SDU |grep Block
Band 1 Block=2600x1 Type=Int16, ColorInterp=Undefined

❯ gdalinfo NETCDF:"SDUds2021010100000042310001I1MA.nc":SDU |grep Block
Band 1 Block=2600x2600 Type=Int16, ColorInterp=Undefined

❯ gdalinfo NETCDF:"SDUds2022010100000042310001I1MA.nc":SDU |grep Block
Band 1 Block=2600x2600 Type=Int16, ColorInterp=Undefined

I see nothing, and wouldn't expect, different in their structure :
❯ gdalinfo NETCDF:"SDUds202001010000004231000101MA.nc":SDU -nomd -mm -hist
Driver: netCDF/Network Common Data Format
Files: SDUds202001010000004231000101MA.nc
Size is 2600, 2600
Origin = (-64.999998473533992,64.999998473533992)
Pixel Size = (0.049999998825795,-0.049999998825795)
Corner Coordinates:
Upper Left  ( -64.9999985,  64.9999985)
Lower Left  ( -64.9999985, -64.9999985)
Upper Right (  64.9999985,  64.9999985)
Lower Right (  64.9999985, -64.9999985)
Center      (   0.0000000,   0.0000000)
Band 1 Block=2600x1 Type=Int16, ColorInterp=Undefined
    Computed Min/Max=0.000,16826.000
0...10...20...30...40...50...60...70...80...90...100 - done.
  256 buckets from -31.2686 to 15978.3:
  467342 40261 52663 52011 46506 31090 27445 25339 23633 23290 22746 23262 24742 21057 19571 19080 18612 18956 18431 19037 19048 17618 17125 16782 17034 16769 16696 16770 16809 16232 15834 15799 15891 15655 15670 15314 14966 15214 15086 15190 14874 15212 14919 15050 14733 14800 15037 14869 15200 14833 15006 15151 15183 14797 15200 14978 15383 15272 15696 15825 15989 15718 15478 15979 15849 16251 16030 16396 16280 16676 16418 16662 16616 17024 17220 16949 17277 17163 17758 17499 17794 17684 18057 17866 18335 18069 18194 18512 18235 19162 18763 19239 19074 19205 18863 19366 19594 20183 19899 20923 20775 21282 21249 21207 21959 22143 22932 22462 23100 23505 24785 24325 24773 24628 25823 26099 26319 26651 27288 27981 27884 29528 29708 31114 31418 30804 31030 32053 32397 32498 34588 34812 34814 33896 35101 36339 37499 36549 37926 36866 38944 40005 40957 44034 42018 43687 44113 46579 49330 50137 50086 53450 54112 54046 59402 60320 60950 73065 78197 72032 66601 56165 53708 53309 57130 48266 49690 45168 37245 30702 26818 26691 25001 25085 24415 23307 22653 22691 23935 23741 22637 21416 20936 20156 19663 18889 18150 16446 16723 15264 14665 13640 13660 12832 13392 12443 10820 10301 9992 10353 10469 9712 9943 10564 8366 7821 8051 7962 8917 7671 8276 7946 8500 7299 7896 7307 3670 2651 2183 1818 1975 1310 701 531 477 457 186 167 132 127 91 73 53 47 47 36 25 24 26 22 22 23 17 21 22 19 29 17 17 22 14 13 12 12 6 87
  NoData Value=-999
  Unit Type: h
  Offset: 0,   Scale:0.00100000004749745

❯ gdalinfo NETCDF:"SDUds2022010100000042310001I1MA.nc":SDU -nomd -mm -hist
Driver: netCDF/Network Common Data Format
Files: SDUds2022010100000042310001I1MA.nc
Size is 2600, 2600
Origin = (-64.999998473533992,64.999998473533992)
Pixel Size = (0.049999998825795,-0.049999998825795)
Corner Coordinates:
Upper Left  ( -64.9999985,  64.9999985)
Lower Left  ( -64.9999985, -64.9999985)
Upper Right (  64.9999985,  64.9999985)
Lower Right (  64.9999985, -64.9999985)
Center      (   0.0000000,   0.0000000)
Band 1 Block=2600x2600 Type=Int16, ColorInterp=Undefined
  Min=0.000 Max=15505.000   Computed Min/Max=0.000,15505.000
  Minimum=0.000, Maximum=15505.000, Mean=6665.472, StdDev=3824.519
  256 buckets from -30.402 to 15535.4:
  365805 32375 43754 45164 42195 28573 24135 22574 22238 21668 21330 21519 22036 19163 18429 17775 17603 17744 16924 17314 18372 16608 16132 15651 15718 15743 15933 16066 15822 16018 15788 15365 15355 15815 15929 16048 16074 15990 15818 16080 15929 16530 16024 16410 16702 16793 16532 16513 16387 16808 16640 16593 16711 16792 16490 16786 16713 16474 16764 16464 17015 17352 17459 17262 17124 17443 16848 17002 17323 17267 17453 17535 17202 17159 17086 17746 17811 17848 17900 17868 17964 18059 17997 18280 18319 18990 18958 19047 18927 18870 19053 19376 19300 19806 19350 19957 20073 20264 20233 20158 20845 21035 21287 21409 21570 21172 21817 22212 22583 22778 22399 23381 23449 23921 24259 24480 25141 25204 25929 26322 26352 27833 27606 27874 28121 28486 29185 30227 29895 30842 30670 31508 31794 31616 32215 32724 32061 33218 34789 36574 36879 38181 37302 37101 38380 38667 41836 44951 45601 44806 44356 45319 47977 48065 50990 52268 51439 54150 57542 61182 67320 74175 70339 76261 65646 51593 54918 50178 50036 47310 44524 39475 32437 27083 28290 28231 26635 26760 27043 27792 26086 26277 26241 26888 26928 24498 22093 22386 21467 21560 21561 21469 20890 18655 17181 17099 16900 16476 15936 15243 13440 11387 10791 11907 12058 11483 10037 9511 9708 9506 9312 9565 8932 6838 5345 4980 4635 3914 4277 3674 3298 2110 1116 939 837 955 1106 1785 1267 948 665 107 82 57 53 52 52 48 21 19 23 17 21 13 14 15 11 3 0 4 4 2 2 0 0 1
  NoData Value=-999
  Unit Type: h
  Offset: 0,   Scale:0.00100000004749745

My understanding is that kerchunk cannot handle this, i.e. cannot edit this attribute. What can/should be done in this case?  Edit the NetCDF attributes before kerchunking ? Also curious on why this difference, but I guess I should ask the producer.

@NikosAlexandris , if you would like the challenge, you could try to follow the example in https://github.com/fsspec/kerchunk/pull/374 to create a "var-chunk" zarr array, which is based on my PR https://github.com/zarr-developers/zarr-python/pull/1483 and allows working around the current limitations of zarr.

Also curious on why this difference, but I guess I should ask the producer.

Some communication from the producer :

From: CMSAF Contact <Contact.Cmsaf@dwd.de>
Sent: Tuesday, October 31, 2023 9:27 AM
To: ALEXANDRIS Nikos
Subject: AW: Different band block sizes in SDU, SARAH3 between years
Dear Nikos Alexandris,
Thank you for your valuable hints.
Our way forward would likely be, to first check the status of the chunk-sizes throughout the data records and then to inform the users.
If you need any assistance to work with the data, please let us know.
Best regards,
The CM SAF Operation Team

and

From: CMSAF Contact <Contact.Cmsaf@dwd.de>
Sent: Monday, October 30, 2023 2:08 PM
..
Dear Nikos Alexandris,
Indeed the chunk-sizes of the NetCDF files between the CDR (until 2020) and the ICDR (2021 onwards) are different. Thank you for pointing us towards this. The difference is not intentionally. With the tools and tests we undertook to the data, this has not shown up. The probable reason for the difference is that the slightly different CDR and ICDR processing environments must have used different default values for the chunk-size at some point.
We apologize for any inconvenience and we noted this to be checked additionally in future releases. We hope that working with the data is still feasible and that the data useful for your applications.

..
For the records :

Von: ALEXANDRIS Nikos <Nikos.Alexandris@ec.europa.eu>
Gesendet: Freitag, 27. Oktober 2023 14:50
An: CMSAF Contact <Contact.Cmsaf@dwd.de>
Betreff: Different band block sizes in SDU, SARAH3 between years
Dear DWD
I found out that, indeed, some source NetCDF files from DOI:10.5676/EUM_SAF_CM/SARAH/V003, differ in their Block definition (as per GDAL) for example between year 2020 (2600 x 1) and years 2021, 2022 (2600 x 2600) :
❯ gdalinfo NETCDF:"SDUds202001010000004231000101MA.nc":SDU |grep Block  # First day
Band 1 Block=2600x1 Type=Int16, ColorInterp=Undefined

❯ gdalinfo NETCDF:"SDUds202012310000004231000101MA.nc":SDU |grep Block  # Last day
Band 1 Block=2600x1 Type=Int16, ColorInterp=Undefined

❯ gdalinfo NETCDF:"SDUds2021010100000042310001I1MA.nc":SDU |grep Block
Band 1 Block=2600x2600 Type=Int16, ColorInterp=Undefined

❯ gdalinfo NETCDF:"SDUds2022010100000042310001I1MA.nc":SDU |grep Block
Band 1 Block=2600x2600 Type=Int16, ColorInterp=Undefined

I see nothing, and wouldn't expect, different in their structure :
❯ gdalinfo NETCDF:"SDUds202001010000004231000101MA.nc":SDU -nomd -mm -hist
Driver: netCDF/Network Common Data Format
Files: SDUds202001010000004231000101MA.nc
Size is 2600, 2600
Origin = (-64.999998473533992,64.999998473533992)
Pixel Size = (0.049999998825795,-0.049999998825795)
Corner Coordinates:
Upper Left  ( -64.9999985,  64.9999985)
Lower Left  ( -64.9999985, -64.9999985)
Upper Right (  64.9999985,  64.9999985)
Lower Right (  64.9999985, -64.9999985)
Center      (   0.0000000,   0.0000000)
Band 1 Block=2600x1 Type=Int16, ColorInterp=Undefined
   Computed Min/Max=0.000,16826.000
0...10...20...30...40...50...60...70...80...90...100 - done.
 256 buckets from -31.2686 to 15978.3:
 467342 40261 52663 .. 12 6 87
 NoData Value=-999
 Unit Type: h
 Offset: 0,   Scale:0.00100000004749745

``` bash
❯ gdalinfo NETCDF:"SDUds2022010100000042310001I1MA.nc":SDU -nomd -mm -hist
Driver: netCDF/Network Common Data Format
Files: SDUds2022010100000042310001I1MA.nc
Size is 2600, 2600
Origin = (-64.999998473533992,64.999998473533992)
Pixel Size = (0.049999998825795,-0.049999998825795)
Corner Coordinates:
Upper Left  ( -64.9999985,  64.9999985)
Lower Left  ( -64.9999985, -64.9999985)
Upper Right (  64.9999985,  64.9999985)
Lower Right (  64.9999985, -64.9999985)
Center      (   0.0000000,   0.0000000)
Band 1 Block=2600x2600 Type=Int16, ColorInterp=Undefined
 Min=0.000 Max=15505.000   Computed Min/Max=0.000,15505.000
 Minimum=0.000, Maximum=15505.000, Mean=6665.472, StdDev=3824.519
 256 buckets from -30.402 to 15535.4:
 365805 32375 43754 .. 0 0 1
 NoData Value=-999
 Unit Type: h
 Offset: 0,   Scale:0.00100000004749745

Curious on why this difference. This poses problems in tools that deal with scanning the metadata and rely on the defined “chunk sizes” to build then a continuous time series.
..

and

Von: ALEXANDRIS Nikos
Gesendet: Montag, 30. Oktober 2023 16:57
..
Please have a look at other products too. There is some [1, 2600, 2600] layout (which seems to be a reasonable default) alternating with [1, 1300, 1300] in SIS/SID products.
The data need some re-chunking which is some costly process, as you well know.
Question : may I share your reply publicly, i.e. in relevant discussions for software that deal with this kind of data?
..

The difference is not intentional

I imagine this happens a lot in practice

We rechunked era5 using rechunker and every chunk is uniform except that last one which has fewer time steps (which should be fine by Zarr v2).   We still get this error when MultiZarrToZarr tried to combine them - is this really expected?
 MultiZarrToZarr(
%%time
_ = MultiZarrToZarr(
        ref_list,
        remote_protocol="abfs",
        remote_options=opts,
        concat_dims=["time"],
        coo_map={"time": "cf:time"},
        identical_dims=identical_dims,
        out=out).translate()
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
File <timed exec>:8

File /anaconda/envs/pangeo-local/lib/python3.12/site-packages/kerchunk/combine.py:604, in MultiZarrToZarr.translate(self, filename, storage_options)
    602     self.store_coords()
    603 if 3 not in self.done:
--> 604     self.second_pass()
    605 if 4 not in self.done:
    606     if self.postprocess is not None:

File /anaconda/envs/pangeo-local/lib/python3.12/site-packages/kerchunk/combine.py:503, in MultiZarrToZarr.second_pass(self)
    501     chunk_sizes[v] = zarray["chunks"]
    502 elif chunk_sizes[v] != zarray["chunks"]:
--> 503     raise ValueError(
    504         f"""Found chunk size mismatch:
    505         at prefix {v} in iteration {i} (file {self._paths[i]})
    506         new chunk: {chunk_sizes[v]}
    507         chunks so far: {zarray["chunks"]}"""
    508     )
    509 chunks = chunk_sizes[v]
    510 zattrs = ujson.loads(m.get(f"{v}/.zattrs", "{}"))

ValueError: Found chunk size mismatch:
                        at prefix 10m_u_component_of_wind in iteration 18 (file None)
                        new chunk: [34560, 20, 20]
                        chunks so far: [17832, 20, 20]

If the last chunk of each input along a concat dimension is truncated, then you have a problem - these would become partial chunks not-at-the-end in the combined dataset. That was why we needed variable chunks. I think vzarr does cope with this situation, though ( @TomNicholas ).
Were you saying that only the last chunk of the last dataset is shorter? Then this ought to be workable, but the logic to get it right may need to be written.

Yes, only the last chunk (the 18th) is shorter!  The other 17 are identical.

I think vzarr does cope with this situation, though

No it doesn't, because that would be outside the (current) zarr model. https://github.com/zarr-developers/VirtualiZarr/issues/12 tracks this.
VirtualiZarr will also currently error if you try to concatenate e.g. 3 arrays with chunks (2,), (2,), (1,), even though that would be allowed in zarr's current model. This could be relaxed to accommodate @rsignell 's case though. See the check here

OK, I'll look to make this work within MZZ. I think it's probably only the check that's incorrect.

We rechunked ERA5 using rechunker into 18 zarr datasets. Each dataset has 360_24_4 hours of data and 20x20 in lon/lat, except the last dataset, which is shorter. So every chunk is uniform except the last one in the time dimension, which should be fine by Zarr v2, right? Yet we still get this error when using MultiZarrToZarr to combine them. This isn't expected behavior, is it?
_ = MultiZarrToZarr(
        ref_list,
        remote_protocol="abfs",
        remote_options=opts,
        concat_dims=["time"],
        coo_map={"time": "cf:time"},
        identical_dims=identical_dims,
        out=out).translate()
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
File <timed exec>:8

File /anaconda/envs/pangeo-local/lib/python3.12/site-packages/kerchunk/combine.py:604, in MultiZarrToZarr.translate(self, filename, storage_options)
    602     self.store_coords()
    603 if 3 not in self.done:
--> 604     self.second_pass()
    605 if 4 not in self.done:
    606     if self.postprocess is not None:

File /anaconda/envs/pangeo-local/lib/python3.12/site-packages/kerchunk/combine.py:503, in MultiZarrToZarr.second_pass(self)
    501     chunk_sizes[v] = zarray["chunks"]
    502 elif chunk_sizes[v] != zarray["chunks"]:
--> 503     raise ValueError(
    504         f"""Found chunk size mismatch:
    505         at prefix {v} in iteration {i} (file {self._paths[i]})
    506         new chunk: {chunk_sizes[v]}
    507         chunks so far: {zarray["chunks"]}"""
    508     )
    509 chunks = chunk_sizes[v]
    510 zattrs = ujson.loads(m.get(f"{v}/.zattrs", "{}"))

ValueError: Found chunk size mismatch:
                        at prefix 10m_u_component_of_wind in iteration 18 (file None)
                        new chunk: [34560, 20, 20]
                        chunks so far: [17832, 20, 20]

Hello @rsignell : I am restarting working on data re-chunking these days, including global ERA5 datasets (temperature and windspeed).  I have left an effort to identify the "optimal" chunking shape for my use-case (on-the-fly reading and processing location-specific time series) in the middle due to other priorities.  Sooner or later will come back to this or other similar threads.
One observation : only nccopy really worked for me to rechunk data in NetCDF files.
One question : why 20 x 20 in space ?  What is the use-case behind this shape ?

source: gharchive/issue

authors: NikosAlexandris TomNicholas agrigoriev martindurant peterm790 rsignell

repo: fsspec/kerchunk

license: MIT

license_type: permissive

license_source: github-api


text:
Update sbt-scalajs to 1.4.0

Updates org.scala-js:sbt-scalajs from 1.3.1 to 1.4.0.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.scala-js", artifactId = "sbt-scalajs" } ]

labels: sbt-plugin-update, semver-minor

Superseded by #145.

source: gharchive/pull-request

repo: fthomas/status-page


text:
Sliders don't stop moving on 'mouseUp'

Hello there! Really nice Plugin!!
There's a major issue in the current release. When you move any of the sliders, then you stop moving it, (mouseUp) the sliders don't stop moving.
I then checked the older version, dated back to  2013.12.18, and there isn't an error there. Source.
Here's a JSFiddle with the current version, and it has the error. The console says:
"Uncaught TypeError: Cannot read property 'blur' of undefined"

Hi Horray, thanks for the feedback! I can confirm that it happens when using jQuery 1.8.3 or later. With jQuery 1.7.2 the issue does not happen. Will look into it for possible deprecated jquery functions used by the plugin.

The following code is deprecated:
$this.data('jQWCP.settings')

You could emulate that if you have control over each place you're setting an event handler. Every time you call $(elem).blur(handler) or more generally $(elem).someevent(handler) or $(elem).on("event", handler), then you can register that handler (i.e. save a reference) in your own object like events[event] = handler

Fixed at commit  6fbb6c6. The code snippet was intended to temporarily unbind blur events set by user while moving slider, because moving slider will blur the input element. For internal use by the plugin, event handlers are already known so it's not a problem.
For now I just disabled this feature when using jquery 1.8+ (data('event') is undefined) and update the documentation.
Thanks!

Fixed at commit  6fbb6c6. The code snippet was intended to temporarily unbind blur events set by user while moving slider, because moving slider will blur the input element. For internal use by the plugin, event handlers are already known so it's not a problem.
For now I just disabled this feature when using jquery 1.8+ (data('event') is undefined) and update the documentation.
Thanks!

Hi, I am using jQuery version 2.1.4 and colorwheelpicker 2.5.1,
I belive I am still facing this issue, when the sliders are moved up or down  till the very end , they still keep moving even if I dont drag. Any solution?

authors: Horray abhi06991 fujaru

repo: fujaru/jquery-wheelcolorpicker

license: mit

license_source: bigquery


text:
Allow using fixed target port for TCP

Currently, for TCP probing, the source port is fixed and the target ports rotations (to track the sequence).  The default for TCP should be the reverse.
Depends on #43

Closed in #134


text:
足し算の修正

1+ -2 = -1が正常に動作
変更後の振る舞い

〜したら
〜になる

変更前の振る舞い

〜したら
〜になる

ok


text:
Create and Advanced Minimalist Fulcro Tutorial

Create and Advanced Minimalist Fulcro Tutorial that goes into routing, form state, UISM and other non-essential but very useful topics.
The work is now under way in the advanced-min-fulcro branch and you can subscribe to this issue to get informed of updates.

Update: Finished Troubleshooting and UI x Data divergence sections. The first rough draft of routing is almost done.

repo: fulcro-community/guides

license: Unlicense


text:
🛑 NGR System Status is down

In ca3d3e1, NGR System Status (https://my.ngr.com.au/api/v2/web_service/service_test) was down:

HTTP code: 0
Response time: 0 ms

Resolved: NGR System Status is back up in 36d0fe4.

repo: fullprofile/agridigital-status-monitor


text:
[3.x] Adds support for Laravel >=11.33.0, Fixes #81

Upgrade modeltyper to V3.x
The problem with previous version comes from laravel 11.33 extracting functionalities from ShowModelCommand to a external class Illuminate\Database\Eloquent\ModelInspector (see #81).
Changes
This pull request drops support for Laravel <11.0 and php 8.1, updates dependencies and include php 8.4 in testbench.
Now, instead of extending ShowModelCommand, modeltyper implements ModelInspector directly, and drops its internal ShowModelCommand.
Dependencies have been updated, larastan v3, orchestra v9, make sure phpstan and test pass (there where a lot of tests that where not implemented, those were updated).
Additions

Added more test, all command flags are now tested.
Added new command artisan model:typer-mappings, it returns a table with current mappings.
Added more config options, to enable to run artisan model:typer and get the data exactly how you want it.
Added option to enable model:typer to run after migrating, disabled by default.
Added UPGRADE.md

Removed

Remove flag --all, in description it implied it "Enable all output options", in reality only 2 (--plurals --api-resources),
Remove flags --resolve-abstract and --throw-exceptions, these are no longer needed.
Remove spatie/laravel-ray as it is incompatible at this time with larastan@v3.

Task list

[x] Update Dependencies, PHP 8.2
[x] Update workflows
[x] Remove deprecated ModelInterface and TypescriptInterface
[x] Update skeleton to default orchestra v9 skeleton
[x] Update expectations for new skeleton
[x] Fix failing tests, remove deprecated phpunit annotations
[x] Change composer.json conflict from ">=11.33.0" to "<11.33.0" and update dependencies
[x] Check if ShowModelCommand can be updated (it can't)
[x] Implement ModelInspector and remove ShowModelCommand
[x] Update config, to allow set project defaults
[x] Check phpstan/test pass
[x] Add migrations Docs
[x] Update readme

Optional
I would like some other improvements for V3, if ok, like:

migrate to Pest (currently the recomended testing framework in spatie/package-skeleton-laravel)
Add Pests Arch testing

Added new class RunModelInspector, runs an extended FumeApp\ModelTyper\Overrides\ModelInspector that in constructor updates the protected $relationMethods array, with those defined in config, but there are no test for it

I'm sorry for so many changes, I know that reviewing so many modifications is quite tedious, let me know if you need me to clarify anything.

I'm sorry for so many changes, I know that reviewing so many modifications is quite tedious, let me know if you need me to clarify anything.

All good, much appreciated! I'll be sure to review this when i can asap thanks again

authors: ayrtonandino tcampbPPU

repo: fumeapp/modeltyper


text:
Why have to implement MS_Deform_Attn in cuda?

You have written "ms_deform_attn_core_pytorch(value, value_spatial_shapes, sampling_locations, attention_weights)" in pure pytorch. Why not use this pytorch version?

See #9.

Noted with thanks.

repo: fundamentalvision/Deformable-DETR


text:
Illegal instruction when starting urbackupsrv

I successfully build urbackupsrv on machine and it runs like a charm.
When I install the same package on a different machine, urbackupsrv stops with an error 'Illegal instruction'
Screenshot:
[root@odin urbackup]# /usr/bin/urbackupsrv run --config /etc/default/urbackupsrv
2017-10-22 12:41:46: Starting HTTP-Server on port 55414
2017-10-22 12:41:46: HTTP: Server started up successfully!
2017-10-22 12:41:46: SQLite: recovered 4 frames from WAL file /var/urbackup/backup_server.db-wal code: 283
2017-10-22 12:41:46: SQLite: recovered 8 frames from WAL file /var/urbackup/backup_server_link_journal.db-wal code: 283
2017-10-22 12:41:46: SQLite: recovered 4 frames from WAL file /var/urbackup/backup_server.db-wal code: 283
2017-10-22 12:41:46: SQLite: recovered 8 frames from WAL file /var/urbackup/backup_server_link_journal.db-wal code: 283
2017-10-22 12:41:46: Started UrBackup...
2017-10-22 12:41:46: Testing if backup destination can handle subvolumes and snapshots...
Backupfolder not set
2017-10-22 12:41:47: Backup destination cannot handle subvolumes and snapshots. Snapshots disabled.
2017-10-22 12:41:47: Testing if backup destination can handle filesystem transactions...
Backupfolder not set
2017-10-22 12:41:47: Image mounting disabled: Backupfolder not set
2017-10-22 12:41:47: WARNING: Backupfolder "C:\urbackup" does not exist. Not running recovery.
2017-10-22 12:41:47: Binding to interface enp0s13 for broadcasting...
2017-10-22 12:41:47: Broadcasting on interface IP 192.168.12.2
2017-10-22 12:41:47: UrBackup Server start up complete.
2017-10-22 12:41:47: Server started up successfully!
2017-10-22 12:41:47: Looking for old Sessions... 0 sessions
Illegal instruction

In /var/log/urbackup.log:
2017-10-22 12:41:46: Starting HTTP-Server on port 55414
2017-10-22 12:41:46: HTTP: Server started up successfully!
2017-10-22 12:41:46: SQLite: recovered 4 frames from WAL file /var/urbackup/backup_server.db-wal code: 283
2017-10-22 12:41:46: SQLite: recovered 8 frames from WAL file /var/urbackup/backup_server_link_journal.db-wal code: 283
2017-10-22 12:41:46: SQLite: recovered 4 frames from WAL file /var/urbackup/backup_server.db-wal code: 283
2017-10-22 12:41:46: SQLite: recovered 8 frames from WAL file /var/urbackup/backup_server_link_journal.db-wal code: 283
2017-10-22 12:41:46: Started UrBackup...
2017-10-22 12:41:46: Testing if backup destination can handle subvolumes and snapshots...
2017-10-22 12:41:47: Backup destination cannot handle subvolumes and snapshots. Snapshots disabled.
2017-10-22 12:41:47: Testing if backup destination can handle filesystem transactions...
2017-10-22 12:41:47: Image mounting disabled: Backupfolder not set
2017-10-22 12:41:47: WARNING: Backupfolder "C:\urbackup" does not exist. Not running recovery.
2017-10-22 12:41:47: Binding to interface enp0s13 for broadcasting...
2017-10-22 12:41:47: Broadcasting on interface IP 192.168.12.2
2017-10-22 12:41:47: UrBackup Server start up complete.
2017-10-22 12:41:47: Server started up successfully!
2017-10-22 12:41:47: Looking for old Sessions... 0 sessions

Build was done on : Linux zeus 4.12.12-1-ARCH #1 SMP PREEMPT Sun Sep 10 10:03:08 CEST 2017 i686 GNU/Linux
Error was on : Linux odin 4.13.7-1-ARCH #1 SMP PREEMPT Tue Oct 17 15:16:10 CEST 2017 i686 GNU/Linux
When I try to build it on th errormachine the file urbackupsrv was not created
I miss something and can't find the clue
Help would be very appreciated

Hello there,  I don't know how you configured your installation but it's clear that

WARNING: Backupfolder "C:\urbackup" does not exist. Not running recovery.

So for me it's clear that the path does not exist ...
Cheers,
N. Chababy

It's running on a linux machine.
Op 23 oktober 2017 13:32:21 +02:00, schreef Nissar Chababy notifications@github.com:

Hello there, I don't know how you configured your installation but it's clear that

WARNING: Backupfolder "C:\urbackup" does not exist. Not running recovery.
So for me it's clear that the path does not exist ...

Cheers,
N. Chababy
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/funilrys/PKGBUILD/issues/9#issuecomment-338630163, or mute the thread https://github.com/notifications/unsubscribe-auth/ATQG_ZolXezLznp7Cu2F_2-B0uacdHzsks5svHlFgaJpZM4QB5vJ.

That's weird ... Gonna check that out once home ...

On the urbackup forum a guy orogor wrote "As i remember on arch there s an issue with the arch compile options. You need to set the cpu the good flags for urbackup and maybe the crypto lib."
I rebuild it completly to capture the output. It can be found http://haezeleer.be/archlinux/build_urbackup_i686.log

I ran the command to compile urbackupsrv seperatly and it compiled without problem.
I copied the line from the logfile of previous 'makepkg -s'
g++ -pthread -DLINUX -DSTATIC_PLUGIN -DWITH_FUSEPLUGIN -DVARDIR='"/var"' -DBINDIR='"/usr/bin"' -DDATADIR='"/usr/share"'   -march=native -O2 -pipe -fstack-protector-strong -ansi
-Wno-deprecated-declarations  -L/usr/lib -Wl,-z,relro -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -L/usr/lib -L/usr/lib -o urbackupsrv ......

authors: bhaezlr funilrys

repo: funilrys/PKGBUILD


text:
fix(server.js): prevent errors from _next/ route

fix #71, fix #72
This prevents the errors in the console from accumulating and enables HMR again while keeping the universal route.
This fix seems like a band-aid, if anyone knows of the right way to handle this, please speak up.

Only call renderAndCache if the :type parameter isn't _next
call next() if it is, to pass the request off to the next function in line (which handles it correctly, the * route to be specific)

edit: working on rework of server.js, to get things working again in meantime remove universal route.

actually fixed it while keeping universal route working.
Also refactored the universal route to use a translation object.

@NickyMeuleman this is brilliant! You are a genius! Thank you!

authors: JesseRWeigel NickyMeuleman

repo: fus-marcom/franciscan-react


text:
Questions About Memory Addresses and Functions in Boot Segment

# Description of a successful boot process of the firmware image
boot:
  # A list of addresses required for a successful boot
  required:
    # An address (or symbol) in this list may indicate the if/else branch of a positive check
    - 0x0800052A
    # Or a function which activates a peripheral which is only called in case all checks were successful
    - activate_uart_peripheral
  # A list of addresses which indicate a failed boot
  avoid:
    # if/else branch of a failed check
    - 0x08000518
    # an error output function logging an error condition
    - log_error
  # Address at which the firmware is considered booted (successfully or unsuccessfully based on the previous config attributes)
  target: idle
# This configuration is used to derive a firmware state which is considered to be booted successfully and continue fuzzing from here

Hi there. I am working on understanding memory addresses and functions in the boot segment and have some questions to ensure I am on the right track.

In the required section, we mention addresses and functions. Does address mean that it tells fuzzware which branch to take to avoid errors? For example, if jumping to LABEL in a bne <LABEL> instruction causes a dead loop that fails to start, and not jumping continues correct execution, does the address help fuzzware to find the right jump target in the boot segment?
I’m not clear why we need to specify certain functions. Could you provide an example to help me understand?

The fuzzware documentation is great, but I need a bit more help to fully grasp these points. Thanks in advance for your assistance!

Hi LittleNewton,
yes, the required addresses indicate all the basic block addresses that need to be visited so that the boot process is considered valid. An example could be: You want the firmware during boot to initialize the network stack successfully, so the function init_network needs to be called during boot. In this case, you would include init_network as part of the required function. Within the init_network function, there may also be a function network_init_finalize which is called at the very end of network initialization where you know the full network initialization has been performed correctly, without errors. In that case, you could also add network_init_finalize to the required list.
Best
Tobi

authors: LittleNewton Scepticz

repo: fuzzware-fuzzer/fuzzware-emulator


text:
Rainmeter crashing on load

Hi first of all thanks for developing this Plugin.
I compiled it using the latest VS version and copied the files after your instructions, rainmeter crashes with the following error when I try to use the code shown below.
Error:
Anwendung: Rainmeter.exe
Frameworkversion: v4.0.30319
Beschreibung: Der Prozess wurde aufgrund einer unbehandelten Ausnahme beendet.
Ausnahmeinformationen: System.Net.Sockets.SocketException
bei System.Net.Sockets.Socket.DoConnect(System.Net.EndPoint, System.Net.SocketAddress)
bei System.Net.Sockets.Socket.Connect(System.Net.EndPoint)
bei uPLibrary.Networking.M2Mqtt.MqttNetworkChannel.Connect()
bei uPLibrary.Networking.M2Mqtt.MqttClient.Connect(System.String, System.String, System.String, Boolean, Byte, Boolean, System.String, System.String, Boolean, UInt16)
Ausnahmeinformationen: uPLibrary.Networking.M2Mqtt.Exceptions.MqttConnectionException
bei uPLibrary.Networking.M2Mqtt.MqttClient.Connect(System.String, System.String, System.String, Boolean, Byte, Boolean, System.String, System.String, Boolean, UInt16)
bei uPLibrary.Networking.M2Mqtt.MqttClient.Connect(System.String, System.String, System.String)
bei MqttPlugin.ParentMeasure.Reload(Rainmeter.API, Double ByRef)
bei MqttPlugin.Plugin.Reload(IntPtr, IntPtr, Double ByRef)
Code:
[Rainmeter]
Update=1
BackgroundMode=2
SolidColor=000000
[mqttServer]
Measure=Plugin
Plugin=MqttPlugin.dll
Server=farmer.cloudmqtt.com
Username=*******
Password=*******
Port=16750
[mqttTopic1]
Measure=Plugin
Plugin=MqttPlugin.dll
ParentName=mqttServer
Topic=thermometer/data
[Text]
Meter=STRING
MeasureName=mqttServer
MeasureName2=mqttTopic1
X=5
Y=5
W=300
H=100
FontColor=FFFFFF
FontSize=24
Text="mqttServer: %1#CRLF#mqttTopic1: %2"

Hi @thorambar, thanks for your bug report.
I believe this is caused by the use of a non-standard port and/or lack of error-checking in the current version of the MqttPlugin. I'm testing on a local client with the default port... So, just now I updated the v0.0.2 branch with a some better error handling for connection failures:

In v0.0.2 there is support for the Port parameter, v0.0.1 (current master) did not.
Added better error handling for connection issues and verbosity in the Rainmeter Log.

Can you please pull the new v0.0.2 version and recompile? Only the MqttPlugin.dll needs to be updated. If you still see the error, please report back with the versions of Windows, .NET and Rainmeter you are using.
Hope this helps, let me know if it does or doesn't. I appreciate you taking the time to test the plug-in!

Tried v0.0.2 worked without perfectly without any other changes.
The update interval was so low from me playing around, but yeah 1ms is silly.
Thanks again for your work this plugin is exactly what I was looking for and for taking the time to reply.

No problem, thanks for confirming the v0.0.2 update (f89470f) works. Glad to help out.

authors: fvanroie thorambar

repo: fvanroie/MqttPlugin


text:
🛑 Importer STAI is down

In d909aa3, Importer STAI (https://importerstai.darunnajah.ac.id) was down:

HTTP code: 502
Response time: 353 ms

Resolved: Importer STAI is back up in 75a5c09 after 17 minutes.


text:
Doesn't appear to support state changes such as re-opened

Hi there, love that this now exists -- I've been wanting a youtrack integration forever but never got around to implementing my own. There doesn't appear to be a workflow trigger when state changes, such as resolved - re-opened. Is this fixable or a limitation of the workflow API?

I wouldn't call this unintended behaviour, just simply missed out from implementation due to the project's needs at the time.
I believe this is something that can be easily added, but we could do with the refactor (#2) before implementing this.

authors: ParadauxIO TomDotBat

repo: g-karts/youtrack-discord-webhook


text:
Notofications

Does this include the notification part when the app is in background? @g0g0l @Bibaswann

Hi, Can I use part of your code in my final year project please ?? I can not find any licensing on it, thats why I am asking.

You are free to use/modify any part of code

Hi, please can you tell me how can I get the message even if my app is in the background?

I have implemented notification using JobScheduler. It might have some bugs as I didn't have time to test it thoroughly, but you will get an idea


text:
so so assessment

still working out where things go... had trouble with migration file

feeling better now, seems like im getting it, but still working on details


text:
Disambiguate organizations for workflows

Feature Request
Desired behaviour
Something to work on while redesigning workflows.
Right now, workflows are "first one wins" when two workflows have the same name and organization, but different registries. We have a bit of a window to fix this to be more like tools.

While reviewing endpoints, we also noticed that Docker registries are returned, which implies that the UI will automatically adjust to new Docker registries. But we're probably not doing the same with source control

Migration plan:
Add an issue for how to review affected workflows before migration and create a redirect when SC is not specified (probably github.com)
Note that we can either wipe out the existing workflow threads on Discourse or use the API to fix them to the new URLs

The webservice portion has been implemented and merged. The UI has been mostly completed.

This is done, I think you mentioned to Gary to write a test.

This seems functional. Follow-up with smaller issues if needed, noticed #1245


text:
Add text that distinguishes the two serialization types

See https://docs.google.com/document/d/1TkgJMttPjKuAwc86B5x6jWw6HP-6waJyZmi7LFq5oYs/edit#heading=h.233cq9yos1lr

Done.


text:
”musicgen-small“  the path to the model file？

Hello, I downloaded the WIN version, it automatically downloads the musicgen-small model file, in the C drive letter in what folder? What is the file name? I searched "musicgen" and couldn't find it

In windows, all the files that the app might download land in
C:\Users\Your User\AppData\Roaming\gabotechs\musicgpt\data

thx~

authors: aimarxjg gabotechs

repo: gabotechs/MusicGPT


text:
Isomorphic react support

I'm trying to use it in my isomorphic react project. But I got a error when I require a jsx file.
locals[0] does not appear to be a `module` object with Hot Module replacement API enabled. You should disable react-transform-hmr in production by using `env` section in Babel configuration. See the example in README: https://github.com/gaearon/react-transform-hmr

Error: locals[0] does not appear to be a `module` object with Hot Module replacement API enabled. You should disable react-transform-hmr in production by using `env` section in Babel configuration. See the example in README: https://github.com/gaearon/react-transform-hmr
    at proxyReactComponents (/Users/xujie/Documents/Github/FTravel/node_modules/react-transform-hmr/lib/index.js:51:11)
    at Object.<anonymous> (/Users/xujie/Documents/Github/FTravel/public/js/components/Root.jsx:47:59)
    at Module._compile (module.js:425:26)
    at loader (/Users/xujie/Documents/Github/FTravel/node_modules/babel-register/lib/node.js:130:5)
    at Object.require.extensions.(anonymous function) [as .jsx] (/Users/xujie/Documents/Github/FTravel/node_modules/babel-register/lib/node.js:140:7)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:311:12)
    at Module.require (module.js:366:17)
    at require (module.js:385:17)
    at Object.<anonymous> (app.js:5:20)

Seems like, babel cant figure out whether the file is required in the server side or not. So it apply the react-transform-hmr on every jsx file.
Any workarounds to solve my problem?

What are the contents of your .babelrc and npm scripts?

.babelrc:
{
  "presets": ["react", "es2015"],
  "env": {
    "development": {
      "plugins": [
        ["react-transform", {
          "transforms": [{
            "transform": "react-transform-hmr",
            "imports": ["react"],
            "locals": ["module"]
          }, {
            "transform": "react-transform-catch-errors",
            "imports": ["react", "redbox-react"]
          }]
        }]
      ]
    }
  }
}

npm scripts:
"scripts": {
    "start": "node ./bin/www",
    "test": "mocha"
  }

I just set up an express server with a webpack-dev-middleware in the start script.
I built the configuration based on gaearon's react-transform-boilerplate.

not use babel-plugin-react-transform in .babelrc. use config in webpack.config.js
example
module: {
    loaders: [
      {
        test: /\.(js|jsx)$/,
        loader: 'babel',
        include: path.join(__dirname, 'src'),
        query: {
          plugins: [
            ["react-transform", {
              transforms: [{
                transform: "react-transform-hmr",
                imports: ["react"],
                locals: ["module"]
              }, {
                "transform": "react-transform-catch-errors",
                "imports": ["react", "redbox-react"]
              }]
            }]
          ]
        }
      }
    ]
  }

it is support in isomorphic apps (y)

@carloscuatin you are brilliant! It works.

@carloscuatin thanks! got me up and running too :+1:

Seems like, babel cant figure out whether the file is required in the server side or not. So it apply the react-transform-hmr on every jsx file.

Babel only checks the NODE_ENV. Look at your .babelrc:
{
  "presets": ["react", "es2015"],
  "env": {
    "development": {
      ...

This means the plugin activates when NODE_ENV is equal to development or absent because this is the default.
So the solution is to tell Babel to use another configuration on the server by specifying NODE_ENV.
# any of these will do!

NODE_ENV=production node app.js
NODE_ENV=server node app.js
NODE_ENV=server-development node app.js
NODE_ENV=whatever node app.js

# but not this!
node app.js

We already say this in README but sadly people look over it. :-(

// this plugin will be included only in development mode, e.g.
// if NODE_ENV (or BABEL_ENV) environment variable is not set
// or is equal to "development"

删除(delete)  .bablerc => env =>"development": {"presets": ["react-hmre"]}
{ 	"presets": ["react","es2015", "stage-0"], 	"plugins": [], 	"env": {} }

authors: Dakuan Deathsteps carloscuatin gaearon olegakbarov webxzy

repo: gaearon/babel-plugin-react-transform


text:
Replace lodash with lodash. deps

Close #516

The CI task exitted correctly, however, there is still some problem with the build. Let me know if you're actually interested in this and I'll update it.

I'm 👎 on this. While the Lodash install size on disk is large, react-dnd uses modular imports (e.g. lodash/shuffle) to pull in only the parts that it needs, which keeps the bundle small.
The proposed change would actually increase the bundle size, since as of Lodash 4.14.0, the individual packages are zero-dependency modules and therefore, unlike modular imports, do not share any code, despite much of it being duplicate (see the changelog). Moreover, the proposed change prevents apps which use both Lodash and react-dnd from sharing Lodash code.
I'm all for decreasing install size, but not at the cost of bundle size.
FYI, the individual packages are on their way out in Lodash 5.0.0 in favor of modular imports, and there are plans to significantly decrease install size.

The individual packages being deprecated is a pretty compelling reason for me not to bring this in.
With 5.0 lodash is aiming to shave 50% off of install size, so maybe we could just rely on that solving people's install size woes?
Alternatively, we could look into other modules instead. (e.g. using native Array.isArray, array-shuffle, is-plain-object).

These are valid points.
I'm closing the PR then. 😉

authors: billyjanitsch jakubzitny kesne

repo: gaearon/react-dnd


text:
Globals

Add React components, sass file structure, and global variables.

Hey, thanks for all the effort. However, I think adding all of Sass/PostCSS/etc is a bit too opinionated for this project, since its purpose is to show a barebones example for using Webpack + React Hot Loader.

authors: calesce josephmallari

repo: gaearon/react-hot-boilerplate


text:
How to clean notification in IOS

I did rmxAudioPlayer.clearAllItems(); but notification remains. why? Can anyone help me?

Hi @dvasea1 ,
I think the plugin don't clear the notification yet, but it totally makes sense to do so.


text:
Abstract HTTP client and wrap errors

Hey,
This PR abstracts HTTP client so that consumer can use any decorated HTTP client. In my case the decorator provides OAuth/JWT authentication for the private Solana validator. I know that this technically can be done by decorating the RoundTripper but the doc in standard library strongly suggest to not modify request in RoundTrip method. It also can't be done just by providing custom headers because the JWT token must be refreshed time to time, so instead of using some sort of standard decorator consumers have to build a logic that will re-instantiate a new client when token expires.
There is also a bunch of places where response error isn't wrapped so it makes it hard to handle context deadline/cancelled errors in a specific way.
Hope this will make this package more flexible.

LGTM

Thanks!

authors: gagliardetto hexdigest

repo: gagliardetto/solana-go


text:
ContractMetadata Serialization/Deserialization failing on Multi-entry source files

Version
v1.0.2
Platform
linux x86_64
Description
Im using the ethers::etherscan lib to download etherscan verified source code, and am cacheing them to a location on disk to preserve to prevent future network requests. Ideally, Id prefer to preserve the response as the ContractMetadata struct using the Serialization and serde_json::to_string_pretty. The written file looks fantastic, but Im having trouble Deserializing it immediately after.
The below example uses the address from the contract.rs tests where the object matches the SourceCodeMetadata::Metadata { .. } enum
Ethers Test Code:
ethers-etherscan/src/contract.rs#L523-L539
let meta = client
    .contract_source_code("0x8d04a8c79cEB0889Bdd12acdF3Fa9D207eD3Ff63".parse().unwrap())
    .await
    .unwrap();

assert_eq!(meta.items.len(), 1);
assert!(matches!(meta.items[0].source_code, SourceCodeMetadata::Metadata { .. }));
let source_tree = meta.source_tree();
assert_eq!(source_tree.entries.len(), 15);

Code example Im running into issues with:
let client = Client::new(*ethers_chain, api_key).unwrap();

//https://api.etherscan.io/api?module=contract&action=getsourcecode&address=0x8d04a8c79cEB0889Bdd12acdF3Fa9D207eD3Ff63
let request_result = client
    .contract_source_code(address) // 0x8d04a8c79cEB0889Bdd12acdF3Fa9D207eD3Ff63
    .await.unwrap();

std::fs::write(
    format!("{}/{}-{:?}.json", cache_path, network, address),
    serde_json::to_string_pretty(&response).unwrap(),
).unwrap();
let file_contents = std::fs::read_to_string(&format!("{}/{}-{:?}.json", cache_path, network, address)).unwrap();
trace!("test - deserializing cache file into contract_metadata");
let contract_metadata: ContractMetadata = serde_json::from_str(&file_contents).unwrap(); // fails here

This is the panic output:
thread 'tokio-runtime-worker' panicked at 'called `Result::unwrap()` on an `Err` value: Error("invalid type: map, expected a string", line: 3, column: 18)'

This is the beginning of the Serialized file.
[
  {
    "SourceCode": {
      "language": "Solidity",
      "sources": {
        "@openzeppelin/contracts/security/ReentrancyGuard.sol": {
          "content": "// SPDX-License-Identifier: MIT\n\npragma solidit ..."
          },
      },
    }
  }
]

It should match ContractMetadata->Metadata->SourceCodeMetadata->Metadata enum as shown below:
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(untagged)]
pub enum SourceCodeMetadata {
    /// Contains metadata and path mapped source code.
    Metadata {
        /// Programming language of the sources.
        #[serde(default, skip_serializing_if = "Option::is_none")]
        language: Option<SourceCodeLanguage>,
        /// Source path => source code
        #[serde(default)]
        sources: HashMap<String, SourceCodeEntry>,
        /// Compiler settings, None if the language is not Solidity.
        #[serde(default, skip_serializing_if = "Option::is_none")]
        settings: Option<serde_json::Value>,
    },
    /// Contains only the source code.
    SourceCode(String),
}

Any idea what could be going wrong? Happy to experiment more

since this is a decoding issue, could please get the actual json response and compare it against the current model?
Then we can fix.

Etherscan's response's SourceCode field is always one big String, we parse that as a JSON object if it starts with 2 { characters and then deserialize into the actual objects, otherwise it is just the source code for one file;
When you serialize, it will use the default Serde derive impl so it will be serialized as a normal object, not a string;
When deserializing this last step, it will fail expecting a String

authors: DaniPopes mattsse plotchy

repo: gakonst/ethers-rs


text:
Add uWSGI wheel

This supercedes #95 and #114.

Jenkins? Hello Jenkins?
@martenson?

@galaxybot test this

@galaxybot test this

boom, shakalaka!

@natefoo a secret was missing from ghprb webhook; when I switched to using it I forgot about Starforge, sorry

@martenson thanks!
Now, why is it claiming failure? =l

maybe this sent error exit code?
Possible sdists missing, build failed?
Build step 'Execute shell' marked build as failure

@martenson indeed.
https://depot.galaxyproject.org/starforge/wheels/build-161/

uWSGI 2.0.15 has been released, meaning that we no longer need to use a development build of uWSGI 2.0. I updated the PR and the new wheels are now on wheels.galaxyproject.org.

Then let's get this in.

authors: bgruening martenson natefoo

repo: galaxyproject/starforge


text:
Hands-on Tutorial on Qiime (Metagenomics)

Hello,
I'm writing a tutorial based on this:
http://nbviewer.jupyter.org/github/biocore/qiime/blob/1.9.1/examples/ipynb/illumina_overview_tutorial.ipynb
Daniel

The biom suit of tools are not available. Would need at least the biom summarize-table for this tutorial.

The perform open-reference OTU picking tool generates an empty file which is necessary for the tutorial

I will work on the missing tools (in biom-format) in the next days
What is the generated empty file by pick_open_reference_otus?

It's the "OTU table with taxonomic assignment and without sequences failing the alignment to the OTU representative sequences". Basically the equivalent to otu_table_mc2_w_tax_no_pynast_failures.biom

@dsobral Are you still working on the tutorial? Do you need any help with it?

I have been busy with other stuff, but I plan to continue working on the tutorial. I'll check if the tools now allow me to complete the qiime overview tutorial.

By the way, are the new biom tools in the main toolshed? Thanks!

Sorry, I saw they are in the Test Toolshed.

We will write tutorials for QIIME2 as soon as it is ready. @blankenberg can you update us on the status of the qiime2 integration?

authors: bebatut dsobral erasche

repo: galaxyproject/training-material


text:
colour blind friendly diff colouring

@natefoo  https://github.com/galaxyproject/training-material/issues/2583#issuecomment-871492929 This is more or less the styling I use in my terminal (as a red-green impaired human) and it works quite well, and should be black-white colourblind friendly
[color "diff"]
	meta = bold
	frag = magenta bold
	old = strike reverse
	new = blue dim reverse

Awesome, this works very nicely even in monochrome 👍🏽

authors: hexylena matuskalas


text:
WIP: New Tutorial Variant Calling (shell commands)

This PR adds a new tutorial on Variant Calling, based on the Data Carpentry Wrangling Genomics lesson (https://datacarpentry.org/wrangling-genomics/04-variant_calling/index.html). It uses the shell provided by the RStudio interactive environment, using conda to install the necessary tools.
The tutorial has been developed in the context of the Gallantries project, by @Sofokli5 and @fpsom .

I'm not sure if it's interesting or not for you all, but you could annotate it as a bash notebook and then every code block annotated bash, could be automatically available in a jupyter notebook?

Also, because by accident the git lesson was merged in this PR, we have now removed the git tutorial files from here and started the new PR #3507 . We should have already incorporated the comments / suggestions you added here, but it felt ... cleaner to have the two tutorials as different PRs. :) Sorry for the mess. :/

https://github.com/BiodataAnalysisGroup/training-material/pull/4 PR here to fix the linting issues @fpsom :)

Ok, sorry to ask this, but can you rebase? I needed to include @Sofokli5 's information in the CONTRIBUTORS file to get the other PR working, and now that causes a git conflict unfortunately.

authors: Sofokli5 fpsom hexylena


text:
New training material - Ai4Life

Ai4Life Training Mateiral
Slides
Example Slide Deck
These slides have been removed.
BioImage Model Zoo: Advanced models in one-click
Contains the slides or training material to get into the BioimageModelZoo. Changed to appear on the GTN website automatically.
Welcome to the BioImage Model Zoo
New slides for getting to know the bioimage.io website.
Note: I can't get the video to be automatically generated from the slides. I've been testing it but I can't find what is wrong.
Others
Fix invalid path error in Git when checking out repository in Windows (":" is not an allowed character).

Note: I can't get the video to be automatically generated from the slides. I've been testing it but I can't find what is wrong.

how are you trying to do this? The process should be 100% automatic once it's merged into this repository, no action should be required (unless you just want to preview the video).

Note: I can't get the video to be automatically generated from the slides. I've been testing it but I can't find what is wrong.

how are you trying to do this? The process should be 100% automatic once it's merged into this repository, no action should be required (unless you just want to preview the video).

Hi! I just write the video: true at the beginning of the slides as indicated. I am using GitPod to preview it before doing the pull request and I can't see the videos. Thanks!

Hi! I just write the video: true at the beginning of the slides as indicated. I am using GitPod to preview it before doing the pull request and I can't see the videos. Thanks!

Ok, in that case it is expected that the video is not available :) Compiling the video takes 5-10 minutes, and requires access to our AWS keys to generate the audio components. Thus they are only generated for merged PRs.

@cfusterbarcelo videos are now live:

https://training.galaxyproject.org/videos/watch.html?v=ai4life/tutorials/welcome-bioimageio/slides
https://training.galaxyproject.org/videos/watch.html?v=ai4life/tutorials/getting-started/slides

Currently we randomly pick a voice every time the video is updated, but if there's one you find you like, we can ensure that the video always renders with that voice. (There's currently no "gallery" of the voices, I'll see if I can fix that tomorrow.)
if you want to re-upload to youtube or elsewhere you're of course welcome to :)

https://training.galaxyproject.org/videos/topics/ai4life/tutorials/getting-started/slides.mp4
https://training.galaxyproject.org/videos/topics/ai4life/tutorials/welcome-bioimageio/slides.mp4

authors: cfusterbarcelo hexylena


text:
permissions issue

I am not sure i have the correct permissions and im unable to tell exactly what the latest permission sets are so that i can enable and disable ss
I am using
- scoreboardstats.use
- scoreboardstats.hide

in my default permissions group and i still get errors
i am going off permissions based on this - https://github.com/games647/ScoreboardStats/blob/master/src/main/resources/plugin.yml
Thanks

Erros? Which ones?
scoreboardstats.use and scoreboardstats are assigned by default
In order to disable scoreboardstats with permissions, you need to give the user the negative permission.


text:
I can't get it to work at all.

To start off I'm really new to Github and Atom so, sorry if it's something totally obvious. I installed the package and updated dependencies. I type <div> in a jsx file and nothing happens.

@JacaL-13 u need disabled the Tree Sitter Parsers options (enabled by default) on Atom core config. (cmd|ctrl + ;)

repo: gandm/language-babel


text:
viztracer --ignore_frozen -m搭配使用

# 目录结构
/viz_test  # 执行viztracer命令所在目录
    - /xxx
        - __main__.py

# __main__.py
def fn():
    return 1
fn()
# 执行命令
viztracer --ignore_frozen -m xxx
# 没有`fn`函数调用记录

(py311) xuxx@xuxx:~/vswork$ viztracer -m xxx
Total Entries: 957
Use the following command to open the report:
vizviewer /home/xuxx/vswork/result.json

viztracer --ignore_frozen -m xxx
Total Entries: 4
Use the following command to open the report:
vizviewer /home/xuxx/vswork/result.json

为什么__main__.py会被调用？它又不在xxx里，而且连__main__.py里都没调用它啊，只是定义了而已。你首先要确定你这个函数被调用了啊。你在你的fn里写个print(1)它会打印么？

傻逼了， 大哥

是我写错了， main.py 在xxx目录下， 并且有调用fn函数

repo: gaogaotiantian/viztracer


text:
AppImage启动太慢，直接发zip包解压缩运行

Search before asking

[X] 在 issues 中没有找到类似的内容。
I searched in the issues and found nothing similar.

feature
AppImage启动太慢
描述 Motivation
AppImage启动太慢，我把文件从 ./mount 复制出来运行，不需要解压，快得很。
Solution
直接发zip包解压缩运行
还有其他内容吗？Anything else?
No response
你是否愿意提交一份 PR？Are you willing to submit a PR?

[ ] 我愿意提供 PR! I'm willing to submit a PR!

AppImage启动太慢,且resources/engine中的aria2c存在“无可执行权限“问题，启动后会报错无法下载云盘文件；
deb包中resources/engine中的aria2c也存在“无可执行权限“问题，建议首次安装完deb包后，执行一次:sudo chmod +x /opt/小白羊云盘/resources/engine/aria2c

authors: shiqishao sonichy

repo: gaozhangmin/aliyunpan


text:
Add Gardener anniversary blog post

What this PR does / why we need it:
This PR adds a general Gardener blog post.
Special notes for your reviewer:
/cc @vasu1124
Release note:

Gardener anniversary blog post was added.

@PavelPavlov46

@PavelPavlov46

Thanks for your review @g-pavlov. I've addressed your comments, PTAL.

Thanks for your review @g-pavlov. I've addressed your comments, PTAL.

/do-not-merge
After some internal feedback, I'm going to rephrase a section of the post.

/do-not-merge
After some internal feedback, I'm going to rephrase a section of the post.

/unhold

/unhold

Update: The post is ready for a final round of review.

Update: The post is ready for a final round of review.

Can we merge @vasu1124 @g-pavlov?

Can we merge @vasu1124 @g-pavlov?

Fine by me.

/lgtm (too late)

authors: g-pavlov timuthy vasu1124

repo: gardener/documentation


text:
dep ensure fails on new dev setup

Hangs while running the command:
(25)      try github.com/google/gofuzz@master
^CSignal received: waiting for 1 ops to complete...
(25)  ✓ select github.com/google/gofuzz@master w/1 pkgs
(26)    ← no more versions of github.com/mailru/easyjson to try; begin backtrack
  ✗ solving failed

Solver wall times by segment:
              b-gmal: 5h3m14.463362824s
  b-deduce-proj-root:      8.316778095s
         b-list-pkgs:      5.721625983s
     b-source-exists:      1.455858749s
    b-rev-present-in:      502.506653ms
     b-list-versions:      302.277318ms
             satisfy:      109.409226ms
         select-atom:       53.654494ms
         select-root:       10.850098ms
            new-atom:        3.678568ms
               other:         778.974µs
            unselect:         326.409µs
            add-atom:          10.278µs
           backtrack:           7.186µs

  TOTAL: 5h3m30.941124855s

dep:
 version     : devel
 build date  :
 git hash    :
 go version  : go1.11
 go compiler : gc
 platform    : darwin/amd64
 features    : ImportDuringSolve=false

go version go1.11 darwin/amd64

Failed when trying to run tests natively on mac

Can you upgrade your version of dep and try again? This is what I have:
$ dep version
dep:
 version     : v0.4.1
 build date  : 2018-01-27
 git hash    : 37d9ea0
 go version  : go1.9.3
 go compiler : gc
 platform    : darwin/amd64

More investigation reveals that this is likely an issue with the dep using some bad cache. I think I'll have a fix soon.

Alright, thank you. Then let's close this one.

Cleaned up the $GOPATH/pkg directory and everything is good now.

authors: harsimranmaan rfranzke

repo: gardener/gardener


text:
Provision via GKE

Issue by kubernetes-jenkins
Wednesday Jul 19, 2017 at 18:44 GMT
Originally opened as https://git.removed/kubernetes-attic/garden-operator/issues/14

Stories

As provider I want to provision Kubernetes via GKE, so that we dont have to provision Kubernetes ourselves on GCP (when SAP or customers need to run it side-by-side) and benefit from a presumably more competitive solution than whatever we can do (in terms of full management and pricing).

Motivation
Do not engage in Kubernetes provisioning when there are competitive native solutions, especially when customers already run on that infrastructure and use/manage/operate the same native solutions.
Acceptance Criteria

[ ] GKE Kubernetes cluster can be created via the Gardener and offers about the same functionality as what we have on AWS (to be clarified)

:information_source: Migrated from Jira issue KUBE-84

Dropped out of scope.

authors: gardener-ci vlerenc


text:
Remove etcd proposal alerts.

How to categorize this PR?
/area monitoring
/kind cleanup
What this PR does / why we need it:
Removes alerts which are built on the proposals_failed_total metric of etcd.
The information that these alerts currently convey are already conveyed to the operator through other means, and there is no action an operator can take to fix the issue that raises these alerts (primarily leader election among members of the etcd cluster). See #10514 for more information.
Which issue(s) this PR fixes:
Fixes #10514
Special notes for your reviewer:
/cc @shreyas-s-rao @ashwani2k @vicwicker @rickardsjp
Release note:

Alerts based on the `proposals_failed_total` metric of the etcd cluster are not raised anymore.

/retest-required

@rickardsjp @vicwicker could you take a look?

/lgtm, thank you!

authors: renormalize rickardsjp


text:
Add architecture diagram to docs

Add an architecture diagram to docs.
Use mermaid

Completed for both web app and CLI app

repo: garethwebber/elixir-tower-hanoi


text:
Unknown command: "@garvae/eslint-plugin-comments"

Does this package actually exist?

Does this package actually exist?

So I'm back. I've been away for quite a while, not sure if you still need this rule. But I fixed the error in the post-installation script. The rule is installed and works according to the documentation, I just tested it on a “clean” react application. 🙏

authors: StanLaurent garvae

repo: garvae/eslint-plugin-comments


text:
Call to undefined method swoole_http_response::gzip()

开启gzip提示错误，swoole版本是1.9.4，laravel 5.2 ,有人报同样的错误吗？

请检查是否有glib扩展

repo: garveen/laravoole


text:
Overflow- INT_MAX reached

Hi everyone,
I'm actually running some of my work on your library ( pretty weel designed by the way), and I'm facing some issue during fit step.
I got arround 100 000 instances split on 5000 bags.
Problem is encountered in quadprog.convert() method at line  P = cvxmat(H), where H is a sample*sample matrix. I got some matrix that contains arround 10 000 000 000 values, and by the ways exceed the authorized number of values (limited to INT_MAX values).
Do someone have some clues on how to solve that?
Best regards

Hi @RomainCendre,
I don't think there's a straightforward way to overcome this issue. Fundamentally, kernel methods are O(n^2) in memory complexity without doing some tricks that would require re-writing the existing MIL algorithms to incorporate these tricks (e.g., only using a subset of the instances as candidate support vectors). Even if you didn't hit this INT_MAX limit, you'd run out of memory storing such a large matrix.
One potential workaround for a large MIL problem like this is to use a bag-level classifier rather than an instance-level classifier (if your problem permits this). For example, an approach like the NSK or STK will only construct a kernel matrix that is O(n_bags^2) instead of O(n_instances^2), which for 5,000 bags is more doable. The disadvantage is that with these methods it's harder to get good instance-level labels on the test set (they only work well at classifying an entire bag at a time).
Hope this helps,
-Gary

Hi @garydoranjr,
Thx for the time you took to answer and for the clarity.
As you said, I will first take a look at NSK / STK and I will tell you
about it.
Take Care,
Best regards
Le jeu. 28 nov. 2019 à 18:16, Gary Doran notifications@github.com a
écrit :

Hi @RomainCendre https://github.com/RomainCendre,
I don't think there's a straightforward way to overcome this issue.
Fundamentally, kernel methods are O(n^2) in memory complexity without doing
some tricks that would require re-writing the existing MIL algorithms to
incorporate these tricks (e.g., only using a subset of the instances as
candidate support vectors). Even if you didn't hit this INT_MAX limit,
you'd run out of memory storing such a large matrix.
One potential workaround for a large MIL problem like this is to use a
bag-level classifier rather than an instance-level classifier (if your
problem permits this). For example, an approach like the NSK or STK will
only construct a kernel matrix that is O(n_bags^2) instead of
O(n_instances^2), which for 5,000 bags is more doable. The disadvantage is
that with these methods it's harder to get good instance-level labels on
the test set (they only work well at classifying an entire bag at a time).
Hope this helps,
-Gary
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/garydoranjr/misvm/issues/21?email_source=notifications&email_token=AEN47KIPOPYQAJVDDV5QJXTQV74GRA5CNFSM4JRFXU6KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFNEULY#issuecomment-559565359,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEN47KLQQG7UR4INACH3NRDQV74GRANCNFSM4JRFXU6A
.

authors: RomainCendre garydoranjr

repo: garydoranjr/misvm


text:
Weird results when applying SemGCN to 2D pose from image

Inference on images in the wild using SemGCN has been partially covered in this thread and others, but only the overall process has been made clear. I.e.:

Step 1: Use a 2D pose estimation network to generate 2D pose in MPII format.
Step 2: Convert 2D pose from MPII format to H36M format as done here.
Step 3: Pre-process the 2D input pose as done here.
Step 4: Use the pre-processed 2D pose in H36M format as input to the SemGCN SH model. It outputs 3D pose in H36M format.

Below i will follow each step, using the test image of size 300x600 to the left.

For Step 1, i use EfficientPose to generate the MPII format 2d pose of the test image as shown above on the right, here's the numeric output:
positions = [[[108. 512.]	# Right ankle
              [114. 428.]	# Right knee
              [124. 320.]	# Right hip
              [186. 324.]	# Left hip
              [178. 426.]	# Left knee
              [176. 512.]	# Left ankle
              [156. 322.]	# Pelvis
              [162. 152.]	# Thorax
              [164. 114.]	# Upper neck
              [166.  24.]	# Head top
              [ 60. 322.]	# Right wrist
              [ 78. 238.]	# Right elbow
              [ 96. 148.]	# Right shoulder
              [230. 154.]	# Left shoulder
              [240. 246.]	# Left elbow
              [224. 326.]]]	# Left wrist

For Step 2, i run this:
positions = positions[:, SH_TO_GT_PERM, :]
To get the output:
positions = [[[156. 322.]
              [124. 320.]
              [114. 428.]
              [108. 512.]
              [186. 324.]
              [178. 426.]
              [176. 512.]
              [162. 152.]
              [164. 114.]
              [166.  24.]
              [230. 154.]
              [240. 246.]
              [224. 326.]
              [ 96. 148.]
              [ 78. 238.]
              [ 60. 322.]]]

For Step 3, i run this:
positions[..., :2] = normalize_screen_coordinates(positions[..., :2], w=300, h=600)
To get the output:
positions = [[[ 0.0399  0.1466 ]
              [-0.1733  0.1333 ]
              [-0.2400  0.8533 ]
              [-0.2799  1.4133 ]
              [ 0.2400  0.1600 ]
              [ 0.1866  0.8399 ]
              [ 0.1733  1.4133 ]
              [ 0.0800 -0.9866 ]
              [ 0.0933 -1.2400 ]
              [ 0.1066 -1.8400 ]
              [ 0.5333 -0.9733 ]
              [ 0.6000 -0.3600 ]
              [ 0.4933  0.1733 ]
              [-0.3600 -1.0133 ]
              [-0.4800 -0.4133 ]
              [-0.6000  0.1466 ]]]

For Step 4, the above is used as input to the SemGCN SH model running this:
inputs_2d = torch.from_numpy(positions)
inputs_2d = inputs_2d.to(device)
outputs_3d = model_pos(inputs_2d).cpu()
outputs = outputs_3d[:, :, :] - outputs_3d[:, :1, :]

Which gives the output:
outputs = [[[ 0.0000  0.0000  0.0000 ]
            [-0.0769 -0.6899 -0.2520 ]
            [ 0.0847 -0.4062 -0.0607 ]
            [ 0.4154  0.2318  0.4062 ]
            [ 0.2708 -0.5181 -0.0504 ]
            [ 0.3431 -0.7337  0.3018 ]
            [ 0.6379  0.6684  0.2033 ]
            [ 0.1650 -0.9141 -0.8496 ]
            [ 0.5825 -2.1341  0.2762 ]
            [ 1.1561 -1.5364 -0.6433 ]
            [ 1.1612 -1.1453 -0.2103 ]
            [ 0.9097 -0.6763  0.2361 ]
            [ 0.8202 -0.2971  0.2679 ]
            [ 0.8008 -1.1936 -0.1120 ]
            [ 0.2124 -1.3246  0.5563 ]
            [ 0.5093 -0.4762  0.3473 ]]]

When visualized this looks completely wrong... See image below. Can anyone highlight on where the problem lies? Is it a problem with the pre-processing, or with the model?

@duckduck-sys I think there are two points to note about the data:

location: the neck should be halfway between the shoulders, and the thorax should be roughly halfway between the neck and the hips.
scale: normalization depends on the image width(mapping to [-1,1] based on the w), thus, the proportion of human body in the image needs to match H36M.

raw output

after scaling the locations

after modifying the location of the neck and the thorax

@duckduck-sys I think there are two points to note about the data:

location: the neck should be halfway between the shoulders, and the thorax should be roughly halfway between the neck and the hips.
scale: normalization depends on the image width(mapping to [-1,1] based on the w), thus, the proportion of human body in the image needs to match H36M.

raw output

after scaling the locations

after modifying the location of the neck and the thorax

hi, how do you calculate the spine point

hi @duckduck-sys
大神，你的数据归一化后有点奇怪，还有我想请教下您现在能正确回归出3d pose了吗，主要是hip和spine节点我不懂计算，还有就是hip节点要假定为(0,0)吗

@develduan Hi Duan, can you share this solution? I'm also facing somewhat same problem.
Thanks

@develduan Hi. I also face the same problem  about how to figure out the spine point, because the stack-hourglass doesn't output the spine point

@lisa676 @dandingol03  Hi, I'm sorry that I stopped following this project because it didn't work very well on my dataset(the wild environment). In my dataset, all pedestrians stand upright, so I simply treated the midpoint of the neck and the pelvis as the thorax/spine: positions_mpii[i_thorax] = (positions_mpii[i_neck] + positions_mpii[i_pelvis]) / 2 . After normalize_screen_coordinates, scale the locations with a factor to fit the proportion of human body in the image of H36M, in my case: positions = positions / 2.
In my case, I want to get the 3D posture directly from the image instead of getting a 2D posture and then a 3D posture,  and I got a better result by following this paper "Angjoo Kanazawa, Michael J. Black, David W. Jacobs, Jitendra Malik. End-to-end Recovery of Human Shape and Pose".

@develduan  Firstly, thanks for your kindly apply. Secondly, the paper " End-to-end Recovery of Human Shape and Pose" is cool, i will delve into the paper soon. And last, here is my email dandingol03@outlook.com, maybe someday we can exchange idea about 3d pose estimation~

repo: garyzhao/SemGCN


text:
Add github action workflow steps to build and push OLM images

This depends on #66.

Fixes #47.

repo: gatekeeper/gatekeeper-operator


text:
PROD-550: Credit Cost Per Question Deck Form Admin

Description
Add new select input in Deck form to allocate credits to each questions - Credit Flow (Featured flag)

What are the steps to test that this code is working?
Create a new deck or edit existing deck, you will see Credit Cost property at second last in the form

Screen shots or recordings for UI changes
NA

Tests are failing due to migration

repo: gator-labs/chomp


text:
feat(gatsby-source-strapi): Graphql filter on one type doesn't work

Hi,
Question:
I use the queries below in the gatsby-graphql playground.
When I use the current query:
query MyQuery {
  allStrapiJob(filter: {data: {elemMatch: {id: {eq: 2}}}}) {
    nodes {
      data {
        attributes {
          slug
          title
        }
        id
      }
      id
    }
  }
}

I get this as a result
{
  "data": {
    "allStrapiJob": {
      "nodes": [
        {
          "data": [
            {
              "attributes": {
                "slug": "helpdesk-engineer-uk",
                "title": "Helpdesk Engineer"
              },
              "id": 1
            },
            {
              "attributes": {
                "slug": "frontend-developer",
                "title": "Frontend developer"
              },
              "id": 2
            }
          ],
          "id": "Job_undefined"
        }
      ]
    }
  },
  "extensions": {}
}

Always 2 results (there are 2 entries in strapi at the moment)
If I change the id to filter, to either 1 or 2, I always get 2 results.
When changing it to anything else then 1 or 2, I get nothing as a result.
What am I doing wrong here?
Also, I got "strapiJob" to use.
query MyQuery {
  strapiJob(data: {elemMatch: {id: {eq: 1}}}) {
    data {
      attributes {
        slug
        title
        description
      }
      id
    }
  }
}

Same result, always 2 entries.

I am assuming this is Strapi 4 data fetched (because of the .data.attributes blocks) by version 1 of gatsby-source-strapi which only supports Strapi 3.  Please create a new issue with new info if you think otherwise.

authors: laurenskling mariovde-IIO

repo: gatsby-uc/plugins


text:
Added support for user provided certificate and updated image.

Changes:

Added user provided ssl certificate support. CA and Server certificate.
changed base image to golang:latest(builder), and ubuntu:20.04(main)-building won;t work in golang:1.9
changed go command so source is downloaded from barnacs/compy
multi arch support.

Image built successfully in my raspberry pi 4 (ubuntu 20.04) , but not tested. leaving it as a draft.

PS: I'm not familiar with travic-ci; you should edit related files accordingly.

repo: gaul/compy

license: ISC


text:
[Vuln-20] Modified README file to include team member info.

UT CS 361 Fall 2015
Team: ABSA Potatoes
WebGoat Vulnerability: Injection Flaws => Database Backdoor (vuln-20)
Team Members: Jeannette Tang (jeannette.tang@utexas.edu) and Kelvin Nguyen (kelvinnguyen@utexas.edu)
Apologies for the multiple pull requests.

Not entirely necessary but thanks for the update.

authors: Xugo mtesauro

repo: gauntlt/gauntlt-demo


text:
mVisualizer.setRawAudioBytes(data) not showing proper data in it..

Hey,
I have a recording app and want to use this as visualizer, but when receiving audio and setting into it, i am not able to find the proper blast form or any thing nicer in that..
![image](https://user-images.githubusercontent.com/24537492/51698235-b1cc0380-202f-11e9-921e-6a133b1534c6.png)

    public int RECORDER_SAMPLERATE = 44100;
    private final int RECORDER_CHANNELS = AudioFormat.CHANNEL_IN_MONO;
    private final int RECORDER_AUDIO_ENCODING = AudioFormat.ENCODING_PCM_16BIT;

 BufferElements2Rec = AudioRecord.getMinBufferSize(RECORDER_SAMPLERATE,
                RECORDER_CHANNELS, RECORDER_AUDIO_ENCODING);

   audioRecord = new AudioRecord(MediaRecorder.AudioSource.MIC,
                RECORDER_SAMPLERATE, RECORDER_CHANNELS,
                RECORDER_AUDIO_ENCODING, BufferElements2Rec * 2);

Thread recordingThread = new Thread("recorder") {
                @Override
                public void run() {
                    super.run();
                    if (Looper.myLooper() == null) {
                        Looper.prepare();
                    }
                    audioRecord.setRecordPositionUpdateListener(recordPositionUpdateListener, new Handler(Looper.myLooper()));

                    audioRecord.setPositionNotificationPeriod((int) BufferElements2Rec);
                    //We need to read first chunk to motivate recordPositionUpdateListener.
                    //Mostly, for lower versions - https://code.google.com/p/android/issues/detail?id=53996
                    bufferReadResult = audioRecord.read(buffer, 0, BufferElements2Rec);
                    Looper.loop();
                }
            };
audioRecord.startRecording();

recordingThread.start();

  private AudioRecord.OnRecordPositionUpdateListener recordPositionUpdateListener = new AudioRecord.OnRecordPositionUpdateListener() {
        @Override
        public void onMarkerReached(AudioRecord recorder) {
            //empty for now
        }

        @Override
        public void onPeriodicNotification(AudioRecord recorder) {
            if (audioRecord.getRecordingState() == AudioRecord.RECORDSTATE_RECORDING
                    && audioRecord.read(buffer, 0, buffer.length) != -1) {
//                mHorizon.updateView(buffer);
                byte[] data = new byte[buffer.length];
                for (int j = 0; j < buffer.length; j++) {
                    byte tmp = (byte) (buffer[j]);
                    data[j] = tmp;
                }
                  mVisualizer.setRawAudioBytes(data);
            }
        }
    };

Help is really appreciate asap.

Already spent almost two for combo..
![screenshot_10](https://user-images.githubusercontent.com/24537492/51698343-eb9d0a00-202f-11e9-9efa-e5404547f185.png)

Привет,
У меня есть приложение для записи, и я хочу использовать его в качестве визуализатора, но при получении звука и настройке в нем я не могу найти правильную форму звука или что-то более приятное в этом ...
![image](https://user-images.githubusercontent.com/24537492/51698235-b1cc0380-202f-11e9-921e-6a133b1534c6.png)

    public int RECORDER_SAMPLERATE = 44100;
    private final int RECORDER_CHANNELS = AudioFormat.CHANNEL_IN_MONO;
    private final int RECORDER_AUDIO_ENCODING = AudioFormat.ENCODING_PCM_16BIT;

 BufferElements2Rec = AudioRecord.getMinBufferSize(RECORDER_SAMPLERATE,
                RECORDER_CHANNELS, RECORDER_AUDIO_ENCODING);

   audioRecord = new AudioRecord(MediaRecorder.AudioSource.MIC,
                RECORDER_SAMPLERATE, RECORDER_CHANNELS,
                RECORDER_AUDIO_ENCODING, BufferElements2Rec * 2);

Thread recordingThread = new Thread("recorder") {
                @Override
                public void run() {
                    super.run();
                    if (Looper.myLooper() == null) {
                        Looper.prepare();
                    }
                    audioRecord.setRecordPositionUpdateListener(recordPositionUpdateListener, new Handler(Looper.myLooper()));

                    audioRecord.setPositionNotificationPeriod((int) BufferElements2Rec);
                    //We need to read first chunk to motivate recordPositionUpdateListener.
                    //Mostly, for lower versions - https://code.google.com/p/android/issues/detail?id=53996
                    bufferReadResult = audioRecord.read(buffer, 0, BufferElements2Rec);
                    Looper.loop();
                }
            };
audioRecord.startRecording();

recordingThread.start();

  private AudioRecord.OnRecordPositionUpdateListener recordPositionUpdateListener = new AudioRecord.OnRecordPositionUpdateListener() {
        @Override
        public void onMarkerReached(AudioRecord recorder) {
            //empty for now
        }

        @Override
        public void onPeriodicNotification(AudioRecord recorder) {
            if (audioRecord.getRecordingState() == AudioRecord.RECORDSTATE_RECORDING
                    && audioRecord.read(buffer, 0, buffer.length) != -1) {
//                mHorizon.updateView(buffer);
                byte[] data = new byte[buffer.length];
                for (int j = 0; j < buffer.length; j++) {
                    byte tmp = (byte) (buffer[j]);
                    data[j] = tmp;
                }
                  mVisualizer.setRawAudioBytes(data);
            }
        }
    };

Help is really appreciate asap.

Already spent almost two for combo..
![screenshot_10](https://user-images.githubusercontent.com/24537492/51698343-eb9d0a00-202f-11e9-9efa-e5404547f185.png)

Hi, you have solved this problem. If so, how.


text:
Subdirectories searching are case sensitive, while parent directories are not

Describe the bug
When I navigate to the first directory, my input is not case sensitive. But for the next subdirectory, it's case sensitive however.
To Reproduce
Imagine you are in the following home directory

/home/user
       ⌞ Foo
         ⌞ Bar

You call fifc, then type foo (first letter lower case).
Then you see directory Foo. as input. Press enter.
Then you have navigated to the directory Foo.
You call fifc again and type bar (first lower letter case).
But the directory Bar however doesn't show up because the input is lower case only, which is strange behaviour compared with how the first directory is looked up.
Expected behavior
I was expecting the same case-insensitive behaviour when filtering subdirectories like with the directories in same operation.

I was wondering if you got any chance to look into this, perhaps, @gazorby ?

repo: gazorby/fifc


text:
Placeholder texts

In the UI we have made room for about texts for each filter. But someone has to write those descriptions.
https://github.com/gbif/gbif-web/blob/master/packages/react-components/src/locales/en.js#L229
We might not want if for all - or we might not have it for all to begin with. So for now we should either add an about text or remove the text (by removing the key, the about option will disappear).
Similar we have placeholder texts for input "placeholder" i.e. the greyed out text in search boxes.

Are these customizable within hosted portal instances?

No - that isn't the intention. The idea is to have a shared version (managed in Crowdin). So that e.g. all spanish version have the same help texts. If there is disagreements on what a help text should be, then that discussion can take place in Crowdin among the translators. Or if different variations of spanish is needed and there is a desire among the translators to have spanish in Pan-American, Caribbean and Catalan then that is fine.
If there is a desire to customise messages on an instance level, then I see no technical challenges in doing so, but I believe we should start with a shared version.
Do you ask because you already now foresee a need to customise them?

I expect that there will arise the need to customize them, but not because
we have one at this moment. I agree that the best starting point is to have
good shared versions. I think we are interested in providing these. Should
we do that through a pull request on this repo?
On Mon, Apr 26, 2021 at 4:43 AM Morten Høfft @.***>
wrote:

No - that isn't the intention. The idea is to have a shared version
(managed in Crowdin). So that e.g. all spanish version have the same help
texts. If there is disagreements on what a help text should be, then that
discussion can take place in Crowdin among the translators. Or if different
variations of spanish is needed and there is a desire among the translators
to have spanish in Pan-American, Caribbean and Catalan then that is fine.
If there is a desire to customise messages on an instance level, then I
see no technical challenges in doing so, but I believe we should start with
a shared version.
Do you ask because you already now foresee a need to customise them?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/gbif/hosted-portals/issues/134#issuecomment-826591840,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AADQ725ZQQJV24ZLYZ473KLTKUKRPANCNFSM43N5HXBQ
.

This question seem to be the same as https://github.com/gbif/hosted-portals/issues/60#issuecomment-827125356 ?
My primary concern with that is my worry that someone/you waste effort because we/I later decide to restructure something in the UI that make that help text unused. But that might be worrying too much.
Secondly, then some texts are still hardcoded, so you might not be able to find everything in the translation files.
And lastly, the format of the translation file will likely change, but that won't influence any text.
You are welcome to do pull requests with text changes to the project. Or just provide them here and I will copy them (if it becomes overwhelming I might back down on that approach and accept pull requests only).

I am in the process of amending commonFilters.js, getCommonSuggests.js, and locales/en.js There are three terms that aren't clear to me to what they refer. I don't see anything in the hosted portals to which they might refer. Can you tell me what they are suppose to be? They are:
name
city
code

I created a pull request with English text for as many terms as I could identify.

First of all - thank you for correcting all those texts!

Can you tell me what they are suppose to be? They are:
name
city
code

The list of filters is used across search for publishers/datasets/occurrences/institutions/collections/literature
name, city and code are specific to institutions and collections. You can see it in use here https://react-components.gbif-staging.org/iframe.html?id=search-collectionsearch--example&args=&viewMode=story

Thanks to @tucotuco I will close this issue. There is likely more placeholder values and mistaken hardcoded help texts lingering, but the large bulk is solved now.

authors: MortenHofft tucotuco

repo: gbif/hosted-portals


text:
Drupal API doesn't expose all participant info (eg. history)

The Drupal API expose the participant, but only some of the content. This is essential to have for the country pages. Ex: http://cms-api.gbif-dev.org/api/v1/gbif_participant/8513

This is due to a content massaging error. Will fix it.

The gbif_participant endpoint now uses participantId from Directory API for calling node information, e.g. http://cms.gbif-dev.org/api/v1/gbif_participant/2
And now the history, vision & mission as well as other textual contents are shown.

authors: MortenHofft burkeker


text:
🛑 IPT - GBIF Spain is down

In 9f2da65, IPT - GBIF Spain (https://ipt.gbif.es/) was down:

HTTP code: 0
Response time: 0 ms

Resolved: IPT - GBIF Spain is back up in 3ba5da7 after 5 minutes.

repo: gbifatlas/status