text:
Theme submission: Pyxill2

Theme submission
Please make sure that you have read the theme submission guidelines before submitting a theme. The guidelines provide all relevant information and requirements that have to be fulfilled before the submission. We strongly suggest that you test your theme with the Hugo Themes Build Script before submitting the theme for review. If a submission does not meet the criteria mentioned in the README it will be closed. You may re-submit once you fix the problems with your submission. However, please note that we have limited resources and can not provide help for general web development issues. For Hugo support questions please refer to the dedicated support forum.
Please tick the relevant boxes for your theme in the checklist below:
Link to my theme repository: https://github.com/IcpMoles/pyxill2
I made sure that...

[x] the repository contains a good README.md describing my theme
[x] an open source license has been added to LICENSE.md
[x] all metadata have been added to theme.toml
[ ] in case I'm using a customized demo via the exampleSite folder that

[ ] https://example.com is set as base url in exampleSite/config.{toml, yaml, json} to avoid the abuse of unused domains
[ ] I tested that my theme's demo works with the content directory of gohugoio/HugoBasicExample

[x] screenshots have been added in the images/ folder with the required dimensions
[x] I tested my theme against the gohugoio/HugoBasicExample

[x] I've checked the developer tools' console in my browser for error messages

[x] in case my theme is using Hugo Pipes features like toCSS and PostCSS that I have committed the /resources directory with all generated assets, for my theme to work in the basic version of Hugo

N.B. By submitting a theme to the Hugo Themes Showcase you understand that you need to maintain your theme. If a theme demo breaks and remains broken then at some point it will be removed from the list without prior warning. If you no longer wish to maintain a theme please let us know.
New themes will usually be promoted on Hugo's official Twitter account. If you would like to be mentioned in the tweet please add your Twitter username to this submission.
**Link to my Twitter account (optional): https://twitter.com/icpmoles
Feel free to ask questions. We're glad to help.

Hello Iacopo,
thank you for sharing this theme with the Hugo community 👍 While reviewing your theme I noticed a few issues:

right now embedded fonts can't be loaded if the website if hosted in a subdirectory, e.g. as in www.example.com/blog. This is the case with demos on www.themes.gohugo.io. Instead of using absolute paths in the stylesheet(s) that would always point to www.example.com, make the fonts relative to the stylesheet by prepending the path with ../font/font.woff.
since your theme is based on the pixyll theme it would be helpful for users if you could add a small section to the README that states what changes have been made compared to the original theme.
links to a privacy policy and credit page are hard-coded in the footer. Users of your theme might don't need such pages or want to link custom ones. Hence I suggest to use Hugo's menu feature to make links more flexible.
your theme links to an apple touch icon and a favicon over here but those files can't be found in the repository.

@digitalcraftsman
Thanks for the review. I think I fixed every you listed.

authors: IcpMoles digitalcraftsman

repo: gohugoio/hugoThemes


text:
Remove alpha v1 from java package name

The package should be renamed before publishing this to public Maven repository.

/ok-to-test

/approve

/lgtm

source: gharchive/pull-request

authors: davidheryanto khorshuheng

repo: gojek/feast

license_type: permissive

license_source: github-api


text:
dep status is not useful if memo is out of date, not even a warning

I've tried dep on a number of projects, some have main packages, others don't.
After I ran dep init, I expected dep status [-v, -detail, etc] to produce some list of packages. Each time I've gotten nothing.
At dep rev 1b193f4439655572d59fe1b87035870f1a7344ca

Okay, so status won't show anything if the memo field is out of date.

Yeah, dep status was one of the first things we implemented, and it's been largely neglected since. Some of that is because waiting on things in gps, but those things are almost all resolved now (sdboyer/gps#127 is the big outstanding one). In any case, though status really needs a full pass.
Would be a great place for someone really wanting to grok this thing/get their hands dirty to dig in 😃 😇

Any suggestions on how we handle this behavior?
In case of outdated memo, shouldn't running status inform the user about outdated memo and ask to run ensure or provide information about how to fix it?
Also, under which scenarios the memo would not be updated and become outdated? Considering #470 would ensure memo is updated after init run.

@darkowlzz status has really fallen by the wayside, which is unfortunate. i do have some sketched-out plans that, hopefully, i can translate into the doc in #277 for status soon, which should hopefully enable a more detailed discussion about it.
in general, though, if there's a memo mismatch (which can occur when a new import is added, an import is removed, or any values in Gopkg.toml are modified), then i think our goal is probably to detect the cause of the mismatch, and report it to the user.

@darkowlzz i think i just merged a PR from you...or there's one in flight...that addresses this?

@sdboyer nope, I don't see the changes in master. It's #611 .

source: gharchive/issue

authors: darkowlzz kardianos sdboyer

repo: golang/dep

license_source: bigquery


text:
spec: bounded type selectors

Spec (https://golang.org/ref/spec#Method_declarations): "The receiver is specified via an extra parameter section preceeding the method name. That parameter section must declare a single parameter, the receiver. Its type must be of the form T or *T (possibly using parentheses) where T is a type name. The type denoted by T is called the receiver base type; it must not be a pointer or interface type and it must be declared in the same package as the method. The method is said to be bound to the base type and the method name is visible only within selectors for that type."
It says for that type, whereas according to http://golang.org/ref/spec#Selectors, it is also visible within pointers to the receiver base type. This is true whether the method has a pointer or a value receiver.  It's being misleading, unless selectors on both T and *T values are considered selectors for T, which should be stated more clearly if it's the case. If it isn't the case, then the spec is incorrect and needs to be fixed.

@griesemer if this gets merged, do I get my name in the contributors file or do I have to be one to submit the actual code?

@aubble No, unless you sign the CLA for legal reasons and submit a change, you're not going to be in the contributors file.
You're indirectly referred to through the issue that's mentioned in that change. I'm happy to mention your name/e-mail address in the change as the person who found the issue if you like. But again, you probably should sign the CLA so that there's no legal issue we the Go team using your e-mail address and name. Thanks.

@griesemer alright I signed it, my name is Anmol Sethi and my email is anmol@aubble.com. thanks :D

Would you like me to mention your e-mail address explicitly in the commit
message? (This will make it visible to the public).
Otherwise I'm just going to say:
"Inconsistency identified by Anmol Sethi (aubble@github)."

gri

On Tue, Aug 4, 2015 at 1:44 PM, aubble notifications@github.com wrote:

Anmol Sethi and my email is anmol@aubble.com

@griesemer well I mean it will be public anyway in the contributors file so its fine. explicit is okk

done. I will submit once prior pending spec changes are ok'ed.

gri

On Tue, Aug 4, 2015 at 2:34 PM, aubble notifications@github.com wrote:

@griesemer https://github.com/griesemer well I mean it will be public
anyway in the contributors file so its fine. explicit is okk
—
Reply to this email directly or view it on GitHub
https://github.com/golang/go/issues/10341#issuecomment-127768834.

sweet now I have something to tell my grandchildren.

authors: aubble griesemer

repo: golang/go


text:
fmt: float64 formatting behaving differently than expected (and than documented)

What version of Go are you using (go version)?
1.7.3
What operating system and processor architecture are you using (go env)?
darwin amd64
What did you do?
https://play.golang.org/p/LEVTU7r4wX
What did you expect to see?
The %g-formatted (or %v) float64 number to be the same as either %f or %e, as per documentation.
What did you see instead?
The %g-formatted number was 24.849999999999998.

The default precision for %e and %f is 6; for %g it is the smallest number of digits necessary to identify the value uniquely.

Probably needs a doc tweak, as the short description of %g does suggest that it is either %e or %f.
CC @robpike @martisch

There's only so much we can say in a one-line summary. The full details of the precision are documented. I tend to think this is fine as is, unless someone can come up with a very short replacement for the current one-line summary.

It may suffice to replace
%g	%e for large exponents, %f otherwise

with
%g	%e for large exponents, %f otherwise (see below for default precision in this case)

authors: ALTree griesemer ianlancetaylor rsc vvekic


text:
cmd/go: list {{.Stale}} was not updated for the change to install

In Go 1.10 we changed go install so that it only installs the named package.  The docs for go list -f {{.Stale}} say that it means "would 'go install' do anything for this package?".  But
> go install golang.org/x/tools/cmd/present
> go list -f '{{.Stale}} {{.StaleReason}}' golang.org/x/tools/cmd/present
true stale dependency: golang.org/x/tools/godoc/static

As we can see, even though go install will not do anything, {{.Stale}} reports that the package is stale.
Setting milestone to 1.10.1 since this is a regression in 1.10.
CC @rsc

I'm not convinced this is a regression. The docs may need updating instead. Is there a larger example that is broken by this behavior?

See #24396, where the mobile tooling wants to know whether a program is up to date.  My suggestion was that go list -f '{{.Stale}}' should report that.  Although that used to work in older releases, it no longer does, unless (I assume) you use go install -i.  I don't see any particular reason it ought to work that way.

Possible dup of #23818 and #23797.

@rsc, the example may be the test I have tried to fix: https://go-review.googlesource.com/c/go/+/103675
I had to pass "-i" to install to get rid of stale message to pass the test.
Does it look right?

Started work on this.

@gopherbot please file this for backport against 1.10. This might be considered a regression (see discussion).

authors: FiloSottile eliasnaur ianlancetaylor rsc ysmolsky


text:
x/tools/go/ssa: wrong ssa referrers with ifstmt

Please answer these questions before submitting your issue. Thanks!
What version of Go are you using (go version)?
go version go1.10.2 linux/amd64
Does this issue reproduce with the latest release?
yes
What operating system and processor architecture are you using (go env)?
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/nabice/.cache/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/nabice/go:/home/nabice/skygo/go"
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build010669814=/tmp/go-build -gno-record-gcc-switches"
What did you do?
Package: golang.org/x/tools/go/ssa
ok.Referrers do not contains Ifstmt
https://play.golang.org/p/wkERfEjeSu1
I found it is replaced here:
https://github.com/golang/tools/blob/e10d6c9a84802dced65cb0278773be159bb7ed07/go/ssa/blockopt.go#L89
What did you expect to see?
ok.Referrers contain Ifstmt's cond -- "ok"
What did you see instead?
wrong

The jumpThreading optimization simplifies degenerate if statements, effectively entirely eliminating the one below:
   cond = ...
   if cond {
   } else {
   }

leaving only the evaluation of the condition, but not its referrers.
This would be a sound code transformation  in a compiler, but because go/ssa strives to stay close to the source code, perhaps it must forego this optimization.

authors: alandonovan nabice


text:
x/dl/version: not setting goroot

What version of Go are you using (go version)?
1.12beta2 but irreverent
Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env)?

set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\alexr\AppData\Local\go-build
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=%USERPROFILE%\go
set GOPROXY=
set GORACE=
set GOROOT=C:\Go-tip
set GOTMPDIR=
set GOTOOLDIR=C:\Go-tip\pkg\tool\windows_amd64
set GCCGO=gccgo
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=%USERPROFILE%\AppData\Local\Temp\go-build619042602=/tmp/go-build -gno-record-gcc-switches

What did you do?
In an effort to reproduce the issue reported in #30117
I ran go get golang.org/dl/gotip followed by gotip run . on the code described in
https://github.com/golang/go/issues/30117#issuecomment-461323858
What did you expect to see?
No output
What did you see instead?
.\test.go:16:86: syntax error: unexpected semicolon, expecting expression
Setting GOROOT to the proper path gives the expected result.

You may need to run  go get -u golang.org/dl/gotip && gotip download to make sure you're on the version of the wrapper that sets it.

right... forgot to mention I did gotip download

Did you run go get -u, with the -u flag?

Never downloaded gotip on this machine before so didn't think it mattered. But just to be sure I did ran it with -u and no change.
gotip env does show GOROOT to be C:\Go instead of %USERPROFILE%\sdk\gotip

Are you setting GOROOT directly anywhere on your system?

Nope, only set GOPATH nothing more.

OHHH the windows installer set GOROOT under system variables and I didn't checked there.
Still one would assume gotip to set GOROOT no?

In general, Go installations nowadays should not set up GOROOT. It is inferred from the location of the go binary you are running.
If you do specify a GOROOT via an env var, it's always used directly. So if your system sets GOROOT, you'll also have to set it up when calling gotip or any other binary Go installation.
gotip doesn't set GOROOT because it isn't necessary. And presumably, also because users can set their own GOROOT, overriding the default.
So in this case, I'd say this is not a bug; you should either not have GOROOT set up, or you should set it up when calling gotip. But I'll leave the issue open for now, in case it's decided that there's something to fix in programs like gotip.

If it's decided that there's nothing to do here I'm fine with that.
Should I file a new report for the official windows installer setting GOROOT?

Should I file a new report for the official windows installer setting GOROOT?

I'd do that; see https://golang.org/doc/go1.10#goroot. Setting it is usually harmless, but can be confusing when one uses multiple Go versions, like you're trying to do.

I've also sent https://go-review.googlesource.com/c/go/+/161758, as I still found pieces of the official documentation which recommeded setting up GOROOT.

Looks like this is tracked in the new issue. Closing.

@agnivade not exactly, they are related yes but quite different.
This issue is about whether x/dl/version should ignore a set GOROOT as the point of x/dl/version is to use a specific Go version.

x/dl/version just installs a go distribution. And the Go distribution will always use a GOROOT env var if it is specifically set. I don't think there is any bug here with x/dl/version.

authors: AlexRouSg agnivade bcmills mvdan


text:
proposal: database/sql: add ConnetcionID()

When I create a connection, I want to get the connection ID, which may have been multiple times.
Is it possible to implement ConnectionID()?
Poor efficiency when using select connection_id() multiple times.
Go version go1.12.1 linux/amd64

Probably not. Either grab a DB.Conn and manually associate the ID once per connection, or wait for a potential conn.raw call.

Was this resolved?

Thanks! the problem has been solved!
When I found that a new connection is used, I will query the connection ID once.
https://github.com/go-sql-driver/mysql/issues/946#issuecomment-480708482

authors: hanchuanchuan kardianos


text:
strconv.UnquoteChar() fails for "\"

What version of Go are you using (go version)?

$ go version
go version go1.12.5 windows/amd64

Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env)?
windows/amd64
go env Output
$ go env

$ go env
set GOARCH=amd64
set GOBIN=f:\Dropbox\golang/bin
set GOCACHE=C:\Users\Malcolm\AppData\Local\go-build
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=f:\Dropbox\golang
set GOPROXY=
set GORACE=
set GOROOT=C:\Go
set GOTMPDIR=
set GOTOOLDIR=C:\Go\pkg\tool\windows_amd64
set GCCGO=gccgo
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=G:\cygwin64\tmp\go-build905078030=/tmp/go-build -gno-record-gcc-switches
What did you do?

https://play.golang.org/p/AlYeYnWcsJZ
What did you expect to see?
I expected string input of "\" to return a rune of value '\'
What did you see instead?
error returned: invalid syntax

Whoops! I see what I did wrong.
Problem solved.
https://play.golang.org/p/LjEMdp_tAQV


text:
go/build: sourcehut requires characters considered unsafe for cgo names

build.go has a set of safe characters which can be present in CFLAGS:
https://github.com/golang/go/blob/c290cb6338fe9ce7d8ade3adbb4acd0f0e6172e4/src/go/build/build.go#L1553-L1559
Sourcehut packages always have ~ in the URL, and in the future will likely have ^ as well. Importing a package from sr.ht which uses ${SRCDIR} in its CFLAGS accordingly causes this error:
widgets/terminal.go:11:2: /home/sircmpwn/.local/share/go/pkg/mod/git.sr.ht/~sircmpwn/go-libvterm@v0.0.0-20190526184212-d093d527d591/vterm.go: malformed #cgo argument: -I/home/sircmpwn/.local/share/go/pkg/mod/git.sr.ht/~sircmpwn/go-libvterm@v0.0.0-20190526184212-d093d527d591/libvterm/include/

~ has a special meaning in the shell, but it is URL safe and, per the comments in build.go, this string is never expanded with a shell.
EDIT: added permalink

ping @ianlancetaylor

Bump? This should be easy to fix, is definitely a bug in golang, and shows unfair favoritism towards other platforms.

Want to send in a patch?

I don't really want to sign the CLA to be honest :/ I've pointed out the line of code which needs updating and it should be a one-line change.

Please note that these changes often aren't a trivial one-line fix. At minimum, it should include a small regression test. And the author should ensure that no other features or tests break because of the change.

Please note that these changes often aren't a trivial one-line fix. At minimum, it should include a small regression test. And the author should ensure that no other features or tests break because of the change.

Yeah, I know. The point I was getting at is that this is about as simple of a change as Golang can get. Thanks a lot for putting up the patch, I appreciate it!

Thank you!

This also affects 8.3 short path on Windows. Glad to see it was fixed.
--- FAIL: TestScript (0.01s)
    --- FAIL: TestScript/mod_case_cgo (0.31s)
        script_test.go:183:
            > [!cgo] skip
            > env GO111MODULE=on
            > go get rsc.io/CGO
            [stderr]
            go: finding rsc.io/CGO v1.0.0
            go: downloading rsc.io/CGO v1.0.0
            go: extracting rsc.io/CGO v1.0.0
            package rsc.io/CGO: $WORK\gopath\pkg\mod\rsc.io\!c!g!o@v1.0.0\cgo.go: malformed #cgo argument: -IC:/DOCUME~1/ADMINI~1/LOCALS~1/Temp/cmd-go-test-056881027/script-mod_case_cgo/gopath/pkg/mod/rsc.io/!c!g!o@v1.0.0
            [exit status 1]
            FAIL: testdata\script\mod_case_cgo.txt:5: unexpected command failure

authors: AlexRouSg ddevault ianlancetaylor mvdan zhangyoufu


text:
net/http: DetectContentType does not detect application/x-bzip2

What version of Go are you using (go version)?

$ go version
go version go1.12.5 linux/amd64

Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env)?
go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/cdaley/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/cdaley/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/lib/golang"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/golang/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/cdaley/Projects/github.com/offlinedocs/offlinedocs/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build110412641=/tmp/go-build -gno-record-gcc-switches"GOARCH="amd64"
GOBIN=""
GOCACHE="/home/cdaley/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/cdaley/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/lib/golang"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/golang/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/cdaley/Projects/github.com/offlinedocs/offlinedocs/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build110412641=/tmp/go-build -gno-record-gcc-switches"

What did you do?
extracted the first 512 bytes of a bzip2 file and ran net/http.DetectContentType on it
https://play.golang.org/p/u4WBPl0bsMS
What did you expect to see?
application/x-bzip2
What did you see instead?
application/octet-stream

Thank you for filing this issue @coreydaley!
So we get the signatures from https://mimesniff.spec.whatwg.org/#zip-based-mime-type so perhaps let's first file an issue on the Github issues https://github.com/whatwg/mimesniff and once that's agreed upon we'll LGTM your CL. Thank you.

I am going to move this to the unreleased milestone since we are depending on browsers and the MIME working group to approve the signatures.

@odeke-em, Unreleased is for code that's not included in Go releases. I changed to Unplanned. We can do this if the upstream spec is updated.

Got it, thanks Brad!
On Mon, Oct 14, 2019 at 1:42 PM Brad Fitzpatrick notifications@github.com
wrote:

@odeke-em https://github.com/odeke-em, Unreleased is for code that's
not included in Go releases. I changed to Unplanned. We can do this if the
upstream spec is updated.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/golang/go/issues/32508?email_source=notifications&email_token=ABFL3V5DF64LIC2JHIJZQODQOTKUXA5CNFSM4HWLAZVKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBGN3ZY#issuecomment-541908455,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABFL3VYA7TQU5DACSFQ3TL3QOTKUXANCNFSM4HWLAZVA
.

authors: bradfitz coreydaley odeke-em


text:
cmd/go: go Mod should use go dep SemVer behavior for < v1.0.0

What version of Go are you using (go version)?
go version go1.13.3 darwin/amd64
Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env)?
GO111MODULE="on"
GOARCH="amd64"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPRIVATE=""
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
What did you do?
I've a project with a dependency on lib A version v0.3.2, I ran:
go get -u ./...
lib A available versions are: v0.3.2 and v0.4.0
What did you expect to see?
Go.sum/go.mod should not update to v0.4.0 since it's a break change based on dep convention described here:
https://github.com/golang/dep/blob/master/docs/Gopkg.toml.md
What did you see instead?
It pulled v0.4.0 instead which lead to compilation errors

The go module behavior is documented at: https://github.com/golang/go/wiki/Modules
This would change the expected behavior for all current users of modules. I've marked it as a feature request.

/cc @bcmills @jayconrod

go get -u explicitly requests that versions be upgraded. If we were to avoid upgrading v0 dependencies with -u, those dependencies would never get upgraded and fixes to bugs would never be applied: according to the Semantic Versioning spec, at version 0 “[a]nything MAY change at any time”, including from v0.3.1 to v0.3.2, so literally no upgrade at all would be safe.
If you don't want to pull in breaking changes, either don't pass the -u flag or don't depend on modules that are still unstable.

So we essentially created a situation where:1. There is no safe migration path from dep,2. There is no safe upgrades for v0 libraries,3. There is no convergence path?
It seems that -reading that v0 is all fair game- makes it way less useful (specially to safely iterate).
And I'm not sure what was so "bad" or "wrong" about Dep's approach that it was decided to diverge from? was it maybe considered to propose an update to SemVer to reflect that behavior....
I would like to use this issue to at least document the thought process that went into this decision...

@EngHabu The thought process and design decisions that went into modules are well described in Russ's series of blog posts. In particular, see The Principles of Versioning in Go.

authors: EngHabu bcmills cagedmantis jayconrod


text:
x/tools/gopls: gofmt -s not formatting correctly on save

What version of Go are you using (go version)?

$ go version
go version go1.13.8 linux/amd64

What version of gopls are you using (gopls version)?

$ gopls version
golang.org/x/tools/gopls v0.3.2
    golang.org/x/tools/gopls@v0.3.2 h1:eP1aj1AvT6ynElQH6KP0mmOT2gnWa1gYclHL4wGUbMo=

Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env)?
go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/ko1dli/.cache/go-build"
GOENV="/home/ko1dli/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/ko1dli/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/linuxbrew/.linuxbrew/Cellar/go/1.13.8/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/linuxbrew/.linuxbrew/Cellar/go/1.13.8/libexec/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc-5"
CXX="g++-5"
CGO_ENABLED="0"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build977859433=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Given settings.json with
    "go.formatTool": "gofmt",
    "go.formatFlags": ["-s"],
    "[go]": {
        "editor.formatOnSave": true,
        "editor.codeActionsOnSave": {
            "source.organizeImports": true,
        },
        // Optional: Disable snippets, as they conflict with completion ranking.
        "editor.snippetSuggestions": "none",
    },

testKey := []*apigateway.ApiKey{
		&apigateway.ApiKey{
			Name:  aws.String("test"),
			Id:    aws.String("test"),
			Value: aws.String("ThisIsMyApiKey"),
		},
	}

I save my file
What did you expect to see?
Properly formatted code
testKey := []*apigateway.ApiKey{
		{
			Name:  aws.String("test"),
			Id:    aws.String("test"),
			Value: aws.String("ThisIsMyApiKey"),
		},
	}

What did you see instead?
Nothing changes, causing our CI to complain that the files was not formatted using gofmt -s
Relevant logs
gopls -rpc.trace -v check #file# Output
2020/02/14 09:19:17 Info:2020/02/14 09:19:17 Build info
----------
golang.org/x/tools/gopls v0.3.2
    golang.org/x/tools/gopls@v0.3.2 h1:eP1aj1AvT6ynElQH6KP0mmOT2gnWa1gYclHL4wGUbMo=
    github.com/BurntSushi/toml@v0.3.1
    github.com/sergi/go-diff@v1.0.0 h1:Kpca3qRNrduNnOQeazBd0ysaKrUJiIuISHxogkT9RPQ=
    golang.org/x/mod@v0.1.1-0.20191105210325-c90efee705ee h1:WG0RUwxtNT4qqaXX3DPA8zHFNm/D9xaBpxzHt1WcA/E=
    golang.org/x/sync@v0.0.0-20190423024810-112230192c58
    golang.org/x/tools@v0.0.0-20200212213342-7a21e308cf6c h1:D2X+P0Z6ychko7xn2jvd38yxQfdU0eksO4AHfd8AWFI=
    golang.org/x/xerrors@v0.0.0-20191011141410-1b5146add898 h1:/atklqdjdhuosWIl6AIbOeHJjicWYPqR9bpxqxYG2pA=
    honnef.co/go/tools@v0.0.1-2019.2.3 h1:3JgtbtFHMiCmsznwGVTUWbgGov+pVqnlf1dEJTNAXeM=
    mvdan.cc/xurls/v2@v2.1.0 h1:KaMb5GLhlcSX+e+qhbRJODnUUBvlw01jt4yrjFIHAuA=
Go info
go version go1.13.8 linux/amd64
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/ko1dli/.cache/go-build"
GOENV="/home/ko1dli/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/ko1dli/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/linuxbrew/.linuxbrew/Cellar/go/1.13.8/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/linuxbrew/.linuxbrew/Cellar/go/1.13.8/libexec/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc-5"
CXX="g++-5"
CGO_ENABLED="0"
GOMOD="/c/Users/ko1dli/git/base/src/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build981242842=/tmp/go-build -gno-record-gcc-switches"
2020/02/14 09:19:20 Info:2020/02/14 09:19:20 go/packages.Load
snapshot = 0
query = [./... builtin]
packages = 223

gopls logs Output
[Trace - 09:11:49.333 AM] Sending request 'textDocument/codeLens - (15)'.
Params: {"textDocument":{"uri":"file:///c/Users/ko1dli/git/base/src/onboarding/sandbox/pre-token-generation/handler_test.go"}}
[Trace - 09:11:49.333 AM] Received response 'textDocument/codeLens - (15)' in 0ms.
Result: {}
[Trace - 09:11:49.543 AM] Sending request 'textDocument/codeLens - (16)'.
Params: {"textDocument":{"uri":"file:///c/Users/ko1dli/git/base/src/onboarding/sandbox/pre-token-generation/handler_test.go"}}
[Trace - 09:11:49.544 AM] Received response 'textDocument/codeLens - (16)' in 1ms.
Result: {}
[Trace - 09:11:49.852 AM] Sending request 'textDocument/codeAction - (17)'.
Params: {"textDocument":{"uri":"file:///c/Users/ko1dli/git/base/src/onboarding/sandbox/pre-token-generation/handler_test.go"},"range":{"start":{"line":0,"character":0},"end":{"line":244,"character":0}},"context":{"diagnostics":[]}}
[Trace - 09:11:49.857 AM] Received response 'textDocument/codeAction - (17)' in 4ms.
Result: {}
[Trace - 09:11:49.864 AM] Sending request 'textDocument/formatting - (18)'.
Params: {"textDocument":{"uri":"file:///c/Users/ko1dli/git/base/src/onboarding/sandbox/pre-token-generation/handler_test.go"},"options":{"tabSize":4,"insertSpaces":false}}
[Trace - 09:11:49.870 AM] Received response 'textDocument/formatting - (18)' in 5ms.
Result: [{"range":{"start":{"line":35,"character":1},"end":{"line":36,"character":1}},"newText":""},{"range":{"start":{"line":43,"character":0},"end":{"line":44,"character":1}},"newText":""}]
[Trace - 09:11:49.938 AM] Sending notification 'textDocument/didChange'.
Params: {"textDocument":{"uri":"file:///c/Users/ko1dli/git/base/src/onboarding/sandbox/pre-token-generation/handler_test.go","version":3},"contentChanges":[{"range":{"start":{"line":43,"character":0},"end":{"line":44,"character":1}},"rangeLength":2,"text":""},{"range":{"start":{"line":35,"character":1},"end":{"line":36,"character":1}},"rangeLength":2,"text":""}]}
[Trace - 09:11:50.033 AM] Sending notification 'textDocument/didSave'.
Params: {"textDocument":{"uri":"file:///c/Users/ko1dli/git/base/src/onboarding/sandbox/pre-token-generation/handler_test.go","version":3}}
[Trace - 09:11:50.102 AM] Sending request 'textDocument/foldingRange - (19)'.
Params: {"textDocument":{"uri":"file:///c/Users/ko1dli/git/base/src/onboarding/sandbox/pre-token-generation/handler_test.go"}}
[Trace - 09:11:50.103 AM] Received response 'textDocument/foldingRange - (19)' in 0ms.
Result: [{"startLine":2,"startCharacter":8,"endLine":13,"endCharacter":37,"kind":"imports"},{"startLine":26,"startCharacter":50,"endLine":58,"endCharacter":103},{"startLine":61,"startCharacter":66,"endLine":114,"endCharacter":103},{"startLine":117,"startCharacter":71,"endLine":169,"endCharacter":93},{"startLine":172,"startCharacter":41,"endLine":183,"endCharacter":93},{"startLine":186,"startCharacter":39,"endLine":197,"endCharacter":93},{"startLine":200,"startCharacter":72,"endLine":233,"endCharacter":25},{"startLine":236,"startCharacter":41,"endLine":240,"endCharacter":48}]
[Trace - 09:11:50.312 AM] Sending request 'textDocument/codeLens - (20)'.
Params: {"textDocument":{"uri":"file:///c/Users/ko1dli/git/base/src/onboarding/sandbox/pre-token-generation/handler_test.go"}}
[Trace - 09:11:50.315 AM] Received response 'textDocument/codeLens - (20)' in 2ms.
Result: {}
[Trace - 09:11:50.482 AM] Sending notification 'workspace/didChangeWatchedFiles'.
Params: {"changes":[{"uri":"file:///c/Users/ko1dli/git/base/src/onboarding/sandbox/pre-token-generation/handler_test.go","type":2}]}
[Trace - 09:11:50.486 AM] Sending request 'textDocument/codeLens - (21)'.
Params: {"textDocument":{"uri":"file:///c/Users/ko1dli/git/base/src/onboarding/sandbox/pre-token-generation/handler_test.go"}}
[Trace - 09:11:50.489 AM] Received response 'textDocument/codeLens - (21)' in 3ms.
Result: {}
[Trace - 09:11:50.639 AM] Sending request 'textDocument/documentSymbol - (22)'.
Params: {"textDocument":{"uri":"file:///c/Users/ko1dli/git/base/src/onboarding/sandbox/pre-token-generation/handler_test.go"}}
[Trace - 09:11:50.642 AM] Received response 'textDocument/documentSymbol - (22)' in 2ms.
Result: [{"name":"apiMock","detail":"*mocks.MockApigwService","kind":13,"range":{"start":{"line":16,"character":0},"end":{"line":16,"character":35}},"selectionRange":{"start":{"line":16,"character":4},"end":{"line":16,"character":11}}},{"name":"r","detail":"CognitoEventUserPoolsPreTokenGenerationRequest","kind":13,"range":{"start":{"line":18,"character":0},"end":{"line":22,"character":1}},"selectionRange":{"start":{"line":18,"character":4},"end":{"line":18,"character":5}}},{"name":"event","detail":"CognitoEventUserPoolsPreTokenGeneration","kind":13,"range":{"start":{"line":24,"character":0},"end":{"line":24,"character":63}},"selectionRange":{"start":{"line":24,"character":4},"end":{"line":24,"character":9}}},{"name":"TestPreTokenGenerationSuccess","detail":"(t *testing.T)","kind":12,"range":{"start":{"line":26,"character":0},"end":{"line":59,"character":1}},"selectionRange":{"start":{"line":26,"character":5},"end":{"line":26,"character":34}}},{"name":"TestPreTokenGenerationSuccessWithMultipleKeys","detail":"(t *testing.T)","kind":12,"range":{"start":{"line":61,"character":0},"end":{"line":115,"character":1}},"selectionRange":{"start":{"line":61,"character":5},"end":{"line":61,"character":50}}},{"name":"TestPreTokenGenerationNoStageFoundWithMultipleKeys","detail":"(t *testing.T)","kind":12,"range":{"start":{"line":117,"character":0},"end":{"line":170,"character":1}},"selectionRange":{"start":{"line":117,"character":5},"end":{"line":117,"character":55}}},{"name":"TestGetApiKeysFailed","detail":"(t *testing.T)","kind":12,"range":{"start":{"line":172,"character":0},"end":{"line":184,"character":1}},"selectionRange":{"start":{"line":172,"character":5},"end":{"line":172,"character":25}}},{"name":"TestPreTokenNoKeys","detail":"(t *testing.T)","kind":12,"range":{"start":{"line":186,"character":0},"end":{"line":198,"character":1}},"selectionRange":{"start":{"line":186,"character":5},"end":{"line":186,"character":23}}},{"name":"TestValidateCognitoEventUserPoolsPreTokenGeneration","detail":"(t *testing.T)","kind":12,"range":{"start":{"line":200,"character":0},"end":{"line":234,"character":1}},"selectionRange":{"start":{"line":200,"character":5},"end":{"line":200,"character":56}}},{"name":"createMocks","detail":"(t gomock.TestReporter)","kind":12,"range":{"start":{"line":236,"character":0},"end":{"line":241,"character":1}},"selectionRange":{"start":{"line":236,"character":5},"end":{"line":236,"character":16}}}]
[Trace - 09:11:50.650 AM] Sending request 'textDocument/codeLens - (23)'.
Params: {"textDocument":{"uri":"file:///c/Users/ko1dli/git/base/src/onboarding/sandbox/pre-token-generation/handler_test.go"}}
[Trace - 09:11:50.651 AM] Received response 'textDocument/codeLens - (23)' in 0ms.
Result: {}
[Trace - 09:11:50.906 AM] Sending request 'textDocument/documentLink - (24)'.
Params: {"textDocument":{"uri":"file:///c/Users/ko1dli/git/base/src/onboarding/sandbox/pre-token-generation/handler_test.go"}}
[Trace - 09:11:50.923 AM] Received response 'textDocument/documentLink - (24)' in 16ms.
Result: [{"range":{"start":{"line":3,"character":2},"end":{"line":3,"character":9}},"target":"https://pkg.go.dev/context"},{"range":{"start":{"line":4,"character":2},"end":{"line":4,"character":8}},"target":"https://pkg.go.dev/errors"},{"range":{"start":{"line":5,"character":2},"end":{"line":5,"character":31}},"target":"https://pkg.go.dev/sas/common/aws/lambda/handler"},{"range":{"start":{"line":6,"character":2},"end":{"line":6,"character":21}},"target":"https://pkg.go.dev/sas/common/aws/xray"},{"range":{"start":{"line":7,"character":2},"end":{"line":7,"character":18}},"target":"https://pkg.go.dev/sas/common/mocks"},{"range":{"start":{"line":8,"character":2},"end":{"line":8,"character":9}},"target":"https://pkg.go.dev/testing"},{"range":{"start":{"line":10,"character":2},"end":{"line":10,"character":31}},"target":"https://pkg.go.dev/github.com/aws/aws-sdk-go/aws"},{"range":{"start":{"line":11,"character":2},"end":{"line":11,"character":46}},"target":"https://pkg.go.dev/github.com/aws/aws-sdk-go/service/apigateway"},{"range":{"start":{"line":12,"character":2},"end":{"line":12,"character":31}},"target":"https://pkg.go.dev/github.com/golang/mock/gomock"},{"range":{"start":{"line":13,"character":2},"end":{"line":13,"character":36}},"target":"https://pkg.go.dev/github.com/stretchr/testify/assert"},{"range":{"start":{"line":20,"character":17},"end":{"line":20,"character":24}},"target":"https://test.se"}]
2020/02/14 09:12:02 background imports cache refresh starting
2020/02/14 09:12:11 background refresh finished after 8.699511s with err:

gopls does not support the -s flag for formatting yet, and since you are using it for formatting, these settings have no effect:
"go.formatTool": "gofmt",
"go.formatFlags": ["-s"],

You can disable gopls formatting and use gofmt directly by setting
"go.experimentalLanguageServerFeatures": {
    "format": false,
}

/cc @ridersofrohan

authors: dan-lind stamblerre


text:
os: if rename system call fails, try ioctl(FICLONE)

What version of Go are you using (go version)?

$ go version
go version go1.15.2 darwin/amd64

Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env)?
go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/danny/Library/Caches/go-build"
GOENV="/Users/danny/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/danny/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/danny/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.15.2/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.15.2/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/danny/workspace/media-sort/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/36/kj75t85j1m57cg047cr5dh4m0000gn/T/go-build628346493=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?
I am cross-compiling on Mac for Linux
Linux execution env: Linux nas 4.4.190.x86_64.1 #1 SMP Mon Oct 28 01:55:46 UTC 2019 x86_64 GNU/Linux
Build command: GOOS=linux GOARCH=amd64 go build -v
func move(src, dst string) error {
	err := os.Rename(src, dst)
	// cross device move
	if err != nil && strings.Contains(err.Error(), "cross-device") {
		if err := copy(src, dst); err != nil {
			return err
		}
		if err := os.Remove(src); err != nil {
			return err
		}
	}
	return nil
}

What did you expect to see?
os.Rename() to succeed because the source and target are on the same filesystem.
What did you see instead?
os.Rename() returns "invalid cross-device link" and the above code falls back to copy-and-delete. If I do an mv on the command line, it completes immediately - even for multi-gigabyte files.

os.Rename is a fairly thin wrapper around syscall.Rename, which just calls the rename system call.  If possible, please run the final GNU/Linux executable under strace -f and see what the rename system call returns.

Thanks for your quick response @ianlancetaylor. To test, I made an empty file, f, in /data/Videos on my NAS, and mv-ed it to /data/Music. Here's the strace:
execve("/bin/mv", ["mv", "f", "/data/Music"], [/* 17 vars */]) = 0
brk(0)                                  = 0x16eb000
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fe1406d8000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=28370, ...}) = 0
mmap(NULL, 28370, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fe1406d1000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libselinux.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20c\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=142728, ...}) = 0
mmap(NULL, 2246896, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fe140295000
mprotect(0x7fe1402b6000, 2097152, PROT_NONE) = 0
mmap(0x7fe1404b6000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x21000) = 0x7fe1404b6000
mmap(0x7fe1404b8000, 6384, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fe1404b8000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libacl.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\200\37\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=35288, ...}) = 0
mmap(NULL, 2130592, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fe14008c000
mprotect(0x7fe140094000, 2093056, PROT_NONE) = 0
mmap(0x7fe140293000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x7000) = 0x7fe140293000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libattr.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\320\23\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=18640, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fe1406d0000
mmap(NULL, 2113912, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fe13fe87000
mprotect(0x7fe13fe8b000, 2093056, PROT_NONE) = 0
mmap(0x7fe14008a000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3000) = 0x7fe14008a000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P\34\2\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1738176, ...}) = 0
mmap(NULL, 3844640, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fe13fadc000
mprotect(0x7fe13fc7d000, 2097152, PROT_NONE) = 0
mmap(0x7fe13fe7d000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1a1000) = 0x7fe13fe7d000
mmap(0x7fe13fe83000, 14880, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fe13fe83000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libpcre.so.3", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\27\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=448440, ...}) = 0
mmap(NULL, 2543976, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fe13f86e000
mprotect(0x7fe13f8da000, 2097152, PROT_NONE) = 0
mmap(0x7fe13fada000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6c000) = 0x7fe13fada000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\320\16\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=14664, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fe1406cf000
mmap(NULL, 2109712, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fe13f66a000
mprotect(0x7fe13f66d000, 2093056, PROT_NONE) = 0
mmap(0x7fe13f86c000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7fe13f86c000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\320n\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=137384, ...}) = 0
mmap(NULL, 2213008, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fe13f44d000
mprotect(0x7fe13f465000, 2093056, PROT_NONE) = 0
mmap(0x7fe13f664000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x17000) = 0x7fe13f664000
mmap(0x7fe13f666000, 13456, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fe13f666000
close(3)                                = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fe1406ce000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fe1406cc000
arch_prctl(ARCH_SET_FS, 0x7fe1406cc800) = 0
mprotect(0x7fe13fe7d000, 16384, PROT_READ) = 0
mprotect(0x7fe13f664000, 4096, PROT_READ) = 0
mprotect(0x7fe13f86c000, 4096, PROT_READ) = 0
mprotect(0x7fe13fada000, 4096, PROT_READ) = 0
mprotect(0x7fe14008a000, 4096, PROT_READ) = 0
mprotect(0x7fe140293000, 4096, PROT_READ) = 0
mprotect(0x7fe1404b6000, 4096, PROT_READ) = 0
mprotect(0x61e000, 4096, PROT_READ)     = 0
mprotect(0x7fe1406da000, 4096, PROT_READ) = 0
munmap(0x7fe1406d1000, 28370)           = 0
set_tid_address(0x7fe1406ccad0)         = 23496
set_robust_list(0x7fe1406ccae0, 24)     = 0
rt_sigaction(SIGRTMIN, {0x7fe13f4539b0, [], SA_RESTORER|SA_SIGINFO, 0x7fe13f45c890}, NULL, 8) = 0
rt_sigaction(SIGRT_1, {0x7fe13f453a40, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x7fe13f45c890}, NULL, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0
statfs("/sys/fs/selinux", 0x7ffc57d353f0) = -1 ENOENT (No such file or directory)
statfs("/selinux", {f_type="EXT2_SUPER_MAGIC", f_bsize=4096, f_blocks=965552, f_bfree=722015, f_bavail=665542, f_files=1048576, f_ffree=1032627, f_fsid={-1827525020, -625743731}, f_namelen=255, f_frsize=4096}) = 0
brk(0)                                  = 0x16eb000
brk(0x170c000)                          = 0x170c000
open("/proc/filesystems", O_RDONLY)     = 3
fstat(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fe1406d7000
read(3, "nodev\tsysfs\nnodev\trootfs\nnodev\tt"..., 1024) = 370
read(3, "", 1024)                       = 0
close(3)                                = 0
munmap(0x7fe1406d7000, 4096)            = 0
open("/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=6839968, ...}) = 0
mmap(NULL, 6839968, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fe13edc7000
close(3)                                = 0
geteuid()                               = 0
ioctl(0, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
stat("/data/Music", {st_mode=S_IFDIR|0777, st_size=38, ...}) = 0
lstat("f", {st_mode=S_IFREG|0666, st_size=0, ...}) = 0
lstat("/data/Music/f", 0x7ffc57d35060)  = -1 ENOENT (No such file or directory)
rename("f", "/data/Music/f")            = -1 EXDEV (Invalid cross-device link)
unlink("/data/Music/f")                 = -1 ENOENT (No such file or directory)
open("f", O_RDONLY|O_NOFOLLOW)          = 3
fstat(3, {st_mode=S_IFREG|0666, st_size=0, ...}) = 0
open("/data/Music/f", O_WRONLY|O_CREAT|O_EXCL, 0600) = 4
fstat(4, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0
ioctl(4, BTRFS_IOC_CLONE, 0x3)          = 0
utimensat(4, NULL, {{1600461257, 619450219}, {1600461257, 619450219}}, 0) = 0
flistxattr(3, NULL, 0)                  = 24
flistxattr(3, "system.posix_acl_access\0", 24) = 24
open("/etc/xattr.conf", O_RDONLY)       = -1 ENOENT (No such file or directory)
fgetxattr(3, "system.posix_acl_access", 0x0, 0) = 68
fgetxattr(3, "system.posix_acl_access", "\x02\x00\x00\x00\x01\x00\x06\x00\xff\xff\xff\xff\x02\x00\x07\x00b\x00\x00\x00\x02\x00\x07\x00c\x00\x00\x00\x04\x00\x07\x00\xff\xff\xff\xff\x08\x00\x07\x00b\x00\x00\x00\x08\x00\x07\x00c\x00\x00\x00\x10\x00\x06\x00\xff\xff\xff\xff \x00\x06\x00\xff\xff\xff\xff", 68) = 68
fsetxattr(4, "system.posix_acl_access", "\x02\x00\x00\x00\x01\x00\x06\x00\xff\xff\xff\xff\x02\x00\x07\x00b\x00\x00\x00\x02\x00\x07\x00c\x00\x00\x00\x04\x00\x07\x00\xff\xff\xff\xff\x08\x00\x07\x00b\x00\x00\x00\x08\x00\x07\x00c\x00\x00\x00\x10\x00\x06\x00\xff\xff\xff\xff \x00\x06\x00\xff\xff\xff\xff", 68, 0) = 0
fgetxattr(3, "system.posix_acl_access", "\x02\x00\x00\x00\x01\x00\x06\x00\xff\xff\xff\xff\x02\x00\x07\x00b\x00\x00\x00\x02\x00\x07\x00c\x00\x00\x00\x04\x00\x07\x00\xff\xff\xff\xff\x08\x00\x07\x00b\x00\x00\x00\x08\x00\x07\x00c\x00\x00\x00\x10\x00\x06\x00\xff\xff\xff\xff \x00\x06\x00\xff\xff\xff\xff", 132) = 68
fsetxattr(4, "system.posix_acl_access", "\x02\x00\x00\x00\x01\x00\x06\x00\xff\xff\xff\xff\x02\x00\x07\x00b\x00\x00\x00\x02\x00\x07\x00c\x00\x00\x00\x04\x00\x07\x00\xff\xff\xff\xff\x08\x00\x07\x00b\x00\x00\x00\x08\x00\x07\x00c\x00\x00\x00\x10\x00\x06\x00\xff\xff\xff\xff \x00\x06\x00\xff\xff\xff\xff", 68, 0) = 0
close(4)                                = 0
close(3)                                = 0
lstat("/", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
newfstatat(AT_FDCWD, "f", {st_mode=S_IFREG|0666, st_size=0, ...}, AT_SYMLINK_NOFOLLOW) = 0
unlinkat(AT_FDCWD, "f", 0)              = 0
lseek(0, 0, SEEK_CUR)                   = -1 ESPIPE (Illegal seek)
close(0)                                = 0
close(1)                                = 0
close(2)                                = 0
exit_group(0)                           = ?
+++ exited with 0 +++

I'm waaaay out of my depth here, but I'm guessing these lines are the important ones:
rename("f", "/data/Music/f")            = -1 EXDEV (Invalid cross-device link)
[... followed shortly by ...]
ioctl(4, BTRFS_IOC_CLONE, 0x3)          = 0

It looks like the rename call fails, and mv falls back to using a BTRFS-specific alternative.

Interesting.  I was not aware of that.  I'll retitle this issue.

I've tried to follow the source for mv, and... well... I'm not a C programmer! 🤣 But ultimately, it ends up here: https://github.com/coreutils/coreutils/blob/master/src/copy.c#L405-L418, which, AFAICT, comes from here: https://github.com/coreutils/coreutils/blob/master/src/copy.c#L1459.
Ultimately, it appears to all be controlled by the defaulting of the following two flags in mv.c:
https://github.com/coreutils/coreutils/blob/master/src/mv.c#L113
https://github.com/coreutils/coreutils/blob/master/src/mv.c#L130

It's interesting that BTRFS reports those two directories as being on different filesystems, but is then happy to do a CoW!

Maybe related: https://superuser.com/questions/948808/why-is-moving-files-between-btrfs-subvolumes-an-expensive-operation

authors: codeaholics ianlancetaylor thanm


text:
proposal: ECDH interface and dev.boringssl

Currently crypto/elliptic exposes an interface that uses points and x and y coordinates. This is convenient enough for many applications, but poses problems when trying to work within the constraints of FIPS 140-2.
I'd like to propose a crypto/ecdh package, that has a PublicKey and PrivateKey types.
The methods are GenerateKeypair(*elliptic.Curve) that returns a keypair, and PrivateKey.Agree(*ecdh.PublicKey) that returns bytes as in the NIST recommended generation method used in TLS. It could also take a KDF and apply that, but in that case I'd like to support auxillary input as SP 800-56C permits. Feel free to bikeshed this further.
My uninformed guess is many callers of elliptic outside the standard library are actually implementing something like the ecdh package and would benefit from having it in the standard library.
If I understand correctly currently dev.boringssl doesn't replace any of the functions in elliptic, but could easily replace these with calls to BoringSSL.
I think this is related to the questions in https://github.com/golang/go/issues/30158

Funny timing. I was just looking into this myself the other day and was surprised to find that an official package doesn't exist, even in x/crypto.
It should be noted, however, that some form ECDH is fairly easy to do manually with the existing system, though it's certainly far from convenient. For example,
// GenerateECDHKey takes your private key and the peer's public key and returns a symmetric key large enough for use with AES256.
func GenerateECDHKey(priv []byte, x, y *big.Int) []byte {
  keycoord, _ := elliptic.P256().ScalarMult(x, y, priv)
  return sha256.Sum256(keycoord.Bytes())[:]
}

Please note: I am not in any way a cryptography expert. If the above is a bad way to do that, please take that as evidence that this should be implemented in the standard library.

Funny timing. I was just looking into this myself the other day and was surprised to find that an official package doesn't exist, even in x/crypto.
It should be noted, however, that some form ECDH is fairly easy to do manually with the existing system, though it's certainly far from convenient. For example,
// GenerateECDHKey takes your private key and the peer's public key and returns a symmetric key large enough for use with AES256.
func GenerateECDHKey(priv []byte, x, y *big.Int) []byte {
  keycoord, _ := elliptic.P256().ScalarMult(x, y, priv)
  return sha256.Sum256(keycoord.Bytes())[:]
}

Please note: I am not in any way a cryptography expert. If the above is a bad way to do that, please take that as evidence that this should be implemented in the standard library.

I don't have yet formed opinions on the color of the bikeshed, but I am interested in working on a set of APIs for NIST P curves that are []byte based. In fact, what would help a lot is a survey of what developers are using crypto/elliptic for, and how.

I don't have yet formed opinions on the color of the bikeshed, but I am interested in working on a set of APIs for NIST P curves that are []byte based. In fact, what would help a lot is a survey of what developers are using crypto/elliptic for, and how.

Does that mean "package elliptic" is not using FIPS 140-2 boringssl code and thus can not be claimed as fips approved? Is there any plan to include boringssl support for elliptic package?

Seems to be a duplicate of #52221.

This proposal is a duplicate of a previously discussed proposal, as noted above,
and there is no significant new information to justify reopening the discussion.
The issue has therefore been declined as a duplicate.
— rsc for the proposal review group

authors: DeedleFake FiloSottile rsc santoshrokade wbl


text:
spec: similar sounding nomenclature is confusing

In the spec we use the following phrases:

1) integer value
2) value of integer type
3) value of type int (or int8, int16, etc)
4) representable as a value of type int (or int8, int16, etc)

They are very similar in English but mean different things in the spec:

1) -> a (typed or untyped) constant value that is integer (1, 2.0, -4.0 + 0i)
2) -> a typed value that has one of the signed/unsigned integer types (byte, int,
uint8, etc.)
3) -> a value of a concrete type int, int8, etc.
4) -> a value that can be represented w/o loss of accuracy as a value of a specific
type

We should clean this up, and/or introduce a section explaining the nomenclature. As is,
the English differences are small, but the spec implications are large.

has been addressed with an explicit definition of "representability" (https://go-review.googlesource.com/57530)
all uses of "value of type T" in the spec are very clear
all uses of "value of integer type" in the spec are also clear (no ambiguities)
all uses of "integer value" have been clarified over time - it's always a value of integer type or an untyped integer constant (never an untyped constant that is not an integer constant but happens to have an integer value, such as 1.0). Furthermore, the section on numeric types defines each set as a set of "integer or floating-point values" which defines the term.

Closing.


text:
x/build: install fossil, bzr, and gpg binaries on linux-*-longtest builders

The go command at least nominally supports five different version control systems: Git, Mercurial, Subversion, Fossil, and Bazaar. Of those, only three appear to be installed on the linux-amd64-longtest builder. That all but guarantees that regressions in support for the other two will not be detected by the builders.
The two missing tools are Fossil (provided by the fossil Debian package), and Bazaar (somewhat confusingly, now provided by the brz Debian package).
CC @golang/release

The linux-x86-bullseye image has been updated after the submission of go.dev/cl/383156. This issue should be resolved after we switch the linux longtest builders to use the updated Debian bullseye image.

The image has been updated. The issue should be resolved after the next coordinator deployment.

authors: bcmills cagedmantis


text:
runtime: Inconsistency Multiplication, Division, and Addition Result

What version of Go are you using (go version)?

$ go version
go version go1.17.6 windows/amd64

Does this issue reproduce with the latest release?
What operating system and processor architecture are you using (go env)?
go env Output
$ go env
set GO111MODULE=
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\user\AppData\Local\go-build
set GOENV=C:\Users\user\AppData\Roaming\go\env
set GOEXE=.exe
set GOEXPERIMENT=
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GOMODCACHE=C:\Users\user\go\pkg\mod
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=C:\Users\user\go
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=C:\Program Files\Go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=C:\Program Files\Go\pkg\tool\windows_amd64
set GOVCS=
set GOVERSION=go1.17.6
set GCCGO=gccgo
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=C:\Users\user\Documents\interview-codes\go.mod
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=C:\Users\user\AppData\Local\Temp\go-build2510967983=/tmp/go-build -gno-record-gcc-switches

What did you do?

What did you expect to see?
I thought that c must be eqal to d
What did you see instead?
In another case i get correct results just like this

but the arithmetic give difference results by this formula

I got same issue with division and addition

test codes

func c1() {
a	:=	0.61	*	275.0
b	:=	0.62	*	275.0
c	:=	0.63	*	275.0
d	:=	0.64	*	275.0
e	:=	0.65	*	275.0
f	:=	0.66	*	275.0
g	:=	0.67	*	275.0
h	:=	0.68	*	275.0
i	:=	0.69	*	275.0
fmt.Printf("%g\n%g\n%g\n%g\n%g\n%g\n%g\n%g\n%g\n", a, b, c, d, e, f, g, h, i)
}

func c2() {
z := 275.
a	:=	0.61	*	z
b	:=	0.62	*	z
c	:=	0.63	*	z
d	:=	0.64	*	z
e	:=	0.65	*	z
f	:=	0.66	*	z
g	:=	0.67	*	z
h	:=	0.68	*	z
i	:=	0.69	*	z
fmt.Printf("%g\n%g\n%g\n%g\n%g\n%g\n%g\n%g\n%g\n", a, b, c, d, e, f, g, h, i)
}

func c3() {
	for i := 0; i < 10; i++ {
		a := float64(i)/100 + 0.5
		fmt.Printf("i = %d; a =  %g\n", i, a)
	}
}

func c4() {
	z := 0.5
	a := 0.01 + z
	b := 0.02 + z
	c := 0.03 + z
	d := 0.04 + z
	e := 0.05 + z
	f := 0.06 + z
	g := 0.07 + z
	h := 0.08 + z
	i := 0.09 + z
	fmt.Printf("%g\n%g\n%g\n%g\n%g\n%g\n%g\n%g\n%g\n", a, b, c, d, e, f, g, h, i)
}

func c5() {
	a := 0.01 + 0.5
	b := 0.02 + 0.5
	c := 0.03 + 0.5
	d := 0.04 + 0.5
	e := 0.05 + 0.5
	f := 0.06 + 0.5
	g := 0.07 + 0.5
	h := 0.08 + 0.5
	i := 0.09 + 0.5
	fmt.Printf("%g\n%g\n%g\n%g\n%g\n%g\n%g\n%g\n%g\n", a, b, c, d, e, f, g, h, i)
}

This is how floating-point calculations work on all computers vs. how constants work in Go. They do not behave the same.
Please read these two links:
https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html
https://go.dev/blog/constants

Please post plain text as plain text, not as images.  Images are difficult to read.  Thanks.
Closing this issue because it is not a bug in Go.  Please comment if you disagree.

authors: gho1b ianlancetaylor robpike


text:
cmd/dist: build cmd with CGO_ENABLED=0, so cmd/go uses pure Go net resolver

Now that we've removed the .a files from the distribution, the only possible leakage of the host build environment into Go Linux releases is the dynamic libc.so.6 path embedded in the command binaries that use the network (only bin/go and pkg/tool/goos_goarch/pprof, I believe).
I wonder whether we should build cmd with CGO_ENABLED=0 so that we end up with purely static cmd/go and cmd/pprof binaries. This would apply to all systems, but since we are planning to cross-compile non-Linux distributions and Mac and Windows don't use cgo anyway for net, the only effect would be on Linux.
We default to the cgo-based net so that people with strange /etc/resolv.conf can still resolve the names they need to resolve. But perhaps that is more for local names like mDNS and such rather than the standard internet names that cmd/go needs.
Does anyone see anything that would break if we made the go command always use the pure Go net resolver?

Would this cause go install cmd to install a different cmd/go from what make.bash produces?
That might be ok, but we would probably at least have to update some tests that assume it is a no-op.
(But it also might provide a reasonable workaround for folks who want a cmd/go that uses the C resolver.)

Would this cause go install cmd to install a different cmd/go from what make.bash produces?
That might be ok, but we would probably at least have to update some tests that assume it is a no-op.

Yes, and agreed.

With a little more work this lets us build completely reproducible distributions, where it doesn't matter which OS the distribution was built on at all. In particular they could all be built on a single trusted system. CL 454836 does this.

We default to the cgo-based net so that people with strange /etc/resolv.conf can still resolve the names they need to resolve. But perhaps that is more for local names like mDNS and such rather than the standard internet names that cmd/go needs.

I don't think that resolv.conf might be a problem, go 1.20 supports most options that are used in the wild.

Does anyone see anything that would break if we made the go command always use the pure Go net resolver?

I think that some users might have problem because of /etc/nsswitch.conf, when we compiled without cgo we defualt to files,dns order, we don't even parse that file.

In what circumstances would nsswitch.conf be needed to resolve public internet names like github.com or proxy.golang.org?

In what circumstances would nsswitch.conf be needed to resolve public internet names like github.com or proxy.golang.org?

Glibc uses nsswitch.conf to resolve all names. Recently linux distros (desktop) started adopting systemd-resolve nss module, instead of a dns one, so that the dns module is not even used to resolve hostnames, the dns queries are send by the systemd-resolved deamon.
I mean if we changed the go binary to use netgo by default, then probably nothing will break, but we don't know how the nsswitch.conf configurations might evolve in the future.

I guess we will find out - there are enough upsides to this change, including cross-compiled reproducible builds, that I think we owe it to ourselves to try it in Go 1.21 and find out whether there are any show-stoppers.

#57328 is an interesting data point here too. There, a user is reporting that building a cgo-enabled dynamically-linked binary on an Ubuntu with a newer glibc causes that binary to fail to run on a system with an older glibc.
That seems to imply that if we build a cgo-enabled cmd/go against a recent glibc, it will similarly fail to run on an older system. But we probably also don't want to build our releases on ancient versions of our supported platforms just to improve the glibc compatibility of the resulting binaries.
Building the release with CGO_ENABLED=0 nicely sidesteps that problem, allowing Go releases to be built on more up-to-date platforms.

failures due to glibc versions were also seen a long time ago in #5203

authors: bcmills mateusz834 rsc seankhliao


text:
x/pkgsite: flag to open browser

When running pkgsite locally it would be great to have a command line flag to automatically load the site in the users browser.

cc @golang/pkgsite

authors: seankhliao stevenh


text:
Only use the default email when name is empty

I've also added angle-bracket email stripping which is also related to the issue. Tests are modified so they pass.
See https://github.com/gollum/gollum/issues/954.

Many thanks!

My pleasure :)

authors: dometto simonzack

repo: gollum/gollum-lib

license: MIT


text:
Determine markup format from format name rather than filename

I wanted to use Gollum's ability to change file extensions associated with markup formats to associate Markdown with the primary extension ".wiki". While I was able to get Gollum to create files with the ".wiki" extension when the Markdown format was selected in the editor, these files would not be rendered as Markdown. This persisted even after I cleared the Mediawiki and other formats.
After digging around in the code I figured out the problem: the github/markup gem's render function determines what format to use based on the filename, and Gollum::Markup.formats is not taken into consideration when running it.
I took a stab at hacking up a solution by providing GitHub::Markup.render with a fake filename using the format's :name as its extension. Unfortunately, I had to include a couple of hardcoded if statements to not break Org-mode and reStructuredText formatting. And really the whole solution is quite a nasty hack. Any suggestions of how to do this in a nicer way?

@dmyates sorry for the extremely late reply. I think a better approach would be to start using github-markup's render_s method, which determines the format based on a symbol. Then all we have to do is make sure the symbols in Gollum::Markups match up with the one's that github-markup uses. Would you still be interested in implementing a cleaner fix? If so, please submit a new PR versus the gollum-lib-5.x development branch. Many thanks in advance!

Cool! 👍

authors: bartkamphorst dmyates dometto


text:
Storing base64 data in a variable

Recording is working however, I'm trying to store the data that I'm receiving through
AudioRecord.on('data', data => {
  // base64-encoded audio data chunks
});

in a variable. However, it's not working for some reason and I'm really struggling with this.
I've tried doing this (within a function):
var tempString = '';
AudioRecord.on('data', data => {
  tempString += data;
});

but it only gives the first part of the data. If I just set it to the data and output it, it does gives different data but never when I try to add all the data together. I've tried arrays, listening to a changing variable and using another function but for some reason they all just gave the first data that is received if I used a similar method to the one I described above.
So how do I store all the data that I get from a 3 second audio recording in a variable?

Did you find any solution to this issue?

It turned out it was working after all, was an issue on my end.

@StevenJW I'm happy that you've found the solution! I'm struggling with the same problem atm : )
Could you tell me please, how did you decide to store data in a var?

authors: CarlosdpdClluc StevenJW xKRUTOIx

repo: goodatlas/react-native-audio-record


text:
Support bootstrap of TypeScript projects

The application can bootstrap TypeScript projects now. All the source code master files are rewritten to TypeScript. Babel transpiler is used to create JavaScript files from TypeScript master templates if user chooses to. The transpiled files are then formatted with Prettier to get nicely looking source code files.
The bootstrapped project flavor can be set via command line option "flavor" that accepts values "ts" (TypeScript) and "js" (JavaScript), by the configuration option with the same name in a config file, or during interactive CLI setup by choosing the option from the list.
JIRA: RAIL-2854

Build failed (check pipeline).

https://checklist.intgdc.com/job/client-libs/job/gooddata-create-gooddata-react-app-unit-tests-zuul-docker-el7/182/ : FAILURE in 49s
https://checklist.intgdc.com/job/compliance/job/gooddata-create-gooddata-react-app-oss-check/150/ : SUCCESS in 8s

Build failed (check pipeline).

https://checklist.intgdc.com/job/client-libs/job/gooddata-create-gooddata-react-app-unit-tests-zuul-docker-el7/183/ : FAILURE in 54s
https://checklist.intgdc.com/job/compliance/job/gooddata-create-gooddata-react-app-oss-check/151/ : SUCCESS in 6s

Build failed (check pipeline).

https://checklist.intgdc.com/job/client-libs/job/gooddata-create-gooddata-react-app-unit-tests-zuul-docker-el7/184/ : FAILURE in 59s
https://checklist.intgdc.com/job/compliance/job/gooddata-create-gooddata-react-app-oss-check/152/ : SUCCESS in 7s

The code coverage improved with your change! 🎉
You are the best! Like literary the best!
Report (compared to build #181)
Covered lines: 38.19%, change: 11.39% (additional 18 lines are not covered)
Covered branches: 40.98%, change: 15.05% (additional 4 branches are covered)
Coverage report
https://checklist.intgdc.com/job/client-libs/job/gooddata-create-gooddata-react-app-unit-tests-zuul-docker-el7/185/artifact/ci/results/coverage/lcov-report/index.html

Build succeeded (check pipeline).

https://checklist.intgdc.com/job/client-libs/job/gooddata-create-gooddata-react-app-unit-tests-zuul-docker-el7/185/ : SUCCESS in 1m 06s
https://checklist.intgdc.com/job/compliance/job/gooddata-create-gooddata-react-app-oss-check/153/ : SUCCESS in 7s

Build succeeded (check pipeline).

https://checklist.intgdc.com/job/client-libs/job/gooddata-create-gooddata-react-app-unit-tests-zuul-docker-el7/186/ : SUCCESS in 20s
https://checklist.intgdc.com/job/compliance/job/gooddata-create-gooddata-react-app-oss-check/154/ : SUCCESS in 6s

Build succeeded (check pipeline).

https://checklist.intgdc.com/job/client-libs/job/gooddata-create-gooddata-react-app-unit-tests-zuul-docker-el7/187/ : SUCCESS in 53s
https://checklist.intgdc.com/job/compliance/job/gooddata-create-gooddata-react-app-oss-check/155/ : SUCCESS in 6s

Build succeeded (check pipeline).

https://checklist.intgdc.com/job/client-libs/job/gooddata-create-gooddata-react-app-unit-tests-zuul-docker-el7/188/ : SUCCESS in 38s
https://checklist.intgdc.com/job/compliance/job/gooddata-create-gooddata-react-app-oss-check/156/ : SUCCESS in 6s

Build succeeded (check pipeline).

https://checklist.intgdc.com/job/client-libs/job/gooddata-create-gooddata-react-app-unit-tests-zuul-docker-el7/189/ : SUCCESS in 18s
https://checklist.intgdc.com/job/compliance/job/gooddata-create-gooddata-react-app-oss-check/157/ : SUCCESS in 11s

authors: gdgate xMort yenkins


text:
CornerRadius

The corner radius is not the same as in system alerts and there is no API to configure it.

I'll add the API to configure it.

System default.
iOS 9: 20
iOS 8: 12

repo: goodpatch/PKAlertController

license: mit


text:
tunning  text_generator_main.cc performance

Description of the bug:
Hi,
when i using text_generator_main.cc to inference, i found the performance is slower ollama or other inference engine, Do you have any suggestion to tuning the model or imporve it in text_generator_main.cc?
i using tinlyllama model
Actual vs expected behavior:
No response
Any other information you'd like to share?
No response

Hi @nigelzzz, can you provide some profiling data in both these cases? I think we care more about using the model/measuring performance in a real setting (i.e. in a mobile app). So I would say go ahead and use ollama if you feel that suits your needs better for now.

Hi @nigelzzz, "computer graph optimization" is very general and I think the entire point for compiling with the current graph computation paradigm ... so we are always doing computer graph optimization... continuously, unless you meant something different or if you had a more specific question?

Hi @pkgoogle ,
thanks for your response again!!
Because i am not familiar with deep learning, if i need to learn computer graph optimation, how to get more information in ai-edge-torch. e.g., how to implement in ai-edge-torch source code.
I think learn it on ai-edge-torcht that can enhance my knowledge on optimaztion in deeplearning field.

Oh I see, no worries. If you are in school, I highly recommend to see what internal resources you may have available first, otherwise, perhaps look at the numerous MOOCS/youtube videos available for deep learning. If you have windows I recommend you start with WSL, otherwise stick w/ your local OS. For AI-Edge-Torch specifically ... I recommend you implement, test, evaluate some basic Neural Networks on your own with PyTorch and Python https://pytorch.org/tutorials/beginner/basics/intro.html. I recommend understanding the history before transformers as well. For graph optimization... understanding the loss function, back propagation, gradient descent will help you understand why backprop optimizes the network towards minimizing the loss function. (This requires some multivariable calculus [backprop is the derivative of the function that the NN computes] & basic linear algebra [mainly to understand the notation]). Currently all optimization is just that, gradient descent on the function (this function has parameter # of variables) that the NN computes. It doesn't seem like this will change soon ... but other techniques to optimize this function may be found in the future.
So for AI-Edge-Torch ... you have to understand a little bit MLIR and compilers. Like how do you convert a graph representation to binary that works on multiple, sometime heterogenous hardware -- how to do this "optimally" -- that is the problem that AI-Edge-Torch in some sense is solving.

Hi @pkgoogle ,
thanks for your suggestion, it's useful for me.
i will try to study multivariable. (https://ocw.mit.edu/courses/18-02sc-multivariable-calculus-fall-2010/)
and try to understand MLIR
does the ai-edge-torch have any performance report. e.g., how many token decode in second,  if using c++ inference in example code.

Hi @nigelzzz, so AI-Edge-Torch is at the end of the day is a Pytorch -> TFLite converter... most of the time people don't care about how performant the conversion takes, but they do care about how fast the produced model is... is that what you are looking for? In which case we have the benchmark tool in TFLite: https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/tools/benchmark.
For decoding tokens per second... I think you will have to time it in C++, if you don't need the highest precision (i.e. you don't actually need to know the exact number of CPU cycles needed to execute that part of the code) you can just time it yourself w/ the <chrono> standard library. Otherwise you'll need a profiling tool, like Valgrind... I would research the various tools and their tradeoffs and figure out which works best for your situation.
For multivariable calculus, understanding what a gradient and partial derivatives are, is the most important thing today. (Once you understand the backprop proof, or compute the gradient of a simple NN manually once -- that's mainly what's needed for ML, but of course feel free to go further).

authors: nigelzzz pkgoogle

repo: google-ai-edge/ai-edge-torch


text:
USB cameras on dev board mini

Hi,
are there any plans to support USB cameras on the dev board mini? I noticed the getting started guide said people are working on it. It there a roadmap or sth.?
I am trying to connect three cameras on a USB 2.0 hub (bandwidth-wise this works fine on other platforms). But when I try it on the dev board mini, only one camera comes up at a time. Is there a known (hardware/driver) limitation to the USB port that prevents this?
Best,
Thomas

@kueblert
I believe the 3 cameras may just be under powered. I'll check with my team to see progresses on this, but do you see anything interesting in dmesg?
Also, what usb cameras are you using?

Thank you for checking, @Namburger .
I tried connecting the cameras via a powered USB hub, but with the same effect.
The cameras are custom uvc cameras connected via a USB 2.0 hub.
They deliver up to 640x480 MJPEG @ 60fps each.
I'll provide a dmesg log later.

Ok, so I am running the following GStreamer pipeline:
gst-launch-1.0 v4l2src device="/dev/video1" ! image/jpeg, width=640, height=480, framerate=60/1 ! queue ! decodebin ! compositor name=comp ! videoconvert ! fakesink v4l2src device="/dev/video4" ! image/jpeg, width=640, height=480, framerate=60/1 ! queue ! decodebin ! comp.
and receive this output:
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Failed to allocate required memory.
Additional debug info:
gstv4l2src.c(660): gst_v4l2src_decide_allocation (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
Buffer pool activation failed
Execution ended after 0:00:00.167497906
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
dmesg returns:
[ 3506.160263] uvcvideo: Failed to query (GET_DEF) UVC control 13 on unit 1: -32 (exp. 8).
[ 3506.182577] uvcvideo: Failed to query (GET_DEF) UVC control 13 on unit 1: -32 (exp. 8).
[ 3506.329979] musb-hdrc musb-hdrc.2.auto: iso hwep alloc failed for 3x280
[ 3506.336819] uvcvideo: Failed to submit URB 0 (-28).

https://e2e.ti.com/support/processors/f/791/t/845765?AM3352-Multiple-USB-cameras it sounds like musb might be the limiting factor (however, this seems a bit beyond my expertise...). Is it possible to enable that xHCI-thing that those people talk about on Mendel OS and can anyone tell whether it will have the expected outcome?

@kueblert wait, could you run this commands?
v4l2-ctl --list-devices
v4l2-ctl --device /dev/video1 --list-formats

I may be wrong, but I believe /dev/video1 is reserved for our camera module

@Namburger should be video0 and video2.
platform:mt8167 (platform:mt8167):
/dev/video0                                                                                14001000.rdma (platform:mt8173):
/dev/video2                                                                                RYS THREE: RYS THREE (usb-musb-hdrc.2.auto-1.2):
/dev/video1
/dev/video3                                                                                RYS ONE: RYS ONE (usb-musb-hdrc.2.auto-1.3):
/dev/video4
/dev/video5                                                                                RYS TWO: RYS TWO (usb-musb-hdrc.2.auto-1.4):
/dev/video6
/dev/video7
video1 formats:
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture                                                                                [0]: 'MJPG' (Motion-JPEG, compressed)

after enabling tracing on the uvc driver I got the following slightly more detailled log (of which I still cannot make much sense):
[ 33.757774] usb0_vbus_old: disabling
[ 33.761344] backlight_vbus: disabling
[ 33.765106] ldo_vm25: disabling
[ 56.782243] uvcvideo: Failed to query (GET_DEF) UVC control 13 on unit 1: -32 (exp. 8).
[ 56.859116] uvcvideo: Failed to query (GET_DEF) UVC control 13 on unit 1: -32 (exp. 8).
[ 57.020651] musb-hdrc musb-hdrc.2.auto: iso hwep alloc failed for 3x280
[ 57.027350] uvcvideo: Failed to submit URB 0 (-28).
[211.075916] uvcvideo: uvc_v4l2_open
[211.210291] uvcvideo: Resuming interface 0
[211.210306] uvcvideo: Resuming interface 1
[211.211574] uvcvideo: uvc_v4l2_release
[211.212801] uvcvideo: uvc_v4l2_open
[211.278443] uvcvideo: Resuming interface 0
[211.278458] uvcvideo: Resuming interface 1
[211.279735] uvcvideo: uvc_v4l2_release
[211.280969] uvcvideo: uvc_v4l2_open
[211.346249] uvcvideo: Resuming interface 0
[211.346265] uvcvideo: Resuming interface 1
[211.347535] uvcvideo: uvc_v4l2_release
[211.405118] uvcvideo: uvc_v4l2_open
[211.408522] uvcvideo: Failed to query (GET_DEF) UVC control 13 on unit 1: -32 (exp. 8).
[211.417013] uvcvideo: Control error 6
[211.426834] uvcvideo: uvc_v4l2_open
[211.430017] uvcvideo: Failed to query (GET_DEF) UVC control 13 on unit 1: -32 (exp. 8).
[211.438362] uvcvideo: Control error 6
[211.440717] uvcvideo: Trying format 0x47504a4d (MJPG): 640x480.
[211.440729] uvcvideo: Using default frame interval 16666.6 us (60.0 fps).
[211.441693] uvcvideo: Trying format 0x47504a4d (MJPG): 640x480.
[211.441704] uvcvideo: Using default frame interval 16666.6 us (60.0 fps).
[211.490024] uvcvideo: Setting frame interval to 1/60 (166666).
[211.495532] uvcvideo: Setting frame interval to 1/60 (166666).
[211.544356] uvcvideo: Control 0x00980927 not found.
[211.550031] uvcvideo: Control 0x00980927 not found.
[211.560957] uvcvideo: Control 0x00980927 not found.
[211.560961] uvcvideo: Control 0x00980927 not found.
[211.568968] uvcvideo: Device requested 320 B/frame bandwidth.
[211.568980] uvcvideo: Selecting alternate setting 4 (840 B/frame bandwidth).
[211.574527] uvcvideo: Device requested 320 B/frame bandwidth.
[211.574543] uvcvideo: Selecting alternate setting 4 (840 B/frame bandwidth).
[211.577174] uvcvideo: Allocated 5 URB buffers of 32x840 bytes each.
[211.577309] uvcvideo: Allocated 5 URB buffers of 32x840 bytes each.
[211.577339] uvcvideo: uvc_v4l2_poll
[211.577347] musb-hdrc musb-hdrc.2.auto: iso hwep alloc failed for 3x280
[211.577382] uvcvideo: Failed to submit URB 0 (-28).
[211.593006] uvcvideo: uvc_v4l2_poll
[211.593223] musb_host_rx 1895: Rx interrupt with no errors or packet!
[211.603692] uvcvideo: uvc_v4l2_release
[211.604066] uvcvideo: uvc_v4l2_release

Humn, so it looks like your cameras are showing up as
/dev/video2,4, and 5. Could you run the pipeline over those devices?
gst-launch-1.0  glvideomixer name=m background=black sink_0::xpos=0 sink_0::ypos=0   sink_0::zorder=1  ! glimagesink
      v4l2src device=/dev/video2 ! video/x-raw,framerate=30/1,width=640,height=480 !  glupload ! glcolorconvert ! m.sink_0

Hi @Namburger , sorry for the delay over the weekend. I am pretty sure I got the camera devices right.
Video 0/2 are the built-in ones. Each of my cameras registers both a raw and a MJPG compressed stream. To be able to transfer all images simultaneously, I need to use the MJPG stream. Otherwise the bandwidth might be problematic.
The pipeline you wrote therefore fails. I modified it as follows, which runs just fine (but extending it to use two cameras results in the same error as before):
gst-launch-1.0  glvideomixer name=m background=black sink_0::xpos=0 sink_0::ypos=0 sink_0::zorder=1 ! glimagesink v4l2src device=/dev/video6 ! image/jpeg,framerate=60/1,width=640,height=480 ! jpegdec ! glupload ! glcolorconvert ! m.sink_0
Here is also the output of gst-device-monitor:
`Probing devices...
Device found:
name  : RYS THREE: RYS THREE
class : Video/Source
caps  : image/jpeg, width=(int)640, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)60/1;
        image/jpeg, width=(int)352, height=(int)288, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)60/1;
        image/jpeg, width=(int)320, height=(int)240, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)60/1;
        image/jpeg, width=(int)176, height=(int)144, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)60/1;
        image/jpeg, width=(int)160, height=(int)120, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)60/1;
properties:
	udev-probed = true
	device.bus_path = platform-musb-hdrc.2.auto-usb-0:1.2:1.0
	sysfs.path = /sys/devices/platform/soc/11100000.usb/musb-hdrc.2.auto/usb1/1-1/1-1.2/1-1.2:1.0/video4linux/video1
	device.bus = usb
	device.subsystem = video4linux
	device.vendor.id = 1421
	device.vendor.name = "RYS\\x20Corp."
	device.product.id = 0806
	device.product.name = "RYS\ THREE:\ RYS\ THREE"
	device.serial = RYS_Corp._RYS_THREE
	device.capabilities = :capture:
	device.api = v4l2
	device.path = /dev/video1
	v4l2.device.driver = uvcvideo
	v4l2.device.card = "RYS\ THREE:\ RYS\ THREE"
	v4l2.device.bus_info = usb-musb-hdrc.2.auto-1.2
	v4l2.device.version = 267133 (0x0004137d)
	v4l2.device.capabilities = 2225078273 (0x84a00001)
	v4l2.device.device_caps = 69206017 (0x04200001)
gst-launch-1.0 v4l2src device=/dev/video1 ! ...

Device found:
name  : RYS ONE: RYS ONE
class : Video/Source
caps  : image/jpeg, width=(int)640, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)60/1;
        image/jpeg, width=(int)352, height=(int)288, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)60/1;
        image/jpeg, width=(int)320, height=(int)240, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)60/1;
        image/jpeg, width=(int)176, height=(int)144, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)60/1;
        image/jpeg, width=(int)160, height=(int)120, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)60/1;
properties:
	udev-probed = true
	device.bus_path = platform-musb-hdrc.2.auto-usb-0:1.3:1.0
	sysfs.path = /sys/devices/platform/soc/11100000.usb/musb-hdrc.2.auto/usb1/1-1/1-1.3/1-1.3:1.0/video4linux/video4
	device.bus = usb
	device.subsystem = video4linux
	device.vendor.id = 1421
	device.vendor.name = "RYS\\x20Corp."
	device.product.id = 0806
	device.product.name = "RYS\ ONE:\ RYS\ ONE"
	device.serial = RYS_Corp._RYS_ONE
	device.capabilities = :capture:
	device.api = v4l2
	device.path = /dev/video4
	v4l2.device.driver = uvcvideo
	v4l2.device.card = "RYS\ ONE:\ RYS\ ONE"
	v4l2.device.bus_info = usb-musb-hdrc.2.auto-1.3
	v4l2.device.version = 267133 (0x0004137d)
	v4l2.device.capabilities = 2225078273 (0x84a00001)
	v4l2.device.device_caps = 69206017 (0x04200001)
gst-launch-1.0 v4l2src device=/dev/video4 ! ...

Device found:
name  : RYS TWO: RYS TWO
class : Video/Source
caps  : image/jpeg, width=(int)640, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)60/1;
        image/jpeg, width=(int)352, height=(int)288, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)60/1;
        image/jpeg, width=(int)320, height=(int)240, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)60/1;
        image/jpeg, width=(int)176, height=(int)144, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)60/1;
        image/jpeg, width=(int)160, height=(int)120, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)60/1;
properties:
	udev-probed = true
	device.bus_path = platform-musb-hdrc.2.auto-usb-0:1.4:1.0
	sysfs.path = /sys/devices/platform/soc/11100000.usb/musb-hdrc.2.auto/usb1/1-1/1-1.4/1-1.4:1.0/video4linux/video6
	device.bus = usb
	device.subsystem = video4linux
	device.vendor.id = 1421
	device.vendor.name = "RYS\\x20Corp."
	device.product.id = 0806
	device.product.name = "RYS\ TWO:\ RYS\ TWO"
	device.serial = RYS_Corp._RYS_TWO
	device.capabilities = :capture:
	device.api = v4l2
	device.path = /dev/video6
	v4l2.device.driver = uvcvideo
	v4l2.device.card = "RYS\ TWO:\ RYS\ TWO"
	v4l2.device.bus_info = usb-musb-hdrc.2.auto-1.4
	v4l2.device.version = 267133 (0x0004137d)
	v4l2.device.capabilities = 2225078273 (0x84a00001)
	v4l2.device.device_caps = 69206017 (0x04200001)
gst-launch-1.0 v4l2src device=/dev/video6 ! ...

`

Did the team provide any hint of why only VGA&MJPG is supported? Is it a hardware limitation of the USB/Soc or a driver thing?

Anyone knows whether USB with the normal, full-scale dev board is any better?

@kueblert allow me to check back with you regarding the dev board mini's usb camera issue, however the original dev board should be able to use the usb camera without issues

@Namburger Your help is much appreciated. I'd be happy to get it working with the mini.

Hi @kueblert ,
The limitation is USB. With the USB2.0 connection, we are limited to this input (when coming from USB) - we can't get pixels in and sent to the GPU fast enough. Theoretically a higher-resolution H264 camera could be used, but trying to decode that in streaming mode isn't very reliable on the MTK platform. These restrictions don't exist for MIPI Cameras on the Dev Board Mini or on the full size Dev Board (where the USB port is USB3.0 and it better handles moving USB frames).

For future reference, editing /sys/module/usbcore/parameters/usbfs_memory_mb at least canges the error message.
Still, only one camera at a time is recording. But gstreamer now does run three separate pipelines at the same time now and then.

authors: Namburger kueblert mbrooksx

repo: google-coral/edgetpu


text:
Authenticating to docker using gcloud isn't working after this action

TL;DR
Authenticating to docker using gcloud isn't working, despite what looks like a successful login.
Expected behavior
Docker to be successfully authenticated.
Observed behavior
Docker authentication doesn't seem to be working in combination with this actions output?
Action YAML
      - name: 'Authenticate to Google Cloud'
        uses: 'google-github-actions/auth@v0'
        with:
          credentials_json: '${{ secrets.GOOGLE_CLOUD_TOKEN }}'

      - name: 'Set up Cloud SDK'
        uses: 'google-github-actions/setup-gcloud@v0'

      - name: Docker Login
        run: "gcloud auth configure-docker --quiet us-central1-docker.pkg.dev"

      - uses: actions/checkout@v2

      - name: Build Docker Image
        run: "docker build ."

Log output
Getting:

ERROR: (gcloud.auth.docker-helper) Failed to load credential file: [/home/runner/work/api/api/39c413352e4b4825126e788d].  File /home/runner/work/api/api/39c413352e4b4825126e788d was not found.

Even though:
    CLOUDSDK_AUTH_CREDENTIAL_FILE_OVERRIDE: /home/runner/work/api/api/39c413352e4b4825126e788d
    GOOGLE_APPLICATION_CREDENTIALS: /home/runner/work/api/api/39c413352e4b4825126e788d
    GOOGLE_GHA_CREDS_PATH: /home/runner/work/api/api/39c413352e4b4825126e788d

checkout should be the first step before doing anything else. This is documented in the readme.
If you move actions/checkout@v2 to before auth, that will fix your issue.

Apologies, it might be worthwhile to reorganize the readme into essential sections at the top and then break out the individual flows away from that.  A lot of it is all blended together, so I took a hunt-and-peck approach while reading it :chicken:
:wink:

authors: atrauzzi sethvargo

repo: google-github-actions/auth


text:
BERT feature extraction for tensorflow serving

I would like to export BERT feature extraction for use via tensorflow serving.
Following what has been done here: https://github.com/bigboNed3/bert_serving, I modified extract_features.py by adding this function
def serving_input_fn():
    label_ids = tf.placeholder(tf.int32, [None], name='label_ids')
    input_ids = tf.placeholder(tf.int32, [None, FLAGS.max_seq_length], name='input_ids')
    input_mask = tf.placeholder(tf.int32, [None, FLAGS.max_seq_length], name='input_mask')
    segment_ids = tf.placeholder(tf.int32, [None, FLAGS.max_seq_length], name='segment_ids')
    input_fn = tf.estimator.export.build_raw_serving_input_receiver_fn({
        'label_ids': label_ids,
        'input_ids': input_ids,
        'input_mask': input_mask,
        'segment_ids': segment_ids,
    })()
    return input_fn

and the following code snippet to the main function.
  if FLAGS.do_export:
    estimator._export_to_tpu = False
    estimator.export_savedmodel(FLAGS.export_dir, serving_input_fn, strip_default_attrs=True)

I then run the following on command line
python extract_features.py --input_file=/tmp/input.txt \
  --output_file=/tmp/output.jsonl \
  --vocab_file=$BERT_BASE_DIR/vocab.txt \
  --bert_config_file=$BERT_BASE_DIR/bert_config.json \
  --init_checkpoint=$BERT_BASE_DIR/bert_model.ckpt \
  --layers=-1,-2,-3,-4 --max_seq_length=128 --batch_size=8 \
  --do_export=True --export_dir='/models'

I get these errors below
Traceback (most recent call last):
  File "extract_features_export.py", line 431, in <module>
    tf.app.run()
  File "/home/m/anaconda3/envs/gpu-dev/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 125, in run
    _sys.exit(main(argv))
  File "extract_features_export.py", line 423, in main
    estimator.export_savedmodel(FLAGS.export_dir, serving_input_fn, strip_default_attrs=True)
  File "/home/m/anaconda3/envs/gpu-dev/lib/python3.6/site-packages/tensorflow/python/estimator/estimator.py", line 663, in export_savedmodel
    mode=model_fn_lib.ModeKeys.PREDICT)
  File "/home/m/anaconda3/envs/gpu-dev/lib/python3.6/site-packages/tensorflow/python/estimator/estimator.py", line 789, in _export_saved_model_for_mode
    strip_default_attrs=strip_default_attrs)
  File "/home/m/anaconda3/envs/gpu-dev/lib/python3.6/site-packages/tensorflow/python/estimator/estimator.py", line 878, in _export_all_saved_models
    raise ValueError("Couldn't find trained model at %s." % self._model_dir)
ValueError: Couldn't find trained model at /tmp/tmp3eh_rldt.

Specifying the directory explicitly by updating run_config to include an output_dir
  run_config = tf.contrib.tpu.RunConfig(
      master=FLAGS.master,
      model_dir=FLAGS.output_dir,
      tpu_config=tf.contrib.tpu.TPUConfig(
          num_shards=FLAGS.num_tpu_cores,
          per_host_input_for_training=is_per_host))

and calling the function by giving --output_dir=$BERT_BASE_DIR still gives me the same ValueError: Couldn't find trained model at ....
Has anyone else faced this issue? How can I fix this?

the same problem : (

any improvement ? I am following https://github.com/bigboNed3/bert_serving.  model
loading is ok. but , failed in serving: err msg is " session run failed: Invalid argument: Expects arg[0] to be string but int64 is provided"

@apurvaasf I found an easiest way to export original bert model to SavedModel.
# load the checkpoint from bert
# create an estimator which contains original bert model and some other ops if you want
estimator = ...
save_hook = tf.train.CheckpointSaverHook(FLAGS.output_dir, save_secs=1)
estimator.predict(input_fn=predict_input_fn, hooks=[save_hook])
# now you will get graph.pbtxt which is used in SavedModel, and then
estimator.export_savedmodel(...)

yeah I'm with you, estimator has to run predict at least one time before export_savedmodel.

I write the fake train code, and solved this problem
 optimizer = tf.train.AdagradOptimizer(learning_rate=0.1)     train_op = optimizer.minimize(loss, global_step=tf.train.get_global_step())     if mode == tf.estimator.ModeKeys.TRAIN:       output_spec = tf.contrib.tpu.TPUEstimatorSpec(         mode=mode, loss=loss, train_op=train_op, scaffold_fn=scaffold_fn     )
and add this in the main function
estimator.train(input_fn)

@apurvaasf I found an easiest way to export original bert model to SavedModel.
# load the checkpoint from bert
# create an estimator which contains original bert model and some other ops if you want
estimator = ...
save_hook = tf.train.CheckpointSaverHook(FLAGS.output_dir, save_secs=1)
estimator.predict(input_fn=predict_input_fn, hooks=[save_hook])
# now you will get graph.pbtxt which is used in SavedModel, and then
estimator.export_savedmodel(...)

I add this but I still can't get the same error. It seems that the model doesn't run the predict at all.

Maybe check this out if you are looking for serving BERT fine-tuned model.
BERT Serving and Inferencing from fine-tuned

I had the same error even with graph.pbtxt present in the model folder, which made me believe that perhaps tf estimator was looking into the wrong folder. Turned out the issue can be fixed by providing the correct "checkpoint_path" to the export_saved_model() function.
For example:
estimator.export_saved_model(..., checkpoint_path='model.ckpt-12345')
And this works even without graph.pbtxt.

I had the same error even with graph.pbtxt present in the model folder, which made me believe that perhaps tf estimator was looking into the wrong folder. Turned out the issue can be fixed by providing the correct "checkpoint_path" to the export_saved_model() function.
For example: estimator.export_saved_model(..., checkpoint_path='model.ckpt-12345')
And this works even without graph.pbtxt.

it's useful! thank you

If someone export_saved_model from origin google_bert's ckpt file use
estimator.export_saved_model(..., checkpoint_path='model.ckpt-12345')
and get error global_step not found in checkpoint
You can make simple pretrain process than new ckpt have global_step
maybe is origin google_bert's ckpt in save process clip adam parameter make global_step also cliped (especially use tpuesitmator)

authors: DawsenWSH NatLee TristanShao fengzuo97 jageshmaharjan jiqiujia mvss80 nlp4whp pqchen xielangtc

repo: google-research/bert