text:
brew install painfully slow if Homebrew not up to date

Follow up on https://github.com/Homebrew/brew/issues/4755.
What you were trying to do (and why)
What: Use Homebrew to install dependencies in a CI environment (CircleCI/macOS in this case).
We're using the CircleCI image for XCode 10.3.0: https://circle-macos-docs.s3.amazonaws.com/image-manifest/build-903/index.html
Why: Because our build scripts need tools not included in macOS, and Homebrew is the same tool we use in our local environments.
What happened (include command output)
Homebrew takes 2-3 minutes to complete a brew install.

Command output

$ /usr/bin/time brew install --verbose --debug jq
Updating Homebrew...
==> Auto-updated Homebrew!
Updated 3 taps (homebrew/core, homebrew/cask and caskroom/versions).
==> New Formulae
aws-cdk
base91
benthos
berglas
bitwise
broot
clp
codespell
coinutils
cpu_features
dive
fastmod
faust
ghc@8.6
git-revise
go-md2man
go@1.12
grpc-swift
kubebuilder
lazydocker
lerna
libsigc++@2
lightgbm
llvm@8
minizip2
msgpack-tools
now-cli
nushell
octant
open-image-denoise
pastel
pdfcpu
pdftk-java
pokerstove
protobuf@3.7
pulp
pygments
ripgrep-all
scc
scs
starship
wiiuse
wtfutil
zrepl
==> Updated Formulae
abcmidi
abyss
ace
ack
activemq
activemq-cpp
adwaita-icon-theme
afflib
afl-fuzz
agedu
aircrack-ng
algernon
aliyun-cli
allure
alpine
amap
amazon-ecs-cli
ammonite-repl
amqp-cpp
amtk
angband
angular-cli
anjuta
annie
ansible
ansible-lint
ansible@1.9
ansible@2.0
ansiweather
ant
anycable-go
apache-arrow
apache-flink
apache-geode
apache-spark
apib
appscale-tools
apr-util
aqbanking
arangodb
aravis
armadillo
arpack
arping
asdf
asio
ask-cli
aspell
astrometry-net
atk
atkmm
atlantis
atomist-cli
auditbeat
autorest
avfs
avro-c
avro-cpp
avro-tools
aws-google-auth
aws-okta
aws-sdk-cpp
awscli
axel
azure-cli
azure-storage-cpp
babel
babl
bacula-fd
badtouch
balena-cli
ballerina
baobab
bareos-client
basex
bash
bash-snippets
bat
bazel
bazelisk
bbftp-client
bcftools
bear
bedtools
bee
berkeley-db
bibtexconv
bigloo
binaryen
bind
biogeme
bison
bit
bitchx
bitcoin
bitrise
blueutil
boost
boost-bcp
boost-build
boost-mpi
boost-python
boost-python3
botan
breezy
bro
btfs
btparse
btpd
buildifier
buildkit
buku
bundletool
burp
c-blosc
cabal-install
cadaver
caddy
caf
caffe
cairomm
calicoctl
camlp4
camlp5
cargo-completion
cash-cli
catch2
ccache
ceres-solver
certbot
certstrap
cfn-lint
cfr-decompiler
cgdb
cglm
cgrep
chakra
chapel
charm-tools
checkbashisms
checkstyle
chronograf
cimg
circleci
citus
cjdns
ckan
clamav
clingo
clojure
closure-compiler
cmake
cmix
cmus
cnats
cockroach
cointop
collectd
collector-sidecar
commandbox
compcert
composer
conan
configen
conjure-up
consul
consul-template
contentful-cli
convox
coturn
cpl
cppcheck
cpprestsdk
cql
credstash
cromwell
crowdin
cryptol
crystal
crystal-icr
csound
cubelib
cucumber-cpp
curl
curl-openssl
cypher-shell
cython
dartsim
datamash
dav1d
davix
davmail
dbmate
dcmtk
dcos-cli
ddgr
deno
dependency-check
desktop-file-utils
devspace
dhall
dhall-bash
dhall-json
dialog
diamond
dieharder
diff-pdf
diff-so-fancy
diffoscope
diskus
distcc
dita-ot
django-completion
dmd
dmg2img
dnscrypt-proxy
dnscrypt-wrapper
dnsperf
dnsviz
docfx
docker
docker-completion
docker-compose
docker-machine
docker-machine-completion
doctl
docutils
doitlive
dosbox-x
dovecot
doxygen
drafter
drone-cli
dtc
dub
duck
duo_unix
duplicity
dyld-headers
dylibbundler
dynare
eccodes
efl
ejabberd
ekg2
elasticsearch
elektra
elinks
elm-format
emacs
embree
embulk
emscripten
encfs
enchant
entr
envconsul
epic5
eralchemy
erlang
erlang@20
erlang@21
eslint
etcd
ethereum
ettercap
exempi
exiftool
exim
exiv2
expat
exploitdb
faas-cli
fabric
fastjar
faudio
fauna-shell
fb-client
fd
fdroidserver
feedgnuplot
feh
ffmpeg
ffmpeg@2.8
ffsend
fibjs
file-roller
findutils
fio
firebase-cli
flac
flintrock
flow
flowgrind
fluid-synth
fluxctl
flyway
fn
folly
fontforge
fonttools
fossil
fpp
fq
fragroute
freeipmi
freeradius-server
freetds
frpc
frps
frugal
fselect
fstrm
fuse-emulator
fuse-zip
futhark
gambit-scheme
gammu
gatsby-cli
gauge
gcc
gdal
gdcm
gdk-pixbuf
gdl
gdub
geant4
gearman
geckodriver
gedit
geoipupdate
gerbil-scheme
get_iplayer
getdns
getmail
getxbook
ghc
ghr
gifski
git
git-annex
git-cola
git-crypt
git-extras
git-quick-stats
git-secret
git-series
gitbucket
gitg
gitlab-runner
gitleaks
gitmoji
gitversion
gjs
gkrellm
glances
glib
glib-networking
glib-openssl
glibmm
globus-toolkit
glooctl
gloox
gmic
gmsh
gmt
gnunet
gnupg
gnupg-pkcs11-scd
gnuradio
gnutls
go
go-jira
go@1.11
gobby
gobject-introspection
gocryptfs
godep
golang-migrate
goofys
gopass
goreleaser
gpac
gphoto2
gpsim
gradle
grafana
grakn
graph-tool
graphene
graphicsmagick
grok
groonga
groovy
grpc
grpcurl
gsettings-desktop-schemas
gsl
gsmartcontrol
gsoap
gspell
gst-plugins-bad
gst-plugins-good
gstreamermm
gtk+3
gtk-doc
gtk-vnc
gtkmm
gtkmm3
gtksourceview4
gtksourceviewmm
gtksourceviewmm3
gtmess
gtranslator
gwenhywfar
gwyddion
h2o
h3
hadolint
hapi-fhir-cli
haproxy
harfbuzz
hashcat
hashpump
hbase
heimdal
helmfile
helmsman
help2man
hfstospell
highlight
hive
hledger
hlint
homebank
htpdate
http_load
httpd
httperf
httpie
httping
httrack
hub
hugo
hydra
hyperfine
hyperkit
hyperscan
i2p
i2pd
i386-elf-gcc
icarus-verilog
icecast
icecream
ike-scan
imagemagick
imagemagick@6
imageoptim-cli
imap-uw
imapfilter
inlets
innotop
insect
interactive-rebase-tool
intercal
ios-sim
ios-webkit-debug-proxy
iperf
iperf3
ipfs
ipmitool
ipopt
ipv6calc
ipython
ircd-hybrid
ircii
irssi
ispc
istioctl
isync
janet
jdupes
jemalloc
jenkins
jenkins-lts
jfrog-cli-go
jhipster
jing-trang
jinja2-cli
john-jumbo
joplin
jose
jpeg-turbo
jruby
jsonnet
jsonrpc-glib
juise
juju
jvgrep
k3d
k6
kafka
kafkacat
kahip
kakoune
kibana
kim-api
knot
kops
kore
kotlin
krb5
kube-aws
kubecfg
kubectx
kubernetes-cli
kubernetes-helm
kubernetes-service-catalog-client
kubeseal
kubespy
kustomize
lablgtk
lammps
lasso
lastpass-cli
launchdns
lazygit
ldapvi
ldc
ldid
ldns
lean-cli
ledger
lego
lftp
lgogdownloader
libarchive
libbitcoin
libbitcoin-blockchain
libbitcoin-client
libbitcoin-consensus
libbitcoin-database
libbitcoin-explorer
libbitcoin-network
libbitcoin-node
libbitcoin-protocol
libbitcoin-server
libbluray
libcouchbase
libdap
libdazzle
libdvbpsi
libevent
libevhtp
libewf
libexosip
libfaketime
libfreefare
libftdi
libgcrypt
libgda
libgee
libgig
libgit2
libglademm
libgnomecanvasmm
libheif
libimagequant
libimobiledevice
libiodbc
libiscsi
libjwt
liblcf
libmaa
libmatio
libmodbus
libmowgli
libmxml
libmypaint
liboauth
libogg
libomp
libopendkim
libopenmpt
libosinfo
libosip
libosmium
libp11
libpeas
libphonenumber
libpq
libpqxx
libpulsar
libqalculate
libraw
librdkafka
libre
librealsense
librsvg
librsync
libsecret
libsigc++
libsignal-protocol-c
libslax
libsoup
libspng
libssh
libssh2
libstrophe
libtasn1
libtins
libtorrent-rasterbar
libtrace
libu2f-host
libu2f-server
libusb
libuv
libvirt
libvisio
libvmaf
libvoikko
libvterm
libwebsockets
libxkbcommon
libxml++
libxml++3
libxmlsec1
libzdb
lighttpd
linkerd
links
liquibase
liquid-dsp
lldpd
llvm
lmod
logrotate
logstash
logtalk
lolcat
lsd
lsof
luvit
lxc
lynx
lz4
mackup
mafft
mage
magic-wormhole
mailutils
makensis
makepkg
mame
mariadb
mariadb-connector-c
mariadb-connector-odbc
mariadb@10.0
mariadb@10.1
mariadb@10.2
mariadb@10.3
maven
mawk
maxwell
mbedtls
md5sha1sum
mdds
media-info
mediaconch
megacmd
megatools
memcached
memcacheq
menhir
mercurial
mesa
meson
metabase
metricbeat
mfterm
mgba
micronaut
midnight-commander
mill
miller
mingw-w64
minio
minio-mc
miniserve
mit-scheme
mitmproxy
mkcert
mkl-dnn
mktorrent
mkvtoolnix
mmark
mmseqs2
moarvm
mockserver
moco
modules
moe
molecule
monero
monetdb
mongo-c-driver
mongoose
monit
monitoring-plugins
monkeysphere
mono
mosh
mosquitto
mp3cat
mpg123
mrboom
msitools
mtr
mupdf
mutt
mycli
mydumper
mysql
mysql-client
mysql-connector-c++
mysql@5.7
mytop
n
nagios
nagios-plugins
nano
nativefier
nats-server
nats-streaming-server
ncdc
ncdu
ncmpcpp
nco
ncrack
ncview
neo4j
neofetch
neomutt
neon
neovim
net-snmp
netdata
netlify-cli
newman
newsboat
nfdump
nghttp2
nginx
ngircd
nickle
nim
nlohmann-json
nmap
nmh
nnn
node
node-build
node@10
node@8
nomad
nopoll
noti
nqp
nrpe
nsd
nspr
nsq
nss
ntl
ntp
nuget
numpy
nwchem
nylon
nyx
nzbget
oath-toolkit
ocaml
ocaml-findlib
ocaml-num
ocamlbuild
ocamlsdl
ocproxy
ocrmypdf
octave
ode
offlineimap
omega
omniorb
oniguruma
ooniprobe
opa
open-mpi
openapi-generator
openblas
opencolorio
openconnect
opencv
opencv@2
opencv@3
openfortivpn
openimageio
openkim-models
openldap
openrct2
opensaml
opensc
openssh
openssl
openssl@1.1
openvpn
operator-sdk
ophcrack
opusfile
orc
orc-tools
ori
osc
oscats
osquery
osslsigncode
owfs
p11-kit
p7zip
packer
packer-completion
pagmo
paket
pandoc-citeproc
pango
pangomm
parallel
passenger
pc6001vx
pcb2gcode
pdfcrack
pdftoipe
pdns
pdnsrec
pegtl
percona-server
percona-toolkit
percona-xtrabackup
petsc
petsc-complex
pev
pgbouncer
pgcli
pgroonga
pgweb
phoronix-test-suite
php
php-cs-fixer
php@7.1
php@7.2
phpmd
phpstan
phpunit
picard-tools
pijul
pilosa
pioneer
pip-completion
pipx
pjproject
pkcs11-helper
planck
plantuml
platformio
pmd
png++
pngquant
poco
podofo
pony-stable
ponyc
poppler
postgis
postgresql
postgresql@10
postgresql@9.4
postgresql@9.5
postgresql@9.6
postgrest
pre-commit
prefixsuffix
presto
procyon-decompiler
profanity
proj
prometheus
proteinortho
protobuf
protobuf-c
protobuf-swift
protobuf@3.6
proxytunnel
pspg
psqlodbc
pulledpork
pulseaudio
pulumi
pure-ftpd
pushpin
puzzles
pwntools
pwsafe
pybind11
pyenv
pygobject3
pypy
pypy3
pyqt
pyside
python
python@2
pyvim
qalculate-gtk
qca
qemu
qmmp
qpdf
qpid-proton
qrupdate
qscintilla2
qsoas
qt
quantlib
quickjs
r
rabbitmq-c
rakudo
range-v3
rc
rclone
rdesktop
re-flex
re2
re2c
readline
rebar3
redo
redsocks
remctl
rename
reop
repo
reposurgeon
riff
ripgrep
rke
rmlint
robot-framework
rom-tools
root
roswell
rpm
rsyslog
rtags
rtmpdump
ruby
ruby-build
ruby@2.4
ruby@2.5
rust
rustup-init
s-lang
s-nail
s2geometry
s3-backer
s3ql
sagittarius-scheme
saldl
salt
sane-backends
saxon
sbcl
sblim-sfcc
sbt
scala
scala@2.12
scalapack
scamper
sccache
scdoc
schismtracker
scipy
scons
scrcpy
scrypt
scw
sdhash
sdl2
serf
serverless
shadowenv
shadowsocks-libev
shairport
shairport-sync
shared-mime-info
shellcheck
shellharden
shellinabox
shibboleth-sp
ship
shogun
siege
signal-cli
sile
silk
simh
simple-scan
singular
sip
sipsak
siril
sk
skaffold
skinny
skopeo
sleuthkit
slowhttptest
slrn
smali
sn0int
snakemake
snapcraft
sngrep
snort
snownews
socat
sofia-sip
softhsm
solarus
solr
sonarqube
sonarqube-lts
sonobuoy
source-highlight
sourcekitten
sourcery
spdlog
spdylay
speedtest-cli
sphinx
sphinx-doc
spice-gtk
spim
spin
spiped
spirv-cross
sqlcipher
sqlmap
squid
src
srt
ssh-audit
ssh-permit-a38
sshfs
ssldump
sslsplit
sstp-client
step
stolon
stone-soup
storm
stow
streamlink
stress-ng
strongswan
stunnel
stuntman
subnetcalc
subversion
subversion@1.8
suite-sparse
sundials
supervisor
svtplay-dl
swagger-codegen
swagger-codegen@2
swi-prolog
swiftformat
swiftlint
swig
sylpheed
sync_gateway
syncthing
synfig
sysbench
sysdig
talloc
tarantool
tarsnap
taskell
tcl-tk
tcpdump
tcpflow
tdlib
tealdeer
tectonic
telegraf
telegram-cli
teleport
template-glib
termius
terraform
terraform-inventory
terraformer
terragrunt
terrahub
tesseract
testdisk
tflint
thrift
tika
tile38
tinc
tintin
tiny-fugue
tinyxml2
tmate
tmux
tomcat
tomcat-native
tomcat@7
tomcat@8
tomee-plus
tomee-webprofile
topgit
topgrade
tor
traefik
trafficserver
transcrypt
translate-shell
triton
ttyd
tunnel
tvnamer
twarc
twemcache
twine-pypi
typescript
u-boot-tools
ucloud
udunits
unbound
ungit
unrar
unrtf
unshield
unzip
upscaledb
uptimed
urh
uwsgi
v8
vala
valabind
valgrind
vapoursynth
vapoursynth-imwri
vapoursynth-ocr
vapoursynth-sub
varnish
vault
vault-cli
vaulted
velero
verilator
vert.x
vgmstream
vifm
vim
vim@7.4
vimpc
vips
virgil
visp
vit
vitetris
vnstat
vte3
vtk
vulkan-headers
wandio
watchexec
watchman
wcslib
wdc
webdis
webpack
webtorrent-cli
weechat
whois
widelands
wine
winetricks
wireguard-go
wireguard-tools
wiremock-standalone
wireshark
wla-dx
wp-cli
wskdeploy
wtf
wxmaxima
x11vnc
x265
xapian
xcodegen
xdot
xmake
xml-security-c
xml-tooling-c
xmount
xmrig
xonsh
xrootd
xsimd
xtensor
xxhash
yamllint
yank
yaws
yelp-tools
ykman
yosys
you-get
youtube-dl
z3
zabbix
zbackup
zboy
zelda-roth-se
zim
zimg
zint
znc
zsdx
zstd
zsxd
==> Renamed Formulae
confluent-oss -> confluent-platform
transmission -> transmission-cli
==> Deleted Formulae
cclive
cherokee
csup
ctunnel
engine_pkcs11
frag_find
freetds@0.91
ftimes
httest
irods
konoha
kontena
liblacewing
libopkele
llvm@3.9
llvm@5
mongodb
mongodb@3.0
mongodb@3.2
mongodb@3.4
mongodb@3.6
mysql-cluster
nordugrid-arc
packetbeat
percona-server-mongodb
pincaster
postgres-xc
skipfish
taisei
thc-pptp-bruter
tlsdate
tn5250
voms
xar-mackyle
/usr/local/Homebrew/Library/Homebrew/brew.rb (Formulary::FormulaLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/jq.rb
Warning: jq 1.6 is already installed and up-to-date
To reinstall 1.6, run brew reinstall jq
199.32 real       141.30 user        59.66 sys

What you expected to happen
Homebrew can complete installing a simple formula in a reasonable amount of time. 30 seconds?
Note that I'm installing an already-installed dependency, so this time doesn't include actually installing the program.
Step-by-step reproduction instructions (by running brew commands)

SSH into a CircleCI XCode 10.3 image
brew install jq

More information
HOMEBREW_VERSION: 2.1.8
ORIGIN: https://github.com/Homebrew/brew
HEAD: 549a40bedb32171912a449d8d0e7bcae02d5b432
Last commit: 9 weeks ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: 888b778d0290c07b77e9796a5be8c72066e05986
Core tap last commit: 9 weeks ago
HOMEBREW_PREFIX: /usr/local
HOMEBREW_NO_ANALYTICS_THIS_RUN: 1
CPU: quad-core 64-bit ivybridge
Homebrew Ruby: 2.3.7 => /System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/bin/ruby
Clang: 10.0 build 1001
Git: 2.22.0 => /usr/local/bin/git
Curl: 7.54.0 => /usr/bin/curl
Java: 1.8.0_222
macOS: 10.14.4-x86_64
CLT: 10.3.0.0.1.1562985497
Xcode: 10.3 => /Applications/Xcode-10.3.app/Contents/Developer
CLT headers: 10.3.0.0.1.1562985497

Running with HOMEBREW_NO_ANALYTICS=1 doesn't have any noticeable impact.
If I run brew update --verbose --debug --force, the bulk of the time seems to be spent looping over all taps:
...
/usr/local/Homebrew/Library/Homebrew/brew.rb (Formulary::FromPathLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/flyway.rb
/usr/local/Homebrew/Library/Homebrew/brew.rb (Formulary::FormulaContentsLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/flyway.rb
/usr/local/Homebrew/Library/Homebrew/brew.rb (Formulary::FromPathLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/fn.rb
/usr/local/Homebrew/Library/Homebrew/brew.rb (Formulary::FormulaContentsLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/fn.rb
/usr/local/Homebrew/Library/Homebrew/brew.rb (Formulary::FromPathLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/folly.rb
/usr/local/Homebrew/Library/Homebrew/brew.rb (Formulary::FormulaContentsLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/folly.rb
/usr/local/Homebrew/Library/Homebrew/brew.rb (Formulary::FromPathLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/fontforge.rb
...

[X] are reporting a bug others will be able to reproduce and not asking a question. If you're not sure or want to ask a question do so on our Discourse: https://discourse.brew.sh
[X] ran a brew command and reproduced the problem with multiple formulae? If it's a problem with a single, official formula (not cask) please file this issue at Homebrew/homebrew-core: https://github.com/Homebrew/homebrew-core/issues/new/choose. If it's a brew cask problem please file this issue at https://github.com/Homebrew/homebrew-cask/issues/new/choose. If it's a tap (e.g. Homebrew/homebrew-php) problem please file this issue at the tap.
[X] ran brew update and can still reproduce the problem?
[X] ran brew doctor, fixed all issues and can still reproduce the problem?
[X] ran brew config and brew doctor and included their output with your issue?

Follow up on #4755.

From that issue:

To be clear, I'm open to this being improved but until we've got something actionable here I don't think it's worth having an open issue.

SSH into a CircleCI XCode 10.3 image

This is an issue with CircleCI not keeping their images reasonable up-to-date, not with Homebrew.
If you want to avoid updating: there's documentation for doing that in man brew (although we won't fix any issues you encounter with that setup). Otherwise: we'll accept pull requests to make this faster but I'd be surprised if there's anything that's going to dramatically improve performance here.

authors: MikeMcQuaid djanowski


text:
--quiet is an unrecognized option

According to the brew docs, --quiet is a global option available to all commands:
GLOBAL OPTIONS

These options are applicable across all sub-commands.

    -q, --quiet: Suppress any warnings.
    -v, --verbose: Make some output more verbose.
    -d, --debug: Display any debugging information.
    -f, --force: Override warnings and enable potentially unsafe operations.

When I attempt to use --quiet to cut down on the irrelevant noise from Brew (from Travis):
brew update --quiet
brew install --quiet autoconf automake libtool

It results in an error:
Updated 2 taps (homebrew/cask and homebrew/core).
==> New Formulae
abseil                                   killswitch
ansible@2.8                              kind
...
homebrew/cask/x-lite
homebrew/cask/ynab

Usage: brew install [options] formula

Install formula. Additional options specific to formula may be appended to
the command.

Unless HOMEBREW_NO_INSTALL_CLEANUP is set, brew cleanup will then be run for

the installed formulae or, every 30 days, for all formulae.

    -d, --debug                      If brewing fails, open an interactive
                                     debugging session with access to IRB or a
    ...
                                     shell inside the temporary build directory.
    -h, --help                       Show this message.

Error: invalid option: --quiet

Please fill in the issue template.

Keep i broke. Great engineering :)

Don't read or fill in issue templates that provide us with the information we need to do our work: great issue reporting.

As simple as the following on Travis:
brew update --quiet
brew install --quiet autoconf automake libtool

From the issue template:

Please note we will close your issue without comment if you delete, do not read or do not fill out the issue checklist below and provide ALL the requested information. If you repeatedly fail to use the issue template, we will block you from ever submitting issues to Homebrew again.

My job is to report the errors. Your job is to reporduce them.
Don't worry about blocking me. I won't waste my time in the future.

source: gharchive/issue

authors: MikeMcQuaid noloader

repo: Homebrew/brew

license_type: permissive

license_source: github-api


text:
brew.sh should use HTTPS

To install Homebrew you're supposed to copy and paste some code from http://brew.sh/ into your terminal. This website should use HTTPS so that an attacker can't modify the command to copy and paste.

brew.sh is a GitHub Pages site with a custom domain and currently it's hard to support HTTPS on such sites. See also https://github.com/Homebrew/homebrew.github.io/issues/98.
There are instructions for the security-minded linked on the homepage too: https://github.com/Homebrew/brew/blob/master/share/doc/homebrew/Installation.md#installation.

Ahh, I see this has been discussed quite a bit elsewhere already.

authors: micahflee zmwangx


text:
apply extend/os to Cask::cmd::install#install_casks

[x] Have you followed the guidelines in our Contributing document?
[x] Have you checked to ensure there aren't other open Pull Requests for the same change?
[ ] Have you added an explanation of what your changes do and why you'd like us to include them?
[ ] Have you written new tests for your changes? Here's an example.
[x] Have you successfully run brew style with your changes locally?
[x] Have you successfully run brew typecheck with your changes locally?
[x] Have you successfully run brew tests with your changes locally?

move Cask::cmd::install#install_casks to extend/os because of #todo.

Thanks again @hyuraku!

source: gharchive/pull-request

authors: MikeMcQuaid hyuraku


text:
Fix flaky Tapioca::Compilers::Args test

[x] Have you followed the guidelines in our Contributing document?
[x] Have you checked to ensure there aren't other open Pull Requests for the same change?
[x] Have you added an explanation of what your changes do and why you'd like us to include them?
[x] Have you written new tests for your changes? Here's an example.
[x] Have you successfully run brew style with your changes locally?
[x] Have you successfully run brew typecheck with your changes locally?
[x] Have you successfully run brew tests with your changes locally?

Resolves https://github.com/Homebrew/brew/actions/runs/8329015201/job/22790407761 – let's just validate the keys, since it seems that the values vary by OS.

Thanks for quick fix @dduugg!

authors: MikeMcQuaid dduugg


text:
[WIP] add support for node module based formulas with homebrew managed resources

Warning: Do not merge this PR! It's in a early prototype state. Please don't see this purely as a PR to get something merged into core/brew. Please see it more like an discussion issue about handling node module formulas with homebrew managed dependencies as resources with an attached diff for review purpose.

[x] Have you followed the guidelines in our Contributing document?
[x] Have you checked to ensure there aren't other open Pull Requests for the same change?
[x] Have you added an explanation of what your changes do and why you'd like us to include them?
[ ] Have you written new tests for your changes? Here's an example.
[ ] Have you successfully ran brew tests with your changes locally?

I know this is a lot of text so feel free to skip any section not of your interest (like the 'How it installs the node modules' section which is a first draft of it's documentation which requires some knowledge about the node module spec )
About handling node module formulas
Inspired by the discussion in legacy-homebrew#50167 I started to work on a prototype of a Language::Node for supporting formulas for node modules (with cmd line tools) were all dependencies are managed by homebrew resources and not by npm as it's done now. Especially because I've believed that most of the concerns mentioned by @DomT4 in this comment are no longer an issue now. As it turned out that is the case for all points except the fact that the node community is taking the unix philosophy literally resulting in a high average count of dependencies / resources (which I have underestimated a little I have to admit). So please let's start the discussion if we want to have such a support, where we want to have it (core vs tap) and who we want to realize it!
How to try it out
I've created a tap for it over at chrmoritz/node so that you can easier try it out (I think this is now the best way to go for testing changes affecting both brew and homebrew-core).
brew tap chrmoritz/node
It also comes with a node based cmd line tool for generating homebrew node formulas in the proposed style. It will install the module globally with npm@3 to a /tmp folder and parses the dep tree output from npm ls --long --json to generate the formula. You can install with brew install chrmoritz/node/node-formulator --HEAD. Then type node-formulator -h for usage instruction (btw: sry for the name). Currently it only fully supports npm registry hosted dependencies and has no support for / is not tested with optional and bundled dependencies yet.
It works really great for small tools like generate-json-schema (currently in homebrew-core), rimraf, coffee-script and coffeelint or the node-formulator itself.
It still works fine for tools with a about average amount of dependencies like less and node-sass.
But is sadly currently hits homebrew limitation (e.g. no concurrent resource fetching) on very dependency heavy modules like azure-cli or autocode (both currently in homebrew-core).
How does it work
The proposed changes consist of 2 parts.

Adding a new Language::Node which include all the module installation logic and some other helper function (e.g. major node version helper for checking bottle compatibility of native addons).
Adding a new Resource::Node which extends the classic resource by two new fields: parent (used to map the dependency tree structure) and install (which is used to patch to install instruction for native addons to not use precompiled binaries, and yes we may should consider renaming it to avoid confusions).

As you might have seen: I'm not a ruby developer. I'd appreciate it if someone could point me into the right direction to get rid of the helper getters in the Resource::Node or give any other feedback to improve to code style.
How it installs the node modules
The actual installation is done by the Language::Node.node_modules_install resources, libexec/"node_modules" method. Just to say it at the beginning: The final structure of the node_modules folder is only partly npm compatible but fully node module spec compliant. I've taken the time to optimize it for our 'one time installation and never updating a dependency without reinstalling everything'-use case. It was slightly inspired by the flat dependency structure of npm@3, but I've gone further to be 100% flat (yes no nested modules for version conflicts) by using versioned dependencies and symlinks to their parents slightly inspired by pnpm (yes we don't have multiple copies of the same nested dependency, we just have multiple symlinks from our versioned dependency at the root level to them).
If you look at the generated formulas over at chrmoritz/homebrew-node/Formula you will see that there are to types of node resources. The first block of alphabetically sorted resources consists of modules simply named by their name, which should get installed at the root level the same way npm@3 would do it. The second block consist of the versioned modules (in npm@3 they would be nested modules) which are named like name@version, which we install in a versioned style at the root level to (similar to the way pnpm handles their modules). Don't worry they can't get accidentally get picked up because every not url safe character (including @) is a reserved character for module names in the npm ecosystem. Now that we have them there at one place (without unnecessary copies) we can create symlinks to their parents node_modules folder with only their names so they get picked up everywhere where they are needed. The information about their parents is read from the new resource metadata field which is filed by my formula generator tool with right information generated from the dependency tree. (Please note: In some very rare cases root dependency can also have parents, namely if a modules X dep in the dependency tree requires the version of module A from the root level, but somewhere in the dep tree branch between X and the root a other version of A is requested by another module and symlinked. Without the symlink from the root version of A this other version would be picked up by X.)
Handling native addons: This is still quite some work in progress in the current version. But the idea is that we have to patch most install scrips to not download prebuild binaries and build them always from source instead to be morw aligned with homebrews philosophy of building everything from source. This is done by adding an override field to the resource metadata describing how to compile the native addon, which should be used instead of the default one found in the scripts.install section of the package.json.
These fields are automatically generated by my tool from a patched version of the default install script in case of node-pre-gyp or prebuild usage, but they need to be edited by hand for native addons using a custom solution for providing binaries (but most native addons should use one of the 2 supported solutions or not provide any binaries at all).
Advantages over the current system
(All italic points could be backborted to the current system without homebrew manged resources)

installing doesn't rely on network access at install time (assuming you did a fetch earlier) as all dependencies are managed as npm resources
installing the same version of a formula with --build-from-source will always result in the same dependency versions to be installed. Currently the dependency are only locked to semver ranges and reinstalling the same version of a formula a week later will result in newer dependency release to be picked up (can bew fixed within the current system by applying a homebrew provided npm_shrinkwrap.json)
better handling for native addons inside the dependency chain:

I've a system in place for patching the install instructions to tell tools like node-pre-gyp or prebuild to always build them from source instead of downloading a prebuild binary (which is currently done for fibers in azure-cli).
I'm also adding a pour_bottle? requirement for a specific node major version for formulas with a native addon in it's dependency chain, because the bottle won't be compatible with a node version which doesn't match the major version number with the node version used to build the bottle. (Don't forget the revision azure-cli with the upgrade to node v5.)
the native addon compilation is independent of npm, so that it will work for users with a broken npm too (Why are still so many people playing russian roulett with their npm installation by running npm -g update npm instead of npm -g install npm@latest?).

I'm reading the package.jsons bin section for generating the bin.install_symlink code because the command name does not have to match the js file name (Just installing symlinks for everything in bin will not always do the correct thing. They don't even have to be located in the bin folder.)
a node module spec compliant 100% flat dependency structure with no duplication: I use versioned dependency (inspired by pnpm) instead of nested dependency to resolve version conflicts and symlink them to their parents node_modules folder instead of copying then multiple times over there. (We can do this more efficient than npm here because we don't need to support the 'updating only a specific dependency'-scenario, because every time we want to update a resource we have to reinstall the complete formula, which should happen only in a few exceptional cases outside of a formula version upgrade anyway.)
Formula generation can be done fully automated with a simple command line tool which fills everything out except for the test. (You may have to edit some fields read from the package.json like desc by hand through if they doesn't match homebrews style.)

Disadvantages of the proposed system
these are caused by the high amount of dependencies / resources for which homebrew wasn't really designed for

long homebrew formulas (over 1000 lines for very dependency heavy modules)
if building from source: long resources fetch time for dependency heavy modules (until homebrew implements concurrent resource fetching)

Summary
As I've sad before: Don't see this as a finished PR. See it more like an prototype, which should start a discussion about how we can improve support for node module based formulas in homebrew.
I would appreciate any feedback about how the current approve can be evolved and how this can / should be supported by homebrew before I invest more time in it.
I'm also prepared that you might say that the amount of resources is a show stopper for you (I wouldn't be disappointed if thats the case at this point, but I would like to get some feedback saying that this has a change before continuing working on it.).
And sorry for writing such a long text ;-).

I'll leave some thoughts here tomorrow, apologies for the silence so far.

No problem - I know that is a quite big topic to go through.
At this point the most important thing for me to know is, if this project will fail on the high number of resources it produces or if it has a change and it's worth to invest more time into it. Also some review of the Ruby part would help, because my Ruby skills are still quite limited. But if you want you can postpone the review of the node part of this (and over 50% of the text from above) until it is more finished.
Also note that native addon compilation is atm party broken, because npm relies on it's magic .bin folders for adding dep executable to the PATH at compilation time. But I've already an idea how to make the native addon compilation npm independent, which should solve this issue.
Should we go for a depends_on :python => :build build dependency if we detect a native addon or should we support system / user installed python (But in this case we can't guarantee that python in the PATH points to python 2.6/7 right?)?

I'm personally in favour of reducing the number of resources that need to be added manually or programmatically to formulae if there's an underlying package manager (i.e. npm) that doesn't just install everything from the tip of the master branch by default (I'm looking at you, go a few years ago), and allows packages to be vendored per-formula. I think npm meets both those criteria. The main area I personally see for improvement is pointing npm towards a cache that it can share between builds.

@mikemcquaid The design from this PR mostly addresses this comment from @DomT4.

This means realistically if a user doesn't want the bottle, they need to be online during install and have sufficient connection strength to install npm-using formulae. This is an annoying problem for a package manager that does caching, and people have come to expect to be able to brew fetch something in advance and have it install offline later. It's a niche problem certainly, but an important niche.

But I agree that there are some improvements from this PR we could apply to our current node module formulas even if we don't want to increase the number of resources in favor of offline instability (after a fetch):

doing global installs with a --prefix set to libexec instead of you current local installs
better handling of native addons

build dep on python
bottle pour requirement to same major node version
however avoiding pulling prebuilt binaries while continue using npm for installs is most likely not possible

Also npm already maintains a cache at ~/.npm for modules, but we could separate it to use a custom independent cache for homebrew install using the --cache=/path/to/cache parameter.

@mikemcquaid Also note that with the current approve two subsequent --build-from-source installs of the same version of a node module based formula can produce different results, because the dependencies aren't usually locked to specific versions (only to semver ranges). IMHO this issue is comparable with the go head install issue.
It could be resolved by either shipping a homebrew provided npm-shrinkwrap.json for every node module based formula or going with this homebrew managed node module resources approve, which is sadly quite resource heavy.

doing global installs with a --prefix set to libexec instead of you current local installs
Also npm already maintains a cache at ~/.npm for modules, but we could separate it to use a custom independent cache for homebrew install using the --cache=/path/to/cache parameter.

These both sound useful.

Also note that with the current approve two subsequent --build-from-source installs of the same version of a node module based formulas can produce different results, because the dependencies aren't usually locked to specific versions (only to semver ranges). IMHO this issue is comparable with the go head install issue you've mentioned.

I think that's actually desirable; if there's an upstream security release I'd hope the buildsystem picks it up.

I've pushed an update to make native addon compilation independnet from npm to this PR an the example formula for azure-cli: https://github.com/chrmoritz/homebrew-node/blob/master/Formula/azure-cli.rb#L1478-L1480

I think that's actually desirable; if there's an upstream security release I'd hope the buildsystem picks it up.

But in this case we should do a revision which updates the bottle too. Currently only --build-from-source builds would benefit from the security update while the bottle would remain unpatched.
Also I think that you can except from over 95% of the current formulas in homebrew-core, that when you  install version x of formula y (from source) you will get exactly the same result as when you repeat it one week later (with still the same formula version).
But in the case of node module based formulas you will atm get a completely different dependency tree a week later (different dep versions and maybe also a different dep tree layout too because of resolving/adding new version conflicts through a dep update).

But in this case we should do a revision which updates the bottle too. Currently only --build-from-source builds would benefit from the security update while the bottle would remain unpatched.

I'm definitely open to bumping a revision when that happens.

But in the case of node module based formulas you will atm get a completely different dependency tree a week later (different dep versions and maybe also a different dep tree layout too because of resolving/adding new version conflicts through a dep update).

Yeh, I'm sympathetic to that and open to solutions that don't involve extra work for either upstream or our formula contributors.

I think we can agree, that this PR on the one hand tries to solves a lot of issues with the current way we handle node modules based formulas, but unfortunately on the other hand introduces a new big issue through the big amount of dependencies it produces, which homebrew can't handle well enough atm. Also some :white_check_mark: (but not all :red_circle:) of the identified issues can be solved in another way too (without moving completely away from npm based installs). An example on how an improved formula can look like can be found here for azure-cli. I think we should move the discussion now more in the direction of these issues, which I've summarized bellow:
1. Online requirement during installs (Refs this comment) :red_circle:

only fixable by homebrew managed resource dependencies as proposed by this PR

2. Global vs local module installations :white_check_mark:

Independent of what way we choose, we should switch our module installation from local style to global style (with setting a custom prefix to libexec) as in this line
this brings our installations more in line with global installations through npm
prunes the obsoletly installed devDependencies (which are unnecessarily installed with our current local installations) and
uses npm for spec conform bin command name resolution (instead of relying on the developer to have a 1:1 mapping in a bin folder)

3. Locking dependencies to specific versions :white_check_mark:

atm when reinstalling the same version of the formula (from source) after a week you will get a completely different dependencies tree, because deps are only locked to semver ranges with the current approve
this can either be fixed by homebrew managed resource deps (as proposed by this PR) or by providing a npm-shrinkwrap.json as done here
this would require the generation of a new npm-shrinkwrap.json with every version upgrade by running npm shrinkwrap inside a global installation and updating the gist

4. Handling of the npm cache :white_check_mark:

atm the npm cache location gets overwritten by superenv to point to buildpath/.brew_home/.npm instead of ~/.npm
this results in over 10x higher build times than necessary making the current approve even less performant than the resource heavy approve proposed by this PR
to fix this (while sticking with npm based installs) we need to tell npm the real location of it's global cache (and not our fake buildpath/.brew_home/.npm cache)
a workaround for this can be found here (Can this be improved somehow (can we access the old home?)?)

5. NPM is not designed as a package manager for global installations :red_circle:

NPM is optimized for the local installation use case and because of this is not creating optimized global installations
the node module spec compliant format proposed in this PR is in 2 ways better than the global format produced by npm:

100% flat (no nested dependencies, just symlinks to their locations)
no duplications (Npm produces multiple copies of the same version of a dependency if it's required at different nested places. In the proposed format we would just create multiple symlinks from our versioned dep to the places where it is needed, resulting in smaller installation sizes)

6. Handling of native addons (inside the dependency tree)
6.1 Not pulling prebuild binaries :red_circle:

Solutions like node-pre-gyp or prebuild for pulling prebuild binaries for native addons are not compatible with homebrews philosophy to build everything from source on the CI / in --build-from-source builds. That's why I have a system in place in this PR for patching the installation step of native addons to tell such tools to always build from source.
Unfortunately it's is not possible to avoid pulling prebuild binaries of c-code for native addons if we stick with npm install based formula installations.

6.2 bottle compatibility with same major node version :white_check_mark:

native addons compiled against a Node version are only compatible with other Node version with the same major version (other Node version are not ABI compatible)
thats why we should consider adding a pour_bottle? requirement for native addons like done here for a specific node major version
also a build time dep to python 2.6/2.7 is required too

Online requirement during installs

As described above I think this is unfortunate but a problem I'd like to be fixed (if we do) without requiring anything else of formula version creators/bumpers than "add URL, add checksum, done".

Independent of what way we choose, we should switch our module installation from local style to global style (with setting a custom prefix to libexec) as in this line
this brings our installations more in line with global installations through npm
prunes the obsoletly installed devDependencies (which are unnecessarily installed with our current local installations) and

Sounds good :+1:

this can either be fixed by homebrew managed resource deps (as proposed by this PR) or by providing a npm-shrinkwrap.json as done here
this would require the generation of a new npm-shrinkwrap.json with every version upgrade by running npm shrinkwrap inside a global installation and updating the gist

I think this is too much overhead for formula creators/updaters. I think if upstream don't provide files for this then it's fine for our versions to change each time you install.

to fix this (while sticking with npm based installs) we need to tell npm the real location of it's global cache (and not our fake buildpath/.brew_home/.npm cache)
a workaround for this can be found here (Can this be improved somehow (can we access the old home?)?)

Seems like a good idea :+1: However, the location should be somewhere under HOMEBREW_CACHE; we don't want to provide write-access to $HOME during install. We could consider copying files in there from $HOME/.npm at some point.

the node module spec compliant format proposed in this PR is in 2 ways better than the global format produced by npm

I think this is something that's better taken upstream to npm.

Unfortunately it's is not possible to avoid pulling prebuild binaries of c-code for native addons if we stick with npm install based formula installations.

Again, this is regrettable but not that serious, currently, and should be taken upstream to npm.

native addons compiled against a Node version are only compatible with other Node version with the same major version (other Node version are not ABI compatible)
thats why we should consider adding a pour_bottle? requirement for native addons like done here for a specific node major version

We don't actually need that requirement; depends_on "node" means that we can rely on the newest version of node always being installed before the package is installed (even if they are pinned). This would only be required if we had a :node or similar that allowed using Nodes from homebrew-versions.

Great write-up here, thanks for all your thought and work.

4. Handling of the npm cache (+ 2. Global vs local module installations)

Seems like a good idea :+1: However, the location should be somewhere under HOMEBREW_CACHE; we don't want to provide write-access to $HOME during install. We could consider copying files in there from $HOME/.npm at some point.

I think I might have an idea of what is causing the current npm install slowness. If no explicit cache path is set npm tries to resolve ~/.npm both in it's main process and it's spawned child processes, but only it's main process is affected by the homebrew patched ENV["HOME"] by superenv. This results in a edge case because the cache is resolved to buildpath/.brew_home/./npm in the main process, but in the spawned child processes it's resolved to the actual /Users/username/.npm.
Setting the cache to somewhere inside HOMEBREW_CACHE doesn't seem to decrease the performance to much. These are the install times of azure-cli on a MBA 2012 (with a decent internet connection):

with prefilled cache in ~/.npm: ~45sec
with empty cache in a tmp dir: ~50sec
with unset cache causing edge case: aborted after 20 min (after running out of my 10gb free disk space)

Do you think the simple HOMEBREW_CACHE/.npm would be a good location or do you have something better in mind? Also do you think that this (and 2. global style module installation) would be worth creating a Language::Node helper similar to pythons setup_install_args which sets the cache (by creating a local .npmrc file with the cache entry set) and returning our npm install args for the global style ("npm", "install", "--global", "--prefix=#{libexec}", "--verbose", buildpath)?
1. Online requirement during installs + 3. Locking dependencies to specific versions
Ok, than I take this as a wont fix, because fixing it would add to much overhead.
5. NPM is not designed as a package manager for global installations

I think this is something that's better taken upstream to npm.

I disagree here. While it's not a common use case to update only a resource of a formula in homebrew, updating only specific dependencies is quite common in npm. What I meant here was, that we could optimize the module format more for the homebrew use case as it could be done for a npm use case.
But I think it's like 1 and 3 something we can't improve without adding to much overhead.
6.1 Native addons: Not pulling prebuild binaries

Again, this is regrettable but not that serious, currently, and should be taken upstream to npm.

Npm official only supports building native addons from source. These are both 3rd party solutions and it might be worth asking them to add a environment variable to force source builds, which we could set.
6.2 bottle compatibility with same major node version

We don't actually need that requirement; depends_on "node" means that we can rely on the newest version of node always being installed before the package is installed (even if they are pinned). This would only be required if we had a :node or similar that allowed using Nodes from homebrew-versions.

Agreed. What we would really need to improve this is a way to enforce them to be executed with a specific node major version, which would require something like a major version specific opt prefix. So I guess this is also something which is not easily fixable.

Do you think the simple HOMEBREW_CACHE/.npm would be a good location or do you have something better in mind? Also do you think that this (and 2. global style module installation) would be worth creating a Language::Node helper similar to pythons setup_install_args which sets the cache (by creating a local .npmrc file with the cache entry set) and returning our npm install args for the global style ("npm", "install", "--global", "--prefix=#{libexec}", "--verbose", buildpath)?

Yeh, this all sounds pretty good. I'd probably go for HOMEBREW_CACHE/npm just to make the directory more obvious.

Npm official only supports building native addons from source. These are both 3rd party solutions and it might be worth asking them to add an environment variable to force source builds, which we could set.

:+1:

Ok, than I take this as a wont fix, because fixing it would add to much overhead.
But I think it's like 1 and 3 something we can't improve without adding to much overhead.
Agreed. What we would really need to improve this is a way to enforce them to be executed with a specific node major version, which would require something like a major version specific opt prefix. So I guess this is also something which is not easily fixable.

Yeh, sadly on these. Thanks for understanding.

I've switched this PR to introduce the more simple Language::Node.npm_install_args helper method with this functionality:

make sure that a working copy of npm and node-gyp from node's libexec is prepended to the PATH (to not rely of a user managed npm installations)
sets the npm cache to HOMEBREW_CACHE/npm, which fixes issues under superenv resulting in long install times and high disk usage (https://github.com/Homebrew/brew/pull/37#issuecomment-208840366)
returns npm install args for a global style module installation to libexec

A working copy of the azure-cli using this new helper method can be found here: https://github.com/chrmoritz/homebrew-node/blob/master/Formula/azure-cli.rb (brew install chrmoritz/node/azure-cli).

Updated formulas for all npm install based formulas in homebrew-core (except tegh which is a candidate for the boneyard imho) using this new helper method can be found here: https://github.com/Homebrew/homebrew-core/compare/master...chrmoritz:node_modules

A couple of comments but I like the direction here :+1:

Addressed your comment and rebased on master.
I've also already changed the proposed formulas over at https://github.com/Homebrew/homebrew-core/compare/master...chrmoritz:node_modules to the new format, which has discarded you inline comments over there:
The current test of azure-cli is failing on node 6.0.0-rc.2, so native addon compatibility to node major version is already covered by it.
Also (node-)gyp requires Python 2.6 or 2.7. (Same as in the v8 formula)

Thinking this further I've added a second method named Language::Node.local_npm_install_args,
which can be used as a replacement for std_npm_install_args in formulas (like elixirscript and kibana), where npm install is only a part of the build step and global installations are therefore not suitable.
It still sets up the PATH for npm and our cache config but only returns the —verbose argument (as of now, but it might be extended in the future and therefore I've keeped it an array).

@chrmoritz Looks good to me now but would like to get @DomT4 to 👍 before I merge.

I'll try and look over and play with this tonight, but I'll get it done at the latest before Tuesday morning. One thing I can spot that'd be nice to get done is to get brew cleanup to clean the npm-cache in the same way we do the Java one.

Summary:
This mostly looks reasonable, from testing. I like the way it ended up. Added a few comments but there's nothing enormous.
I agree with Mike in particular that avoiding adding a wad of resources to every npm-using formula was an important step to getting this 🚢-able; we've discovered that heavy resource use on things like Python formulae tends to result in people not updating them unless something explicitly breaks, which is a solid way to end up missing upstream resource security or bugfix releases.
It is an annoyance that many formulae can be fetched and then installed offline, and then some very much cannot, and that's something I'd like to eventually see if we can stamp out. I consider it a lesser evil  though to require someone to be online than implementing another system where resource security updates are potentially missed.
Improvements:
One thing that would be nice to prevent is .brew_home being coped into every npm-using formula's libexec/lib/node_modules/name under the new system. It shouldn't cause any harm, but it could be confusing to users and is unnecessary clutter.
That directory will always contain at least a blob of folders followed by one file, the Python helper that makes sure Homebrew's site-packages for Python remain accessible to the system Python, as documented here.
Things that'd be nice to handle one day:
One thing that does make me wonder if we've room for improvement are these comments in some formulae such as here:

needs a revision for every node major version bump (native addon)

I worry a bit that a lot of people are going to forget those lines exist and we'll end up doing revisions after-the-fact (if at all) rather than at-the-time.
It's probably overly complex and we may just have to live with this new mechanism being an improvement in stability over the status quo but it'd be nice if audit detected Node was being bumped more than x amount (which should be easy enough given Node's versioning system) and there weren't expected changes to formulae with that notation.
I don't think that idea is particularly workable immediately though, it's more of a "nice to have in future" than "let's get this done today".

I've addressed 2 of you comments and I'm now looking into removing .brew_home from libexec.

I worry a bit that a lot of people are going to forget those lines exist and we'll end up doing revisions after-the-fact (if at all) rather than at-the-time.

As of now every node major version bump would trigger a test of all formulas which depend on node on the CI. If we would do a node major bump without adding a revision to azure-cli the test would fail because it would be compiled against node 5 but tested with node 6 for example. So this should be already covered by the current brew testbot implementation.

In "Dom is incredibly annoying" could you perhaps write a document like this (without needing to be anywhere near as extensive) that serves as guidance for formulae authors moving to the new system?
Particularly on how and when to use each helper method, when to revision formulae that use native addons and a short example Foo formula.
That's my last point of concern here about moving the system across. If we can get that done I'm personally happy to ship.

I've addressed all comments from above and squashed all fixup commits.
I will look into creating a documentation for the "annoying Don" tomorrow.

I've added a first draft of the Node for formula authors documentation to this PR.

Great work here, @chrmoritz. I'm happy to merge this if @DomT4 is.

Mostly documentation notes at this point. Nearly there on getting this wrapped.

I've updated the documentation to address your comments and rebased on master.

There's nothing I can see here that I've a problem with at this point. If nobody cries wolf over the next 12 hours or so I'll merge this then.

Accidentally squashed it, which is a minor annoyance in this situation, but merged in https://github.com/Homebrew/brew/commit/bc98fd37882c64c896dc2243fcc6e129f170a32a.
Thanks for sticking with us on this review journey @chrmoritz, appreciate your patience and dedication in getting this done. If you wanted to follow-up by changing a few of the Node-using formulae in homebrew/core to use the new system that'd be ❤ but entirely up to you.

Thx for merging this. I've submitted a rebase of the formulas (as proposed here) to core to make use of the new Language::Node in all node module based formulas (except tegh).

Thanks! Spotted that and left a comment on the tegh situation.

authors: DomT4 chrmoritz mikemcquaid


text:
Test officially supported cmd taps.

[x] Have you followed the guidelines in our Contributing document?
[x] Have you checked to ensure there aren't other open Pull Requests for the same change?
[x] Have you added an explanation of what your changes do and why you'd like us to include them?
[x] Have you written new tests for your changes? Here's an example.
[x] Have you successfully run brew tests with your changes locally?

All of these taps use Homebrew internal APIs (or will shortly) and we autoinstall them all from brew $CMD. We should adjust our CI to ensure that we never accidentally break these taps when making changes to core code so that these taps can rely more on this core code rather than having to e.g. vendor equivalent code that never changes on our end.
CI may well by wrong here as I've not tested this on the actual bot and it won't run brew tests --official-cmd-taps (although I should fix that so we can test new brew test-bot changes here).
The approach may well be terrible; we could perhaps get away with a simpler integration tests or a subset of their tests. Alternatively, we may also want to consider running these tests in parallel with something like https://github.com/grosser/parallel_tests.
This (in some form) will be a requirement for @anastasiasulyagina's GSoC project to be able to remove duplicated/vendored core code from Caskroom/cask and eventually move it into Homebrew itself and it's something we should have done for Homebrew/bundle a while ago. This also perhaps overlaps with @eirinikos's integration testing work.
CC @Homebrew/maintainers for thoughts as this is a fairly big change in how we do CI for this repository but it's something we'll have to do in some form.

@UniqMartin @xu-cheng Neither of you are opposed to the overall concept of this PR?

@UniqMartin @xu-cheng Neither of you are opposed to the overall concept of this PR?

I'm not opposed. Makes a lot of sense to me, though I personally don't care that much about these external commands because I don't use them. How long does this additional step approximately take to complete compared to a regular brew tests run? Does running this on our Mavericks test bot (with Ruby 1.8.7) make sense? Do those external commands even support Ruby 1.8.7?

I'm not oppose this PR's concept as well.

Have addressed feedback.

How long does this additional step approximately take to complete compared to a regular brew tests run? Does running this on our Mavericks test bot (with Ruby 1.8.7) make sense? Do those external commands even support Ruby 1.8.7?

Will get you the numbers for this after this PR runs the CI job.

Have you been able to run brew official-cmd-tap-tests successfully? If I try that locally on my machine (OS X 10.11) in a Homebrew installation without any pre-installed taps except homebew/core, the Homebrew Cask tests actually fail with:
$ brew official-cmd-tap-tests
==> Running caskroom/cask tests:
==> Tapping caskroom/cask
[…snip…]
8307 tests, 10105 assertions, 1 failures, 4 errors, 2395 skips
[…snip…]

The failure and those errors of course result in a nonzero exit code. On the other hand, the tests run for homebrew/bundle finish just fine.

Ignoring the failures for a moment, here are some numbers when running this locally:
$ /usr/bin/time brew tests --no-compat
[…snip…]
       74.16 real        57.36 user        16.19 sys
$ /usr/bin/time brew tests --no-compat
[…snip…]
       67.33 real        54.76 user        13.66 sys
$ /usr/bin/time brew official-cmd-tap-tests
[…snip…]
      353.80 real       142.69 user        39.52 sys
$ /usr/bin/time brew official-cmd-tap-tests
[…snip…]
      193.10 real        76.73 user        30.89 sys

Those extra tests are not exactly cheap, particularly on the first run when the taps need to be installed and the Gems required for testing need to be fetched and installed via bundle install. On our test bot we'll obviously always have the slower times because all the taps and Gems will need to be installed anew on every test bot run.

Cask's tests seem very oriented towards their Travis setup so I decided it was a bad idea to run the entire test suite (and probably overkill) so I decided to just write integration tests to smoke tests these three commands; in my experience if we e.g. break an API they are relying on we'll realise pretty quickly. I think this gives us the best of both worlds; fast integration tests (less than a second added in total for all three assuming they are already tapped) whilst making sure we don't obliterate the internal methods they rely on.

Commented some code, that is probably causing the test bot run to fail, and on a few details.

Cask's tests seem very oriented towards their Travis setup so I decided it was a bad idea to run the entire test suite (and probably overkill)

I agree.

so I decided to just write integration tests to smoke tests these three commands; in my experience if we e.g. break an API they are relying on we'll realise pretty quickly.

Makes sense and works for me.

I'm sorry, but I had to revert this in dba1958bd79c1c9d18f215dfc2b806ea62edd1c8. This needs more work and local testing via, e.g.,
brew tests --official-cmd-taps

before it can be merged (as the test bot won't run this code until it is merged, at which point it's too late). To quote from the revert commit:

Taps installed prior to running the test suite are not visible to the test suite as most Homebrew paths are redefined as to not mess up the local installation.

I just realized that there is/was another issue with the current implementation. The entire test suite now runs thrice: once with coverage in no-compat mode, once regularly, and now once more with the three integration tests from this PR enabled. The latter two runs are basically identical except for those newly added integration tests, resulting in some unnecessary and wasteful redundancy.

Addressed this in a series of commits ending in https://github.com/Homebrew/brew/commit/a1b0ef1300ff8ac07ef904479b0321a2a5fc0368 as currently the bot isn't doing CI testing with the correct version so it was easier to just ensure it was working properly on all platforms by pushing to master. The green builds for Travis and Jenkins are: https://travis-ci.org/Homebrew/brew/builds/142266951 http://bot.brew.sh/job/Homebrew Testing/1029/

authors: MikeMcQuaid UniqMartin xu-cheng


text:
DEFAULT_PREFIX: Use HOMEBREW_FORCE_USR_LOCAL_ON_LINUX

[x] Have you followed the guidelines in our Contributing document?
[x] Have you checked to ensure there aren't other open Pull Requests for the same change?
[x] Have you added an explanation of what your changes do and why you'd like us to include them?
[ ] Have you written new tests for your changes? Here's an example.
[x] Have you successfully run brew style with your changes locally?
[x] Have you successfully run brew tests with your changes locally?

This change was introduced in 51af87f85fe9ace55ecc6b86ba15773819e2f0da.

Great, thanks @sjackman 👍

authors: MikeMcQuaid sjackman


text:
Delete unused formulae

Delete all homebrew/gui formulae that show no installations since analytics records began.

Hello,
I actually need one of these formulae. Is there any way to, for example, copy it somewhere on my machine and then tell Homebrew to also look there when looking for available formulae so that it would be able to install it as if it was coming from an official tap?
Thanks in advance for any help!

@miclf which formula?

@miclf (as we may re-add it)

@dunn @MikeMcQuaid It’s Klavaro.
Note that I’m not asking to re-add it. If I’m the only one who needs this formula, I would be fine with an alternate way to make my Homebrew installation able to use it.

@miclf Reverted/readded in f2de0c0.

@MikeMcQuaid Thanks!

authors: MikeMcQuaid dunn miclf

repo: Homebrew/homebrew-gui

license_source: bigquery


text:
formulae_dependents: improve texlive handling

The texlive bottle is gigantic and has about a thousand dependencies.
This tends to make GitHub-hosted runners run out of storage by the time
they get around to testing it, particularly since it comes late in the
alphabet (and we test dependents in alphabetical order).
To try to minimise usage of our self-hosted runner, let's test texlive
as early as possible to increase the chances of the runner having
sufficient storage to be able to test texlive.

Success 🎉 https://github.com/Homebrew/homebrew-core/actions/runs/11030334118/job/30635055781?pr=191772#step:4:119

repo: Homebrew/homebrew-test-bot


text:
Fallback to HEAD and origin/master

Closes: #282
In comparison to the patch attached in issue post I added this:
... || diff_start_sha1.blank?

because I forgot to set GITHUB_BASE_REF in local Docker container and it was set in Github Actions job. So, looks like when GITHUB_BASE_REF is set, diff_start_sha1 becomes blank probably because git merge-base returns fatal: Not a valid object name #{diff_end_sha1}.
With these changes I am able to successfully run test-bot on Github Actions.
Successful job using these changes:
https://github.com/dawidd6/homebrew-tap/runs/237362215
Unsuccessful jobs not using these changes:
https://github.com/dawidd6/homebrew-tap/runs/236568969
https://github.com/dawidd6/homebrew-tap/runs/237214506
Obtaining PR patch differs between these 2 unsuccessful jobs but I think it works similarly anyway.

because I forgot to set GITHUB_BASE_REF in local Docker container and it was set in Github Actions job. So, looks like when GITHUB_BASE_REF is set, diff_start_sha1 becomes blank probably because git merge-base returns fatal: Not a valid object name #{diff_end_sha1}.

Can you test https://github.com/Homebrew/homebrew-test-bot/pull/284 instead? I was going to leave them as comments on this PR but I figured it might be quicker to create my own.

Closing in favour of https://github.com/Homebrew/homebrew-test-bot/pull/284 but happy to keep talking about this if it doesn't address the issue(s).

It slightly improved with your patch, thanks, but it's still not this.
Current situation:
Testing tap dawidd6/tap with:
  origin/master   38b267e (workflow: test upstream test-bot)
  HEAD            b571f31 (neofetch: test build with upstream test-bot)
  diff_start_sha1 b571f31
  diff_end_sha1   34a0815c9ed21a0aebe8eb2b7617a83366aa6098

Formula changes to be tested:
  added formulae    (empty)
  modified formulae (empty)
  deleted formulae  (empty)

Previous proper:
Testing tap dawidd6/tap with:
  origin/master   e32c27e (workflow: handle multiple packages)
  HEAD            07f46b5 (neofetch: test)
  diff_start_sha1 e32c27e
  diff_end_sha1   07f46b5

Formula changes to be tested:
  added formulae    (empty)
  modified formulae dawidd6/tap/neofetch
  deleted formulae  (empty)

You need to pass through the following variables: https://github.com/Homebrew/brew/blob/2c7091f06d801502bc0e55e4b9b876ff3007619d/Dockerfile.yml#L19-L26
Have you passed through all of those correctly? If not, I'll probably improve the error messaging again.

Just made a check:
https://github.com/dawidd6/homebrew-tap/runs/237767418
Run echo "GITHUB_ACTIONS    $GITHUB_ACTIONS"
  echo "GITHUB_ACTIONS    $GITHUB_ACTIONS"
  echo "GITHUB_BASE_REF   $GITHUB_BASE_REF"
  echo "GITHUB_EVENT_NAME $GITHUB_EVENT_NAME"
  echo "GITHUB_REF        $GITHUB_REF"
  echo "GITHUB_REPOSITORY $GITHUB_REPOSITORY"
  echo "GITHUB_SHA        $GITHUB_SHA"
  echo "HEAD_GITHUB_REF   $HEAD_GITHUB_REF"
  shell: /bin/bash -e {0}
GITHUB_ACTIONS    true
GITHUB_BASE_REF   master
GITHUB_EVENT_NAME pull_request
GITHUB_REF        refs/pull/37/merge
GITHUB_REPOSITORY dawidd6/homebrew-tap
GITHUB_SHA        6aa743b1aa236bf4079b5dbc2aa42f0018adbd3c
HEAD_GITHUB_REF

It seems like I don't have HEAD_GITHUB_REF set, but judging by code of test-bot it is not used anywhere.


text:
Resaving document with fontkit after adding text results in error

What were you trying to do?
I was trying to reuse already created PDFDocument by removing all pages and working with it as if it is new (the speedup, when it works, is around 30% in my case). The example would not contain removing and readding pages, because it has no effect on the error showing up as far as I tested.
How did you attempt to do it?

Create the document;
register fontkit;
embed font;
add pages, shapes, texts etc.
save the document;
add pages, shapes, texts etc.
save the document.

What actually happened?
Saving the document the second time resulted in exception, the exact place and type of the error depended on the font being embedded in the document
The exceptions:

Cannot read properties of undefined (reading 'advanceWidth')
Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'advanceWidth')
    at CustomFontEmbedder.computeWidths (CustomFontEmbedder.js:230:40)
    at CustomFontSubsetEmbedder.<anonymous> (CustomFontEmbedder.js:141:37)
    at step (tslib.es6.js:100:23)
    at Object.next (tslib.es6.js:81:53)
    at fulfilled (tslib.es6.js:71:58)
CustomFontEmbedder.computeWidths    @   CustomFontEmbedder.js:230
(anonymous) @   CustomFontEmbedder.js:141
step    @   tslib.es6.js:100
(anonymous) @   tslib.es6.js:81
fulfilled   @   tslib.es6.js:71
await in fulfilled (async)
(anonymous) @   test3.html:22

Uncaught RangeError: Index out of range
Uncaught RangeError: Index out of range
    at checkInt (fontkit.umd.js:1531:43)
    at Uint8Array.writeUInt16BE (fontkit.umd.js:1596:21)
    at TTFSubset._addGlyph (fontkit.umd.js:40914:17)
    at TTFSubset.encode (fontkit.umd.js:40952:13)
    at fontkit.umd.js:40692:14
    at Item.run (fontkit.umd.js:2632:13)
    at drainQueue (fontkit.umd.js:2597:35)
checkInt    @   fontkit.umd.js:1531
writeUInt16BE   @   fontkit.umd.js:1596
_addGlyph   @   fontkit.umd.js:40914
encode  @   fontkit.umd.js:40952
(anonymous) @   fontkit.umd.js:40692
Item.run    @   fontkit.umd.js:2632
drainQueue  @   fontkit.umd.js:2597
setTimeout (async)
runTimeout  @   fontkit.umd.js:2509
nextTick    @   fontkit.umd.js:2622
encodeStream    @   fontkit.umd.js:40691
(anonymous) @   CustomFontSubsetEmbedder.js:57
CustomFontSubsetEmbedder.serializeFont  @   CustomFontSubsetEmbedder.js:54
(anonymous) @   CustomFontEmbedder.js:195
step    @   tslib.es6.js:100
(anonymous) @   tslib.es6.js:81
(anonymous) @   tslib.es6.js:74
__awaiter   @   tslib.es6.js:70
CustomFontEmbedder.embedFontStream  @   CustomFontEmbedder.js:189
(anonymous) @   CustomFontEmbedder.js:154
step    @   tslib.es6.js:100
(anonymous) @   tslib.es6.js:81
(anonymous) @   tslib.es6.js:74
__awaiter   @   tslib.es6.js:70
CustomFontEmbedder.embedFontDescriptor  @   CustomFontEmbedder.js:149
(anonymous) @   CustomFontEmbedder.js:127
step    @   tslib.es6.js:100
(anonymous) @   tslib.es6.js:81
(anonymous) @   tslib.es6.js:74
__awaiter   @   tslib.es6.js:70
CustomFontEmbedder.embedCIDFontDict @   CustomFontEmbedder.js:123
(anonymous) @   CustomFontEmbedder.js:95
step    @   tslib.es6.js:100
(anonymous) @   tslib.es6.js:81
(anonymous) @   tslib.es6.js:74
__awaiter   @   tslib.es6.js:70
CustomFontEmbedder.embedFontDict    @   CustomFontEmbedder.js:91
CustomFontEmbedder.embedIntoContext @   CustomFontEmbedder.js:88
(anonymous) @   PDFFont.js:113
step    @   tslib.es6.js:100
(anonymous) @   tslib.es6.js:81
(anonymous) @   tslib.es6.js:74
__awaiter   @   tslib.es6.js:70
PDFFont.embed   @   PDFFont.js:108
(anonymous) @   PDFDocument.js:1321
step    @   tslib.es6.js:100
(anonymous) @   tslib.es6.js:81
(anonymous) @   tslib.es6.js:74
__awaiter   @   tslib.es6.js:70
PDFDocument.embedAll    @   PDFDocument.js:1312
(anonymous) @   PDFDocument.js:1203
step    @   tslib.es6.js:100
(anonymous) @   tslib.es6.js:81
(anonymous) @   tslib.es6.js:74
__awaiter   @   tslib.es6.js:70
PDFDocument.flush   @   PDFDocument.js:1200
(anonymous) @   PDFDocument.js:1258
step    @   tslib.es6.js:100
(anonymous) @   tslib.es6.js:81
(anonymous) @   tslib.es6.js:74
__awaiter   @   tslib.es6.js:70
PDFDocument.save    @   PDFDocument.js:1241
(anonymous) @   test3.html:18
await in (anonymous) (async)
(anonymous) @   test3.html:22

(filename is test3.html, previous two are also bugs)
What did you expect to happen?
Save the document without receiving exceptions, since I haven't found any signs that I cannot save the document multiple times.
How can we reproduce the issue?
<html><head>
<script src="https://unpkg.com/pdf-lib@1.17.1/dist/pdf-lib.js"></script>
<script src="https://unpkg.com/@pdf-lib/fontkit@1.1.1/dist/fontkit.umd.js"></script>
<script>
(async() => {
    const link = 'https://fonts.cdnfonts.com/s/29105/ARIAL.woff' //Cannot read properties of undefined (reading 'advanceWidth')
    //const link = 'https://fonts.gstatic.com/s/roboto/v30/KFOkCnqEu92Fr1MmgVxMIzIFKw.woff2' //index out of range
    const fontBytes = await fetch(link).then(res => res.arrayBuffer());
    const pdfDoc = await PDFLib.PDFDocument.create();
    pdfDoc.registerFontkit(window.fontkit);

    const font = await pdfDoc.embedFont(fontBytes, { subset: true });

    const c1 = String.fromCodePoint(1072) //cyrrilic 'а'
    const c2 = String.fromCodePoint(1073) //cyrrilic 'б'
    const page = pdfDoc.addPage([1,1]);
    page.drawText(c1, { font });
    await pdfDoc.save();
    page.drawText(c2, { font });
    await pdfDoc.save();
    console.log('done')
})()
</script>
</head></html>

Version
pdf-lib: 1.17.1, fontkit: 1.1.1
What environment are you running pdf-lib in?
Browser
Checklist

[X] My report includes a Short, Self Contained, Correct (Compilable) Example.
[X] I have attached all PDFs, images, and other files needed to run my SSCCE.

Additional Notes
Browser: Chrome 114
OS: ChromeOS 114

Hi. I've got the similar error but in different scenario.
How did you attempt to do it?

Load PDF document from a file
Register new font
Embed the font
Copy the first page
Fill in with text and image
Add the page to the same document
Save the document

What actually happened?

Same error reported
```
/node_modules/pdf-lib/src/core/embedders/CustomFontEmbedder.ts:231
      currSection.push(currGlyph.advanceWidth * this.scale);
                                 ^
TypeError: Cannot read properties of undefined (reading 'advanceWidth')
    at CustomFontSubsetEmbedder.CustomFontEmbedder.computeWidths (/node_modules/pdf-lib/src/core/embedders/CustomFontEmbedder.ts:231:34)
    at CustomFontSubsetEmbedder. (/node_modules/pdf-lib/src/core/embedders/CustomFontEmbedder.ts:155:15)
    at step (/node_modules/pdf-lib/node_modules/tslib/tslib.js:141:27)
    at Object.next (/node_modules/pdf-lib/node_modules/tslib/tslib.js:122:57)
    at fulfilled (/node_modules/pdf-lib/node_modules/tslib/tslib.js:112:62)
```

Version
pdf-lib: 1.17.1
fontkit: 1.1.1
What environment are you running pdf-lib in?
node 18.13.0

What I found
My error was caused by this line:
const [page] = await doc.copyPages(doc, [0]);
// Fill in texts and images
doc.addPage(page);

Which I attempted to duplicate the first page to the new page, fill text and image, then add the page back to the same doc.
I resolved this by
Instead of copying to the same doc, I create a new doc and copy the page to the new doc instead. Something similar to this:
const targetedDoc = await PDFDocument.create();
const [page] = await targetedDoc.copyPages(doc, [0]);
// Fill in texts and images
targetedDoc.addPage(page);

Hope this info might be useful for this bug.

authors: VanaIgr mixth

repo: Hopding/pdf-lib

license: MIT


text:
run local_test.sh    "slow_conv2d_cpu" not implemented for 'Half'

runbash local_test.sh sparse4dv3_temporal_r50_1x8_bs6_256x704  path/*pt
BUG log：
No CUDA runtime is found, using CUDA_HOME='/root/miniconda3/envs/sprase4D'
/root/miniconda3/envs/sprase4D/lib/python3.8/site-packages/mmcv/init.py:20: UserWarning: On January 1, 2023, MMCV will release v2.0.0, in which it will remove components related to the training process and add a data transformation module. In addition, it will rename the package names mmcv to mmcv-lite and mmcv-full to mmcv. See https://github.com/open-mmlab/mmcv/blob/master/docs/en/compatibility.md for more details.
warnings.warn(
projects.mmdet3d_plugin
{'version': 'JRDB_v1.0'}
distributed: False
/root/miniconda3/envs/sprase4D/lib/python3.8/site-packages/mmdet/models/backbones/resnet.py:401: UserWarning: DeprecationWarning: pretrained is deprecated, please use "init_cfg" instead
warnings.warn('DeprecationWarning: pretrained is deprecated, '
load checkpoint from local path: /root/sparse4D_track/work_dirs/JRDB_sparse4dv3_temporal_r50_1x8_bs6_480x768_local/iter_46480.pth
[                                                  ] 0/27661, elapsed: 0s, ETA:/root/miniconda3/envs/sprase4D/lib/python3.8/site-packages/torch/amp/autocast_mode.py:202: UserWarning: User provided device_type of 'cuda', but CUDA is not available. Disabling
warnings.warn('User provided device_type of 'cuda', but CUDA is not available. Disabling')
Traceback (most recent call last):
File "./tools/test.py", line 313, in
main()
File "./tools/test.py", line 258, in main
outputs = single_gpu_test(model, data_loader, args.show, args.show_dir)
File "/root/miniconda3/envs/sprase4D/lib/python3.8/site-packages/mmdet/apis/test.py", line 29, in single_gpu_test
result = model(return_loss=False, rescale=True, **data)
File "/root/miniconda3/envs/sprase4D/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
return forward_call(*input, **kwargs)
File "/root/miniconda3/envs/sprase4D/lib/python3.8/site-packages/mmcv/parallel/data_parallel.py", line 49, in forward
return self.module(*inputs[0], **kwargs[0])
File "/root/miniconda3/envs/sprase4D/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
return forward_call(*input, **kwargs)
File "/root/miniconda3/envs/sprase4D/lib/python3.8/site-packages/mmcv/runner/fp16_utils.py", line 236, in new_func
output = old_func(*new_args, **new_kwargs)
File "/root/sparse4D_track/projects/mmdet3d_plugin/models/sparse4d.py", line 97, in forward
return self.forward_test(img, **data)
File "/root/sparse4D_track/projects/mmdet3d_plugin/models/sparse4d.py", line 113, in forward_test
return self.simple_test(img, **data)
File "/root/sparse4D_track/projects/mmdet3d_plugin/models/sparse4d.py", line 116, in simple_test
feature_maps = self.extract_feat(img)
File "/root/miniconda3/envs/sprase4D/lib/python3.8/site-packages/mmcv/runner/fp16_utils.py", line 149, in new_func
output = old_func(*new_args, **new_kwargs)
File "/root/sparse4D_track/projects/mmdet3d_plugin/models/sparse4d.py", line 75, in extract_feat
feature_maps = self.img_backbone(img)
File "/root/miniconda3/envs/sprase4D/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
return forward_call(*input, **kwargs)
File "/root/miniconda3/envs/sprase4D/lib/python3.8/site-packages/mmdet/models/backbones/resnet.py", line 636, in forward
x = self.conv1(x)
File "/root/miniconda3/envs/sprase4D/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
return forward_call(*input, **kwargs)
File "/root/miniconda3/envs/sprase4D/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 463, in forward
return self._conv_forward(input, self.weight, self.bias)
File "/root/miniconda3/envs/sprase4D/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 459, in _conv_forward
return F.conv2d(input, weight, bias, self.stride,
RuntimeError: "slow_conv2d_cpu" not implemented for 'Half'
my torch version is :1.13.0
Name: torch
Version: 1.13.0
Summary: Tensors and Dynamic neural networks in Python with strong GPU acceleration
Home-page: https://pytorch.org/
Author: PyTorch Team
Author-email: packages@pytorch.org
License: BSD-3
Location: /root/miniconda3/envs/sprase4D/lib/python3.8/site-packages
Requires: typing_extensions
Required-by: torchaudio, torchvision
cuda vesrion：
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Tue_Mar__8_18:18:20_PST_2022
Cuda compilation tools, release 11.6, V11.6.124
Build cuda_11.6.r11.6/compiler.31057947_0
nvidia-smi：
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.67                 Driver Version: 550.67         CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 3090        On  |   00000000:20:00.0 Off |                  N/A |
| 32%   39C    P8             25W /  350W |      11MiB /  24576MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
|   1  NVIDIA GeForce RTX 3090        On  |   00000000:21:00.0 Off |                  N/A |
| 32%   40C    P8             20W /  350W |      11MiB /  24576MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
+-----------------------------------------------------------------------------------------+
test cuda ：
``import torch
print(torch.version)
print(torch.cuda.is_available())
print(torch.cuda.amp.common.amp_definitely_not_available())
``
result
1.13.0
True
False

This seems to be a common issue. It is recommended to look for answers in other forums. Maybe the cuda_home setting is incorrect.

Make sure CUDA is available in you mmdet and mmcv packages:
python /usr/local/lib/python3.8/dist-packages/mmdet/utils/collect_env.py to see that mmcv and mmdet support CUDA.
You should see at the bottom of the printout:
TorchVision: 0.14.0+cu117
OpenCV: 4.10.0
MMCV: 1.7.1
MMCV Compiler: GCC 9.3
MMCV CUDA Compiler: 11.7
If not try installing from this whl or build from source with required flags (solved the issue for me):
pip install mmcv-full==1.7.0 -f https://download.openmmlab.com/mmcv/dist/cu117/torch1.13/index.html
good luck

Make sure CUDA is available in you mmdet and mmcv packages: python /usr/local/lib/python3.8/dist-packages/mmdet/utils/collect_env.py to see that mmcv and mmdet support CUDA. You should see at the bottom of the printout: TorchVision: 0.14.0+cu117 OpenCV: 4.10.0 MMCV: 1.7.1 MMCV Compiler: GCC 9.3 MMCV CUDA Compiler: 11.7
If not try installing from this whl or build from source with required flags (solved the issue for me): pip install mmcv-full==1.7.0 -f https://download.openmmlab.com/mmcv/dist/cu117/torch1.13/index.html good luck

Thank you for your help
I have to fix this problem because the local_test.sh script default export CUDA_VISIBLE_DEVICES=3, I only have one GPU device, and when I set it to export CUDA_VISIBLE_DEVICES=0, everything works.

g: User provided device_type of 'cuda', but CUDA is not available. Disabling
warnings.warn('User provided device_type of 'cuda', but CUDA is not available. Disabling')

echo $CUDA_VISIBLE_DEVICES   will help.


text:
Husky pre-commit hook stopped working on all packages

💥 Regression Report
When commiting husky isn't properly called anymore.
The error is:
Can't find Husky, skipping pre-commit hook
You can reinstall it using 'npm install husky --save-dev' or delete this hook

To Reproduce
Steps to reproduce the behavior: use any yarn commit- script from monorepo or just commit from inside the specific package using cd packages/server && yarn commit
Expected behavior
pre-commit hook working and checking build and tests
Your Environment

os: MacOS
Node.js: v12.14.0
yarn: v1.19.1

@fox1t, this is what I get when I try to do yarn commit in the frontend repo:
husky > commit-msg (node v13.5.0)
⧗   input: feat(toolbar): add button toolbar
✖   subject may not be empty [subject-empty]
✖   type may not be empty [type-empty]

✖   found 2 problems, 0 warnings
ⓘ   Get help: https://github.com/conventional-changelog/commitlint/#what-is-commitlint

os: MacOs
Node.js: v13.5.0
yarn: 1.21.1

This is a different issue that has been fixed by @tehKapa: pull the last version and make sure to do a yarn install in order to force that package to be upgraded.

OK I found the reason of this.
https://github.com/typicode/husky/issues/667

Another issue that came to life: husky stops working if package dependencies were installed using yarn workspaces. However it is not consistent.

repo: HospitalRun/hospitalrun


text:
コーヒー取得API・レビュー取得APIのページネーション対応

クエリパラメータにoffsetを使い何件目から取得するかを指定できるようになった

動作確認どこからすればいいだろ

方法1

localhost:5000 でバックエンドのサーバーが動いている場合。

curl http://localhost:5000/reviews?offset=10
curl http://localhost:5000/coffees?offset=10

のようにCurlコマンドで確かめられるよ
方法2
フロントのページネーションPRと組み合わせて、フロントでの動作を確認。

動作確認できた！

repo: Hotsukai/HigaCoffeeDBBack


text:
Supported versions of Java?

Which versions of Java does HtmlUnit support?
I did not find any mention if it does support Java9+ and no luck building it myself so far, keep getting
java.lang.module.FindException: Unable to derive module descriptor for \xalan-2.7.2.jar

It isn't clear what exactly doesn't work but expected to from this issue.

The Package netscape inside htmlunit package will conflict with jdk.jsobject.
And will getting XXX module reading netscrape.javascrape from both htmlunit and jdk.jsobject error.
The netscape folder inside htmlunit should remove and use the jdk.jsobject version.

Well that would also require an update of the project to java 9.
I guess the easiest way to achieve functionality for java 8 and java9++ is to extract the netscape.javascript into it's own project, release it and reference it in the pom.
By that any Java9++ project with modules can exclude the reference to netscape.javascript and will no longer face issues of duplicated modules.

authors: baha2046 skyhirider slonopotamus twendelmuth

repo: HtmlUnit/htmlunit


text:
test make static

What problem does this PR solve?

Issue Number: Close #xxx
What is changed and how does it work?

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No code

Code changes

Has the configuration change
Has HTTP API interfaces changed (Don't forget to add the declarative for the new API)
Has persistent data change

Side effects

Possible performance regression
Increased code complexity
Breaking backward compatibility

Related changes

PR to update pingcap/docs/pingcap/docs-cn:
PR to update pingcap/tiup:
Need to cherry-pick to the release branch

Release note

None.

Welcome to Codecov :tada:
Once merged to your default branch, Codecov will compare your coverage reports and display the results in this comment.
Thanks for integrating Codecov - We've got you covered :open_umbrella:

authors: HuSharp codecov-commenter

repo: HuSharp/pd


text:
pre-trained model for video style transfer

Thanks for the great work! It looks like this repo currently only hosts code and model for image style transfer. Could you provide a pre-trained model (and perhaps test scripts) for video style transfer as discussed in the paper? Thank you.

Is the Video version is available now?


text:
defaultTdHeight配置好像没有起作用

version
0.9.23
Link to minimal reproduction
No response
Steps to reproduce
defaultTdHeight配置项没有起作用
What is expected?
defaultTdHeight配置项起作用
What is actually happening?
defaultTdHeight配置项没有起作用
System Info
No response
Any additional comments?
No response

0.9.28
相似issues：https://github.com/Hufe921/canvas-editor/issues/162


text:
设置图片当做背景

在初始化编辑器时，能否支持设置一张图片作为背景

想了一下 是否直接css设置canvas的背景图更为方便高效且灵活？

可以


text:
Russian translation update

Very small changes actually.

Wow, thanks!
Will be in the next beta coming very soon.

authors: HugoGresse dimqua

repo: HugoGresse/Anecdote


text:
Issue with visualisation

BCIO tool
Query or bug description
I am trying to use the tool to make a visualisation for an evidence synthesis paper but some BCTs do not fit properly in the hierarchy.
Relevant BCTs are: BCIO:007002,BCIO:007003,BCIO:007008,BCIO:007010,BCIO:007012,BCIO:007014,BCIO:007015,BCIO:007017,BCIO:007019,BCIO:007022,BCIO:007023,BCIO:007025,BCIO:007027,BCIO:007029,BCIO:007031,BCIO:007034,BCIO:007036,BCIO:007037,BCIO:007039,BCIO:007040,BCIO:007042,BCIO:007043,BCIO:007050,BCIO:007051,BCIO:007052,BCIO:007055,BCIO:007057,BCIO:007058,BCIO:007062,BCIO:007063,BCIO:007066,BCIO:007068,BCIO:007069,BCIO:007070,BCIO:007072,BCIO:007073,BCIO:007074,BCIO:007075,BCIO:007077,BCIO:007081,BCIO:007089,BCIO:007094,BCIO:007095,BCIO:007097,BCIO:007100,BCIO:007138,BCIO:007142,BCIO:007143,BCIO:007157,BCIO:007158,BCIO:007160,BCIO:007161,BCIO:007171,BCIO:007172,BCIO:007173,BCIO:007175,BCIO:007177,BCIO:007179,BCIO:007189,BCIO:007192,BCIO:007194,BCIO:007204,BCIO:007209,BCIO:007234,BCIO:007239,BCIO:007241,BCIO:007245,BCIO:007252,BCIO:007290,BCIO:007302,BCIO:007303,BCIO:050333,BCIO:050334,BCIO:050337,BCIO:050350,BCIO:050410
Some BCTs are floating off of the main hierarchy. Any fixes would be appreciated for use in the paper.

OK, I have now realised some items coded were actually behaviours and not BCTs - thus they were not in the hierarchy of BCTs. Please ignore!

I noticed one item was not a BCT but rather a behaviour but removing it did not fix the visualisation

repo: HumanBehaviourChangeProject/ontologies


text:
new technique ? HiP-CT

Hirachical i? phase contrast tomography

https://mecheng.ucl.ac.uk/hip-ct/

solved with #426


text:
Create Masks Using RectangleLabels and Export to COCO Format

Is your feature request related to a problem? Please describe.
I am unable to train my Mask-RCNN using bounding box annotations created in Label Studio. This is because there are no masks created when using the RectangleLabels to annotate. There is also no mask export happening when using COCO export.
Describe the solution you'd like
I previously used a different tool called COCO Annotator that allowed me to annotate using bounding boxes and export annotations that had both bboxes and segmentation components of the COCO data format. Label studio exports do not have segmentation components when exporting in COCO format.
Describe alternatives you've considered
My alternative is at this point to switch to a Faster-RCNN that doesnt utilize masking. It should be fine but I would like to be able to do either.
Additional context
Annotation produced using label-studio:

Annotation produced using COCO Annotator:

This feature would be very helpful for work I am trying to do too (training a Mask-RCNN model). Are there any alternatives to generating both bounding boxes and segmentation masks for training?

What should be in segmentation in case of bboxes?

You can calculate the bounding boxes by having the min and max of the coordinates from the masks.

@kodecreer do you have examples with similar coco exports?

The same process from the PyTorch object detection tutorials if you are able to extract just each mask from the image into a black and white photo like traditional Mask-R CNN
pos = np.where(mask)
xmin = np.min(pos[1])
xmax = np.max(pos[1])
ymin = np.min(pos[0])
ymax = np.max(pos[0])
boxes.append([xmin, ymin, xmax, ymax])

@kodecreer Maybe you could contribute this?

Have you solved the problem?

authors: Cazual kodecreer makseq njflove p4tr1ckc4rs0n

repo: HumanSignal/label-studio


text:
fix: DIA-699: [FE] We are still using 'Items' instead of 'Records' on some places monorepo

Hi @yyassi-heartex!
This PR was created in a response to PRs in upstream repos:

https://github.com/HumanSignal/dm2/pull/273

It's better don't merge this PR RN, It would rollback changes that I did this morning!

It's better don't merge this PR RN, It would rollback changes that I did this morning!

good to know, but keep in mind this PR was auto generated so you'll likely be getting more through out the day/week, etc


text:
Impact Area editor has some odd behavior

The OK button is enabled to start with:

But if you click it, you get this message and then the ok button becomes not enabled:

The file selector is still enabled which means that you could select a new file, but the unique column selector never shows up so I don't think you could ever save it with a new file.

You can't ever save this editor because there is no unique column visible.

So if imported, should we disable the OK button and then file selection button? we don't allow any editing so it makes sense that this screen would be mostly read only

Yeah, we could allow a name change, but the user can always right click the element and select "rename". We could just lock it all down and make it read only, or we could lock down the file selection and allow a name and description change. I think ideally we would allow the user to edit the name and description.

Let's allow the user to edit name and description. How much time do you anticipate this task will take?

1 to 2 hrs

Ok. Let's keep this in back log for now, saving our pennies for a rainy day...

repo: HydrologicEngineeringCenter/HEC-FDA


text:
Login Page is not working

Hello All,
I am not able to login after providing my details.

It is not showing my profile details..

Any update for this issue? i am facing same issue

authors: ctgdevops trinathtiru

repo: Hygieia/Hygieia


text:
I cannot clear my trash pokemons

I do pokelog before clean_trash and after it finishes the scanning the pokemons, I do clean_trash and it scans again, checks ping. Nothing happens at the end. Is there a trash list or something ? pls help
edit: ive just updated the bot but it releases one and it stops.

Sometimes, pokecord or your bot might lag down. This is usually prominent in the clean_trash command.
I will rewrite this function if Miles adds a mass release option.
In your case, it either might be the lag, or it may be the case where the computed duplicates itself are wrong.
After line 535, add a print(pokes) to see which is the case.

I tried that code after line 535. First try, I wanted to clean_trash but pokecord didnt answer then checked the ping. Then it stopped. At the second try, it released 15 pokemons. Then checked the pokemon list.

After everytime when i try, it shows [ ] this and it says Succesfully cleared the junk. There's too many trash, it doesnt see them i think.

So the problem is with populating the trash list.
Rewriting the mass trasher is a long term goal after Pokecord 3.0 is released. Till then stick to other commands.
I'll leave this issue open but won't be working on it for a while.

Fixed in https://github.com/Hyperclaw79/PokeBall-SelfBot/commit/150170291d89ed001fa80d18080a71c44fd7bc0c.