text:
Perform L0 compaction before creating new image layers

Problem
If there are too many L0 layers before compaction, the compaction process becomes slow because of slow timeline::get. As a result of the slowdown, the pageserver will generate even more L0 layers for the next iteration, further exacerbating the slow performance.
Summary of changes
Perform L0 -> L1 compaction before of creating new images. It simple change speedups compaction time and timeline::get to 5x. timeline::get is faster on top of L1 layers.
For layers map:

reconstruct data latency before:

and after:

Checklist before requesting a review

[x] I have performed a self-review of my code.
[ ] If it is a core feature, I have added thorough tests.
[ ] Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
[ ] If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

[ ] Do not forget to reformat commit message to not include the above checklist

In other words, we could merge of the WAL records from different L0 layers "on-the-fly", as part of image layer creation, similar to how the L0 compaction does.

Yep, we'd like to do this at some point in the medium-term future: https://github.com/neondatabase/neon/issues/4979
In the near term, the measurements in the PR description are quite persuasive... conversely we might need to think a bit and see if we can identify write patterns where this model becomes particularly painful.

The idea with the current code is that if you're about to create image layers anyway, it's a waste of time and unnecessary write amplification to compact the deltas, because subsequent getpage requests will make use of the image layers and won't look at the deltas anymore.

@hlinnaka , thank you. Is it correct to assume that when PITR horizon is set, we will perform L0 to L1 compaction anyway?

We're reconstruct the pages with individual "retail" getpage calls, but if you know you're going to call getpage for every page, you could surely do it more efficiently.

Interesting. Btw, even after L0->L1 compaction, creating of new images is long process. It's CPU bound to 1 CPU. Disk usage is low:
iostat -xmdy 10 5 /dev/vda
Linux 5.15.0-88-generic (pageserver2) 	11/28/2023 	_x86_64_	(12 CPU)

Device            r/s     rMB/s   rrqm/s  %rrqm r_await rareq-sz     w/s     wMB/s   wrqm/s  %wrqm w_await wareq-sz     d/s     dMB/s   drqm/s  %drqm d_await dareq-sz     f/s f_await  aqu-sz  %util
vda            106.50     13.11     0.00   0.00    0.19   126.05   30.30     12.87     8.00  20.89    6.47   434.79    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.22  43.28

Device            r/s     rMB/s   rrqm/s  %rrqm r_await rareq-sz     w/s     wMB/s   wrqm/s  %wrqm w_await wareq-sz     d/s     dMB/s   drqm/s  %drqm d_await dareq-sz     f/s f_await  aqu-sz  %util
vda            105.00     12.99     0.00   0.00    0.20   126.69   29.90     12.84     5.50  15.54    1.67   439.83    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.07  42.24

Device            r/s     rMB/s   rrqm/s  %rrqm r_await rareq-sz     w/s     wMB/s   wrqm/s  %wrqm w_await wareq-sz     d/s     dMB/s   drqm/s  %drqm d_await dareq-sz     f/s f_await  aqu-sz  %util
vda            105.90     13.06     0.00   0.00    0.21   126.24   32.60     12.84     5.40  14.21    1.55   403.41    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.07  43.00

Device            r/s     rMB/s   rrqm/s  %rrqm r_await rareq-sz     w/s     wMB/s   wrqm/s  %wrqm w_await wareq-sz     d/s     dMB/s   drqm/s  %drqm d_await dareq-sz     f/s f_await  aqu-sz  %util
vda            106.20     13.09     0.00   0.00    0.22   126.25   37.80     12.86     7.40  16.37    1.74   348.48    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.09  43.20

Device            r/s     rMB/s   rrqm/s  %rrqm r_await rareq-sz     w/s     wMB/s   wrqm/s  %wrqm w_await wareq-sz     d/s     dMB/s   drqm/s  %drqm d_await dareq-sz     f/s f_await  aqu-sz  %util
vda            106.10     13.08     0.00   0.00    0.20   126.27   33.80     12.86     7.60  18.36    1.62   389.61    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.08  43.44

Should the compacting be reworked for multi-threading, or is there still potential for further optimizations on single core?

Btw, even after L0->L1 compaction, creating of new images is long process. It's CPU bound to 1 CPU. Disk usage is low:

My apologies, I mistakenly compared the release build (before) with the debug build (after) in the description 🙂. After L0 compaction, image creation works much better. Although it remains CPU-limited, it now performs more IO:
$ iostat -xmdy 10 5 /dev/vda
Linux 5.15.0-88-generic (pageserver2) 	11/28/2023 	_x86_64_	(12 CPU)

Device            r/s     rMB/s   rrqm/s  %rrqm r_await rareq-sz     w/s     wMB/s   wrqm/s  %wrqm w_await wareq-sz     d/s     dMB/s   drqm/s  %drqm d_await dareq-sz     f/s f_await  aqu-sz  %util
vda           1314.49    158.81     0.00   0.00    0.17   123.71  270.73    119.48    25.17   8.51    6.97   451.90    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    2.11  99.90

Device            r/s     rMB/s   rrqm/s  %rrqm r_await rareq-sz     w/s     wMB/s   wrqm/s  %wrqm w_await wareq-sz     d/s     dMB/s   drqm/s  %drqm d_await dareq-sz     f/s f_await  aqu-sz  %util
vda           1156.90    143.07     0.00   0.00    0.16   126.64  253.40    141.08    23.40   8.45   10.30   570.11    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    2.79  99.96

authors: ivaxer jcsp


text:
Pg stat statements reset for neon superuser

Problem
Extension pg_stat_statements has function pg_stat_statements_reset().
In vanilla Postgres this function can only be called by superuser role or other users/roles explicitly granted.
In Neon no end user can use superuser role.
Instead we have neon_superuser role.
We need to grant execute on pg_stat_statements_reset() to neon_superuser
Summary of changes

Modify the Postgres v14, v15, v16 submodule pg_Stat_statements contrib to grant execute to superuser
for existing installations that have already "CREATE EXTENSION pg_stat_statements()" modify compute.rs to apply the config change (GRANT EXECUTE) when compute node is restarted

Checklist before requesting a review

[ ] I have performed a self-review of my code.
[ ] If it is a core feature, I have added thorough tests.
[ ] Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
[ ] If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

[ ] Do not forget to reformat commit message to not include the above checklist

You don't need to use approved-for-ci-run label, we use it for running CI for external contributors (because we don't want to run CI automatically for untrusted code)
See https://github.com/neondatabase/neon/blob/main/CONTRIBUTING.md#how-to-run-a-ci-pipeline-on-pull-requests-from-external-contributors

https://github.com/neondatabase/neon/actions/runs/7302179882/job/19900348069?pr=6232 job failed:
Expected postgres-v14 rev to be at '0bb356aa0cd1582112926fbcf0b5370222c2db6d', but it is at '7de983ea08e2d9c11be79093b9620adfc9b479e2'
Expected postgres-v15 rev to be at '24333abb81a9ecae4541019478f0bf7d0b289df7', but it is at 'd9d6957148ec46b0cadae49e2c3a9e5f7573e4b5'
Expected postgres-v16 rev to be at '863b71572bc441581efb3bbee2ad18af037be1bb', but it is at '6700d915d6f9e2c762f51d6dd0744807902c0ce5'
Please update vendors/revisions.json if these changes are intentional

source: gharchive/pull-request

authors: Bodobolero bayandin

repo: neondatabase/neon

license_type: permissive

license_source: github-api


text:
pageserver: fix timeline metrics lingering after timeline deletion

Problem
Metrics for timelines lingered after their deletion, resulting in oversized prometheus scrapes on pageservers that had run for some time.
Closes: #7221
Summary of changes

Implement a Drop for TimelineMetrics that deregistered metrics if shutdown() wasn't called
Call TimelineMetrics::shutdown during deletion

This is intentionally a "double fix" -- users of TimelineMetrics should politely call shutdown(), but we also shouldn't rely on them doing so.
Checklist before requesting a review

[ ] I have performed a self-review of my code.
[ ] If it is a core feature, I have added thorough tests.
[ ] Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
[ ] If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

[ ] Do not forget to reformat commit message to not include the above checklist

https://github.com/neondatabase/neon/pull/7233 will have the same net effect.
@problame we could still merge this to get the deregister-on-drop safety net?  Or I can close this and wait for 7233.

I've been stalling the review because I wanted to wait for #7233 .
I would like to get #7233 landed first, see if that solves the issue.
I would rather not have the impl Drop that this PR (re-)adds:
https://github.com/neondatabase/neon/pull/7222/files#diff-e4df5e2a32ef2508a0eddb2adf019fbb4760f582192d97910116c1bb552c273aR2084-R2090
We should just get the object/task lifecycles right, using gates + cancel.
One approach would be to make the TimelineMetrics sensitive to Timeline::cancel.

authors: jcsp problame


text:
pageserver: batch InMemoryLayer puts, remove need to sort items by LSN during ingest

Problem/Solution
TimelineWriter::put_batch is simply a loop over individual puts.  Each put acquires and releases locks, and checks for potentially starting a new layer.  Batching these is more efficient, but more importantly unlocks future changes where we can pre-build serialized buffers much earlier in the ingest process, potentially even on the safekeeper (imagine a future model where some variant of DatadirModification lives on the safekeeper).
Ensuring that the values in put_batch are written to one layer also enables a simplification upstream, where we no longer need to write values in LSN-order.  This saves us a sort, but also simplifies follow-on refactors to DatadirModification: we can store metadata keys and data keys separately at that level without needing to zip them together in LSN order later.
Why?
In this PR, these changes are simplify optimizations, but they are motivated by evolving the ingest path in the direction of disentangling extracting DatadirModification from Timeline.  It may not obvious how right now, but the general idea is that we'll end up with three phases of ingest:

A) Decode walrecords and build a datadirmodification with all the simple data contents already in a big serialized buffer ready to write to an ephemeral layer <-- this part can be pipelined and parallelized, and done on a safekeeper!
B) Let that datadirmodification see a Timeline, so that it can also generate all the metadata updates that require a read-modify-write of existing pages
C) Dump the results of B into an ephemeral layer.

Related: https://github.com/neondatabase/neon/issues/8452
Caveats
Doing a big monolithic buffer of values to write to disk is ordinarily an anti-pattern: we prefer nice streaming I/O.  However:

In future, when we do this first decode stage on the safekeeper, it would be inefficient to serialize a Vec of Value, and then later deserialize it just to add blob size headers while writing into the ephemeral layer format.  The idea is that for bulk write data, we will serialize exactly once.
The monolithic buffer is a stepping stone to pipelining more of this: by seriailizing earlier (rather than at the final put_value), we will be able to parallelize the wal decoding and bulk serialization of data page writes.
The ephemeral layer's buffered writer already stalls writes while it waits to flush: so while yes we'll stall for a couple milliseconds to write a couple megabytes, we already have stalls like this, just distributed across smaller writes.

Benchmarks
This PR is primarily a stepping stone to safekeeper ingest filtering, but also provides a modest efficiency improvement to the wal_recovery part of test_bulk_ingest.
test_bulk_ingest:
test_bulk_insert[neon-release-pg16].insert: 23.659 s
test_bulk_insert[neon-release-pg16].pageserver_writes: 5,428 MB
test_bulk_insert[neon-release-pg16].peak_mem: 626 MB
test_bulk_insert[neon-release-pg16].size: 0 MB
test_bulk_insert[neon-release-pg16].data_uploaded: 1,922 MB
test_bulk_insert[neon-release-pg16].num_files_uploaded: 8
test_bulk_insert[neon-release-pg16].wal_written: 1,382 MB
test_bulk_insert[neon-release-pg16].wal_recovery: 18.981 s
test_bulk_insert[neon-release-pg16].compaction: 0.055 s

vs. tip of main:
test_bulk_insert[neon-release-pg16].insert: 24.001 s
test_bulk_insert[neon-release-pg16].pageserver_writes: 5,428 MB
test_bulk_insert[neon-release-pg16].peak_mem: 604 MB
test_bulk_insert[neon-release-pg16].size: 0 MB
test_bulk_insert[neon-release-pg16].data_uploaded: 1,922 MB
test_bulk_insert[neon-release-pg16].num_files_uploaded: 8
test_bulk_insert[neon-release-pg16].wal_written: 1,382 MB
test_bulk_insert[neon-release-pg16].wal_recovery: 23.586 s
test_bulk_insert[neon-release-pg16].compaction: 0.054 s

Checklist before requesting a review

[ ] I have performed a self-review of my code.
[ ] If it is a core feature, I have added thorough tests.
[ ] Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
[ ] If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

[ ] Do not forget to reformat commit message to not include the above checklist

This should be good to go, but I plan on merging it after Monday's release branch so that it gets a week in staging.


text:
Double operator fails

Hello,
I am having problems when I have formula with "double" operators.
E.g:

--4+5 -> ParseError: syntax error, unexpected '4' (T_LNUMBER)
+-4+5 -> Exception: NXP\Exception\UnknownOperatorException
-+4+5 -> Exception: NXP\Exception\UnknownOperatorException

Thanks for the issue.
I won't be able to look at this for about 2 weeks, but I thought I might be able to help you with a few observations.
First, MathExecutor will not parse anything that is not valid in PHP.  So write a PHP script that does what you want and see if it is valid PHP.
Second (but related to above), I suspect that you need to do something like 10 - -4 + 5.  I think the issue is the first - (or plus) sign is not associated with a value to operate on.  The second +/- is associated with the following value (or should be).  This could be a case of precedence which should be easy to fix if that is the case.
Hope this helps and I will take a look in July.

I think that, all expressions talked in this issue (--4+5, +-4+5, -+4+5, 10--4+5) are not valid and MathExecutor lib should have no code support them.
Minus or plus operators work great with both positive and negative variables and there is no problem.
    $calculator = new MathExecutor();
    $calculator->setVar("prevYearEarning",1000);
    $calculator->setVar("thisYearEarning",-500);
    echo $calculator->execute("$prevYearEarning - $thisYearEarning"); // outputs 1500

If anyone wants to execute an expression like "--4+5" successfully, the expression must be normalized before executation.
For example:
    function normalizeDoubleMinusPlusOperators(string $expression):string{
        $expression = str_replace(" ", "", $expression);
        return str_replace(["--","-+","+-","++"],["+","-","-","+"], $expression);
    }
    $calculator = new MathExecutor();
    echo $calculator->execute(normalizeDoubleMinusPlusOperators("--4+5")); // outputs 9

Another option is surrounding all numbers with brackets to ensure it will execute successfully:
    $calculator = new MathExecutor();
    echo $calculator->execute("-(-4)+(5)"); // outputs 9
    echo $calculator->execute("(10)-(-4)+(5)"); // outputs 19

The real test is to replicate what PHP does. Some of these expressions don't work in PHP, so no need to support them.  Others are easily worked around, so not a huge priority at this point.

source: gharchive/issue

authors: fatihkizmaz johnrazeur phpfui

repo: neonxp/MathExecutor

license: MIT


text:
Do we need ProcessManager / Saga in the first implementation?

Just a bookmark issue, to not forget this ;)
Personally I hate Saga, and ProcessManager describe more clearly what it is. I think that we can avoid the concept in the first implementation, but at some point I feel that we will need something in the area.

Related to this discussion https://github.com/neos/Neos.Cqrs/issues/3#issuecomment-245561196

I don't think that we'd need any specific support for this as ProcessManagers can be modeled just like aggregates AFAIK.

I'm also against implementing them at this point. Especially long running (persisted) process managers are tricky.
I think the difference between a process manager and an aggregate is that the latter actually issues commands and waits for resulting events from other aggregates. But I'm not completely into that topic.

I think the difference between a process manager and an aggregate is that the latter actually issues commands and waits for resulting events from other aggregates

Exactly my gut feeling but I also haven't really worked with them yet.
BTW: My point was not that PM == AR (even though that could be argued) but that we can use the already existing mechanics for those

Chatting about this in the #guild-cqrs channel I realized that there is a missing mechanism in order to work with PMs (it's not exactly related to PM/Saga but something we'd need anyways IMO):
Currently aggregates can only reconstitute their state from own events. For a PM we'd need a mechanism to listen only to a specific set of events.
With the current mechanism we have for EventListeners we can already "pro-actively" determine the event type that is handled by a particular listener. For a PM we'd probably need a more advanced mechanism so that the system can determine where to forward a certain event to. I.E. an OrderProcessManager instance might only be interested in events of a specific Order. See https://github.com/neos/Neos.EventStore/issues/1 for a related requirement.

So one thing I found is that there is some difference between the Saga concept in CQRS (long running transaction) and what I personally see as ProcessManager:
A ProcessManager for me just defines a reaction (normally a new Command, but could IMO also be an Event - more later) to a specific series of Events that have happened, without any implications to transactional behaviour. The transactional behaviour would mean that the events in the process either all happen, or none at all. I think this is not required in a lot (if not most) cases and that actually leads to making ProcessManagers hard to implement.
Also in most writing one finds about this concept, a ProcessManager dispatches a new Command only. However, a Command is just a Message that implicates that it can be rejected, while an Event is also a Message but one that just can't be rejected. An Event is a fact, deal with it or ignore it.
I find there are use-cases where a process will not trigger something that is only possibly done, but a process can also just model a very fixed thing that will inevitably happen if a series of things happened before. In the simplest case, such a process could just model a transformation/translation of a single event (for example between two BCs).
So a ProcessManager signature for me looks like: (Event[]) -> Event|Command
On the implementation side, it would just be an EventListenerInterface that has a CommandBus (there's the use-case for the CommandBus ;)) and an EventBus injected and available for sending messages to the system.
The events the ProcessManager emits should be considered transient, so are not persisted in the EventStore (otherwise you should just make it an Aggregate or command another Aggregate). For replay, the ProcessManagers are registered, so they will still take part (and therefore should not contain side effects, see #24).
Of course, this opens up the ProcessManager for "altering history" in the sense of any changes to the events they emit will change the outcome of replaying the history.
What do you think? Is this a reasonable concept of a ProcessManager or am I trying to make the ProcessManager something that it isn't?

I don't think that we need to implement anything special for Sagas/ProcessManagers.
As @albe said, it's just a EventListener that can have state and access to CommandHandlers.
See #24 for a related discussion

authors: Akii albe bwaidelich dfeyer


text:
#2673 Breaks gopls

Description
mod_cache does not seem to be set correctly with the performance improvement using async_run_command. Introduced in 9a2cc569c88662fa41d414bdb65b13ea72349f86 (#2673). Reverting the changes fix this.
Neovim version
NVIM v0.8.3
Build type: RelWithDebInfo
LuaJIT 2.1.0-beta3
Compiled by runneradmin@fv-az171-224
Features: -acl +iconv +tui
See ":help feature-compile"
system vimrc file: "$VIM\sysinit.vim"
fall-back for $VIM: "C:/Program Files (x86)/nvim/share/nvim"
Run :checkhealth for more info
Nvim-lspconfig version
No response
Operating system and version
Windows 10
Affected language servers
gopls
Steps to reproduce

go mod init example/hello
nvim hello.go

Actual behavior
[lspconfig] unhandled error: ...-lspconfig/lua/lspconfig/server_configurations/gopls.lua:16: attempt to get length of upval
ue 'mod_cache' (a nil value)

Expected behavior
No response
Minimal config
local on_windows = vim.loop.os_uname().version:match 'Windows'

local function join_paths(...)
  local path_sep = on_windows and '\\' or '/'
  local result = table.concat({ ... }, path_sep)
  return result
end

vim.cmd [[set runtimepath=$VIMRUNTIME]]

local temp_dir = vim.loop.os_getenv 'TEMP' or '/tmp'

vim.cmd('set packpath=' .. join_paths(temp_dir, 'nvim', 'site'))

local package_root = join_paths(temp_dir, 'nvim', 'site', 'pack')
local lspconfig_path = join_paths(package_root, 'test', 'start', 'nvim-lspconfig')

if vim.fn.isdirectory(lspconfig_path) ~= 1 then
  vim.fn.system { 'git', 'clone', 'https://github.com/neovim/nvim-lspconfig', lspconfig_path }
end

vim.lsp.set_log_level 'trace'
require('vim.lsp.log').set_format_func(vim.inspect)
local nvim_lsp = require 'lspconfig'
local on_attach = function(_, bufnr)
  local function buf_set_option(...)
    vim.api.nvim_buf_set_option(bufnr, ...)
  end

  buf_set_option('omnifunc', 'v:lua.vim.lsp.omnifunc')

  -- Mappings.
  local opts = { buffer = bufnr, noremap = true, silent = true }
  vim.keymap.set('n', 'gD', vim.lsp.buf.declaration, opts)
  vim.keymap.set('n', 'gd', vim.lsp.buf.definition, opts)
  vim.keymap.set('n', 'K', vim.lsp.buf.hover, opts)
  vim.keymap.set('n', 'gi', vim.lsp.buf.implementation, opts)
  vim.keymap.set('n', '<C-k>', vim.lsp.buf.signature_help, opts)
  vim.keymap.set('n', '<space>wa', vim.lsp.buf.add_workspace_folder, opts)
  vim.keymap.set('n', '<space>wr', vim.lsp.buf.remove_workspace_folder, opts)
  vim.keymap.set('n', '<space>wl', function()
    print(vim.inspect(vim.lsp.buf.list_workspace_folders()))
  end, opts)
  vim.keymap.set('n', '<space>D', vim.lsp.buf.type_definition, opts)
  vim.keymap.set('n', '<space>rn', vim.lsp.buf.rename, opts)
  vim.keymap.set('n', 'gr', vim.lsp.buf.references, opts)
  vim.keymap.set('n', '<space>e', vim.diagnostic.open_float, opts)
  vim.keymap.set('n', '[d', vim.diagnostic.goto_prev, opts)
  vim.keymap.set('n', ']d', vim.diagnostic.goto_next, opts)
  vim.keymap.set('n', '<space>q', vim.diagnostic.setloclist, opts)
end

-- Add the server that troubles you here
local name = 'gopls'
-- local cmd = { 'pyright-langserver', '--stdio' } -- needed for elixirls, lua_ls, omnisharp
if not name then
  print 'You have not defined a server name, please edit minimal_init.lua'
end
if not nvim_lsp[name].document_config.default_config.cmd and not cmd then
  print [[You have not defined a server default cmd for a server
    that requires it please edit minimal_init.lua]]
end

nvim_lsp[name].setup {
  cmd = cmd,
  on_attach = on_attach,
}

print [[You can find your log at $HOME/.cache/nvim/lsp.log. Please paste in a github issue under a details tag as described in the issue template.]]

LSP log
[START][2023-07-26 09:12:41] LSP logging initiated
[INFO][2023-07-26 09:12:41] .../lua/vim/lsp.lua:1808	"exit_handler"	{}

output of go env GOMODCACHE

# go env GOMODCACHE
C:\Users\connor.chang\go\pkg\mod

problem is why there does not get this value ..although we can add a nil check but it should be have a value.

In here:
https://github.com/neovim/nvim-lspconfig/blob/654dd9e95c49f64726b8751d75cbe147ec626875/lua/lspconfig/server_configurations/gopls.lua#L11-L14
It looks like result is nil. I'm not sure how async_run_command works, but somehow it's failing. I think we should change the code to fall back to using vim.fn.system for now.
        local result = util.async_run_command 'go env GOMODCACHE'
        if result and result[1] then
          mod_cache = vim.trim(result[1])
        else
          mod_cache = vim.fn.system 'go env GOMODCACHE'
        end

I don't think it make sense to have a code path where mod_cache is nil anyway

vim.fn.system is sync. and problem not is nil check.  it's only in windows async get command output failed. in *nix it works fine !

Sorry, I'm not sure what you are trying to say. vim.fn.system works fine. That's why I proposed the code above, to fall back to run in sync when async fails. I can create a pr, but this wouldn't fix the issue with async_run_command.

fallback to sync will cause an performance issue . could you try the cmd like i comment on above ?

Did not work :(. I did some more testing and this only happens when launching from msys2 environment, so some environment variable might be interfering. Does not happen when launching from cmd or using neovim qt.

If it helps at all, I'm having the same problem with WSL2 using Ubuntu 22.04.2 LTS.
When I run go env GOMODCACHE I get the following output:
go env GOMODCACHE
/home/gilad/go/pkg/mod

I was using Window Terminal with Git Bash, and I was getting the same error. If I keep using Windows Terminal but use PowerShell instead of Bash, the error goes away.

snap go has the same issue

I'm having the same problem since updating to nvim v0.9.x, I temp fixed by setting the value of mod_cache at the top of gopls.lua file with the value thrown by go env GOMODCACHE
local util = require 'lspconfig.util'
local mod_cache = '/home/myuser/go/pkg/mod'

In another station, which runs nvim v0.8.1, works fine, with the latest lspconfig update.  I dont want to update nvim on that one yet.

I'm having the same problem since updating to nvim v0.9.x, I temp fixed by setting the value of mod_cache at the top of gopls.lua file with the value thrown by go env GOMODCACHE
local util = require 'lspconfig.util'
local mod_cache = '/home/myuser/go/pkg/mod'

In another station, which runs nvim v0.8.1, works fine, with the latest lspconfig update. I dont want to update nvim on that one yet.

Had same issue with nvim  0.9.1 with lazy-nvim, replaced my nvim-data/lazy/nvim-lspconfig/lua/lspconfig/server_configurations/gopls.lua and set the mod_cache default from nil to whatever prints out in your terminal when you run go env GOMODCACHE, for windows remember you will need to escape the backslashes, so replace \ with \\

Hey! Is there some outcome here? This is still open and even with the most recent changes and neovim version 0.9.2 on MacOS it does cause issues...

For anyone having this issue, you can overwrite the root_dir in your config:
local util = require'lspconfig.util'
require'lspconfig'.gopls.setup {
   capabilities = default_capabilities(),
   on_attach = on_attach,
   root_dir = require'lspconfig'.gopls.setup {
      -- ...some other setups
      root_dir = function(fname)
         -- see: https://github.com/neovim/nvim-lspconfig/issues/804
         local mod_cache = vim.trim(vim.fn.system 'go env GOMODCACHE')
         if fname:sub(1, #mod_cache) == mod_cache then
            local clients = vim.lsp.get_active_clients { name = 'gopls' }
            if #clients > 0 then
               return clients[#clients].config.root_dir
            end
         end
         return util.root_pattern 'go.work'(fname) or util.root_pattern('go.mod', '.git')(fname)
      end,
   }
}

or if you're still having issues:
require'lspconfig'.gopls.setup {
   -- ...some other setups
   root_dir = function(fname)
      return util.root_pattern 'go.work'(fname) or util.root_pattern('go.mod', '.git')(fname)
   end,
}

For those who are experiencing the same issue, you can fix the error by manually providing the config with the proper value set for the variable mod_cache. To do this, first obtain the value by running::
go env GOMODCACHE
# for me result is:
# /root/go/pkg/mod

Now, provide the configuration for gopls with the variable value set to the result of calling the previous command (you can do this in your init.lua, for example).
-- loading module to provide config for a server following steps from guide here
-- https://github.com/neovim/nvim-lspconfig/blob/ede4114e1fd41acb121c70a27e1b026ac68c42d6/doc/lspconfig.txt#L326
local configs = require 'lspconfig.configs'

-- copy paste from
-- https://github.com/neovim/nvim-lspconfig/blob/ede4114e1fd41acb121c70a27e1b026ac68c42d6/lua/lspconfig/server_configurations/gopls.lua
local util = require 'lspconfig.util'
local async = require 'lspconfig.async'
-- -> the following line fixes it - mod_cache initially set to value that you've got from `go env GOMODCACHE` command
local mod_cache = '/root/go/pkg/mod'

-- setting the config for gopls, the contents below is also copy-paste from
-- https://github.com/neovim/nvim-lspconfig/blob/ede4114e1fd41acb121c70a27e1b026ac68c42d6/lua/lspconfig/server_configurations/gopls.lua
configs.gopls = {
    default_config = {
      cmd = { 'gopls' },
      filetypes = { 'go', 'gomod', 'gowork', 'gotmpl' },
      root_dir = function(fname)
        -- see: https://github.com/neovim/nvim-lspconfig/issues/804
        if not mod_cache then
          local result = async.run_command 'go env GOMODCACHE'
          if result and result[1] then
            mod_cache = vim.trim(result[1])
          end
        end
        if fname:sub(1, #mod_cache) == mod_cache then
          local clients = vim.lsp.get_active_clients { name = 'gopls' }
          if #clients > 0 then
            return clients[#clients].config.root_dir
          end
        end
        return util.root_pattern 'go.work'(fname) or util.root_pattern('go.mod', '.git')(fname)
      end,
      single_file_support = true,
    },
    docs = {
      description = [[
  https://github.com/golang/tools/tree/master/gopls

  Google's lsp server for golang.
  ]],
      default_config = {
        root_dir = [[root_pattern("go.work", "go.mod", ".git")]],
      },
    },
  }

svitiashchuk Thanks a lot! That worked indeed. Had this issue on windows 11 , even though on macOS there was no such issue

~/.local/share/nvim/site/pack/packer/start/nvim-lspconfig/lua/lspconfig/server_configurations/gopls.lua
just wanted to add this here, (linux).  I had trouble finding the file location. (I use packer as my package manager.
like others have the same setup on another machine. no issues. but when I booted this machine up it gave me this issue.  thanks @svitiashchuk !

I'm having this issue too and have tried all the solutions here to no avail.
This is only happening on nvim on windows.
on my nvim on fedora and ubuntu i don't have this issue

@EJammy Thanks alot for providing the override snippet. Even though i dont understand all of it, it solved the issue.

For the record, this is happening to me on an Ubuntu machine (uname -a: Linux -- 6.5.0-15-generic #15~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Fri Jan 12 18:54:30 UTC 2 x86_64 x86_64 x86_64 GNU/Linux) as well, so it's not just on windows.

Happened for me also on Ubuntu 22.04
Noticed that go commands were not printing anything when redirected to file, this file was empty:
go env GOMODCACHE > file
The issue was gone after removing go/gopls installed through ubuntu (sudo snap remove go), and installing manually from https://go.dev/doc/install .
Possibly caused by https://bugs.launchpad.net/ubuntu/+source/snapd/+bug/1849753 - the symptoms are the same as in duplicate of this bug -- https://bugs.launchpad.net/snapd/+bug/1921110

Hello everyone,
I am having the same problem i tried to override the mod_cache in ~/.local/share/nvim/lazy/nvim-lspconfig/lua/lspconfig/server_configurations/gops.lua but then i got the following error
[lspconfig] Cannot access configuration for gopls. Ensure this server is listed in server_configurations.md or added as a custom server.
I checked the server_configuration.md, gopls is listed there
Is there any solution for this

So... is this still an issue? https://github.com/neovim/nvim-lspconfig/issues/2733 was a "partial" fix.

So... is this still an issue? #2733 was a "partial" fix.

Yeah, I just hit this, an updated nvim-lspconfig let gopls run but it wasn't fully working.
I had neovim and go installed via snaps, running go env GOMODCACHE was working outside neovim but not from inside (i.e:  :! go env ...)
I took hauserx's hint from above and reinstalled go without snap and everything worked perfectly, even on the older nvim-lspconfig.

authors: Aaronik BaktashGorgani BalanceBalls CalebJohnHunt DownerCase EJammy MikeDepinto bartektelec giladsher glepnir hauserx juparave justinmk muazmd nitschmann svitiashchuk tashish-soni x000zh

repo: neovim/nvim-lspconfig


text:
fix(elixirls): smarter detection of root_dir for umbrella apps

Should fix issues identified following merge of https://github.com/neovim/nvim-lspconfig/pull/2910

Great work everyone!

authors: georgeguimaraes giddie


text:
BUG: depretated package: neptune and kedro-datasets

Describe the bug
Running Kedro neptune throws deprecation warnings
Reproduction
pip install -U kedro-neptune

Add the example code from https://docs.neptune.ai/integrations/kedro/
kedro run

Traceback
$ kedro run
[06/29/23 14:47:33] WARNING  /Users/sagenos/mambaforge/lib/python3.10/s warnings.py:109
                             ite-packages/kedro_neptune/__init__.py:32:
                              DeprecationWarning:
                             `kedro.extras.datasets` is deprecated and
                             will be removed in Kedro 0.19, install
                             `kedro-datasets` instead by running `pip
                             install kedro-datasets`.
                               from kedro.extras.datasets.text import
                             TextDataSet

                    WARNING  /Users/sagenos/mambaforge/lib/python3.10/s warnings.py:109
                             ite-packages/botocore/httpsession.py:41:
                             DeprecationWarning:
                             'urllib3.contrib.pyopenssl' module is
                             deprecated and will be removed in a future
                             release of urllib3 2.x. Read more in this
                             issue:
                             https://github.com/urllib3/urllib3/issues/
                             2680
                               from urllib3.contrib.pyopenssl import
                             orig_util_SSLContext as SSLContext

                    WARNING  /Users/sagenos/mambaforge/lib/python3.10/s warnings.py:109
                             ite-packages/neptune/internal/backends/hos
                             ted_client.py:50:
                             NeptuneDeprecationWarning: The
                             'neptune-client' package has been
                             deprecated and will be removed in the
                             future. Install the 'neptune' package
                             instead. For more, see
                             https://docs.neptune.ai/setup/upgrading/
                               from neptune.version import version as
                             neptune_client_version

Environment
The output of pip list:
~/code/ltv-prediction add-terraform* ❯ pip list                           10s 14:47:42
Package                       Version     Editable project location
----------------------------- ----------- ------------------------------------------------------------
absl-py                       1.4.0
aiobotocore                   2.4.2
aiofiles                      22.1.0
aiohttp                       3.8.4
aioitertools                  0.11.0
aiokafka                      0.8.0
aiosignal                     1.3.1
alabaster                     0.7.13
alembic                       1.8.1
altair                        5.0.1
analytics-python              1.4.post1
antlr4-python3-runtime        4.9.3
anyconfig                     0.10.1
anyio                         3.6.2
appdirs                       1.4.4
appnope                       0.1.3
argilla                       1.3.2
argon2-cffi                   21.3.0
argon2-cffi-bindings          21.2.0
arrow                         1.2.3
asgiref                       3.6.0
astroid                       2.15.5
asttokens                     2.2.1
astunparse                    1.6.3
async-timeout                 4.0.2
attrs                         23.1.0
autoflake                     2.1.1
autoflake8                    0.4.0
autopep8                      2.0.1
azure-common                  1.1.28
azure-core                    1.27.1
azure-mgmt-core               1.4.0
azure-mgmt-resource           23.0.1
Babel                         2.12.1
backcall                      0.2.0
backoff                       1.10.0
bcrypt                        4.0.1
beautifulsoup4                4.12.2
binaryornot                   0.4.4
black                         22.12.0
blacken-docs                  1.13.0
bleach                        6.0.0
blinker                       1.6.2
boto3                         1.24.59
botocore                      1.27.59
bravado                       11.0.3
bravado-core                  5.17.1
Brotli                        1.0.9
brotlipy                      0.7.0
bs4                           0.0.1
build                         0.10.0
cachetools                    5.3.1
certifi                       2023.5.7
cffi                          1.15.1
cfgv                          3.3.1
chardet                       5.1.0
charset-normalizer            3.1.0
chromadb                      0.3.21
click                         8.1.3
click-help-colors             0.9.1
click-params                  0.3.0
clickhouse-connect            0.5.20
cloudpickle                   2.2.1
colorama                      0.4.6
comm                          0.1.3
commonmark                    0.9.1
conda                         23.1.0
conda-package-handling        2.0.2
conda_package_streaming       0.7.0
cookiecutter                  2.1.1
cookiecutter-project-upgrader 1.3.1
copier                        8.0.0
copilot                       0.0.4
coverage                      7.2.7
cryptography                  39.0.1
cycler                        0.11.0
databricks-cli                0.17.7
dataclasses-json              0.5.7
debugpy                       1.6.7
decorator                     5.1.1
defusedxml                    0.7.1
Deprecated                    1.2.13
dill                          0.3.6
distlib                       0.3.6
distro                        1.8.0
dnspython                     2.3.0
docker                        6.1.3
docker-pycreds                0.4.0
docstring-parser              0.15
docutils                      0.20.1
duckdb                        0.7.1
dunamai                       1.17.0
dynaconf                      3.1.12
ecdsa                         0.18.0
entrypoints                   0.4
et-xmlfile                    1.1.0
evidently                     0.2.2
exceptiongroup                1.1.1
executing                     1.2.0
fake-useragent                1.1.3
fastapi                       0.75.2
fastapi-utils                 0.2.1
fastjsonschema                2.16.3
filelock                      3.12.0
fire                          0.5.0
flake8                        3.9.2
Flake8-pyproject              1.2.3
Flask                         2.2.3
flatbuffers                   23.5.26
fonttools                     4.39.3
fqdn                          1.5.1
frozenlist                    1.3.3
fsspec                        2022.11.0
funcy                         2.0
future                        0.18.3
gast                          0.4.0
gevent                        22.10.2
geventhttpclient              2.0.2
gitdb                         4.0.10
GitPython                     3.1.31
google-api-core               2.11.0
google-api-python-client      1.12.11
google-auth                   2.17.3
google-auth-httplib2          0.1.0
google-auth-oauthlib          0.4.6
google-cloud-core             2.3.2
google-cloud-storage          2.9.0
google-crc32c                 1.5.0
google-pasta                  0.2.0
google-resumable-media        2.5.0
google-search-results         2.4.2
googleapis-common-protos      1.59.0
graphql-core                  3.2.3
great-expectations            0.15.47
greenlet                      2.0.2
grpcio                        1.54.0
gunicorn                      20.1.0
h11                           0.14.0
h5py                          3.8.0
hnswlib                       0.7.0
html2text                     2020.1.16
httpcore                      0.16.3
httplib2                      0.19.1
httptools                     0.5.0
httpx                         0.23.3
huggingface-hub               0.13.4
identify                      2.5.24
idna                          3.4
imagesize                     1.4.1
importlib-metadata            6.6.0
importlib-resources           5.12.0
iniconfig                     2.0.0
ipykernel                     6.23.0
ipython                       8.13.2
ipython-genutils              0.2.0
ipywidgets                    7.7.5
isodate                       0.6.1
isoduration                   20.11.0
isort                         5.10.1
itsdangerous                  2.1.2
jaraco.classes                3.2.3
jax                           0.4.11
jedi                          0.18.2
Jinja2                        3.1.2
jinja2-ansible-filters        1.3.2
jinja2-time                   0.2.0
jmespath                      0.10.0
joblib                        1.2.0
json5                         0.9.14
jsonpatch                     1.32
jsonpointer                   2.3
jsonref                       1.1.0
jsonschema                    4.17.3
jupyter                       1.0.0
jupyter_client                8.2.0
jupyter-console               6.6.3
jupyter_core                  5.3.0
jupyter-events                0.6.3
jupyter-server                1.24.0
jupyter_server_terminals      0.4.4
jupyterlab                    3.5.3
jupyterlab-pygments           0.2.2
jupyterlab_server             2.15.2
jupyterlab-widgets            1.1.4
kafka-python                  2.0.2
kedro                         0.18.10
kedro-datasets                1.2.0
kedro-neptune                 0.2.0
kedro-telemetry               0.2.4
kedro-viz                     6.3.1
keras                         2.12.0
keyring                       23.13.1
kfp                           1.8.16
kfp-pipeline-spec             0.1.16
kfp-server-api                1.8.5
kiwisolver                    1.4.4
kubernetes                    18.20.0
langchain                     0.0.191
lazy-object-proxy             1.9.0
libclang                      16.0.0
libcst                        1.0.1
libmambapy                    1.1.0
littleutils                   0.2.2
llama-cpp-python              0.1.23
llvmlite                      0.40.0
lxml                          4.9.2
lz4                           4.3.2
makefun                       1.15.1
Mako                          1.2.4
mamba                         1.1.0
Markdown                      3.4.3
markdown-it-py                2.2.0
MarkupSafe                    2.1.3
marshmallow                   3.19.0
marshmallow-enum              1.5.1
matplotlib                    3.5.1
matplotlib-inline             0.1.6
mccabe                        0.6.1
mdit-py-plugins               0.4.0
mdurl                         0.1.2
mistune                       2.0.5
ml-dtypes                     0.2.0
mlflow                        2.2.2
mlserver                      1.3.2
mlserver-mlflow               1.3.2
mlxtend                       0.22.0
monotonic                     1.6
more-itertools                9.1.0
mpmath                        1.3.0
msg-parser                    1.2.0
msgpack                       1.0.5
multidict                     6.0.4
mypy                          0.991
mypy-extensions               1.0.0
nbclassic                     1.0.0
nbclient                      0.7.4
nbconvert                     7.4.0
nbformat                      5.8.0
nbpreview                     0.9.1
nbstripout                    0.6.1
neptune-client                1.3.1
neptune-tensorflow-keras      0.9.9
nest-asyncio                  1.5.6
networkx                      3.1
nltk                          3.8.1
no-implicit-optional          1.3
nodeenv                       1.8.0
notebook                      6.5.4
notebook_shim                 0.2.3
numba                         0.57.0
numexpr                       2.8.4
numpy                         1.24.3
oauthlib                      3.2.2
olefile                       0.46
omegaconf                     2.3.0
openai                        0.27.7
openapi-schema-pydantic       1.2.4
openpyxl                      3.1.2
opt-einsum                    3.3.0
orjson                        3.8.12
packaging                     23.1
pandas                        1.5.3
pandocfilters                 1.5.0
parquet                       1.3.1
parso                         0.8.3
passlib                       1.7.4
pathspec                      0.11.0
pathtools                     0.1.2
patsy                         0.5.3
pexpect                       4.8.0
picharsso                     2.0.1
pickleshare                   0.7.5
Pillow                        9.5.0
pip                           23.1.2
pip-tools                     6.13.0
pkginfo                       1.9.6
platformdirs                  3.5.1
plotly                        5.14.1
pluggy                        1.0.0
plumbum                       1.8.2
ply                           3.11
posthog                       3.0.0
pre-commit                    3.1.1
prometheus-client             0.16.0
prompt-toolkit                3.0.38
protobuf                      3.20.3
psutil                        5.9.5
psycopg2-binary               2.9.6
ptpython                      3.0.23
ptyprocess                    0.7.0
pure-eval                     0.2.2
py-d2                         0.0.2
py-grpc-prometheus            0.7.0
pyarrow                       11.0.0
pyasn1                        0.5.0
pyasn1-modules                0.3.0
pycodestyle                   2.7.0
pycosat                       0.6.4
pycparser                     2.21
pydantic                      1.10.8
pydeck                        0.8.1b0
pyflakes                      2.3.1
PyGithub                      1.58.2
Pygments                      2.15.1
PyJWT                         2.7.0
pylance                       0.3.12
pylatexenc                    2.10
pylint                        2.17.3
pylint-per-file-ignores       1.2.1
pylint-plugin-utils           0.8.2
pylint-pydantic               0.1.8
pymongo                       4.3.3
Pympler                       1.0.1
PyMySQL                       1.0.3
PyNaCl                        1.5.0
pynvim                        0.4.3
pyOpenSSL                     23.0.0
pypandoc                      1.11
pyparsing                     2.4.7
pypdf                         3.8.0
pyproject-api                 1.5.2
pyproject_hooks               1.0.0
pyrsistent                    0.19.3
PySocks                       1.7.1
pysrt                         1.1.2
pytest                        7.2.1
pytest-cov                    3.0.0
pytest-mock                   1.13.0
python-dateutil               2.8.2
python-docx                   0.8.11
python-dotenv                 1.0.0
python-jose                   3.3.0
python-json-logger            2.0.7
python-magic                  0.4.27
python-multipart              0.0.6
python-pptx                   0.6.21
python-rapidjson              1.10
python-slugify                8.0.1
python-terraform              0.10.1
pytoolconfig                  1.2.5
pytz                          2022.7.1
pytz-deprecation-shim         0.1.0.post0
PyYAML                        5.4.1
pyyaml-include                1.3
pyzmq                         25.0.2
qtconsole                     5.4.3
QtPy                          2.3.1
querystring-parser            1.2.4
questionary                   1.10.0
readme-renderer               37.3
regex                         2023.3.23
requests                      2.28.2
requests-oauthlib             1.3.1
requests-toolbelt             0.10.1
rfc3339-validator             0.1.4
rfc3986                       1.5.0
rfc3986-validator             0.1.1
rfc3987                       1.3.8
rich                          12.6.0
rope                          1.8.0
rsa                           4.9
ruamel.yaml                   0.17.17
ruamel.yaml.clib              0.2.7
ruff                          0.0.270
s3fs                          2022.11.0
s3transfer                    0.6.0
scikit-learn                  1.2.1
scipy                         1.9.3
seaborn                       0.12.2
secure                        0.3.0
semver                        2.13.0
Send2Trash                    1.8.2
sentence-transformers         2.2.2
sentencepiece                 0.1.98
sentry-sdk                    1.22.2
setproctitle                  1.3.2
setuptools                    65.6.3
shap                          0.41.0
simplejson                    3.19.1
six                           1.16.0
slicer                        0.0.7
smmap                         5.0.0
sniffio                       1.3.0
snowballstemmer               2.2.0
sorcery                       0.2.2
soupsieve                     2.4.1
Sphinx                        7.0.1
sphinxcontrib-applehelp       1.0.4
sphinxcontrib-devhelp         1.0.2
sphinxcontrib-htmlhelp        2.0.1
sphinxcontrib-jsmath          1.0.1
sphinxcontrib-qthelp          1.0.3
sphinxcontrib-serializinghtml 1.1.5
SQLAlchemy                    1.4.41
SQLAlchemy-Utils              0.38.3
sqlalchemy2-stubs             0.0.2a34
sqlmodel                      0.0.8
sqlparse                      0.4.4
stack-data                    0.6.2
starlette                     0.17.1
starlette-exporter            0.16.0
statsmodels                   0.14.0
strawberry-graphql            0.177.1
streamlit                     1.23.1
strip-hints                   0.1.10
sty                           1.0.4
swagger-spec-validator        3.0.3
sympy                         1.11.1
tabulate                      0.9.0
tenacity                      8.2.2
tensorboard                   2.8.0
tensorboard-data-server       0.6.1
tensorboard-plugin-wit        1.8.1
tensorflow-estimator          2.12.0
tensorflow-macos              2.12.0
term-image                    0.7.0
termcolor                     2.3.0
terminado                     0.17.1
text-unidecode                1.3
threadpoolctl                 3.1.0
thriftpy2                     0.4.16
tinycss2                      1.2.1
tokenizers                    0.13.3
toml                          0.10.2
toml-cli                      0.4.2
tomli                         2.0.1
tomlkit                       0.11.6
toolz                         0.12.0
toposort                      1.10
torch                         1.13.1
torchvision                   0.15.1
tornado                       6.3.2
tox                           4.3.5
tqdm                          4.65.0
traitlets                     5.9.0
transformers                  4.28.1
tritonclient                  2.33.0
twine                         4.0.2
typer                         0.5.0
types-click                   7.1.8
types-Pillow                  9.5.0.4
types-PyYAML                  6.0.12.9
types-regex                   2023.6.3.0
types-requests                2.31.0.1
types-setuptools              67.8.0.0
types-simplejson              3.19.0.1
types-tabulate                0.9.0.2
types-toml                    0.10.8.6
types-tqdm                    4.65.0.1
types-urllib3                 1.26.25.13
typing_extensions             4.6.3
typing-inspect                0.9.0
tzdata                        2023.3
tzlocal                       4.3
unstructured                  0.5.13
uri-template                  1.2.0
uritemplate                   3.0.1
urllib3                       1.26.16
uvicorn                       0.17.6
uvloop                        0.17.0
validators                    0.18.2
virtualenv                    20.23.0
wandb                         0.15.2
watchfiles                    0.19.0
watchgod                      0.8.2
wcwidth                       0.2.6
webcolors                     1.13
webencodings                  0.5.1
websocket-client              1.5.1
websockets                    11.0.2
Werkzeug                      2.2.3
wheel                         0.38.4
widgetsnbextension            3.6.4
wrapt                         1.14.1
XlsxWriter                    3.1.0
yarl                          1.9.2
zenml                         0.40.3
zipp                          3.15.0
zope.event                    4.6
zope.interface                6.0
zstandard                     0.19.0

The operating system you're using:
OSX.
The output of python --version:.
python --version.
Additional context
Add any other context about the problem here.

Hello @MarkusSagen 👋
Thank you for #71 . I've forwarded it to the developers to review and add to the next release. 🚀
Regarding the NeptuneDeprecationWarning you mentioned, it appears that you are currently using neptune-client instead of neptune.
To resolve this, I recommend uninstalling neptune-client and installing neptune instead. You can do this by running the following commands in your terminal:
pip uninstall neptune-client -y
pip install neptune

By following these steps, you should be able to eliminate the NeptuneDeprecationWarning you're encountering.
For more detailed instructions on upgrading from neptune-client to neptune, you can refer to our guide: Upgrade Guide.
Please let me know if this solution helps resolve your issue. Feel free to ask if you have any other questions or need further assistance. I'm here to help!

authors: MarkusSagen SiddhantSadangi

repo: neptune-ai/kedro-neptune


text:
automated mapping of higher level elements

There is a code repo by that automatically takes the UN/TDID specification text and writes it as xml (results here).
It would be cool to include these automated mappings into pydifact.
Thanks to @sabas for this idea.

Feel free to reach out also by email if you need advice!

authors: nerdoc sabas

repo: nerdocs/pydifact


text:
determine branching and releasing strategy

Currently rez follows a simplistic branch/release process:

Changes are merged into master via PR;
Releases are performed from master;
Releases follow semantic versioning.

Pros:

It's easy to track down regressions, since one release typically pulls in a small number of updates (and often a single PR).
It's simple

Cons:

It doesn't guarantee stable releases. Many releases have known issues that later releases have fixed.
Does not convey a sense of which release should be used to the community

Questions:

What strategy would be better, and why?
What does the ASWF recommend/prefer, or do they have anything to say about this? Should they?
Is there existing tooling we can use to support our chosen strategy? (Specifically, to automate as much as possible, and to guarantee well structured changelog and corresponding github release notes).

Crossposting from Slack:
We do similar internally with a few additions:

We tag branches (bug/name , feat/name, chore/name )
We have a develop branch
We merge PRs into develop
Once we want to release we merge to main and tag
Develop SHOULD always be kept usable, main is guaranteed (well there is no guarantee of course, but that's the goal)

This has some advantages:

It's more reliable for releases to be stable as you mentioned (though that of course also depends on the level of actual testing
It's helpful if you need to combine multiple PRs for something to work (or if you would break something if you don't combine - PRs)
It makes a clear distinction between "Here we work" and "Here we release"
It gives an easy way to treat PRs differently in CI if that is something we want

If this is worth the overhead i don't know. Depends also on how many people contribute and how many people maintain I guess. Also +1 for JC's comment in regards to "micro" releases.

I am a huge fan of semantic commit / branch naming, and allowing that semantic naming to trickle downward as helpful information.
i.e, if any of the merges since the last release contain a "feat/" branchname or commit name,  automatically ensure that the next release is also a feature-semver-bump, and/or ensure proper formatting of something makes it into the changelog.
At the very least, a specific list of chosen tokens like fix, feature, docs, style, refactor, test, etc, would be great.
I also hate clicking on the "branches" dropdown on the rez repo because it has many different styles of branch naming, and we don't delete branches after being merged. I think we're due for a spring cleaning.

I also hate clicking on the "branches" dropdown on the rez repo because it has many different styles of branch naming, and we don't delete branches after being merged.

That can be fixed by enabling auto-deletion of branches on merge: https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/configuring-pull-request-merges/managing-the-automatic-deletion-of-branches.

I'm in favor of the git-flow-named branches, and I think @JeanChristopheMorinPerso has a point about releasing less-frequently-with-more-changes.
This also gives us an opportunity to change the primary origin branch to main instead of master to have some more inclusive language.

Releasing less often and having releases with more PRs/changes would help to convey more information and help know, in some way, which version they should use. Micro releases can make sense inside a studio, but in open source it makes less sense because I'm sure most studios don't update rez as soon as there is a new release available.

Can we talk about this more please? Having smaller releases doesn't prevent anyone from say, updating Rez to the latest when they're ready to do so. However if there's less releases, when they're ready to make a move, there'll be many more merge conflicts to deal with versus smaller releases. That's bitten us in the past. "We can't update to the latest Rez yet because there's a bunch of merge conflicts we need to QA and test".

Speaking from experience, updating rez is "painful" because it's difficult to keep track of what changed because there is too many one PR releases.

I would rather diff between 5, 200 line release tags than diff between 2, 1000 line release tags with features (unrelated features or not) jumbled all together. I would much rather update from an older Rez to a later, non-latest (of those 5) Rez releases than delay updating because there's to many merge conflicts between my fork and the latest branch to be tackled all at once.
Having smaller releases gives us more options while upgrading while not preventing anyone from working the way they choose. But less releases gives people less options.

Is there any benefit to considering separate stable-vs-unstable release streams?
(Personally, I'm in the prefer-small-releases-camp, but I'm suggesting ideas that accommodate worries from the larger-releases-camp)

There are different aspects and as always advantages and disadvantages. While it might be easier to do a git diff evaluation it is a whole lot more hassle to just read release notes. One of the examples you gave, going from 2.100.0 to latest involves 3 pages of releases and release notes where most of the space is redundant information (like the download links etc.). I recently did that and usually i read release notes rather than code. I usually also just use a release and deploy that and there are no merge conflicts. Do you have that because you also maintain internal changes that you need to combine with official releases? And if so is that because you lack fixes or features? (I think it would be desirable to try and enable everyone to just use of the shelf releases. For one it is less hassle for the affected people and it also helps with support as supporting code we don't know is tricky at times).
My take is, that the difference is typically the same code wise but i find it more hassle to just read through changes and understand implications. In practice i never released every version - as mentioned before there might be multiple a day and i typically release every few weeks rather than multiple times a day - so i do a big upgrade anyways. It's just harder to tell what exactly is happening unless i want to look at code for individual features. You still get to check individual commits that are also associated with the issues (in the future hopefully ;) )
Another aspect is that it can be hard to tell the current state as in "will there be more short term releases that i should wait for? Or ss the current release the end of the current set of releases?"

One of the examples you gave, going from 2.100.0 to latest involves 3 pages of releases

The release notes is fine for getting an overview but without reading and auditing a change. But I would not advocate for any new Rez upgrade to an artist floor without reading the source code changes to know exactly what it's doing. The alternative that you're suggesting, what I understand from your reply, is you'd prefer to collapse all of these into a single release or just less releases. It'd still be basically the same amount of reading, just now it's in one diff that we cannot separate anymore since it's one release. Seems like a demerit to me, overall.
Unrelated side note, for viewing diffs more easily, I use exclude wildcards help for the files that I know aren't as important (e.g. benchmark files).

Do you have that because you also maintain internal changes that you need to combine with official releases? And if so is that because you lack fixes or features?

Yes and yes. We / I wanted those changes in the main Rez but sometimes issues / PRs may take longer than production is willing to wait. It ends up going in the fork. And sometimes we need a feature that isn't well received in the open source community (which is fine, of course). In those cases forking is the only option.

You still get to check individual commits that are also associated with the issues (in the future hopefully ;) )

If Rez adopts a "all PRs must be rebased and squashed prior to merge" policy, which I think you can actually set directly in the GitHub repo settings, commits can still be WIP, formatting / newline changes, pylint, "fixed unittest", etc. But if we have that guarantee that each commit is clean and contains exactly one bug fix / one feature (one PR's content), then I can see that being viable.
Btw I'm not necessarily advocating for rebase + squash as a requirement. I'm just saying, in the absence of that, diffing WIP commits in a larger release is more frustrating than diffing release tags that you know are atomic.

Our setup makes it easy to mix and match and roll back and forth (we tried to adopt a similar strategy for our Rez deployment than Rez does: Provide all versions supported and make it easy to choose and switch). So i am not auditing every code change. I treat Rez similar to other software and libraries:

I check the changelog for relevant changes and possible deal breakers
If i find none i deploy to staging and have testers test in a production like environment
If testers approve we deploy to production

I would like to add another comment to clarify in regards to the "amount of reading". Here is a shot from the top of the releases page. This is on 1440p:

I get 2 releases per screen. And the relevant information is this:

Implement 1196 1199 (instinct-vfx)
Re-enable py3 workflow in windows.yaml 1232 (instinct-vfx)
fix: address zsh install message displaying wrong completion script 1235 (maxnbk)
Explicit fail when python executable is not found 1236 (aboellinger)

I get around 4-5 pages per screen and 3 pages to reach 2.100.0. That's really hard to just read and get a quick overview of changes.
Just to make that clear: I am not a dire advocate of this, and i can live with what we have now too, i am just pointing out why i would slightly prefer somewhat bigger releases. In the past releases were usually in groups when Allan was spending time on Rez. That typically results in 2-3 releases on that day. So i am not saying only release once a month and make releases huge, but i would see benefit in grouping these 2-3 releases that are released the same day anyways if that makes sense.

(As a side note, documenting release notes in a dedicated page in the docs or alike might also help mitigate this i guess)

Sorry for the wall of text... But I felt there was a lot of ground to cover.

Colin: It will be worse if they're all between 2 tagged releases. With separate tags, 2.100.0, 2.101.0, 2.102.0, etc, you can do git diff 2.100.0...2.105.0 to view the full thing or git diff 2.102.0...2.103.0 to view each feature's change incrementally. If there's only one, uber tag containing all 5 releases code, then you're stuck with just git diff 2.100.0...2.101.0. And the diff will likely contain multiple unrelated feature's code all at once that you'd have to guess / reason through.

That's what commits are for. If there is a release that contains multiple changes, normally the release notes will contain links to PRs and it's also quite easy to inspect the commits between two tags.
As for comments about forks that are hard to maintain because of conflicts, I don't think there is much we can do. Anyone forking a repo and doing changes to that fork should know that keeping the fork up to date will require time, efforts and merge conflict resolution skills. I don't think micro releases help in any way with having less conflicts.

Colin: Having smaller releases doesn't prevent anyone from say, updating Rez to the latest when they're ready to do so.

Indeed, and my suggestion for having less frequent and bigger releases isn't about that. It was slightly related since I talked about release notes, but @instinct-vfx already covered the pain points around these and I don't have more things to add on that than what he said.

Colin: The alternative that you're suggesting, what I understand from your reply, is you'd prefer to collapse all of these into a single release or just less releases. It'd still be basically the same amount of reading, just now it's in one diff that we cannot separate anymore since it's one release.

(saying "just now it's in one diff that we cannot separate anymore since it's one release" is incorrect because there would still be at least one commit per merged PR)
If we look at it that way, yes there isn't "any" big benefit. At the end of the day, if there is for example 10 PRs, no matter if they are split between 1 or 10 releases, it's the same amount of code. But I don't think it should be seen with that perspective. In the issue description, @nerdvegas asks questions and lists pros and cons. My suggestion of moving away from micro releases is to try answering how we could solve the con "Does not convey a sense of which release should be used to the community".
Let me try to explain things differently in hope that it will highlight more clearly my view of how things can be improved. I'll try to explain how we could better help our community to choose which version to pick and also how to give a better sense of stability (or in other words, how we can increase the trust our users have when it comes to updating Rez).
Right now, Rez releases feel like rolling/nightly releases because they contain really small set of changes and can happen multiple times a day. The condition for a release to happen is to have at least one PR merged (no matter what kind of change it is. There's been releases just for extremely small changes in the past that wasn't affecting the behaviour of Rez, for example a change in a test only). Hence the reason why I'm making a parallel with rolling/nightly releases.
Slowing the release cycle and having bigger releases has multiple benefits, some of which are:

Reduction of the noise generated by multiple micro releases in a short amount of time. Note the use of the word noise which I carefully chose to help demonstrate my point. This leads to the next point.
Reducing the release cadence would help improve the impression of stability and would potentially give more confidence in the project to some users. That's a quite personal opinion, but I tend to "trust" a little bit less projects which release too often because I can't keep up with the release cadence and I endup missing some crucial information. Which leads to the next point.
We would be able to better communicate the changes since there is more possibilities that they would be grouped. So instead of saying A was released in version X, B in version Y and C in version Z, we could say A, B and C where released in X. This applies to both users and contributors/maintainers. I'm pretty sure that almost nobody can say in which versions a specific thing was introduced or fixed without looking at the release notes because of the sheer amount of releases that makes release versions almost meaning less.
Give meaning to versions. Right now the only meaning that the versions convey is semantic information.
We would be able to have release announcements, which would tremendously help our community.
Remove some work on the maintainer shoulders (maintainers with an "s" once TSC is formed)

And with more time I could probably find more benefits and I could also explain more in-depth some of them.

maxnbk: Is there any benefit to considering separate stable-vs-unstable release streams?

We could have dev/unstable releases, but the  how do we determine that something if a version needs to be a dev or an official release? Also I don't think anyone would install dev releases (because the sole purpose of tagging a release as dev is to tell  everyone to not use it in production) which means we would have the same result as having bigger releases.

Again, sorry for the wall of text.

@JeanChristopheMorinPerso
I read through the whole post however I'm mostly going to reply the last 60%, because I think there's important points you've made and I want to clarify those.

(or in other words, how we can increase the trust our users have when it comes to updating Rez)...Reducing the release cadence would help improve the impression of stability

In general, when I talk to artists and stakeholders about assets or code features, they trust my word when I say something is fixed, works, "is ready", etc.
The reason I read through Rez's diff is not because I don't trust its changes but because Rez permeates through the entire pipeline, touching every user, and even runs on the farm with elevated permissions. It's unique compared to other Open Source projects in that way. Having larger releases will not make this responsibility go away and will make maintaining that level of care for our users harder during upgrades.

I'm pretty sure that almost nobody can say in which versions a specific thing was introduced or fixed without looking at the release notes

I agree and I'd like to expand that by saying if we used git tags Rez more effectively, we could still achieve the same clarity without requiring to group releases. git tag -n is pretty great and I wish Rez's tags summarized each change better. They're mostly "changelog update" etc.

Right now, Rez releases feel like rolling/nightly releases ... Slowing the release cycle and having bigger releases has multiple benefits

I realize that what I'm about to say may come off poorly and I apologize in advance. Rez has been propped up by Allan and as a community effort. For many, entirely in their own free time. However I cannot help but point this out. Does slowing Rez's release cycle make sense for Rez? Is everyone happy enough with Rez's current set of features that we're okay with waiting longer for releases?
There's currently 34 open PRs, and 63 open enhancement issues. Many dating back several years. If we can do something to help Rez implement & merge those faster, I would like to seek out what that could be. The thought of slowing down, especially with the recent momentum following ASWF adoption, IMO would be a wasted opportunity.
Our current branch structure suits Rez well, I think. We just need to be more effective with the processes that are already in place.

I realize that what I'm about to say may come off poorly and I apologize in advance. Rez has been propped up by Allan and as a community effort. For many, entirely in their own free time. However I cannot help but point this out. Does slowing Rez's release cycle make sense for Rez? Is everyone happy enough with Rez's current set of features that we're okay with waiting longer for releases?

Just a quick comment here: I don't think slowing down the release cadence has a direct relation to the speed of development. My take would be that the development of features is stable and all we are deciding is how often we package a release. Depending on the overhead the release process has it might even be less effort to release less often.

Without being overly verbose, I prefer @ColinKennedy 's take, but I do sympathize that it is currently difficult to figure out what happened in what release.
That said, my use-case-scenario is usually much more of a "I just want to know if this particular feature or bugfix is in this particular version or not", and I have much less of a problem finding out that information with basic git skills.
If I was going to suggest any change to the rez release schedule, I would propose something like this: merging/releasing only bugfix PRs as one large batch, once every-approximate-fortnight, and only merging/releasing only features as one large batch, every-other-fortnight (if any features are ready to be merged). It wouldn't really slow down the development/merge/release cycle, but it would organize the fact that "after we release features, we release bug fixes to those features", kind-of, and would have the tendency to group fixes with other fixes, features with other features, making for slightly easier release/changelog review, while not overly impacting how long it takes for a PR to reach release status.

As @instinct-vfx said, slowing down the release cadence doesn't mean slowing down on development. And I will point out that if someone wants a feature badly, then that person can communicate that to the future TBD TSC members.
And @maxnbk, I was also imagining something like you described (though I'm not sure if you imply that releases would either contain features or bugfixes, but not both... Anyway, that's details.).

Righto, my thoughts in quasi random order:
To @maxnbk 's idea of releasing every other fortnight (btw "every other" is an Americanism that left me confused for longer than I care to admit!). I don't mind it, but the problem with this is that currently, I'm the sole maintainer (in reality), and it's more of a case of get-it-done-when-I-can. So until there are maintainers plural, a more scheduled release approach will not work. I'd suggest we leave discussion around scheduled releases until if/when that becomes practically possible.
I generally agree with points raised in this thread. I like small releases personally, because it makes tracking down what happened in what version very straightforward. However, I also agree that larger and less frequent releases have their advantages. Specifically I do think larger releases help in terms of general communication and impression of stability. Also, what hasn't been mentioned yet is that it should result in more studios running a common version of rez at any given time.
Here's what I don't like about larger releases though:

if a studio wants a fix sooner than the next release, they'll need to either pull from master or pull a specific commit. Now we're in a situation where it's much harder to describe "where" a studio was when their version of rez broke. Rather than saying, "this feature stopped working in 2.105.4", we say, "it stopped working in the master I pulled a few weeks ago", or, "it stopped working at this commit hash."
If a studio uses a version of rez based on a commit, we need to be careful about commits and ensure that a commit isn't actually a partial implementation of a fix. Ideally that should never be the case, but there's no assurance, and this is just something else to worry about. Currently we don't have this worry because people pull versions, not commits.

I wonder then, what we might be able to do in order to have the best of both worlds. What if we had two kinds of releases? We could follow the current release method, but let's call these versions (eg) 2.104.5.beta. Then, in the TSC meetings, and at regular intervals (once a month perhaps), we agree on which version most people are/should be using. We then do a "production" release for this version (eg "2.105.1"). This release doesn't add anything new, it just tags the repo at the appropriate commit (which would be the latest beta < the production version), and generates a set of release notes that collates all the changes since the last production release.
So it'd look something like this:
# in commit order
* (<- master)
* 2.108.2.beta
* 2.108.1.beta
* 2.108.0.beta
* 2.107.1  <- production release
* 2.107.0.beta
* 2.106.1.beta
* 2.106.0.beta
* 2.105.1  <- production release
* 2.105.0.beta
* ...

Here the 2.107.1 release would:

have release notes that collate all changes since 2.105.1;
have "source diff" link that shows diff wrt 2.105.1

Of course, the releases would not appear in github in this order (releases are shown in order of creation date), they might appear like so:
# in date order
* 2.107.1
* 2.108.2.beta
* 2.108.1.beta
* 2.108.0.beta
* 2.105.1
* 2.107.0.beta
* 2.106.1.beta
* 2.106.0.beta
* 2.105.0.beta

I'm ok with this though, it conveys that betas are the latest releases, and non-beta are the "most recent good and stable" version.
The only thing that's a bit odd is how the production releases are all a .1 patch - that's required because otherwise 2.105.0.beta > 2.105.0 (in rez versioning terms).
Thoughts?

Hm. While this would mostly adress this specific issue it would make things worse in regards to perceived stability and communication for everything except non-beta tagged releases. I am also rather strongly against using the specific term beta in this context. This should only be used if there is an actual difference. Typically i would expect non beta releases to be actually tested or pass certain criteria to be labled production. If the only difference is an arbitrary TSC decision that seems wrong to me.
I just had a different thought (as i am not generally opposing the idea of the approach just the way it would be implemented). Tags and Releases are a different concept in Github. How about having tags for all in between releases and releases for actual releases. A common naming scheme have often seen is having a build number. So it could be:
2.105.0 <- release
2.105.0.001  <- Tag (created upon merging a PR)
2.105.0.002  <- Tag (created upon merging a PR)
2.105.0.003  <- Tag (created upon merging a PR)
2.105.1 <- release
2.105.1.001  <- Tag (created upon merging a PR)
2.105.1.002  <- Tag (created upon merging a PR)
2.105.1.003  <- Tag (created upon merging a PR)
2.106.1 <- release
2.106.1.001  <- Tag (created upon merging a PR)
2.106.1.002  <- Tag (created upon merging a PR)
2.106.1.003  <- Tag (created upon merging a PR)
I have not fully thought this through, i just had that idea while reading your post.
This way you get tags for everything, but releases with changelog for every n-th tag as an actual release. We can reference a breakpoint by tag and still get only releases in the releases page.

I dunno, how are you going to get the production testing done on 2.105.0 to
be able to make it a proper release in the first place? I suspect people
won't install tags, they'll just install the previous release, and the tags
will just become a notional concept that don't really have any practical
use. Why would you install a tag when you don't really know what it is?
I don't think agreeing on a prod release in the TSC meeting would be
arbitrary. It would be based on feedback over the last month about what
versions studios are using and how stable they're reporting them to be.
Now, there's a similar argument to what I just mentioned though - how many
studios are even going to be using these beta/whatever releases, would they
not just tend to use the last prod release? I would say to that that,
again, an actual release represents a lower barrier of entry to use than a
tag. An actual release has release notes and can simply be downloaded
directly from github. Those seem like minor things but I suspect they'll
matter in terms of what gets used.
The trick here, I think, is to have easy access to smaller releases, so
that we get enough real-world testing in to then be able to make larger,
extra stable releases at semi-defined intervals.
On Sat, Mar 19, 2022 at 9:18 AM Thorsten Kaufmann @.***>
wrote:

Hm. While this would mostly adress this specific issue it would make
things worse in regards to perceived stability and communication for
everything except non-beta tagged releases. I am also rather strongly
against using the specific term beta in this context. This should only be
used if there is an actual difference. Typically i would expect non beta
releases to be actually tested or pass certain criteria to be labled
production. If the only difference is an arbitrary TSC decision that seems
wrong to me.
I just had a different thought (as i am not generally opposing the idea of
the approach just the way it would be implemented). Tags and Releases are a
different concept in Github. How about having tags for all in between
releases and releases for actual releases. A common naming scheme have
often seen is having a build number. So it could be:
2.105.0 <- release
2.105.0.001 <- Tag (created upon merging a PR)
2.105.0.002 <- Tag (created upon merging a PR)
2.105.0.003 <- Tag (created upon merging a PR)
2.105.1 <- release
2.105.1.001 <- Tag (created upon merging a PR)
2.105.1.002 <- Tag (created upon merging a PR)
2.105.1.003 <- Tag (created upon merging a PR)
2.106.1 <- release
2.106.1.001 <- Tag (created upon merging a PR)
2.106.1.002 <- Tag (created upon merging a PR)
2.106.1.003 <- Tag (created upon merging a PR)
I have not fully thought this through, i just had that idea while reading
your post.
This way you get tags for everything, but releases with changelog for
every n-th tag as an actual release. We can reference a breakpoint by tag
and still get only releases in the releases page.
—
Reply to this email directly, view it on GitHub
https://github.com/nerdvegas/rez/issues/1247#issuecomment-1072860441,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAMOUSUFR6EGXUR5XRPIUZTVAT6KZANCNFSM5Q2MBEEA
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you were mentioned.Message ID:
@.***>

Hm. I am not sure actually. I can see how people might not install tags. Will they be more likely to install releases tagged beta though? And if they don't then we will not get a lot of production feedback and testing of beta releases would not really make non-beta releases a whole lot better, no?
I am really not fond of the term "beta" there and i also really don't like the idea of releases not being in order (though that might just be ocd(

I'm not sold on beta either tbh, I was just using that to illustrate the concept, to be clear.

Micro tagging or micro releasing are pretty much the same IMO and they have the same problems. And micro releasing + creating release bundles is probably worst then micro releases alone. Not only would it add extra work, it would also require extra mental overhead and load just to process what's going on.
I think that if a user wants a fix really badly, he will probably fork the repo to fix the issue, create a PR, etc. That means they can install the fix from their fork and then go back to the official releases once the fix is officially released. That's the beauty of open sourced code. Also, any user that doesn't use an official release shouldn't be "supported". We can't garantee that everything will work if they picked a specific commit instead of a known version. Or If someone has a critical bug, that person can always revert to the previous version that worked for them. Additionally, I tend to think that most users of open source projects don't go as far as using non release versions.

As for "If a studio uses a version of rez based on a commit, we need to be careful about commits and ensure that a commit isn't actually a partial implementation of a fix.", as my previous reply said, it should be the other way around. If someone wants to pick a specific commit to use on their prod, it's up to them to make sure what they chose works.

Using GitHub releases is an interesting idea. Are releases always associated with a tag this way you can automate downloads just using Git clone, tag checkout, and install.py? or is release data only available via the GitHub API and Web UI?

Another point that I don't think has been made for larger, more deliberate releases is that it would help create some uniformity between user installs. Right now, you could go to a dozen different sites and there is a good chance that a different version of rez is in production at each site. From the standpoint of developing and sharing an ecosystem of other tools or services around rez, and even for rez itself, it creates a much larger maintenance burden.
As far as a specific release cadence, it would ideally be closely aligned with the update cadence for users. How often can we reasonably expect users to install new versions of rez, test them, and put them into production? They likely aren't going to do that every week, right? Monthly is probably about the highest frequency you could reasonably expect I imagine. The cadence would just be for planned minor releases. It would still allow you to push out patch releases for critical bugs or regressions outside of the normal cadence.
As far as automating changelog and release notes, that would likely be easier if rez adopted some form of Conventional Commits, so that commits could be reliably parsed in a structured way.

authors: ColinKennedy JeanChristopheMorinPerso bpabel instinct-vfx jasoncscott maxnbk nerdvegas sirfuzzalot

repo: nerdvegas/rez

license_source: bigquery