text:
When updating cluster_dns, k3s-agent (on worker nodes) needs to be restarted

When I was investigating #351, I noticed that after changing cluster_dns and re-running hetzner-k3s create, the cluster DNS address was updated on the control plane nodes, but not on the worker nodes.
On the control plane nodes, the cluster DNS address is provided through a k3s flag (--cluster-dns=...) in /etc/systemd/system/k3s.service, and it looks like when we re-run hetzner-k3s create, that unit file gets correctly updated, reloaded, and restarted. Yay!
However, on the worker nodes, the cluster DNS address is not specified anywhere. I suppose that k3s-agent obtains that parameters from the control plane somehow.
When we update cluster_dns and re-run hetzner-k3s create, it does not restart k3s-agent on the worker nodes, and as a result, the worker nodes still use the old DNS address when creating new pods.
If I systemctl restart k3s-agent.service manually on the worker nodes, they seem to pick up the new DNS address, and new pods get the new DNS address.
I don't know if something should be changed in the code, or if it should simply be documented; but I wanted to report the issue so you could decide what was best for the project.
Thank you!

hetzner-k3s re-runs the installation scripts with updated settings on both masters and workers, so if some setting changed on master nodes is not propagated to workers (provided that like in this case the setting you change is only available for masters), then it's more of an issue for the k3s repository IMO. Can you open one there?

Ah, okay! I was assuming that hetzner-k3s was directly updating and restarting these systemd services; I didn't realize some k3s scripts were involved.
I'm going to dig a bit deeper to make sure I understand what scripts are called, so that I can open a relevant issue on the k3s repo.
Thanks!

Yep, I run the official scripts and they then determine if something has changed. If nothing has changed they don't restart k3s. So in this case it's likely something that needs to be addressed there.

Closing since it's more likely an issue with k3s itself :)

authors: jpetazzo vitobotta


text:
Add GitHub action to update DigitalOcean deployment

Add GitHub Action workflow to update DigitalOcean deployments on push to the main branch.
Update and create deployments for imgproxy, cms, and site-main apps using .digitalocean/deploy.sh script.

Why can't we use the DigitalOcean action for this? https://docs.digitalocean.com/products/app-platform/how-to/deploy-from-github-actions/

36bb5396a0dc7a2f8a7c0963742232e43bbe8691

source: gharchive/pull-request

repo: vivid-planet/comet-starter

license_type: permissive

license_source: github-api


text:
Add preload for scope functionality to PageTreeReadApi

This avoids many db-roundtrips when loading a menu structure with children.

the preload has to be started manually

the cache is only used for a single request

~the ReadApi object must not be kept across requests~ automatically done by using a request scoped service

Also added dataloader for loading attached document (and avoiding loading documents from non-requested type)

This improves performance for the main page request (that loads menu, breadcrumbs etc) of "one of our customers":
before:
┌─────────┬─────────┬─────────┬─────────┬─────────┬───────────┬────────────┬─────────┐
│ Stat    │ 2.5%    │ 50%     │ 97.5%   │ 99%     │ Avg       │ Stdev      │ Max     │
├─────────┼─────────┼─────────┼─────────┼─────────┼───────────┼────────────┼─────────┤
│ Latency │ 2103 ms │ 4272 ms │ 6460 ms │ 6460 ms │ 3954.9 ms │ 1530.51 ms │ 6460 ms │
└─────────┴─────────┴─────────┴─────────┴─────────┴───────────┴────────────┴─────────┘

after:
┌─────────┬────────┬────────┬─────────┬─────────┬──────────┬───────────┬─────────┐
│ Stat    │ 2.5%   │ 50%    │ 97.5%   │ 99%     │ Avg      │ Stdev     │ Max     │
├─────────┼────────┼────────┼─────────┼─────────┼──────────┼───────────┼─────────┤
│ Latency │ 398 ms │ 793 ms │ 1219 ms │ 1219 ms │ 740.5 ms │ 302.46 ms │ 1219 ms │
└─────────┴────────┴────────┴─────────┴─────────┴──────────┴───────────┴─────────┘

(tested with npx autocannon --header "Authorization: Basic xxxx" --header "Content-Type: application/json" --method POST --input payload https://api-xxx.niko.dev.vivid-planet.cloud:8443/graphql --amount 20 --connections 2

@nsams this PR has too many commits. Did you change target from main to next?

@johnnyomair rebased now correctly.
You should review the individual commits of this PR

Another question: Should preloadNodes support category as well?

difficult but might be possible. Needs a real use case that benefit from this.

Needs a real use case that benefit from this.

The use case I'm thinking of is loading the page tree in the admin, which always filters by category. But maybe it's not necessary there

authors: johnnyomair nsams

repo: vivid-planet/comet


text:
Implement new Patch skin

font
color

no functionality changes

Sketch file Patch UI 150915

ok so can't assign to multiple people. So just gonna mention @imgrby here too & you guys can sort it out :)

source: gharchive/issue

authors: kschzt mattatgit

repo: vizorvr/vizor-create

license: mit

license_source: bigquery


text:
Revolver: Inactive inputs aren't loaded

Trying to make a button that changes between 3 different 3D objects:
http://destroy.vizor.io/fthr/gaze-click-to-change-object/edit
So you have three Object3D's in a carousel. Only the first one shows up, others will show nothing.

Fixed by #1134

authors: fthr kkananen

repo: vizorvr/vizor

license: MIT


text:
Task10 matrix

Задача 10. Матричный алгоритм решения задачи достижимости с КС ограничениями

[x] Реализовать функцию, основанную на матричном алгоритме, решающую задачу достижимости между всеми парами вершин для заданного графа и заданной КС грамматики.

Результат работы функции --- множество троек вида (вершина, нетерминал, вершина).
Предусмотрите возможность получать граф из различных источников в различных форматах.
Предусмотрите возможность получать грамматику из различных источников (из файла, как результат работы других функций).
Для преобразования грамматики в ОНФХ использовать результаты предыдущих работ.
Для реализации матричных операций использовать sciPy.

[x] На основе предыдущей функции реализовать функцию, позволяющую решать задачу достижимости для заданного набора стартовых и финальных вершин, и заданного нетерминала.
[x] Добавить необходимые тесты.

Добавил небольшую правку в алгоритм, а также вынес в отдельную функции общие части cfpq.


text:
[Ｏｆｆｉｃｉａｌ！ＬｉＶＥ] OKTAGON 61 Ｌｉｖｅ Ｓｔｒｅａｍｓ Ｆｒｅｅ Ｃｏｖｅｒａｇｅ ＯＮ 21 September ２０２４

Oktagon MMA is in action on Saturday with Oktagon 61 taking place at the Winning Group Arena in Brno, Czech Republic. Oktagon 61 MMA LiveStream MMA Fights on TV
A wide range of action saw a former champ in Scott Askham competing at middleweight, facing former UFC fighter Makhmud Muradov, who had parted ways with his former promotion rather suddenly despite having re-signed with the company relatively recently. The Tipsport Gamechanger tournament also continued, with Losene Keita taking on Mateusz Legierski in semifinal action.
In the main event, meanwhile, Ronald Paradeiser of Slovakia face off with Spain’s Acoidan Duque. The winner would move on to the final of the European MMA Champions League.
OKTAGON 61 is set to take place on Saturday, September 21, 2024, featuring 12 fights at the Winning Group Arena in Brno, Czech Republic.
Event Overview:
Event: OKTAGON 61.
Date: September 21, 2024.
Venue: Winning Group Arena, Brno, Czech Republic.
Number of Fights: 12 bouts.
Highlights:
MMA Action: OKTAGON, one of Europe's leading MMA promotions, brings together top fighters from the region and beyond for a night of intense action across multiple weight classes.
Live Fight Updates
Ronald Paradeiser vs. Acoidan Duque
Makhmud Muradov vs. Scott Askham
Losene Keita vs. Mateusz Legierski
Vladimír Lengál vs. Corey Fry
Andrej Kalašnik vs. John Palaiologos
Jan Stanovský vs. Gleb Shadrin
Mate Sanikidze vs. Jack Cartwright
Attila Korkmaz vs. Agy Sardari
Jakub Tichota vs. George Staines
Tomáš Hron vs. Fred Sikking

SPORTS BOXing

        ✅▶ 𝗥𝗲𝗮𝗹𝗹𝘆 𝗔𝗺𝗮𝘇𝗶𝗻𝗴 𝗬𝗼𝘂 𝗖𝗮𝗻 𝗧𝗿𝘆 𝗧𝗵𝗶𝘀 ◀✅✅

**[[📺📱 🔴 ➤►LIVE👉 STREAM PLAY NOW ▶
📺📱 🔴 ➤►Go👉 LIVE ONLINE ▶

⬆
⬆
⬆

        ✅▶ 𝗥𝗲𝗮𝗹𝗹𝘆 𝗔𝗺𝗮𝘇𝗶𝗻𝗴 𝗬𝗼𝘂 𝗖𝗮𝗻 𝗧𝗿𝘆 𝗧𝗵𝗶𝘀 ◀✅✅

**[[📺📱 🔴 ➤►LIVE👉 STREAM PLAY NOW ▶
📺📱 🔴 ➤►Go👉 LIVE ONLINE ▶

⬆
⬆
⬆
⬆
⬆
⬆
⬆
⬆
⬆

SPORTS BOXing

        ✅▶ 𝗥𝗲𝗮𝗹𝗹𝘆 𝗔𝗺𝗮𝘇𝗶𝗻𝗴 𝗬𝗼𝘂 𝗖𝗮𝗻 𝗧𝗿𝘆 𝗧𝗵𝗶𝘀 ◀✅✅

**[[📺📱 🔴 ➤►LIVE👉 STREAM PLAY NOW ▶
📺📱 🔴 ➤►Go👉 LIVE ONLINE ▶

⬆
⬆
⬆

        ✅▶ 𝗥𝗲𝗮𝗹𝗹𝘆 𝗔𝗺𝗮𝘇𝗶𝗻𝗴 𝗬𝗼𝘂 𝗖𝗮𝗻 𝗧𝗿𝘆 𝗧𝗵𝗶𝘀 ◀✅✅

**[[📺📱 🔴 ➤►LIVE👉 STREAM PLAY NOW ▶
📺📱 🔴 ➤►Go👉 LIVE ONLINE ▶

⬆
⬆
⬆
⬆
⬆
⬆
⬆
⬆
⬆

SPORTS BOXing

        ✅▶ 𝗥𝗲𝗮𝗹𝗹𝘆 𝗔𝗺𝗮𝘇𝗶𝗻𝗴 𝗬𝗼𝘂 𝗖𝗮𝗻 𝗧𝗿𝘆 𝗧𝗵𝗶𝘀 ◀✅✅

**[[📺📱 🔴 ➤►LIVE👉 STREAM PLAY NOW ▶
📺📱 🔴 ➤►Go👉 LIVE ONLINE ▶

⬆
⬆
⬆

        ✅▶ 𝗥𝗲𝗮𝗹𝗹𝘆 𝗔𝗺𝗮𝘇𝗶𝗻𝗴 𝗬𝗼𝘂 𝗖𝗮𝗻 𝗧𝗿𝘆 𝗧𝗵𝗶𝘀 ◀✅✅

**[[📺📱 🔴 ➤►LIVE👉 STREAM PLAY NOW ▶
📺📱 🔴 ➤►Go👉 LIVE ONLINE ▶

⬆
⬆
⬆
⬆
⬆
⬆
⬆
⬆
⬆

SPORTS BOXing

        ✅▶ 𝗥𝗲𝗮𝗹𝗹𝘆 𝗔𝗺𝗮𝘇𝗶𝗻𝗴 𝗬𝗼𝘂 𝗖𝗮𝗻 𝗧𝗿𝘆 𝗧𝗵𝗶𝘀 ◀✅✅

**[[📺📱 🔴 ➤►LIVE👉 STREAM PLAY NOW ▶
📺📱 🔴 ➤►Go👉 LIVE ONLINE ▶

⬆
⬆
⬆

        ✅▶ 𝗥𝗲𝗮𝗹𝗹𝘆 𝗔𝗺𝗮𝘇𝗶𝗻𝗴 𝗬𝗼𝘂 𝗖𝗮𝗻 𝗧𝗿𝘆 𝗧𝗵𝗶𝘀 ◀✅✅

**[[📺📱 🔴 ➤►LIVE👉 STREAM PLAY NOW ▶
📺📱 🔴 ➤►Go👉 LIVE ONLINE ▶

⬆
⬆
⬆
⬆
⬆
⬆
⬆
⬆
⬆

SPORTS BOXing

        ✅▶ 𝗥𝗲𝗮𝗹𝗹𝘆 𝗔𝗺𝗮𝘇𝗶𝗻𝗴 𝗬𝗼𝘂 𝗖𝗮𝗻 𝗧𝗿𝘆 𝗧𝗵𝗶𝘀 ◀✅✅

**[[📺📱 🔴 ➤►LIVE👉 STREAM PLAY NOW ▶
📺📱 🔴 ➤►Go👉 LIVE ONLINE ▶

⬆
⬆
⬆

        ✅▶ 𝗥𝗲𝗮𝗹𝗹𝘆 𝗔𝗺𝗮𝘇𝗶𝗻𝗴 𝗬𝗼𝘂 𝗖𝗮𝗻 𝗧𝗿𝘆 𝗧𝗵𝗶𝘀 ◀✅✅

**[[📺📱 🔴 ➤►LIVE👉 STREAM PLAY NOW ▶
📺📱 🔴 ➤►Go👉 LIVE ONLINE ▶

⬆
⬆
⬆
⬆
⬆
⬆
⬆
⬆
⬆

SPORTS BOXing

        ✅▶ 𝗥𝗲𝗮𝗹𝗹𝘆 𝗔𝗺𝗮𝘇𝗶𝗻𝗴 𝗬𝗼𝘂 𝗖𝗮𝗻 𝗧𝗿𝘆 𝗧𝗵𝗶𝘀 ◀✅✅

**[[📺📱 🔴 ➤►LIVE👉 STREAM PLAY NOW ▶
📺📱 🔴 ➤►Go👉 LIVE ONLINE ▶

⬆
⬆
⬆

        ✅▶ 𝗥𝗲𝗮𝗹𝗹𝘆 𝗔𝗺𝗮𝘇𝗶𝗻𝗴 𝗬𝗼𝘂 𝗖𝗮𝗻 𝗧𝗿𝘆 𝗧𝗵𝗶𝘀 ◀✅✅

**[[📺📱 🔴 ➤►LIVE👉 STREAM PLAY NOW ▶
📺📱 🔴 ➤►Go👉 LIVE ONLINE ▶

⬆
⬆
⬆
⬆
⬆
⬆
⬆
⬆
⬆

SPORTS BOXing

        ✅▶ 𝗥𝗲𝗮𝗹𝗹𝘆 𝗔𝗺𝗮𝘇𝗶𝗻𝗴 𝗬𝗼𝘂 𝗖𝗮𝗻 𝗧𝗿𝘆 𝗧𝗵𝗶𝘀 ◀✅✅

**[[📺📱 🔴 ➤►LIVE👉 STREAM PLAY NOW ▶
📺📱 🔴 ➤►Go👉 LIVE ONLINE ▶

⬆
⬆
⬆

        ✅▶ 𝗥𝗲𝗮𝗹𝗹𝘆 𝗔𝗺𝗮𝘇𝗶𝗻𝗴 𝗬𝗼𝘂 𝗖𝗮𝗻 𝗧𝗿𝘆 𝗧𝗵𝗶𝘀 ◀✅✅

**[[📺📱 🔴 ➤►LIVE👉 STREAM PLAY NOW ▶
📺📱 🔴 ➤►Go👉 LIVE ONLINE ▶

⬆
⬆
⬆
⬆
⬆
⬆
⬆
⬆
⬆

SPORTS BOXing

        ✅▶ 𝗥𝗲𝗮𝗹𝗹𝘆 𝗔𝗺𝗮𝘇𝗶𝗻𝗴 𝗬𝗼𝘂 𝗖𝗮𝗻 𝗧𝗿𝘆 𝗧𝗵𝗶𝘀 ◀✅✅

**[[📺📱 🔴 ➤►LIVE👉 STREAM PLAY NOW ▶
📺📱 🔴 ➤►Go👉 LIVE ONLINE ▶

⬆
⬆
⬆

        ✅▶ 𝗥𝗲𝗮𝗹𝗹𝘆 𝗔𝗺𝗮𝘇𝗶𝗻𝗴 𝗬𝗼𝘂 𝗖𝗮𝗻 𝗧𝗿𝘆 𝗧𝗵𝗶𝘀 ◀✅✅

**[[📺📱 🔴 ➤►LIVE👉 STREAM PLAY NOW ▶
📺📱 🔴 ➤►Go👉 LIVE ONLINE ▶

⬆
⬆
⬆
⬆
⬆
⬆
⬆
⬆
⬆

SPORTS BOXing

        ✅▶ 𝗥𝗲𝗮𝗹𝗹𝘆 𝗔𝗺𝗮𝘇𝗶𝗻𝗴 𝗬𝗼𝘂 𝗖𝗮𝗻 𝗧𝗿𝘆 𝗧𝗵𝗶𝘀 ◀✅✅

**[[📺📱 🔴 ➤►LIVE👉 STREAM PLAY NOW ▶
📺📱 🔴 ➤►Go👉 LIVE ONLINE ▶

⬆
⬆
⬆

        ✅▶ 𝗥𝗲𝗮𝗹𝗹𝘆 𝗔𝗺𝗮𝘇𝗶𝗻𝗴 𝗬𝗼𝘂 𝗖𝗮𝗻 𝗧𝗿𝘆 𝗧𝗵𝗶𝘀 ◀✅✅

**[[📺📱 🔴 ➤►LIVE👉 STREAM PLAY NOW ▶
📺📱 🔴 ➤►Go👉 LIVE ONLINE ▶

⬆
⬆
⬆
⬆
⬆
⬆
⬆
⬆
⬆

vvljkljkl11

repo: vllm-project/llm-compressor


text:
AWQ modifier implementation

Adds a AWQModifier to llmcompressor
Depends on

#179 (For the test script attached below, otherwise can land independently of this)
#182

The test script has been attached:
# local/awq/AWQ/scripts/awq_one_shot.py
from llmcompressor.modifiers.awq.base import AWQModifier
from llmcompressor.modifiers.quantization import QuantizationModifier
from llmcompressor.transformers import oneshot
from llmcompressor.transformers import SparseAutoModelForCausalLM
from transformers import AutoTokenizer

MODEL_ID = 'meta-llama/Llama-2-7b-hf'
DATASET_ID, SPLIT = "mit-han-lab/pile-val-backup", {"calibration": "validation"}
NUM_CALIBRATION_SAMPLES = 128
OUTPUT_DIR = MODEL_ID.split("/")[-1] + f"-awq-llm-compressor-{NUM_CALIBRATION_SAMPLES}"
MAX_SEQUENCE_LENGTH = 512

recipe = [
    AWQModifier(bits=4),
    QuantizationModifier(scheme="W4A16", targets="Linear", ignore=["lm_head"]),

]

model = SparseAutoModelForCausalLM.from_pretrained(
    MODEL_ID,  device_map="auto", torch_dtype="auto"
)

oneshot(
    model=model,
    dataset=DATASET_ID,
    splits=SPLIT,
    recipe=recipe,
    save_compressed=True,
    output_dir=OUTPUT_DIR,
    overwrite_output_dir=True,
    max_seq_length=MAX_SEQUENCE_LENGTH,
    num_calibration_samples=NUM_CALIBRATION_SAMPLES,
)

print("Done! model saved to", OUTPUT_DIR)

print("Runnning Evaluation")
from ppl_eval import evaluate_perplexity

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
ppl = evaluate_perplexity(model, tokenizer)
print(f"Perplexity: {ppl}")

# local/awq/AWQ/scripts/ppl_eval.py

import torch
from datasets import load_dataset
from tqdm import tqdm
from transformers import AutoTokenizer
from llmcompressor.transformers import SparseAutoModelForCausalLM

def evaluate_perplexity(model, tokenizer):
    def _perplexity(nlls, n_samples, seqlen):
        return torch.exp(torch.stack(nlls).sum() / (n_samples * seqlen))

    # load and prepare dataset
    data = load_dataset("wikitext", "wikitext-2-raw-v1", split="test")
    data = tokenizer("\n\n".join(data["text"]), return_tensors="pt")
    data = data.input_ids.to(model.device)

    seqlen = 2048
    model = model.eval()
    n_samples = data.numel() // seqlen

    nlls = []

    with tqdm(range(n_samples), desc="Perplexity -") as progress_bar:
        for i in progress_bar:
            start_index = i * seqlen
            end_index = (i + 1) * seqlen
            batch = data[:, start_index:end_index].to(model.device)
            with torch.no_grad():
                logits = model(batch).logits
            shift_logits = logits[:, :-1, :].contiguous().float()
            shift_labels = data[:, start_index:end_index][:, 1:]
            loss_fct = torch.nn.CrossEntropyLoss()
            loss = loss_fct(
                shift_logits.view(-1, shift_logits.size(-1)), shift_labels.view(-1)
            )
            neg_log_likelihood = loss.float() * seqlen
            nlls.append(neg_log_likelihood)

            curr_ppl = _perplexity(nlls, i + 1, seqlen)
            progress_bar.set_description(f"Perplexity {curr_ppl:.3f}")

    ppl = _perplexity(nlls, n_samples, seqlen)

    return ppl.item()

def parse_args():
    import argparse

    parser = argparse.ArgumentParser()
    parser.add_argument("--model", type=str, required=True)
    return parser.parse_args()

def main():
    args = parse_args()
    model = SparseAutoModelForCausalLM.from_pretrained(
        args.model,
        torch_dtype="auto",
        device_map="auto"
        )
    tokenizer = AutoTokenizer.from_pretrained(args.model)
    ppl = evaluate_perplexity(model, tokenizer)
    print(f"Perplexity: {ppl}")

if __name__ == "__main__":
    main()

Reported Perplexity:

Model
Perplexity (wikitext)
Relative Increase (from Baseline)

llama-2-7b-awq (AutoAWQ Baseline)
5.614
0

Llama-2-7b-hf-awq-llm-compressor-128
5.706
1.61%

Llama-2-7b-hf-awq-llm-compressor-64
5.709
1.66%

Model
Perplexity (wikitext)
Relative Increase (from Baseline)

hugging-quants/Meta-Llama-3.1-8B-Instruct-AWQ-INT4
7.547
0

Meta-Llama-3.1-8B-Instruct-awq-llm-compressor-128
7.782
3.1%

All models are located under: /network/rahul/awq-models

Love the API and simplicity of this! Great work.

authors: casper-hansen rahul-tuli


text:
[Bug]: speculative_draft_tensor_parallel_size=4 cannot be other value than 1

Your current environment

The output of `python collect_env.py`
vllm 0.6.3 docker image
RTX 4090

use speculative decoding and set tensor-parallel=4 for draft model.

Model Input Dumps
no input, launch error
🐛 Describe the bug
No module named 'vllm._version'
  from vllm.version import __version__ as VLLM_VERSION
INFO 11-19 23:29:07 api_server.py:528] vLLM API server version dev
INFO 11-19 23:29:07 api_server.py:529] args: Namespace(host=None, port=14050, uvicorn_log_level='info', allow_credentials=False, allowed_origins=['*'], allowed_methods=['*'], allowed_headers=['*'], api_key=None, lora_modules=None, prompt_adapters=None, chat_template=None, response_role='assistant', ssl_keyfile=None, ssl_certfile=None, ssl_ca_certs=None, ssl_cert_reqs=0, root_path=None, middleware=[], return_tokens_as_token_ids=False, disable_frontend_multiprocessing=False, enable_auto_tool_choice=False, tool_call_parser=None, tool_parser_plugin='', model='/models/Qwen2.5-72B-Instruct-AWQ/', tokenizer=None, skip_tokenizer_init=False, revision=None, code_revision=None, tokenizer_revision=None, tokenizer_mode='auto', trust_remote_code=False, download_dir=None, load_format='auto', config_format='auto', dtype='bfloat16', kv_cache_dtype='auto', quantization_param_path=None, max_model_len=8192, guided_decoding_backend='outlines', distributed_executor_backend=None, worker_use_ray=False, pipeline_parallel_size=1, tensor_parallel_size=4, max_parallel_loading_workers=None, ray_workers_use_nsight=False, block_size=16, enable_prefix_caching=True, disable_sliding_window=False, use_v2_block_manager=True, num_lookahead_slots=0, seed=42, swap_space=4, cpu_offload_gb=0, gpu_memory_utilization=0.9, num_gpu_blocks_override=None, max_num_batched_tokens=None, max_num_seqs=64, max_logprobs=20, disable_log_stats=False, quantization=None, rope_scaling=None, rope_theta=None, enforce_eager=False, max_context_len_to_capture=None, max_seq_len_to_capture=8192, disable_custom_all_reduce=False, tokenizer_pool_size=0, tokenizer_pool_type='ray', tokenizer_pool_extra_config=None, limit_mm_per_prompt=None, mm_processor_kwargs=None, enable_lora=False, max_loras=1, max_lora_rank=16, lora_extra_vocab_size=256, lora_dtype='auto', long_lora_scaling_factors=None, max_cpu_loras=None, fully_sharded_loras=False, enable_prompt_adapter=False, max_prompt_adapters=1, max_prompt_adapter_token=0, device='auto', num_scheduler_steps=1, multi_step_stream_outputs=True, scheduler_delay_factor=0.0, enable_chunked_prefill=None, speculative_model='/models/Qwen2.5-14B-Instruct-AWQ', speculative_model_quantization=None, num_speculative_tokens=3, speculative_disable_mqa_scorer=False, speculative_draft_tensor_parallel_size=4, speculative_max_model_len=8192, speculative_disable_by_batch_size=None, ngram_prompt_lookup_max=None, ngram_prompt_lookup_min=None, spec_decoding_acceptance_method='rejection_sampler', typical_acceptance_sampler_posterior_threshold=None, typical_acceptance_sampler_posterior_alpha=None, disable_logprobs_during_spec_decoding=None, model_loader_extra_config=None, ignore_patterns=[], preemption_mode=None, served_model_name=['default'], qlora_adapter_name_or_path=None, otlp_traces_endpoint=None, collect_detailed_traces=None, disable_async_output_proc=False, override_neuron_config=None, scheduling_policy='fcfs', disable_log_requests=False, max_log_len=None, disable_fastapi_docs=False)
INFO 11-19 23:29:07 api_server.py:166] Multiprocessing frontend to use ipc:///tmp/e5ce14c2-12ac-46d7-be7a-d30ffab51be3 for IPC Path.
INFO 11-19 23:29:07 api_server.py:179] Started engine process with PID 76
WARNING 11-19 23:29:07 config.py:1674] Casting torch.float16 to torch.bfloat16.
/usr/local/lib/python3.12/dist-packages/vllm/connections.py:8: RuntimeWarning: Failed to read commit hash:
No module named 'vllm._version'
  from vllm.version import __version__ as VLLM_VERSION
WARNING 11-19 23:29:11 config.py:1674] Casting torch.float16 to torch.bfloat16.
(base) chenchunhui@node1:~$ docker logs d94e5aedc94a3d86638a4d983245ca1f226620de2cf4d7476ac355b9d179984e
/usr/local/lib/python3.12/dist-packages/vllm/connections.py:8: RuntimeWarning: Failed to read commit hash:
No module named 'vllm._version'
  from vllm.version import __version__ as VLLM_VERSION
INFO 11-19 23:29:07 api_server.py:528] vLLM API server version dev
INFO 11-19 23:29:07 api_server.py:529] args: Namespace(host=None, port=14050, uvicorn_log_level='info', allow_credentials=False, allowed_origins=['*'], allowed_methods=['*'], allowed_headers=['*'], api_key=None, lora_modules=None, prompt_adapters=None, chat_template=None, response_role='assistant', ssl_keyfile=None, ssl_certfile=None, ssl_ca_certs=None, ssl_cert_reqs=0, root_path=None, middleware=[], return_tokens_as_token_ids=False, disable_frontend_multiprocessing=False, enable_auto_tool_choice=False, tool_call_parser=None, tool_parser_plugin='', model='/models/Qwen2.5-72B-Instruct-AWQ/', tokenizer=None, skip_tokenizer_init=False, revision=None, code_revision=None, tokenizer_revision=None, tokenizer_mode='auto', trust_remote_code=False, download_dir=None, load_format='auto', config_format='auto', dtype='bfloat16', kv_cache_dtype='auto', quantization_param_path=None, max_model_len=8192, guided_decoding_backend='outlines', distributed_executor_backend=None, worker_use_ray=False, pipeline_parallel_size=1, tensor_parallel_size=4, max_parallel_loading_workers=None, ray_workers_use_nsight=False, block_size=16, enable_prefix_caching=True, disable_sliding_window=False, use_v2_block_manager=True, num_lookahead_slots=0, seed=42, swap_space=4, cpu_offload_gb=0, gpu_memory_utilization=0.9, num_gpu_blocks_override=None, max_num_batched_tokens=None, max_num_seqs=64, max_logprobs=20, disable_log_stats=False, quantization=None, rope_scaling=None, rope_theta=None, enforce_eager=False, max_context_len_to_capture=None, max_seq_len_to_capture=8192, disable_custom_all_reduce=False, tokenizer_pool_size=0, tokenizer_pool_type='ray', tokenizer_pool_extra_config=None, limit_mm_per_prompt=None, mm_processor_kwargs=None, enable_lora=False, max_loras=1, max_lora_rank=16, lora_extra_vocab_size=256, lora_dtype='auto', long_lora_scaling_factors=None, max_cpu_loras=None, fully_sharded_loras=False, enable_prompt_adapter=False, max_prompt_adapters=1, max_prompt_adapter_token=0, device='auto', num_scheduler_steps=1, multi_step_stream_outputs=True, scheduler_delay_factor=0.0, enable_chunked_prefill=None, speculative_model='/models/Qwen2.5-14B-Instruct-AWQ', speculative_model_quantization=None, num_speculative_tokens=3, speculative_disable_mqa_scorer=False, speculative_draft_tensor_parallel_size=4, speculative_max_model_len=8192, speculative_disable_by_batch_size=None, ngram_prompt_lookup_max=None, ngram_prompt_lookup_min=None, spec_decoding_acceptance_method='rejection_sampler', typical_acceptance_sampler_posterior_threshold=None, typical_acceptance_sampler_posterior_alpha=None, disable_logprobs_during_spec_decoding=None, model_loader_extra_config=None, ignore_patterns=[], preemption_mode=None, served_model_name=['default'], qlora_adapter_name_or_path=None, otlp_traces_endpoint=None, collect_detailed_traces=None, disable_async_output_proc=False, override_neuron_config=None, scheduling_policy='fcfs', disable_log_requests=False, max_log_len=None, disable_fastapi_docs=False)
INFO 11-19 23:29:07 api_server.py:166] Multiprocessing frontend to use ipc:///tmp/e5ce14c2-12ac-46d7-be7a-d30ffab51be3 for IPC Path.
INFO 11-19 23:29:07 api_server.py:179] Started engine process with PID 76
WARNING 11-19 23:29:07 config.py:1674] Casting torch.float16 to torch.bfloat16.
/usr/local/lib/python3.12/dist-packages/vllm/connections.py:8: RuntimeWarning: Failed to read commit hash:
No module named 'vllm._version'
  from vllm.version import __version__ as VLLM_VERSION
WARNING 11-19 23:29:11 config.py:1674] Casting torch.float16 to torch.bfloat16.
INFO 11-19 23:29:11 awq_marlin.py:97] The model is convertible to awq_marlin during runtime. Using awq_marlin kernel.
INFO 11-19 23:29:11 config.py:887] Defaulting to use mp for distributed inference
WARNING 11-19 23:29:11 config.py:1674] Casting torch.float16 to torch.bfloat16.
INFO 11-19 23:29:11 awq_marlin.py:97] The model is convertible to awq_marlin during runtime. Using awq_marlin kernel.
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/openai/api_server.py", line 585, in <module>
    uvloop.run(run_server(args))
  File "/usr/local/lib/python3.12/dist-packages/uvloop/__init__.py", line 109, in run
    return __asyncio.run(
           ^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/asyncio/runners.py", line 194, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "uvloop/loop.pyx", line 1517, in uvloop.loop.Loop.run_until_complete
  File "/usr/local/lib/python3.12/dist-packages/uvloop/__init__.py", line 61, in wrapper
    return await main
           ^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/openai/api_server.py", line 552, in run_server
    async with build_async_engine_client(args) as engine_client:
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/contextlib.py", line 210, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/openai/api_server.py", line 107, in build_async_engine_client
    async with build_async_engine_client_from_engine_args(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/contextlib.py", line 210, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/openai/api_server.py", line 184, in build_async_engine_client_from_engine_args
    engine_config = engine_args.create_engine_config()
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/vllm/engine/arg_utils.py", line 975, in create_engine_config
    speculative_config = SpeculativeConfig.maybe_create_spec_config(
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/vllm/config.py", line 1292, in maybe_create_spec_config
    SpeculativeConfig.create_draft_parallel_config(
  File "/usr/local/lib/python3.12/dist-packages/vllm/config.py", line 1383, in create_draft_parallel_config
    raise ValueError(
ValueError: speculative_draft_tensor_parallel_size=4 cannot be other value than 1

Before submitting a new issue...

[X] Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

It is not a bug, the ValueError report is as expected. Currently, speculative_draft_tensor_parallel_size only supports 1 or equal to target model tensor_parallel_size by default. Maybe you should try the latest release version, which has clearer error messages.

fix in version v0.6.4.post1

repo: vllm-project/vllm


text:
[Bug]: unhandled system error with NCCL on v0.5.0.post1

Your current environment
Collecting environment information...
PyTorch version: 2.3.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.4 LTS (x86_64)
GCC version: (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Clang version: Could not collect
CMake version: version 3.29.2
Libc version: glibc-2.35

Python version: 3.11.9 (main, Apr 19 2024, 16:48:06) [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-6.5.0-1022-azure-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: Could not collect
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: Tesla T4
Nvidia driver version: 535.171.04
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture:                       x86_64
CPU op-mode(s):                     32-bit, 64-bit
Address sizes:                      48 bits physical, 48 bits virtual
Byte Order:                         Little Endian
CPU(s):                             4
On-line CPU(s) list:                0-3
Vendor ID:                          AuthenticAMD
Model name:                         AMD EPYC 7V12 64-Core Processor
CPU family:                         23
Model:                              49
Thread(s) per core:                 1
Core(s) per socket:                 4
Socket(s):                          1
Stepping:                           0
BogoMIPS:                           4890.88
Flags:                              fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl tsc_reliable nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext perfctr_core ssbd vmmcall fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 clzero xsaveerptr rdpru arat umip rdpid
Hypervisor vendor:                  Microsoft
Virtualization type:                full
L1d cache:                          128 KiB (4 instances)
L1i cache:                          128 KiB (4 instances)
L2 cache:                           2 MiB (4 instances)
L3 cache:                           16 MiB (1 instance)
NUMA node(s):                       1
NUMA node0 CPU(s):                  0-3
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit:        Not affected
Vulnerability L1tf:                 Not affected
Vulnerability Mds:                  Not affected
Vulnerability Meltdown:             Not affected
Vulnerability Mmio stale data:      Not affected
Vulnerability Retbleed:             Mitigation; untrained return thunk; SMT disabled
Vulnerability Spec rstack overflow: Vulnerable: Safe RET, no microcode
Vulnerability Spec store bypass:    Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:           Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:           Mitigation; Retpolines; STIBP disabled; RSB filling; PBRSB-eIBRS Not affected; BHI Not affected
Vulnerability Srbds:                Not affected
Vulnerability Tsx async abort:      Not affected

Versions of relevant libraries:
[pip3] numpy==1.26.4
[pip3] nvidia-nccl-cu12==2.20.5
[pip3] sentence-transformers==2.7.0
[pip3] torch==2.3.0+cu121
[pip3] transformers==4.40.1
[pip3] triton==2.3.0
[pip3] vllm-nccl-cu12==2.18.1.0.4.0
[conda] numpy                     1.26.4                   pypi_0    pypi
[conda] nvidia-nccl-cu12          2.20.5                   pypi_0    pypi
[conda] sentence-transformers     2.7.0                    pypi_0    pypi
[conda] torch                     2.3.0+cu121              pypi_0    pypi
[conda] transformers              4.40.1                   pypi_0    pypi
[conda] triton                    2.3.0                    pypi_0    pypi
[conda] vllm-nccl-cu12            2.18.1.0.4.0             pypi_0    pypi
ROCM Version: Could not collect
Neuron SDK Version: N/A
vLLM Version: 0.4.2
vLLM Build Flags:
CUDA Archs: Not Set; ROCm: Disabled; Neuron: Disabled
GPU Topology:
GPU0    NIC0    CPU Affinity    NUMA Affinity   GPU NUMA ID
GPU0     X      SYS     0-3     0               N/A
NIC0    SYS      X

Legend:

  X    = Self
  SYS  = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
  NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
  PHB  = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
  PXB  = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
  PIX  = Connection traversing at most a single PCIe bridge
  NV#  = Connection traversing a bonded set of # NVLinks

NIC Legend:

  NIC0: mlx5_0

🐛 Describe the bug
vllm won't load with v0.5.0 or v0.5.0.post1, when downgrading to 0.4.2 it works fine.
v0.5.0.post1 throws this error: RuntimeError: NCCL Error 2: unhandled system error. More details below:
INFO 06-25 15:26:04 api_server.py:177] vLLM API server version 0.5.0.post1
INFO 06-25 15:26:04 api_server.py:178] args: Namespace(host=None, port=8000, uvicorn_log_level='info', allow_credentials=False, allowed_origins=[''], allowed_methods=[''], allowed_headers=['*'], api_key=None, lora_modules=None, chat_template=None, response_role='assistant', ssl_keyfile=None, ssl_certfile=None, ssl_ca_certs=None, ssl_cert_reqs=0, root_path=None, middleware=[], model='/ai/Meta-Llama-3-8B-Instruct', tokenizer=None, skip_tokenizer_init=False, revision=None, code_revision=None, tokenizer_revision=None, tokenizer_mode='auto', trust_remote_code=False, download_dir=None, load_format='auto', dtype='half', kv_cache_dtype='auto', quantization_param_path=None, max_model_len=4096, guided_decoding_backend='outlines', distributed_executor_backend=None, worker_use_ray=True, pipeline_parallel_size=1, tensor_parallel_size=2, max_parallel_loading_workers=None, ray_workers_use_nsight=False, block_size=16, enable_prefix_caching=True, disable_sliding_window=False, use_v2_block_manager=False, num_lookahead_slots=0, seed=0, swap_space=4, gpu_memory_utilization=0.96, num_gpu_blocks_override=None, max_num_batched_tokens=None, max_num_seqs=256, max_logprobs=20, disable_log_stats=True, quantization=None, rope_scaling=None, rope_theta=None, enforce_eager=True, max_context_len_to_capture=None, max_seq_len_to_capture=8192, disable_custom_all_reduce=False, tokenizer_pool_size=0, tokenizer_pool_type='ray', tokenizer_pool_extra_config=None, enable_lora=False, max_loras=1, max_lora_rank=16, lora_extra_vocab_size=256, lora_dtype='auto', long_lora_scaling_factors=None, max_cpu_loras=None, fully_sharded_loras=False, device='auto', image_input_type=None, image_token_id=None, image_input_shape=None, image_feature_size=None, image_processor=None, image_processor_revision=None, disable_image_processor=False, scheduler_delay_factor=0.0, enable_chunked_prefill=False, speculative_model=None, num_speculative_tokens=None, speculative_max_model_len=None, speculative_disable_by_batch_size=None, ngram_prompt_lookup_max=None, ngram_prompt_lookup_min=None, model_loader_extra_config=None, preemption_mode=None, served_model_name=None, qlora_adapter_name_or_path=None, engine_use_ray=False, disable_log_requests=True, max_log_len=None)
WARNING 06-25 15:26:04 config.py:1222] Casting torch.bfloat16 to torch.float16.
2024-06-25 15:26:04,681 INFO worker.py:1568 -- Connecting to existing Ray cluster at address: 10.1.0.10:6379...
2024-06-25 15:26:04,687 INFO worker.py:1753 -- Connected to Ray cluster.
INFO 06-25 15:26:04 llm_engine.py:161] Initializing an LLM engine (v0.5.0.post1) with config: model='/ai/Meta-Llama-3-8B-Instruct', speculative_config=None, tokenizer='/ai/Meta-Llama-3-8B-Instruct', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, rope_scaling=None, rope_theta=None, tokenizer_revision=None, trust_remote_code=False, dtype=torch.float16, max_seq_len=4096, download_dir=None, load_format=LoadFormat.AUTO, tensor_parallel_size=2, disable_custom_all_reduce=False, quantization=None, enforce_eager=True, kv_cache_dtype=auto, quantization_param_path=None, device_config=cuda, decoding_config=DecodingConfig(guided_decoding_backend='outlines'), seed=0, served_model_name=/ai/Meta-Llama-3-8B-Instruct)
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
INFO 06-25 15:26:09 selector.py:131] Cannot use FlashAttention-2 backend for Volta and Turing GPUs.
INFO 06-25 15:26:09 selector.py:51] Using XFormers backend.
(RayWorkerWrapper pid=67451, ip=10.1.0.7) INFO 06-25 15:26:09 selector.py:131] Cannot use FlashAttention-2 backend for Volta and Turing GPUs.
(RayWorkerWrapper pid=67451, ip=10.1.0.7) INFO 06-25 15:26:09 selector.py:51] Using XFormers backend.
INFO 06-25 15:26:10 utils.py:637] Found nccl from library libnccl.so.2
INFO 06-25 15:26:10 pynccl.py:63] vLLM is using nccl==2.20.5
(RayWorkerWrapper pid=67451, ip=10.1.0.7) INFO 06-25 15:26:10 utils.py:637] Found nccl from library libnccl.so.2
(RayWorkerWrapper pid=67451, ip=10.1.0.7) INFO 06-25 15:26:10 pynccl.py:63] vLLM is using nccl==2.20.5
WARNING 06-25 15:26:10 custom_all_reduce.py:117] Custom allreduce is disabled because this process group spans across nodes.
INFO 06-25 15:26:10 selector.py:131] Cannot use FlashAttention-2 backend for Volta and Turing GPUs.
INFO 06-25 15:26:10 selector.py:51] Using XFormers backend.
(RayWorkerWrapper pid=67451, ip=10.1.0.7) WARNING 06-25 15:26:10 custom_all_reduce.py:117] Custom allreduce is disabled because this process group spans across nodes.
(RayWorkerWrapper pid=67451, ip=10.1.0.7) INFO 06-25 15:26:10 selector.py:131] Cannot use FlashAttention-2 backend for Volta and Turing GPUs.
(RayWorkerWrapper pid=67451, ip=10.1.0.7) INFO 06-25 15:26:10 selector.py:51] Using XFormers backend.
INFO 06-25 15:26:13 model_runner.py:160] Loading model weights took 7.4829 GB
(RayWorkerWrapper pid=67451, ip=10.1.0.7) INFO 06-25 15:26:16 model_runner.py:160] Loading model weights took 7.4829 GB
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148] Error executing method determine_num_available_blocks. This might cause deadlock in distributed execution.
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148] Traceback (most recent call last):
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]   File "/home/efleisc222/miniconda3/envs/ai/lib/python3.11/site-packages/vllm/worker/worker_base.py", line 140, in execute_method
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]     return executor(*args, **kwargs)
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]            ^^^^^^^^^^^^^^^^^^^^^^^^^
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]   File "/home/efleisc222/miniconda3/envs/ai/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]     return func(*args, **kwargs)
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]            ^^^^^^^^^^^^^^^^^^^^^
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]   File "/home/efleisc222/miniconda3/envs/ai/lib/python3.11/site-packages/vllm/worker/worker.py", line 162, in determine_num_available_blocks
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]     self.model_runner.profile_run()
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]   File "/home/efleisc222/miniconda3/envs/ai/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]     return func(*args, **kwargs)
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]            ^^^^^^^^^^^^^^^^^^^^^
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]   File "/home/efleisc222/miniconda3/envs/ai/lib/python3.11/site-packages/vllm/worker/model_runner.py", line 844, in profile_run
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]     self.execute_model(seqs, kv_caches)
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]   File "/home/efleisc222/miniconda3/envs/ai/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]     return func(*args, **kwargs)
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]            ^^^^^^^^^^^^^^^^^^^^^
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]   File "/home/efleisc222/miniconda3/envs/ai/lib/python3.11/site-packages/vllm/worker/model_runner.py", line 758, in execute_model
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]     logits = self.model.compute_logits(hidden_states, sampling_metadata)
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]   File "/home/efleisc222/miniconda3/envs/ai/lib/python3.11/site-packages/vllm/model_executor/models/llama.py", line 377, in compute_logits
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]     logits = self.logits_processor(self.lm_head.weight, hidden_states,
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]   File "/home/efleisc222/miniconda3/envs/ai/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]     return self._call_impl(*args, **kwargs)
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]   File "/home/efleisc222/miniconda3/envs/ai/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]     return forward_call(*args, **kwargs)
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]   File "/home/efleisc222/miniconda3/envs/ai/lib/python3.11/site-packages/vllm/model_executor/layers/logits_processor.py", line 52, in forward
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]     logits = self.get_logits(hidden_states, embedding, embedding_bias)
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]   File "/home/efleisc222/miniconda3/envs/ai/lib/python3.11/site-packages/vllm/model_executor/layers/logits_processor.py", line 69, in get_logits
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]     logits = tensor_model_parallel_gather(logits)
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]   File "/home/efleisc222/miniconda3/envs/ai/lib/python3.11/site-packages/vllm/distributed/communication_op.py", line 24, in tensor_model_parallel_gather
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]     return get_tp_group().gather(input, dst, dim)
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]   File "/home/efleisc222/miniconda3/envs/ai/lib/python3.11/site-packages/vllm/distributed/parallel_state.py", line 301, in gather
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]     torch.distributed.gather(input,
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]   File "/home/efleisc222/miniconda3/envs/ai/lib/python3.11/site-packages/torch/distributed/c10d_logger.py", line 75, in wrapper
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]     return func(*args, **kwargs)
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]            ^^^^^^^^^^^^^^^^^^^^^
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]   File "/home/efleisc222/miniconda3/envs/ai/lib/python3.11/site-packages/torch/distributed/distributed_c10d.py", line 3133, in gather
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]     work = group.gather(output_tensors, input_tensors, opts)
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(RayWorkerWrapper pid=67451, ip=10.1.0.7) ERROR 06-25 15:28:30 worker_base.py:148] RuntimeError: NCCL Error 2: unhandled system error (run with NCCL_DEBUG=INFO for details)

can you follow https://docs.vllm.ai/en/latest/getting_started/debugging.html to dig out the cause of the error?

I solved the problem setting NCCL_SOCKET_IFNAME=eth0 on both ray head and nodes. Now it loads with no errors. P.S.: another error message I found in the log was: "Error executing method determine_num_available_blocks. This might cause deadlock in distributed execution"
but again, settting NCCL_SOCKET_IFNAME solved the problem and v0.5.0.post1 loads with no errors.

glad it works. I just added NCCL_SOCKET_IFNAME in https://docs.vllm.ai/en/latest/getting_started/debugging.html

authors: eByteTheDust youkaichao


text:
[Usage]: Load local model from local path

How would you like to use vllm
Does vLLM support huggingface local pytorch pt model or onnx model? How can I load them in both offline python code  and OpenAI Completions API? I can see an error saying config.json file was not found. Does vllm only support vanilla hf model?

You could load local models, but the file structure should follow that of a standard HuggingFace model repo.

Do you know how can I convert pt model or onnx model to huggingface model format? Seems I need to register it to huggingface first: https://discuss.huggingface.co/t/convert-pytorch-model-to-huggingface-transformer/16965

Sorry for very late reply, the model just needs to inherit from PreTrainedModel. You can see some example conversion scripts like https://github.com/huggingface/transformers/blob/main/src/transformers/models/llava_next/convert_llava_next_weights_to_hf.py

@xiaoyu-work is your issue resolved?

@xiaoyu-work is your issue resolved?

Yes i got the answer for this question. Thanks!

authors: DarkLight1337 hmellor xiaoyu-work


text:
Memory leak while using tensor_parallel_size>1

Can you provide more details on what model are you using, and how many GPUs are you using? Any more details can be helpful. Thank you!

I'm running starcoder on 2*A10, The command is as follows:

same question when loading llama2 70b models on 4 gpus

same question when loading llama2 70b models on 2 gpus

Same issue w/ Mixtral 8x7B Instruct 0.1 (non-quantized)

we met the same issue with mistral7b (tp = 4)

tensor_parallel_size also meet memory leak.
TP=1, GPU = 1*A30 vllm = 0.3.3

tensor_parallel_size also meets memory leak.
TP=2, GPU = 2*V100 vllm = 0.4.2

TP = 2, GPU=2 V100,llama3.1-8B-Instruct, vllm version = 0.6.2  . When I close the server it shows Warning as leaked memo.

authors: ChristineSeven PeterWang1986 austingg ekmekovski haiasd wangcho2k wonderseen yarinlaniado zhuohan123


text:
[Core] Reduce TTFT with concurrent partial prefills

Replaces #10061, as inspired by @njhill and @comaniac's comments. Co-authored by @prashantgupta24
Context: our customers running large multi-tenanted SaaS deployments of vLLM have a problem where high volumes of small-prompt requests are usually processed smoothly, but quickly pile up in a giant queue when a small number of large-prompt requests are submitted. We see the decoding throughput drop to zero on multiple replicas when this happens.
The current chunked prefill implementation only allows a single sequence to be partially prefilled at a time. This has a few limitations:

Multiple medium-sized prompts must wait to be prefilled serially, increasing TTFT for those in the back of the queue
A single very large prompt will block all other prompts from prefilling for many iterations. This can eventually starve decoding- for example a 130k token prompt with —max-num-batched-tokens=512 will take about 250 iterations to prefill, in which time the currently decoding sequences may all finish. Send a few of these requests at once and very quickly nothing will be decoding.

This PR implements both

An explicit setting for the number of sequences that can be partially prefilled concurrently. This can be configured with --max-num-partial-prefills=N
A limit on the number of “very long prompt” sequences that can be prefilled concurrently. This can be configured with

--max-long-partial-prefills=N to set the limit on the number of long sequences that can be concurrently prefilled. This defaults to 1 sequence.
--long-prefill-threashold=x% to set a percentage of the context length that determines which sequences are considered "long". This defaults to 4%

This is implemented in the v0 scheduler. We’re aware that the v1 implementation is underway and will later become the default, but we need a fix for our customers soon and we hope that what we discover here may help inform a different, better solution in the v1 scheduler.
To test this we created three scenarios, a “medium request” case, a “large request” case, and a “mixed” case.
For the medium request case, we created a subset of the sharegpt dataset with 900 small requests (<50 prompt characters) and 100 of the largest requests (typically between 10k and 20k prompt characters, which we call “medium” sized). We modified the benchmark_serving.py test to not filter out any of the small or large requests, and ran it with this dataset. What we expect to find is similar throughput compared to the main branch, but much lower TTFT on the small requests. Since 10% of the requests are larger than the rest, we should see better TTFT at p90 and below, with comparable TTFT above p90.
For the large request case, we took 990 of the smallest requests from the sharegpt dataset, and then took 10 of the largest requests and duplicated the prompts until they were around 100k characters in length. We ran this in the same way as the medium request case, and here we expect to see smaller TTFT across the board since the small requests will no longer be blocked from prefilling by the few very large requests.
For the mixed case, we used 850 “small”, and 140 “medium” requests, as well as 10 "large" requests where we duplicated the prompts up to 200k characters.
All tests were run on a single 80GB A100, with the command:
python benchmarks/benchmark_serving.py --model meta-llama/Meta-Llama-3.1-8B-Instruct --dataset-path ${test_case} --metric-percentiles 80,85,90,95,99 --request-rate 12

We ran the tests against the main branch (commit 874f551b3626321f6bf9a902b8fd9fc1fa7c7f2e), as well as this PR with the new optimization both disabled (--max-num-partial-prefills=1), and enabled (--max-num-partial-prefills=4)
The results are shown here:

The TTFT improvements are very easy to see- in the medium case we cut the p90 TTFT in half, and in the large case we cut it nearly 30x. In both cases we did not measure a throughput drop when run with  --max-num-partial-prefills=1, and the throughput drop with --max-num-partial-prefills=4 is minimal.
Surprisingly, along with the massive TTFT improvements in the "mixed" test case, we also see a 4% throughput improvement (3506 tokens/s up from 3368 tokens/s). Based on the fact that ITL still looks a little slower, it seems that the throughput is higher simply because more requests were able to be successfully scheduled at the same time.
cc @rickyyx

 PR Checklist (Click to Expand)
Thank you for your contribution to vLLM! Before submitting the pull request, please ensure the PR meets the following criteria. This helps vLLM maintain the code quality and improve the efficiency of the review process.
PR Title and Classification
Only specific types of PRs will be reviewed. The PR title is prefixed appropriately to indicate the type of change. Please use one of the following:

[Bugfix] for bug fixes.
[CI/Build] for build or continuous integration improvements.
[Doc] for documentation fixes and improvements.
[Model] for adding a new model or improving an existing model. Model name should appear in the title.
[Frontend] For changes on the vLLM frontend (e.g., OpenAI API server, LLM class, etc.)
[Kernel] for changes affecting CUDA kernels or other compute kernels.
[Core] for changes in the core vLLM logic (e.g., LLMEngine, AsyncLLMEngine, Scheduler, etc.)
[Hardware][Vendor] for hardware-specific changes. Vendor name should appear in the prefix (e.g., [Hardware][AMD]).
[Misc] for PRs that do not fit the above categories. Please use this sparingly.

Note: If the PR spans more than one category, please include all relevant prefixes.
Code Quality
The PR need to meet the following code quality standards:

We adhere to Google Python style guide and Google C++ style guide.
Pass all linter checks. Please use format.sh to format your code.
The code need to be well-documented to ensure future contributors can easily understand the code.
Include sufficient tests to ensure the project to stay correct and robust. This includes both unit tests and integration tests.
Please add documentation to docs/source/ if the PR modifies the user-facing behaviors of vLLM. It helps vLLM user understand and utilize the new features or changes.

Adding or changing kernels
Each custom kernel needs a schema and one or more implementations to be registered with PyTorch.

Make sure custom ops are registered following PyTorch guidelines: Custom C++ and CUDA Operators and The Custom Operators Manual
Custom operations that return Tensors require meta-functions. Meta-functions should be implemented and registered in python so that dynamic dims can be handled automatically. See above documents for a description of meta-functions.
Use torch.libary.opcheck() to test the function registration and meta-function for any registered ops.  See tests/kernels for examples.
When changing the C++ signature of an existing op, the schema must be updated to reflect the changes.
If a new custom type is needed, see the following document: Custom Class Support in PT2.

Notes for Large Changes
Please keep the changes as concise as possible. For major architectural changes (>500 LOC excluding kernel/data/config/test), we would expect a GitHub issue (RFC) discussing the technical design and justification. Otherwise, we will tag it with rfc-required and might not go through the PR.
What to Expect for the Reviews
The goal of the vLLM team is to be a transparent reviewing machine. We would like to make the review process transparent and efficient and make sure no contributor feel confused or frustrated. However, the vLLM team is small, so we need to prioritize some PRs over others. Here is what you can expect from the review process:

 After the PR is submitted, the PR will be assigned to a reviewer. Every reviewer will pick up the PRs based on their expertise and availability.
 After the PR is assigned, the reviewer will provide status update every 2-3 days. If the PR is not reviewed within 7 days, please feel free to ping the reviewer or the vLLM team.
 After the review, the reviewer will put an  action-required label on the PR if there are changes required. The contributor should address the comments and ping the reviewer to re-review the PR.
 Please respond to all comments within a reasonable time frame. If a comment isn't clear or you disagree with a suggestion, feel free to ask for clarification or discuss the suggestion.

Thank You
 Finally, thank you for taking the time to read these guidelines and for your interest in contributing to vLLM. Your contributions make vLLM a great tool for everyone!

@prashantgupta24 it looks like the regression test is hanging for some reason, after prompt processing starts.
[2024-11-21T18:53:40Z] Running 4 items in this shard: tests/test_regression.py::test_duplicated_ignored_sequence_group, tests/test_regression.py::test_max_tokens_none, tests/test_regression.py::test_gc, tests/test_regression.py::test_model_from_modelscope
...
[2024-11-21T18:53:51Z] INFO 11-21 10:53:51 model_runner.py:1399] Capturing cudagraphs for decoding. This may lead to unexpected consequences if the model is not static. To run the model in eager mode, set 'enforce_eager=True' or use '--enforce-eager' in the CLI.
[2024-11-21T18:53:51Z] INFO 11-21 10:53:51 model_runner.py:1403] If out-of-memory error occurs during cudagraph capture, consider decreasing `gpu_memory_utilization` or switching to eager mode. You can also reduce the `max_num_seqs` as needed to decrease memory usage.
[2024-11-21T18:54:02Z] INFO 11-21 10:54:02 model_runner.py:1517] Graph capturing finished in 10 secs, took 0.12 GiB
Processed prompts:  50% 1/2 [00:00<00:00,  1.70it/s, est. speed input: 10.17 toks/s, output: 434.01 toks/s]
# Received cancellation signal, interrupting
[2024-11-21T20:52:13Z] 🚨 Error: The command exited with status -1

@prashantgupta24 it looks like the regression test is hanging for some reason, after prompt processing starts.
[2024-11-21T18:53:40Z] Running 4 items in this shard: tests/test_regression.py::test_duplicated_ignored_sequence_group, tests/test_regression.py::test_max_tokens_none, tests/test_regression.py::test_gc, tests/test_regression.py::test_model_from_modelscope
...
[2024-11-21T18:53:51Z] INFO 11-21 10:53:51 model_runner.py:1399] Capturing cudagraphs for decoding. This may lead to unexpected consequences if the model is not static. To run the model in eager mode, set 'enforce_eager=True' or use '--enforce-eager' in the CLI.
[2024-11-21T18:53:51Z] INFO 11-21 10:53:51 model_runner.py:1403] If out-of-memory error occurs during cudagraph capture, consider decreasing `gpu_memory_utilization` or switching to eager mode. You can also reduce the `max_num_seqs` as needed to decrease memory usage.
[2024-11-21T18:54:02Z] INFO 11-21 10:54:02 model_runner.py:1517] Graph capturing finished in 10 secs, took 0.12 GiB
Processed prompts:  50% 1/2 [00:00<00:00,  1.70it/s, est. speed input: 10.17 toks/s, output: 434.01 toks/s]
# Received cancellation signal, interrupting
[2024-11-21T20:52:13Z] 🚨 Error: The command exited with status -1

Whoops fixed that!

ugh this is a tough merge conflict

@comaniac @njhill any chance we could get this merged? All tests passed

What's the overheads of recomputing partial prefill metadata on each scheduler step? If it's not significant, could we avoid it? I feel the information we needed for the partial metadata could be accumulated as some very limited amount of internal state of the scheduler.

@rickyyx good question! I kinda already answered why we went this way in this comment but another reason we need to do some amount of up-front processing is that we have to first peek through the waiting queue at the beginning of each step to see if there are prefills that we can schedule so that we can then properly budget the tokens when we schedule the running queue. So, we could keep info around in the scheduler state about the number of prefills currently running, but we still have to peek at the waiting queue each step.
As for exactly how long the from_queues method takes, I don't know. The results we measured from the serving benchmark suggest that there's no significant overhead, but if you want we could instrument this method with some timing and measure exactly how long it takes.

Related recent issue that I think would be improved by this: https://github.com/vllm-project/vllm/issues/11286

maybe related to https://github.com/vllm-project/vllm/issues/10774

@rickyyx Here are some numbers from running the serving benchmark and measuring the time of from_queues
Metadata Overhead (mean/median/p95/max): 0.005/0.005/0.010/0.027 ms

This took at most 27 nanoseconds, so I don't think we need to worry about this overhead. What do you think?

@rickyyx Here are some numbers from running the serving benchmark and measuring the time of from_queues
Metadata Overhead (mean/median/p95/max): 0.005/0.005/0.010/0.027 ms

This took at most 27 nanoseconds, so I don't think we need to worry about this overhead. What do you think?

Oh nice, this is rather negligible. Thanks for collecting that - do you know roughly how many elements are in the queue? Or what's the load when serving?

@rickyyx It was the "medium case" from the pr description at 12qps, but I didn't grab any logs from the server about the size of the queues while it was running. I can run the other cases and do it at higher qps to verify it doesn't get out of hand

@rickyyx Some more numbers, with the "mixed case" under 24qps load:
INFO 12-19 21:15:37 metrics.py:467] Avg prompt throughput: 6380.4 tokens/s, Avg generation throughput: 532.1 tokens/s, Running: 91 reqs, Swapped: 0 reqs, Pending: 230 reqs, GPU KV cache usage: 98.7%, CPU KV cache usage: 0.0%.
Metadata Overhead (mean/median/p95/max): 0.031/0.025/0.070/0.106 ms

With many requests running and a large waiting queue it maxes out at 100 nanoseconds. So it's a little worse, but I think still sub-millisecond should be fine here.

authors: joerunde njhill noooop prashantgupta24 rickyyx


text:
[v1] fix compilation cache

compilation cache is not working due to a bug, self.to_be_compiled_sizes should be self.compile_sizes.copy() rather than self.compile_sizes.union(self.capture_sizes) .
this should be merged after https://github.com/vllm-project/vllm/pull/11596 . our compilation cache does not consider the code change in the model's code.
before this pr:

Compiling a graph for general shape takes 12.69 s

After this pr:

Compiling a graph for general shape takes 2.27 s

cc @tlrmchlsmth how can we take all these code into consideration? like rope / activation etc.

our compilation cache does not consider the code change in the model's code.

I thought this would be handled by adding the version number here. Maybe we need to explicitly add the git hash as well?
https://github.com/vllm-project/vllm/blob/faef77c0d69c5429182f475a57127676e6bcb230/vllm/config.py#L3000

I thought this would be handled by adding the version number here. Maybe we need to explicitly add the git hash as well?

that would be kind of overkill, for the developers (like me), the git hash changes from time to time, and it means I cannot reuse any compilation cache.

authors: tlrmchlsmth youkaichao


text:
GCP/A3 doesn't work with torch<2.2 - suggesting to change the deps

GCP/A3 doesn't work with torch<2.2 - so we can't use vllm - this PR is suggesting to relax the deps

Oh, I see there is a PR already: https://github.com/vllm-project/vllm/pull/2804


text:
Sync huggingface modifications of qwen Moe model

nearly huggingface merged my pr：https://github.com/huggingface/transformers/pull/30552/files
i introduced a new config "mlp_only_layers" to qwen Moe model,
i think vllm should keep the same model forward logic as huggingface model definations.
so this pr is for sync huggingface modifications of qwen Moe model.
the new config "mlp_only_layers" is to set cut which layers' experts, for fit to limited HBM or other creative scenarios.

change in Qwen2MoeDecoderLayer is to sync huggingface modifications of the model.
change in load_weights is when mlp_only_layers is not empty, "self.named_parameters()" do not have all weights(because some experts have been cut), it'll cause exception when doing "param = params_dict[name]".
it's also a bug in the original version of qwen2_moe.py, when set decoder_sparse_step > 1, experts also been cut, and  name is not in params_dict, params_dict[name] will cause expceptions.
what ever, check the params_dict's key "name" exsist or not before read params_dict[name] is safer.

i'm willing to contribute to great vllm ~~ any reply is welcomed~ thks ^_^

i'll fix the CI warns soon.

this issue can be solved：
https://github.com/vllm-project/vllm/issues/4369
and these issues can be solved when using MoE models：
https://github.com/vllm-project/vllm/issues/3931
https://github.com/vllm-project/vllm/issues/3563

@WoosukKwon @zhuohan123
could you please arrage someone to review my pr ?
thanks~~

amd test said:
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/meta-llama/Llama-2-7b-chat-hf/resolve/main/config.json
Failed to connect to localhost port 8000: Connection refused
timeout 600 bash -c 'until curl localhost:8000/v1/models; do sleep 1; done'
it's likely to be a bug of the scanner, i try it again

@WoosukKwon @zhuohan123 now ci passed, could you please give a review？thanks  ^_^

@JustinLin610 can you help take a look at this?

@JustinLin610 can you help take a look at this? Thank you! 🙏

thank you for replying!
@JustinLin610 HI ^_^ ，qwen member, could you please give this pr a review? thanks!

@yangapku @simonJJJ @logicwong @JianxinMa @hzhwcmhf @fyabc @huybery
各位阿里千问member灯塔级大神，谁能帮忙确认一下这个pr嘛？多谢啦！！！

I think the PR is functionally okay.

thank you for your approvement!!
@simon-mo HI ~ it seems qwen members are all very busy ...

bo is our member. yes, i just noticed that this is merged into transformers. somehow it is not necessary cuz we do not have this setup for our models, but functionally it is ok. i think it is okay to merge it. @eigen2017 @simon-mo

bo is our member. yes, i just noticed that this is merged into transformers. somehow it is not necessary cuz we do not have this setup for our models, but functionally it is ok. i think it is okay to merge it. @eigen2017 @simon-mo

thank you very much！yes it's functional ok and don't change anything if not set.
@simon-mo if anything else needed for merg this pr please tell me,

config.mlp_only_layers should have a default value.


text:
[Kernel] Re-tune Mixtral MoE configurations for FP8 on H100

Re-tune FP8 MoE configs based on https://github.com/vllm-project/vllm/pull/4921
Results for ITL of Mixtral-8x7B-v0.1, prompt-len 1000, gen-len 50:
TP2:
qps = 1: 10.5 ms (was 11.6 ms)
qps = 2: 10.2 ms (was 11.6 ms)
qps = 4: 11.8 ms (was 11.9 ms)
qps = 6: 12.6 ms (was 12.7 ms)

TP4:
qps = 1: 10.9 ms (was 11.7 ms)
qps = 2: 11.1 ms (was 12.0 ms)
qps = 4: 12.0 ms (was 12.2 ms)
qps = 6: 12.9 ms (was 13.0 ms)

As you can see, for Mixtral 8x7B and FP8 one should be running TP2 and not TP4 (unlike for FP16).

 PR Checklist (Click to Expand)
Thank you for your contribution to vLLM! Before submitting the pull request, please ensure the PR meets the following criteria. This helps vLLM maintain the code quality and improve the efficiency of the review process.
PR Title and Classification
Only specific types of PRs will be reviewed. The PR title is prefixed appropriately to indicate the type of change. Please use one of the following:

[Bugfix] for bug fixes.
[CI/Build] for build or continuous integration improvements.
[Doc] for documentation fixes and improvements.
[Model] for adding a new model or improving an existing model. Model name should appear in the title.
[Frontend] For changes on the vLLM frontend (e.g., OpenAI API server, LLM class, etc.)
[Kernel] for changes affecting CUDA kernels or other compute kernels.
[Core] for changes in the core vLLM logic (e.g., LLMEngine, AsyncLLMEngine, Scheduler, etc.)
[Hardware][Vendor] for hardware-specific changes. Vendor name should appear in the prefix (e.g., [Hardware][AMD]).
[Misc] for PRs that do not fit the above categories. Please use this sparingly.

Note: If the PR spans more than one category, please include all relevant prefixes.
Code Quality
The PR need to meet the following code quality standards:

We adhere to Google Python style guide and Google C++ style guide.
Pass all linter checks. Please use format.sh to format your code.
The code need to be well-documented to ensure future contributors can easily understand the code.
Include sufficient tests to ensure the project to stay correct and robust. This includes both unit tests and integration tests.
Please add documentation to docs/source/ if the PR modifies the user-facing behaviors of vLLM. It helps vLLM user understand and utilize the new features or changes.

Notes for Large Changes
Please keep the changes as concise as possible. For major architectural changes (>500 LOC excluding kernel/data/config/test), we would expect a GitHub issue (RFC) discussing the technical design and justification. Otherwise, we will tag it with rfc-required and might not go through the PR.
What to Expect for the Reviews
The goal of the vLLM team is to be a transparent reviewing machine. We would like to make the review process transparent and efficient and make sure no contributor feel confused or frustrated. However, the vLLM team is small, so we need to prioritize some PRs over others. Here is what you can expect from the review process:

 After the PR is submitted, the PR will be assigned to a reviewer. Every reviewer will pick up the PRs based on their expertise and availability.
 After the PR is assigned, the reviewer will provide status update every 2-3 days. If the PR is not reviewed within 7 days, please feel free to ping the reviewer or the vLLM team.
 After the review, the reviewer will put an  action-required label on the PR if there are changes required. The contributor should address the comments and ping the reviewer to re-review the PR.
 Please respond to all comments within a reasonable time frame. If a comment isn't clear or you disagree with a suggestion, feel free to ask for clarification or discuss the suggestion.

Thank You
 Finally, thank you for taking the time to read these guidelines and for your interest in contributing to vLLM. Your contributions make vLLM a great tool for everyone!

@WoosukKwon can you have another look / approve?


text:
version 1.21

Seems that I can't set backup mode anymore
Barrys-MacBook-Pro:~ barrymurphy$ curl -sk --header "Authorization: Bearer TjrnGcgW9qE4NUE69l6ed6GOz01N7nbAoB_CEZbLbyglSpqUr2VxcTd-iqlaby2YzwJw7TZ4EDkCTRZpzfh9zA==" -X POST -d '{"mode":"backup","backup_reserve_percent":15}' http://192.168.33.9:81/api/operation
{"code":500,"error":"RealMode must be either self_consumption or backup","message":"Unable to set operational mode"}Barrys-MacBook-Pro:~ barrymurphy$

seems sending real_mode backup fixed this issue

Confirmed; real_mode resolves the 1.21 issue
Thanks @icepicknz ;)

Real_mode works for backup or self_consumption but not autonomous. It only seems possible to set that through the app now. This is the case regardless which mode the unit is currently in.
Anyone achieved a workaround?
{"code":500,"error":"RealMode must be either self_consumption or backup","message":"Unable to set operational mode"}

authors: icepicknz simwood-simon spoonwzd


text:
Image title attribute is always "# {title }"

![Some image](/some_image.png)
and
![Some image](/some_image.png "With a title")
both yield:
<img src="/some_image.png" alt="Some image" title="# {title }">

Hello @benwalsh,
It looks like you're using a custom render object that has a kind of syntax issue ; the default Redcarpet stack isn't using Ruby interpolation for rendering. I'm giving it a close but if I'm wrong, let us know and we can reopen.

Ah, looks like that's what it is. Thank you.

repo: vmg/redcarpet


text:
attack doesn't like unicode nick names

when you are using non-ascii nick names, the attack plugin fails to generate an insult

actually the issue wasn't unicode, but with spaces

fixed by T83e437e4f4eaaa8af88de53e1ebdac10ac129b70

authors: tsearle vmorsiani

repo: vmorsiani/jabbot


text:
Allow external hostname to be passed through when specifying an ingress.

Is your feature request related to a problem? Please describe.
When specifying additional ingresses for a workshop, internally the hostname of the target is passed in the Host header in case it uses virtual hosts. It would be useful in some cases to have the external workshop ingress hostname passed through for situations where target acts based on that.
Describe the solution you'd like
Add changeOrigin option to ingresses. It should default to true to preserve existing behaviour, but should be able to be set to false in the workshop definition to override behaviour.
Describe alternatives you've considered
No response
Additional information
No response

This isn't going to be trivial to handle as currently use one http-proxy-middleware instance for all ingress proxies. The changeOrigin property can only be set on the proxy object itself, and cannot be overridden by return an override value from the router function on a case by case basis. This is probably why never did it originally. Returning issue to backlog for now.

This done in https://github.com/vmware-tanzu-labs/educates-training-platform/pull/286


text:
CRD to package a software

Describe the problem/challenge you have
Package authors do not have a consistent way to package their software which then can be distributed for consumption.
Describe the solution you'd like
Package authors should be able to create a new Package CR to package their software that can then be distributed for consumption by others. They should be able to define the following in this CR:

Reference to a Package Contents Bundle (containing all the files, OCI images)
Name and version of the package
Authors and description
Describe how to install the package

The Package CR can be deployed on a cluster directly or bundled in a package repository.
Anything else you would like to add:

In progress on the dev-packaging branch.

This is available in release v0.17.0-alpha.1+.
Installation instructions for the latest alpha versions containing this can be found here
Release yaml can be found here
For tracking when this is available in a major release, please follow the epic issue

authors: danielhelfand ewrenn8 vibhas

repo: vmware-tanzu/carvel-kapp-controller


text:
updated go text pkg to v0.3.7 (trivy vulnerability)

Updated go text package (golang.org/x/text) to v0.3.7 as trivy reported vulnerability in v0.3.6

@sethiyash I think you might have missed running go mod vendor or building the package once.

Looks like there are no changes in the vendor files is that expected?
But I believe the verify-no-dirty-files script would have caught that.

Looks like there are no changes in the vendor files is that expected?

I assumed that that's because it's part of Go itself. We can confirm to be sure.

authors: 100mik praveenrewar sethiyash

repo: vmware-tanzu/carvel-kapp


text:
Investigate dropping polling

@bryanl commented on Mon May 27 2019
Polling is hard due to pile ups. Instead of polling, the frontend should register its current context (e.g. viewing a list of lists, viewing a list, viewing an object), and send changes only when there is an update.
This would require additional changes:

timestamps should autoincrement in the front end
possibly would need to be able to stop the current context. i.e. tell the backend to stop sending updates

@bryanl commented on Wed Jun 05 2019
Instead of polling, why not send updates when things change rather than always.

We need a way to detect the change on the backend so we can signal the frontend.

We no longer do external polling and stream updates are only sent when data changes. I think essence of what was captured in this issue is now implemented in Octant as of 0.16 and any further enhancements and improvements should be drafted as separate issues.

repo: vmware-tanzu/octant


text:
Convert clr-modal to cds-modal

What this PR does / why we need it:
Shifts usage of clr-modal to cds-modal. Minor style adjustments and additional accessibility features are gained from this update.
Which issue(s) this PR fixes

See https://kubernetes.slack.com/archives/CM37M9FCG/p1618414173115100

Special notes for your reviewer:
Places of interest;

Build and release information on the top right menu
Preferences
Quick switcher
Confirmation dialog when deleting an object (from datagrids and object summary pages)
a modal opened by a button

Signed-off-by: Sam Foo foos@vmware.com

Hi @GuessWhoSamFoo, I see that you have some conflicting files due to my previous merge, if you need help with the rebase, just let me know

authors: GuessWhoSamFoo ftovaro


text:
Basic Core example doesn´t work

Describe the bug
I have followed these instructions  and there is no modal component in the node_modules @clr/core, so the import fails.

The modal was just recently merged into master and is not yet available in the @clr/core package. It will be published with the 4.0 release.
Try the instructions in our online documentation: https://clarity.design/storybook/core/?path=/docs/documentation-getting-started--page

authors: luisartola mathisscott

repo: vmware/clarity


text:
clrAlertClosed input is not respecting value change

Describe the bug
The alert component's clrAlertClosed input is not respecting a value change.
How to reproduce
https://stackblitz.com/edit/clarity-v4-light-theme-nspfiu
https://clarity-v4-light-theme-nspfiu.stackblitz.io
Steps to reproduce the behavior:

When the app first loads, the alert is closed.
Click the Test button.
In the console, you'll see that this.closed went from true to false.

This was working fine in v2 but when I moved up to v4, I noticed that was no longer the case.
Expected behavior
The alert did not open. It should have opened because its input value changed from true to false.
Versions
App

Angular: 10
Node: 12
Clarity: 4

Device:

Type: Any
OS: Any
Browser: Any
Version: Any

Hello, just checking in to see if this could be looked at? This is impacting applications that utilize the alert component, which is not appearing when it should be. Thank you!

Hi @nightwolfdev - I'm sorry, this hasn't been investigated yet. It may just be a lifecycle issue but I'm not sure without debugging the alert code. As a workaround, you could add an *ngIf to the alerts tied to the open state for each one. Here is a fork of your stackblitz.

authors: hippee-lee nightwolfdev


text:
Styling issue with clr-control-container within form

There is a styling issue with controls within a form.  When the control is focused the focus (underline) extends beyond the control to fill the entire width of the form.
Not in focus:

In focus:

Reproduce via stackblitz.io:
https://clarity-v4-light-theme-iv2dee.stackblitz.io
Angular: 10.2
Project Clarity: 4.0.12
Regards,
David

It's common issue for several different form elements. I think it was already fixed for datalist at least.

I found it:
https://github.com/vmware/clarity/issues/5269
There is an open PR for the above.
Closing this one as duplicate. Thanks for reporting.


text:
[NG] [DO-NOT-MERGE] Datepicker Dev

Notes:

No tests yet.
I wanted to add a demo link but I am facing issues with the KS-App.
New design not finalized yet.

Closing this PR. New PR open: https://github.com/vmware/clarity/pull/1980
Checked with @youdz before closing.


text:
chore(card): add a11y demonstration and social post like card

Signed-off-by: Bogdan Bogdanov bbogdanov@vmware.com
PR Checklist
PR Type
What kind of change does this PR introduce?

[ ] Bugfix
[ ] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] CI related changes
[x] Documentation content changes
[ ] clarity.design website / infrastructure changes
[ ] Other... Please describe:

Does this PR introduce a breaking change?

[ ] Yes
[x] No

#5563


text:
Invalid characters in name error during volume creation.

Hit intermittently
https://ci.vmware.run/vmware/docker-volume-vsphere/648
2017/06/20 18:47:56 START: VolumeAccessTestSuite.TestAccessUpdate
2017/06/20 18:47:56 Creating volume [vol_access_volume_l33x03nwf9uh@datastore1] on VM [192.168.31.81]
2017/06/20 18:48:04 Writing message_by_host1 to file test.txt on volume [vol_access_volume_l33x03nwf9uh@datastore1] from VM[192.168.31.81]
2017/06/20 18:48:09 Reading from file test.txt on volume [vol_access_volume_l33x03nwf9uh@datastore1] from VM[192.168.31.85]
2017/06/20 18:48:10 Destroying volume [vol_access_volume_l33x03nwf9uh@datastore1]

----------------------------------------------------------------------
FAIL: volume_access_test.go:76: VolumeAccessTestSuite.TestAccessUpdate

volume_access_test.go:91:
    c.Assert(err, IsNil, Commentf(out))
... value *exec.ExitError = &exec.ExitError{ProcessState:(*os.ProcessState)(0xc4201263e0), Stderr:[]uint8(nil)} ("exit status 125")
... docker: Error response from daemon: create vol_access_volume_l33x03nwf9uh@datastore1: "vol_access_volume_l33x03nwf9uh@datastore1" includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed. If you intended to pass a host directory, use absolute path.
See 'docker run --help'.

2017/06/20 18:48:11 START: VolumeAccessTestSuite.TestAccessUpdate_R_RW

Initial impression : Looks like volume creation was redirected to local driver instead of vsphere.

This is related to #1401 .... it has introduced the regression. E2E test (basic_test) doesn't check whether config rm --local --confirm removes the DB or not and left out vmgroup created by testcase in the system hence the volume_access_test is running into the issue.
I will raise a separate PR to add more verification steps for basic_test to help us finding root cause easily.
/CC @msterin @shaominchen

So why I am getting it on every CI run on https://github.com/vmware/docker-volume-vsphere/pull/1401 ?

can you please refer https://github.com/vmware/docker-volume-vsphere/pull/1401#issuecomment-310015744? I tried my best to provide detailed analysis.

ah, crossed in the mail :-). Thanks !

authors: msterin pshahzeb shuklanirdesh82

repo: vmware/docker-volume-vsphere


text:
store log in fs, provide api for getting log of a job

@ywk253100 please review as I made update to utils/log.go

The callDepth in line 65 of utils/log/logger.go should be 3.

repo: vmware/harbor


text:
Finish porting and BB article translation

With some questions(marked doubt and for topic that includes case)

Thank you very much for your translation of MIR documentation.  I really appreciate your efforts.

authors: HackLinjiuyue vnmakarov

repo: vnmakarov/mir


text:
CTA图形化回测模块--数据下载问题

环境

操作系统: 如Windows 10或者Ubuntu 18.04
Anaconda版本: 如Anaconda 18.12 Python 3.7 64位
vn.py版本: 如v2.0发行版或者dev branch 20190101（下载日期）

Issue类型
三选一：Bug/Enhancement/Question
预期程序行为
希望使用RQData下载IF88的1小时数据，程序报错；并且CTA回测模块停止运行（不能继续下载其他数据）
原因
RQData里面get_price()函数的frequency字段是对应图形化界面的“K线周期”。
而frequency字段只接受分钟/日线级别，即1m、2m、1d。
建议
1）在图形化界面是“K线周期“上规定只能选择1m、1h、1d；并且在cta_backtester里面的engine上加入对K线周期数值进行转换，如1h要转换成60m，frequency=60m是可以使用RQData下载的。
2）使用try--catch来补足字段填写错误的异常，并且显示在日志上。

已修复

repo: vnpy/vnpy


text:
onorder 和 ontrade的 race condition

环境

操作系统: 如Windows 7或者Ubuntu 16.04
Anaconda版本: 如Anaconda 4.0.0 Python 2.7 32位
vn.py版本: 如v1.7发行版或者dev branch 20170920（下载日期）

Issue类型
三选一：Bug/Enhancement/Question
预期程序行为
实际程序行为
重现步骤
针对Bug类型Issue，请提供具体重现步骤以及报错截图
由于底层接口实现不同，程序无法保证先收到OnOrder 还是 OnTrade， 在CTAengine里面对orderid的删除动作会导致后面来的ontrade逻辑无法完成，需要改在ontrade里面删除orderid ， 理论上canncel和
rejected不受影响，是否不做删除动作也正常不确定，有待进一步测试

请提供下具体错误的截图。
这边检查了下，即使Order先回来，委托和策略的映射关系是通过orderStrategyDict维护的，其中内容并不会因为委托成交或者撤销发生影响。
另外目前CTP的数据推送应该是Order先于Trade到达的

这个是你修改后的CtaEngine代码把，报错时候的信息是什么？？

确认是OKEX的问题了
On Sat, Jun 9, 2018 at 11:22 PM, vn.py notifications@github.com wrote:

Closed #872 https://github.com/vnpy/vnpy/issues/872.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/vnpy/vnpy/issues/872#event-1672296777, or mute the
thread
https://github.com/notifications/unsubscribe-auth/AKuzAgG9PaI4TddohjEQk8qBCl67fLp7ks5t6-glgaJpZM4T_xL1
.

authors: rwayan vnpy


text:
User hakları

Hocam merhaba,
yeni eklediğim modulu X1 user için "edit user permissions" ekranında yetkilendirme yapmama rağmen, X1 user ile login olunca sol menüde modul görünmüyor, nerde hata yapıyorumdur,yardımcı olabilirseniz çok memnun olurum
Teşekkürler

Page deki yetkiye bakın

Çok teşekkürler , çözdüm :)

authors: volkanceylan volkanturk

repo: volkanceylan/Serenity


text:
Tutorial

This kind of library would really benefit from a tutorial how to use it.
There's one here, but it is outdated: https://vorner.github.io/2018/12/09/Spirit-Tutorial.html. There are some examples in the repository (examples/hws-complete.rs is probably the richest one).
Rough ideas:

Start with spirit-less example (simple TCP echo server or message-of-the-day server is enough).
Step by step add support for spirit, logging, metrics, reloading of the listening ports...

To prevent it from getting outdated, it might be a good idea to create a dummy module that has no code, but only the tutorial as one big doc-comment. This way it will not only be in the repository and the examples and code snippets will be compiled & checked, but it'll also render on docs.rs.
If someone wants to give it a try, please claim the issue. I'll provide any help or guidance necessary. If you don't have that much experience with Rust or the spirit library, that's probably even better as you have better idea what you need to know.

@vorner I wouldn't mind giving this a shot.
I feel like mdbook would be a good platform for this. Instead of creating a dummy module, you could make a set of programs inside the example/ folder, then use the include portions of a file function to insert snippets into the guide while using CI to make sure the original examples still compile.

Hello
Thank you that you're looking into this. I'll assign the task to you.
The reason I don't really like mdbook much is, besides having to host it somewhere and setting up the CI and everything, it necessarily covers only the current newest version. I find some value in the fact that even older versions would have their own versions of tutorial relevant for them. Furthermore, books tend to end up quite long and I hoped to something of moderate length. But I'm OK listening to arguments why mdbook would indeed be better.
I've noticed you've already sent a pull request, which is great. I'll find the time to look through it during the next few days.

You raise a really good point about mdbook in that it tends to promote lengthier documents when a shorter one would suffice. I guess the only way to counteract that tendency is by having conciseness as a goal and being quite aggressive about it during code review?
The point about having documentation for previous versions is quite a good one. Using rustdoc essentially gives tutorial versioning to you for free, whereas mdbook tends to promote a "living" document which reflects how things are done on the current master (not latest release). Although with the popularity of tools like cargo outdated I would question how common it is to be stuck on an older version of a library?
Also, having to prefix every line with //!  and having Rust syntax highlighting when you're writing Markdown might get annoying.
Regardless of whether we use mdbook or rustdoc, the tutorial will probably consist of a handful of Markdown pages (one big page might get a bit hard to maintain?) and switching between the two will be a case of inserting/removing //!  from the start of every line.

In terms of the tutorial itself, is there an outline of the areas we'd like to touch on? The original blog post gives us a strong foundation to build from, although looking at the API docs there's quite a large ecosystem of crates which can be plugged into spirit to provide nice things (dipstick, reqwest, hyper, etc).
Also, in general how does spirit work with existing tools like systemd? I'm guessing you'd just need to write a small systend unit file and everything would Just Work, but it's worth mentioning how to install spirit applications as an OS-level service.

I would question how common it is to be stuck on an older version of a library?

I don't know. I've done some changes between versions that need non-trivial porting. If there was a company that had some (unpublished) spirit-something plugins and the company wouldn't be very fast in porting, it might be convenient for new employees to have the relevant tutorial somewhere. But I guess as it being tutorial that is aimed at new users, having to use older version is probably rare.

In terms of the tutorial itself, is there an outline of the areas we'd like to touch on? The original blog post gives us a strong foundation to build from, although looking at the API docs there's quite a large ecosystem of crates which can be plugged into spirit to provide nice things (dipstick, reqwest, hyper, etc).

I'd propose starting with as little as possible, just enough to explain the principles. Using specific crates once the principles are known should be matter of API documentation.
But if it turns out putting more in the tutorial is advantageous, it can be added in later versions.

Also, in general how does spirit work with existing tools like systemd?

I don't think there's a specific support needed. I don't use systemd myself, but from what I know about it, just writing the service file should be enough. For older init systems, spirit-daemonize should give everything needed.
If there's something special needed for systemd, it should be hopefully possible to add as some spirit-systemd crate. If you know about anything specific, would you create an issue?

Hello, @Michael-F-Bryan. I wanted to check ‒ do you plan on continuing with the guide? There was some silence time 😇.

Cleaning up: we do have tutorial now, in the docs.rs pages.

Cleaning up: we do have tutorial now, in the docs.rs pages.

authors: Michael-F-Bryan vorner

repo: vorner/spirit


text:
Frodriguezhervias/agora 1449 view my profilebutton from the dropdown has a broken link

Here is the preview link https://agora-next-21zj02qek-voteagora.vercel.app/
Solves:

"View my profile"button from the dropdown has a broken link
Some minor styles on hover on profile dropdown

Hi @kent, thanks for your comments and video.
It is solved now on this commit https://github.com/voteagora/agora-next/commit/69aa7eaf9a8278350b8d3f193764883933a8a60a you can try it out here https://agora-next-76psz8r4u-voteagora.vercel.app/

repo: voteagora/agora-next


text:
Add support for viewing group slices within embedding panel

Uses the current group slice to properly build a view for loading plot data for the embeddings panel. With this change you will now see corresponding samples that match the following:

the currently selected group slice
exist within the currently selected brain key (visualization)

Note: even with this change there is no simple "1 click" ux for showing multiple slices, even if they meet criteria 2.

Reminder: before merging this will need some regression testing against non-group embeddings plots.


text:
Refactor BasicSide

Description
This refactoring is based on changes and feedback in PR #29. It relies on breaking changes in the model shape spec. I updated Gubu to reflect the new shape spec.
New model:
app: web: part: side: {
  logo: img: string
  section: &: {
    title: string
    item: &: {
      kind: string
      label: string
      icon: string
      path: string
      access: {
        &: boolean
      }
    }
  }
}

@rjrodger I incorporated the feedback you shared.

repo: voxgig/model-react


text:
[skip-ci]release 2.0.0

:stuck_out_tongue:

authors: bastelfreak raphink

repo: voxpupuli/puppet-amanda


text:
donate this module to voxpupuli

since you seem to not maintain this module would you be willing to move this repo to https://github.com/voxpupuli where we could look after this module?

[x] Ask one of the Administrators to add you to the incoming-migrations team.
[x] At that point you can transfer your own repository.
[x] If migrating a module from puppetlabs, re-enable github issues.
[ ] Verify that all webhooks are disabled.
[x] Update the README.md with a description of the deprecation and a link to the new module location.
[x] Release a copy of your module to the ‘puppet’ forge account.
[x] Add the module to our modulesync setup.
- [x ] Add the module to our plumbing repository(plumbing).
[x] Ask an admin to add the collaborators team to the module’s Collaborators & Teams ‘Teams’ list with Write permissions (e.g. https://github.com/voxpupuli/puppet-gitlab/settings/collaboration)).
[x] Execute modulesync for this module.
[x] Create a Jira issue at https://tickets.puppetlabs.com and ask to deprecate the old module (and approve the new one if the old one was approved as well).
[x] Ensure that there isn’t a CONTRIBUTING.md file in the docroot. Our modulesync will place one at .github/CONTRIBUTING.md. Please enhance our existing template if the version in the docroot contains useful parts.
[x] Ensure that the module has a correct LICENSE file in the docroot that matches the mentioned license in the metadata.json.

@dudemcbacon this would be great ?

@attachmentgenie How do I donate the module?

@dudemcbacon => https://voxpupuli.org/docs/migrate_module/
If you are interested in helping maintaining the module in vox pupuli i can ping the right person to get you access. If not i am already a member of the incoming-migrations teams and would be able to take it from there

If you give me the appropriate permissions, I will transfer it. I no longer have the time to maintain it.

@dudemcbacon happy to help out once transferred.

@dudemcbacon you should have an invite to get permissions now.

@dudemcbacon can i help with anything to get the migration going?

@attachmentgenie I have transfered this repo. Apologies it took so long, I had mistakenly transfered it to 'voxpopuli' and didn't realize until a few days ago.

Thanks!

@attachmentgenie will you continue the migration process (
https://voxpupuli.org/docs/migrate_module/ https://github.com/voxpupuli/voxpupuli.github.io/pull/231/files )?

@bastelfreak yes I will!

@bastelfreak can you add the collaborators team to this repo please?

@attachmentgenie done, permissions should be fine now.

@bastelfreak if you could check the webhooks than i think i got through all of them. just need to get github actions to pass.

@bastelfreak if you could check the webhooks than i think i got through all of them. just need to get github actions to pass.

authors: attachmentgenie bastelfreak dudemcbacon traylenator

repo: voxpupuli/puppet-nomad


text:
Terminology

This is an update to the terminology through the README and the tutorials.
In terms of tutorials, note that it is a slight step back from the previous update, in that it seeks to by a temporary stepping stone keeping the tutorials consistent with the code as it is currently, but mapping to the terminology used in the paper. There are also various small fixes and so on.
Once the code moves in terminology, it is a quick revision to move the tutorials to be fully consistent with the new terminology.

Codecov Report

Merging #84 into master will increase coverage by 0.13%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master      #84      +/-   ##
==========================================
+ Coverage    97.3%   97.43%   +0.13%
==========================================
  Files          33       33
  Lines        1482     1482
==========================================
+ Hits         1442     1444       +2
+ Misses         40       38       -2

Impacted Files
Coverage Δ

fooof/group.py
96.93% <0%> (+2.04%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d059820...841fc1d. Read the comment docs.

authors: TomDonoghue codecov-io

repo: voytekresearch/fooof


text:
Setup prettier

In many places code has inconsistent formatting which makes developing
quite hard. In this diff I setup prettier as linter. Users should format
code by their editors. Or with format script.
Tell me if you want to change some rules.
https://prettier.io/docs/en/options.html

Coverage decreased (-0.08%) to 94.811% when pulling bbed04150b22dd36b52c6dd72a69d6560e274713 on TrySound:prettier into 5fc0710b22043ce4f1d473e0a4edecc718898ec0 on vpulim:master.

No idea why coveralls fails.

authors: TrySound coveralls

repo: vpulim/node-soap


text:
Add RMP for Radio Navgation and VHF Radio (Radio Management Panel)

Add Radio Management Panel for Radio Navigation and VHF Radio

[ ] VHF Radio
[ ] Radio Navgation (VOR/ILS/ADF(NDB)...)

after two thousand years...........


text:
Update localization for zh-CN (synchronized with crowdin too)

add new localization strings

uhhh....I want to update the README too. So wait a minute.

done.

repo: vrcx-team/VRCX


text:
Lika 410 new subsystem form

Description
Updated the new & edit subsystem forms to reflect the new designs.
Jira ticket reference: LIKA-###
LIKA-410
What has changed:

Major changes in the new & edit subsystem forms.
Combined access_restriction_level and private fields in dataset.json to one visibility field. Note: the private field is still retained in dataset.json (though it's set to display:none on the actual rendered page) for backwards compatibility reasons.
Some changes in the "backend" code to reflect the aforementioned visibility field change.

Checklist

[X] Contains schema changes.
[ ] Schema changes require migration of existing data.
[ ] Contains migration script for existing data.
[ ] Changes should be covered by tests.
[ ] Changes are covered by tests.

Does this have a design impact ?

[X] Yes
[ ] No

Add screenshots of design changes here if yes.
The whole new & edit subsystem has been altered so the screenshots don't seem to provide any extra value.

after_search and after_show functions can keep using visibility field as those are only used for the resource visibility. Dataset visibility is done using get_dataset_labels function.

authors: Zharktas atteraty

repo: vrk-kpa/api-catalog