text:
Pre-rendered Responses

This project seems very cool, great job :smile:
I'm interested in building something with a React front-end, and a backend using something like JAWS. It seems if all I wanted was a fat static client, I could use JAWS as-is. But it's possible, and in my case, required, that we render initial requests first, to both speed up the initial page load and to support clients without JavaScript.
It seems like it would be trivial to set up a lambda function that takes care of an initial render, but who calls that function? API Gateway seems to be JSON only, so that seems like a non-starter. It seems like in this case that having a server might be unavoidable.
Any thoughts would be greatly appreciated :smile: Thanks!

Lava iris

I came to the same conclusion as you.
Until Amazon extends the capabilities of API Gateway this is impossible, like you said.

It actually looks like it would be possible to return an html response from the API gateway, so this might be possible already.

@devknoll Where do you see this?

@ac360 https://forums.aws.amazon.com/message.jspa?messageID=650072#650072 This + a couple other posts imply that non-JSON requests/responses are possible. Should just be a matter of setting up a text/html response, but I haven't tried it.

It's definitely possible. I wrote a blog post about it: https://medium.com/@devknoll/rendering-react-with-amazon-lambda-e4e85a788257

@devknoll you are the man. When I tried I got stuck on the mapping template and could only ever get json back.
Thanks a ton for taking the time to write down your methods.

API Gateway has an option for setting up Domain Names for an API. Is there another reason you're using Route 53?

You could also shortcut having to make an entry for each possible route by using resources and just making a GET/POST method for /{resourceName} up to the desired depth, just ignoring it, and pulling in the full path to pass along to your router.

@devknoll you are absolutely right, my understanding of API gateways capabilities was wrong.
To anyone else that may be interested here is the relevent link to the api gateway docs for setting up a custom domain.
My biggest concern was that you would no be able to leverage cloudfront by directly exposing the api surface.
However:
"API Gateway is not currently integrated with domain name registrars or managed DNS providers. You must register your custom domain name with a domain name registrar before setting it up in API Gateway. After you’ve set up your domain name and base path mapping(s) in the Amazon API Gateway console, you must update your domain name with your authoritative DNS name server provider to direct traffic to the CloudFront distribution API Gateway created for your domain name. The domain name of this CloudFront distribution is displayed as the “Distribution Domain Name” in the Amazon API Gateway console."
It seems like api gateway automatically sets up a cloudfront distribution, awesome.

@devknoll have you noticed that redeploying the lambda breaks the api created in the method you described? If you delete the method and recreate it using the same method then everything works again.

No, I haven't done much else with it yet since getting the proof of concept running.
I'm a little worried about using something like this in production at the moment, given the range of response times I'm seeing. Warm starts seem to be around ~130ms, while cold starts can be up to 1.3 seconds. Include any APIs that are cold as well, with any initialization they need to do (e.g. connect to a DB or cache) and you're probably looking at at least 2 seconds before the first byte is delivered...
I'd love to see your progress or your own metrics though!

So here is my progress on this so far.
Create react page component for ssr lambda
var React = require('react');

var RootComponent = React.createClass({
  render: function () {
    return (
      React.createElement("html", null,
        React.createElement("head", null,
          React.createElement("title", null, this.props.title)
        ),
        React.createElement("body", null,
          React.createElement("img", {src: this.props.imgSrc})
        )
      )
    )
  }
});

RootFactory = React.createFactory(RootComponent);

exports.handler = function(event, context) {
  var root = RootFactory({
    title: 'test-title',
    imgSrc: 'https://raw.githubusercontent.com/jaws-stack/JAWS-graphics/master/jaws_logo_javascript_aws.png'
  });
  context.succeed(React.renderToStaticMarkup(root));
};

Local output via jaws-run, notice that none of the quotes are escaped

Deploy Lambda, and test, notice quotes are now escaped

Configure api, and test get request, quotes are still escaped

Deploy api and test endpoint in browser
Escaped and quoted string is sent as response

Conclusion
I'm not sure why Lambda seems to be auto escaping and quoting the response string, which causes problems when delivering the html, the browser interprets it as a string instead of markup.

@devknoll Did you check out the Optimization Guide?  There are some good tips in there.  Increasing the Lambda MemorySize is a big one.  I'm not sure what you mean about "APIs being cold".  Do you mean you are seeing that API Gateway also has cold-start issues, aside from Lambda and other services?
Overall, the lambda cold-start issue is a serious problem for JAWS and AWS.  I'm talking to the AWS Lambda team tomorrow and I will bring up the cold-start issue and attempt to get optimization tips from them.  Everything I find out will be added to the Optimization Guide.

I created issue #39 and pull #40 that fixes the inconsistencies I was seeing between the local jaws run and my results on aws.
However it is a problem that the results of every lambda are JSON.strigified, it is going to make returning markup problematic :/

@austinrivas This happens when the mapping template isn't set up correctly. I just tested and can confirm that the resulting text isn't escaped.
@ac360 Excited to hear the results of your chat.
In general, what I mean is:

Hit cold react rendering lambda: +1 second
React rendering lambda hits cold API lambda: +1*n seconds (depending on the shape of your request tree)

I haven't tried adjusting the memory size yet. I wonder if any cold start improvements with it could be attributable to a relative lower number of users with higher memory sizes, leading to functions being cached longer...
In general, I'd love to be able to pay a little extra to have a guaranteed cold start time, just to be able to develop in a JAWS-like paradigm.

Strange, I didn't notice any drastic improvements with higher memory usage through API Gateway. The lambda itself definitely seems to execute quicker with everything over 128mb. But a cold start, even at 1536mb takes at least 1 sec...

@devknoll I had a conversation with the Lambda team today.  I brought up this issue because it is critical for their success and the success of JAWS.  We discussed it for ~20 mins and I learned a lot.  Tonight, I tried to add some further clarity on this issue and optimization tips in the JAWS Optimization page.  Perhaps something in there can help you.
Most importantly, the AWS Lambda team is well aware of this and they are working on several strategies to improve this issue.  They even mentioned that we should see improved Lambda start-up performance within 2 weeks.
As for now, I will continue to make JAWS-Lambda support as optimized as possible.  If you stumble across anything in particular that helps.  PLEASE let me know!!!

I'm a big dumb.
pasting $input.path(‘$’) from the blog post was breaking the aws mapping template. ' from medium are dangerous.
That's what I get for being lazy. Everything is working as expected now and I think we can safely close this one @ac360, unless you want to keep it open for conversations sake.

authors: Surasakeathan ac360 austinrivas devknoll

repo: jaws-stack/JAWS


text:
cond error by using Tracer as a condition

Description
  def take_step(acting_state):
        # TODO when the environment is done, a dummy step should be used to save computation time.
        #  Especially useful for knapsack environment where real number of steps << max number of steps
        #  theoretically possible.

        key, act_key = random.split(acting_state.key, 2)

        action, logprob = policy(acting_state.timestep.observation, act_key)

        problem_size = problem.shape[0]

        result = jax.lax.cond(
        jax.numpy.less(problem_size - 1,action),  #
        lambda _: agents_help(agent_index, action-problem_size, acting_state.timestep.observation, act_key),  #
        lambda _: (action,logprob), #
        operand=0 #
        )

        action, logprob = result

        state, timestep = environment.step(acting_state.state, action)
        info = Information(extras={"logprob": logprob}, metrics={}, logging={})
        acting_state = ActingState(state=state, timestep=timestep, key=key)
        return acting_state, (timestep, info)

System info (python version, jaxlib version, accelerator, etc.)
The two conditional branches of cond will both be executed.action is a Tracer.

It's hard to tell what's going on without more information. Could you edit your question to add a minimal reproducible example of the problem you're seeing, and also paste the full traceback that you're seeing? Thanks!

import sys
sys.path.append('/home/stu12_hzx/poppy-main1')
from dataclasses import field
from typing import TYPE_CHECKING, List, Optional, Tuple, Union
import chex
import omegaconf
if TYPE_CHECKING:
from dataclasses import dataclass
else:
from chex import dataclass
import functools
import time
import math
import acme
import haiku as hk
import hydra
import jax
import jax.numpy as jnp
import numpy as np
import jmp
import optax
import rlax
import random
import numpy
from functools import partial
from sklearn.metrics.pairwise import rbf_kernel, linear_kernel
from sklearn.gaussian_process.kernels import Matern, RationalQuadratic
from chex import Array, PRNGKey
from jax import random
from jumanji.environments.packing.knapsack.types import State as StateKnapsack
from jumanji.environments.routing.cvrp.types import State as StateCVRP
from jumanji.environments.routing.tsp.types import State as StateTSP
from jumanji.types import TimeStep
import poppy.trainers.validation as validation
from poppy.environments.cvrp.types import Observation as ObservationCVRP
from poppy.environments.knapsack.types import Observation as ObservationKnapsack
from poppy.environments.poppy_env import PoppyEnv
from poppy.environments.tsp.types import Observation as ObservationTSP
from poppy.networks import DecoderBase, EncoderBase, Networks
from poppy.utils.checkpoint import (
create_checkpoint_directory,
load_checkpoint,
save_checkpoint,
)
from poppy.utils.data import prepare_problem_batch
from poppy.utils.utils import (
fetch_from_first_device,
fetch_from_devices,
generate_zeros_from_spec,
reduce_from_devices,
)
##get_sorted_agents
from poppy.utils.metrics import get_sorted_agents
State = Union[StateTSP, StateKnapsack, StateCVRP]
Observation = Union[ObservationTSP, ObservationKnapsack, ObservationCVRP]
@dataclass
class TrainingState:  # type: ignore
"""Container for data used during the acting in the environment."""
params: hk.Params
optimizer_state: optax.OptState
num_steps: jnp.int32
key: PRNGKey
extras: Optional[dict] = field(default_factory=dict)

@dataclass
class ActingState:  # type: ignore
"""Container for data used during the acting in the environment."""
state: State
timestep: TimeStep
key: PRNGKey

@dataclass
class Information:  # type: ignore
extras: Optional[dict] = field(default_factory=dict)
metrics: Optional[dict] = field(default_factory=dict)
logging: Optional[dict] = field(default_factory=dict)
def get_optimizer(cfg: omegaconf.DictConfig) -> optax.GradientTransformation:
encoder_mask_fn = functools.partial(
hk.data_structures.map, lambda m, n, p: "encoder" in m
)
decoder_mask_fn = functools.partial(
hk.data_structures.map, lambda m, n, p: "encoder" not in m
)
optimizer = optax.chain(
    optax.masked(
        optax.adamw(
            learning_rate=cfg.encoder.lr,
            weight_decay=cfg.encoder.l2_regularization,
        ),
        encoder_mask_fn,
    ),
    optax.masked(
        optax.adamw(
            learning_rate=cfg.decoder.lr,
            weight_decay=cfg.decoder.l2_regularization,
        ),
        decoder_mask_fn,
    ),
)
optimizer = optax.MultiSteps(optimizer, cfg.num_gradient_accumulation_steps)

return optimizer

def get_networks(cfg) -> Networks:
def encoder_fn(problem: chex.Array):
encoder = hydra.utils.instantiate(cfg.encoder, name="shared_encoder")
return encoder(problem)
def decoder_fn(observation: Observation, embeddings: Array):
    decoder = hydra.utils.instantiate(cfg.decoder, name="decoder")
    return decoder(observation, embeddings)

return Networks(
    encoder_fn=hk.without_apply_rng(hk.transform(encoder_fn)),
    decoder_fn=hk.without_apply_rng(hk.transform(decoder_fn)),
)

def init_training_state(
cfg: omegaconf.DictConfig, networks: Networks, environment: PoppyEnv
) -> TrainingState:
key = random.PRNGKey(cfg.seed)
encoder_key, decoder_key, training_key = random.split(key, 3)
(
    encoder_params,
    decoder_params,
    optimizer_state,
    keys,
    num_steps,
    extras,
) = load_checkpoint(cfg)

environment_spec = acme.make_environment_spec(environment)
_dummy_obs = environment.make_observation(
    *jax.tree_map(
        generate_zeros_from_spec,
        environment_spec.observations.generate_value(),
    )
)

if not encoder_params:
    encoder_params = networks.encoder_fn.init(encoder_key, _dummy_obs.problem)

if not decoder_params:
    embeddings = networks.encoder_fn.apply(encoder_params, _dummy_obs.problem)
    decoder_params = jax.vmap(networks.decoder_fn.init, in_axes=(0, None, None))(
        random.split(decoder_key, cfg.pop_size), _dummy_obs, embeddings
    )

if not keys:
    keys = list(random.split(training_key, cfg.num_devices))

# Distribute parameters over devices as required.
devices = jax.local_devices()
encoder_params = jax.device_put_replicated(encoder_params, devices)
if cfg.rollout.decoder_pmap_axis == "batch":
    # Decoding is parallelised over the batch --> every agent needs to be on every device.
    decoder_params = jax.device_put_replicated(decoder_params, devices)
elif cfg.rollout.decoder_pmap_axis == "pop":
    # Decoding is parallelised over the population --> subset of agent needs to be on every device.
    assert (
            cfg.pop_size >= cfg.num_devices
    ), f"Population of size {cfg.pop_size} too small for distribution over {cfg.num_devices} devices."
    assert (
            cfg.pop_size % cfg.num_devices == 0
    ), f"Population of size {cfg.pop_size} isn't divisibile by number of devices ({cfg.num_devices})."

    def distribute_params(p):
        shp = p.shape
        p = list(p.reshape(cfg.num_devices, shp[0] // cfg.num_devices, *shp[1:]))

        return jax.device_put_sharded(p, devices)

    decoder_params = jax.tree_map(distribute_params, decoder_params)
else:
    raise ValueError(
        f"config.rollout.decoder_pmap_axis of {cfg.rollout.decoder_pmap_axis} not recognised"
    )

params = hk.data_structures.merge(encoder_params, decoder_params)
if not optimizer_state:
    optimizer_state = get_optimizer(cfg.optimizer).init(
        fetch_from_first_device(params)
    )

training_state = TrainingState(
    params=params,
    optimizer_state=jax.device_put_replicated(optimizer_state, devices),
    num_steps=jax.device_put_replicated(num_steps, devices),
    key=jax.device_put_sharded(keys, devices),
    extras=jax.device_put_replicated(extras, devices),
)

return training_state

def generate_trajectory(
decoder_apply_fn,
policy_temperature,
environment,
problem,
embeddings,
params,
start_position,
acting_key,
agent_index,
decoder_params,
pop_size,
):
"""Decode a single agent, from a single starting position on a single problem.
With decorators, the expected input dimensions are:
    problems: [N, problem_size, 2]
    embeddings: [N, problem_size, 128]
    params (decoder only): {key: [K, ...]}
    start_position: [N, K, M]
    acting_key: [N, K, M, 2]
"""
# def select_params_by_index(decoder_params, index):
# # 初始化结果变量
#     result = jax.lax.full(decoder_params[0].shape, 0)  # 使用第一个参数的形状和值初始化结果

# # 定义循环条件
#     def cond_fun(i, _):
#      return i < index

# # 定义循环主体
#     def body_fun(i, current_params):
#      return i + 1, jax.lax.cond(
#         i == index,
#         lambda: current_params,
#         lambda: decoder_params[i],  # 默认情况，选择当前索引的参数
#         operand=None
#      )

# # 运行循环
#     _, result = jax.lax.while_loop(cond_fun, body_fun, (0, result))

#     return result
def get_decoder_params_by_index(all_decoders_params, index):
# 使用 lax.cond 来构建条件选择结构
   def select_params(i):
    print("xxxxx:",i)
    if(i>=6): return None
    return jax.lax.cond(
        jnp.equal(i, index),
        lambda _:  all_decoders_params[i],
        lambda _:  jax.lax.cond(
                          i<=6,
                   lambda _:  all_decoders_params[i],
                   lambda _:  params,
                   operand=None
                   ),
        operand=None
    )
   return select_params(0)
def policy(
        observation: Observation,
        key,
) -> Array:
    logits = decoder_apply_fn(params, observation, embeddings)
    print("asdasdasdshape:",logits.shape)
    print("asdasdasd:",logits)
    logits -= 1e30 * observation.action_mask

    if policy_temperature > 0:
        action = rlax.softmax(temperature=policy_temperature).sample(key, logits)
        print("actionshape:",action.shape)
    else:
        action = rlax.greedy().sample(key, logits)
        print("actionshape:",action.shape)

    logprob = rlax.softmax(temperature=1).logprob(sample=action, logits=logits)

    return action, logprob

def policyWithoutHelp(
        observation: Observation,
        key,
        index,
) -> Array:
    logits = decoder_apply_fn(decoder_params[index], observation, embeddings)
    logits = logits[:problem.shape(1)]
    logits -= 1e30 * observation.action_mask
    if policy_temperature > 0:
        action = rlax.softmax(temperature=policy_temperature).sample(key, logits)
    else:
        action = rlax.greedy().sample(key, logits)
    logprob = rlax.softmax(temperature=1).logprob(sample=action, logits=logits)
    return action, logprob

def agents_help(agent_index,action, observation: Observation, key):
    print("action:",action)
    result = jax.lax.cond(
    jnp.less(action, agent_index)==False,  # 条件
    lambda _:policyWithoutHelp(observation,key,action),
    lambda _:policyWithoutHelp(observation,key,jax.lax.sub(action,1)), # 如果条件为False时的操作
    operand=0 #操作数
    )
    return result

    # count = count_decoders(decoder_params)
    # if  (action-count >= agent_index):
    #     return policyWithoutHelp(observation,key,agent_index-count+1)
    # else:
    #     return policyWithoutHelp(observation,key,agent_index-count)

def f_host(x):
   # 调用 numpy（而不是 jax.numpy）操作：
   return np.sin(x).astype(x.dtype)

def f(x):
  result_shape = jax.ShapeDtypeStruct(x.shape, x.dtype)
  return jax.pure_callback(f_host, result_shape, x)

def take_step(acting_state):
    # TODO when the environment is done, a dummy step should be used to save computation time.
    #  Especially useful for knapsack environment where real number of steps << max number of steps
    #  theoretically possible.
    key, act_key = random.split(acting_state.key, 2)
    action, logprob = policy(acting_state.timestep.observation, act_key)
    problem_size = problem.shape[0]
    result = jax.lax.cond(
    jax.numpy.less(problem_size - 1,action),  # 条件
    lambda _: agents_help(agent_index, action-problem_size, acting_state.timestep.observation, act_key),  # 如果条件为True时的操作
    lambda _: (action,logprob), # 如果条件为False时的操作
    operand=0 # 操作数
    )
    action, logprob = result
    state, timestep = environment.step(acting_state.state, action)
    info = Information(extras={"logprob": logprob}, metrics={}, logging={})
    acting_state = ActingState(state=state, timestep=timestep, key=key)
    return acting_state, (timestep, info)

state, timestep = environment.reset_from_state(problem, start_position, pop_size)

acting_state = ActingState(state=state, timestep=timestep, key=acting_key)

acting_state, (traj, info) = jax.lax.scan(
    lambda acting_state, _: take_step(acting_state),
    acting_state,
    xs=None,
    length=environment.get_episode_horizon(),
)
# traj = []
# info = []

# # 获取 episode horizon 长度
# episode_horizon = environment.get_episode_horizon()

# # 使用 for 循环模拟迭代
# for _ in range(episode_horizon):
#  acting_state, step_info = take_step(acting_state)
#  traj.append(step_info['traj'])  # 假设 take_step 返回的 info 包含 traj
#  info.append(step_info['info'])  # 假设 take_step 返回的 info 包含 info
return acting_state, (traj, info)

def rollout(
cfg: omegaconf.DictConfig,
environment: PoppyEnv,
params: chex.ArrayTree,
networks: Networks,
problems: jnp.ndarray,
start_positions: jnp.ndarray,
acting_keys: jnp.ndarray,
) -> Tuple[ActingState, Tuple[TimeStep, Information]]:
"""Rollout a batch of agents on a batch of problems and starting points.
Args:
    cfg: The rollout config.
    environment: The environment to rollout.
    params: Dictionary of parameters for all Networks.  Encoder params are assumed to be shared
      across all agents, decoder params are assumed to have a leading dimension of shape K.
    networks: The required networks.
    problems: A batch of N problems ([N, problem_size, 2]).
    start_positions: M starting positions for each problem-agent pair ([N, K, M]).
    acting_keys: M acting keys for each problem-agent pair ([N, K, M, 2]).

Returns:
    # TODO
"""
# def count_decoders(params):

#     print("params：",params)

#     for key, value in params.items():
#         if isinstance(value, dict)==False:
#             return value.shape[0]
#     return None
def print_shapes(dictionary):
        for key, value in dictionary.items():
            if isinstance(value, dict):
                print(f"Key: {key}, Shape: (nested dict)")
                print_shapes(value)
            else:
                try:
                    shape = value.shape
                    print(f"Key: {key}, Shape: {shape}")
                except AttributeError:
                    print(f"Key: {key}, Shape: Not applicable (not an array)")

def extract_decoder_params(params, i):
    extracted_params = {}
    for key, value in params.items():
        if isinstance(value, dict):
                # 递归调用以处理嵌套的字典
                extracted_params[key] = extract_decoder_params(value, i)
        else:
            if len(value.shape) == 2:
                extracted_params[key] = value[i:i + 1, :]
            elif len(value.shape) == 3:
                # 如果数组的第二个维度为3，则提取出第i个decoder的部分
                extracted_params[key] = value[i:i + 1, :, :]
    return extracted_params
def count(params):

    c = 0;
    for key, value in params.items():
        if isinstance(value, dict):

            c =  count(value)
        else:
            return value.shape[0]

    return c
# Initialise the embeddings for each problem.
encoder_params, decoder_params = hk.data_structures.partition(
    lambda m, n, p: "encoder" in m, params
)

embeddings = jax.vmap(networks.encoder_fn.apply, in_axes=(None, 0))(
    encoder_params, problems
)

all_decoders_params = []

n = count(decoder_params)

# 通过循环提取每个decoder的参数，并将其存储到all_decoders_params中
for i in range(n):
  decoder_i_params = extract_decoder_params(decoder_params, i)
  all_decoders_params.append(decoder_i_params)
print_shapes(decoder_params)

@functools.partial(jax.vmap, in_axes=(0, 0, None, 0, 0,None))  # over N problems
@functools.partial(jax.vmap, in_axes=(None, None, 0, 0, 0,None), axis_name="agents")  # over K agents
@functools.partial(jax.vmap, in_axes=(None, None, None, 0, 0,None))  # M starting pos.
def generate_trajectory_fn(
        problem, embeddings, decoder_params, start_position, acting_key,decoder_params1
):
    agent_index = jax.lax.axis_index("agents")
    return generate_trajectory(
        networks.decoder_fn.apply,
        cfg.policy.temperature,
        environment,
        problem,
        embeddings,
        decoder_params,
        start_position,
        acting_key,
        agent_index,
        decoder_params1,
        n,
    )

# generate the traj
acting_state, (traj, info) = generate_trajectory_fn(
    problems, embeddings, decoder_params, start_positions, acting_keys,all_decoders_params
)

return acting_state, (traj, info)

def calculate_loss(traj, info, use_poppy='pomo') -> chex.Array:
returns = traj.reward.sum(-1)  # [N, K, M, t] --> [N, K, M]
logprob_traj = info.extras["logprob"].sum(-1)  # [N, K, M, t] --> [N, K, M]
# Calculate advantages.
if returns.shape[-1] > 1 and use_poppy != 'poppy':
    advantages = returns - returns.mean(-1, keepdims=True)
else:
    advantages = returns

if use_poppy == 'poppy_with_pomo_baseline':
    train_idxs = returns.argmax(axis=1, keepdims=True)
    advantages = jnp.take_along_axis(advantages, train_idxs, axis=1)
    logprob_traj = jnp.take_along_axis(logprob_traj, train_idxs, axis=1)

elif use_poppy == 'poppy':
    sorted_idxs = jnp.argsort(returns, axis=1)  # [N, K, M]

    # Get the indices of the maximum and second maximum return
    max_idxs = sorted_idxs[:, -1:, :]  # [N, 1, M]
    second_max_idxs = sorted_idxs[:, -2:-1, :]  # [N, 1, M]

    # Get the maximum and second maximum returns
    max_returns = jnp.take_along_axis(returns, max_idxs, axis=1)  # [N, 1, M]
    second_max_returns = jnp.take_along_axis(returns, second_max_idxs, axis=1)  # [N, 1, M]

    # Calculate advantages as the difference between the maximum and second maximum returns
    advantages = max_returns - second_max_returns  # [N, 1, M]

    # Here, we still take the log probabilities along the max return path
    logprob_traj = jnp.take_along_axis(logprob_traj, max_idxs, axis=1)  # [N, 1, M]

loss = -jnp.mean(advantages * logprob_traj)
return loss

def get_policy(use_half=True, is_norm_layer=False):
"""Get a jmp.Policy.
Args:
    use_half: Whether we are configured to use half (bf16) precision.
    is_norm_layer: Whether this policy should be that for a normalisation layer.

Returns: A configured jmp.Policy.
"""
half = jnp.bfloat16  # only support A100 GPU and TPU for now
full = jnp.float32
if use_half:
if is_norm_layer:
# Compute normalisation layers (batch/layer etc) in full precision to avoid instabilities.
policy = jmp.Policy(compute_dtype=full, param_dtype=full, output_dtype=half)
else:
policy = jmp.Policy(compute_dtype=half, param_dtype=full, output_dtype=half)
else:
policy = jmp.Policy(compute_dtype=full, param_dtype=full, output_dtype=full)
return policy

def set_policy(modules: Union[List[hk.Module], hk.Module], use_half: bool = True):
"""Set the jmp.Policy of the passed modules.
Args:
    modules: A list of (or single) haiku modules.
    use_half: Whether we are configured to use half (bf16) precision.

Returns: None
"""
if type(modules) is not list:
    modules = [modules]
for mod in modules:
    policy = get_policy(use_half, is_norm_layer=False)
    hk.mixed_precision.set_policy(mod, policy)
if use_half:
    # Ensure some layers are always in full precision.
    policy_norm = get_policy(use_half=True, is_norm_layer=True)
    hk.mixed_precision.set_policy(hk.BatchNorm, policy_norm)
    hk.mixed_precision.set_policy(hk.LayerNorm, policy_norm)

def explore(params, b):
def apply_mutation(v):
key = jax.random.PRNGKey(42)
mutation = jax.random.choice(key, jnp.array([1 / math.exp(1 - b), math.exp(1 - b)]))  # 使用 JAX 提供的随机选择函数
return mutation * v
return jax.tree_map(apply_mutation, params)

class Trainer:
def init(
self,
cfg: omegaconf.DictConfig,
logger,
):
self.cfg = cfg
self.logger = logger
self.environment = hydra.utils.instantiate(cfg.environment)
self.networks = get_networks(cfg.networks)
create_checkpoint_directory(cfg, self.logger)
self.training_state = init_training_state(cfg, self.networks, self.environment)
    self.cfg.validation.num_devices = self.cfg.num_devices

    if (
            self.cfg.validation.use_augmentations
            and "Knapsack" in cfg.environment._target_
    ):
        raise ValueError(
            "Knapsack's problem instances cannot be augmented, set "
            "'use_augmentations' in config.validate to False."
        )

    # def get_det(pop, params):

    #     if params['dpp_kernel'] == 'Matern5':
    #         kernel = Matern(length_scale=1.0, length_scale_bounds=(1e-1, 10.0),
    #                         nu=2.5)
    #         K = kernel(pop)
    #     elif params['dpp_kernel'] == 'Matern3':
    #         kernel = Matern(length_scale=1.0, length_scale_bounds=(1e-1, 10.0),
    #                         nu=1.5)
    #     elif params['dpp_kernel'] == 'Exponential':
    #         kernel = Matern(length_scale=1.0, length_scale_bounds=(1e-1, 10.0),
    #                         nu=0.5)  # matern 1/2 = exponential kernel
    #         K = kernel(pop)
    #     elif params['dpp_kernel'] == 'RQ':
    #         kernel = RationalQuadratic()
    #         K = kernel(pop)
    #     elif params['dpp_kernel'] == 'Linear':
    #         pop = normalize2(pop, pop)
    #         row_sums = pop.sum(axis=1)
    #         pop = pop / (row_sums[:, np.newaxis] + 1e-8)
    #         K = linear_kernel(pop)
    #     else:
    #         K = rbf_kernel(pop)

    #     return (np.linalg.det(K))

    def sgd_step(training_state):
        def loss_and_output(params, problems, start_positions, acting_keys):
            state, (traj, info) = rollout(
                self.cfg.rollout,
                self.environment,
                params,
                self.networks,
                problems,
                start_positions,
                acting_keys,
            )

            # Mask logprob's for steps where the environement was done.
            #  - traj.observation.is_done = [0,0,...,0,1,1,...] with the first 1 at the terminal step.
            #  - we want to mask everything *after* the last step, hence the roll & setting the
            #    first step to always (obviously) not be done.
            # TODO: should this be done inside of the rollout function by default?
            is_done = (
                jnp.roll(traj.observation.is_done, 1, axis=-1).at[..., 0].set(0)
            )
            info.extras["logprob"] *= 1 - is_done

            loss = calculate_loss(
                traj, info, use_poppy=self.cfg.loss_objective
            )

            # Log loss and returns. 控制台输出的消息
            info.metrics["loss"] = loss

            episode_return = traj.reward.sum(-1)  # [N, K, M]
            if self.environment.is_reward_negative():
                ret_sign = -1
            else:
                ret_sign = 1
            return_str = self.environment.get_reward_string()

            info.metrics[f"{return_str}"] = (
                    ret_sign * episode_return.max((-1, -2)).mean()
            )
            if self.cfg.pop_size > 1:
                info.metrics[f"{return_str}_rand_agent"] = (
                        ret_sign * episode_return.max(-1).mean()
                )
            if self.cfg.num_starting_positions != 1:
                info.metrics[f"{return_str}_rand_start"] = (
                        ret_sign * episode_return.max(-2).mean()
                )
            if (self.cfg.pop_size > 1) and (self.cfg.num_starting_positions != 1):
                info.metrics[f"{return_str}_rand_agent+start"] = (
                        ret_sign * episode_return.mean()
                )

            return loss, (state, (traj, info))

        # Prepare batch of problems, start positions and acting keys.
        key, problem_key = random.split(training_state.key, 2)

        num_problems = self.cfg.batch_size // self.cfg.num_devices
        duplicate_problems_on_each_device = False

        if self.cfg.rollout.decoder_pmap_axis == "pop":
            num_agents = self.cfg.pop_size // self.cfg.num_devices
        else:
            num_agents = self.cfg.pop_size

        problem_key, start_key, act_key = random.split(problem_key, 3)
        if duplicate_problems_on_each_device:
            # Distribute the problem key from the first device to all devices.
            problem_key = jax.lax.all_gather(problem_key, "i", axis=0)[0]

        problems, start_positions, acting_keys = prepare_problem_batch(
            prng_key=problem_key,
            environment=self.environment,
            num_problems=num_problems,
            num_agents=num_agents,
            num_start_positions=self.cfg.num_starting_positions,
            duplicate_problems_on_each_device=duplicate_problems_on_each_device,
        )

        params = training_state.params
        optimizer_state = training_state.optimizer_state

        grads, (state, (traj, info)) = jax.grad(
            loss_and_output,
            has_aux=True,
        )(
            params,
            problems,
            start_positions,
            acting_keys,
        )

        if self.cfg.num_devices > 1:
            # Taking the mean across all devices to keep params in sync.
            grads = jax.lax.pmean(grads, axis_name="i")

        # TODO: mask optimizer updates for non-trained decoder heads.
        updates, optimizer_state = get_optimizer(self.cfg.optimizer).update(
            grads, optimizer_state, params=params
        )

        params = optax.apply_updates(params, updates)

        training_state = TrainingState(
            params=params,
            optimizer_state=optimizer_state,
            key=key,
            num_steps=training_state.num_steps + 1,
            extras=training_state.extras,
        )

        return training_state, info.metrics

    @functools.partial(jax.pmap, axis_name="i")
    def n_sgd_steps(training_state):
        training_state, metrics = jax.lax.scan(
            lambda state, xs: sgd_step(state),
            init=training_state,
            xs=None,
            length=self.cfg.num_jit_steps,
        )

        # Average metrics over all jit-ted steps.
        metrics = jax.tree_map(lambda x: x.mean(0), metrics)

        return training_state, metrics

    self.n_sgd_steps = n_sgd_steps

###

def exploit_explore(self, a, b):
    def extract_decoder_params(params, i):
        extracted_params = {}
        for key, value in params.items():
            if isinstance(value, dict):
                # 递归调用以处理嵌套的字典
                extracted_params[key] = extract_decoder_params(value, i)
            elif isinstance(value, jnp.ndarray) and len(value.shape) == 4:
                # 如果数组的第二个维度为4，则提取出第i个decoder的部分
                extracted_params[key] = value[:, i:i + 1, :, :]
            elif isinstance(value, jnp.ndarray) and len(value.shape) == 3:
                extracted_params[key] = value[:, i:i + 1, :]
        return extracted_params

    def create_empty_dict_with_structure(d):
        result = {}
        for key, value in d.items():
            if isinstance(value, dict):
                result[key] = create_empty_dict_with_structure(value)
            else:
                result[key] = None
        return result

    def print_shapes(dictionary):
        for key, value in dictionary.items():
            if isinstance(value, dict):
                print(f"Key: {key}, Shape: (nested dict)")
                print_shapes(value)
            else:
                try:
                    shape = value.shape
                    print(f"Key: {key}, Shape: {shape}")
                except AttributeError:
                    print(f"Key: {key}, Shape: Not applicable (not an array)")

    def remove_zero_dims(arr):
        if jnp.any(jnp.array(arr.shape) == 0):
            # 如果存在大小为 0 的维度，则使用 jnp.nonzero() 找到非零维度索引
            nonzero_dims = jnp.nonzero(jnp.array(arr.shape) != 0)
            # 使用切片操作选择非零维度
            arr = arr[nonzero_dims]
        return arr

    params = self.training_state.params
    encoder_params, decoder_params = hk.data_structures.partition(
        lambda m, n, p: "shared_encoder" in m, params
    )
    # for key, value in decoder_params.items():
    #    print(key, ":", value)

    # print("KEY1值为：",len(decoder_params['decoder/mha_dec/key']))
    pop_size = self.cfg.pop_size
    agent_params = []

    for i in range(pop_size):
        agent_params.append(extract_decoder_params(decoder_params, i))
        # print_shapes(extract_decoder_params(decoder_params,i))

    num = self.cfg.pop_size * self.cfg.truncation_ratio

    print("size值为：", len(agent_params))
    for i in range(int(num)):
        agent_params[a[pop_size - i - 1]] = explore(agent_params[a[i]], b)
        # print_shapes(agent_params[i])

    decoder_params = create_empty_dict_with_structure(decoder_params)
    for key in decoder_params.keys():
        # 叠加b
        decoder_params[key]['b'] = jnp.stack([a[key]['b'] for a in agent_params], axis=1).squeeze()
        # 叠加w
        decoder_params[key]['w'] = jnp.stack([a[key]['w'] for a in agent_params], axis=1).squeeze()

    # print("1size值为：")
    # print_shapes(decoder_params)

    # for key, value in agent_params.items():
    #    print(f"  {key}: {value}")
    ##merged_params = tuple(zip(*agent_params))
    ##decoder_params = jax.tree_multimap(lambda *args: args, *merged_params)

    params = hk.data_structures.merge(encoder_params, decoder_params)

    self.training_state.params = params

def train(self):  # noqa: CCR001
    def get_n_steps():
        if self.cfg.num_devices > 1:
            n_steps = fetch_from_first_device(self.training_state.num_steps)
        else:
            n_steps = self.training_state.num_steps
        return n_steps
    def print_shapes(dictionary):
        for key, value in dictionary.items():
            if isinstance(value, dict):
                print(f"Key: {key}, Shape: (nested dict)")
                print_shapes(value)
            else:
                try:
                    shape = value.shape
                    print(f"Key: {key}, Shape: {shape}")
                except AttributeError:
                    print(f"Key: {key}, Shape: Not applicable (not an array)")

    def extract_decoder_params(params, i):
        extracted_params = {}
        for key, value in params.items():

            if isinstance(value, dict):
                # 递归调用以处理嵌套的字典
                extracted_params[key] = extract_decoder_params(value, i)
            else:
                if len(value.shape) == 2:
                 extracted_params[key] = value[i:i + 1, :]
                elif len(value.shape) == 3:
                # 如果数组的第二个维度为3，则提取出第i个decoder的部分
                 extracted_params[key] = value[i:i + 1, :, :]

        return extracted_params

    def log(metrics, key=None):
        metrics["step"] = get_n_steps()
        if self.logger:
            if key:
                metrics = {f"{key}/{k}": v for (k, v) in metrics.items()}
            self.logger.write(metrics)

    set_train_policy = lambda: set_policy(
        modules=[EncoderBase, DecoderBase, hk.MultiHeadAttention],
        use_half=self.cfg.use_half_precision,
    )
    set_validation_policy = lambda: set_policy(
        modules=[EncoderBase, DecoderBase, hk.MultiHeadAttention],
        use_half=self.cfg.validation.use_half_precision,
    )

    set_train_policy()
    # encoder_params, decoder_params = hk.data_structures.partition(
    #    lambda m, n, p: "encoder" in m, params
    # )
    # params = hk.data_structures.merge(encoder_params, decoder_params)
    while get_n_steps() <= self.cfg.num_steps:
        if get_n_steps() % self.cfg.validation_freq == 0:
            set_validation_policy()
            t = time.time()
            training_state = fetch_from_first_device(self.training_state)
            encoder_params, decoder_params = hk.data_structures.partition(
                lambda m, n, p: "encoder" in m, training_state.params
            )

            metrics, episode_return = validation.validate(
                self.cfg.validation, training_state.params
            )
            jax.tree_map(
                lambda x: x.block_until_ready(), metrics
            )  # For accurate timings.
            metrics["total_time"] = time.time() - t
            if self.cfg.num_devices > 1:
                metrics = reduce_from_devices(metrics, axis=0)
            log(metrics, "validate")
            ##
            # a = get_sorted_agents(episode_return, self.cfg.pop_size)
            # print("aaaaa:", a)
            # self.exploit_explore(a, get_n_steps() / self.cfg.num_steps)

            set_train_policy()
            reward_str = self.environment.get_reward_string()
            if self.cfg.checkpointing.save_checkpoint:
                training_state = fetch_from_first_device(
                    self.training_state
                ).replace(key=self.training_state.key)
                save_checkpoint(
                    self.cfg,
                    training_state,
                    self.logger,
                )

                if (
                        metrics[reward_str] > training_state.extras["best_reward"]
                        and self.cfg.checkpointing.keep_best_checkpoint
                ):
                    save_checkpoint(
                        self.cfg,
                        training_state,
                        self.logger,
                        fname_prefix="best_",
                    )

                    extras = self.training_state.extras
                    extras.update(
                        {
                            "best_reward": jnp.ones_like(extras["best_reward"])
                                           * metrics[reward_str]
                        }
                    )

                    self.training_state = TrainingState(
                        params=self.training_state.params,
                        optimizer_state=self.training_state.optimizer_state,
                        num_steps=self.training_state.num_steps,
                        key=self.training_state.key,
                        extras=extras,
                    )

                print(f"Saved checkpoint at step {get_n_steps()}")

        t = time.time()
        self.training_state, metrics = self.n_sgd_steps(self.training_state)
        jax.tree_map(
            lambda x: x.block_until_ready(), metrics
        )  # For accurate timings.

        if self.cfg.num_devices > 1:
            metrics = reduce_from_devices(metrics, axis=0)
        metrics["step_time"] = (time.time() - t) / self.cfg.num_jit_steps
        log(metrics, "train")

this is all code,I use the vmap causing "action" to be a tracer,but i want to use it as a index,if the "action" >"poblem.shape(0)", i will use "action-problem.shape(0)" as a index to the array of dict "decoder_params" for generate another action,but it  errors

i have try much method to make the action to be a index for next action but failed ,can anyway for this to come true?

That can't be all the code – it doesn't include any imports or function calls. Without an executable example and/or a traceback from the issue you're reporting, it will be very difficult to give you any help.

sorry,my fault,the original code is here https://github.com/instadeepai/poppy,I have modified the function I mentioned above in the trainer class.I want to use the action generated by the fuction "policy"  as a index to generate another new action，but it failed a lot times，I‘m  very confused, if you help me sovled this problem,I will give you some reward.Thank you Very much, it's important to me

source: gharchive/issue

repo: jax-ml/jax

license_type: permissive

license_source: github-api


text:
App reload itself after uploading done

I'm not really sure what happen to my client app (frontend)..
If the resolver in the backend perform await processUpload(...) and succeed the client will reload itself.
It's hard to debug what the response from the backend was.
Because using Chrome Devtools, the network tab also reloaded.
Does this package sendRefresh HTTP Header or something else happen in the backend that cause this?
PS: I'm using express with middlewares: (in order) bodyParser.json() bodyParser.urlencoded({ extended: true }) cookieParser(theSecret) compress() methodOverride() session({..}) passport.initialize() passport.session() helmet() cors() expressWinston.logger({..}) rest graphql graphiql

Nothing jumps to mind, there is a lot going on there. Are you able to narrow down the minimal steps needed to reproduce the issue?

I think the problem is in this code below (the resolver):

resolvers: {
Mutation: {
createUser: async (_, { inputData: { picture } }) => {

    try {
      await processUpload({ picture })

      // This line was never called
      return {msg: 'Success'}

    } catch (error) {
      throw error
    }
  },
}
}

after the processUpload() done so is the execution of this resolver.
Anyidea?

Perhaps there is something wrong in processUpload causing it to never resolve.

Aha!
2 problems:

The order of event listener on picture stream (.on('error' and on.('end') is wrong. It listened on those events after writeStream

This is the most important. The uploaded files transfered to my app directory, which cause my app server to reload because it detects file chages

#Damn

authors: Altiano jaydenseric

repo: jaydenseric/apollo-upload-server

license: MIT


text:
localvars 对齐

TODO

https://github.com/jayli/vim-easydebugger/commit/ca12901a8f7f31e8f0f5bbfa6386b3d3363da417?diff=unified

repo: jayli/vim-easydebugger


text:
fix(throttle): use last known arguments for trailing: true call

fixes #94
Cc/ @bcowgill

@bcowgill the specific test for your use case is this one

Published as v0.15.0

source: gharchive/pull-request

license: mit

license_source: bigquery


text:
Coda is a IDE for Web, not a text editor

Text Editors / IDE section:
Perhaps this is an opinion, but I believe the Coda 2 listing belongs a few lines lower on the list, under the IDE section.
(P.S. - thanks for the list, it's pretty good.)

thx!

authors: briandigital jaywcjlove

repo: jaywcjlove/awesome-mac


text:
[WIP] 请求添加备忘清单: OpenSSL

生成公钥和私钥
生成 RSA 密钥
生成 2048 位 RSA 私钥并保存为 KEY1.pem
$ openssl genrsa -out KEY1.pem 2048

生成 4096 位 RSA 私钥，并使用 AES128 加密密钥
$ openssl genrsa -out KEY2.pem -aes128 4096

密钥长度必须置于末尾
如不指定 -out <FILE> 参数则默认输出到 stdout
也支持以下加密算法

-aes128
-aes192
-aes256
-des3
-des

生成 DSA 密钥
生成 DSA 参数文件
$ openssl dsaparam -out DSA-PARAM.pem 1024

用 DSA 参数文件生成 DSA 密钥
$ openssl gendsa -out DSA-KEY.pem DSA-PARAM.pem

将  DSA 参数文件和密钥生成于同一文件
$ openssl dsaparam -genkey -out DSA-PARAM-KEY.pem 2048

生成 Elliptic Curve （椭圆曲线）密钥
生成 EC 参数文件
$ openssl genkey -genparam -algorithm EC -pkeyopt ec_paramgen_curve:secp384r1 -out EC-PARAM.pem

用 EC 参数文件生成 EC 密钥
$ openssl genkey -paramfile EC-PARAM.pem -out EC-KEY.pem

直接生成 EC 密钥
$ openssl genkey -algorithm EC -pkeyopt ec_paramgen_curve:P-384 -out EC-KEY.pem

查看支持的椭圆曲线
$ openssl ecparam -list_curves

推荐的曲线：

prime256v1
secp384r1
secp521r1
分别对应于 P-256/P-384/P-521

检视 RSA、DSA 和 Elliptic Curve （椭圆曲线）的秘钥
【WIP】
生成证书签名请求（CSR）和自签名证书
【WIP】
检视证书签名请求（CSR）和自签名证书
【WIP】
文件格式与格式间转换 (PEM, DER, PFX)
【WIP】
另见

linux-command openssl (wangchujiang.com)

参考的这个cheatsheet
OpenSSL+Cheat+Sheet+-+v1.7+-+Practical+Networking+.net.pdf

@mofelee 已添加欢迎更新补充修正

OpenSSL 官网 (openssl.org)
OpenSSL Cheat Sheet (cheatography.com)
OpenSSL Cheat Sheet (megamorf.gitlab.io)

@mofelee 已添加欢迎更新补充修正

OpenSSL 官网 (openssl.org)
OpenSSL Cheat Sheet (cheatography.com)
OpenSSL Cheat Sheet (megamorf.gitlab.io)

:joy: 你太快了，还好我没完成。。

@mofelee 下次你想搞可以说一下 :)

下次先把标题挂出来，憋一天，然后你一个没忍住就做完了 :smile:

这个也不错 https://github.com/cloudflare/cfssl
https://blog.cloudflare.com/introducing-cfssl/

authors: jaywcjlove mofelee

repo: jaywcjlove/reference


text:
Add a test for FakeRepository.get_hashes()

What's the problem this feature will solve?

FakeRepository.get_hashes() has no test in tests/test_fake_index.py.
Describe the solution you'd like

Add a test to tests/test_fake_index.py:
def test_get_hashes(from_line, repository):
    ireq = from_line("django==1.8")
    assert repository.get_hashes(ireq) == ...

Alternative Solutions

N/A
Additional context

https://github.com/jazzband/pip-tools/pull/383/files#r79352023
https://github.com/jazzband/pip-tools/commit/4d002e517265481d8c550c739cee24f9d895d213

I made a PR for this.

authors: atugushev hramezani

repo: jazzband/pip-tools


text:
Parameterize readme.md and readme.txt or make them case insensitive

Currently if -f tests are case sensitive.

Done in 1.0


text:
Move remaining C linked constants over to libc crate constants

Due to a recent pull request, the rest of the netlink constants used in this library should now be in libc. Move these all over to the libc crate and rip out the C dependency.

Resolved by b8c8130efa67004ba3d696cb0f453f7d862d3beb

repo: jbaublitz/neli


text:
issues installing version 0.99.20

I've previously used qiime2R (in a different R version), and found it really useful. I have to re-install it, and I tried following the instructions posted:
if (!requireNamespace("devtools", quietly = TRUE)){install.packages("devtools")} devtools::install_github("jbisanz/qiime2R")
But I keep getting errors like this:
Error: Failed to install 'unknown package' from GitHub:   Timeout was reached: [api.github.com] Operation timed out after 16369 milliseconds with 0 out of 0 bytes received
The time changes, but it is otherwise the same error message.
I'm using 3.6.3 in RStudio on Windows.
Is there any alternative way to do the installation? Thanks!

Hi Emilie, I'm not sure why this didn't work for you. One idea: perhaps the quotation marks are copy/pasting oddly into your session. Alternatively, there is something off with the network connection of the computer you are trying to install on. You can get around this by downloading the source of the package and then installing manually as detailed here. Let me know if this works for you.

Thanks for the speedy reply! I was able to download the file, and then I tried install.packages("qiime2R-master.zip", repos=NULL, type="source") (after changing my working directory to downloads, as that is where the file is).
The only feedback on the screen was Installing package into ‘C:/Users/Emilie/Documents/R/win-library/3.6’ (as ‘lib’ is unspecified)
But the package doesn't show up in my list of packages, and I can't load the library. So I don't think it worked. I've never installed a package this way before, so maybe I'm missing something else?

It is strange that there would be no error message but you could try manually installing the dependencies using install.packages() and BiocManager::install() and then try reinstalling again. You could also take a step back and look at using devtools::install_github(). Do you have a working install of devtools?

All the dependencies listed for it on github I have installed.  I tried again with install.packages("../Downloads/qiime2R-master.zip", repos=NULL, type="source")
No error message. I can see the folder in C:\Users\Emilie\Documents\R\win-library\3.6, but it does not show up in the list of packages with library().
I tried your suggestion of using devtools. I installed that package today (an downloaded RTools).
> install_github("jbisanz/qiime2R") Downloading GitHub repo jbisanz/qiime2R@master Error in utils::download.file(url, path, method = method, quiet = quiet,  :    cannot open URL 'https://api.github.com/repos/jbisanz/qiime2R/tarball/master'
At least there is an error message this time.

I also just tried with BiocManager
BiocManager::install("qiime2R")
It asked me to update some packages:
Bioconductor version 3.10 (BiocManager 1.30.10), R 3.6.3 (2020-02-29) Installing package(s) 'qiime2R' Installation path not writeable, unable to update packages: class, foreign, lattice, nlme, nnet,   survival Old packages: 'igraph', 'locfit', 'Rcpp'
I said update all, and it did that but then there was this: Warning message: package ‘qiime2R’ is not available (for R version 3.6.3)
Is my R version the issue?

I don't think the issue is with your version of R (and note that qiime2R is not in bioconductor so the BiocMAnager::install("qiime2R") wouldn't be expected to work. Maybe try unzipping master.zip and then running: install.packages("qiime2R-master", repos=NULL, type="source").

Hello Jordan
I moved the zip folder to my default working directory (Documents) and then unzipped it. I then tried to command as suggested:install.packages("qiime2R-master", repos=NULL, type="source")
I got a different error message this time:
Installing package into ‘C:/Users/Emilie/Documents/R/win-library/3.6’ (as ‘lib’ is unspecified) Warning: invalid package 'qiime2R-master' Error: ERROR: no packages specified Warning in install.packages :   installation of package ‘qiime2R-master’ had non-zero exit status
And I can see it did not get installed into the library.
So I tried with the zip file (in Documents this time) and it installed it , but R doesn't recognize it as a valid package. I tried this both in R studio and in R itself. The folder is installed in C:/Users/Emilie/Documents/R/win-library/3.6 but when I try to load it, I get the same message I saw before: library('qiime2R-master')
Error in library("qiime2R-master") :  	  there is no package called ‘qiime2R-master’
I thought maybe the hyphen was the issue, so I changed the folder name and tried again:
library('qiime2R')
Error in library("qiime2R") : ‘qiime2R’ is not a valid installed package
So I think R isn't recognizing it as a valid package for some reason.

Installing package into ‘C:/Users/Emilie/Documents/R/win-library/3.6’ (as ‘lib’ is unspecified) Warning: invalid package 'qiime2R-master' Error: ERROR: no packages specified Warning in install.packages : installation of package ‘qiime2R-master’ had non-zero exit status

Probably means it could not find the folder. If you use list.files() do you see it listed in your working directory? If not use setwd() to change to the directory that contains the qiime2R-master. Idea: try removing the folder from C:/Users/Emilie/Documents/R/win-library/3.6. Then try installing the package as:
devtools::install("C:\\path\to\qiime2R-master")

This ended up working! The folder was within another folder of the same name once it was unzipped, so I think that was giving me some issues before.
In case it is useful to anyone else, the command that worked was
devtools::install(pkg = "C:/Users/Emilie/Documents/qiime2R-master/qiime2R-master")
Thanks so much for your help :)

I had the same problem with R4.0.1 on Windows10.
It was solved when I tried the last command you shared as below.
devtools::install("C:/Users/maruyama/Documents/qiime2R-master/")
Thank you for the very helpful information!

Hi,
I had the same problems using R4.0.4 on Windows10.
However when I tried to use devtools::install("C:/Users/angel/OneDrive/Documents/qiime2R-master/")
I received this message :

Error in (function (command = NULL, args = character(), error_on_status = TRUE,  :
System command 'Rcmd.exe' failed, exit status: 1, stdout + stderr

I have done all the above steps mentioned and none of it seems to work.
Please let me know what I can do to fix this!

authors: angepoch espasov hmaru jbisanz


text:
Error constructing handler for request of type MediatR.IRequestHandler

I have 3 projects, API, Application e Infra.IoC. The settings below work when run by VS Code and Visual Studio. However, when I run inside a docker container, the error occurs:
Error constructing handler for request of type MediatR.IRequestHandler`2[Monefy.Application.Features.AccountFeatures.Commands.CreateAccountCommand,Monefy.Application.Common.CommandResult]. Register your handlers with the container. See the samples in GitHub for examples. Not Handled
Anyone can help me?
ProjectApi -  Program.cs
builder.Services.RegisterMediatRDependencies();
ProjectApplication - AccountCommandHandler.cs
ProjectDependecyResolver - DependecyContainer.cs
public static class ServicesExtensions {     public static void RegisterMediatRDependencies(this IServiceCollection services)     {         services.AddMediatR(typeof(CreateAccountCommandHandler).Assembly);         services.AddTransient<IRequestHandler<CreateAccountCommand, CommandResult>, CreateAccountCommandHandler>();     } }

Is DC.cs part of Infra.IoC project?
Looks like CreateAccountCommandHandler dependency should accept some paramers?
services.AddTransient<IRequestHandler<CreateAccountCommand, CommandResult>, CreateAccountCommandHandler>(); } }
I'd try resolving IRequestHandler<CreateAccountCommand, CommandResult> above the line you're adding MediatR.
Good luck with it

authors: KostetskyiYaroslav cmssantos

repo: jbogard/MediatR


text:
[RHDM-1419] s2i build ignores JVM properties

https://issues.redhat.com/browse/RHDM-1419
@spolti @jakubschwan

why updating it?

@spolti to avoid an override with the 1.6.0 version where the deps are provided by the jdk 1.8, but if isn't an issue we could stay on the same version even with this changes

replaced by https://github.com/jboss-container-images/jboss-kie-modules/pull/388


text:
fix: move counter to right side

What does this PR do and why?
Provide a description of what this PR does. Link to any relevant GitHub issues, Notion tasks or Discord discussions.
Screenshots or screen recordings
If applicable, provide screenshots or screen recordings to demonstrate the changes.
Acceptance checklist

[ ] I have evaluated the Approval Guidelines for this PR.
[ ] (if relevant) I have tested this PR in all supported browsers.
[ ] (if relevant) I have tested this PR in dark mode and light mode (if applicable).

Current dependencies on/for this PR:

main

PR #3895

PR #3896

PR #3897   👈

This comment was auto-generated by Graphite.

@wraeth-eth started a stack merge that includes this pull request via Graphite.

Graphite rebased this pull request as part of a merge.

@wraeth-eth merged this pull request with Graphite.

repo: jbx-protocol/juice-interface


text:
Old allowed IP addresses list.

In Przelewy24 newest documentation (https://www.przelewy24.pl/files/cms/2/przelewy24_specyfikacja_3.1.pdf) it clearly states there are more allowed IP addresses than there are defined.

My bad, there's new documentation version with only the third IP range.


text:
Apple Two factor Authentication update

So if you're a tech geek like me you've probably already seen the news.
'Apple-Craig' said 5 minutes ago on the #WWDC2015 keynote that Apple have remade their TFA system so that "all devices"(iOS 9 devices I'd guess) support it now.
As I have Apple dev access I'll post updates concerning Apple's TFA solution here.

Super excited for this! I'll update too as I find new Apple ID enabled sites and check iOS 9 out. I'll also keep my list up to date, and hopefully, I won't need it anymore!

repo: jdavis/twofactorauth


text:
fix hexadecimal example

https://github.com/jdf/Processing.py-Bugs/issues/296

Maybe I should change it to make clear that in Python mode the hexadecimal notation does not work with color(), only with fill() and stroke()

Currently the wording is a bit awkward. Saying something like:
"In Python Mode, color() does not support hexadecimal notation." might be a bit more blunt and to the point. Since fill() and stroke() both go over using hexadecimal values, I don't know if it's worth mentioning here as well.
I also just noticed that the link to color datatype leads to a 404: https://py.processing.org/reference/color_datatype.html
Not sure if you want to tackle adding that in this PR? Or make a new issue?

That's odd. Builds are now failing on Travis with:

$ curl -L -O http://py.processing.org/processing.py-linux64.tgz
curl: (60) SSL certificate problem: certificate has expired
More details here: http://curl.haxx.se/docs/sslcerts.html

Thank you

authors: FeXd jdf jeremydouglass villares

repo: jdf/processing-py-site


text:
Rendering a Func

Consider the following snippet:
Nustache.Core.Render.StringToString("{{ foo }}, {{ bar }}", new {
            foo = "bar",
            bar = new Func<string>( () => {return "foo";} )
});

//Result:  "bar, System.Func`1[System.String]"

Rather than literally calling .toString() on the Func, should it instead invoke it and call .toString() on the results, similar to how if this was a normal class that contained a method "bar"?  This was tested on the version of Nustache available from NuGet.

Hey,
This is related to #84 and I've done some work so that delegates (Funcs) that are not typed to the new Lambda<object> and Lambda<string, object> types will be wrapped and invoked to the given type allowing them to work exactly the same.
Sorry this took so long to get resolved. It'll go in the next release with the Lambda functionality.

Released in version 1.15.0.4

authors: Romanx dcherman

repo: jdiamond/Nustache


text:
Close #1165: Add shop as Easy

Issue #1165

Thanks for your contribution!

authors: mbtrs tupaschoal

repo: jdm-contrib/jdm


text:
Add Auchan as hard

French supermarket chain.

Thanks for your contribution!

authors: JohnnyJayJay tupaschoal


text:
能否添加自动登陆功能

每次都打账号好麻烦噢。。

@YYYYWD  其实是自动登录的，但是因为LeetCode网站有限制，只要再别处登录之后，之前的session就会被撤销。

authors: YYYYWD jdneo

repo: jdneo/vscode-leetcode


text:
Added Steam Player Number

Added the number of total players online in Steam under the Gaming category

Love this. Thanks!

authors: jdorfman philhartung

repo: jdorfman/awesome-json-datasets


text:
Cannot read property 'StandardUsed' of undefined

Hello,
j'ai l'erreur suivante durant la collecte:
TypeError: Cannot read property 'StandardUsed' of undefined
    at /usr/src/app/server.js:196:65

Je suppose que c'est parce que j'ai la fibre...?
Idéalement cette section pourrait être facultative...

Hello,
Je n'ai pas testé sur une Livebox fibre, n'en disposant pas, mais effectivement, la majorité - voire la totalité - des métriques remontées relèvent de la technologie ADSL et non de la fibre.
Je crains que cet exporter ne vous soit pas d'une grande utilité.

j'ai essayé de trouver une autre mib que dsl.
A priori ce serait eth en fibre, cependant elle ne contient aucune statistique sur les paquets...
Dommage, merci pour quand même pour le projet!

@jeanfabrice pour info, on peut collecter les statistiques réseau en UPnP. Il faut activer le service UPnP de la box, et utiliser cet exporter: https://github.com/yrro/igd-exporter/ (il y a quelques modifications à faire, tickets ouverts à ce sujet)

authors: jeanfabrice phpbg

repo: jeanfabrice/livebox-exporter


text:
Wrong order of events when changing maximumValue and the value prop

When both maximumValue and value prop are adjusted, the thumb's position is set to value on the old min/max slider track, then the slider's track adjusts to the new maximumValue. But order of events should occur the other way around.
e.g. scenario:
slider's initial props are set to {:minimumValue 0 :maximumValue 100 :value 50}
update slider component props to {:value 100 :maximumValue 200}
current result:
slider component draws the thumb at the end of the slider even though the max value is 200; and interacting with the slider will readjust slider's value to new min/max value (e.g. dragging left slightly will recalibrate the thumb's current position to be near the 200 mark)

a workaround is to set new min/max first and slider values afterwards:
this.setState({ sliderMax: NEW_MAX }, () => { this.setState({ sliderValue: VALUE }) }
this is not ideal but fixes the wrong behavior of the slider

repo: jeanregisser/react-native-slider


text:
How to use FindPETSc.cmake?

Hello, I have copied the FindPETSc.cmake to /usr/share/cmake-3.7/Modules (where all the other .cmake files are). However, when I run cmake the following message is shown:
"  By not providing "FindPETSC.cmake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "PETSC", but
CMake did not find one. "
This is how FindPETSc.cmake is called in my top level CMakeLists:
find_package(PETSC)
if(PETSC_FOUND)
include_directories(${Boost_INCLUDE_DIR})
add_definitions( "-DHAS_PETSC" )
endif()
Could you please provide more information?
Thank you.

CMake is case-sensitive so there is a difference between PETSc and PETSC.
Also note that it's unreasonable to ask your users to copy FindPETSc.cmake to the system module path. One would normally include the file (and its dependencies) in your source repository (you can use a git submodule, git subtree, or just copy the files in), then have
project (MyProject)
list (APPEND CMAKE_MODULE_PATH "${MyProject_SOURCE_DIR}/cmake-modules")

Issue solved, thank you.

repo: jedbrown/cmake-modules


text:
Update dnscrypt-resolvers.csv ; new public key for ipredator.se

update Ipredator.se Provider public key

Thanks!
Did you lose the signing key or was it leaked?
Public keys in the dnscrypt-resolvers.csv are usually not supposed to change. New certificates with other expiration dates can be created using the previous key.

I have no connect to this server via dnscrypt and i see on the website a other public key.
https://blog.ipredator.se/ubuntu-dnscrypt-howto.html#configuring_dnscrypt_proxy

Thanks a lot for the pull request, Paul!

license: ISC


text:
JEditableTable中disabledRows属性如何使用，官方文档中没有使用方式

版本号：
2.2
问题描述：
disabledRows属性不知如何使用
截图&代码：

友情提示： 未按格式要求发帖，会直接删掉。

全文搜索，找例子

全文搜索，找例子

全文搜索，找例子

全文搜索例子，貌似是存放在了npm包中，我无法查看

全文搜索，找例子

全文搜索例子，貌似是存放在了npm包中，我无法查看

你好，请问你的问题现在解决了吗？我目前遇到和你相同的问题，不知道如何配置disabledrows。如果你已解决，可以指导我一下吗，谢谢

repo: jeecgboot/jeecg-boot


text:
请问，online表单设置按钮，绑定JAVA增强或SQL增强，无法多选，只能一次选一条

版本号：2.4.5
问题描述：
online表单设置按钮，绑定JAVA增强或SQL增强，无法多选，对次表单数据只能一次选一条，有办法解决吗？一次选择多条，将多条JSON对象的JSON传送给后台
截图&代码：

友情提示： 未按格式要求发帖，会直接删掉。

我也想这样问！支持你

同问

已完成，下个版本发布

请问大佬，下个版本大概什么时候发布，是否可以在2.4.5上增量
------------------ 原始邮件 ------------------
发件人:                                                                                "jeecgboot/jeecg-boot"                                                                                @.>;
发送时间: 2021年8月3日(星期二) 下午3:31
@.>;
@.@.>;
主题: Re: [jeecgboot/jeecg-boot] 请问，online表单设置按钮，绑定JAVA增强或SQL增强，无法多选，只能一次选一条 (#2766)
已完成，下个版本发布
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.


text:
省市区组件的字典怎么维护

版本号：
2.4.5
问题描述：
从area-data更新到china-area-data后，新增的区划显示异常
截图&代码：

友情提示： 未按格式要求发帖，会直接删掉。

jeecg-boot-module-demo/org/jeecg/modules/demo/mock/json/area.json

authors: i-Hu zhangdaiscott


text:
Nullptr dereference in Bot::ChangeWptIndex()

Seems that waypoints.GetPath may return NULL in some cases.'
YaPB just crashed in this function with 0x4 fault address.

Well, its possible ofcourse, but GetPath could return nullptr only  if requested index is out of bounds, but bounds are checked in ChangeWptIndex, or malloc has failed inside waypoint loader (but it's should throw exception).
Another case - broken waypoint file. What map?

Unfortunately, I have only crash log.
*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
Build fingerprint: 'samsung/a5xeltextc/a5xeltextc:6.0.1/MMB29K/A510YDXS3BPK1:user/release-keys'
Revision: '3'
ABI: 'arm'
pid: 16881, tid: 16918, name: EngineThread >>> in.celest.xash3d.hl <<<
signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x4
r0 00000000 r1 da118b84 r2 fffffcd0 r3 fffffcd4
r4 cd604800 r5 da117ef0 r6 da15b000 r7 da118b84
r8 00000000 r9 95a17d9a sl ffffffff fp da118b84
ip 00000000 sp dc8ff240 lr da0e5673 pc da0e5676 cpsr 88070030

backtrace:
#00 pc 00045676 /data/app/in.celest.xash3d.cs16client-2/lib/arm/libyapb_hardfp.so (_ZN3Bot14ChangeWptIndexEi+97)

repo: jeefo/yapb


text:
Bots hear each other's footsteps from a mile away!

When you have just spawned, the bots say 'I hear something'. Which means they hear the footsteps of the enemy from a very long distance.
Another sign that your bots cheat.

fixed in 8448bb3


text:
Epäluku EUR - NaN errors

Thanks for creating this! Looks really promising.
I just tried with a CSV file i got from Coinbase Pro. For some reason the amount of tax (veron määrä) is always shown as zero or as "epäluku":

Every row in the UI shows as either zero or as epäluku. Any idea what goes wrong?
The CSV file has 76 rows.

It seems to be firing requests to Coinbase in an endless loop.

Hmm. Is it possible that you post the CSV rows that are causing the issue?

Here it is. Thanks for looking into this!

Ok, did some debugging. Here's where I'm at:
According to the vero.fi guidance:

The market value is the quoted value of the virtual currency in euros at the time of the transaction

This means that one of the sides of the transaction should either be EUR, or other one of the pairs should be converted to EUR to get the proper taxable value.
The transactions that seem to be causing issues are between USDC and BTC, so the report does not have an EUR value for either.
In this case the website makes a request to Coinbase to ask for a historical known EUR price one of the pairs, in this case USDC. The request being:
https://api.pro.coinbase.com/products/USDC-EUR/candles?start=2020-12-22Z00:00:00.00&end=2020-12-23Z00:00:00.00&granularity=86400
However the response seems to be empty for some reason — perhaps there's no trading data for that day at all for USDC.
I'll try to fix this issue by trying the other pair too (in this case BTC) if the first one does not yield a result.
Also there seems to be an issue with trying to backfill multiple rows at the same time which causes exponential growth in requests. Fixing that too.

That transaction to USDC was actually a mistake I made. I was not able to sell USDC for EUR in Coinbase and looks like they are not offering the rate in their API either. Hopefully there is a way to make this work.

Ok, I think I have fixed both issues now.
Digging even deeper I found another issue: The pairs from your CSV are between two cryptos (and not euros at any point).
So the taxes are calculated based on the purchase price, there must be an initial EUR → crypto purchase so we can calculate gains based on the initial price. That seems to be missing from the CSV, so I assume one of these different scenarios:

The Coinbase Fills CSV report did not contain all purchases before 2020-12-17 OR
You have transferred the initial BTC to Coinbase before 2020-12-17 from some other exchange or wallet

You can resolve the 1. case by downloading a report that contains all transactions. The 2. case can be resolved by adding transactions leading the first Coinbase transaction manually from "Lisää uusi rivi..." button.
So the updated kryptovero.fi will now show an error when it finds that you're missing some transactions, like so:

Looks better already but now it hans up after the progress bar has progressed to the end. Here is the CSV file with my initial purchases added to the top.

I think you have edited the file using a spreadsheet program that changed comma separators (,) to semicolons (;). Also the timestamps have extra : in them. The CSV import is pretty naive at the moment to expect the CSV to be of the specific format that's exported by Coinbase.
A better way than manually editing the CSV file would be to use the form from kryptovero.fi ("Lisää uusi rivi..." button) to add the missing rows first, then import the CSV.
Meanwhile I'll add some extra checks to the CSV parsing to fix the issue and to give a proper error message if the CSV is in an invalid fomat 👍

Thanks! Indeed it was because i had mangled the CSV using the Numbers spreadsheet app. I did some find/replacing in the file and after that it was correctly handled in kryptoveri.fi.
The reason why i have so much rows in my CSV file is that i've used limit orders when buying and selling. These limit orders end up filled in multiple batches. These fills occur within 1-2 seconds so that the timestamps are almost equal. I wonder if I should combine these before feeding them to kryptovero, so that there are less transactions to be reported to the taxman. Any suggestions how this should be done? I guess each of the result rows shown in this app needs to be manually reported to the tax office.
Some kind of limit order combine feature might be a useful for kryptovero.fi :-)

authors: anssip jehna


text:
Update README.md

Github -> GitHub

Thanks but I will close this. It's generated by https://github.com/kefranabg/readme-md-generator

authors: eltociear jellydn

repo: jellydn/dapp-starter


text:
Change typo for "HTML Semantics" on documentation

I've noticed a typo on the documentation in the following sentence:
It applies a few basic styles to the HTML following the HTML *Semenatics.*
This pull request should update the documentation

thanks @emattiazzi

authors: emattiazzi jenil

repo: jenil/chota


text:
Delete application skiapp from this environment

The command jx delete application was run by omedina and it generated this Pull Request

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approvers:
If they are not already assigned, you can assign the PR to them by writing /assign  in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


text:
Delete application bdd-spring-1574109597 from this environment

The command jx delete application was run by  and it generated this Pull Request

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approvers:
If they are not already assigned, you can assign the PR to them by writing /assign  in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


text:
Delete application bdd-spring-1572798306 from this environment

The command jx delete application was run by  and it generated this Pull Request

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approvers:
If they are not already assigned, you can assign the PR to them by writing /assign  in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


text:
Plugin displays the cli login information in output

While executing the plugin, it does a cli login and the spn details including the secret are also displayed in the output. Also there should be a way to suppress the cli command output on jenkins as people would run cli commands to list publish profiles etc and that should not be displayed

Disable print login #4 .

repo: jenkinsci/azure-cli-plugin


text:
Success rates are same for each template

Hi,
We got following screenshot from our customer. They are reporting that success rates are same for all templates. I checked the code here and there seems to be some typos: ch -> th. Could you have a look at that?
Thanks!

I'm the customer named above, and can confirm that I'm still seeing this. The three metrics listed above appear to be repeated for every template rather than being tracked/calculated for each one individually.

It looks like https://github.com/jenkinsci/cloud-stats-plugin/commit/a28517c351ab877cb4167c06d5f848ea87bef29f was meant to fix this issue, although even after installing Cloud Statistics 0.18, I still see the same Overall Success Rate and Current Success Rate values repeated for all templates.

Is this something that will only resolve itself with newly tracked statistics after 0.18 is installed? Shall I wipe out the existing statistics and start over?

The fix in https://github.com/jenkinsci/cloud-stats-plugin/commit/a28517c351ab877cb4167c06d5f848ea87bef29f was incomplete as it addresses sample count but not the success rates. Fixed in a483a5d.

Released as 0.19, it should be in update center soon.

Confirmed as fixed in 0.19. Thank you!

authors: ArchangelSDY medianick olivergondza

repo: jenkinsci/cloud-stats-plugin


text:
koalaman/shellcheck reports an immediate error

Building this docker image from the included Jenkinsfile fails due to an error in the current image of koalaman/shellcheck.
Docker commands that I execute
$ docker run --rm -t koalaman/shellcheck shellcheck -e SC1091 *.sh

Actual result
shellcheck: shellcheck: openBinaryFile: does not exist (No such file or directory)
install-plugins.sh: install-plugins.sh: openBinaryFile: does not exist (No such file or directory)
jenkins.sh: jenkins.sh: openBinaryFile: does not exist (No such file or directory)
plugins.sh: plugins.sh: openBinaryFile: does not exist (No such file or directory)
publish.sh: publish.sh: openBinaryFile: does not exist (No such file or directory)
update-official-library.sh: update-official-library.sh: openBinaryFile: does not exist (No such file or directory)

Expected outcome
Successful shellcheck of the shell scripts

Have you tried a non-dockerized Jenkins and get the expected outcome?
I'm unable to compile koalaman/shellcheck from source, though will continue trying.

Output of docker version
Docker version 17.05.0-ce, build 89658be

Other relevant information

you are not mounting the current dir  in the container, so it has no files
docker run --rm -w /mnt -v `pwd`:/mnt -t koalaman/shellcheck -e SC1091 *.sh

authors: MarkEWaite carlossg

repo: jenkinsci/docker


text:
GCloud SDK auto-installer

I was working on a gcloud-sdk plugin at same time
hopefully adopted a very comparable approach so can propose this "merge" contribution

Thank you for a pull request! Please check this document for how the Jenkins project handles pull requests

@reviewbybees

This pull request originates from a CloudBees employee. At CloudBees, we require that all pull requests be reviewed by other CloudBees employees before we seek to have the change accepted. If you want to learn more about our process please see this explanation.

:bug: - binary compatibility, Windows handking, getSDK() installation. The code needs some improvement.
Another red flag - usage of space indenting in a tab-spaced source code. The coding style is defined by the plugin owner and every contributor should follow it.

authors: jenkinsadmin ndeloof oleg-nenashev reviewbybees

repo: jenkinsci/gcloud-sdk-plugin


text:
Server error in  Jenkins 2.19.1

Hi!,
I tried the API with the las lts ( 2.19.1) and it gives an error server (error code 500).
If I try with the current code in git, then it goes OK.
So, Can we have a new release of the API?
Thank you!

Can you give a little bit more details about what you have tested and which code you have called?

repo: jenkinsci/java-client-api


text:
Integration with credentials plugin

This pull request provides integration with credentials plugin to replace framework principal/secret.

@reviewbybees

This pull request originates from a CloudBees employee. At CloudBees, we require that all pull requests be reviewed by other CloudBees employees before we seek to have the change accepted. If you want to learn more about our process please see this explanation.

Thank you for a pull request! Please check this document for how the Jenkins project handles pull requests

@vinodkone ping

:bee:

:bee:
Assuming the build failure is expected/flaky.

@reviewbybees done

This pull request has completed our internal processes and we now respectfully request the maintainers of this repository to consider our proposal contained within this pull request for merging.

There are some compatibility concerns with this change! Lets fix those.
More importantly, I don't understand the motivation for this change. Can you please include that in the description?
We definitely need feedback from the community regarding this change. Please send an email to the mailing list with the motivation and link to the PR.
Thanks.

Hi,
I think I have addressed your review comments, and I offloaded some items to other pull requests as requested.

:+1:
If I understand this change correctly, it would solve a related issue I have with using the plugin in our environment.
We're standing up a Jenkins master for any dev team that wants one, and they'll run slave jobs on Mesos. I would like to be able to give these teams Admin to configure the masters as they see fit.
The problem is that the secret for the Mesos cluster is right there in plain text on the Jenkins Config page. From a security perspective it would be much better if this were handled like a proper credential. I currently can't let them see this page or they'd have the creds for the Mesos cluster.
Thanks for the good work!

authors: Vlatombe brism jenkinsadmin reviewbybees stephenc tfennelly vinodkone

repo: jenkinsci/mesos-plugin


text:
Optional based on build parameter

We've got Jenkins setup to deploy to a "dev" lane when an option boolean is checked. It'd be nice if we could set notifications to only happen when that checkbox is toggled.

If the slack plugin worked with the flexible publish plugin then this would be easy.  See and follow #41.

Since a contribution in #41 would resolve this I'm going to close this open issue.  Feel free to subscribe to #41 for any updates.
I do not actively develop this plugin.  I will be maintaining pull requests and releases from now on.

repo: jenkinsci/slack-plugin


text:
HmIP-BBL Jalosieaktor: Einstellungen lassen sich nicht speichern

Hallo *,
das o.a. Gerät ist ziemlich neu und lässt sich auch erst seit dem letzten Release anlernen.
Es funktioniert jedoch nur teilweise, da beim speichern der Einstellungen der Vorgang spätestens bei Channel:3 hängen bleibt.
F5 beendet der Vorgang - die vorausgegangenen Kanäle sind auch gespeichert aber insbesondere Channel:7 (Timer) lässt sich so nicht speichern/einrichten.
Mache ich etwas falsch oder muss hier etwas angepaßt werden?
Gruß
Ralf

leider funktioniert auch der Zugriff über die WebUi nicht zuverlässig (auch nach Cache Löschen). Hoch/Runter z.B. erst nach ändern der Lamellenstellung mal gar nicht, mal funktioniert es. Dabei ist es egal ob über ein Script aufgerufen oder direkt in der WebUi

Hallo MgicEye66,
das kann ich nicht bestätigen - via Script (Astro Funktion) funktioniert bei mir alles seht gut.
Auch die Skripte zur manuellen Steuerung (z.B. 0%, 55%, 100%) laufen ohne Probleme.

Allerdings steuere ich damit einen Rolladen - keine Jalousie.

jupp, beim Rolladen keine Probleme, das Problem tritt nur bei Jalousien auf. Habe davon 6 Stück im Einsatz und immer der gleiche Effekt.

mit dem kommenden release sollte dies behoben sein.

Leider besteht der Fehler immer noch.
Mit dem aktuellen Release bleibt der Speichervorgang nun bei Channel 5 oder auch 6 hängen.
Diese beiden Kanäle sind in der Übersich gar nicht vorhanden.
Allerdings Channel 7 - selbiger ist für ein Wochenschaltprogramm vorgesehen damit der Rolladen auch autark mit der Astrofunktion laufen kann.

authors: MgicEye66 banjamann jens-maus

repo: jens-maus/RaspberryMatic


text:
Filter gehen verloren

Describe the bug
Ein gesetzter Filter bei "Startseite > Programme und Verknüpfungen > Programme" in der Spalte "Name" und auch in der Spalte "Beschreibung" ist nach Aufruf von "Bearbeiten" und anschliessender Rückkehr (egal ob mit Abbruch oder OK) der Filter nicht mehr vorhanden.
To Reproduce
Steps to reproduce the behavior:

Menüpunkt Startseite > Programme und Verknüpfungen > Programme aufrufen
Filter in der Spalte "Name" oder Beschreibung setzten
Dann eines der angezeigten Programme mittel "Bearbeiten" öffnen
Schließen des Programms mit "Abbruch" oder "Ok"
Filter ist weg und alle Programm werden angezeigt.

Expected behavior
In der letzten RM Version wurden die Filter beibehalten
Screenshots
If applicable, add screenshots to help explain your problem.
System information (please complete the following information):

Version : RaspberryMatic 3.47.10.20190713
Hardware [RaspberryPi3+ und Tinkerboard S]

Additional context
Add any other context about the problem here.

@psi-4ward Can you reproduce and perhaps fix that filter related issue?

Jetzt bitte in Deutsch.
Aber ich kann dies jederzeit reproduzieren - gerade noch gemacht

Sorry ich habe den Klammeraffen  nicht gesehen...war gar nicht für mich...

@HM-klana Also ich hab das gerade selbst mal mit Chrome probiert und es geht hier problemlos. Ich kann einen Filter unter Programme gesetzt haben und dann in eins der Programme zum editieren wechseln, dann Abbruch drücken und er kehrt problemlos zurück und zeigt weiterhin nur die vom filter ausgewählten Programme an.

Hallo Jens,
habe mir den Chrome gerade runter geladen und nochmal probiert.
Hier geht es auch bei mir...die Filter werden behalten wie es sein soll.
Ich verstehe nur nicht, dass es bei allen anderen Filtern quer durch die Menüs funktioniert und nur bei den Programmen nicht mehr (hat doch schon mal funktioniert)
Gruß
Klaus

Das müsste sich @psi-4ward einmal genauer anschauen warum in dem Falle es nur mit Chrome funktioniert. Du hattest aber auch noch nicht gesagt mit welchem Browser es bei dir NICHT geht.

Safari Version 12.1.2 (14607.3.9)
Mojave macOS 10.14.6 (18G87)

Hab leider keinen Safari. Kannst du mal in die Dev-Console schauen und sehen ob hier vllt iwelche Errors fliegen?
Allgemein muss ich gestehen, dass dieses ganze Filter-Zeug schon etwas hacky ist. Leider ist die WebUI stark in die Jahre gekommen.

Hi, es hat etwas gedauert bis ich die richtigen Anzeigen hatte...kenne mich mit den Dev Tools in Safari gar nicht aus. Es erscheint nirgends ein Fehler oder Warning (siehe Screenshot)
Aber das seltsamste kommt ja noch....ich bin seit fast 40 Jahren in der EDV tätig, aber manchmal muss man nicht alles verstehen...
Nachdem ich die Dev-Konsole offen hatte und versuchte den Fehler nachzustellen ...geht es plötzlich wieder!! ich brech' zusammen! Erklären kann ich es nicht!
Der Fehler bestand schon seit der Installation der aktuellen Firmware RM und jetzt ist es weg.
@jens-maus "Ich wurde sagen ...mach das Ticket zu"
Sorry für viel Wind um nix (oder was auch immer)
Klaus

authors: HM-klana jens-maus psi-4ward


text:
New Use Case: Chaos Proxy

In the vains of New Relics article about chaos testing with graphql we had the idea to use the proxy to add this feature to any graphql api.
Current ideas:
@chaos directive on field or type level to replace original response with error message
@delay directive on field or type level to slow down the whole response as soon as a field/type with this directive is used in the query
This is a placeholder issue and needs to be filled with more info!

I'd rather use something more descriptive like:
"""
injectError will inject an error at the given error rate when a client requests the field
this is useful e.g. for chaos testing clients/consumers
"""
directive @injectError(
    errorRate: Float!
    errorMessage: String!
) on FIELD

enum TimeUnit {
    Milliseconds
    Seconds
}

"""
injectDelay will artificially inject a delay whenever the annotated type is requested
"""
directive @injectDelay(
    delay: Integer!
    unit: TimeUnit!
) on TYPE

What do you think @mvid ?

I would say something like @injectErrorChance or @injectedErrorPercent or something so that it is clear that an error won't occur every time.
For the delay, I don't think a TimeUnit is required. Standardizing on seconds or milliseconds is probably simpler. @injectDelaySeconds or @injectDelayMilliseconds
Otherwise makes sense to me.

I think "injectedErrorPercent" is a good idea.
"@injectDelaySeconds" and "@injectDelayMilliseconds" look simple and concise to me.

Nope looks great

authors: flexzuu jensneuse mvid

repo: jensneuse/graphql-go-tools


text:
I have a problem when I use groupBy and distinct

Hi, first sorry for my english, and congrats for this package, is awesome!
I have a problem when I use groupBy and Distinct, I have saved in a variable a Jenssegers\Mongodb\Relations\HasMany, because I use in differents functions,
$this->periodScores =  $this->scores()->where('period', $this->period);
For example when I want the average of the scores I only use this $this->periodScores->avg('score'); But in one function I need to Use distinct to get a especify values, when I use that $this->periodScores->distinct('student')->get()->count(); Works fine, but when I need to use again $this->periodScores allways use distinct values. So when I need to reuse $this->periodScores, I dont have the values that I need. Maybe I forgot something, please help me!
Thanks from Chile.

Please add details to narrow the answer set or to isolate an issue that can be answered in a few paragraphs. As it's currently written, it’s hard to tell exactly what you're asking.

authors: askmatey mdeorue

repo: jenssegers/laravel-mongodb


text:
🛑 Wanted Gigs is down

In a88e34b, Wanted Gigs (https://www.wanted.co.kr/gigs) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Wanted Gigs is back up in 16cfefd.

repo: jeongsk/upptime


text:
peripheral manager wrapper class

Have a "peripheral manager" to help manage peripherals such as I2C and SPI that may be shared between several chips. The idea is to make it easy to keep track of when to switch on / off the peripherals when needed or not, in particular to save power.
I think it may be a simple wrapper class, something like:
size_t max_nbr_users = 8;
using userid = char[3];

template<typename T>
PeriphepheralManager{
  public:
    begin(userid)
    end(userid)
    peripheral
    force_end
    active_users
  private:
    etc::vector<char[3], max_nbr_users> active_users
}

implementation ongoing


text:
Relicensing

I would like to relicense LibreSubstratum to GPLv3 or Apache 2.0 (I'm still on the fence about it). Because of legal reasons I have to ask contributors for their agreement. Please reply to this issue with "I agree" if you agree or "I don't agree" if not.
Current contributors:
@dimqua
@Poussinou

I agree.

I agree (if GPLv3)

I agree
And I prefer GPLv3 because it's a stronger copyleft license if you want my opinion.

authors: Poussinou TheComputerGuy96 dimqua jereksel

repo: jereksel/LibreSubstratum


text:
[FP]: Wrongly reporting vulnerability CVE-2016-1405 on clamav-client-1.0.1.jar

Package URl
pkg:maven/fi.solita.clamav/clamav-client@1.0.1
CPE
cpe:2.3:a:clamav:clamav::::::::
CVE
CVE-2016-1405
ODC Integration
{"label"=>"Maven Plugin"}
ODC Version
10.0.2
Description
Per NVD Affected component: cpe:2.3:a:clamav:clamav::::::::
This https://nvd.nist.gov/vuln/detail/CVE-2016-1405 impacts only on "openid.net" packages as per NVD and cpe provided. But tool is reporting https://nvd.nist.gov/vuln/detail/CVE-2016-1405 on "fi.solita.clamav/clamav-client@1.0.1" jars as well which is wrong.
From Dependency Check tool team, we need confirmation on these false positives. Could you please validate and confirm?

Correction on above description - Impacts on "clamav:clamav" packages, not on open id.

This is basically the same as #7018 - no need to create one FP per CVE as the problem is the product/CPE matching.

FP handled under #7018

authors: aikebah chadlwilson prabutdr

repo: jeremylong/DependencyCheck


text:
[FP]: osgi-resource-locator detected as eclipse glassfish

Package URl
pkg:maven/org.glassfish.hk2/osgi-resource-locator@1.0.3
CPE
cpe:2.3:a:eclipse:glassfish:1.0.3:*:*:*:*:*:*:*
CVE
CVE-2024-9329
ODC Integration
{"label"=>"Maven Plugin"}
ODC Version
10.0.1
Description
Another dependency under the glassfish org is being detected as eclipse glassfish, which recently had a published CVE released.

Dupe of #7015, already fixed in #7016 but needs another "automated" FP report to be merged before it gets published.

authors: chadlwilson davidmstirn


text:
Add dependency files to repo to enable offline developing.

Right now, all the dependencies are being served from CDNs. This prevents us from developing when offline. So it would be better to have the file served from our servers itself.
Major dependencies:

[ ] jQuery

[ ] Material.css

[ ] Material

[ ] Scroll Reveal.js

@digi0ps We can serve it locally for development, but I think it would rather be better to stick with the CDNs for serving production. How do we isolate it only for development though?

That seems good!
We can do it through an if condition in the base template.

repo: jeremyphilemon/uniqna


text:
Stack overflow in ecma_is_lexical_environment

JerryScript revision
git hash: 392ee717128f6daaf0b94e54fc9512eb26d680e2
Test case
function a() { new new Proxy(a, {}) }
JSON.parse("[]", a)

Execution steps
./jerry poc.js
Build cmd
python tools/build.py --compile-flag="-fsanitize=address"
Stack dump:
ASAN:DEADLYSIGNAL
=================================================================
==180140==ERROR: AddressSanitizer: stack-overflow on address 0x7fff1ed99fe8 (pc 0x5632f4db5751 bp 0x7fff1ed9a000 sp 0x7fff1ed99fe0 T0)
    #0 0x5632f4db5750 in ecma_is_lexical_environment /home/yongheng/SQLFuzzer/jerry_clean/jerry-core/ecma/base/ecma-helpers.c:174
    #1 0x5632f4db5c78 in ecma_get_lex_env_type /home/yongheng/SQLFuzzer/jerry_clean/jerry-core/ecma/base/ecma-helpers.c:274
    #2 0x5632f4ca68fa in ecma_op_resolve_reference_value /home/yongheng/SQLFuzzer/jerry_clean/jerry-core/ecma/operations/ecma-reference.c:276
    #3 0x5632f4d219de in vm_loop.lto_priv.485 /home/yongheng/SQLFuzzer/jerry_clean/jerry-core/vm/vm.c:1036
    #4 0x5632f4cf8723 in vm_execute /home/yongheng/SQLFuzzer/jerry_clean/jerry-core/vm/vm.c:4670
    #5 0x5632f4cf8d17 in vm_run /home/yongheng/SQLFuzzer/jerry_clean/jerry-core/vm/vm.c:4778
    #6 0x5632f4da077e in ecma_op_function_call_simple /home/yongheng/SQLFuzzer/jerry_clean/jerry-core/ecma/operations/ecma-function-object.c:942
    #7 0x5632f4da17ea in ecma_op_function_construct /home/yongheng/SQLFuzzer/jerry_clean/jerry-core/ecma/operations/ecma-function-object.c:1366
    #8 0x5632f4d7a201 in ecma_proxy_object_construct /home/yongheng/SQLFuzzer/jerry_clean/jerry-core/ecma/operations/ecma-proxy-object.c:1779
    #9 0x5632f4da1459 in ecma_op_function_construct /home/yongheng/SQLFuzzer/jerry_clean/jerry-core/ecma/operations/ecma-function-object.c:1264
    #10 0x5632f4d20f37 in opfunc_construct.lto_priv.481 /home/yongheng/SQLFuzzer/jerry_clean/jerry-core/vm/vm.c:863
    #11 0x5632f4cf87d8 in vm_execute /home/yongheng/SQLFuzzer/jerry_clean/jerry-core/vm/vm.c:4697
    #12 0x5632f4cf8d17 in vm_run /home/yongheng/SQLFuzzer/jerry_clean/jerry-core/vm/vm.c:4778
    #13 0x5632f4da077e in ecma_op_function_call_simple /home/yongheng/SQLFuzzer/jerry_clean/jerry-core/ecma/operations/ecma-function-object.c:942
    #14 0x5632f4da17ea in ecma_op_function_construct /home/yongheng/SQLFuzzer/jerry_clean/jerry-core/ecma/operations/ecma-function-object.c:1366
    #15 0x5632f4d7a201 in ecma_proxy_object_construct /home/yongheng/SQLFuzzer/jerry_clean/jerry-core/ecma/operations/ecma-proxy-object.c:1779
    #16 0x5632f4da1459 in ecma_op_function_construct /home/yongheng/SQLFuzzer/jerry_clean/jerry-core/ecma/operations/ecma-function-object.c:1264
    #17 0x5632f4d20f37 in opfunc_construct.lto_priv.481 /home/yongheng/SQLFuzzer/jerry_clean/jerry-core/vm/vm.c:863
    #18 0x5632f4cf87d8 in vm_execute /home/yongheng/SQLFuzzer/jerry_clean/jerry-core/vm/vm.c:4697
    #19 0x5632f4cf8d17 in vm_run /home/yongheng/SQLFuzzer/jerry_clean/jerry-core/vm/vm.c:4778
    #20 0x5632f4da077e in ecma_op_function_call_simple /home/yongheng/SQLFuzzer/jerry_clean/jerry-core/ecma/operations/ecma-function-object.c:942
    #21 0x5632f4da17ea in ecma_op_function_construct /home/yongheng/SQLFuzzer/jerry_clean/jerry-core/ecma/operations/ecma-function-object.c:1366
    #22 0x5632f4d7a201 in ecma_proxy_object_construct /home/yongheng/SQLFuzzer/jerry_clean/jerry-core/ecma/operations/ecma-proxy-object.c:1779
    #23 0x5632f4da1459 in ecma_op_function_construct /home/yongheng/SQLFuzzer/jerry_clean/jerry-core/ecma/operations/ecma-function-object.c:1264
    #24 0x5632f4d20f37 in opfunc_construct.lto_priv.481 /home/yongheng/SQLFuzzer/jerry_clean/jerry-core/vm/vm.c:863
    #25 0x5632f4cf87d8 in vm_execute /home/yongheng/SQLFuzzer/jerry_clean/jerry-core/vm/vm.c:4697
    #26 0x5632f4cf8d17 in vm_run /home/yongheng/SQLFuzzer/jerry_clean/jerry-core/vm/vm.c:4778
    #27 0x5632f4da077e in ecma_op_function_call_simple /home/yongheng/SQLFuzzer/jerry_clean/jerry-core/ecma/operations/ecma-function-object.c:942
    #28 0x5632f4da17ea in ecma_op_function_construct /home/yongheng/SQLFuzzer/jerry_clean/jerry-core/ecma/operations/ecma-function-object.c:1366
    #29 0x5632f4d7a201 in ecma_proxy_object_construct /home/yongheng/SQLFuzzer/jerry_clean/jerry-core/ecma/operations/ecma-proxy-object.c:1779
    #30 0x5632f4da1459 in ecma_op_function_construct /home/yongheng/SQLFuzzer/jerry_clean/jerry-core/ecma/operations/ecma-function-object.c:1264
    #31 0x5632f4d20f37 in opfunc_construct.lto_priv.481 /home/yongheng/SQLFuzzer/jerry_clean/jerry-core/vm/vm.c:863
    #32 0x5632f4cf87d8 in vm_execute /home/yongheng/SQLFuzzer/jerry_clean/jerry-core/vm/vm.c:4697
    #33 0x5632f4cf8d17 in vm_run /home/yongheng/SQLFuzzer/jerry_clean/jerry-core/vm/vm.c:4778

Hi @Changochen!
That's a general stack-overflow issue. The engine has no internal stack-overflow protection by default, but you can enable it by passing --stack-limit X to build.py where X is the limit of stack usage in KB. Whenever this limit is reached a range error is thrown.

Can you tell what value can we use 32bit linux based machine please.

authors: Changochen rerobika vinay54321

repo: jerryscript-project/jerryscript