text:
CBC encrypt and decrypt issue?

Hello there,
I hate to ask such a silly question, but did I miss something to use the AES CBC encryption?
I am using AES128 CBC encryption and modified the main() of "test.c" as follows:
`
int main(void)
{
#ifdef AES128
printf("\nTesting AES128\n\n");
#elif defined(AES192)
printf("\nTesting AES192\n\n");
#elif defined(AES256)
printf("\nTesting AES256\n\n");
#else
printf("You need to specify a symbol between AES128, AES192 or AES256. Exiting");
return 0;
#endif
//test_encrypt_cbc();
//test_decrypt_cbc();
//test_encrypt_ctr();
//test_decrypt_ctr();
//test_decrypt_ecb();
//test_encrypt_ecb();
//test_encrypt_ecb_verbose();
//test_encrypt_ecb_verbose();
uint8_t iv[] = { 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f };

uint8_t i, buf[64], buf2[64];

// 128bit key
uint8_t key[16] = { (uint8_t)0x2b, (uint8_t)0x7e, (uint8_t)0x15, (uint8_t)0x16, (uint8_t)0x28, (uint8_t)0xae, (uint8_t)0xd2, (uint8_t)0xa6, (uint8_t)0xab, (uint8_t)0xf7, (uint8_t)0x15, (uint8_t)0x88, (uint8_t)0x09, (uint8_t)0xcf, (uint8_t)0x4f, (uint8_t)0x3c };

// 512bit text
uint8_t plain_text[64] = { (uint8_t)0x6b, (uint8_t)0xc1, (uint8_t)0xbe, (uint8_t)0xe2, (uint8_t)0x2e, (uint8_t)0x40, (uint8_t)0x9f, (uint8_t)0x96, (uint8_t)0xe9, (uint8_t)0x3d, (uint8_t)0x7e, (uint8_t)0x11, (uint8_t)0x73, (uint8_t)0x93, (uint8_t)0x17, (uint8_t)0x2a,
	(uint8_t)0xae, (uint8_t)0x2d, (uint8_t)0x8a, (uint8_t)0x57, (uint8_t)0x1e, (uint8_t)0x03, (uint8_t)0xac, (uint8_t)0x9c, (uint8_t)0x9e, (uint8_t)0xb7, (uint8_t)0x6f, (uint8_t)0xac, (uint8_t)0x45, (uint8_t)0xaf, (uint8_t)0x8e, (uint8_t)0x51,
	(uint8_t)0x30, (uint8_t)0xc8, (uint8_t)0x1c, (uint8_t)0x46, (uint8_t)0xa3, (uint8_t)0x5c, (uint8_t)0xe4, (uint8_t)0x11, (uint8_t)0xe5, (uint8_t)0xfb, (uint8_t)0xc1, (uint8_t)0x19, (uint8_t)0x1a, (uint8_t)0x0a, (uint8_t)0x52, (uint8_t)0xef,
	(uint8_t)0xf6, (uint8_t)0x9f, (uint8_t)0x24, (uint8_t)0x45, (uint8_t)0xdf, (uint8_t)0x4f, (uint8_t)0x9b, (uint8_t)0x17, (uint8_t)0xad, (uint8_t)0x2b, (uint8_t)0x41, (uint8_t)0x7b, (uint8_t)0xe6, (uint8_t)0x6c, (uint8_t)0x37, (uint8_t)0x10 };

memset(buf, 0, 64);
memset(buf2, 0, 64);

// print text to encrypt, key and IV
printf("CBC encrypt test:\n\n");
printf("plain text:\n \n \n");

for (i = 0; i < 4; ++i)
{
	phex(plain_text + (i * 16));
}

struct AES_ctx ctx;

AES_init_ctx_iv(&ctx, key, iv);

uint8_t encypted_text[128];

printf("\n \n \n");
printf("encrypted and  decrypted text:\n \n \n");
for (i = 0; i < 4; ++i)
{
	//AES_init_ctx_iv(&ctx, key, iv);				// <<<< UNCOMMENT this line -> working
	AES_CBC_encrypt_buffer(&ctx, plain_text + (i * 16), 32);

	//AES_init_ctx_iv(&ctx, key, iv);				// <<<< UNCOMMENT this line -> working
	AES_CBC_decrypt_buffer(&ctx, plain_text + (i * 16), 32);
	phex(plain_text + (i * 16));
}
printf("\n");

return 0;

}`
The problem is that the text which was encrypted and then decrypted isn't identical to the plain text.
For now, I figured out, that I have to call "AES_init_ctx_iv" before each encrypt / decrypt function call. Can someone confirm this as an issue? Or am I doing something wrong?!

Hi @mp1609 and sorry for the slow answer - I am quite busy at the time being.
It doesn't look right that you are encrypting and decrypting at the same time.
I would expect that to ruin the IV since two "sessions" are now using and updating it.
Pull the call to AES_CBC_decrypt_buffer out of the loop, so you have two loops: one encrypting and one decrypting.

Hi kokke, thanks for your response.
Okay I tried what you said and took the AES_CBC_decrypt_buffer into another loop, but unfortunatelly the decrypted text doesn't match the source text. I've zipped my project and attached it here. I only modified the main() in test.c
tinyAES_exe.zip
So the current outpoot looks like this:
plain text:

6bc1bee22e409f96e93d7e117393172a
ae2d8a571e03ac9c9eb76fac45af8e51
30c81c46a35ce411e5fbc1191a0a52ef
f69f2445df4f9b17ad2b417be66c3710

encrypted text:

7649abac8119b246cee98e9b12e9197d
7df76b0c1ab899b33e42f047b91b546f
7df76b0c1ab899b33e42f047b91b546f
7df76b0c1ab899b33e42f047b91b546f

decrypted text:

1997ac7bf0198500f69b2102a9732a13
1637d7ed30fd0022df76845dc6854d4a
1637d7ed30fd0022df76845dc6854d4a
1637d7ed30fd0022df76845dc6854d4a

@mp1609 try re-initializing the IV from the same source between the encryption and decryption loops

@dwinant Thanks, but the issue remains the same:
Tried following:
int main(void)
{

#ifdef AES128
    printf("\nTesting AES128\n\n");
#elif defined(AES192)
    printf("\nTesting AES192\n\n");
#elif defined(AES256)
    printf("\nTesting AES256\n\n");
#else
    printf("You need to specify a symbol between AES128, AES192 or AES256. Exiting");
    return 0;
#endif

	//test_encrypt_cbc();
	//test_decrypt_cbc();
	//test_encrypt_ctr();
	//test_decrypt_ctr();
	//test_decrypt_ecb();
    //test_encrypt_ecb();
    //test_encrypt_ecb_verbose();
	//test_encrypt_ecb_verbose();

	uint8_t iv[] = { 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f };

	uint8_t i, buf[64], buf2[64];

	// 128bit key
	uint8_t key[16] = { (uint8_t)0x2b, (uint8_t)0x7e, (uint8_t)0x15, (uint8_t)0x16, (uint8_t)0x28, (uint8_t)0xae, (uint8_t)0xd2, (uint8_t)0xa6, (uint8_t)0xab, (uint8_t)0xf7, (uint8_t)0x15, (uint8_t)0x88, (uint8_t)0x09, (uint8_t)0xcf, (uint8_t)0x4f, (uint8_t)0x3c };

	// 512bit text
	uint8_t plain_text[64] = { (uint8_t)0x6b, (uint8_t)0xc1, (uint8_t)0xbe, (uint8_t)0xe2, (uint8_t)0x2e, (uint8_t)0x40, (uint8_t)0x9f, (uint8_t)0x96, (uint8_t)0xe9, (uint8_t)0x3d, (uint8_t)0x7e, (uint8_t)0x11, (uint8_t)0x73, (uint8_t)0x93, (uint8_t)0x17, (uint8_t)0x2a,
		(uint8_t)0xae, (uint8_t)0x2d, (uint8_t)0x8a, (uint8_t)0x57, (uint8_t)0x1e, (uint8_t)0x03, (uint8_t)0xac, (uint8_t)0x9c, (uint8_t)0x9e, (uint8_t)0xb7, (uint8_t)0x6f, (uint8_t)0xac, (uint8_t)0x45, (uint8_t)0xaf, (uint8_t)0x8e, (uint8_t)0x51,
		(uint8_t)0x30, (uint8_t)0xc8, (uint8_t)0x1c, (uint8_t)0x46, (uint8_t)0xa3, (uint8_t)0x5c, (uint8_t)0xe4, (uint8_t)0x11, (uint8_t)0xe5, (uint8_t)0xfb, (uint8_t)0xc1, (uint8_t)0x19, (uint8_t)0x1a, (uint8_t)0x0a, (uint8_t)0x52, (uint8_t)0xef,
		(uint8_t)0xf6, (uint8_t)0x9f, (uint8_t)0x24, (uint8_t)0x45, (uint8_t)0xdf, (uint8_t)0x4f, (uint8_t)0x9b, (uint8_t)0x17, (uint8_t)0xad, (uint8_t)0x2b, (uint8_t)0x41, (uint8_t)0x7b, (uint8_t)0xe6, (uint8_t)0x6c, (uint8_t)0x37, (uint8_t)0x10 };

	memset(buf, 0, 64);
	memset(buf2, 0, 64);

	// print text to encrypt, key and IV
	printf("CBC encrypt test:\n\n");
	printf("plain text:\n \n \n");

	for (i = 0; i < 4; ++i)
	{
		phex(plain_text + (i * 16));
	}

	struct AES_ctx ctx;

	AES_init_ctx_iv(&ctx, key, iv);

	uint8_t encypted_text[128];

	AES_init_ctx_iv(&ctx, key, iv);
	printf("\n \n \n");
	printf("encrypted text:\n \n \n");
	for (i = 0; i < 4; ++i)
	{
		AES_CBC_encrypt_buffer(&ctx, plain_text + (i * 16), 32);
		phex(plain_text + (i * 16));
	}
	printf("\n");

	// print text to decrypt
	printf("\n \n \n");
	printf("decrypted text:\n \n \n");

	AES_init_ctx_iv(&ctx, key, iv);
	for (i = 0; i < 4; ++i)
	{
		AES_CBC_decrypt_buffer(&ctx, plain_text + (i * 16), 32);
		phex(plain_text + (i * 16));
	}
	printf("\n");

    return 0;
}

plain text:

6bc1bee22e409f96e93d7e117393172a
ae2d8a571e03ac9c9eb76fac45af8e51
30c81c46a35ce411e5fbc1191a0a52ef
f69f2445df4f9b17ad2b417be66c3710
encrypted text:
7649abac8119b246cee98e9b12e9197d
7df76b0c1ab899b33e42f047b91b546f
7df76b0c1ab899b33e42f047b91b546f
7df76b0c1ab899b33e42f047b91b546f
decrypted text:
1004fcc386ad013eac8a9dde1506041f
69e216649fedbac3040cfa4e6aec8f47
6df295cc98109e8a9ac16792a0105e7f
69e216649fedbac3040cfa4e6aec8f47

@mp1609 I think you need to change the length parameter from 32 to 16 in your loops.  With a length of 32 it is encrypting most of your data twice, plus touching data past the end of your plain_text array.
When I made that change, in addition to resetting the IV before the decryption, your code ran and gave the correct decrypted text.

@dwinant : Thanks for your answer and also for this cool support!
That worked for me, have a nice weekend.

Thank you very much @dwinant for helping closing this issue :) Very much appreciated.
Have a nice weekend.

i got error when i try to compile test.c is there any documentation for this library .

authors: Mohammedyasein dwinant kokke mp1609

repo: kokke/tiny-AES-c

license: Unlicense


text:
🛑 Leo Pizza is down

In 836f8bb, Leo Pizza (https://leo-pizzaservice.de) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Leo Pizza is back up in c4d933c after 8 minutes.

source: gharchive/issue

repo: koko-io/up

license: MIT

license_type: permissive

license_source: github-api


text:
Swiping from image to video is lagging

My DCIM folder contains fix of images and videos . When I swipe from image to a video there is a stuck or lag .
Images to images swipe has no problem.
Images to video and video images has this problem.

Closing this issue because the issue is no longer available after 1.5.2 update.

repo: kollerlukas/Camera-Roll-Android-App

license_source: bigquery


text:
Rework network interation in JS code

Right now it's too much of callbacks.
Something like async/await-based client class would be much more appropriate.

Done in V2 project

repo: komodorio/helm-dashboard


text:
Give user a better feedback when commands are executed

use spinner on all commands that modify system (spinner waits state change when possible)
improve deploy tracking

I'll give this a spin tomorrow morning

@jakolehm would it be possible to have automated docker builds of feature branches like these for easier testing (with contena for example)

We install kontena cli with Gemfile for the project so the old gemset will conflict with my symlinked "devtena"...

Rebased against master and refactored/unified spinner & error messages

@nevalla ping

Looks good!

source: gharchive/pull-request

authors: jakolehm matti nevalla

repo: kontena/kontena


text:
Release v1.4.0.pre7

Merge commit from master

travis CLI omnibus build https://travis-ci.org/kontena/kontena/jobs/277350268
travis release build https://travis-ci.org/kontena/kontena/jobs/277350271
https://hub.docker.com/r/kontena/agent/tags/
https://hub.docker.com/r/kontena/server/tags/
https://rubygems.org/gems/kontena-cli/versions/1.4.0.pre7
https://bintray.com/kontena/ubuntu/kontena-agent/1.4.0.pre7
https://bintray.com/kontena/ubuntu/kontena-server/1.4.0.pre7


text:
(bug): Fetch list (cluster/storage/plan) fails when a secret is missing

When you delete a secret without deleting the migResource, the fetch request will fail silently. We need to add some sort of retry logic or alert to the user when this occurs. _(This issue only happens when you delete a secret manually from the command line. Nothing in the UI solely deletes the secret)
_

TODO: https://github.com/konveyor/mig-ui/blob/master/src/app/common/duck/sagas.ts#L41

repo: konveyor/mig-ui


text:
[RFR] Stakeholder group CRUD test

Signed-off-by: Karishma Punwatkar kpunwatk@redhat.com
new file:   mta/tests/migration/controls/test_stakeholder_group.py
modified:   pytest.ini

https://polarion.engineering.redhat.com/polarion/redirect/project/MTAPathfinder/workitem?id=MTA-325 TC implemented

repo: konveyor/tackle-api-tests


text:
koord-yarn: add pkg dir for YARN related modules

Ⅰ. Describe what this PR does

add some pkg dirs for YARN related modules, see https://github.com/koordinator-sh/koordinator/discussions/1297 for detailed desgins.
Ⅱ. Does this pull request fix one issue?

Ⅲ. Describe how to verify it
Ⅳ. Special notes for reviews
V. Checklist

[ ] I have written necessary docs and comments
[ ] I have added necessary unit tests and integration tests
[ ] All checks passed in make test

/verify-owners

Is it more reasonable to create a yarn directory specifically, and then create a copilot/operator below it? The subsequent yarn-specific code can be converged. This keeps the same style as webhook/descheduler/scheduler/koordlet.

/lgtm

operator is a little weird name as a sub package under pkg, which is usually the name of the whole project or component. How about controller or controllers?

authors: FillZpp eahydra hormes zwzhang0107

repo: koordinator-sh/koordinator


text:
ERROR: could not unmarshal event: unknown PrivateNetworkRequestPolicy value

for some reasons all request fails with
ERROR: could not unmarshal event: unknown PrivateNetworkRequestPolicy value

Could you please help me to solve that?

Could you show what the input looks like?

@idoo can I close this issue? I haven't got any additional information from you.

authors: idoo kosmosec

repo: kosmosec/proto-find


text:
Turn VisUI into a standalone UI toolkit

VisUI (and Scene2D-UI) is an amazing piece of software. It would be even better if you turned it into and advertise it as a standalone UI toolkit that covers ALL major platforms.
It may be "healthy" to abstract away libgdx rendering and its reliance on libgdx internal state. Libgdx would be "just" the main "supplier" for VisUI rendering. It should be relatively easy to port VisUI to another "supplier", let's say Java2D for example.
VisUI would be THE universal UI toolkit for ALL major platforms. Not just the ui for libgdx games where most people don't even consider using it.

You are underestimating the complexity of doing this.
Also why not go ahead with a proof of concept?
This would show that it really is relatively easy to port VisUI to another "supplier".

I am not advocating to implementing another "supplier". I am advocating primarily to turn VisUI into a standalone UI toolkit and advertise it as such. While disentangling VisUI from libGDX one could at the same time think about a clean interface to implement new "suppliers". LibGDX would be an implementation of that interface.
VisUI is such a genius project and lives in the shadow of libGDX. Moreover if VisUI gets more attention, libGDX can only benefit.

VisUI is tied too closely to libGDX for this to be realistic. If I ever were to do a universal UI toolkit I'd start from scratch probably using some vector renderer.

authors: intrigus kotcrab suthcark

repo: kotcrab/vis-ui


text:
Redefinition of NegativeInt

Description
Similarly to the issue #228, we would like to redefine the NegativeInt with a minimal API.
Dependencies
This issue is blocked by the following ones:

[ ] #227

Checklist

[ ] Define the API with an interface.
[ ] Implement and test the builder and the toInt function, then update the public API binaries and check the generated documentation.
[ ] Implement and test the equals function, then update the public API binaries and check the generated documentation.
[ ] Implement and test the hashCode function, then update the public API binaries and check the generated documentation.
[ ] Implement and test the toString function, then update the public API binaries and check the generated documentation.
[ ] Make the type serializable.
[ ] Make the type comparable.
[ ] Refactor.
[ ] Update the Unreleased section in changelog.

We will not break the API doing this: it brings no tangible value to the library.

repo: kotools/types


text:
Document the WebAssembly story

As far as I can tell this library currently does not support wasm. But since Rust now supports wasm on stable it would be useful if documentation existed somewhere detailing this libraries future plans for supporting wasm, or if it already does and I missed that, there should be docs on how to use this library with wasm.
Edit: After fiddling a bit, I have determined that at the very least the todomvc example can run under wasm. This just means that it is unclear if wasm is actually supported or if this is something that is not guarenteed to work going forward.

I managed to make a basic example work, but not todomvc:
$ wargo build --release
$ cp target/wasm32-unknown-emscripten/release/deps/todomvc-*.{js,wasm} static/  # Hacky, I know :)
$ mv static/todomvc-*.js static/todomvc.js
$ mv static/todomvc-*.wasm static/todomvc.wasm

Then I modified <script src="js/app.js"></script> into(inspiration):
<script type='text/javascript'>
	var Module = {};

	fetch('todomvc.wasm').then(response =>
		response.arrayBuffer()
		).then(buffer => {

		Module.wasmBinary = buffer;

		var script = document.createElement('script');
		script.src = "todomvc.js";
		script.onload = function() {
		console.log("Emscripten boilerplate loaded.")
		}
		document.body.appendChild(script);
	});
</script>

This basic page would work then:
<!doctype html>
<html lang="en">
	<head>
		<meta charset="utf-8">
		<title>stdweb • Alert</title>
	</head>
	<body>
		<input class="the-input">
		<script type='text/javascript'>
			var Module = {};

			fetch('todomvc.wasm').then(response =>
				response.arrayBuffer()
				).then(buffer => {

				Module.wasmBinary = buffer;

				var script = document.createElement('script');
				script.src = "todomvc.js";
				script.onload = function() {
				console.log("Emscripten boilerplate loaded.")
				}
				document.body.appendChild(script);
			});
		</script>
	</body>
</html>

For this code(did not plan initially  to post the code: at this point I realise I should better push the code somewhere instead of creating a big comment, but this is the last part):
fn main() {
    stdweb::initialize();

    let state = window().local_storage().get("state").and_then(|state_json| {
        serde_json::from_str(state_json.as_str()).ok()
    }).unwrap_or_else(State::new);

    let state = Rc::new(RefCell::new(state));

    let message: InputElement = document().query_selector(".the-input").unwrap().try_into().unwrap();
    message.add_event_listener(enclose!( (state, message) move |event: KeypressEvent| {
        if event.key() == "Enter" {
            event.prevent_default();

            let msg: String = message.value().try_into().unwrap();
            js! {
                alert( @{msg} );
            }
        }
    }));

    stdweb::event_loop();
}

I created a basic example of stdweb + wasm32: https://github.com/lilianmoraru/stdweb-alert-example
I did something wrong or the support is not great(stdweb or/and Rust + emscripten) but while trying to make todomvc to work, I had all kinds of missing symbols issues or things like that(even with the basic example sometimes - weird)...

I created a basic example of stdweb + wasm32: https://github.com/lilianmoraru/stdweb-alert-example

I did something wrong or the support is not great(stdweb or/and Rust + emscripten) but while trying to make todomvc to work, I had all kinds of missing symbols issues or things like that(even with the basic example sometimes - weird)...

@lilianmoraru Yes, the way you're using it isn't really supported (yet).
Basically, Rust's Emscripten-based wasm target requires a hefty runtime which you can find in the .js file generated alongside the .wasm. That .js file already includes the code necessary to load the .js file, so right now you're supposed to just load that .js and it will fetch the .wasm by itself, not the other way around.
Rust is going to have a native wasm target where it'll only generate a standalone .wasm file; I'll add support for that (to both stdweb and cargo-web) once I'll be able to download a reasonably working toolchain through rustup.

Currently it might not be, but I see no reason why we wouldn't be able to work something out. (I'll add it myself if I have to.)

I believe that wasm32-unknown-unknown is now available from rustup on nightly.

@emosenkis stweb wasm32-unknown-unknown does not currently work with it because it requires an event loop that is available only inside the emscripten environment.
wasm32-unknown-unknown is pretty basic right now, the std is mostly stubbed because compared to emscripten which offers File IO API for example(in RAM), this environment is basic, just wasm32.
If you want a crate to work under this target, the safest way is to not require std.

Since WebAssembly is now supported on every target that Rust supports - closing!

authors: dgriffen emosenkis koute lilianmoraru

repo: koute/stdweb


text:
State doesn’t change to paused when paused

I don’t know if this is a Jellyfin server issue but when I pause media the state in Home Assistant remains as “Playing”. The state changes to “Idle” if I stop playing the media completely.
The client is NVIDIA Shield

Well status updates are no more working since few weeks without any evident reasons :(

authors: tanc vincegre


text:
[BUG] Opaque background on transparent terminals

Describe the bug 🐛
If Dooit is run on a terminal with a (semi-)transparent background, instead if respecting the transparency, Dooit will draw the background as completely opaque.
To Reproduce 🐣

Install a terminal that supports transparency (ex. St with alpha patch, Kitty, Alacritty).
Set an non-zero transparency.
Run Dooit on that terminal.

Expected behavior  🤔
A curses application should respect the transparency of the terminal, so, if the terminal is semi-transparent, so should be the background drawn by the curses application. (Look programs such as Less, Vim, and Nano for reference.)
Screenshots 🧐

Desktop (please complete the following information):  🤖

OS: Artix GNU/Linux
Version: Supposedly any (tested 2.0.0 and 2.0.1)

Additional context 📝
Add any other context about the problem here.

Hey @SuperDuperDeou!
Unfortunately this was done intentionally by the TUI framework which dooit is based on :(
Check out this for more details

authors: SuperDuperDeou kraanzu

repo: kraanzu/dooit


text:
Beta firmware (6.1.10) connection issue

Similar to https://github.com/krahabb/meross_lan/issues/181 Mss 310  smart plugs not resetting their energy usage to 0 at midnight. I raised the issue with meross - they said they had a beta firmware that should resolve the issue.
They uploaded it to one of my plugs and how HA can't interface to it anymore.
Device info: mss310 6.0.0
by Meross: Firmware: 6.1.10

Version of the custom_component

Configuration

{
  "home_assistant": {
    "installation_type": "Home Assistant Container",
    "version": "2022.9.0",
    "dev": false,
    "hassio": false,
    "virtualenv": false,
    "python_version": "3.10.5",
    "docker": true,
    "arch": "x86_64",
    "timezone": "Australia/Adelaide",
    "os_name": "Linux",
    "os_version": "4.4.59+",
    "run_as_root": true
  },
  "custom_components": {
    "meross_lan": {
      "version": "2.6.2",
      "requirements": []
    },

Describe the bug
A clear and concise description of what the bug is.
Debug log

Logger: custom_components.meross_lan
Source: custom_components/meross_lan/config_flow.py:199
Integration: Meross LAN ([documentation](https://github.com/krahabb/meross_lan), [issues](https://github.com/krahabb/meross_lan/issues))
First occurred: 8:30:12 PM (3 occurrences)
Last logged: 8:32:25 PM

Error (Server disconnected) configuring meross device (host:<ip>)

Hello @ffrog8 ,
Did you retrieve the 'cloud key' correctly? When you configure your device (since it appears to be paired to the Meross cloud).
If you set the key to an empty field and try using 'Hack mode' in order to bypass the configuration check, the device might reject the connection attempts.
If you instead went through the cloud retrieval process with success and the key is set then the issue might lie in a bugged firmware

Here's the sequence of events.

Meross central updated the firmware
Device power cycled as part of firmware update
HA lost contact
Hours later HA was restarted
HA config for plug was unchanged (it would have been hack mode and auto connection protocol)
Attempted to reconfigure device, using hack mode. Got error "Key error: select 'Cloud retrieve' to get from Meross account"
Attempted to reconfigure device, using cloud retrieve. seemed to 1/2 work in that for a while entity values would flash a value then go back to undefined
At this point a bunch of random button clicks to get shit to work was attempted

Most recently

Power cycled device
Cleared the device key
Tried hack mode - same error as above
Tried cloud mode - key retrieved

And now HA entities are populated! Go figure! I'm guessing step 1 power cycle did the trick (or step 8).
Thanks for the assist.


text:
Add toggle to sort queue by queue positions

At the moment, elements in the queue are sorted by it's name.
Please add a toggle or something to sort elements by it's queue position.

Ordered by priority by default as i would expect this order be default. Feeling different?
Can you please test it together with the changes from #19 (all included in current master branch)?

It is ordered by priority now, but I don't know if it was before.

It wasn't. Closing this :)

authors: kraiz mmrose

repo: kraiz/icecult


text:
Unregister postRobot listener

I there a way to unregister a specific postRobot listener?
If I am using postRobot in an angular 2 app for example and want to register onto a postMessage, but only if a specific Component is loaded. And I want to unload it if the Component isn't present anymore.
I added the postRobot.on('x') listener but I cannot "destroy" it.
Is there a way to do this?
Thanks.

You can do:
    var listener = postRobot.on('ON_MESSAGE', { window: window.parent }, function(event) {
      ...
    });

    listener.cancel();

I'll add a section for the docs for this too.

thanks.

I wonder if an API like postRobot.off('x') could be suitable?

@samwalshnz - there's some discussion here about why off() isn't offered: https://github.com/krakenjs/post-robot/issues/21#issuecomment-303808080

authors: bluepnume janis91 samwalshnz

repo: krakenjs/post-robot


text:
fixed scroll to find footer bug

Summary
This PR fixes a bug where users have to scroll to find the footer.
Description
This PR aims to enhance the user experience by removing the scroll on pages which has less content.
Images
Include any relevant images or diagrams that can help reviewers visualize the changes, if applicable
Issue(s) Addressed
Closes #121
Prerequisites

[x] Have you followed all the CONTRIBUTING GUIDELINES?

@kami-codes This PR LGTM.
@krishnaacharyaa, would like to hear your thoughts.

@Mohitsen11 thanks
let me know if any other issue i can take

repo: krishnaacharyaa/wanderlust


text:
Insertion Sort in C

Assigned Issue Number ?
3
Language Opted ?
C
Number of files added or updated ?
1
Name of these files ?
Algorithms=>Sorting=> InsertionSort => InsertionSort.c
Checklist

[ ✔] I've read the contribution guidelines.
[ ✔] I've referred the correct issue number.
[ ✔] I've fill up this entire template correctly.

I am accepting PR this time next time, please first raise an issue or comment in open issue.
only raise PR after getting assigned.


text:
feat: added fonts & @font-face import

Description

Added the Open Sans fonts Regular & Bold to the folder fonts.
Added a package called ncp to copy the fonts from the src to the _site map.
Added dev watchers for changes to the fonts folder.
Changed the way SCSS imports modules. SCSS will now import all the scss files inside the scss folder that start with an underscore: _myScssSubModule.scss will work.

Testing
npm run dev & npm run build
Change the fonts inside the src/assets/fonts folder, or change the name chokidar will handle the change.
To do after merging this PR (optional)
Go make components I guess.

Closing since #31 fixes all this

authors: gijslaarman kriskuiper

repo: kriskuiper/cmd-digital-playground


text:
Problem with automatic global id expansion

we use GlobalId internally to pass object links in events rather than the whole objects. activejob gem automatically expands those gid values.
The problem is code which works in sync mode, no longer work in async.
class Listeners::Mailer
  def self.create_comment_successful opts
    comment = GlobalID::Locator.locate opts.fetch(:comment_gid)

    # sync:
    # opts[:comment_gid] === String
    # comment == SomeObject

    # async:
    # opts[:comment_gid] === SomeObject
    # comment == nil
  end
end

This issue makes listeners very easy to break as they will receive different values based on the way the event is passed

as a workaround only objects have to be passed and never their gid equivalent

If you broadcast an ActiveRecord object doesn't ActiveJob automatically serialize and deserialize it? So not need for GlobalID::Locator.locate inside the listener, it would get the object.

yes @krisleech, you are right. We've started with the rule to only serialize primitive types and later added activejob support, which broke our existing implementation.
I don't think it's a big deal, but was worth it to raise that in case people come across the same issue

authors: antulik krisleech

repo: krisleech/wisper-activejob

license: mit


text:
problem 48

feat: problem 45
feat: problem 46
feat: problem 48

Pull Request Test Coverage Report for Build 1935606095

255 of 269 (94.8%)  changed or added relevant lines in 4 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.4%) to 91.414%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/problem_46/mod.rs
76
79
96.2%

src/problem_47/mod.rs
51
54
94.44%

src/problem_48/mod.rs
92
95
96.84%

src/problem_45/mod.rs
36
41
87.8%

Totals

Change from base Build 1929870125:
0.4%

Covered Lines:
2470

Relevant Lines:
2702

💛  - Coveralls

authors: coveralls kristof-mattei

repo: kristof-mattei/leet-code


text:
Node-gyp install failure on macOS

When installing cbor-extract for cbor-x I've run into the following error:
gyp info it worked if it ends with ok
gyp info using node-gyp@7.1.2
gyp info using node@16.0.0 | darwin | x64
gyp info find Python using Python version 2.7.16 found at "/System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python"
(node:6061) [DEP0150] DeprecationWarning: Setting process.config is deprecated. In the future the property will be read-only.
(Use `node --trace-deprecation ...` to show where the warning was created)
gyp info spawn /System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python
gyp info spawn args [
gyp info spawn args   '/usr/local/Cellar/node/16.0.0/libexec/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '[...]node_modules/cbor-extract/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/usr/local/Cellar/node/16.0.0/libexec/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '[...]Library/Caches/node-gyp/16.0.0/include/node/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=[...]Library/Caches/node-gyp/16.0.0',
gyp info spawn args   '-Dnode_gyp_dir=/usr/local/Cellar/node/16.0.0/libexec/lib/node_modules/npm/node_modules/node-gyp',
gyp info spawn args   '-Dnode_lib_file=[...]Library/Caches/node-gyp/16.0.0/<(target_arch)/node.lib',
gyp info spawn args   '-Dmodule_root_dir=[...]node_modules/cbor-extract',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.'
gyp info spawn args ]
gyp info spawn make
gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
  CXX(target) Release/obj.target/cbor-extract/src/extract.o
In file included from ../src/extract.cpp:8:
In file included from [...]Library/Caches/node-gyp/16.0.0/include/node/v8.h:30:
[...]Library/Caches/node-gyp/16.0.0/include/node/v8-internal.h:452:38: error: no template named 'remove_cv_t' in namespace 'std'; did you mean 'remove_cv'?
            !std::is_same<Data, std::remove_cv_t<T>>::value>::Perform(data);
                                ~~~~~^~~~~~~~~~~
                                     remove_cv
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/type_traits:697:50: note: 'remove_cv' declared here
template <class _Tp> struct _LIBCPP_TEMPLATE_VIS remove_cv
                                                 ^
../src/extract.cpp:170:17: warning: unused variable 'context' [-Wunused-variable]
        Local<Context> context = Nan::GetCurrentContext();
                       ^
1 warning and 1 error generated.
make: *** [Release/obj.target/cbor-extract/src/extract.o] Error 1
gyp ERR! build error
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/usr/local/Cellar/node/16.0.0/libexec/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:194:23)
gyp ERR! stack     at ChildProcess.emit (node:events:365:28)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (node:internal/child_process:290:12)
gyp ERR! System Darwin 20.3.0
gyp ERR! command "/usr/local/Cellar/node/16.0.0/bin/node" "/usr/local/Cellar/node/16.0.0/libexec/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd [...]node_modules/cbor-extract
gyp ERR! node -v v16.0.0
gyp ERR! node-gyp -v v7.1.2
gyp ERR! not ok

System:

macOS: 11.2.3 (Big Sur)
node: v16.0.0
cbor-extract: v0.2.0
node-gyp: v7.1.2

Hope this helps to resolve the issue, cbor-x is faster than every other encoder out there without, but I'm excited to see how fast it can get!

This issue is resolved so no need to keep this around 👍🏻 thanks for your effort!

repo: kriszyp/cbor-extract


text:
Fix for add-and-commit

The value for "push" can be true/false, but it can also be something like "--force --set-upstream origin dependency-update" which is passed as an argument for git push

Good catch, thanks!


text:
AccessLevel does not (and cannot) conform to NSCoding, causing issues with Swift templates

I've been playing with Sourcery a bit together with @Joannis and we have been following your progress quite close. Trying to play with the Swift templates in 0.5.2, we couldn't get them to work due to the following exception:

2017-01-10 09:25:13.513 Sourcery[44543:41067027] -[_SwiftValue encodeWithCoder:]: unrecognized selector sent to instance 0x7fb84f532e10
2017-01-10 09:25:13.513 Sourcery[44543:41067027] -[_SwiftValue encodeWithCoder:]: unrecognized selector sent to instance 0x7fb84f532e10
2017-01-10 09:25:13.515 Sourcery[44543:41067027] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '-[_SwiftValue encodeWithCoder:]: unrecognized selector sent to instance 0x7fb84f532e10'
*** First throw call stack:
(
	0   CoreFoundation                      0x00007fffcb75de7b __exceptionPreprocess + 171
	1   libobjc.A.dylib                     0x00007fffe0348cad objc_exception_throw + 48
	2   CoreFoundation                      0x00007fffcb7dfcb4 -[NSObject(NSObject) doesNotRecognizeSelector:] + 132
	3   CoreFoundation                      0x00007fffcb6cffb5 ___forwarding___ + 1061
	4   CoreFoundation                      0x00007fffcb6cfb08 _CF_forwarding_prep_0 + 120
	5   Foundation                          0x00007fffcd140a2a _encodeObject + 1241
	6   Sourcery                            0x00000001039bc4cc _TTSf4g_n___TFC8Sourcery6Method6encodefT4withCSo7NSCoder_T_ + 508
	7   Sourcery                            0x00000001039bac29 _TToFCC8Sourcery6Method9Parameter6encodefT4withCSo7NSCoder_T__merged + 41
	8   Foundation                          0x00007fffcd140a2a _encodeObject + 1241
	9   Foundation                          0x00007fffcd141fdc -[NSKeyedArchiver _encodeArrayOfObjects:forKey:] + 460
	10  Foundation                          0x00007fffcd140a2a _encodeObject + 1241
	11  Sourcery                            0x00000001039e63f2 _TTSf4g_n___TFC8Sourcery4Type6encodefT4withCSo7NSCoder_T_ + 770
	12  Sourcery                            0x00000001039c3b75 _TToFC8Sourcery5Class6encodefT4withCSo7NSCoder_T_ + 37
	13  Foundation                          0x00007fffcd140a2a _encodeObject + 1241
	14  Foundation                          0x00007fffcd141fdc -[NSKeyedArchiver _encodeArrayOfObjects:forKey:] + 460
	15  Foundation                          0x00007fffcd140a2a _encodeObject + 1241
	16  Sourcery                            0x00000001039ca9f8 _TTSf4g_n___TFC8Sourcery17GenerationContext6encodefT4withCSo7NSCoder_T_ + 120
	17  Sourcery                            0x00000001039c7f35 _TToFC8Sourcery17GenerationContext6encodefT4withCSo7NSCoder_T_ + 37
	18  Foundation                          0x00007fffcd140a2a _encodeObject + 1241
	19  Foundation                          0x00007fffcd17c562 +[NSKeyedArchiver archivedDataWithRootObject:] + 156
	20  Sourcery                            0x00000001039ed67d _TTSf4g_g_n___TFC8Sourcery13SwiftTemplate6renderfzT5typesGSaCS_4Type_9argumentsGVs10DictionarySSCSo8NSObject__SS + 1821
	21  Sourcery                            0x00000001039e9183 _TTWC8Sourcery13SwiftTemplateS_8TemplateS_FS1_6renderfzT5typesGSaCS_4Type_9argumentsGVs10DictionarySSCSo8NSObject__SS + 51
	22  Sourcery                            0x00000001039d1485 _TTSf4gs_gs_n_n___TFC8Sourcery8SourceryP33_DB5C4364BFA94633EE953DD412A08C308generatefzT12templatePathV7PathKit4Path6outputS2_5typesGSaCS_4Type__T_ + 2677
	23  Sourcery                            0x00000001039d1b0b _TTSf4gs_gs_s_n_n___TFC8Sourcery8Sourcery12processFilesfzTV7PathKit4Path14usingTemplatesS2_6outputS2_14watcherEnabledSb_GSqGSaCOS_13FolderWatcher5Local__ + 347
	24  Sourcery                            0x000000010398cb4a _TTSf4n_n_gs_gs_gs_g___TFF8Sourcery6runCLIFT_T_U_FTSbSbV7PathKit4PathS1_S1_VS_15CustomArguments_T_ + 426
	25  Sourcery                            0x0000000103988f1f _TFF8Sourcery6runCLIFT_T_U_FTSbSbV7PathKit4PathS1_S1_VS_15CustomArguments_T_ + 79
	26  Sourcery                            0x0000000103990c7d _TPA__TTRXFo_dSbdSboV7PathKit4PathoS0_oS0_oV8Sourcery15CustomArguments_zoPs5Error__XFo_iSbiSbiS0_iS0_iS0_iS2__zoPS3___ + 173
	27  Commander                           0x0000000103b334de _TFF9Commander7commandu4_RxS_18ArgumentDescriptor_S0_0_S0_1_S0_2_S0_3_S0_rFTxq_q0_q1_q2_q3_FzTwx9ValueTypew_S1_w0_S1_w1_S1_w2_S1_w3_S1__T__PS_11CommandType_U_FzCS_14ArgumentParserT_ + 2830
	28  Commander                           0x0000000103b33c93 _TPA__TFF9Commander7commandu4_RxS_18ArgumentDescriptor_S0_0_S0_1_S0_2_S0_3_S0_rFTxq_q0_q1_q2_q3_FzTwx9ValueTypew_S1_w0_S1_w1_S1_w2_S1_w3_S1__T__PS_11CommandType_U_FzCS_14ArgumentParserT_ + 515
	29  Commander                           0x0000000103b2511f _TFE9CommanderPS_11CommandType3runfGSqSS_Os5Never + 287
	30  Sourcery                            0x0000000103988ec8 _TF8Sourcery6runCLIFT_T_ + 1624
	31  Sourcery                            0x00000001039883d1 main + 241
	32  libdyld.dylib                       0x00007fffe0c2c255 start + 1
	33  ???                                 0x0000000000000005 0x0 + 5
)
libc++abi.dylib: terminating with uncaught exception of type NSException
/Users/robbert/Documents/Tools/sourcery: line 4: 44543 Abort trap: 6           /Applications/Sourcery.app/Contents/MacOS/Sourcery "$@"

This actually happens because of this line in Method.swift, which tries to encode the AccessLevel enum using NSCoder. However, AccessLevel does not conform to NSCoding, which is a class protocol.
I don't really know how you guys would like to fix this, so that's why I'm opening an issue instead of submitting a pull request.

Turning access level into regular NSObject subclass as every other model we have would be fine, I'll need to be able to persist that regardless of a templating engine we use as I want to cache artifacts.
If you want to submit PR for that, it would be awesome :) You'll need to add markup as we do for other nscoding items and run sourcery on itself (using latest release binary) and then find Sourcery -name \"*.swift\" | xargs ./Scripts/update-placeholders ./Sourcery/CodeGenerated/Coding.Generated.swift
PS. My general recommendation is still to use Stencil rather than Swift templates, they are very much an early prototype in situations where there is no way to express something in Stencil (really rare).
Let me know if you want to try implementing that fix

Maybe we can just encode/decode rawValue?

We could but then we'd need to add support for that to the coding template, NSObject would be more consistent with general architecture

I'll adjust this, thanks for report @Obbut

Actually much simpler, I forgot we don't really use AccessLevel outside of Method (which I missed), we only store rawValue of it anyway, so just updating method should fix the issue, we can optimize it for swift templates in the future but for now just to make it consistent in both templating engines I'll update to String as its everywhere else /cc @ilyapuchka

authors: Obbut ilyapuchka krzysztofzablocki

repo: krzysztofzablocki/Sourcery


text:
Changes in Seldon open source license to BSL

We need to take some actions on the recent change of license in Seldon projects since we have Alibi integration for the explainer.
Here's the full email from Seldon regarding the license change:

Since Seldon was founded in 2014, its mission has been to accelerate the adoption of machine learning. Thanks to unwavering support and guidance from our customers, community members and investors, this vision has turned into a reality, with over 10 million machine learning models deployed across the world's most innovative companies.
Seldon has always committed itself to an open core business model, i.e. using open source as the foundation of the product suite, with advanced features, support and governance under commercial licensing models. As the market and Seldon have matured, it has become clear the value the Core and Alibi projects deliver is extensive, and no longer commensurate with the licensing model. To enable us to deliver even more value across our products, have better control over how they are commercialized, and reduce the threat of others benefitting from our efforts, we have decided it is appropriate to change the license of Core and Alibi. Our entry level serving product, MLServer, will remain open source under Apache 2.0.
What's changed?
From today, January 22, 2024, the license for future releases of Seldon's projects Core 1, Core 2, Alibi Detect and Alibi Explain will change from the Apache License Version 2.0 to the source available Business Source License v1.1 ("BSL").
What does this mean?
In practice, this means that use of the Core and Alibi projects under the BSL will be for non-production uses only, i.e. pre-production, staging and testing, with production use requiring payment under a commercial license.
We believe this change is a re-commitment to our open core principles. We now have an open source foundation with the MLServer project, a source available step-up with the Core projects, and commercially licensed enterprise grade products, with pricing across the suite that reflects a fair exchange of value.
The world of MLOps and AI is ever-changing, and we're excited to continue pushing the boundaries of innovation alongside our community and customers. For further information about the open source licensing change, or if you have any questions, please check out my full blog post, our FAQs or message your Community Account Manager in the Slack community.
Kind regards,
James Perry
CEO, Seldon

/king issue

Two action items I think:

Sunset Seldon Alibi as we did for AIX https://github.com/kserve/kserve/issues/2823.

Remove alibi explainer from kserve InferenceService specification
Remove alibi explainer python runtime.

Remove Seldon MLServer as we have equivalent features in KServe as well now, we already started removing MLServer reference in KServe docs.

@yuzisun are we losing anything significant if we get rid of alibi?
Should we start an effort to make something similar in a truly open source way? (I'm sure we can get organizations to provide resources if necessary)
We could even start with the code base from the previously Apache 2.0 licensed alibi (if it's any good).

I am not sure how many KServe users are using Alibi, currently KServe is depending on Alibi 0.9.4 which is still under Apache license according to FAQ.
I think what we can do is to remove alibi type from the inference service yaml specification. User can still use previous alibi versions(<0.9.5) under apache license via custom explainer spec. We did the same when we deprecated AIX explainer and left an example using custom explainer spec with AIX

@yuzisun Sounds good. We'll work on a PR to address this.

Should we start an effort to make something similar in a truly open source way? (I'm sure we can get organizations to provide resources if necessary)

@thesuperzapper I have a separate proposal to integrate with TrustyAI. See https://github.com/kserve/kserve/issues/3381 for details.

@yuzisun me and my team would like to keep maintaining an fork of Alibi under Apache license (or alternative library, to keep explainability present in KServe in a user friendly way). What would be needed in order to align our efforts?

Let's use this issue to discuss Seldon-related dependencies. We are working on the removal of Alibi as outlined by @yuzisun https://github.com/kserve/kserve/issues/3380#issuecomment-1905354242.
Regarding a fork, I doubt that we'd want to rely on vendor-maintained fork and don't want to have the risk of the fork going unmaintained again at some point.

@terrytangyuan @TimKleinloog if there is sufficient user-demand, and a group willing to maintain a fork of Alibi, I don't see a harm in at least starting the discussion.

@terrytangyuan I also would like to use this issue to propose an alternative, since I don’t see a custom explainer as a good alternative. Especially in the EU where we are active explainability will increase in importance (due to transparency obligations in AI regulation). I would like to strengthen the position of KServe in this field in stead of weakening it.

@terrytangyuan I also would like to use this issue to propose an alternative, since I don’t see a custom explainer as a good alternative. Especially in the EU where we are active explainability will increase in importance (due to transparency obligations in AI regulation). I would like to strengthen the position of KServe in this field in stead of weakening it.

We can still bring in explainer runtime installed out of the box from kserve when we have a good alternative just like we provide triton, torchserve predictor runtimes. I am suggesting to remove the hard coded explainer type(alibi) from inference service spec and make it more extensible so later we can bring in the official explainer runtime. If people still want to alibi they can use custom explainer, does this sound good to you ?

I see that there is lot of traction about explainability. While we are go ahead to remove the alibi, we should already set some targets for what alternative can be provided. So I agree with @TimKleinloog @thesuperzapper maybe we should already start working upon other options. Because I think explainability in production level setting is still one of USPs of kserve, I would not like it become completely custom because it really reduces reusability of whatever standards are out there for explainability.

@yuzisun that sounds like a solid plan. I would like to align our efforts to be able to create and maintain an explainer runtime, so we immediately can have an alternative.

I see that there is lot of traction about explainability. While we are going ahead to remove the alibi, we should already set some targets for what alternative can be provided. So I agree with @TimKleinloog @thesuperzapper maybe we should already start working upon other options. Because I think explainability in production level setting is still one of USPs of kserve, I would not like it become completely custom because it really reduces reusability of whatever standards are out there for explainability.

Just to be clear it is not custom, we want to make explainer pluggable as predictors so user can easily add explainer runtimes. kserve still ships a few official explainer runtimes out of the box. I am only suggesting using custom for alibi if people want to.

@yuzisun Just to be clear. Do you mean creating different runtimes and plugging it in. Like these runtimes and explainer spec similar to model spec where we can choose runtimes build for different explainability libraries

@yuzisun Just to be clear. Do you mean creating different runtimes and plugging it in. Like these runtimes and explainer spec similar to model spec where we can choose runtimes build for different explainability libraries

Yes exactly.

Sounds good, so in this issue we just scope removal of alibi. And in the other issues like one suggested by @TimKleinloog #3991. @terrytangyuan #3381 we should take up integration of new runtime approaches for explainability.

I am moving the pluggable explainer to a separate issue to track https://github.com/kserve/kserve/issues/3398

/assign

Remove Seldon MLServer as we have equivalent features in KServe as well now, we already started removing MLServer reference in KServe docs.

Hi @yuzisun / @terrytangyuan is this the plan going forward, even though MLServer is still Apache 2.0? We are actually considering migrating from SeldonCore to KServe cause we could "just" switch the backend over to KServe but keep the models running on MLServer (for now, while we migrate them 1 by 1 later).

@yuzisun @terrytangyuan can we confirm if we are planning to keep using MLServer in KServe?
It seems to still be Apache 2.0:

https://www.seldon.io/seldon-licensing
https://github.com/SeldonIO/MLServer

I don’t think we can remove it yet. See https://github.com/kserve/kserve/pull/3443#discussion_r1519025379

I know it;s a closed issue but does normal kserve serving runtime have the option of injecting version like MLFlow had? Currently am using MlServe. So just wanted to know this before migration.

authors: TimKleinloog bcvanmeurs ksgnextuple nilayaishwarya terrytangyuan thesuperzapper yuzisun

repo: kserve/kserve


text:
Custom ClusterServingRuntime not being selected based on modelFormat Name/Version without runtime specification

/kind bug
I created a ClusterServingRuntime that looks like this:
apiVersion: serving.kserve.io/v1alpha1
kind: ClusterServingRuntime
metadata:
  name: nvidia-nim-llm-24.01
spec:
  supportedModelFormats:
    - name: nvidia-nim-llm
      version: "24.01"
      autoSelect: true
      priority: 1

And an InferenceService like this:
kind: InferenceService
metadata:
  name: my-model
spec:
  predictor:
    model:
      modelFormat:
        name: nvidia-nim-llm
        version: "24.01"

And I get this error:
v1beta1Controllers  no runtime found to support predictor with model type: {nvidia-nim-llm 0xc002c46540}

Based on my understanding of the docs, if I specify the same model format/version as supported in my runtime as I list in my predictor, KServe should select it for use. I was not expecting to get this error. If I manually specify the name of the runtime in my yaml it works fine, but I would like to just specify the modelFormat name/version.
Running this through Kubeflow v1.8.0 and on the latest KServe v1.12.1.

Also the version here means for the model framework version like pytorch version or tensorrt version, I think what you specified is the nim serving time version which we have another field named “runtimeVersion”.  It is not the reason why this fails though but just want to clarify.

authors: supertetelman yuzisun


text:
Add multiModel field to ServingRuntime spec

This multiModel field will be used to define if a model is meant for SMS vs MMS.
Related: https://github.com/kserve/modelmesh-serving/pull/89

/cc @yuzisun

/lgtm
/approve

authors: pvaneck yuzisun


text:
Settings not respected and m,/m. don't work

Hi, I don't know if something is wrong on my side, but I'm having issues with two options (the only I tried so far) and two mappings (m, and m.).
For the command and dot mappings I get the following error:

Error detected while processing function signature#mark#Toggle[10]..76_ReportNoAvailableMarks
Signature: No free marks left.

Maybe I didn't understand how they work but they should add the next available marker yes?
And the settings I have are:
let g:SignatureDeleteConfirmation = 1
let g:SignaturePurgeConfirmation = 1

But none of these are respected, m space deletes all markers without confirmation. I have Space set as my leader key, could that be a conflict?
EDIT: Forgot to mention, m- doesn't work either.

This should be fixed now.

Thank you, it's much more usable now! I've noticed another bug, if you press m- or m you get a Y/N prompt to confirm deletion. If you press Esc, it counts as Yes. It's a minor thing but in Vim we are used to spam the Esc key and I keep doing it by habit. If you could fix that it'd be perfect. Thanks again for the quick response.

@shaggyrogers Thanks for the fix.
@dotfrag Sorry, I've been busy at work; I'll take a look at that this weekend.

Hi @dotfrag, sorry for the delay. Give this a try now.

No problem at all. Works perfectly, thank you @kshenoy !

authors: dotfrag kshenoy shaggyrogers

repo: kshenoy/vim-signature


text:
WIP: Add sub-command registry remove

Add the ability to remove a registry, through new sub-command registry remove.
This is a work in progress, but I would like your feedback on whether this feature is desired, and if it is, input on how to finalize this PR with tests etc.
Note to self: Also consider removing packages installed from registry.

@aknuds1 Hello, before we can accept this PR, you'll have to sign off your commit. Please see https://github.com/ksonnet/ksonnet/blob/master/CONTRIBUTING.md for more details.

@bryanl Thanks, took care of it.

repo: ksonnet/ksonnet


text:
Environment URI normalization

Go doesn't seem to have a standard way / default library to normalize URIs. As part of the environments feature, we want to be able to deploy to a specific cluster given the environment's URI. We cross-check against kubecfg for this URI's location. For fail-safe comparison, we probably want to normalize these URIs.

This issue was moved to ksonnet/ksonnet#10

repo: ksonnet/kubecfg


text:
Report client-go version in version subcommand

Adds output like:
client-go version: v1.6.8-beta.0+$Format:%h$

(Apparently $Format:%h$ is a misplaced feature - and requires us to declare the client-go git sha at link time, which is too awkward for us at this point.  The preceding version text is useful, however.)

Fwiw, this was broken by kubernetes/kubernetes#55353 :cry:


text:
Fix Carthage support for Mac target

Fixes #131. See also https://github.com/Carthage/Carthage/issues/1289.

Thanks!

authors: ikesyo kstenerud

repo: kstenerud/KSCrash


text:
why onConnectionEtablished?

why onConnectionEtablished?
Maybe onConnectionEstablished?

Has this been switched out yet?

no

Sorry for the late guys ! Merged ! @Lunar-Void @xzripper @zai1208

Thank!

authors: Lunar-Void kstzl xzripper

repo: kstzl/UrsinaNetworking


text:
编译没报错，运行崩溃

libsystem_platform.dylib`_os_once_gate_recursive_abort:
0x1110a7c57 <+0>:  movl   %edi, %eax
0x1110a7c59 <+2>:  leaq   0x2052(%rip), %rcx        ; "BUG IN CLIENT OF LIBPLATFORM: Trying to recursively lock an os_once_t"
0x1110a7c60 <+9>:  movq   %rcx, 0x3659(%rip)        ; gCRAnnotations + 8
0x1110a7c67 <+16>: movq   %rax, 0x3682(%rip)        ; gCRAnnotations + 56
->  0x1110a7c6e <+23>: ud2
什么问题

使用的sdk版本号是多少，xcode版本还有测试用手机系统版本都是多少？
是编译我们的demo后运行出错呢还是将sdk引入自己的工程后的问题？
崩溃堆栈又是什么样的呢？

sdk版本 2.9.0
xcode版本：xcode 9
手机系统:ios11
将sdk引入自己的工程，刚启动就崩溃了，崩溃堆栈如下
#0	0x0000000117a02c6e in _os_once_gate_recursive_abort ()
#1	0x0000000117a02984 in os_once_gate_wait_slow ()
#2	0x0000000117a11b2b in pthread_once ()
#3	0x000000010a64f989 in CRYPTO_THREAD_run_once ()
#4	0x000000010a644db0 in OPENSSL_init_crypto ()
#5	0x000000010a6418e4 in ERR_get_state ()
#6	0x000000010a641753 in ERR_put_error ()
#7	0x000000010a631d4d in DSO_dsobyaddr ()
#8	0x000000010a645166 in ossl_init_base_ossl ()
#9	0x0000000117a11b8c in __pthread_once_handler ()
#10	0x00000001179feac1 in _os_once ()
#11	0x0000000117a11b2b in pthread_once ()
#12	0x000000010a64f989 in CRYPTO_THREAD_run_once ()
#13	0x000000010a644db0 in OPENSSL_init_crypto ()
#14	0x000000010a5f9b43 in OPENSSL_init_ssl ()
#15	0x000000010a66bf49 in mg_mgr_init ()
#16	0x000000010a674156 in _log_run ()
#17	0x0000000117a1193b in _pthread_body ()
#18	0x0000000117a11887 in _pthread_start ()
#19	0x0000000117a1108d in thread_start ()

集成动态库没有崩溃，但是用了动态库App Store审核貌似不能通过。

项目中是不是用到了其他的openssl的库？我们的sdk中有使用openssl的库license
应该是库冲突了，使用动态库可以解决冲突问题


text:
为什么我搭建的SERO中转，都是无效份额？

开兼容模式


text:
Updated type, price, release date for MssqlMerge

corresponds to the latest v 1.19.3 from 2021-02-26

@semenenkov Thanks!

authors: ktaranov semenenkov

repo: ktaranov/sqlserver-kit


text: Images for README.md

repo: ktht/scicomp


text:
CI failure: V0.4.2-RC9 offline create cluster fail

envronment info
kubean-version: v.4.2-rc9
k8s version:1.24.7
containerd version: 1.6.16
crictl version: 1.24.0
os: redhat7
error msg

debug
login the master node excute kubeadm:
/usr/local/bin/kubeadm  init  --config=/etc/kubernetes/kubeadm-config.yaml  --ignore-preflight-errors=all --skip-phases=addon/coredns  --upload-certs

/kind test

Wait for https://github.com/kubernetes-sigs/kubespray/pull/9743

HI @tu1h , it has fixed.

success ci recored of the new release: https://github.com/kubean-io/kubean/actions/runs/4182604509/jobs/7262679969

authors: tu1h wenting-guo yankay

repo: kubean-io/kubean


text:
Investigate the impact of deserializing JSON in the API answer

Currently when we return an answer from the API the process queries the database and then deserializes the JSON into a unstructured.Unstructured struct using json.Unmarshal.  This process happens for all objects returned, but we don't do anything with the objects. Investigate the cost of this unmarshalling using benchmarks with Go.
Acceptance Criteria

Use Go benchmarking (and maybe other more rudimentary tools) to see the performance of serializing vs not-deserializing.
Don't change the API answer code yet, we just want information.

Notes:

It does not even need to be in our code, maybe a script that does it and that it mimics what do we do could be enough.
Make sure its the closest possible so we get a reasonable comparison.
The best scenario is to run this in our code, so we get the "real" comparison.
This may be OK with queries that are limited in size, but probably is worth optimizing for very large queries.

I ran some tests with locust. Looks like there is some impact when there are resources on the database. I have to check if all the increase in time is due to DB latency, or us deserializing.
`locust --headless --users 1 -r 1 -t 30s -H https://localhost:8081 --processes 1 GetPods`

*0 on database*

Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s
--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
GET      /api/v1/pods                                                                   22264     0(0.00%) |      1       1      32      1 |  741.36        0.00
--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
         Aggregated                                                                     22264     0(0.00%) |      1       1      32      1 |  741.36        0.00

Response time percentiles (approximated)
Type     Name                                                                                50%    66%    75%    80%    90%    95%    98%    99%  99.9% 99.99%   100% # reqs
--------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------
GET      /api/v1/pods                                                                            1      1      1      1      2      2      2      2      3      6     32  22264
--------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------
         Aggregated                                                                              1      1      1      1      2      2      2      2      3      6     32  22264

*100 on database*

Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s
--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
GET      /api/v1/pods                                                                     190     0(0.00%) |    156     100     300    120 |    6.38        0.00
--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
         Aggregated                                                                       190     0(0.00%) |    156     100     300    120 |    6.38        0.00

Response time percentiles (approximated)
Type     Name                                                                                50%    66%    75%    80%    90%    95%    98%    99%  99.9% 99.99%   100% # reqs
--------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------
GET      /api/v1/pods                                                                          130    190    200    200    210    290    290    300    300    300    300    190
--------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------
         Aggregated                                                                            130    190    200    200    210    290    290    300    300    300    300    190

*10_000 on database*

Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s
--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
GET      /api/v1/pods                                                                     197     0(0.00%) |    151      69     300    140 |    6.58        0.00
--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
         Aggregated                                                                       197     0(0.00%) |    151      69     300    140 |    6.58        0.00

Response time percentiles (approximated)
Type     Name                                                                                50%    66%    75%    80%    90%    95%    98%    99%  99.9% 99.99%   100% # reqs
--------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------
GET      /api/v1/pods                                                                          140    170    170    190    200    230    270    270    300    300    300    197
--------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------
         Aggregated                                                                            140    170    170    190    200    230    270    270    300    300    300    197

* 10_000 on database, limit=1000

Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s
--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
GET      /api/v1/pods?limit=1000                                                           18     0(0.00%) |   1600    1294    2000   1600 |    0.62        0.00
--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
         Aggregated                                                                        18     0(0.00%) |   1600    1294    2000   1600 |    0.62        0.00

Response time percentiles (approximated)
Type     Name                                                                                50%    66%    75%    80%    90%    95%    98%    99%  99.9% 99.99%   100% # reqs
--------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------
GET      /api/v1/pods?limit=1000                                                              1700   1700   1700   1800   1800   2000   2000   2000   2000   2000   2000     18
--------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------
         Aggregated                                                                           1700   1700   1700   1800   1800   2000   2000   2000   2000   2000   2000     18

* 10_000 on database, limit=1

Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s
--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
GET      /api/v1/pods?limit=1                                                           12554     0(0.00%) |      2       1      56      2 |  417.85        0.00
--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
         Aggregated                                                                     12554     0(0.00%) |      2       1      56      2 |  417.85        0.00

Response time percentiles (approximated)
Type     Name                                                                                50%    66%    75%    80%    90%    95%    98%    99%  99.9% 99.99%   100% # reqs
--------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------
GET      /api/v1/pods?limit=1                                                                    2      2      2      2      2      3     12     17     29     34     57  12554
--------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------
         Aggregated                                                                              2      2      2      2      2      3     12     17     29     34     57  12554

For those that aren't sure of how to read the posted results:

avg (average), min and max are the time that the request took in milliseconds
the percentile table is read as: the x% of the requests are solved in less than y milliseconds

e.g.
Response time percentiles (approximated)
Type     Name                                                                                50%    66%    75%    80%    90%    95%    98%    99%  99.9% 99.99%   100% # reqs
--------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------
GET      /api/v1/pods?limit=1000                                                              1700   1700   1700   1800   1800   2000   2000   2000   2000   2000   2000     18
--------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------
         Aggregated                                                                           1700   1700   1700   1800   1800   2000   2000   2000   2000   2000   2000     18

The 95% of the requests made take less than 2 seconds

To add a little bit more: usually the target percentile is 95% or 99%. The idea is that you want most of requests to take less than X, so normally you monitor those high percentiles to ensure most of people will be happy with their response times.

After reading about serialize/deserialize json performance it seems that there are several libraries for dealing with these operations faster than the encoding/json native library.
Just for reference, in case we want to keep the deserialization while keeping it performant:

Alternative libraries:

https://github.com/json-iterator/go
https://github.com/bytedance/sonic
https://github.com/mailru/easyjson
https://github.com/goccy/go-json
https://github.com/pquerna/ffjson
https://github.com/segmentio/encoding

Benchmark tests that compare some of them

https://github.com/go-json-experiment/jsonbench

Considering the speed, the size and the maintenance of  the project my first two picks if we want to test different performance would be:

https://github.com/bytedance/sonic
https://github.com/json-iterator/go

Note that the structure of the deserialized objects is important when measuring performance so, in case we want to change the native encoding/json library to another library we should do our own performance test

Currently when we return an answer from the API the process queries the database and then deserializes the JSON into a unstructured.Unstructured struct using json.Unmarshal. This process happens for all objects returned, but we don't do anything with the objects.

I have to ask this:  if  don't do anything with these deserialized objects, then why are we deserializing them?
Also, we should not be making any performance decisions based on a database containing 10K objects. That size is so small that the difference in the sizes being tested here is really negligible.  We should be using the 7.8 million record database for testing.  Maybe we'll find out the performance is the same, but if we see a performance impact from 0 records to 10K records, it's likely that millions of records will be worse. Maybe it levels out at some point, but we need to know this.

I have to ask this: if don't do anything with these deserialized objects, then why are we deserializing them?

I initially introduced the deserialization on retrieval because it made sense to me from a logical POV (let's retrieve a list of resources and not a random JSON) and also because I expect we will need that for supporting some things that the Kubernetes API support like retrieving the information in a table format.
This is something that isn't used yet, so we could directly remove the deserialization and just return JSONs for now (and I believe that this ticket was opened to check if we should do that change), but I think we should keep in mind that we might want to use deserialization at some point.

This is something that isn't used yet, so we could directly remove the deserialization and just return JSONs for now (and I believe that this ticket was opened to check if we should do that change), but I think we should keep in mind that we might want to use deserialization at some point.

I find your view here at odds with the one you've taken in https://github.com/kubearchive/kubearchive/pull/609#discussion_r1833528674!  :-)

I find your view here at odds with the one you've taken in https://github.com/kubearchive/kubearchive/pull/609#discussion_r1833528674

Hmm, I don't see it that way. In that comment I am giving priority to the number of queries done (read operations) over "readability" (although I don't think that the code looked that bad or was that complex, maybe that's because I am too close to it) but here I am open to remove something that isn't performant although I believe we may want to support features that need the objects to be deserialized (doing more queries isn't needed for any current or potential future feature, that's why I see it different)

We should be using the 7.8 million record

I tested with the 7.8 million records and the performance is worse overall.

I tested with the 7.8 million records and the performance is worse overall.

We may have two problems then, but I believe that we should isolate each thing to get useful conclusions. For example, using a mock driver database that has fixed data and returns the requested number of items to test how bad is the deserializing problem.

I'm not sure why we are wasting time testing the performance of something we don't need and probably should not be doing. If we need it in the future, we can investigate it then.  If it's a performance problem now, we should take it out.

authors: ggallen maruiz93 rh-hemartin

repo: kubearchive/kubearchive


text:
add lock while deleting hash ring

It's better to add  lock while deleting hash ring.

CI有问题，我重新开启一下这个PR试试能不能恢复

/close

/reopen

/lgtm

/approve

authors: Poorunga khalid-jobs muxuelan

repo: kubeedge/edgemesh


text:
Console is losing additional Cloudflare textboxes visibility on screen change

Which version of kubefirst are you using?
2.3.5
Which cloud provider?
None specific
Which DNS?
Cloudflare
Which installation type?
UI (Console app)
Which distributed Git provider?
None specific
Did you use a fork of gitops-template?
No
Which Operating System?
None specific
What is the issue?
On the "Cluster details" screen in console, if you select Cloudflare as your DNS provider, it will show you two additional textboxes. If you move back to the previous screen, and come back to the third one, those textboxes aren't shown anymore. You have to select again the DNS provider value of Cloudflare for them to reappear. The content is still there so it's just a visibility state that is lost between steps.
https://github.com/kubefirst/kubefirst/assets/1170989/8e1a9dbb-36bb-446c-93cf-e04487f5fc02
Code of Conduct

[X] I agree to follow this project's Code of Conduct

Fixed by @D-B-Hawk with https://github.com/kubefirst/console/pull/403

repo: kubefirst/kubefirst


text:
Expose TimeoutSeconds to Inference Spec

/kind feature
Describe the solution you'd like
Request timeouts are currently statically set to 60 seconds https://github.com/kubeflow/kfserving/blob/92c5f3c9df662c853beef86bddf7f30ab5d8b996/pkg/constants/constants.go#L63
This is way to long for most real-time use cases, and can together with the retry settings lead to self-DoS #808.
A field on the component (predictor, transformer, explainer) spec should be added to make the timeout configurable.

Agreed! @cliveseldon 's explanation use case actually is opposite and needs to make the timeout longer, so we should make it configurable.

authors: salanki yuzisun

repo: kubeflow/kfserving


text:
evaluator should be set in TF_CONFIG when using Estimator distribute strategy

Evaluator was excluded when generatingTF_CONFIG  environment  in tf-operator, see:https://github.com/kubeflow/tf-operator/blob/master/pkg/controller.v1/tensorflow/tensorflow.go#L110
However,  when use Estimator with distribute strategy ，TF 1.12 will  raise an error:
[2020-03-09 23:29:07,474] [INFO] [113583#MainThread] [tensorflow/python/estimator/training.py:460] Running `train_and_evaluate` with Distribute Coordinator.
[2020-03-09 23:29:07,474] [INFO] [113583#MainThread] [tensorflow/python/distribute/distribute_coordinator.py:720] Running Distribute Coordinator with mode = 'independent_worker', cluster_spec = {u'ps': [u'mbj-ps-0.default.svc:20002'], u'chief': [u'mbj-chief-0.default.svc:20001'], u'worker': [u'mbj-worker-0.default.svc:20000']}, task_type = u'evaluator', task_id = 0, environment = u'cloud', rpc_layer = 'grpc'
[2020-03-09 23:29:07,474] [WARNING] [113583#MainThread] [tensorflow/python/distribute/distribute_coordinator.py:772] `eval_strategy` is not passed in. No distribution strategy will be used for evaluation.
Traceback (most recent call last):
  File "train.py", line 136, in <module>
    tf.app.run()
  File "/usr/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 128, in run
    _sys.exit(main(argv))
  File "train.py", line 130, in main
    eval_spec)
  File "/usr/lib/python2.7/site-packages/tensorflow/python/estimator/training.py", line 462, in train_and_evaluate
    estimator, train_spec, eval_spec, _TrainingExecutor)
  File "/usr/lib/python2.7/site-packages/tensorflow/python/distribute/estimator_training.py", line 279, in train_and_evaluate
    session_config=run_config.session_config)
  File "/usr/lib/python2.7/site-packages/tensorflow/python/distribute/distribute_coordinator.py", line 786, in run_distribute_coordinator
    environment=environment)
  File "/usr/lib/python2.7/site-packages/tensorflow/python/distribute/distribute_coordinator.py", line 374, in _run_std_server
    target = cluster_spec.task_address(task_type, task_id)
  File "/usr/lib/python2.7/site-packages/tensorflow/python/training/server_lib.py", line 422, in task_address
    raise ValueError("No such job in cluster: %r" % job_name)
ValueError: No such job in cluster: u'evaluator'

similar error occured in TF 1.15:
Traceback (most recent call last):
  File "train.py", line 136, in <module>
    tf.app.run()
  File "/home/pai/envs/compat/lib/python2.7/site-packages/tensorflow_core/python/platform/app.py", line 40, in run
    _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
  File "/home/pai/envs/compat/lib/python2.7/site-packages/absl/app.py", line 299, in run
    _run_main(main, args)
  File "/home/pai/envs/compat/lib/python2.7/site-packages/absl/app.py", line 250, in _run_main
    sys.exit(main(argv))
  File "train.py", line 105, in main
    train_distribute=tf.contrib.distribute.ParameterServerStrategy(),
  File "/home/pai/envs/compat/lib/python2.7/site-packages/tensorflow_core/contrib/distribute/python/parameter_server_strategy.py", line 90, in __init__
    ParameterServerExtended(self, num_gpus_per_worker))
  File "/home/pai/envs/compat/lib/python2.7/site-packages/tensorflow_core/contrib/distribute/python/parameter_server_strategy.py", line 132, in __init__
    container_strategy, cluster_resolver=cluster_resolver)
  File "/home/pai/envs/compat/lib/python2.7/site-packages/tensorflow_core/python/distribute/parameter_server_strategy.py", line 145, in __init__
    parameter_device=parameter_device)
  File "/home/pai/envs/compat/lib/python2.7/site-packages/tensorflow_core/python/distribute/parameter_server_strategy.py", line 156, in _initialize_strategy
    self._initialize_multi_worker(cluster_resolver)
  File "/home/pai/envs/compat/lib/python2.7/site-packages/tensorflow_core/python/distribute/parameter_server_strategy.py", line 239, in _initialize_multi_worker
    task_id)
  File "/home/pai/envs/compat/lib/python2.7/site-packages/tensorflow_core/python/distribute/multi_worker_util.py", line 120, in is_chief
    _validate_cluster_spec(cluster_spec, task_type, task_id)
  File "/home/pai/envs/compat/lib/python2.7/site-packages/tensorflow_core/python/distribute/multi_worker_util.py", line 80, in _validate_cluster_spec
    raise ValueError("`task_type` %r not found in cluster_spec." % task_type)
ValueError: `task_type` 'evaluator' not found in cluster_spec.

TF code with distribute strategy （1 ps 1 chief 1 worker 1 evaluator）:
# -*- coding: utf-8 -*-
"""Example for mnist training using Estimator."""

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import tensorflow as tf
import pai
import os
import json

tf.app.flags.DEFINE_string('works', './train.tfrecords', 'separated by ,')
tf.app.flags.DEFINE_string('model_dir', './', 'model directory')
tf.app.flags.DEFINE_integer('batch_size', 128, 'batch size')
tf.app.flags.DEFINE_integer('max_steps', 50000, 'max training steps')

FLAGS = tf.flags.FLAGS

def model_fn(features, labels, mode):
  """Model function."""
  with tf.variable_scope('lr_softmax'):
    weights = tf.get_variable('weights', initializer=tf.zeros([784, 10]))
    biases = tf.get_variable('biases', initializer=tf.zeros([10]))
    logits = tf.matmul(features, weights) + biases
  loss = tf.reduce_mean(
      tf.nn.softmax_cross_entropy_with_logits_v2(labels=labels, logits=logits),
      name='loss')

  if mode == tf.estimator.ModeKeys.TRAIN:
    global_step = tf.train.get_or_create_global_step()
    opt = tf.train.AdamOptimizer(0.1, name='adam')
    train_op = opt.minimize(loss, global_step=global_step, name='train')
    return tf.estimator.EstimatorSpec(
        mode=mode,
        loss=loss,
        train_op=train_op)
  elif mode == tf.estimator.ModeKeys.EVAL:
    return tf.estimator.EstimatorSpec(
        mode=mode,
        loss=loss,
        eval_metric_ops={'no_eval': (tf.no_op(), tf.no_op())})
  else:
    raise ValueError(
        "Only TRAIN and EVAL modes are supported: %s" % (mode))

def train_input_fn():
  image = tf.zeros([784], dtype=tf.float32)
  label = tf.zeros([10], dtype=tf.int64)
  d = tf.data.Dataset.from_tensors((image, label)).repeat().batch(FLAGS.batch_size).prefetch(64)
  return d

def eval_input_fn():
  image = tf.zeros([784], dtype=tf.float32)
  label = tf.zeros([10], dtype=tf.int64)
  d = tf.data.Dataset.from_tensors((image, label)).repeat().batch(FLAGS.batch_size).prefetch(64)
  return d

def main(_):
  tf.logging.set_verbosity(tf.logging.INFO)

  estimator = tf.estimator.Estimator(
      model_fn=model_fn,
      model_dir=FLAGS.model_dir,
      config=tf.estimator.RunConfig(
          train_distribute=tf.contrib.distribute.ParameterServerStrategy(),
          session_config=tf.ConfigProto(
              log_device_placement=False,
              allow_soft_placement=True,
              gpu_options=tf.GPUOptions(
                  allow_growth=True,
                  force_gpu_compatible=True))))
  tf.logging.info('Estimator created.')

  train_spec = tf.estimator.TrainSpec(
      input_fn=train_input_fn,
      max_steps=50000)
  eval_spec = tf.estimator.EvalSpec(
      input_fn=eval_input_fn)

  tf.logging.info('Tran and Eval specs created.')

  tf.estimator.train_and_evaluate(
      estimator,
      train_spec,
      eval_spec)

if __name__ == '__main__':
  env_dist = os.environ
  print(env_dist.get('TF_CONFIG'))

  tf.app.run()

Note: If train_distribute attribute in tf.estimator.RunConfig is set to None, it works well.

similar issue in tensorflow repo: https://github.com/tensorflow/tensorflow/issues/30121

@johnugeorge @richardsliu @gaocegege Looks like tf-operator needs to support evaluator in addition to ps and worker. An example cluster spec that has all these roles:
cluster = {'chief': ['host0:2222'],
                 'evaluator': ['host6:2222'],
                 'ps': ['host1:2222', 'host2:2222'],
                 'worker': ['host3:2222', 'host4:2222', 'host5:2222']}

On evaluator node, TF_CONFIG should be similar to the following in order to tell TensorFlow to use this node for performing model evaluation:
os.environ['TF_CONFIG'] = json.dumps(
      {'cluster': cluster,
       'task': {'type': 'evaluator', 'index': 0}})

@johnugeorge @richardsliu @gaocegege Looks like tf-operator needs to support evaluator in addition to ps and worker. An example cluster spec that has all these roles:
cluster = {'chief': ['host0:2222'],
                 'evaluator': ['host6:2222'],
                 'ps': ['host1:2222', 'host2:2222'],
                 'worker': ['host3:2222', 'host4:2222', 'host5:2222']}

On evaluator node, TF_CONFIG should be similar to the following in order to tell TensorFlow to use this node for performing model evaluation:
os.environ['TF_CONFIG'] = json.dumps(
      {'cluster': cluster,
       'task': {'type': 'evaluator', 'index': 0}})

@terrytangyuan If evaluator contains TF_CONFIG cluster spec,  some changes should be made in tensorflow 1.12  when train_distribute attribute in tf.estimator.RunConfig is set to None. Otherwise other node will  wait for evaluator forever：

@rbrishabh  I'm glad to  take time to fix this problem if possible.

@richardsliu  I'm glad to take time to fix this problem if possible.

@richardsliu @gaocegege   I'm glad to take time to fix this problem if possible.

LGTM

LGTM

Hi，you mean the way TF_CONFIG environment  set by tf-operator  currently  is correct?

@meibenjin are you working on this? We are blocked on this too and we'd like to create a patch if you haven't already(would be glad to test your patch if there's an image).

@meibenjin Welcome the PR. I think it is a problem, but one question: How about the old version TF?

@meibenjin Welcome the PR. I think it is a problem, but one question: How about the old version TF?

I think some changes would be made in old version TF if we add evaluator in TF_CONFIG cluster (remove evaluator from TF_CONFIG cluster_spec)，In our test when train_distribute attribute in tf.estimator.RunConfig is set to None. Other nodes will wait for evaluator forever：

@meibenjin are you working on this? We are blocked on this too and we'd like to create a patch if you haven't already(would be glad to test your patch if there's an image).

please see my reply to gaocegege ，We should think about the compatibility in old version TF like Tensorflow1.12。

@meibenjin Thanks for the reply. I will comment soon after a deep dive into the TF code.

@johnugeorge @richardsliu @gaocegege Looks like tf-operator needs to support evaluator in addition to ps and worker. An example cluster spec that has all these roles:
cluster = {'chief': ['host0:2222'],
                 'evaluator': ['host6:2222'],
                 'ps': ['host1:2222', 'host2:2222'],
                 'worker': ['host3:2222', 'host4:2222', 'host5:2222']}

On evaluator node, TF_CONFIG should be similar to the following in order to tell TensorFlow to use this node for performing model evaluation:
os.environ['TF_CONFIG'] = json.dumps(
      {'cluster': cluster,
       'task': {'type': 'evaluator', 'index': 0}})

@terrytangyuan We already support it if there is no DistributionStrategy. DistributionStrategy requires a validate function call to validate that the evaluator should be in cluster spec.
https://github.com/tensorflow/tensorflow/issues/30121

@johnugeorge @richardsliu @gaocegege Looks like tf-operator needs to support evaluator in addition to ps and worker. An example cluster spec that has all these roles:
cluster = {'chief': ['host0:2222'],
                 'evaluator': ['host6:2222'],
                 'ps': ['host1:2222', 'host2:2222'],
                 'worker': ['host3:2222', 'host4:2222', 'host5:2222']}

On evaluator node, TF_CONFIG should be similar to the following in order to tell TensorFlow to use this node for performing model evaluation:
os.environ['TF_CONFIG'] = json.dumps(
      {'cluster': cluster,
       'task': {'type': 'evaluator', 'index': 0}})

@terrytangyuan We already support it if there is no DistributionStrategy. DistributionStrategy requires a validate function call to validate that the evaluator should be in cluster spec.
https://github.com/tensorflow/tensorflow/issues/30121

I think some changes would be made in old version TF if we add evaluator in TF_CONFIG cluster (remove evaluator from TF_CONFIG cluster_spec)，In our test when train_distribute attribute in tf.estimator.RunConfig is set to None. Other nodes will wait for evaluator forever：

Yeah. It is what I worry about. All the replicas will wait for the evaluator session.
I am not sure why distribute strategy needs such a validate function to keep evaluators in cluster_spec. Not sure if it is a bug or feature.
/cc @terrytangyuan Do you have any idea about it?

@gaocegege Not sure. It's probably due to some requirements in higher level APIs. We can bring this up in tensorflow/tensorflow#30121.

I have a PR fix here: https://github.com/kubeflow/tf-operator/pull/1146
But this will run into the issue mentioned by @meibenjin.

Please have a look at this https://github.com/tensorflow/tensorflow/issues/27857#issuecomment-488924749. Master node is not officially supported. Because master will start an evaluator itself.

hey, do you guys have a clear solution? I still came out with this problem.

i have the same problem,  is there one solution or workaround please?  @terrytangyuan
from estimator.run_config.py:
Example of evaluator node (evaluator is not part of training cluster):
    ```
      cluster = {'chief': ['host0:2222'],
                 'ps': ['host1:2222', 'host2:2222'],
                 'worker': ['host3:2222', 'host4:2222', 'host5:2222']}
      os.environ['TF_CONFIG'] = json.dumps(
          {'cluster': cluster,
           'task': {'type': 'evaluator', 'index': 0}})
      config = RunConfig()
      assert config.master == ''
      assert config.evaluator_master == ''
      assert config.task_id == 0
      assert config.num_ps_replicas == 0
      assert config.num_worker_replicas == 0
      assert config.cluster_spec == {}
      assert config.task_type == 'evaluator'
      assert not config.is_chief
    ```

@pengyuan There is a workaround here. But it seems a little hacky. An estimator parses all the cluster information from the environment variable TF_CONFIG. You can try to override the TF_CONFIG to a reasonable string before starting your own program. But be careful about the exiting logic of the whole tensorflow program. For example, you have 1 ps, 1 master, 2 workers. You can create a valid TF_CONFIG with 1 ps, 1 chief, 1 evaluator, 1 worker. The evaluator should not appear in the cluster section of TF_CONFIG. You can choose any machine from the worker list as an evaluator.
original TF_CONFIG may be
TF_CONFIG = {
    "cluster": {
      "ps": ["localhost:port1"],
      "worker":["localhost:port4", "localhost:port2"],
      "master": ["localhost:port3"]
    },
    "task": {
      "type": "master",
      "index": 0
    }
},

We choose the first worker from the worker list as the evaluator.
The TF_CONFIG for each role is:
for chief
TF_CONFIG = {
  "cluster": {
  "ps": ["localhost:port1"],
  "worker": ["localhost:port2"],
  "chief": ["localhost:port3"]
  },
  "task": {
  "type": "chief",
  "index": 0
  }
}

for ps
TF_CONFIG = {
  "cluster": {
    "ps": ["localhost:port1"],
    "worker": ["localhost:port2"],
    "chief": ["localhost:port3"]
  },
  "task": {
    "type": "ps",
    "index": 0
  }
}

for worker
TF_CONFIG = {
  "cluster": {
    "ps": ["localhost:port1"],
    "worker": ["localhost:port2"],
    "chief": ["localhost:port3"]
  },
  "task": {
    "type": "worker",
    "index": 0
  }
}

for evaluator
TF_CONFIG = {
  "cluster": {
    "ps": ["localhost:port1"],
    "worker": ["localhost:port2"],
    "chief": ["localhost:port3"]
  },
  "task": {
    "type": "evaluator",
    "index": 0
  }
}

authors: Mesilenceki ashahab chunyang-wen gaocegege johnugeorge meibenjin pengyuan richardsliu terrytangyuan

repo: kubeflow/tf-operator


text:
crd: Add validation using OpenAPI 3.0

Close #437
/assign @jlewi @ScorpioCPH
The feature is restricted and I only add validation for replicas. This avoids the crash problem when replicas < 0.
Signed-off-by: Ce Gao gaoce@caicloud.io

This change is

/retest

If we could find the validation for pod template spec, then we could reuse it here. While I do not find it.

/retest

/retest

/retest

I think the test failures could be kubeflow/kubeflow#869

/hold
Do you still want to submit this in addition to the ksonnet changes? If so then just cancel the hold.
/lgtm
/approve

/hold cancel
I will file a new PR if I find a way to check the podtemplatespec

Coverage remained the same at 55.646% when pulling 1175791881a7415a61a08c455b5874122a688575 on gaocegege:validation into 98b777eba0b9e91d55be2fd917f25a198cdddeef on kubeflow:master.

authors: coveralls gaocegege jlewi


text:
WIP: Refactor tf-operator E2E tests (part 2)

This is part 2 of the work to refactor E2E tests and make them easier to write.
This change moves all of the test-specific code out of the test_runner.py code. Instead, the following test suites are created:

simple_tfjob_tests
shutdown_policy_tests
cleanpod_policy_tests
estimator_runconfig_tests

Each file contains self-contained steps and verifications. The original test_runner invokes the test method specified in the workflows.libsonnet file.
In addition the workflows.libsonnet file has been refactored to reduce the amount of boiler-plate code.
Next step is to write a guide on how to author new E2E tests.

This change is

/test all

/assign @jlewi

Fixed all the other tests, please take a look.

/retest

/test kubeflow-tf-operator-presubmit

A couple high level questions:
This test is moving us in the direction of making our tests look more like standard Python unittests. I think this is a huge improvement.
Given that goal
Do we need our own "main()"?  Could we just use unittest.main()? Or if need be subclass unittest.main()?
I think one of the advantages of reusing unittest.main() is that out of the box it supports a lot of useful parameters. e.g. we can run a single test
python my_module.py TestClass.test_func
Should Test Case subclass unittest.TestCase? This would allow us to use the stand assert macros to do test validation.
Instead of creating our own TestCase should we just be creating our own TestRunner suitable for reporting results to Prow (https://docs.python.org/3/library/unittest.html)?
Then we could just change the TestRunner when running under Prow but in all other ways our tests would just be Py Unittests and it would be much easier to run them locally or without Prow.

I think if we had our own TestRunner we could do something like this
if name == "main":
test_runner = None
if os.getenv("IS_PROW):
test_runner = test_util.TestRunner()
unittest.main(testRunner=test_runner)
https://github.com/python/cpython/blob/master/Lib/unittest/main.py#L55

/retest

Opened https://github.com/kubeflow/tf-operator/issues/851 to track further enhancements to the test harness.

/lgtm
/approve

authors: jlewi krzyzacy richardsliu


text:
Expands introduction to Kubeflow Pipelines quickstart

Fixes https://github.com/kubeflow/website/issues/648

This change is

@iyera This change is in response to your feedback on a friction report. Please take a look.

Preview: https://deploy-preview-705--competent-brattain-de2d6d.netlify.com/docs/pipelines/pipelines-quickstart/

/assign @abhi-g

I have an LGTM from @iyera on chat.

/lgtm
/approve

authors: richardsliu sarahmaddox

repo: kubeflow/website


text:
Store Runtime image references in configmaps

Use Case
As a kubeless administrator I would like to define custom runtime Images to be used by developers. These images would be approved by the organization and would be available on private docker registry.
Description
Currently the runtime images are hard-coded into the code. Move this into the configmaps kubeless-config. This should allow Kubeless Administrator to provide runtime images to all the developers easily. Additionally it should be possible to pass the reference to the secrets. As this would allow developers to use images from private docker hubs.
Solutions Proposed

The hardcoded images inside pkg/langruntime/langruntimes.go could be moved to the configmap kubeless-config
Inside the function init() in pkg/langruntime/langruntimes.go we can get the values for variables httpImage, pubsubImage, initImage from the configmap.
Each function runtime image config can have additional field for passing the secrets. When this is passed we can update the function deployment yaml to include the reference to the secret.

Closing the issue, since desired functionality is already added in #575

authors: murali-reddy rakesh-garimella

repo: kubeless/kubeless


text:
Allow upstream cluster autoscaler flags configuration

What this PR does / why we need it:
This PR is to allow the configuration of the upstream cluster-autoscaler supported flags via KubeOne provided cluster-autoscaler addon to address https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/FAQ.md#my-cluster-is-below-minimum--above-maximum-number-of-nodes-but-ca-did-not-fix-that-why and https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/FAQ.md#im-running-cluster-with-nodes-in-multiple-zones-for-ha-purposes-is-that-supported-by-cluster-autoscaler issue as faced at the one of the customer end.
Which issue(s) this PR fixes:

Fixes #
What type of PR is this?
/kind bug
/kind chore
Special notes for your reviewer:
Does this PR introduce a user-facing change? Then add your Release Note here:

Allow the configuration of the upstream cluster-autoscaler flags  `--enforce-node-group-min-size` and `--balance-similar-node-groups`

Documentation:

TBD

@archups it's not a bug it's a feature :joy:
/approve
/lgtm

@archups it's not a bug it's a feature 😂
/approve /lgtm

Oops, I was confused whether to add as feature or bug..  Thanks @kron4eg  for the clarification and review. :)

/backport-needed

/cherrypick release/v1.8

repo: kubermatic/kubeone


text:
Only update kernel commandline to add quota on RHEL

Fedora amis are still ext4 based and ext4 doesn't like this flag.
Signed-off-by: Mrunal Patel mrunalp@gmail.com

@stevekuznetsov @rhatdan PTAL

LGTM

LGTM

/test all

/test e2e_rhel

I'm going to bring this into #963 -- I made bad Fedora AMIs without it

Okay sounds good.

On Sep 30, 2017, at 7:59 PM, Steve Kuznetsov notifications@github.com wrote:
I'm going to bring this into #963 -- I made bad Fedora AMIs without it
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

Can probably close this. Brought it into #963, Fedora boots fine with this change.

authors: mrunalp rhatdan stevekuznetsov

repo: kubernetes-incubator/cri-o