text:
first field in an optional submessage don't get encoded

Whenever a submessage is set to optional the first field in the submessage is not encoded. This does apply whenever a field has the has_???? property.
So in proto2:
syntax = "proto2";

package update_protocol;

message SubMessage {
    required int32 test = 4;
    required int32 var3 = 6;
    required int32 var4 = 7;
}

message Test {
    required int32 var1 = 1;
    required int32 var2 = 2;
    optional SubMessage sub = 3;
}

the field SubMessage.test is not encoded,
or in proto3:
syntax = "proto3";

package update_protocol;

message SubMessage {
    int32 test = 4;
    int32 var3 = 6;
    int32 var4 = 7;
}

message Test {
    int32 var1 = 1;
    int32 var2 = 2;
    SubMessage sub = 3;
}

the Test message gets an implicit has_sub property and the field SubMessage.test is also not encoded....
Code Snippet where used:
   update_protocol_Test test = update_protocol_Test_init_zero;
    pb_ostream_t stream = pb_ostream_from_buffer(prototestbuffer, sizeof(prototestbuffer));

    test.var1 = 1;
    test.var2 = 2;
    test.has_sub = true;
    test.sub.var3 = 3;
    test.sub.var4 = 4;
    test.sub.test = 5;

    OSAL_LOG_ERR("var1: %d",test.var1);
    OSAL_LOG_ERR("var2: %d",test.var2);
    OSAL_LOG_ERR("sub.var3: %d",test.sub.var3);
    OSAL_LOG_ERR("sub.var4: %d",test.sub.var4);
    OSAL_LOG_ERR("sub.test: %d",test.sub.test);

    bool status = pb_encode(&stream, update_protocol_Test_fields, &test);
    OSAL_LOG_ERR("Status: %d, Encode Length = %d",status, stream.bytes_written);

    OSAL_LOG_HEXDUMP_ERR(prototestbuffer,32,"buffer");

    update_protocol_Test test_decode = update_protocol_Test_init_zero;
    pb_istream_t istream = pb_istream_from_buffer(prototestbuffer, stream.bytes_written);
    status = pb_decode(&istream, update_protocol_Test_fields, &test_decode );

    OSAL_LOG_ERR("Status :%d",status);
    OSAL_LOG_ERR("var1: %d",test_decode.var1);
    OSAL_LOG_ERR("var2: %d",test_decode.var2);
    OSAL_LOG_ERR("sub.var3: %d",test_decode.sub.var3);
    OSAL_LOG_ERR("sub.var4: %d",test_decode.sub.var4);
    OSAL_LOG_ERR("sub.test: %d",test_decode.sub.test);

and generated output:
[00:07:43.972,000] <err> flash: var1: 1
[00:07:43.973,000] <err> flash: var2: 2
[00:07:43.974,000] <err> flash: sub.var3: 3
[00:07:43.974,000] <err> flash: sub.var4: 4
[00:07:43.975,000] <err> flash: sub.test: 5
[00:07:43.976,000] <err> flash: Status: 1, Encode Length = 16
[00:07:43.977,000] <err> flash: buffer
                                08 01 10 02 1a 0a 20 80  06 30 80 08 38 80 8a 03 |...... . .0..8...
                                00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 |........ ........
[00:07:43.980,000] <err> flash: Status :1
[00:07:43.981,000] <err> flash: var1: 1
[00:07:43.982,000] <err> flash: var2: 2
[00:07:43.982,000] <err> flash: sub.var3: 3
[00:07:43.983,000] <err> flash: sub.var4: 4
[00:07:43.984,000] <err> flash: sub.test: 0

Decoding the hex dump using https://protobuf-decoder.netlify.app/ shows that the fields get encoded, but with unexpected values.
Typically this means that the compiled version of .pb.c and the header .pb.h that is included do not match in some way. Make sure you have recompiled everything and there are no old file versions that could get accidentally included.
Another possibility is if you are using both C and C++ in the project, check if sizeof(bool) is different between C and C++ files. That would cause the structs to appear differently in memory when accessed by C code vs. C++.

Hi, tanks a lot for your fast answer.
I tried recompiling all files and the problem still exists.
The value gets encoded as the same unexpected value, no matter what i try to set as value.
There are no c or c++ files mixed, all is plain C.
I had to include the gcc library as the 64bit shift opertators where not nativly available on my target.
Can you point me to some code where to debug this problem?

Output:
[00:00:05.127,000] <err> flash: var1: 1
[00:00:05.127,000] <err> flash: var2: 2
[00:00:05.128,000] <err> flash: sub.var3: 3
[00:00:05.129,000] <err> flash: sub.var4: 4
[00:00:05.129,000] <err> flash: sub.tst: 5
[00:00:05.130,000] <err> flash: Status: 1, Encode Length = 16
[00:00:05.131,000] <err> flash: buffer
                                08 01 10 02 1a 0a 20 80  06 30 80 08 38 80 8c 03 |...... . .0..8...
                                00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 |........ ........
[00:00:05.134,000] <err> flash: message struct
                                01 00 00 00 02 00 00 00  01 05 00 00 00 03 00 00 |........ ........
                                00 04 00 00 00                                   |.....
[00:00:05.138,000] <err> flash: message descriptor
                                04 11 00 40 08 11 04 40  0d 18 01 40 0c 00 0c 00 |...@...@ ...@....
                                00 00 00 00 c0 b9 00 00  ec b9 00 00 00 00 00 00 |........ ........
                                00 00 00 00 03 00 00 00  03 00 00 00 a4 b9 00 00 |........ ........
                                00 00 00 00 30 2e 31 2e  30 2d 30 00 45 55 46 45 |....0.1. 0-0.EUFE
[00:00:05.144,000] <err> flash: Status :1
[00:00:05.145,000] <err> flash: var1: 1
[00:00:05.145,000] <err> flash: var2: 2
[00:00:05.146,000] <err> flash: sub.var3: 3
[00:00:05.147,000] <err> flash: sub.var4: 4
[00:00:05.147,000] <err> flash: sub.tst: 0

Ok, so let's see.
Message struct:
01 00 00 00 // var1
02 00 00 00 // var2
01          // has_sub
05 00 00 00 // test
03 00 00 00 // var3
04 00 00 00 // var4

Descriptor (comparing to pb.h definition):
04 11 00 40  Field 1, proto3 varint, data_offset 0, data_size 4

08 11 04 40  Field 2, proto3 varint, data_offset 4, data_size 4

0d 18 01 40  Field 3, optional submessage, size_offset 4
0c 00 0c 00  data_offset 12, data_size 12

00 00 00 00  End of descriptor

So yeah, the descriptor says the submessage begins at offset 12, so there is 4 bytes received for the bool. But the message struct only has 1 byte reserved. This means that there is some significant difference between how the .pb.c file is compiled and how your main .c file is compiled.
Typically this would be because of something wrong with compiler configuration, e.g. if you have different compilation options for different files and they make the structs incompatible. Something like #pragma pack can also affect it.

Thank you so much!
It turned out that it was a #pragma pack(1) directive in some other header file included.
Changes this to #pragma pack(push,1) and #pragma pack(pop) at the end and now eveything is working as expected.

repo: nanopb/nanopb

license: Zlib


text:
Questions about the stored binary files "_ps_and_gamma.npy”

Dear developers,
I did some calculation of the 3rd phonon bandwidths for tersoff-Si using the force constants from lammps with kaldo. I had finished the calculation but encounterd some confusing problems. I am writing to seek your help and the following are my questions.

What's the meaning of data stored in file “_ps_and_gamma.npy”? I found that the second column is the phonon bandwidth but the first column is smaller.
Is it possible for kaldo to export the information of generated k points? The Phonons API only defines the number of k points and doesn’t return any information of k points.
I took the primitive cell (not orthogonal) and conventional cell (8 atoms, orthogonal) of diamond silicon respectively but get quite different density of phonon states and bandwidth/lifetime results. I did the test on using different  broadening_shape, kpts, temperature, broadening_shape parameters and the situation is the same. Does kaldo have any restrictions for the input cell?

Thank you very much!
Best regards

Hi @OliverYL ,
I am replying here to both the issue and the email.

If you select to store your files in Numpy format,_ps_and_gamma.npy has the information about phase space and bandwidth (gamma). For RTA and QHGK the first column is the phase space, while the second is the bandwidth, analogous to Phonons.bandwidth and Phonons.phase_space.
Yes, you can save any property of the Phonons, Conductivity or ForceConstants objects. Once you created the Phonons object with something like phonons = Phonons(...), use np.save('filename', phonons.kpts) to store it as a NumPy array.
As far as I know, kALDo doesn't have restrictions for the input cell. Can you share the input files via email, please?

source: gharchive/issue

authors: OliverYL gbarbalinardo

repo: nanotheorygroup/kaldo

license_type: permissive

license_source: github-api


text:
task: move documentation on napari-hub/description.md to hub wiki

@potating-potato, I think it would make more sense to put the long instructions in description.md on the hub wiki page, in a section describing best practices on customizing a listing on the hub.  There, it would be easier to customize, and keep up to date, and would stay relevant long after someone had cloned this cookiecutter repo.
The description.md here could just have a link to those docs.  thoughts?

By description.md are you referring to https://github.com/napari/cookiecutter-napari-plugin/blob/main/{{cookiecutter.plugin_name}}/.napari-hub/DESCRIPTION.md?
supposedly it is a placeholder that the plugin owner would modify to add the description they would like to have.
Do you mean we would drop this placeholder file and add a link (I assume in the cookie-cutter readme?) to the hub wiki instead to explain how to add a customized description?

Do you mean we would drop this placeholder file and add a link (I assume in the cookie-cutter readme?) to the hub wiki instead to explain how to add a customized description?

no, the placeholder file would stay... but it would just have much less text in it, mostly just providing a link like "for tips on how to leverage this file to customize how your page looks on the hub, see ".  That way you can keep it up to date, and iterate, and long after someone has cloned this repo it would still be modern and useful for them

Ohh I was confused because I didn't realize there is much more content than what GitHub displays.
The developer has not yet provided a napari-hub specific description.

that makes sense, I can take care of this

repo: napari/cookiecutter-napari-plugin


text:
Deprecations

[x] Design deprecations x three levels
Consider differentiating icons for accessibility? Or else align with team if we need all three levels, or just one or two?

PyData default:

Here's a link to the designs for Deprecations in Figma (scroll right to see all three of the depcration levels):
https://www.figma.com/file/ahjEJjeaFL6oTjeQaueGDQ/napari-Sphinx-Theme?node-id=89%3A249
And here's where the icons for each level are in Figma:
https://www.figma.com/file/ahjEJjeaFL6oTjeQaueGDQ/?node-id=89%3A284

The colors look good, but there are few other updates to make it match the designs:

[ ] Space between edge of box and text should be 25px altogether; right now it is larger. Looks like there is a combination of margin + padding making it wider:

[ ] Square corners

[ ] Font size should be 14px (for wide screens) / 9 px for narrow screens (<= 495px wide)

[ ] Font weight should be Barlow Medium (500)

These are looking good, but I noticed a couple of things I didn't catch the first time, sorry!

[ ] Should not have shadows
[ ] The 'header' text of the deprecations that follows right after the icons (e.g. "Deprecated since version v0.3:"), should actually be larger (size 17 in sider screens / size 11 in mobile screens)
[ ] The header text should also not be italics, (but maintain the bold / 700 font weight, that is correct!)

Thanks!

repo: napari/napari-sphinx-theme


text:
behavior of translate in view_image

🐛 Bug/Question
it's possible that I just don't understand how I should be using translate in view_image, but after #1360, using translate in view_image (when viewing a single volume) no longer shifts the image relative to the camera.  It may be a strange case of my particular dataset that the image needed to be translated at all, but I always needed to use translate to get the dask map_blocks example to be centered in the frame.  Now, using translate no longer does that (see two examples below).  Lemme know if I should just be doing something different with this particular dataset, or whether translate should always shift the image relative to the camera

Incidentally, it's interesting that the layer list doesn't appear in the screenshot!
Other than that this is pretty concerning. 😅 Do the coordinates under the cursor make sense in both cases? That might tell us whether this is more of a camera issue or a data issue...

ok, turns out the main thing I was seeing here was due to me incorrectly updating chunk size while chaining dask map_blocks calls that actually modify the shape of the underlying data.  Since dask calculates the shape of the array based on those chunks, it resulted in a dask array that reported an incorrect shape.  It's (kinda) interesting that it all still works "fine" in napari... that is, the computed numpy array still renders correctly, but because it's not the shape that the dask array was reporting, it just gets rendered off center (in all three axes).
So, I don't think we need to change anything in napari... but it's worth "remembering" that this kind of thing can happen if the end-user is not careful about updating chunks/shape when mapping shape-modifying functions.
here was the necessary update to the dask-napari notebook notebook
# map and chain those functions across all dask blocks
deskewed = dask_stack.map_blocks(deskew, dtype="uint16", chunks=(1,65,256,533))
deconvolved = deskewed.map_blocks(deconv, dtype="float32", chunks=(1,64,256,525))
cropped = deconvolved.map_blocks(crop, dtype="float32", chunks=(1,62,226,500))

repo: napari/napari


text:
Viewer buttons pannel dissapear when oppening a image in napari (windows 11)

🐛 Bug
After launching napari viewer from the anaconda terminal and open an image (any image, even the samples) the canvas windows resize, and buttons panel disappears behind the windows taskbar turning them unaccessible.
Buttons can be shown when reducing the window and maximizing it again but as soon the cursor moves the window is resized and the buttons panel disappears again.
To Reproduce
Steps to reproduce the behavior:

Launch napari from anaconda terminal
Open any image
Try to use the buttons panel

Expected behavior
When opening the image I do not expect any change in the viewer window nor in the canvas
Environment

Please copy and paste the information at napari info option in help menubar here:
napari: 0.4.17
Platform: Windows-10-10.0.22621-SP0
Python: 3.9.16 (main, Jan 11 2023, 16:16:36) [MSC v.1916 64 bit (AMD64)]
Qt: 5.15.2
PyQt5: 5.15.9
NumPy: 1.23.5
SciPy: 1.10.0
Dask: 2022.7.0
VisPy: 0.11.0
magicgui: 0.6.1
superqt: unknown
in-n-out: 0.1.6
app-model: 0.1.1
npe2: 0.6.2

OpenGL:

GL version: 4.6.0 - Build 30.0.101.1692
MAX_TEXTURE_SIZE: 16384

Screens:

screen 1: resolution 960x540, scale 2.0

Any other relevant information:

Additional context
Windows information
Edition	Windows 11 Home
Version	22H2
Installed on	09/02/2023
OS build	22621.1265
Serial number	PF2ZYHT6
Experience	Windows Feature Experience Pack 1000.22638.1000.0

screen 1: resolution 960x540, scale 2.0

I think that the source of the problem is here. Until now, we do not support very small napari windows. Could you share your use case for why you need to use napari on such a small screen?

I´m actually using a Lenovo Ideapad 5 15ITL05 with 15.6'' screen and a 1920 x1080 resolution (same as resolution 960x540, scale 2.0?) which is my working laptop, I wouldn't consider it as a small screen, it is rather a standard size for laptop.

960x540 is a small area.
It means that I do not understand. What is the resolution visible in your system settings?

Maybe some HIDPI setting gone amok?

@Czaki as I said my resolution is 1920x1080. Not 960x540. Seems to me that napari is providing my resolution as 960x540 times 2, as it says screen 1: resolution 960x540, scale 2.0
@psobolewskiPhD it could be, but at the moment this mismatch between app size window and window display only happens with napari.

Maybe you can try:
set QT_SCALE_FACTOR=1
before launching napari?

Thanks @psobolewskiPhD for suggesting but it didn't work.
For clarity some screenshots of my issue:
Before opening the image:

And then when opening the image happens this:

Buttons panel disappear and it is impossible to use them.

Oh, OK, that's different.
Can you try without fullscreen mode and see if the window size also changes?
But I think there is still something wrong with the scaling--the napari UI stuff looks huge compared to your Windows stuffs.
another thing to try:
QT_AUTO_SCREEN_SCALE_FACTOR=1

Thanks @psobolewskiPhD the issue is fixed!!
Using:
set QT_AUTO_SCREEN_SCALE_FACTOR=0
Before launching napari fixed the issue.
For reference, before fixing the issue, when not using full screen mode I could'n make small enough to fit in the screen.

Yay! Out of curiosity can you post napari info again with fixed behavior?

I´m actually using a Lenovo Ideapad 5 15ITL05 with 15.6'' screen and a 1920 x1080 resolution (same as resolution 960x540, scale 2.0?) which is my working laptop, I wouldn't consider it as a small screen, it is rather a standard size for laptop.

In general, it is a standard size, but for scaling 1.0.
Did you scale the interface, or is scale 2.0 some mistake? I remember that sometimes Qt wrong interprets fractional scaling. Or you do not have scaling and for some reason, Qt treats your display as high DPI?

Yay! Out of curiosity can you post napari info again with fixed behavior?

Yes @psobolewskiPhD here is the napari info, and as expected the resolution is also fixed with a scaling factor of 1, not 2 as before:
napari: 0.4.17
Platform: Windows-10-10.0.22621-SP0
Python: 3.9.16 (main, Jan 11 2023, 16:16:36) [MSC v.1916 64 bit (AMD64)]
Qt: 5.15.2
PyQt5: 5.15.9
NumPy: 1.23.5
SciPy: 1.10.0
Dask: 2022.7.0
VisPy: 0.11.0
magicgui: 0.6.1
superqt: unknown
in-n-out: 0.1.6
app-model: 0.1.1
npe2: 0.6.2
OpenGL:

GL version: 4.6.0 - Build 30.0.101.1692
MAX_TEXTURE_SIZE: 16384

Screens:
- screen 1: resolution 1920x1080, scale 1.0

@Dr2-JMM
Glad it's working and yes now the scaling info is correct. This isn't really a napari bug, but rather an issue with your environment, perhaps you migrated computers or used external monitors with some other Qt app (not just python)? It's that or something in your Windows settings...
As your problem is resolved, I'm going to close this, but if the issue recurs or you have a way to replicate it, please feel free to re-open.

authors: Czaki Dr2-JMM psobolewskiPhD


text:
add option to disable overwrite of labels during painting, also an eraser

Description
Add toggling during painting of labels, to turn off overwriting of existing label, and eraser to erase labels instead of overwriting with background label as described in https://github.com/napari/napari/issues/1142
when overwrite is disabled, existing labels(other than background label - 0) won't be replaced,
also added an eraser to replace overwrite of background label
Type of change

[ ] Bug-fix (non-breaking change which fixes an issue)
[X] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

References
https://github.com/napari/napari/issues/1142
How has this been tested?

[X] created test to verify paint and erase with overwrite toggled off correctly mark only empty areas

Final checklist:

[X] My PR is the minimum possible work for the desired functionality
[X] I have commented my code, particularly in hard-to-understand areas
[X] I have made corresponding changes to the documentation
[X] I have added tests that prove my fix is effective or that my feature works

Very exciting for your first PR @ziyangczi!!! Thanks for the initial review too @tlambert03, I've left a couple minor comments too. Great job :-)

Thanks everyone for your feedback! I give it more thought and added the eraser function as suggested in the issue, I think it is a better solution to address overwriting with background label

adding self.selectionSpinBox.setMinimum(1) in the Qt controller doesn't prevent someone from setting it manually with layer.selected_label = 0 ... so it introduces the possibility that the GUI can become out of sync with the model.  For instance, set layer.selected_label = 0 in the console and you'll get this:

(the label in the spinbox does not agree with the color swatch and the radio button is still on zoom, even though it should be on erase)
certain key bindings are also now out of sync: pressing the "e" key for instance, should select the eraser tool and not change the label swatch... (it's the same as the previous point, but with a hotkey instead of going into the console).
the "d" key (decrease_label_id) also lets the user get the GUI out of sync now, by decreasing the label down to 0...
I actually really liked the shift key behavior!  personally, I would leave it in there as a toggle to the overwrite checkbox (i.e. the shift key swaps the overwrite behavior while pressed).
I think it would also be nice to have eraser toggle key, perhaps the option key similar to photoshop and other "deselect" tools .... but maybe we don't need both that and the e key, which permanently sets the eraser too?  thoughts?

Thanks for the feedback! I am personally thinking we can disallow using of label 0, other than an eraser use case, but I am very new to the project and can miss other's perspectives. Is there a valid case where someone would "paint" background than "erasing it"? I am thinking we should restrict every function from access the background element

I actually really liked the shift key behavior!  personally, I would leave it in there as a toggle to the overwrite checkbox (i.e. the shift key swaps the overwrite behavior while pressed).
I think it would also be nice to have eraser toggle key, perhaps the option key similar to photoshop and other "deselect" tools .... but maybe we don't need both that and the e key, which permanently sets the eraser too?  thoughts?

I think it would also be nice to have eraser toggle key, perhaps the option key similar to photoshop and other "deselect" tools .... but maybe we don't need both that and the e key, which permanently sets the eraser too?  thoughts?

icons are now too tightly packed on only the labels:

(everything else):

we could potentially change the "shuffle colors" button to a push button with a standard "shuffle" logo...  might give us some space

I think it would also be nice to have eraser toggle key, perhaps the option key similar to photoshop and other "deselect" tools .... but maybe we don't need both that and the e key, which permanently sets the eraser too?  thoughts?

I got this idea from "City Skylines" 😄 maybe other imaging tools have similar feature too, we can remove eraser as a button, but use right-click/mouse drag as "erasing" behavior, it might be a pain to erase on large area, maybe we can also implement double left click to fill and double right click to unfill/erasing? how's this idea sound?

I got this idea from "City Skylines" smile maybe other imaging tools have similar feature too, we can remove eraser as a button, but use right-click/mouse drag as "erasing" behavior, it might be a pain to erase on large area, maybe we can also implement double left click to fill and double right click to unfill/erasing? how's this idea sound?

Buttons are good, they are explicit for beginners. Mouse click combos are hard to discover, and right click should almost always be "open context menu". Middle click is flexible. =)
e could be "toggle eraser mode" rather than "set eraser mode". Indeed, maybe all mode shortcuts should work as toggles...?
@ziyangczi the other thing I would add is that scope creep in PRs is generally a bad idea — this PR was almost ready to merge but now we are discussing a totally unrelated UI issue! So I would recommend in the future to open a new branch + PR for each new feature.

Hi @ziyangczi, I just went through - overall looking good - but a couple comments.
It's great that you've begun to add erase mode, but I think this really belongs as two separate PRs. In general we try and keep our PRs as short and modular as possible - i.e. one piece of functionality at a time - as it makes review much easier and discussions more focused. This PR now adds both the overwrite toggle and the eraser. For the cases like this one when there are two related things and you want to work on both at the same time with functionality from the first one in the second one, then we make two PRs where the second one contains the changes from the first.
In this case if you could change this PR back to only having the overwrite and addition of the background label, and then make a new PR which had those changes + the addition of the erase button that would be great. I think we were very close to merging the overwrite toggle, but the eraser will require some more discussion as there are some quirks with who we think about erasing as a paintbrush, and erasing as a fill bucket (see some of my comments in the PR).
As to having a shift key vs the checkbox for the overwrite - I agree with @tlambert03 that having the modifier is much more natural, and it's what other tools in this space do. Maybe we can try having both, so the checkbox changes the default, and the modifier is a temporary toggle (which also updates the checkbox) but reverts on release. That way you have a visual representation of the state but you can very quickly change it using a hotkey. How does that sound?

Ah as I was writing my comment @jni commented too, but I didn't see it till now!! My apologies!!

reduced scope as suggested, going to open up another PR to discuss eraser

This is almost done @ziyangczi thanks for pulling out other code. There is one event that needs to be connected, then I think it is ready.
Maybe a final question for @tlambert03 and @jni though is do we like the word overwrite. I worry that overwrite sounds very intense, and this is the behavior that most people will expect by default. We could alternatively have block_painting which then get's set to True when you want the blocking behavior. I prefer this as it is more explicit and by default it is False.

This is almost done @ziyangczi thanks for pulling out other code. There is one event that needs to be connected, then I think it is ready.
Maybe a final question for @tlambert03 and @jni though is do we like the word overwrite. I worry that overwrite sounds very intense, and this is the behavior that most people will expect by default. We could alternatively have block_painting which then get's set to True when you want the blocking behavior. I prefer this as it is more explicit and by default it is False.

Definitely welcome suggestion over naming, I felt "overwrite" is very broad, too, and can lead to misunderstanding. "block_painting" also sounds like we are disabling painting all together, how about "replace_existing_label" ?

Definitely welcome suggestion over naming, I felt "overwrite" is very broad, too, and can lead to misunderstanding. "block_painting" also sounds like we are disabling painting all together, how about "replace_existing_label" ?

How about preserve_existing_labels and have it be False by default. I like having the default behavior as False because then people who don't want to use this functionality don't need to learn about it.

how about preserve_existing_labels

I was gonna say the same thing :)

Thanks everyone for your thorough review and very useful feedback! I finalized the PR and please take another look and let me know if things look good. Sorry the PR is messier than it should have been. Hopefully, as I ramp up speed, I would be more thoughtful and productive contributing on other issues

Thanks for that ! Very useful.

Awesome work @ziyangczi ! Amazing first PR!

source: gharchive/pull-request

authors: VolkerH jni kevinyamauchi sofroniewn tlambert03 ziyangczi


text:
Add pre-commit check for basic formattings bugs

Description
This PR add few checks from https://github.com/pre-commit/pre-commit-hooks repository to catch basing errors.

This got the "Ready-to-merge" label 2 days ago and got no objections. Merging to move the queue forward.
Thanks.

authors: Carreau Czaki


text:
feat(signal-slice): add signalSlice

This was created in collaboration with Chau, it adds signalSlice which is loosely inspired by the createSlice API from Redux Toolkit. The general idea is that it allows you to declaratively create a "slice" of state. This state will be available as a readonly signal.
The key motivation, and what makes this declarative, is that all the ways for updating this signal are declared upfront with sources and reducers. It is not possible to imperatively update the state.
This PR also makes minor changes to connect in order to expose types that are shared by this utility.
Example using all API features:
  state = signalSlice({
    initialState: this.initialState,
    sources: [this.sources$],
    reducers: {
      add: (state, checklist: AddChecklist) => ({
        checklists: [...state.checklists, this.addIdToChecklist(checklist)],
      }),
      edit: (state, update: EditChecklist) => ({
        checklists: state.checklists.map((checklist) =>
          checklist.id === update.id
            ? { ...checklist, title: update.data.title }
            : checklist
        ),
      }),
      remove: (state, id: RemoveChecklist) => ({
        checklists: state.checklists.filter((checklist) => checklist.id !== id),
      }),
    },
    selectors: (state) => ({
      loadedAndError: () => state().loaded && state().error,
      whatever: () => 'hi',
    }),
  });

Key features:

Sources will automatically update signal when they emit
Reducers will automatically create actions on the state object that can be called (e.g. this.state.remove(id))
Top-level properties of the initial state are automatically exposed as selectors (computeds) on the state object (e.g. this.state.loaded()`)
Additional selectors can be supplied which will also be added to the state object

For my own purposes this API is "feature complete" and does everything I need but I'm very open to comments/revisions/improvements.

@all-contributors please add @joshuamorony for code

authors: joshuamorony nartc

repo: nartc/ngxtension-platform

license: MIT


text:
Fix #581, Propagate the OSAL compile definitions to CFE build

Describe the contribution
Use the INTERFACE_COMPILE_DEFINITIONS and INTERFACE_INCLUDE_DIRECTORIES properties from the osal target and apply them to the entire CFE build as a directory-scope property.
At this time, the OSAL library build does not use/export these properties so this is effectively a no-op for the CFE build and can be merged with no effect.  However, in a future version (specifically after nasa/osal#312 gets fixed), the OSAL library will export these interface properties and this will become important.
Fixes #581
Testing performed
Built for all three test platforms (ppc-vxworks6.9, i686-rtems4.11, native/x86-64 Linux).  Sanity Check CFE build boots and runs, unit tests run.
Expected behavior changes
No impact to behavior - build is identical because these properties are currently empty/not set in the current OSAL build.
System(s) tested on
Ubuntu 18.04 LTS 64 bit.
Additional context
Also locally tested against a version of OSAL+PSP that includes all the latest proposed changes and confirm that this still works.
Contributor Info - All information REQUIRED for consideration of pull request
Joseph Hickey, Vantage Systems, Inc.

CCB 20200408 - APPROVED

Rebased on master so it can be tested with the bundle

authors: astrogeco jphickey skliper

repo: nasa/cFE


text:
anomaly/gmm: Add GMM-based change detection algorithm

Adding the GMM change detection code to anomaly/
Still needs work:

[ ] pickling GMMs (caching)
[ ] removal of old code (old clustering algorithm)
[ ] streamlining of results
[ ] testing of ground truth localized data

this code was merged in https://github.com/nasa/isaac/pull/100

authors: jamiesantos marinagmoreira

repo: nasa/isaac


text:
thing before 1.0

main things needed for 1.0 launch.
ban system.
main home page.
make a password admin password in install.php
admin post view, should look like my uploader draft.
make board conf forms in admin panel
universal admin panel (requires no board to log in with)
make super basic module system.
edit post button.
convert form to html multi form
modules, i want to have each module be a folder that contains a main script and a library for them to talk about and store board data with out having to design it to much. there should be a wiki to help figure it out
i can really call a 1.0 release when i got ban and home made.

exitif data stripping, imagemagic moduels should do that. apply duing image prossesing.

repo: nashikouen/kotatsuBBS


text:
T142 -  Recode update product function, delete media when update

Before update product:

Start update product:

After update product, media will be deleted in database:

actually here we can create 2 pull request, 1 media refactoring and 1 for change in product
I will learn from experience.

repo: nashtech-garage/yas


text:
Corrected spelling

https://github.com/CSSEGISandData/COVID-19/issues/124#issuecomment-592957638

@ChooseYourPlan Thank you so much.

@all-contributors please add @ChooseYourPlan for spelling checking

@all-contributors please add @ChooseYourPlan for translation


text:
Vivint Integration Broken after update 2023.8.0

After update home assistant flags the vivint devices as "invalid" and they do let load; the entities pull in fine but now I have no devices that I can automate or interact with, just entities- can you help?

Can you enable debug logs and post any relevant information from it? Right now, it's difficult for me to guess what the issue is and I haven't updated to 2023.8 yet myself.

i can attach the full log if you want, thank you so much for your quick response

repo: natekspencer/hacs-vivint


text:
BGP Route Dampening

What feature would you like to see?
Can pathvector setting up BGP Route Dampening features to BIRD2 configuration?
I think implementing BGP Route Dampening is really good idea, since sometimes when configurator did something fatal, it will cause flapping too much :(

BIRD (the underlying BGP daemon that Pathvector uses) doesn't currently support route dampening. This would be a feature request for the BIRD team.

authors: natesales rapdodge

repo: natesales/pathvector


text:
Serving

What do cool people such as yourself do when they want to prcl -w concurrently with running, say, a local HTTP server? :-)

http-server -s & prcl -w main.js bundle.js

Cheers! I hadn't realised you could use & in NPM scripts -- this used to require a separate library like concurrently, AFAICT. Thanks also for the -s tip :-)

I probably would not put that all in one NPM script; I'd have a separate one for serving and watching and then do npm start serve& npm start watch (or use make).

Okay, thanks :)

authors: nathan tjvr

repo: nathan/parcel

license: mit

license_source: bigquery


text:
Mac OS Sierra cannot run PhantomJS 1.9.7

When running mocha-phantomjs (or plain PhantomJS for that matter) on the new Mac OS Sierra, it fails with the following error:

phantomjs terminated with signal SIGSEGV

Upgrading to a newer PhantomJS version solves this issue (i'm using 2.1.7 right now).
I found several issues in the past which requested a newer PhantomJS version, but these were rejected.
I do think that we cannot reject it any longer, as Mac OS Sierra no longer works with this older version.
@nathanboktae what do you think? I can file a PR if needed.

same issue here, no idea what to do

Use the -p option to specify a different phantomjs binary.
Switch to mocha-phantomjs-core. I will eventually add the same CLI interface to that project as this one and then it will be basically the same as this one but without a direct binary dependency.

distributing the phantomjs binary has been a bag of hurt (#167, #231, #224) and I'm not going to do it going forward. mocha-phantomjs-core will be the future, and with the CLI and even some shell wrappers it'll feel the same project.

I switched to mocha-phantomjs-core

authors: latel nathanboktae paulrutter

repo: nathanboktae/mocha-phantomjs


text:
Update aws-java-sdk-dynamodb to 1.12.711 in scala-steward-dependencies

About this PR
📦 Updates com.amazonaws:aws-java-sdk-dynamodb from 1.12.710 to 1.12.711
Usage
✅ Please merge!
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

⚙ Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "com.amazonaws", artifactId = "aws-java-sdk-dynamodb" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "30 days" },
  dependency = { groupId = "com.amazonaws", artifactId = "aws-java-sdk-dynamodb" }
}]

labels: test-library-update, early-semver-patch, semver-spec-patch, commit-count:1

Superseded by #198.

repo: nationalarchives/da-reference-generator


text:
Update tdr-graphql-client to 0.0.175 in scala-steward-dependencies

About this PR
📦 Updates uk.gov.nationalarchives:tdr-graphql-client from 0.0.174 to 0.0.175
📜 GitHub Release Notes - Version Diff
Usage
✅ Please merge!
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

⚙ Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "uk.gov.nationalarchives", artifactId = "tdr-graphql-client" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "30 days" },
  dependency = { groupId = "uk.gov.nationalarchives", artifactId = "tdr-graphql-client" }
}]

labels: library-update, early-semver-major, semver-spec-patch, commit-count:1

Superseded by #526.


text:
Update rds, s3, ssm to 2.24.8 in scala-steward-dependencies

About this PR
📦 Updates

software.amazon.awssdk:rds
software.amazon.awssdk:s3
software.amazon.awssdk:ssm

from 2.24.4 to 2.24.8
Usage
✅ Please merge!
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

⚙ Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "software.amazon.awssdk" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "30 days" },
  dependency = { groupId = "software.amazon.awssdk" }
}]

labels: library-update, early-semver-patch, semver-spec-patch, commit-count:1

Closing as already upgraded.

authors: TomJKing tna-digital-archiving-jenkins

repo: nationalarchives/tdr-statuses


text:
Can permissions (e.g. to clipboard access / notifications) be granted automatically?

Is it possible to automatically grant permissions like clipboard access or notifications?

@AgentSmith0 I assume you are talking about macOS permissions. To my knowledge, no, that'd defeat the purpose. Closing, but feel free to come back with more info and we can re-open if there's something worth adding.

I didn't mean macOS permissions specifically, I mean permissions in general. For example if your website requires microphone access, the browser is asking the user for permission. But I didn't see any popup in my nativefier app, so I wanted to ask if there is a way to force allowing some or all permissions.
Here is an example for such a permission popup:
https://developer.mozilla.org/en-US/docs/Web/API/WebRTC_API/Build_a_phone_with_peerjs/Connect_peers/Get_microphone_permission/1oieizmfmllnygbsdu4f1iq.png
Here are some links:
https://developer.mozilla.org/en-US/docs/Web/API/Permissions_API
https://developer.mozilla.org/en-US/docs/Web/API/WebRTC_API/build_a_phone_with_peerjs/connect_peers/get_microphone_permission
https://developer.mozilla.org/de/docs/Web/API/notification/permission
https://developer.mozilla.org/en-US/docs/Web/API/Clipboard/read

@AgentSmith0 right, got confused about the question.
Did you try it? Nativefier grants these permissions already. Try it for yourself:

✔ https://async-clipboard-api.glitch.me/ → both Copy/Paste buttons work
✔ https://www.html5rocks.com/en/tutorials/getusermedia/intro/ → "Capture video" and "Stop" work
✘ https://developer.mozilla.org/en-US/docs/Web/API/Notifications_API/Using_the_Notifications_API → "Notify me!" button doesn't work, this may be https://github.com/nativefier/nativefier/issues/1078

, so I guess your question becomes "Can/should permissions like clipboard / microphone / video/notifications be denied (maybe by default), and allowed/denied via a flag". Seems legit, renaming the issue.

Are the permissions also automatically granted when the website is not using a https protocol?

Are the permissions also automatically granted when the website is not using a https protocol?

I'd be unsure without a test site to try on. You could also add the --insecure flag during app creation to see if that helps.

Local server without https do not have all permissions automatically granted. I tried it with a server in my network, it's address is something like http://192.168.0.10
I already tried adding the --insecure flag, but this doesn't solves my issue. Is there a way to fix this?

I was able to use https on my server and everything is working fine with nativefier. I will close this issue for now.
Thank you all for your great help!

authors: AgentSmith0 TheCleric ronjouch

repo: nativefier/nativefier


text:
ui-material-tabs break on back navigation (ios)

Which platform(s) does your issue occur on?

iOS 15

Please, provide the following version numbers that your issue occurs with:

CLI:8.1.4
Cross-platform modules:  "@nativescript/core": "8.1.5",
Runtime(s): JSC and 8.1.0
Plugin(s): "@nativescript-community/ui-material-tabs": "~6.2.2", I tried down to 5.x versions, same problem with older packages as well. I suspect it to be related to IOS 15, IOS 11 on emulator does not show the problem

Please, tell us how to recreate the issue in as much detail as possible.
As can be seen in the attached video, when transitioning back to the view, the top tabbar is shifted by a certain amount into the page. Could have to do with OverflowSaveArea?
Forward/Backword navigation with vertical slide transition is responsible for the bug to show up, other page transitions do not lead to this error
https://user-images.githubusercontent.com/43253483/139599962-45c453cd-12a8-412f-b1c8-febfa60fa942.mov
<MDTabs animationEnabled="false" offscreenTabLimit="4" swipeEnabled="false" [selectedIndex]="_selectedIndex"
		(selectedIndexChanged)="onSelectedIndexChanged($event)" class="border-top">
		<MDTabStrip>
			<MDTabStripItem>

			</MDTabStripItem>

			<MDTabStripItem>

			</MDTabStripItem>

			<MDTabStripItem>

			</MDTabStripItem>

			<MDTabStripItem>

			</MDTabStripItem>
		</MDTabStrip>

		<MDTabContentItem>
			<ContentView>
				<ns-search-tab *ngIf="_showSearchTab"></ns-search-tab>
			</ContentView>
		</MDTabContentItem>

		<MDTabContentItem>
			<ContentView>
				<ns-custom-tab *ngIf="_showCustomTab"></ns-custom-tab>
			</ContentView>
		</MDTabContentItem>

		<MDTabContentItem>
			<ContentView>
				<ns-meals-tab *ngIf="_showMealsTab"></ns-meals-tab>
			</ContentView>
		</MDTabContentItem>

		<MDTabContentItem>
			<ContentView>
				<ns-quickly-add-tab *ngIf="_showQuicklyAddTab"></ns-quickly-add-tab>
			</ContentView>
		</MDTabContentItem>
	</MDTabs>

@cjohn001 would be good to try and replicate that issue with N core version (no plugin). It could help pinpoint where the issue comes from

@farfromrefug. I do not know how to work with N core, only Angular.

@cjohn001 you can use ns create to create a simple example on which you can try and replicate the issue. It is essential to be able to pinpoint the issue

Well I currently do not have the time to look into it. I need to see how to find a workaround the problem first.

@farfromrefug : I will close the issue as there seems to be black magic happening (behavior was 100% repeatable yesterday).
I have rebuild the app this morning and cannot see the reported behavior anymore. As all packages are in the same version like they were yesterday and I made no code changes, I do not understand whats happening. I think it makes no sense to debug something which is not repeatable, therefore I close the issue


text:
BottomNvigationBar : Tabpressed not fired on the first Item with Vue+NS+Android

Hello,
I use BottomNavigationBar with Vue.js.
I set isSelectable to False to each item so that I can use the TabPressed event, the problem I found is that this event is not fired on the first item only (the two others work fine!)  :
 <MDBottomNavigationBar id="id_tab_nav" ref="my_tab_navigation_bottom" backgroundColor="#70152b" activeColor="#FFF" inactiveColor="#FFF" row="1"  @tabPressed="tabPressedEvent"    >

        <MDBottomNavigationTab color="#FFF" title="Home" icon="res://home" attr-link="Home"
        isSelectable="false"   />

        <MDBottomNavigationTab color="#FFF" title="Search" icon="res://search" attr-link="Search"
        isSelectable="false"   />

        <MDBottomNavigationTab color="#FFF" title="User"  icon="res://user" attr-link="Login"
        isSelectable="false"    />

    </MDBottomNavigationBar>

Here is my function for the tabPressed event :

        tabPressedEvent(e)
    {
         console.log('element=>refs',this.itemTabNavView.items);

         console.log('navigateTo Item =>');
         console.log(e.index);
        this.$navigateTo(this.items_navs[e.index]);

    }

Is there any solution to solve this problem ?
Thanks.

tabPressed is not emited if the  tabs are not selectable

@Echoju thanks for.pointing that out. if you can create a PR if you find the fix that would be awesome!

authors: Echoju farfromrefug


text:
Unexpected changes of default timeout value in NatsRequestCompletableFuture

Defect
In recent PR https://github.com/nats-io/nats.java/pull/659, which was dedicated to fix issue https://github.com/nats-io/nats.java/issues/658, the default waiting for response timeout was decreased from 10 MINUTES to 2 SECONDS in NatsRequestCompletableFuture.
I know that there is possibility to use Connection#request with explicitly specified timeout as method parameter, but a lot of developers which were using this library previously may relay on default 10m timeout. So, to start using new version io.nats:jnats they must to review all places in their code when any request is sending to NATS and explicitly specify needed timeout. Moreover, such implicit changes of timeout may lead to unexpected bugs. Fortunately, I've been lucky to notice this problem by failing integration test before merging version update to master branch.
As for me, 2s delay is not enough to consider it as error scenario and cancel NatsRequestCompletableFuture. But if this is not a bug from your point of view and there is no possibility to revert timeout changes in NatsRequestCompletableFuture, it would be nice to have some configurable connection property dedicated for default response timeout rather than just use Options.DEFAULT_CONNECTION_TIMEOUT constant value. Such approach will make migration to newer version of io.nats:jnats less painful.
Versions of io.nats:jnats and nats-server:
2.15.5
OS/Container environment:
Steps or code to reproduce the issue:
Invoke Connection#request method without specifying timeout
Expected result:
Response timeout more then 2s
Actual result:
Response timeout 2s

I will change this to be the request timeout plus a small amount. I see how this is a problem for devs using longer than 2 sec timeout for their request.

@osadchuk-roman For now, you can use the Connection#requestWithTimeout

Thanks @scottf, yes, I know that Connection#requestWithTimeout could be used for now.
As for me, it will be great to have separate option instead of Options.DEFAULT_CONNECTION_TIMEOUT, since "connection timeout" is not the same as "response timeout".  Or at least use options.getConnectionTimeout(), which returns value defined by developer, instead of default constant.

@osadchuk-roman can you look at this PR and comment?

@scottf In general I'm OK with such solution.
But I still believe that better introduce new property rather than use "ConnectionTimeout". For example: as a developer I may want to have connection timeout to NATS = 2s, but handling of my requests may take up to 2m. So, I want to be able to await for response for 2m, if during sending of request connection to NATS was established successfully in 2s.
With your current approach in PR https://github.com/nats-io/nats.java/pull/691, same property will be used for these 2 different purposes.

Ok changed to use RequestCleanupInterval

https://github.com/nats-io/nats.java/pull/693/

authors: osadchuk-roman scottf


text:
Closing the NATS instance with no connection established throws NatsError: Subject must be supplied

When attempting to close a node-nats-streaming instance that is trying to connect to a cluster because waitOnFirstConnect is set to true, it throws an "NatsError: Subject must be supplied" error every time.
Attached is a script that showcases the problem. Am I using close in an incorrect way? If yes, what is the best practice of closing a nats connection that might not be connected yet because the application is closing?
// docker run -p 4222:4222 -p 8223:8223 nats-streaming
const natsstreaming = require('node-nats-streaming')
const host = `nats://192.168.99.100:4224`
const opts = {
  url: host,
  maxReconnectAttempts: 12,
  reconnect: true,
  reconnectTimeWait: 6000,
  verbose: true,
  waitOnFirstConnect: true
}

const stan = natsstreaming.connect('test-cluster', `some client id`, opts)

stan.on('connect', () => {
  console.log('NATS connection established.')
})

stan.on('reconnect', () => {
  console.log('NATS reconnected.')
})

stan.on('reconnecting', () => {
  console.log('NATS reconnecting...')
})

stan.on('disconnect', () => {
  console.log('NATS disconnected.')
})

stan.on('error', (err) => {
  console.error(err)
})

stan.on('close', () => {
  console.log('NATS connection closed.')
})

process.on('SIGINT', () => {
  console.log('Received SIGINT, closing NATS instance')
  stan.close()
})

process.on('SIGTERM', () => {
  console.log('Received SIGTERM, closing NATS instance')
  stan.close()
})

setTimeout(() => {
  console.log('attempting to close nats')
  stan.close()
}, 2500)

Output when application is run without further user interaction:
NATS disconnected.
attempting to close nats

[...]\node_modules\nats\lib\nats.js:1379
            throw (new NatsError(BAD_SUBJECT_MSG, BAD_SUBJECT));
            ^
NatsError: Subject must be supplied
    at Client.publish ([...]node_modules\nats\lib\nats.js:1379:20)
    at Client.request ([...]node_modules\nats\lib\nats.js:1606:10)
    at Client.requestOne ([...]node_modules\nats\lib\nats.js:1685:17)
    at Stan.close ([...]node_modules\node-nats-streaming\lib\stan.js:364:11)
    at Timeout.setTimeout [as _onTimeout]
    [...]

You can also remove the setTimeout code block and issue a SIGTERM at some point in time. Output:
NATS disconnected.
NATS reconnecting...
NATS disconnected.
NATS reconnecting...
NATS disconnected.
NATS reconnecting...
NATS disconnected.
Received SIGINT, closing NATS instance

[...]node_modules\nats\lib\nats.js:1379
            throw (new NatsError(BAD_SUBJECT_MSG, BAD_SUBJECT));
            ^
NatsError: Subject must be supplied
    at Client.publish ([...]node_modules\nats\lib\nats.js:1379:20)
    at Client.request ([...]node_modules\nats\lib\nats.js:1606:10)
    at Client.requestOne [...]node_modules\nats\lib\nats.js:1685:17)
    at Stan.close ([...]node_modules\node-nats-streaming\lib\stan.js:364:11)
    at process.on ([...]\connection_natsonly.js:42:8)
    at process.emit (events.js:197:13)

Edit:
node-nats-streaming version: 0.0.52

The STAN protocol requires for the stan connection to form before any of the STAN API is called.
@aknuds1 thanks for the report.

Fixed by #95

authors: anXieTyPB aricart

repo: nats-io/node-nats-streaming


text:
YouTube API key verification failed

My error log shows the following errors:
jun 19 20:30:19 raspberrypi mopidy[27280]: ERROR    [YouTubeBackend-3] mopidy_youtube Failed to verify YouTube API key, disabling API
jun 19 20:30:19 raspberrypi mopidy[27280]: INFO     [YouTubeBackend-3] mopidy_youtube using bs4API
I created an API key for 'YouTube Data API v3' on https://console.developers.google.com/apis/
[youtube]
enabled = true
youtube_api_key = ***************************************
api_enabled = true
search_results = 30
The amount of stars in my configuration above represents the length of the API key which I copied from Google's API library.
I tried to delete the API key and create another one. I also tried to use another email address to create the API key. Both with no luck.
Do you have any suggestions?
Thanks in advance

Strange. How would you feel about sending me an API key that you have created and that you have confirmed doesn't work. I could test it, and then you can cancel it.

Thank you for your quick reply. I sent you an email with extra information including the full API key. Please let me know if this is sufficient for you to test.

Hi @Archiefkast - thanks for the above and for your email. I have tested your API key; the error I get is:
2020-06-20 20:46:46,691 ERROR [16374:YouTubeBackend-6] mopidy_youtube: search error "{'code': 403, 'message': 'The request cannot be completed because you have exceeded your quota.', 'errors': [{'message': 'The request cannot be completed because you have exceeded your quota.', 'domain': 'youtube.quota', 'reason': 'quotaExceeded'}]}"
2020-06-20 20:46:46,693 ERROR [16374:YouTubeBackend-6] mopidy_youtube: Failed to verify YouTube API key, disabling API

I'm not sure what your quota is, or what you might use it for, or how much using mopidy-youtube uses it up, but it looks like it has been exceeded?  Maybe you can have a look at your usage in the google console?

Thank your for your research!
The strange thing is, I see that there are a few requests on the API key that I generated yesterday. All these request results in an error:

In the quota section, the usage indicates 0 and it seems that no quota has been reached:

The limit of 0 queries per day could be your problem.  Mine has a limit of 10000 per day, and mopidy-youtube does use them.  You can see the small number of queries I ran with my API key before I tried with your API key:

This should be the cause. I will look for more information about this 0 requests per day limit. I like the API connection because of the personal playlists.
This issue can be closed now, it looks like it has nothing to do with the plugin.
Thank you very much for your effort @natumbri !

I deleted my project on https://console.developers.google.com/apis/ and created a new one. Then I created a new API key, entered it in my Mopidy configuration and restarted Mopidy. Now it is working! See the log below:
jun 20 14:13:57 raspberrypi mopidy[7249]: INFO     [YouTubeBackend-3] mopidy_youtube YouTube API key verified
Again, thank you very much @natumbri ! You made my day :)

Glad it is working for you! Would you mind just explaining a bit how you use personal playlists? I'm interested to know how people actually use the plugin, especially if they are using it in a way different to how I use it.

My setup:

Mopidy with the YouTube plugin
Icecast: a free, open source media streaming server (https://icecast.org/)
Home Assistant: a free, open source home automation solution (https://www.home-assistant.io/)
Google chromecast audio (in several rooms of my home)

Streaming through Icecast
I have configured the Mopidy audio output to stream through Icecast:
(https://docs.mopidy.com/en/release-2.2/audio/#streaming-through-icecast)
Select a playlist in Home Assistant to play in a room
In Home Assistant I created a dropdown for selecting a playlist (that will automatically load a YouTube playlist in Mopidy):

In Home Assistant I also created a dropdown for each room with a Google chromecast audio to select the Mopidy input:

The only thing I have to figure out is how I can let Home Assistant shuffle Mopidy's playlist after loading a YouTube playlist.
Please let me know if you have any further questions.

Thanks for the detailed description.  I'm doing something similar, but with snapcast instead of icecast, and domoticz instead of home assistant.
I don't know how home assistant works, but if it can do this
curl -d '{"jsonrpc": "2.0", "id": 1, "method": "core.tracklist.shuffle"}' -H 'Content-Type: application/json' http://localhost:6680/mopidy/rpc

... your playlist will be shuffled.

You're welcome. Thank you for the suggestion, I will try that one.

Also, if you want random play mode, instead of shuffling the playlist, you could do this:
curl -d '{"jsonrpc": "2.0", "id": 1, "method": "core.tracklist.set_random", "params": {"value": true}}' -H 'Content-Type: application/json' http://localhost:6680/mopidy/rpc

One good thing about snapcast is the client runs on android (so you can use an old android phone as a receiver) and on openwrt, so you can use just about any device that is supported by openwrt and has a usb port as a client.
I've done it with some success using a TL-WR703N (which is a very, very basic router) with a very cheap CM108 usb sound card.

That's really a good thing, it offers a way to reuse old devices. This is a nice one for my holiday :)
Thank you very much for your help and your suggestions!

Random play mode works like a charm! :)

authors: Archiefkast natumbri

repo: natumbri/mopidy-youtube


text:
Error on Homebrewery. Got directed here.

Error: Bad Request
Error: Bad Request
    at Request.<anonymous> (https://homebrewery.naturalcrit.com/homebrew/bundle.js:71842:19)
    at Request.Emitter.emit (https://homebrewery.naturalcrit.com/homebrew/bundle.js:1768:20)
    at XMLHttpRequest.xhr.onreadystatechange (https://homebrewery.naturalcrit.com/homebrew/bundle.js:72151:10)
{
  "status": 400,
  "method": "POST",
  "url": "/api/newGoogle/"
}

Check out the steps in discussion #1580 for resolving this.  Will close this Issue for now, but it can be reopened if those steps don't work for you.

repo: naturalcrit/homebrewery


text:
Issue saving

Error Code
Error: Forbidden

Error: Forbidden
    at Request.<anonymous> (https://homebrewery.naturalcrit.com/homebrew/bundle.js:80465:19)
    at Emitter.emit (https://homebrewery.naturalcrit.com/homebrew/bundle.js:1966:20)
    at xhr.onreadystatechange (https://homebrewery.naturalcrit.com/homebrew/bundle.js:80774:10)
{
  "status": 403,
  "method": "PUT",
  "url": "/api/update/LUTrc4yrVNp-"
}

### Your description of what happened:

It won't let me save any of my documents. I've tried logging out of Google and back in, but it hasn't helped.

Are you signed in to the correct Google account? Google is telling us "The user does not have sufficient permissions for this file."

authors: calculuschild elaverick


text:
Support for JSONField in Nautobot Jobs

As ...
P.D. - Plugin Developer
I want ...
the ability to have a JSON field as a job input
So that ...
I can pass JSON data to my job
I know this is done when...

Pass in valid JSON and inside the job I can interact with the serialized data

Optional - Feature groups this request pertains to.

[X] Automation
[ ] Circuits
[ ] DCIM
[ ] IPAM
[ ] Misc (including Data Sources)
[ ] Organization
[ ] Plugins (and other Extensibility)
[ ] Security (Secrets, etc)
[ ] Image Management
[ ] UI/UX
[ ] Documentation
[ ] Other (not directly a platform feature)

Database Changes

None

External Dependencies

None

Implemented for 2.1 by #4927.

repo: nautobot/nautobot


text:
Support dynamic definition of powerport and interface types

User Stories
1
As Nelly the Network Engineer,
I want to be able to accurately represent the hardware present in my network, including any number of newly introduced power port types and interface types,
So that my source of truth provides as much detailed and accurate information as possible.
I will know this is done when I can add new hardware to Nautobot without needing to wait for a new release of Nautobot to support this hardware definition.
2
As Cora the Coder,
I want to get out of the business of needing to write new Nautobot code (even if only a few lines) every time a hardware manufacturer or standards body develops a new interface hardware type or power port connector type,
So that I can focus my time and development effort on other Nautobot features and fixes.
I will know this is done when I no longer need to release a new version of Nautobot to support a new interface hardware type or power port connector type.

Database Changes (Optional)
This would follow a similar model to the customizable Status model, wherein we replaced a hard-coded list of supported values with a new database model. Probably we will need one model for InterfaceType and another model for PowerPortType.
We'll need to be cautious about how we migrate existing data to use the new model, as the way that we autogenerate Status values at install time has occasionally been problematic with regard to testing and reuse of test databases.

External Dependencies (Optional)
We'll need to think about how to retain compatibility with https://github.com/netbox-community/devicetype-library as it would be a shame to lose the ability to import hardware types from that repository.

I would give an upvote to this concept.  If we can abstract the interface and data type being passed over this interface, nautobot would be much more flexible to progress made in networking and also could expand into other areas such as media interfaces.  Right now we have different groups of interfaces such as Ethernet, Serial, SONNET, Wireless.  I would love to be able to add "Media" with interfaces like XLR, BNC, RCA Pin Plug, 1/4" TRS, etc.  This would open up other applications and markets.

We also require this feature to define a custom interface group because the current nautobot version has LAG grouping for physical interfaces but we need to group virtual interfaces also where SD-WAN is being used.

I wonder if this would also apply to things like cable types as well or should that be an additional feature request

Sure, it would be a good idea to review all of the static lists in nautobot/dcim/choices.py and see which of the following would make sense to make extensible:

ConsolePortTypeChoices
PowerPortTypeChoices
PowerOutletTypeChoices
InterfaceTypeChoices
PortTypeChoices
CableTypeChoices

authors: aaashok1225 glennmatthews nniehoff pozar


text:
🛑 BarefootEdu is down

In f480664, BarefootEdu (https://barefootedu.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: BarefootEdu is back up in 2efc0fb after 17 minutes.

repo: navadhiti/upptime


text:
🛑 BarefootEdu is down

In bf354ba, BarefootEdu (https://barefootedu.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: BarefootEdu is back up in d80f0fa after 38 minutes.


text:
🛑 Harmony Bot Website is down

In 18e49d0, Harmony Bot Website ($HARMONY_WEB) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Harmony Bot Website is back up in d3350ed.


text:
🛑 Harmony Bot Website is down

In 0a3b29a, Harmony Bot Website ($HARMONY_WEB) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Harmony Bot Website is back up in c337584.


text:
🛑 Harmony Bot Website is down

In 8fef602, Harmony Bot Website ($HARMONY_WEB) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Harmony Bot Website is back up in 731d90d.


text:
🛑 Harmony Bot Website is down

In 615b314, Harmony Bot Website ($HARMONY_WEB) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Harmony Bot Website is back up in 34b765d.


text:
🛑 Harmony Bot Website is down

In 950a8d8, Harmony Bot Website ($HARMONY_WEB) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Harmony Bot Website is back up in 3a1bf3a.


text:
🛑 Harmony Bot Website is down

In 6b8813e, Harmony Bot Website ($HARMONY_WEB) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Harmony Bot Website is back up in af88a70.


text:
🛑 Harmony Bot is down

In 38322a8, Harmony Bot ($HARMONY) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Harmony Bot is back up in 1bfd067.


text:
🛑 Harmony Backup Bot is down

In a9f985f, Harmony Backup Bot ($HARMONY_BACKUP) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Harmony Backup Bot is back up in 17c5402.


text:
🛑 Harmony Bot Website is down

In b6739e3, Harmony Bot Website ($HARMONY_WEB) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Harmony Bot Website is back up in 078349e.


text:
🛑 Harmony Bot Website is down

In 9662483, Harmony Bot Website ($HARMONY_WEB) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Harmony Bot Website is back up in 05ab287.


text:
🛑 Harmony Bot Website is down

In 5db8c13, Harmony Bot Website ($HARMONY_WEB) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Harmony Bot Website is back up in 0dfb5ad.


text:
Brandbok i design.nav.no

Overføring av hele brandbok https://navno.sharepoint.com/sites/intranett-kommunikasjon/SitePages/Design-og-profil.aspx?web=1 til design.nav.no
Trenger noen sider å være private?

Brandbok har blitt flyttet til Frontify. Vi må linke til Frontify fra design.nav.no i stedet.

authors: randisolli sergiohaisch

repo: navikt/Designsystemet


text:
Ikke 'refetch on focus' i redigeringsskjemaene

Litt usikker på om vi skulle gjort overalt hvor useAvtale brukes? Men f. eks i notat oversikt er det vel greit at den refetcher på focus?

Ja, refetching er der jo by default for at man ønsker at ting har riktig tilstand. Det kan evt. være at issuet her ikke er refetch, men at vi har litt for ivrig rendre-logikk av loading-tilstanden? Hvordan oppfører skjema seg hvis vi lar være å rendre spinner hvis avtale allerede har lastet data tidligere - blir skjema resatt da?

Ja, refetching er der jo by default for at man ønsker at ting har riktig tilstand. Det kan evt. være at issuet her ikke er refetch, men at vi har litt for ivrig rendre-logikk av loading-tilstanden? Hvordan oppfører skjema seg hvis vi lar være å rendre spinner hvis avtale allerede har lastet data tidligere - blir skjema resatt da?

Ja. Jeg endret fra å bruke isFetching til isLoading når man viser  og det fiksa også problemet

authors: fredrikpe sondrele

repo: navikt/mulighetsrommet


text:
🛑 NBB is down

In 8cbeefb, NBB (https://stat.nbb.be/restsdmx/sdmx.ashx/GetDataStructure/ALL) was down:

HTTP code: 429
Response time: 747 ms

Resolved: NBB is back up in e603e11.

repo: nbbrd/sdmx-upptime


text:
🛑 OECD is down

In e2b5065, OECD (https://stats.oecd.org/restsdmx/sdmx.ashx/GetDataStructure/ALL) was down:

HTTP code: 0
Response time: 0 ms

Resolved: OECD is back up in 25c98df.


text:
🛑 ISTAT is down

In 3614a89, ISTAT (https://esploradati.istat.it/SDMXWS/rest/dataflow/all/all/latest) was down:

HTTP code: 0
Response time: 0 ms

Resolved: ISTAT is back up in b1b81ac after 1 hour, 6 minutes.


text:
🛑 ISTAT is down

In bb7dc70, ISTAT (https://esploradati.istat.it/SDMXWS/rest/dataflow/all/all/latest) was down:

HTTP code: 0
Response time: 0 ms

Resolved: ISTAT is back up in 4c2dd11 after 1 hour, 15 minutes.


text:
🛑 ISTAT is down

In cd01f3f, ISTAT (https://esploradati.istat.it/SDMXWS/rest/dataflow/all/all/latest) was down:

HTTP code: 0
Response time: 0 ms

Resolved: ISTAT is back up in 327a715 after 1 hour, 16 minutes.


text:
🛑 ISTAT is down

In b5aa7fb, ISTAT (https://esploradati.istat.it/SDMXWS/rest/dataflow/all/all/latest) was down:

HTTP code: 0
Response time: 0 ms

Resolved: ISTAT is back up in 80da09b after 4 minutes.


text:
🛑 ESTAT is down

In f7b7f5e, ESTAT (https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/dataflow/ESTAT/all/latest) was down:

HTTP code: 0
Response time: 0 ms

Resolved: ESTAT is back up in 507607c after 5 minutes.


text:
🛑 IMF is down

In 467f1c3, IMF (http://dataservices.imf.org/REST/SDMX_XML.svc/Dataflow) was down:

HTTP code: 0
Response time: 0 ms

Resolved: IMF is back up in 528aeb0 after 8 minutes.


text:
🛑 ISTAT is down

In 3d3c442, ISTAT (https://esploradati.istat.it/SDMXWS/rest/dataflow/all/all/latest) was down:

HTTP code: 0
Response time: 0 ms

Resolved: ISTAT is back up in bd7bdc7 after 10 minutes.


text:
🛑 ISTAT is down

In 08b8e83, ISTAT (https://esploradati.istat.it/SDMXWS/rest/dataflow/all/all/latest) was down:

HTTP code: 0
Response time: 0 ms

Resolved: ISTAT is back up in 5582cfd after 35 minutes.


text:
🛑 ESTAT is down

In 8211ca0, ESTAT (https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/dataflow/ESTAT/all/latest) was down:

HTTP code: 0
Response time: 0 ms

Resolved: ESTAT is back up in 01d0100 after 9 minutes.


text:
🛑 ESTAT is down

In 44f0aad, ESTAT (https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/dataflow/ESTAT/all/latest) was down:

HTTP code: 0
Response time: 0 ms

Resolved: ESTAT is back up in 470d17b after 11 minutes.


text:
🛑 ESTAT is down

In 76fdf42, ESTAT (https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/dataflow/ESTAT/all/latest) was down:

HTTP code: 0
Response time: 0 ms

Resolved: ESTAT is back up in 11959e0 after 10 minutes.


text:
🛑 NBB is down

In b79fde1, NBB (https://stat.nbb.be/restsdmx/sdmx.ashx/GetDataStructure/ALL) was down:

HTTP code: 429
Response time: 634 ms

Resolved: NBB is back up in 5cc12d8.


text:
🛑 ESTAT is down

In cedd239, ESTAT (https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/dataflow/ESTAT/all/latest) was down:

HTTP code: 0
Response time: 0 ms

Resolved: ESTAT is back up in 4f880f1 after 5 minutes.


text:
🛑 ISTAT is down

In 8a11ed5, ISTAT (https://esploradati.istat.it/SDMXWS/rest/dataflow/all/all/latest) was down:

HTTP code: 0
Response time: 0 ms

Resolved: ISTAT is back up in dc1b22a after 15 minutes.


text:
🛑 NBB is down

In 89a0212, NBB (https://stat.nbb.be/restsdmx/sdmx.ashx/GetDataStructure/ALL) was down:

HTTP code: 429
Response time: 668 ms

Resolved: NBB is back up in 766e244 after 29 minutes.


text:
🛑 INEGI is down

In 3c3ae20, INEGI (https://sdmx.snieg.mx/service/Rest/dataflow/all/all/latest) was down:

HTTP code: 0
Response time: 0 ms

Resolved: INEGI is back up in 7a6ee9d.


text:
🛑 ESTAT is down

In 0dfbdd4, ESTAT (https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/dataflow/ESTAT/all/latest) was down:

HTTP code: 0
Response time: 0 ms

Resolved: ESTAT is back up in 24bd9ed.


text:
🛑 File access (beta) is down

In 53ff6ce, File access (beta) (https://beta.fyle.uk/file/fish) was down:

HTTP code: 503
Response time: 190 ms

Resolved: File access (beta) is back up in 73673ca.

repo: nbitzz/status


text:
未定義の機関(コード番号: 50)がみちびきから送信される

みちびきから未定義の機関(コード番号: 50)を受信し、azarashiがQzssDcrDecoderExceptionを送出する。

NMEAの例:
$QZQSM,55,C6B2E40DE00000000000000000000000000000000000000000000003D1F0734*01

これはDCXの試験メッセージだった。DCXをサポートしたので解決。

repo: nbtk/azarashi


text:
restore correct version of MPVClient.pas

The contents of MPVClient.pas were overwritten by the contents of MPVBasePlayer.pas back in March.
This restores the correct contents from 9th March, 2023.
Both demo projects now build and run correctly.

I'll take a look a few days later. Thanks.
Edward Guo

From: Baz Cuda @.>
Sent: Thursday, July 20, 2023 5:16:01 PM
To: nbuyer/LibMPVDelphi @.>
Cc: Subscribed @.***>
Subject: [nbuyer/LibMPVDelphi] restore correct version of MPVClient.pas (PR #2)
The contents of MPVClient.pas were overwritten by the contents of MPVBasePlayer.pas back in March.
This restores the correct contents from 9th March, 2023.
Both demo projects now build and run correctly.

You can view, comment on, or merge this pull request online at:
https://github.com/nbuyer/LibMPVDelphi/pull/2
Commit Summary

54d7475https://github.com/nbuyer/LibMPVDelphi/pull/2/commits/54d747512d674007be40bd3f91b02261e9deca22 restore correct version of MPVClient.pas

File Changes
(1 filehttps://github.com/nbuyer/LibMPVDelphi/pull/2/files)

M MPVClient.pashttps://github.com/nbuyer/LibMPVDelphi/pull/2/files#diff-2f76cae5383cc1bad08090aaab8982a7f89677c0b171e67d9cc2f80f9d42cf49 (3729)

Patch Links:

https://github.com/nbuyer/LibMPVDelphi/pull/2.patch
https://github.com/nbuyer/LibMPVDelphi/pull/2.diff

—
Reply to this email directly, view it on GitHubhttps://github.com/nbuyer/LibMPVDelphi/pull/2, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABINTKPBE7BAA2JDVRKVUM3XRG33DANCNFSM6AAAAAA2SCYXH4.
You are receiving this because you are subscribed to this thread.Message ID: @.***>

authors: BazzaCuda nbuyer

repo: nbuyer/LibMPVDelphi


text:
flintrock constant failure : Exception: Failed to install Spark

Flintrock version: 0.11
Python version: 3.7.3
OS: Ubuntu 18.0.4

Here  is the log file:
ilendra@ubuntu-server1:/usr/local/bin$ flintrock --debug launch
boomi-platform-analysis --ec2-identity-file  ~/Downloads/spark-cluster.pem
--ec2-user ec2-user
2019-07-19 11:28:02,105 - flintrock.ec2       - INFO  - Launching 3
instances...
2019-07-19 11:28:15,239 - flintrock.ec2       - DEBUG - 3 instances not in
state 'running': 'i-0b765794ecb34f060', 'i-0242cd797a72a2966',
'i-09fae853dc92916d3', ...
2019-07-19 11:28:18,766 - flintrock.ec2       - DEBUG - 3 instances not in
state 'running': 'i-0b765794ecb34f060', 'i-09fae853dc92916d3',
'i-0242cd797a72a2966', ...
2019-07-19 11:28:22,067 - flintrock.ec2       - DEBUG - 1 instances not in
state 'running': 'i-0242cd797a72a2966', ...
2019-07-19 11:28:28,371 - flintrock.ssh       - DEBUG - [54.87.10.29] SSH
timeout.
2019-07-19 11:28:28,371 - flintrock.ssh       - DEBUG - [54.174.33.164] SSH
timeout.
2019-07-19 11:28:28,375 - flintrock.ssh       - DEBUG - [3.84.223.206] SSH
timeout.
2019-07-19 11:28:33,458 - flintrock.ssh       - DEBUG - [54.174.33.164] SSH
exception: [Errno None] Unable to connect to port 22 on 54.174.33.164
2019-07-19 11:28:33,463 - flintrock.ssh       - DEBUG - [3.84.223.206] SSH
exception: [Errno None] Unable to connect to port 22 on 3.84.223.206
2019-07-19 11:28:33,465 - flintrock.ssh       - DEBUG - [54.87.10.29] SSH
exception: [Errno None] Unable to connect to port 22 on 54.87.10.29
/home/shailendra/.local/lib/python3.7/site-packages/paramiko/kex_ecdh_nist.py:39:
CryptographyDeprecationWarning: encode_point has been deprecated on
EllipticCurvePublicNumbers and will be removed in a future version. Please use
EllipticCurvePublicKey.public_bytes to obtain both compressed and uncompressed
point encoding.
  m.add_string(self.Q_C.public_numbers().encode_point())
/home/shailendra/.local/lib/python3.7/site-packages/paramiko/kex_ecdh_nist.py:96:
CryptographyDeprecationWarning: Support for unsafe construction of public
numbers from encoded data will be removed in a future version. Please use
EllipticCurvePublicKey.from_encoded_point
  self.curve, Q_S_bytes
/home/shailendra/.local/lib/python3.7/site-packages/paramiko/kex_ecdh_nist.py:111:
CryptographyDeprecationWarning: encode_point has been deprecated on
EllipticCurvePublicNumbers and will be removed in a future version. Please use
EllipticCurvePublicKey.public_bytes to obtain both compressed and uncompressed
point encoding.
  hm.add_string(self.Q_C.public_numbers().encode_point())
2019-07-19 11:28:39,295 - flintrock.ssh       - INFO  - [54.87.10.29] SSH
online.
2019-07-19 11:28:39,405 - flintrock.ssh       - INFO  - [3.84.223.206] SSH
online.
2019-07-19 11:28:39,478 - flintrock.ssh       - INFO  - [54.174.33.164] SSH
online.
2019-07-19 11:28:40,440 - flintrock.core      - INFO  - [54.87.10.29]
Configuring ephemeral storage...
2019-07-19 11:28:40,702 - flintrock.core      - INFO  - [3.84.223.206]
Configuring ephemeral storage...
2019-07-19 11:28:40,836 - flintrock.core      - INFO  - [54.174.33.164]
Configuring ephemeral storage...
2019-07-19 11:28:44,294 - flintrock.core      - INFO  - [54.174.33.164]
Installing Java 1.8...
2019-07-19 11:28:44,670 - flintrock.core      - INFO  - [54.87.10.29]
Installing Java 1.8...
2019-07-19 11:28:44,919 - flintrock.core      - INFO  - [3.84.223.206]
Installing Java 1.8...
2019-07-19 11:29:17,524 - flintrock.services  - INFO  - [54.87.10.29]
Installing HDFS...
2019-07-19 11:29:17,783 - flintrock.services  - INFO  - [54.174.33.164]
Installing HDFS...
2019-07-19 11:29:20,342 - flintrock.services  - INFO  - [3.84.223.206]
Installing HDFS...
2019-07-19 11:29:43,652 - flintrock.services  - INFO  - [54.87.10.29]
Installing Spark...
2019-07-19 11:29:44,619 - flintrock.services  - INFO  - [54.174.33.164]
Installing Spark...
2019-07-19 11:29:46,084 - flintrock.services  - INFO  - [3.84.223.206]
Installing Spark...
Do you want to terminate the 3 instances created by this operation? [Y/n]: y
Terminating instances...
Traceback (most recent call last):
  File
"/home/shailendra/.local/lib/python3.7/site-packages/flintrock/core.py", line
599, in setup_node
    cluster=cluster,
  File
"/home/shailendra/.local/lib/python3.7/site-packages/flintrock/services.py",
line 333, in install
    """)
  File "/home/shailendra/.local/lib/python3.7/site-packages/flintrock/ssh.py",
line 145, in ssh_check_output
    message=stdout_output + stderr_output)
flintrock.exceptions.SSHError: [3.84.223.206] ln: failed to create symbolic
link ‘/usr/local/bin/beeline’: File exists

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/bin/flintrock", line 11, in <module>
    sys.exit(main())
  File
"/home/shailendra/.local/lib/python3.7/site-packages/flintrock/flintrock.py",
line 1187, in main
    cli(obj={})
  File "/home/shailendra/.local/lib/python3.7/site-packages/click/core.py",
line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/shailendra/.local/lib/python3.7/site-packages/click/core.py",
line 717, in main
    rv = self.invoke(ctx)
  File "/home/shailendra/.local/lib/python3.7/site-packages/click/core.py",
line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/shailendra/.local/lib/python3.7/site-packages/click/core.py",
line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/shailendra/.local/lib/python3.7/site-packages/click/core.py",
line 555, in invoke
    return callback(*args, **kwargs)
  File
"/home/shailendra/.local/lib/python3.7/site-packages/click/decorators.py",
line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File
"/home/shailendra/.local/lib/python3.7/site-packages/flintrock/flintrock.py",
line 456, in launch
    tags=ec2_tags)
  File "/home/shailendra/.local/lib/python3.7/site-packages/flintrock/ec2.py",
line 53, in wrapper
    res = func(*args, **kwargs)
  File "/home/shailendra/.local/lib/python3.7/site-packages/flintrock/ec2.py",
line 955, in launch
    identity_file=identity_file)
  File
"/home/shailendra/.local/lib/python3.7/site-packages/flintrock/core.py", line
625, in provision_cluster
    run_against_hosts(partial_func=partial_func, hosts=hosts)
  File
"/home/shailendra/.local/lib/python3.7/site-packages/flintrock/core.py", line
492, in run_against_hosts
    future.result()
  File "/usr/lib/python3.7/concurrent/futures/_base.py", line 432, in result
    return self.__get_result()
  File "/usr/lib/python3.7/concurrent/futures/_base.py", line 384, in
__get_result
    raise self._exception
  File "/usr/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File
"/home/shailendra/.local/lib/python3.7/site-packages/flintrock/core.py", line
681, in provision_node
    cluster=cluster)
  File
"/home/shailendra/.local/lib/python3.7/site-packages/flintrock/core.py", line
605, in setup_node
    ) from e
Exception: Failed to install Spark.
Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 63, in
apport_excepthook
    from apport.fileutils import likely_packaged, get_recent_crashes
  File "/usr/lib/python3/dist-packages/apport/__init__.py", line 5, in
<module>
    from apport.report import Report
  File "/usr/lib/python3/dist-packages/apport/report.py", line 30, in <module>
    import apport.fileutils
  File "/usr/lib/python3/dist-packages/apport/fileutils.py", line 23, in
<module>
    from apport.packaging_impl import impl as packaging
  File "/usr/lib/python3/dist-packages/apport/packaging_impl.py", line 24, in
<module>
    import apt
  File "/usr/lib/python3/dist-packages/apt/__init__.py", line 23, in <module>
    import apt_pkg
ModuleNotFoundError: No module named 'apt_pkg'

Original exception was:
Traceback (most recent call last):
  File
"/home/shailendra/.local/lib/python3.7/site-packages/flintrock/core.py", line
599, in setup_node
    cluster=cluster,
  File
"/home/shailendra/.local/lib/python3.7/site-packages/flintrock/services.py",
line 333, in install
    """)
  File "/home/shailendra/.local/lib/python3.7/site-packages/flintrock/ssh.py",
line 145, in ssh_check_output
    message=stdout_output + stderr_output)
flintrock.exceptions.SSHError: [3.84.223.206] ln: failed to create symbolic
link ‘/usr/local/bin/beeline’: File exists

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/bin/flintrock", line 11, in <module>
    sys.exit(main())
  File
"/home/shailendra/.local/lib/python3.7/site-packages/flintrock/flintrock.py",
line 1187, in main
    cli(obj={})
  File "/home/shailendra/.local/lib/python3.7/site-packages/click/core.py",
line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/shailendra/.local/lib/python3.7/site-packages/click/core.py",
line 717, in main
    rv = self.invoke(ctx)
  File "/home/shailendra/.local/lib/python3.7/site-packages/click/core.py",
line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/shailendra/.local/lib/python3.7/site-packages/click/core.py",
line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/shailendra/.local/lib/python3.7/site-packages/click/core.py",
line 555, in invoke
    return callback(*args, **kwargs)
  File
"/home/shailendra/.local/lib/python3.7/site-packages/click/decorators.py",
line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File
"/home/shailendra/.local/lib/python3.7/site-packages/flintrock/flintrock.py",
line 456, in launch
    tags=ec2_tags)
  File "/home/shailendra/.local/lib/python3.7/site-packages/flintrock/ec2.py",
line 53, in wrapper
    res = func(*args, **kwargs)
  File "/home/shailendra/.local/lib/python3.7/site-packages/flintrock/ec2.py",
line 955, in launch
    identity_file=identity_file)
  File
"/home/shailendra/.local/lib/python3.7/site-packages/flintrock/core.py", line
625, in provision_cluster
    run_against_hosts(partial_func=partial_func, hosts=hosts)
  File
"/home/shailendra/.local/lib/python3.7/site-packages/flintrock/core.py", line
492, in run_against_hosts
    future.result()
  File "/usr/lib/python3.7/concurrent/futures/_base.py", line 432, in result
    return self.__get_result()
  File "/usr/lib/python3.7/concurrent/futures/_base.py", line 384, in
__get_result
    raise self._exception
  File "/usr/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File
"/home/shailendra/.local/lib/python3.7/site-packages/flintrock/core.py", line
681, in provision_node
    cluster=cluster)
  File
"/home/shailendra/.local/lib/python3.7/site-packages/flintrock/core.py", line
605, in setup_node
    ) from e
Exception: Failed to install Spark.

ModuleNotFoundError: No module named 'apt_pkg'

What kind of AMI are you using in your cluster config? Flintrock only supports Amazon Linux and similar OSes (like CentOS).
If you want to use an apt-based distribution, for example, I can only provide some general guidance to help you, but you'll have to tweak Flintrock yourself to get it to work.

flintrock.exceptions.SSHError: [3.84.223.206] ln: failed to create symbolic
link ‘/usr/local/bin/beeline’: File exists

It also looks like you may be using an AMI that has Spark already installed on it. Is that the case?

I am using the default ami that flintlock configure specified. I just changed the bare minimum fields on the yamal file. Namely the spark download link, hdfs link,  pem file, ec2 user and number of slaves.

Here is the config,yaml for flintrock
services:
spark:
version: 2.4.3
# git-commit: latest  # if not 'latest', provide a full commit SHA; e.g. d6dc12ef0146ae409834c78737c116050961f350
# git-repository:  # optional; defaults to https://github.com/apache/spark
# optional; defaults to download from from the official Spark S3 bucket
#   - must contain a {v} template corresponding to the version
#   - Spark must be pre-built
#   - must be a tar.gz file
#download-source: "https://www.example.com/files/spark/{v}/spark-{v}.tar.gz"
download-source: "http://mirrors.advancedhosters.com/apache/spark/spark-2.4.3/spark-2.4.3-bin-hadoop2.7.tgz"
# executor-instances: 1

hdfs:
version: 2.7.6
# optional; defaults to download from a dynamically selected Apache mirror
#   - must contain a {v} template corresponding to the version
#   - must be a .tar.gz file
# download-source: "https://www.example.com/files/hadoop/{v}/hadoop-{v}.tar.gz"
# download-source: "http://www-us.apache.org/dist/hadoop/common/hadoop-{v}/hadoop-{v}.tar.gz"
# download-source: "http://www-us.apache.org/dist/hadoop/common/hadoop-{v}/hadoop-{v}.tar.gz"
download-source: "http://mirrors.advancedhosters.com/apache/spark/spark-2.4.3/spark-2.4.3-bin-hadoop2.7.tgz"
provider: ec2
providers:
ec2:
key-name: spark-cluster
identity-file: /root/anaconda3/spark-cluster.pem
instance-type: m3.medium
region: us-east-1
# availability-zone:
ami: ami-0b8d0d6ac70e5750c  # Amazon Linux 2, us-east-1
user: ec2-user
# ami: ami-61bbf104  # CentOS 7, us-east-1
# user: centos
# spot-price:
# vpc-id:
# subnet-id:
# placement-group:
# security-groups:
#   - group-name1
#   - group-name2
# instance-profile-name:
# tags:
#   - key1,value1
#   - key2, value2  # leading/trailing spaces are trimmed
#   - key3,  # value will be empty
# min-root-ebs-size-gb:
tenancy: default  # default | dedicated
ebs-optimized: no  # yes | no
instance-initiated-shutdown-behavior: terminate  # terminate | stop
# user-data: /path/to/userdata/script
launch:
num-slaves: 2
install-hdfs: True
install-spark: True
debug: false

How did you install Flintrock? Can you try installing Flintrock using pip in a dedicated virtual environment, and then retrying the cluster launch?

I have a Ubuntu virtual machine. I installed flintrock on it using pip3 on this machine.

Can someone guide me as to what the issue could be? I have followed the instructions exactly but still ut keeps on failing. The machines get created and then all get deleted. I had sent the log files earlier. It seems like Spark is not getting installed properly for some reason (a lnk is trying to get created and it already exists.) Please help me. I am stuck for quite some time now.

The errors about apt_pkg and beeline that I referred to earlier are strange, and I haven't seen them before. To eliminate the possibility that the errors are somehow caused by how you installed Flintrock, I suggested in my message just above that you try installing Flintrock into a dedicated virtual environment using pip.
Did you try that? Please show the console output from when you installed Flintrock.

Nick, I did try from a new virtual environment. I think I may have figured out the issue. The spark tarball does include Hadoop. I think in my script file I was installing both hdfs and spark(which already includes Hadoop. I believe this was causing the issue. I opted to not install hdfs and now the scripts runs fine. I am new to Spark installation so do not know if this will cause issues, but the cluster is running now. Please let me know if I do not install hdfs, will that cause any issues.

There should be no conflict between Spark and HDFS. Are the tarballs hosted by mirrors.advancedhosters.com (which I see in your config above) the same as the official ones hosted by Apache?
Do you have the same issue if you use the official Apache mirrors instead of the host you currently have configured?

I took that mirror from https://www.apache.org/dyn/closer.lua/spark/spark-2.4.3/spark-2.4.3-bin-hadoop2.7.tgz.  I did not do a comparison though. I will try and let you know.

For the record, I just successfully launched a few clusters using Flintrock master, which I think is functionally identical to the 0.11 release.
I just noticed that you are using an older version of HDFS in your config. The configuration template specifies HDFS 2.8.5, and that's what I recommend.
Have you tried that? It might resolve the beeline error, though I don't think it would help with the ModuleNotFoundError you are also seeing.
If updating your configured version of HDFS has the effect I expect, then please reinstall Flintrock into a new virtual environment and show me the entire log of how you did it. Maybe there's a clue in there about the second error.

Still having the same issue, @shail-burman? If so, please try the things I suggested in my previous message and let me know how they turned out.

Sorry Nick for the tardy response, I have been stuck on a  problem involving loading millions of small files. Saw yr note on that forum that there is no good way.
Anyways, Thanks so much Hadoop 2.8.5 solved the beeline issue. You may close this issue.
I had a few questions aroiund flint rock.
A) Can we  create regular instances instead of spot instances, so we could use them in semi prod environment.
B) Can we use different configurations for driver and workers?\
Thanks,
‘Shail

Glad to hear the HDFS version fixed your issues.
A) Yes, Flintrock by default creates on-demand instances if you don't specify a spot price in your config.
B) Unfortunately, no. There's some discussion on this in #199 and several issues linked to or from there.

authors: nchammas shail-burman shailburman

repo: nchammas/flintrock


text:
Accessibility: Adding alt text to images in scores documentation

It would be great if alt text could be added to all images (plots, charts etc.) in the scores documentation.
At the moment, I believe all images in the scores documentation are currently from Jupyter Notebook tutorials.
Ideally the Alt Text should work in:

Jupyter (JupyterLab/Juptyter Notebook)
Tutorials as rendered in Read the Docs
Binder

(Note: if I have overlooked any images in the scores documentation that aren't in the tutorials, please post a comment below saying so!)

I did a bit of research on this.
Matplotlib has issue 24309, open as of this writing, to add alt-text to their docs. As near as I can understand, there's a checklist of items needed to make that possible.
Also, note that images can have a title attribute, which is displayed as a tooltip and an alt attribute used in place of the image for accessibility. See usage info here: https://accessibility.psu.edu/images/imageshtml/

https://github.com/jupyter/nbconvert/pull/1112 may also be relevant

authors: Steph-Chong cbare tennlee

repo: nci/scores


text:
Add HTAN ID validation rules

Closes #268

Thanks @elv-sb. Let's discuss one at sprint close out today before merging.

@elv-sb leaving this one open as I want to review some notes in the linked issues at an ops call

In our sprint close I think I am happy that this can go to staging -(ie be merged)

repo: ncihtan/data-models