text:
"Not a valid RQ worker key" error when using custom classes

$ docker-compose -f docker-compose.yml -f docker-compose.custom.yml up
Recreating rq-exporter_grafana_1                  ... done
Starting rq-exporter_redis_1        ... done
Recreating rq-exporter_prometheus_1               ... done
Starting rq-exporter_dashboard_1                  ... done
Starting rq-exporter_rq_exporter_1                ... done
Starting rq-exporter_rq_exporter_custom_classes_1 ... done
Starting rq-exporter_enqueue_1                    ... done
Starting rq-exporter_enqueue_custom_classes_1     ... done
Starting rq-exporter_worker_1                     ... done
Starting rq-exporter_worker_2                     ... done
Starting rq-exporter_worker_custom_classes_1      ... done
Starting rq-exporter_worker_custom_classes_2      ... done
Attaching to rq-exporter_redis_1, rq-exporter_grafana_1, rq-exporter_dashboard_1, rq-exporter_rq_exporter_1, rq-exporter_rq_exporter_custom_classes_1, rq-exporter_prometheus_1, rq-exporter_enqueue_1, rq-exporter_enqueue_custom_classes_1, rq-exporter_worker_1, rq-exporter_worker_2, rq-exporter_worker_custom_classes_1, rq-exporter_worker_custom_classes_2
dashboard_1                   | RQ Dashboard version 0.6.0
enqueue_custom_classes_1      | Using queues with custom RQ classes
enqueue_custom_classes_1      | Enqueuing job "process_data" on queue "high"
enqueue_1                     | Enqueuing job "process_data" on queue "low"
dashboard_1                   |  * Running on 0.0.0.0:9181
grafana_1                     | t=2020-11-30T17:24:16+0000 lvl=warn msg="phantomJS is deprecated and will be removed in a future release. You should consider migrating from phantomJS to grafana-image-renderer plugin. Read more at https://grafana.com/docs/grafana/latest/administration/image_rendering/" logger=rendering renderer=phantomJS
dashboard_1                   |  * Serving Flask app "rq_dashboard.cli" (lazy loading)
dashboard_1                   |  * Environment: production
dashboard_1                   |    WARNING: This is a development server. Do not use it in a production deployment.
dashboard_1                   |    Use a production WSGI server instead.
dashboard_1                   |  * Debug mode: off
redis_1                       | 1:C 30 Nov 2020 17:24:13.503 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis_1                       | 1:C 30 Nov 2020 17:24:13.503 # Redis version=5.0.10, bits=64, commit=00000000, modified=0, pid=1, just started
redis_1                       | 1:C 30 Nov 2020 17:24:13.503 # Configuration loaded
rq_exporter_1                 | [2020-11-30 17:24:18] [rq_exporter] [INFO]: Serving the application on 0.0.0.0:9726
rq_exporter_custom_classes_1  | [2020-11-30 17:24:18] [rq_exporter] [INFO]: Serving the application on 0.0.0.0:9726
redis_1                       | 1:M 30 Nov 2020 17:24:13.505 * Running mode=standalone, port=6379.
redis_1                       | 1:M 30 Nov 2020 17:24:13.505 # Server initialized
redis_1                       | 1:M 30 Nov 2020 17:24:13.505 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
worker_2                      | 17:24:20 Worker rq:worker:c1f6df2c68ed486292a0deccabc6a049: started, version 1.6.1
worker_2                      | 17:24:20 Subscribing to channel rq:pubsub:c1f6df2c68ed486292a0deccabc6a049
redis_1                       | 1:M 30 Nov 2020 17:24:13.506 * Ready to accept connections
worker_2                      | 17:24:20 *** Listening on high, default, low...
worker_2                      | 17:24:20 Cleaning registries for queue: high
worker_2                      | 17:24:20 Cleaning registries for queue: default
worker_2                      | 17:24:20 Cleaning registries for queue: low
worker_2                      | 17:24:20 low: jobs.process_data(2) (55375777-b6d9-4ca5-80d9-3f3008366af1)
rq-exporter_prometheus_1 exited with code 1
worker_custom_classes_2       | 17:24:20 Worker rq:custom:worker:cc69e2c9c6c145ea823ccf84981bae67: started, version 1.6.1
worker_custom_classes_2       | 17:24:20 Subscribing to channel rq:pubsub:cc69e2c9c6c145ea823ccf84981bae67
worker_custom_classes_2       | 17:24:20 *** Listening on high, default, low...
worker_custom_classes_2       | 17:24:20 Cleaning registries for queue: high
worker_1                      | 17:24:20 Worker rq:worker:358aca2223bd4d2aa04c8ab57044f34e: started, version 1.6.1
worker_1                      | 17:24:20 Subscribing to channel rq:pubsub:358aca2223bd4d2aa04c8ab57044f34e
worker_custom_classes_1       | Traceback (most recent call last):
worker_custom_classes_1       |   File "/usr/local/bin/rq", line 8, in <module>
worker_custom_classes_1       |     sys.exit(main())
worker_custom_classes_1       |   File "/usr/local/lib/python3.8/site-packages/click/core.py", line 829, in __call__
worker_custom_classes_1       |     return self.main(*args, **kwargs)
worker_custom_classes_1       |   File "/usr/local/lib/python3.8/site-packages/click/core.py", line 782, in main
worker_custom_classes_1       |     rv = self.invoke(ctx)
worker_custom_classes_1       |   File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
worker_custom_classes_1       |     return _process_result(sub_ctx.command.invoke(sub_ctx))
worker_custom_classes_1       |   File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
worker_custom_classes_1       |     return ctx.invoke(self.callback, **ctx.params)
worker_custom_classes_1       |   File "/usr/local/lib/python3.8/site-packages/click/core.py", line 610, in invoke
worker_custom_classes_1       |     return callback(*args, **kwargs)
worker_custom_classes_1       |   File "/usr/local/lib/python3.8/site-packages/rq/cli/cli.py", line 78, in wrapper
worker_custom_classes_1       |     return ctx.invoke(func, cli_config, *args[1:], **kwargs)
worker_custom_classes_1       |   File "/usr/local/lib/python3.8/site-packages/click/core.py", line 610, in invoke
worker_custom_classes_1       |     return callback(*args, **kwargs)
worker_custom_classes_1       |   File "/usr/local/lib/python3.8/site-packages/rq/cli/cli.py", line 230, in worker
worker_custom_classes_1       |     cleanup_ghosts(cli_config.connection)
worker_custom_classes_1       |   File "/usr/local/lib/python3.8/site-packages/rq/contrib/legacy.py", line 25, in cleanup_ghosts
worker_custom_classes_1       |     for worker in Worker.all(connection=conn):
worker_custom_classes_1       |   File "/usr/local/lib/python3.8/site-packages/rq/worker.py", line 120, in all
worker_custom_classes_1       |     workers = [cls.find_by_key(as_text(key),
worker_custom_classes_1       |   File "/usr/local/lib/python3.8/site-packages/rq/worker.py", line 120, in <listcomp>
worker_custom_classes_1       |     workers = [cls.find_by_key(as_text(key),
worker_custom_classes_1       |   File "/usr/local/lib/python3.8/site-packages/rq/worker.py", line 146, in find_by_key
worker_custom_classes_1       |     raise ValueError('Not a valid RQ worker key: %s' % worker_key)
worker_custom_classes_1       | ValueError: Not a valid RQ worker key: rq:custom:worker:cc69e2c9c6c145ea823ccf84981bae67
worker_1                      | 17:24:20 *** Listening on high, default, low...
worker_custom_classes_2       | 17:24:20 Cleaning registries for queue: default
worker_custom_classes_2       | 17:24:20 Cleaning registries for queue: low
worker_custom_classes_2       | 17:24:20 high: jobs.process_data(2) (d04e44ff-31f8-4306-a987-331f0579ac36)
rq-exporter_worker_custom_classes_1 exited with code 1
worker_2                      | process_data: sleeping for 2 seconds
worker_2                      | 17:24:22 low: Job OK (55375777-b6d9-4ca5-80d9-3f3008366af1)

Hi,
Thank you for reporting this error, actually this error is coming from the function rq.contrib.legacy.cleanup_ghosts, it is using the default Worker class, that's why the class method Worker.find_by_key is raising this error because it's checking if the worker name starts with the original prefix Worker.redis_worker_namespace_prefix (rq:worker:) and not the custom class' prefix CustomWorker.redis_worker_namespace_prefix (rq:custom:worker:).
This has to be fixed in the rq repo, the cleanup_ghosts must be fixed to use the custom worker class cli_config.worker_class.
If you want a temporary fix, use a custom worker prefix that starts with rq:worker:, for example:
class CustomWorker(Worker):
    redis_worker_namespace_prefix = 'rq:worker:custom'

authors: PaulFlanaganGenscape mdawar

repo: mdawar/rq-exporter


text:
Feat: Modifying Solve page's UI

Describe the feature
Since the UI of solve page looks simple.I would like to make it more attractive and visually appealing for users.
Add ScreenShots

WILL MODIFY THIS SECTION
Record

[X] I agree to follow this project's Code of Conduct
[X] I'm a GSSOC'24 contributor
[ ] I'm a VSoC'24 contributor
[X] I want to work on this issue

@mdazfar2 Please assign this issue to me.
also assign the labels to my issue.

@AshmitaBarthwal can you please elaborate it how can you make is more attractive

@mdazfar2 I will create a glass like affect around the txt and will add effects to make it dynamic,change the fonts,will also try to add a good background.

@AshmitaBarthwal It would be great if you share some demo video. It will help me to assign you.

source: gharchive/issue

license: MIT

license_type: permissive

license_source: github-api


text:
Query params are not set on relationship when reloading page

I noticed in the ModelMixin that the relationship is not reloaded if it is not yet loaded.
    //if the hasMany content is already loaded, we must reload it
    if (value.get('content').get('isLoaded')) {
      value = value.reload();
    }

In my use case I have a route that is loading the model but on the very first load of the model the route wants to pass query params for when the hasMany relationship is fetched (the hasMany relationship looks like it is being called automatically when I display the relationship data in my template).
I was trying to do something like this in the tag-entity-pair model hook of my router:
    return this.store.findRecord('tag-entity-pair', params.tag_entity_pair_id, {adapterOptions: {query: params}}).then(function(tagEntityPair){
      // force the first fetch of the relationship to set the sort value of the relationship being fetched
      return tagEntityPair.query('comments', {sort: 'comment.last-edited'}).then(function(comments){
        return tagEntityPair;
      });
    });

However, because the comments have not been loaded at the time I run tagEntityPair.query the values are not reloaded and no query params are used.
Is there a way to use the query params on the very first load of the relationship if it's being loaded from the model hook of the router?
Apologies if this question makes no sense as I'm still trying to wrap my head around Ember Data (I have a feeling I might be setting up something wrong here).
Thank you for your effort on this plugin!

The reason for the value.reload() is, if the hasMany relationship data is already loaded, Ember Data will not reload from the backend again. If the relationship has not yet been loaded, the query method will automatically request data from the server, and therefore reload is not needed.
I suspect something else is going on. What does your tag-entity-pair model look like?
You could try returning the tag-entity-pair and the comments separately in the model hook (and change the template accordingly):
    return this.store.findRecord('tag-entity-pair', params.tag_entity_pair_id, {adapterOptions: {query: params}}).then(function(tagEntityPair){
      // force the first fetch of the relationship to set the sort value of the relationship being fetched
      return tagEntityPair.query('comments', {sort: 'comment.last-edited'}).then(function(comments){
        return {
          tagEntityPair: tagEntityPair,
          comments: comments
        };
      });
    });

@mdehoog Thank you for the information on value.reload().  That makes a lot of sense.  Here are the relevant parts of my two models:
// models/tag-entity-pair.js
export default DS.Model.extend(HasManyQueryModelMixin, {
  comments: DS.hasMany('comment')
});

// models/comment.js
export default DS.Model.extend(HasManyQueryModelMixin, {
  tagEntityPair: DS.belongsTo('tag-entity-pair'),
});

I added some output statements your two mixins and it seems like the following events are happening:
MODEL MIXIN: Enter
model.js:29 MODEL MIXIN: Set _commentsQueryParams to:
model.js:30 Object {tag_entity_pair_id: "6", sort: "-comment.last-edited", page: Object}
model.js:42 MODEL MIXIN: Setting _commentsQueryParams to undefined

Refresh the page
Model Mixin gets called
The model mixin is retrieving the query params correctly

{
   sort: comment.last-edited
}

The model mixin sets _commentsQueryParams properly
The content is not already loaded so we don't reload it
value.finally returns and _commentsQueryParams is set to undefined
The Adapter mixin runs
The Adapter retrieves _commentsQueryParam and it is undefined
The Adapter calls this.ajax with no query params since it was undefined

Based on the steps above, the ModelMixin appears to be setting the value of the comments query params to undefined before the AdapterMixing has a chance to run.

Thanks for the excellent analysis. Which version of Ember Data are you using? Does it help if you explicitly set async: true in your DS.hasMany options hash?

Here are the versions I'm running:
Ember             : 2.1.0
Ember Data        : 2.1.0
jQuery            : 1.11.3

Setting {async:true} explicitly doesn't seem to change anything.  The Adapter calls are still executed after the model mixin has set the query params back to undefined

Just updated to latest and still having the issue.  Will continue to play around with it and see if I can get any further with the debugging.
Ember             : 2.2.0
Ember Data        : 2.2.1
jQuery            : 1.11.3

I'm having trouble reproducing this issue. Any chance of a cut down example that demonstrates the issue?
The model mixin query function calls get for the passed hasMany attribute, which should trigger a the adapter to fetch the data, appending the query params to the URL. Here is my call stack:

You can see that the query function (currently selected) calls this.get(propertyName);, which ends up in this addon's rest-adapter mixin's findHasMany function. This adds the query parameters to the URL and calls ajax before the value.finally clears the query params. Perhaps you could set a breakpoint in the rest-adapter mixin findHasMany function and include your call stack here?

Thank you for investigating the issue.  I set breakpoints in both model.js where the _queryParamProperty gets set to null
//model.js
self.set(_queryParamPropertyName, undefined);

and then in the rest-adapter where the queryPramPropertyName is read back
//rest-adapter.js
var queryParams = snapshot.record.get(queryParamPropertyName(relationship.key));

and for whatever reason in my case the queryParam is set to undefined before the rest-adapter grabs the value.
My call stack looks identical to yours.  I'm thinking it must be the way I'm trying to load the relationship in the model:
    return this.store.findRecord('tag-entity-pair', params.tag_entity_pair_id).then(function(tagEntityPair){
      return tagEntityPair.query('comments', params).then(function(comments) { //model.js query gets called here and the promise returns and sets the parameter to undefined
        return tagEntityPair; // this gets returned before any rest-adapter calls are made
      });
    });

I'll see if I can recreate it in a jsbin.

I think I've run into this same problem. Has anyone found a solution?

Sorry I'm still having the issue.  We've just lived with it for now until we have some cycles to look into it again:
We're on the following versions now:
Ember             : 2.3.1
Ember Data        : 2.3.3
jQuery            : 2.1.4

@sarus An update to the latest version of this addon and to ember-data >= 2.5 may help, I have switched the reloading logic to use the new ds-references implementation.
Note that this addon only works with an async HasMany relationship that is included in the parent model server response in the links attribute, e.g.
{
    tag-entity-pair: {
        ...
        links: {comments: "comments"}
    }
}

Thank you for the info.  I'll get our app updated to 2.5 and report back.

I found a bug which is similar to this issue in that, when making two queries in quick succession, the first query is aborted and then query parameters are cleared, which causes them to be cleared for the second query too. I've fixed this issue in v0.1.5 of this addon; it may help with this issue.

authors: mdehoog octabode sarus

repo: mdehoog/ember-data-has-many-query

license: mit

license_source: bigquery


text:
Addressing recent QAP PR comments

This should address pmla's recent comments on the PR. If I've forgotten to include something lmk.
Didn't address how/if we want to change the init parameters since it seems a bit up for debate at the moment.
I am down for leaving it as is, seems clear what it is doing and gives the user flexibility.
Also shall we both rebase so we are at the same scipy commit stamp?  It seems like I've rebased more recently than you.

the commits are a mess my bad

In the future the way to do this would be to check out my branch qap_merge and then work on the changes. Rather than trying to merge this, I will just copy the changes over.

So I won't merge this but I was able to cherry-pick the relevant commit.

source: gharchive/pull-request

repo: mdhaber/scipy


text:
Unable to open URL

Describe the bug
After a TV show is ocnverted and the info is passed back to Sickgear, it gets a system exit.  Looking at SabNZBD's log is says it cannot open the URL back to Sickgear,
Command or context you are trying to run
Auto conversion within SabNZBD.  Attaching log from SabNZBD.
sabLog.txt
autoProcess.ini settings
autoProcess.txt
Log files
smaLog.txt
FFMpeg headers
built with gcc 9 (Ubuntu 9.4.0-1ubuntu1~20.04.1)
configuration: --prefix=/usr --extra-version=0ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
WARNING: library configuration mismatch
avcodec     configuration: --prefix=/usr --extra-version=0ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared --enable-version3 --disable-doc --disable-programs --enable-libaribb24 --enable-liblensfun --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libtesseract --enable-libvo_amrwbenc
libavutil      56. 31.100 / 56. 31.100
libavcodec     58. 54.100 / 58. 54.100
libavformat    58. 29.100 / 58. 29.100
libavdevice    58.  8.100 / 58.  8.100
libavfilter     7. 57.100 /  7. 57.100
libavresample   4.  0.  0 /  4.  0.  0
libswscale      5.  5.100 /  5.  5.100
libswresample   3.  5.100 /  3.  5.100
libpostproc    55.  5.100 / 55.  5.100
Hyper fast Audio and Video encoder
System Information

OS: Ubuntu 20.04.6 LTS x86_64
Python version: Python 3.8.10

Expected behavior
After the MKV file is processed to an MP4, I was expecting the file to communicate with Sickgear and move the file to the process location on my server.
Additional context
I have an older version of the software from several years back that seems to work fine, but does not convert HDR files properly.  The HDR conversion works perfectly now.  Thank you so much for this.  It has been a standard staple on my server for years.

I will try Sickrage again.  Was having an issue with it earlier not converting any of the files and not copying it over.  I got further with Sickbeard and didn't go back.  End result may need to be that I switch back to either Sickbeard or Sickrage.  Will update over the weekend.

I fixed the bug you were having with SR so hopefully it should work

That did it!  Amazing!  Thank you.

authors: mdhiggins nogardvfx


text:
Document AudioTrack and AudioTrackList

https://bugzilla.mozilla.org/show_bug.cgi?id=1454268
This is the HTML DOM AudioTrack, not the Media Capture and Streams AudioStramTrack. Represents an audio track on an HTML <audio> or <video> element.
Tasks

[x] Create overview page for AudioTrack
[x] Document the 5 attributes of AudioTrack
[x] Create overview page for AudioTrackList
[x] Document the length attribute
[x] Document the getTrackById() method
[x] Document the indexed getter
[x] Document the onchange event handler
[x] Document the onaddtrack event handler
[x] Document the onremovetrack event handler
[x] Document the change, addtrack, and removetrack events
[x] Make sure the HTMLMediaElement.audioTracks property is documented properly and links to the new content appropriately

The indexed getter is as documented as we ever really do, by simply mentioning in all the right places that you can use that syntax to access items.
These docs are now complete. Hurray!

(Reopened just to make sure it shows up under “Done” for now.

repo: mdn/sprints


text:
419 Page expired

I have mod_headers enabled for my Apache server and the configuration below gives me 419 Page expired when I try to use any form with the module enabled.
Is there any solution that I can use even when mod_headers is enabled? I have searched the internet for a few days and tried some possible solutions (can't remember which ones) and have now given up. Posting this issue is my "only" chance to fix this. And with "only" I mean that I haven't seen every single solution out there xD
Define E my@email.com
Define P 80004

Protocols h2 h2c http/1.1

<VirtualHost *:80>
  ServerAdmin ${E}
  ServerName img.airikr.me

  ProxyPreserveHost On
  ProxyPass / http://127.0.0.1:${P}/
  ProxyPassReverse / http://127.0.0.1:${P}/

  RewriteEngine On
  RewriteCond %{SERVER_NAME} =img.airikr.me
  RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>

<VirtualHost *:443>
  ServerAdmin ${E}
  ServerName img.airikr.me

  ProxyPreserveHost On
  ProxyPass / http://127.0.0.1:${P}/
  ProxyPassReverse / http://127.0.0.1:${P}/

  ErrorLog ${APACHE_LOG_DIR}/img.airikr.me.error.log

  RewriteEngine On

  SSLCertificateFile /.../img.airikr.me/fullchain.pem
  SSLCertificateKeyFile /.../img.airikr.me/privkey.pem
  Include /.../options-ssl-apache.conf
</VirtualHost>

419 is normally a CSRF Token issue.
Hopefully this answer helps from Stack.
https://stackoverflow.com/questions/52583886/post-request-in-laravel-error-419-sorry-your-session-419-your-page-has-exp?rq=4

Thanks, but adding -e SESSION_DRIVER=file -e SESSION_DOMAIN=img.airikr.me -e SESSION_SECURE_COOKIE=true to the docker run command didn't fix the issue. Same for SESSION_SECURE_COOKIE=false and when I removed -e SESSION_DRIVER=file -e SESSION_SECURE_COOKIE=true and only leaved SESSION_DOMAIN untouched.
The rest of the answers is for JavaScript or other stuff that must be added inside the actual source code. Didn't bother to do anything with that part :)

Is it a Caddy 419 or an Apache 419?

Apache 419. I use "Run in HTTP mode" with -e FORCE_HTTPS=true.

I'm not super familiar with Apache in general, but what is mod_headers being used to do here?
And I don't think that really makes sense... Apache doesn't generate a 419 Page-Expired, Apache's 419 error is Insufficient-Space-On-Resource. 419 Page-Expired is a Laravel/Caddy error.

Thank you for the information. I will get back to you after I got some sleep. Am very sleepy xD

Alright! I use mod_headers to hardening my web server and strength the security of it. I use the following guides:

https://geekflare.com/apache-web-server-hardening-security/
https://www.tecmint.com/apache-security-tips/
https://www.acunetix.com/blog/articles/10-tips-secure-apache-installation/

So the 419 error is Laravel/Caddy and not Apache after all? Then I have no clue of how to fix this issue :( I don't have any knowledge at all of Caddy nor Laravel (more than Laravel is PHP based).
I will ask someone to help me with all this, or if I have the mental strength to search the net for a solution.
Thank you for your help :)

While looking up information about this for nginx, I stumbled over this SO answer which kind of explains the issue entirely! So in your apache config you'll need to include SameSite=none in the cookie line.

Oho! But what cookie line? I don't have any in my configuration file for my domain.

Number 10 on this page describes setting a cookie to handle the header setting. You'd need to set a cookie anyway for this kind of security to work either way.

Aha, ok :) I'm sorry, can't think straight now.
I added Header edit Set-Cookie ^(.*)$ $1;HttpOnly;Secure below ProxyPassReverse for both *:80 and *:443 in the config file for my domain and reloaded Apache. Same issue.

After Secure set ;SameSite=none

Here's the line in apache.conf now: Header edit Set-Cookie ^(.*)$ $1;HttpOnly;Secure;SameSite=none
Same issue.

Sadly I've already overextended my knowledge here. I will say that nginx's proxy_cookie_flags ~ secure; worked for me but I'd imagine you don't want to learn a whole new server just for this.

I am sort of interested in switching to nginx, but that's when I am ready for it.
Many thanks for your help anyway :) You have guided me to the right path (I hope) so I will dig deeper into this when my brain is more cooperative.

This issue has now been solved. I tried to comment out Header edit Set-Cookie ^(.*)$ $1;HttpOnly;Secure;SameSite=none and restarted Apache and everything just worked!

authors: EmberHeartshine KevinGuest ai-ri-kr

repo: mdshack/shotshare


text:
codeBlockPlugin now can set theme

codeBlockPlugin now can set theme
It can be "light" | "dark" | "auto",
or the theme in "@codesandbox/sandpack-themes" package,
or you can also custom one,
just like: sandpack docs

That's really cool and necessary, thank you for the PR!

@newproplus after further thought, I moved the implementation to the codeMirrorPlugin. It does not make sense to keep it on a codeBlock level, because it is specific to the codeMirror editor specifics.
https://github.com/mdx-editor/editor/commit/4aa63e30d2fe697ab99de12a6cba8eda2cc44d82.
Hope this makes sense.

authors: newproplus petyosi

repo: mdx-editor/editor


text:
npm import player is not working

// To import only MediaElementPlayer class and $.fn.mediaelementplayer plugin
// (includes the HTML5 and Flahs renderers ONLY)
import 'mediaelement/player';

There is no player.js

Yes to import the full player just use 'mediaelement'. I'll remove this from the docs thanks for pointing that out

Element removed in master branch. Thanks

repo: mediaelement/mediaelement


text:
Allow for slot suppression from logging

Some slot values may include PII. This configuration allows the ability to suppress certain slots, by name, so that they aren't reported to GA, and so no PII is stored.

Coverage decreased (-9.0%) to 54.741% when pulling 446dff438729ebf9ed8a781caf11e60652a3a19f on suppress-slots into 97a59d4d6a7ac028be4190623cc36ab17d55133e on master.

It says that coverage decreased, but I actually added unit tests, so I suspect this is because someone did a direct push w/o a PR that decreased test coverage.

Seems like it's missing documentation

Looks good to me. Just  need to add how to use on the README file. @heneryville

🤦♂  Well, that's why we do code reviews. Thanks for reminding me. Another look?

authors: armonge coveralls heneryville rmberrios

repo: mediarain/voxa-ga


text:
Update tsconfig.json to use "react-jsx" for JSX.

This allows React to be used within JSX files without having to import it directly. This has been possible since v17.
Without it, opening this repo in an editor with TS typechecking available shows a lot of red lines.

Hi @jckw  - thanks for the contribution!
This repo is derived from our monorepo at https://github.com/medplum/medplum - it is automatically synchronized on versioned releases.
Would you mind opening a PR there?  No worries if that's too much trouble, we can do it as well.
Thanks, and apologies for the confusion.

It looks like it already exists in your monorepo example: https://github.com/medplum/medplum/blob/main/examples/medplum-hello-world/tsconfig.json#L16
Possibly an issue with the syncing then / due a re-sync

authors: codyebberson jckw

repo: medplum/medplum-hello-world


text:
[Photon Health] Sync Patient

Create a one page sample app that allows the user to sync patients between Medplum and Photon Health.
An identifier from Photon Health should be added to the patient and users should only be able to sync patients if they are not already synced.
https://docs.photon.health/docs/sync-patients

@ksmith94 Just to clarify, this logic should be in a Bot, that calls the Photon API for the sync, and updates the patient's identifier

It should also make sure that syncing multiple times doesn't cause any issues (idempotent)

repo: medplum/medplum


text:
Implements 2-opt in numpy

Calls and quickly returns short_travel_cutcode_2opt from within short_travel_cutcode rather than using the regular code present.
This firstly implements this code to do a 2-opt algorithm and secondly, uses numpy to do it.

Due to the skipping of the candidate code it does not perform inner first optimizations.
Due to the numpy requirement it doesn't really work per usual without numpy.

This code should live and die as a draft.  The job here is mostly to show the speed and utility of two-opt implemented in this fashion.
--
Uses code I wrote for vpype: https://github.com/abey79/vpype/commit/7b1fad6bd0fcfc267473fdb8ba2166821c80d9cd this isn't an issue since I wrote that code too. Also, vpype is also MIT licensed.

I tried creating a candidate list and sort it to see if it would be faster than the current short_travel_cutcode and it wasn't any faster than the current main. I tried it both creating the candidate list at every need to travel and creating it at the start of the method and removing candidates as they cut. But neither was faster. In both cases it needed to calculate the distance from current point every time it had to find a travel point and this wasn't twice as fast as part of a python sort as the current loop iteration (average half of the candidates) and in the latter case it needed to find and delete same-subpath candidates which was extra overhead.
If I read your code correctly you are implementing 2-opt without greedy (presumably to try it out and to understand the performance implications). It might be worth trying it with and without greedy first to see which is faster. If you prefer I would be happy to try numpy for greedy.
Hopefully we will find that greedy using numpy is faster because it uses all cores when calculating the distances. I am not sure whether numpy can parallise the actual sort, but since you only want to find the minimum distance for greedy there is a partial sort (to find the lowest x i.e. 1 value) or perhaps there is a numpy function to find the minimum.

2-opt here should be faster and better optimize the results. Yeah, I'm not doing anything fancy except forcing it in there as a weird step. I might be able to run it parallel to normal optimize or something to have it an option. But, it does a pretty nice thing and prints the times etc.

If 2-opt is faster in most non-trivial situations, then I guess that should be preferred over greedy.
If I remember correctly, I think you said that numpy is pre-requisite with Pillow but optional when you run MK without a gui. Is that correct? If so, then I guess we can test for numpy import at start-up and set a NUMPY_AVAILABLE global variable which we can then use to choose which algorithm to run.
I have never used numpy myself - and don't really know either its capabilities nor which bits are capable of using multiple cores, so I am not really able to understand or review your 2-opt code in detail without doing some research. But if you would like me to do that, I would be happy to put the time in and see if I can suggest potential speed optimisations.

Suppose we did a greedy optimisation, then treated all burns and shortest travels (shortest 80% or shorter than xxxx mil) as fixed in order to try to shorten the longer travels.
Would it help to pre-calculate (using numpy) the distance between all pairs of endpoints of these remaining travels in order to be able to give numpy the data it needs to find the optimum solution?

Essentially numpy helps you do matrix math. Not all problems can be solved only by affine transformations (ax+c), but those that can be (which can be expressed by vector math) can be made very very fast using SIMD instructions and if it so supports it, direct GPU calculation. (3D graphics is done entirely that way in matrix math). If your problem isn't representable in matrix form, then numpy is useless to you.
The devil is always in the details. If exactly this type of problem were easy to solve, everything would be a multi-core program -- and it isn't.
Good luck, Tat, you're well beyond me. I never liked matrix math much, preferring explicit systems of equations and trig functions over matrices of coefficients and rotation matrices.

I switched this up to checking if numpy is available. In theory this could be checked for correctness if opt_2opt was set to True and then we could default opt_2opt to True.
This would potentially allow rather marked improvements for a lot of cases. Since without inner first checked the 2opt algorithm actually works fine.

Hyper-optimized some more stuff there.
Renamed greedy, nearest neighbor. There's other greedy algorithms with stuff.
Since 2-opt suffers other defects with non-reversible paths it is set to false.

I think this might be merge ready. It depends on a couple factors the default 2-opt is off and it fails with a couple methods but since it's defaulted to off that shouldn't really matter. If the defaults for nearest neighbor are good. We're good.

authors: Sophist-UK tatarize tiger12506


text:
🛑 Capture the Flag is down

In e12d1e0, Capture the Flag (https://aaronmeese.dev) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Capture the Flag is back up in ae94105 after 22 minutes.

repo: meese-enterprises/uptime-monitor


text:
fixes #297

use color-all mixin to set the fill property in addition to the color property in inverted context

The thought was that if you are in a <div class="stripe--inverted" > it should fill the svg icon as well with the needed inverted color vs adding new css.

authors: akdetrick eilinora


text:
Feature: Improve SEO

The site is currently written using Vite and react-router-dom, and no form of SSR (Server-Side Rendering). This means that there's one HTML for each page, having the same title, description, etc for meta fields.
This is less than optimal, but requires a bit of work to get to properly work with the current setup. I've worked a bit using Rakkas, but didn't get to the end successfully (I've pushed my work to the branch ssr-rakkas, but it's quite outdated now).
There's also a recent addition to Remix that allows you to use Vite, but the integration is unstable, so not sure if I want to try that route.

I've started work on this, check out https://github.com/megoth/rdfjs/tree/testing.

repo: megoth/rdfjs


text:
channel shuffle

V1里用了通道重排，为啥v2没有使用，2里面的通道重排函数只是一个split，block最后concat之后就没有channel shuffle

channel shuffle写在了每个block最开始，刚好就是上一个block concat之后的地方：
https://github.com/megvii-model/ShuffleNet-Series/blob/aa91feb71b01f28d0b8da3533d20a3edb11b1810/ShuffleNetV2/blocks.py#L49

感谢回复，我明白了

@megvii-model 你好，请问repo中关于shufflenetV2的channel_shuffle的使用是否和论文中的与torchvision中的实现不同？
repo中仅在stride==1时经过channel_shuffle，但论文和torchvision实现中均为总是经过channel_shuffle，无论stride是1还是2:
https://github.com/megvii-model/ShuffleNet-Series/blob/aa91feb71b01f28d0b8da3533d20a3edb11b1810/ShuffleNetV2/blocks.py#L47-L54
https://github.com/pytorch/vision/blob/d0e16b7607f8ce53e190fe286dcc338c6dc8a3cc/torchvision/models/shufflenetv2.py#L99
导出ONNX模型用Netron观察也是和从torchvision导出的不同

authors: WuSiYu fengchen321 megvii-model

repo: megvii-model/ShuffleNet-Series


text:
Bump meilisearch-ruby (v0.18.0)

Update meilisearch-ruby to v0.18.0
Remove get_or_create_index method usage
Changes regarding task API changes

This PR is breaking because uses meilisearch-ruby v0.18.0 that only works with MeiliSearch v0.25.0 and later -> it enforces the users to upgrade their MeiliSearch version

repo: meilisearch/meilisearch-rails


text:
Add multi search

Pull Request
Related issue
Fixes #246
What does this PR do?

Add's multi search method

PR checklist
Please check if your PR fulfills the following requirements:

[ ] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
[x] Have you read the contributing guidelines?
[x] Have you made sure that the title is accurate and descriptive of the changes?

Opening this PR to discuss the implementation, because not sure what are the expectations, so not ready to merge :)

@brunoocasali I'm not sure if we should have a totally separate SearchQuery that is converted to the client one or extend it.. which one you prefer? I chose former one because I thought there is not point to have a setIndexUid method, but then we must duplicate everything.

Why you don't think the setIndexUid is not useful for the multi-search? From what I saw in the code you're going to send a multi-search for all the configured indices right?
If yes, I'm not sure it is the right approach... Since I could have a lot of defined models but want to search in a subset, in this case I would have a lot of useless data being sent over the network.

Why you don't think the setIndexUid is not useful for the multi-search?

That's because in bundle we operate on the class level, so you instantiate a SearchQuery with for example test fixture Post::class which will be later then converted to the real index name test_posts etc.
But sure, if you have the full index name at hand, maybe it could be useful :thinking:

Oh, okay I see it now!

Just a comment + fixing the styling, and we're good to go!

aggregator support needs to be added I think also, but first I need to understand it how to make it work :)

Any up for this ? Really looking for it, would be really useful
❤

@brunoocasali do you have any suggestions how to make this work with aggregators?
For example if you define Post index and then an aggregator that is also uses Post then there is an ambiguity where to search.
Do we need to return for both indexes? What if I want to really only search in Post index and avoid searching in aggregated index?

For example if you define Post index and then an aggregator that is also uses Post then there is an ambiguity where to search.

Can you remind me about what the "aggregators" should do in the first place? Is it the ability to map two models into the same index?
Because in any case, I think we could release without that and figure out that later, WDYT?

Can you remind me about what the "aggregators" should do in the first place? Is it the ability to map two models into the same index?

I think so..

Because in any case, I think we could release without that and figure out that later, WDYT?

Okay maybe :) also I think it's possible to define same class as multiple indexes,so there would be same problem that it would search also in all of those 🤔

Okay maybe :) also I think it's possible to define same class as multiple indexes,so there would be same problem that it would search also in all of those 🤔

Do you mean, even without this PR? Because I think this could be a problem indeed, so we would not need to worry about that possibility here, right?

I think if you feel the code for the feature is good enough you can push forward and then grab some user feedback about the feature, WDYT?
I also encouraged the rails maintainer to do a similar approach, instead of waiting to have the best public API in town 😅.

Yeah ok,just need to find some time to finish it:)

Hi, do you need help here? Maybe I'll be able to do something. Multisearch is an important functionality for me :)

Hi, if you want to continue the work of this then go ahead.

What is the work required? Is it to remove the unnecessary comment and add getters to fix the code style issues?

To decide properly how it should behave :)

authors: SirMishaa brunoocasali czachor mrmadhat norkunas

repo: meilisearch/meilisearch-symfony


text:
Uncomment clippy from the ci check

The issue has been fixed in the latest release of rust. See https://github.com/rust-lang/rust-clippy/issues/8662
Fix #2305

bors try

authors: curquiza irevoire

repo: meilisearch/meilisearch


text:
Chore/sonarqube lint

Fix some issues reported by sonarqube

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (cafcd45) 99.73% compared to head (e6943a7) 99.86%.
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #19      +/-   ##
==========================================
+ Coverage   99.73%   99.86%   +0.13%
==========================================
  Files          13       13
  Lines         754      754
  Branches       20       20
==========================================
+ Hits          752      753       +1
+ Misses          1        0       -1
  Partials        1        1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

authors: codecov-commenter melgish

repo: melgish/bingo-board


text:
🛑 Siege Green Books is down

In 913f12f, Siege Green Books (https://book.siege-green.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Siege Green Books is back up in 94f2a2f after 4 minutes.

repo: meloncafe/status-sggrn-upptime


text:
🛑 Siege Green Auth is down

In 1ee06f5, Siege Green Auth (https://auth.siege-green.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Siege Green Auth is back up in dc80d25 after 2 hours, 43 minutes.


text:
🛑 Siege Green Books is down

In ab6a7ce, Siege Green Books (https://book.siege-green.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Siege Green Books is back up in f357e67 after 7 minutes.


text:
1.18.2 0.7 forge crash at startup

The new version does not start.
Latest: https://0x0.st/oYjl.txt
Crash: https://0x0.st/oYjU.txt

That's interesting... It works fine on its own.
I can't tell you what's going on right now. I'll look closer later.
Sorry for the inconvenience 😅

No problem, take your time and keep up your good work!

same for forge 1.19.2 and 0.7, but version 0.6 already didn't work (crash at startup):
crash: https://pastebin.com/xt4nh9BV
debug.log

No problem, take your time and keep up your good work!

@agentwolf88 sooo, I think I fixed the 1.18 version.
Could you please check out if this version works for you?

Works fine now. :D THX!

Should I create a separate issue for 1.19.2?

Yeah.
Also, I still don't know what the problem with 1.19 is. Seems like forge changed more things internally. I'm not sure how soon (or if at all) I can fix this.

authors: agentwolf88 melontini thimblebird


text:
Meltano UI - Error in Analyze Menu

Migrated from GitLab: https://gitlab.com/meltano/meltano/-/issues/460
Originally created by @iroussos on 2019-03-07 18:15:49

What is the current bug behavior?
Using the latest version from master (c8cabe49a054ebb45f2dedf50590a4d95ccbf640)
When I run meltano ui and I go to the Analyze tab, I get an Error Result instead of the supported topics.
What is the expected correct behavior?
I Should see all the compiled topics.
Steps to reproduce

Pull latest version from master (c8cabe49a054ebb45f2dedf50590a4d95ccbf640)
run meltano ui
Visit http://localhost:5000/
Hover over the Analyze menu

Relevant logs and/or screenshots

Further regression test
(Ensure we automatically catch similar issues in the future)

View 11 previous comments from the original issue on GitLab

repo: meltano/meltano


text:
Docs: New section for Replicating Data > Troubleshooting > Performance Benchmarking

Some recent conversations with community members have sparked the idea for a performance benchmarking/troubleshooting section.
Possible rough content outline

Start by listing all the possible areas of slowdown: CLI startup, tap throughput, tap rate limits,  target throughput, target batch frequency, network latency, network bandwidth, RAM limitations and buffer overruns.
Benchmark a data movement noop such as meltano config PLUGIN_NAME list.
Benchmark the EL processing times from a single tap to a single target.
Benchmark the tap alone. This would echo STDOUT to file(s) or to a null buffer.
Benchmark the target alone. This would take data from pre-generated file(s) to isolate target performance from the performance of the tap.

Iterative approaches
We could absolutely start this off with some very basic instructions on just a few of the above bullets. A good first iteration would at minimum give basic instructions to run the tap and target separately, since that is the first step in identifying where the EL pipeline is being throttled.
Related discussions

https://github.com/meltano/sdk/discussions/906 from @BuzzCutNorman
https://github.com/meltano/meltano/issues/6597 from @dwallace0723
https://github.com/meltano/sdk/issues/9 which would benefit from before-and-after benchmark data.

cc @edgarrmondragon, @visch, @pandemicsyn, @afolson, @tayloramurphy

From a users perspective a guide would be very helpful maybe something like this (Which might help drive some of this?)
Title: How to find performance issues?

We're going to focus on how to answer the question:
- The total time to run the job is very long, why?

The next question we need to answer is where is the bottle neck?
1. Is it the tap?
2. Is it the target?
3. Is it meltano?

To find out of the issue is the tap we should run just the tap for some subset of records. Ideally a subset that can fit on your disk easily. Steps:
1. First, we'll get a baseline by running `time {meltano run tap-name target-name}`
2. Second, let's test just the tap. We'll run `time { meltano invoke tap-name > tap.singer }` This will output the tap data into `tap.singer` and give you a time for how long the tap takes to run.
3. Third,  let's test how long the target takes to run with data already pulled from the tap by running `time { cat tap.singer | meltano invoke target-name }`.

Look at these time differences the time from number 1 above should match either number 2 or number 3. From this we can tell whether the issue is with our tap or with our target.

@sbalnojan - Following from the slack thread here, do you think this is something you might be able to pick up in a future iteration? We can all provide our own tips/patterns if helpful.

@aaronsteers I think so. I already made a note to include a kind of "write up" in the next newsletter, following that up with an addition to the docs should be easy.

@sbalnojan this was on the Meltano Engineering Board but on a super old iteration. I'm going to remove it to help keep that focused. Likely it should go on the devrel board if we want to prioritize it.

authors: aaronsteers sbalnojan tayloramurphy visch


text:
custom dimensions not supported in tap-google-analytics

Migrated from GitLab: https://gitlab.com/meltano/meltano/-/issues/833
Originally created by @kylewill on 2019-07-29 20:38:06

What is the current bug behavior?
Adding custom dimensions in the reports.json generates and error and does not call the Google Analytics API to extract.
What is the expected correct behavior?
What should be happening?
tap-google-analytics should allow for custom dimensions as described by the Google Analytics API.
Steps to reproduce

Add a custom dimension to reports.json
run elt
error generated by tap_google_analytics/reports_helper.py

How to find custom dimensions (ours are generated by Clearbit Reveal):

Click Admin from GA ui
Under Propery Column, click "Custom Definitions", then "Custom Dimensions"
Add the dimension index to reports.json based on the Google Analytics API Docs (ga:dimensionXX where XX is the index number from Step 2)

Relevant logs and/or screenshots
Example reports.json with proper custom dimension format:
[
  { "name" : "events",
    "dimensions" :
    [
      "ga:dimension3",
      "ga:date",
      "ga:week",
      "ga:eventCategory",
      "ga:eventAction",
      "ga:eventLabel"
    ],
    "metrics" :
    [
      "ga:totalEvents",
      "ga:uniqueEvents",
      "ga:sessionsWithEvent",
      "ga:eventsPerSessionWithEvent",
      "ga:users",
      "ga:sessions"
    ]
  }
]

Error when using the proper custom dimension format:
CRITICAL 'ga:dimension3'
Traceback (most recent call last):
  File "/meltanoexp/carbon/.meltano/extractors/tap-google-analytics/venv/bin/tap-google-analytics", line 11, in <module>
    load_entry_point('tap-google-analytics==0.1.0', 'console_scripts', 'tap-google-analytics')()
  File "/.meltano/extractors/tap-google-analytics/venv/lib/python3.7/site-packages/singer/utils.py", line 225, in wrapped
    return fnc(*args, **kwargs)
  File "/.meltano/extractors/tap-google-analytics/venv/lib/python3.7/site-packages/tap_google_analytics/__init__.py", line 184, in main
    catalog = discover(args.config)
  File "/.meltano/extractors/tap-google-analytics/venv/lib/python3.7/site-packages/tap_google_analytics/__init__.py", line 43, in discover
    return reports_helper.generate_catalog()
  File "/.meltano/extractors/tap-google-analytics/venv/lib/python3.7/site-packages/tap_google_analytics/reports_helper.py", line 85, in generate_catalog
    data_type = self.lookup_data_type('dimension', dimension)
  File "/.meltano/extractors/tap-google-analytics/venv/lib/python3.7/site-packages/tap_google_analytics/reports_helper.py", line 155, in lookup_data_type
    attr_type = self.valid_dimensions[attribute]
KeyError: 'ga:dimension3'
[MainThread][ERROR][2019-07-29 16:12:18,580] Subprocesses didn't exit cleanly: tap(1), target(0)

Example reports.json with hardcoded ga:dimensionXX as returned from Metadata API:
[
  { "name" : "events",
    "dimensions" :
    [
      "ga:dimensionXX",
      "ga:date",
      "ga:week",
      "ga:eventCategory",
      "ga:eventAction",
      "ga:eventLabel"
    ],
    "metrics" :
    [
      "ga:totalEvents",
      "ga:uniqueEvents",
      "ga:sessionsWithEvent",
      "ga:eventsPerSessionWithEvent",
      "ga:users",
      "ga:sessions"
    ]
  }
]

Error from hardcoded ga:dimensionXX
ERROR Giving up query_api(...) after 1 tries (googleapiclient.errors.HttpError: <HttpError 400 when requesting https://analyticsreporting.googleapis.com/v4/reports:batchGet?alt=json returned "Unknown dimension(s): ga:dimensionXX
For details see https://developers.google.com/analytics/devguides/reporting/core/dimsmets.">)
ERROR Skipping stream: 'events' due to invalid report definition.
[MainThread][ERROR][2019-07-29 16:33:54,737] Subprocesses didn't exit cleanly: tap(1), target(0)

Possible fixes
tap-google-analytics should be updated to allow for dimensions with the ga:dimensionXX format where XX is an integer up to 200.
I noticed that tap-google-analytics is calling the "metadata" endpoint to check for valid dimensions / metrics. The Metadata response does NOT include the custom dimensions IDs for a given GA account, instead it includes the format ga:dimensionXX where XX = the custom dimension index.
If you hardcode ga:dimensionXX reports_helper.py does not trigger any errors but the actual call to Google Analytics API fails (since ga:dimensionXX is the format not a specific id).
Here is the line from the Metadata call for custom dimensions:
{
    "id": "ga:dimensionXX",
    "kind": "analytics#column",
    "attributes":
    {
        "type": "DIMENSION",
        "dataType": "STRING",
        "group": "Custom Variables or Columns",
        "status": "PUBLIC",
        "uiName": "Custom Dimension XX",
        "description": "The value of the requested custom dimension, where XX refers to the number or index of the custom dimension.",
        "minTemplateIndex": "1",
        "maxTemplateIndex": "20",
        "premiumMinTemplateIndex": "1",
        "premiumMaxTemplateIndex": "200",
        "allowedInSegments": "true",
        "addedInApiVersion": "3"
    }
}

View 11 previous comments from the original issue on GitLab


text:
Add paradigm for stream filter constraints and max_records constraints

Currently, developers are left to inspect self.replication_key, self.get_starting_replication_key_value(), self.get_starting_timestamp() themselves, within get_records() or get_batches(). This isn't great for a number of reasons.
To make the filters expected much more explicit, this proposal suggests that we pass something like a StreamFilter object to methods like get_records() and get_batches(), and perhaps also to methods like get_url_params() which may be able to pass down queries to the API call.

By localizing the filter to each method call, we also open up options for multiple partitions of the same dataset to be queried simultaneously.
By handling the filter rulesets generically, we unlock use cases that also want a "max" constraint, such as #922, which in turn unlocks parallel processing of time partitions, as noted above.
Developers get the option of using the generic apply, like filtered_records = filters.apply(unfiltered_records) and include: bool = filters.eval(record_dict).
Alternatively, developers can loop through the StreamFilter.filters set, and handle each filter in a custom manner if needed. (Such as sending eligible constraints as filters to the remote API.)

Some possible psuedocode to get a feel for how this might look:

Details
class Stream
    # ...
    def get_batches(
        self,
        batch_config: BatchConfig,
        context: dict | None = None,
        filters: StreamFilters,
    ) -> Iterable[tuple[BaseBatchFileEncoding, list[str]]]:
        """Batch generator function.

        Developers are encouraged to override this method to customize batching
        behavior for databases, bulk APIs, etc.

        Args:
            batch_config: Batch config for this stream.
            context: Stream partition or context dictionary.
            filters: A StreamFilters object defining any restrictions which should be applied to the dataset.

        Yields:
            A tuple of (encoding, manifest) for each batch.
        """

from operator import le, lt, gt, ge, eq, ne

class RecordFilter:
    def __init__(self, property_name: str, property_value: Any, operator: Callable[[Any, Any], bool]):
        # Only support a finite set of operators:
        if operator not in [le, lt, gt, ge, eq, ne]:
            raise ValueError("Unsupported operator: {operator.__name__}")

        self.property_name = property_name
        self.property_value = property_value
        self.operator = operator

    def eval(self, record:dict) -> bool:
        """Return True to keep, False to exclude."""
        return self.operator(record[self.property_name], self.property_value)

class StreamFilter:
    filters: list[RecordFilter]
    max_record_limit: int | None

    def eval(self, record: dict) -> bool:
        """Return True to keep the record, False to exclude."""
        return all((filter.eval(record) for filter in self.filters))

    def apply(self, records: Iterable[dict]) -> Iterable[dict]
         """Can be called against a set of records to return only those which match."""
         if self.max_record_limit:
             yield from itertools.islice((record for record in records if self.filter(record), self.max_record_limit)
         else:
             yield from (record for record in records if self.filter(record))

Implementation for SQL taps
For SQL taps, we obviously would not use the inline Python-based evaluators, but instead we could map the filter constraints to WHERE clause filters and LIMIT restrictions, passed generically to SQLAlchemy.
As it relates to the get_batches() method, this could be introduced as a breaking change (sooner the better). Since there are no 'stable' BATCH message implementations as of this writing, it should be acceptable to make this change.
Implementing for get_records() implementations in a backwards-compatible manner.
In regards to existing taps that already implement get_records():

Internally we can add a new Stream.filter_records() method that automatically applies the filterset to the records produced by Stream.get_records() - probably after Stream.post_process(), to ensure the properties are in the expected place.
For performance reasons, we can advise developers to override Stream.filter_records() to no-op any filters they've already handled in get_records().
When SDK 1.0 releases, we would update the signature of get_records(), perhaps still preserving the generic Stream.filter_records() for convenience in the default implementation.

Related
This probably also resolves:

https://github.com/meltano/sdk/issues/226

And would make this a pretty easy fast follow:

https://github.com/meltano/sdk/issues/922

@edgarrmondragon  and @kgpayne - I updated the above with a possible get_records() implementation proposal. Would love to get your thoughts.

Still relevant

authors: aaronsteers edgarrmondragon

repo: meltano/sdk


text:
Singer SDK Version
0.13.1
Python Version
3.10
Bug scope
Configuration (settings parsing, validation, etc.)
Operating System
MacOS
Description
Logger cannot be configured with env var LOGGING_CONF_FILE.
If we have the following log config:
# logging.conf
...
format=LOGGED USING logging.conf: %(message)s
...

... and run our tap, the output does not use our requested format:
$ LOGGING_CONF_FILE=logging.conf poetry run tap-mytap
2022-11-16 11:02:03,828 Beginning incremental sync of 'groups'...

This used to work pre-12.0 - I think it's due to the dedicated metrics logger added in 0.12.0. If we use the new env var (and different yml file format) for that, then it works:
# logging.yml
...
formatters:
  child:
    class: logging.Formatter
    format: 'LOGGED USING logging.json: %(message)s'
...

$ SINGER_SDK_LOG_CONFIG=logging.yml poetry run tap-mytap
LOGGED USING logging.json: Beginning incremental sync of 'groups'...

It looks like one logging config is overriding the other.
Full log configs added below, I think this can be reproduced on any tap/target.
Code
# logging.conf
[loggers]
keys=root

[handlers]
keys=stderr

[formatters]
keys=child

[logger_root]
level=INFO
handlers=stderr
formatter=child
propagate=0

[handler_stderr]
level=INFO
class=StreamHandler
formatter=child
args=(sys.stderr,)

[formatter_child]
class=logging.Formatter
format=LOGGED USING logging.conf: %(message)s
datefmt=%Y-%m-%d %H:%M:%S

# logging.yml
version: 1
disable_existing_loggers: false
loggers:
  root:
    level: DEBUG
    handlers:
      - consoleHandler
    formatter: child
    propagate: true
formatters:
  child:
    class: logging.Formatter
    format: 'LOGGED USING logging.json: %(message)s'
    datefmt: '%Y-%m-%d %H:%M:%S'
handlers:
  consoleHandler:
    level: DEBUG
    class: logging.StreamHandler
    formatter: child
    stream: ext://sys.stderr

@edgarrmondragon @aaronsteers should we warn here when duplicate logging is configured? Seems like some taps used LOGGING_CONF_FILE but we're now recommending logging.yml, right?

@tayloramurphy @aelred AFAICT it was only ever possible to configure SDK-based taps using LOGGING_CONF_FILE by accident when we called any method in the pipelinewise-singer-python library that called get_logger, now removed via https://github.com/meltano/sdk/pull/979
It's seems rather easy to support both somewhere in https://github.com/meltano/sdk/blob/b55494ee85b9ac4dea5963265fdaf34dc8e1b0cd/singer_sdk/metrics.py#L394-L408
by preferring SINGER_SDK_LOG_CONFIG but accepting LOGGING_CONFIG_FILE too if the first is not present.
@aelred would you be willing to contribute this?

Thanks for the context @edgarrmondragon ! @aelred we'll definitely take a PR to add support but likely won't prioritize this ourselves.

Still accepting contributions if anyone really need this

authors: aelred edgarrmondragon tayloramurphy


text:
feat: SSH Tunnel support for all target-sql targets

Feature scope
Targets (data type handling, batching, SQL object generation, etc.)
Description
We have an implementation here https://github.com/MeltanoLabs/tap-postgres/pull/94 that should be generic enough for every sql based target with sqlalchemy. We'd have to polish this implementation up a bit most likely but the general idea seems good to me!

Another implementation here https://github.com/MeltanoLabs/tap-mysql/pull/6


text:
docs: Document an example of date-range pagination

Documentation type
How-to guides
Description
from __future__ import annotations

import datetime
import typing as t
from urllib.parse import parse_qsl, urlparse

from requests import Response

from singer_sdk.helpers._compat import date_fromisoformat
from singer_sdk.pagination import BaseAPIPaginator

class PageValue(t.NamedTuple):
    """A page value for DateRangePage."""

    start_date: datetime.date
    end_date: datetime.date
    page: int

class DateRangeAndPagePaginator(BaseAPIPaginator[PageValue]):
    """Pagination class for DateRangePage."""

    def __init__(
        self,
        start_date: datetime.date,
    ) -> None:
        self.today = datetime.datetime.now(tz=datetime.timezone.utc).date()
        start_value = self._get_next_date_range(start_date)
        super().__init__(start_value=start_value)

    def _get_next_date_range(self, start_date: datetime.date) -> PageValue | None:
        end_date = min(self.today, start_date + datetime.timedelta(days=60))

        # End date is inclusive, so we need to start the next range a day after
        start_date = start_date + datetime.timedelta(days=1)

        return None if end_date == self.today else PageValue(start_date, end_date, 1)

    def get_next(self, response):
        """Return the next page tuple or None."""
        next_url: str = response.links.get("next", {}).get("url")
        if next_url:
            parsed_url = urlparse(next_url)
            query = dict(parse_qsl(parsed_url.query))

            start_date = date_fromisoformat(query.get("start_date"))
            end_date = date_fromisoformat(query.get("end_date"))
            page = int(query.get("page"))

            return PageValue(start_date, end_date, page)

        return self._get_next_date_range(self.current_value.end_date)

paginator = DateRangeAndPagePaginator(start_date=datetime.date(2021, 1, 1))
print(paginator.current_value)
# PageValue(start_date=datetime.date(2021, 1, 2), end_date=datetime.date(2021, 3, 2), page=1)  # noqa: ERA001, E501

next_link_template = '<https://example.com?start_date={start_date}&end_date={end_date}&page={page}>; rel="next"'  # noqa: E501
response = Response()
response.headers = {
    "link": next_link_template.format(
        start_date=paginator.current_value.start_date,
        end_date=paginator.current_value.end_date,
        page=paginator.current_value.page + 1,
    ),
}

paginator.advance(response)
print(paginator.current_value)
# PageValue(start_date=datetime.date(2021, 1, 2), end_date=datetime.date(2021, 3, 2), page=2)  # noqa: ERA001, E501

response = Response()
paginator.advance(response)
print(paginator.current_value)
# PageValue(start_date=datetime.date(2021, 3, 3), end_date=datetime.date(2021, 5, 1), page=1)  # noqa: ERA001, E501

The risk with this is that the last page in a date range comes empty and pagination is terminated prematurely:
https://github.com/meltano/sdk/blob/950f391b3b64755cfe8e8250b66eed726d560353/singer_sdk/streams/rest.py#L389-L395

Related:

https://github.com/meltano/sdk/issues/1663

One way to circumvent breaking the loop when a date range is empty would be to make the break behavior in that loop an opt-out based on a new attribute of the RESTStream class. PRs welcome!
https://meltano.slack.com/archives/C068YBQQF1V/p1720057717641439?thread_ts=1720043007.794179&cid=C068YBQQF1V


text:
Unable to get options.configPath working

Is it possible to add a warning log when the custom config path does not exist?
I am unable to troubleshoot why the jest angular builder cannot find my custom jest config.
The current custom-config.resolver.ts
static resolveForProject(projectRoot: Path, configPath: string){
  const jestConfigPath = getSystemPath(join(projectRoot, configPath));
  return existsSync(jestConfigPath) && require(jestConfigPath) || {};
}

I propose adding:
console.log(`warning: unable to locate custom jest configuration at path "${jestConfigPath}"`);

The final code might look like:
static resolveForProject(projectRoot: Path, configPath: string){
  const jestConfigPath = getSystemPath(join(projectRoot, configPath));
  const exists = existsSync(jestConfigPath);
  if (!exists) {
    console.log(`warning: unable to locate custom jest configuration file at path "${jestConfigPath}"`);
    return {};
  }
  return require(jestConfigPath);
}

I would be more than happy to submit the PR myself if the concept is well received.
Thanks!

Sounds like a reasonable requirement to me.
The implementation though shall be a bit different. You should use BuilderContext logger (like in this gist) instead of console.log.
If you feel like implementing it - sure, go ahead!
Also, as far as I understand, there are two problems here:

Absence of warning
Your specific problem with jest.config path

Did you manage to solve 2 or you need help with that?

Thank you for the quick feedback!
I'll see if i can get something up for review this week.
As for the jest.config.js path - I am still not able to resolve this.
Currently, I have it working but only because I place the config at the default path (project root)
Below is a snippet of my file tree:
├── projects
│   ├── project1
│   └── project2
│       └── src
│           └── project2-jest.config.js
├── tools
│   └── jest
│       ├── jest-global-mocks.ts
│       └── setup-jest.ts
├── jest.config.js
└── angular.json

Here is the angular.json snippet:
{
  "projects": {
    "project1": {
      "targets": {
        "test": {
          "builder": "@angular-builders/jest:run",
          "options": {
            "preset": "jest-preset-angular",
            "setupTestFrameworkScriptFile": "<rootDir>/tools/jest/setup-jest.ts",
            "configPath": "<rootDir>/project1/src/project1-jest.config.js"
          }
        },
      }
    }
  }
}

For the config path I have also tried ./projects/project1/src/project1-jest.config.js but this silently fails as well.

Cool thanks! I figured it out after fiddling with the source code actually xD
Please checkout the PR: https://github.com/meltedspark/angular-builders/pull/87

Why does this warning make sense? What if I don't need custom config and the default one works fine? What if I configure custom configuration in package.json and not in jest.config.js?

Yeah, probably it should be displayed only in certain circumstances. The question is in which circumstances.
Indeed, if you don't need custom config you're fine. But if you do need it and you place it in a different place... You probably want to see this warning.
The builder hides complexity behind Jest setup. Part of it is that the user doesn't have to specify config path explicitly - he can just use the default path and put the custom config under the directory specified in the docs. The builder, however, cannot distinguish between config path that was specified explicitly and the default one since the default comes from the JSON schema which is parsed by architect.
I'm open for suggestions though.

authors: denis-itskovich esotrope meltedspark

repo: meltedspark/angular-builders


text:
Fix examples

In this PR:

Use membrane_h26x_plugin and make it only available on test env.
Get width and height from H265 format in cmaf muxer
Fix examples by adding start_link since it's not  generated anymore by Membrane.Pipeline

Hi @gBillal, thanks for the fixes! It seems you updated Credo and it's not passing now, please either downgrade or fix the warnings, thanks

@mat-hek fixed

authors: gBillal mat-hek


text:
List devices

closes #9 🎉

Thank you @mat-hek! ❤

authors: mat-hek thbar


text:
Adding unit tests for watch deletions when calling mcp.internal()

The watch deletions command is not printing the deletion logs correctly when sending deletion requests through mcp.internal(). The attached unit test file illustrates this issue.

Thanks!

@YufeiHu
Thanks! Any chance you could integrate this test into t/proxyinternal.t instead of having it be its own file? If not I'll do it; but just offering in case you feel up to contributing.
That test is listening on CMD_ANY so it should be pretty trivial to add the watcher bits as a subtest

@dormando
I've addressed the suggestion! The test now passes after I applied the same fix as in your branch (https://github.com/memcached/memcached/pull/1178).
If there are any further adjustments needed, I’ll take care of them promptly. Thanks!

Looks rad, thanks!
I need to port more of these test files to the object based one though, sorry!

merged

authors: YufeiHu dormando

repo: memcached/memcached


text:
Is this package still being worked on

We're really keen to use this library with the OGM model however on inspection of the typescript examples, all files seem to contain only copy past version of a single example. It would be really great if we could get OGM system working with Node.js/typescript. Regards.

Hi @ninnjak!
Yes, it is 😄 But we don't have many users here, so the project was maintained in a limited capacity so far. But, super happy to help make it work 😄
In general, 2 biggest topics are OS support since this package depends on mgclient (mgclient already has a bigger list of supported operating systems), and the API. So my two main questions are:

Which operating systems are you planning to use?
Any feedback on how to improve the current API?

Hi @gitbuda, thanks for the quick reply.
We are running our existing ops on cloud services using Kubernetes and Docker (Alpine Linux image) with whilst we use MacOS internally to develop. We haven’t used the Python version (GQLAlchemy) of the API, however we currently use Neo4j with @neo4j/graphql and @neo4j/graphql-ogm and there OGM API is great, maybe you could make the API as close as possible which could aid users to migrate more seamlessly.

authors: gitbuda ninnjak

repo: memgraph/nodemgclient


text:
🛑 Kérwá is down

In 5797a7b, Kérwá (https://kerwa.ucr.ac.cr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Kérwá is back up in 8d8aa63 after 1 hour, 4 minutes.

repo: memoemg/howyoudoing


text:
WIP: Text box web & native on input event property

The core text box widget will include an on input event property starting from MX 8.13.

Please label this PR as work-in-progress. It's not certain whether we will release this for MX 8.13

@IIsaku is this going into 8.13, or is it still being debated?

authors: ConnorLand IIsaku

repo: mendix/docs


text:
Updating pics to show App Model instead of Project Model

This is related to https://mendix.atlassian.net/browse/AA-3679.
It's Project Model instead of App Model on original pics

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Pavel Shirvanov seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

authors: CLAassistant shirvanoff


text:
/ignore or /filter command

It would be useful to be able to ignore /part /join /quit messages.

@nsjph good idea - I'll get something added for it.

authors: mephux nsjph

repo: mephux/komanda-cli


text:
模型验证

您好，想问一下您实现的这个1D的swinformer有在什么数据集上验证有效性吗？谢谢

你好，这些模型我是在一些脑电的数据集上进行了验证。老实说，效果并不好，原因主要是数据量太少，并不能很好发挥特性。

authors: meraks wuyefa


text:
⚠ SkyfallMC Minecraft Server has degraded performance

In 5a2b08e, SkyfallMC Minecraft Server (play.skyfallmc.ga) experienced degraded performance:

HTTP code: 200
Response time: 1757 ms

Resolved: SkyfallMC Minecraft Server performance has improved in d26047c.


text:
Implement first pass of very basic data visualization in Queries area

See the comments in this issue: https://github.com/mergestat/internal/issues/86 we've done a Figma pass at basic data visualizations in the Queries section.
This ticket is to track the UI work for the first pass at implementing basic data viz. In fact, for this ticket let's only focus on:

The overall structure changes we need to make to the query results area (to support the tabs for new "views"
The single metric "visualization" type

We will tackle the charts and other options in subsequent tickets.

@patrickdevivo @emilyvandewalle Is the "Number" drop down in this design meant to be a reference to a column name or to a  number format? I think this should be for formatting but I could also imagine a query that returns a row with several metrics but we only display the one so we would need to reference a column. That being said I think for a first pass the single metric viz should expect a query that only returns one row with one metric and that we do not do column selection. Thoughts?

This is going to be very basic. This would be something I think we need to have live on a branch or if we had feature flags I would suggest we use them there. Apex does not have a single metric component so this one is going to have to be homegrown. I would also suggest we don't implement download on this for now as it seems like early optimization on this homegrown component but I won't fight anyone on that.

@german-mergestat here are some decisions that @patrickdevivo and I made about this feature.

The number drop down should list the available columns from the query to use for the metric
The download and save to dashboards are not needed for now
We should go ahead and add the Line and Bar chart options but grey them out until we implement them
The number that we show in the metric will be the first number returned in the column that is selected. So on this example only 1 or 2 could be shown based on what column is selected

authors: amenowanna patrickdevivo

repo: mergestat/mergestat


text:
Add Azure AD in Azure Portal

Add Azure AD in Azure portal command (separate from Entra portal).
Fixes #23

Thanks @sam-cogan . Sorry to double up, I just added this.

authors: merill sam-cogan

repo: merill/cmd


text:
[UI] Display Kubernetes Context Name in MeshSync Flush Data confirmation modal

Current Behavior
In the Settings page under the Environments tab, users can click to flush the MeshSync data for a specific Kubernetes context that they have connected to Meshery. When they click to flush this data, only the data for that context is removed, but the confirmation model that shows displays to the user does not show the name of the Kubernetes context.
Desired Behavior
The confirmation model should display the Kubernetes cluster name in quotation marks, so that the user is confident that only MeshSync data for the specific (currently selected) Kubernetes cluster will be deleted.
Implementation

Acceptance Tests

Confirmation title bar should be:
Flush MeshSync data for "<k8s cluster name">?
Detailed message shown should be:

Are you sure that you want to remove MeshSync data for the "<k8s cluster name>"?

Fresh MeshSync data will be repopulated for this context, if MeshSync is actively running on this cluster.

Screenshot
Here's a screenshot of the current implementation that is missing the cluster name.

Contributor Guides and Handbook

🎨 Wireframes and designs for Meshery UI in Figma
🖥 Contributing to Meshery UI
🙋🏾🙋🏼 Questions: Discussion Forum and Community Slack

hello @leecalcote in 2 days  , as my exams come to a halt . I will be back here to finish this one 💯 .
PS:- I am missing this place.

Can I work on this?

hi @leecalcote I am on this now can you please assign?

repo: meshery/meshery


text:
I have made changes to the scrollbar css and now it is not overflowin…

I have made changes to the scrollbar css and now it is not overflowing the parent container
I have made changes to the MesheryTreeView.js file.
Notes for Reviewers
This is my first PR in meshery
This PR fixes #10434
Signed commits

[ ] Yes, I signed my commits.

@Yashsharma1911 I have moved the scroll bar

@kunal-511 any screenshot or recording?

@sudhanshutech Above are the screenshots for the fix made
screenshot1- is the error
screenshot2- is after the fix

@kunal-511 Let's discuss this in Meshery dev call today. I'm adding this as an agenda item to the meeting minutes.

@kunal-511 Let's discuss this in Meshery dev call today. I'm adding this as an agenda item to the meeting minutes.

ok I will be there in the meeting

@sudhanshutech @Yashsharma1911

Thankyou for this @kunal-511 this is small item but should be covered, can you make sure that the way this line connect on right side it also does on left? make sure it does affect the other UI

In the original UI, there is a gap from both sides so I have to make that gap or join it from left side as in right side?

@kunal-511, please know that @Yashsharma1911 is out for exams for a few days, but will be back early next week.


text:
[MeshModel] Add workflow and script to generate components | CU-3yvwk9h

Description
This PR adds a script that generates meshery components from artifacthub packages and adds a workflow that can be triggered manually to generate and persist those generated component in meshery/meshery repository.
Notes for Reviewers
Signed commits

[x] Yes, I signed my commits.

Task linked: CU-3yvwk9h Cronjob for pre-generating components in GitHub Action in meshery-extensions-packages

Codecov Report
Base: 17.26% // Head: 17.26% // Decreases project coverage by -0.00% :warning:

Coverage data is based on head (44eee08) compared to base (e06ed47).
Patch coverage: 0.00% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6387      +/-   ##
==========================================
- Coverage   17.26%   17.26%   -0.01%
==========================================
  Files         116      116
  Lines       10893    10885       -8
==========================================
- Hits         1881     1879       -2
+ Misses       8549     8543       -6
  Partials      463      463

Flag
Coverage Δ

e2etests
∅ <ø> (?)

gointegrationtests
5.87% <0.00%> (-0.02%)
:arrow_down:

integrationtests
∅ <ø> (?)

unittests
12.92% <0.00%> (+<0.01%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

mesheryctl/internal/cli/root/system/config.go
4.95% <0.00%> (+0.18%)
:arrow_up:

mesheryctl/internal/cli/root/system/start.go
58.33% <0.00%> (-0.88%)
:arrow_down:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


text:
Feature/exp components command

Notes for Reviewers
This PR fixes #
#9625
Signed commits

[x] Yes, I signed my commits.

Thanks for raising PR @RipulHandoo, please add this in Docs as your item to discuss it in the Meshery Development Meet today at 8:30 pm IST.
Check out Community Calendar for meet Details.

@RipulHandoo will you have a look at the failing lint checks?

@leecalcote  Sir, Currently, we are encountering a lint check failure attributed to missing fields (Category, Name, Version) in the ComponentDefinition used in our Model struct.
Please take a look at this.
I would greatly appreciate your insights on whether it is advisable to incorporate these fields.

🚀 Preview for commit fc9b2ad4f2d6c28ce7a22654501a112acda541a0 at: https://658d511ec7fcc49f46c83ab9--meshery-docs-preview.netlify.app

@RipulHandoo watch out for accidentally intermixing commits between branches

Hi @RipulHandoo since this is an exp command, I suggest you to add this in this entry in the file marked below...
https://github.com/meshery/meshery/blob/047800b0c42b9c100821f59601e11ac20d30b568/mesheryctl/internal/cli/root/experimental/experimental.go#L46-L49

@RipulHandoo are expecting these build failures?

@leecalcote Yes sir. I am expecting these failures. As some of the fields are not defined in the ComponentDefinition.

Thanks much for confirming, @RipulHandoo. Makes sense.
@Yashsharma1911 your 👀 on this would be helpful.

@RipulHandoo there's a spot with your name on it here - https://docs.google.com/document/d/1Dg4Uz9rGTuGHmYZwIt47z1j5x3LmnpxD1M1SgypAkQ0/edit

@RipulHandoo there's a spot with your name on it here - https://docs.google.com/document/d/1Dg4Uz9rGTuGHmYZwIt47z1j5x3LmnpxD1M1SgypAkQ0/edit

Which you have now deleted. Ok, never mind.

@RipulHandoo there's a spot with your name on it here - https://docs.google.com/document/d/1Dg4Uz9rGTuGHmYZwIt47z1j5x3LmnpxD1M1SgypAkQ0/edit

Which you have now deleted. Ok, never mind.

@leecalcote Sir.
Unfortunately, I won't be able to provide a live demonstration of the new Mesheryctl commands during today's meeting as I am currently traveling and away from my keyboard.
However, if it's feasible, I can still join the meeting via my mobile device to discuss any updates or answer questions.

@RipulHandoo here's a quick reference - https://docs.google.com/presentation/d/1SQMfyu5shjpGKlYONdVzOtd7UYTgLWBcgUvHMLCZ2tY/edit#slide=id.g232ac6f646e_0_158

https://github.com/meshery/schemas/tree/master/schemas/meshmodel/schemas

https://github.com/meshery/schemas/tree/master/schemas/meshmodel/schemas

This helped a lot in understanding the struct fields.
Thanks!!

@leecalcote sir, I've successfully tested the mesheryctl exp components commands locally, and they are functioning as expected. However, golangci-lint is encountering issues. I would greatly appreciate your guidance in resolving this matter. Thank you

Please demo this command at the next Meshery dev meeting.

Please demo this command at the next Meshery dev meeting.

Sure Sir. 🚀

@RipulHandoo good job on demo. Current items you need to look now are as follows:

UX message for invalid search. Eg: If a user entered a wrong/misspelled component, the command should print a message, not left void.
Pagination stuff you did is good. But I feel something is missing there, so I'll look it up and tell what needs to be done.

I'll create a design spec doc in meantime to track whatever we're doing. You can involve that as well.

@alphaX86 Thanks for the feedback on the demo! I'll work on adding the UX message for invalid search. Also, I'll be waiting for your guidance on the pagination part. I'm ready to help with the design spec doc too.
Thanks again!

@alphaX86 @RipulHandoo have either of you worked up a design spec?

@alphaX86 @RipulHandoo have either of you worked up a design spec?

A doc is made. I'll work with @RipulHandoo as I'm free tomorrow mostly.

Works fine with me. Thanks for the update.

Design spec doc link: https://docs.google.com/document/d/1pxBDl0qgA6RmfH2fm_9j9Cp556QDZaqPaAtPIY3CHX4/edit?usp=sharing

Hi @leecalcote and @alphaX86,
Could you please review the failing Mesheryctl/golangci-lint test in relation to the latest PR? I'm considering closing the current PR and opening a new one since the tests are failing due to some undefined fields in docs_test.go. Notably, this file hasn't been modified in this PR.
Your assistance in resolving this matter is greatly appreciated.

Hi @leecalcote and @alphaX86,
Could you please review the failing Mesheryctl/golangci-lint test in relation to the latest PR? I'm considering closing the current PR and opening a new one since the tests are failing due to some undefined fields in docs_test.go. Notably, this file hasn't been modified in this PR.
Your assistance in resolving this matter is greatly appreciated.

I'll look on the golangci-lint and the unit test issue. For now, implement the changes I've requested previously. If all fails up even after fixes etc, then we might consider a new PR. Until then, I suggest you to continue.

@leecalcote @alphaX86, should I add pagination to the mesheryctl exp components list command?
https://github.com/meshery/meshery/assets/107461226/da009d2e-6f3c-4f71-9ebb-05774c9a7124

@alphaX86 were you able to figure this out?

@RipulHandoo after you address this final merge conflict, let's merge.


text:
Support for username / password credentials for Cassandra

Chronos supports Cassandra as a statistic / metrics backend.
If you are running Cassandra with an auth required you are not able to use Cassandra with Chronos, because it don`t support username / password configurations.
The configuration settings are missing in https://github.com/mesos/chronos/blob/88522b574c491489662509fae09153c43bb61397/src/main/scala/org/apache/mesos/chronos/scheduler/config/CassandraConfiguration.scala
The connection is not build with username / password in https://github.com/mesos/chronos/blob/88522b574c491489662509fae09153c43bb61397/src/main/scala/org/apache/mesos/chronos/scheduler/config/JobStatsModule.scala
In the builder the withCredentials method is handy here.

@andygrunwald,  how about apply a PR

authors: andygrunwald xiaods

repo: mesos/chronos


text:
next: mesos.Client API should reflect changes in httpcli package

.. and the demo (sched/exec) code should be updated to use mesos.Client instead of httpcli.Client

some fixes here: e26bc076e0385bdb01923b816c0cda4945389ec6
demo scheduler and executor still maintain ref's to the *httpcli.Client instead of mesos.Client

repo: mesos/mesos-go


text:
per-topology supervisor logs

Since storm-mesos launches separate supervisors for each topology on a host,
we should be separating out the supervisor logs on a per-topology basis.
Towards this end, I have leveraged the STORM-1056 change I introduced into the storm project.  This has been released in storm-0.9.6 and storm-0.10.0.
With this change we can pass an env variable to bin/storm (or bin/storm.py in storm-0.10.0+)
which sets the name of the supervisor log file:

STORM_SUPERVISOR_LOG_FILE
Notably, the location that the logs are written to is unchanged.

Previously the logs were written (by default) to:
/tmp/mesos/slaves/SLAVE_UUID/frameworks/FRAMEWORK_UUID/executors/TOPOLOGY_ID/runs/RUN_UUID/storm-mesos-*/logs/supervisor.log

Now the logs will be written (by default) to:
/tmp/mesos/slaves/SLAVE_UUID/frameworks/FRAMEWORK_UUID/executors/TOPOLOGY_ID/runs/RUN_UUID/storm-mesos-*/logs/TOPOLOGY_ID-supervisor.log

If you set storm.log.dir in storm.yaml to a common place (e.g., for ingesting into splunk from a common dir), then it might look like this:
 # in storm.yaml
 storm.log.dir: "/var/tmp/storm/logs"

 # supervisor log file path:
 /var/tmp/storm/logs/TOPOLOGY_ID-supervisor.log

@dsKarthick & @jessicaLHartog: please :eyes:

:shipit:

LGTM! :ship:

authors: JessicaLHartog dsKarthick erikdw

repo: mesos/storm


text:
Added integration tests for validating JMX_PORT config

Added two integration tests for validating:

the default JMX_PORT being set to 7199 (test_jmx_default)
the ability to configure JMX_PORT (test_jmx_override)

Note: The test automation will report test_jmx_default as PASSING and test_jmx_override as FAILED. test_jmx_override will fail for this PR because we have not updated the https://downloads.mesosphere.com/cassandra/assets/apache-cassandra-3.0.8-bin-dcos.tar.gz yet, which is used by default in the stub universe package. We intend to do that when we'll release our next version.

Would it make sense to just mark the expected-to-fail test as disabled for now?

@nickbp Let me mark that test as disabled, and add a task to remind that we need to enable this test.

@mesosphere-ci retest this please

@mesosphere-ci retest this please

@mesosphere-ci  retest this please

@mesosphere-ci retest this please

@mesosphere-ci retest this please

@mohitsoni:  The needed change in binary dependency has been made.  All tests have passed.  Can you get this PR across the line if it is still needed?

retest this please

authors: gabrielhartmann mohitsoni nickbp triclambert

repo: mesosphere/dcos-cassandra-service


text:
Integrate API to get list of reported numbers

Integrate list to get reported numbers
Api: https://www.loom.com/share/3663614cd70d4c19ac3d823e4bc46156?sid=47ca159e-3d33-4450-aeb5-8832fcf0da1e

Hi @alexboateng it's Vincent here. I'm working on this

repo: mestafrica/mobile-wallet-fraud-database


text:
[Evals API][4/n] evals with generation meta-reference impl

-- continuation of https://github.com/meta-llama/llama-stack/pull/296
TL;DR

full evals jobs with generation

Tests
PROVIDER_ID=test-meta PROVIDER_CONFIG=llama_stack/providers/tests/eval/provider_config_example.yaml pytest -s llama_stack/providers/tests/eval/test_eval.py --tb=short --disable-warnings

Client Flow
App Evals Flow

scoring input dataset registration

client.datasets.register_dataset(
     DatasetDefWithProvider(
         identifier="test-dataset",
         provider_id="meta0",
         url=URL(
              uri=test_url,
         ),
         # Expect scoring input dataset schema to obey this format
         dataset_schema={
              "generated_answer": StringType(),
              "expected_answer": StringType(),
              "input_query": StringType(),
         },
      )
 )

scoring

client.scoring.score_batch(
    dataset_id="test-dataset",
    scoring_functions=["equality"],
)

# Example Response Output
results={
  'equality': ScoringResult(score_rows=[
    {
      'score': 0.0
    },
    {
      'score': 1.0
    },
    {
      'score': 1.0
    },
    {
      'score': 0.0
    }
  ],
  aggregated_results={
    'accuracy': 0.5,
    'num_correct': 2.0,
    'num_total': 4
  })
}

Benchmark Evals Flow
datasets_impl = eval_settings["datasets_impl"]
await register_dataset(
    datasets_impl,
    include_generated_answer=False,
    dataset_id="test_dataset_for_eval",
)

response = await datasets_impl.list_datasets()

eval_impl = eval_settings["eval_impl"]
response = await eval_impl.evaluate_batch(
    dataset_id=response[0].identifier,
    candidate=ModelCandidate(
        model="Llama3.1-8B-Instruct",
        sampling_params=SamplingParams(),
    ),
    scoring_functions=["inclusion"],
)
assert response.job_id == "0"
job_status = await eval_impl.job_status(response.job_id)
assert job_status.value == "completed"

eval_response = await eval_impl.job_result(response.job_id)

Example EvalResponse

generations=[
  {
    'generated_answer': 'The capital of France is Paris.'
  },
  {
    'generated_answer': "Mark Zuckerberg is the Meta CEO, however it appears he is currently on leave.  On October 25, 2022, Mark Zuckerberg announced that he would be taking a break from being the chair and CEO of the company's board. He is on leave from the board of directors and day-to-day operations as CEO of the firm he co-founded until at least the end of the year."
  },
  {
    'generated_answer': 'The largest planet in our solar system is Jupiter. It is a gas giant planet and is the fifth planet from the Sun. Jupiter has a diameter of approximately 142,984 kilometers (88,846 miles) making it more than 11 times the diameter of Earth.'
  },
  {
    'generated_answer': 'The smallest country in the world, both in terms of population and land area, is the Vatican City, an independent city-state located within Rome, Italy. \n\n1. Population: The Vatican City has a population of approximately 825 people (as of 2021). This includes officials of the Holy See, such as cardinals, bishops, priests, nuns, soldiers of the Swiss Guard, and a few lay workers. \n\n2. Land Area: The Vatican City covers an area of about 0.44 km² (0.17 sq mi). This is roughly the size of a golf course and makes it the smallest internationally recognized sovereign state in the world.'
  },
  {
    'generated_answer': 'The currency of Japan is the Japanese Yen (JPY).'
  }
]scores={
  'inclusion': ScoringResult(score_rows=[
    {
      'score': 1.0
    },
    {
      'score': 1.0
    },
    {
      'score': 1.0
    },
    {
      'score': 1.0
    },
    {
      'score': 1.0
    }
  ],
  aggregated_results={
    'accuracy': 1.0,
    'num_correct': 5.0,
    'num_total': 5
  })
}

Can you include how the client cli can be used to run app evals vs benchmark evals?

@raghotham  Updated in PR summary.

repo: meta-llama/llama-stack


text:
molecule (v2) with ansible arguments

Hello,
I would like to know if there is a command wich enables me to launch molecule command with ansible arguments.
i saw the official documentation but i didn't found that.
Thank you for your help.

Not sure what you're referring to.

I want to run molecule tests on ansible roles,
knowing that my roles are done with parameters,
so, I want to specify these parameters when launching the command "molecule test".

repo: metacloud/molecule


text:
How to use nextPenSlugs with HTML divs

Hi Dave
I'm struggling to get this to work with HTML divs in a template
Here's my live PEN
I can't work out how to change the following code so it works with my HTML divs.
var nextPenSlugs = [
  '3d9a3b8092ebcf9bc4a72672b81df1ac',
  '2cde50c59ea73c47aec5bd26343ce287',
  'd83110c5f71ea23ba5800b6b1a4a95c4',
];

How do i use <div class="item"> in replace of a slug in the above section of code as this template has no pagination?
I tried this but it doesn't work
path: function() {

    var pageNumber = ( this.loadCount + 3 );

    return '?page=';

    },

I need it to load 3 HTML items on button click each time not next page slugs. Thanks

Hello! Thanks for reporting this issue.
The CodePen is a hack as it loads content from different URLs, rather than a typical site which would load content from page2, page3, page4, etc.
Can you describe what content you would like to load?

Just images as seen in the HTML, 1 row at a time

Okay. Where do the images URLs come from?

Custom fields

Sorry, I don't understand what you're trying to achieve. Let me reset.
If you are loading content based on text or fields that you already have available, you may not need to use Infinite Scroll's loading behavior. You can just use it for scrollThreshold event and add content with custom code. recommend looking through

I'd like to add a load more button so only 3 items display on page load and then more on button click.

Ah, in that case, if you want to change path to return a different URL, and that URL is contained within the loaded page HTML, you can do so. See https://github.com/metafizzy/infinite-scroll/issues/688#issuecomment-316403688
var nextURL;

function updateNextURL( doc ) {
  nextURL = $( doc ).find('.pagination__next').attr('href');
}
// get initial nextURL
updateNextURL( document );

// init Infinite Scroll
var $container = $('.container').infiniteScroll({
  // use function to set custom URLs
  path: function() {
    return nextURL;
  },
});

// update nextURL on page load
$container.on( 'load.infiniteScroll', function( event, response ) {
  updateNextURL( response );
});

Thanks, i also need to load 3 items on click.

authors: braddalton desandro samhayman

repo: metafizzy/infinite-scroll


text:
weighted random sampler

This adds a weighted sampler for use with the random select op.

Pull Request Test Coverage Report for Build 1240128885

3 of 3 (100.0%)  changed or added relevant lines in 2 files are covered.
5 unchanged lines in 2 files lost coverage.
Overall coverage increased (+0.2%) to 83.025%

Files with Coverage Reduction
New Missed Lines
%

mlir_graphblas/engine.py
1
75.1%

mlir_graphblas/tests/test_profile.py
4
51.95%

Totals

Change from base Build 1238878930:
0.2%

Covered Lines:
2607

Relevant Lines:
3140

💛  - Coveralls

Pull Request Test Coverage Report for Build 1240128885
Warning: This coverage report may be inaccurate.
This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

3 of 3 (100.0%)  changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.3%) to 83.185%

Totals

Change from base Build 1238878930:
0.3%

Covered Lines:
2612

Relevant Lines:
3140

💛  - Coveralls

authors: coveralls seibert

repo: metagraph-dev/mlir-graphblas