text:
2.1.1 - unable to copy token when creating API token

Steps to reproduce:

Install influxdb 2.1.1
quick start
create API token

Expected behavior:
After creating the token a modal box appears with the token in, to copy to clipboard
Actual behavior:
No modal appears, but the token is added to the token list
A message box saying "API token was created successfully" appears top right
Environment info:

InfluxDB version: 2.1.1

This error means I am unable to do almost anything from the CLI tool, as I cannot attach a token to a request

Hi,
same here, Docker image influxdb:2.1-alpine

click on "Generate API Token"
select "All Access API Token"
add a description
click "Save"
Token is not shown

There is a workaround:

open the developer console (F12)
select the network tab
create a token
you will find your token in the response:

the issue is still there, v2.3.0

Just got bit by this after finally deciding to upgrade from 1.8
Cant believe it still exists, thx @sgramsch for the work around

The existence of this bug is a truly ridiculous waste of time.

This issue is still present in the most recent version (2.6.1).

Closing this issue as resolved.
On versions prior to 2.5, after creating a token, the user can click on the token name to see the token and copy from there.
On versions 2.5+, the token will be show one time upon creation.
These are the expected behaviors.

authors: JustANewGuy KillingJacky MattBrauer aaronmarcus jeffreyssmith2nd sbachstein sgramsch


text:
How many metrics we are collecting?

i have one influxdb instance. the version is 0.12.2. and i am wonder how many metrics do i collect per second ?
Currently i use telegraf for gathering statistics.
And i have several candidates for name "metric per second". that is
pointReqLocal  under influxdb_write
writePointsOk under influxdb_shard
both of them according to count from servers showing different but incorrect value.

Sorry that it's taken so long to respond to this, but questions like this should go to the mailing list where they will get a more prompt response. GitHub is reserved for feature requests and bug reports.

source: gharchive/issue

authors: freeseacher jsternberg

repo: influxdata/influxdb

license_type: permissive

license_source: github-api


text:
continuous query not work if setting time range in where !

System info:
Amazon AWS Linux InfluxDB 1.2.0

datas

> select * from raw
name: raw
time                aid cid mt    value
----                --- --- --    -----
1487024460000000000 c   a   event 12
1487161260000000000 c   a   image 20
1487233199000000000 c   a   event 1
1487233199000000000 c   a   text  1
1487233199000000000 b   a   text  10

create the continuous queries

CREATE CONTINUOUS QUERY hours_raply2 ON t4 BEGIN SELECT sum(value) AS value INTO t4.autogen.hours_2 FROM t4.autogen.raw WHERE time > now() - 2d AND time <= now() - 1h GROUP BY time(1h), cid, aid, mt END

contioues queries log :

[I] 2017-02-16T10:00:00Z finished continuous query hours_raply2 (2017-02-16 09:00:00 +0000 UTC to 2017-02-16 10:00:00 +0000 UTC) in 404.387µs service=continuous_querier

question

not any data in hours_2.but the query

SELECT sum(value) AS value INTO t4.autogen.hours_2 FROM t4.autogen.raw WHERE time > now() - 2d AND time <= now() - 1h GROUP BY time(1h), cid, aid, mt

its working !!!!

Why do we need to calculate the data of longer before ？

In real case , we report the raw data is usually delayed (such as network failure, equipment failure, etc.), over a period of time after all data will be reported.
====================================
i want calculate something delay sometime(such as 1d,2d,6h,etc.),and we have another options ?

@imiskolee There's a bit of an issue with your CQ. Try changing it to
CREATE CONTINUOUS QUERY hours_raply2 ON t4 BEGIN SELECT sum(value) AS value INTO t4.autogen.hours_2 FROM t4.autogen.raw GROUP BY time(1h), cid, aid, mt END

The issue with the CQ as you have it is
WHERE time > now() - 2d AND time <= now() - 1h

CQs do not require that time ranges be specified.

Hello @desa,
sry for necrobumping this issue.
I've got the same issue as @imiskolee (running influxdb 1.6.2) but I need the where time clause in my CQs, so just dropping the clause would be a problem. If I run the query by hand it works as expected:
SELECT mean(value) AS value INTO "test"."agg_1h".:MEASUREMENT FROM "test"."autogen"./.*/ WHERE time < now() -2w GROUP BY time(1h),*
But if I run it as this CQ:
CREATE CONTINUOUS QUERY "cq_agg_1h" ON "test" BEGIN SELECT mean(value) AS value INTO "test"."agg_1h".:MEASUREMENT FROM "test"."autogen"./.*/ WHERE time < now() -2w GROUP BY time(1h),* END
It starts aggregating data which is only an hour old and not which is older than 2 weeks.
My question is, is my CQ worng or is it just no possible to use time clauses in CQs? If the latter applys, are there plans for this feature in CQs in the future?
Thanks a lot for your time & help!

authors: desa imiskolee zxyz


text:
fix: managed function has /api/v2 prefix

The managed functions has to have /api/v2. Try to:
curl -v -X GET 'https://us-west1-1.gcp.cloud2.influxdata.com/api/v2/functions'  --header Authorization <some_token> -H 'accept: application/json'

vs
curl -v -X GET 'https://us-west1-1.gcp.cloud2.influxdata.com/functions'  --header Authorization <some_token> -H 'accept: application/json'

This PR is no longer need due the https://github.com/influxdata/openapi/pull/213.

source: gharchive/pull-request

repo: influxdata/openapi

license: MIT


text:
SOIMP changes

Customer Order Num (field #29) should be 30 chars, not 10
Address details - now 'all or nothing' - if all address fields are blank use NAM, otherwise use what is in file.

Description should be 60 characters, not 20

Customer Order Num = 30
(confirmed 30/05/2017)

Fixed 03/07/2017

license_source: bigquery


text:
jQuery Version Issue

After updating the jQuery version we're seeing odd behavior from our apps. When you go to any of the new apps, for a second or two, you can see all the unrendered templates that we have used. This is more visible in recordedit.

Following the discussion about the version of dependencies that we're using I spent a little time looking at our dependencies. The following are our js dependencies:
Angular-ui Bootstrap
At first I though angular-ui is using bootstrapjs and therefore jQuery. But it seems like it doesn't require bootstrapjs/jQuery at all. Also angular-ui is feature-complete and the version that we have included is the latest. That dependency won't be updated and we don't need to worry about it.
plotly.js
Doesn't rely on jQuery and can be ignored from this discussion.
Bootstrap
bootstrap itself doesn't rely on jQuery and it's the js component that needs jQuery. We should make sure the jQuery version that we're using is compatible with our bootstrapjs version.
Chaise Code And Test cases
Currently we're using jQuery selectors in all of our apps. The use of jQuery is minimal and it's mainly just the jQuery selector itself (and not any of its dependencies that might have been deprecated with the new version). So the jQuery update shouldn't affect any of them. Although the fact that we saw weird behavior from recordedit shows otherwise.
AngularJS
AngularJS doesn't rely on jQuery but if it's present in the app, it will use it. Based on AngularJS documentation "jQuery 1.7 and newer might work correctly with AngularJS but we don't guarantee that.".  So if we really want to take care of "updating" our dependencies and realize that the newest version of jQuery won't actually work correctly with the new version of Angularjs, we might want to completely drop jQuery from our project.

We had a discussion about this a while ago. We decided that we can do any of these three different possible options to resolve this. Since we didn't document it at a time and I wanted to make sure that is documented here, I'm going to add them here.
Option 1: Minimal required task to get the login/navbar apps going (so they can be integrated with deriva web apps):

[ ] Remove jQuery dependency from login/navbar apps and all the directives that they are using.
[ ] Remove jQuery from login and navbar app (chaise should stick to the current version that is using).
[ ] Add ng-jq directive to login/navbar to make sure they are not using any jQuery.

Option 2: Upgrade Chaise jQuery

[ ] Find jQuery version that is compatible with bootstrapjs
[ ] Make sure non of chaise angularjs apps are using jQuery (add ng-jq)
[ ] Make sure the jQuery usage in the code and test are compatible with the new version.

Option 3 (preferred): Remove jQuery from Chaise

[ ] Make sure non of chaise angularjs apps are using jQuery (add ng-jq)
[ ] Remove bootstrapjs and change any where in the code that is using it directly to use angular-ui
[ ] Rewrite chaise code and tests to avoid using jQuery.

What We Have Done
The following are the few experiments that we have done to tackle this issue:

Upgrading jQuery and using the ng-jq in boolean-search:

@Divyata-Singh please complete this section.

Removing jQuery injection from login and navbar, trying the boolean-search:

@Divyata-Singh
Future Plan
Based on these experiments and what we learned from them, the following are the steps that we should follow:

[ ] Remove jQuery usage from navbar/login apps and directives completely. To test this, you can use the standalone navbar/login apps.

Create a simple static page that doesn't include jQuery by itself.
Include the navbar/login app in your static page.
Remove the included jQuery from navbar.app.js/login.app.js
Make sure they work properly (By modifying the chaise code).

[ ] Do some more experiments.

[ ] Make sure we can use jQuery selectors in protractor environment without including jQuery in the page.
[ ] Figure out why ng-jq didn't work. When we upgraded the jQuery, the forms where still broken even with the ng-jq. We should figure out what was the issue exactly? Was the ng-jq ignored or it was broken because of some other issues.
[ ] Upgrade the included jQuery in boolean-search and figure out why the un-rendered forms are displayed before the page completely loads.
[ ] Use CSS to hide the elements before load in boolean-search. This is just to make sure master branch is stable. We should tag the current code so we can get back to it later (if we decided to upgrade the jQuery).

The next step after these experiments could be either "upgrading jQuery" or "removing jQuery". This relies on the result of the experiments.

Update
The jQuery usage from navbar/login apps has been removed [PR].(https://github.com/informatics-isi-edu/chaise/pull/1803).
This function has been removed from navbar and login as the navbar is assumed to be primarily at the top of the webpage. In general to test the usage of the same, the dropdown of Require DOI? on this page will produce the behavior based on the size of the viewport.

Old Code with the issue
The old code which uses jquery3 has the  issue. To replicate it,

Checkout this commit to get the chaise app.
Checkout this commit to get the compatible ermrestjs.
The reason for the issue is that jquery3.3.1 uses timeout function for the initial call to setup, whereas jquery1.11.1 does not.
The following line of code in jquery3.3.1 was changed from :

t?c():(w.Deferred.getStackHook&&(c.stackTrace=w.Deferred.getStackHook()),e.setTimeout(c))

to

t?c():(w.Deferred.getStackHook&&(c.stackTrace=w.Deferred.getStackHook()),c())

By doing so, the un-rendered forms are not seen.
Usage of ng-jq:
Adding of tag ng-jq on the html tag forces the angular app to use jqLite instead of using the available jquery.
This is verified by adding the ng-jq tag on html tag and the page is not loading as  with the following error:

[jqLite:nosel] http://errors.angularjs.org/1.6.9/jqLite/nosel

This means that "Looking up elements via selectors is not supported by jqLite". More information the issue.

As mentioned in the comment https://github.com/informatics-isi-edu/chaise/issues/1637#issuecomment-506554324 about updating the jquery to 3.4.1
There are couple of bugs found while doing the local testing which are mentioned in the PR.

@RFSH @vipul-21 Are we done with this issue? If not, what are the remaining tasks to finish?

The only remaining task was upgrading jQuery version of deriva-webbapps which seems to be done. I am going to close this issue.

authors: RFSH hongsudt vipul-21

repo: informatics-isi-edu/chaise


text:
Add edit link to record page (#402)

The link redirects user to the recordedit (data entry) app.
The redirect url is determined via ".../data-entry" + window.location.hash.
The Edit link only appears if the catalog ACL for content_write_user has an * or matches one of the user's session attribute IDs.

To Test
Visit a Record page in my user dir, such as this record from Yang Chai. Next to the Permalink link, click the "Edit Record" link. You should be redirected to the recordedit (data entry) app where the input fields are pre-filled with this record's values.
The code can be pulled down from branch record-edit-btn.
Normally I wouldn't have worked on a separate branch for this relatively small enhancement, but I thought it would take more work than was actually necessary when I started. Well now that we have the ACLs, this is no longer the case!

Shouldn't the redirect be using a relative URL or otherwise figuring out the entire path?  This hard-coded absolute path means you cannot have several Chaise deployments and keep them separate from each other.

@karlcz That's true, I didn't think about the possibility of multiple Chaise deployments. The redirect url is now created via ".../data-entry" + window.location.hash, based on the assumption that a Chaise app lives on same level as other apps (i.e. /chaise/data-entry|recordset|record ...).
I also switched to this relative url in the recordedit/data-entry app, which redirects to record or recordset upon form submission.

I tested this and looked at the code and it makes sense. However, I think @robes mentioned wanting to do some ACL check to decide whether to display the edit button.  Do we want that before we merge the PR?

Right @karlcz, this was underspecified... We should check whether the user is in the content_write_user ACL before displaying the Edit button. I think it would add confusion for deployments that are public read-only sites, even to display the Edit button.
@howdyjessie This would be a good opportunity to make a change in ermrestjs because I don't think it exposes the meta resource. Can you talk with @shinyichen about what you need to do and send her a PR with the changes?

Okay I've updated this PR. Now the edit link only appears if the catalog ACL for content_write_user has an * or matches one of the user's session attribute IDs.

I think we decided that this feature should obey the ignore annotation and hide the edit button if the table or schema is set to ignore in contexts of edit and entry.

From today's standup: In addition to the catalog ACL check, the record app must additionally check whether the current table has an ignore annotation with an entry context with an edit sub-context. If so, don't show the edit link. Otherwise, show the link.

I've updated the top of the thread to reflect the additional check for an ignore annotation at the table level in terms of changes and test info.

I verified that this honors the tag:isrd.isi.edu,2016:ignore annotation on the table, but it does not seem to observe the same annotation at the schema level.
According to our specification this annotation should also apply to whole schemas, i.e. every table in the schema. I think this should be further defined to follow the same override characteristics we've said for other annotation tags. The logic should be something like:

Default ignore list is [] i.e. nothing ignored
Schema-level annotation (if any) replaces the ignore list
Table-level annotation (if any) replaces the ignore list

Thus a schema-level annotation could be ["entry"] meaning that most tables in the schema will have no data-entry enabled. But, if one table has its own ignore list ["create"] then that table would expose an edit record button.  Does that make sense?  I can update the annotation spec if this sounds good...?

@karlcz Agh I missed the "or schema" part of your previous comment! My bad.
But yes this hierarachy makes sense to me; it's like CSS, where rules with greater specificity take precedence over those with lower specificity.
So for this issue, when checking whether to display the Edit link, is it the best/quickest way to do that by:

Checking the catalog ACL for write access, and if the authenticated user is allowed write access;
Then checking the table for a relevant ignore annotation, and, if I don't find one;
Then checking the schema for a relevant ignore?

I've pushed the code that also checks for schema for any ignore annotations and updated the top of the thread to reflect this addition.

I'm not sure about the status of this PR. I tried the first link "this record from Yang Chai" and I get the following error after clicking on the edit link from the record page.

@robes Sorry about the red ugliness. I had uploaded code from another branch into my user dir when you were testing. The testing info remains the same. Can you try again?

@karlcz Can you please review this up to the synapse use case?

I tried to test the record-edit-btn branch on my machine and get an error app.js:1290:17  SyntaxError: expected expression, got ')'.  This is using master branch of everything else.

@karlcz It turns out I botched a merge conflict; can you test this branch again?

It doesn't crash anymore but I am not seeing an edit button even though I think the ACL allows it and the chaise config enables it.

Thanks to @howdyjessie who clarified the config option is editRecord not recordEdit and that I had a stray 2016:ignore annotation suppressing the edit link, I can now see the edit record link and see that it is disabled if I set the editRecord config to false.
However, the actual edit page is not showing any existing values when it loads. Every field is blank. At the same time, it doesn't show the + button so I think it knows it is in the edit mode rather than new record mode...?

By the way, this other issue may be a bug in the recordedit app since the URL looks OK to me (with a predicate restricting it to the existing record ID).  So, it might be right to merge this PR and chase that other bug separately...

@karlcz When going to the recordedit page that the edit links directs you to, the console says a couple ermrestjs files are missing (errors/networkerrors.js, errors/validationerrors.js ), which is making a number of error messages clog up the console. I checked that those two files are in ermrestjs' master, so could you re-pull it and see if that changes anything? (I'd do it myself but I see unstaged changes in the folder and I don't want to mess up anything.)

authors: hongsudt howdyjessie karlcz robes


text:
Add support for Google disk snapshots

Terraform resource: google_compute_snapshot, the resource is already in the codebase, just returns nil as it's not been implemented.
We can add one cost component "Name: Storage" with "Units: GB-months". The following Pricing API filters should be used. The storage_gb infracost-usage.yml file parameter should be added/used too. The google_compute_disk_resource_policy_attachment resource looks like it's free, we can add it to the free resource list as part of this ticket too.
    "productFilter": {
      "service": "Compute Engine",
      "productFamily": "Storage",
      "region": "us-east1",
      "attributeFilters": [
				{
          "key": "description",
          "value": "Storage PD Snapshot"
        }
      ]
    }
  }

Pricing details: see here, "Snapshot storage" is the relevant price. Cloud Storage already includes the network transfer prices so we don't need to include those in the snapshot resource.
I can't find the pricing filters for "Multi-regional snapshot storage" using the following, @aliscott any ideas where it might be hiding?
db.products.distinct("attributes.description", {"vendorName": "gcp", "region": "us-east1", "service": "Compute Engine"})

db.products.distinct("attributes.description", {"vendorName": "gcp", "region": "us-east1", "service": "Compute Engine", "productFamily": "Storage"})

db.products.distinct("attributes.description", {"vendorName": "gcp", "region": "us-east1", "service": "Compute Engine", "productFamily":"Compute"})

@alikhajeh1 hey, I want to work on it!

Released in v0.8.3

repo: infracost/infracost


text:
Don't warn about missing usage file unless usage syncing enabled

If you run the infracost breakdown command and set the log-level to warn, a warning is output about the usage file not being specified, even if the --sync-usage-file flag hasn't been specified.
Here's an example:
$ infracost breakdown --path plan.json --format=json --log-level=warn > infracost.json
time="2021-06-25T16:38:15+01:00" level=warning msg="Can't sync usage file as it's not specified"

I'd only expect this if the --sync-usage-file flag was specified, for example:
infracost breakdown --path spacelift.json --format=json --log-level=warn --sync-usage-file

This is with Infracost 0.9.1.

@alikhajeh1 here's a PR if you want it: https://github.com/infracost/infracost/pull/833
I've left a bit of a question in the description because it looks to me like the warning could just be removed, but I didn't wanna take that step without checking first.

Thanks @adamconnelly! This was released in v0.9.2 today.


text:
🛑 Sitio WEB Unimonserrate is down

In c012047, Sitio WEB Unimonserrate (https://www.unimonserrate.edu.co/) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Sitio WEB Unimonserrate is back up in 51c28bb after 12 minutes.

repo: infraestructuraidt/status


text:
🛑 Revistas y publicaciones is down

In b090b50, Revistas y publicaciones (https://revistas.unimonserrate.edu.co/) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Revistas y publicaciones is back up in 7f201c2 after 26 minutes.


text:
Please invite me to the community

Name
libraiger
Additional Context
I bring a wealth of experience in the realm of AI and web technologies, accumulated over several years of dedicated involvement. My expertise shines prominently in AI endeavors, covering a spectrum that includes image processing, NLP applications, chatbots, and time series forecasting, and more.
Over the last 7 years, I've fine-tuned my skills in Full Stack Development, navigating through frontend frameworks like React, Next.js, TypeScript, and backend technologies such as Node.js, Java, and GoLang, Python, among others. What distinguishes me is my advanced proficiency in no-code, low-code development in AI like CrewAI, n8n, YouAI, Chainlit.
Beyond the realms of coding and development, my competencies extend to cloud computing, hosting, and cybersecurity – pivotal components for delivering robust, well-structured, and secure projects.
Please check my portfolio and hope you will consider my proposal.
https://westchain.org
I am enthusiastic about embracing fresh challenges and would be thrilled to elaborate further on how I can contribute value to your upcoming projects.
Let's connect and delve into discussions on how we can collaboratively turn ideas into outstanding results.
Looking forward to hearing from you.
Best Regards

Invitation sent to join the GitHub Organization. Welcome to Infraform 🎉Please check your mail to accept the invitation sent by GitHub! 🤖About usInfraform is a remarkable open-source solution tailored for serverless platforms, aimed at empowering creators to craft their infrastructure solutions. 💻 We're thrilled to welcome you into our vibrant community! ✨ We eagerly anticipate your valuable contributions! 🤓 Don't forget to explore our community on Github and connect with us on LinkedIn 💜 Feel free to spread the word about our community to like-minded individuals interested in development, DevOps, technology, and collaborative contributions. 👉💜👈

authors: devenes libraiger

repo: infraform/contribute


text:
Improve the Kubernetes Api Functionality

Our service discovery classes are badly implemented, refactoring a business dsl is required

A single common library for service/pod discovery for akka cluster, remote interactions, remote event listeners

repo: ing-bank/baker

license: mit


text:
Fix type of "ref" property on custom component

Description:
The type of "ref" property on the class component always be a reference to an instance of a class component. But in react-pixi, it's a Pixi instance, so should set it to a function component and specify the type of ref property.
Related issue (if exists):

Hi @vimcaw,
The ref type are already handled correctly for all components, the Stage however should result in a class instance to the Stage itself.
I’ll close this PR, thanks for collab!

It's correct in preset component (Stage, Graphics, Sprite, etc), but not in a custom component.
Look at this example: https://codesandbox.io/s/react-pixi-ref-type-error-on-custom-component-5fwot?file=/src/App.tsx
Custom component:

Preset component：

That's because Preset Components use React.FC to define ref type:
https://github.com/inlet/react-pixi/blob/c049ca6c52bcc765add26c88cf18eec0fca93aac/index.d.ts#L310-L320
export const Text: React.FC<_ReactPixi.IText>;
export const Sprite: React.FC<_ReactPixi.ISprite>;
export const Container: React.FC<_ReactPixi.IContainer>;
export const Graphics: React.FC<_ReactPixi.IGraphics>;
export const BitmapText: React.FC<_ReactPixi.IBitmapText>;
export const NineSlicePlane: React.FC<_ReactPixi.INineSlicePlane>;
export const ParticleContainer: React.FC<_ReactPixi.IParticleContainer>;
export const TilingSprite: React.FC<_ReactPixi.ITilingSprite>;
export const SimpleRope: React.FC<_ReactPixi.ISimpleRope>;
export const SimpleMesh: React.FC<_ReactPixi.ISimpleMesh>;
export const AnimatedSprite: React.FC<_ReactPixi.IAnimatedSprite>;

https://github.com/inlet/react-pixi/blob/master/index.d.ts#L139-L161
  type Container<T extends PIXI.DisplayObject, U = {}> = Partial<
    Omit<T, 'children' | P | ReadonlyKeys<T> | keyof U> &
    WithPointLike<P>
    > & U & InteractionEvents & { ref?: React.Ref<T> };

  type IContainer = Container<PIXI.Container>;
  type ISprite = Container<PIXI.Sprite, WithSource>;
  type IText = Container<PIXI.Text, WithSource>;
  type IGraphics = Container<PIXI.Graphics, {
    /**
     * Draw a graphic with imperative callback.
     *
     * @param {PIXI.Graphics} graphics - The graphics instance to draw on
     * @example
     *
     * draw={g => {
     *   g.beginFill(0xff0000);
     *   g.drawRect(0,0,100,100);
     *   g.endFill();
     * }}
     */
    draw?(graphics: PIXI.Graphics): void;
  }>;

Oops, I closed this PR too soon.. yes you’re right, ref is missing for custom components. Thanks for letting me know..
I’ll review this PR as soon as I’ve time

authors: inlet vimcaw

repo: inlet/react-pixi


text:
Output file not found: result.avi

I'm running the first test using a command line:
 nexrender-cli --file example.json --workpath 'C:\CODING\GSheet2AE\gsheets2ae\WS\'
and I'm getting the following error:
[dJCCVSifvBDFHkBPMUQn2] setting up job...                                                                                [dJCCVSifvBDFHkBPMUQn2] working directory is: C:\CODING\GSheet2AE\gsheets2ae\WS\dJCCVSifvBDFHkBPMUQn2                                                  [dJCCVSifvBDFHkBPMUQn2] applying predownload actions...                                                                                [dJCCVSifvBDFHkBPMUQn2] downloading assets...                                                                                [dJCCVSifvBDFHkBPMUQn2] applying postdownload actions...                                                                                [dJCCVSifvBDFHkBPMUQn2] applying prerender actions...                                                                                [dJCCVSifvBDFHkBPMUQn2] running script assemble...                                                                                [dJCCVSifvBDFHkBPMUQn2] rendering job...                                                                                [dJCCVSifvBDFHkBPMUQn2] rendering took ~9.594 sec.                                                                                [dJCCVSifvBDFHkBPMUQn2] writing aerender job log to: C:\CODING\GSheet2AE\gsheets2ae\WS\aerender-dJCCVSifvBDFHkBPMUQn2.log                              [dJCCVSifvBDFHkBPMUQn2] cleaning up...                                                                                > job rendering failed                                                                                Error: Output file not found: C:\CODING\GSheet2AE\gsheets2ae\WS\dJCCVSifvBDFHkBPMUQn2\result.avi                                                           at ChildProcess.<anonymous> (C:\nodejs\nvm\v14.21.2\node_modules\@nexrender\cli\node_modules\@nexrender\core\src\tasks\render.js:223:31)               at ChildProcess.emit (events.js:400:28)                                                                                at maybeClose (internal/child_process.js:1088:16)                                                                                at Socket.<anonymous> (internal/child_process.js:446:11)                                                                                at Socket.emit (events.js:400:28)                                                                                at Pipe.<anonymous> (net.js:686:12)                                                                                [dJCCVSifvBDFHkBPMUQn2] Temporary AfterEffects project deleted. If you want to inspect it for debugging, use "--skip-cleanup"
What shall I do? There's no reference to a "result.avi" anywhere. Shall I specify an output path somewhere?

Usually when it says result.avi not found it means that AE was not able to complete the render successfully. Do check that your composition is able to be rendered from AE itself and sharing the job JSON file will help us debug your problem easier.

authors: APZmedia limzykenneth

repo: inlife/nexrender


text:
Move DB-related functions and classes to inmanta.db.util module (#5397)

Pull request opened by the merge tool on behalf of #5397

Merged in 6e7e51da8c8683dc0d7bdb47879c9115cf570f2c

repo: inmanta/inmanta-core


text:
Issue/892 vs cde support workspace adr

Description
Add ADR
Self Check:
Strike through any lines that are not applicable (~~line~~) then check the box

[ ] Attached issue to pull request
[x] Changelog entry
[ ] Type annotations are present
[ ] Code is clear and sufficiently documented
[ ] No (preventable) type errors (check using make mypy or make mypy-diff)
[ ] Sufficient test cases (reproduces the bug/tests the requested feature)
[ ] Correct, in line with design
[ ] End user documentation is included or an issue is created for end-user documentation (add ref to issue here: )
[ ] If this PR fixes a race condition in the test suite, also push the fix to the relevant stable branche(s) (see test-fixes for more info)

As discussed, this adr should go into the vs-code extension repo. This adr has been added as a part of this PR


text:
5038 highlight rows tables (#5293)

Pull request opened by the merge tool on behalf of #5293

Merged in 175409dbedb6c0984a610296a659ef05cca6da4a

repo: inmanta/web-console


text:
🛑 wangpan is down

In 56513eb, wangpan (https://husky-twisty-lightyear.glitch.me/onedrive) was down:

HTTP code: 504
Response time: 16671 ms

Resolved: wangpan is back up in 807c651 after 12 minutes.

repo: inotiawu/upptimes


text:
feat: Implement EventStream using mio

This branch changes the implementation of Stream for EventStream
to use mio and tokio_reactor to register interest in events on
the Inotify file descriptor with the reactor, rather than using
task::current().notify() to unpark the task in the executor when
polling the fd would block.
Since the reactor is intended to drive all IO resources (such as the
inotify FD) and to be the bridge between tasks in the executor and
events from the OS, using PollEvented is a more correct way to
implement the stream.
This branch shouldn't result in any noticeable changes in behaviour,
but will hopefully allow tokio to manage Inotify's IO events more
intelligently.

Thanks for the PR! I'll take a closer look once #104 has been merged.

Now that #104 is merged, this PR is no longer blocked.
@hawkw I'm a bit clueless about mio/futures/tokio, so I'm not too confident in my ability to provide a good review. That said, I can't find anything wrong with this PR.
There's one thing that's keeping me from merging this right now: There's an open PR (#101) that is a follow-up to the initial futures work (#99). It was blocked back then (may still be?), so I couldn't merge right away. It's my impression that your PR supersedes #101. Is that correct?
@mathstuf This PR seems to supersede your open PR #101. Do you have any thoughts on this PR? I'm considering merging this PR and closing #101. Do you see any reason not to do that? Please note that this is based on a previous PR, and the first 4 commits are already merged. As of this writing, only fa6bf8a and 2c63299 are relevant.

First, just FYI, I've rebased this branch onto master, so the diff should no longer include diffs from PR #104.
@hannobraun

It's my impression that your PR supersedes #101. Is that correct?

Yeah, that's correct. I haven't looked too closely at #101, but here are some of the important differences from my perspective:

It looks like #101 uses the old version of tokio (i.e. the tokio-core and tokio-io crates), while this branch uses the new tokio crate. These are incompatible, and IMO since a majority of the ecosystem has migrated to the new tokio, it's better to use that.
On master, Inotify::event_stream method returns a concrete EventStream type. On #101, this changes to impl Stream<...>, while on this branch, we still return a named EventStream type. Personally, I think that while impl Trait is nice to use in applications or in internal code that's not part of a library's public API, it's not ideal for library methods to return an impl Trait, as it limits the way users of the library can use that returned type. For example, if I wanted to put the stream returned by Inotify::event_stream in a struct field, or use it as a type parameter to some other type, I'd have to box it. This could be a potentially breaking change.
#101 depends on the tokio_file_unixcrate, while my branch doesn't. I'm not familiar with that crate, and this isn't a value judgement, just a difference I noticed.

Thank you for the explanation, @hawkw, this is great! I'll give @mathstuf a few days to respond before making a decision, but so far everything looks great.

For what it's worth: we just found that switching to this branch from master fixed an issue where inotify-rs was causing our application to pin CPU use at 100%.

In particular, this line is what causes this stream to burn CPU, since the executor can't leave this stream as parked (since the task is always notified).

@olix0r Thank you for the info. If I remember correctly, we decided to land a preliminary version of futures support, as the more efficient version was blocked for various reasons.
Looking back, this seems like a mistake on my part. We should have documented this extensively and hidden it behind a Cargo feature, at the very least. I'm sorry for the trouble this caused you!

@hannobraun No worries! Luckily we caught this in testing ;)
Thanks for putting together this very useful library.

@hannobraun I think releasing the futures support experimentally was a good choice on your part, given that otherwise, we wouldn't have found this issue at all.
Honestly, I'm mostly just surprised nobody else hit this issue earlier, as I'm fairly sure could occur any time the EventStream was used (rather than being triggered by some particular uncommon condition). I guess nobody was using the futures support seriously enough to care about its performance until us.
By the way, thanks for being so willing to work with me on landing all the patches I've submitted, I really appreciate it!

@olix0r @hawkw Thank you both for your kind words!

I guess nobody was using the futures support seriously enough to care about its performance until us.

Yeah, probably. Judging from the downloads numbers on crates.io, this library must see a fair bit of use. Unfortunately, not a lot gets back to me.

By the way, thanks for being so willing to work with me on landing all the patches I've submitted, I really appreciate it!

No problem. Thank you for the patches!

I'm merging this. Sorry for the additional delay.
@hawkw Thank you again for your contributions!

@hannobraun

Sorry for the additional delay.

No problem at all!

@hawkw Thank you again for your contributions!

My pleasure! It was fun. :D Thanks for all your reviews.

@mathstuf This PR seems to supersede your open PR #101. Do you have any thoughts on this PR?

Sorry, was on an extended vacation. Superseding is fine. This does look more idiomatic than my PR. Thanks @hawkw!

authors: hannobraun hawkw mathstuf olix0r

repo: inotify-rs/inotify

license: ISC


text:
[FR] - Layered Docker container image

Internal/External
External
Area
Other - Development Experience & Deployment Runtime
Describe the feature you'd like
As a cardano-node developer, when I'm working on code, I'd want to deploy new Docker-based builds of my changes as quickly as possible to reduce the cycle time between a change and observing that change in a local build. To do so, I want the Docker image that is produced by the build to create multiple layers so the underlying container runtime can have a chance at caching frequently used layers. When I get a cache hit, there's less time involved in building, pulling, and starting up a new image.
Describe alternatives you've considered
None - will investigate and see if there are other approaches.
Additional context / screenshots
This is an optimization of a containerized build, so may be lower in priority if there are other Docker container-related issues in the backlog.
Neat article about optimizing nix builds with Docker layering - https://grahamc.com/blog/nix-and-layered-docker-images.

Closing this. If this is still relevant please reopen.

repo: input-output-hk/cardano-node


text:
[CBR-312] Refined exceptions

Description
This PR refines the type of exceptions that the new wallet API code throws.
Linked issue
https://iohk.myjetbrains.com/youtrack/issue/CBR-312
Type of change

[] 🐞 Bug fix (non-breaking change which fixes an issue)
[] 🛠 New feature (non-breaking change which adds functionality)
[] ⚠ Breaking change (fix or feature that would cause existing functionality to change)
[x] 🏭 Refactoring that does not change existing functionality but does improve things like code readability, structure etc
[ ] 🔨 New or improved tests for existing code
[] ⛑ git-flow chore (backport, hotfix, etc)

Developer checklist

[ ] I have read the style guide document, and my code follows the code style of this project.
[ ] If my code deals with exceptions, it follows the guidelines.
[ ] I have updated any documentation accordingly, if needed. Documentation changes can be reflected in opening a PR on cardanodocs.com, amending the inline Haddock comments, any relevant README file or one of the document listed in the docs directory.

Testing checklist

[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.

QA Steps

Screenshots (if available)

Ignoring appveyor failure here, we aren't merging to develop and it's seriously 10 lines of comments in a file -___- ...

authors: KtorZ parsonsmatt

repo: input-output-hk/cardano-sl


text:
[CBR-453] Fix bug in backfilling

Description
ApplyBlock takes an arg Maybe (Set HdAccountId) to allow callers to restrict the accounts.
In the backfilling code we were calling applyBlock with restricted accounts = Just {}
The effect was that the first block in backfilling would not be applied, and all subsequent blocks would fail because of the "missing predecessor block" that was not applied. Since we retry failures, this was causing an infinite loop with backfilling never happening.
This fix:

removes the bug that was creating the empty restrictions Just {}
changes the type of the restrictions in the ApplyBlock  signature to Maybe (NonEmpty HdAccountId) to prevent invalid restrictions
add info log messages for the start and end of Backfilling, add a warning message when we recursively handle errors (so that an infinite loop occurring here is at least visible in the logs)

Linked issue
https://iohk.myjetbrains.com/youtrack/issue/CBR-453
Type of change

[x] 🐞 Bug fix (non-breaking change which fixes an issue)
[X] 🏭 Refactoring that does not change existing functionality but does improve things like code readability, structure etc

QA Steps
This PR fixes backfilling - we can now test that a wallet catches up with a node if it falls behind.

@uroboros will open a separate PR with other improvements, so that we can expedite the bug fix here.

authors: adinapoli-iohk uroboros


text:
Add getNewBlocks to diffusion layer and call it from dedicated wallet worker

Context
See https://github.com/input-output-hk/cardano-wallet/wiki/ApplyingBlocks-Rollback-After-Refactoring-Byron
Decision
We want to add diffusion layer mediated way of getting blocks, at this stage in parallel to the existent mechanism in order to compare results
Acceptance Criteria

[ ] API call, getNewBlocks, must be added to diffusion layer. It may take header and return all blocks on top of it,

[ ] Worker that calls the trusted node periodically for new blocks using diffusion layer must be added to the wallet.

[ ] Respective legacy solution results and the new ones must be compared during integration tests (Trace.Debug like logs may be used)

PR

Number
Base

develop

Consequences

Included in #88

authors: KtorZ paweljakubas

repo: input-output-hk/cardano-wallet


text:
Make sure there is collateral on the wallet before e2e plutus tests

[x] Make sure there is collateral on the wallet before e2e plutus tests (925cc09e1cca3c449381a1ccc3503263137d7e87)

Comments
Sometimes wallets turn out to have no pure ADA utxo and plutus tests fail. This is to make sure there are always some pure ADA utxos.
#maintenance
Issue Number
n/a

bors r+

bors r+

bors r+

bors r+

bors r+

bors r+

bors r+

bors r+

authors: Anviking piotr-iohk


text:
[DDW-326] Restore deleted package-lock.json file

This PR restores deleted package-lock.json file. The file was deleted by mistake and looks like we still need it...

Review Checklist:
Basics

[ ] PR is updated to the most recent version of target branch (and there are no conflicts)
[ ] PR has good description that summarizes all changes and shows some screenshots or animated GIFs of important UI changes
[ ] CHANGELOG entry has been added and is linked to the correct PR on GitHub
[ ] Automated tests: All acceptance tests are passing (npm run test)
[ ] Manual tests (minimum tests should cover newly added feature/fix): App works correctly in development build (npm run dev)
[ ] Manual tests (minimum tests should cover newly added feature/fix): App works correctly in production build (npm run package / CI builds)
[ ] There are no flow errors or warnings (npm run flow:test)
[ ] There are no lint errors or warnings (npm run lint)
[ ] Text changes are proofread and approved (Jane Wild)
[ ] There are no missing translations (running npm run manage:translations produces no changes)
[ ] UI changes look good in all themes (Alexander Rukin)
[ ] Storybook works and no stories are broken (npm run storybook)
[ ] In case of npm dependency changes both package-lock.json and yarn.lock files are updated

Code Quality

[ ] Important parts of the code are properly documented and commented
[ ] Code is properly typed with flow
[ ] React components are split-up enough to avoid unnecessary re-rendering
[ ] Any code that only works in Electron is neatly separated from components

Testing

[ ] New feature / change is covered by acceptance tests
[ ] All existing acceptance tests are still up-to-date
[ ] New feature / change is covered by Daedalus Testing scenario
[ ] All existing Daedalus Testing scenarios are still up-to-date

After Review:

[ ] Merge PR
[ ] Delete source branch
[ ] Move ticket to done on the Youtrack board

why do we need it ?

authors: DominikGuzei nikolaglumac

repo: input-output-hk/daedalus


text:
Base the definitions of bridges, loss servers and duplication servers on distributors

This resolves #130.

In many places you replaced simp by blast. What’s the reason for that?

In many places you replaced simp by blast. What’s the reason for that?

Because simp suddently stopped working, I guess it has to do with the new definitions.

repo: input-output-hk/fm-ouroboros


text:
io-classes-mtl-0.1.2.0 is not buildable on GHC 9.8.2

io-classes-mtl-0.1.2.0 is not buildable on GHC 9.8.2. See the attached error.log.
Taken from: https://github.com/IntersectMBO/cardano-cli/actions/runs/9254721425/job/25457077489?pr=777#step:11:973

This is a known bug which has been fix: https://github.com/input-output-hk/io-sim/pull/163. We'll try to release it asap

Reproduces on GHC 9.6.3.

There was a revision made today, could you please make sure it is fixed now and then close the issue? cc @carbolymer @uhbif19

Testing in https://github.com/IntersectMBO/cardano-node/pull/5869

@bolt12 seems to be working, thanks!

Works for me too.

repo: input-output-hk/io-sim


text:
Use CardanoWsClient in Lace

Checklist

[x] JIRA - <link>
[ ] Proper tests implemented
[ ] Screenshots added.

Proposed solution
Conditionally use the CardanoWsClient on Lace to improve performance.

Closing in favor of #1469

authors: AngelCastilloB iccicci

repo: input-output-hk/lace


text:
Rewinding the PBFT ChainState to an EBB fails

See https://github.com/input-output-hk/ouroboros-network/pull/1284#issuecomment-560976105

We discussed three options today.

Option A. Complicate the PBFT chain state and/or code to track EBBs.
Option B. Complicate the CS client to exclude EBBs from the intersection point candidates.
Option C. Relax the PBFT chain state code to assume that all rewind: rollback to block not previously applied errors are due to rewinding to an EBB, and so shouldn't be treated as errors. (What should happen in that case, rewind to the latest previously applied block prior to that EBB?)

Right now, as of PR 1284, master has a hybrid A/C, where it assumes a rewind to slot 0 can only fail with rewind: rollback to block not previously applied due to the epoch 0 EBB and so falls back to rewinding to Origin. This works because the test infrastructure only ever forges the epoch 0 EBB.
We're leaning towards option C but are hesitant since it could mask other bugs.

In production, "by the time our nodes take over, the EBBs will be much longer than K back" (-Duncan). So I had thought there was another option to just update the test infrastructure to only have "historical" EBBs, by starting the (first) nodes with a non-empty ImmDB. However, "Thomas is right of course that [...] we can lose data and roll back into the old region of the chain." (-Duncan). It took me a while to process that and realize that even if only one node truncates its ImmDb (eg disk failure), the resulting CS interactions may involve MsgIntersectFounds that target EBBs -- and those messages' handlers will incur failing calls to rewind.

Option C. [..] (What should happen in that case, rewind to the latest previously applied block prior to that EBB?)

That's my first thought. Now I'm thinking about the following: say we have the following chain:
slots: 10    20          20 + k
.. ->  A  -> EBB  -> .. -> X

There is no block between A at slot 10 and the EBB at slot 20. If we now need to rewind k (our maximum rollback) slots from X to the EBB (because we found an intersection with the EBB), we should not return the ChainState of the latest previously applied block prior to that EBB, i.e. the ChainState for A, because that ChainState is older than k blocks.
Note that this scenario is very unlikely to happen in practice as there are nearly no empty slots (in the Byron-era that has EBBs).

If we now need to rewind k (our maximum rollback) slots from X to the EBB (because we found an intersection with the EBB), we should not return the ChainState of the latest previously applied block prior to that EBB, i.e. the ChainState for A, because that ChainState is older than k blocks.

Fortunately, I think that we'll do the right thing automatically if we recursively call rewind for the slot (of the block) before the EBB when finding out that there's only an EBB in that slot.

I agree with that assessment. If we have correctly implemented "rewind to the latest previously applied block prior to that EBB", then "rewind to 20" is effectively equivalent to "rewind to 10", and so both of those should fail. (This is what I want and think is happening on PR 1317.)

authors: mrBliss nfrisby

repo: input-output-hk/ouroboros-network


text:
Cardano eras transitions tests

[x] Write tests for all the Cardano transitions.

[x] Byron to Shelley
[x] Shelley to Allegra
[x] Allegra to Mary
[x] Mary to Alonzo
[x] Alonzo to Babbage

[x] Add to the AddingAnEra.md document instructions to add a new era translation test when adding a new era.

If we can generate arbitrary ledger states of each Cardano era we can write property tests. Otherwise, we'll have to resort to unit tests.
Byron to Shelley
Ensure that all the UTxO is transferred from in-memory Byron state (which has no tables) to the ledger tables of the Shelley state. The result must consist only of insertions (which are present in the resulting diffs).
Shelley to Allegra
Ensure that the avvm addresses appear as deletions in the resulting diffs.
All remaining transitions
Ensure that the resulting diff is empty, which means no modification to the ledger tables will be made from one era to the next.

Should we close this issue as completed? @bartfrenk @dnadales

Indeed.

authors: Jasagredo dnadales


text:
better nsswitch file that allows queries to the internet...

The previous nsswitch.conf didn't seem to let cabal update succeed, because I think it couldn't get to the internet.
This one works; I just tested it on Ubuntu; I might try and test it on Windows first (but I'll have to set up the dev environment, etc, etc, or we can just merge this, and I'll test it via the dockerhub image, which is much easier).

Pre-submit checklist:

Branch

[x] Commit sequence broadly makes sense
[x] Key commits have useful messages
[x] Relevant tickets are mentioned in commit messages
[x] Formatting, materialized Nix files, PNG optimization, etc. are updated

PR

[x] Self-reviewed the diff
[x] Useful pull request description
[x] Reviewer requested

Pre-merge checklist:

[ ] Someone approved it
[ ] Commits have useful messages
[ ] Review clarifications made it into the code
[ ] History is moderately tidy; or going to squash-merge

this is not necessary.

repo: input-output-hk/plutus


text:
TypeError: sequence item 0: expected str instance, bytes found

Installed with 'sudo pip3 install gcalcli'. First run generates a token and outputs "Authentication successful', then dumps a stack trace ending with the subject line.

Stack trace:

Authentication successful.
Traceback (most recent call last):
  File "/usr/local/bin/gcalcli", line 11, in
    sys.exit(main())
  File "/usr/local/lib/python3.4/dist-packages/gcalcli/cli.py", line 147, in main
    cal_names=cal_names, printer=printer, **vars(parsed_args)
  File "/usr/local/lib/python3.4/dist-packages/gcalcli/gcal.py", line 66, in __init__
    self._get_cached()
  File "/usr/local/lib/python3.4/dist-packages/gcalcli/gcal.py", line 193, in _get_cached
    self.get_cal_service().calendarList().list()
  File "/usr/local/lib/python3.4/dist-packages/gcalcli/gcal.py", line 157, in get_cal_service
    http=self._google_auth())
  File "/usr/local/lib/python3.4/dist-packages/googleapiclient/_helpers.py", line 130, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/usr/local/lib/python3.4/dist-packages/googleapiclient/discovery.py", line 224, in build
    requested_url, discovery_http, cache_discovery, cache, developerKey)
  File "/usr/local/lib/python3.4/dist-packages/googleapiclient/discovery.py", line 274, in _retrieve_discovery_doc
    resp, content = http.request(actual_url)
  File "/usr/local/lib/python3.4/dist-packages/oauth2client/transport.py", line 175, in new_request
    redirections, connection_type)
  File "/usr/local/lib/python3.4/dist-packages/oauth2client/transport.py", line 282, in request
    connection_type=connection_type)
  File "/usr/lib/python3/dist-packages/httplib2/__init__.py", line 1138, in request
    headers = self._normalize_headers(headers)
  File "/usr/lib/python3/dist-packages/httplib2/__init__.py", line 1106, in _normalize_headers
    return _normalize_headers(headers)
  File "/usr/lib/python3/dist-packages/httplib2/__init__.py", line 194, in _normalize_headers
    return dict([ (key.lower(), NORMALIZE_SPACE.sub(value, ' ').strip())  for (key, value) in headers.items()])
  File "/usr/lib/python3/dist-packages/httplib2/__init__.py", line 194, in
    return dict([ (key.lower(), NORMALIZE_SPACE.sub(value, ' ').strip())  for (key, value) in headers.items()])
TypeError: sequence item 0: expected str instance, bytes found

Try again...

 Authentication successful.
Traceback (most recent call last):
  File "/usr/local/bin/gcalcli", line 11, in
    sys.exit(main())
  File "/usr/local/lib/python3.4/dist-packages/gcalcli/cli.py", line 147, in main
    cal_names=cal_names, printer=printer, **vars(parsed_args)
  File "/usr/local/lib/python3.4/dist-packages/gcalcli/gcal.py", line 66, in __init__
    self._get_cached()
  File "/usr/local/lib/python3.4/dist-packages/gcalcli/gcal.py", line 193, in _get_cached
    self.get_cal_service().calendarList().list()
  File "/usr/local/lib/python3.4/dist-packages/gcalcli/gcal.py", line 157, in get_cal_service
    http=self._google_auth())
  File "/usr/local/lib/python3.4/dist-packages/googleapiclient/_helpers.py", line 130, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/usr/local/lib/python3.4/dist-packages/googleapiclient/discovery.py", line 224, in build
    requested_url, discovery_http, cache_discovery, cache, developerKey)
  File "/usr/local/lib/python3.4/dist-packages/googleapiclient/discovery.py", line 274, in _retrieve_discovery_doc
    resp, content = http.request(actual_url)
  File "/usr/local/lib/python3.4/dist-packages/oauth2client/transport.py", line 175, in new_request
    redirections, connection_type)
  File "/usr/local/lib/python3.4/dist-packages/oauth2client/transport.py", line 282, in request
    connection_type=connection_type)
  File "/usr/lib/python3/dist-packages/httplib2/__init__.py", line 1138, in request
    headers = self._normalize_headers(headers)
  File "/usr/lib/python3/dist-packages/httplib2/__init__.py", line 1106, in _normalize_headers
    return _normalize_headers(headers)
  File "/usr/lib/python3/dist-packages/httplib2/__init__.py", line 194, in _normalize_headers
    return dict([ (key.lower(), NORMALIZE_SPACE.sub(value, ' ').strip())  for (key, value) in headers.items()])
  File "/usr/lib/python3/dist-packages/httplib2/__init__.py", line 194, in
    return dict([ (key.lower(), NORMALIZE_SPACE.sub(value, ' ').strip())  for (key, value) in headers.items()])
TypeError: sequence item 0: expected str instance, bytes found

Linux asdf 3.16.0-8-amd64 #1 SMP Debian 3.16.64-2 (2019-04-01) x86_64 GNU/Linux

python --version
Python 2.7.9

python3 --version
Python 3.4.2

expected str instance, bytes found

That's a classic py2 vs py3 error message.  I wonder if something in setup.py isn't written correctly for your setuptools.  I can install gcalcli with pip3 on my debian system without an error, so I may need some help figuring out how to repro this.

OK, I need to upgrade my distro I guess.

authors: PhilHudson jcrowgey

repo: insanum/gcalcli


text:
Support DLA

Support DLA in the framework

Implemented in #781 #780

repo: insight-platform/Savant


text:
A question regarding multi label classification and others...

First of all: Great tool. I have been playing around with it for a while now and ended up having a lot of questions, thank you already for the effort to answer some of them :)
A question regarding the default choice of "top_k_categorical_accuracy" I think the default value for the keras implementation is k=5 in this case. When doing multi label training, how does it work? Is it based on all labels being in the top 5? What if there are more valid labels?
And furthermore, switching to categorical_accuracy I am even less sure how it works with multi labels - does it test if the top predicted label is among the true labels or does it actually calculate the fraction of correct labels on the first i ranks, i being the number of labels? In which case it would be the better choice for mulit label prediction I guess? What would be the "best" choice for a metric for expected labels between 1 and 5?
Another question: The multi label document, is it seen as a document belonging to both classes? Or is it a new class, formed by the exact combinations of labels (because that would explain my poor accuracy on my val set) I have 500 labels but 8000 unique combinations in 50k data points.
Last but not least: Assuming I want to train on larger documents, is there any experience whether "standard" text mining preprocessing make sense to get rid of stop words etc. that do not contribute to the information in the easily huge input layer (Assuming the documents have 3000+ tokens) Otherwise I easily run out of memory on 250GB machines when creating the model. (Embedding size 300). Interestingly so far the token sample size retained had only an impact on how quick in terms of epochs the net learned towards its plateau - it doesn't change the final accuracy. (I tested from 100 - 2500)
Actually, one more: I sampled the validation set so that it has the same distribution of "single label" occurrence but I did not pay attention to the multi label combinations (way to many unique combinations, as stated above) still in all my runs using a wide spread of parameters my internal accuracy is almost way better than my validation accuracy, 0.7 top-k internally and 0.4 top-k on validation set, is such a discrepancy expected?
Thank you very much, your tool finally made me start looking into keras ;).
Alex

@Tabernakel glad you're using and enjoying Magpie! There's a lot of questions, I'll try to answer them one by one :)
1 and 2) We don't do any fancy logic on top of Keras evaluation metrics, only using them out-of-the-box, so you can look into their code/docs to validate your assumptions. As far as I recall, if there's only one correct label throughout the document and it's in the top K, the value is 100%. If there's more than K correct labels, but all of the top K are correct, you also get 100%. Otherwise your score is a ratio of correct to all labels within the top K ranks. Value of K is arbitrary and dependent on your problem, but it's usually 5. Feel free to call my bluff and check the code!

Not sure what you mean, but rather the second option. Although from the evaluation perspective, if you score half of your labels right it is not the same as if you scored all of them wrong - you'll get a better score.

My experience says that removing stopwords doesn't help. You definitely want to have them for computing word2vec vectors - in theory you could remove them from the training later which perhaps could lead to better results (build w2v vectors on the corpus with stopwords, feed to the net the corpus without them).

300-size embedding sounds really large to me. We didn't see a difference above 70 for scientific paper abstracts. For the token sample size - that is not unlikely if the beginning of the document contains most (all) of the information that a document conveys. For instance it's enough to process an abstract of a scientific paper to figure out whether it's about astronomy, you don't need to read the whole thing.

It's difficult to say, but my intuition would be that it's not due to label distribution in your sets, but your model might just be overfitting. Try reducing dimensionality and maybe adding some regularization layers (you can try to increase the Dropout ratio) and see whether that helps.

In general, I think evaluation metrics are the weakest side of Magpie at the moment. We use vanilla Keras functions for computing multi-label metrics, which can be a bit confusing. At some undefined point in the future, I'll try to write some code and documentation to make it simpler and easier to use :)

Nice, thanks for the answers @Tabernakel !
Have you tried running the code with embeddings of size 200 or 100? I'd be curious about the results.
And TensorBoard support sounds like a very sound idea, we should integrate with it as well. If you have some code ready for that, feel free to open a PR.
Cheers,
Jan

I am also a bit limited here in the way I can interact with github - but it is really simple for what I did, I am calling Magpie.train like this, first I create the callback object and just pass it to the train function:
 mycallbacks=keras.callbacks.TensorBoard(log_dir='/mylog/', histogram_freq=0, batch_size=1000, write_graph=True, write_grads=False, write_images=False, embeddings_freq=0, embeddings_layer_names=None, embeddings_metadata=None) magpie.train('/training', labels, epochs=50,test_dir='/eval', **logger_callbacks=mycallbacks**)
For main.py the changes are:
def train(self, train_dir, vocabulary, test_dir=None, **logger_callbacks=None**,               nn_model=NN_ARCHITECTURE, batch_size=BATCH_SIZE, test_ratio=0.0,               epochs=EPOCHS, verbose=1):
...
 return self.keras_model.fit(             x_train,             y_train,             batch_size=batch_size,             epochs=epochs,             validation_data=test_data,             validation_split=test_ratio,             **callbacks=[logger_callbacks]**,             verbose=verbose,         )
that should be all, firing up tensorboard
 tensorboard --logdir /mylog/
You see exactly what Magpie is doing. Sorry for the crude way of interaction, maybe you can pick it up from here, this is what you will get:

@Tabernakel you seem to be right, especially the categorical_accuracy doesn't make sense for this context. It should be fixed.
Multi-label problems have some standard evaluation metrics such as micro and macro-averaged accuracy, however I don't think that they are great for large problems like the ones you would use Magpie for. If the problem is large enough, the probabilities will be always very low and binarising them rarely makes sense.
When I use Magpie, I use ranking metrics for evaluation. I use the probabilities that the NN outputs for sorting and look for positive labels on the top of the ranking. This approach of looking at the problem is well studied in the information retrieval domain (search engines) and there are many established metrics for that. Here is a post about their differences.
My take is that Magpie should provide standard binary accuracy metrics (micro & macro average) and ranking metrics out-of-the-box as callbacks.

Hi Jan,
I am also using precision/recall and F1 measure for my retrieved labels.  Interestingly, these numbers are much higher than what the categorical accuracy wants me to believe (but thats due to the effect I described above, it is not really meant for multi-labels.)
Did you know that this branch of keras: https://github.com/keras-team/keras/commit/a56b1a55182acf061b1eb2e2c86b48193a0e88f7
Actually did have  recall/precision/Fmeasure as metrics, but they have been taken out with an argument I cannot really follow, the release notes mention it here: https://github.com/keras-team/keras/wiki/Keras-2.0-release-notes
The argument goes
"Basically these are all global metrics that were approximated batch-wise, which is more misleading than helpful. This was mentioned in the docs but it's much cleaner to remove them altogether. It was a mistake to merge them in the first place."
But that argument for me doesn`t make sense actually, any idea what they are referring to?

authors: Tabernakel jstypka

repo: inspirehep/magpie


text:
Issue #27: Add a hough transform implementation ported from opencv

I took a stab at this and it's working for me.

Great thanx for this.
can you also add a demo/example so that people can try it with webcam for example (see jsfeat docs/examples)?

authors: charleslaw inspirit

repo: inspirit/jsfeat


text:
The error ocurred when I run dcnv3/tests/test_dcnv3_module.py and test_grad.py

The error occurred when I run dcnv3/tests/test_dcnv3_module.py
#####################################################
DCNv3(
(dw_conv): Sequential(
(0): Conv2d(4, 4, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=4)
(1): Sequential(
(0): to_channels_last()
(1): LayerNorm((4,), eps=1e-06, elementwise_affine=True)
)
(2): GELU(approximate='none')
)
(offset): Linear(in_features=4, out_features=36, bias=True)
(mask): Linear(in_features=4, out_features=18, bias=True)
(input_proj): Linear(in_features=4, out_features=4, bias=True)
(output_proj): Linear(in_features=4, out_features=4, bias=True)
)
Traceback (most recent call last):
File "D:\project(yolo)\dcnv3\tests\test_dcnv3_module.py", line 17, in
test_dcnv3_module()
File "D:\project(yolo)\dcnv3\tests\test_dcnv3_module.py", line 12, in test_dcnv3_module
output = m(x)
File "D:\anaconda3\envs\yolo\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "D:\anaconda3\envs\yolo\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "D:\anaconda3\envs\yolo\Lib\site-packages\dcnv3\ops_dcnv3.py", line 214, in forward
x = self.output_proj(x)
File "D:\anaconda3\envs\yolo\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "D:\anaconda3\envs\yolo\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "D:\anaconda3\envs\yolo\Lib\site-packages\torch\nn\modules\linear.py", line 114, in forward
return F.linear(input, self.weight, self.bias)
TypeError: linear(): argument 'input' (position 1) must be Tensor, not NoneType
############################################################
Input is NoneType? Why?
The error ocurred when I run dcnv3/tests/test_grad.py
###############################################################
Traceback (most recent call last):
File "D:\project(yolo)\dcnv3\tests\test_grad.py", line 61, in
test_dcnv3()
File "D:\project(yolo)\dcnv3\tests\test_grad.py", line 40, in test_dcnv3
c_output.sum().backward()
AttributeError: 'NoneType' object has no attribute 'sum'
#####################################################
Original code below:
c_output = dcnv3.ops.dcnv3(input, offset, mask,
                           kernel_h, kernel_w,
                           stride_h, stride_w,
                           pad_h, pad_w,
                           dilation_h, dilation_w,
                           group, group_channels,
                           offset_scale,
                           im2col_step,
                           remove_center)
c_output.sum().backward()

c_output is Nonetype? Why?

Sorry, I forgot this repo is mine and almost ignored GitHub's notification 😂.
Honestly, I don't know what could possibly go wrong as it still works on my machine. Did you install the TestPyPI wheel or compile from source? Note that the TestPyPI wheel only works if your configuration matches that of the GitHub Action server where I built them. Even if you installed from source, there are a lot of possibilities.
I just updated the TestPyPI wheel to make it compatible with torch>=2.1.0 and Cuda 12.1 (previously it could run on 1.9.0<=torch<=2.0.1 and Cuda 11.8). When you run the script again if you see this error, then you need to recompile from source:
Detected that PyTorch and Extension were compiled with different CUDA versions. PyTorch has CUDA Version={t_major}.{t_minor} and Extension has CUDA Version={ext_major}.{ext_minor}. Please reinstall the Extension that matches your PyTorch install.


text:
Error occurred on dashboard when first sign in to Board with no collected data.

In GitLab by @liyanq528 on Apr 12, 2018, 10:38

In GitLab by @liyanq528 on Nov 8, 2019, 07:49
fixed by pull request #1181

In GitLab by @wknet123 on Nov 8, 2019, 07:49
Closing it as fixed.

In GitLab by @sokril on Nov 8, 2019, 08:20
closed

In GitLab by @sokril on Nov 8, 2019, 08:20
assigned to @wknet123

In GitLab by @liyanq528 on Nov 11, 2019, 08:52
mentioned in merge request !489

In GitLab by @liyanq528 on Nov 12, 2019, 02:23
mentioned in merge request !876

In GitLab by @liyanq528 on Nov 13, 2019, 10:48
mentioned in merge request !728

repo: inspursoft/board


text:
Jenkins is hung when building images, but no body is building image at this time

In GitLab by @luotian-github on Mar 16, 2018, 16:54
too long time to update the total pod number in dashboard, even the total has already been got in service section.

In GitLab by @luotian-github on Nov 8, 2019, 07:47
will re-test it on the devops version with multiple user repo

In GitLab by @luotian-github on Nov 8, 2019, 07:47
waiting for multiple kvm in DesOps

In GitLab by @wknet123 on Nov 8, 2019, 07:47
Please reproduce it by using current release.

In GitLab by @luotian-github on Nov 8, 2019, 07:47
verified

In GitLab by @sokril on Nov 8, 2019, 08:19
closed

In GitLab by @sokril on Nov 8, 2019, 08:19
assigned to @luotian-github


text:
Combine image selection and container configuration step into one step.

In GitLab by @luotian-github on Jun 12, 2018, 15:35
If you are reporting a problem, please make sure the following information are provided:

Version of board (last git commit number) , docker engine and docker-compose.
Config files of board, you can get them by packaging "board.cfg" and files in the same directory, including subdirectory.
Log files, you can get them by package the /var/log/board/.
Other necessery information.

In GitLab by @yhua123 on Nov 8, 2019, 07:52
merged step 2 and step 3

In GitLab by @liyanq528 on Nov 8, 2019, 07:52
pr#1682

In GitLab by @yhua123 on Nov 8, 2019, 07:52
step 1: update back service for combine 2 steps
step 2: update UI again

In GitLab by @sokril on Nov 8, 2019, 07:52
fix with #1811

In GitLab by @wknet123 on Nov 8, 2019, 07:52
Closing it as fixed.

In GitLab by @sokril on Nov 8, 2019, 08:23
closed

In GitLab by @sokril on Nov 8, 2019, 08:23
assigned to @wknet123


text:
[Issue] Blank Search should Return All, Better Using Logo Rutun Homepage

In GitLab by @liyanq528 on Sep 13, 2017, 18:11

In GitLab by @wknet123 on Nov 8, 2019, 07:36
Fixed with PR #303

In GitLab by @yhua123 on Nov 8, 2019, 07:36
LGTM

In GitLab by @sokril on Nov 8, 2019, 08:10
closed

In GitLab by @sokril on Nov 8, 2019, 08:10
assigned to @wknet123

In GitLab by @wknet123 on Nov 13, 2019, 03:54
mentioned in merge request !127

In GitLab by @wknet123 on Nov 13, 2019, 09:58
mentioned in merge request !202


text:
[reverb/pybind.cc:447] Tensor can't be extracted from the source represented as ndarray: INTERNAL: Unsupported object type DiscreteArray

Hi, I'm trying to get Mava to run on a custom environment of ours. I've written a wrapper for it but I've been having some issues in getting it to work properly. I'm using the Decentralized MAPPO implementation for this. The traceback I'm currently dealing with is:
[executor/0] [reverb/pybind.cc:447] Tensor can't be extracted from the source represented as ndarray: INTERNAL: Unsupported object type DiscreteArray
[executor/0] Traceback (most recent call last):
[executor/0]   File "/home/mohib/anaconda3/lib/python3.9/site-packages/launchpad/nodes/python/process_entry.py", line 110, in <module>
[executor/0]     app.run(main)
[executor/0]   File "/home/mohib/anaconda3/lib/python3.9/site-packages/absl/app.py", line 312, in run
[executor/0]     _run_main(main, args)
[executor/0]   File "/home/mohib/anaconda3/lib/python3.9/site-packages/absl/app.py", line 258, in _run_main
[executor/0]     sys.exit(main(argv))
[executor/0]   File "/home/mohib/anaconda3/lib/python3.9/site-packages/launchpad/nodes/python/process_entry.py", line 105, in main
[executor/0]     functions[task_id]()
[executor/0]   File "/home/mohib/anaconda3/lib/python3.9/site-packages/launchpad/nodes/python/node.py", line 72, in _construct_function
[executor/0]     return functools.partial(self._function, *args, **kwargs)()
[executor/0]   File "/home/mohib/anaconda3/lib/python3.9/site-packages/launchpad/nodes/courier/node.py", line 124, in run
[executor/0]     instance.run()
[executor/0]   File "/home/mohib/anaconda3/lib/python3.9/site-packages/mava/environment_loop.py", line 582, in run
[executor/0]     result = self.run_episode()
[executor/0]   File "/home/mohib/anaconda3/lib/python3.9/site-packages/mava/environment_loop.py", line 433, in run_episode
[executor/0]     self._executor.observe_first(timestep, extras=env_extras)
[executor/0]   File "/home/mohib/anaconda3/lib/python3.9/site-packages/mava/systems/tf/mappo/execution.py", line 173, in observe_first
[executor/0]     self._adder.add_first(timestep)
[executor/0]   File "/home/mohib/anaconda3/lib/python3.9/site-packages/mava/adders/reverb/base.py", line 387, in add_first
[executor/0]     self._writer.append(
[executor/0]   File "/home/mohib/anaconda3/lib/python3.9/site-packages/reverb/trajectory_writer.py", line 301, in append
[executor/0]     flat_column_data_references = self._writer.AppendPartial(flat_column_data)
[executor/0] TypeError: AppendPartial(): incompatible function arguments. The following argument types are supported:
[executor/0]     1. (self: reverb.libpybind.TrajectoryWriter, arg0: List[Optional[tensorflow::Tensor]]) -> List[Optional[reverb.libpybind.WeakCellRef]]

The error would suggest that I need the observations need to be in the form of a pure TF Tensor, but I can't do that since earlier parts of the code require agent observations to be encased in an OLT. If I directly replace each observations[agent] with our environment's state as a Tensor, I'd get a traceback like this:
[executor/0] AttributeError: in user code:
[executor/0]
[executor/0]     File "/home/mohib/anaconda3/lib/python3.9/site-packages/mava/systems/tf/mappo/execution.py", line 134, in _select_actions  *
[executor/0]         log_prob, action = self._policy(agent, observation)
[executor/0]     File "/home/mohib/anaconda3/lib/python3.9/site-packages/mava/systems/tf/mappo/execution.py", line 94, in _policy  *
[executor/0]         observation = tf2_utils.add_batch_dim(observation_olt.observation)
[executor/0]
[executor/0]     AttributeError: 'Tensor' object has no attribute 'observation'

Any ideas on how to fix this? For reference, the example decentralized MAPPO script seems to run just fine. Our wrapper currently looks like this:
class ILocusEnv(dm_env.Environment):
    def __init__(self, size_r=5, size_c=5, driver_sim=Drivers(), gamma=0.56, objective=None):
        self.action_space = None
        self.observation_space = None
        self.reward_range = None
        self.size_r = size_r
        self.size_c = size_c
        self.gamma = gamma
        self.env = Environment(driver_sim, (self.size_r, self.size_c), objective)
        self.action_queue = []
        self.curr_obs = None

        agent_list = []
        for i in range(size_r * size_c):
            agent_list.append("agent_" + str(i))
        self.possible_agents = agent_list

    def step(self, action):
        # First, add the action to our queue
        self.action_queue.append(action + 1)

        # Next, check if our queue has a submitted action for every single agent
        if len(self.action_queue) == (self.size_r * self.size_c):
            # If so, assemble all the actions into a size_r * size_c array
            action_arr = np.empty((self.size_r, self.size_c))
            for r in range(self.size_r):
                for c in range(self.size_c):
                    action_arr[r, c] = self.action_queue[(r * self.size_c) + c]

            # Submit this action to the environment, then reset the queue
            state, reward, term = self.env.step(action_arr)
            self.action_queue = []
            self.curr_obs = tf.convert_to_tensor(state)

            # Make sure to return it as a timestep!
            if term:
                step_type = dm_env.StepType.LAST
            else:
                step_type = dm_env.StepType.MID
            rewards = {}
            observations = {}
            gammas = {}
            for agent in self.possible_agents:
                rewards[agent] = tf.convert_to_tensor(reward)
                observations[agent] = types.OLT(
                    observation = self.curr_obs,
                    legal_actions = self.action_spec()[agent],
                    terminal = np.array(term)
                )
                gammas[agent] = tf.convert_to_tensor(self.gamma)
            return dm_env.TimeStep(
                observation=observations,
                reward=rewards,
                discount=gammas,
                step_type=step_type,
            )
        else:
            rewards = {}
            observations = {}
            gammas = {}
            for agent in self.possible_agents:
                rewards[agent] = tf.convert_to_tensor(0.0)
                observations[agent] = types.OLT(
                    observation = self.curr_obs,
                    legal_actions = self.action_spec()[agent],
                    terminal = np.array(False)
                )
                gammas[agent] = tf.convert_to_tensor(self.gamma)
            return dm_env.TimeStep(
                observation=observations,
                reward=rewards,
                discount=gammas,
                step_type=dm_env.StepType.MID,
            )

    def reset(self):
        self.curr_obs = tf.convert_to_tensor(self.env.reset(None))
        rewards = {}
        observations = {}
        gammas = {}
        for agent in self.possible_agents:
            rewards[agent] = tf.convert_to_tensor(np.array(0.0))
            observations[agent] = types.OLT(
                observation = self.curr_obs,
                legal_actions = self.action_spec()[agent],
                terminal = np.array(False)
            )
            gammas[agent] = tf.convert_to_tensor(np.array(self.gamma))
        return dm_env.TimeStep(
            observation=observations,
            reward=rewards,
            discount=gammas,
            step_type=dm_env.StepType.FIRST,
        )

    def action_spec(self) -> specs.DiscreteArray:
        action_spec_dict = {}
        for agent in self.possible_agents:
            action_spec_dict[agent] = specs.DiscreteArray(5)
        return action_spec_dict

    def observation_spec(self) -> specs.Array:
        obs_spec_dict = {}
        for agent in self.possible_agents:
            obs_spec_dict[agent] = types.OLT(
                observation = specs.Array(shape=[self.size_r, self.size_c],
                                          dtype=np.float),
                legal_actions = self.action_spec()[agent],
                terminal = specs.Array(shape=(1,), dtype=np.bool_),
            )
            # print(agent)
        return obs_spec_dict

    def reward_spec(self):
        # Based on the _environment code of dm_env on Github
        reward_spec_dict = {}
        for agent in self.possible_agents:
            reward_spec_dict[agent] = specs.Array(shape=(1,), dtype=float, name='reward')
        return reward_spec_dict

    def discount_spec(self):
        # Based on the _environment code of dm_env on Github
        dis_spec_dict = {}
        for agent in self.possible_agents:
            dis_spec_dict[agent] = specs.BoundedArray(shape=(1,), dtype=float, minimum=0., maximum=1., name='discount')
        return dis_spec_dict

    def extra_spec(self):
        ex_spec_dict = {}
        return ex_spec_dict

Hi @MohibAzam , thanks for your detailed issue.
Can you print a sample of your state and self.curr_obs? It can be a valid numpy array or tensor. Also, can you print the observations that get passed in your reset? This seems to be where the error is occurring. It looks like one of your observations might be a spec, i.e. specs.DiscreteArray ,  as opposed to an actual numpy array or tensor.

@MohibAzam I'm closing this issue for now. I hope @KaleabTessera's comment solved your issue. Please feel free to open another if you're still having problems.

authors: KaleabTessera MohibAzam mmorris44

repo: instadeepai/Mava


text:
Request return 429 error! - after 5 days (ultimate_schedule) - log available

I have just received "Request return 429 error" after 5 days of boting with ultimate_schedule.py without any problems trough proxy.
Windows 8.1, Anaconda Python 3.6, multiple accounts.
For each IG account i have a copy of Bot script. This bot is using version='0.3.3.16'
log_1.txt

I had the same experience and I realized that is normal to have this warning from Instagram if we are going to exaggerate in the use of the bot. I do not know what is the threshold yet, but would be interesting to understand it, collecting the experiences of everyone.

Duplicated: #212

authors: maverjk ohld rebbu

repo: instagrambot/instabot


text:
Downloading :saved ended unexpectedly.

Your question here...
% instaloader --login=andy91129 --dirname-pattern=/Users/yao-antsai/Applications/Box\ Sync/Instaloader/{profile} --filename-pattern={profile}_{date_utc:%Y-%m-%d_%H-%M-%S} --no-captions --no-metadata-json --resume-prefix resume :saved
...
...
...
[27663/32115] /Users/yao-antsai/Applications/Box Sync/Instaloader/0907sonia/xxxxxxxxx_2016-09-29_00-55-29.jpg exists
Iteration complete, deleted resume information file /Users/yao-antsai/Applications/Box Sync/Instaloader/:saved/resume_uPLl6Txc.json.xz.
Saved session to /Users/yao-antsai/.config/instaloader/session-andy91129.

I used instaloader to download all of my saved posts.
However, it ended in the middle of downloading at 2XXXXX posts.
I already tried to update to the lastest version and tried to download the saved posts from the beginning twice.
Is there anyway to solve this problem?
Thx.

Did anyone encounter similar issues?

Did you count the posts that you can actually see in your saved posts? Instagram says that I have around 1300 saved posts, but I can only see around 1100, so only 1100 get downloaded. I guess around 200 posts have been deleted/archived/private accounts that you now don't have access to/blocked accounts, but Instagram still counts those in the total number of saved posts, but if you don't have permission to the posts, they won't get downloaded. That's just my guess tho

I have similar problem.
This is my output.
Loaded session from D:\*****.
Logged in as :\*****..
Retrieving saved posts...
Saved session to D:\*****.

When I open Saved from instagram app there is a lot of posts.

repo: instaloader/instaloader


text:
Get followers of photos from closed profile instagram

Hello, it is possible to get many infromation from closed account but account picture, count of followes/followers and etc?

what are many information?
if you are not an approved follower of a private account, you only get the very basic things like profile picture and the information on top like count of followes/followers or biography. nothing more. check the json file you got after grabbing a private account.


text:
ilab should implicitly use a local config.yaml file, if present, from the current working directory.

What is the current experience of ilab you've found issue with?
I am looking for an option to use a ./config.yaml file from $PWD instead of the default one from $HOME/.config/instructlab/config.yaml. I am aware of the --config flag, however, it is not the best solution for me. I have to run every command with this --config flag.
How do you think the described experience should be?
ilab should use automatically use configurations from a local ./config.yaml file if present in current working directory.
What is the benefit of the change you are proposing?
I want to contain all the files and configurations for an AI project into a single folder.

@svghadi
you can set the ILAB_GLOBAL_CONFIG env var to point to the config file you want
please try that and see if it solves your use case

repo: instructlab/instructlab


text:
Enable usage of ic-modal with Ember 1.9.0.

See http://emberjs.com/blog/2014/10/16/handlebars-update.html for details, but Ember 1.9.0 will require the usage of Handlebars 2.0. This PR depends on https://github.com/toranb/broccoli-ember-hbs-template-compiler/pull/6 and bumps the major version (as precompiled templates are not compatible between Handlebars 1.x and 2.0).
In the interim users can specify my branch for usage with Ember 1.9.0 by entering the following in their bower.json:
 "ic-modal": "rwjblue/ic-modal#ember-1-9-compat"

This really helped me out. Wondering if this will be merged in soon!

authors: darkzeroman rwjblue

repo: instructure/ic-modal


text:
🛑 Shopintake is down

In 60cd78b, Shopintake (https://www.shopintake.com) was down:

HTTP code: 403
Response time: 476 ms

Resolved: Shopintake is back up in 040909a after 7 minutes.


text:
Fix PKCS5Padding of openssl

Fix PKCS5Padding of openssl, and use ReflectionUtils create the cipher instance since CipherFactory will use JCECipher when the real cipher created failed

@dianfu Could you help to take a look?

Patch updated.

+1

Thank @dianfu and @winningsix for your review.

authors: dianfu sundapeng

repo: intel-hadoop/chimera


text:
Stop partprobing so much

IML currently runs partprobe every time it kicks off a device scan.
It does this because it doesn't know if the other node in an active / passive pair has any new partitions that have been created / destroyed.
A side effect of this is that udev thrashes back and forth during a partprobe and reading devices is non-deterministic. There are two ways we can solve this in tandem.

[x] Use udevadm settle to wait for the event queue to empty
[ ] Use shared device state to inform the other node of any new partition creations / deletions. By doing this, we only need to partprobe in those specific situations instead of every full scan, which will considerably lower the churn.

Partprobe has been completely removed. We will need to update the docs to notify users that this is a now a manual activity when creating / destroying zpools.
We will add this back in a more measured fashion when we take on fully managed ZFS.

repo: intel-hpdd/intel-manager-for-lustre


text:
Pass the Protex scan for 4.0.2

Pass protex scan. This has a pre-req that someone takes the protex training or we find an outside source to perform the scans.

No need for protex in 4.0.2

authors: jgrund petertix


text:
Drive configuration section name in device services is incorrect

THe custom Driver configuration section name is Driver not DriverConfig
https://github.com/intel-retail/automated-vending/blob/Edgex-3.0/ds-card-reader/res/configuration.yaml#L10
https://github.com/intel-retail/automated-vending/blob/Edgex-3.0/ds-controller-board/res/configuration.yaml#L20C6-L20C6

This are true custom config sections.

repo: intel-retail/automated-vending


text:
[POAE7-2526] Fix Non Groupby Agg Arrow Format null value.

What changes were proposed in this pull request?

Fix Non Groupby Agg Arrow Format null value.
Why are the changes needed?

Fix Non Groupby Agg Arrow Format null value.
Does this PR introduce any user-facing change?

No.
How was this patch tested?

Uts.
Which label does this PR belong to?

BUG,cider,test

Plz help to take a review. @jikunshang @yma11 @kd97

repo: intel/BDTK


text:
Question about memory

(1) What's the necessary of qat_config_mem.ko ?  I simply rmmod it while Nginx was still running and found Nginx worked and reloaded correctly.
(2) Next few days after I did (1), I found Nginx did't work, dmesg shows:
[1564771.408920] Nginx: page allocation failure: order:9, mode:0x3040d0
[1564771.416519] CPU: 29 PID: 13537 Comm: Nginx Tainted: G           OE  ------------   3.10.0-327.el7.x86_64 #1
[1564771.428579] Hardware name: Huawei Technologies Co., Ltd. Tecal RH2288H V2-12L/BC11SRSG1, BIOS RMIBV503 03/09/2015
[1564771.439347] Call Trace:
[1564771.442148]  [<ffffffff81650e6a>] dump_stack+0x19/0x1b
[1564771.447628]  [<ffffffff81179ac0>] warn_alloc_failed+0x110/0x180
[1564771.453892]  [<ffffffff8164b57d>] ? __alloc_pages_direct_compact+0x186/0x1b5
[1564771.461452]  [<ffffffff8164bb9b>] __alloc_pages_slowpath+0x5ef/0x826
[1564771.468152]  [<ffffffff8117dcba>] __alloc_pages_nodemask+0x41a/0x440
[1564771.474841]  [<ffffffff8164d062>] kmalloc_large_node+0x60/0x8d
[1564771.481017]  [<ffffffff811d1c32>] __kmalloc_node+0x222/0x280
[1564771.487006]  [<ffffffff81176265>] ? filemap_fault+0x225/0x430
[1564771.493290]  [<ffffffffa0502dfc>] dev_mem_alloc.isra.6+0x12c/0x5f0 [usdm_drv]
[1564771.501108]  [<ffffffffa0503427>] mem_ioctl+0x167/0x200 [usdm_drv]
[1564771.507752]  [<ffffffff81204575>] do_vfs_ioctl+0x2e5/0x4c0
[1564771.513670]  [<ffffffff812047f1>] SyS_ioctl+0xa1/0xc0
[1564771.519113]  [<ffffffff816630fd>] system_call_fastpath+0x16/0x1b
[1564771.525535] Mem-Info:

[1564772.301905] usdm_drv: userMemAlloc:380 Unable to allocate memory slab or wrong alignment:           (null)
[1564772.312064] usdm_drv: dev_mem_alloc:566 userMemAlloc failed

Seemed memory leaked but the memory usage of Nginx was low(I event restarted the Nginx but it didn't work still).
Then I found that Cached in /proc/meminfo is very large. Yes, I do a lot of log action, and I think the system would reclaim it.
(3) I removed the log file, then the Cached in /proc/meminfo recovered, and restarted Nginx, it works now!
(4) Does it related tormmod qat_config_mem

@stevelinsell Hello Steve, I have a similar issue. userMemAlloc:380 can allocate memory but is with the problem of wrong alignment. What's likely root cause? Is it still caused by USDM failing to allocate 2MB slab? Many thanks in advance.
Besides, if the wrong alignment is caused by USDM failing to allocate 2MB slab, can I use kvzalloc() to replace kmalloc() in QAT driver/USDM? Does QAT driver/USDM require "physically contiguous memory in the kernel's own address space"?
https://lwn.net/Articles/711653/
Version information:
QAT driver: QAT1.7.Upstream.L.1.0.3-42
QAT engine: QAT_Engine-0.5.40, uses qat_contig_mem as memory driver.

Intel QAT driver uses USDM to allocate memory. "The USDM Memory Driver works by allocating 2MB slabs and dividing them up for individual allocations. The failures you are seeing in your log are from not being able to allocate a 2MB slab of contiguous memory from the kernel."
The device is highly fragmented because there're few chunks equal to or greater than 2MB.
cat /proc/buddyinfo
Node 0, zone DMA 2 1 0 1 3 2 2 1 2 0 0 0
Node 0, zone DMA32 3721 455 163 120 41 15 7 3 2 7 5 1
Node 0, zone Normal 273 237 187 128 66 32 28 5 4 0 0 0
Nov 26 13:01:27  user.err kernel: [4999514.599784] userMemAlloc:380 Unable to allocate memory slab or wrong alignment: 000000006faebeda
Nov 26 13:01:27  user.err kernel: [4999514.707096] dev_mem_alloc:566 userMemAlloc failed
Nov 26 13:01:27  user.info kernel: [4999514.765474] xxxxx[15568]: segfault at 40 ip 00007f8ffa1938d0 sp 00007ffc35618518 error 4 in libjemalloc.so.2[7f8ffa13b000+76000]
Nov 26 13:01:27  user.info kernel: [4999514.765478] Code: 66 2e 0f 1f 84 00 00 00 00 00 48 8b 06 48 8d 15 06 15 02 00 48 c1 e8 12 0f b6 c8 48 8b 04 ca c3 66 2e 0f 1f 84 00 00 00 00 00 <48> 8b 46 40 c3 66 66 2e 0f 1f 84 00 00 00 00 00 48 89 56 40 c3 66
Nov 26 13:01:29  user.warn kernel: [4999516.600569] warn_alloc: 4 callbacks suppressed
Nov 26 13:01:29  user.warn kernel: [4999516.600570] xxxxx: page allocation failure: order:9, mode:0x40cc0(GFP_KERNEL|__GFP_COMP), nodemask=(null)
Nov 26 13:01:29  user.debug kernel: [4999516.600572] CPU: 2 PID: 15572 Comm: xxxxx Kdump: loaded Tainted: G W O 5.4.0 1
Nov 26 13:01:29  user.debug kernel: [4999516.600573] Hardware name: GIGABYTE MN32-EC2-F5/MN32-EC2-F5, BIOS F03 06/04/2019
Nov 26 13:01:29  user.debug kernel: [4999516.600573] Call Trace:
Nov 26 13:01:29  user.debug kernel: [4999516.600577] dump_stack+0x50/0x70
Nov 26 13:01:29  user.debug kernel: [4999516.600578] warn_alloc.cold+0x73/0xd7
Nov 26 13:01:29  user.debug kernel: [4999516.600580] __alloc_pages_slowpath+0x8e3/0xaa0
Nov 26 13:01:29  user.debug kernel: [4999516.600581] ? cdev_put.part.0+0x20/0x20
Nov 26 13:01:29  user.debug kernel: [4999516.600582] __alloc_pages_nodemask+0x222/0x250
Nov 26 13:01:29  user.debug kernel: [4999516.600584] kmalloc_large_node+0x40/0xa0
Nov 26 13:01:29  user.debug kernel: [4999516.600585] __kmalloc_node+0x12b/0x290
Nov 26 13:01:29  user.debug kernel: [4999516.600587] 0xffffffffa02848cd
Nov 26 13:01:29  user.debug kernel: [4999516.600588] 0xffffffffa02849cf
Nov 26 13:01:29  user.debug kernel: [4999516.600589] do_vfs_ioctl+0x3e4/0x640
Nov 26 13:01:29  user.debug kernel: [4999516.600590] ksys_ioctl+0x3a/0x70
Nov 26 13:01:29  user.debug kernel: [4999516.600592] __x64_sys_ioctl+0x16/0x20
Nov 26 13:01:29  user.debug kernel: [4999516.600593] do_syscall_64+0x68/0x3c0
Nov 26 13:01:29  user.debug kernel: [4999516.600594] ? __do_page_fault+0x23d/0x480
Nov 26 13:01:29  user.debug kernel: [4999516.600596] entry_SYSCALL_64_after_hwframe+0x44/0xa9
2.search regular expression "^.started . acceleration engines\n" in dmesg info.
In the configuration file:
[SHIM]
NumberCyInstances = 22
But it seems that only 16 Crypto instances are obtained each time.
Line 887: Sep 12 03:39:16  user.info kernel: [ 49.046552] c6xx 0000:53:00.0: qat_dev0 started 8 acceleration engines
Line 893: Sep 12 03:39:16  user.info kernel: [ 50.768547] c6xx 0000:53:00.0: qat_dev0 started 8 acceleration engines
Line 1818: Sep 25 04:41:17  user.info kernel: [ 48.279127] c6xx 0000:53:00.0: qat_dev0 started 8 acceleration engines
Line 1824: Sep 25 04:41:17  user.info kernel: [ 50.000120] c6xx 0000:53:00.0: qat_dev0 started 8 acceleration engines
Line 3075: Sep 29 17:17:18  user.info kernel: [ 48.391143] c6xx 0000:53:00.0: qat_dev0 started 8 acceleration engines
Line 3081: Sep 29 17:17:18  user.info kernel: [ 50.114146] c6xx 0000:53:00.0: qat_dev0 started 8 acceleration engines

authors: Syllinia mrpre


text:
Update required Ansible version to 2.7.12

Fixes CVE-2019-10156 vulnerability.

Ansible 2.7.12 verified to be compatible with the existing code and 3rd party components.
PLAY RECAP *******************************************************************************
127.0.0.1                  : ok=6    changed=0    unreachable=0    failed=0
master1                    : ok=494  changed=166  unreachable=0    failed=0
master2                    : ok=414  changed=128  unreachable=0    failed=0
master3                    : ok=416  changed=129  unreachable=0    failed=0
node1                      : ok=420  changed=155  unreachable=0    failed=0
node2                      : ok=376  changed=129  unreachable=0    failed=0
node3                      : ok=392  changed=141  unreachable=0    failed=0

repo: intel/container-experience-kits


text:
Support phi series SQ

Type of Change
support phi-1, phi-1.5, phi-2, phi-3 do smoothquant with ipex 2.3, transformers 4.38.1
Description
detail description
JIRA ticket: xxx
Expected Behavior & Potential Risk
the expected behavior that triggered by this PR
How has this PR been tested?
how to reproduce the test (including hardware information)
Dependency Change?
any library dependency introduced or removed

https://inteltf-jenk.sh.intel.com/view/nlp-toolkit-validation/job/nlp-toolkit-validation-top-mr-extension/1671/

repo: intel/intel-extension-for-transformers


text:
I'm dying to know how to implement the int8 winograd in conv3x3s1

I find the mkl-dnn has been supported the int8 winograd in convolution.
Is there any paper to introduce how to quantization winograd.
Thanks a lot!

@BUG1989, there's a section in documentation describing quantization approach Intel MKL-DNN supports. The same approach is applied to Winograd convolution algorithm.

Summoning @tprimak or @nshustrov to comment on the nuances of the implementation.

@BUG1989, the implementation of int8 winograd convolution in MLK-DNN does not require special quantization. The approach to quantization of input data and weights is the same with quantization used for direct int8 convolution with unsigned int8 input and signed int8 weights.  All needed data adjustment happens in the weights reorder and transformation of input and output data.

I see. Thank you very much for your help.And the quantize of weights before or after winograd transformation ?

@BUG1989 can show the opensource project name or address, which you find int8 winograd in it?

@xiaoweiChen
https://github.com/Tencent/ncnn/blob/master/src/layer/x86/convolution_3x3_int8.h

@BUG1989 Thanks for your sharing.  :)

authors: BUG1989 nshustrov vpirogov xiaoweiChen

repo: intel/mkl-dnn