text:
Fileprovider.Watch("") Does not work as expected

Hello again, ive been using this in memory file magic some more, and ive found that the root directory (in this case i mean the root directory of any given provider) does not function properly, i cannot watch for changes in this root directory forexample, i get a null argument exception, which i dont get with the physical file provider forexample..
Its quite unsettling for me that my tests have completely different behaviors in this regard than my actual physical file provider.
        [Fact]
        public async Task GetFilesToBackupQuery_NewApproach_FilewatchIsApplied()
        {
            // Arrange
            var rootLocation = "/RAP";
            var directory = new InMemoryDirectory(rootLocation);
            var fp = new InMemoryFileProvider(directory);
            var watcher = fp.Watch(""); // This line throws, but i expect it to watch rootLocation
            var changeMe = false;
            watcher.RegisterChangeCallback((state) => { changeMe = true; }, new object());
            // Act
            directory.AddFile(rootLocation, new StringFileInfo("Test", "Test.txt"));
            await Task.Delay(2000);
            // Assert
            changeMe.Should().Be(true);
        }

Interesting. Does Watch("/") work?

It doesent throw if i use it with "/", however this is different from how the physical file provider works (:
It also dident register any change when i added that file a couple lines later (:

What do you think?

Sounds like its a bug if its not behaving consistently with how Watch works with other providers like the PhysicalFileProvider.
I'd welcome a PR or added test coverage in this area.
Otherwise it may be a while until I can look at it, but I'll add it to my list!

You can see there are various tests for watch here: https://github.com/dazinator/Dazinator.AspNet.Extensions.FileProviders/blob/master/src/Dazinator.AspNet.Extensions.FileProviders.Tests/InMemoryFileProviderTests.cs#L215
These were produced at the time based on the physical file watcher tests.
However I can't see one that tests watching for a new file - only tests are for watching changes to an existing file.
It would be interesting to see how PhysicalFileWatcher handles that.
Also I am pretty sure watching("") used to be correct, are you using asp.net core 1.X or 2.X ? Perhaps things have changed with physicalfileprovider in 2.X  compared to the 1.X version?

My manager has agreed to me spending some of my work time doing a PR, so ill see what i can figure out (:
Im using 2.X of asp.net core (:

After digging around in your code a bit, it seems to me that this code was never designed to support watching directories, only files, and the tests seem to back this theory since there arent any that watches a directory (: So i think its gonna take a bit of a redesign.. This whole filters concept seems to work really nice for checking if a file has changed, but i cant really make it work with directories, since its not what its designed for sadly..
Im totally down for helping, but i think you might need to make some design considerations (:
From what ive learned so far it seems that we can check if a path has a file extension (.txt ect) and if it does not then we can add a "/" wildcard to make the globbing thing work, but then we get another design issue later when it then tries to notify the watcher cause it cannot find a watcher with the path "/" cause the watcher would be watching "".
Im gonna keep going for a while here, i just wanted to put down these thoughts while they were still fresh!

I am up for changes in design that facilitate the end goal of mirroring physicalfileprovider behaviours in terms of watching.
It would be good initially just to get a few failing tests up that we want to pass, and that PhysicalFileProvider passes with - if you get a chance to add a few such tests we can then discuss any design changes in terms of making those tests pass.

Closing here, thanks for this, it was my first contribution to an OS framework, and it was a fun and educating experience (:

@dazinator Just one last comment, any plans on making a new release, or should i just use the unstable version for a while? (:

If you can use the latest unstable nuget package for now that would be great. I will issue a new release but I want to ensure there has been some time for this change to sink in - if you hit any further issues please let me know.

authors: RPaetau dazinator


text:
Incorrect format of generated binary literals

System information:

Operating system (distribution) and version: any
DBeaver version: 7.2.3 and earlier
Java version: any
Additional extensions: none

Connection specification:

Database name and version: HANA Cloud (4.0), also earlier
Driver name: com.sap.db.jdbc.Driver, 2.7.9 and earlier
Do you use tunnels or proxies (SSH, SOCKS, etc)? no

Describe the problem you're observing:
In short: Binary literals are generated as 0x<value> instead of X'<value>', where <value> is the value in hexadecimal format.
Long: Generated binary literals use the non-standard format: 0xAABB. However, the SQL standard specifies that  X'AABB' should be used instead. Some of the DBs (like MariaDB) still support the non-standard format probably for compatibility reasons, but others (like HANA) don't (HANA actually treats 0xAABB as an hexadecimal integer value).
Snippet from the ANSI SQL '92 Standard:
         <hex string literal> ::=
              X <quote> [ <hexit>... ] <quote>
                [ { <separator>... <quote> [ <hexit>... ] <quote> }... ]

HANA 2.0.5 docs: https://help.sap.com/viewer/4fe29514fd584807ac9f2a04f6754767/2.0.05/en-US/20a1569875191014b507cf392724b7eb.html
Steps to reproduce, if exist:

Create a table with a binary primary key.
Go to table data and try to delete a row. Save and refresh the data. The row is still there.
Alternatively, go to table data and right click a row, then generate delete query. Run the generated query to get the following error:

SQL Error [266] [07006]: SAP DBTech JDBC: [266] (at 43): inconsistent datatype: INT type is not comparable with VARBINARY type.: line 2 col 12

verified

source: gharchive/issue

authors: alumni uslss

repo: dbeaver/dbeaver

license_type: permissive

license_source: github-api


text:
Function header and trailler missing for PostgreSQL

I suspect this is similar to https://github.com/dbeaver/dbeaver/issues/3892 but it impacts PostgreSQL versions well above 8.4.
System information:
Windows
Tested on:
DBeaver 7.3.0
DBeaver 21.1.0
Connection specification:
Tested on:
PostgreSQL 9.5 and 10.10:
PostgreSQL 9.5.23 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39), 64-bit
PostgreSQL 10.10 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-36), 64-bit

Driver name
PostgreSQL JDBC Driver

Do you use tunnels or proxies (SSH, SOCKS, etc)?
Happens both on and off SSH.

Describe the problem you're observing:
Sometimes when loading a function in PostgreSQL you do not get the header.
You end up with:

Whereas it should look like:

Also missing from the end is:
$function$;
Steps to reproduce, if exist:

Open up a PostgreSQL connection.
Open the functions list in a schema with functions (more than 1 would show the results best).
Open up (view function) one of the functions in the schema. This should load as expected, close the function.
Leave DBeaver running for at least an hour with no interaction.
Open up one of the functions that was not opened in step 3. This will not load the header.
Opening up the same function that was opened in step 3 still loads correctly.

Workaround:
Restart of DBeaver.
Or just Disconnect the connection and reconnect (Invalidate/Reconnect does not seem to be enough).
Include any warning/errors/backtraces from the logs
Logs seem to show:
2021-06-04 12:50:21.729 - Error reading procedure body
org.postgresql.util.PSQLException: This connection has been closed.
at org.postgresql.jdbc.PgConnection.checkClosed(PgConnection.java:767)
at org.postgresql.jdbc.PgConnection.prepareStatement(PgConnection.java:1659)
at org.postgresql.jdbc.PgConnection.prepareStatement(PgConnection.java:373)
at org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCConnectionImpl.prepareStatement(JDBCConnectionImpl.java:244)
at org.jkiss.dbeaver.model.impl.jdbc.JDBCUtils.queryString(JDBCUtils.java:624)
at org.jkiss.dbeaver.ext.postgresql.model.PostgreProcedure.getObjectDefinitionText(PostgreProcedure.java:400)
at org.jkiss.dbeaver.ui.editors.sql.SQLSourceViewer.getSourceText(SQLSourceViewer.java:85)
at org.jkiss.dbeaver.ui.editors.sql.SQLEditorNested$ObjectDocumentProvider$1.lambda$0(SQLEditorNested.java:271)
at org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:169)
at org.jkiss.dbeaver.ui.editors.sql.SQLEditorNested$ObjectDocumentProvider$1.run(SQLEditorNested.java:269)
at org.jkiss.dbeaver.model.runtime.AbstractJob.run(AbstractJob.java:105)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)

thanks for the report
closed as the duplicate of #12649

authors: HeikkiVesanto uslss


text:
IBMi DB2 / AS400 - Auto Generated / Auto Increment not correctly showing

System information:

Operating system (distribution) and version:
Windows 10 / 64bit
DBeaver version:
21.0.1 and 21.0.2. Also verified issue in 6.0.0
Additional extensions:
None

Connection specification:

Database name and version:
com.ibm.as400.access.AS400JDBCDriver, jt400-10.5.jar (10.5)
Driver name:
JT400 / com.ibm.as400.access.AS400JDBCDriver
Do you use tunnels or proxies (SSH, SOCKS, etc)?
No

Describe the problem you're observing:
When viewing the column properties on the table it is not identifying identity columns on the table properly and showing us they auto-increment.
Example:

You can see the fields are not marked as auto incrementing, even though it is an identity column which if I use the native ACS viewer from IBM I see this:

Steps to reproduce, if exist:
Double click a table with an identity column in DBeaver to bring up that tables property window.
Include any warning/errors/backtraces from the logs

No warnings.

Hello @xantari
Please add table DDL info for tests. I can't reproduce it for now for DB2 LUW.

This is for DB2 for IBMi, which is a different DB2 dialect then LUW and uses a different JDBC driver then the LUW version.
Here is the DDL:
CREATE TABLE ARRTFLIB.CEACTIVITYAPIUSERS FOR SYSTEM NAME CEACTAPIU (
	IDSPROVIDERID FOR COLUMN IDSPROID   INTEGER GENERATED ALWAYS AS IDENTITY (
	START WITH 1 INCREMENT BY 1
	NO MINVALUE NO MAXVALUE
	NO CYCLE NO ORDER
	NO CACHE )
	,
	CLIENTID VARCHAR(200) CCSID 37 NOT NULL ,
	ALLSPONSORACCESS FOR COLUMN ALLSPAXS   SMALLINT NOT NULL DEFAULT 0 ,
	CONSTRAINT ARRTFLIB.Q_ARRTFLIB_CEACTAPIU_IDSPROID_00001 PRIMARY KEY( IDSPROVIDERID ) )

	RCDFMT CEACTAPIU  ;

LABEL ON TABLE ARRTFLIB.CEACTIVITYAPIUSERS
	IS '-CEACTIVITYAPIUSERS' ;

LABEL ON COLUMN ARRTFLIB.CEACTIVITYAPIUSERS
( IDSPROVIDERID TEXT IS 'Identity Server PoviderID' ,
	CLIENTID TEXT IS 'Client ID' ,
	ALLSPONSORACCESS TEXT IS 'All sponsor access flag' ) ;

Ok, thanks for the bug report.
For AS400 DBeaver shows only what the driver gives. So maybe this is a driver issue.
We don't have a DB2 AS400 test environment for testing, unfortunately. Therefore we can't fix it now. Maybe someday.

@xantari This driver can be very sensitive to it's properties and changing their default values can fix some issues for some people, but break something for others. So my advice to you for now is to try carefully tweaking them (or first looking in the documentation for a needed property) to test if it can be changed

@Matvey16 Thanks, I did some experimentation and found a result that does show the auto increment properties properly. But then it messes up the column retrieval information.
Here is what I had:

metadata source: 0
translate binary: true
date format: iso
time format: iso
extended metadata: false

With the above settings you get the column names in addition to the column text underneath the column as follows when you do a select * from table:

To fix the identity column information issue I then changed it to the following:

metadata source: 1
translate binary: true
date format: iso
time format: iso
extended metadata: true

So lines #1 and #5 above were changed. You can not leave metadata source: 0 and just leave "extended metadata" to true. As that still doesn't allow you to view the identity column information, even though the documentation says it should.
The problem now though, is with #1 and #5 above changed I now get this when viewing that same table as above:

As you can see in the above image, though I've fixed the issue with the identity column property info now being displayed in DBeaver, I have completely lost the column names on the result sets.

So a bit more experimentation.
Used the following driver properties:
metadata source: 1
translate binary: true
date format: iso
time format: iso
extended metadata: true
When I have this unmarked:

I get this:

When marking it:

I get the column names back:

Now, I figured how do I get the column headers back. And I noticed this property in DBeavber:

It was already marked. and should show the column description/labels in the header right? Kinda wondering if this is now a DBeaver bug.

Closing this in favor of #13335 since the original issue this report is about is fixed, but a separate issue has now cropped up.

Ok, thanks for the bug report.
For AS400 DBeaver shows only what the driver gives. So maybe this is a driver issue.
We don't have a DB2 AS400 test environment for testing, unfortunately. Therefore we can't fix it now. Maybe someday.

You may be able to get a free IBM i account at https://pub400.com/.


text:
No shortcut to open a table from the References panel

Is your feature request related to a problem? Please describe.
When following references between multiple tables, the References panel comes useful to go from table A to table B. However, if table C has a foreign key refering to table B, it is then inconvenient to get to C.
I see no way to do that with DBeaver 22.0.1.
Describe the solution you'd like
A simple way would be to allow opening the References panel in its own full tab, so that we can then use References again.
Describe alternatives you've considered
Perhaps a more elaborate solution like Toad's Master-Detail Browser would be even better.

Hello @Chealer,
Sorry for the late response.
I'm unsure if I understand you correctly. Can you please describe (or show using a video) a use case for your feature request?

Nothing to be sorry about @ShadelessFox
A use case would involve 2 entities indirected linked with foreign keys, through a third entity.
For example, an organization can have members, and a member can have skills. If a table for skills has a foreign key to a table for members, and the members table has a foreign key to an ORGANIZATION table, it would be great to be able to quickly go from SKILL to ORGANIZATION through MEMBER, by selecting SKILL's foreign key, then MEMBER's foreign key.

So, basically, you want this combo menu to show references from the table that is currently shown in that panel?

I'm not sure what data that panel should display in such case. Should it result in query that looks something like this?
SELECT * FROM skills WHERE member_id IN (SELECT member_id FROM members WHERE organization_id = <selected row>);

Thanks.

That is not really what I wanted @ShadelessFox , but please excuse me and disregard my previous comment. I haven't used DBeaver in a while and was confused. I retested, and here is the actual use case which is problematic:
For example, an organization can have members, and a member can have skills. If a table for skills has a foreign key to a table for members, and the members table has a foreign key to an ORGANIZATION table, it would be great to be able to quickly go from ORGANIZATION to SKILL through MEMBER, by selecting MEMBER's reverse foreign key from ORGANIZATION, then SKILL's reverse foreign key. This would allow to quickly find skills found in an organization.
I don't know the best way to do this, but one way would be to add a button in the References panel allowing to make it a full-fledged tab (rather than a sub-tab). There could be some "Separate into new tab" button. To achieve the above, I would go to ORGANIZATION, select a row, open the References panel, select a member in the References panel, use the new "Separate into new tab" button, and from there open the References panel again.
By the way, while Toad's Master Detail can serve as inspiration, its design is not that intuitive, so I would not advise to replicate it exactly.

The provided solution is actually pretty useful, so we can stick with it.
I was unable to take a glance of Toad's Master Detail. Can you please, if possible, provide a footage that shows its functionality?
There is a workaround that involves using the References tab in the metadata editor (it's not the same as the References panel):

authors: Chealer ShadelessFox


text:
is there a way not to have to reinstall plugins after every updates ?

Hi,
i'm using DBeaver in zip version (Windows)
every time I migrate to the newer version I have to redownload et reinstall the plugins.
Fortunatly I'm using only one.
having this message for instance with the Office addon:

thanks for the suggestion
closed a a duplicate of #5317

authors: itphonim uslss


text:
连接doris一段时间后，再也连接不上

Description
第一次连接doris后，过一段时间后，doris再也连接不上，doris集群没任何问题，电脑重启后，打开debaver后恢复正常，过一段时间后，会再次出现连接不上的问题，断开重新连接也连接不上doris，请问这个问题怎么解决?
DBeaver Version
社区版本24.1.0
Operating System
windows11
Database and driver
No response
Steps to reproduce
doris版本2.1.2，社区版本24.1.0
Additional context
No response

已解决


text:
I can't modify cell in resultset. Earlier this function was work. It is very necessary function and it not work.

Can't reproduce. Please provide more details.

What is your database
How do you edit cells (in table editor or in custom query results).
Are you sure that exactly the same results were editable in earlier versions?

Sorry. I rechecked and made sure that I can edit cell in the result of query for MySQL and PostgreSQL.
I can't edit cell for Amazon Redshift, after update the driver for this DB.
It is driver issue (i think) or interaction between DBeaver and Driver Amazon Redshift.
I tried edit cell in the custom query results.

This was fixed in 5.1.5.

authors: nkiseev serge-rider


text:
SQL Editor Issue

Even ever i press multiple enter, and then do backspace their are scroll bar coming on editor screen, its irritating after few occurrence.

Please check workaround in #3916

authors: khushalc serge-rider


text:
Additional Sound Notifications

Hey Serge,
I see that in 3.6.7 you added sound notification support "beep after query finish".
Is it possible we can expand on this capability, for example rather than beep after each query could we have a notification sound after the run of all queries within one SQL editor?
Also could there be a different notification sound based on the outcome per query/per SQL script.
See example notification sounds below:

Success: http://www.soundsnap.com/node/92951
Failed: http://www.soundsnap.com/error_tone_chime_blastwavefx_16379

This would really help me and hopefully other people when you running many queries, want to monitor but don't always have to be at your desk.
Thanks,
Ben

Agreed, that would be a good feature.

Hello All, +1 vote for this...

Hello All, I also +1 vote for this.
This small feature can contribute so much for efficient time utilization while waiting for long running SQL.

+1 for this!

+1
I would also suggest the ability to trigger a macOS notification on query completion too. (Should that be a separate issue?)  I sometimes need to kick off a long-running query and then I go do work in some other app while I'm waiting.  For various reasons, sound notifications aren't always viable.

+1
I would also suggest the ability to trigger a macOS notification on query completion too. (Should that be a separate issue?) I sometimes need to kick off a long-running query and then I go do work in some other app while I'm waiting. For various reasons, sound notifications aren't always viable.

i don't even know how to set the notification:(

What's the latest on this feature?  Is it still planned?

+1
dont't forget us!

Any updates on this? Has it been abandoned?

authors: SirBenJammin andreescastano dburtonDRW earsonlau eng543 lalato serge-rider shungabubus xenago yonisade


text:
DBeaver: When editing an INTEGER value in a grid, the cell is not opened if start typing a minus or plus sign

DBeaver - Version 7.1.4 CE
DBeaver driver: MS SQL Server  / Microsoft Driver (the new Microsoft Driver)
Operating System: Windows 7 / Windows 8.1 / Windows 10
Database Server: Microsoft SQL Express 2014, 2016, 2017
When editing an INTEGER value in a grid, the cell is not opened if start typing a minus "-" or a plus "+", this workes if I enter a number "0".."9" instead.
The grid ought to set the INTEGER cell in edit mode not only when I press "0".."9" on the keyboard, pressing "+" or "-" ought to open the cell in edit mode too.

Create a test table and fill it with test data:

CREATE TABLE TestInteger (Id INTEGER, Value INTEGER);
GO

INSERT INTO TestInteger
VALUES (1, 10), (2, 20);

View the table data TestInteger in a grid.

Click on the 1st row Value column (with the value of 10).

Press number "5" on the keyboard.

The cell is changed to edit mode. This is OK.

Click on the 2nd row Value column (with the value of 20).

Press minus sign (the "-" character) or the plus sign (the "+" character) on the keyboard.

Nothing happens. This is an ERROR.

My guess is that the "-" and the "+" character should be added to characters that change the grid cell into edit mode.

`thanks for the bug report

Fixed

+ still doesn't change int cell in edit mode

Keypad buttons handle was added

verified

authors: kseniiaguzeeva pdanie serge-rider uslss


text:
[BUG] helper command fails due to losreader problem

Describe the bug
When running the 2nd raiderDelay helper command, it fails due to array dimension error in the losreader.py file.
To Reproduce
Steps to reproduce the behavior:

Command used
raiderDelay.py --date 20200103 --time 23:00:00 -b 39 40 -79 -78 --model GMAO --zref 15000 --heightlvs 0 100 200 -v
Error Output

Weather model GMAO is available from 2014-02-20 00:00:00-Present
WARNING: Rounded given hour from  23 to 0
Traceback (most recent call last):
  File "/opt/anaconda3/envs/RAiDER/bin/raiderDelay.py", line 4, in <module>
    __import__('pkg_resources').run_script('RAiDER==0.0.1', 'raiderDelay.py')
  File "/opt/anaconda3/envs/RAiDER/lib/python3.7/site-packages/pkg_resources/__init__.py", line 665, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/opt/anaconda3/envs/RAiDER/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1463, in run_script
    exec(code, namespace, namespace)
  File "/opt/anaconda3/envs/RAiDER/lib/python3.7/site-packages/RAiDER-0.0.1-py3.7-macosx-10.9-x86_64.egg/EGG-INFO/scripts/raiderDelay.py", line 12, in <module>
    parseCMD()
  File "/opt/anaconda3/envs/RAiDER/lib/python3.7/site-packages/RAiDER-0.0.1-py3.7-macosx-10.9-x86_64.egg/RAiDER/runProgram.py", line 196, in parseCMD
    _tropo_delay(new_args)
  File "/opt/anaconda3/envs/RAiDER/lib/python3.7/site-packages/RAiDER-0.0.1-py3.7-macosx-10.9-x86_64.egg/RAiDER/runProgram.py", line 208, in _tropo_delay
    (_, _) = tropo_delay(args_copy)
  File "/opt/anaconda3/envs/RAiDER/lib/python3.7/site-packages/RAiDER-0.0.1-py3.7-macosx-10.9-x86_64.egg/RAiDER/delay.py", line 153, in tropo_delay
    los = getLookVectors(los, lats, lons, hgts, zref)
  File "/opt/anaconda3/envs/RAiDER/lib/python3.7/site-packages/RAiDER-0.0.1-py3.7-macosx-10.9-x86_64.egg/RAiDER/losreader.py", line 338, in getLookVectors
    look_vecs = _getZenithLookVecs(lat, lon, hgt, zref=zref)
  File "/opt/anaconda3/envs/RAiDER/lib/python3.7/site-packages/RAiDER-0.0.1-py3.7-macosx-10.9-x86_64.egg/RAiDER/losreader.py", line 320, in _getZenithLookVecs
    zenLookVecs = (np.array((e, n, u)).T * (zref - heights)[..., np.newaxis])
ValueError: operands could not be broadcast together with shapes (2,3) (3,1)

@leiyangleon yes I saw this bug last night when I was testing as well. I think I have the fix in place, will push asap.

@jlmaurer the command still fails but with a new error now:
Traceback (most recent call last):
  File "/opt/anaconda3/envs/RAiDER/bin/raiderDelay.py", line 4, in <module>
    __import__('pkg_resources').run_script('RAiDER==0.0.1', 'raiderDelay.py')
  File "/opt/anaconda3/envs/RAiDER/lib/python3.7/site-packages/pkg_resources/__init__.py", line 665, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/opt/anaconda3/envs/RAiDER/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1463, in run_script
    exec(code, namespace, namespace)
  File "/opt/anaconda3/envs/RAiDER/lib/python3.7/site-packages/RAiDER-0.0.1-py3.7-macosx-10.9-x86_64.egg/EGG-INFO/scripts/raiderDelay.py", line 12, in <module>
    parseCMD()
  File "/opt/anaconda3/envs/RAiDER/lib/python3.7/site-packages/RAiDER-0.0.1-py3.7-macosx-10.9-x86_64.egg/RAiDER/runProgram.py", line 154, in parseCMD
    args = checkArgs(args, p)
  File "/opt/anaconda3/envs/RAiDER/lib/python3.7/site-packages/RAiDER-0.0.1-py3.7-macosx-10.9-x86_64.egg/RAiDER/checkArgs.py", line 42, in checkArgs
    lat, lon, llproj, bounds, flag, pnts_file = readLL(args.query_area)
  File "/opt/anaconda3/envs/RAiDER/lib/python3.7/site-packages/RAiDER-0.0.1-py3.7-macosx-10.9-x86_64.egg/RAiDER/llreader.py", line 33, in readLL
    fname = ' '.join(*args)
TypeError: sequence item 0: expected str instance, float found

@leiyangleon This is actually not the same bug, but I can't reproduce the error either way because I'm getting a KeyError in the _load_model_level function in gmao.py. Can you tell me if there is a quick fix for this?

(RAiDER) jlmd9g@rt01jlmd9g tmp1 % raiderDelay.py --date 20200103 --time 23:00:00 -b 39 40 -79 -78 --model GMAO --zref 15000 --heightlvs 0 100 200 -v
Weather model GMAO is available from 2014-02-20 00:00:00-Present
WARNING: Rounded given hour from  23 to 0
ERROR: Unable to save weathermodel to file
Traceback (most recent call last):
  File "/Users/jlmd9g/software/miniconda3/envs/RAiDER/lib/python3.8/site-packages/RAiDER-0.0.1-py3.8-macosx-10.9-x86_64.egg/RAiDER/models/gmao.py", line 143, in _fetch
    writeWeatherVars2NETCDF4(self, lats, lons, h, q, p, t, outName=out)
  File "/Users/jlmd9g/software/miniconda3/envs/RAiDER/lib/python3.8/site-packages/RAiDER-0.0.1-py3.8-macosx-10.9-x86_64.egg/RAiDER/utilFcns.py", line 730, in writeWeatherVars2NETCDF4
    nc_outfile = write2NETCDF4core(nc_outfile, dimension_dict, dataset_dict, tran, mapping_name='WGS84')
  File "/Users/jlmd9g/software/miniconda3/envs/RAiDER/lib/python3.8/site-packages/RAiDER-0.0.1-py3.8-macosx-10.9-x86_64.egg/RAiDER/utilFcns.py", line 801, in write2NETCDF4core
    dataset_dict[data]['dataset'][np.isnan(dataset_dict[data]['dataset'])] = FillValue
TypeError: only integer scalar arrays can be converted to a scalar index
Traceback (most recent call last):
  File "/Users/jlmd9g/software/miniconda3/envs/RAiDER/bin/raiderDelay.py", line 4, in
    __import__('pkg_resources').run_script('RAiDER==0.0.1', 'raiderDelay.py')
  File "/Users/jlmd9g/software/miniconda3/envs/RAiDER/lib/python3.8/site-packages/pkg_resources/__init__.py", line 650, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/Users/jlmd9g/software/miniconda3/envs/RAiDER/lib/python3.8/site-packages/pkg_resources/__init__.py", line 1446, in run_script
    exec(code, namespace, namespace)
  File "/Users/jlmd9g/software/miniconda3/envs/RAiDER/lib/python3.8/site-packages/RAiDER-0.0.1-py3.8-macosx-10.9-x86_64.egg/EGG-INFO/scripts/raiderDelay.py", line 12, in
    parseCMD()
  File "/Users/jlmd9g/software/miniconda3/envs/RAiDER/lib/python3.8/site-packages/RAiDER-0.0.1-py3.8-macosx-10.9-x86_64.egg/RAiDER/runProgram.py", line 196, in parseCMD
    _tropo_delay(new_args)
  File "/Users/jlmd9g/software/miniconda3/envs/RAiDER/lib/python3.8/site-packages/RAiDER-0.0.1-py3.8-macosx-10.9-x86_64.egg/RAiDER/runProgram.py", line 208, in _tropo_delay
    (_, _) = tropo_delay(args_copy)
  File "/Users/jlmd9g/software/miniconda3/envs/RAiDER/lib/python3.8/site-packages/RAiDER-0.0.1-py3.8-macosx-10.9-x86_64.egg/RAiDER/delay.py", line 113, in tropo_delay
    weather_model_file = prepareWeatherModel(
  File "/Users/jlmd9g/software/miniconda3/envs/RAiDER/lib/python3.8/site-packages/RAiDER-0.0.1-py3.8-macosx-10.9-x86_64.egg/RAiDER/processWM.py", line 91, in prepareWeatherModel
    f = weather_model.load(
  File "/Users/jlmd9g/software/miniconda3/envs/RAiDER/lib/python3.8/site-packages/RAiDER-0.0.1-py3.8-macosx-10.9-x86_64.egg/RAiDER/models/weatherModel.py", line 201, in load
    self.load_weather(*args, **kwargs)
  File "/Users/jlmd9g/software/miniconda3/envs/RAiDER/lib/python3.8/site-packages/RAiDER-0.0.1-py3.8-macosx-10.9-x86_64.egg/RAiDER/models/gmao.py", line 156, in load_weather
    self._load_model_level(f)
  File "/Users/jlmd9g/software/miniconda3/envs/RAiDER/lib/python3.8/site-packages/RAiDER-0.0.1-py3.8-macosx-10.9-x86_64.egg/RAiDER/models/gmao.py", line 168, in _load_model_level
    h = np.array(f.variables['H'][:])
KeyError: 'H'

@jlmaurer not sure what the cause is as I have never seen that error before. Would suggest to try other models. I suspect this is not related to GMAO only as I haven't touched the GMAO code for months...

authors: jlmaurer leiyangleon

repo: dbekaert/RAiDER


text:
Training yolact on resnet18 and got some error

Hi,
I was training the yolact model on resnet18 backbone and it was going all good but then suddenly I got some error and the training got aborted-
`[  2]   32930 || B: 3.996 | C: 4.955 | M: 4.408 | S: 1.036 | T: 14.395 || ETA: 1 day, 4:21:46 || timer: 0.127
[  2]   32940 || B: 4.020 | C: 5.072 | M: 4.458 | S: 1.104 | T: 14.655 || ETA: 1 day, 4:23:08 || timer: 0.132
[  2]   32950 || B: 4.063 | C: 5.246 | M: 4.552 | S: 1.199 | T: 15.060 || ETA: 1 day, 4:23:34 || timer: 0.151
[  2]   32960 || B: 4.057 | C: 5.435 | M: 4.608 | S: 1.271 | T: 15.371 || ETA: 1 day, 4:23:31 || timer: 0.158
[  2]   32970 || B: 4.071 | C: 5.574 | M: 4.675 | S: 1.306 | T: 15.626 || ETA: 1 day, 4:24:23 || timer: 0.166
[  2]   32980 || B: 4.033 | C: 5.746 | M: 4.758 | S: 1.381 | T: 15.918 || ETA: 1 day, 4:26:38 || timer: 0.140
[  2]   32990 || B: 4.031 | C: 5.817 | M: 4.741 | S: 1.411 | T: 15.999 || ETA: 1 day, 4:25:21 || timer: 0.139
[  2]   33000 || B: 4.055 | C: 5.763 | M: 4.799 | S: 1.412 | T: 16.028 || ETA: 1 day, 4:25:51 || timer: 0.128
Traceback (most recent call last):
File "/home/gangwa/miniconda3/lib/python3.9/multiprocessing/queues.py", line 245, in _feed
obj = _ForkingPickler.dumps(obj)
File "/home/gangwa/miniconda3/lib/python3.9/multiprocessing/reduction.py", line 51, in dumps
cls(buf, protocol).dump(obj)
File "/home/gangwa/miniconda3/lib/python3.9/site-packages/torch/multiprocessing/reductions.py", line 364, in reduce_storage
shared_cache[cache_key] = StorageWeakRef(storage)
File "/home/gangwa/miniconda3/lib/python3.9/site-packages/torch/multiprocessing/reductions.py", line 65, in setitem
self.free_dead_references()
File "/home/gangwa/miniconda3/lib/python3.9/site-packages/torch/multiprocessing/reductions.py", line 70, in free_dead_references
if storage_ref.expired():
File "/home/gangwa/miniconda3/lib/python3.9/site-packages/torch/multiprocessing/reductions.py", line 35, in expired
return torch.Storage._expired(self.cdata)  # type: ignore[attr-defined]
File "/home/gangwa/miniconda3/lib/python3.9/site-packages/torch/storage.py", line 757, in _expired
return eval(cls.module)._UntypedStorage._expired(*args, **kwargs)
AttributeError: module 'torch.cuda' has no attribute '_UntypedStorage'
Traceback (most recent call last):
File "/home/gangwa/miniconda3/lib/python3.9/multiprocessing/queues.py", line 245, in _feed
obj = _ForkingPickler.dumps(obj)
File "/home/gangwa/miniconda3/lib/python3.9/multiprocessing/reduction.py", line 51, in dumps
cls(buf, protocol).dump(obj)
File "/home/gangwa/miniconda3/lib/python3.9/site-packages/torch/multiprocessing/reductions.py", line 364, in reduce_storage
shared_cache[cache_key] = StorageWeakRef(storage)
File "/home/gangwa/miniconda3/lib/python3.9/site-packages/torch/multiprocessing/reductions.py", line 65, in setitem
self.free_dead_references()
File "/home/gangwa/miniconda3/lib/python3.9/site-packages/torch/multiprocessing/reductions.py", line 70, in free_dead_references
if storage_ref.expired():
File "/home/gangwa/miniconda3/lib/python3.9/site-packages/torch/multiprocessing/reductions.py", line 35, in expired
return torch.Storage._expired(self.cdata)  # type: ignore[attr-defined]
File "/home/gangwa/miniconda3/lib/python3.9/site-packages/torch/storage.py", line 757, in _expired
return eval(cls.module)._UntypedStorage._expired(*args, **kwargs)
AttributeError: module 'torch.cuda' has no attribute '_UntypedStorage'
[  2]   33010 || B: 4.178 | C: 5.768 | M: 4.934 | S: 1.417 | T: 16.296 || ETA: 1 day, 4:49:09 || timer: 0.126
Traceback (most recent call last):
File "/home/gangwa/miniconda3/lib/python3.9/multiprocessing/queues.py", line 245, in _feed
obj = _ForkingPickler.dumps(obj)
File "/home/gangwa/miniconda3/lib/python3.9/multiprocessing/reduction.py", line 51, in dumps
cls(buf, protocol).dump(obj)
File "/home/gangwa/miniconda3/lib/python3.9/site-packages/torch/multiprocessing/reductions.py", line 364, in reduce_storage
shared_cache[cache_key] = StorageWeakRef(storage)
File "/home/gangwa/miniconda3/lib/python3.9/site-packages/torch/multiprocessing/reductions.py", line 65, in setitem
self.free_dead_references()
File "/home/gangwa/miniconda3/lib/python3.9/site-packages/torch/multiprocessing/reductions.py", line 70, in free_dead_references
if storage_ref.expired():
File "/home/gangwa/miniconda3/lib/python3.9/site-packages/torch/multiprocessing/reductions.py", line 35, in expired
return torch.Storage._expired(self.cdata)  # type: ignore[attr-defined]
File "/home/gangwa/miniconda3/lib/python3.9/site-packages/torch/storage.py", line 757, in _expired
return eval(cls.module)._UntypedStorage._expired(*args, **kwargs)
AttributeError: module 'torch.cuda' has no attribute '_UntypedStorage'
Traceback (most recent call last):
File "/home/gangwa/miniconda3/lib/python3.9/multiprocessing/queues.py", line 245, in _feed
obj = _ForkingPickler.dumps(obj)
File "/home/gangwa/miniconda3/lib/python3.9/multiprocessing/reduction.py", line 51, in dumps
cls(buf, protocol).dump(obj)
File "/home/gangwa/miniconda3/lib/python3.9/site-packages/torch/multiprocessing/reductions.py", line 364, in reduce_storage
shared_cache[cache_key] = StorageWeakRef(storage)
File "/home/gangwa/miniconda3/lib/python3.9/site-packages/torch/multiprocessing/reductions.py", line 65, in setitem
self.free_dead_references()
File "/home/gangwa/miniconda3/lib/python3.9/site-packages/torch/multiprocessing/reductions.py", line 70, in free_dead_references
if storage_ref.expired():
File "/home/gangwa/miniconda3/lib/python3.9/site-packages/torch/multiprocessing/reductions.py", line 35, in expired
return torch.Storage._expired(self.cdata)  # type: ignore[attr-defined]
File "/home/gangwa/miniconda3/lib/python3.9/site-packages/torch/storage.py", line 757, in _expired
return eval(cls.module)._UntypedStorage._expired(*args, **kwargs)
AttributeError: module 'torch.cuda' has no attribute '_UntypedStorage'
`
Anyone has any idea why I got this after around 1-2 hours of training.
Thanks

This repo did not updated 3 years.
It's better for you to train with yolov8-seg

I am able to make it run but getting very slow training speed

Solution -
Don't use torch1.12. Either upgrade or degrade the version of torch with cuda.

authors: abd-gang sdimantsd

repo: dbolya/yolact

license: MIT


text:
Oauth/IClaimsPrinicpal collector

gab the current IClaimsPrinicpal for a ASPNET application
extract the name and id, using the approved XML namespaces

https://github.com/dbones-labs/auditable/commit/d88e68c6e2a46e7919ef6cc36458f1c36f5f743e

repo: dbones-labs/auditable


text:
Add support for concrete nested serializers

This is a proposal to add support for editing the fields of nested serializers not on a per-request basis, but at instantiation time.
Here is an example. There is a serializer for user called UserSerializer it has rather a lot of fields
class UserSerializer(...):
 class Meta:
  fields = ('id', 'url', 'name', 'email', 'accounts', 'friends', 'most_recent_activity')

The UserSerializer is nested inside another serializer
class MessageSerializer(...):
 from_users = UserSerializer(many=True, read_only=True)
 class Meta:
  fields = ('id', 'url', 'from_users', 'to_users', 'account', 'created_at', 'modified_at')

But we only want a few bits of info for each user in the MessageSerializer, just name, email, id and url. Just enough context to help the front end render without relying on a user lookup.
This proposal is made to solve that situation:
class MessageSerializer(...):
 from_users = UserSerializer(many=True, read_only=True, fields=('id', 'url', 'name', 'email'))

I have already coded something like this up, and can I can see how there is some overlap with this project. Enough to justify putting it in, I think. Unfortunately it doesn't directly support the purpose of this project, which is dynamic per-request fields. This is more like dynamic fields at runtime.

I'm not interested in this feature anymore. Nested ModelSerializers pay a significant penalty when instantiated. This was my main use-case. Instead I create a normal Serializer for my nested serializers. The performance benefits are huge.
In case anyone was wondering, it is the get_fields function in ModelSerializer that is particularly taxing.

repo: dbrgn/drf-dynamic-fields


text:
Upgrade Jinja2 dependency version specification to address CVE-2024-22195

resolves CVE-2024-22195

Description
CVE-2024-22195 identified an issue in Jinja2 versions <= 3.1.2. As such we've gone and changed our dependency requirement specification to be 3.1.2 or greater (but less than 4).
Note: Preivously we were using the ~= version specifier. However due to some issues with the ~= we've moved to using >= in combination with <. This gives us the same range that ~= gave us, but avoids a pip resolution issue when multiple packages in an environment use ~= for the same dependency.
Checklist

[x] I have read the contributing guide and understand what's expected of me
[x] I have signed the CLA
[x] I have run this code in development and it appears to resolve the stated issue
[x] I have opened an issue to add/update docs, or docs changes are not required/relevant for this PR
[x] I have run changie new to create a changelog entry

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 54.04%. Comparing base (c61d318) to head (7b3f164).

Additional details and impacted files
@@           Coverage Diff           @@
##             main      #85   +/-   ##
=======================================
  Coverage   54.04%   54.04%
=======================================
  Files          49       49
  Lines        2866     2866
=======================================
  Hits         1549     1549
  Misses       1317     1317

Flag
Coverage Δ

unit
54.04% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

source: gharchive/pull-request

authors: QMalcolm codecov-commenter

repo: dbt-labs/dbt-common


text:
[CT-794] Export Lineage Graph as SVG or PDF

Describe the feature
Problem: if we want to generate an image for our Lineage Graph, there is the option to export the graph as PNG, but this image has no quality options.
Example:

To add more flexibility, dbt can allow generating not just the PNG, but an SVG or PDF file with vectorized images. Reference: Scalable Vector Graphics
Describe alternatives you've considered
You can try to add PNG configurations as ppi and dimensions, but a scalable image format is a more general solution.
Who will this benefit?

Users who want to use Lineage Graphs in slides or documentations.
Users who want to edit Lineage Graphs using graphic design programs as Illustrator.

Are you interested in contributing this feature?
Maybe, but I'm not so familiar with the dbt repo.

@nelsoncardenas Sorry for the delay getting back to you!
The logic for the graph export is neatly self-contained in just a few lines of code. We use the cytoscape library's built-in .png() function to create a PNG here:
https://github.com/dbt-labs/dbt-docs/blob/85dec858c5d213699fbc2cefa388ba1e80c94889/src/app/components/graph/graph-viz.js#L210-L214
It looks like the cytoscape library has built-in support for PNG, JPG, and JSON as export options (no SVG): https://js.cytoscape.org/#core/export
But it also looks like someone has developed an extension to the cytoscape library, for SVG exports: https://github.com/kinimesi/cytoscape-svg
Is that something you'd be interested in experimenting with?

@nelsoncardenas  @jtcohen6 Is this still open?

@abhijithp05 sorry, I have been busy, and I don't think I'll have any time soon to work on this problem.

@nelsoncardenas I have issue while running the project related to assets/css reference.

@nelsoncardenas Can you tell me where to find the export button

@nelsoncardenas Created a PR for the issue. Please review. and merge

repo: dbt-labs/dbt-docs


text:
ADAP-56: Python 3.11 Support

resolves #524
Description
Add support for Python 3.11
Checklist

[x] I have read the contributing guide and understand what's expected of me
[x] I have signed the CLA
[ ] I have run this code in development and it appears to resolve the stated issue
[x] This PR includes tests, or tests are not required/relevant for this PR
[ ] I have opened an issue to add/update docs, or docs changes are not required/relevant for this PR
[x] I have run changie new to create a changelog entry

We have a better way of doing this.

We have a better way of doing this.

@mikealfare Can you link to the better way if ready please?

We have a better way of doing this.

@mikealfare Can you link to the better way if ready please?

Fair point. Here's the PR we merged: https://github.com/dbt-labs/dbt-spark/pull/818

authors: followingell mikealfare

repo: dbt-labs/dbt-spark


text:
A large enterprise moving from Core to Cloud to enable more dbt developers

Contact Details
@boxysean
I have read the dbt Developer Blog contribution guidelines.

[X] I have read the dbt Developer Blog contribution guidelines.

Which of these best describes you?

[ ] I am a dbt Community member or partner contributing to the Developer Blog
[X] I work for dbt Labs and am creating this issue for a community or marketing approved piece.

What is the topic of your post?
This post is a success story targeted towards dbt Core users at large enterprises who are looking to expand their dbt usage by adopting dbt Cloud. It will include key technical challenges and solutions used to solve them that others can follow.
Link to an initial outline.
https://www.notion.so/dbtlabs/848888d520f541a78c11e9e147a31581

Hey @boxysean - this is an awesome topic for a post. We have been wanting to do a guide on moving for Core to Cloud for a while and I still think we should do that, but starting with a single example and going deep makes a ton of sense. Let's plan on you, me and @dave-connors-3 spending some time digging into this in early Jan.

authors: boxysean jasnonaz


text:
add an EAX preprint badge

cosmetic

ta very mooch

authors: dbuscombe-usgs ebgoldstein


text:
thrimshim: use send_file to serve templates from database

I don't claim to understand why this works, but this makes downloading the thumbnail templates (to preview on the thumbnail management page, or to select the "hole" in the advanced crop settings in the editor) way faster. Something to do with how flask is chunking the memoryview object to serve.
Applying this changed the timings to download "fiddling.png" from 818ms waiting and 6770ms receiving, to 777ms waiting and 14ms receiving.
Resolves #458

It will not spot all Python bugs but running a linter such as Pyflakes before pushing is good practice.

ekim has a better fix (bytes(image) apparently is enough), closing

authors: chrusher dcollinsn

repo: dbvideostriketeam/wubloader


text:
add stronger typing to poetic plotting, fixes #882

Related to #881 and #871

Coverage increased (+0.2%) to 65.67% when pulling 47336cb3de67c6fdff901d0d08834033d0ce758b on tlnagy:pull-request/24f5d936 into 4a3683797227463b2a5bb4f736ce64fc19fd016d on dcjones:master.

Coverage increased (+0.02%) to 65.458% when pulling 47336cb3de67c6fdff901d0d08834033d0ce758b on tlnagy:pull-request/24f5d936 into 4a3683797227463b2a5bb4f736ce64fc19fd016d on dcjones:master.

Wow this is a pretty action at a distance kind of problem. I guess this fix is warranted though.

Indeed. I would've preferred a different workaround to this (something more like #874), but that was a lucky fix. However, this is a much more robust fix for this error and it looks like the tests pass. Hopefully no one is passing anything too funky to the a and b parameters.
Also, based on your suggestion in #882, I added a test for the ambiguity method error.

Coverage increased (+0.02%) to 65.458% when pulling 8804dde2af4781692cacbdd0333a291ec1762e81 on tlnagy:pull-request/24f5d936 into 4a3683797227463b2a5bb4f736ce64fc19fd016d on dcjones:master.

Coverage increased (+0.02%) to 65.458% when pulling 8804dde2af4781692cacbdd0333a291ec1762e81 on tlnagy:pull-request/24f5d936 into 4a3683797227463b2a5bb4f736ce64fc19fd016d on dcjones:master.

Hopefully no one is passing anything too funky to the a and b parameters.

julia> brightness(x::RGB) = (x.r+x.g+x.b)/3
brightness (generic function with 1 method)

julia> plot([brightness], colorant"black", colorant"white")
ERROR: MethodError: `isless` has no method matching isless(::ColorTypes.RGB{FixedPointNumbers.UFixed{UInt8,8}}, ::ColorTypes.RGB{FixedPointNumbers.UFixed{UInt8,8}})
Closest candidates are:
  isless(::DataArrays.NAtype, ::Any)
  isless(::Any, ::DataArrays.NAtype)
 in plot at /home/shashi/.julia/v0.4/Gadfly/src/poetry.jl:44

haha.
yeah, if someone's using funky a and b then they can open an issue.

authors: coveralls shashi tlnagy

license: mit

license_source: bigquery


text:
Try rack cas

This branch removes devise and devise_cas_authenticatable in favor of the rack-cas gem. The devise_cas_authenticatable gem relies on the ruby-cas client gem, which is no longer maintained and produces deprecation warnings in Rails 6.
Aside from the initial setup for rack-cas, the branch introduces 4 new methods into the Application Controller:

authenticate_user! - directs visitors to either the login or the unregistered page
set_current_user - updates the login stats for a user and sets a @current_user, if available
get_current_user - gets the user from a CAS session
current_user - helper method for use in views

In order to maintain the user statistics that Devise's trackable module provided, the update_login_stats method has been introduced on the User model. This method gets called in the set_current_user method. A sessions controller was also introduced to handle closing these out before sending the user to the CAS logout path.
The authenticated root we used with Devise has been replicated using a condition in StaticPagesController#home
The test suite has been updated to reflect these changes and should be passing.
To handle visitors that may be authenticated at the central level by CAS but are not registered in the app, an unregistered path has been introduced. The replicates devise_cas_authenticatable's unregistered method but provides a custom layout.

In manual testing, a registered user should see no difference in their experience with the app. An unregistered user should now get the custom unregistered page.
Definitely appreciate the additional eyes on this!! :)

I think this is good to go. I manually resolved conflicts in the gem lock file.


text:
Doc changes for installing DCOS v1.9.0 on Google GCE

Description

Urgency

[ ] Blocker
[ ] High
[x] Medium

Requirements

Test all commands and procedures.
Build content locally and test for formatting/links.
Add redirects to dcos-website/redirect-files.
Change all affected versions (e.g. 1.7, 1.8, and 1.9).
See the contribution guidelines.

Can one of the admins verify this patch?

Please can somebody review the changes?

I think the dcos_installer_filename usage is just confusing here. If you change it to dcos_generate_config.sh everywhere it'll be easier to follow and copy/paste.

Alright. this is fine, but needs to be rebased.

Can one of the admins verify this patch?

Hello @karlkfi, I'm not sure I've followed the correct process here

Sorry for the slow turnaround! I thought I had already merged this...

authors: ajazam karlkfi mesosphere-ci

repo: dcos/dcos-docs


text:
Prevent stop action for scheduler tasks

Scheduler tasks don't like to be stopped so we hide the stop button if one is selected.
@leemunroe - should we hide this button or disable it? If we disable it, I presume we'd also want a tooltip to explain why.

https://mesosphere.atlassian.net/browse/DCOS-10693

@MatApple Good point. I think disabled with a tooltip is a better UX in this case. Stopping a scheduler task is not supported.

Thanks @leemunroe - will make the appropriate changes.

@jfurrow - I'm having trouble getting the tooltip to work with the "Stop" button. I've tried wrapping the button with the Tooltip and also having the Tooltip as the immediate child of the button. Either way, the button doesn't like it and the Tooltip doesn't work correctly - especially with the "disabled" attribute on the button. Any suggestions? Thanks

@mesosphere-ci retest this please

@MatApple Functionally looks good but why does the disabled state have a grey background and hover state? It makes it stand out more 😪
We should remove the background if disabled. And no hover state needed (i.e. no underline).

@leemunroe - completely agree, the button is weird. Talked to @ashenden about this. Instead of adding custom CSS, we want to handle this by updating the button styles for the disabled state in CNVS.

authors: MatApple leemunroe

repo: dcos/dcos-ui


text:
fix(TasksView): disable restart from tasksViews

Disable restart in TasksView when is SDK, this is part of #2343
Closes DCOS-16564
Checklist

[x] Did you add a JIRA issue in a commit message or as part of the branch name?
[ ] Did you add new unit tests?
[ ] Did you add new integration tests?
[ ] If this is a regression, did you write a test to catch this in the future?

Nice catch @bstavroulakis @weblancaster 👍

authors: MatApple weblancaster


text:
DCOS-52747 Disable package installation until "manager" requirement is satisfied

https://jira.mesosphere.com/browse/DCOS-52747
⚠ depends on https://github.com/dcos/dcos-ui/pull/3864, you might want to get that merged first.
some packages (currently only kubernetes-cluster) have a manager that must be
installed before they can be used. currently the API restricts the installation
of kubernetes-cluster if kubernetes is not installed yet. you have to go through
the whole installation-process in the UI to afterwards be notified that stuff
did not work. as a remedy a check has been implemented that renders an infobox
with a nice warning as well as the reviewAndRun-button disabled - so now we
can't waste a lot of work in the first place when we don't know about
dependencies upfront.
Review
The functionality has been implemented in two steps (commit). The first commit (i mean the one that starts with refactor) refactors geReviewAndRunButtons -> renderReviewAndRunButton in the PackageDetailTab as i can not deal with lets somehow... it also does some dummy-changes that cause huge whitespace-changes.
the second commit should contain all the logic related to dependency-checking. it should enable to get an overview of the approach taken.
Testing
Try to install kubernetes-cluster on a cluster that does not have a kubernetes service running to see the warning. install kubernetes and confirm that the message disappears (it should disappear in realtime as soon as the kubernetes-task has the state "TASK_RUNNING").
To make the warning show up programatically you can make hasUnresolvedDependency return true.

@TattdCodeMonkey I did notice that we're checking that the manager package exists, but not the status or it. So if its deploying you will get an error when trying to run the package with the dependency.

We're currently looking for a TASK_RUNNING, which was the best i could come up with. There seems to be no accurate service status for the kubernetes package yet, so this currently is the best we can do, right? would it be ok for you, if i opened an issue that reminds us of implementing a more sophisticated check once a package with a manager implements those statuses?
i'll happily implement more/different checks though if you have something in mind!

Finding it disquieting that the cosmos system test failed here. Could be relevant! 👀 I'll watch it

Frustrating because they pass locally

@pierrebeitz Cypress can't seem to complete mesos stream proxy requests so all the universe system tests fail with this change. So in order to preserve the universe system tests, I've added a commit that modifies the PackageDetailsTab to consult the DCOSStore (marathon groups endpoint) to check for an installed kubernetes package.
Unfortunately, this is a worse solution because "kubernetes" could appear to be satisfied even if it has failed to start. This can result in the failure to install kubernetes-cluster. I don't think this is a huge deal because in the end we give the user an appropriate error message after kubernetes-cluster fails to install.
Another alternative is to disable universe system tests but I opted to not do this.
Another thought I had was that there's not enough information to figure out if "manager dependencies" are truly satisfied. Would it require that all or any tasks of that package be running?  For kubernetes, this is not an issue because there's only one task.

The button looks disabled, but I can still click it. Is this intentional?

@brandonc , I approve of your current solution.

The button looks disabled, but I can still click it. Is this intentional?

no, not at all! should be fixed now!

Why do we have "kubernetes" in quotation marks?

apparently i worked with a wrong design doc in the beginning. removed them!

I ran "kubernetes", tried to install "kubernetes-cluster", but got this error. Maybe we need to show the infobox in case "kubernetes" isn't configured correctly?

that seems to be an error from the server. any idea on how to find out whether kubernetes is configured correctly upfront?

Are there designs for the infobox in this case? I think gray is sort of easily ignored color, I think yellow or red are better.

@mperrotti talked to design. they want it to be gray. here's what i consider the design doc: https://mesosphere.invisionapp.com/share/2TRTTM5SZQX#/screens/361260526_k8cluster-Details-Page-Disabled

@pierrebeitz , thank you for addressing my feedback and for the quick response.

I have one concern though. If our logic for checking if the dependency is somewhat incorrect, we prevent the user from installing a certain package. I think we should just display a warning. We have a server error in case the user insists in trying to install the package.

Also, the tooltip message isn't in the message catalog and cannot be translated.

I have one concern though. If our logic for checking if the dependency is somewhat incorrect, we prevent the user from installing a certain package. I think we should just display a warning. We have a server error in case the user insists in trying to install the package.

i really admire that idea! we need to talk to design about this.

Also, the tooltip message isn't in the message catalog and cannot be translated.

@GeorgiSTodorov updated the fixup-commit

@GeorgiSTodorov @TattdCodeMonkey the ServiceTree().getLabels method flattens out those running labels so we should be able to detect. The only thing we can't detect is whether or not the underling task requirement is satisfied. But I think that is a limitation of the API.

@pierrebeitz I think this is ready to go

I'm merging this because the "installed" requirement seems like the correct one to focus on since there are many scenarios we can't detect and this is meant as a helpful shortcut for someone who definitely does not have kubernetes installed before installing kubernetes-cluster

:tada: This PR is included in version 2.96.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

authors: GeorgiSTodorov brandonc mesosphere-ci pierrebeitz


text:
[master] Automated bump DC/OS UI v2.40.8

This bump contains the following JIRAs:
DCOS-14285 DCOS-46169

@mesosphere-mergebot bump-ee

@mesosphere-mergebot label [Ready for Review]

repo: dcos/dcos


text:
[1.12][DCOS-45428] Bump cosmos to latest version

High-level description
What features does this change enable? What bugs does this change fix?
Backport of #4409
Corresponding DC/OS tickets (obligatory)
These DC/OS JIRA ticket(s) must be updated (ideally closed) in the moment this PR lands:

DCOS-45428 Cosmos decodes URL parameter and it breaks resource links

Related tickets (optional)
Other tickets related to this change:

DCOS_OSS- Foo the Bar so it stops Bazzing.

Checklist for all PRs

[x] Added a comprehensible changelog entry to CHANGES.md or explain why this is not a user-facing change: No user visible changes.
[x] Included a test which will fail if code is reverted but test is not. If there is no test please explain here: No test included in dcos integration tests. However, package installation for dcos-ui would fail if code is reverted.
[x] Read the DC/OS contributing guidelines
[x] Followed relevant code rules Rules for Packages and Systemd

Checklist for component/package updates:
If you are changing components or packages in DC/OS (e.g. you are bumping the sha or ref of anything underneath packages), then in addition to the above please also include:

[x] Change log from the last version integrated (this should be a link to commits for easy verification and review): View diff
[x] Test Results: link to CI job test results for component
[x] Code Coverage (if available): link to code coverage report

PLEASE FILL IN THE TEMPLATE ABOVE / DO NOT REMOVE ANY SECTIONS ABOVE THIS LINE
Instructions and review process
What is the review process and when will my changes land?
All PRs require 2 approvals using GitHub's pull request reviews.
Reviewers should be:

Developers who understand the code being modified.
Developers responsible for code that interacts with or depends on the code being modified.

It is best to proactively ask for 2 reviews by @mentioning the candidate reviewers in the PR comments area. The responsibility is on the developer submitting the PR to follow-up with reviewers and make sure a PR is reviewed in a timely manner. Once a PR has 2 ship-it's, no red reviews, and all tests are green it will be included in the next train.

@mesosphere-mergebot bump-ee

@mesosphere-mergebot label Ready For Review


text:
[master] Bump Mesos to nightly 1.8.x 85462fc

High-level description
This is a routine bump to the latest Mesos and mesos-modules.
Related JIRA Issues
Checklist for all PRs

[ ] Included a test which will fail if code is reverted but test is not. If there is no test please explain here:
[x] Read the DC/OS contributing guidelines
[x] Followed relevant code rules Rules for Packages and Systemd

Checklist for component/package updates:
If you are changing components or packages in DC/OS (e.g. you are bumping the sha or ref of anything underneath packages), then in addition to the above please also include:

[x] Changelog:
https://github.com/apache/mesos/compare/0c503b01d3a9428ec9db35d09da5e237d737c570...85462fc183a60ae18d85729bccb1fffb59aa572c

[ ] Test Results: [link to CI job test results for component]

[ ] Code Coverage (if available): [link to code coverage report]

@mesosphere-mergebot bump-ee


text:
exhibitor: bump

High-level description
What features does this change enable? What bugs does this change fix?
Corresponding DC/OS tickets (required)

D2IQ-ID JIRA title / short description.

Related tickets (optional)

D2IQ-ID JIRA title / short description.

@mesosphere-mergebot test teamcity/dcos/build/dcos teamcity/dcos/build/tox

@mesosphere-mergebot test all

@mesosphere-mergebot test all

@mesosphere-mergebot test all


text:
Improve suggested tags

We want to add improved weighting that gives better suggestions.
In order of bias where tags are seen:

Bottle
Brand/Distillery
Region
Country

Added randomization to the list so at least selection bias goes away. Currently this is only weighted by Bottle, and likely requires tags to be materialized/indexed to do more.

repo: dcramer/peated


text:
[WIP] Added decode tx page and view raw link

For issues #190 and #136 . The rpc api has no "getrawblock" function so that can't be added, but the view raw link works as intended now, showing the txid and hex while the view decoded link shows the json output. The decode tx pages works as intended as well but doesn't look so good for now.

Just pushed up a bit more styling for this https://github.com/dcrdata/dcrdata/tree/raw-tx

@gozart1 thanks looks pretty good right now. @chappjc this is ready for merge.

Might be nice to pass the rejection reason out to the UI.
In the server log I see:
Rejected transaction 005969df2bc92ac33b97255521762ec4d69a27d6188e958c3246f01856d78259: transaction already exists

But the UI currently just shows
Error: Could not send hex
Try decoding it first to see if it's valid

Added that

We need to be cautious with accepting and processing external data.  Some thoughts:

broadcast needs to be rate limited (per IP and server-wide)
websocket input needs to somehow stop receiving data after X bytes (otherwise an attacker can send an infinite data stream, eating bandwidth and eventually filling up the server's memory)
the html form should screen the data before sending, but an attacker would just ignore the form and send directly to the websocket
the decode handler needs to have a final check on the size of the data before decoding it

Rate limiting may not be needed as the node won't broadcast junk or duplicates, and txns cost $$.

Took a shot at abusing this.  My initial attempt was passing 460,000 valid hex characters to decode transaction.  It completely crippled dcrdata.  Local network request, just major CPU usage, forever.
I assume it's at least as costly to do this to send transaction.

I started with some valid transaction, like this one:

01000000020000000000000000000000000000000000000000000000000000000000000000ffffffff00ffffffffc0c343c6fb9ed34f26dd6a2bc7e233f22f82b4da4f363a7c5d437f7a555d30c60000000001ffffffff0400000000000000000000266a2463769d4b15965779ec9e542f888848426ec49dd80c0d175f3500000000000000b8df020000000000000000000000086a06010005000000a6e567000000000000001abb76a914a92c9ac541dd5ac40c630e6659ecce25e9fdd70e88aca53471d90100000000001abb76a91454ee8f3f4ceb3dfbdf38b3f55ec34c318c55b65c88ac0000000000000000028ffa46080000000000000000ffffffff020000bd1f92d10100000082d702000600000091483045022100f317885cfed85eb7355ea25a6fee0125b5fcd81afdcad91071a8e8ecc991f3ee02201683600dedd352d88a2ae0e11a5550ddfde68f87d53f9e072963f984f3bb8b1d014751210306559376b41006b6e16341b003c8d957cd3974ec665cea4e932826d9e7f1c7c72102f536ba2b34501eb3d3522c4397272d99803223087034a04f687ac6424c5aa8b652ae

Then I pasted that in the text box, holding down CTRL+V for a few seconds, then select all, and paste for another couple of seconds.  😆

We also need to limit the length of the hex logged: Received decodetx signal for hex:  .
But what I was seeing is the process chugging away with multiple CPU cores after printing the offending hex.

It seems easy to clog up the websocket.
2017-11-21 17:25:27.351 [ERR] SQLT: SendRawTransaction failed: -1: Rejected transaction 24a719667306c2284af9cb564be3178417bd4906da09746d3f73ed34df94b2be: transaction already exists
2017-11-21 17:25:27.351 [DBG] EXPR: Failed to encode WebSocketMessage decodedtx: write tcp 127.0.0.1:7777->127.0.0.1:44582: i/o timeout
2017-11-21 17:25:27.921 [DBG] EXPR: Failed to encode WebSocketMessage 3: write tcp 127.0.0.1:7777->127.0.0.1:44582: i/o timeout
2017/11/21 17:25:27 "GET http://127.0.0.1:7777/explorer/decodetx/ws HTTP/1.1" from 127.0.0.1:44582 - 000 0B in 1m48.000770061s

I'm just clicking both buttons like a maniac.

Going in the right direction, but it still chokes.  Try this:  https://pastebin.com/raw/FRAfMiqb

@RogueElement nice job with this.  I'm just being picky on this because of the attack surface it exposes.

authors: RogueElement chappjc gozart1

repo: dcrdata/dcrdata

license: isc


text:
🛑 Danbs is down

In 0ee1e52, Danbs (https://danbs.net) was down:

HTTP code: 404
Response time: 171 ms

Resolved: Danbs is back up in 5a6f73d after 6 minutes.

repo: ddanbs/upptime


text:
🛑 Danbs is down

In ec89686, Danbs (https://danbs.net) was down:

HTTP code: 404
Response time: 234 ms

Resolved: Danbs is back up in 48d105e after 13 minutes.


text:
🛑 Danbs is down

In 5043183, Danbs (https://danbs.net) was down:

HTTP code: 404
Response time: 3785 ms

Resolved: Danbs is back up in a15b34d after 15 minutes.


text:
🛑 Danbs is down

In f3e6b13, Danbs (https://danbs.net) was down:

HTTP code: 404
Response time: 169 ms

Resolved: Danbs is back up in 8d055a7 after 37 minutes.


text:
🛑 Danbs is down

In c45545c, Danbs (https://danbs.net) was down:

HTTP code: 404
Response time: 146 ms

Resolved: Danbs is back up in 26941a9 after 13 minutes.


text:
🛑 Danbs is down

In 98fdee2, Danbs (https://danbs.net) was down:

HTTP code: 404
Response time: 195 ms

Resolved: Danbs is back up in 7494774.


text:
🛑 Danbs is down

In 09b4627, Danbs (https://danbs.net) was down:

HTTP code: 404
Response time: 146 ms

Resolved: Danbs is back up in e8a976b.


text:
https://remiguieditor.daviderosa.repl.co/ no module named remi

Hello, just letting you know that https://remiguieditor.daviderosa.repl.co/ is not loading :)

@awesomebytes thank you a lot for the info, I will now fix

@awesomebytes  Fixed thank you ;-)

It seems that because of recent repl.it updates it is not possible to run the editor in a stable way. I removed that link from the readme. In the future I will look for a different solution.

authors: awesomebytes dddomodossola

repo: dddomodossola/remi


text:
Item typehint is strictly array in Writers

So, if the item can be an array, object or anything then shouldn't we remove the array typehint from the Writer interface? Because currently it only accepts arrays.

In other words: do we have to convert the item to an array when it arrives at the Writer?

It's not really easy to remove this, because some writers accept only arrays. I removed some month the array type hints from the converts/items.
For example the ExcelWriter use phpexcel to create the spreadsheet. And i have no idea how we can extract the properties.

In this case, shouldn't we make the assumption that by the end of the process (aka the workflow gets to the writer part) the item should be converted into an array?
I think we should clear this (in the documentation) so that everyone knows why is this.
See this library: https://github.com/plumphp/plum/
It is inspired by data-import and it says it accepts any kind of data.

I am leaving this open as a reminder for documentation.

See this library: https://github.com/plumphp/plum/

Interesting, I didn’t know about plumphp. I’ll contact Florian and see whether it makes sense to combine our efforts.

In this case, shouldn't we make the assumption that by the end of the process (aka the workflow gets to the writer part) the item should be converted into an array?

I think we should clear this (in the documentation) so that everyone knows why is this.

Agreed. I decided to have arrays as the data element format: most readers can output it, most writers can handle it (and some, such as PHPExcel require it).

authors: Baachi ddeboer sagikazarmark

repo: ddeboer/data-import


text:
Cardinalities missing in XMI export

There are a few cardinalities in the content, that are not included in the output as XMI. One example is the property target cardinality 2..2 of the propperts maps in AgentSimilarityPair.

some more examples on: https://ddi-alliance.atlassian.net/browse/DMT-108

as i wrote in DMT-108, is this just a matter of adding 2..2 in the list of cardinalities?

deployed https://github.com/ddialliance/lion/commit/6b7b1c7e1ec9f5bb2124430b2064edb0d919b6a1 to production. @OliverHopt is this resolved?

Solved

authors: OliverHopt borsna

repo: ddialliance/lion


text:
Foreman in production: how to preload rails app?

I've been toying with using a kind of rails app preloader to share memory between process types. In a rails environment, one uses rails to preload the code. In theory, a foreman engine could access the environment.rb file and use it to preload it, before forking the different web/worker/etc processes. This would be therefore not a lot different than what the unicorn and puma (in cluster mode) do.
The caveat is that it seems that foreman forks shell commands (which will later load different instances of the VM). Therefore, this:
class Foreman::Engine::RailsCLI < Foreman::Engine::CLI
  def startup
    super
    require File.expand_path('../../config/environment', __FILE__)
  end
end

is not enough.
Has anyone ever played around with such a thing? This is a rails-specific solution, so I don't hope to see it in foreman (or should I? Ideally every framework could create its own foreman engine and script).
I can envision that instead of Process.spawn, one would use fork.

I'd prefer not to add things like this to foreman proper. If you want to combine processes together do it one level before foreman as a single process.

Well, I also agree that this isn't a foreman concern. Nevertheless, I was eyeing a workflow that could be "lobbied" into frameworks like rails, i.e. a kind of foreman engine that each framework could implement and reuse. I post here my POC for Rails:
# bin/foreman-rails
#!/usr/bin/env ruby
require 'foreman/cli'
module Foreman
  class Engine::RailsCLI < Engine::CLI
    def startup
      super
      require File.expand_path('../../config/environment', __FILE__)
    end

    def register(name, command, options={})
      options[:env] ||= env
      options[:cwd] ||= File.dirname(command.split(" ").first)
      process = RailsProcess.new(command, options)
      @names[process] = name
      @processes << process
    end
  end

  class RailsCLI < CLI
    no_tasks do
      def engine
        @engine ||= begin
          engine_class = Engine::RailsCLI
          engine = engine_class.new(options)
          engine
        end
      end
    end
  end

  class RailsProcess < Process
    def run(options={})
      env    = @options[:env].merge(options[:env] || {})
      output = options[:output] || $stdout
      runner = "#{Foreman.runner}".shellescape
      final_command = expanded_command(env)

      Dir.chdir(cwd) do
        fork do
          env.each do |k, v|
            ENV[k] ||= v
          end

          log_args = output.is_a?(IO) ? [output] : [output, 'w']
          $stdout.reopen(*log_args)
          $stderr.reopen(*log_args)

          argv = final_command.split(/\s+/).reject { |s| %w(bundle exec).include?(s) }
          executable = argv.shift

          ARGV.clear
          argv.each do |v|
            ARGV << v
          end
          load Bundler.which(executable)
        end
      end
    end
  end
end

Foreman::RailsCLI.start

This works... mostly. Basically by loading rails environment.rb, I'm eagerloading/kickstarting the initialization process, which is interpreted as "web process is starting" by some dependencies, which take decisions based on whether the current process is the web process or not (I have problems with sidekiq because of that). This is mainly because Rails itself doesn't provide proper hooks to signalize this IMO. The memory savings however, would be huge, if such a process could be implemented.
I've created this discussion, but sadly not a lot of follow up, maybe I'll reopen it as an issue.

repo: ddollar/foreman


text:
set -o nounset causes unbound variable warnings

With source ~/.hishtory/config.sh in .bashrc.
$> set -u
-bash: HISHTORY_AT_PROMPT: unbound variable
-bash: HISHTORY_FIRST_PROMPT: unbound variable

Reason:
https://github.com/ddworken/hishtory/blob/cc123854a02374a7e4ee7fc87a974b19566fa142/client/lib/config.sh#L10
Solutions: https://stackoverflow.com/questions/7832080/test-if-a-variable-is-set-in-bash-when-using-set-o-nounset

This should be fixed! If you run hishtory update you'll get the latest version with the fix. If you're still experiencing this issue (or run into anything else!) please reopen this so I can take another look.

Seems to be working; thanks for the quick fix @ddworken!

repo: ddworken/hishtory


text:
feat: 🎸 Support passing through tags

Adding more flexibility by adding the ability to add tags to all used resources that support tag metadata - useful for billing/project grouping etc.

Released in v0.5.2.

Released in v0.5.2.

authors: maael ofhouse

repo: dealmore/terraform-aws-next-js


text:
Errror decode()  argument 1 must be string, not None when run textract.process

I am having trouble when convertTotext with UTF-8 file...
text = textract.process("1.pdf", method='pdfminer')
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 150, in maybeDeferred
result = f(*args, **kw)
File "/usr/local/lib/python2.7/dist-packages/pydispatch/robustapply.py", line 55, in robustApply
return receiver(*arguments, **named)
File "td-net.py", line 178, in close
textData = convertToText(path,self.date) # convert pdf to text after download
File "td-net.py", line 239, in convertToText
text = textract.process("data/pdf/{1}/{0}.pdf".format(path,sDate), method='pdfminer')
File "/usr/local/lib/python2.7/dist-packages/textract/parsers/init.py", line 58, in process
return parser.process(filename, encoding, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/textract/parsers/utils.py", line 46, in process
unicode_string = self.decode(byte_string)
File "/usr/local/lib/python2.7/dist-packages/textract/parsers/utils.py", line 65, in decode
return text.decode(result['encoding'])
TypeError: decode() argument 1 must be string, not None

That's odd. Looks like chardet could not determine an encoding for your file 1.pdf. Can you try running chardet 1.pdf to see what the output looks like?
I wonder if this is related to #133 somehow...

This is exactly the problem I was having.

I just pinned chardet to 2.1.1 to address #107. I think this will likely address your issue as well. Try pulling from the latest master on github to see if that fixes it. I'm going to close this, but feel free to reopen if it remains a problem.

Hello,
I have this issue. I went back to 2.1.1 and now I got another error: ModuleNotFoundError: No module named 'universaldetector' which happens because chardet 2.1.1 is too old. What should I do?

authors: SwedishBotMafia deanmalmgren frostchick mstanojevic118

repo: deanmalmgren/textract


text:
Cluttered README

I think this project is awesome. It combines so many blacklists to create a comprehensive ad blocking list. Unfortunately, when I was reading the README, I found it to be rather confusing and hard to navigate. Maybe it could be split up into separate files with the stats placed somewhere else?

The script is still in an "active development state"
I have some major additions and changes on the horizon, and for now, the stats help me visualize the information at-a-glance.
I like the stats being there, and that is probably going to stay. However, I'll take any suggestions regarding the rest of the main README.md

authors: InnovativeInventor deathbybandaid

repo: deathbybandaid/piholeparser

license: WTFPL


text:
fatal error:: configuration file not found at /root/instapy-config.json

After lots of trial and error, I have finally made it to the very last step...but now I can't get passed this error message:
user123@computer123:/usr/share/wordlists$ sudo instagram-py -u username123 -pl rockyou.txt.gz
Instagram-Py 2.0.7 , Slick Instagram brute force command line tool.
Copyright (C) 2018 The Future Shell , Antony Jr.
[+] Started @ 2018-11-22 07:45:28.707465
fatal error:: configuration file not found at /root/instapy-config.json

python instagram-py -dc
AnalJesus

just move  instagram-config.json file  to  /root ..
cp instagram-config.json  /root/

authors: Hacker-spe TheMercyless1 analjesus

repo: deathsec/instagram-py


text:
DBZ-1720 Move CI to GH Actions

@Naros need to configure a secrets for WEBHOOK_URL for Gitter.

@ani-sha Hi, this looks very nice! I have just few comments for improvements

make sure that the order in paths field is same for all workflows, first the changed module and then dependencies. Ideally the dependencies will be graphically separated for example via comment. IIUC the deps block should be the same for all workflows
would it be possible to use matrix for jobs that executes more than once per connector?
would it be possible to add one more workflow that will trigger a new docs build when documentation is changed?

Great work!

Thanks, @jpechane for the suggestions.

In which case would we be needing to run the connectors more than once? I believe depending on the no of keys in a matrix, that many no of times a job will be executed.
Surely would be adding a workflow for docs. Also had a discussion with @Naros; thought of implementing this after fixing GH actions for website.

@ani-sha See for example MongoDB connector - it is keyed by version.mongo.server Maven property

@ani-sha See for example MongoDB connector - it is keyed by version.mongo.server Maven property

yep, I can create a matrix depending on the versions for mongodb. Anything for postgres?

@ani-sha For postgres there is version.postgres.server. Unfortunately profile names are changed as well so it needs to be somehow woven together.

@jpechane I tried using a matrix for mongodb locally, but it fails the build with this error.
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Debezium Parent POM 1.4.0-SNAPSHOT:
[INFO]
[INFO] Debezium Checkstyle Rules .......................... FAILURE [  2.882 s]
[INFO] Debezium IDE Formatting Rules ...................... SKIPPED
[INFO] Debezium Revapi Rules .............................. SKIPPED
[INFO] Debezium Parent POM ................................ SKIPPED
[INFO] Debezium API ....................................... SKIPPED
[INFO] Debezium Core ...................................... SKIPPED
[INFO] Debezium Assembly Descriptors ...................... SKIPPED
[INFO] Debezium Embedded .................................. SKIPPED
[INFO] Debezium Connector for MongoDB ..................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  58.448 s
[INFO] Finished at: 2020-11-09T09:46:02Z
[INFO] ------------------------------------------------------------------------
Error:  Unknown lifecycle phase "4.2". You must specify a valid lifecycle phase or a goal in the format <plugin-prefix>:<goal> or <plugin-group-id>:<plugin-artifact-id>[:<plugin-version>]:<goal>. Available lifecycle phases are: validate, initialize, generate-sources, process-sources, generate-resources, process-resources, compile, process-classes, generate-test-sources, process-test-sources, generate-test-resources, process-test-resources, test-compile, process-test-classes, test, prepare-package, package, pre-integration-test, integration-test, post-integration-test, verify, install, deploy, pre-clean, clean, post-clean, pre-site, site, post-site, site-deploy. -> [Help 1]
Error:
Error:  To see the full stack trace of the errors, re-run Maven with the -e switch.
Error:  Re-run Maven using the -X switch to enable full debug logging.
Error:
Error:  For more information about the errors and possible solutions, please read the following articles:
Error:  [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/LifecyclePhaseNotFoundException
Error: Process completed with exit code 1.

@ani-sha Could you please show the maven command you use?

@ani-sha Could you please show the maven command you use?

mvn clean install -B -pl debezium-connector-mongodb -am -Passembly -Dcheckstyle.skip=true -Dformat.skip=true -Drevapi.skip -Dversion.mongo.server= ${{ matrix.version-mongo-server }} -Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=warn

@ani-sha Try removing the space between = and ${ in -Dversion.mongo.server= ${{ matrix.version-mongo-server }}

@ani-sha Try removing the space between = and ${ in -Dversion.mongo.server= ${{ matrix.version-mongo-server }}

@jpechane Overcoming the first error throws a new error.
[INFO] Tests run: 113, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
[INFO] --- maven-jar-plugin:3.0.2:jar (default-jar) @ debezium-connector-mongodb ---
[INFO] Building jar: /home/runner/work/debezium/debezium/debezium-connector-mongodb/target/debezium-connector-mongodb-1.4.0-SNAPSHOT.jar
[INFO]
[INFO] --- maven-source-plugin:3.1.0:test-jar-no-fork (attach-test-sources) @ debezium-connector-mongodb ---
[INFO] Building jar: /home/runner/work/debezium/debezium/debezium-connector-mongodb/target/debezium-connector-mongodb-1.4.0-SNAPSHOT-test-sources.jar
[INFO]
[INFO] --- maven-jar-plugin:3.0.2:test-jar (test-jar) @ debezium-connector-mongodb ---
[INFO] Building jar: /home/runner/work/debezium/debezium/debezium-connector-mongodb/target/debezium-connector-mongodb-1.4.0-SNAPSHOT-tests.jar
[INFO]
[INFO] --- maven-assembly-plugin:3.1.1:single (default) @ debezium-connector-mongodb ---
[INFO] Building tar: /home/runner/work/debezium/debezium/debezium-connector-mongodb/target/debezium-connector-mongodb-1.4.0-SNAPSHOT-plugin.tar.gz
[INFO] Building zip: /home/runner/work/debezium/debezium/debezium-connector-mongodb/target/debezium-connector-mongodb-1.4.0-SNAPSHOT-plugin.zip
[INFO]
[INFO] --- docker-maven-plugin:0.31.0:build (start) @ debezium-connector-mongodb ---
[INFO]
[INFO] --- docker-maven-plugin:0.31.0:start (start) @ debezium-connector-mongodb ---
[INFO] DOCKER> Pulling from library/mongo
[INFO] DOCKER> Digest: sha256:efc408845bc917d0b7fd97a8590e9c8d3c314f58cee651bd3030c9cf2ce9032d
[INFO] DOCKER> Status: Downloaded newer image for mongo:4
[INFO] DOCKER> Pulled mongo:4 in 8 seconds
Error:  DOCKER> Error occurred during container startup, shutting down...
Error:  DOCKER> I/O Error [Unable to pull 'debezium/mongo-initiator:4' : {"message":"manifest for debezium/mongo-initiator:4 not found: manifest unknown: manifest unknown"} (Not Found: 404)]
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Debezium Parent POM 1.4.0-SNAPSHOT:
[INFO]
[INFO] Debezium Checkstyle Rules .......................... SUCCESS [01:31 min]
[INFO] Debezium IDE Formatting Rules ...................... SUCCESS [  0.778 s]
[INFO] Debezium Revapi Rules .............................. SUCCESS [  0.084 s]
[INFO] Debezium Parent POM ................................ SUCCESS [01:10 min]
[INFO] Debezium API ....................................... SUCCESS [ 31.257 s]
[INFO] Debezium Core ...................................... SUCCESS [01:06 min]
[INFO] Debezium Assembly Descriptors ...................... SUCCESS [  0.057 s]
[INFO] Debezium Embedded .................................. SUCCESS [ 30.882 s]
[INFO] Debezium Connector for MongoDB ..................... FAILURE [ 37.975 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  06:31 min
[INFO] Finished at: 2020-11-09T10:13:45Z
[INFO] ------------------------------------------------------------------------
Error:  Failed to execute goal io.fabric8:docker-maven-plugin:0.31.0:start (start) on project debezium-connector-mongodb: I/O Error: Unable to pull 'debezium/mongo-initiator:4' : {"message":"manifest for debezium/mongo-initiator:4 not found: manifest unknown: manifest unknown"} (Not Found: 404) -> [Help 1]
Error:
Error:  To see the full stack trace of the errors, re-run Maven with the -e switch.
Error:  Re-run Maven using the -X switch to enable full debug logging.
Error:
Error:  For more information about the errors and possible solutions, please read the following articles:
Error:  [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
Error:
Error:  After correcting the problems, you can resume the build with the command
Error:    mvn <args> -rf :debezium-connector-mongodb
Error: Process completed with exit code 1.

@ani-sha It seems that 4.0 was converted to 4 only when passed to the mvn command

@ani-sha It seems that 4.0 was converted to 4 only when passed to the mvn command

Yes. But 4.0 was provided in the matrix it took 4.

@jpechane Fixed the matrix for mongodb.

@ani-sha Nice! Do you think you'll manage to do it for postgres as well? It might be a bit more complicated as two things are updated

@ani-sha Nice! Do you think you'll manage to do it for postgres as well? It might be a bit more complicated as two things are updated

Well the first issue I am facing right now is with the dependencies in the postgres-connector. So I am not able to run or test anything locally for postgres.

@ani-sha Could you please share the full error message in the log? This should work the same way as MongoDB does.

@ani-sha Could you please share the full error message in the log? This should work the same way as MongoDB does.

Error:  Errors:
Error:    PostgresConnectorIT.shouldResumeStreamingFromSlotPositionForCustomSnapshot:1524->waitForSnapshotToBeCompleted:2378->AbstractConnectorTest.waitForSnapshotToBeCompleted:1035 » ConditionTimeout
[INFO]
Error:  Tests run: 219, Failures: 0, Errors: 1, Skipped: 3
[INFO]
[INFO]
[INFO] --- docker-maven-plugin:0.31.0:stop (stop) @ debezium-connector-postgres ---
07:14:53.469 postgresLOG:  received smart shutdown request
07:14:53.469 postgresLOG:  autovacuum launcher shutting down
07:14:53.469 postgresFATAL:  terminating autovacuum process due to administrator command
07:14:53.759 postgresLOG:  shutting down
07:14:53.829 postgresLOG:  database system is shut down
[INFO] DOCKER> [debezium/postgres-server-test-database:latest]: Stop and removed container 0f9e4630a416 after 0 ms
[INFO]
[INFO] --- maven-source-plugin:3.1.0:jar-no-fork (attach-sources) @ debezium-connector-postgres ---
[INFO] Building jar: /home/runner/work/debezium/debezium/debezium-connector-postgres/target/debezium-connector-postgres-1.4.0-SNAPSHOT-sources.jar
[INFO]
[INFO] --- maven-checkstyle-plugin:3.1.1:checkstyle (check-style) @ debezium-connector-postgres ---
[INFO] Starting audit...
Audit done.
[INFO]
[INFO] --- maven-failsafe-plugin:3.0.0-M3:verify (verify) @ debezium-connector-postgres ---
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Debezium Parent POM 1.4.0-SNAPSHOT:
[INFO]
[INFO] Debezium Checkstyle Rules .......................... SUCCESS [  2.313 s]
[INFO] Debezium IDE Formatting Rules ...................... SUCCESS [  0.282 s]
[INFO] Debezium Revapi Rules .............................. SUCCESS [  0.069 s]
[INFO] Debezium Parent POM ................................ SUCCESS [  1.406 s]
[INFO] Debezium API ....................................... SUCCESS [  5.895 s]
[INFO] Debezium Core ...................................... SUCCESS [01:22 min]
[INFO] Debezium Assembly Descriptors ...................... SUCCESS [  0.107 s]
[INFO] Debezium Embedded .................................. SUCCESS [ 14.299 s]
[INFO] Debezium Connector for PostgreSQL .................. FAILURE [12:15 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  14:02 min
[INFO] Finished at: 2020-11-11T07:14:55Z
[INFO] ------------------------------------------------------------------------
Error:  Failed to execute goal org.apache.maven.plugins:maven-failsafe-plugin:3.0.0-M3:verify (verify) on project debezium-connector-postgres: There are test failures.
Error:
Error:  Please refer to /home/runner/work/debezium/debezium/debezium-connector-postgres/target/failsafe-reports for the individual test results.
Error:  Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.

@ani-sha This looks like an intermittent failure issue so you can ignore it for now. But if it is reliable reproducibale on your machine then we are going to borrow it to find the root cause :-)

@ani-sha This looks like an intermittent failure issue so you can ignore it for now. But if it is reliable reproducibale on your machine then we are going to borrow it to find the root cause :-)

ok sure; for postrges we would be using two matrixes one being the version.postgres.server with [9.6, 10] and other being plugin matrix would contain?

It seems to me that -Dversion.postgres.server=9.6-devel could be removed and matrix will be made out of different profile settings

assembly
assembly,wal2json
assembly,postgres-10,pgoutput

and the strings from above will be passed as -P{...} to the maven command

@ani-sha I think there is no need to delay this later - looks good and let's give it a try!

@ani-sha I think there is no need to delay this later - looks good and let's give it a try!
Absolutely! 🚀

authors: ani-sha jpechane

repo: debezium/debezium


text:
DBZ-6803 Add REPEAT function for MySQL

https://issues.redhat.com/browse/DBZ-6803

Upstream - https://github.com/antlr/grammars-v4/pull/3667

@ani-sha Applied, thanks


text:
DBZ-720 Updating required permissions for connector user for snapshot…

…ting
@jpechane Small update regarding grants for initial snapshotting.
Btw. this might make us re-think whether the flashback query is the best way to do the initial snapshot. We might also get away with reading within a transaction using the right isolation level. I found the "AS OF SCN ..." approach quite attractive, though, so I went for it. We can re-evaluate later on, but I wanted to bring it to your attention.

@gunnarmorling Applied, thanks!

authors: gunnarmorling jpechane

repo: debezium/oracle-vagrant-box


text:
Fetch NFT prices and use them to calculate treasury total

https://docs.moralis.io/web3-data-api/evm/reference/get-nft-sale-prices

@tomstuart123 from a product perspective, do users want to see NFTs in their Safes count toward their total balance?

Just logging here

Do we still want to do this?

authors: Da-Colon adamgall mudrila tomstuart123

repo: decentdao/decent-interface


text:
Feat/billboard

Introduced Billboard feature as Explorer/Assets/DCL/Billboard

Locate at

Nullables enabled

Covered with tests

For demo created scenes

Assets/DCL/Billboard/Demo/BillboardDemoTest.unity - shows multiple

Assets/DCL/Billboard/Demo/BillboardPlayground.unity - makes possible to tweak options in runtime and to see a difference

IDemoWorld for easy reuse

Looks very cool, man 💪 I have just 3 suggestions:

It would be nice to have all SDK-related components under one root folder. Maybe Explorer/Assets/DCL/Scenes/Billboard or Explorer/Assets/DCL/SDKScenes/Billboard
Since you already have nice integration test environment, can you assemble then a Performance Test (for 200 and 500 entities). As we already have Unity Performance Testing imported in the project
Please, rename the header of the PR

authors: NickKhalow popuz

repo: decentraland/unity-explorer


text:
Messages and generators notation update for readability

First attempt to close issue #54 (see in the issue for other options).
This PR proposes to pass the generators directly to functions instead of their indexes, with the goal of increasing spec readability.
IMO this simplifies notation. For example, using indexes in spkGen will require notation for 5 different lists of indexes. This can get confusing. By passing the generators as a list, only requires notation for 2 lists of indexes (and one less input argument in SpkGen). Also,

Passing the messages as a map between the message and the index of the generator will still require a lot of notation for indexes.
Mentioning that those generators are not necessarily the L first elements from the global (or not) generators list, also preserves the flexibility required from the blind signatures.
Mentioning that implementations may choose to pass the indexes of the generators instead and pointing to a reference implementation or perhaps a more detailed explanation in the Appendix IMO will be enough to address the efficiency of the applications concerns, while keeping the spec more readable.

Also, changes in this PR use the terminology from PR #62 to some places, but I will update it elsewhere after that PR is merged.

Discussed on WG call 21st of Feb, awaiting review from other WG members

@BasileiosKal can you please update this PR to resolve the conflicts?

Multiple approvals, PR open for 2 weeks and discussed on WG call, massive improvement in notation across the spec, merging

authors: BasileiosKal tplooker

repo: decentralized-identity/bbs-signature


text:
Feature extraction

Hi, thanks for the great codebase.
Could you kindly provide the code to extract features from custom videos using pre-trained models?

@Finspire13 Sorry for late reply. You can easily modify the config files (\eg remove the cls head) and test_video.py or  test_recognizer.py to extract features.

repo: decisionforce/TPN


text:
Thumbnails not loading

I encountered three different problems:

All thumbnails in a folder are visible
No thumbnail in a folder is visible
In the same folder, some photos have thumbnails while others do not.

In case for the failing thumbnail, a dark grey square is shown instead.

The failing thumbnails are for common formats like JPG and PNG, or for
something else?
Do they fail for files that no longer exist but are still registered in the
media store?
On Mon, Jan 4, 2021 at 12:24 PM Donkey-Doug notifications@github.com
wrote:

I encountered three different problems:

All thumbnails in a folder are visible
No thumbnail in a folder is visible
In the same folder, some photos have thumbnails while others do not.

In case for the failing thumbnail, a dark grey square is shown instead.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/deckerst/aves/issues/30, or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADKBEXNTBWXLWFMIU35LAMDSYEYGRANCNFSM4VSNNUMQ
.

The failing thumbnails are for common formats like JPG and PNG, or for
something else?
Do they fail for files that no longer exist but are still registered in the
media store?
On Mon, Jan 4, 2021 at 12:24 PM Donkey-Doug notifications@github.com
wrote:

I encountered three different problems:

All thumbnails in a folder are visible
No thumbnail in a folder is visible
In the same folder, some photos have thumbnails while others do not.

In case for the failing thumbnail, a dark grey square is shown instead.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/deckerst/aves/issues/30, or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADKBEXNTBWXLWFMIU35LAMDSYEYGRANCNFSM4VSNNUMQ
.

I tried to find a pattern in the failing thumbnails before reporting the issue, but failed to find any.

I tried to find a pattern in the failing thumbnails before reporting the issue, but failed to find any.

Some of the thumbnails that were not visible earlier started to show. Apparently it takes very long for some thumbnails to become visible.

Some of the thumbnails that were not visible earlier started to show. Apparently it takes very long for some thumbnails to become visible.

Thanks for the update. Indeed some apps delete files without properly removing them from the Media Store.
Ideally, these broken files should be handled more gracefully by Aves. When the app detects such file, it could even suggest to fix the situation by removing them from the Media Store.

Thanks for the update. Indeed some apps delete files without properly removing them from the Media Store.
Ideally, these broken files should be handled more gracefully by Aves. When the app detects such file, it could even suggest to fix the situation by removing them from the Media Store.

authors: Donkey-Doug deckerst

repo: deckerst/aves


text:
Update Japanese translations

Fixed Japanese mistranslation and HTML tag mistakes.

Nice catch, thanks.

authors: HiSubway deckerst


text:
Use CompilerConfig model in Scratch model

A little refactor.
Make sure to not break the existing create scratch endpoint's interface (#148 would help here!)

@ethteck did we get anywhere with this

I've been working on it

This isn't needed anymore since we store Presets in the backend, which are essentially CompilerConfig

authors: bates64 ethteck nanaian


text:
Hook up fee estimation to transaction creation.

This change requires a newer version of the wallet's RPC API.
Fixes #10.
Fixes #11.

Two issues; you see Estimated Remaining Balance at 1.0, should be 0. And the crash of course

If priority checks were run at all that means the fee was not high enough.  I'll look through it for bugs.

Rebased over master and fixed a magnitude error with the default fee.  Shouldn't see any more tx rejected errors for low priority.

Coins were unconfirmed, and therefore not usable to fund a transaction.  The same transaction went through after change received the required number of block confirmations.  We should display the spendable balance next to the selected account, instead of the user relying on the total balance of all accounts in the corner.

so I am ok with this going in.  It isn't perfect but certainly works much better.

just fixing up an issue in the transaction authoring code where it could create dust change outputs.  Will merge after that is fixed.

authors: jrick marcopeereboom

repo: decred/Paymetheus