text:
Fix a couple issues in benchmark program

Initialize g_num_copies and g_num_hashes when test starts. Also speedup
hash computation for large objects.

I'm still a bit confused why you want to truncate the hash computations given different lengths of key are being tested:
https://github.com/greg7mdp/sparsehash/blob/55e56904c9490a779ec94fdb6ac84d6b6cf51daa/src/time_hash_map.cc#L730-L733
If you are really convinced of the need to test different hash input lengths, then perhaps a more template-friendly option is to add a HashLength parameter to the HashObject template and then you can test all manner of variable inputs rather than clamping the hash input to 4 bytes?

In fact, is that not what HashSize actually does? :-)

Yes, you are right, I'll update my change.

From: Donovan Hide [mailto:notifications@github.com]
Sent: Saturday, July 23, 2016 11:02 AM
To: sparsehash/sparsehash
Cc: greg7mdp; Author
Subject: Re: [sparsehash/sparsehash] Fix a couple issues in benchmark
program (#127)
In fact, is that not what HashSize actually does? :-)

You are receiving this because you authored the thread.
Reply to this email directly, view
https://github.com/sparsehash/sparsehash/pull/127#issuecomment-234722684
it on GitHub, or mute
<https://github.com/notifications/unsubscribe-auth/ABbXS51EN357QM-Uxw_bNAb-z
HRTg4S3ks5qYizhgaJpZM4JTO-Q> the thread.
<https://github.com/notifications/beacon/ABbXS4wKHFxbyCHlSV-7buSMtt3FIEUmks5
qYizhgaJpZM4JTO-Q.gif>

Hi Donovan,
The reason for the change is that, even when large objects are inserted into
a hash table, the hash computation does not normally require iterating
through the whole object, and even if it did, then it would make more sense
to store the computed hash value rather than recomputing it every time. So
having a test with a very slow hash computation makes hash tables look worse
than they really are compared to maps.
greg

From: Donovan Hide [mailto:notifications@github.com]
Sent: Saturday, July 23, 2016 11:00 AM
To: sparsehash/sparsehash
Cc: greg7mdp; Author
Subject: Re: [sparsehash/sparsehash] Fix a couple issues in benchmark
program (#127)
I'm still a bit confused why you want to truncate the hash computations
given different lengths of key are being tested:
https://github.com/greg7mdp/sparsehash/blob/55e56904c9490a779ec94fdb6ac84d6b
6cf51daa/src/time_hash_map.cc#L730-L733
If you are really convinced of the need to test different hash input
lengths, then perhaps a more template-friendly option is to add a HashLength
parameter to the HashObject template and then you can test all manner of
variable inputs rather than clamping the hash input to 4 bytes?

You are receiving this because you authored the thread.
Reply to this email directly, view
https://github.com/sparsehash/sparsehash/pull/127#issuecomment-234722588
it on GitHub, or mute
<https://github.com/notifications/unsubscribe-auth/ABbXS3Z-IjSo3kUlYJeJgxU55
50bpzgcks5qYixsgaJpZM4JTO-Q> the thread.
<https://github.com/notifications/beacon/ABbXSwACzqC0trYELX8pgNjbQbsivfO8ks5
qYixsgaJpZM4JTO-Q.gif>

Fair enough, but the original point I was trying to make is that you only need to change the HashSize parameters in lines:
https://github.com/greg7mdp/sparsehash/blob/55e56904c9490a779ec94fdb6ac84d6b6cf51daa/src/time_hash_map.cc#L730-L733
to achieve your goal. Also you can't compare the benchmarks of std:map or std:unordered_map to this particular benchmark because the code does not exist :-) It sounds like you are trying to benchmark sparse and dense hash maps with other maps. If you want to do that, consider either creating a new set of tests which do so :-)
Finally, not sure what the last commit is achieving. You need to include explanative commit messages for each change if you want to have them merged.

Hi Donovan,
I do agree with your original point. You are right that it is simpler to just change the Hashsize template parameter. This is what I did with my last commit, so I thought it would be obvious from the diffs..
If you run the time_hash_map benchmark, you'll see that it indeed does benchmark unordered_map and map (the unordered_map is included using the HASH_MAP_H macro.

I'm starting to get confused :-)
https://github.com/sparsehash/sparsehash/pull/127/files
Does not change the HashSize template parameter for the 256 byte key. Specifically, I'm suggesting adding a new line, perhaps:
if (FLAGS_test_256_shortkey_bytes)  test_all_maps< HashObject<256,4> >(256, iters/32);

with the necessary adjustments for the command line flag parsing, rather than hard-coding a 4 byte limit in the template itself.
Apologies for missing the existence of the existing map and unordered_map code. It's been a long time since I actually looked at this code :-)

My bad, it is the first time I'm submitting changes on github and I don't know what I'm doing. I thought I submitted a second commit but maybe it didn't go through. I'll try again.

OK, I think overall my combined commits are good. If you'd like I can restart from scratch and submit a new pull request.

Github and git can be a bit fiddly the first few times :-)
Ok, had a bit of a think about this. You've assumed that std::hash for std::unordered_map and std::less for std::map are examining fewer bytes than densehash or sparsehash in the test. Can you support the truth of that assertion? I honestly don't know the answer...
Also, it will be better to separate out the Reset() changes into a different commit and pull request with a comment about how that affects the test output.
Sorry for the caution with this change. Benchmarks are contentious and also I'm weary of making changes without consensus.

repo: sparsehash/sparsehash


text:
unicode characters are not handled correctly

e.g. (from https://github.com/voku/portable-utf8/blob/master/src/voku/helper/UTF8.php)
  private static $BOM = [
      "\xef\xbb\xbf"     => 3, // UTF-8 BOM
      'ï»¿'              => 6, // UTF-8 BOM as "WINDOWS-1252" (one char has [maybe] more then one byte ...)
      "\x00\x00\xfe\xff" => 4, // UTF-32 (BE) BOM
      '  þÿ'             => 6, // UTF-32 (BE) BOM as "WINDOWS-1252"
      "\xff\xfe\x00\x00" => 4, // UTF-32 (LE) BOM
      'ÿþ  '             => 6, // UTF-32 (LE) BOM as "WINDOWS-1252"
      "\xfe\xff"         => 2, // UTF-16 (BE) BOM
      'þÿ'               => 4, // UTF-16 (BE) BOM as "WINDOWS-1252"
      "\xff\xfe"         => 2, // UTF-16 (LE) BOM
      'ÿþ'               => 4, // UTF-16 (LE) BOM as "WINDOWS-1252"
  ];

... becomes ...
    private static $BOM = [
        "" => 3,
        // UTF-8 BOM
        'ï»¿' => 6,
        // UTF-8 BOM as "WINDOWS-1252" (one char has [maybe] more then one byte ...)
        "\0\0��" => 4,
        // UTF-32 (BE) BOM
        '  þÿ' => 6,
        // UTF-32 (BE) BOM as "WINDOWS-1252"
        "��\0\0" => 4,
        // UTF-32 (LE) BOM
        'ÿþ  ' => 6,
        // UTF-32 (LE) BOM as "WINDOWS-1252"
        "��" => 2,
        // UTF-16 (BE) BOM
        'þÿ' => 4,
        // UTF-16 (BE) BOM as "WINDOWS-1252"
        "��" => 2,
        // UTF-16 (LE) BOM
        'ÿþ' => 4,
    ];

We are not maintaining this package any more. Feel free to fork our code and maintain your own copy.

source: gharchive/issue

authors: freekmurze voku

license: mit

license_type: permissive

license_source: bigquery


text:
Structure caching doesn't find classes

✏ Describe the bug
I have a specific architecture where my code is not stored in app, but rather in src and nested subdirectories. Code is not architectured by type (eg. Data, Controllers, etc) but by domain (eg. src/Feature/Authentication/UserData.php).
I tried running data:cache-structures, but it can't find any data class. All my data classes extend Spatie\LaravelData\Data, which does implement DataObject/BaseData.
Despite tweaking config/data.php appropriately, php-structure-discoverer can't seem to find my classes, with or without reflection discovery.
'structure_caching' => [
    'enabled' => env('DATA_STRUCTURE_CACHE_ENABLED', true),
    'directories' => [base_path('src/Feature')],
    'cache' => [
        'store' => env('CACHE_DRIVER', 'file'),
        'prefix' => 'laravel-data',
    ],
    'reflection_discovery' => [
        'enabled' => false,
        'base_path' => base_path('src/Feature'),
        'root_namespace' => null,
    ],
],

It seems to be an issue in ImplementsDiscoverCondition, because my classes are filtered out in StructureResolver:

In fact, if I change the following line, caching works:
$discoverer = Discover::in(...$this->directories)
-     ->implementing(BaseData::class)
+     ->extending(Data::class);

I wasn't sure where to raise the issue (there or at php-structure-discoverer), I decided here since ultimately php-structure-discoverer is an implementation detail, but let me know if I should move the issue.
🖥 Versions
Laravel: 10.43.0
Laravel Data: 3.11.0
Structure Discoverer: 2.0.1
PHP: 8.3.1

Hi @innocenzi, could you provide me a test project showing this issue? Makes it a bit simpler to debug these kinds of things.

Hey Ruben, here's a commit that reproduces it: https://github.com/hybridly/demo/commit/1adeb432b7dbaf1766b57e6546b37a36af0a24f2
You could follow the instructions here but I think it would be easier to add a src directory in your own project to reproduce it, like I've done in the commit

Mhh, structure discoverer is in your example configured to not use reflection. So it only has parsed PHP to build its knowledge.
Add base_path('vendor/spatie/laravel-data/src') to the directories and the structure discoverer should find your data class. I get a ReflectionException because   Class "Foo\Bar\BarData" does not exist but that could also be my local config.
I've also added some extra docs on this.

Reflection was not working either, and was executing some code twice (a namespaced function file, specifically). I'll try again with your specified base_path and report back

authors: innocenzi rubenvanassche

repo: spatie/laravel-data

license: MIT

license_source: github-api


text:
send_queries_to_ray is still performing DB action when disabled

Describe the bug
Perhaps it shoudn't be marked as bug, but the way it is implemented at te moment, you cannot really change the loading behavior of Ray.
The QueryWatcher is able to be activated by a setting, see:
https://github.com/spatie/laravel-ray/blob/master/src/Watchers/QueryWatcher.php#L20
Our booting process is a bit specific by setting database connections, so at that time the connection is not available yet. We can solve this at our side by changing the boot process, but in my opinion, the setting check could be performed before the DB::listen call, as there is no need for this call if the setting is disabled.
Versions
Ray version (you can see this in "About Ray"):
You can use composer show to get the version numbers of:

spatie/ray package version: 1.19
spatie/laravel-ray package version: 1.12.1

PHP version: 7.4
Laravel version: 7.30.4
To Reproduce
Bit difficult to reproduce if you don't have the specific database boot process as we have.
x
Expected behavior
Skip function logic when setting is disabled, move the setting check to line 18

Perhaps solution implemented in this issue (https://github.com/spatie/laravel-ray/issues/135) will also solve this issue in another way, but still, is the DB call needed when the setting is disabled :)

Could you submit a non-breaking PR that would fix this issue for you?

Hereby: https://github.com/spatie/laravel-ray/pull/140/files

We'll continue the conversation in #140

authors: esseremmerik freekmurze

repo: spatie/laravel-ray


text:
Added a verification that the Yii class exists

When trying to use Laravel Envoy in a CraftCMS project got me this error:
PHP Fatal error:  Uncaught Error: Class "Yii" not found in /.../vendor/spatie/ray/src/helpers.php:32

Thanks!

source: gharchive/pull-request

repo: spatie/ray


text:
Production system unavailable

The production online tools was unavailable on 25 July 2022.
The system is back up through a restart of the AWS EC2 instance and restarting the docker image.
The original cause of the downtime is under investigation.

The following error was found in the std-err container logs - not sure if this is related to the downtime:

Traceback (most recent call last):
  File "/spdx/src/app/formatxml.py", line 253, in <module>
    process(fname)
  File "/spdx/src/app/formatxml.py", line 101, in process
    ser = fmt(blocks)
  File "/spdx/src/app/formatxml.py", line 166, in fmt
    indent = l * config['lvl_indent']
UnboundLocalError: local variable 'l' referenced before assignment
Traceback (most recent call last):
  File "/spdx/src/app/formatxml.py", line 253, in <module>
    process(fname)
  File "/spdx/src/app/formatxml.py", line 101, in process
    ser = fmt(blocks)
  File "/spdx/src/app/formatxml.py", line 166, in fmt
    indent = l * config['lvl_indent']
UnboundLocalError: local variable 'l' referenced before assignment
Traceback (most recent call last):
  File "/spdx/src/app/formatxml.py", line 253, in <module>
    process(fname)
  File "/spdx/src/app/formatxml.py", line 101, in process
    ser = fmt(blocks)
  File "/spdx/src/app/formatxml.py", line 166, in fmt
    indent = l * config['lvl_indent']
UnboundLocalError: local variable 'l' referenced before assignment
Traceback (most recent call last):
  File "/spdx/src/app/formatxml.py", line 253, in <module>
    process(fname)
  File "/spdx/src/app/formatxml.py", line 101, in process
    ser = fmt(blocks)
  File "/spdx/src/app/formatxml.py", line 166, in fmt
    indent = l * config['lvl_indent']
UnboundLocalError: local variable 'l' referenced before assignment
Traceback (most recent call last):
  File "/spdx/src/app/formatxml.py", line 253, in <module>
    process(fname)
  File "/spdx/src/app/formatxml.py", line 101, in process
    ser = fmt(blocks)
  File "/spdx/src/app/formatxml.py", line 166, in fmt
    indent = l * config['lvl_indent']
UnboundLocalError: local variable 'l' referenced before assignment
Traceback (most recent call last):
  File "/spdx/src/app/formatxml.py", line 253, in <module>
    process(fname)
  File "/spdx/src/app/formatxml.py", line 101, in process
    ser = fmt(blocks)
  File "/spdx/src/app/formatxml.py", line 166, in fmt
    indent = l * config['lvl_indent']
UnboundLocalError: local variable 'l' referenced before assignment
Traceback (most recent call last):
  File "/spdx/src/app/formatxml.py", line 253, in <module>
    process(fname)
  File "/spdx/src/app/formatxml.py", line 101, in process
    ser = fmt(blocks)
  File "/spdx/src/app/formatxml.py", line 166, in fmt
    indent = l * config['lvl_indent']
UnboundLocalError: local variable 'l' referenced before assignment

The following errors were in the std-out for the Docker image - likely unrelated to the downtime:
08:04:13.379 [Thread-65] ERROR org.apache.jena.rdf.model.impl.RDFDefaultErrorHandler - (line 687 column 75): {E211} Base URI is null, but there are relative URIs to resolve.: <LicenseRef-BSD-3-Clause-UC>
08:04:13.380 [Thread-65] ERROR org.apache.jena.rdf.model.impl.RDFDefaultErrorHandler - (line 706 column 67): {E211} Base URI is null, but there are relative URIs to resolve.: <LicenseRef-GPL-2.0>
08:04:13.380 [Thread-65] ERROR org.apache.jena.rdf.model.impl.RDFDefaultErrorHandler - (line 713 column 68): {E211} Base URI is null, but there are relative URIs to resolve.: <LicenseRef-LGPL-2.0>
08:04:13.380 [Thread-65] ERROR org.apache.jena.rdf.model.impl.RDFDefaultErrorHandler - (line 720 column 65): {E211} Base URI is null, but there are relative URIs to resolve.: <LicenseRef-Xming>
08:04:13.417 [Thread-65] ERROR org.spdx.spdxRdfStore.RdfSpdxDocumentModelManager - Error getting single value.  Multiple values for property fileName ID SPDXRef-file7b3eac4e7def3b3bffa60a3870b50034.
08:04:13.417 [Thread-65] ERROR org.spdx.spdxRdfStore.RdfSpdxDocumentModelManager - Error getting single value.  Multiple values for property licenseConcluded ID SPDXRef-file7b3eac4e7def3b3bffa60a3870b50034.
08:04:13.419 [Thread-65] ERROR org.spdx.spdxRdfStore.RdfSpdxDocumentModelManager - Error getting single value.  Multiple values for property fileName ID SPDXRef-file7b3eac4e7def3b3bffa60a3870b50034.
08:04:13.419 [Thread-65] ERROR org.spdx.spdxRdfStore.RdfSpdxDocumentModelManager - Error getting single value.  Multiple values for property fileName ID SPDXRef-file7b3eac4e7def3b3bffa60a3870b50034.
08:04:13.425 [Thread-65] ERROR org.spdx.spdxRdfStore.RdfSpdxDocumentModelManager - Error getting single value.  Multiple values for property fileName ID SPDXRef-filef1c730a52fed8182847b459c8a7cccef.
08:04:13.426 [Thread-65] ERROR org.spdx.spdxRdfStore.RdfSpdxDocumentModelManager - Error getting single value.  Multiple values for property licenseConcluded ID SPDXRef-filef1c730a52fed8182847b459c8a7cccef.
08:04:13.427 [Thread-65] ERROR org.spdx.spdxRdfStore.RdfSpdxDocumentModelManager - Error getting single value.  Multiple values for property fileName ID SPDXRef-filef1c730a52fed8182847b459c8a7cccef.
08:04:13.427 [Thread-65] ERROR org.spdx.spdxRdfStore.RdfSpdxDocumentModelManager - Error getting single value.  Multiple values for property fileName ID SPDXRef-filef1c730a52fed8182847b459c8a7cccef.
08:04:13.434 [Thread-65] ERROR org.spdx.spdxRdfStore.RdfSpdxDocumentModelManager - Error getting single value.  Multiple values for property fileName ID SPDXRef-file215ba888327b0e6333d6419b8de60e8e.
08:04:13.435 [Thread-65] ERROR org.spdx.spdxRdfStore.RdfSpdxDocumentModelManager - Error getting single value.  Multiple values for property licenseConcluded ID SPDXRef-file215ba888327b0e6333d6419b8de60e8e.
08:04:13.435 [Thread-65] ERROR org.spdx.spdxRdfStore.RdfSpdxDocumentModelManager - Error getting single value.  Multiple values for property copyrightText ID SPDXRef-file215ba888327b0e6333d6419b8de60e8e.
08:04:13.436 [Thread-65] ERROR org.spdx.spdxRdfStore.RdfSpdxDocumentModelManager - Error getting single value.  Multiple values for property fileName ID SPDXRef-file215ba888327b0e6333d6419b8de60e8e.
08:04:13.437 [Thread-65] ERROR org.spdx.spdxRdfStore.RdfSpdxDocumentModelManager - Error getting single value.  Multiple values for property fileName ID SPDXRef-file215ba888327b0e6333d6419b8de60e8e.
08:51:30.225 [Thread-17] ERROR org.spdx.spreadsheetstore.SpdxSpreadsheet - I/O error reading SPDX Spreadsheet
java.io.IOException: Your InputStream was neither an OLE2 stream, nor an OOXML stream
        at org.apache.poi.ss.usermodel.WorkbookFactory.create(WorkbookFactory.java:258) ~[tool.jar:?]
        at org.apache.poi.ss.usermodel.WorkbookFactory.create(WorkbookFactory.java:221) ~[tool.jar:?]
        at org.spdx.spreadsheetstore.SpdxSpreadsheet.<init>(SpdxSpreadsheet.java:107) [tool.jar:?]
        at org.spdx.spreadsheetstore.SpreadsheetStore.deSerialize(SpreadsheetStore.java:339) [tool.jar:?]
        at org.spdx.tools.Verify.verify(Verify.java:126) [tool.jar:?]
12:59:08.482 [Thread-90] ERROR org.spdx.spreadsheetstore.SpdxSpreadsheet - Spreadsheet version SPDX-2.2 not supported.
13:46:35.869 [Thread-12] ERROR org.spdx.spreadsheetstore.SpdxSpreadsheet - Spreadsheet version SPDX-2.2 not supported.
WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
16:22:37.898 [main] ERROR org.apache.jena.rdf.model.impl.RDFDefaultErrorHandler - (line 6073 column 73): {E211} Base URI is null, but there are relative URIs to resolve.: <SPDX Document created by ScanCode Toolkit>
16:22:38.145 [main] ERROR org.apache.jena.rdf.model.impl.RDFDefaultErrorHandler - (line 14751 column 50): {E211} Base URI is null, but there are relative URIs to resolve.: <SPDXRef-001>
16:22:38.788 [main] ERROR org.spdx.spdxRdfStore.SpdxOwlOntology - http://spdx.org/rdf/terms#licenseName is not an SPDX property
19:06:55.343 [Thread-67] ERROR org.apache.jena.rdf.model.impl.RDFDefaultErrorHandler - (line 2 column 11): Open quote is expected for attribute "xmlns:rdf" associated with an  element type  "rdf:RDF".
Verification errors were found in /spdx/src/app/media/AnonymousUser/1657706197/talko.spdx.  See verification errors sheet for details.
Verification errors were found in /spdx/src/app/media/AnonymousUser/1657706197/talko-premium.spdx.  See verification errors sheet for details.
WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
18:05:52.251 [Thread-25] ERROR org.apache.jena.rdf.model.impl.RDFDefaultErrorHandler - (line 1 column 1): Content is not allowed in prolog.
18:07:02.015 [Thread-9] ERROR org.apache.jena.rdf.model.impl.RDFDefaultErrorHandler - (line 1 column 1): Content is not allowed in prolog.
23:41:16.876 [Thread-29] ERROR org.spdx.spreadsheetstore.SpdxSpreadsheet - Required cell Package Download Location missing for row 1 in PackageInfo sheet.
04:19:22.626 [Thread-76] ERROR org.spdx.spreadsheetstore.SpdxSpreadsheet - I/O error reading SPDX Spreadsheet
java.io.IOException: Your InputStream was neither an OLE2 stream, nor an OOXML stream
        at org.apache.poi.ss.usermodel.WorkbookFactory.create(WorkbookFactory.java:258) ~[tool.jar:?]
        at org.apache.poi.ss.usermodel.WorkbookFactory.create(WorkbookFactory.java:221) ~[tool.jar:?]
        at org.spdx.spreadsheetstore.SpdxSpreadsheet.<init>(SpdxSpreadsheet.java:107) [tool.jar:?]
        at org.spdx.spreadsheetstore.SpreadsheetStore.deSerialize(SpreadsheetStore.java:339) [tool.jar:?]
        at org.spdx.tools.SpdxConverter.convert(SpdxConverter.java:151) [tool.jar:?]
04:20:21.294 [Thread-32] ERROR org.apache.jena.rdf.model.impl.RDFDefaultErrorHandler - (line 1 column 1): Content is not allowed in prolog.
WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
11:15:29.693 [Thread-1] ERROR org.apache.jena.rdf.model.impl.RDFDefaultErrorHandler - (line 1 column 1): Content is not allowed in prolog.
11:41:10.685 [Thread-157] ERROR org.spdx.spdxRdfStore.RdfSpdxDocumentModelManager - ID  does not exist in the model.
21:37:21.917 [Thread-39] ERROR org.spdx.jacksonstore.JacksonSerializer - Invalid ID SPDXRef-@vue/babel-sugar-v-model.  Must be an SPDX Identifier or Anonymous
21:40:25.111 [Thread-41] ERROR org.spdx.jacksonstore.JacksonSerializer - Invalid ID SPDXRef-vue/babel-sugar-functional-vue.  Must be an SPDX Identifier or Anonymous
21:42:22.244 [Thread-42] ERROR org.spdx.jacksonstore.JacksonSerializer - Invalid ID SPDXRef-vue/babel-sugar-functional-vue.  Must be an SPDX Identifier or Anonymous
21:43:04.784 [Thread-43] ERROR org.spdx.jacksonstore.JacksonSerializer - Invalid ID SPDXRef-types/q.  Must be an SPDX Identifier or Anonymous
21:44:09.224 [Thread-44] ERROR org.spdx.jacksonstore.JacksonSerializer - Invalid ID SPDXRef-types/q.  Must be an SPDX Identifier or Anonymous
21:44:49.699 [Thread-45] ERROR org.spdx.jacksonstore.JacksonSerializer - Invalid ID SPDXRef-ttskch/select2-bootstrap4-theme.  Must be an SPDX Identifier or Anonymous
21:45:23.496 [Thread-46] ERROR org.spdx.jacksonstore.JacksonSerializer - Invalid ID SPDXRef-soda/get-current-script.  Must be an SPDX Identifier or Anonymous
21:45:49.099 [Thread-47] ERROR org.spdx.jacksonstore.JacksonSerializer - Invalid ID SPDXRef-sweetalert2/theme-bootstrap-4.  Must be an SPDX Identifier or Anonymous
XML Parse Error: syntax error: line 1, column 0
XML Parse Error: syntax error: line 1, column 0
XML Parse Error: no element found: line 1, column 38
XML Parse Error: undefined entity &xxe;: line 4, column 51
XML Parse Error: undefined entity &xxe;: line 4, column 51
14:40:31.235 [Thread-164] ERROR org.apache.jena.rdf.model.impl.RDFDefaultErrorHandler - (line 1 column 1): Content is not allowed in prolog.

I found this in the syslog:
Jul 25 17:16:34 ip-172-31-6-122 systemd[1]: Started Unattended Upgrades Shutdown.

And several networking related messages:

Jul 25 17:16:37 ip-172-31-6-122 systemd-timesyncd[593]: Network configuration ch
anged, trying to establish connection.
Jul 25 17:16:37 ip-172-31-6-122 systemd-udevd[1344]: link_config: autonegotiatio
n is unset or enabled, the speed and duplex are not writable.
Jul 25 17:16:37 ip-172-31-6-122 systemd-udevd[1344]: Could not generate persiste
nt MAC address for veth3d8a564: No such file or directory
Jul 25 17:16:37 ip-172-31-6-122 systemd-udevd[1352]: link_config: autonegotiatio
n is unset or enabled, the speed and duplex are not writable.
Jul 25 17:16:37 ip-172-31-6-122 systemd-udevd[1352]: Could not generate persiste
nt MAC address for veth309a3b7: No such file or directory
Jul 25 17:16:37 ip-172-31-6-122 systemd-timesyncd[593]: Network configuration ch
anged, trying to establish connection.
Jul 25 17:16:37 ip-172-31-6-122 systemd-timesyncd[593]: Network configuration ch
anged, trying to establish connection.

Not sure if it is related

More evidence of network errors from syslog:
Jul 25 06:35:14 ip-172-31-6-122 snapd[9129]: autorefresh.go:528: Cannot prepare
auto-refresh change due to a permanent network error: persistent network error:
Post https://api.snapcraft.io/v2/snaps/refresh: dial tcp: lookup api.snapcraft.i
o: Temporary failure in name resolution
Jul 25 06:35:14 ip-172-31-6-122 snapd[9129]: stateengine.go:149: state ensure er
ror: persistent network error: Post https://api.snapcraft.io/v2/snaps/refresh: d
ial tcp: lookup api.snapcraft.io: Temporary failure in name resolution
Jul 25 07:00:14 ip-172-31-6-122 snapd[9129]: autorefresh.go:528: Cannot prepare
auto-refresh change due to a permanent network error: persistent network error:
Post https://api.snapcraft.io/v2/snaps/refresh: dial tcp: lookup api.snapcraft.i
o: Temporary failure in name resolution
Jul 25 07:00:14 ip-172-31-6-122 snapd[9129]: stateengine.go:149: state ensure er
ror: persistent network error: Post https://api.snapcraft.io/v2/snaps/refresh: d
ial tcp: lookup api.snapcraft.io: Temporary failure in name resolution

From the code, looks like UnboundLocalError: local variable 'l' referenced before assignment can happen if m.group('level') is false
def fmt(blocks):
    bregexp = re.compile(r'((?P<level>\d+)#)?(?P<paragraph>.*)')
    ser = ''
    for line in blocks.split('\n'):
        if line == '':
            continue
        m = bregexp.match(line)
        if m.group('level'):
            l = int(m.group('level'))
        else:
            logger.error('Block without level: "{}"'.format(line))
        par = m.group('paragraph')
        if par == '':
            continue
        indent = l * config['lvl_indent']
        width = config['max_width'] - indent
        for fmtline in to_lines(par, width):
            ser += indent * ' ' + fmtline + '\n'
    return ser

Hi Gary,
thank you for providing the online service of the SPDX tools. This is really great. However I am currently no able to access them anymore. I am using as URL https://tools.spdx.org/ but I always get "ERR_TIMED_OUT". Do I have the wrong URL?
Another question: I recently tried to validate a mid sized SPDX Tag-Value file (11.2 MB) and always got the error cannot connect to the system - is there a limitation in size for validation?

Thanks @OliverFendt for reporting the error.  I just checked and the server itself is down.  I'm rebooting it now.  Later tonight I'll see if I can find a cause for the outage.

@OliverFendt This server is back up

@goneall yes, it is working again. Thank you very much for taking care

I took a look at the system logs and didn't see any clues as to why the server was down.
I added an alert to send me an email if this situation happens again.

Closing this issue as it hasn't happened in a while

authors: OliverFendt goneall rtgdk

repo: spdx/spdx-online-tools


text:
Performance Optimization Enhancement for Verification

One way to speed up the verification algorithm is to cache the results of each element being verified.  It will take up more memory, but save having to re-parse and traverse the graph for elements if they are referenced more than once (e.g. within a relationship).

I did a some experimentation and found the issue is really spdx/Spdx-Java-Library#40 when using tag/value
If I comment out the check for removing in-use SPDX ID's, the time it take to verify keycloak-13.0.1.zip-too-big goes from 1:42 down to less than 6 seconds on my machine - a 16X improvement in performance.

@puerco - Can you point me to one of the output files for Kubernetes referenced in your tweet?  - I'll verify it is the same issue.
I suspect the issue is related to the tag/value format - it may not occur in JSON format, but I'm not certain.

Yes glad to! Here is a copy of one of them:
kubernetes-source.spdx.gz
This file is almost the same as the Kubernetes v.1.22.0-beta.2  SBOM we will generate tomorrow.

BTW, I think I found an interesting fact that perhaps will be useful:
When I run the verification with tools-java-1.0.1-jar-with-dependencies.jar it takes 12m12.034s to complete.
On another computer, I had a copy of the deprecated spdx-tools-2.2.5-jar-with-dependencies.jar version. This one runs much much faster. It takes 2m19.739s  to process the file.
I hope that helps and let me know if I can help with anything else!

Thanks @puerco  - This is almost certainly related to the delete performance issue referenced above.  I'll do some analysis and see if I can come up with a fix.

I just finished running some experiments with 3 scenarios:

Baseline: Current version of the SPDX tools
No in use: Remove in use check on delete (hack to work around the delete performance problem)
No in use + no related element: In addition to working around the in use check performance problem, don't verify the related SPDX element (works around the performance issue this issue covers)

Here's the results:

Baseline: 1,163 seconds
No in use: 174 seconds
No use no related element: 162

From the above results, it looks like there is a very significant issue with the deserialization of tag/value due to the delete performance issue.
This problem should not occur if other serialization formats are used (e.g. JSON).
I have a few ideas on how to fix this which I'll research over the next couple of days.

I just created https://github.com/spdx/Spdx-Java-Library/pull/56 which resolves the delete performance issue.
The kubernetes-source.spdx now validates in 109 seconds, more than a 10X improvement.

Moving this issue to the Spdx Java Library

authors: goneall puerco

repo: spdx/tools-java


text:
SPEAR -> MuJoCo scene export

TODO

[x] move environment.yml to setup.py
[x] improve README.md
[x] reverse joint parent-child order, based on new SPEAR content guideline
[ ] replace all Blender operations with trimesh

Closing this PR because we have a functional MuJoCo interop demo checked into main.

repo: spear-sim/spear


text:
Webworker fx

Would be nice to have out of box effects working in webworker.
Sample package https://github.com/ampproject/worker-dom

not spect concern.

repo: spectjs/spect


text:
Feature/gmm model native

Feature introduces the Gaussian Mixture Model structure that will serve as return type generated by GMM-family algorithms.

 Native GmmModel

repo: spectre-team/spectre


text:
How to use TestConsole for testing CLI prompts workflows?

Hello,
I am working on a CLI tool and would like to test the workflow of it.
After some trial and error, and by looking at the source code of this project I was able to find the TestConsole class.
It allowed me to write code like that:
open Spectre.Console.Testing
open Spectre.Console

[<Test>]
let ``Prompt.commitType works`` () =
    let console = new TestConsole()
    console.Profile.Capabilities.Interactive <- true

    AnsiConsole.Console <- console

    console.Input.PushKey(ConsoleKey.DownArrow)
    console.Input.PushKey(ConsoleKey.Enter)

    let selectedCommitType = Prompt.commitType CommitParserConfig.Default

    Expect.equal
        selectedCommitType
        CommitParserConfig.Default.Types[1]

[<Test>]
let test () =
    let console = new TestConsole()
    console.Profile.Capabilities.Interactive <- true

    AnsiConsole.Console <- console

    console.Input.PushKey(ConsoleKey.DownArrow)
    console.Input.PushKey(ConsoleKey.Enter)
    console.Input.PushTextWithEnter("an amazing feature")

    let selectedCommitType = Prompt.commitType CommitParserConfig.Default
    let shortMessage = Prompt.shortMessage ()

    printfn "%A" selectedCommitType
    printfn "%A" shortMessage

I am not sure if using AnsiConsole.Console <- console is the correct way of doing it? I am asking because I didn't see something similar in spectre.console repository.
I am also wondering if I should not revert AnsiConsole.Console to its original value after the test?

You should test using the TestConsole, and not use AnsiConsole.Console at all.

Also, your control should accept an instance of IAnsiConsole, not working with AnsiConsole directly.

Oh I see, I didn't think about it this way.
So you mean that I should convert:

let shortMessage () =
    AnsiConsole.Clear()

    TextPrompt<string>("Short message:") |> AnsiConsole.Prompt

to
let shortMessage (console :IAnsiConsole) =
    console.Clear()

    TextPrompt<string>("Short message:") |> console.Prompt

and then pass either AnsiConsole.Console or new TestConsole()
Thank you

authors: MangelMaxime patriksvensson


text:
Allow table title to not be capitalized

Is your feature request related to a problem? Please describe.
I'm converting an app from System.Console to Spectre.Console and in migrating a custom table output to use Spectre's Table resulted in a failing unit test because the table title (which in my case is a filename) no longer matches the actual filename, since the first character of table titles are capitalized in TableRenderer.
Describe the solution you'd like
Even if capitalizing the first letter is the default, I'd love an option to override it for when it's not desired
Describe alternatives you've considered
My current workaround is just to pad the string with a space so that the first character isn't uppercased, but that's not ideal.
Additional context
Thanks for building such a wide-ranging and configurable library!

@SteveDesmond-ca That makes perfect sense to me.
When I think about it, I don't think we should capitalize the first letter at all.

if no one has picked this, can i get started with this as my first contribution ?
@patriksvensson

@zmrfzn That would be awesome. Much appreciated!

Awesome!
thanks for the quick response, i am gonna setup and start with the setup today

authors: SteveDesmond-ca patriksvensson zmrfzn


text:
Update interfaces.py

The modification in the threshold is very important to ensure the accuracy for speaker verfication through SpeakerRecognition class.

The modification in the threshold is very important to ensure the accuracy for speaker verfication through SpeakerRecognition class. In the former code the threshold is defined to be 0.25, which gives an erroneous result. Redefining it to 0.5 gives quite better results.

IMO if this value is to be changed it should be done very very carefully, especially as this is defined globally rather than at model level.
Given the applications of speaker recognition, I think it may be more worthwhile to add a big fat warning to tell users that they should carefully evaluate the threshold for their particular model and data.
I am using speechbrain/spkrec-ecapa-voxceleb for a university assignment, using a subset of VoxCeleb for evaluation. Coincidentally, I had been testing threshold values. Unless I got something wrong, some of my findings are as follows (where "seuil" stands for "threshold"):

where
    false_reject = false_negative / (true_positive + false_negative) # FR
    false_accept = false_positive / (false_positive + true_negative) # FA

As you can see, the value of 0.25 is much, much closer to the EER (i.e. where FA/FR intersect) than to 0.5. That is not to say that a higher threshold might not be desirable for applications with much stronger FR requirements, but I don't think the default should be changed.

Closing for the reason I provided, IMO the fix would be better documentation

authors: AsuMagic asumagic joshipunitram

repo: speechbrain/speechbrain


text:
Fix regression of .Truncated evaluation in manual summaries

This fixes the behavior of .Truncated that was introduced with commit
bef496b97e1dc2df8c52b080152d539a58752801 which was later broken.  The
desired behavior is that .Truncated would evaluate to false when there
was nothing after the user defined summary marker.
This also adds a simple unit test to ensure that this feature isn't
broken again.  The check for content after the user defined summary
marker is done on the raw content instead of the working copy because
some of the markup renderers add elements after the marker, making it
difficult to determine if there is actually any content.
The behavior (evaluating to false when there is no content, just
summary) is also now documented.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Anton Staaf seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.

Hi Bjørn, I've updated to only do the raw data split once, I've also included in this version the removal of some dead code, that might have been a mistake though.  Let me know if you would like me to make a separate pull request for the removal.

authors: CLAassistant socialhacker


text:
lazyQuote option for stringSlice

possible fix for https://github.com/spf13/pflag/issues/370

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

authors: CLAassistant dbenque


text:
faster plugin manager

I feel like there was an issue on this topic but I can't find it...
Is there still talk of moving towards a faster (threaded) plugin manager with lazy loading?
Some good candidates would be: Neobundle or vim-plug
Neobundle is the one that Neovim seems to be going with: neovim/neovim#247
I was thinking we could just wrap the Bundle/UnBundle function we use to call Neobundle instead of Vundle (if that is possible in VimL)

I just want to call some attention to this issue. There's a big potential for speeding things up, vim-plug allows you to do this:
Plug 'tpope/vim-fireplace', { 'for': 'clojure' }

Which means a plugin only gets loaded for clojure files. I have big configurations for multiple languages, I think this could really cut down both startup time and general performance.

I would also really like to see vim-plug added. Or at least, some tutorial provided for setting it up as an alternative. I'd rather not mutilate my .vimrc.bundles, but I will.

1 .  remove vundles reference in .vimrc.bundles
2 .  add the following in your .vimrc.before.local
function! s:deregister(repo)
  let repo = substitute(a:repo, '[\/]\+$', '', '')
  let name = fnamemodify(repo, ':t:s?\.git$??')
  call remove(g:plugs, name)
  call remove(g:plugs_order, index(g:plugs_order, name))
endfunction

command! -nargs=1 -bar UnPlug call s:deregister(<args>)

" compatibility with Vundle
command! -nargs=+ -bar Bundle call plug#(<args>)
command! -nargs=1 -bar UnBundle call s:deregister(<args>)

3 .  add
call plug#begin()

and
call plug#end()

around  source ~/.vimrc.bundles in .vimrc

authors: haifengkao mwilliammyers skorlir tinco


text:
Pretty print tooltips that are shown in Vega

Ideally, named modules should have their names printed, with new lines for different parameters (e.g., convolution stride, etc). Should also have input/output sizes (this would be per invocation, which might be tricky).

Other things to potentially include: the pamaeter count

repo: spfrommer/torchexplorer


text:
Visitor Pass Generation

Visitor Details Form:
Design and implement a form where users can input their details, such as name and time.
Ensure the form is user-friendly and includes validation to ensure all required fields are filled out correctly.
Generate Visitor Pass:
Integrate the visitor details from the form with the QR code generation process.
Dynamically generate a QR code containing the visitor's details using the selected QR code generation library.
Download and Display Visitor Pass:
Implement functionality to allow users to download the generated visitor pass as an image file.
Display the visitor pass on the webpage for users to see and verify.

Visitor form have
Name
Name of the company
Phone Number
These three  details will be displayed in Visitor pass
other details such as
email address and address are required to be provided but will be handled in backend later.
the random id will be generated along with the pass details which will act as the key for each data.
the date and time will be added to pass dynamically via js function.

The visitor pass is generated like this.

THis is how the interface is looking.


text:
Separate modules for plugin and ProtoData model

This PR introduces two separate modules: one for the prototype definitions and the other for the plugin itself. This is necessary because the ProtoData plugin should be compiled and deployed before the ProtoData model can be processed.

@Oleg-Melnik Let's fix the PR description: we don't have any "prototype definitions". We have Protobuf definitions.

authors: Oleg-Melnik armiol

repo: spine-examples/Hello-ProtoData


text:
[BUG]data loss: using file/mvfile/seqfile

eventgen.conf

[film.json]
disabled = false
index = test_123
count = 1000
mode = sample
outputMode = splunkstream
interval = 10
end = 1
earliest = -1m
autotimestamp = true
sourcetype = _json
sampletype = raw
source = film.json
outputMode = httpevent
generatorWorkers =1
httpeventServers = {"servers": [{"protocol": "https", "port": "8088", "key": "6ef943c6-cb4c-4763-abf8-1806cb825b6c", "address": "10.224.68.179"}]}
httpeventOutputMode = mirror
httpeventMaxPayloadSize = 10000
token.0.token = "FILM_ID":(\d+)
token.0.replacementType = integerid
token.0.replacement = 0
token.1.token = "REGION_ID":(\d+)
token.1.replacementType = seqfile
token.1.replacement = /root/data/default/samples/count10.txt

eventgen cmd:
splunk_eventgen generate eventgen.conf
content of count10.txt

0
1
2
3
4
5
6
7
8
9

film.json(1K lines)

{"FILM_ID":0,"DIRECTOR_ID":0,"REGION_ID":0,"TITLE":"CENTERDINOSAUR","DESCRIPTION":"ABeautifulCharacterStudyofaSumoWrestlerAndaDentistwhomustFindaDoginCalifornia","RELEASE_YEAR":2006,"LANGUAGE_ID":1,"ORIGINAL_LANGUAGE_ID":"(null)","RENTAL_DURATION":5,"RENTAL_RATE":4.99,"LENGTH":152,"REPLACEMENT_COST":12.99,"RATING":"PG","SPECIAL_FEATURES":"DeletedScenes","LAST_UPDATE":"2/15/0605:03"},

Expected results:
1000 events generated
Actual results:
910 events generated

@YifengMao ,could you attache the bundle including eventgen.conf and sample files?

@YifengMao ,could you attache the bundle including eventgen.conf and sample files?

already included in description

I was using a similar film.json file that I found in our large test samples to reproduce the issue, and I noticed that the first event was missing an opening bracket:
"FILM_ID":0,"DIRECTOR_ID":0,"REGION_ID":0,"TITLE":"CENTERDINOSAUR","DESCRIPTION":"ABeautifulCharacterStudyofaSumoWrestlerAndaDentistwhomustFindaDoginCalifornia","RELEASE_YEAR":2006,"LANGUAGE_ID":1,"ORIGINAL_LANGUAGE_ID":"(null)","RENTAL_DURATION":5,"RENTAL_RATE":4.99,"LENGTH":152,"REPLACEMENT_COST":12.99,"RATING":"PG","SPECIAL_FEATURES":"DeletedScenes","LAST_UPDATE":"2/15/0605:03","_time":"Mon Jul 13 09:30:00 PDT 2017"},
I see all the events when I output to stdout or a file, and seqfile was not affecting the behavior. Eventgen was parsing the line as an event (which counts towards the 1000), however Splunk never indexes the event because of the json formatting. After fixing the json, I was able to generate all 1000 events with your eventgen.conf no problem.
Can you check your sample file for any json formatting errors or provide the full sample file for reference? I suspect this is the root cause.

@sjacobs-splunk i don't find any json misformat issues in my film.json.
Actually, if you use my eventgen.conf, and remove the lines of seqfile token replacement, it can produce 1000 events. (seqfile or mvfile both have this issue)
token.1.token = "REGION_ID":(\d+)
token.1.replacementType = seqfile
token.1.replacement = /root/data/default/samples/count10.txt
film.txt
please rename it to film.json

I can not reproduce it with above conf and sample file with the latest version. I will show you tomorrow. @YifengMao

authors: YifengMao jmeixensperger li-wu

repo: splunk/eventgen


text:
Dfds servicebuilder https enabled v2

Hey, I just made a Pull Request!

:heavy_check_mark: Checklist

[X] All tests are passing yarn test
[X] Screenshots attached (for UI changes)
[X] Relevant documentation updated
[X] Prettier run on changed files
[X] Tests added for new functionality
[X] Regression tests added for bug fixes

Sorry for adding the "microsite edits" to this PR. But it came in last night from your master and I couldnt be asked to undo the commits. :)

CLI test are timing out after 5hrs and X mins. Not sure what to do from here. :)

Let us know if there's anything we can do to assist today.

authors: ZaradarDFDS freben

repo: spotify/backstage


text:
cli: introduce jest transformModules options for providing a list of modules to transform

Jest tranform options are messy af. This is the best method of providing people with a possibility to transform some modules that I could come up with.

@freben people don't always publish nicely transpiled CommonJS modules, which is what jest needs. In the case of @kyma-project/asyncapi-react it publishes ESM modules, meaning we need jest to transform them. We also don't want people to have to send PRs for each package that needs an override, so we allow it to be specified in the jest config of each project instead.
Hopefully we can get rid of all of this one jest supports modules.


text:
Can't read file on image build

Issue:
Step 1 : ADD //home/bes/repo/colada.ms/LogAggregator/target/LogAggregator-1.0.0-SNAPSHOT.jar /opt/LogAggregator-1.0.0-SNAPSHOT.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 5.285s
[INFO] Finished at: Mon Sep 21 17:07:50 MSK 2015
[INFO] Final Memory: 30M/387M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.spotify:docker-maven-plugin:0.3.2:build (default-cli) on project LogAggregator: Exception caught: /home/bes/repo/colada.ms/LogAggregator/target/LogAggregator-1.0.0-SNAPSHOT.jar: no such file or directory -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal com.spotify:docker-maven-plugin:0.3.2:build (default-cli) on project LogAggregator: Exception caught
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:217)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:320)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:537)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:196)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:141)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoExecutionException: Exception caught
at com.spotify.docker.AbstractDockerMojo.execute(AbstractDockerMojo.java:95)
at com.spotify.docker.BuildMojo.execute(BuildMojo.java:79)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:101)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:209)
... 19 more
Caused by: com.spotify.docker.client.DockerException: /home/bes/repo/colada.ms/LogAggregator/target/LogAggregator-1.0.0-SNAPSHOT.jar: no such file or directory
at com.spotify.docker.client.AnsiProgressHandler.progress(AnsiProgressHandler.java:51)
at com.spotify.docker.client.DefaultDockerClient.build(DefaultDockerClient.java:830)
at com.spotify.docker.BuildMojo.buildImage(BuildMojo.java:517)
at com.spotify.docker.BuildMojo.execute(BuildMojo.java:307)
at com.spotify.docker.AbstractDockerMojo.execute(AbstractDockerMojo.java:93)
... 22 more
but
bes@debian:~/repo/colada.ms/LogAggregator$ ls -l //home/bes/repo/colada.ms/LogAggregator/target/LogAggregator-1.0.0-SNAPSHOT.jar
-rw-r--r-- 1 bes bes 7239613 Sep 21 17:07 //home/bes/repo/colada.ms/LogAggregator/target/LogAggregator-1.0.0-SNAPSHOT.jar

Sorry, dockerfile context missed

I have the same issue. How did you solved it?

ybalt.. Can you please share the solution.. I am also facing the same ussue.

authors: bapetc brijchavda ybaltouski

repo: spotify/docker-maven-plugin


text:
Support an optional init script

Suggestion - allow the writing of a shell script into the container that is launched by the entrypoint.
The shell script would then be responsible for launching the java process. This allows for easy interpolation of things like JAVA_OPTS as an environment variable much like Tomcat.

If you supply your own Dockerfile, this sort of thing should be pretty easy to accomplish. The part of the plugin that generates a Dockerfile for you only really for convenience and I think we would be wary of adding more logic to it.

authors: jmkgreen mattnworb


text:
Luigi pipeline

Hi,
I'm trying for the human exome dataset, it was exactly 29 hrs still it running. can you check this ....
CMD
Luigi python -m luigi --module Luigipipeline Custom_Genome_Pipeline --local-scheduler
DEBUG: Checking if Custom_Genome_Pipeline() is complete
DEBUG: Checking if Convert_Bcf_Vcf(sample=SRR098401_1) is complete
DEBUG: Checking if Convert_Bcf_Vcf(sample=SRR098401_2) is complete
INFO: Informed scheduler that task   Custom_Genome_Pipeline__99914b932b   has status   PENDING
DEBUG: Checking if Call_Variant(sample=SRR098401_2) is complete
INFO: Informed scheduler that task   Convert_Bcf_Vcf_SRR098401_2_1153ce33d1   has status   PENDING
DEBUG: Checking if Index_Bam(sample=SRR098401_2) is complete
Indexing ... Done
/home/likithreddy/miniconda3/lib/python3.6/site-packages/luigi/worker.py:334: UserWarning: Task Index_Bam(sample=SRR098401_2) without outputs has no custom complete() method
is_complete = task.complete()
INFO: Informed scheduler that task   Call_Variant_SRR098401_2_1153ce33d1   has status   PENDING
DEBUG: Checking if Sort_Bam(sample=SRR098401_2) is complete
/home/likithreddy/miniconda3/lib/python3.6/site-packages/luigi/worker.py:334: UserWarning: Task Sort_Bam(sample=SRR098401_2) without outputs has no custom complete() method
is_complete = task.complete()
INFO: Informed scheduler that task   Index_Bam_SRR098401_2_1153ce33d1   has status   PENDING
DEBUG: Checking if Convert_Sam_Bam(sample=SRR098401_2) is complete
INFO: Informed scheduler that task   Sort_Bam_SRR098401_2_1153ce33d1   has status   PENDING
DEBUG: Checking if Bwa_Mem(sample=SRR098401_2) is complete
INFO: Informed scheduler that task   Convert_Sam_Bam_SRR098401_2_1153ce33d1   has status   PENDING
INFO: Informed scheduler that task   Bwa_Mem_SRR098401_2_1153ce33d1   has status   PENDING
DEBUG: Checking if Call_Variant(sample=SRR098401_1) is complete
INFO: Informed scheduler that task   Convert_Bcf_Vcf_SRR098401_1_7300c92048   has status   PENDING
DEBUG: Checking if Index_Bam(sample=SRR098401_1) is complete
Indexing ... Done
/home/likithreddy/miniconda3/lib/python3.6/site-packages/luigi/worker.py:334: UserWarning: Task Index_Bam(sample=SRR098401_1) without outputs has no custom complete() method
is_complete = task.complete()
INFO: Informed scheduler that task   Call_Variant_SRR098401_1_7300c92048   has status   PENDING
DEBUG: Checking if Sort_Bam(sample=SRR098401_1) is complete
/home/likithreddy/miniconda3/lib/python3.6/site-packages/luigi/worker.py:334: UserWarning: Task Sort_Bam(sample=SRR098401_1) without outputs has no custom complete() method
is_complete = task.complete()
INFO: Informed scheduler that task   Index_Bam_SRR098401_1_7300c92048   has status   PENDING
DEBUG: Checking if Convert_Sam_Bam(sample=SRR098401_1) is complete
INFO: Informed scheduler that task   Sort_Bam_SRR098401_1_7300c92048   has status   PENDING
DEBUG: Checking if Bwa_Mem(sample=SRR098401_1) is complete
INFO: Informed scheduler that task   Convert_Sam_Bam_SRR098401_1_7300c92048   has status   PENDING
INFO: Informed scheduler that task   Bwa_Mem_SRR098401_1_7300c92048   has status   PENDING
INFO: Done scheduling tasks
INFO: Running Worker with 1 processes
DEBUG: Asking scheduler for work...
DEBUG: Pending tasks: 13
INFO: [pid 3826] Worker Worker(salt=905540885, workers=1, host=CURIUM, username=likithreddy, pid=3826) running   Bwa_Mem(sample=SRR098401_2)
[M::bwa_idx_load_from_disk] read 0 ALT contigs
[M::process] read 131580 sequences (10000080 bp)...
[M::process] read 131580 sequences (10000080 bp)...
[M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (0, 0, 0, 0)
[M::mem_pestat] skip orientation FF as there are not enough pairs
[M::mem_pestat] skip orientation FR as there are not enough pairs
[M::mem_pestat] skip orientation RF as there are not enough pairs
[M::mem_pestat] skip orientation RR as there are not enough pairs
[M::mem_process_seqs] Processed 131580 reads in 11.895 CPU sec, 11.829 real sec
[M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (0, 0, 0, 0)
[M::mem_pestat] skip orientation FF as there are not enough pairs
[M::mem_pestat] skip orientation FR as there are not enough pairs
[M::mem_pestat] skip orientation RF as there are not enough pairs
[M::mem_pestat] skip orientation RR as there are not enough pairs
[M::mem_process_seqs] Processed 131580 reads in 13.610 CPU sec, 13.611 real sec
Since yesterday, its showing the same prompt..
Here is Luigi.py script.
from subprocess import check_output
import subprocess
import luigi
import os
import random
#faking the pipline for now
sample_list = ["SRR098401_1","SRR098401_2"]
genome = "reference/hg38.fa"
def run_cmd(cmd):
p = subprocess.Popen(cmd, shell=False, universal_newlines=True, stdout=subprocess.PIPE)
p.wait()
output = p.communicate()[0]
return output
class Bwa_Mem(luigi.Task):
# Ideally we have to show this but for now I am gonna fake it by creating a list of sample IDs
# fastq_path = luigi.Parameter()

sample = luigi.Parameter()
#genome = luigi.Parameter()

#def requires(self):
#    return [LoadFastq(sample) for sample in sample_list]

def output(self):
    return luigi.LocalTarget("sam/%s.sam" % self.sample)

def run(self):
    tmp = run_cmd(["bwa",
                 "mem",
                 genome,
                 "fastq/"+self.sample+".fastq",
                 "fastq/"+self.sample+".fastq"])
    with self.output().open('w') as sam_out:
        sam_out.write(tmp)
    print ("====== Running BWA ======")

class Convert_Sam_Bam(luigi.Task):
sample = luigi.Parameter()
#genome = luigi.Parameter()

def requires(self):
    return Bwa_Mem(self.sample)
def output(self):
    return luigi.LocalTarget("bam/%s.bam" % self.sample)
def run(self):
    tmp = run_cmd(["samtools",
                   "view",
                   "-bS",
                   "sam/"+self.sample+".sam"])
    with self.output().open('w') as bam_out:
        bam_out.write(tmp)
    print ("===== Converting Sam file to Bam file ======")

class Sort_Bam(luigi.Task):
sample = luigi.Parameter()

def requires(self):
    return Convert_Sam_Bam(self.sample)
def run(self):
    tmp = run_cmd(["samtools",
                   "sort",
                   "bam/"+self.sample+".bam",
                   "bam/"+self.sample+".sorted"])
    print ("===== Sorting Bam =======")

class Index_Bam(luigi.Task):
sample = luigi.Parameter()

def output(self):
    print ("Indexing ... Done")
def requires(self):
    return Sort_Bam(self.sample)
def run(self):
    tmp = run_cmd(["samtools",
                   "index",
                   "bam/"+self.sample+".sorted.bam"])
    print ("==== Indexing Bam ====")

class Call_Variant(luigi.Task):
sample = luigi.Parameter()
def requires(self):
    return Index_Bam(self.sample)
def output(self):
    return luigi.LocalTarget("bcf/%s.bcf" % self.sample)
def run(self):
    tmp = run_cmd(["samtools",
                   "mpileup",
                   "-u",
                   "-f",
                   genome,
                   "bam/"+self.sample+".sorted.bam"])
    with self.output().open("w") as bcf_file:
        bcf_file.write(tmp)
    print ("=====  Calling Variants =======")

class Convert_Bcf_Vcf(luigi.Task):
sample = luigi.Parameter()

def requires(self):
    return Call_Variant(self.sample)
def output(self):
    return luigi.LocalTarget("vcf/%s.vcf" % self.sample)
def run(self):
    tmp = run_cmd(["bcftools",
                   "view",
                   "-v",
                   "-c",
                   "-g",
                   "bcf/"+self.sample+".bcf"])
    with self.output().open("w") as vcf_file:
        vcf_file.write(tmp)

    print ("===== Creating VCF =======")

class Custom_Genome_Pipeline(luigi.Task):
def requires(self):
    return [Convert_Bcf_Vcf(sample) for sample in sample_list]

def output(self):
    return luigi.LocalTarget("log.txt")

def run(self):
    print ("running...")

for sample in sample_list:
print(sample + "\n")
return Convert_Bcf_Vcf(sample)
if name=='main':
luigi.run()

closing this issue.

[ ] It has been inactive for +4 months.
[ ] It's not about luigi core, so not as many users are affected about this.
[ ] The change seems quite big, it's unlikely to be sporadically picked up.
[ ] The owner haven't responded or disappeared.
[ ] I don't understand what this issue is about.
[ ] There exists a reasonable workaround for this.
[ ] We need to check if this hasn't been fixed by now (for old issues).
[ ] This is kind of by design and not a bug.
[ ] Resolving this would probably add a lot of complexity.
[x] This is a question not an issue. Use SO or the mailing list.

Every open issue adds some clutter, and we try to make the issues fewer and make it easier for new collaborators to find. Currently we try to close any issue that meets the first checkbox + one other.
Feel free to reopen this issue at any point if you have the intent to continue to work this. :)

repo: spotify/luigi


text:
Cannot find krb5.h, enusre you have installed krb5 development library.

I hit a krb5.h error when pip install. It looks like there is a hard dependency on the python-krbV library. Is this absolutely required for snakebite to work? In the spotify blog post, it advertises itself as a a pure Python HDFS client.
Anyway, I commented out any references to sasl and kerberos. Snakebite seems to work this way.

Snakebite itself is pure python but relies on other packages, which imho is fine. Not sure what to do about the dependencies of the python-krbV library though. My pip foo isn't that great, but maybe @ravwojdyla can help out.

Also, we currently rely on python-prototobuf  which is a C package as well.

This does create a problem however. I'm running into this issue when building the docs.

I've pushed a PR that should address this:
https://github.com/spotify/snakebite/pull/170

Wonderful! Can you post to pypi?

One workaround until 2.7.0 is uploaded is to use
pip install snakebite==2.5.5

Done!

authors: tungwaiyip wouterdebie

repo: spotify/snakebite


text:
Add support for EKS Blueprints in examples

Add support for EKS Blueprints in our examples for Ocean Spark on AWS. EKS Blueprints are a much easier way to deploy EKS clusters via Terraform in AWS.

authors: joshdlee loganjohnlong

repo: spotinst/terraform-spotinst-ocean-spark


text:
统一各组件dubbo版本

spring-cloud-alibaba-sentinel 使用的是alibaba-dubbo-2.6.5，但它已经兼容apache-dubbo-2.7.1了。
spring-cloud-alibaba-dubbo用的是apache-dubbo-2.7.1。

Yes, see 0e6565b461d619d2b7645f96f8fcc0e1dfd6c297.


text:
@RemoteApplicationEventScan usage alters behaviour

Hello,
spring boot: 2.1.2.RELEASE
spring-cloud: Greenwich.RELEASE
When I use following annotation in main class
@RemoteApplicationEventScan(basePackages =
  { "tr.com....", "org.springframework.cloud.bus.event" })

org.springframework.messaging.converter.MappingJackson2MessageConverter

is instantiated before
org.springframework.cloud.bus.jackson.BusJacksonAutoConfiguration

as a result UnknownRemotEventListener is not thrown. instead serialiazation error is thrown.
If RemoteApplicationEventScan is ommitted UnknownRemotEventListener is thrown. But since custom packages are not scanned exception is UnknownRemotEventListener for all custom events.
As a workaround I've copied BusJacksonAutoConfiguration and modified packagesToScan member then everything started work. Is there a better option? What am I missing?

Can you provide a complete, minimal, verifiable sample that reproduces the problem? It should be available as a GitHub (or similar) project or attached to this issue as a zip file.

attaching sample Comment out line 27 in ClientApp.java in order to observe behaviour
cloud-sample.zip

I'm still confused even though I have the sample.
You have two events defined in the client project, Event and UnknownEvent. Why would you not have @RemoteApplicationEventScan(basePackages = { "org.ugurlu" }) in that project? Without it how would you expect this to work?

Sorry for the confusion commenting out of RemoteApplicationEventScan should be done in ServerApp.java.
When we comment out scanning of base packages both events are captured by UnknownListener. If I enable base package scan in ServerApp Event is captured properly however instead of trigerring UnknownListener
Caused by: org.springframework.amqp.AmqpRejectAndDontRequeueException: org.springframework.messaging.converter.MessageConversionException: Could not read JSON: Could not resolve type id 'UnknownEvent' as a subtype of [simple type, class org.springframework.cloud.bus.event.RemoteApplicationEvent]: known type ids = [AckRemoteApplicationEvent, EnvironmentChangeRemoteApplicationEvent, Event, RefreshRemoteApplicationEvent, UnknownRemoteApplicationEvent]

exception is thrown.

seems like call order MappingJackson2MessageConverter, BusJacksonAutoConfiguration changes behaviour. use of  @RemoteApplicationEventScan can cause skipping of events. Event is received from queue however is not forwarded to application

RemoteApplicationEventScan shouldn't be used on spring cloud packages.

RemoteApplicationEventScan shouldn't be used on spring cloud packages.

I got incredibly confused here, as spring cloud documentation recommends using RemoveApplicationScan as follows:

If you cannot or don’t want to use a subpackage of org.springframework.cloud.bus.event for your custom events, you must specify which packages to scan for events of type RemoteApplicationEvent using @RemoteApplicationEventScan. Packages specified with @RemoteApplicationEventScan include subpackages.

Can you be more specific, please?

authors: msakirc ryanjbaxter spencergibb ugurlu

repo: spring-cloud/spring-cloud-bus