text:
Laravel 5.3 When seeding database: "This cache store does not support tagging."

I have been going nuts trying to figure out how to create users with my current setup.  I have Redis enabled, which does support tagging, and I have been scratching my head figuring out how to set this up with Entrust, which requires a cache with tagging.  I included all files that could be relevant.
composer.json
...
    "require": {
        "php": ">=5.6.4",
        "laravel/framework": "5.3.*",
        "tymon/jwt-auth": "^0.5.9",
        "zizaco/entrust": "5.2.x-dev",
        "predis/predis": "^1.1"
    },
...

.env
...
CACHE_DRIVER=redis
SESSION_DRIVER=redis

database/seeds/DatabaseSeeder.php
<?php

use Illuminate\Database\Seeder;
use Illuminate\Database\Eloquent\Model;
use App\User;

class DatabaseSeeder extends Seeder
{
    /**
     * Run the database seeds.
     *
     * @return void
     */
    public function run()
    {
        Model::unguard();

        DB::table('users')->delete();

        $users = array(
            ['name' => 'Ryan Chenkie', 'email' => 'ryanchenkie@gmail.com', 'password' => Hash::make('secret')],
            ['name' => 'Chris Sevilleja', 'email' => 'chris@scotch.io', 'password' => Hash::make('secret')],
            ['name' => 'Holly Lloyd', 'email' => 'holly@scotch.io', 'password' => Hash::make('secret')],
            ['name' => 'Adnan Kukic', 'email' => 'adnan@scotch.io', 'password' => Hash::make('secret')],
        );

        foreach ($users as $user)
        {
            User::create($user);

            Model::reguard();
        }
    }
}

database\migrations\2016_09_08_014637_create_users_table.php
<?php

use Illuminate\Support\Facades\Schema;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Database\Migrations\Migration;

class CreateUsersTable extends Migration
{
    /**
     * Run the migrations.
     *
     * @return void
     */
    public function up()
    {
        Schema::create('users', function (Blueprint $table) {
            $table->increments('id');
            $table->string('name');
            $table->string('email')->unique();
            $table->string('password', 60);
            $table->rememberToken();
            $table->timestamps();
        });
    }
    /**
     * Reverse the migrations.
     *
     * @return void
     */
    public function down()
    {
        Schema::drop('users');
    }
}

database/migrations/2016_09_08_014640_entrust_setup_tables.php
<?php
use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;

class EntrustSetupTables extends Migration
{
    /**
     * Run the migrations.
     *
     * @return  void
     */
    public function up()
    {
        // Create table for storing roles
        Schema::create('roles', function (Blueprint $table) {
            $table->increments('id');
            $table->string('name')->unique();
            $table->string('display_name')->nullable();
            $table->string('description')->nullable();
            $table->timestamps();
        });

        // Create table for associating roles to users (Many-to-Many)
        Schema::create('role_user', function (Blueprint $table) {
            $table->integer('user_id')->unsigned();
            $table->integer('role_id')->unsigned();

            $table->foreign('user_id')->references('id')->on('users')
                ->onUpdate('cascade')->onDelete('cascade');
            $table->foreign('role_id')->references('id')->on('roles')
                ->onUpdate('cascade')->onDelete('cascade');

            $table->primary(['user_id', 'role_id']);
        });

        // Create table for storing permissions
        Schema::create('permissions', function (Blueprint $table) {
            $table->increments('id');
            $table->string('name')->unique();
            $table->string('display_name')->nullable();
            $table->string('description')->nullable();
            $table->timestamps();
        });

        // Create table for associating permissions to roles (Many-to-Many)
        Schema::create('permission_role', function (Blueprint $table) {
            $table->integer('permission_id')->unsigned();
            $table->integer('role_id')->unsigned();

            $table->foreign('permission_id')->references('id')->on('permissions')
                ->onUpdate('cascade')->onDelete('cascade');
            $table->foreign('role_id')->references('id')->on('roles')
                ->onUpdate('cascade')->onDelete('cascade');

            $table->primary(['permission_id', 'role_id']);
        });
    }

    /**
     * Reverse the migrations.
     *
     * @return  void
     */
    public function down()
    {
        Schema::drop('permission_role');
        Schema::drop('permissions');
        Schema::drop('role_user');
        Schema::drop('roles');
    }
}

app/User.php
<?php

namespace App;

use Illuminate\Auth\Authenticatable;
use Illuminate\Database\Eloquent\Model;
use Illuminate\Auth\Passwords\CanResetPassword;
use Illuminate\Contracts\Auth\Authenticatable as AuthenticatableContract;
use Illuminate\Contracts\Auth\Access\Authorizable as AuthorizableContract;
use Illuminate\Contracts\Auth\CanResetPassword as CanResetPasswordContract;
use Zizaco\Entrust\Traits\EntrustUserTrait;

class User extends Model implements AuthenticatableContract,
                                    AuthorizableContract,
                                    CanResetPasswordContract
{
    use Authenticatable, CanResetPassword, EntrustUserTrait;

    /**
     * The attributes that are mass assignable.
     *
     * @var array
     */
    protected $fillable = [
        'name', 'email', 'password',
    ];

    /**
     * The attributes that should be hidden for arrays.
     *
     * @var array
     */
    protected $hidden = [
        'password', 'remember_token',
    ];
}

config/database.php
...
    'redis' => [

        'cluster' => false,

        'default' => [
            'host' => '127.0.0.1',
            'port' => 6379,
            'database' => 0,
        ],

    ],

config/cache.php
...
'default' => env('CACHE_DRIVER', 'redis'),
...
    'stores' => [

        'apc' => [
            'driver' => 'apc',
        ],

        'array' => [
            'driver' => 'array',
        ],

        'database' => [
            'driver' => 'database',
            'table' => 'cache',
            'connection' => null,
        ],

        'file' => [
            'driver' => 'file',
            'path' => storage_path('framework/cache'),
        ],

        'memcached' => [
            'driver' => 'memcached',
            'persistent_id' => env('MEMCACHED_PERSISTENT_ID'),
            'sasl' => [
                env('MEMCACHED_USERNAME'),
                env('MEMCACHED_PASSWORD'),
            ],
            'options' => [
                // Memcached::OPT_CONNECT_TIMEOUT  => 2000,
            ],
            'servers' => [
                [
                    'host' => env('MEMCACHED_HOST', '127.0.0.1'),
                    'port' => env('MEMCACHED_PORT', 11211),
                    'weight' => 100,
                ],
            ],
        ],

        'redis' => [
            'driver' => 'redis',
            'connection' => 'default',
        ],

    ],

@kylelamse I came across same issue when I tried with file caching. I changed it to CACHE_DRIVER=array in .env and it worked.

@kylelamse
U need patch the files by extending the model files User.php

public function cachedRoles()
    {
        $userPrimaryKey = $this->primaryKey;
        $cacheKey = 'entrust_roles_for_user_'.$this->$userPrimaryKey;
        if(Cache::getStore() instanceof TaggableStore) {
            return Cache::tags('role_user')->remember($cacheKey, Config::get('cache.ttl'), function () {
                return $this->roles()->get();
            });
        } else {
             return $this->roles()->get();
        }
    }

    public function save(array $options = [])
    {   //both inserts and updates
        if(Cache::getStore() instanceof TaggableStore) {
            Cache::tags('role_user')->flush();
        }
        return parent::save($options);
    }
    public function delete(array $options = [])
    {   //soft or hard
        if(Cache::getStore() instanceof TaggableStore) {
            Cache::tags('role_user')->flush();
        }
        return parent::delete($options);;
    }
    public function restore()
    {   //soft delete undo's
        if(Cache::getStore() instanceof TaggableStore) {
            Cache::tags('role_user')->flush();
        }
        return parent::restore();;
    }

Extend the rule model and patch the function

public function cachedPermissions()
    {
        $rolePrimaryKey = $this->primaryKey;
        $cacheKey = 'entrust_permissions_for_role_'.$this->$rolePrimaryKey;
        if(Cache::getStore() instanceof TaggableStore) {
            return Cache::tags('permission_role')->remember($cacheKey, Config::get('cache.ttl'), function () {
                return $this->perms()->get();
            });
        } else {
            return $this->perms()->get();
        }
    }
    public function save(array $options = [])
    {   //both inserts and updates
        if(Cache::getStore() instanceof TaggableStore) {
            Cache::tags('permission_role')->flush();
        }
        return parent::save($options);
    }
    public function delete(array $options = [])
    {   //soft or hard
        if(Cache::getStore() instanceof TaggableStore) {
            Cache::tags('permission_role')->flush();
        }
        return parent::delete($options);
    }
    public function restore()
    {   //soft delete undo's
        if(Cache::getStore() instanceof TaggableStore) {
            Cache::tags('permission_role')->flush();
        }
        return parent::restore();
    }

Pull request has been given by few but still maintainers  have not still merged

@sunel Thanks for that, but isn't Redis an instance of a TaggableStore?  I would like to use Redis as my cache, but for some reason, it doesn't seem the Cache facade supports tagging even though it is set to use Redis.

@kylelamse yes
remember to add use Illuminate\Cache\TaggableStore;

@kylelamse
check this

class RedisStore extends TaggableStore implements Store

its belong  TaggableStore

@sunel
Where do I add use Illuminate\Cache\TaggableStore;?
I just want to make sure my Cache is using Redis.  Right now it seems like it's not using RedisStore at all with how I have it set up.

@kylelamse ok i will guide you lets go step by step
set this in .env
CACHE_DRIVER=redis
``

add this in ``` composer.json ```

"predis/predis": "^1.1"

do a ``` composer update```

now in ``` User.php ```

``` php

use Illuminate\Foundation\Auth\User as Authenticatable;
use Illuminate\Cache\TaggableStore;
use Illuminate\Support\Facades\Cache;
use Illuminate\Support\Facades\Config;

class User extends Authenticatable
{
     use EntrustUserTrait;

    public function cachedRoles()
    {
        $userPrimaryKey = $this->primaryKey;
        $cacheKey = 'entrust_roles_for_user_'.$this->$userPrimaryKey;
        if(Cache::getStore() instanceof TaggableStore) {
            return Cache::tags('role_user')->remember($cacheKey, Config::get('cache.ttl'), function () {
                return $this->roles()->get();
            });
        } else {
             return $this->roles()->get();
        }
    }

    public function save(array $options = [])
    {   //both inserts and updates
        if(Cache::getStore() instanceof TaggableStore) {
            Cache::tags('role_user')->flush();
        }
        return parent::save($options);
    }
    public function delete(array $options = [])
    {   //soft or hard
        if(Cache::getStore() instanceof TaggableStore) {
            Cache::tags('role_user')->flush();
        }
        return parent::delete($options);;
    }
    public function restore()
    {   //soft delete undo's
        if(Cache::getStore() instanceof TaggableStore) {
            Cache::tags('role_user')->flush();
        }
        return parent::restore();;
    }
}

now in Role.php

use Illuminate\Cache\TaggableStore;
use Illuminate\Support\Facades\Cache;
use Illuminate\Support\Facades\Config;
use Zizaco\Entrust\EntrustRole;

class Role extends EntrustRole
{
      public function cachedPermissions()
    {
        $rolePrimaryKey = $this->primaryKey;
        $cacheKey = 'entrust_permissions_for_role_'.$this->$rolePrimaryKey;
        if(Cache::getStore() instanceof TaggableStore) {
            return Cache::tags('permission_role')->remember($cacheKey, Config::get('cache.ttl'), function () {
                return $this->perms()->get();
            });
        } else {
            return $this->perms()->get();
        }
    }
    public function save(array $options = [])
    {   //both inserts and updates
        if(Cache::getStore() instanceof TaggableStore) {
            Cache::tags('permission_role')->flush();
        }
        return parent::save($options);
    }
    public function delete(array $options = [])
    {   //soft or hard
        if(Cache::getStore() instanceof TaggableStore) {
            Cache::tags('permission_role')->flush();
        }
        return parent::delete($options);
    }
    public function restore()
    {   //soft delete undo's
        if(Cache::getStore() instanceof TaggableStore) {
            Cache::tags('permission_role')->flush();
        }
        return parent::restore();
    }
}

@sunel I did all of that, but I still get
  [BadMethodCallException]
  This cache store does not support tagging.

When doing php artisan db:seed from that Seed file I posted.

@kylelamse
dd(get_class(Cache::getStore()))
post the output u get

@sunel
"Illuminate\Cache\FileStore"
So... It appears it's not even using Redis from that output

@kylelamse Yeah some config issue .. please check with it
dd(env('CACHE_DRIVER', 'redis'))

@sunel
Did you mean dd(env('CACHE_DRIVER'))?
dd(env('CACHE_DRIVER', 'redis'))
returns
"redis".
and dd(env('CACHE_DRIVER'))
returns
null

I ended up fixing it.
Here's what the issue was:
I had changed CACHE_DRIVER to CACHE_DRIVER=redis in .envbut never ranphp artisan config:cache. Doing that made dd(get_class(Cache::getStore()));return"Illuminate\Cache\RedisStore"`

👍

authors: Sarav-S kylelamse sunel

repo: Zizaco/entrust


text:
Table reload callback not working

according to issue #181:
I have the same issue and studied your doc about this topic. If I set the attrib bind-reload="getResource" the entire table is not rendered:

If I remove the attribs, it woks pretty fine.
My table:

                          {{ row.Name }}
                          {{ row.Basename }}
                          {{ row.Menge  | volumne }}
                          <uib-progressbar type="{{row._TYPE}}" value="(row.Restmenge / row.Menge) * 100">{{ row.Restmenge  | volumne }}</uib-progressbar>
                          {{ row.Nikotin }}mg
                          {{ row.Reifezeit }} {{ 'DAYS' | translate:translationData }}
                          {{ row.Preis | currency }}
                          {{ row.Herstellungsdatum | amUtc | amDateFormat:'L' }}
                          {{ row.Benutzer }}

My controller:
function databaseRecipeCtrl($rootScope, $scope, Item, Data, Notify, $timeout) {
    // Notify.send("ghfhfhhhfhhf");

    // $scope.radioModel = 10;
    $scope.reloadCallback = function () {};
    $scope.init = {
      'count': 10,
      'page': 1,
      'sortBy': 'name',
      'sortOrder': 'dsc',
      'filterBase': 1 // set false to disable
    };
    $scope.listItemsPerPage = [10, 25, 50, 100];

    $scope.getResource = function (params, paramsObj) {

      return Data.post('recipes', paramsObj).then(function(results) {
        //
        return {
          'rows': results.rows,
          'header': results.header,
          'pagination': results.pagination[0]
        }
      });
    }
    $rootScope.$on('UpdateRecipeTable',   $scope.reloadCallback());
}

Mentioned in #219, so this is closed, sorry!

source: gharchive/issue

repo: Zizzamia/ng-tasty

license: mit

license_type: permissive

license_source: bigquery


text:
Soft Navigations support

Hello!
Looks like that Soft Navigations is not so far to get out of the experimental status.
Maybe you already have some integration plans?
Or we can discuss it here)

Let's discuss :)
cc @lkuangPR

Hi Any advise on how to reset each metrics during soft navigation? Can this be supported currently by perfume.js until chrome is done with its experiment?
Thanks

authors: SuperOleg39 Zizzamia ashu-g

license: MIT

license_source: github-api


text:
[FEATURE] .git_hooks -> .githooks

提案の詳細

.git_hooks から .githooks に名前を変える
動機

こちらの方が一般的らしい
あと、ケバブケースのファイルが多いのでそれにそろえる目的もある
その他

これ #17 終わってからじゃないと README の更新しにくない？

repo: Zli-UoA/ryouomoi-checker-frontend


text:
changed Treznor NPC ID to reflect recent update

fixes #1303
Treznor's npc id was apparently updated  in late 2022, and I confirmed the new id 11957 to be accurate on my character (who did already have the quest done).

Potentially also fixes #1176

source: gharchive/pull-request

repo: Zoinkwiz/quest-helper


text:
Feature/location

Gets delivery address from current location
User can edit the field and it will look for a matching address based on their query

Fixes #59

Looks good in terms of UI

authors: Mysdreivus ZonalWings

repo: ZonalWings/hotfoot


text:
Fix sign/transfer test on zemu

It is not working right now. It is not running the signing algorithm. It is only taking screenshoots and comparing them.

@emmanuelm41  is this still relevant? This issue should already be fixed. Please check latest release.

Cryptographic functions take a really LONG time to be executed in Zemu. We will update Zemu test to run this in CI but it is working if you wait long enough

authors: ainhoa-a emmanuelm41 ftheirs

repo: Zondax/ledger-arweave


text:
Bug with Schedule (Broadcast interval)

Hello,
I noticed that the cooldown option does not apply well, basically the cooldown does not respect the minutes I feel, I do not know if this is a bug or not but a priori, during my tests on my server, we were 3 and all receive the message but never in the agreed time.
My config:
 Enabled - with this false this schedule will be ignored
 From and Until is in 24 hours format with : signs to separate minutes and seconds
 Exp and Money determines how big is boost, to reset it. keep it at 1.0. so if you use 2.0 that means all money/exp is doubled
 Days variables: Monday, Tuesday, Wednesday, Thursday, Friday, Saturday, Sunday, All
 Jobs can be any of your settup job or use All to give for all jobs at once
 BroadcastOnStart or BroadcastOnStop - set it false to disable message when boost starts/stops
 MessageOnStart or MessageOnStop - optional messages, if not given, then message from locale file will be shown
 BroadcastInterval - how often in minutes to broadcast message about money/exp boost for jobs
 BroadcastMessage - message to show every x minutes

Boost:
ShortExpBoost:
Enabled: true
From: '12:00:00'
Until: '00:00:00'
Exp: 2.0
Money: 1.0
Days:
- Sunday
Jobs:
- Mineur
- Chasseur
- Pêcheur
- Bûcheron
- Aubergiste
- Fermier
- Assassin
- Sorcier
BroadcastOnStart: true
MessageOnStart:
- '&a***********************************************'
- '    &2Boost Xp x2 pour tous les métiers !'
- '    &2De 12h00 à 00h00 !'
- '&a***********************************************'
BroadcastOnStop: true
MessageOnStop:
- '&a***********************************************'
- '    &2Boost métiers terminé !'
- '    &2De retour Dimanche de 12h00 à 00h00 !'
- '&a***********************************************'
BroadcastInterval: 30
BroadcastMessage:
- '&a***********************************************'
- '    &2Boost Xp x2 pour tous les métiers !'
- '    &2De 12h00 à 00h00 !'
- '&a***********************************************'

Try update to the latest version.

repo: Zrips/Jobs


text:
Tagging hairdresser=barber

Could support for the tag hairdresser=barber please be added, it was recently introduced for tagging barber shops.
Many thanks for creating this app, I use it on a daily basis!

https://github.com/Zverik/every_door#presets-and-translations
There is a preset for this tag, but you need to wait for the next Every Door update
https://github.com/openstreetmap/id-tagging-schema/pull/1165

Yup, will be available in 5.2.

authors: Zverik codeinabox deevroman

license: ISC


text:
High memory consumption in writePileUps

Hi Arne,
[...] The job seems to be stuck in “dentist collect”.  The file “workdir/pile-ups.db” was created but it is empty.  The node that it is running on shows that 71G of memory is used and 52G available.
Here are the last entries in the “collect.log” file.
{"thread":140737354013504,"timestamp":637255135504311334,"numPileUps":260,"numAlignmentChains":3186}
{"thread":140737354013504,"timestamp":637255135504323016,"state":"exit","function":"dentist.commands.collectPileUps.PileUpCollector.buildPileUps","timeElapsed":25742682}
{"thread":140737354013504,"timestamp":637255135504332559,"state":"enter","function":"dentist.commands.collectPileUps.PileUpCollector.writePileUps"}
Do you have any ideas?  Could there be a problem with the pipeline before it hit dentist collect?  Watching it run is a thing of beauty!
Regards,
Randy
Originally posted by @BradleyRan in https://github.com/a-ludi/dentist/issues/3#issuecomment-632727668

Hi Randy,
I don't have a clue so far. It looks like some kind of memory leak in the writer for the binary pileups format. Here are some questions to get some light into the dark:

Is the memory consumption rising over time?
How big is the input .las files? numAlignmentChains is just 3186 so I guess it's rather small.
Is it possible to share the input files? If yes, please drop them into my ownCloud so I can try reproducing the bug.
Can you please share the full command that is being executed (you may remove sensitive names from the paths)? You can make snakemake report the command by running it with options -np.

Hi Arne,
CC: Tim Smith
I’ll try to send you the entire directory.  I’m running tar on it now.  The .las file is 7.2 G.  We are trying this on an SGE cluster.  I think, from memory and random peeks while it was running, that the memory use on the node running the collect phase increased to about 50G.  After some time, I logged into the node.  “ps” showed that the process was still running but “top” didn’t show that it was using any CPU.  I should have saved the output of “ps” and “lsof” but I didn’t think of that before I deleted the job.  I’d be happy to rerun it and do anything to diagnose the problem.
Regards,
Randy
[randy.bradley@arsnecla0fshome:~ 09:47:26]tail -15 dentist-test/joblog/dentistCore.out
Finished job 1308.
1237 of 1243 steps (100%) done
[Fri May 22 23:57:40 2020]
checkpoint collect:
input: workdir/dentist.json, workdir/scaffolds_FINAL.dam, workdir/.scaffolds_FINAL.bps, workdir/.scaffolds_FINAL.hdr, workdir/.scaffolds_FINAL.idx, workdir/haplotype-Simmental_dam.reformat.dam, workdir/.haplotype-Simmental_dam.reformat.bps, workdir/.haplotype-Simmental_dam.reformat.hdr, workdir/.haplotype-Simmental_dam.reformat.idx, workdir/scaffolds_FINAL.haplotype-Simmental_dam.reformat.las, workdir/.scaffolds_FINAL.dentist-self.anno, workdir/.scaffolds_FINAL.dentist-self.data, workdir/.scaffolds_FINAL.tan.anno, workdir/.scaffolds_FINAL.tan.data, workdir/.scaffolds_FINAL.dentist-reads.anno, workdir/.scaffolds_FINAL.dentist-reads.data
output: workdir/pile-ups.db
log: logs/collect.log
jobid: 1303
reason: Missing output files: workdir/pile-ups.db; Input files updated by another job: workdir/.scaffolds_FINAL.dentist-self.data, workdir/scaffolds_FINAL.haplotype-Simmental_dam.reformat.las, workdir/.scaffolds_FINAL.dentist-reads.data, workdir/.scaffolds_FINAL.dentist-self.anno, workdir/.scaffolds_FINAL.dentist-reads.anno
threads: 8
Downstream jobs will be updated after completion.
dentist collect --config=workdir/dentist.json  --threads=4 --auxiliary-threads=2 - - - - 2> logs/collect.log
Submitted job 1303 with external jobid 'Your job 377542 ("dentist.collect") has been submitted'.
[randy.bradley@arsnecla0fshome:~ 09:41:49]cat dentist-test/workdir/dentist.json
{"default": {"read-coverage": 73.0, "max-coverage-self": 3, "verbose": 2, "reference": "workdir/scaffolds_FINAL.dam", "reads": "workdir/haplotype-Simmental_dam.reformat.dam", "result": "gap-closed.fasta", "ref-vs-reads-alignment": "workdir/scaffolds_FINAL.haplotype-Simmental_dam.reformat.las", "mask": ["dentist-self", "tan", "dentist-reads"], "pile-ups": "workdir/pile-ups.db", "insertions": "workdir/insertions.db"}, "output": {"fasta-line-width": 80}, "mask-repetitive-regions": {"reads": null}}[randy.bradley@arsnecla0fshome:~ 09:42:33]ll -h dentist-test/workdir/
total 13G
-rw-rw-r-- 1 randy.bradley randy.bradley  503 May 22 15:02 dentist.json
-rw-rw-r-- 1 randy.bradley randy.bradley  25K May 22 17:00 haplotype-Simmental_dam.reformat.dam
-rw-rw-r-- 1 randy.bradley randy.bradley    0 May 23 01:31 pile-ups.db
-rw-rw-r-- 1 randy.bradley randy.bradley  421 May 22 15:03 scaffolds_FINAL.dam
-r--r--r-- 1 randy.bradley randy.bradley 7.2G May 22 23:41 scaffolds_FINAL.haplotype-Simmental_dam.reformat.las
-r--r--r-- 1 randy.bradley randy.bradley 9.6G May 22 19:40 scaffolds_FINAL.scaffolds_FINAL.las
-rw-rw-r-- 1 randy.bradley randy.bradley   12 May 22 15:03 TAN.scaffolds_FINAL.las
From: Arne notifications@github.com
Reply-To: a-ludi/dentist reply@reply.github.com
Date: Monday, May 25, 2020 at 2:54 AM
To: a-ludi/dentist dentist@noreply.github.com
Cc: "Bradley, Randy" randy.bradley@usda.gov, Mention mention@noreply.github.com
Subject: Re: [a-ludi/dentist] Hi Arne, (#6)
Hi Randy,
I don't have a clue so far. It looks like some kind of memory leak in the writer for the binary pileups format. Here are some questions to get some light into the dark:

Is the memory consumption rising over time?
How big is the input .las files? numAlignmentChains is just 3186 so I guess it's rather small.
Is it possible to share the input files? If yes, please drop them into my ownCloudhttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcloud.mpi-cbg.de%2Findex.php%2Fs%2FAa4Jf89vjEDVMKV&data=02|01||12642b0408a54f46229208d80080d960|ed5b36e701ee4ebc867ee03cfa0d4697|0|0|637259900695914951&sdata=728IClQtrkZlWBKvpB%2B4eW0O5sbcvnK7Mc7yFXAzrIA%3D&reserved=0 so I can try reproducing the bug.
Can you please share the full command that is being executed (you may remove sensitive names from the paths)? You can make snakemake report the command by running it with options -np.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fa-ludi%2Fdentist%2Fissues%2F6%23issuecomment-633434436&data=02|01||12642b0408a54f46229208d80080d960|ed5b36e701ee4ebc867ee03cfa0d4697|0|0|637259900695914951&sdata=sCxHI44OaHMVB8Dgj8W%2BQfxfor8d4xrKjM0GIv8uuu4%3D&reserved=0, or unsubscribehttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAKFTC4QJ4VGX5Z6M4JRCTBTRTIP3FANCNFSM4NJJQYWA&data=02|01||12642b0408a54f46229208d80080d960|ed5b36e701ee4ebc867ee03cfa0d4697|0|0|637259900695914951&sdata=3xBIe6Y2FH%2FM9wU2R%2FrVxcO8ZVsgLuX6x4EwcCwccc4%3D&reserved=0.
This electronic message contains information generated by the USDA solely for the intended recipients. Any unauthorized interception of this message or the use or disclosure of the information it contains may violate the law and subject the violator to civil or criminal penalties. If you believe you have received this message in error, please notify the sender and delete the email immediately.

Hi Arne,
We’ve been having problems trying to upload the entire workdir to your owncloud due to the filesize.  Aaron had the idea of splitting the .tar file up into chunks that you could cat back together.  Is that OK?
Thanks.
If he is willing to handle a split file, I could potentially upload it to his ownCloud in chunks rather than Box. That might be better for him.
Randy, you want to see if he would take that?  I’d use split to break the file and provide md5sums for each part and the full file.  Have to cat > the files back together before decompressing.
From: "Rogge, Aaron" aaron.rogge@usda.gov
Date: Wednesday, May 27, 2020 at 11:39 AM
To: "Bradley, Randy" randy.bradley@usda.gov
Cc: "Anderson, Phil" phil.anderson@usda.gov, "Smith, Tim - ARS" tim.smith2@usda.gov
Subject: RE: [a-ludi/dentist] Hi Arne, (#6)
Second try crashed out at about the same point (6.5 MB difference).  I think the file is too large for his ownCloud.

<d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns">
<s:exception>Sabre\DAV\Exception\BadRequest</s:exception>
<s:message>expected filesize 67073086930 got 18478247936</s:message>
</d:error>
Summary:

Source file is 63G (gzipped)
File is too large for Box – 15G max file size
File crashes out at just of 17G transferring to ownCloud – max file size?
Globus isn’t licensed for shared end points
OneDrive is internal only also max file size issues
Any other ideas? I can split the file into chunks and upload to Box but they would have to be reassembled on the other end….

From: Rogge, Aaron
Sent: Wednesday, May 27, 2020 11:10 AM
To: Bradley, Randy randy.bradley@usda.gov
Cc: Anderson, Phil phil.anderson@usda.gov; Smith, Tim - ARS tim.smith2@usda.gov
Subject: RE: [a-ludi/dentist] Hi Arne, (#6)
After uploading all night the upload “timed out”. I am beginning to suspect a file size limit.  The file I’m trying to upload is 63G.
Below is the curl command. Sadly I am not getting a progress bar, so I can’t monitor progress.  My first transfer died at 17.2G. I am trying again to see if it drops out at about the same point.
curl -k -u "Aa4Jf89vjEDVMKV:" -H 'X-Requested-With: XMLHttpRequest' "https://cloud.mpi-cbg.de/public.php/webdav/https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcloud.mpi-cbg.de%2Fpublic.php%2Fwebdav%2F&data=02|01||f81ceada8bee4d696e1b08d8025c8a62|ed5b36e701ee4ebc867ee03cfa0d4697|0|0|637261943762448765&sdata=LibxbKpCS22eSrj8Tfe2iWUG3bn6Girw3KIrlIBk67k%3D&reserved=0" -T dentist-workdir.tar.gz
Here is the error I received the first time

<d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/nshttps://gcc02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsabredav.org%2Fns&data=02|01||f81ceada8bee4d696e1b08d8025c8a62|ed5b36e701ee4ebc867ee03cfa0d4697|0|0|637261943762458761&sdata=Bzmi6HQ%2B%2Bw%2F%2B%2Bp53Oy%2FDtbJwiNeSCTElDNAP%2FNFGghs%3D&reserved=0">
<s:exception>Sabre\DAV\Exception\BadRequest</s:exception>
<s:message>expected filesize 67073086930 got 18485112832</s:message>
</d:error>
From: Bradley, Randy <randy.bradley@usda.govmailto:randy.bradley@usda.gov>
Sent: Wednesday, May 27, 2020 8:45 AM
To: Rogge, Aaron <aaron.rogge@usda.govmailto:aaron.rogge@usda.gov>
Cc: Anderson, Phil <phil.anderson@usda.govmailto:phil.anderson@usda.gov>; Smith, Tim - ARS <tim.smith2@usda.govmailto:tim.smith2@usda.gov>
Subject: Re: [a-ludi/dentist] Hi Arne, (#6)
I’m beginning to see why the scientists are so frustrated trying to transfer data!
Maybe I should have asked if he had a CLI command to upload data to owncloud.  Then we might be able to use our data transfer node to send it over I2.  I did search a little but didn’t find anything simple.  So I investigated using Globus a little but kept hitting roadblocks but I did finally get 2FA working from home.  I knew they were going to drop the data sharing feature of Globus.  I must be missing something because they keep recommending Globus to transfer data on SciNet.  ☹
[A screenshot of a cell phone  Description automatically generated]
From: "Rogge, Aaron" <aaron.rogge@usda.govmailto:aaron.rogge@usda.gov>
Date: Wednesday, May 27, 2020 at 8:22 AM
To: "Bradley, Randy" <randy.bradley@usda.govmailto:randy.bradley@usda.gov>
Subject: RE: [a-ludi/dentist] Hi Arne, (#6)
Still slowly uploading.
From: Bradley, Randy <randy.bradley@usda.govmailto:randy.bradley@usda.gov>
Sent: Tuesday, May 26, 2020 5:48 PM
To: Rogge, Aaron <aaron.rogge@usda.govmailto:aaron.rogge@usda.gov>
Subject: Re: [a-ludi/dentist] Hi Arne, (#6)
OK.  Thanks!
From: "Rogge, Aaron" <aaron.rogge@usda.govmailto:aaron.rogge@usda.gov>
Date: Tuesday, May 26, 2020 at 5:40 PM
To: "Bradley, Randy" <randy.bradley@usda.govmailto:randy.bradley@usda.gov>
Subject: RE: [a-ludi/dentist] Hi Arne, (#6)
The file is still uploading. I’ll send you an update in the morning.
If it fails, I may be able to move it in chunks via Box (15G max file size limit) and try uploading from home.
Have a good evening,
aaron
This electronic message contains information generated by the USDA solely for the intended recipients. Any unauthorized interception of this message or the use or disclosure of the information it contains may violate the law and subject the violator to civil or criminal penalties. If you believe you have received this message in error, please notify the sender and delete the email immediately.

Sure, perfect!

Arne,
Would you check to see if all the files made it?  You should have the following files (size given in bytes and GiB)
    382b  382 dentist.md5

15032385536b  14G dentist-workdir.tar.gz.part-aa
15032385536b  14G dentist-workdir.tar.gz.part-ab
15032385536b  14G dentist-workdir.tar.gz.part-ac
15032385536b  14G dentist-workdir.tar.gz.part-ad
6943544786b 6.5G dentist-workdir.tar.gz.part-ae
The md5 file has the md5sums of the individual files and the original file before splitting:
fb81a55c846be3fdbe36c83b0a49c22d  dentist-workdir.tar.gz (unsplit file)
c8fa2aa5fb4cf346bda64b683e56e917  dentist-workdir.tar.gz.part-aa
5bb012d5b2b7ff7a0ec18370b8cc53ae  dentist-workdir.tar.gz.part-ab
f33179f01fc3478fe015882f7e775ebb  dentist-workdir.tar.gz.part-ac
1db5be4178c3da3f24c195dfe99aeb49  dentist-workdir.tar.gz.part-ad
1f16bfff96d491b747a545cfc8276a5b  dentist-workdir.tar.gz.part-ae
From: Bradley, Randy randy.bradley@usda.gov
Sent: Wednesday, May 27, 2020 11:49 AM
To: a-ludi/dentist reply@reply.github.com
Cc: Rogge, Aaron aaron.rogge@usda.gov
Subject: FW: [a-ludi/dentist] Hi Arne, (#6)
Hi Arne,
We’ve been having problems trying to upload the entire workdir to your owncloud due to the filesize.  Aaron had the idea of splitting the .tar file up into chunks that you could cat back together.  Is that OK?
Thanks.
If he is willing to handle a split file, I could potentially upload it to his ownCloud in chunks rather than Box. That might be better for him.
Randy, you want to see if he would take that?  I’d use split to break the file and provide md5sums for each part and the full file.  Have to cat > the files back together before decompressing.
From: "Rogge, Aaron" <aaron.rogge@usda.govmailto:aaron.rogge@usda.gov>
Date: Wednesday, May 27, 2020 at 11:39 AM
To: "Bradley, Randy" <randy.bradley@usda.govmailto:randy.bradley@usda.gov>
Cc: "Anderson, Phil" <phil.anderson@usda.govmailto:phil.anderson@usda.gov>, "Smith, Tim - ARS" <tim.smith2@usda.govmailto:tim.smith2@usda.gov>
Subject: RE: [a-ludi/dentist] Hi Arne, (#6)
Second try crashed out at about the same point (6.5 MB difference).  I think the file is too large for his ownCloud.

<d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns">
<s:exception>Sabre\DAV\Exception\BadRequest</s:exception>
<s:message>expected filesize 67073086930 got 18478247936</s:message>
</d:error>
Summary:

Source file is 63G (gzipped)
File is too large for Box – 15G max file size
File crashes out at just of 17G transferring to ownCloud – max file size?
Globus isn’t licensed for shared end points
OneDrive is internal only also max file size issues
Any other ideas? I can split the file into chunks and upload to Box but they would have to be reassembled on the other end….

From: Rogge, Aaron
Sent: Wednesday, May 27, 2020 11:10 AM
To: Bradley, Randy <randy.bradley@usda.govmailto:randy.bradley@usda.gov>
Cc: Anderson, Phil <phil.anderson@usda.govmailto:phil.anderson@usda.gov>; Smith, Tim - ARS <tim.smith2@usda.govmailto:tim.smith2@usda.gov>
Subject: RE: [a-ludi/dentist] Hi Arne, (#6)
After uploading all night the upload “timed out”. I am beginning to suspect a file size limit.  The file I’m trying to upload is 63G.
Below is the curl command. Sadly I am not getting a progress bar, so I can’t monitor progress.  My first transfer died at 17.2G. I am trying again to see if it drops out at about the same point.
curl -k -u "Aa4Jf89vjEDVMKV:" -H 'X-Requested-With: XMLHttpRequest' "https://cloud.mpi-cbg.de/public.php/webdav/https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcloud.mpi-cbg.de%2Fpublic.php%2Fwebdav%2F&data=02|01||1cac58dda5454f2a995c08d8025dd502|ed5b36e701ee4ebc867ee03cfa0d4697|0|0|637261949324808241&sdata=Kzk00tR5CyQ8E8XKK8n8e0HQ85gkMHfc0H%2BjohaXHCA%3D&reserved=0" -T dentist-workdir.tar.gz
Here is the error I received the first time

<d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/nshttps://gcc02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsabredav.org%2Fns&data=02|01||1cac58dda5454f2a995c08d8025dd502|ed5b36e701ee4ebc867ee03cfa0d4697|0|0|637261949324808241&sdata=wkl9x1YxgbFiY6jp2i8RNpnB3wKDX2KIDq8bm2uBp%2B8%3D&reserved=0">
<s:exception>Sabre\DAV\Exception\BadRequest</s:exception>
<s:message>expected filesize 67073086930 got 18485112832</s:message>
</d:error>
From: Bradley, Randy <randy.bradley@usda.govmailto:randy.bradley@usda.gov>
Sent: Wednesday, May 27, 2020 8:45 AM
To: Rogge, Aaron <aaron.rogge@usda.govmailto:aaron.rogge@usda.gov>
Cc: Anderson, Phil <phil.anderson@usda.govmailto:phil.anderson@usda.gov>; Smith, Tim - ARS <tim.smith2@usda.govmailto:tim.smith2@usda.gov>
Subject: Re: [a-ludi/dentist] Hi Arne, (#6)
I’m beginning to see why the scientists are so frustrated trying to transfer data!
Maybe I should have asked if he had a CLI command to upload data to owncloud.  Then we might be able to use our data transfer node to send it over I2.  I did search a little but didn’t find anything simple.  So I investigated using Globus a little but kept hitting roadblocks but I did finally get 2FA working from home.  I knew they were going to drop the data sharing feature of Globus.  I must be missing something because they keep recommending Globus to transfer data on SciNet.  ☹
[A screenshot of a cell phone  Description automatically generated]
From: "Rogge, Aaron" <aaron.rogge@usda.govmailto:aaron.rogge@usda.gov>
Date: Wednesday, May 27, 2020 at 8:22 AM
To: "Bradley, Randy" <randy.bradley@usda.govmailto:randy.bradley@usda.gov>
Subject: RE: [a-ludi/dentist] Hi Arne, (#6)
Still slowly uploading.
From: Bradley, Randy <randy.bradley@usda.govmailto:randy.bradley@usda.gov>
Sent: Tuesday, May 26, 2020 5:48 PM
To: Rogge, Aaron <aaron.rogge@usda.govmailto:aaron.rogge@usda.gov>
Subject: Re: [a-ludi/dentist] Hi Arne, (#6)
OK.  Thanks!
From: "Rogge, Aaron" <aaron.rogge@usda.govmailto:aaron.rogge@usda.gov>
Date: Tuesday, May 26, 2020 at 5:40 PM
To: "Bradley, Randy" <randy.bradley@usda.govmailto:randy.bradley@usda.gov>
Subject: RE: [a-ludi/dentist] Hi Arne, (#6)
The file is still uploading. I’ll send you an update in the morning.
If it fails, I may be able to move it in chunks via Box (15G max file size limit) and try uploading from home.
Have a good evening,
aaron
This electronic message contains information generated by the USDA solely for the intended recipients. Any unauthorized interception of this message or the use or disclosure of the information it contains may violate the law and subject the violator to civil or criminal penalties. If you believe you have received this message in error, please notify the sender and delete the email immediately.

Hey Randy,
got the files and they are OK. I will write to you once I have news.

So, in my setup it worked. It took 08:55 hours with 2 CPUs and consumed max. RSS of 86.5 GB. Most of the time (6.7h) was spent reading the large alignment file. The routine is not very optimized, I have to admit.
Now that you know how many resources are required, can you try again? I ran the job with 24h time limit and max allowed RSS of 128G.

Hi Arne,
I’m pretty sure I’ve got it narrowed down to NFS options.  Yesterday, I copied the workdir to a directory on another file system (mounted with different NFS OPTIONS) and ran “dentist collect” from there.  It didn’t finish but it was able to write to pile-ups.db.  Looks like it got hung up at 1:10 am this morning.  I’m running it again today from a SSD array that is mounted with only the default options.
-rw-rw-r-- 1 randy.bradley randy.bradley     5171172 Jun  7 01:10 pile-ups.db
I’ll let you know.
Thanks,
Randy
From: Arne notifications@github.com
Reply-To: a-ludi/dentist reply@reply.github.com
Date: Thursday, June 4, 2020 at 4:26 AM
To: a-ludi/dentist dentist@noreply.github.com
Cc: "Bradley, Randy" randy.bradley@usda.gov, Mention mention@noreply.github.com
Subject: Re: [a-ludi/dentist] High memory consumption in writePileUps (#6)
So, in my setup it worked. It took 08:55 hours with 2 CPUs and consumed max. RSS of 86.5 GB. Most of the time (6.7h) was spent reading the large alignment file. The routine is not very optimized, I have to admit.
Now that you know how many resources are required, can you try again? I ran the job with 24h time limit and max allowed RSS of 128G.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fa-ludi%2Fdentist%2Fissues%2F6%23issuecomment-638732300&data=02|01||cf9225330058478862b408d808694e03|ed5b36e701ee4ebc867ee03cfa0d4697|0|0|637268595666556977&sdata=shDkMCaoWqWtD9iVLL%2BIVedOAFS4XYSUyUmkPovjFtQ%3D&reserved=0, or unsubscribehttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAKFTC4RO3DKBDKFKCTEZGVTRU5SCXANCNFSM4NJJQYWA&data=02|01||cf9225330058478862b408d808694e03|ed5b36e701ee4ebc867ee03cfa0d4697|0|0|637268595666566933&sdata=F43sIJE7afXpOXNzhHABRouEGSC%2B38Fv%2FS1P2OsSLw4%3D&reserved=0.
This electronic message contains information generated by the USDA solely for the intended recipients. Any unauthorized interception of this message or the use or disclosure of the information it contains may violate the law and subject the violator to civil or criminal penalties. If you believe you have received this message in error, please notify the sender and delete the email immediately.

Hi Arne,
Some good news.  I was able to get the pipeline to run to completion by using a directory that resides directly on the master node.  Maybe speculative, but I think the problem was an interface MTU packet size difference.  Our home directory file server was left at the default MTU of 1500 while our cluster and compute nodes are set at 9000.  It could also be that plus a combination of NFS options such as rsize, wsize, sync, etc.  I have more testing to do but I’ll keep you posted.
Thanks,
Randy
Tail of job log:
[Mon Jun  8 15:17:11 2020]
rule output:
input: workdir/dentist.json, workdir/scaffolds_FINAL.dam, workdir/.scaffolds_FINAL.bps, workdir/.scaffolds_FINAL.hdr, workdir/.scaffolds_FINAL.idx, workdir/insertions.db

output: gap-closed.fasta

log: logs/output.log

jobid: 17

reason: Missing output files: gap-closed.fasta; Input files updated by another job: workdir/.scaffolds_FINAL.hdr, workdir/scaffolds_FINAL.dam, workdir/insertions.db, workdir/.scaffolds_FINAL.bps, workdir/.scaffolds_FINAL.idx

dentist output --config=workdir/dentist.json  - - - 2> logs/output.log
Submitted job 17 with external jobid 'Your job 383493 ("dentist.output") has been submitted'.
[Mon Jun  8 15:18:11 2020]
Finished job 17.
1306 of 1307 steps (100%) done
[Mon Jun  8 15:18:11 2020]
localrule ALL:
input: gap-closed.fasta

jobid: 0

reason: Input files updated by another job: gap-closed.fasta

[Mon Jun  8 15:18:11 2020]
Finished job 0.
1307 of 1307 steps (100%) done
Complete log: /ext/randy.bradley/dentist-test/.snakemake/log/2020-06-08T093217.330730.snakemake.log
From: Arne notifications@github.com
Reply-To: a-ludi/dentist reply@reply.github.com
Date: Thursday, June 4, 2020 at 4:26 AM
To: a-ludi/dentist dentist@noreply.github.com
Cc: "Bradley, Randy" randy.bradley@usda.gov, Mention mention@noreply.github.com
Subject: Re: [a-ludi/dentist] High memory consumption in writePileUps (#6)
So, in my setup it worked. It took 08:55 hours with 2 CPUs and consumed max. RSS of 86.5 GB. Most of the time (6.7h) was spent reading the large alignment file. The routine is not very optimized, I have to admit.
Now that you know how many resources are required, can you try again? I ran the job with 24h time limit and max allowed RSS of 128G.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fa-ludi%2Fdentist%2Fissues%2F6%23issuecomment-638732300&data=02|01||cf9225330058478862b408d808694e03|ed5b36e701ee4ebc867ee03cfa0d4697|0|0|637268595666556977&sdata=shDkMCaoWqWtD9iVLL%2BIVedOAFS4XYSUyUmkPovjFtQ%3D&reserved=0, or unsubscribehttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAKFTC4RO3DKBDKFKCTEZGVTRU5SCXANCNFSM4NJJQYWA&data=02|01||cf9225330058478862b408d808694e03|ed5b36e701ee4ebc867ee03cfa0d4697|0|0|637268595666566933&sdata=F43sIJE7afXpOXNzhHABRouEGSC%2B38Fv%2FS1P2OsSLw4%3D&reserved=0.
This electronic message contains information generated by the USDA solely for the intended recipients. Any unauthorized interception of this message or the use or disclosure of the information it contains may violate the law and subject the violator to civil or criminal penalties. If you believe you have received this message in error, please notify the sender and delete the email immediately.

Hi Arne,
The complete snakemake pipeline ran in less than 6 hours on one NFS filesystem.  On two other NFS filesystems, the pipeline gets stuck in “dentist collect”.  I’ve tried to make the failed NFS mount options identical to the successful filesystem but the symptoms remain the same.  “dentist collect” runs for about 2 hours.  During that time, I periodically log into the compute node that it is running on and I see that the top processes are “dentist collect” and “LAdump”.  Then the “pile-ups.db” file gets created but remains empty.  Comparing the logs, it may be that it gets stuck trying to write-protect the pile-ups.db file?
/ext: works
dentist collect --config=workdir/dentist.json  --threads=4 --auxiliary-threads=2 - - - - 2> logs/collect.log
Submitted job 120 with external jobid 'Your job 383483 ("dentist.collect") has been submitted'.
Write-protecting output file workdir/pile-ups.db.
Updating job 5 (extend_dentist_config_for_merge).
Updating job 121 (process).
Updating job 119 (merge).
[Mon Jun  8 15:14:51 2020]
/mnt/shared_tmp: doesn’t work
dentist collect --config=workdir/dentist.json  --threads=4 --auxiliary-threads=2 - - - - 2> logs/collect.log
Submitted job 1303 with external jobid 'Your job 387584 ("dentist.collect") has been submitted'.
Thanks,
Randy
From: Arne notifications@github.com
Reply-To: a-ludi/dentist reply@reply.github.com
Date: Thursday, June 4, 2020 at 4:26 AM
To: a-ludi/dentist dentist@noreply.github.com
Cc: "Bradley, Randy" randy.bradley@usda.gov, Mention mention@noreply.github.com
Subject: Re: [a-ludi/dentist] High memory consumption in writePileUps (#6)
So, in my setup it worked. It took 08:55 hours with 2 CPUs and consumed max. RSS of 86.5 GB. Most of the time (6.7h) was spent reading the large alignment file. The routine is not very optimized, I have to admit.
Now that you know how many resources are required, can you try again? I ran the job with 24h time limit and max allowed RSS of 128G.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fa-ludi%2Fdentist%2Fissues%2F6%23issuecomment-638732300&data=02|01||cf9225330058478862b408d808694e03|ed5b36e701ee4ebc867ee03cfa0d4697|0|0|637268595666556977&sdata=shDkMCaoWqWtD9iVLL%2BIVedOAFS4XYSUyUmkPovjFtQ%3D&reserved=0, or unsubscribehttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAKFTC4RO3DKBDKFKCTEZGVTRU5SCXANCNFSM4NJJQYWA&data=02|01||cf9225330058478862b408d808694e03|ed5b36e701ee4ebc867ee03cfa0d4697|0|0|637268595666566933&sdata=F43sIJE7afXpOXNzhHABRouEGSC%2B38Fv%2FS1P2OsSLw4%3D&reserved=0.
This electronic message contains information generated by the USDA solely for the intended recipients. Any unauthorized interception of this message or the use or disclosure of the information it contains may violate the law and subject the violator to civil or criminal penalties. If you believe you have received this message in error, please notify the sender and delete the email immediately.

Hi Randy,
the write-protection is likely not the cause of your issue because it is made by snakemake after dentist finished successfully. You may double-check this by verifying if dentist is still active.
I guess it might be file-locking: dentist tries to lock files it reads or writes via flockfile. If an error occurs (like it does on our cluster because file locking is not implemented) it will just open the file without locking and continue. In contrast, it will just get stuck if the file lock cannot be acquired for some reason.
I created another version of dentist that allows skipping the locking step entirely. I hope it just works because I did not test it at all.
dentist.v1.0.0-beta.1-6-gce64df8.x86_64.tar.gz

Thanks Arne!  Do I need to pass dentist an option to skip the locking?
From: Arne notifications@github.com
Reply-To: a-ludi/dentist reply@reply.github.com
Date: Friday, June 12, 2020 at 8:01 AM
To: a-ludi/dentist dentist@noreply.github.com
Cc: "Bradley, Randy" randy.bradley@usda.gov, Mention mention@noreply.github.com
Subject: Re: [a-ludi/dentist] High memory consumption in writePileUps (#6)
Hi Randy,
the write-protection is likely not the cause of your issue because it is made by snakemake after dentist finished successfully. You may double-check this by verifying if dentist is still active.
I guess it might be file-locking: dentist tries to lock files it reads or writes via flockfilehttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmanpage.me%2F%3Fq%3Dflockfile&data=02|01||8877979ee27044c3790e08d80ed0af7c|ed5b36e701ee4ebc867ee03cfa0d4697|0|0|637275636786381996&sdata=j9wwoiC42hy1qrAvhJGac%2FWxWa%2BvB2Hh78cBIYYFrmU%3D&reserved=0. If an error occurs (like it does on our cluster because file locking is not implemented) it will just open the file without locking and continue. In contrast, it will just get stuck if the file lock cannot be acquired for some reason.
I created another version of dentist that allows skipping the locking step entirely. I hope it just works because I did not test it at all.
dentist.v1.0.0-beta.1-6-gce64df8.x86_64.tar.gzhttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fa-ludi%2Fdentist%2Ffiles%2F4770897%2Fdentist.v1.0.0-beta.1-6-gce64df8.x86_64.tar.gz&data=02|01||8877979ee27044c3790e08d80ed0af7c|ed5b36e701ee4ebc867ee03cfa0d4697|0|0|637275636786381996&sdata=%2BNe0TN%2BnCenel6XoGF9MKjBlbQv%2FCYIIW6lFnJWsYw4%3D&reserved=0
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fa-ludi%2Fdentist%2Fissues%2F6%23issuecomment-643257676&data=02|01||8877979ee27044c3790e08d80ed0af7c|ed5b36e701ee4ebc867ee03cfa0d4697|0|0|637275636786391984&sdata=wdjXx7PNd9W0c00UMvDkbG%2FnzstRS%2BtuKb8GhCnrQ9Q%3D&reserved=0, or unsubscribehttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAKFTC4TXDU2EQGXD65GFHX3RWIRJRANCNFSM4NJJQYWA&data=02|01||8877979ee27044c3790e08d80ed0af7c|ed5b36e701ee4ebc867ee03cfa0d4697|0|0|637275636786391984&sdata=%2FOsZsriSWDvLAR3mgrhnAK5Urndpa6AraXTaFAZQnBo%3D&reserved=0.
This electronic message contains information generated by the USDA solely for the intended recipients. Any unauthorized interception of this message or the use or disclosure of the information it contains may violate the law and subject the violator to civil or criminal penalties. If you believe you have received this message in error, please notify the sender and delete the email immediately.

Hi Arne,
I applied an update to nfs-utils on our home directory server.  I also made sure that rpcbind, nfs-lock, nfs-server, and nfs-idmap were started on it.  Now, “dentist collect” works there too.
The third NFS file server that I tried it on is a CentOS 8 OS with SSD drives.  It still doesn’t work there but the nfs-utils package for CentOS 8 doesn’t include the nfs-lock and nfs-idmap daemons.  We just use that system for temp storage.
Thanks,
Randy
From: Arne notifications@github.com
Reply-To: a-ludi/dentist reply@reply.github.com
Date: Friday, June 12, 2020 at 8:01 AM
To: a-ludi/dentist dentist@noreply.github.com
Cc: "Bradley, Randy" randy.bradley@usda.gov, Mention mention@noreply.github.com
Subject: Re: [a-ludi/dentist] High memory consumption in writePileUps (#6)
Hi Randy,
the write-protection is likely not the cause of your issue because it is made by snakemake after dentist finished successfully. You may double-check this by verifying if dentist is still active.
I guess it might be file-locking: dentist tries to lock files it reads or writes via flockfilehttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmanpage.me%2F%3Fq%3Dflockfile&data=02|01||8877979ee27044c3790e08d80ed0af7c|ed5b36e701ee4ebc867ee03cfa0d4697|0|0|637275636786381996&sdata=j9wwoiC42hy1qrAvhJGac%2FWxWa%2BvB2Hh78cBIYYFrmU%3D&reserved=0. If an error occurs (like it does on our cluster because file locking is not implemented) it will just open the file without locking and continue. In contrast, it will just get stuck if the file lock cannot be acquired for some reason.
I created another version of dentist that allows skipping the locking step entirely. I hope it just works because I did not test it at all.
dentist.v1.0.0-beta.1-6-gce64df8.x86_64.tar.gzhttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fa-ludi%2Fdentist%2Ffiles%2F4770897%2Fdentist.v1.0.0-beta.1-6-gce64df8.x86_64.tar.gz&data=02|01||8877979ee27044c3790e08d80ed0af7c|ed5b36e701ee4ebc867ee03cfa0d4697|0|0|637275636786381996&sdata=%2BNe0TN%2BnCenel6XoGF9MKjBlbQv%2FCYIIW6lFnJWsYw4%3D&reserved=0
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fa-ludi%2Fdentist%2Fissues%2F6%23issuecomment-643257676&data=02|01||8877979ee27044c3790e08d80ed0af7c|ed5b36e701ee4ebc867ee03cfa0d4697|0|0|637275636786391984&sdata=wdjXx7PNd9W0c00UMvDkbG%2FnzstRS%2BtuKb8GhCnrQ9Q%3D&reserved=0, or unsubscribehttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAKFTC4TXDU2EQGXD65GFHX3RWIRJRANCNFSM4NJJQYWA&data=02|01||8877979ee27044c3790e08d80ed0af7c|ed5b36e701ee4ebc867ee03cfa0d4697|0|0|637275636786391984&sdata=%2FOsZsriSWDvLAR3mgrhnAK5Urndpa6AraXTaFAZQnBo%3D&reserved=0.
This electronic message contains information generated by the USDA solely for the intended recipients. Any unauthorized interception of this message or the use or disclosure of the information it contains may violate the law and subject the violator to civil or criminal penalties. If you believe you have received this message in error, please notify the sender and delete the email immediately.

Sorry, I forgot to mention: SKIP_FILE_LOCKING=1 should do the trick.

Yes, that worked perfectly.
Thanks!
From: Arne notifications@github.com
Reply-To: a-ludi/dentist reply@reply.github.com
Date: Saturday, June 13, 2020 at 7:59 AM
To: a-ludi/dentist dentist@noreply.github.com
Cc: "Bradley, Randy" randy.bradley@usda.gov, Mention mention@noreply.github.com
Subject: Re: [a-ludi/dentist] High memory consumption in writePileUps (#6)
Sorry, I forgot to mention: SKIP_FILE_LOCKING=1 should do the trick.
This electronic message contains information generated by the USDA solely for the intended recipients. Any unauthorized interception of this message or the use or disclosure of the information it contains may violate the law and subject the violator to civil or criminal penalties. If you believe you have received this message in error, please notify the sender and delete the email immediately.

authors: BradleyRan a-ludi

repo: a-ludi/dentist


text:
getting nNameError: name 'module' is not defined while running any ansible playbook

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: NameError: name 'module' is not defined
fatal: [localhost]: FAILED! => {"changed": false, "module_stderr": "Traceback (most recent call last):\n  File "/tmp/ansible_F4TsRV/ansible_module_a10_monitor.py", line 133, in \n    module.fail_json(msg="Import Error:{0}".format(ex))\nNameError: name 'module' is not defined\n", "module_stdout": "", "msg": "MODULE FAILURE", "rc": 1}

Usually comes from the requests library not being installed. PR#71 addresses this.


text:
⚠ WSINGRESOS - GENERICO has degraded performance

In 257934e, WSINGRESOS - GENERICO (https://srvshyweb.yucatan.gob.mx/cgi-bin/wspd_cgi.sh/WService=wsingresos/internet/servicios/wsGenericoPagosWeb.r) experienced degraded performance:

HTTP code: 200
Response time: 2164 ms

Resolved: WSINGRESOS - GENERICO performance has improved in 79b5832.

repo: aafyyuc/upptime


text:
Licensing and request

Hi Aaron, I am trying to contact with regarding the "gerber_rs274x_parser", my company may be interested in leveraging that technology, we'd like to contact you to discuss more if that's okay with you? Thanks, Tian Ling

Added the BSD 2-clause license to the project, which is what it was always intended to be under, just hadn't gotten around to it yet.


text:
Optionally bz2 encode clone files

https://github.com/mholt/archiver

Available as of v0.0.5 release.

repo: aaronland/go-smithsonian-openaccess


text:
Remix app is unable to connect to DB when deployed to Azure.

Bug Description
When I deployed this repo to Azure using azd up as recomended in the /prisma/seed.ts file, I get the error below. For some reason the remix app is unable to connect to the postgresql instance when it is running in Azure.
Steps to reproduce:

Clone this repo and run the dev container inside of it.
When the dev container is initialized, log into azd with the azd login --use-device-code command.
After a successful login, run the command azd up and answer the questions in the following dialog.
After a successful deployment, navigate to the produced Endpoint and try to log in with the credentials provided in the README.
See this error in the browser and the error below in the console:

Console Error:

2023-01-21T05:14:45.78782  Successfully Connected to container: 'main' [Revision: 'ca-web-zpj7g4hsn3q5e--33qs0uf', Replica: 'ca-web-zpj7g4hsn3q5e--33qs0uf-8486759696-2g887']
2023-01-21T05:13:27.144270489Z
2023-01-21T05:13:27.144274861Z Can't reach database server at `psql-db-zpj7g4hsn3q5e.postgres.database.azure.com`:`5432`
2023-01-21T05:13:27.144278759Z
2023-01-21T05:13:27.144283149Z Please make sure your database server is running at `psql-db-zpj7g4hsn3q5e.postgres.database.azure.com`:`5432`.
2023-01-21T05:13:27.144287778Z     at RequestHandler.handleRequestError (/myapp/node_modules/@prisma/client/runtime/index.js:34316:13)
2023-01-21T05:13:27.144291493Z     at RequestHandler.request (/myapp/node_modules/@prisma/client/runtime/index.js:34293:12)
2023-01-21T05:13:27.144295489Z     at PrismaClient._request (/myapp/node_modules/@prisma/client/runtime/index.js:35273:16)
2023-01-21T05:13:27.144299564Z     at verifyLogin (/myapp/build/index.js:111:26)
2023-01-21T05:13:27.144303596Z     at action2 (/myapp/build/index.js:479:14)
2023-01-21T05:13:27.144306942Z     at Object.callRouteAction (/myapp/node_modules/@remix-run/server-runtime/dist/data.js:35:14)
2023-01-21T05:13:27.144310339Z     at handleDataRequest (/myapp/node_modules/@remix-run/server-runtime/dist/server.js:116:18)
2023-01-21T05:13:27.144313941Z     at requestHandler (/myapp/node_modules/@remix-run/server-runtime/dist/server.js:48:18)
2023-01-21T05:13:27.144317795Z     at /myapp/node_modules/@remix-run/express/dist/server.js:39:22 {
2023-01-21T05:13:27.144321674Z   clientVersion: '4.6.1',
2023-01-21T05:13:27.144325256Z   errorCode: undefined
2023-01-21T05:13:27.144329396Z }
2023-01-21T05:13:27.146333004Z POST /login?_data=routes%2Flogin 500 - - 5045.256 ms
2023-01-21T05:13:27.229181638Z GET /login?_data=root 200 - - 1.297 ms
2023-01-21T05:14:44.818299223Z GET /login?_data=routes%2Flogin 200 - - 2.002 ms
2023-01-21T05:14:44.819928833Z GET /login?_data=root 200 - - 0.999 ms

2023-01-21T05:15:17.502527331Z PrismaClientInitializationError:
2023-01-21T05:15:17.502560236Z Invalid `prisma.user.findUnique()` invocation:
2023-01-21T05:15:17.502565036Z
2023-01-21T05:15:17.502569184Z
2023-01-21T05:15:17.502573770Z Can't reach database server at `psql-db-zpj7g4hsn3q5e.postgres.database.azure.com`:`5432`
2023-01-21T05:15:17.502577328Z
2023-01-21T05:15:17.502583345Z Please make sure your database server is running at `psql-db-zpj7g4hsn3q5e.postgres.database.azure.com`:`5432`.
2023-01-21T05:15:17.502586974Z     at RequestHandler.handleRequestError (/myapp/node_modules/@prisma/client/runtime/index.js:34316:13)
2023-01-21T05:15:17.502590850Z     at RequestHandler.request (/myapp/node_modules/@prisma/client/runtime/index.js:34293:12)
2023-01-21T05:15:17.502594398Z     at PrismaClient._request (/myapp/node_modules/@prisma/client/runtime/index.js:35273:16)
2023-01-21T05:15:17.502598911Z     at verifyLogin (/myapp/build/index.js:111:26)
2023-01-21T05:15:17.502603695Z     at action2 (/myapp/build/index.js:479:14)
2023-01-21T05:15:17.502607903Z     at Object.callRouteAction (/myapp/node_modules/@remix-run/server-runtime/dist/data.js:35:14)
2023-01-21T05:15:17.502611587Z     at handleDataRequest (/myapp/node_modules/@remix-run/server-runtime/dist/server.js:116:18)
2023-01-21T05:15:17.502631731Z     at requestHandler (/myapp/node_modules/@remix-run/server-runtime/dist/server.js:48:18)
2023-01-21T05:15:17.502635831Z     at /myapp/node_modules/@remix-run/express/dist/server.js:39:22 {
2023-01-21T05:15:17.502641064Z   clientVersion: '4.6.1',
2023-01-21T05:15:17.502645171Z   errorCode: undefined
2023-01-21T05:15:17.502649064Z }
2023-01-21T05:15:17.504787869Z POST /login?_data=routes%2Flogin 500 - - 5005.932 ms

2023-01-21T05:15:17.594855249Z GET /login?_data=root 200 - - 1.197 ms

I have figured out the cause of this issue. The database is configured incorrectly. I believe that the Virtual Networking stuff in infra/core/database/postresl.bicep is getting in the way of connecting the other other resources in the stack.
I have been able to work around this issue by adding the 'Allow all Azure resources' option in the Network option in the POSTGRESQL dashboard.
You can also edit the infra/core/database/postgres.bicep file to include this resource:
  resource SQLAllowAllWindowsAzureIps 'firewallRules' = {
    name: 'AllowAllWindowsAzureIps'
    properties: {
      startIpAddress: '0.0.0.0'
      endIpAddress: '0.0.0.0'
    }
  }

My bad - I even knew I needed to add that but just forgot.
Put it in the main branch now.

No worries here! This is what Open Source is all about! We help each other build the best product possible!

repo: aaronpowell/azure-remix-stack


text:
Transformer?

Just a suggestion, encoder-decoder are not transformer models. Perhaps add a transformer section too.

Transformer models in this repository refer to deep learning networks.
See architecture for more details on encoder-decoder: https://en.m.wikipedia.org/wiki/Transformer_(machine_learning_model)

authors: abacaj vmajor

repo: abacaj/awesome-transformers


text:
Refactor user persistence

Instead of selecting the settings from the DB for each "get" (on unbuffered table), user settings are now kept in instance memory. Simplifies code, too.
Added unit tests.

Why are you using DB_COMMIT instead of COMMIT WORK avoiding the SAP LUW ?

I might have had several sessions open during my tests. seems to work now.

@flaiker any additional comments?

Seems fine to me.
I guess the risk level of the unit test would be debatable as DB changes are made. https://help.sap.com/viewer/ba879a6e2ea04d9bb94c7ccd7cdac446/202009.000/en-US/4925667929ac16b7e10000000a42189d.html

yea, agree on the risk level

@mbtools if you change the risk level, I think this one can be merged

just a side idea: maybe refactor settings (repo, user, ...) to a string map ? setter/getter for each setting look a bit overwhelming imho. A simple param validation would work well instead.

agree. or ajson. we could get the settings with paths and have arrays.

json in xml ? ufff ! on the other hand ... would be quite flexible. One of our products supposes customizing ... but the customizing shape is not obvious at this stage of development (tables, maintenance views ...). So we store the customizing in json in the way similar to abapgit (just that it is json, not xml) and respectively wrote a customzing transaction, which is able to put it to transports and such things ... not that bad so far ... very very flexible

no, just json (no more xml). it would improve flexibility as you wrote, and make things easier to consume (no individual classes or get/set method).
at the same time, we should put the checksums into a transparent table to improve efficiency esp if we want to store by branch or commit.
anyway, there's always another refactor

thanks 👍

authors: flaiker larshp mbtools sbcgua

repo: abapGit/abapGit


text:
[Feat Request] Make available search engines calibratable

Currently the model call can choose from all engines, but in reality some engines may not be available (e.g., no google API). We should make the available engines a parameter in parameter.py. Then, in examples/browse.py, calibrate it to duckduckgo.

#15

authors: daiwaid zenodflow

repo: abcsys/libem


text:
How do I undo 'remember my choice'?

It seems to be literally "once and for all". related to #1

Hi @jan-glx, you can use the Tablr: Clear Csv Choice command for that purpose. I'll update the docs with a mention about since I forgot to add one.

Thanks!


text:
insertUseGeneratedKeys设置自增主键，主键名不为id，有没有更好的办法解决

文档说明：
/**
* ======如果主键不是id怎么用？==========
* 假设主键的属性名是uid,那么新建一个Mapper接口如下
*
public interface InsertUidMapper {
@Options(useGeneratedKeys = true, keyProperty = "id")
@InsertProvider(type = SpecialProvider.class, method = "dynamicSQL")
int insertUseGeneratedKeys(T record);
}
* 只要修改keyProperty = "uid"就可以
*
* 然后让你自己的Mapper继承InsertUidListMapper即可
我按照这种方式解决，但是有点麻烦，我需要专门创建一个接口去继承这个接口并且在新的接口不能调用原本的常用方法。然后需要在service中注入了两个mapper，很麻烦有没有简单的解决方案?

有 但是现在作者还没有去做 我已经提出issue建议这么做了 目前我用着是很成功的 再不做 我就自己写 然后发起个pull request了
请看我的issue #445

authors: CaoPengflying sunhao-java


text:
sqlserver 表名包装错误

连接sqlserver数据库，然后表名是 abc-def这样。
数据库终端里面直接查 select * from [abc-def] 是没问题的
但是生成的sql 是 select a,b,c,d from abc-def 这样。会出现错误

所以说 规范很重要， 尽量不要特殊化命名方式

authors: dwhgygzt tomasWade


text:
#TK73 Agregar funcionalidad de favorito

En estos momentos no se puede marcar como favorito un curso, agregar la funcionalidad para grabar en base de datos que el curso ha sido marcado como favorito por el estudiante.

Duplicada de #33

repo: abelardog/bootcampar


text:
created a branch, created a file and made a commit, and opened a pull request

created a branch, created a file and made a commit, and opened a pull request

created a branch, created a file and made a commit, and opened a pull request


text:
Add async API signatures

To support async / await callbacks.
Also add Async callback signatures.
Should just mirror existing names but append "Async", e.g.  MakeAssertionAsync

Or should we leave the storage of counter / public key creds responsibility to the call site? Might be better when I think about it.
Async might be needed for metdata services however. Leaving this open.

Wouldn't hurt to provide a sample storage facility for counter/public keys in the form of an in-memory database. Would be excellent for proof of concept.

Sure, but I'm still leaning to have it at the call site as separate methods. Async might be needed in the callback for ICredentialUniqueToUser though (since that will need to scan storage)

Done. DevStorage added in https://github.com/abergs/fido2-net-lib/commit/2d866a51fd3b67428ea37cc006211a0bda4a8aec#diff-43a3af1c06c320b60ef91be4cfbf21f0
Unfortunalte ReadOnlySpans are not allowed when methods are async which increases byte[] copying.

authors: abergs aseigler


text:
Converted to python3

Made a branch so the program works in python 3. Not sure how to request for a new branch to be created, but here is the converted code.

Hi @idenc
Thank you for converting the source to Python3. I have made a new branch with your source named PyQuiz-Python3. I have also updated the same in README.md and Contributing.md. Thank you for the contribution.

authors: abhijitnathwani idenc

repo: abhijitnathwani/PyQuiz


text:
[Question] Dealing with background noise/music

Any thoughts?
https://github.com/tyiannak/pyAudioAnalysis - says it can detect unkown sounds and has speaker diarization.
Can it use speaker diarization to extract only vocals and ignore other sources of sound?
Also if 2 people are speaking having speaker diarization would be great

I'll need to go through it once. This currently just splits based on speech and doesn't distinguish between speakers. Other sources like music are automatically ignored.

repo: abhirooptalasila/AutoSub


text:
🛑 Biro Pemerintahan is down

In c02b648, Biro Pemerintahan (https://biropemerintahan.kalbarprov.go.id) was down:

HTTP code: 500
Response time: 1646 ms

Resolved: Biro Pemerintahan is back up in 0ef4f44 after 2 hours, 57 minutes.

repo: abimanyu/Kalbarnetworks


text:
Switch back to upstream request package

Use upstream request and request-deferred package instead of ycmd-request and ycmd-request-deferred. Keep third-party folder until MELPA recipe is updated to not download files from third-party

Sorry for the slow response...at a conference this week. I'm glad we're finally able to do this!

authors: abingham ptrv

repo: abingham/emacs-ycmd


text:
fix(org-roam): autosync

fix bug

Thanks! Merged!

authors: abougouffa donneyluck

repo: abougouffa/minemacs


text:
Compile error with Clang 5 for Android

As part of gRPC C++ we currently get a build error when building gRPC v1.27.2 using Clang 5 for Android arm (using dockcross).
2020-02-26T16:09:34.1998811Z [ 85%] Building CXX object third_party/abseil-cpp/absl/synchronization/CMakeFiles/absl_synchronization.dir/mutex.cc.o
2020-02-26T16:09:34.5554197Z /work/build/android-arm/third_party/grpc/grpc/src/grpc/third_party/abseil-cpp/absl/synchronization/mutex.cc:431:36: warning: declaration shadows a local variable [-Wshadow]
2020-02-26T16:09:34.5554397Z       static bool pred(SynchEvent *ev) {
2020-02-26T16:09:34.5554512Z                                    ^
2020-02-26T16:09:34.5554903Z /work/build/android-arm/third_party/grpc/grpc/src/grpc/third_party/abseil-cpp/absl/synchronization/mutex.cc:405:43: note: previous declaration is here
2020-02-26T16:09:34.5555258Z static void PostSynchEvent(void *obj, int ev) {
2020-02-26T16:09:34.5555369Z                                           ^
2020-02-26T16:09:34.5607969Z /work/build/android-arm/third_party/grpc/grpc/src/grpc/third_party/abseil-cpp/absl/synchronization/mutex.cc:1108:1: error: 'xray_log_args' attribute is invalid for the implicit this argument
2020-02-26T16:09:34.5608300Z ABSL_XRAY_LOG_ARGS(1) void Mutex::Block(PerThreadSynch *s) {
2020-02-26T16:09:34.5608415Z ^                  ~
2020-02-26T16:09:34.5608824Z /work/build/android-arm/third_party/grpc/grpc/src/grpc/third_party/abseil-cpp/absl/base/attributes.h:516:38: note: expanded from macro 'ABSL_XRAY_LOG_ARGS'
2020-02-26T16:09:34.5609000Z     [[clang::xray_always_instrument, clang::xray_log_args(N)]]
2020-02-26T16:09:34.5609098Z                                      ^                    ~
2020-02-26T16:09:34.5658164Z /work/build/android-arm/third_party/grpc/grpc/src/grpc/third_party/abseil-cpp/absl/synchronization/mutex.cc:1455:1: error: 'xray_log_args' attribute is invalid for the implicit this argument
2020-02-26T16:09:34.5658464Z ABSL_XRAY_LOG_ARGS(1) void Mutex::Lock() {
2020-02-26T16:09:34.5658571Z ^                  ~
2020-02-26T16:09:34.5659140Z /work/build/android-arm/third_party/grpc/grpc/src/grpc/third_party/abseil-cpp/absl/base/attributes.h:516:38: note: expanded from macro 'ABSL_XRAY_LOG_ARGS'
2020-02-26T16:09:34.5659316Z     [[clang::xray_always_instrument, clang::xray_log_args(N)]]
2020-02-26T16:09:34.5659429Z                                      ^                    ~
2020-02-26T16:09:34.5659843Z /work/build/android-arm/third_party/grpc/grpc/src/grpc/third_party/abseil-cpp/absl/synchronization/mutex.cc:1473:1: error: 'xray_log_args' attribute is invalid for the implicit this argument
2020-02-26T16:09:34.5660010Z ABSL_XRAY_LOG_ARGS(1) void Mutex::ReaderLock() {
2020-02-26T16:09:34.5660114Z ^                  ~
2020-02-26T16:09:34.5660495Z /work/build/android-arm/third_party/grpc/grpc/src/grpc/third_party/abseil-cpp/absl/base/attributes.h:516:38: note: expanded from macro 'ABSL_XRAY_LOG_ARGS'
2020-02-26T16:09:34.5660662Z     [[clang::xray_always_instrument, clang::xray_log_args(N)]]
2020-02-26T16:09:34.5660779Z                                      ^                    ~
2020-02-26T16:09:34.5676281Z /work/build/android-arm/third_party/grpc/grpc/src/grpc/third_party/abseil-cpp/absl/synchronization/mutex.cc:1586:1: error: 'xray_log_args' attribute is invalid for the implicit this argument
2020-02-26T16:09:34.5676519Z ABSL_XRAY_LOG_ARGS(1) bool Mutex::TryLock() {
2020-02-26T16:09:34.5676627Z ^                  ~
2020-02-26T16:09:34.5677022Z /work/build/android-arm/third_party/grpc/grpc/src/grpc/third_party/abseil-cpp/absl/base/attributes.h:516:38: note: expanded from macro 'ABSL_XRAY_LOG_ARGS'
2020-02-26T16:09:34.5683037Z     [[clang::xray_always_instrument, clang::xray_log_args(N)]]
2020-02-26T16:09:34.5683162Z                                      ^                    ~
2020-02-26T16:09:34.5687888Z /work/build/android-arm/third_party/grpc/grpc/src/grpc/third_party/abseil-cpp/absl/synchronization/mutex.cc:1615:1: error: 'xray_log_args' attribute is invalid for the implicit this argument
2020-02-26T16:09:34.5688227Z ABSL_XRAY_LOG_ARGS(1) bool Mutex::ReaderTryLock() {
2020-02-26T16:09:34.5688309Z ^                  ~
2020-02-26T16:09:34.5688679Z /work/build/android-arm/third_party/grpc/grpc/src/grpc/third_party/abseil-cpp/absl/base/attributes.h:516:38: note: expanded from macro 'ABSL_XRAY_LOG_ARGS'
2020-02-26T16:09:34.5688897Z     [[clang::xray_always_instrument, clang::xray_log_args(N)]]
2020-02-26T16:09:34.5688998Z                                      ^                    ~
2020-02-26T16:09:34.5689374Z /work/build/android-arm/third_party/grpc/grpc/src/grpc/third_party/abseil-cpp/absl/synchronization/mutex.cc:1661:1: error: 'xray_log_args' attribute is invalid for the implicit this argument
2020-02-26T16:09:34.5689496Z ABSL_XRAY_LOG_ARGS(1) void Mutex::Unlock() {
2020-02-26T16:09:34.5689684Z ^                  ~
2020-02-26T16:09:34.5690171Z /work/build/android-arm/third_party/grpc/grpc/src/grpc/third_party/abseil-cpp/absl/base/attributes.h:516:38: note: expanded from macro 'ABSL_XRAY_LOG_ARGS'
2020-02-26T16:09:34.5690296Z     [[clang::xray_always_instrument, clang::xray_log_args(N)]]
2020-02-26T16:09:34.5690396Z                                      ^                    ~
2020-02-26T16:09:34.5741016Z /work/build/android-arm/third_party/grpc/grpc/src/grpc/third_party/abseil-cpp/absl/synchronization/mutex.cc:1713:1: error: 'xray_log_args' attribute is invalid for the implicit this argument
2020-02-26T16:09:34.5741208Z ABSL_XRAY_LOG_ARGS(1) void Mutex::ReaderUnlock() {
2020-02-26T16:09:34.5741325Z ^                  ~
2020-02-26T16:09:34.5741751Z /work/build/android-arm/third_party/grpc/grpc/src/grpc/third_party/abseil-cpp/absl/base/attributes.h:516:38: note: expanded from macro 'ABSL_XRAY_LOG_ARGS'
2020-02-26T16:09:34.5741904Z     [[clang::xray_always_instrument, clang::xray_log_args(N)]]
2020-02-26T16:09:34.5742026Z                                      ^                    ~
2020-02-26T16:09:34.5816877Z [ 85%] Building CXX object CMakeFiles/grpc_cronet.dir/src/core/ext/filters/client_channel/local_subchannel_pool.cc.o
2020-02-26T16:09:34.5966379Z 1 warning and 7 errors generated.
2020-02-26T16:09:34.5998119Z third_party/abseil-cpp/absl/synchronization/CMakeFiles/absl_synchronization.dir/build.make:140: recipe for target 'third_party/abseil-cpp/absl/synchronization/CMakeFiles/absl_synchronization.dir/mutex.cc.o' failed
2020-02-26T16:09:34.5999509Z make[5]: *** [third_party/abseil-cpp/absl/synchronization/CMakeFiles/absl_synchronization.dir/mutex.cc.o] Error 1
2020-02-26T16:09:34.5999960Z make[4]: *** [third_party/abseil-cpp/absl/synchronization/CMakeFiles/absl_synchronization.dir/all] Error 2

https://github.com/mavlink/MAVSDK/pull/982/checks?check_run_id=470191975#step:4:5120

@derekmauro would it be possible to backport this fix to the abseil LTS release?
This was suggested by @veblush in https://github.com/grpc/grpc/issues/22338#issuecomment-624235182. Thanks.

It looks like there is already a workaround available. Try building with -DABSL_NO_XRAY_ATTRIBUTES.

@derekmauro thanks! That worked (see https://github.com/mavlink/MAVSDK/pull/982/commits/32bea3f726856846cba41cacfca267cb31c36848).

authors: derekmauro julianoes

repo: abseil/abseil-cpp


text:
async rpc in put message to cluster

Signed-off-by: 元守 lulin@youzan.com
change rpc call in async way in PutMessageToCluster & PutMessagesToCluster
async in put messages to cluster

@absolute8511 for your review again.

@absolute8511 for your review

@absolute8511 for your review. async rpc response wait incorporated in SlaveWriteResult.GetResult()

@DoraALin  Could you please compare the detail benchmark before and after this changes. (both for benchmark test and the real server test.)

@absolute8511 benchmark shows async rpc call does NOT fatser than replica synchronization. I will try with server test.
Sync RPC call
10000            211161 ns/op           0.61 MB/s
--- BENCH: BenchmarkNsqdCoordPub1Replicator128-4
10000            279250 ns/op           0.46 MB/s
--- BENCH: BenchmarkNsqdCoordPub2Replicator128-4
10000            297141 ns/op           0.43 MB/s
--- BENCH: BenchmarkNsqdCoordPub3Replicator128-4
5000            251618 ns/op           4.07 MB/s
--- BENCH: BenchmarkNsqdCoordPub1Replicator1024-4
10000            331672 ns/op           3.09 MB/s
--- BENCH: BenchmarkNsqdCoordPub2Replicator1024-4
5000            307006 ns/op           3.34 MB/s
--- BENCH: BenchmarkNsqdCoordPub3Replicator1024-4
3000            380150 ns/op           0.34 MB/s
--- BENCH: BenchmarkNsqdCoordPub5Replicator128-4
5000            427857 ns/op           2.39 MB/s
--- BENCH: BenchmarkNsqdCoordPub5Replicator1024-4
5000            325689 ns/op           0.39 MB/s
--- BENCH: BenchmarkNsqdCoordPub4Replicator128-4
5000            344935 ns/op           2.97 MB/s
--- BENCH: BenchmarkNsqdCoordPub4Replicator1024-4
Async RPC call
PASS
10000            223244 ns/op           0.57 MB/s
--- BENCH: BenchmarkNsqdCoordPub1Replicator128-4
5000            344346 ns/op           0.37 MB/s
--- BENCH: BenchmarkNsqdCoordPub2Replicator128-4
5000            350341 ns/op           0.37 MB/s
--- BENCH: BenchmarkNsqdCoordPub3Replicator128-4
5000            205962 ns/op           4.97 MB/s
--- BENCH: BenchmarkNsqdCoordPub1Replicator1024-4
5000            322390 ns/op           3.18 MB/s
--- BENCH: BenchmarkNsqdCoordPub2Replicator1024-4
5000            457972 ns/op           2.24 MB/s
--- BENCH: BenchmarkNsqdCoordPub3Replicator1024-4
3000            441307 ns/op           0.29 MB/s
--- BENCH: BenchmarkNsqdCoordPub5Replicator128-4
3000            392285 ns/op           2.61 MB/s
--- BENCH: BenchmarkNsqdCoordPub5Replicator1024-4
5000            361425 ns/op           0.35 MB/s
--- BENCH: BenchmarkNsqdCoordPub4Replicator128-4
3000            377754 ns/op           2.71 MB/s
--- BENCH: BenchmarkNsqdCoordPub4Replicator1024-4


text:
🛑 blog.accubits.com is down

In d37b76a, blog.accubits.com (https://blog.accubits.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: blog.accubits.com is back up in af6b669 after 163 days, 3 hours, 46 minutes.

repo: accubits-nightswatch/upptime


text:
🛑 AceBlock Swagger UI is down

In f657039, AceBlock Swagger UI ($ACEBLOCK_SWAGGER_UI) was down:

HTTP code: 0
Response time: 0 ms

Resolved: AceBlock Swagger UI is back up in 2e66d81.

repo: aceblockID/monitoring


text:
🛑 AceBlock Swagger UI is down

In 9103054, AceBlock Swagger UI ($ACEBLOCK_SWAGGER_UI) was down:

HTTP code: 0
Response time: 0 ms

Resolved: AceBlock Swagger UI is back up in dd99444.


text:
🛑 AceBlock Swagger UI is down

In de54e73, AceBlock Swagger UI ($ACEBLOCK_SWAGGER_UI) was down:

HTTP code: 0
Response time: 0 ms

Resolved: AceBlock Swagger UI is back up in c2562f5.


text:
Can someone introduce what’s the purpose of this project? I do not understand it.

It seems that there is no specific function introduction, which makes me a little confused. Can someone introduce it?

i am using for automation. like i have so many question  only one parameter are different . this project is very usefull.

I use it on my telegram bot, instead of using Microsoft Edge I just use the bot to interact with BingChat (EdgeGPT)

It allows you to easily integrate bing ai into various projects. For example, you may use it in your Telegram bot, or you might want to write a better frontend/application for bing ai.

authors: EsqerYasen PRO-2684 alanjustalan scmanjarrez

license: Unlicense


text:
Unexplained iCloud communication, virus??

For some reason the solution is trying to connect to: 94598331.members.btmm.icloud.com?
Virus?
//

There is nothing in the source code that would result in the app attempting to connect to that URL.
Can you provide further information on the methods you used to detect the alleged communication?

Hi
It came up a error windows with a lot of references to parts of the code and there I can see this link. I think that I can provide a screen shot from it and upload so you can take a look.
I got worried when I saw this call.

Oh, it must have somehow discovered an iCloud service and crashed

authors: acidhax luigiplr pentiumii400mhz

repo: acidhax/chromecast-audio-stream


text:
bug:  Edge ve Chrome'da çeviri fonksiyonları kullanıldığındaki Application Error hatası

Hata Tanımı
Edge ve Chrome'da üzerindeki Çeviri fonksiyonları kullanıldığında uygulama Application Error veriyor. Github'dan kontrol edildiğinde bu konuyla ilgili issue #532 açıldığı görüldü ancak bu issue'nun sorunu açıklamadığı görüldüğünden yeni bir issue açılması kararlaştırıldı.
Edge ile birlikte Chrome'da da browserdaki çeviri fonksiyonunu açarak bu hatayı reproduce ettiğimizde rc de yenileme butonunun çalışmadığını prodda ise sitenin hiç çalışmadığını ve Application Error alındığını gördük.
-- bu bilgiyi ekleyiniz
notDepresseDeveloper#0919
## Hata Ortamı
Hatanın ortaya çıktığı ortamı açıklayın. Örneğin:
rc.afetharita.com (sadece güncelleme hatası #532 )
afetharita.com
Hatayı nasıl aldığınızı adım adım açıklayın

afetharita.com sitesine gidin
Browser üzerinden çeviri için ilgili özelliği aktifleştirin

Beklenen Davranış
Uygulamadaki Türkçe olan bölümlerin çeviri özelliği yardımıyla İngilizce veya diğer dillerde de görüntülenebilmesi
Ekran Görüntüleri
Chrome 109.0.5414.121 deki test için ekran görüntüsü ve console çıktısı görseldeki gibi olmaktadır.

Masaüstü Bilgileri

İşletim Sistemi: Windows
Tarayıcı: Chrome
Sürüm: 109.0.5414.121

Selamlar,
Discordda triaj ettik durumu. Kullanilan kutuphanelerle alakali ve cabukca cozulemeyen bir sorun oldugu icin ilk etapta translation ozelligini kapatacagiz. PR acilacak birazdan.
ilgili linkler:
https://github.com/facebook/react/issues/11538
https://bugs.chromium.org/p/chromium/issues/detail?id=872770

Selamlar, şimdi pr açıyorum ilgili konuyla ilgili. Yapılan güncelleme @efegurkan'ın yazdığı gibi translation özelliğini kapatma ile ilgili.

@usirin @canerakin111 Bu issue'yu tekrar açabilir miyiz? Translate kapatılmış fakat bu çözüm değil, Alanı tara butonu ile ilgili bir fix lazım bunun için, şu anda fixini yaptım pr'ını açabilirim

@usirin kontrol edebilir misin?

@canerakin111 #603

repo: acikkaynak/deprem-yardim-frontend


text:
[MOD] Show check mark next to threads from subscribed mags

Would it be possible to add some sort of indicator, maybe like a check mark or something, next to the name of the magazine/community in threads to show whether you're subscribed to it or not? Is this something that could be done in RES?
Reposted from kbin:
https://kbin.social/m/kbinMeta/t/273347/Subscribed-indicator-next-to-magazine

this is a nice idea 👍🏼

authors: aclist minnieoo

repo: aclist/kbin-kes


text:
Add documentation of list of ports used

Summary
Add documentation of list of ports used
Purpose
To make it easier to manage the ports used
Contents
Add documentation of list of ports used
Bring bts envoy from QuickMPC-BTS
Testing Methods Performed

@KotaTakahashi9320
Could you rebase this branch?

@mdonaka
I rebased it!
Please check.

repo: acompany-develop/QuickMPC


text:
Add a link to GET endpoints without arguments.

This means that its very easy to see the contents of an endpoint like this.

You were right about the second condition which was an error. Have replaced with if + else. Happy to squash the commits if you prefer, haven't done so yet to keep the discussion clear in case there are more comments.

Assigning myself to try this out when I find some time ...

LGTM. Here's a screenshot of the new functionality (note the links):

Thanks @jwg4!

repo: acoomans/flask-autodoc


text:
Add label and annotation propagation from project

Signed-off-by: Daishan Peng daishan@acorn.io
Context
We need the ability to propagate annotations and labels from project namespace to appNamespace. The primary goal to drive this is to allow linkerd service mesh to inject sidecar to acorn app by default. This is done by propagating annotations linkerd.io/inject: enabled from project to all acorn app namespaces.
Acorn linkerd plugin will be responsible for add linkerd.io/inject: enabled to all acorn projects, and acorn will just propagate the labels down to app namespaces.
Usage
To enable propagations, run acorn install with
acorn install --annotations-propagation "linkerd.io/inject"

Then install linkerd plugin
acorn run ghcr.io/acorn-io/acorn-linkerd-plugin:main

Changes
It addes a seperate controller for propagating annotations and labels, and also corresponding tests for it.

Darren has asked some requests so I made the following modification:

Move the logic to AddNamespace so that the change is owned by appInstance, not a separate handler that modifies namespaces annotations and labels.
Move AddNamespace into a separate handler to make code clean. Also move the test to accommodate.

End up doing a large test refactoring. Since we want to move namespace out of DeploySpec, a lot of existing tests won't work since we no longer create namespace in deploySpec.
I have removed namespace.yaml in the expected.yaml, at the same time, created new tests specifically for labeling logic(mostly around label and annotation scopes) to ensure we have the same test coverage as before.

Addressed all darren's comments and donnie's too.
The current flag:
--propagate-project-annotation strings   The list of keys of annotations to propagate from acorn project to app namespaces
--propagate-project-label strings        The list of keys of labels to propagate from acorn project to app namespaces

Going to merge it.

@StrongMonkey please open an issue for this and put it into the v0.6.0 milestone, in the QA column

Issue: https://github.com/acorn-io/acorn/issues/1172

authors: StrongMonkey cjellick

repo: acorn-io/acorn


text:
Limit body buffering to avoid resource exhaustion

This PR implement a limit for body buffering in order to avoid resource exhaustion using a very large body (see #7). The default limit is 10MB (which should be enough for non-uploads requests), but it's configurable.

Awesome, although you should specify the new option in the .traefik,yaml file
https://github.com/acouvreur/traefik-modsecurity-plugin/blob/19cdb477b8cee1966ad95278d168ae90a93df663/.traefik.yml#L8-L9
And probably add a documentation about it, the default value and the reason inside the readme.
Thank you very much for this!

Done! :+1:
(this is the first time I see the code for a traefik plugin)
I took the liberty of adding a new section in the README, named "Configuration", showing both the configuration parameters. I also added the configuration for maxBodySize inside the docker-compose files and the traefik YAML.
Let me know if you have any comment :-)

repo: acouvreur/traefik-modsecurity-plugin


text:
feat: Add support for names to scale down multiple containers

Original issue
Using Name is still allowed, but should be removed in the next major bump. Using Names as a one-element array does the same thing.
I looked into adding the services list to the default loading page. But it required a rework on the variables passed to the strategies. I'll work towards this in the future.
I factorized the test cases so that it's easier to add more tests / test cases.

Thanks for the pull request, I'm quit busy right now, but it seems promising!
I'll give it a better look soon.
And I think we'll merge this feature into beta channel first.

Alright awesome !
I'm testing it further this week.
I'll also change the base to beta .

:tada: This PR is included in version 1.0.0-beta.1 :tada:
The release is available on:

v1.0.0-beta.1
GitHub release

Your semantic-release bot :package::rocket:

authors: Nastaliss acouvreur

repo: acouvreur/traefik-ondemand-plugin


text:
Combine results

As there is not really any documentation, would anyone please kindly tell me how to combine two Parsers in a way that the result will be a list of them? So e.g. I have a Parser<char,char> and a different Parser<char,char> and want a Parser<char, IEnumerable<char>>.

Sorry for your inconvenience, as there is no well-constructed documentations.
If you want to combine two or more parsers that have a same type, you can use
Parser<TToken, IEnumerable<T>> Sequence<TToken, T>(params Parser<TToken, T>[] parsers)
method. like this
var combined = Sequence(parser1, parser2) // Parser<char, IEnumerable<char>>

And also, if you actually want to combine parsers that have different types, it is better choice to use LINQ query notation.
var combined =
    from x in parser1
    from y in parser2
    select new MyRecord(x, y); // any classes as you like

Ok thanks ^^

authors: acple trueFireblade

repo: acple/ParsecSharp


text:
Add new intervention type: Street furniture and urban realm

Including:

[ ] Benches
[ ] Cycle parking
[ ] Pavement decluttering/widening
[ ] Pocket parks

See values here in the upstream version: https://acteng.github.io/schema/v1 and https://github.com/acteng/schema/blob/main/schemas/furniture.json

Good point. It's covered by Other.

repo: acteng/atip


text:
how to use match action name of intent of Dialogflow to handler in new SDK v2?

I'm migrating from SDK v1 to v2. And found that new SDK doesn't catch Dialogflow action names.
app.intent('action-name', handler);

Doesn't work.
I can match manually action name to handle in fallback handler, but maybe it could be fixed in SDK?
And what is the plan for Dialogflow action name, is it deprecated and shouldn't be used anymore?

I could but it won't be very flexible as it was before. Sometimes I need to route intents to the same handle (for example when few intents are parts of single follow-up flow), in v1 I could change routing/matching in Dialogflow by changing action name, but in v2 I need to change code and redeploy app which has few app.intent(action-name-1, handler); ...; app.intent(action-name-n, handler);  because we can't have identical intent names.

You can have map several intents to each other so you can have one handler. Alternatively, make the handler a function and provide it for multiple intents.
app.intent('My First Intent', handler);
app.intent('My Second Intent', 'My First Intent');

or
const handler = (conv) => {
  // Do a thing
};

app.intent('My First Intent', handler);
app.intent('My Second Intent', handler);

Same here - this makes migrating to the new version much more complicated an error-prone.

I posted my proposal to add a new method app.action so that developers can register my handler function with an action name as #164.

Also needed for us.  We are using actions-on-google-kotlin sdk, and I am in process of updating to V2.  I am going to default to using action instead intent, due to our need to have multiple DF intents trigger the same handler.  If an alternate solution is added to the nodejs lib I will update to match.

authors: Fleker hyzhak patjackson52 urish yoichiro


text:
Transfer this repository to a dedicated GitHub organization

A few months back I had talked with @summerwind and he was willing to transfer this repository to me.
I don't really mind who owns the repository but recently I got some motivation to have administrative privilege myself to do:

Enable stale bot #487 cc/ @callum-tait-pbx
Update our workflow and publish our chart in another GitHub repository https://github.com/summerwind/actions-runner-controller/issues/402#issuecomment-821934353
Enable discussions #443 @callum-tait-pbx @igorbrigadir @erikkn

For the second point I'd like to have another sibling repository for publishing Helm charts, too. So, I think it's more plausible and understandable to have this repository and the new repository for charts under a clean GitHub organization, rather than having it under my crowed GitHub user account :)
I've already created https://github.com/actions-runner-controller/ so if there's no strong objection I'd like to talk with @summerwind once again for the move.

Sounds good, an org will help scaling this project quite a bit. Awesome, @mumoshu !

Super pleased to hear we are making progress! I really think this project could be huge if the release, issue management and testing could be improved. I spent a long time looking at solutions for adding a self-hosted capability to GitHub Actions and this project as well as philips-labs were far and away the best projects out there and both have their use cases, neither project was objectively better than the other.

For the second point I'd like to have another sibling repository for publishing Helm charts, too. So, I think it's more plausible and understandable to have this repository and the new repository for charts under a clean GitHub organization, rather than having it under my crowed GitHub user account :)

I definitely think this needs thought through if you do want to go down that route, whilst better than the current setup I still don't love it tbh but there can be a thread once discussions is enabled :D

I've already created https://github.com/actions-runner-controller/ so if there's no strong objection I'd like to talk with @summerwind once again for the move.

Great idea, just a thought, the org might be worth being named slightly more generically e.g. actions-runner, maybe down the road someone makes a runner chart or something and we end up with a second repository called actions-runner-runners 🤷♂  probably bikeshedding though :D

Fully agree to transfer this repository to org. I will transfer the repository as soon as I can.
Thanks to all users who have used it so far, and to the contributors who have made various improvements. Especially a big thanks to @mumoshu for his continuous development!

The repository transfer is complete. actions-runner-controller org is the new home!

@erikkn @callum-tait-pbx Thanks for our comments and thanks for taking your time @summerwind!
I'm already a happy user of Discussions and Stale app 😄

authors: callum-tait-pbx erikkn mumoshu summerwind


text:
Add optional volumes and volumeMounts to Helm chart

Is your feature request related to a problem? Please describe.
I am using the secret store CSI driver (https://github.com/kubernetes-sigs/secrets-store-csi-driver) to populate pods with secrets from a third party secret store. In order to sync it with Kubernetes secrets, I need to mount a volume to the target pod (see https://secrets-store-csi-driver.sigs.k8s.io/topics/sync-as-kubernetes-secret.html). The Helm chart currently does not provide me an easy way to quickly add an extravolume and volumeMount. Some other popular charts offer this flexibility already (look at external-dns, neo4j, nginx, etc).
In the future, a user might want to add additional volumes for other purposes.
Describe the solution you'd like
Add 2 new fields in values.yaml that will add additional volumes and volume mounts to the deployment yaml (https://github.com/actions-runner-controller/actions-runner-controller/blob/master/charts/actions-runner-controller/templates/deployment.yaml).
## specify additional volumes to mount in the core container, this can be used
## to specify additional storage of material or to inject files from ConfigMaps
## into the running container
additionalVolumes: []

## specify where the additional volumes are mounted in the core container
additionalVolumeMounts: []

In the deployment.yaml template, it suffices to add:
        volumeMounts:
        - mountPath: "/etc/actions-runner-controller"
          name: secret
          readOnly: true
        - mountPath: /tmp
          name: tmp
        - mountPath: /tmp/k8s-webhook-server/serving-certs
          name: cert
          readOnly: true
      {{- if .Values.additionalVolumeMounts }}
{{ toYaml .Values.additionalVolumeMounts | indent 8}}
      {{- end }}

and
      volumes:
      - name: secret
        secret:
          secretName: {{ include "actions-runner-controller.secretName" . }}
      - name: cert
        secret:
          defaultMode: 420
          secretName: {{ include "actions-runner-controller.servingCertName" . }}
      - name: tmp
        emptyDir: {}
        {{- if .Values.additionalVolumes }}
{{ toYaml .Values.additionalVolumes | indent 8}}
        {{- end }}

Describe alternatives you've considered
Patching an existing deployment with the additional volumes.
Or specifically for my use case: not using the secret store csi driver, and manually adding the kubernetes secret. Or use other ways to get secrets into a k8s namespace from a third party provider.

@brunocous Hey! Thanks a lot for writing this up. This sounds like a valid feature request to our chart. I think I've seen this additionalVolumes and additionalVolumeMounts pair many times in various Helm stable charts so the values API seems good too.
Would you mind submitting a PR for this, if you have any time/experience writing charts?

Fixed in #952

authors: brunocous mumoshu