text:
Difficulties to follow your development process under Windows

I have a few issues with following the suggested work flow here
on my Windows machine (Windows 10, Anaconda python 3.7.3).
Steps to reproduce

run pip install -e '.[all]' in the nevergrad folder
did not work at all:

C:\Users\dimo\Desktop\nevergrad>pip install -e `.[all]`
`.[all]` should either be a path to a local project or a VCS url beginning with svn+, git+, hg+, or bzr+

running pip install -e . however, installed nevergrad without problems
C:\Users\dimo\Desktop\nevergrad>pip install -e .
Obtaining file:///C:/Users/dimo/Desktop/nevergrad
Requirement already satisfied: numpy>=1.15.0 in c:\users\dimo\anaconda3\lib\site-packages (from nevergrad==0.3.0) (1.16.2)
Requirement already satisfied: pandas>=0.23.4 in c:\users\dimo\anaconda3\lib\site-packages (from nevergrad==0.3.0) (0.24.2)
Requirement already satisfied: cma>=2.6.0 in c:\users\dimo\anaconda3\lib\site-packages (from nevergrad==0.3.0) (2.7.0)
Requirement already satisfied: bayesian-optimization>=1.0.1 in c:\users\dimo\anaconda3\lib\site-packages (from nevergrad==0.3.0) (1.0.1)
Requirement already satisfied: typing_extensions>=3.6.6 in c:\users\dimo\anaconda3\lib\site-packages (from nevergrad==0.3.0) (3.7.4)
Requirement already satisfied: python-dateutil>=2.5.0 in c:\users\dimo\anaconda3\lib\site-packages (from pandas>=0.23.4->nevergrad==0.3.0) (2.8.0)
Requirement already satisfied: pytz>=2011k in c:\users\dimo\anaconda3\lib\site-packages (from pandas>=0.23.4->nevergrad==0.3.0) (2018.9)
Requirement already satisfied: scikit-learn>=0.18.0 in c:\users\dimo\anaconda3\lib\site-packages (from bayesian-optimization>=1.0.1->nevergrad==0.3.0) (0.20.3)
Requirement already satisfied: scipy>=0.14.0 in c:\users\dimo\anaconda3\lib\site-packages (from bayesian-optimization>=1.0.1->nevergrad==0.3.0) (1.2.1)
Requirement already satisfied: six>=1.5 in c:\users\dimo\anaconda3\lib\site-packages (from python-dateutil>=2.5.0->pandas>=0.23.4->nevergrad==0.3.0) (1.12.0)
Installing collected packages: nevergrad
  Found existing installation: nevergrad 0.3.0
    Uninstalling nevergrad-0.3.0:
      Successfully uninstalled nevergrad-0.3.0
  Running setup.py develop for nevergrad
Successfully installed nevergrad

So this looks rather a documentation issue than an actual code issue.

However, the next step in the development process is to run pytest nevergrad --cov=nevergrad which, for me, results in quite some problems, mainly related to the operating system (output below from the master branch, cut for length):

C:\Users\dimo\Desktop\nevergrad>pytest nevergrad --cov=nevergrad
Test session starts (platform: win32, Python 3.7.3, pytest 4.3.1, pytest-sugar 0.9.2)
rootdir: C:\Users\dimo\Desktop\nevergrad, inifile:
plugins: sugar-0.9.2, remotedata-0.3.1, openfiles-0.3.2, mock-2.0.0, doctestplus-0.3.0, cov-2.8.1, arraydiff-0.3
collecting ...
 nevergrad/benchmark/test_core.py ✓                                                                         0%            nevergrad/benchmark/test_core.py ✓✓                                                                        0%            nevergrad/benchmark/test_core.py ✓✓✓                                                                       0%            nevergrad/benchmark/test_core.py ✓✓✓✓                                                                      0% ▏          nevergrad/benchmark/test_core.py ✓✓✓✓✓                                                                     1% ▏

――――――――――――――――――――――――――――――――――――――――――――――― test_commandline_launch ――――――――――――――――――――――――――――――――――――――――――――――――
subprocess.CalledProcessError: Command '['python', '-m', 'nevergrad.benchmark', 'additional_experiment', '--cap_index', '2', '--num_workers', '2', '--output', 'C:\\Users\\dimo\\AppData\\Local\\Temp\\tmpjuk3d1f8\\benchmark_launch_test.csv', '--imports', 'C:\\Users\\dimo\\Desktop\\nevergrad\\nevergrad\\benchmark\\additional\\example.py']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

    def test_commandline_launch() -> None:
        with tempfile.TemporaryDirectory() as folder:
            output = Path(folder) / "benchmark_launch_test.csv"
            # commandline test
            CommandFunction(command=["python", "-m", "nevergrad.benchmark", "additional_experiment",
                                     "--cap_index", "2", "--num_workers", "2", "--output", str(output),
>                                    "--imports", str(Path(__file__).parent / "additional" / "example.py")])()

nevergrad\benchmark\test_core.py:46:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <nevergrad.instrumentation.utils.CommandFunction object at 0x0000024EFF2C6518>, args = (), kwargs = {}
full_command = ['python', '-m', 'nevergrad.benchmark', 'additional_experiment', '--cap_index', '2', ...]
outlines = ['Starting repetition 1 / 1'], process = <subprocess.Popen object at 0x0000024EFF2C6D30>
line = b'Starting repetition 1 / 1\r\n'
stderr = b'concurrent.futures.process._RemoteTraceback: \r\n"""\r\nTraceback (most recent call last):\r\n  File "C:\\Users\\dim...\futures\\_base.py", line 384, in __get_result\r\n    raise self._exception\r\nKeyError: \'additional_experiment\'\r\n'
stdout = 'Starting repetition 1 / 1', retcode = 1
subprocess_error = CalledProcessError(1, ['python', '-m', 'nevergrad.benchmark', 'additional_experiment', '--cap_index', '2', '--num_work...rk_launch_test.csv', '--imports', 'C:\\Users\\dimo\\Desktop\\nevergrad\\nevergrad\\benchmark\\additional\\example.py'])

    def __call__(self, *args: Any, **kwargs: Any) -> str:
        """Call the cammand line with addidional arguments
        The keyword arguments will be sent as --{key}={val}
        The logs are bufferized. They will be printed if the job fails, or sent as output of the function
        Errors are provided with the internal stderr
        """
        # TODO make the following command more robust (probably fails in multiple cases)
        full_command = self.command + [str(x) for x in args] + ["--{}={}".format(x, y) for x, y in kwargs.items()]
        if self.verbose:
            print(f"The following command is sent: {full_command}")
        outlines: List[str] = []
        with subprocess.Popen(full_command, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
                              shell=False, cwd=self.cwd, env=self.env) as process:
            try:
                for line in iter(process.stdout.readline, b''):
                    if not line:
                        break
                    outlines.append(line.decode().strip())
                    if self.verbose:
                        print(outlines[-1], flush=True)
            except Exception:  # pylint: disable=broad-except
                process.kill()
                process.wait()
                raise FailedJobError("Job got killed for an unknown reason.")
            stderr = process.communicate()[1]  # we already got stdout
            stdout = "\n".join(outlines)
            retcode = process.poll()
            if stderr and (retcode or self.verbose):
                print(stderr.decode(), file=sys.stderr)
            if retcode:
                subprocess_error = subprocess.CalledProcessError(retcode, process.args, output=stdout, stderr=stderr)
>               raise FailedJobError(stderr.decode()) from subprocess_error
E               nevergrad.instrumentation.utils.FailedJobError: concurrent.futures.process._RemoteTraceback:
E               """
E               Traceback (most recent call last):
E                 File "C:\Users\dimo\Anaconda3\lib\concurrent\futures\process.py", line 232, in _process_worker
E                   r = call_item.fn(*call_item.args, **call_item.kwargs)
E                 File "C:\Users\dimo\Desktop\nevergrad\nevergrad\benchmark\core.py", line 173, in compute
E                   for local_ind, (index, xp) in enumerate(self):
E                 File "C:\Users\dimo\Desktop\nevergrad\nevergrad\benchmark\core.py", line 127, in __iter__
E                   maker = registry[self.name]
E                 File "C:\Users\dimo\Desktop\nevergrad\nevergrad\common\decorators.py", line 61, in __getitem__
E                   return self.data[key]
E               KeyError: 'additional_experiment'
E               """
E
E               The above exception was the direct cause of the following exception:
E
E               Traceback (most recent call last):
E                 File "C:\Users\dimo\Anaconda3\lib\runpy.py", line 193, in _run_module_as_main
E                   "__main__", mod_spec)
E                 File "C:\Users\dimo\Anaconda3\lib\runpy.py", line 85, in _run_code
E                   exec(code, run_globals)
E                 File "C:\Users\dimo\Desktop\nevergrad\nevergrad\benchmark\__main__.py", line 93, in <module>
E                   repetitions=args.repetitions)
E                 File "C:\Users\dimo\Desktop\nevergrad\nevergrad\benchmark\__main__.py", line 79, in repeated_launch
E                   seed=None if seed is None else seed + k)
E                 File "C:\Users\dimo\Desktop\nevergrad\nevergrad\benchmark\__main__.py", line 28, in launch
E                   df = core.compute(experiment, seed=seed, cap_index=cap_index, executor=executor, num_workers=num_workers)
E                 File "C:\Users\dimo\Desktop\nevergrad\nevergrad\benchmark\core.py", line 279, in compute
E                   dfs = [j.result() for j in jobs]
E                 File "C:\Users\dimo\Desktop\nevergrad\nevergrad\benchmark\core.py", line 279, in <listcomp>
E                   dfs = [j.result() for j in jobs]
E                 File "C:\Users\dimo\Anaconda3\lib\concurrent\futures\_base.py", line 432, in result
E                   return self.__get_result()
E                 File "C:\Users\dimo\Anaconda3\lib\concurrent\futures\_base.py", line 384, in __get_result
E                   raise self._exception
E               KeyError: 'additional_experiment'

nevergrad\instrumentation\utils.py:114: FailedJobError
------------------------------------------------- Captured stderr call -------------------------------------------------
concurrent.futures.process._RemoteTraceback:
"""
Traceback (most recent call last):
  File "C:\Users\dimo\Anaconda3\lib\concurrent\futures\process.py", line 232, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
  File "C:\Users\dimo\Desktop\nevergrad\nevergrad\benchmark\core.py", line 173, in compute
    for local_ind, (index, xp) in enumerate(self):
  File "C:\Users\dimo\Desktop\nevergrad\nevergrad\benchmark\core.py", line 127, in __iter__
    maker = registry[self.name]
  File "C:\Users\dimo\Desktop\nevergrad\nevergrad\common\decorators.py", line 61, in __getitem__
    return self.data[key]
KeyError: 'additional_experiment'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\dimo\Anaconda3\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Users\dimo\Anaconda3\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\dimo\Desktop\nevergrad\nevergrad\benchmark\__main__.py", line 93, in <module>
    repetitions=args.repetitions)
  File "C:\Users\dimo\Desktop\nevergrad\nevergrad\benchmark\__main__.py", line 79, in repeated_launch
    seed=None if seed is None else seed + k)
  File "C:\Users\dimo\Desktop\nevergrad\nevergrad\benchmark\__main__.py", line 28, in launch
    df = core.compute(experiment, seed=seed, cap_index=cap_index, executor=executor, num_workers=num_workers)
  File "C:\Users\dimo\Desktop\nevergrad\nevergrad\benchmark\core.py", line 279, in compute
    dfs = [j.result() for j in jobs]
  File "C:\Users\dimo\Desktop\nevergrad\nevergrad\benchmark\core.py", line 279, in <listcomp>
    dfs = [j.result() for j in jobs]
  File "C:\Users\dimo\Anaconda3\lib\concurrent\futures\_base.py", line 432, in result
    return self.__get_result()
  File "C:\Users\dimo\Anaconda3\lib\concurrent\futures\_base.py", line 384, in __get_result
    raise self._exception
KeyError: 'additional_experiment'

 nevergrad/benchmark/test_core.py ⨯                                                                         1% ▏          nevergrad/benchmark/test_core.py ⨯✓                                                                        1% ▏
[...]
nevergrad/common/test_tools.py ✓✓✓✓✓✓✓✓✓✓✓✓✓✓                                                             11% █▏         nevergrad/common/test_tools.py ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓                                                            11% █▏         nevergrad/common/test_tools.py ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓                                                           11% █▏

――――――――――――――――――――――――――――――――――――――――――――――――――――― test_sleeper ―――――――――――――――――――――――――――――――――――――――――――――――――――――

    def test_sleeper() -> None:
        min_sleep = 1e-5
        sleeper = tools.Sleeper(min_sleep=min_sleep)
        np.testing.assert_almost_equal(sleeper._get_advised_sleep_duration(), min_sleep, decimal=5)
        sleeper.start_timer()
        np.testing.assert_almost_equal(sleeper._get_advised_sleep_duration(), min_sleep, decimal=5)
        sleeper.stop_timer()
>       np.testing.assert_almost_equal(sleeper._get_advised_sleep_duration(), min_sleep, decimal=5)
E       AssertionError:
E       Arrays are not almost equal to 5 decimals
E        ACTUAL: 4.661083221435547e-05
E        DESIRED: 1e-05

nevergrad\common\test_tools.py:87: AssertionError

nevergrad/common/test_tools.py ⨯                                                                          11% █▏
nevergrad/functions/test_corefuncs.py ✓                                                                   11% █▏
[...]
nevergrad/instrumentation/test_discretization.py ✓✓✓✓✓✓✓✓✓✓✓✓✓                                            27% ██▊
nevergrad/instrumentation/test_discretization.py ✓✓✓✓✓✓✓✓✓✓✓✓✓✓                                           27% ██▊

――――――――――――――――――――――――――――――――――――――――――――――― test_symlink_folder_tree ―――――――――――――――――――――――――――――――――――――――――――――――

    def test_symlink_folder_tree() -> None:
        path = Path(__file__).absolute().parents[1]
        assert path.name == "nevergrad"
        with tempfile.TemporaryDirectory() as folder:
>           instantiate.symlink_folder_tree(path, folder)

nevergrad\instrumentation\test_instantiate.py:20:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
nevergrad\instrumentation\instantiate.py:94: in symlink_folder_tree
    symlink_folder_tree(fp, shadow_fp)
nevergrad\instrumentation\instantiate.py:94: in symlink_folder_tree
    symlink_folder_tree(fp, shadow_fp)
nevergrad\instrumentation\instantiate.py:96: in symlink_folder_tree
    shadow_fp.symlink_to(fp)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = WindowsPath('C:/Users/dimo/AppData/Local/Temp/tmp7le_w28a/benchmark/additional/example.py')
target = WindowsPath('C:/Users/dimo/Desktop/nevergrad/nevergrad/benchmark/additional/example.py')
target_is_directory = False

    def symlink_to(self, target, target_is_directory=False):
        """
        Make this path a symlink pointing to the given path.
        Note the order of arguments (self, target) is the reverse of os.symlink's.
        """
        if self._closed:
            self._raise_closed()
>       self._accessor.symlink(target, self, target_is_directory)
E       OSError: symbolic link privilege not held

..\..\Anaconda3\lib\pathlib.py:1330: OSError

nevergrad/instrumentation/test_instantiate.py ⨯                                                           27% ██▊
nevergrad/instrumentation/test_instantiate.py ⨯✓                                                          28% ██▊
[...]
nevergrad/instrumentation/test_instantiate.py ⨯✓✓✓✓✓✓✓✓                                                   28% ██▉
nevergrad/instrumentation/test_instantiate.py ⨯✓✓✓✓✓✓✓✓✓                                                  28% ██▉

―――――――――――――――――――――――――――――――――――――― test_folder_instantiator[with_clean_copy] ―――――――――――――――――――――――――――――――――――――――

clean_copy = True

    @testing.parametrized(
        with_clean_copy=(True,),
        without_clean_copy=(False,),
    )
    def test_folder_instantiator(clean_copy: bool) -> None:
        path = Path(__file__).parent / "examples"
        ifolder = instantiate.FolderInstantiator(path, clean_copy=clean_copy)
        testing.printed_assert_equal(ifolder.placeholders, _EXPECTED)
        np.testing.assert_equal(len(ifolder.file_functions), 1)
>       with ifolder.instantiate(value1=12, value2=110., string="") as tmp:

nevergrad\instrumentation\test_instantiate.py:68:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
..\..\Anaconda3\lib\contextlib.py:112: in __enter__
    return next(self.gen)
nevergrad\instrumentation\instantiate.py:215: in instantiate
    self.instantiate_to_folder(subtempfolder, kwargs)
nevergrad\instrumentation\instantiate.py:204: in instantiate_to_folder
    symlink_folder_tree(self.folder, outfolder)
nevergrad\instrumentation\instantiate.py:96: in symlink_folder_tree
    shadow_fp.symlink_to(fp)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = WindowsPath('C:/Users/dimo/AppData/Local/Temp/tmp64j87q_g/examples/script.py')
target = WindowsPath('C:/Users/dimo/AppData/Local/Temp/tmp_clean_copy_tmko4nqc/examples/script.py')
target_is_directory = False

    def symlink_to(self, target, target_is_directory=False):
        """
        Make this path a symlink pointing to the given path.
        Note the order of arguments (self, target) is the reverse of os.symlink's.
        """
        if self._closed:
            self._raise_closed()
>       self._accessor.symlink(target, self, target_is_directory)
E       OSError: symbolic link privilege not held

..\..\Anaconda3\lib\pathlib.py:1330: OSError

 nevergrad/instrumentation/test_instantiate.py ⨯                                                           29% ██▉

――――――――――――――――――――――――――――――――――――― test_folder_instantiator[without_clean_copy] ―――――――――――――――――――――――――――――――――――――

clean_copy = False

    @testing.parametrized(
        with_clean_copy=(True,),
        without_clean_copy=(False,),
    )
    def test_folder_instantiator(clean_copy: bool) -> None:
        path = Path(__file__).parent / "examples"
        ifolder = instantiate.FolderInstantiator(path, clean_copy=clean_copy)
        testing.printed_assert_equal(ifolder.placeholders, _EXPECTED)
        np.testing.assert_equal(len(ifolder.file_functions), 1)
>       with ifolder.instantiate(value1=12, value2=110., string="") as tmp:

nevergrad\instrumentation\test_instantiate.py:68:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
..\..\Anaconda3\lib\contextlib.py:112: in __enter__
    return next(self.gen)
nevergrad\instrumentation\instantiate.py:215: in instantiate
    self.instantiate_to_folder(subtempfolder, kwargs)
nevergrad\instrumentation\instantiate.py:204: in instantiate_to_folder
    symlink_folder_tree(self.folder, outfolder)
nevergrad\instrumentation\instantiate.py:96: in symlink_folder_tree
    shadow_fp.symlink_to(fp)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = WindowsPath('C:/Users/dimo/AppData/Local/Temp/tmpsteavf_a/examples/script.py')
target = WindowsPath('C:/Users/dimo/Desktop/nevergrad/nevergrad/instrumentation/examples/script.py')
target_is_directory = False

    def symlink_to(self, target, target_is_directory=False):
        """
        Make this path a symlink pointing to the given path.
        Note the order of arguments (self, target) is the reverse of os.symlink's.
        """
        if self._closed:
            self._raise_closed()
>       self._accessor.symlink(target, self, target_is_directory)
E       OSError: symbolic link privilege not held

..\..\Anaconda3\lib\pathlib.py:1330: OSError

nevergrad/instrumentation/test_instantiate.py ⨯                                                           29% ██▉
nevergrad/instrumentation/test_instantiate.py ⨯✓                                                          29% ██▉
[...]
nevergrad/instrumentation/test_instantiate.py ⨯✓✓✓✓✓✓                                                     29% ██▉
nevergrad/instrumentation/test_instantiate.py ⨯✓✓✓✓✓✓✓                                                    29% ██▉

――――――――――――――――――――――――――――――――――――――――――――――――― test_folder_function ―――――――――――――――――――――――――――――――――――――――――――――――――

    def test_folder_function() -> None:
        folder = Path(__file__).parent / "examples"
        func = instantiate.FolderFunction(str(folder), ["python", "examples/script.py"], clean_copy=True)
>       output = func(value1=98, value2=12, string="plop")

nevergrad\instrumentation\test_instantiate.py:114:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
nevergrad\instrumentation\instantiate.py:279: in __call__
    with self.instantiator.instantiate(**kwargs) as folder:
..\..\Anaconda3\lib\contextlib.py:112: in __enter__
    return next(self.gen)
nevergrad\instrumentation\instantiate.py:215: in instantiate
    self.instantiate_to_folder(subtempfolder, kwargs)
nevergrad\instrumentation\instantiate.py:204: in instantiate_to_folder
    symlink_folder_tree(self.folder, outfolder)
nevergrad\instrumentation\instantiate.py:96: in symlink_folder_tree
    shadow_fp.symlink_to(fp)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = WindowsPath('C:/Users/dimo/AppData/Local/Temp/tmpf2n3_rpg/examples/script.py')
target = WindowsPath('C:/Users/dimo/AppData/Local/Temp/tmp_clean_copy_lsj82by7/examples/script.py')
target_is_directory = False

    def symlink_to(self, target, target_is_directory=False):
        """
        Make this path a symlink pointing to the given path.
        Note the order of arguments (self, target) is the reverse of os.symlink's.
        """
        if self._closed:
            self._raise_closed()
>       self._accessor.symlink(target, self, target_is_directory)
E       OSError: symbolic link privilege not held

..\..\Anaconda3\lib\pathlib.py:1330: OSError

nevergrad/instrumentation/test_instantiate.py ⨯                                                           30% ██▉
nevergrad/instrumentation/test_multivariables.py ✓                                                        30% ███
[...]
✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓                 52% █████▎                                                 ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓                52% █████▎                                                 ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓               52% █████▎                                                 capi_return is NULL
Call-back cb_calcfc_in__cobyla__user__routines failed.
                                             ✓✓capi_return is NULL█████▎
Call-back cb_calcfc_in__cobyla__user__routines failed.
                                             ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓            52% █████▎                                                 ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓           52% █████▎
[...]
████████▊                                              ✓✓✓✓✓✓✓✓✓                                                     88% ████████▊                                              ✓✓✓✓✓✓✓✓✓✓                                                    88% ████████▊                                              ✓✓✓✓✓✓✓✓✓✓✓                                                   88% ████████▊                                              ✓✓✓✓✓capi_return is NULL
Call-back cb_calcfc_in__cobyla__user__routines failed.
                                             ✓✓✓✓✓✓capi_return is NULL
Call-back cb_calcfc_in__cobyla__user__routines failed.
                                             ✓✓✓✓✓✓✓✓✓✓✓✓✓✓                                                89% ████████▉                                              ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓                                               89% ████████▉                                              ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓                                              89% ████████▉                                              ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓
[...]
99% ██████████ nevergrad/optimization/test_utils.py ✓✓✓✓✓✓✓✓✓✓                                                           99% ██████████ nevergrad/optimization/test_utils.py ✓✓✓✓✓✓✓✓✓✓✓                                                         100% ██████████
=================================================== warnings summary ===================================================
C:\Users\dimo\Anaconda3\lib\site-packages\cma\utilities\utils.py:8
  C:\Users\dimo\Anaconda3\lib\site-packages\cma\utilities\utils.py:8: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
    from collections import MutableMapping  # since Python 2.4?

nevergrad/benchmark/test_experiments.py::test_experiments_registry[mlda]
nevergrad/benchmark/test_experiments.py::test_experiments_registry[mldaas]
nevergrad/benchmark/test_experiments.py::test_experiments_registry[realworld]
  C:\Users\dimo\Desktop\nevergrad\nevergrad\functions\mlda\problems.py:45: RuntimeWarning: invalid value encountered in true_divide
    self._points /= np.std(self._points, axis=0, keepdims=True)

nevergrad/benchmark/test_experiments.py::test_experiments_registry[mlda]
nevergrad/benchmark/test_experiments.py::test_experiments_registry[mldaas]
nevergrad/benchmark/test_experiments.py::test_experiments_registry[realworld]
  C:\Users\dimo\Desktop\nevergrad\nevergrad\functions\mlda\problems.py:181: RuntimeWarning: invalid value encountered in true_divide
    raw_data /= np.std(raw_data, axis=0, keepdims=True)

nevergrad/optimization/test_optimizerlib.py::test_optimizers_suggest[DEwithLHS]
nevergrad/optimization/test_optimizerlib.py::test_optimizers_suggest[DEwithLHSdim]
nevergrad/optimization/test_optimizerlib.py::test_optimizers_suggest[DEwithLHS30]
nevergrad/optimization/test_optimizerlib.py::test_optimizers_recommendation[DEwithLHS]
nevergrad/optimization/test_optimizerlib.py::test_optimizers_recommendation[DEwithLHSdim]
nevergrad/optimization/test_optimizerlib.py::test_optimizers_recommendation[DEwithLHS30]
  C:\Users\dimo\Desktop\nevergrad\nevergrad\optimization\differentialevolution.py:216: InefficientSettingsWarning: DE algorithms are inefficient with budget < 60
    warnings.warn("DE algorithms are inefficient with budget < 60", base.InefficientSettingsWarning)

nevergrad/optimization/test_optimizerlib.py::test_optimizers_suggest[PSOwithLHS30]
nevergrad/optimization/test_optimizerlib.py::test_optimizers_recommendation[PSOwithLHS30]
  C:\Users\dimo\Desktop\nevergrad\nevergrad\optimization\optimizerlib.py:633: InefficientSettingsWarning: PSO is inefficient with budget < 60
    warnings.warn("PSO is inefficient with budget < 60", base.InefficientSettingsWarning)

-- Docs: https://docs.pytest.org/en/latest/warnings.html

----------- coverage: platform win32, python 3.7.3-final-0 -----------
Name                                              Stmts   Miss  Cover
---------------------------------------------------------------------
nevergrad\__init__.py                                 7      0   100%
nevergrad\benchmark\__init__.py                       2      0   100%
nevergrad\benchmark\__main__.py                      51      5    90%
nevergrad\benchmark\additional\example.py            21      1    95%
nevergrad\benchmark\core.py                         116      9    92%
nevergrad\benchmark\execution.py                     77      2    97%
nevergrad\benchmark\experiments.py                  249      0   100%
nevergrad\benchmark\frozenexperiments.py            151      0   100%
nevergrad\benchmark\plotting.py                     293     17    94%
nevergrad\benchmark\xpbase.py                       127      5    96%
nevergrad\common\__init__.py                          0      0   100%
nevergrad\common\decorators.py                       38      0   100%
nevergrad\common\testing.py                          65      5    92%
nevergrad\common\tools.py                            92      6    93%
nevergrad\common\typetools.py                        16      3    81%
nevergrad\functions\__init__.py                       3      0   100%
nevergrad\functions\arcoating\__init__.py             1      0   100%
nevergrad\functions\arcoating\core.py                43      1    98%
nevergrad\functions\corefuncs.py                    200      5    98%
nevergrad\functions\functionlib.py                  111      2    98%
nevergrad\functions\mlda\__init__.py                  1      0   100%
nevergrad\functions\mlda\datasets.py                 59     11    81%
nevergrad\functions\mlda\problems.py                106      2    98%
nevergrad\functions\multiobjective\__init__.py        1      0   100%
nevergrad\functions\multiobjective\core.py           47      0   100%
nevergrad\functions\multiobjective\pyhv.py          169     89    47%
nevergrad\functions\photonics\__init__.py             1      0   100%
nevergrad\functions\photonics\core.py                43      2    95%
nevergrad\functions\rl\__init__.py                    3      0   100%
nevergrad\functions\rl\agents.py                    105      3    97%
nevergrad\functions\rl\base.py                      130      8    94%
nevergrad\functions\rl\envs.py                       68      0   100%
nevergrad\functions\utils.py                         25      2    92%
nevergrad\instrumentation\__init__.py                 6      0   100%
nevergrad\instrumentation\core.py                    87      4    95%
nevergrad\instrumentation\discretization.py          43      0   100%
nevergrad\instrumentation\instantiate.py            177     34    81%
nevergrad\instrumentation\multivariables.py         132      3    98%
nevergrad\instrumentation\transforms.py             120      5    96%
nevergrad\instrumentation\utils.py                   57      8    86%
nevergrad\instrumentation\variables.py              177      9    95%
nevergrad\optimization\__init__.py                    4      0   100%
nevergrad\optimization\base.py                      302     30    90%
nevergrad\optimization\callbacks.py                  57      2    96%
nevergrad\optimization\differentialevolution.py     150      3    98%
nevergrad\optimization\families.py                    9      0   100%
nevergrad\optimization\mutations.py                  44      1    98%
nevergrad\optimization\oneshot.py                   175      2    99%
nevergrad\optimization\optimizerlib.py              803     22    97%
nevergrad\optimization\recaster.py                  133     16    88%
nevergrad\optimization\recastlib.py                  46      2    96%
nevergrad\optimization\requirements_check.py         10     10     0%
nevergrad\optimization\rescaledoneshot.py            39      0   100%
nevergrad\optimization\sequences.py                 134      4    97%
nevergrad\optimization\utils.py                     204      9    96%
---------------------------------------------------------------------
TOTAL                                              5330    342    94%

Results (157.22s):
     900 passed
       6 failed
         - nevergrad\benchmark/test_core.py:40 test_commandline_launch
         - nevergrad\common/test_tools.py:80 test_sleeper
         - nevergrad\instrumentation/test_instantiate.py:16 test_symlink_folder_tree
         - nevergrad\instrumentation/test_instantiate.py:59 test_folder_instantiator[with_clean_copy]
         - nevergrad\instrumentation/test_instantiate.py:59 test_folder_instantiator[without_clean_copy]
         - nevergrad\instrumentation/test_instantiate.py:111 test_folder_function

Observed Results

see above: the tests did not pass on my Windows machine.

Expected Results

What did you expect to happen? That all tests pass in the first place.
Suggestion: set up AppVeyor (or similar) tests to also cover issues on Windows machines (often the most problematic ones)

Relevant Code

see above

Hi,
Thank you for raising the issue!
We're developing on MacOS and Ubuntu here, so as you point out, we've not been able to check everything worked well on Windows. If you have time to help, we can for sure make it work though.

pip install -e '.[all]'
Can you check if pip install -e .[all]
works better for you? With MaxOS and Ubuntu, the first one works on both bash and zsh, while the second only works for bash. If the second works on windows, I'll update the documentation and make a specific case for zsh.
Note that with pip install -e . you only install the main dependencies so many things are not expected to work (all the benchmark part + all the dev related packages such as pytest, and numpy-stubs since #450)

OSError: symbolic link privilege not held
It seems Windows requires admin rights to create a symlink. This test should probably just be deactivated under Windows, not sure we can do much more about it, and not sure it would matter anyway.

KeyError: 'additional_experiment'
This is due to a dynamic import that seems to fail on Windows, no idea why. This is a feature that is probably hardly used at all, so I see two options:

you're willing to investigate and find a way to make it work on Windows (on my side, I can't iterate fast enough on it without actually working on Windows)
we just ignore the test on Windows and if somebody needs the feature, they'll help :)

4. test_sleeper
This test is messy because it depends on the current load and resources of the computer... I'll probably just remove most of it.

Adding CI for Windows: indeed. When we are able to solve all this, we can add it. There's an example of how to do it in a related project: https://github.com/facebookresearch/hydra/blob/master/.circleci/config.yml

I think I can provide quick fixes for most of it easily so I'll do it. For 3., I'll just ignore the test for now, if you're willing to investigate just don't hesitate ;) but I'm not sure it's really necessary.
I'll probably need help testing this though if you have some time.

@brockho can you check if #451 works for you? And as mention above, let me know if pip install -e .[all] also works on Windows

pip install -e .[all] seems to work on Windows. #488 also improves a bit on the test compatibility.

I agree, with the newest version, the dependency with tools.nnwrap is gone and the pip install -e .[all] works also for me under Windows. Nice :-)
Many thanks!

No problem ;) I think there are still a few errors in the test on Windows though.
I'll try again when I have time to experiment on Windows again, or I'll add a Windows CI if it's straightforward enough.

CI can be activated for Windows #494
However, it is very flaky and I don't have much time to investigate right now :(

I've activated the CI since multiple runs seemed to work just fine, I'll close this task now and create a new one for solving flaky tests under Windows when/if we identify some.
Also, don't hesitate to create new ones if you encounter any issue!

authors: brockho jrapin

repo: facebookresearch/nevergrad


text:
enable alltoall primitive

Summary: enable alltoall primitive, which takes a list of tensors for input and another list of tensors for output. We use Alltoallv in UCC to avoid flattening the tensors.
Differential Revision: D35159293

cc @Sergei-Lebedev - could you please help review this PR? Thanks!

source: gharchive/pull-request

license: MIT

license_type: permissive

license_source: github-api


text:
host on fly.io

Is there any way to use this image on fly.io?

Hello, this is the source code for a docker image. I did a quick search and it appears that fly.io is a host for deploying docker images. Following their docs, you should be able to host this on fly.io.
I see the command should be:
fly launch --image factoriotools/factorio:stable. This would start the hosted image for this repo with the stable version tag. I am unsure what ports/permissions fly.io will allow, so hopefully you can expose the correct ports (sometimes this may cost extra or not be available if you do not have a VPS/server with the full port range available).

The tl;dr is "yes".
Quick technical correction -- while fly.io processes docker images for app deployments, they don't actually use docker (or any other container engine) on the backend for running the final product. They extract the filesystem and run it on a tiny Firecracker VM, and you have wide permissions in there.
Ports can be exposed via [[services]] in the fly.toml configuration file for your Fly app.
In this case you'd also need a dedicated IPv4 address because Fly's infra doesn't support UDP over shared IPv4s or IPv6.

source: gharchive/issue

authors: IslamZaoui fisher60 miend

repo: factoriotools/factorio-docker


text:
URLDownloadToCacheFileW and URLDownloadToFileW

These two should have String arguments changed to LongPtr I think:
Public Declare PtrSafe Function URLDownloadToCacheFileW Lib "urlmon" (ByVal lpUnkcaller As stdole.IUnknown, ByVal szURL As String, ByVal szFileName As LongPtr, ByVal dwBufLength As Long, ByVal dwReserved As Long, ByVal pbsc As IBindStatusCallback) As Long
Public Declare PtrSafe Function URLDownloadToFileW Lib "urlmon" (ByVal pCaller As stdole.IUnknown, ByVal szURL As String, ByVal szFile As String, ByVal dwReserved As Long, ByVal lpfnCB As IBindStatusCallback) As Long

Well thats annoying; I made those changes and somehow they didn't take... maybe forgot to save, or tB did that fakeout...
Fixed in v8.0.404, available now.
Hey is this something regex would be able to help with? If so, if you're up for it and whenever you have the time, could you check for other W functions that have String arguments but not DeclareWide?

I ran the search for unAliased W's with String Args and included in list below. For each of those found, I also examined their corresponding A and Aliased siblings and included what looked to be incorrect in those too (for example, some A/W versions were misidentified as W/A respectively).
Public Declare PtrSafe Function CommandLineToArgvW Lib "shell32" (ByVal lpCmdLine As String, pNumArgs As Long) As LongPtr

'the DeclareWide Aliased version has LongPtr that should be String
Public DeclareWide PtrSafe Function PathIsSystemFolder Lib "shlwapi" Alias "PathIsSystemFolderW" (ByVal pszPath As LongPtr, ByVal dwAttrb As Long) As BOOL
Public Declare PtrSafe Function PathIsSystemFolderW Lib "shlwapi" (ByVal pszPath As String, ByVal dwAttrb As Long) As BOOL

Public Declare PtrSafe Function UrlCreateFromPathW Lib "shlwapi" (ByVal pszPath As String, ByVal pszUrl As String, ByRef pcchUrl As Long, ByVal dwFlags As Long) As Long

'the DeclareWide Aliased version has LongPtr that should be String
Public Declare PtrSafe Function IsBadStringPtr Lib "kernel32" Alias "IsBadStringPtrW" (ByVal lpsz As LongPtr, ByVal ucchMax As LongPtr) As BOOL
Public Declare PtrSafe Function IsBadStringPtrW Lib "kernel32" (ByVal lpsz As String, ByVal ucchMax As LongPtr) As BOOL
Public Declare PtrSafe Function CreateWaitableTimerExW Lib "kernel32" (lpTimerAttributes As SECURITY_ATTRIBUTES, ByVal lpTimerName As String, ByVal dwFlags As CreateWaitableTimerExFlags, ByVal dwDesiredAccess As Long) As LongPtr

'switch the args between W and A versions?
Public Declare PtrSafe Function CreateServiceA Lib "advapi32" (ByVal hService As LongPtr, ByVal lpServiceName As LongPtr, ByVal lpDisplayName As LongPtr, ByVal dwDesiredAccess As Long, ByVal dwServiceType As ServiceType, ByVal dwStartType As ServiceStartType, ByVal dwErrorControl As ServiceErrorControlType, Optional ByVal lpBinaryPathName As LongPtr, Optional ByVal lpLoadOrderGroup As LongPtr, Optional ByRef lpdwTagId As Long, Optional ByVal lpDependencies As LongPtr, Optional ByVal lpServiceStartName As LongPtr, Optional ByVal lpPassword As LongPtr) As BOOL
Public Declare PtrSafe Function CreateServiceW Lib "advapi32" (ByVal hService As LongPtr, ByVal lpServiceName As LongPtr, ByVal lpDisplayName As LongPtr, ByVal dwDesiredAccess As Long, ByVal dwServiceType As ServiceType, ByVal dwStartType As ServiceStartType, ByVal dwErrorControl As ServiceErrorControlType, Optional ByVal lpBinaryPathName As String, Optional ByVal lpLoadOrderGroup As String, Optional ByRef lpdwTagId As Long, Optional ByVal lpDependencies As String, Optional ByVal lpServiceStartName As String, Optional ByVal lpPassword As String) As BOOL

Public Declare PtrSafe Function mmioRenameW Lib "winmm" (ByVal szFileName As LongPtr, ByVal SzNewFileName As String, lpmmioinfo As MMIOINFO, ByVal dwRenameFlags As Long) As MMIOERR

Public Declare PtrSafe Function GetPrintProcessorDirectoryW Lib "winspool.drv" (ByVal pName As LongPtr, ByVal pEnvironment As LongPtr, ByVal Level As Long, ByVal pPrintProcessorInfo As String, ByVal cbBuf As Long, pcbNeeded As Long) As BOOL

Public Declare PtrSafe Function SetFormW Lib "winspool.drv" (ByVal hPrinter As LongPtr, ByVal pFormName As String, ByVal Level As Long, pForm As Any) As BOOL

'switch the args between W and A versions?
Public Declare PtrSafe Function AddPortA Lib "winspool.drv" (ByVal pName As LongPtr, ByVal hwnd As LongPtr, ByVal pMonitorName As LongPtr) As BOOL
Public Declare PtrSafe Function AddPortW Lib "winspool.drv" (ByVal pName As String, ByVal hwnd As LongPtr, ByVal pMonitorName As String) As BOOL

'these two do not have corresponding A and Aliased W
Public Declare PtrSafe Function CredUIStoreSSOCredW Lib "credui.dll" (ByVal pszRealm As String, ByVal pszUsername As String, ByVal pszPassword As String, ByVal bPersist As BOOL) As Long
Public Declare PtrSafe Function CCredUIReadSSOCredW Lib "credui.dll" (ByVal pszRealm As String, ppszUsername As LongPtr) As Long

Public Declare PtrSafe Function GetUrlCacheEntryInfoExW Lib "wininet" (ByVal lpszUrl As LongPtr, lpCacheEntryInfo As INTERNET_CACHE_ENTRY_INFOW, lpcbCacheEntryInfo As Long, ByVal lpszRedirectUrl As String, lpcbRedirectUrl As Long, ByVal lpReserved As LongPtr, ByVal dwFlags As Long) As BOOL

'these two do not have corresponding A and Aliased W
Public Declare PtrSafe Function PrivacySetZonePreferenceW Lib "wininet" (ByVal dwZone As URLZONE, ByVal dwType As PrivacyType, ByVal dwTemplate As INET_PRIVACY_TEMPLATES, Optional ByVal pszPreference As String) As Long
Public Declare PtrSafe Function PrivacyGetZonePreferenceW Lib "wininet" (ByVal dwZone As URLZONE, ByVal dwType As PrivacyType, pdwTemplate As INET_PRIVACY_TEMPLATES, Optional ByVal pszBuffer As String, Optional pdwBufferLength As Long) As Long

There's no CredUIStoreSSOCred[A} defined in the Windows API, same for the one below that, and PrivacyGet/SetZonePreferenceW, and the CmdLine...-- but there's only 1 C at the start of CredUIReadSSOCredW so glad that was caught. I'm going to use overloads so either String or LongPtr can be used with W.
Thanks again, really great working finding all these bugs 💯

All identified issues fixed in v8.0.406, available now.

repo: fafalone/WinDevLib


text:
循环列表急停

循环列表
滚动时会瞬间停止，而且每次都停在第二章那张图片上，说明计算位置时出现了bug

在刚好跨越边界时会出现突然停止的情况。这个边界通常是6倍于列表项目数的大小。


text:
[fetaure] Add encoder/decoder chain implementation

Closes #11

@els0r FYI: I extended the tests a bit in the last commit, apart from that g2g - I'll go ahead an merge, thx for filing this one!


text:
new(content/en/docs): add Python client docs

Signed-off-by: Mattia Meleleo melmat@tuta.io
What type of PR is this?
/kind cleanup
/kind content
Any specific area of the project related to this PR?
/area documentation
What this PR does / why we need it:
Adds Python client documentation
Which issue(s) this PR fixes:
Fixes https://github.com/falcosecurity/falco-website/issues/126

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To complete the pull request process, please assign leodido
You can assign the PR to them by writing /assign @leodido in a comment when ready.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

LGTM label has been added.  Git tree hash: 4d217b3293cd862e036f6de07d074efe60d4b9e4

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: leodido
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [leodido]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

repo: falcosecurity/falco-website


text:
Release/0.26.1 + installation fixes

What type of PR is this?
/kind content
Any specific area of the project related to this PR?
/area blog
/area documentation
What this PR does / why we need it:
A blog post for Falco 0.26.1 release.
Fixes for installation of binary release.
Which issue(s) this PR fixes:
NONE
Special notes for your reviewer:

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To complete the pull request process, please assign leodido
You can assign the PR to them by writing /assign @leodido in a comment when ready.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

LGTM label has been added.  Git tree hash: 22a58cd3ccf63fab31a49723f0fe9e11dbd9b21e

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: fntlnz
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [fntlnz]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: fntlnz, leogr
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [fntlnz,leogr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

New changes are detected. LGTM label has been removed.

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To complete the pull request process, please assign fntlnz
You can assign the PR to them by writing /assign @fntlnz in a comment when ready.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To complete the pull request process, please assign fntlnz
You can assign the PR to them by writing /assign @fntlnz in a comment when ready.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To complete the pull request process, please assign fntlnz
You can assign the PR to them by writing /assign @fntlnz in a comment when ready.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

LGTM label has been added.  Git tree hash: c800b7cc5902561df8011d807208e5017795b4d0

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: leogr
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [leogr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

authors: leodido poiana


text:
Rule not working: Create/Modify Configmap With Private Credentials

Rule - Create/Modify Configmap With Private Credentials seems not working by default following https://falco.org/docs/event-sources/kubernetes-audit/ on version 0.22.1.
Other k8s audit rules would flag without problems.
Command in use to create configmap:
kubectl create -f configmap.yaml
Audit log output:
{
"kind": "Event",
"apiVersion": "audit.k8s.io/v1",
"level": "RequestResponse",
"auditID": "59a27ad6-5164-46a9-8c63-0a71bcc2ed8e",
"stage": "ResponseComplete",
"requestURI": "/api/v1/namespaces/default/configmaps",
"verb": "create",
"user": {
"username": "kubernetes-admin",
"groups": [
"system:masters",
"system:authenticated"
]
},
"sourceIPs": [
"10.211.55.12"
],
"userAgent": "kubectl/v1.18.2 (linux/amd64) kubernetes/52c56ce",
"objectRef": {
"resource": "configmaps",
"namespace": "default",
"name": "my-config1",
"apiVersion": "v1"
},
"responseStatus": {
"metadata": {},
"code": 201
},
"requestObject": {
"kind": "ConfigMap",
"apiVersion": "v1",
"metadata": {
"name": "my-config1",
"namespace": "default",
"creationTimestamp": null
},
"data": {
"access.properties": "aws_access_key_id = MY-ID\naws_secret_access_key = MY-KEY\npassword = NICK\n"
}
},
"responseObject": {
"kind": "ConfigMap",
"apiVersion": "v1",
"metadata": {
"name": "my-config1",
"namespace": "default",
"selfLink": "/api/v1/namespaces/default/configmaps/my-config1",
"uid": "66cd8571-fa16-40fb-b485-293e6c2d4d61",
"resourceVersion": "219832",
"creationTimestamp": "2020-05-05T07:17:06Z",
"managedFields": [
{
"manager": "kubectl",
"operation": "Update",
"apiVersion": "v1",
"time": "2020-05-05T07:17:06Z",
"fieldsType": "FieldsV1",
"fieldsV1": {
"f:data": {
".": {},
"f:access.properties": {}
}
}
}
]
},
"data": {
"access.properties": "aws_access_key_id = MY-ID\naws_secret_access_key = MY-KEY\npassword = NICK\n"
}
},
"requestReceivedTimestamp": "2020-05-05T07:17:06.458299Z",
"stageTimestamp": "2020-05-05T07:17:06.461035Z",
"annotations": {
"authorization.k8s.io/decision": "allow",
"authorization.k8s.io/reason": ""
}
}

Update: this rule is not working when the "user" is not in allowed_k8s_users, in my case "kubernetes-admin"

When user "kubernetes-admin" is not in list allowed_k8s_users:

kubectl delete configmap
kubectl create configmap config --from-file=configmap

Alerts flagged:
May  5 15:57:18 k8s-controlplane1 falco: 15:57:04.389830912: Warning K8s Operation performed by user not in allowed list of users (user=kubernetes-admin target=nickconfig/configmaps verb=create uri=/api/v1/namespaces/default/configmaps resp=201)
May  5 15:57:18 k8s-controlplane1 falco: 15:56:59.000089088: Warning K8s Operation performed by user not in allowed list of users (user=kubernetes-admin target=nickconfig/configmaps verb=delete uri=/api/v1/namespaces/default/configmaps/nickconfig resp=200)

When user "kubernetes-admin" is in list allowed_k8s_users:

kubectl delete configmap
kubectl create configmap config --from-file=configmap

Alers flagged:
May  5 15:59:18 k8s-controlplane1 falco: 15:59:17.949396992: Notice K8s ConfigMap Deleted (user=kubernetes-admin configmap=nickconfig ns=default resp=200 decision=allow reason=)
May  5 15:59:48 k8s-controlplane1 falco: 15:59:22.695921920: Warning K8s configmap with private credential (user=kubernetes-admin verb=create configmap=nickconfig config={"test-config1":"apiVersion: v1\ndata:\n  access.properties: |\n    aws_access_key_id = MY-ID\n    aws_secret_access_key = MY-KEY\n    password = NICK\nkind: ConfigMap\nmetadata:\n  name: my-config1\n  namespace: default\n"})

@yinynick I had the exact same issue with Create/Modify Configmap With Private Credentials rule. Solved it by adding kubernetes-admin to the list of allowed users.

Please, keep open.
I have had the same issue.

@yinynick: There is not a label identifying the kind of this issue.
Please specify it either using /kind <group> or manually from the side menu.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

authors: bzurkowski leogr poiana yinynick

repo: falcosecurity/falco


text:
new(driver-bpf,driver-kmod,libscap,libscap-engine-bpf,libscap-engine-kmod): extend buffer event drop metrics

Signed-off-by: incertum melissa.kilby.oss@gmail.com
What type of PR is this?
/kind feature
Any specific area of the project related to this PR?
/area driver-bpf
/area libscap
/area libscap-engine-bpf
What this PR does / why we need it:

Add n_drops_buffer_* stats for categories of system calls when the perf buffer between kernel and userspace is full (eBPF).
Proposed changes hold the potential to more accurately measure the impact of future optimizations that address kernel side event drops than is currently possible solely based on the total n_drops_buffer.
Breakdown of event drops per system call category can serve as reflection of the servers / clusters workload footprint, e.g. expect higher drops for open system calls, because this is a more high volume system call. However, the ratios between the categories vary significantly across workloads. For instance for the open system call category we observed ratios of 2000:1 or 5:3 (open system call drops:next highest category drops).

Kicking off a discussion around possible future optimizations to reduce kernel side event drops:

When using Falco on bare-metal production servers with 50-60% average load in simpleconsumer mode event drops likely won't be much of an issue. On the other hand bare-metal servers with higher average loads and/or unique workloads may experience event drops on a fraction of the fleet.
Several issues in the past around event drops have been raised. However, it appears more work needs to be done to address the issue of kernel side event drops in order to prevent them from happening even under the most extreme conditions.
More specifically, would like to re-start a conversation around buffer size (e.g. see https://github.com/falcosecurity/falco/issues/813), but tailored towards eBPF. eBPF driver currently uses a perf buffer

perf_map of type BPF_MAP_TYPE_PERF_EVENT_ARRAY, see https://github.com/falcosecurity/libs/blob/master/driver/bpf/maps.h#L23 (note that .max_entries is set to number of cpus in userspace)
per-CPU buffer size is specified in userspace, see https://github.com/falcosecurity/libs/blob/master/userspace/libscap/engine/bpf/scap_bpf.c#L149 and defined as:

static const int BUF_SIZE_PAGES = 2048;

int page_size = getpagesize();
int ring_size = page_size * BUF_SIZE_PAGES;
int header_size = page_size;
int total_size = ring_size * 2 + header_size;

Assuming page_size is 4096 bytes we get a total of ~16.78 MB hard-coded buffer size [(4096 bytes * 2048 * 2) + 4096 bytes]
What experiments led to the choice of the current default settings? What are the trade-offs of increasing / decreasing buffer size with respect to both

(1) throughput rate and
(2) CPU utilization?

Would there be appetite to make BUF_SIZE_PAGES (aka number of pages, needs to be power of 2) configurable over a cmdline flag like suggested in the earlier issue listed above? That way end-users could optimize deployments for both bare-metal and the world of hypervisors where Falco runs in each guest OS (VMs, micro-VMs etc). At the same time, recommendations and crowd-sourced experimental results for choosing the right buffer size in a particular environment would be awesome to guide decision making.
Alternative of BPF_MAP_TYPE_RINGBUF got introduced in kernel 5.8, therefore still a bit forward leaning, but could initial experimental results of the benefits be shared beyond @fntlnz post?

Finally, besides adjusting buffer size pushing targeted pre-filters into the kernel (eBPF option) beyond simpleconsumer mode may be another viable alternative in order to drop the uninteresting portion of events for the system calls of interest before mapping to userspace while not generating unnecessary overhead in the kernel program - looking into it.

Special notes for your reviewer:

Finalized by https://github.com/falcosecurity/falco/pull/2079.
Possibly more categories could be added or different buckets of system calls can be implemented instead.

Does this PR introduce a user-facing change?:
NONE

How was this tested?:
Deployed to a production environment - working as expected, generates useful insights.
Related / Tracking:

https://github.com/falcosecurity/falco/issues/1403

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: incertum
To complete the pull request process, please assign andreagit97 after the PR has been reviewed.
You can assign the PR to them by writing /assign @andreagit97 in a comment when ready.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Hi @incertum. Thanks for your PR.
I'm waiting for a falcosecurity member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.
Once the patch is verified, the new status will be reflected by the ok-to-test label.
I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

/area driver-bpf

Hi @incertum this PR will be essential to understand the reason behind the huge numbers of drops in some environments, thank you very much for it!  The ratios between the syscalls categories you have found are really interesting, if you have collected some data, could you please share them with us? Maybe in this PR or where it is most comfortable for you.
We know that under high average loads users may experience huge event drops, unfortunately, this is still an open issue. The kernel-side simple consumer is an attempt to mitigate this phenomenon but in some cases, it is not enough.
The most effective way to reduce the number of drops is to reduce the number of events sent to userspace and we are working on it in 2 different ways:

On one side @Stringy is trying to solve performance problems by reducing the number of syscalls caught by our actual probe, this will inevitably lead to a considerable decrease in the number of events dropped. He recently merged his solution into RedHat ACS Collector Probe with pretty good results as far as I know.

On the other side we are working on a modern BPF probe to exploit all new BPF features like BPF_MAP_TYPE_RINGBUF, bpf global variables, BTF enabled programs, ... you can find more details here modern BPF probe proposal. This probe should solve a lot of problems related to performance and it could address also the drops issue. In a nutshell, the new probe will allow you to choose which syscalls you want to capture avoiding to pollute the userspace with useless events.  The first PR is already there, our hope is to merge it completely ASAP. You can find some discussion on this new simple consumer logic here

Coming back to your topics, the perf buffer size was probably chosen to fit the most common use cases but obviously is not easy to find the correct trade-off that works for all scenarios. I think that increasing the buffer size could be a possible way to mitigate the drops issue, hypothetically the system will slow down a little bit, and this gives more time to userspace to consume events. The main downside of this approach is that the whole system will slow down and CPU usage may grow slightly. Here you can find some other references to this topic.
Making BUF_SIZE_PAGES configurable over a command-line flag seems a great idea to me. Users could try to mitigate the drops issue by finding the best trade-off for their environments, in this way they could configure Falco according to their needs. Big +1 on this.
To address your last point, I'm not sure that the BPF_MAP_TYPE_RINGBUF could really help us on this topic... this new feature will surely increase the performance kernel-side, speeding up the whole system, but reducing the BPF instrumentation cost means also producing more events, so the userspace may face even more events to process, causing further drops. Obviously, this is just a possible scenario, it is not easy to predict the behavior of the running system, but I think that right now we have two main weapons to fight against the drops issue:

reducing the number of captured syscalls (new probe approach or @Stringy's solution for example)
sizing our buffer according to the user's environments and so allowing them to configure the size at runtime with a command-line option for example.

In the end, I can say thank you again for this PR, hoping this will be the first step to finally solving this annoying issue.

/ok-to-test

/test build-libs-bundled-deps

Thanks a bunch for all the super useful additional links!
Thoughts re option (1) - turning off all uninteresting system calls kernel side: Falco introduced optimizations with the so needed simple consumer in Dec 2021 https://github.com/falcosecurity/libs/pull/115. simple consumer mode can be taken to the extreme via hard patching syscall_table with the UF_UNINTERESTING flag - suspecting drops reported in https://github.com/falcosecurity/falco/issues/669 can be completely eliminated with the simple consumer.
Then there is the lucky ones (like myself) who deploy Falco to very large environments with extreme workloads where option (1) is not enough and CPU utilization in general needs to be lower as well, so goal shall actually be to not needing to increase buffer size as sole option.
Some more data-driven insights gathered from a large / diverse fleet based on proposed new metrics w/ customized simple consumer mode and default buffer size:

No drops at all can be achieved on over 90-99% of your fleet and many end users likely now (with the simple consumer) have no problems anymore at all. It's hard to generalize as it all depends on so many factors and ultimately on what system calls you care about and what your SLOs are, but wanted to emphasize that this is about optimizing the long tail. Unfortunately in security we do have to care about every single dropped event.
While it's no secret that everything is a file in Linux, wasn't surprised, but still a bit amazed that when looking at the stats derived from large numbers the open* system calls were the most prominent single source of problem (on average accounting for >65% of drops from the categories of system calls with targeted counters and on average accounting for >30% of all drops). And ironically, recent changes addressing TOCTOU vulnerabilities https://github.com/falcosecurity/falco/security/advisories/GHSA-6v9j-2vm2-ghf7 now also support full arg retrieval for enter events for open and connect system calls - this highlights that open system calls are difficult, but essential for many critical classes of vulnerabilities - and yet we push 99% of unnecessary open events to userspace while risking to drop that one important event.
While these overall stats may help guiding decisions on what and where to optimize, on a per host basis distributions may vary significantly depending on the actual workload, therefore generalizations should be taken with a grain of salt and an array of tools is needed (drop uninteresting syscalls right away, perhaps pre-filter some interesting syscalls kernel side, and adjust buffer size if all else fails)

Looking forward to finding out what the right balance / trade-offs around what to map to userspace in the first place can look like and hoping in 1 yr community can empirically demonstrate significant overall improvements (lower CPU and memory footprint, zero drops in extreme conditions while advancing not sacrificing threat detection capabilities).
Andreagit97 modern-bpf for >= 5.8 looks very promising (hope CPU usage drops significantly as a result) and as you mentioned, could be that as bpf instrumentation is being optimized, there may be more events, so current stats may be an underestimate.

Nice write up re benefits of staying with the per CPU architecture while still benefitting from the new ring buffer for kernels >= 5.8.
read-only global variable will open up safer options of introducing more adventurous approaches, already a fan of it!
Love the proposition of distinguishing between variable size vs fixed sized events pushed onto the buffer.

@Andreagit97

Support configurable buffer size at run-time as you proposed.

Could this be added to existing efforts you mentioned under (2) to have this feature asap in 0.33 release? Would be much appreciated (personal preference would be falco.yaml over cmdline flag or both for easier DevOps).

Yeah sure, this bullet list was just to split different topics, we will surely add it in the next Falco release. :)

Enhance simple consumer logic by dropping other uninteresting syscalls:

update(driver): add getrlimit and ugetrlimit to simple consumer #382
update(libsinsp): drop also untracked syscalls in kernel-side simple consumer mode #211

Enable the simple consumer to be configurable by the user, in order to catch just the interesting syscall for that context (this is not completely possible since sinsp needs at least a minimum set of syscalls to work smooth, but anyway we can try to do something)

Currently parses rules and then tags the ones not in any rule with UF_UNINTERESTING assuming syscall is not tagged with UF_SIMPLEDRIVER_KEEP in syscall_table right?

Unfortunately, this is not exactly what we do right now. This first simple consumer implementation statically selects a subset of syscalls that have a security value or are relevant for the sinsp state and catches only them. This set of syscall is fixed and we have a table in which we mark the uninteresting syscalls: https://github.com/falcosecurity/libs/blob/3e640097903b68e73635ec8e11b8a224cba978cb/userspace/libscap/syscall_info_table.c#L25 The syscalls with flag EF_DROP_SIMPLE_CONS are the ones not interesting so the simple consumer approach doesn't catch them during the capture. As I said this was a first attempt pretty easy to implement, I think that it's time to switch to a more dynamic approach that allows users much freedom as you suggested. This is why I mentioned it in the [3] point.

Seems like a really great approach to me, maybe just a small touch up needed to be able to overwrite at own risk.
* all clone/fork/execve system calls needed for bare minimum state engine build up

* __NR_socket needed to have ip tuple available for other network system calls such as __NR_connect, but maybe end user may want to debug without needing to change rules files or patch src

Forward-leaning (unknown release milestone):
Would be up for it to support exploring some self-aware tuning (for eBPF) with respect to performance and have Falco re-kick itself with updated configs to cut losses in a more controlled manner. Also because of the fact that in practice there are DevOps limits to configuring and updating Falco for different environments.
Factors could be stricter syscall priorities, buffer size and throughput speed. All of which sounds great, but it's not gonna be easy to do and find a balance. One of my co-workers Eric Sage and I shared during the eBPF Summit 2021 that bpftool (bpftool-prog) is great to measure "average time spent in each bpf", available since 5.1, but note that it does not (yet) support resolution at the bpf tail call level. Having a suspicion that there is a tricky relationship between event volume, buffer size and CPU utilization as we saw that for servers with lower loads and lower CPU the average time spent in each bpf was remarkably more volatile (all settings - buffer size and syscalls to be traced were constant). To make matters worse also need to dig more into how userspace stuff truly impacts the tracing flow / pressure. This was mentioned already in some issues here and there.

I think that this approach could be very interesting, reloading Falco with a new configuration when it starts not working well seems a smart idea. The complex thing here is to find the right threshold to restart Falco and one downside of this approach is that with a reload we lose all the current state of the system.  We have also to say that if Falco becomes unusable we do not care a lot of the current system state... IMHO this could be a great feature to have, but I want also to hear what other maintainers think about it @FedeDP @leogr @gnosek @jasondellaluce, and all the others...

/area driver-kmod
/area libscap-engine-kmod

Was doing some more inspection and wondering if it also makes sense to add another new drop statistics "n_drops_sampling". Is this a feature that will be carried over to modern-bpf? Ideally would not want to need random kernel side sampling if the tool is used for threat detection.

Uhm that's a good point, probably you have noticed that today there are 2 dropping logic in our drivers: the simple consumer mode and a logic based on the dropping ratio (the [one]((https://github.com/falcosecurity/libs/blob/master/driver/bpf/plumbing_helpers.h#L446) that you linked in this comment.). Having 2 logics is quite confusing so we are thinking to merge them together! It should be quite simple. For what concerns the modern probe, yes, also the modern probe will have the simple consumer logic, but here we can say we have a simpler and more powerful logic... As soon as our BPF programs start (sys_enter/sys_exit hooks) they will read inside a table of booleans if the syscall must be considered... the cool thing is that this table can be modified a run-time, so you can see it as dynamic-filter that allow you to choose which syscall to take during the execution. Obviously, we need to match this logic with the Falco behavior but we can think of something cool :)

I suppose it all comes back to the question of what direction simple consumer mode is gonna take in the future :) https://github.com/falcosecurity/libs/blob/master/driver/bpf/plumbing_helpers.h#L446

Yeah right now I would leave only the actual stats, and maybe in the next future we could add something else to fit our simple consumer logic

Uhm that's a good point, probably you have noticed that today there are 2 dropping logic in our drivers: the simple consumer mode and a logic based on the dropping ratio (the one that you linked in this comment.). Having 2 logics is quite confusing so we are thinking to merge them together! It should be quite simple. For what concerns the modern probe, yes, also the modern probe will have the simple consumer logic, but here we can say we have a simpler and more powerful logic... As soon as our BPF programs start (sys_enter/sys_exit hooks) they will read inside a table of booleans if the syscall must be considered... the cool thing is that this table can be modified a run-time, so you can see it as dynamic-filter that allow you to choose which syscall to take during the execution. Obviously, we need to match this logic with the Falco behavior but we can think of something cool :)

Truly appreciate all the efforts all of you are putting into getting the simple consumer logic to the next level 🚀 .
dynamic-filtering -> not just cool, but amazing! ... me thinking out loud as ML person :)

Yeah right now I would leave only the actual stats, and maybe in the next future we could add something else to fit our simple consumer logic

Small tweaks you will see in the last commit. TLDR added bunch of comments to help "de-confuse" the stats definition (also wrt already existing fields), but stats typically always remain mysterious. In addition, added many more comments also to the falco code changes. Finally, the dir_file and other_interest surely have potential to be improved, but since we are only interested in coarse grained insights, maybe it's fine. Happy to make more modifications to the switch statements.

LGTM label has been added.  Git tree hash: 8897550ffa6b5bd4869612ba874c34f5f42bd263

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: FedeDP, incertum
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [FedeDP]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Uhm that's a good point, probably you have noticed that today there are 2 dropping logic in our drivers: the simple consumer mode and a logic based on the dropping ratio (the one that you linked in this comment.). Having 2 logics is quite confusing so we are thinking to merge them together! It should be quite simple. For what concerns the modern probe, yes, also the modern probe will have the simple consumer logic, but here we can say we have a simpler and more powerful logic... As soon as our BPF programs start (sys_enter/sys_exit hooks) they will read inside a table of booleans if the syscall must be considered... the cool thing is that this table can be modified a run-time, so you can see it as dynamic-filter that allow you to choose which syscall to take during the execution. Obviously, we need to match this logic with the Falco behavior but we can think of something cool :)

Truly appreciate all the efforts all of you are putting into getting the simple consumer logic to the next level rocket . dynamic-filtering -> not just cool, but amazing! ... me thinking out loud as ML person :)

Yeah right now I would leave only the actual stats, and maybe in the next future we could add something else to fit our simple consumer logic

Small tweaks you will see in the last commit. TLDR added bunch of comments to help "de-confuse" the stats definition (also wrt already existing fields), but stats typically always remain mysterious. In addition, added many more comments also to the falco code changes. Finally, the dir_file and other_interest surely have potential to be improved, but since we are only interested in coarse grained insights, maybe it's fine. Happy to make more modifications to the switch statements.

Thank you very much for the comments, we really need these everywhere!

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: Andreagit97, FedeDP, incertum
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [Andreagit97,FedeDP]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

authors: Andreagit97 incertum leogr poiana

repo: falcosecurity/libs


text:
feat: add more Cloudtrail fields

Add more fields as defined at
https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-event-reference-record-contents.html
What type of PR is this?
/kind feature
Any specific area of the project related to this PR?
/area plugins
What this PR does / why we need it:
The current CloudTrail plugin is missing fields. This PR adds all currently defined fields according to https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-event-reference-record-contents.html.
The fields ct.addendum.* and ct.edgedevicedetails are untested due to missing events with these fields.
Special notes for your reviewer:
The plugin has only been tested with falcodump, Logray and CloudTrail access via S3.

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: uhei
Once this PR has been reviewed and has the lgtm label, please assign jasondellaluce for approval. For more information see the Kubernetes Code Review Process.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Welcome @uhei! It looks like this is your first PR to falcosecurity/plugins 🎉

cc @Issif

cc @Issif @LucaGuerra

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: Issif, uhei
Once this PR has been reviewed and has the lgtm label, please assign jasondellaluce for approval. For more information see the Kubernetes Code Review Process.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

LGTM label has been added.  Git tree hash: 3b1c543ed36edbb69f6a64bd4ae6312d54770666

/lgtm
@leogr do you think we should embed the plugin version bump up in that PR too?

I know there should be other planned changes to this plugin, so we can bump up in a later PR.
Btw, the PR SGTM, too. I will do a second look soon and then approve.
Thanks!

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: Issif, leogr, uhei
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [leogr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

authors: leogr poiana uhei

repo: falcosecurity/plugins


text:
Upgrade auf Java 8

Möglichkeiten von Lamdas bzw. Methodenreferenzen ausloten für das Verbinden von Signalen und Slots.

Resolved

license: mit

license_source: bigquery


text:
Responsiveness : Form height when petition is short

The petition page should be resized to the form's height when the petition is too short (See screens below)
How it renders after the example provided by @fallen :

How it should render :

This relates to https://github.com/fallen/Pytition/issues/44. No need to fix this if the petition template is rewritten, but the follow up implementation should take this into consideration

repo: fallen/Pytition


text:
Subscription Based API

Hey @falood, I wanted to get your feedback on this so far

I created a subscription-based API to ExFSWatch.Worker, so you can now subscribe to get messages from it.
I rewrote the Module based API to spawn a process that subscribes to events and calls the callbacks accordingly
I added a behaviour for ExFSWatch.Backend

Questions:

Do we need a supervisor still? Should subscription clients get a supervised worker or just spawn_link it directly?
Should we move subscribe/1 and spawn_link/1 to the main ExFSWatch module for a friendlier API?

TODO:

[ ] Write tests for both subscription & module based api.

@mveytsman Thank you for your great job!
I think it's better to move subscribe/1 and start_link/1 to main ExFSWatch module 🙂
For the supervisor, I think we should keep it, at least we need it for the child_spec now, but we can remove it after we change the package name.
btw, is there a recommend name?

close this issue, move to #26

authors: falood mveytsman

repo: falood/exfswatch


text:
添加 cdn 刷新

加上了 cdn 刷新功能，第一次抢票时需要修改 /etc/hosts 的权限，以便 cdn ip 的变更，所以 AppStore 审核应该无法通过，个人认为这个 App 也不需要上架到商店，毕竟 12306 变化频繁。
因为修改的 hosts 来改变 dns 解析地址，所以不能用异步并发去做多次查询，只能查询结束一次才切换 cdn ip 进行下一次查询，轮询完 cdn 列表大概需要 1 分钟左右，个人感觉相比原来的单个 cdn 频繁查询得到余票的几率大些，毕竟 12306 放票的时候肯定是每个 cdn 都会分配一定的数量，缺点就是一个 cdn 的下次查询周期是1分钟左右，在这期间如果 12306 对此 cdn 放票就无法抢到。
若是可以针对每个 request 做 dns 动态解析，就能大大提升抢到票的几率了，我今天花了半天时间搜索 dns 动态解析，但是未找到有效的解决方案。
另外添加到 cdn ip 是 hard code，最好还是做动态的 ip 列表，最近工作比较忙时间有限暂时未做，请 review 下代码看是否能合并。
ps：代码结构里闭包和Promise看的我都快晕了～只粗略看了 查询 模块

@1oo1
感谢你的尝试，也试用了一下你改的版本。另外这篇文章里也有讨论一些思路http://www.cnblogs.com/guozili/p/6144561.html
关于这个功能我自己也思考过很久，总感觉有点钻漏洞的感觉（频繁切CDN查询也容易被封IP）。（尤其我这还开源了，这样做影响也不好）
所以我在想我这个软件就不做抢票的处理了，好好做好订票和捡漏票的功能。看看楼主的想法。

确实是钻漏洞了。谢谢你写的这个工具，能坚守着不做抢票处理很佩服你👍

@1oo1 谢谢支持哈～～


text:
dropdown-item  组件,events类型错误？

dropdown-item  组件,events类型错误？remaxjs直接使用change无效 ,

这里确实是类型问题，vant的事件几乎都要加上bind前缀
这会后续修改，或者你pr一个？

authors: WangXang fanhaoyuan

repo: fanhaoyuan/vant-weapp-typings


text:
Create and publish gem

[ ] Init gemspec
[ ] Finalize gem name
[ ] publish gem

Done

repo: faraazahmad/trains


text:
Reset countdown time

Hello, After the SlideCountdown component is added, set the countdown time. When the countdown time expires, the device will not move until a new countdown time is set.

Hi @yj229201093 , when the countdown is done it will restart to the initialization duration?
Yes, it is a continuous countdown, for example, after the end of a countdown, the background will return a new countdown time. It should be back to the initialization state. At present, it is found that after the end of a countdown, the next countdown will not be followed, and I do not know how to deal with it.

是的 ，就是连续倒计时，如，一个倒计时结束后，后台会重新返回一个新的倒计时时间。应该就是回到初始化状态。目前发现一个倒计时结束后，不会接着下一个倒计时，不知道如何处理。

return SlideCountdown(
key: UniqueKey(),
duration:
TimeUtils.getCountdownTime(widget.nowAt ?? 0, widget.endAt ?? 0),
padding: const EdgeInsets.only(left: 0),
slideDirection: SlideDirection.none,
separatorType: SeparatorType.title,
showZeroValue: true,
withDays: true,
durationTitle: const DurationTitle(
days: '天',
hours: '小时',
minutes: '分',
seconds: '秒',
),
textStyle: YWTextStyle.setTextStyle(YWColor.white, fontSize: 12),
onDone: () {
Log.d('完成'); // There's a reassignment after that, and then the new data doesn't count down
if (widget.onDone != null) {
widget.onDone!();
}
},
onChanged: (value) {
Log.d('value = $value');
},
decoration: const BoxDecoration(
color: Colors.transparent,
borderRadius: BorderRadius.all(Radius.circular(0)),
));

add   UniqueKey(), ok, sorry for my question. Thanks for your guidance

您好 大神，问下，我用，倒计时结束后，我开启新的倒计时，不会更新。我添加 key: UniqueKey()可以更新，但是在倒计时期间我刷新页面的话，会闪缩，一闪一闪。这个有解决防范。
Hello God, ask, I use, after the end of the countdown, I open a new countdown, will not update. I add key: UniqueKey() to update, but when I refresh the page during the countdown, it will blink, blink, blink. This has a solution guard.
SlideCountdown(
// key: UniqueKey(),
duration:
TimeUtils.getCountdownTime(nowAt ?? 0, tempSaleStartedAt ?? 0),
padding: const EdgeInsets.only(left: 10),
// fade: true,
slideDirection: SlideDirection.none,
separatorStyle: YWTextStyle.setTextStyle(YWColor.white,
fontSize: 11, fontWeight: FontWeight.normal, height: 1.1),
separatorType: SeparatorType.title,
separatorPadding: const EdgeInsets.all(0),
showZeroValue: true,
withDays: false,
durationTitle: const DurationTitle(
days: ':',
hours: ':',
minutes: ':',
seconds: '',
),
textStyle: YWTextStyle.setTextStyle(YWColor.white, fontSize: 14),
onDone: () {
Log.d('完成');
if (onDone != null) {
onDone!();
}
},
onChanged: (value) {
Log.d('value = $value');
},
decoration: const BoxDecoration(
color: Colors.transparent,
borderRadius: BorderRadius.all(Radius.circular(0)),
));

@yj229201093 You can change UniqueKey to ValueKey
here an example

onDone important for called setState or you can use StateManagement for triger build
make sure value in ValueKey(value) is diffrent from old ValueKey(oldValue)

const kInitialDuration = Duration(seconds: 10);
const kNextDuration = Duration(seconds: 15);

class SlideCountdownPage extends StatefulWidget {
  const SlideCountdownPage({Key? key}) : super(key: key);

  @override
  State<SlideCountdownPage> createState() => _SlideCountdownPageState();
}

class _SlideCountdownPageState extends State<SlideCountdownPage> {
  int _completed = 0;

  @override
  Widget build(BuildContext context) {
    final duration = _completed % 2 == 0 ? kInitialDuration : kNextDuration;
    return Scaffold(
      body: Center(
        child: SlideCountdown(
          key: ValueKey(_completed),
          duration: duration,
          showZeroValue: true,
          withDays: false,
          onDone: () {
            setState(() {
              _completed++;
            });
          },
        ),
      ),
    );
  }
}

感谢，感谢，这个问题可以解决，但是又有了一个新的问题，就是我app在后台或者在切换到其他页面的时候，倒计时就不会走，在到这个页面才会走，会导致倒计时时间不准确。 这个好解决吗？


text:
Running pre-trained model on unlabeled data

I'd like to apply your pretrained model to a set of .txt files I have. I've cloned the repo and downloaded the model using the instructions in the LargeScaleRelationExtractionPipeline README.
However, it's unclear to me how I would go about running the model on unseen data. I tried just running the script with the following:
python run_ls_pipeline.py \
 --configs_file_path ComplexTome_configs.json \
 --log_file_path logs/drought.log \
 --model_folder_path the_best_model \
 --input_folder_path /mnt/scratch/lotrecks/drought_
drought_and_des_1000_subset_15Apr2024/      drought_and_desiccation_combined_22Mar2024/ drought_full_dataset_06Mar2024/
 --input_folder_path /mnt/scratch/lotrecks/drought_and_des_1000_subset_15Apr2024/ \
 --output_folder_path outputs/output_drought

where /mnt/scratch/lotrecks/drought_and_des_1000_subset_15Apr2024/ is a folder of .txt files. However, I'm getting a model path error that looks like its' because there's a hardcoded path somewhere, and am having trouble determining where it's coming from:
[2024-04-16 16:23:44] [<class 'helpers.logger.Logger'>.lp_halt]:
      ********************************************************************************
      HALT REQUESTED BY FUNCTION: program_halt
      HALT MESSAGE:
      Error loading the model. Error: Incorrect path_or_model_id: '/scratch/project_2001426/farmeh/bert_models/RoBERTa-large-PM-M3-Voc-hf/'. Please provide either the path to a local folder or the repo_id of a model on the Hub.
      HALTING PROGRAM!!!
      ********************************************************************************

[2024-04-16 16:23:44] [<class 'large_scale_prediction_pipeline_tf.Large_Scale_Prediction_Pipeline_tensorflow'>.exit]:
      EXITING PROGRAM ...

Haven't found a hardcoded path in any of large_scale_prediction_pipeline_tf.py, ComplexTome_configs.json, or run_ls_pipeline.py.
Pointers appreciated!

Hi, thanks for your question!
First you need to run get_model.sh script to download the model into a local folder and then provide the folder address in -----model_folder_path argument. Check here: https://github.com/farmeh/ComplexTome_extraction/blob/main/LargeScaleTriggerDetectionPipeline/single.sh
Then, the large-extraction pipeline does not include an NER part. So, it does not work on pure texts, but you'll need to run an NER system which detects Protein entities from texts.
Then you need to provide the --input_folder_path: here there are sub-folders, each containing one or more .tar.gz files, and inside, there are as many as you like documents in BRAT standoff format (each document will have a .txt file and .ann file for the detected entities).
Once you do that let us know if you still face any problems. Thanks!

Hi, thanks for your question.
First, you need to run get_model.sh script to download the model into a local folder on your cluser/gpu-machine, and then provide it in --model_folder_path argument.
Second, the large-scale relation extraction program does not have an NER component inside, i.e., it will not work on pure texts. You need to run an NER system to detect Protein entities inside the texts, and once you do that then you can run the relation extraction system. (For example see here).
Third, the --input_folder_path that you provide, should have specific format. Inside, there can be different sub-folders, in each one or more .tar.gz file, each containing documents in BRAT standoff format (each document includes a .txt file for text, and one .ann file for the Protein entities inside).
If you follow aforementioned steps and still see problems please let us know,
Thanks!

authors: farmeh serenalotreck


text:
技术指标 RSI,KDJ

运行代码 如下
st = sm['SH000001']
kd = st.get_kdata(Query(-100,ktype=Query.WEEK))
r1,r2,r3 = RSI(kd)
报错在RSI里

你可以先CLOSE 改成 CLOSE()，稍后更新代码

authors: XuWeiKiven fasiondog

repo: fasiondog/hikyuu


text:
bokeh绘图问题

bokeh_draw.py kplot方法：
axes.add_tools(HoverTool(tooltips=[("index", "$index"), ('日期', get_date_format(k)),
("开盘价", "@open{0.0000}"), ("最高价", "@high{0.0000}"),
("最低价", "@low{0.0000}"),("收盘价", "@close{0.0000}"),
("成交金额", "@amount{0.0000}"), ("成交量", "@volume{0.0000}")],
formatters = { "datetime": "datetime"}))
formatters 少了一个@符号
get_date_format方法返回逻辑似乎写反了

已修改

authors: fasiondog hahacaptain


text:
Add complete hints for important directives

https://user-images.githubusercontent.com/1483922/229269368-f4738e52-49ca-40a5-8ef5-6e839a2dc70b.mp4
cc: @jph00 @seeM

Wow nifty!

repo: fastai/nbdev-vscode


text:
Add an option to simply pass AwilixContainer via the plugin options

Prerequisites

[X] I have written a descriptive issue title
[X] I have searched existing issues to ensure the feature has not already been requested

🚀 Feature Proposal
I want to propose an option to pass a custom AwilixContainer to the plugin.
Motivation
Hi, thanks for the great plugin.
Currently, this plugin uses a singleton container declared inside it.
While we can directly access it via import { diContainer } from '@fastify/awilix' and call diContainer.dispose() to reset its contents, I think it's more intuitive if we can just pass a new AwilixContainer when we initialize the plugin.
This proposal is mainly for testing. Sure, we can dispose and re-initialize deps on-the-fly, but the option to provide the diContainer makes the test far easier.
Example
export type FastifyAwilixOptions = {
  disposeOnResponse?: boolean
  disposeOnClose?: boolean
  asyncInit?: boolean
  asyncDispose?: boolean
  eagerInject?: boolean
  container?: AwilixContainer<Cradle> // <- default to `diContainer` defined in the plugin
}

// register AwilixContainer with plugin option.
// this way we can easily swap the dependencies in testing environment.
fastify.register(fastifyAwilixPlugin, {
  container: newDIContainer
})

(and with this implementation, we can merge plugin code for each injectionMode)

Sure, I'll look into it!

@yshrsmz It was released in 5.0.0, could you please test it and report back if it works as expected?

authors: kibertoad yshrsmz

repo: fastify/fastify-awilix


text:
Gym fails by Code Signing Error: No profile for team 'xxxx' matching

Issue Description
Code Signing Error: No profile for team 'xxxx' matching

Code Signing Error: No profile for team 'xxxx' matching

Hi @jp928, you should provide a bit more context. Does the profile actually exist? What is the Fastfile and lane that is throwing that error? Did you read what @fastlane-bot posted?

楼主解决了么?

Closing this since there hasn't been any response 😊

authors: janpio joshdholtz jp928 uncleRX

repo: fastlane/fastlane